Rich format of python comments

Suggestion
My suggestion is to add a special python code format named, for example, .richpy.
It would allow for not only the code itself but for markdown comments (as in git) and images.
During its execution we would ignore comments as we do currently.

Reasons
If you are familiar with Jupyter notebooks (.ipynb file format), you have probably already experienced the comfort of markdown cells and of having results in charts always in front of you. And all these perks are layed out hand in hand with the code itself.
I would like to see a similar format for python modules. So that all documentation (including charts) would be situated right next to the code itself.

Example
I have been practicing in creating AVL trees and it turned out that some pieces of code can be much easily described by a chart rather than comments only, for example, rotating a node right or left.

Intuition
Why do we comment code? Because we can faster comprehend the algorithm implemented. Why not take another step in that direction and introduce other features that would make comments even more comprehendable?
Consider the following analogy. Back in the days we used to have a primitive notepad editor. It only allowed for plain text. But later on the world has come up MS Word, which had a lot of useful features: headings, bold fonts, colored fonts, images, etc. These features provide more means of expression of our thoughts. So why not use them?
We have currently markdown files with comments in github. It is super popular. It means that the need is there. But for the moment we are forced to hold all this beauty in a separate file, not in the code itself, where it belong imho.

““Possible troubles””
It would be more difficult to track changes or rather depict changes between two versions of a code. My guess is that an option to ignore all comments during comparison would be helpful.

What do you think?

But the comments are as important as the code so the comments must not be ignored when comparing versions surely?

I think this is a very interesting idea. I don’t see why it needs a new file type though. As it is, docstrings are rendered as .rst by Sphinx. You could have a tool that rendered comments and docstrings in whatever rich format you want.

3 Likes

I very quickly looked around if I could find something like this that already exists. The closest I could find is this:

Is it more or less something like this that you have in mind? The comments are rendered inline in the same view as the code itself.

Check out Jupytext.

You can either have markdown comments (Notebooks as code — Jupytext documentation) or literate programming where the code to execute is in markdown fences (Notebooks as Markdown — Jupytext documentation). Both can be used directly as Jupyter notebooks or converted among formats.

When I want to write this way, I tend to use the .Rmd (R markdown) format.

Example: textbook/arrays_and_images.Rmd at main · nipraxis/textbook · GitHub

Rendered: Arrays as images, images as arrays — Practice and theory of brain imaging

You can already put whatever markup you like in a comment. What you want depends only on what you use to view your source code.

There’s codechat which interprets comments in restructuredText which Sphinx turns into HTML. It used to be plugable into Enki so that you’d get a live render although that got dropped. sphinx-autobuild gives a pretty good approximation though.

Personally, whenever I have tried to explain fiddly algorithms using tools like this, I’ve found that often the comments and diagrams can make it impossible to track where each function/for loop/conditional block starts and ends so the code becomes hard to follow and the explanation is also barely coherent because it’s forced to follow the order/structure of the code and to jump between the high level architectural design to low level implementation details. I’ve got much better readability by writing a separate how it works.md document and then just using moderate comments in the code itself. That’s not to say that you shouldn’t use these tools – just to be wary of using them for the wrong reasons.

1 Like

I’ve had a similar experience trying to read code written this way–in the end it’s easier to open up the docs in a browser, with the docstrings pulled out and formatted separately.

It would be somewhat problematic to depict the changes in a concise manner. Basicaly what we are talking here is a track changes option in MS Word. When you choose “show all markup” it is often overwhelming. That is why I personaly would prefer to have on option to see the difference in code only.

I think this feature you refer to is alike, but very limited and does not support images. Imho, we do not need to reinvent something here. We have markdown already. The only thing that is missing for the moment is an opportunity to place it right next to the code and to add an image.

That is what I am talking about. That for the moment the preferred modus operandi is to have little to no comments in the code itself and a separate markdown file with description. Can we live like that? Shure. But the very existence and popularity of ipynb signals that we could do better.
This approach we use now has the following drawback: as soon as we rename a function or method in the code, we have to update it in the markdown file. And it may appear with that name in several places. If we had a single file we would just press Ctrl+H (or alike) and rename it instantly.

so close and yet so far)

Markdown works nicely with git tracking. The images would be referenced rather than embedded, but unless the images themselves are changing frequently, that shouldn’t be a problem.

Markdown also tends to read fairly well just as text. Even if it isn’t being rendered, it still looks like formatted text.

1 Like

Perl has Pod which gave it the ability to add documentation to code that could easily be rendered as ASCII coloured text at the command line terminal, which was cool.
We already have the Sphinx toolchain, but since then Jupyter’s Markup flavour has become very popular and there are already tools that when exporting A notebook as a Python script, will store the markup as a block comment.
So I’d vote on support for Jupyter-style markup in comments, but would also like the Perl-Pod style support for nice output in POSIX terminals as well as HTML, etc.

What we do as programmers? We docompose the task in sub-tasks (funtions or classes) so that these sub-tasks would be independent, as independent as posible (high cohesion, low coupling mantra). Now as far as comments are concerned, they are tightly bound to the code. Having them in 2 separate places is equivalent to having high coupling. As soon as you change here you have to jump and change there.

This doesn’t require any changes to Python itself, does it? A tool can extract and display comments however it wants, as long as it defines a format and people use it.

Consider how Sphinx et al will format docstrings and comments for creating documentation–this is just another interface to the same thing.

Even the original post (defining a new file format) does not require any change to Python. It just needs someone to write the tool that does that thing.

3 Likes

Great insight! We can basically add all special syntax to the comments, so that standard python engine would not notice them at all, while our custom solution (e.g. VS Code plugin) whould render it in a way that we define. So the file will remain .py.
The only hindrance here is that git will not be able to recognize our custom syntax and would mark too much as changes in diff - font, formatting, etc.

I even think this little trouble with git diff can be overcome by a custom IDE extension for comparing files compatible with git. So a custom IDE extension could be a way to go to fully impelement this idea of mine.