Revisiting attribute docstrings

Ideas
1

PEP 224 (Attribute Docstrings) proposed a syntax for class attribute docstrings:

class A:
    b = 42
    """Some documentation."""

    c = None

This was rejected because of ambiguity for readers about which attribute the docstring referred to.

With the prevalence of Sphinx, it is now understood that the docstring refers to the immediate prior symbol (see the docs).

Some in the community don’t like the approach introduced in PEP 727 (Documentation in Annotated Metadata), where a symbol’s documentation is a field in its Annotated annotation, and wish to introduce more Pythonic syntax to address the problems raised in that PEP. Below is a proposal which does just that.

I propose an extended form of PEP 224 to document symbols: module attributes, class attributes, and function parameters.

module_attribute = "spam"
"""A string."""

class AClass:
    class_attribute = 42
    """An integer."""

def foo(
    bar,
    """A required parameter."""

    baz: int = None,
    """An optional parameter."""
):
    ...

A docstring will always document the immediately-prior symbol at the same indentation level. Note how the comma , following a parameter definition must go before the docstring to prevent string-concatenation with string-typed parameter defaults.

The docstring’s value will be stored in a new attribute __docstrings__, defined only on usage of this proposal (injected into parent’s __dict__ after the parent is defined and processed). For module and class attributes, __docstrings__ is set on the module and class respectively. For function parameters, __docstrings__ is set on the function. [1]

I’ve found one usage on GitHub already using the name __docstrings__

inspect.Parameter would gain a new instance attribute docstring which has the parameter’s corresponding docstring value. A search shows some potential conflict with existing code.

  1. perhaps type, ModuleType and FunctionType could learn a __docstrings__ getter-property; this is an implementation detail ↩︎

14 Likes
2

I’d definitely use this if it were accepted since I prefer having docstrings closer to the variable, and I dislike repeating parameter names.

A few questions:

  • Is there any way to put the docstring on the same line as the variable or parameter? That ability alone makes comments a close competitor.
  • Are you making any recommendation about where the comma should with parameter docstrings (your example shows two places)?
  • If we have programmatic access to parameter docstrings, will they available in inspect.Parameter?
2 Likes
3

I personally can’t stand when the docstring is below a non-function-like variable. Especially a lot of the time these would be one line comments. I’d say either do them on the same line or above.

8 Likes
4

I can’t think of a good solution which looks legible, so I’d prefer to stick to the existing forms with class and module attributes.

I don’t mind what the final solution is, but the proposal in my original post explicitly allows for either

Yes, I’ll update the original post to make a comment on Parameter

The doc-string is parsed by Sphinx (and other tools, like PyCharm (an IDE)) and used as the documentation for that symbol, which a comment can’t currently (nor do I think should) do. This proposal would further make that available at runtime

5

I understand but I personally prefer the other way: have the docstring above the definition.

Even if they can be parsed by tooling, I have to look at it, and don’t like the way docstring below variable looks.

I don’t think I’ve ever seen docstring below variable besides class/function declaration.

If we decide to formalize a format, if prefer it be above or on the same line.

Edit:

I think I’ve seen tools parse

V = 'hello' #: this is the docstring for V

As a way of doing one liner

Similarly also for two lines:

#: this is the docstring for V
V = 'hello'
7 Likes
6

You’re probably referring to Sphinx’s autoattribute, which also supports docstrings after the attribute

2 Likes
7

I’d consider that a showstopper. The comma is an explicit separator. Having a docstring after a comma grouped with a parameter before the comma seems red-alert ultra-confusing to me.

12 Likes
8

Same, for parity for where I would normally write the comment. I’ve never seen comment-below ever referring to the line above.

10 Likes
9

Would it be worth contrasting this PEP’s proposed notation with the #: notation in the PEP?

2 Likes
10

Placing the comma after the docstring introduces a syntax ambiguity:

def foo(
    param: str = "some default value"
    """Some documentation""",
):
    ...

Is that a string concatenation? Or is the second one a docstring? With today’s syntax, it’s a string concatenation, so if it’s now a docstring then that’s a backwards-incompatible syntax change.

On the other hand, I would agree with @BrenBarn that separating the parameter from its docstring with a comma would be egregious.

8 Likes
11

No PEP yet, but I likely will if and when I make one. The original post is simply motivation and syntax.

From GitHub searches:

  • symbol documentation using special comments #: is in 33 900 files (search)
  • symbol documentation using multi-line strings """ is in 68 200 files (search) (although this has an indeterminate number of false-positives)

I’m personally not a fan of the #: syntax as I consider all parts of the code starting with hash # to be stripped from the runtime (but potentially used by some tools: the most popular ones I can think of are black and coverage). I also prefer the consistency of having docstrings after declarations (ie the case right now with functions, classes and modules).

That’s a problem; I don’t think we should change string-concatenation semantics. I’ll update the original post to remove that option.

Perhaps there’s opportunity to add a delimiter, eg semicolon ;.

4 Likes
12

Would it be stored as a dictionary with the attribute/parameter name as key and the doc string as value? Or what did you have in mind?

13

A delimiter would be ideal since it would allow you to put the docstring on the same line for parameters and variables. E.g.,

bias: float = 4;  "The bias of the model"
weights: list[float]
"""The weights of model.
Initialized to zero.
"""

and similarly for parameters (with a comma after the docstring). Is that what you have in mind?

So, essentially a parameter x has an optional type annotation indicated by :, an optional default indicated by =, and an optional docstring indicated by (perhaps) ;?

14

In Flask and my other projects, I’m moving away from #: above to """ below attributes to document them.

#: has the problem of messing with indentation levels, since it’s 3 characters before you start typing, so if you need to indent further for some reason, you have to manually add an extra few spaces after pressing tab to get things properly indented. #: also isn’t handled by IDEs well, so typing that before every line of a multiline doc is tedious.

""" avoids those two issues, and also matches what documentation looks like and where it’s found for classes and functions already. It’s also easier to modify lines and reflow text later.

I guess this sort of defeats the purpose of this discussion as opposed to the docs-in-annotations PEP, but I’d personally leave out parameters from a proposal. Aside from the ambiguity in can create, I still think parameter documentation looks better in the class/function docstring rather than next to each parameter.

8 Likes
15

Clearly we need TOCs - triple octothorp comments.

6 Likes
16

Yes, specifically a mapping from strings to strings (don’t need to require all dict methods).

That’s a good option, and was something I was thinking about. I’d imagine many aren’t comfortable with slightly altering the definition of the semicolon ; though.

Parameter documentation is my entire motivation: the module and class attribute was just a bonus.

I agree that parameter docs look better in the docstring, there are situations where the docs are better suited near the parameter and unambiguous, for example when needed at runtime.

2 Likes
17

My main point was a preference towards """ over #: style (or both, as Sphinx does now). If you’re confident about parameter docstrings as well, that’s fine.

1 Like
18

Having docstrings above attributes creates an ambiguity with module docstrings:

# ambiguous.py
"""Am I the module docstring, or the docstring of `hello`?"""

hello = "hello"
5 Likes
19

We could compromise on syntax that goes in the same line as the attribute. It would also work for function arguments. I didn’t know about #:, and I like it, except that it could interfere with comments aimed at guiding commonly-used linters.

20

Maybe ambiguous to a computer, but to a human the extra newline tells us.

1 Like