In PEP362, support for inspecting callable signatures was added. It described the __signature__ attribute as an override to the inspect.signature function : any object found in __signature__ and not None would be returned by inspect.signature.
But in fact, since 2014 apparently, a type guard allowing only instances of Signature to pass, or else raise an exception. (None is treated as if the attribute isn’t present.)
None of this ended up in the documentation.
More than a year ago, the behavior changed (in a backwards-compatible way) by gaining undocumented support for strings and callables, as a special shoehorn for Enum. That’s something I’m trying to roll back, but that’s not the point here.
Not knowing that, in october 23, I authored a merge request to document the __signature__ as it was specified in the PEP. It was accepted and merged, but it was inaccurate as it did not account for the type guard or for the later support of strings and callables.
As a response to this discrepancy, the documentation was patched to write it all off as an implementation detail, advising people to see the exact semantics in the code, and demoting the ability to pass a Signature object as __signature__ from a feature to an implementation detail.
I think it should be re-documented as a feature in the following way :
Signature objects in __signature__ are returned as-is by the inspect.signature function
None is ignored and the behavior is as if __signature__ was not set
Behavior for any other value is an implementation detail.
That way, uses of __signature__ as a cache or as a way to hide undocumented parameters remains guaranteed, while allowing the strings and callables support to be decided and to evolve separately.
Is there reasons why we would like to keep more complex handling for this special attribute?
Given above, the documentation could be rephrased as: “If the object has a __signature__ attribute and if it is not None, this function returns it.” Just as it was designed in the PEP, IIUIC.
Also, while in practice, probably, this will be tedious and inconvenient (see Signatures for extension modules (PEP draft), for some alternative) — this attribute actually the only way how people could support introspection for extension modules.
But it worth to note, that the PEP 362 has limitations, the inspect module can’t yet represent functions with multiple signatures (see e.g. Signatures, a call to action). Maybe this is an argument for we shouldn’t expose __signature__ as public? Wouldn’t be the __signatures__ as “list of Signature objects or None” - better?
Sorry, if this a bit off-topic and changes in logic shouldn’t be discussed here. @erlendaasland suggested me to start jet another thread on d.p.o, but I think it will be overkill:)
I too think discussion on the merits of potential behavior changes to __signature__ should be kept off-topic. As you mentioned, it is being discussed elsewhere, with an associated PEP draft, and I wish here to offer a stopgap change that will document the current behavior in the best possible way, and be compatible with some (though not all) possible future changes.
First, I’d like to point out that the inspect module doesn’t fully and natively support C-defined functions (there are ways around it, notably __text_signature__).
Second, maybe apart from these C-defined functions which I don’t know enough about, so in normal Python, “functions with multiple signatures” don’t exist. There are functions where one parameter changes its meaning based upon other parameters, and you may want to document these functions using several signatures (the best example is range), but as an introspection question, multiple signatures simply do not exist. It may be (I don’t know) that the “call to action” thread is leaning on introducing multiple signatures, but that’s not the current state of the language.
So, the documentation shouldn’t account for speculative changes, in my opinion.
In addition, even if a list of signature were to be supported for a single callable, with the __signatures__ you mentioned, the function to return that would be inspect.signatures, not inspect.signature. So that seems like a moot point to me.
For now, the type check is in place in the function, so that sentence (which I wrote, lol) is inaccurate. If it ever were to change in the way you wish, the phrasing I’m proposing now in #116124 would remain true : if you put a Signature object, it will be returned by the function, if you put None it will be ignored, if you (somehow) put something else that’s not documented.
This is making a distinction that is not very useful IMHO. Users of introspection are interested in the signature(s) that a callable actually supports (i.e. can be successfully called with), not in how the function was physically defined.
I.e. if a function was physically defined as taking (*args, **kwargs) but accepts a logical signature of (x, y, *, some_option=None) (for example because it’s doing argument parsing by hand), I as an introspection user am interested in the latter information, not the former.
That said, I agree that multiple signatures are a distraction from other more pressing concerns, such as documenting actual __signature__ and __text_signature__ semantics, and making it easier for C extensions to expose correct signatures.
(We’re drifting slightly off-topic but this is interesting.)
Yes, I agree. That’s why I support the use of __signature__ to white-lie about a function’s signature, potentially to hide certain parameters and so on.
My point is slightly different : there’s a lot of different ways to document signatures in a human-readable way : one is documenting optional parameters with = and the default value, one uses square brackets to denote optional parameters without specifying their value, and another would use a list of signatures.
All three are perfectly valid to use in documentation, provided some measure of consistency, but the fact that only the first one has a syntax support in Python language is the distinction which, in my opinion, sets it apart from the others.
So, can we move forward with this ? Is there comments, feedback, objections to the way I propose features and implementation details should be split, or to the phrasing I’m using in the pull request ?
Hmm, maybe you should also describe an impact of __signature__ attribute on follow_wrapped parameter of the inspect.signature(). This is mentioned somehow in the inspect.unwrap() docstring, but I doubt it’s the right place…
That’s true. However, the behavior regarding that is weird :
The unwrapping process will stop at the first callable having a __signature__ attribute. That is, if that attribute has any value, including None, when a None value is otherwise ignored…
In other words, that None value has the effect of stopping the unwrapping recursive process, but if the None value is ignored when interpreting a callable’s signature, then it should be ignored all the way.
This started being only a doc issue, but I think this tiny detail should be changed with it… if only to make the behavior consistent. It only requires changing the following :
to using (lambda f: (getattr(f, "__signature__", None) is not None) and the following lines unchanged.
I don’t think it’s “weird”, rather this piece of code assumed, that if this attribute is set — it’s value is a correct Signature object. This constraints could be enforced by an appropriate setter for this magic attribute. Slightly simpler logic for __signature__ processing, that has on a pros also: you will get a TypeError exception from the setter, right at the time you break things, not from the inspect.signature().
That would be backwards-incompatible, and also magic.
It breaks the feature of having clbl.__signature__ = None to make sure that the true signature of the callable is computed - since del clbl.__signature__ will fail when the attribute wasn’t already set.
By being also magic it feels bigger than such a small change requires.