Sorry for the late reply on these topics. I clearly should visit my local planning department more often
Edge case: decorators that use inspect.signature
+ forward references, even though client code is not using annotations directly
This is somewhat similar to the forward-referencing annotations in dataclass, though we’re not looking at the contents of the annotation here.
inspect.signature
is the stlib’s high-level way to work with introspection of callables. Currently it evaluates __annotations__
eagerly when given a function.
Here’s an example in which a decorator tries to report a parameter that it is effectively adding to a wrapped function’s signature.
import inspect
def with_loglevel_param(func):
sig = inspect.signature(func)
def wrapper(*args, loglevel=1, **kwargs):
...
func(*args, **kwargs)
wrapper_sig = inspect.signature(wrapper)
wrapper.__signature__ = sig.replace(
parameters = [
*sig.parameters.values(),
wrapper_sig.parameters["loglevel"]
]
)
return wrapper
@with_loglevel_param
def my_func(spam, ham: MyParamType):
...
class MyParamType:
...
print(inspect.signature(my_func))
Compatibility
-
Eager evaluation (PEP-484, PEP-3107): includes a forward reference
-
Stringified postponed evaluation (PEP-563): it runs, though you could imagine issues figuring out with which globals to evaluate each parameters’ annotations with, should
with_loglevel_param
and my_func
both have annotations and be defined in separate modules
-
Descriptor postponed evaluation (PEP-649): Using
inspect.signature
defeats the postponed evaluation
I could imagine a backwards-compatible change to inspect.Signature
/Parameter
introduced by PEP-649’s implementation would resolve this. For example:
- it would continue to postpone the evaluation of
__annotations__
until it is accessed on a parameter
-
replace(get_[return_]annotation=X)
would treat X as a callable that returns the annotation value
-
replace([return_]annotation=X)
would treat X as the value of the annotation (equivalent to replace(get_[return_]annotation=lambda: X)
)
This would do nothing to solve cases where the decorator would evaluate the annotation directly, unless they also find ways to defer when the annotation gets evaluated. This doesn’t really work if you immediately need annotations, for instance I’m not sure dataclass can implement its ClassVar/InitVar support without requiring them until instantiation.
Another caveat is that this dissociates errors in annotation definitions even further, but from what I understand PEP-649 would end up showing the site of annotations’ definition in the stack trace. (Speaking of stack traces, this solution would add a few functions and frames when using inspect.)
Context
I introduced sigtools.modifiers
a while ago while migrating clize
to rely on Python 3 features (annotations and keyword-only parameters) to maintain Python 2 compatibility. At the time, the docs recommended modifiers.kwoargs, modifiers.autokwoargs, modifiers.annotate prominently. Over time, I updated the docs to prioritize Python 3 syntax, then eventually removed mentions of sigtools.modifiers
completely,
During this time, the usage of it spread, including in third-party tutorials, so I am committed to keep supporting it in future Python versions. (There are some usages detectable in a GitHub public search, but I imagine most users of clize
don’t publish their work or run their scripts in a way that they would see DeprecationWarnings.) Anecdotally, clize and sigtools still use them despite officially dropping Python 2.7 support, as specifically removing code used to guarantee Python 2 compatibility isn’t generally a high priority.
I recognize that it is somewhat unlikely that you’ll see code that mixes both styles like this:
@sigtools.kwoargs("two")
def version_whiplash(one: Int, two: Int = 2):
...
But I imagine it could occur across modules in larger codebases. I suspect, but haven’t confirmed that sigtools.modifiers
could be updated to avoid eager evaluation.
modifers.annotate
also has the same problem, and in addition needs to find a way (under PEP-563) to assign arbitrary values as strings into annotations (it assigns only the __signature__
attribute rather than __annotations__
) in a way that won’t confuse 3rd party tools reading from __signature__
.
As a side-note, I’ve been working on sigtools
to have it support PEP-563, and gravitated to a solution that pairs up annotations with where they have been defined, which seems to build something that is somewhat similar to what PEP-649 proposes, except on a per-parameter basis instead of on the whole __annotations__
dict (some of sigtools
’ function is to attribute each parameter to the function that originally created it, e.g. through decorators and such, so two parameters’ annotations could be evaluated differently).
It does seem odd that PEP-563 changes the meaning of __annotations__
completely, making it more difficult to support both versions (or modules compiled with different future flags) than necessary.