Problem
While working on gh-103131: Convert `sys.getsizeof` and `sys.set_asyncgen_hooks` to AC by sobolevn · Pull Request #103132 · python/cpython · GitHub I’ve noticed that NULL default is a big problem for current defaults in AC.
Right now, inspect.signature will fail for any function with NULL as the default. Let’s take builtins.iter (on 3.12) as an example:
>>> iter.__text_signature__
'($module, object, sentinel=<unrepresentable>, /)'
>>> import inspect
>>> inspect.signature(iter)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/sobolev/Desktop/cpython/Lib/inspect.py", line 3362, in signature
return Signature.from_callable(obj, follow_wrapped=follow_wrapped,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/sobolev/Desktop/cpython/Lib/inspect.py", line 3106, in from_callable
return _signature_from_callable(obj, sigcls=cls,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/sobolev/Desktop/cpython/Lib/inspect.py", line 2599, in _signature_from_callable
return _signature_from_builtin(sigcls, obj,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/sobolev/Desktop/cpython/Lib/inspect.py", line 2400, in _signature_from_builtin
return _signature_fromstr(cls, func, s, skip_bound_arg)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/sobolev/Desktop/cpython/Lib/inspect.py", line 2261, in _signature_fromstr
raise ValueError("{!r} builtin has invalid signature".format(obj))
ValueError: <built-in function iter> builtin has invalid signature
This is how AC converts NULL to be <unrepresentable> in __text_signature__:
/*[clinic input]
test_str_converter
a: str = NULL
/
[clinic start generated code]*/
PyDoc_STRVAR(test_str_converter__doc__,
"test_str_converter($module, a=<unrepresentable>)\n"
"--\n"
"\n");
Right now we have ~52 files with <unrepresentable> signatures.
There’s also a user-reported issue about bytes.hex having incorrect inspect.Signature: inspect.signature(bytes.hex) raises ValueError "builtin has invalid signature" · Issue #87233 · python/cpython · GitHub
And incorrect signatures in builtins module: Help text of builtin functions – missing signatures · Issue #107526 · python/cpython · GitHub
In typeshed we use ... to specify default values. For example, here’s how dir is defined: https://github.com/python/typeshed/blob/1c0500a57050815102c702efd053e09770a5ee88/stdlib/builtins.pyi#L1320
Proposed solution
I propose adding a special signleton value inspect.unrepesentable to be used instead. We can customize its __repr__ to be <unrepresentable>, ..., or whatever. Here’s how a hypothetical patch would look like:
diff --git Lib/inspect.py Lib/inspect.py
index c8211833dd0..64e1b8f0839 100644
--- Lib/inspect.py
+++ Lib/inspect.py
@@ -2238,10 +2238,29 @@ def _signature_strip_non_python_syntax(signature):
add(string)
if (string == ','):
add(' ')
- clean_signature = ''.join(text).strip().replace("\n", "")
+ clean_signature = ''.join(text).strip().replace("\n", "").replace(
+ # Handle `NULL` defaults:
+ "<unrepresentable>",
+ "__unrepresentable__",
+ )
return clean_signature, self_parameter
+class _Unrepresentable:
+ _instance = None
+
+ def __new__(cls):
+ if cls._instance is not None:
+ return cls._instance
+ cls._instance = super().__new__(cls)
+ return cls._instance
+
+ def __repr__(self):
+ return "<unrepresentable>"
+
+unrepresentable = _Unrepresentable()
+
+
def _signature_fromstr(cls, obj, s, skip_bound_arg=True):
"""Private helper to parse content of '__text_signature__'
and return a Signature based on it.
@@ -2309,6 +2328,8 @@ def visit_Attribute(self, node):
def visit_Name(self, node):
if not isinstance(node.ctx, ast.Load):
raise ValueError()
+ if node.id == "__unrepresentable__":
+ return unrepresentable
return wrap_value(node.id)
def visit_BinOp(self, node):
@@ -2331,7 +2352,10 @@ def p(name_node, default_node, default=empty):
if default_node and default_node is not _empty:
try:
default_node = RewriteSymbolics().visit(default_node)
- default = ast.literal_eval(default_node)
+ if default_node is unrepresentable:
+ default = unrepresentable
+ else:
+ default = ast.literal_eval(default_node)
except ValueError:
raise ValueError("{!r} builtin has invalid signature".format(obj)) from None
parameters.append(Parameter(name, kind, default=default, annotation=empty))
This will allow us to parse and inspect this signature:
>>> import inspect
>>> sig = inspect.signature(iter)
>>> sig.parameters
mappingproxy(OrderedDict({'object': <Parameter "object">, 'sentinel': <Parameter "sentinel=<unrepresentable>">}))
>>> sig.parameters['sentinel']
<Parameter "sentinel=<unrepresentable>">
>>> sig.parameters['sentinel'].default
<unrepresentable>
>>> sig.parameters['sentinel'].default is inspect.unrepresentable
True
Related:
- https://github.com/python/cpython/pull/13933
- Issue 37206: Incorrect application of Argument Clinic to dict.pop() - Python tracker
- Signatures, a call to action - #34 by larry
I others agree, I can submit my patch + tests + docs.