Builtins module for a different python version

Python Help
1

I’m making a code analysis tool which is capable of specifying the Python version, which may be different from the version the tool is running on.
If I see an identifier that might be a builtin, I want to examine the builtins (or builtin) module for the specified python version.
Any suggestions?

One possibility is to get the typeshed stub file (stdlib/builtins.pyi or stdlib/@python2/__builtin__.pyi) and then parse it into an AST, then analyze that. The stub has several if sys.version_info ...: statements that need to be evaluated.
Is there a tool somewhere that will do all this for me, and basically give me the type information for each module level name for the given python version? I could copy some code from mypy for both py2 and py3 targets. But is there anything else?

2

You could take a look at GitHub - JelleZijlstra/typeshed_client: Experimental package for retrieving information from typeshed (I don’t know if it does precisely what you’re looking for; I haven’t really played with it. @Jelle will be able to tell you more :slight_smile: )

3

Yes, typeshed_client should be able to help here. Example:

In [10]: print(sorted(name for name, info in typeshed_client.get_stub_names("builtins", search_context=typeshed_client.get_search_context(version=(3, 11))).items() if info.is_export
    ...: ed))
['ArithmeticError', 'AssertionError', 'AttributeError', 'BaseException', 'BaseExceptionGroup', 'BlockingIOError', 'BrokenPipeError', 'BufferError', 'BytesWarning', 'ChildProcessError', 'ConnectionAbortedError', 'ConnectionError', 'ConnectionRefusedError', 'ConnectionResetError', 'DeprecationWarning', 'EOFError', 'Ellipsis', 'EncodingWarning', 'EnvironmentError', 'Exception', 'ExceptionGroup', 'FileExistsError', 'FileNotFoundError', 'FloatingPointError', 'FutureWarning', 'GeneratorExit', 'IOError', 'ImportError', 'ImportWarning', 'IndentationError', 'IndexError', 'InterruptedError', 'IsADirectoryError', 'KeyError', 'KeyboardInterrupt', 'LookupError', 'MemoryError', 'ModuleNotFoundError', 'NameError', 'NotADirectoryError', 'NotImplemented', 'NotImplementedError', 'OSError', 'OverflowError', 'PendingDeprecationWarning', 'PermissionError', 'ProcessLookupError', 'RecursionError', 'ReferenceError', 'ResourceWarning', 'RuntimeError', 'RuntimeWarning', 'StopAsyncIteration', 'StopIteration', 'SyntaxError', 'SyntaxWarning', 'SystemError', 'SystemExit', 'TabError', 'TimeoutError', 'TypeError', 'UnboundLocalError', 'UnicodeDecodeError', 'UnicodeEncodeError', 'UnicodeError', 'UnicodeTranslateError', 'UnicodeWarning', 'UserWarning', 'ValueError', 'Warning', 'ZeroDivisionError', 'abs', 'aiter', 'all', 'anext', 'any', 'ascii', 'bin', 'bool', 'breakpoint', 'bytearray', 'bytes', 'callable', 'chr', 'classmethod', 'compile', 'complex', 'copyright', 'credits', 'delattr', 'dict', 'dir', 'divmod', 'ellipsis', 'enumerate', 'eval', 'exec', 'exit', 'filter', 'float', 'format', 'frozenset', 'function', 'getattr', 'globals', 'hasattr', 'hash', 'help', 'hex', 'id', 'input', 'int', 'isinstance', 'issubclass', 'iter', 'len', 'license', 'list', 'locals', 'map', 'max', 'memoryview', 'min', 'next', 'object', 'oct', 'open', 'ord', 'pow', 'print', 'property', 'quit', 'range', 'repr', 'reversed', 'round', 'set', 'setattr', 'slice', 'sorted', 'staticmethod', 'str', 'sum', 'super', 'tuple', 'type', 'vars', 'zip']

In [11]: print(sorted(name for name, info in typeshed_client.get_stub_names("builtins", search_context=typeshed_client.get_search_context(version=(3, 6))).items() if info.is_exporte
    ...: d))
['ArithmeticError', 'AssertionError', 'AttributeError', 'BaseException', 'BlockingIOError', 'BrokenPipeError', 'BufferError', 'BytesWarning', 'ChildProcessError', 'ConnectionAbortedError', 'ConnectionError', 'ConnectionRefusedError', 'ConnectionResetError', 'DeprecationWarning', 'EOFError', 'Ellipsis', 'EnvironmentError', 'Exception', 'FileExistsError', 'FileNotFoundError', 'FloatingPointError', 'FutureWarning', 'GeneratorExit', 'IOError', 'ImportError', 'ImportWarning', 'IndentationError', 'IndexError', 'InterruptedError', 'IsADirectoryError', 'KeyError', 'KeyboardInterrupt', 'LookupError', 'MemoryError', 'ModuleNotFoundError', 'NameError', 'NotADirectoryError', 'NotImplemented', 'NotImplementedError', 'OSError', 'OverflowError', 'PendingDeprecationWarning', 'PermissionError', 'ProcessLookupError', 'RecursionError', 'ReferenceError', 'ResourceWarning', 'RuntimeError', 'RuntimeWarning', 'StopAsyncIteration', 'StopIteration', 'SyntaxError', 'SyntaxWarning', 'SystemError', 'SystemExit', 'TabError', 'TimeoutError', 'TypeError', 'UnboundLocalError', 'UnicodeDecodeError', 'UnicodeEncodeError', 'UnicodeError', 'UnicodeTranslateError', 'UnicodeWarning', 'UserWarning', 'ValueError', 'Warning', 'ZeroDivisionError', 'abs', 'all', 'any', 'ascii', 'bin', 'bool', 'bytearray', 'bytes', 'callable', 'chr', 'classmethod', 'compile', 'complex', 'copyright', 'credits', 'delattr', 'dict', 'dir', 'divmod', 'ellipsis', 'enumerate', 'eval', 'exec', 'exit', 'filter', 'float', 'format', 'frozenset', 'function', 'getattr', 'globals', 'hasattr', 'hash', 'help', 'hex', 'id', 'input', 'int', 'isinstance', 'issubclass', 'iter', 'len', 'license', 'list', 'locals', 'map', 'max', 'memoryview', 'min', 'next', 'object', 'oct', 'open', 'ord', 'pow', 'print', 'property', 'quit', 'range', 'repr', 'reversed', 'round', 'set', 'setattr', 'slice', 'sorted', 'staticmethod', 'str', 'sum', 'super', 'tuple', 'type', 'vars', 'zip']

This is based on the typeshed stubs, so it’s not precise for versions that are no longer supported by typeshed (e.g., 3.5).

1 Like
4

I am surprised that you are bothering with Python 2. Do you also support Python 1.x? Python 2 is likewise obsolete.

You could try launching the target Python interpreter in a subprocess, import the builtins, and grab the list of builtins using dir() or vars().

I’m confident that will work for any installed Python, including non-CPython implementations. (Tested on Jython, IronPython, MicroPython and PyPy.)

Otherwise, you could keep your own list, built from the docs. If you start with Python 1.0, there are (so far) only 27 versions to track, which mostly changes slowly. If you start with Python 2.7, that makes only 13 so far.

You could track changes to builtins rather than keep multiple redundant lists. E.g. start with the builtins for Python 3.0, then keep a list of additions and removals for each version since then.

That would mean that your tool would only be able to look backwards, to older Python versions, not forwards, to newer ones.

5

I was interested in Python 2.7 because mypy supports 2.7 based code. I want the stuff I’m writing to be usable someday as part of mypy.

This means that any users of my code would have to have the target version of Python installed on their computer, which seems an unacceptable burden on the user.

6

After looking at Jelle’s typeshed_client, I have a better understanding of the workings of typeshed.
I don’t expect I will do the work in my tool to get older versions of builtins. Rather, I’ll make a note that typeshed_client can be used to get the equivalent builtins for a specified version and platform.
I also noticed that each stub file applies to all python versions and platforms using statements like if sys.python_version <= (3, 8): .... So to get the right version of builtins, one would have to get the AST for the stub and determine which definitions are actually executed on the platform.
@Jelle, I think it would be nice for typeshed_client to interpret those if statements and come up with an AST that is pruned for the given SearchContext.

7

typeshed_client.get_stub_names already supports that (when given an appropriate context, it filters by Python version and platform).