Retrieve AST From `exec()`

Hi!

As part of our Python library, we have a decorator for parsing user functions into an AST and transpiling the code into another language.

This works well when user modules are evaluated the standard way, but we can’t figure out how to get that to work when the code is stored as a string in memory and users evaluate it with exec().

See the following snippet as an example.

# Library Code
# ------------

import ast
import inspect

def lib_decorator(func):
    source = inspect.getsource(func)
    tree = ast.parse(source)
    print(tree.body[0])

# User Code
# ---------

# This works.
@lib_decorator
def user_function():
    pass

# This doesn't work.
exec("""
@lib_decorator
def user_function():
    pass
""")

The issue being that inspect.getsource(func) fails to retrieve the source code since it relies on it being stored in a file on disk, and it looks like exec() doesn’t store the string being evaluated anywhere.

I couldn’t find any official helper to build an AST from a function’s bytecode, which I’m assuming is expected since it doesn’t seem trivial.

Am I missing any trick to retrieve the AST of the func argument passed to lib_decorator regardless of how the user code is evaluated?

Thank you!

No. The source code and/or AST might not exists at all anymore and is therefore irretrievable in general. If you need the AST, you can’t really support exec.

Thanks for the prompt reply, @MegaIng!

I wonder if this kind of workflow is something that could be supported by Python in the future since there are quite a few frameworks existing nowadays that transpile Python code into other languages, or would it be not feasible to implement?

It’s ofcourse feasible at a pretty decent memory cost. I doubt that there are enough usecases to justify it. Most python code is not done via exec and therefore doesn’t need support for such rare decorators.

If libraries really care, they can provide custom exec functions[1] that correctly cache the source code.


  1. or even overwrite the builtin one by assinging to the builtins module. ↩︎

Got it, thanks again @MegaIng!