How can i pass linecache over an exec mthod local/global scope

Python Help
1

Hi

I have a usecase where i create some code at runtime and execute it,
i am facing issue in getting proper error traceback when i run the generated code using exec
consider below MRC

code = """
def my_method():
    def internal_method():
        raise RuntimeError("absxsx")

    internal_method()
"""

if __name__ == "__main__":
    import linecache

    scope = {}
    filename = "<dummy-filename>"
    compiled_code = compile(code, filename, "exec")
    if filename not in linecache.cache:
        linecache.cache[filename] = (
            len(code),
            None,
            code.splitlines(keepends=True),
            filename,
        )
    exec(compiled_code , scope, scope)  # noqa: S102
    fun = scope["my_method"]
    fun()

This gives me below error traceback

Traceback (most recent call last):
  File "temp2.py", line 28, in <module>
    fun()
  File "<dummy-filename>", line 6, in my_method
  File "<dummy-filename>", line 4, in internal_method
RuntimeError: absxsx

However if i invoke my_method directly i get proper traceback

def my_method():
    def internal_method():
        raise RuntimeError("absxsx")

    internal_method()

if __name__ == "__main__":
    my_method()

resultant traceback is

Traceback (most recent call last):
  File "temp2.py", line 12, in <module>
    my_method()
  File "temp2.py", line 8, in my_method
    internal_method()
  File "temp2.py", line 6, in internal_method
    raise RuntimeError("absxsx")
RuntimeError: absxsx

i tried to look into it and found that FrameSummary.line attribute is empty because it is not able to get source code but if i am adding it to linecache it should be able to get the source ig?

currently i tried to get it working by writting source code in tempfile and compiling it then it works but i was wondering if there is a way to do this without writting a tempfile
something like

scope = {linecache: linecache}  # does not work :(
exec(source_code, scope, scope)

so that linecache is available
wanted to know if there is better way to do this
i am working with systems having py3.8+

Thanks.

2

It was hard to do because there was no object in the traceback module, that wasn’t read-only :frowning:
But I think I made what you want:

import traceback, linecache

code = """
def my_method():
    def internal_method():
        raise RuntimeError("absxsx")

    internal_method()
"""

scope = {}
filename = "<dummy-filename>"
compiled_code = compile(code, filename, "exec")
c_lines = code.splitlines()

exec(compiled_code , scope, scope)  # noqa: S102
fun = scope["my_method"]
try:
	fun()
except Exception as e:
	
	tb = traceback.TracebackException.from_exception(e)
	new_st = list()
	for fs in tb.stack:
		if fs.filename == filename and len(fs.line) == 0:
			if len(c_lines) >= fs.lineno:
				fs = traceback.FrameSummary(filename, fs.lineno, fs.name, line=c_lines[fs.lineno-1])
		new_st.append(fs)
		
	st = traceback.StackSummary.from_list(new_st)
	print("Traceback (most recent call last):")
	print(* st.format(), sep="")
	print(* tb.format_exception_only(), sep="")

Tell me if it works like You expected :smiley:

3

In python3.13 this works correctly, so I guess this is a bug that was fixed at some point.

4

This solution is great but doesn’t work for me :frowning:
actually in my case there is some massaging that i do in traceback exception inside code string itself that is execed,
for me i was wondering if somehow i can provide the linecache.cache info inside the scope of execed method which might work ig,

because i saw traceback FrameSummary uses linecache to get content of line cpython/Lib/traceback.py at 3.8 · python/cpython (github.com)

while python 3.8 doesn’t say it but i saw in from py3.10 documentation says if source is unavailable line attribute is None, i was just wondering if there is some way to make source available to framesummary for exec code

5

i didn’t get a chance to try with 3.13,
i tried with 3.12 and 3.7 and 3.8 behavior seem to be available till 3.12 ig

6

I don’t see another approach other than updating the linecache - refer to _extract_from_extended_frame_gen in traceback.py - we operate on the code object of the frame, which does not contain the original source.