Add a semi-interactive mode to launch a REPL on exception

It would be tremendously useful if CPython can drop me to a REPL when an uncaught exception happens so that I can poke around and examine the environment at that point. While we cannot make it the default behavior for the sake of backward compatibility, at least we can add a flag --semi-interactive to make it possible to preserve the “crime scene”. It’s called semi-interactive since the interpreter enters REPL only when an exception takes place. Additionally, we can respect an environment variable called PYTHONSEMIINTERACTIVE so that developers can set it in ~/.bash_profile and make it the default behavior locally.

Currently, we can do python -m pdb -c continue myscript.py, but that doesn’t work when you are executing a module, i.e. python3 -m pdb is incompatible with something like python3 -m mymodule.main. There doesn’t exist an environment variable to make launching pdb the default behavior either.

Maybe python -i an PYTHONINSPECT?

That’s the interactive mode. It drops me in a REPL even when no exceptions took place. The proposed semi-interactive mode only drops you to a REPL when there is an uncaught exception.

Normally I want the interpreter to exit(0) after executing the script, but in case things go wrong a REPL would be handy, hence the proposal.

python -m pdb -m mymodule.main works just fine.

Cool, didn’t know that’s possible! However, it seems that using debuggers can have a huge performance overhead, which somehow limits its adoption. Simply entering REPL is a lightweight alternative more suitable for production use.

As a researcher which deals with a lot of scientific code, I typically wrap the Python interpreter process in a long-running tmux session so that I don’t need to keep an active SSH session when the code is running. It is frustrating when I do a tmux attach -t 0, only to be greeted with a stack trace without sufficient information to tell what exactly went wrong. I don’t want to use a debugger if it slows down my code significantly since I have a lot of computation to do.

Additionally, developers won’t expect their code to be wrong, so nobody will use python -m pdb in production if it has a high cost. On the other hand, the proposed semi-interactive mode doesn’t have any obvious performance implications, so I think people will happily adopt it.

How do you expect to be able to inspect the state of the execution at the place where the exception occurred if the code is not run under a debugger that is able to rewind the stack? Without the debugger, the only thing the interpreter can do is to give you access to the outermost scope, which is most likely completely uninformative regarding the reason of the exception (it is the analogous to having only the first line of the stack trace in the exception report).

If having access to the outermost frame is all you need, you can just change this:

def main():
    ...
if __name__ == '__main__':
   main()

into

def main():
    ...
if __name__ == '__main__':
    try:
        main()
    except Exception:
        breakpoint()

That’s a good point. We cannot know whether an exception is uncaught until it propagates to the outermost scope, but by that time we must rewind the stack to access the variables at the place where the exception was originally thrown…

See last_type, last_value, and last_traceback in the sys module. They’re used by post-portem debugging, e.g. import pdb; pdb.pm().

If you’re running your code under an IPython kernel (either directly or via QtConsole, Jupyter, Spyder, etc), you can jump immediately to the exception site post portem either automatically whenever one occurs with the %pdb magic, or afterward with the %debug magic. In Spyder at least, you can launch a kernel on the remote machine (which shouldn’t die if the SSH session closes), and then connect to it on your local Spyder, and use it as if you were connected locally, including the debug functionality mentioned above. Not sure if something like that would fully meet your requirements as it might be too high level for what you’re looking for, but given these are all scientific-oriented tools I figured I’d mention them.

1 Like