What controls the pdb history (the up arrow key)

Python Help
1

(Asked a question on Stackoverflow with no luck, so retrying here, apologies if this breaks some protocol)

To summarize, I’m looking to understand what controls the pdb reaction to the uparrow key to get the command history.

The context is that the pdb history works fine from the interpreter command prompt, but does work not when run through several layers of wrappers (pytest, pydevtool, doit, click). pytest --pdb works fine.

Clearly, the issue is somewhere in these layers of wrappers, and I’m willing to look through them — the question is what to look for!

The OS is a bog-standard ubuntu linux, if that matters.

There are some SO questions which point to readline or pyreadline or gnureadline packages — e.g., Python pdb command history not working on Windows — pip installing gnureadline did not change anything, and import readline seems to work.

To reproduce: drop a breakpoint somewhere

$ git diff
diff --git a/scipy/interpolate/tests/test_interpolate.py b/scipy/interpolate/tests/test_interpolate.py
index 1e22ff9c9d..f7500d0b5a 100644
--- a/scipy/interpolate/tests/test_interpolate.py
+++ b/scipy/interpolate/tests/test_interpolate.py
@@ -22,6 +22,10 @@ from scipy.integrate import nquad
 from scipy.special import binom
 
 
+def test_pdb():
+    import pdb; pdb.set_trace()
+
+
class TestInterp2D:

And run the pydevtool-based CLI:

$ python dev.py test -t scipy/interpolate/tests/test_interpolate.py 
💻  ninja -C /home/br/repos/scipy/scipy/build
ninja: Entering directory `/home/br/repos/scipy/scipy/build'
[2/2] Generating scipy/generate-config with a custom command
Build OK
💻  meson install -C build --only-changed
Installing, see meson-install.log...
Installation OK
SciPy from development installed path at: /home/br/repos/scipy/scipy/build-install/lib/python3.8/site-packages
Running tests for scipy version:1.10.0.dev0+2161.a637553, installed at:/home/br/repos/scipy/scipy/build-install/lib/python3.8/site-packages/scipy
================================= test session starts ==================================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/br/repos/scipy/scipy, configfile: pytest.ini
collected 131 items                                                                    

scipy/interpolate/tests/test_interpolate.py 
>>>>>>>>>>>>>>>>>>>>>>> PDB set_trace (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>
--Return--
> /home/br/repos/scipy/scipy/build-install/lib/python3.8/site-packages/scipy/interpolate/tests/test_interpolate.py(26)test_pdb()->None
-> import pdb; pdb.set_trace()
(Pdb) import readline
(Pdb) <press the up arrow>^[[A

This scipy issue has the original context: doit based dev interface garbles pdb command history (in some cases) · Issue #16452 · scipy/scipy · GitHub
And the SO question is python - what controls pdb history (uparrow) - Stack Overflow

Any pointers would be much appreciated!

2

I’m not sure if this answers your question.
When you run pdb on the normal interpreter, Pdb object waits user input in:

C:/Python310/Lib/pdb.py:342:
    def interaction(self, frame, traceback):

which calls

C:/Python310/Lib/cmd.py:98:
    def cmdloop(self, intro=None):

where the stdin is usually the sys.stdin and the python interpreter records the command history.

I sometimes run pdb in another shell e.g. wx.py.shell has its own interpreter <class 'code.InteractiveInterpreter'> object and pseudo-file-like stdin, waiting for user input in the main thread loop. It doesn’t have its own history list, so the up arrow won’t work unless you override it.

I don’t know how other 3rd party libraries deal with this.

3

The interpreter doesn’t handle this. If sys.stdin and sys.stdout are terminal/console files, and their file descriptors match those of the stdin and stdout file streams in C, then input() calls the C API function PyOS_Readline(). If the readline module is imported (e.g. interactive mode), assuming it exists, then it implements the core functionality of PyOS_Readline(). On most POSIX systems it uses GNU Readline. On Windows, the standard library doesn’t include a readline module, but a third-party pyreadline package is available.

If the readline module isn’t imported, and PyOS_Readline() isn’t otherwise hooked, then it’s implemented by PyOS_StdioReadline(). On POSIX, the latter is based on C fgets(), which has no support for command-line editing keys (e.g. home, end, cursor keys) or history.

On Windows, PyOS_StdioReadline() is based on WinAPI ReadConsoleW(). A console session in Windows implements command-line editing and an input history buffer for each attached application (e.g. F7 popup list, F3 completion, up/down history navigation), but history doesn’t persist across console sessions. Note that ReadConsoleW() is also used by sys.stdin if it’s a console file. In this case, sys.stdin.read() and sys.stdin.readline() support command-line editing and history, whereas on POSIX they do not.

1 Like
4

Thank you for pointing this out.
I see that the record of command history is on the OS side.

5

Thank you Eryk!

Just to close the loop, this explanation gave us a lead, doit based dev interface garbles pdb command history (in some cases) · Issue #16452 · scipy/scipy · GitHub