Currently Tools/Scripts/pydoc3 has a shebang line of /usr/bin/env python3, but Makefile.pre.in modifies it to /usr/bin/python3.10 (or whatever equivalent for the version). This means that when someone creates a virtual environment with python3 -m venv, pydoc3 fails to provide documentation for any non-standard libraries installed in the venv.
This is probably not a big deal since python3 -m pydoc provides the correct functionality, but clearly someone before me thought it was worth making the shortcut in the first place. It’s a one-line fix, and the alternative of subclassing venv.EnvBuilder just seems a bit extra…
Tools/Scripts/pydoc3 is the source for the pydoc3 script that gets installed alongside python3 in the target dir.
Our makefile rewrites its shebang to be a specific path, which is good for altinstalls and downstream redistributors (who typically offer a few parallel versions of python and a main one).
I thought it was a mistaken decision. I do understand the recommendation for people to use python3.x -m pip, the general appeal of -m scripts, but regret that we lost perfectly fine functioning scripts for other operating systems (and non-beginner users who are not confused about their python installations and PATH).
I still find myself regularly trying pydoc thing or pydoc3 thing in an activated venv, then having to correct to the longer, less convenient command. (python3 -m venv is fine since I rarely run it directly.) It would be great to see pydoc in $venv/bin, next to python and pip.
To stop rewriting the shebang in the main pydoc script would be another way to solve this, but I am not sure of all implications.
I was also going to say that redistributors would probably continue to rewrite shebangs, but was surprised to see (on debian) that the pdb, pdb3.11 and pdb3.12 scripts do use /usr/bin/env python3 (incorrectly for the versioned scripts, but that’s another issue). So maybe there is hope!
For pydoc specifically, or do you think that other modules which can be run using -m should also have shortcuts? It seems excessive, especially for venvs, as perhaps relatively few users would actually use e.g. pydoc in a venv. It seems that those who want to could easily set up an alias to do this, or a pydoc.cmd on Windows. One alias or pydoc.cmd could then serve for all venvs on the system. And on Windows, I don’t believe we currently install pydoc as a runnable script, anyway.
True, I could do that, but it would be nice if I didn’t have to! I run pip and python -m pdb in virtual envs, and I still think of pydoc as a top-level utility (a script), not a hidden feature (stdlib module with -m like json.tool).
Well, this is a bit of a self-fulfilling prophecy. Reading this topic, I realised I have basically stopped using pydoc since it’s not picking up the packages in venvs.
I can’t remember the last time I used pydoc (I think it was back in the Alta Vista days).
Always assume other people develop differently than you and won’t be surprised anymore. (And I’m only half joking; you wouldn’t believe the various ways people do development that you would never think of that I have come across since the Python extension for VS Code came to Microsoft and I was put on the team for it.)
No, what I mean is the need to actually have it installed in every venv, rather than using an alias which would work in every venv and installed Pythons too. An alias is a one-time setup (per machine), after all. Everybody’s workflow is different - some regularly use pydoc, others regularly use some other module which can be invoked via -m. It’s not IMO the job of venv to ensure that all these workflows are catered for in a no-work-required-by-the-user (“nice if I didn’t have to”) fashion.
If typing python -m pydoc is so much more work than typing pydoc or setting up a one-time alias that you stopped using pydoc, how much value was pydoc really giving to you? Rhetorical question.
I was mostly reflecting on the point that I stopped using pydoc without even realising it. As you so kindly point out, it’s not a lot of effort, but it’s not surprising that a small quality-of-life tool like pydoc sees less use when there’s a barrier to entry.
What’s the “one-line fix”? On my system (macOS, framework install) I have multiple versions of pydoc installed with different versions in their name (e.g. pydoc3.10 and pydoc3.11). Changing the #! line to be version independent would break this.
An alternative to changing the #! line is to install pydoc inside virtual environments. The impact on disk usage should be minimal.
The funny thing is that my first approach to the problem was to have the venv module just write a script to $VIRTUAL_ENVIRONMENT/bin/pydoc3, but it was pointed out that it would likely be better fixed at “whatever installs pydoc”, i.e. Makefile.pre.in, so I redid my changes appropriately.
then that will run the appropriate pydoc in an activated venv, won’t it, without actually having to install pydoc in the venv? On my machine, I set up this alias, and I can invoke pydoc XXX in activated venvs with different Python versions (including 2.x) and get documentation on modules that have it. None of those venvs have pydoc installed in them. Why is anything more needed? On Windows, you could have a one-line pydoc.cmd file with an equivalent one-line invocation @python -m pydoc %*.
Note that there are dozens of modules that can be run with -m in the stdlib, and some of them might be used more often by some people than pydoc. Does venv have to change to support them, too? See what Brett said about workflows. And it’s not just about the disk space - it’s also about the maintenance/support effort. Note that there was an issue, linked by Éric above, which is why pydoc is no longer installed in venvs.
I wasn’t sure of your exact environment when I suggested that, and I didn’t specifically point to Makefile.pre.in - I assumed some script was installing pydoc as an executable in the venv. Changing Makefile.pre.in doesn’t seem to work for all environments, does it? On Windows, I’ve installed Python using python.org installers, and I don’t seem to get a runnable pydoc script at all. So even installing pydoc into system Pythons seems platform- and perhaps environment-specific, at least with default options for installation.
(And I’m only half joking; you wouldn’t believe the various ways people do development that you would never think of that I have come across since the Python extension for VS Code came to Microsoft and I was put on the team for it.)