The Python Security Response Team received a report of a shell code injection vulnerability in the Tools/scripts/get-remote-certificate.py script. I decided to fix it: issue #97612.
Honestly, I don’t think that it’s still relevant to maintain such script in Python, and I would suggest to remove it in Python 3.12. For example, this one is a thin wrapper to the openssl s_client -connect HOST:PORT -showcert command. IMO using directly the openssl command is simple enough.
There are 74 Python scripts in Tools/scripts/. How many are outdated? The dutree.py comes with a documentation (an email) written in… 1992. Many scripts are short examples of what can be done in a few lines of Python to enhance some existing Unix commands. The which.py script can now be replaced with the shutil.which() function which has a better implementation.
I’m not sure how to proceed. Should we go through each script and decide if it should be removed or not? Or remove all of them and then wait for complains?
Some scripts are used to build Python, like check_extension_modules.py and deepfreeze.py. Maybe they should live in a separated directory? Is win_add2path.py used by the Windows installer?
macOS installs these scripts
The Windows installer don’t install them (correct me if I’m wrong)
On Linux, ./configure && make && make install installs them, but many Linux distributions package these examples in a separated package which is not installed by default (ex: python3-examples on Debian)
If some scripts are useful, maybe the code should be moved into the stdlib? For example, should diff.py becomes python3 -m difflib CLI? Currently, python3 -m difflib runs a self-test, IMO such test should belong to Lib/test/test_difflib.py instead.
Or should we move these scripts to a separated project/Git repository?
LOL diff.py, let’s just get rid of that while also moving -m difflib test code into a unit test. No need for -m difflib to do anything. We’d have to support it’s use as a diff tool forever I’d we did that. Every command line system has a diff tool already.
Just as a kind of data point, a “mature” (20+ yrs) project (written in Python) I work on has been moving old snippets of this nature off into a separate repository of unsupported tools and tricks, and out of the main repo where it might end up getting distributed with the project and considered something that needs supporting.
I would have no objection to also removing them from our macOS installers. I added them years ago as a convenience to users but deliberately did not make them very visible. I don’t recall ever seeing any feedback from users; I doubt anyone would miss them there. And if anyone does, I suppose they could be bundled up as a PyPI-installable package.
I think we should remove everything from those directories that isn’t needed to build. I don’t know how to tell what that is. For sure the Tools/demo directory can go – there’s even a demo of remote execution there!
There are also some benchmarks (ccbench, iobench, stringbench) that probably ought to be moved to pyperformance so they can be used and maintained.
I suggest any of us who actually use these tools (aside from the ones invoked by build scripts which are thus clearly referred to by our build systems) edit them to add a top level comment with our name and date describing what we use it for and why. Consider it a freshness indicator. We don’t need to be shipping/installing most of these, but that at least helps indicate which subset are owned and used.
At worst we can write up the list of files on a whiteboard at the core dev sprints and people can erase the files and directories they want to keep. That might the fastest way to get consensus on the list of things to drop.
File a GitHub issue with the current list as check box bullet items and let all of us committers modify the checkboxes for a week. That way it is global.
It’s more complicated than what I expected: diff.py and ndiff.py are copied in the difflib documentation. I will propose a separated PR for these scripts.
“win_add2path.py” is not used by the installer. I don’t know whether any project uses this script. PR 1645 improves it in several ways, but this PR has been open for over 5 years without getting merged. The one thing this script does that the installer doesn’t do is add the user scripts directory to PATH. That’s for --user installations, which pip enables automatically if it isn’t permitted to install to the system site packages.