Are python package developers just not supposed to package shell scripts as a general practice?

I am trying to move from setup.py to pyproject.toml, and the one thing I can’t move for one of my builds is a shell script.

In setup.py, you can have scripts= and install a shell script, but setup.py is deprecated in favor of pyproject.toml

The setuptools docs (Configuring setuptools using pyproject.toml files - setuptools 67.6.1.post20230328 documentation) say you can have script-files in pyproject.toml., but it’s deprecated.

As far as I can tell, there are no “future-proof” ways to package scripts, only python entry-points. Is this understanding correct?

Essentially yes. The Python packaging standards are focused on packaging Python code in a portable way - shell scripts are inherently platform-specific, and as such the current thinking is that they should be shipped using platform-specific mechanisms (.deb/.rpm files or similar) rather than Python wheels.

That’s a very strict viewpoint of course, and in reality you can get such things to work (the scripts keyword is deprecated, but still works at least for now), it’s just that you won’t necessarily find much explicit support for it.

And even if scripts is removed, there’s nothign stopping you from shiping a wheel (with a custom build backend, or manually put together) with the shell script in the .data/scripts directory. This is essentially what setuptools does anyway, and any installer (e.g. pip) will continue to support this for the foreseeable future.

We’ll want to keep supporting these in wheels long-term I think – this is also how wheels are used to distribute non-python binaries like ruff · PyPI

Maybe we should just document that scripts= is deprecated for Python scripts where you could use an entry point instead, and treat the deprecation as “this is no longer best practice” rather than anything that will ever stop working?

3 Likes

The only relevant standard is the rewriting of scripts that start #!python here (and even that’s only “recommended”). That rewrite is something I think installers should drop, TBH. But having a part of the wheel that gets installed into the “scripts” location is useful and should be something backends support.

Most of what the OP is concerned about, is specific to setuptools - the scripts= paramater and any deprecation warnings in the docs. IMO, setuptools should probably not be deprecating this quite so aggressively, but that’s for them to decide.

The goal is that the script will be run with the Python to which it’s been installed, not whatever’s first on the PATH (e.g. because of an active virtual environment), right?

… Isn’t that part of how pip works? Such that if you explicitly specify a pip somewhere else, it will install to its corresponding Python, not the currently-default one?

How else would you get that kind of behaviour, without the installer rewriting? If I ship a driver script with my library code, I’m pretty sure I’m going to want it to run with the same Python where the package was installed, even if the user invokes it directly rather than going through python -m. If the latter were the Right Way, why bother with a shebang and an executable bit at all?

If the script is in Python, define a console_scripts entrypoint instead. Such scripts are generated without rewriting, and thus more reliable.

2 Likes

:man_facepalming: I knew that. I’ve used it before. Oh well :slight_smile: