Tutorial about Python Packaging offering different use case (Request to Review)


I would like to point to my Python Packaging Tutorial explaining several common use cases using minimal demo projects.

I am not an expert and assume that some of my solutions might not be the best. So I would appreciate if you can review them.

Background of that project: I am member of FOSS maintainer team and prepare the migration of Back In Time from a makefile based build-system to a modern Python Build system using pyproject.toml & Co. To explore some expect able problems and possible solutions I created this minimal examples. I also do plan a tutorial repo about Debian Python Packaging using the same approach with minimal examples illustrating different use cases.

Thanks in advance,

PS: Posted this without response also on python-list@python.org and r/pythoncoding and several other places. I was pointed to this discussion forum from the setuptools-people.

On the topic of internationalization, you have basically two approaches: either you add code to compile the MO catalogs during the sdist → wheel build, as you did, or you pre-compile them and add them to the sdist. I’ve personally used the second approach as it felt a bit simpler.

The first approach might be more elegant, but in that case I’d recommend to use babel in order to not unnecessarily require a Unixy tool on the target system where the sdist is installed.

I also see that you are mentioning python3 -m pip install .. Done outside of a virtual environment, this is not recommended, and will error out by default on recent versions of Debian, Ubuntu and other distros, since it can shadow Python packages installed by the system and thus break system packages written in Python. Either use a virtual environment, or install with pipx instead of pip. See Externally managed environments in the packaging user guide.

In the demo 5, you might want to link to Distribution package vs. import package in the packaging user guide, merged this morning :slight_smile:


Dear Jean,
thanks for your feedback.

Sorry, but I don’t understand your second approach. How do I pre-compile? You mean this as a manual step before running pip(x)?

The GNU gettext tools are also available for Windows & Co. But I wouldn’t get so far in this tutorial and make it to complex. But I mentioned polib in a code comment in the hatch_build.py file.

IMHO it is more preferable using OS tools instead of increasing Python-package dependencies.

I am aware of PEP 668 and always trust the Python folks when the come to a decision no matter if I understand it or not. But it is new to me and I still struggle with it and don’t know how to integrate that into my workflow. I will open a separate thread here about that.

1 Like

Please see my separate thread about PEP 668.

Yes (or before running python -m build or hatch build or similar.)

As you wish. Note, though, that Gettext needs installation on macOS and is, as far as I know, not installed by default on popular Linux distros either, so this isn’t really saving a dependency, it’s trading a Python dependency for an OS one.

Issues I see with your tutorial as I come across them:

  • Here, using helloworld-cli with a hyphen for the name of an import package is a bad idea that encourages bad practices. While it’s not a dealbreaker for a package that only exists to provide a command, having a hyphen in the name of a library package means that import package-name in code that uses the package won’t work, as hyphens are not allowed in Python identifiers. Import package names should always be valid Python identifiers, so change that hyphen to an underscore.

    Note that the spelling of the import package is independent of the spelling of the distribution package and the spelling of any commands the package installs, so even if you rename the folder to helloworld_cli, you can still have the project name set to “helloworld-cli” with a hyphen, and the package can still provide a command named “helloworld-cli” with a hyphen.

  • “The solution is the Development Mode …” — Development mode is a solution, not the solution. The general solution is to install your package inside a virtualenv — either using editable mode or not — and do your testing inside the venv. The only advantage of editable mode over non-editable is that changes to the source code are reflected in the environment immediately without needing to reinstall[1], but if you’re using tox or nox or similar to run your tests, your package will get reinstall on each run anyway, so there’s not much benefit to editable mode.

  • I would advise eliminating the “Start application ‘as root’” section entirely. Installing Python packages as root should not be done lightly, and it’s better to keep that idea out of beginners’ heads in the first place. You don’t even explain what you’re trying to accomplish by installing as root or why it’s necessary for your use case in the first place.

  • “The file __main__.py does produce this output with meta data retrieved from pyproject.toml” — Technically, the metadata is retrieved from the METADATA file installed alongside the package, which is generated from pyproject.toml, which is not included when installing. By the time hellocentralcli is run, the pyproject.toml file is irrelevant.

  • PEP 643 & dynamic metadata are irrelevant to retrieving metadata with importlib.metadata. It would be better to just delete the paragraph about dynamic metadata entirely.

  1. Though adding, removing, or renaming files in your project’s source can require you to reinstall anyway even in editable mode ↩︎

1 Like

I understand your point. From my point of view the installation of gettext is a dependency that will be handled by the distros package maintainer. My users are intended to install my “packages” via their official distro repository. From upstream or from PyPi (where I do not publish) is vorbiden by “normal users”.

I fixed this in the README.md. The source itself is correct.

From my point of view the benefit is that I don’t need to deal with virtual environments, their side effects and redundant packages. But that topic is discussed in another thread.

I understand your point. But in the background of my real project (Back In Time) there is a real use case for this. That is why this example does exist.
The fact that this repo can be used by other persons as a tutorial is just a secondary side effect. The main purpose of that repo is to support the migration of the Back In Time project from its make-based build system to a modern Python build-system. As a backup application it needs to be able to get started as root.

The repo is GPL. Please feel free to re-use it somewhere else.

The “as root” example is not finished yet as the warning boxes trying to indicate. So I am open for alternative solutions reaching the same goals.

I am aware of that. The intention of the repo is not to give deeper understanding of the concepts. There are much better ressources for that and I try to link them.

Can you explain why? I gave my arguments about “dynamic meta data”.

Thanks a lot for your feedback. I will take that into account. And you gave me a lot of things to think about. I am still learning (and not surprised by that).

That’s not a benefit of editable installs. Editable vs. non-editable installs are orthogonal to virtual environments. If you’re not using a virtual environment, then pip install -e . and pip install . will both install your package system-wide, and both will let you test it with pytest.

If the reason for installing your package as root is solely so that you can run it with sudo[1], I would suggest instead installing as non-root (using pipx or pip install --user or a virtualenv) and passing the absolute path to the installed program to sudo, e.,g., sudo ~/.local/bin/my-command. Depending on how your sudo installation is configured, you may be able to instead preserve your PATH (and HOME?) when sudo-ing, letting you just do sudo my-command, but that may come with subtler security issues.

Then again, you’ve also mentioned that your users are intended to install your packages via official distro repositories, so you could just install the distro packages as root (which I believe is the default or only option for most distros) instead of using pip in the first place.

You don’t need to give deeper understanding, but you should at least not give incorrect understanding. importlib.metadata does not retrieve anything from pyproject.toml.

Because dynamic metadata has nothing to do with the section you’ve placed those arguments in. If you want to say things about dynamic metadata, have a dedicated section for it instead of tacking it on to the end of something unrelated.

  1. Is that why you’re installing as root? The README and your comments here aren’t clear. The README should be more explicit about this. ↩︎

1 Like

Pretty sure it’s bad practice to run apps as sudo when they can be modified by regular users. That’s a privilege escalation just waiting to happen.

Inside your own user directory means it’s just as secure as your own account, which presumably is also the one that gets used for random internet browsing, etc.

Best security practice (which not everyone needs) will be to require sudo to modify the apps that you’re going to run with sudo, and to not pass through environment variables referring to user-modifiable locations.


I really appreciate your contributions to this discussion and dealing with a stubborn one like me.
You gave me a lot new insides and things to think about.
It will take some time to come to a conclusion for my self but I will modify my tutorial repo.

1 Like

@wvxvw Thanks for chiming in but, perhaps unsurprisingly, I disagree with most of what you’ve written.

Please consider reading at least Packaging Python Projects - Python Packaging User Guide before providing more opinions here.

This is exactly how things work, if you use any of Python packaging tools other than the things we actively tell people to not do, e.g. directly do python setup.py install.

It’s, for example, exactly how wheel-based installs work and how pip would install packages from a source distribution.

Those definitions are taken from packaging.python.org and are a useful tool for unambiguously referencing the concepts. See Glossary - Python Packaging User Guide



This post is among other things some of the worst advice I’ve ever read, a disingenuous take on the history of packaging in the Python ecosystem, and a flat insult to the hundreds of hours of completely volunteer effort contributed in this space.

I’m glad you’ve found a workflow that works for you. Packaging in Python has a long and winding history. We all know it’s a mess that needs improvement, which is why this forum is filled with posts about it. If you wish to contribute, do us all the service of not actively misleading others.


Just a sidenote about “Demo 04” about run a GUI application as root via pkexec.

I opened an Issue about the whole topic.