I wanted to start a similar discussion and I’ve found this topic. FWICS there hasn’t been any real progress made here, particularly the recommendation doesn’t seem to have eventually made it into the guide.
I am one of the more active developers responsible for the maintenance of Python packages in Gentoo, and I’d like to add a few data points. Downstream testing was always at heart of Gentoo, and its specificity as a source distribution means that the vast majority of our users deal with sources and therefore are impacted by how these sources are distributed. On top of that there are some very strict restrictions — we need to be able to prefetch all the sources necessary for the build via HTTPS and no Internet access is allowed during the build.
For a long time, we were replacing PyPI sdist archives with GitHub generated snapshots as sources of our packages, primarily because of successive projects not including tests, documentation sources or other files in their sdist.
However, I should note that this is not limited to files intentionally omitted — only yesterday I’ve noticed that simplejson stopped including the C sources in their sdist. The modern build systems seem to have been focusing on a certain kind of tricky minimalism that sometimes causes files to be unexpectedly omitted. When most of the testing pipelines operate either directly on the repository or use generated wheels, and so do users, it’s easy to miss problems with sdist archives.
Using GitHub archives come with a number of issues. Just to list a few I’ve seen over the years:
- There’s no real guarantee that the archives will be reproducible. Only recently a major mayhem was caused when GitHub upgraded git and caused all archives to be generated differently. They reverted the change for now but we knew it was coming, and it will hit us again in the future.
- GitHub archives are often missing generated artifacts that are non-trivial to regenerate. Just to recall some problems we had recently: packages that required Cython 3.0 (which is a problem when Cython is installed system-wide and we’re not ready to have our users upgrade to alpha versions), packages that used npm to generate some assets (true horror to get working offline).
- setuptools_scm use gets problematic. For a long time, we’ve been working around it by setting
SETUPTOOLS_SCM_PRETEND_VERSION. Only recently someone noticed that this actually causes setuptools_scm to skip installing some package data files that would be installed otherwise (e.g.py.typedfiles in some packages). We still haven’t been able to estimate how widespread the problem with our packages is.
All things considered, I think using GitHub archives to workaround the problem of sdists missing files that we need is not a good long-term solution, and we really need to start urgently looking for a better solution. Ideally, one that doesn’t require every downstream packager to repackage everything.
Personally, I’m not convinced that a recommendation would be sufficient to solve the problem but having an official recommendation to use as an argument would certainly help the case.
An extreme option would be to extend PyPI to allow two different kinds of source distributions — “minimal” distributions that focus on size and include only files that are needed by the build system, and “full” distributions that include all files in the repository (or at least a useful subset of them).
In any case, I think we need to do something, and we need to do it urgently because GitHub archives just blew up in our faces, and we can’t just keep pretending it won’t happen again.