To get back on track, I think the question we are trying to answer at this point is whether Source distribution format - Python Packaging User Guide should have a line saying something like, “sdists SHOULD include the files necessary to run the distributions test suite and to build its documentation.”
I’m +1 on that change.
+1 from me as well.
FWIW, I’m +1 as well.
It seems like the discussion further above agreed that the onus should be on the package author, not the package build tool. I think the change should then be: “distribution authors SHOULD include the files necessary to run the distribution’s test suite and to build its documentation in the sdist.”
Every time I update our apps’ dependency pins, pip-tools has to download any new version’s sdist (and all wheels), as well as all current internal packages’ sdist, to recompute the hash. To be fair, that’s most definitely a limitation of pip-tools, but it is a pain point, so much that I explicitly remove tests from the sdists
Really? The hash is available from PyPI itself, so I’m surprised it does this. I would assume the download is to recompute dependencies based on the metadata (at least until PEP 658 get implemented).
You’re right, it gets the hash of packages from PyPI, but refuses to use the hash provided by our internal index
At least in terms of the sdist specification, I don’t think it needs to be prescriptively specific how the files get added to the sdist, only that they should be in there; and indeed that wasn’t the gist I gathered from our discussion, in that tools should make it easy to “do the right thing” by default. Furthermore, that framing doesn’t really fit with the rest of the spec, which is defining the format and structure of a sdist, rather than offering recommendations to package authors. Finally, given the overwhelming majority of the latter are unlikely to read the spec itself, direct advice targeting them should be in the packaging guide, not in the spec, which just establishes the basis for such.
Agreed. Adding a SHOULD recommendation to the spec would mean that tools building sdists should do this, not that users of those tools should configure their projects that way.
If the intent is to advise users how to configure their projects, that’s something that should probably go in the Packaging Python Projects section of the guide.
Although I always include the tests in the sdists on my personal projects (via setuptools-scm), this is an important aspect that I never thought about…
I have the impression that currently there is no tooling that will allow easily running the tests from the sdist right?
Does it mean that even when the developers distribute their tests, the fact that the tests themselves work is purely accidental, because the majority of people is not running the tests from the sdist?
In Jupyter we have a convention to use pytest and have a test extra, and we recently started to use a common set of actions to build and test sdists on GitHub Actions.
from me only if either:
Agreed. I thought a lot about dependency groups for the Hatch rewrite and became confident that they only should be used for features. Deps for testing, docs, etc. should not be there but rather should be part of config for an environment manager
I also think sdists should be the source of (code) truth because they can always be made to accurately reflect the project at a particular version whereas you cannot do the same easily with a Git checkout if you have a private repo, a monorepo, extra build steps, etc.
Hatchling does this as well but to be more precise: all non-VCS-ignored files
That is critical to include when building wheels from sdists if that influences the backend on what to ship
If the backend refers to .gitignore (even when run from a plain directory, not a git checkout) then yes, sure. I was talking about cases where the backend doesn’t use the file, and so it’s irrelevant if you’re not in a git checkout.
, agreed
The issue with this is twofold, as I describe above: there are number of different environment managers and no standard way to invoke them, and even if there were, using an environment manager directly conflicts with the primary purpose of downstream testing (the main motivating case for including such tests), which is to test whether the package functions as intended when built and integrated into the downstream’s environment, not whether the package code functions in isolation (which is the domain of the upstream).
Without either standardized test, doc, etc. extras, or some other yet-to-be-proposed and implemented mechanism, downstream users that want to build the docs or run the tests have to manually or programmatically determine which environment manager is being used and where its config is stored, determine the appropriate job/env, and extract the docs/test deps from it (which is particularly complex for executable config formats, like noxfile.py), instead of simply reading it out of the static package metadata or leveraging pip or other tooling that understands such.
While introducing a new PEP 517-like mechanism to retrieve dependencies and invoke the project’s doc, test, etc. backends would be ideal, that’s a rather complex process, whereas simply standardizing test, doc, etc. addresses the first half and perhaps at least some of the second, with a minimal amount of added specification or implementation complexity or requiring any new tooling support on either end.
What baffles me about this statement is the presumption that the only thing downstream needs is the list of dependencies. Even with those extras, they still have to look at the config to see what exact command to use to run the tests, what options need to be passed to that command, any environment settings needed (env variables, run from a particular working directory, etc) and so on.
This means that the benefit is extremely small for the cost of using a feature that’s normally used to indicate optional runtime features for development tasks. I’d much prefer we keep runtime and development-time separate. I thought we learned this lesson with setuptools and pkg_resources, where it took years to disentangle the runtime support parts of setuptools from the development-time part…
I was typing a response but this articulates my position more succinctly.
Recommending what to include or standardizing a broad default inclusion sounds reasonable, but anything beyond that fails my cost-benefit analysis.
include all VCS-tracked files in the sdist by default
all non-VCS-ignored files
I think you’re talking past one another… files ignored by the VCS
are not tracked by the VCS, so it seems like you’re in violent
agreement on this point.
No, you can explicitly track ignored files (at least with Git) so there is a meaningful difference there.
@ofek was just stating more precisely how Hatch/ling handles this, and there is a distinction—there can be untracked but not ignored files that aren’t in the repository, and also normally-ignored files that can be in force-added to the index and the repository. Furthemore, for “files ignored by VCS”, the .gitingore file as the source of truth, whereas for “files tracked by VCS”, the repostiory is, which means the former works even when not run from a repository, e.g. sdists so long as the .gitignore is packaged (a point which I believe was made above). This helps get around the issues with build tooling that uses the latter not working as consistently as indeed when not run from a repo checkout.
there can be untracked but not ignored files that aren’t in the
repository, and also normally-ignored files that can be in
force-added to the index and the repository. Furthemore, for
“files ignored by VCS”, the.gitingorefile as the source of
truth, whereas for “files tracked by VCS”, the repostiory is
Oh, this is a great point, thanks. I did indeed conflate the two
ideas. For probably 99% of cases, patterns are added to .gitignore
specifically to keep people from checking those files in, but that
doesn’t mean it can’t be bypassed or simply discouraging applying
changes to listed files which are actually tracked.
As for which I’d prefer my sdists to include/exclude, it’s hard to
surmise. I’m one of those boring folk who simply don’t check in
files listed in .gitignore and don’t add things to .gitignore if
they’re already tracked (or git rm them when adjusting the ignore
list).
It’s not necessarily accidental, but more about researching as to how to run the tests. So things like RPM I believe let you specify how to run the test suite, so whomever repackages the Python project has to look that up.
I view it as a 3-step process:
test extra)I’m not sure if people view steps 1-2 not useful w/o 3, or if step 3 is a separate thing but isn’t useful w/o steps 1-2.