Metadata 2.2 is specifically about putting statically generated metadata into the sdist wherever possible. Build backend processing that generates a version from something like version.attr or VCS tags runs when building the sdist, and the resulting static version is stored in the metadata of the sdist.
Most cases where people are reading pyproject.toml to extract static metadata would be better served by reading the sdist metadata (as long as it’s version 2.2 or later). In cases where that’s not possible, to be correct you have to call the build backend (or replicate the backend’s algorithm) when fields are declared as dynamic. Of course, for a tool not intended for general use, you can say that you don’t support dynamic data in pyproject.toml, but that’s not allowed for standards-compliant tools.
Yes, although I’d be surprised if there’s much incentive to make any of the other core metadata features dynamic. I just don’t see the point in a static, standardized file if it’s neither static nor standardized! And I avoiding mentioning it but since you ask, I use Docker a lot and being forced to install git and use a deep clone with tags into what used to be a nice little, quick to build, ~20MB container – all to get one tiny little version number when just putting whatever the modern equivalent of git tag "v$(python setup.py --version)" into a release script would have had more or less the same effect (yes, you loose the per-commit versions but I thought they were superfluous too) – makes me very sad. On the other hand though, there’s less potential for this to break on anyone but the packager (as long as they don’t roll their own way of doing it). So yes, I disapprove of using git tags for anything more than tagging things, but not so much for the same reasons as before.
You should install from sdist then, rather than from the source tree, as that is specified to be installable without a full clone. You don’t need to care how, the standards should handle that for you.
This is something that confused me from previous comments. If you install from sdist then isn’t the version literally in the filename (or at least the metadata therein)? Or is it somehow more complicated than that?
If you’re installing from git then it makes sense that some extra step is needed to get from git checkout to sdist. The extra step is also needed in e.g. autotools projects where you need to run say ./bootstraph.sh from git before you can even ./configure but that step is already done for you if you have the release tarball.
That’s basically it. There might be some complexity needed in the build backend to “know” that it should look at sdist metadata rather than wherever it normally gets the version from (I think setuptools_scm needs to have some flag set to tell it not to try to look at the VCS, for example) but that’s what I meant by “you don’t need to care how”…
Maybe I do not understand something crucial here, but the problem I encounter is that if a package is installed using “pip install -e .” then changing version will be immediately visible while there seems to be no trick to make importlib.metadata.version(NAME) update without doing another “pip install -e .”
As others have mentioned, the recommented way of using importlib may require a name which is different from what is actually getting imported.
On the other hand, it seems that for namespace packages, the recommended way is the only way to actually provide a version info for the package.
So I think both approaches are unfortunately flawed and suboptimal. I want the source code, not some metainformation at build/installation time to reflect the version of the source package. And I want this to work for everything including namespace pacakges.
if a package is installed using “pip install -e .” then changing version will be immediately visible
Thanks for that – that’s a good point that I had forgotten about !
Note that since this discussion paused out 2 months ago, the PyPA docs have been updated with clearer and modern discussion about handling version that discusses, but does not specifically recommend, using a __version__ string int he source.
Meanwhile, a draft PEP is in the works to revive the discussion about __version__, so this can be hopefully put to bed.
Hmm, I had thought (and hoped) that both sides of the make __version__ mandatory vs purge __version__ from existence fence had accepted that neither were going to get their way. The middle ground of just trying to steer people who want to set a __version__ towards the right ways of doing it seemed as good as we were going to get.
I certainly haven’t seen a consensus in favour of either position, so IMO any PEP picking a side should fail. And a PEP saying “you can do either” is pretty pointless. So I think that anyone working on a PEP should save their effort unless they can direct the discussion here to a point where there is a clear consensus.
@ChrisBarker-NOAA If you want me to be PEP delegate, and you don’t expect to see consensus, feel free to mark the PEP as rejected right now, to save some time
a PEP saying “you can do either” is pretty pointless.
Well, yes. But a PEP saying “IF you put a version attribute in your pacakge, you should do it this way”, is not.
I hope there’s consensus that some people using VERSION, and some using __version__, and some using MY_WAY_TO_SPELL_VERSION isn’t good for anyone.
unless they can direct the discussion here to a point where there is a clear consensus.
Consensus is HARD – we may never get there, but if there’s any hope, we need a clearly written out proposal to discuss – so writing it out is not a wasted effort.
But I’m not leading that effort – we’ll see what comes of it.
That’s the sort of thing that could be added to the packaging guide, if we have consensus. It doesn’t need a PEP. But OK, if someone wants to write a PEP, that’s fine.
If someone does write a PEP, saying that if you want to include a version attribute, you must do it in such-and-such a way, then that PEP needs to cover transition[1]. For projects that currently use a different convention, how do they migrate (maintain both old and new approaches in parallel, and if so, for how long?), how should consumers handle the migration (check for the new approach, and if it’s not present fall back to the old one?)
Personally, I think having a documented recommendation in the guide, and letting projects choose for themselves how and when to transition, is a much less disruptive approach. But sure, if someone wants to write a PEP to change my mind, then go for it.
because that’s a key part of what a PEP defines ↩︎
Given that there’s a draft PEP in the works, I’d say we should hold off on merging that specific PR since it’s clear that consensus has not been established here.
#1276 is just removing some questionable advice on old ways to set __version__ which is replaced by Chris’s already merged #1580 giving the less fallible methods to do the same thing. Contrary to the branch name, #1276 is no longer about purging __version__ from the face of the Python ecosystem.
I don’t know where the best location in the documentation is now, but for use cases requiring low startup time (CLIs and serverless functions) and knowledge of the package version I would love for us to document that persisting the version in source code is recommended.
Personally, I’m not convinced that the older page is that “questionable”[1] - and I’m definitely not sure that the new page (which is more general and much lighter on specifics) is unarguably an improvement. I’m very much in favour of @pradyunsg’s suggestion, that we hold off on any changes until there’s either an accepted PEP or a clear consensus (which we’re frankly very far from right now).
I don’t think that anyone is served by presenting as “official guidance” something that even the packaging community can’t agree on amongst ourselves.
It’s old, and rather too setuptools-specific, but that’s not the same as being wrong↩︎
If the community is divided on the content, can the community at least agree that having two pages with very similar names in two different spaces with one listed as TODO and the other listed as completed is a bad idea and will unnecessarily confuse people who are exposed to the packaging website for the first time (aka me)?
If we can agree on that, then can we merge the contents from one page to another?
As it currently reads to someone not deeply familiar with packaging and standards of the packaging community, the first page reads as outdated (because there’s a TODO acknowledging that there is stuff uncovered) while the second reads as “Here are some high level points, please refer to your specific backend’s documentation.” But having them in disparate places with a newer date on one of them implies more authority to the second one implicitly, even if unintentionally.
That’s a reasonable point. Unfortunately, the mistake was adding the second page without consensus that the old page should be removed, and we can’t get agreement on what to do now.
That’s the whole problem here. No, we can’t, because people won’t agree on what the wording on the merged page should be.
Your characterisation of the two pages exactly matches my opinion. Personally, I’d remove the newer page, and go back to the original (even with the caveats that it’s outdated) until someone can get the community to agree on a position. Alternatively, just delete both to avoid giving any impression that there’s an agreed “best practice”. I’m deliberately not suggesting that we document what I think is the best practice, as we’ll just get a bunch of people telling me I’m wrong, and the cycle will continue
The one thing I don’t think we should do is change anything. There’s no agreement on what form a change should take, so promoting anything as official advice would be misleading.
I’m sorry that this results in a bad experience for newcomers like yourself. But IMO continually changing the advice we give as people fight over what’s best is worse.
