# Post and developmental release version specifier handling in distros **URL:** \ **Category:** Standards\ **Created:** [November 11, 2023, 4:50pm UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461 "2023-11-11T16:50:46Z")\ **Posts on this page:** 20\ **Page:** 2
**Author:** ![pf\_moore](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pf_moore/32/35_2.png) [@pf\_moore](https://discuss.python.org/u/pf_moore)\ **Post date:** [November 14, 2023, 10:30am UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/21 "2023-11-14T10:30:24Z")
Python version caps don’t work that way, because the resolver will simply pick 0.1.4 over 0.1.4post1 on Python 3.12. There’s lots of discussion on why version caps are bad - you should be able to find discussions reasonably easily. I did a quick check on PyPI data - of the released versions of all projects on PyPI, about 1% are “post” versions. And 6% are “dev” versions\[1\]. So removing those version segments would have a significant impact. * * * 1. Yes, that seems high. From what I can see, releasing pre-alpha releases with `.dev` version numbers is not uncommon. ---
**Author:** ![ntessore](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/ntessore/32/10786_2.png) [@ntessore](https://discuss.python.org/u/ntessore)\ **Post date:** [November 14, 2023, 10:34am UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/22 "2023-11-14T10:34:01Z")
> [@pf\_moore](#): > > Python version caps don’t work that way, because the resolver will simply pick 0.1.4 over 0.1.4post1 on Python 3.12. There’s lots of discussion on why version caps are bad - you should be able to find discussions reasonably easily. I am aware of these — the maintainer would currently also have to yank every uncapped release for the post-release to make sense. My question was intended to clarify whether post-releases are an “acceptable“ thing or a “good” thing/best practice. ---
**Author:** ![fungi](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/fungi/32/5963_2.png) [@fungi](https://discuss.python.org/u/fungi)\ **Post date:** [November 14, 2023, 2:21pm UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/23 "2023-11-14T14:21:51Z")
The main use case I’ve seen for “post” versions is when you need to upload a rebuild that doesn’t involve changes to the project’s source code (and so can’t logically be a separate real release). For example, you want to link some C extension against newer library headers, or rebuild with a newer build backend dependency in order to fix some bug in the packaging of the associated release. I’m dubious about the use case for adjusting metadata since I always consider that to be fundamentally part of the source code still, but I suppose not everyone does. For downstream repackagers, the “post” release ought to be fundamentally identical to the original release since they’re going to be rebuilding it all again anyway, it’s more for the benefit of people installing packages directly from PyPI. But of course, whether that’s how everyone _actually_ uses “post” versions is the real question. ---
**Author:** ![jamestwebber](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/jamestwebber/32/12799_2.png) [@jamestwebber](https://discuss.python.org/u/jamestwebber)\ **Post date:** [November 14, 2023, 3:26pm UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/24 "2023-11-14T15:26:45Z")
> [@vishwin](#): > > which apparently isn’t strong enough. The benefit lies in external consumers of this ecosystem that have to consider a case in a general way that is oddball almost anywhere else. I think you have misread the text from the PyPA. They aren’t strongly discouraged _in general_. They are strongly discouraged _for bug fixes_. > [@vishwin](#): > > Special casing every individual package with these version segments is not a good way when such cases exist systemically. Definitely agree with this…given that it’s part of the spec, it seems like the best solution is to support that spec. ---
**Author:** ![vishwin](https://avatars.discourse-cdn.com/v4/letter/v/ecb155/32.png) [@vishwin](https://discuss.python.org/u/vishwin)\ **Post date:** [November 14, 2023, 3:46pm UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/25 "2023-11-14T15:46:19Z")
Anything, even metadata fixes, can be considered a bug fix depending on who you ask, especially those outside _this_ ecosystem. ---
**Author:** ![effigies](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/effigies/32/35835_2.png) [@effigies](https://discuss.python.org/u/effigies)\ **Post date:** [November 14, 2023, 3:56pm UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/26 "2023-11-14T15:56:47Z")
> Both `.post` and `.dev` are supposed to be semantically newer than their referenced final release, `.postN` is semantically newer than final, but `.devN` is oldest of all: ```python >>> from packaging.version import Version >>> sorted(["1.0", "1.0a", "1.0.dev", "1.0b", "1.0rc1", "1.0.post2"], key=Version) ['1.0.dev', '1.0a', '1.0b', '1.0rc1', '1.0', '1.0.post2'] ``` Could you make the following mapping? | PEP440 suffix | FreeBSD suffix | | --- | --- | | .devN | .dN | | aN | aN | | bN | bN | | rcN | rN | | .postN | pN | By the rules posted above, I think you would achieve the same ordering. If you have build numbers, another approach could be to use the following mapping: | PyPI version | Distro version | | --- | --- | | x.y.z | x.y.z build 1 | | x.y.z.postN | x.y.z build N+1 | If you need space to have multiple builds of the same release and don’t want to keep a table of paired versions/build numbers, you could use `(N + 1) * 100 + sub-build`. | PyPI version | Distro version | | --- | --- | | x.y.z | x.y.z build 1 | | x.y.z.post0 | x.y.z build 102 | | x.y.z.post0 | x.y.z build 103 | | x.y.z.post1 | x.y.z build 204 | ---
**Author:** ![vishwin](https://avatars.discourse-cdn.com/v4/letter/v/ecb155/32.png) [@vishwin](https://discuss.python.org/u/vishwin)\ **Post date:** [November 14, 2023, 3:58pm UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/27 "2023-11-14T15:58:11Z")
> [@fungi](#): > > For downstream repackagers, the “post” release ought to be > fundamentally identical to the original release since they’re going > to be rebuilding it all again anyway, it’s more for the benefit of > people installing packages directly from PyPI. But of course, > whether that’s how everyone _actually_ uses “post” versions is the > real question. For us distros, we update to every newest released version (subject to manpower availability) as we are alerted. Even if a post-release may be fundamentally identical to the referenced final release, as long as a release newer than the final exists, we update. Distro maintainers and our userbases by and large do not care about the nuances we are discussing here, it’s just “new version available, please update”. Additionally, in other non-Python projects that employ similar post-release version notation like sudo, such _do_ include bug fixes in the code. ---
**Author:** ![vishwin](https://avatars.discourse-cdn.com/v4/letter/v/ecb155/32.png) [@vishwin](https://discuss.python.org/u/vishwin)\ **Post date:** [November 14, 2023, 4:45pm UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/28 "2023-11-14T16:45:30Z")
When specifying as the “distribution version”, the PEP 440 suffixes `aN`, `bN` and `rcN` are already supported and mapped to our `.aN`, `.bN` and `.rN` respectively. In fact, the existing logic for “distribution version” mapping treats any trailing letter sequence as a pre-release and converts to `[DOT][first letter]`. As these three are pre-releases, they need to have the `[DOT]` on our end to be semantically older than a final release. I apparently missed the part where `.devN` is semantically older, so in this case the existing logic already fits. However, problems still arise with `.postN` and the `.postN.devM` combo, which cannot be specified as a “distribution version”. They must instead be specified as “port version” which is not mapped/converted, but already follow our conventions. In this case, the challenge is mapping “port version” to the Python version specifier to reference the correct sdist and generated bdist names. In my prototype implementation, this involves blanket overriding the “distribution name” (here, the sdist filename) variable. I especially don’t like this part, since “distribution name” is a generic variable for individual ports’ exclusive use, and is needed for some individual Python cases where mixed-case package name propagates to the sdist and generated bdist despite being semantically identical. We don’t do build numbers. ---
**Author:** ![sirosen](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/sirosen/32/8478_2.png) [@sirosen](https://discuss.python.org/u/sirosen)\ **Post date:** [November 14, 2023, 7:51pm UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/29 "2023-11-14T19:51:29Z")
> [@vishwin](#): > > Additionally, in other non-Python projects that employ similar post-release version notation like sudo, such _do_ include bug fixes in the code. I’m curious, what is your solution in this case? (I’m not familiar with `sudo`’s versioning, living the charmed life of a user who mostly trusts his platforms’ package managers to update _that tool_ appropriately and correctly 😉). * * * I don’t see a realistic solution here if you don’t have some part of the package version which is reserved for the redistributor. For example, let’s set aside the proposals to add a mapping scheme and instead consider how you handle these issues if you add a specifier for the redistributor’s version, separated by a hyphen: ```auto 1.2.4 # no distro version, default to 0 1.2.4-0 # semantically identical to the above 1.2.4-1 # successor to the above ``` Then every post release can bump the number. If you want to ship a dev package (not sure that’s wise but 🤷), you can start at `-0` and simply increment with each release. Python code which knows it’s own version still behaves appropriately, not knowing about the “distro version” field. Key to all of the above is that it’s an _additive_ change which supports all current use-cases. We don’t have to try to remove something which people are already using (and yes, people use post versions, albeit rarely – isort comes to mind as a place where I saw one recently, maybe last year). I would also note that having distro-package owned version into gracefully handles cases like a bad build/repackaging, simply by bumping the distro number. ---
**Author:** ![vishwin](https://avatars.discourse-cdn.com/v4/letter/v/ecb155/32.png) [@vishwin](https://discuss.python.org/u/vishwin)\ **Post date:** [November 14, 2023, 7:52pm UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/30 "2023-11-14T19:52:57Z")
I just spoke to the package manager maintainer, and this will not be considered for support. Responses from the rest of the community range from “why should we care” to “this is dead stupid”. In light of this, I am considering enacting a policy in our distro prohibiting the distribution of `.post` releases. ---
**Author:** ![vishwin](https://avatars.discourse-cdn.com/v4/letter/v/ecb155/32.png) [@vishwin](https://discuss.python.org/u/vishwin)\ **Post date:** [November 14, 2023, 8:07pm UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/31 "2023-11-14T20:07:49Z")
> [@sirosen](#): > > I don’t see a realistic solution here if you don’t have some part of the package version which is reserved for the redistributor. We have such a revision parameter, but is reserved for distro use only. There are other cases that warrant bumping that number, mostly rebuilds after a dependency breaks API. The crux remains that `.postN` are _upstream releases_. The metadata needs to align end-to-end since it is used directly for fetching/extracting the sdist before build and extracting the generated bdist for staging/packaging. (Ab)using the revision parameter at best results in only ever using the original final release, and at worst unable to bump revisions for forced rebuilds else fetching the sdist fails. > [@sirosen](#): > > I’m curious, what is your solution in this case? (I’m not familiar with `sudo`’s versioning, living the charmed life of a user who mostly trusts his platforms’ package managers to update _that tool_ appropriately and correctly 😉). sudo is an isolated case where the upstream version is specified verbatim into the “port version” variable, which does not perform any conversion/mapping. Their version specifiers (currently 1.9.15p2) follow our semantic rules so no further processing is necessary. Not sure if there will be a solution. The general prototype implementation I have is very hacky, ugly and introduces other contentious trade-offs wrt the rest of our framework (outside the package manager), and based on community responses, am considering prohibiting the distribution of `.postN` releases entirely. ---
**Author:** ![sirosen](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/sirosen/32/8478_2.png) [@sirosen](https://discuss.python.org/u/sirosen)\ **Post date:** [November 14, 2023, 8:10pm UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/32 "2023-11-14T20:10:53Z")
> [@vishwin](#): > > Responses from the rest of the community range from “why should we care” to “this is dead stupid” I just want to note that you’re trying to bridge two communities – and I appreciate that you’re getting feedback from both sides which is less than ideal for you – but that these aren’t things we can meaningfully engage with in this forum. Python packages have their own versioning scheme. It isn’t a strict subset of your distro’s versioning scheme. We can pitch ideas for how to resolve this, but most, maybe all, will require changes on the distro end. I don’t see an easy resolution. Even if we all agreed to deprecate post-versions today, and go through the pain of getting rid of them, I don’t see any reasonable way to expect that Python packages versions will remain a strict subset of your valid distro versions. What if we have some new, well-justified idea for expanding our version spec in 30 years (a long time in software)? So will python give you the promise you really want? No. What do we do now? That’s the only productive path to pursue. ---
**Author:** ![vishwin](https://avatars.discourse-cdn.com/v4/letter/v/ecb155/32.png) [@vishwin](https://discuss.python.org/u/vishwin)\ **Post date:** [November 14, 2023, 8:29pm UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/33 "2023-11-14T20:29:36Z")
> [@sirosen](#): > > What do we do now? That’s the only productive path to pursue. Then it is becoming more clear that the productive path forward is for the distro to prohibit distributing `.postN` versions. ---
**Author:** ![jeanas](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/jeanas/32/8115_2.png) [@jeanas](https://discuss.python.org/u/jeanas)\ **Post date:** [November 14, 2023, 8:34pm UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/34 "2023-11-14T20:34:57Z")
Since you say that this is a problem for “distros” in general, do you have actual data on the extent to which this is a problem in distros other than FreeBSD? Petr mentioned that RPM handles it fine. What about APT, Pacman, YAST, Homebrew, MacPorts, …? ---
**Author:** ![fungi](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/fungi/32/5963_2.png) [@fungi](https://discuss.python.org/u/fungi)\ **Post date:** [November 14, 2023, 8:35pm UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/35 "2023-11-14T20:35:17Z")
> We have such a revision parameter, but is reserved for distro use > only. There are other cases that warrant bumping that number, > mostly rebuilds after a dependency breaks API. The crux remains > that `.postN` are _upstream releases_. The metadata needs to align > end-to-end since it is used directly for fetching/extracting the > sdist before build and extracting the generated bdist for > staging/packaging. (Ab)using the revision parameter at best > results in only ever using the original final release, and at > worst unable to bump revisions for forced rebuilds else fetching > the sdist fails. This is the crux of your dilemma, I think. The Python community is hosting “upstream releases” on PyPI but also treats PyPI as “a distribution” of consumable/installable packages, and so like your distribution, needs a way to indicate that some versions are basically the upstream release but slightly later, though not a full new release. By consuming packages from PyPI in order to create your own distribution’s packages, you’re essentially repackaging one distribution into another. Cue references to Gödel’s incompleteness theorems. ---
**Author:** ![vishwin](https://avatars.discourse-cdn.com/v4/letter/v/ecb155/32.png) [@vishwin](https://discuss.python.org/u/vishwin)\ **Post date:** [November 14, 2023, 8:38pm UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/36 "2023-11-14T20:38:38Z")
> [@jeanas](#): > > What about APT, Pacman, YAST, Homebrew, MacPorts, …? Arch/pacman does a [manual override](https://gitlab.archlinux.org/archlinux/packaging/packages/python-hidapi/-/commit/9b2426736d930a0b5f2f276bf4db49feebe84e57). This is exactly something completely undesirable for something systemic as promulgated in the version specifier standard. ---
**Author:** ![jeanas](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/jeanas/32/8115_2.png) [@jeanas](https://discuss.python.org/u/jeanas)\ **Post date:** [November 14, 2023, 8:42pm UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/37 "2023-11-14T20:42:12Z")
OK. What about the others? ---
**Author:** ![vishwin](https://avatars.discourse-cdn.com/v4/letter/v/ecb155/32.png) [@vishwin](https://discuss.python.org/u/vishwin)\ **Post date:** [November 14, 2023, 8:52pm UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/38 "2023-11-14T20:52:48Z")
The sources are not always fetched from PyPI. As long as the metadata (`pyproject.toml` mostly) specifies a package name and version, or something like `setuptools_scm` constructs a version dynamically, we have to match it, end of story. If the version field, specified or constructed, contains a `.postN`, that’s the version. ---
**Author:** ![vishwin](https://avatars.discourse-cdn.com/v4/letter/v/ecb155/32.png) [@vishwin](https://discuss.python.org/u/vishwin)\ **Post date:** [November 14, 2023, 8:54pm UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/39 "2023-11-14T20:54:52Z")
Gentoo/portage also does a [manual override](https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-python/outcome/outcome-1.3.0_p0.ebuild?id=0913d55376f6ecdf5d07c23d11a4495466e75eb1). Though, I’m not sure how far they go wrt end-to-end metadata alignment. I know Arch/pacman just wildcards the bdist file passed into `installer` so no metadata alignment at that end. ---
**Author:** ![sirosen](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/sirosen/32/8478_2.png) [@sirosen](https://discuss.python.org/u/sirosen)\ **Post date:** [November 14, 2023, 11:24pm UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/40 "2023-11-14T23:24:16Z")
I am struggling in this thread with a lack of knowledge and context. I suspect others are as well. You say you have to match the version number exactly here, as a logical conclusion. For me, without your familiarity with FreeBSD packaging, that feels like a big logical _jump_. 1. I don’t think I followed why `sudo`’s versions can be coerced correctly into package versions, but Python post releases cannot be so-coerced in the same way. You said something about both systems following the same semantic rules. Can you explain what you meant a little more? 1. There is a version number already reserved for the distro, but you’ve said it’s off-limits to update it with a post release. Can you explain why at all? It sounds to me like a perfect solution. Sure, it may mix updates to the source with updates to the package build into a single stream, but so what? It doesn’t need to be perfect on that account. What is the core problem with using this field? * * * There may be no solution here if FreeBSD is unwilling to budge, but I find that a little unlikely. There’s probably no solution without compromising on your goals or vision for this integration a bit though. (And yes, I put it on FreeBSD being unwilling here. That I don’t think the Python community should _remove_ features which have been in active use for ~10 years is more or less a given. I use dev releases and I’m not the only one.) [Previous page](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461.md?page=1) [Next page](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461.md?page=3)