# Post and developmental release version specifier handling in distros

**URL:** <https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461>\
**Category:** Standards\
**Created:** [November 11, 2023, 4:50pm UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461 "2023-11-11T16:50:46Z")\
**Posts on this page:** 11\
**Page:** 3

<div class="post-metadata">

**Author:** ![encukou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/encukou/32/2461_2.png) [@encukou](https://discuss.python.org/u/encukou)\
**Post date:** [November 15, 2023, 12:34pm UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/41 "2023-11-15T12:34:04Z")

</div>

Let me write some more about the Fedora/RPM.  
RPM and Python version schemes are _incompatible_. When doing any kind of automation, we always keep in mind which system’s used, and explicit conversion (which is carefully designed so that equivalent versions keep the sort order). RPM versions are powerful enough that if the version is “simple enough” it’ll look the same in both systems. If it’s not simple enough, the version converted from a `pyproject.toml` entry will look weird, e.g. have double tildes in the middle and such.

It’s similar with package _names_, except the names converted from a `pyproject.toml` entry never look like regular package names – for example, _`python3dist(requests)`_, with parentheses. They’ll never conflict with a hand-picked RPM name.

Perhaps you need to design a FreeBSD/Python version mapping where equivalent versions don’t look the same.

---

<div class="post-metadata">

**Author:** ![vishwin](https://avatars.discourse-cdn.com/v4/letter/v/ecb155/32.png) [@vishwin](https://discuss.python.org/u/vishwin)\
**Post date:** [November 15, 2023, 5:20pm UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/42 "2023-11-15T17:20:44Z")

</div>

> [@sirosen](#):
>
> I don’t think I followed why `sudo`’s versions can be coerced correctly into package versions, but Python post releases cannot be so-coerced in the same way. You said something about both systems following the same semantic rules. Can you explain what you meant a little more?

sudo does not place a `[DOT]` before the `p`, which alone is the difference maker. Version comparison run:

```auto
% pkg version -t 1.9.15 1.9.15p2
<
% pkg version -t 1.9.15 1.9.15.p2
>
% pkg version -t 1.9.15 1.9.15.post2
>
%

```

The package manager maintainer will not budge from this logic, any letter in the beginning of a version segment (ie `[DOT]whatever`) is a pre-release and semantically treated as such. In the first example, the second version string’s last segment is `15p2`, but the latter examples’ last segments are `p2` and `post2` respectively.

> [@sirosen](#):
>
> You say you have to match the version number exactly here, as a logical conclusion. For me, without your familiarity with FreeBSD packaging, that feels like a big logical _jump_.

The sdist and generated bdist filenames are taken directly from the name and version fields on our end. We have two ways of specifying version: “distribution version” (preferred) and “port version”. “Distribution version” is the form present on the source archive and is mapped/converted into our acceptable form if necessary. “Port version” is the form present in the distro package and is where the converted/mapped result is stored if “distribution version” is specified. Only one form can be specified. Specifying `1.9.15p2` as “distribution version” results in “port version” `1.9.15.p2`, which is a problem. `1.9.15.post2` “distribution version” converts/maps to the same result.

The only good hack I’ve figured out so far is just that, and it is extremely ugly and unwieldy. Any Python package at a post-release would have to be specified as `1.9.15p2` “port version” to avoid automatic conversion/mapping, but then have logic in the framework to override the source archive/sdist name, namely the operation `s/p/.post/`. This bit is problematic, as the ability to override the source archive name is reserved at an individual case level, not meta. It also presents problems when sdist names are not normalised, most often mixed case names.

> [@sirosen](#):
>
> There is a version number already reserved for the distro, but you’ve said it’s off-limits to update it with a post release. Can you explain why at all? It sounds to me like a perfect solution. Sure, it may mix updates to the source with updates to the package build into a single stream, but so what? It doesn’t need to be perfect on that account. What is the core problem with using this field?

We use the revision parameter for our own bumps for forced rebuilds, metadata changes, etc. It is also not part of the default source archive filename format. Best illustrated with an example:

Given Python package `oxidised-1.0.post1`. With the thought of (ab)using the revision parameter, our metadata:

```auto
NAMEPREFIX=${PYTHON_NAMEPREFIX}
NAME=oxidised
DISTVERSION=1.0
REVISION=1

```

In this form, the `oxidised-1.0` would get fetched and operated on. To actually get the post-release, we have to override the source archive name parameter:

```auto
NAMEPREFIX=${PYTHON_NAMEPREFIX}
NAME=oxidised
DISTVERSION=1.0
REVISION=1
DISTNAME=${NAME}-${DISTVERSION}.post${REVISION}

```

Now, `oxidised` has Rust code. New Rust (toolchain) release drops, we need to force rebuild everything in our tree containing Rust code, so every such case’s revision gets bumped:

```auto
NAMEPREFIX=${PYTHON_NAMEPREFIX}
NAME=oxidised
DISTVERSION=1.0
REVISION=2
DISTNAME=${NAME}-${DISTVERSION}.post${REVISION}

```

This will now fail because `oxidised-1.0.post2` doesn’t exist.

The common theme here is individual manual overrides. Fine for isolated cases like sudo, not acceptable for an entire class.

Another failure mode is `oxidised-1.0.post0`. The revision parameter is always set to 0 by default, and 0 values are hidden.

> [@sirosen](#):
>
> There may be no solution here if FreeBSD is unwilling to budge, but I find that a little unlikely. There’s probably no solution without compromising on your goals or vision for this integration a bit though.

The solution may very well be prohibiting packaging of post-releases.

> [@sirosen](#):
>
> (And yes, I put it on FreeBSD being unwilling here. That I don’t think the Python community should _remove_ features which have been in active use for ~10 years is more or less a given. I use dev releases and I’m not the only one.)

I brought the longevity part up and only received more ridicule. Post-releases for metadata changes and not bug fixes in particular are perceived as poor release engineering practice.

---

<div class="post-metadata">

**Author:** ![effigies](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/effigies/32/35835_2.png) [@effigies](https://discuss.python.org/u/effigies)\
**Post date:** [November 15, 2023, 5:31pm UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/43 "2023-11-15T17:31:24Z")

</div>

Is there a place where this metadata configuration is documented? Out of curiosity I spent a bit of time yesterday looking for it, but couldn’t find it.

Nevermind, with the variables you provided, I found [Chapter 5. Configuring the Makefile | FreeBSD Documentation Portal](https://docs.freebsd.org/en/books/porters-handbook/makefiles/).

---

<div class="post-metadata">

**Author:** ![vishwin](https://avatars.discourse-cdn.com/v4/letter/v/ecb155/32.png) [@vishwin](https://discuss.python.org/u/vishwin)\
**Post date:** [November 15, 2023, 6:22pm UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/44 "2023-11-15T18:22:05Z")

</div>

Yes, the Porter’s Handbook is about the most guide-like prose you will get. Everything else is more-or-less Makefile (BSD Make, not GNU Make) implementation details, with best-effort documentation in comments, that are learnt along the way. For this thread’s purposes, I tried to be generic as possible with variable names and such so those familiar with other similar systems have a chance of understanding.

---

<div class="post-metadata">

**Author:** ![vishwin](https://avatars.discourse-cdn.com/v4/letter/v/ecb155/32.png) [@vishwin](https://discuss.python.org/u/vishwin)\
**Post date:** [November 15, 2023, 6:37pm UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/45 "2023-11-15T18:37:37Z")

</div>

> [@encukou](#):
>
> explicit conversion

This is what I’d be trying to implement at a meta/framework level in an automated manner, not relying on doing this on individual bases, where existing cases in our tree are already wildly inconsistent.

> [@encukou](#):
>
> If it’s not simple enough, the version converted from a `pyproject.toml` entry will look weird, e.g. have double tildes in the middle and such.

This kind of difference is fine, as long as all the metadata is aligned after any automated processing such that we are not misrepresenting things. There are cases in our tree where metadata is not aligned properly.

> [@encukou](#):
>
> It’s similar with package _names_, except the names converted from a `pyproject.toml` entry never look like regular package names – for example, _`python3dist(requests)`_, with parentheses. They’ll never conflict with a hand-picked RPM name.

This is almost entirely a solved problem for us as well, apart from the mixed-casing part of Python package name normalisation.

---

<div class="post-metadata">

**Author:** ![sirosen](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/sirosen/32/8478_2.png) [@sirosen](https://discuss.python.org/u/sirosen)\
**Post date:** [November 15, 2023, 11:51pm UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/46 "2023-11-15T23:51:32Z")

</div>

> [@vishwin](#):
>
> I brought the longevity part up and only received more ridicule. Post-releases for metadata changes and not bug fixes in particular are perceived as poor release engineering practice.

I know this isn’t directly helpful, but I just want to say that I’m sorry you’re having to deal with this.  
Even if the invective isn’t directed at you, it’s unpleasant to deal with it.

Dev releases sort before alphas under the Python versioning rules. You might have more traction pointing at that as a related issue. But honestly, I’d give the friendly advice of “don’t bother” – it sounds to me like this is not going to get you much success no matter what.

* * *

Your answers to my questions were quite helpful – I have a better handle on the situation now. Thank you for those.

I haven’t thought about this enough yet to be sure I can offer a clear resolution, but I’ll note that the way you showed a possible use of the REVISION field did not match my expectations.

I was thinking you would have, for the oxidized case,

```auto
NAMEPREFIX=${PYTHON_NAMEPREFIX}
NAME=oxidised
DISTVERSION=1.0
REVISION=2
DISTNAME=${NAME}-${DISTVERSION}.post1

```

and then if you need a rebuild (e.g. new build tools), the following version would be

```auto
NAMEPREFIX=${PYTHON_NAMEPREFIX}
NAME=oxidised
DISTVERSION=1.0
REVISION=3
DISTNAME=${NAME}-${DISTVERSION}.post1

```

That is, the post version is not tied to those revisions at all.

I’m sure that being unaware of the broader automation context leaves me unclear on the challenges you would face implementing this. But that’s the sort of state I meant to suggest as desirable.

---

<div class="post-metadata">

**Author:** ![vishwin](https://avatars.discourse-cdn.com/v4/letter/v/ecb155/32.png) [@vishwin](https://discuss.python.org/u/vishwin)\
**Post date:** [November 16, 2023, 1:11am UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/47 "2023-11-16T01:11:21Z")

</div>

> [@sirosen](#):
>
> Dev releases sort before alphas under the Python versioning rules. You might have more traction pointing at that as a related issue.

Now knowing that `.devN` are also considered pre-releases, this is no longer an issue. Especially when it has to be the last version element anyway, the sorting works out without any treatment.

> [@sirosen](#):
>
> That is, the post version is not tied to those revisions at all.

Which would be a manual override.

---

<div class="post-metadata">

**Author:** ![sirosen](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/sirosen/32/8478_2.png) [@sirosen](https://discuss.python.org/u/sirosen)\
**Post date:** [November 16, 2023, 8:04am UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/48 "2023-11-16T08:04:36Z")

</div>

> [@vishwin](#):
>
> Now knowing that `.devN` are also considered pre-releases, this is no longer an issue. Especially when it has to be the last version element anyway, the sorting works out without any treatment.

Are you sure about that? Python versioning defines the sort order to put 1.0-dev1 before 1.0a1 . If your distro packaging reflects these versions, does it also put them in that order? My expectation would be that a naive approach would sort alphabetically, which is wrong for this case.

> [@vishwin](#):
>
> > [@sirosen](#):
> >
> > That is, the post version is not tied to those revisions at all.
> 
> Which would be a manual override.

Uh, I guess? I don’t know, with 100% confidence, what you mean by “manual override”.  
You’re saying that this approach is incompatible with automating your packaging? I don’t see why that should be the case.

You could parse the base version string and cut off the post release number, check the last release to see if it’s a post release with the same post release number, and drive your logic that way.  
I was assuming that your packaging toolchain already has access to the last release of your package.

Is that the issue? That you cannot read the metadata from your last release at the time that you are building a package?  
If that’s the problem, then yes, my idea doesn’t work. But it strikes me as solvable – surely these data are visible to your package manager, for published releases?

---

<div class="post-metadata">

**Author:** ![effigies](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/effigies/32/35835_2.png) [@effigies](https://discuss.python.org/u/effigies)\
**Post date:** [November 16, 2023, 4:44pm UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/49 "2023-11-16T16:44:28Z")

</div>

I’ve looked up the BSD make [variable syntax](https://man.openbsd.org/make.1#VARIABLE_ASSIGNMENTS). What about something like the following?

> ```auto
> NAMEPREFIX=${PYTHON_NAMEPREFIX}
> NAME=oxidised
> DISTVERSION=1.0
> REVISION=1
> DISTNAME=${NAME}-${DISTVERSION}.post${REVISION}
> 
> ```

```auto
NAMEPREFIX=${PYTHON_NAMEPREFIX}
NAME=oxidised
DISTVERSION=1.0p1 # Converted from 1.0.post1 on generation
REVISION=1
DISTNAME=${NAME}-${DISTVERSION:S/p/.post/}

```

Bumping `REVISION` should not interfere, and as I understand it, `1.0 < 1.0p1 < 1.0p2`

For `.postM.devN`, would `1.0pM.devN` not work out to `1.0 < 1.0p1.dev1 < 1.0p1 < 1.0p2.dev1`?

```auto
NAMEPREFIX=${PYTHON_NAMEPREFIX}
NAME=oxidised
DISTVERSION=1.0p1.dev20
REVISION=1
DISTNAME=${NAME}-${DISTVERSION:S/p/.post/} # 1.0.post1.dev20

```

Apologies if I’m missing something very obvious here.

---

<div class="post-metadata">

**Author:** ![vishwin](https://avatars.discourse-cdn.com/v4/letter/v/ecb155/32.png) [@vishwin](https://discuss.python.org/u/vishwin)\
**Post date:** [November 17, 2023, 12:42pm UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/50 "2023-11-17T12:42:18Z")

</div>

All of your examples were considered before starting this thread, but they all have manually overriding `${DISTNAME}` per individual case. If these were one-off cases not documented in any sort of policy or living standard, sure, but since these are part of the standard, a more general way is needed to keep consistent.

Additionally, overriding `${DISTNAME}` is not enough, the full Python package name convention is also needed after `build` generates the bdist for `installer` to consume. [That bdist name will contain the full `.postN`, etc](https://packaging.python.org/en/latest/specifications/binary-distribution-format/). I have experimented with some “private” variables to achieve this, but they chiefly involve overriding `${DISTNAME}` at the framework level. As `${DISTNAME}` at that level is assigned by `?=`, it cannot be overriden by another `?=` elsewhere unless an explicit assignment exists. Doing this kind of override in the Python package section of the framework would preclude one-off overrides at the individual case level, as framework-level override would be an explicit assignment `=`.

---

<div class="post-metadata">

**Author:** ![vishwin](https://avatars.discourse-cdn.com/v4/letter/v/ecb155/32.png) [@vishwin](https://discuss.python.org/u/vishwin)\
**Post date:** [November 17, 2023, 2:10pm UTC](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461/51 "2023-11-17T14:10:17Z")

</div>

> [@sirosen](#):
>
> Are you sure about that? Python versioning defines the sort order to put 1.0-dev1 before 1.0a1 . If your distro packaging reflects these versions, does it also put them in that order? My expectation would be that a naive approach would sort alphabetically, which is wrong for this case.

Your last sentence is correct when only `.devN` is present. Some further tweaking would be needed.

> [@sirosen](#):
>
> You could parse the base version string and cut off the post release number, check the last release to see if it’s a post release with the same post release number, and drive your logic that way.  
> I was assuming that your packaging toolchain already has access to the last release of your package.
> 
> Is that the issue? That you cannot read the metadata from your last release at the time that you are building a package?  
> If that’s the problem, then yes, my idea doesn’t work. But it strikes me as solvable – surely these data are visible to your package manager, for published releases?

No knowledge of any previous builds or releases exist whatsoever. Furthermore, at the most basic level, there’s not really a toolchain apart from `make`, the rest of the base operating system and the package manager itself (not part of the base system); anything needed to produce a built package is put together on the fly (preferably in our operating system equivalent of a virtualenv), which is parsed from metadata provided only at invocation.

> [@sirosen](#):
>
> Uh, I guess? I don’t know, with 100% confidence, what you mean by “manual override”.  
> You’re saying that this approach is incompatible with automating your packaging? I don’t see why that should be the case.

See specific genericised examples elsewhere in the thread. The gist, something of this nature, present in a living standard, should be handled at the framework level, not as individual one-offs.

[Previous page](https://discuss.python.org/t/post-and-developmental-release-version-specifier-handling-in-distros/38461.md?page=2)