Let me write some more about the Fedora/RPM.
RPM and Python version schemes are incompatible. When doing any kind of automation, we always keep in mind which system’s used, and explicit conversion (which is carefully designed so that equivalent versions keep the sort order). RPM versions are powerful enough that if the version is “simple enough” it’ll look the same in both systems. If it’s not simple enough, the version converted from a pyproject.toml entry will look weird, e.g. have double tildes in the middle and such.
It’s similar with package names, except the names converted from a pyproject.toml entry never look like regular package names – for example, python3dist(requests), with parentheses. They’ll never conflict with a hand-picked RPM name.
Perhaps you need to design a FreeBSD/Python version mapping where equivalent versions don’t look the same.
sudo does not place a [DOT] before the p, which alone is the difference maker. Version comparison run:
% pkg version -t 1.9.15 1.9.15p2
<
% pkg version -t 1.9.15 1.9.15.p2
>
% pkg version -t 1.9.15 1.9.15.post2
>
%
The package manager maintainer will not budge from this logic, any letter in the beginning of a version segment (ie [DOT]whatever) is a pre-release and semantically treated as such. In the first example, the second version string’s last segment is 15p2, but the latter examples’ last segments are p2 and post2 respectively.
The sdist and generated bdist filenames are taken directly from the name and version fields on our end. We have two ways of specifying version: “distribution version” (preferred) and “port version”. “Distribution version” is the form present on the source archive and is mapped/converted into our acceptable form if necessary. “Port version” is the form present in the distro package and is where the converted/mapped result is stored if “distribution version” is specified. Only one form can be specified. Specifying 1.9.15p2 as “distribution version” results in “port version” 1.9.15.p2, which is a problem. 1.9.15.post2 “distribution version” converts/maps to the same result.
The only good hack I’ve figured out so far is just that, and it is extremely ugly and unwieldy. Any Python package at a post-release would have to be specified as 1.9.15p2 “port version” to avoid automatic conversion/mapping, but then have logic in the framework to override the source archive/sdist name, namely the operation s/p/.post/. This bit is problematic, as the ability to override the source archive name is reserved at an individual case level, not meta. It also presents problems when sdist names are not normalised, most often mixed case names.
We use the revision parameter for our own bumps for forced rebuilds, metadata changes, etc. It is also not part of the default source archive filename format. Best illustrated with an example:
Given Python package oxidised-1.0.post1. With the thought of (ab)using the revision parameter, our metadata:
In this form, the oxidised-1.0 would get fetched and operated on. To actually get the post-release, we have to override the source archive name parameter:
Now, oxidised has Rust code. New Rust (toolchain) release drops, we need to force rebuild everything in our tree containing Rust code, so every such case’s revision gets bumped:
This will now fail because oxidised-1.0.post2 doesn’t exist.
The common theme here is individual manual overrides. Fine for isolated cases like sudo, not acceptable for an entire class.
Another failure mode is oxidised-1.0.post0. The revision parameter is always set to 0 by default, and 0 values are hidden.
The solution may very well be prohibiting packaging of post-releases.
I brought the longevity part up and only received more ridicule. Post-releases for metadata changes and not bug fixes in particular are perceived as poor release engineering practice.
Is there a place where this metadata configuration is documented? Out of curiosity I spent a bit of time yesterday looking for it, but couldn’t find it.
Yes, the Porter’s Handbook is about the most guide-like prose you will get. Everything else is more-or-less Makefile (BSD Make, not GNU Make) implementation details, with best-effort documentation in comments, that are learnt along the way. For this thread’s purposes, I tried to be generic as possible with variable names and such so those familiar with other similar systems have a chance of understanding.
This is what I’d be trying to implement at a meta/framework level in an automated manner, not relying on doing this on individual bases, where existing cases in our tree are already wildly inconsistent.
This kind of difference is fine, as long as all the metadata is aligned after any automated processing such that we are not misrepresenting things. There are cases in our tree where metadata is not aligned properly.
This is almost entirely a solved problem for us as well, apart from the mixed-casing part of Python package name normalisation.
I know this isn’t directly helpful, but I just want to say that I’m sorry you’re having to deal with this.
Even if the invective isn’t directed at you, it’s unpleasant to deal with it.
Dev releases sort before alphas under the Python versioning rules. You might have more traction pointing at that as a related issue. But honestly, I’d give the friendly advice of “don’t bother” – it sounds to me like this is not going to get you much success no matter what.
Your answers to my questions were quite helpful – I have a better handle on the situation now. Thank you for those.
I haven’t thought about this enough yet to be sure I can offer a clear resolution, but I’ll note that the way you showed a possible use of the REVISION field did not match my expectations.
I was thinking you would have, for the oxidized case,
That is, the post version is not tied to those revisions at all.
I’m sure that being unaware of the broader automation context leaves me unclear on the challenges you would face implementing this. But that’s the sort of state I meant to suggest as desirable.
Now knowing that .devN are also considered pre-releases, this is no longer an issue. Especially when it has to be the last version element anyway, the sorting works out without any treatment.
Are you sure about that? Python versioning defines the sort order to put 1.0-dev1 before 1.0a1 . If your distro packaging reflects these versions, does it also put them in that order? My expectation would be that a naive approach would sort alphabetically, which is wrong for this case.
Uh, I guess? I don’t know, with 100% confidence, what you mean by “manual override”.
You’re saying that this approach is incompatible with automating your packaging? I don’t see why that should be the case.
You could parse the base version string and cut off the post release number, check the last release to see if it’s a post release with the same post release number, and drive your logic that way.
I was assuming that your packaging toolchain already has access to the last release of your package.
Is that the issue? That you cannot read the metadata from your last release at the time that you are building a package?
If that’s the problem, then yes, my idea doesn’t work. But it strikes me as solvable – surely these data are visible to your package manager, for published releases?
All of your examples were considered before starting this thread, but they all have manually overriding ${DISTNAME} per individual case. If these were one-off cases not documented in any sort of policy or living standard, sure, but since these are part of the standard, a more general way is needed to keep consistent.
Additionally, overriding ${DISTNAME} is not enough, the full Python package name convention is also needed after build generates the bdist for installer to consume. That bdist name will contain the full .postN, etc. I have experimented with some “private” variables to achieve this, but they chiefly involve overriding ${DISTNAME} at the framework level. As ${DISTNAME} at that level is assigned by ?=, it cannot be overriden by another ?= elsewhere unless an explicit assignment exists. Doing this kind of override in the Python package section of the framework would preclude one-off overrides at the individual case level, as framework-level override would be an explicit assignment =.
Your last sentence is correct when only .devN is present. Some further tweaking would be needed.
No knowledge of any previous builds or releases exist whatsoever. Furthermore, at the most basic level, there’s not really a toolchain apart from make, the rest of the base operating system and the package manager itself (not part of the base system); anything needed to produce a built package is put together on the fly (preferably in our operating system equivalent of a virtualenv), which is parsed from metadata provided only at invocation.
See specific genericised examples elsewhere in the thread. The gist, something of this nature, present in a living standard, should be handled at the framework level, not as individual one-offs.