Local version identifiers for custom builds and transitive dependencies

Packaging Standards
1

I’ve got a quick question on the handling of “Local version identifiers” and how they are handled transitively. I am about to setup a local index to play around, but thought that I’d ask the question first so that I don’t go on a wild goose chase if my understanding is incorrect.

My understanding, is that if I am to build wheels of upstream packages (original source or forked source) to a private index, that I SHOULD publish them with “Local version identifiers”. This is to make it easy to identify our downstream builds from the official upstream builds during incidents (such as CVE etc) or debugging issues. This seems like a good idea.

My question is, does this work transitively for installation of packages?

i.e. if I have this scenario

  1. “foo-pkg 2.0.0” requires “bar-pkg >= 1.0”
  2. There is a “bar-pkg 1.0.0” which I want to patch and publish to a private index
  3. I have published a custom version of “bar-pkg” as “bar-pkg 1.0.0+custom” to a private index
  4. My requirements.txt is as follows
bar-pkg==1.0.0+custom
foo-pkg

Will that result in my desired result of “foo-pkg==2.0.0” and “bar-pkg==1.0.0+custom” being installed?

2

Use a new package name with patched module code that installs into the same package file/folder.

Use bar-pkg-custom in your examples.

3

Yeah, I think that should work. You should test, though, as I’m not 100% sure.

Edit: @barry-scott why do you think the OP’s original approach won’t work?

1 Like
4

If i see a standard package with a version that upstream did not release I would suspect an error in the dependencies. Coming up with a version that upstream will never use seems a risk.

If i see a repackaged version with a new name, myorg-bar, then it is clear what is going on.

I think its the same effort both ways. Also consider that you may be carrying the local patch for a number of releases as upstream issues updates you want with out the fix.
Knowing that myorg-bar 1.0.3 is the same as bar 1.0.3 but with local patches help with the audit trail.

5

Using new package name only can work for direct dependencies. If you need patch to override usage for transitive dependencies then changing package name is not an option. Also even if you only need to change your direct usages, if any of your dependencies later has indirect usage of that package I think behavior may be confusing if values get passed around through both packages.

I have done the make a fake/custom version to patch package and load to internal package index. Has worked pretty well for me across several packages. In practice I’ve never seen version issues. You can pick very weird version numbers like version 12345 or any silly value.

2 Likes
6

I realised that I did not come back to this topic to confirm my findings.

tl;dr I am happy to confirm that local version identifiers will satisfy transitive dependencies.

I’ve created a test repository here: GitHub - groodt/py-local-version-identifiers: This project contains some fake data and a test script to confirm the behaviour of local version identifiers

One thing that might be a little surprising, is that local version identifiers will be preferred if any exist. The only way that I was able to direct pip to skip a local version identifier was to use an exact match bar-pkg===1.0.0