PEP 665, take 2 -- A file format to list Python dependencies for reproducibility of an application

I’ve tried to explain my perspective on what I expect out of a lockfile

But it didn’t really seem to get traction or my point across.

I think that discussions about true reproducibility go off-track, because they mean different things to different people, and in the strictest sense, are probably impossible to achieve in a practical way (or would require boiling the ocean). I don’t believe that byte-for-byte reproducibility is required for a lockfile to be useful. I think it’s a noble goal to pursue, but I don’t see it being a requirement for a lockfile standard. This can be demonstrated to be true because pip-tools and poetry (and others I’m sure) have created non-reproducible lockfiles that are still very useful. And if / when true reproducibility comes along for python wheel builds, then even better, it still wouldn’t make the lockfile standard any less useful.

I’ll try again with another metaphor for what I’d like to see from a lockfile standard.

I would like a lockfile to be a standalone specification for a “wheelhouse” that I can use across platforms. It would contain a list of wheels and sdists that are required at build time and runtime. The key point being that NO other artifacts or indexes should be consulted that aren’t in the lockfile.

pip install --no-index --find-links=/tmp/wheelhouse --requirement lockfile.txt

I think many would find such a lockfile useful, particularly if it was possible to produce them across platforms (I know this isn’t possible at the moment without some tweaks to PEPs around what metadata can be dynamic across platforms).

I personally would have loved PEP 665 as an incremental step by the way and would be comfortable if sdist support came later, however I do recognise and understand that there would likely be many who wouldn’t use a lockfile until sdist support came along.

6 Likes