Requested lz4~=4.3 from file, but installing version 0.0.0

Python Help
1

So, I’m trying to make a FlatPak of the Amulet Minecraft Map Editor, which is built in Python, (Flatpak requires everything to be built from source to maintain sandboxing, including all the Python modules). The flatpak build process runs smoothly until it comes to lz4 at which point I get this fatal error during the build:

========================================================================
Building module python3-lz4 in /home/evilsupahfly/src/Flatpak Testing/.flatpak-builder/build/python3-lz4-2
========================================================================
Running: pip3 install --verbose --exists-action=i --no-index --find-links="file://${PWD}" --prefix=${FLATPAK_DEST} "lz4~=4.3" --no-build-isolation
Using pip 24.0 from /usr/lib/python3.11/site-packages/pip (python 3.11)
Looking in links: file:///run/build/python3-lz4
Processing ./lz4-4.3.3.tar.gz
  Running command Preparing metadata (pyproject.toml)
  /usr/lib/python3.11/site-packages/setuptools/_distutils/dist.py:266: UserWarning: Unknown distribution option: 'use_scm_version'
    warnings.warn(msg)
  running dist_info
  creating /tmp/pip-modern-metadata-aum609f4/lz4.egg-info
  writing /tmp/pip-modern-metadata-aum609f4/lz4.egg-info/PKG-INFO
  writing dependency_links to /tmp/pip-modern-metadata-aum609f4/lz4.egg-info/dependency_links.txt
  writing requirements to /tmp/pip-modern-metadata-aum609f4/lz4.egg-info/requires.txt
  writing top-level names to /tmp/pip-modern-metadata-aum609f4/lz4.egg-info/top_level.txt
  writing manifest file '/tmp/pip-modern-metadata-aum609f4/lz4.egg-info/SOURCES.txt'
  reading manifest file '/tmp/pip-modern-metadata-aum609f4/lz4.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  adding license file 'LICENSE'
  writing manifest file '/tmp/pip-modern-metadata-aum609f4/lz4.egg-info/SOURCES.txt'
  creating '/tmp/pip-modern-metadata-aum609f4/lz4-0.0.0.dist-info'
  Preparing metadata (pyproject.toml) ... done
  WARNING: Requested lz4~=4.3 from file:///run/build/python3-lz4/lz4-4.3.3.tar.gz, but installing version 0.0.0
Discarding file:///run/build/python3-lz4/lz4-4.3.3.tar.gz: Requested lz4~=4.3 from file:///run/build/python3-lz4/lz4-4.3.3.tar.gz has inconsistent version: expected '4.3.3', but metadata has '0.0.0'
ERROR: Could not find a version that satisfies the requirement lz4~=4.3 (from versions: 4.3.3)
ERROR: No matching distribution found for lz4~=4.3
Error: module python3-lz4: Child process exited with code 1

I’ve checked all the references to lz4 in the python source, and they all say 4.3 so I don’t know where version 0.0.0 is coming from nor do I know how to remedy this.

** Edit:
In case anybody needs it, my requires.txt is as follows:

lz4~=4.3
versioneer>=0.29
meson-python==0.16.0
meson==0.63.3
cython>=3.0.0a9
setuptools>=60
pre-commit>=1.11.1
wheel>=0.37.1
pillow>=10.0.1
wxpython>=4.2.1
numpy>=1.17
pyopengl>=3.1.0
pyopengl-accelerate~=3.0
packaging>=21.3
amulet-core>=1.9
amulet-nbt>=1.1
amulet-map-editor>=0.10.34
pymctranslate>=1.2
minecraft-resource-pack>=1.3
platformdirs>=3.1
sphinx>=1.7.4
sphinx-autodoc-typehints>=1.3.0
sphinx-rtd-theme>=0.3.1
2

I think some build requirements need to be installed, or an old pip that’s incompatible with lz4 4.3.3 is being used.

I downloaded lz4-4.3.3.tar.gz and the version tag (4.3.3) is in there, both in lz4-4.3.3\lz4\version.py
and in PKG-INFO, but in what you’ve posted pip isn’t looking in either file.

You could also try cloning the repo with git, installing the build requirements in the build env (if pip’s not doing that automatically for some reason):

"setuptools>=45", "wheel", "setuptools_scm[toml]>=6.2", "pkgconfig"

and see if setuptools_scm will pull the version tag from the commit history.

3

I tweaked my “requirements.txt” and I upgraded PIP (was v22, now v24) and I get the same result. Honestly, I’m confused. In order to keep the build environment clean, nothing is cached - all the requirements and components are downloaded fresh every run, yet it trips and falls over lz4 every time and I don’t understand why. The flatpak-builder script itself is just a wrapper that calls PIP when dealing with Python requirements, and I can run the PIP lines myself without issue. If I run pip3 install --verbose --exists-action=i --no-index --find-links="file://${PWD}" --prefix=${FLATPAK_DEST} "lz4>=4.3" --no-build-isolation and substitute the strings for pathnames, it runs fine.

========================================================================
Building module python3-lz4 in /home/evilsupahfly/src/Flatpak Testing/.flatpak-builder/build/python3-lz4-1
========================================================================
Running: pip3 install --verbose --exists-action=i --no-index --find-links="file://${PWD}" --prefix=${FLATPAK_DEST} "lz4>=4.3" --no-build-isolation
Using pip 24.0 from /usr/lib/python3.11/site-packages/pip (python 3.11)
Looking in links: file:///run/build/python3-lz4
ERROR: Could not find a version that satisfies the requirement lz4>=4.3 (from versions: none)
ERROR: No matching distribution found for lz4>=4.3
Error: module python3-lz4: Child process exited with code 1
4

So I changed lz4>=4.3 to lz4~=4.3 and to lz4==4.3.3 and it fails no matter what, though I see the wording of the error message has changed slightly to versions: none.

ERROR: Could not find a version that satisfies the requirement lz4==4.3.3 (from versions: none)
ERROR: No matching distribution found for lz4==4.3.3
Error: module python3-lz4: Child process exited with code 1
5

My interpretation:

You don’t have cached metadata in your specified --find-links folder, only the cached tarball. When Pip tries to build the package, it doesn’t care about the filename for the tarball - it only gets metadata either from a separate metadata file, or from the build process.

as can be seen on GitHub, the pyproject.toml file doesn’t specify a version number. pyproject.toml is in fact quite minimal in this case; it’s only specifying some build dependencies, and expecting the setup.py process to create all the needed metadata.

But this project wants to compute the version using setuptools-scm, an extension to setuptools that checks version control to retrieve information about the project and compute things like the version number.

More specifically, it’s configured to use setuptools-scm to write a version number into a lz4/version.py file in the source directory, and then use that during building to set the version for metadata. (The file is then also available for the program to use internally at runtime.) The GitHub repository doesn’t contain this file (version control wouldn’t be tracking it on the devs’ machine) and neither does the source tarball. This is a problem, because it prevents anyone else from building from the source tarball - when unpacked on someone else’s machine, there’s no Git repo from which to extract the version info.

So when the package is built from the sdist, it can’t produce the necessary file, therefore can’t inform the metadata creation process, and therefore falls back to a default version number.

The documentation proposes that it should be possible for users to build from the sdist, so this can reasonably be considered a bug to report on the issue tracker (if it isn’t there already). For now, please use a wheel instead if at all possible.

Just for the record (it doesn’t appear to relate to the problem), using --no-index --find-links= is the opposite of that: it’s forcing Pip to use a local copy of the lz4 component, rather than checking PyPI for any updates.

2 Likes
6

I hadn’t noticed that, since it’s in flatpak-pip-generator.py which I didn’t write, but I’ll change it. Also, since flatpak-pip-generator.py itself determines wheel vs archive internally, I’ll have to work around that, but it shouldn’t be an issue.

Thanks for the head up! I’m filing a bug report with lz4 before I go to work today.

7

FWIW, there are probably very good reasons why your generator script wants to use only local files. (i.e., reproducible builds: a given version of a package on PyPI could have a post-release, or get yanked.)

8

True, which is why I’m not going to leave it like that, but for the sake of being thorough, I’ll do a test build with those options disabled, in case someone asks, although since it looks like the error can be reproduced, I don’t think I’ll need the results of that test run.

9

As I thought, disabling those options didn’t do squat.

flatpak-pip-generator.py calls PIP and my understanding is that PIP by default prefers wheels, so I added wheel and removed --no-index but now PIP is complaining because wheel doesn’t like --prefix=${FLATPAK_DEST}, which breaks the flatpak-builder wrapper since wheel takes different options than install, which flatpak-pip-generator.py was originally meant to do and I can’t run wheel and install together because install requires --prefix which breaks wheel.

So it looks like my options are to wait for an upstream fix for lz4, doctor up flatpak-pip-generator.py with a special case work-around for lz4, or search for an alternative to it (which would involve major changes to the original Amulet project.

10

So, on a whim, I added

[project]
version = "4.3.3"

to lz4’s pyroject.toml and tar-gzipped it back up, dropped it into the cache as R-O so it wouldn’t get overwritten, and it still failed with the same old message. So now I’m going to bed, and in the morning, I’ll remove lz4 from requirements.txt to see what happens, and pending those results tomorrow, maybe I’ll leave this alone for a while either until lz4 gets patched upstream or they offer a better solution.