PEP 711: PyBI: a standard format for distributing Python Binaries

Ha, whoops, thanks for redirecting me. :upside_down_face: I still don’t see that that is true in a necessary sense, though, and I’m not even sure it’s true in a practical sense.

On the practical side, it may be true that conda generates a wheel, but it isn’t true that all the conda metadata is simply a “projection” from the pypi metadata. In particular, the dependency information probably won’t be. It’s perfectly possible to make a conda package where all the dependency info is specified “from scratch” in the meta.yaml. You still need a setup.py or pyproject.toml to get the build to work, but it can list no dependencies at all.

And if pypi-level dependencies do exist for the project,they’re likely to be useless for a conda package, because the conda equivalents may have different names and may be split up in different ways (because non-Python deps can be separated). To some extent this translation can be done automatically, but often it involves a human looking at the pyproject.toml and figuring out what the appropriate conda packages are to reproduce the dependency info in conda land. So in this sense the PyPI metadata is no more special than a text-file readme like from the bad old days pre-pip, where the author would say “before installing this you better install that and t’other, but that’s on you”; it’s just something for a human to read to tell them what to do. There’s nothing essential about the format or even the content[1]. The package author’s intent is important, but I don’t see that the pypi metadata format has special importance as a means of expressing that[2]. (I have no idea whether any of what I’m saying here is also true of adapting a pypi package to something like debian or nix, although it certainly seems like it could be,)

When I’ve done this, I see it as simply a nuisance that I have to write a pyproject.toml at all to get conda to build my project. As near as I can tell the only really necessary part is specifying a build backend so conda can use it to build the wheel; there doesn’t actually need to be any contentful pypi-level metadata. I think even the version info there doesn’t matter, because the info in meta.yaml will take precedence over it. So I don’t see this as a super important role for wheels qua wheels; it’s just that wheel-building has been repurposed as an intermediate step in a conda build, and that could well be replaced by something else.

On the conceptual side, I still don’t think that basing something like pybi on that pypi metadata is even a good idea, because it doubles down on the limitations of that system[3]. In particular, it does not solve the larger issue of wanting to depend on arbitrary things that aren’t written in Python.

As far as I can tell the PEP is basically only about installing the Python interpreter. It’s true that that’s probably the most important non-Python dependency[4] for many people, but it’s far from the only one. Also, because it maintains the Python-first model[5], it doesn’t allow authors to actually depend on pybi in the same way they would depend on a normal package. So this would let people install Python itself from pypi (using posy or whatever), but for all the other things they might want to install from pypi that aren’t Python libraries, they’re still out of luck.

So, all in all, I don’t think it’s actually the case that conda packages “derive” (or must derive) their metadata from pypi metadata; and even if it were, I don’t think it’s a good idea to double down on a metadata system that is completely unable to handle non-Python dependencies. What you call the “abstract” pypi metadata is, to my mind, not abstract enough, because (among other things) it conflates a Python package with its bundled non-python dependencies. If that larger problem were solved, then pybi could be just one among many installable things that aren’t Python libraries.


  1. i.e., the actual package names depended on ↩︎

  2. again, apart from the fact that a lot of people use it ↩︎

  3. cogently discussed by @steve.dower here ↩︎

  4. in the sense that Python itself is not a Python library ↩︎

  5. i.e., the environment is inside Python rather than Python inside the environment ↩︎