PEP 665: Specifying Installation Requirements for Python Projects

Thanks for pointing this out. My comment was only meant to illustrate the idea behind required-by, but I failed to communicate that it’s only the basic idea, not the complete logic to do that—in general, you need not required-by itself, but the sections referenced in it (and the top-level dependency specification, as you pointed out). The idea is to limit what needs to be traversed when a locker only intends to modify a part of the graph, which is a common operation for tools like Dependabot (to upgrade a dependency away from a vulnarable version), and not currently handled well by tools like pip and its derivatives.

This reminds me though; in my old proposal back in 2019, I used an empty string to designate the top level dependencies. This never caught on back then, but I’d definitely not object if we bring it back :slightly_smiling_face:

[[package.jinja2]]
required-by = ["", "flask"]  # Jinja2 is specified by the user, and is also depended by Flask.

In a perfect world, reproducibility means running the same code, with the same dependencies, on the same Python interpreter, in the same runtime environment. And that’s definitely a popular and valuable definition (also why containers are so popular nowadays). In reality though, projects tend to come with some level of assumptions and want to bend the rules. The runtime environment does not need to be exactly the same, as long as we don’t config the environment in a way that significantly impacts the behaviour (a popular definition for web services). Maybe the application can run on multiple operating systems (and/or architectures) and they don’t need to all install the same wheel, as long as the dependency’s maintainers promise all those wheels behave the same, so we can develop on macOS and deploy to Ubuntu. In extreme situations, even a dependency’s version doesn’t need to be exactly the same, as long as only one unique resolution is possible for each platform, and the different resolved versions don’t affect the end application’s behaviour. These are all feature requests that came up during development of existing tools, and PEP 665’s design. You can argue some of those are bad practice (I don’t think PEP 665 handles that final example without workarounds, for example), but those are popular enough that we are convinced they need to be possible, otherwise the format won’t be able to catch on and we’ll be back to square one.

3 Likes