Building distributions and drawing the Platypus

Packaging
41

I was again wondering where exactly we are at right now toward this Platypus thing, and ended up drawing this graph:

Blocks marked green are components we already have, and others yet to be standardised (has non-official competing solutions, or not even built yet). All blocks beside pip install and PEP 518 are clearly only needed by only either package and application development, which is a strong indication to me that we probably want two tools (or one tool with two distinct aspects), one for people releasing packages, and one for people installing them (package developers probably need both for their development workflow, but wouldn’t be using them at the same time).

Some personal thoughts:

  • There are a lot of talks about a new manifest and lock file format (i.e. declare dependencies in pyproject.toml, but those alone won’t solve the problem, and can wait until other things are solidified.
  • We are close to a universal package development frontend a la Flit’s command line interface. The only essential missing part is editable installs; others (e.g. incremental builds, extrapolating with external build tools) are all backend-only and can be improved incrementally.
  • Interpreter discovery (how to find Python interpreters in the host system) is a vastly underspecified space, especially on POSIX. There are multiple efforts now, including PEP 397 (py.exe on Windows), @brettcannon’s Python launcher, and Vritualenv 20.0’s Python discovery, but at some point some universal rules are needed for components to interop.
  • Given the relatively steep learning curve to virtual environments, the tool probably needs to hide the implementation behind some abstractions. Interpreter discovery from the previous point would help, but it still leaves the topic of how to manage multiple environments. I’ve been experiementing stuff in this area, but there more interest is needed.

Edit (2020-02-19): I added a new branch tool management, which is basically to standardise (and improve if necessary) pipx currently does. I feel most people already agree it is a good way to install Python applications (e.g. Black) if we continue to deploy them on PyPI, so what’s left is not much different from the virtual environment management thing (described in the next few messages) so tools can interop instead of relying pipx (or whatever we standardise it into) to support every possible use case.

1 Like