Wanting a singular packaging tool/vision

Packaging
25

Here’s an attempt at user story to give some concrete examples of what kind of unified/rallied tooling vision I’m imagining for a future new Python developer.

Disclaimer: This is obviously very inspired by Rust, which I think does a fantastic job in this area, but some of the minor details may need to be tweaked for Python.

The new developer finds out about Python and wants to try it creating a Python project. They visit the official python.org website and are presented with a Getting Started page that provides them with a cross-platform interpreter installer/manager that has the same UI/CLI on every major platform supported. This could be similar to pyenv but Windows is a first class citizen and the installations could be pre-compiled rather than needing to be built.

We’ll call it pyup to sound similar to rustup.

Idea: Remove the duplicity of ways new developers are told to install Python: HomeBrew, stand-alone installer, deadsnakes PPA, build from source, Linux system package, etc.

Upon installing pyup, the latest stable release of Python is then available to them with a common, cross-platform name, e.g. py.

Idea: Remove the confusion regarding launching Python as python3 on *Nix and py on Windows (unless it’s the Windows store and then it’s python).

pyup can be used to install and manage multiple versions of Python. The developer decides they need to test their application on an older version of:

pyup install 3.10

Or change their default Python to 3.10:

pyup default 3.10

Python 3.11.1 is released, and the user can easily update to a new patch release:

pyup update 3.11

Idea: Make it simple and consistent across platforms how to install, update, and manage multiple Python interpreters.

Included with each interpreter installation that pyup installs is a package manager that looks something like (Poetry or hatch-with-lockfile) + pipx. We’ll call it pyrgo for now to keep with our silly naming conventions.

A coworker mentions that httpie would be a great tool to try out an API they will be interfacing with. They are able to easily install a tool globally on their system from PyPI (essentially vendoring a pipx-like experience:

pyrgo global-install httpie

Idea: Make bootstrapping installing Python tools something that is included out of the box. Rust has this with cargo install, Node has this with npx bundled, etc.

The developer is now ready to create their first Python project, so they run:

pyrgo new

This is similar to poetry new and hatch new in that it creates a new, standard project structure for them, including a populated pyproject.toml

demo
├── pyproject.toml
├── README.md
├── demo
│   └── __init__.py
└── tests
    └── __init__.py

The developer learns they will be using FastAPI for their application, so they add it as a dependency and automatically update pyproject.toml:

pyrgo add fastapi

They can then lock their requirements into a cross-platform lockfile, similar to Poetry, for reproducibility:

pyrgo lock

And install those into a virtual environment (which is managed for them, again similar to Poetry and hatch):

pygro install

Idea: Avoid the common papercut of how venvs are activated different on Windows vs *Nix.

The developer would like to automatically format their code to community standards. They find that pyfmt (e.g. black/isort) come preinstalled from pyup.

pyfmt

And one could imagine extending this to a linter or type checker (e.g. mypy) depending on community consensus.

The developer is ready to publish a new open source project to PyPI, they build the sdist and wheel using a single build command similar to Poetry/hatch build:

pyrgo build

They then upload the sdist and wheel to PyPI using a single publish command similar to Poetry/hatch publish:

prygo publish

Idea: Avoid the developer having to discover the existence of the build and twine PyPI packages, find and read their separate user guides, install them into a virtual environment, and invoke them using different commands.

Conclusion

The entire tooling workflow started with Python.org. This avoids a bootstrapping problem and many platform-specific steps that trip up new and seasoned developers alike. The community rallies around these central tools and reduces duplicated effort by pooling ideas/resources.

Importantly, it creates a cohesive ecosystem where a developer is much more likely to be able to drop into a new Python project and already know the workflow/tools.

Since all of these tools are standard across platforms, IDEs and editors have an easier time integrating and keeping up with updates and changes.

18 Likes