Hatch 1.0.0 is available

Hello, I just wanted to announce that my multi-year rewrite is over and Hatch is now ready to use!

There were 3 main reasons I rewrote the project from scratch:

  1. Timing - I’m a stickler for standards and 5 years ago there wasn’t much that existed or things were experimental. Now we have well thought out PEPs 517/518, 621/631, 639, 643, 660, etc. and implementing those will not lead to extra work later on since everything is mostly finalized.
  2. Scope - I kept getting feature requests for actual features and due to my mistake of making things too opinionated, which would have caused burn out if I didn’t halt development.
  3. Environments - I’ve always wanted to run things in not just virtual environments but in arbitrary environments, and have a way to statically defined matrices without using ini files.

So now:

  1. The build backend Hatchling was created and implements all the relevant packaging standards.
  2. All functionality is plugin-based, with defaults based on community standards, so users can extend as desired like getting the version from Git or building things other than wheels and sdists.
  3. There’s already a plugin for running things within Docker and much to my surprise TOML turned out to be a fantastic format for expressing environment configuration.

I consider the project to be mostly feature-complete (due to the plugin system) except for the following things that I couldn’t do without user input:

  • Lock file support, though it is currently blocked
  • Ability to download/install/manage Python versions on all platforms that virtual environments could then use (my old idea was to use/wrap Miniconda but they added a warning that cannot be silenced so we can’t)
  • Ability to parallelize command execution in multiple environments, I want it to be pretty and have an interactive option to expand to full size output based on keypresses so you can check on the progress of any at will
  • A dep show graph command

I also plan on finding a way to ship binaries for each platform.

Anyway, I’d appreciate any feedback :slightly_smiling_face:

17 Likes

Regarding Mailman 3 Vote on adopting the hatch project under pypa - PyPA-Committers - python.org I fully support the proposal and think this will be a great thing for the community at large.

2 Likes

I think this looks fantastic! It clearly has had a lot of work and attention! Thank you!

It’s wonderful to see how many of the modern standards can integrate to create a smooth development experience. I think this is how many experienced with python implicitly understand how things can work together in our own projects like this, but for beginners, it can be incredibly frustrating getting lost in the weeds of PEPs and never knowing which standard applies in which circumstances or what is deprecated or not.

I particularly like that it can be used for both application and library development. The plugin mechanism seems quite extensive as well for the situations where you do need to deviate from the basics / conventions.

Now for the slightly negative opinion. Unfortunately, I feel that many day-to-day python programmers working in industry are crying out for a lock-file experience. See the success of poetry, PDM, pip-tools (and probably others). I think Hatch will struggle to gain much adoption without that because it’s the most prominent missing piece. I see we are both waiting anxiously for metadata-only resolve with `pip download --dry-run --report`! by cosmicexplorer · Pull Request #10748 · pypa/pip · GitHub If that looks too far away, is there any thought in using something light-weight like pip-tools in the short-term to give you a pretty solid lockfile? In next release of pip it will give access to the internal resolver of pip, so it’s as close to a standard as one could reasonably achieve at the moment.

3 Likes

Thanks for the kind words!

I’m not too against the idea, but I’d much prefer opening a new PR and rebasing + spearheading the effort to get it merged.

2 Likes

Agree. I think that’s the best place to start as well. I hope that can get some traction.

1 Like

Hello, congratulations for 1.0 milestone. Install multi version is always pain in Linux and windows because you need to install other tools like pyenv. How hatch support install more than one python version on linux and windows? Is the interface like pyenv?

1 Like

No idea yet, which is why I’d like input :slightly_smiling_face:

In my mind though it would only add them to PATH to be used by virtualenv or other tools

Looks really great, beautiful docs too! Congratulations! :partying_face:

Just from looking at the plugin interface, it looks like has all the necessary knobs to support conda environments as well, which is awesome!

1 Like

Agree, for a beginner in programming like me, pyenv is good enough but too many options. In Linux if python built from source tools need to inform what package needs to be installed and in windows maybe just use official installer from python.org. Also, pyenv don’t show progress bar when installing python, maybe using rich progress bar :wink: will give a good visual.

❯❯❯ pyenv install 3.8.13
Downloading Python-3.8.13.tar.xz...
-> https://www.python.org/ftp/python/3.8.13/Python-3.8.13.tar.xz
Installing Python-3.8.13...

Maybe Nuitka Python Compiler can help with this, the maintainer is pretty active

Thank you for the project :heart:

1 Like

Have you seen this?

It needs some work to push it over the finish line – I know at least Brett and I are both interested in seeing it happen, but we’re both pretty distracted. But perhaps you’d like to help? And in any case, you’re welcome to use the PyPI-compatible repo at https://pybi.vorpus.org/ to pull down prebuilt Python binaries.

3 Likes

I hadn’t seen that, thanks! I was actually looking for prebuilt binaries and ended up finding what GitHub Actions uses:

Just bear in mind (unless they’ve changed something) that these are built specifically for their own images, and may not transfer to other OS versions or layouts (and I believe the Windows packages just run the regular installer and assume Admin privileges).

Nathaniel’s efforts are much more likely to be what you want.

1 Like

I’ve dragged @barry into this as well at work (but that’s all the progress as of late :sweat_smile:), but that’s as far as I will go hear as I don’t want to drag this topic into a tangent on building relocatable CPython binaries.

3 Likes

FYI, @jjhelmus put together give-me-python · PyPI for a PyCon lightning talk (in a day) after seeing my “think outside the box” slide at PyCon. It’s a manylinux compatible Python wheel. (as in, it’s Python as a wheel). Don’t known if expanding this would be a viable way to provide alternate Python implementations for hatch/nox/tox/etc, but interesting concept.

2 Likes

@njs has a repo that does something similar (not sure if he’s sharing that repo around yet).

I just linked it a few messages above :slight_smile:

1 Like

The transfer is complete GitHub - pypa/hatch: Modern, extensible Python project management

4 Likes

It’s very possible to use PDM to manage (PEP 621/631) dependencies in a lockfile, while using Hatch as the build backend.