Announcement: Hatch v1.8.0

I am pleased to announce the release of Hatch v1.8.0 Hatch v1.8.0 - Hatch

This release was a long time in the making and required me slowing down on development for some time to work on binary distribution techniques. Now Hatch can be installed easily by any user like most other software. Special thanks to Ee for getting the certificates set up for the macOS installer!

There is now also a fmt command backed by Ruff with default settings that I have painstakingly enabled individually to provide as perfect an experience as possible. This is the equivalent of what other package managers like Cargo and Go have.

Finally, I’m very excited that Hatch can now manage Python distributions all by itself which I think users and enterprise IT teams will begin to prefer over pyenv.

Please let me know what you think :smile:

22 Likes

Why installing Python interpreters in ~/.pythons instead of some platform recommended location like what platformdirs offers? Or am I misunderstanding things?

The new python command group allows for easy installation of various distributions to arbitrary locations (default ~/.pythons)

Hatch v1.8.0 - Hatch

4 Likes

It seems there was already precedent for that so I followed suit. I’m open to changing the default since that wouldn’t really break anything.

If you mean precedent for .pythons specifically, then you probably don’t want to follow that (unless it’s a defined protocol and the other apps can manage Hatch’s installs and vice versa).

If you mean precedent for ~/.<name>, in my experience that’s POSIX-centric devs not considering that other platforms have other conventions, and so following platformdirs is probably a better starting point (in this particular case, I’d expect something like %LocalAppData%\Hatch\Pythons on Windows, for example, which should be roughly where platformdirs leads you to).

6 Likes

Sorry perhaps I should have been more explicit with my intent! By default I am fine with putting it in a platform specific directory but I don’t want it to be under a Hatch namespace because I thought it would be advantageous for all tools to share a directory that contains Python versions.

Is that not very useful actually?

2 Likes

If you’re using a shared definition of install so that you can have interoperability with other installers (so they can safely/correctly choose an install to reuse, or update/delete them), sure, it might be okay. But currently no such definition exists.

Think of it like pip and conda trying to install into the same directory. It only works as well as it does[n’t] because there’s a shared .dist-info definition. Without that, you get chaos.

Namespacing specifically to Hatch is safest, for sure. People who are using different tools for different projects probably don’t want them to overlap anyway, even if they happen to request the same version.

3 Likes

Okay, thank you both for the feedback! I will put out a patch release changing the default and fixing one other thing.

3 Likes

I think it is. But it should still use platform specific locations. Now the question is under what namespace (instead of the “hatch” namespace)? I do not know. Is there any other tool doing this or something similar currently?

1 Like

platformdirs is actually kind of a PITA on macOS. Great idea in theory, but not a very convenient directory to get to. That said, if there’s good tool UX to manage it, maybe it doesn’t matter that terminal paths are inconvenient.

1 Like

Once/if/when PyBI becomes a standard, then we’ve got a chance of being able to share management of interpreter installs.

Otherwise, the best we’ve got are the platform-specific mechanisms, which are explicitly rejected by this approach (in that the main feature is to manage it consistently across platforms).

It seems the point here isn’t to make the base install easily accessible, but to have it there so that Hatch can use it. Cleaning up old/unused versions is the only thing you’re likely to need, but I daresay it’s accessible enough for that.

(I’m familiar with the open XDGPATH-or-whatever-it’s-called discussions on macOS, but last I heard platformdirs didn’t consider it worth the breaking change and ensuing inconsistency between platforms.)

On a unix system like linux follow the XDG spec.
You should not create ~/.anything these days. For the XDG spec use ~/.local/<something> (for .config/ if its config).

The reason, I believe, is the clutter that legacy code was creating it ~/ will a lot of . files.

2 Likes

Fantastic! I will give this a try.

On the topic of where to put things my own preference is for these things to be usually project local. In other words if I have a directory ~/work/myproject.git then I want the environments associated with myproject to be inside that directory. Then I can see how much space each project is using and I know what the implications are of deleting either the environments contained there or just deleting the whole myproject.git directory.

I dislike the fact that pyenv and hatch both store virtual environments somewhere else so that I need to manage them separately with no way to clean up unused environments automatically. Storing the base Python environments outside of the project does make more sense than for virtual environments but in practice I find that the virtual environments end up taking up more disk space than the base Python installs anyway.

Also I think it would also be possible to make pyenv’s base Python installs about 3x smaller by removing bits that are not commonly used e.g. this is all wasted space as far as I am concerned:

$ du -sh ~/.pyenv/versions/3.12.0/lib/python3.12/test/
126M	/home/oscar/.pyenv/versions/3.12.0/lib/python3.12/test/

That is mostly .pyc files. Even if it is worth shipping the CPython test suite there is no need to generate 100MB of .pyc files from it eagerly given the low likelihood that they would ever be used.

Strong +1 to this. My pet peeve with Hatch is that it creates environments in a shared location by default, meaning I have something else to clean up when I archive a project, beyond just deleting the project directory.

Having said this, I’m not quite sure what the point of managing Python distributions is - are the distributions managed by hatch expected to be used to create environments, to be run manually by the user, or what? Because if I use hatch to install Python 3.10, it presumably won’t be on $PATH by default, and I’ll need to find it to add it to $PATH. Also, the py launcher won’t be able to find it (I assume), so py -3.10 won’t work. And yet there’s no hatch command to invoke a Python interpreter installed by hatch either.

So while I think it’s a nice enough feature, I’m rather confused as to how it’s intended to be used in practice. To be fair, though, this is a problem I have in general with hatch - all the features seem useful enough, but I have no sense of how they are intended to work together to provide a unified experience. So maybe it’s just me.

This is easily configured Hatch - Hatch

The user can do whatever they want. Hatch however never exposes them directly for use in environments but rather they are used to create virtual environments that environments actually use.

The installation command adds it for you by default.

2 Likes

In my global shell configuration? Hopefully not. In my current shell? How, given subprocesses can’t affect the parent shell’s environment?

Sorry, I should probably experiment but I haven’t had time yet and the docs don’t say. If I have to invoke every hatch python install command with --private to stop hatch messing with my shell configuration, I’m going to be a bit cross, TBH. Or at least, it conflicts sufficiently with my preferences to put me off using it - which leads me to

Yeah, tools that are easy to configure the way you like them, but have defaults that are annoying for one reason or another, tend to be frustrating to use in practice (at least in my experience). I know it’s possible to configure per-project environments, but I said “by default” deliberately, as I think the default experience should be the one with the fewest hidden surprises (and I consider left-over clutter on my disk a “hidden surprise”).

Choosing good defaults is hard, and having people complaining from the sidelines is not very helpful (so I apologise for doing so) but in this instance I think that ignoring the general trend towards storing environments in the project directory rather than in a global location, is a mistake.

Forgive me for jumping in, but I will bring a different perspective and say that I like Hatch’s default and dislike tox’s and nox’s default on this.

I understand the argument with deleting the project directory, but

  • I don’t like having a .tox directory in git status output. I always need to add it manually to .gitignore. For me, it’s an advantage of using an interpreted language that I don’t have to care about some build directory in my repository.
  • I’m a heavy user of git clean -xdf, e.g., to start from scratch when I develop a script that generates output files, or as a sledgehammer to fix problems with incremental builds. With tox, if I forget --exclude .tox, this clears out .tox/ and it needs to re-download all the dependencies.
  • When I back up my computer, I don’t want to save all virtual environments, which can take a lot of space. With Hatch, I can just do hatch env prune.
5 Likes

You would have to restart your shell. I could work around us in Hatch though, it’s possible on every platform. Alternatively, you can just rely on environments automatically downloading the right Python distributions if necessary.

This default will never change but will always be configurable and I’m open to feedback on how to make configuration as easy as possible for everything!

Thanks for doing so. People do have different preferences, that’s part of the difficulty :slightly_smiling_face: I could argue that in-project by default is safer, because a newcomer doesn’t know to change the config, and would assume that “delete the project directory” deletes everything. So conforming to a newcomer’s expectations makes for an easier learning experience. But ultimately, the point remains that people do have different preferences.

I think that deletes all environments, not just the project’s, is that right? How do I just delete the environments for the current project? And more importantly (because I used to do it a lot with pipenv) when I’ve deleted the project directory and only then remembered about the environments, how do I delete the environments after the project directory is gone?

I just tried this out, and got in a mess because hatch was creating environments in .venvs. Turns out I had a config.toml buried in my %LOCALAPPDATA% directory that I’d forgotten about. That’s another reason I dislike configuring tools - you forget you’re not seeing what the docs say will happen. And on Windows, at least, the config directory is horribly difficult to find (hidden directory, cluttered with all sorts of crap including caches, working data, etc, as well as config files you’re expected to manually edit). That’s an OS problem, not a tool issue, but the tool suffers as a consequence of it.

1 Like

No that would be horrible! That only affects the current project.

There is no built-in way currently other than removing your data directory.

So (because I don’t want to accidentally change a global config in my main working machine by running a tool that doesn’t document what it does under the command docs) if I run hatch python install 3.10 what precisely will get changed on my PC?

Edit: Sorry for the sudden batch of usage questions. The point here is that I don’t want to just experiment, because my experience is that as a matter of principle, hatch expects to be allowed to put things in unexpected places on my PC, and that’s unacceptable to me.

1 Like