Nice! Could this be split into a separate thread? I don’t think it’s reasonable or fair to mix the two discussions, and I don’t think having “slow mode” on for discussion of a new PEP is a good idea. @davidism I know you’re a moderator, and apparently I’m not allowed to tag the moderators group as a whole, so maybe you could do this if no-one else sees it?
Regarding the PEP itself, I will note that (not surprisingly!) it directly clashes with a number of the “rejected alternatives” in PEP 722. I’d really rather not post all of the arguments from that PEP into Discourse, just to re-hash old debates, as that will waste a lot of everyone’s time. Can I therefore ask that you respond in your PEP to all of those points (maybe in a big “Rejected Alternatives” section of your own?) as if I had raised them all here. Even if all you want to say is that you don’t agree with the objections raised in PEP 722, then could you do that explicitly so that it’s clear that your position is that the objections I give are simply a matter of opinion that you disagree with?
Also, could you make it clear in your PEP that the whole “pyproject.toml for projects that don’t build into a wheel” discussion is still unresolved, as your PEP rather critically depends on the outcome of that…
I’ve added a couple more points to the PEP draft as review comments. I know that’s not the ideal place to do so, but until your PEP has a dedicated thread, that seemed like the best approach.
Other posts
Some other points, related to earlier posts about PEP 722. Again, sorry for the “combined” post, but I didn’t want to wait another 8 hours to cover these.
You did see the point I made in PEP 722 that we can’t assume all clients will have access to the Python AST module? And that the Python 3.11 AST module (for example) might fail to parse Python 3.12?
I assume you disagree with those points, so I guess yeah, your code seems fine if you want to take that stance.
And most of what’s on such places will not use TOML for quite some time yet, as it will date from before that was added as a standard. So copy/paste won’t be enough, they’ll have to add a TOML wrapper. As opposed to PEP 722, where they will have to add a header, and a ## to each line. Yes, both need editing, but IMO PEP 722 is easier.
It’s not, but there are a number of other ways of writing the latter:
project.dependencies = ["rich", "numpy"]
or
[project]
dependencies = [
"rich",
"numpy", # Note trailing comma and comment
]
or even
"""[project]
dependencies = [
"ri\
ch", "numpy"]
"""
(Yes, I know the last one is pathological, but for a requirement with complex markers someone might do that sort of thing).
The point is that TOML is a complex format, and saying it’s easy based on the fact that you assume people will “only encounter the easy bits” is at best risky, and at worst naive.
Well, the fact that very few libraries offer editing suggests that it’s true. Personally, I wouldn’t even know how to spec code to parse, change and write arbitrary TOML without losing at least some formatting choices (for example that string split over 2 lines with a backslash…)
I have no idea what tools you are thinking about, but most of the ones I am thinking of[1] wouldn’t.
I want to address this specifically, as I think it’s a key point in the whole “why are you ignoring the survey?” argument as well.
I absolutely agree that the packaging ecosystem is confusing and unwelcoming for beginners at the moment. There are multiple tools claiming to be the “right way” to do workflow, and they are all competing with each other. New users are faced with everyone telling them that “tool X is best, you should use it” (and often “you should change your workflow to fit how it does things”!). And they don’t even agree on what X is! Having a single, well-understood and “official” approach would be a significant improvement. But that’s a long way off. I doubt Ofek would be willing to abandon hatch to support poetry, for example, and I’m sure the same is true the other way around - so how do we choose?
But use cases like single-file scripts are largely outside of this conflict. There are currently no well-known solutions in this area (pip-run is niche, and pipx support isn’t released yet). If I’m honest, I think that having poetry and hatch support PEP 722 would worsen the situation, because it would suck people writing single scripts into the whole “which workflow tool” debate. That’s why I chose the terms “innovation” and “experimentation”. I’d like to see smaller tools look at solutions that meet user needs in this area, before tools like hatch come along and say “you should all run your scripts in ~/.local/bin with hatch, and configure your environment cache like this, and actually, why not stop writing scripts and just define your scripts in pyproject.toml, and now that you use hatch why not switch your poetry projects to hatch…”
(Sorry @ofek, that’s a dreadful caricature of some of the more evangelical advocates of workflow tools - I only used hatch because it’s the tool I know best, and because normally, everyone has a go at conda over this. I certainly don’t want to suggest this is common, or that hatch is a particularly bad example. But it does happen.)
But putting aside hyperbole, I confidently expect that ultimately, there will be a single “best practice” for running scripts. Personally, I’d like it not to be one of the big workflow tools, but for it to get built into something like the py launcher. But before that can happen we need developers to experiment, in order to work out the best ways of doing things like managing and caching environments[2]. I didn’t mean anything more than that, and I certainly wasn’t advocating for multiple solutions in the longer term.
Actually, I was going to make this a footnote, but I want to call it out explicitly. One of the reasons I see “must be usable in tools not written in Python, that can’t easily parse Python source” as an important requirement is because I’d ultimately like to see the py launcher handle this sort of “script runner” role. And the existing launchers are written in C and Rust.
Also, as per my previous comment, if we can get some sort of “run script with dependencies” into the launcher, why use a workflow tool for intended project developers, rather than just the standard launcher? (An if the launcher is the long-term goal but doesn’t handle this yet, why not use a small dedicated tool as an interim solution?)
Runners like
pipxandpip-run, audit tools that scan a directory to find out what libraries are used, environment builders that build a combined environment to support a set of scripts… ↩︎And by the way, the priorities and trade-offs for managing environments for scripts are very different than those for managing project environments. ↩︎