I voted for Option 2 mostly because of what it would look like in a standalone script. I expect users to mostly write this out separately without necessarily caring about the conversion to pyproject.toml. Therefore, I don’t want them to have to write out a gratuitous-looking line like “[run]” or “[project]”.
I voted for my favorite of these in the PEP 723 context, but I have some questions about how each of these options would work in a pyproject.toml file.
In Option 1, what do run.dependencies and run. requires-python mean in pyproject.toml? What are their modes of use?
In Option 2, assuming these become allowed fallbacks for package data, I think it’s mostly simple. Would they be mutex with the project variants?
Would a PEP for this have to do anything other than declare these top level fields as fallbacks? (I can’t think of anything offhand.)
In Option 3, I think I have the same question as in Option 1. What does this kind of content in pyproject.toml mean and how can I use it? Without name and version I can’t pip install ., so how do I get at those dependencies and for what purpose?
(This one feels the most confusing to me.)
I have voted for option 3, because that has long seemed the easiest solution to this problem. Names and versions are required for building a wheel, so let’s make them required for building a wheel. Nothing else needs to change, no one needs to learn anything new.
If we’re keen on option 2, I would suggest that it is contained within an implicit [run] block; mucking up pyproject’s namespacing on account of [run] feels regressive - namespaces are a honking good idea someone once said. My personal vote is for “none of the above”, and I haven’t kept up with the PEP 723 discussions to understand why it is being revised.
I answered the poll as if it were a question solely about PEP 723, but I agree, I don’t actually think we’re any closer to consensus that this should be in pyproject.toml at all, at this point.
If we’re all adding context, I voted for 2 because it is less boilerplate and agree with Paul that not using pyproject there would help avoid confusion with pyproject.toml.
I think that none of these options cover either my failed proposals, nor the implications of PEP 735. Either of those, I think, would result in either:
Option 2a: there is a top-level requires-python key and top-level dependencies (or similarly named) table with a more complex structure, which would be more general and cover other kinds of dependencies. Contexts that require knowing “runtime dependencies” would then use their own logic to select a dependency list from that table.
Option 4: the existing project.requires-python is used to determine the runtime Python version requirement, while a dependencies table is added as described above.
That said, I am now putting my support behind option 3, though my reasoning is a bit involved.
Insofar as the status of PEP 723 is what it is, I consider it correct to evaluate this choice based on the effect on pyproject.toml, rather than based on the effect on PEP 723.
To my understanding, the entire reason PEP 723 wasn’t accepted outright - and the reason PEP 722 was rejected - is because there’s an explicit goal of enabling a migration/transition path. That is, the reason TOML formatting is being tolerated (and preferred, in the wake of the user study) is because of the potential benefits of having pyproject.toml-ready TOML, or at least of doing things the same way in each place. As such, if all of these options are rejected and there is no alternative that actually impacts on pyproject.toml, my understanding is that that would mean rejecting PEP 723.
However, pyproject.toml already exists, and script runners will want and need to care about data that is put there directly, not just about inline script metadata. Further, anything that modifies the pyproject.toml spec will become the concern of anyone whose tool works with it. So, changes in that file should inform changes in the design of inline script metadata - not the other way around.
It is far too late to close the barn doors: everyone accepts already that pyproject.toml stores non-wheel-related information, and indeed that is why we are accepting the idea of declaring “runtime dependencies and Python requirements” there at all. As such, if the file already contains data that makes sense in non-wheel contexts, it will be a fool’s errand to try to prevent people from using it in non-wheel contexts - such as a script runner. So we might as well standardize it.
[project.requires-python] already specifies the version(s) of Python required by the Python code. The fact that this is because a built wheel needs to record that information in core metadata, and that said wheel will then be subject to a compatibility check on installation, is not actually relevant. There is no conceivable reason why executing the Python code in a different context, would have a different Python version requirement. It’s the same code, ultimately being used in the same way - bytecode-compilation by a Python interpreter and subsequent execution of those bytecodes. (The Python requirement has nothing to do with the wheel-building process itself.)
[project.dependencies] already specifies third-party libraries required by the Python code. While it’s vaguely plausible that directly running the code might require different third-party libraries vs. using the code (running it via a driver script, or importing it as a library) from a built and installed wheel, that would be a truly exceptional case. We should wait until we know that this is a real phenomenon, who runs into issues because of it, and why.
My proposal before, and PEP 735, seem predicated on the idea of having separate lists of dependencies for different contexts. To me, this makes sense if we either know and decide that there are specific supported contexts; know and decide that there are specific privileged contexts with defined semantics, while everyone else uses their own logic to choose a dependency list; or know and decide that every dependency list is created equal, including the one that (ultimately) tells Pip what to install.
But it doesn’t seem like there’s any consensus here; and the strongest indications I can parse are that “what should Pip install” will always be considered privileged, and the conditional acceptance of PEP 723 suggests that “what should Pipx, pip-run etc. install” is also about to be considered privileged; and anything else won’t be, because they’ll be considered related to “development”.
Thus, it’s hard to see a reason to worry right now about a design that adds one more “special case” - because it won’t prevent building in a general-case option later. For example, PEP 735-like lists could be stored under project.other-dependencies or something like that, and we could be explicit that the “main” [project.dependencies] takes precedence where that makes sense.
Therefore, since the needed information is already expected to be present, and the file is already acceptably used for non-wheel purposes, it makes far more sense to drop the requirement of the file also being usable for wheel purposes - considering that nothing actually compels people to try to build a wheel, anyway. Anything else duplicates the information, and requires policy for resolving conflicts if there is a redundant specification of that information.
I will note that, apart from the implied link to pyproject.toml, any one of these would have been acceptable to me as a compromise candidate replacing both of PEPs 722 and 723. The dealbreaker for me was, and still is, the suggestion that what is agreed here must end up in pyproject.toml.
I really appreciated the care @ofek took to ensure that PEP 723 did not dictate any specific choice for pyproject.toml - the PEP simply says that ifpyproject.toml ever gains a [run] section, it must contain 2 particular keys with specific semantics. A proposal that doesn’t conform to that can simply choose a name other than [run] and everything’s good.
I’m disappointed that there’s still a push to make a decision on pyproject.tomlwithout doing due dilligence on the question and actually addressing the various concerns raised in the “projects not meant to produce a wheel” thread. I understand that it’s triggered by the combination of Brett’s conditional acceptance and an eagerness to have PEP 723 implemented (for scripts) but I don’t think it’s healthy for the ecosystem. It’s certainly not what I want to happen when I say that I favour incremental improvements.
@brettcannon - Just in case I’m misunderstanding your intention here, can you clarify? Is this poll solely about changing the syntax of PEP 723, or is your expectation that you (or someone else) will be submitting a PEP for adding the poll result to pyproject.toml, citing this poll as support for the idea?
It hurts to read this - because the way I see it, I tried to enumerate those concerns (based on my extensive reading of that thread and related threads, my own experience as a user of packaging tools, some anecdotes I’ve heard off-forum, and applying a bit of logical reasoning), and I got shut down and gatekept.
I think the key question here is: are PEP 723 idioms A) meant to be usable only in the context of standalone scripts, or B) do we envision them to be applicable to a regular pyproject.toml?
I voted Option 2 under the assumption that the answer to the above question was A.
The problem wasn’t what you were trying to do, it was simply how you did it. You brought too many things into the mix, and everyone’s burned out with mega-threads that try to solve everything. That’s precisely why I don’t think we’re going to get a solution for pyproject.toml quickly. But that doesn’t mean we can’t look at concerns one at a time.
I’m sorry if it looked that way to you. It was intended to give you guidance on how to better channel your enthusiasm and ideas. We need people with the energy to work on these sorts of problems. But we also need people to keep discussions focused and manageable, otherwise all the energy we have will get wasted in unproductive debates.
Even if this current poll is (A), which I think it should be, should we not also consider PEP 723 in some sense a “pilot phase” for the entire non-wheel project endeavour? Because I can imagine that pretty quickly after PEP 723 is accepted (if that happens), someone will ask the logical next question: I have multiple scripts in a project folder, they all have the same runtime requirements, why can I not copy the PEP 723 block into pyproject.toml, instead of duplicating it?
Edit: To clarify, I don’t mean to implicitly broaden the discussion to (B). I only mean to consider not choosing something that is known incompatible with (B).
To me, this is morally the same question as “I have multiple scripts, they all use the same utility functions, why can’t I have them share those utility functions”? To which the answer is: if you’re sharing something (functions, metadata…) accross individual scripts, it’s time to create an actual Python package for them.
I did not vote, as I don’t care which one PEP 723 ends up using. I just want PEP 723 accepted and widely implemented, and I feel that blocking acceptance until a corresponding change is made to pyproject.toml is unnecessary.
I rewrote a script last week. Did I reach for PEP 723 to document my dependencies? No. The multiple cognitive hurdles involved (forward slash / or backslash \, how many slashes?, pyproject but not pyproject.toml, [run], remember to close the block again) just doesn’t seem worth it while hurrying to solve a problem.
And if I use it as documentation only (rather than use a script-runner), I can’t un-comment and copy-paste into a requirements.txt to install deps using standard pip.
Anything to reduce the multiple 723-unique requirements is a big plus. So I voted for 2.
I mostly agree with your other points, I think.
But I either disagree on this or urge more caution on how you approach it.
If I have a project which publishes to readthedocs, and RTD is configured to use py3.12, I want my local doc build to use 3.12
There are similar cases like that. Environment managers like hatch and tox let me specify which python version is used where.
And markers let me control the dependencies on a per version basis.
So there are certainly multiple run contexts for a project with different version constraints. The only thing that [project.requires-python] specifies is what goes into your wheel. It can be used for other purposes, but it’s not obvious that it should be. My own opinion is that it shouldn’t be reused.
Just FYI about Option 1 (PEP 723), I understand nothing exists yet but the goal is to have the contents managed by tooling so you never have to manually type anything except on the command line.
This is an interesting point, because I did not consider this specification to be about environment selection at all, but only about requirements. If my script needs at least Python 3.10 to run, and I invoke $TOOL run myscript.py then I expect $TOOL to come up with an environment that has at least Python 3.10, but I expect nothing else. On the other hand, if I wanted to run my script in a special build-docs environment that mirrors my RTD setup, then I would expect having to do something akin to $TOOL run -e build-docs myscript.py. But that seems orthogonal to the dependency and Python requirement specification.
