PEP 722: Dependency specification for single-file scripts

I felt like I was just conceding more and more, and when I stopped to think about it, I had given up on more than I was actually happy with, between the TOML format, the parallels with pyproject.toml (implied rather than explicit, but still there), not deferring requires-python to a later spec, etc. And none of the objections I raised in PEP 722 had actually been addressed, they’d simply been ignored in the interests of compromise.

And on the level of intention, it felt like I wanted to standardise existing practice, and you wanted to design a new feature, and I think that’s an important distinction which we’d made no progress on resolving.

Ultimately, I guess I felt that I’d moved my position far more than you had, and that didn’t feel right.

But regardless, thanks for being open to the idea of a compromise solution. I hope you can come up with a good final version of 723, which makes your arguments the way you want it to. And then it’s up to Brett.

8 Likes

I am worried that there’s a chance we miss the forest for the trees when “choosing” between PEP 722/723.

I voiced this in PEP 723: Embedding pyproject.toml in single-file scripts which got split to PEP 723: use a new `[run]` table instead of `[project]`?, but it applies here as well.

I can speak personally, that if I had to choose between these two PEPs I would likely settle on PEP 722, yet it might not be as robust or flexible as a solution that finds a way to embed structured metadata into the script (that isn’t labeled as “pyproject.toml”, as I know you and others have concerns). Then we’re forced to either find a way to “extend” the embedded metadata approach of PEP 722, or invent a new one to replace or live alongside PEP 722 [1] once we want to embed more.

I honestly have the capacity and stamina to co-author a PEP to lay the groundwork taking everything said here (and the other 3 or 4 parallel discussions), but I also know that’s somewhat rude and stressful to you (Paul), Ofek, and Brett. So I’ll just voice my concern, and won’t pursue that unless asked by one of y’all.

EDIT: (I’m backpedaling on the “accidental” wording here, and the expectation of this syntax being extended after re-reading the PEP)


Regardless of my concern, I’m personally excited that we’re addressing a gap in support for what I perceive is a decent chunk of Python’s usage. So :tada:
And professionally, I’ll parrot myself from the PEP 723 discussion

Speaking on behalf of Pex and Pantsbuild, we will support whatever decision is accepted, and neither seem technically infeasible.


  1. and then people lob tomatoes and scream fragmentation. ↩︎

This is the crux I think (not picking on you personally but using this message to point this out):
PEP 722’s goal is not to embed metadata into a script. It just wants to allow script authors to write down their dependencies so that script runners can make them available.

3 Likes

Maybe it’s pedantic, but that most certainly is metadata (whether its called that or not on the tin).

From Wikipedia:

Metadata (or metainformation) is “data that provides information about other data”,[1] but not the content of the data

The dependencies of a script is data that provides information about the script.

…Actually, it IS called that on the tin! The PEP itself in the “Rationale” section:

We define the concept of a metadata block that contains information about a script. The only type of metadata defined here is dependency information, but making the concept general allows expansion in the future, should it be needed.

(That’s not even my emphasis, “metadata” is already emphasized in the PEP)


And that’s the entire point of my concern… We accidentally standardized a way to embed metadata.

EDIT: Although, from the PEP I should backpedal the “accidental” part. I’ll own up to that.

3 Likes

(I need to make a dedicated comment for this, apologies for the noise)

I’d like to rephrase my concern (and apologize for my earlier wording), since I see now we’re explicitly standardizing a way of embedding metadata.

My concern is that we standardize this way of embedding metadata through the lens of this use-case and not others. So still the same forest/trees concern, but certainly not accidental (and again, my apologies)

1 Like

I am concerned about that too. However, @brettcannon indicated on a related thread that there will solicitation of user feedback in the process, and that it remains quite possible that that feedback will indicate a solution that differs from both proposals. That alleviates my worries somewhat.[1]


  1. I do still think it would often be useful, not just for this PEP but for many ideas in the past and future, to get such user feedback at an earlier stage and use it to inform the initial drafting of a PEP, but still, getting it and taking account of it at any stage is valuable. ↩︎

2 Likes

This is absolutely correct.

To expand a little, PEP 722’s goal is to standardise existing practice in the area of allowing script authors to write down what distributions they need available in order to run.

It seems to me that PEP 723 is trying more to address the question of “how do we expand the idea of metadata (pyproject.toml style) to cover single Python files”. That’s a much bigger question, and while I think it’s potentially worthwhile to address, I don’t think we yet have sufficient evidence that people need that capability[1], and I definitely don’t think we’re even close to a good idea of what such an expansion should look like.

I understand that some people are uncomfortable with small, incremental improvements, as they are concerned that such changes ignore the longer term. I disagree with this, personally. My view is that incremental change is crucial if we are to progress - endlessly debating the “long term” will simply drain everyone’s energy, and responding to every proposal with “but what about the bigger picture” ignores the reality of how volunteer-driven projects tend to progress, which is in terms of small, focused PRs and closely scoped feature developments.


  1. Outside of the limited situation where it addresses the “running a script with its dependencies” case. ↩︎

5 Likes

You’re reading an out of date version of PEP 722. The current version removed the concept of a “metadata block”. This was all covered earlier in this thread, although I can understand if you missed it - there’s a lot to keep up with (I should know, I’m exhausted!)

I should’ve known better than to atke the link from the very top OP.

So I redact my redaction? This is getting confusing :sweat_smile:

I think everyone gets the gist (I hope) I’ll tap out before I hit strike 3

I’ve edited the original post, to give the correct URL.

1 Like

I think that’s more of a discussion between you and @ofek about if he agrees with you and wants to change PEP 723 to align with what you’re suggesting. Otherwise I’m not sure how your PEP would differ from PEP 723 beyond what TOML data is included or some bikeshed on how the TOML is embedded.

1 Like

I agree with what Brett said but Joshua feel free to message me here, X (formerly Twitter) or Discord

1 Like

I do not like specifying requirements in the comments block. It looks very strange that comments, which should be normally ignored, in this case have a special meaning.
For Python newbies not familiar with packaging, this PEP is not the solution. It takes additional effort to figure out that this kind of syntax is used. Do not forget that if this PEP is accepted (which I would like to avoid), this practice has to spread across the community, which may take a long, long time.

A newbie who would like to package his one-file script would encounter solutions based on requirements.txt, pyproject.toml, etc.) on the web much more frequently because they exist for a long time. It may occur that this solution is one more niche, which for insiders is yet another way to package (but they may prefer more cleaner and standardized ways like requirements.txt, pyproject.toml etc) and for the rest, it may be only an unknown curiosity or thing to ignore (because it looks like usual comments).

To summarize. Please do not accept that and do not introduce “yet another way” to package.

This PEP isn’t meant to be a solution specifically for “newbies
unfamiliar with packaging” nor is it a way of packaging anything at
all (much less another way of anything).

Many of us who write quick Python scripts already put comments in
the beginning of them with a list of the non-stdlib dependencies the
script needs preinstalled. There are even tools which will read such
comments and autocreate suitable ephemeral environments before
running the script. This PEP merely standardizes that practice so
that these tools can have a consistent format to collaborate on
rather than each doing it their own way.

Not approving this PEP isn’t going to stop those of us who already
do what it documents, nor is it going to make the tools which use
this information suddenly disappear, all it will do is increase the
odds that two different runners will each expect a different syntax.

13 Likes

It seems there’s still misunderstanding: This is not a way of packaging at all.

This is a way for someone who writes a script to provide a comment to users (“Your environment will need these dependencies to run”), but structure it so that script runners can build an appropriate environment (or verify that the environment has the dependencies).

Critically, this capability already exists, but in different forms in different tools. By agreeing to a common syntax, the script writer who chooses to use it does not have to say “run this with pipx” or “run this with pip-run”, but “If you use pipx or pip-run (or X other runner), it will just work. Otherwise, read the comment and make sure you have these things installed.” I would probably actually make this explicit in a comment:

# This can be run directly with pipx or in any environment with these installed.
# If you need to edit this list, refer to PEP 722 to ensure it remains runnable.
#
# Script dependencies:
#    pandas
#    matplotlib

There’s no requirement that any tool that is not aiming for compatibility to implement this, and there’s no requirement for a user to use those tools.

3 Likes

I think this is true. But it’s probably still worth considering the outside perspective, even if it’s technically incorrect. If this PEP (or 723) is perceived as complicating the ecosystem, that’s not a good outcome.

Not that one person’s comment is representative of anything, but I hope the user studies that Brett’s group is working on will clarify whether this is happening.

6 Likes

Despite the misunderstanding I am not against the idea. I am against the proposed solution. I still think that there is a better way to achieve that, than comments block which may be easily misspelled etc.

Can you share what that better way would be?

1 Like

This is a way for someone who writes a script to provide a comment to users

Probably also worth reiterating, in many cases the “user” is just
your future self.

I don’t typically give these sorts of scripts to others, it’s purely
so I can remember what the script needs the next time I want to run
it. Having a tool that can automatically create the environment that
script needs from the comments I added to it is a bonus, but as much
as anything they’re notes I’ve made to myself within the script so
that I don’t lose them if I move the script to another directory or
to another one of my systems at some point in the future.

5 Likes

I agree with several of the points recently made, that this is not packaging, that it only standardizes an existing practice, etc.

But the note that this may be perceived – whether or not it’s technically correct – as “packaging” or “part of the packaging ecosystem” resonates with me particularly strongly.
Most python users are probably hazy about the boundary between “packaging” and “workflow tools” and so forth. For them, these are all just “packaging tools for python”.

This line of reasoning leads me to slightly favor PEP 722 over 723 if we must choose one.
My rationale is that by staying intentionally distant from pyproject.toml data, we better help users analogize the feature and its usage with requirements.txt files, which is more accurate than analogizing it with building a package with dependencies.


I’m very concerned that voices calling for “no new ways of doing things” effectively leads to stalled progress.

Rather than standardizing on a new behavior which enhances, replaces, and improves upon past art – like requirements.txt files – we’ll be stuck with only the current standards and no new tooling.


As for use of special comments vs any other mechanism…
There are mechanical problems with basically any other solution. This ground has been trod pretty thoroughly here and in the PEP 723 thread, but I’ll try to summarize.

  • it needs to be possible to parse the data in any language, not just python, so it can’t just be some runtime value or attribute
  • if the value is visible at runtime, the runtime value might not match the verbatim values seen in a file, leading to a misleading discrepancy between runtime information and the spec
  • shebangs, encoding comments, and other languages’ solutions to similar problems (e.g. embedded cargo.toml proposed for Rust) are a precedent for magic comments
  • multiline strings introduce additional questions and confusion for users, f-strings would not work and escaping rules become more complex
5 Likes