Pre-PEP: Locking a PEP 723 single-file script

Does targeting zipapps make more sense at this stage? Other than the concern @pf_moore expressed as to whether we were in a position to say anything regarding zipapps as it’s only really defined in the stdlib docs via PEP 441, people keep bringing up zipapps.

They include a build step, which side-steps any concerns about peoples’ general expectation around scripts. It also keeps the lock file with the code it is meant to lock.

Probably the biggest question would be what tools should do if the shebang line is set to a Python interpreter version and the zipapp contains a lock file that could support a different Python version? Otherwise saying that if there’s a pylock.toml in the zip file then tools like pipx can create a virtual environment and use the lock file to populate the environment is the extent of what I’m suggesting.

2 Likes

I’m unsure. I don’t particularly have a usecase that overlaps with some of the people in support of this, and I don’t particularly agree with the ideas people have about what this is meant to accomplish. That isn’t to say that the goals aren’t valid, but that I’m not sure that any amount of further lockfile specification actually helps here. zipapps can already be distributed as a single file that vendors all needed dependencies. This is inherently more reproducible than relying on a fallible external package index. There’s also tools like pex and nuitka that turn existing python environments with an entrypoint into standalone executables. These executables can be built in such a way that they are a reproducible and verifiable build artifact by using the already existing pylock specification to lock the environment used to produce the standalone executable.

I would want to hear from someone who wants reproducibility and singlefile deployments simultaneously why those and other existing solutions are not more appropriate.

1 Like

That’s not necessarily true; if the dependencies include extension modules then it won’t work and you have to use another tool to do the packaging like pex and shiv which will effectively unzip into a temporary directory and then have Python execute the code from the temp directory.

1 Like

You could do everything shiv does yourself and produce that zipapp, or just use shiv to do it, but it’s still a zipapp containing all dependencies [1] as I understand it.


  1. At least all of the ones that would ever be covered by pylock, things like using ctypes to open and call into a shared library assumed to be present, are still not. ↩︎

1 Like

I think original post (and the subsequent few messages) already says this, no? (Unless you meant “hear from someone other than the original poster”)

But in case it needs to be said, distributing bytes-but-not-text is a nonstarter. (If I’m gonna distribute a binary I might as well use Go :wink:) Especially considering bundling the dependencies in the script bloats the distributable.

I don’t know if this helps the discussion, but I’ve been thinking a bit about the distinction between a “script” and an “application” recently.

For me, the key thing that means my Python code changes from being a “script” to an “application” is when I need any form of build step. A “script”, for me, needs no build step - what you distribute is your source. Whereas with an “application”, what you distribute is something you built from your source - you can’t just take the code on the user’s machine and edit it to fix an issue, for example, you need to create a source directory, make the fix, and rebuild the application.

In that context, distributing bytes rather than text is an application. Even if all you did was zip up the source directory, zipping is still that build step[1].

With that in mind, a locked script is right on the borderline between a script and an application, in a way that a normal script with PEP 723 dependencies isn’t. Because there’s creating the lockfile data, which is effectively a build step. Ideally a tool can refresh the lockfile from the PEP 723 dependencies in place, so there’s no “source” and “distributable” distinction[2]. But you still have to “do a build” after making changes, and you have the question of whether a change “needs a rebuild”. You also might need to rebuild when a new version of a dependency is released, once again emphasising the difference between the “source” and the “distributed” version.

So what makes me most nervous about this proposal is that it brings the overheads of managing an application to the process of publishing a script. And I’m not sure that’s worthwhile - maybe we should simply accept that “if you need an application, you should build one”. You say that “bytes rather than text is a nonstarter” - but why? Is it because the user you distributed the script to can’t make changes to it without needing a build setup? But that’s true of a script with a lockfile block - add a new import and you might need to relock, which could need a tool you don’t have (as an end user). We could add limitations, and say that as long as changes don’t require a rebuild, you can just edit the script - and maybe that’s OK, but it’s a fine distinction that users (both developers and users of locked scripts) need to understand. And with the same provisos, you can also do that with a zipapp - unzip, edit, rezip. It’s not that hard.

Obviously, this is just my personal view on where the difference between “script” and “application” lies. I’m not actually too attached to using the individual words in this way (although I do think it’s a useful distinction to have). But what I do believe is that this PEP needs to discuss, and properly evaluate, the consequences of “making a script need a build step”.


  1. And you probably had to assemble the dependencies as well, which is also part of the build ↩︎

  2. In fact, I’d probably consider having such a tool to be essential for locked scripts to be considered “scripts” ↩︎

6 Likes

One of the powers of python is the textacutable (just made that up, proud of it too). All of my scripts live in one git repository or another. I can read em. My colleagues can read em. When I’m ready to share them, I can just point to the “perma-” URL for the version I want to share.

Binaries in this matter poopoo all over the place.
I can’t read em, I can’t diff them, tools like git are harmful towards them.

And it’s odd to me that if I want to combine a text script and a text dependency spec my only option is binary.

1 Like

That being said. I’m starting to think maybe the drawbacks of a full lock file inside a script is likely going to out weight the pros.

I wonder if instead, specifying how to find an associated lock file alongside a script (i.e. just a convention for the accompanying file’s name), which uv already does, is the way to go.

It’s simple. It’s straightforward. And I think still covers all the points from OP, except for the single-fileness.

(Should I make a new post or just pivot this one?)

It’s certainly a lot simpler. And it doesn’t even need a standard - a per-tool convention is probably sufficient (“please follow the same naming convention as uv, to improve interoperability” seems like a very reasonable feature request for any other tool implementing this).

Once the convention is established and well known, then is probably the time to consider making it a standard. Much like the .venv naming convention for virtual environments, which has managed for years now without needing to be standardised, but might now get a PEP.

Personally I’d be OK with continuing the discussion here. But as I say, there’s not much to discuss, if uv already supports this. Although from a quick investigation, uv doesn’t use standard lockfiles for this, and it doesn’t support fetching lockfiles over HTTP (uv run https://...../script.py). Getting those aspects addressed is probably going to be more about raising feature requests with uv than about discussing a standard. In particular, it would be rather impolite to try to create a standard that would force uv to change without getting them to agree to the change first…

4 Likes

So then what we may be looking for is something a person can do that can still somehow help with reproducibility in any ephemeral environment that gets created for a script. The solution might not be a rigorous as a lock file, but maybe what we want is reproducibility for ease of use (i.e. not being broken by some new version of a project).

One possible solution would be uv’s exclude-newer key (a quick check didn’t turn up a similar feature in PDM, Hatch, or pipx). That sets an upper-bound on what versions to consider based on a provided timestamp. While that isn’t exactly a lock file, it does help with reproducibility with a setting that a person can set/edit while not requiring a user to have to set manual upper-bounds on everything (including indirect dependencies which might be a cause of breakage).

4 Likes

So my original motivating case was executing a script, from a GitHub URL as a part of CI/CD automation, at work.

If I break down my shopping list of features:

  • Version reproducibility (e.g. no surprising failures due to dep versions): I think Brett’s suggestion holds here. I could put a timestamp of (last known good run) and bound the versions that way
  • Version Reproducibility (hashes): I don’t think this is possible without dependency pinning, but the point is to add some supply chain trust
  • Cheeseshop URL(s): I think this is pretty easy to do

So then I think what I’m looking at is either forcing callers to provide these values or embed them in the script. I’d imagine the stepping stone here would be convincing uv to allow me to embed these in a [tool] block in the script metadata.

1 Like

I’m still not clear why it’s not possible to have a lockfile that’s separate, but whose location (name, url) is derivable from the script name. Something like the lockfile for https://github.com/owner/project/path/script.py being https://github.com/owner/project/path/pylock.script.toml.

Then you can do uv run --locked https://github.com/owner/project/path/script.py.

That seems like a perfectly reasonable feature request for uv, pipx, or any other script runner.

3 Likes

(That was mentioned above so I didn’t remention it - I’m also supportive of that)

2 Likes

One potential issue is when the URL of the script is pre-authenticated, with the auth part only valid for one URL. This is surmountable if the tool allows the user to provide the URL to the lock file as well.

This is, of course, possible.

In fact this is basically already implemented in uv, I’m just not sure if it automatically works via http.

However that’s not the request here, the request is to be able to embed the lock file within the script. So I assume the question you mean is: Why doesn’t that meet people’s use cases?

Here is are slightly more verbose descriptions of what I describe in the uv feature request:

  1. Convenience - It’s a lot easier to send someone one file than two file (my use case would be sending over a messaging service not providing over a web server). It’s a lot easier to maintain one file than two files.
  2. Prevention of Decoupling - When there are two files, where one is metadata for the other, it’s easy to lose one file and for the user to just run the main file without the --locked option. E.g. they just download the script and not the lock file from the messaging service.
  3. Prevention of Desyncing - Similar to decoupling it would be a lot easier the lock file to desync, e.g. a second user grabs an older version of a lock file from the script and the dependencies and lock file don’t align any more because the first user updated the script without the presence of the lock file, but now there are conflicts that the tool never got to warn about when the first user initially added those dependencies but the second user doesn’t know how to handle that.

These reasons for me make keeping the script and the lock file as two separate files very unappealing to use in practice.

4 Likes

Hello everyone from 2026!

an accompanying post has brought up this subject again, and almost 1 whole rotation around the sun later and now that “let me sleep on it” has given me “let me sleep 300-or-so times on it”, I think it’s time we continue.

After re-reading all (most :shushing_face:) of the comments here, I see a clear trend on “yes this is good” along with a lot of discussion around some details as well as a lot of discussion around what I’ll call “locking in general”.

I don’t see a reason why not to make a PEP, so I’ll start that.

Then for some boundaries/parameters:

  • I’m only interested in embedding pylock toml blocks inside single-file Python scripts. Put another way, I’m not interested in zipapps or similar. (To be very clear, I think there is merit in some proposal(s) which solve gaps using zipapps, I’m just not interested in them for this proposal)
  • I’m not (particularly) interested in nuanced discussion of the cons of locking in general. These aren’t going to serve us in furthering the discussion. That being said, useful suggestions around guidance I think is helpful (example: an earlier suggestion on encouraging tools to provide mechanisms to check and update the locked section
  • The spec already specifies that inline metadata MUST be continuous. I think the spec should tighten this, but that’d be a different thread so I’ll start that discussion and we can wash our hands of it here

Other than that I need a sponsor. :smiley:

2 Likes

Agreed.

I don’t think you can ignore this totally. If nothing else, the likelihood that script locks will not be updated in a timely manner, simply because people don’t have the infrastructure around scripts that they do around applications or libraries, means that out of date locks is a security issue, and should be covered under “Security implications” in the PEP. It may be that all we can do is educate people about the risks (which means we cover it under “How to teach this”) but it does need to be in there.

I’m also still concerned that the ergonomics of a 50-line script with a 300-line lock data block are pretty terrible. “Don’t use it if you don’t like it” is a fair answer here, but I think we need to acknowledge that it’s an issue.

Personally, I don’t have a good feel for how people will create scripts with embedded lock data. It’s not necessarily part of the PEP, but it is important context. Do you imagine some tool that takes a script with dependency metadata, and calculates a lockfile for it and appends that lockfile to the script? Or do you see people creating the block manually (i.e., creating a lockfile for the script dependencies using existing tools like pip lock, copy/pasting the file into the script, prepending a hash to each line and manually adding # /// lock# ///)?

On the subject of tooling, what happens if the lock data doesn’t apply to the environment the script is running in? Does the answer differ if there’s also (unlocked) dependency metadata that can be used as a fallback? What if the dependency data and the lock data don’t match? Are tools expected to check? If they don’t, there’s another attack vector to consider.

Also, I’ll note that the current spec doesn’t say what tools must do when there’s more than one standardised block type. Blocks that have a non-standard type must be ignored, but if script and lock types are both standard, are tools allowed to only process one type? I think we have to allow that, for backward compatibility. So that means that a script with script and lock metadata would behave differently depending on the tool used to run it. Is that OK?

Regardless, the new PEP must specify what happens when a file has multiple metadata block types present. Simply because the current spec doesn’t say.

I’m not sure I agree - the fact that you can have multiple metadata blocks (of different types) in a script is a feature, and the ability to put them in different places is necessary (the fact that dependency data belongs at the top and lock data belongs at the bottom seems pretty self-evident to me). I’ll object strongly to any attempt to remove that flexibility. So we’re left with a situation where once we normalise the existence of a massive, opaque, script metadata block, we make it rather too easy to “hide” executable code:

# /// script
# dependencies = [ "requests" ]
# ///
import sys
import requests

requests.get(some_url)

# /// lock
# <lockfile data>
# ///
import os
os.system("rm -rf ~")
# /// dummy
# <more data that looks like a lockfile...>
# ///

Yes, “people should audit scripts before running them”, and yes, “once you run a script off the internet all bets are off”, but we shouldn’t be making attachers’ lives easy :slightly_frowning_face:

I’d argue that the original intent of PEP 723 was that metadata blocks would be human-readable and human-edited. Using them for machine-generated, opaque data blocks isn’t prohibited, but it doesn’t fit well with how I see script metadata being used. @sirosen mentioned this explicitly [above]

Assuming you manage to address these points in your PEP, though, I’m willing to sponsor it.

4 Likes

I don’t think this is entirely fair, actually. How is this any different to the current sidecar-lockfile situation? There’s no inline script metadata field for indicating that a lock file is expected, and it’s just as easy to neglect to update a sidecar lock file. Putting the lock data inside the script itself can only be an improvement in that respect, since it makes it significantly harder to lose and easier to “ship” so to speak.

Well, yes, because that’s almost exactly how the tooling already works. You manage single-file script dependencies/venvs/etc via something like uv --script or similar, and (for example) uv lock --script <scriptfile.py> will generate a lockfile next to the script. I’d expect something like uv lock --script <scriptfile.py> --inline would instead create/update the embedded lock data.

No tool is required to process any type of block at all, so I don’t think this is really an issue in practice. Could also be covered by a new optional top-level field in script, e.g. requires-lock: "inline" or something along those lines, to inform tools which understand it that they should expect lock data to be present - and that would help mitigate some of the security concerns for both inline and side-by-side lock data, i’d think.

Personally I’d argue that inline metadata blocks should only be permitted at the start or end of a script, i.e. any inline metadata that occurs between the first python statement and last python statement within a file should throw some kind of warning. But that’s a fairly big hammer and one that the runtime can’t reasonably be expected to enforce considering these blocks are just comments. A more reasonable solution would be having the script-runner tools check for buried code like this and yell about it, probably.

1 Like

Having different types (/// script & /// lock) at different ends (top & bottom) makes sense, and even other types /// foo can be allowed to exist elsewhere in the file, but beyond that, I don’t see what the flexibility you mention buys us, except headaches.

IMO it’d be completely reasonable to require that /// lock blocks MUST be at the end (which doesn’t prohibit other comments or blocks in the middle of the file). In fact, I think it’d be unreasonable to intentionally end up in

There would have to be a strong argument in favour of concrete missed use-cases (rather than generic flexibility) to outweigh the burden we’d be saddling everyone with by allowing this massive footgun. I just don’t see such a case existing – though perhaps I’m not imaginative enough – so I was hoping you could describe what aspect of this flexibility is so important to you, and what concrete scenario would be rendered infeasible by requiring /// lock sections to be at the end & contiguous.

2 Likes

The behaviour you describe here is all I covered in my comment, so I’m not sure if we’re agreeing or disagreeing here…?

Making lock blocks a special case feels odd to me.

I can’t, really, because the script block spec is written to support general metadata block types, but currently we only have “script” blocks. The proposal to add lock data uses the generality to add a new block type without needing new parsing rules, but the rules are still generic, and I can’t tell you what future block types people might find a use for.

I will note that PEP 723 was accepted in preference to PEP 722 at least in part because it offered a more general mechanism. Removing that generality “because we don’t currently need it” feels to me like it’s missing the point…

“Contiguous” makes no sense here. The spec only allows a single “lock” section in a file:

When there are multiple comment blocks of the same TYPE defined, tools MUST produce an error.

As far as requiring the lock section to be at the end of the file, how about those editor variable blocks that Emacs users like to include in their files? Those go at the end of the file, I believe. Is it OK for those to go after the lock section?

To be clear, I don’t have any problem with a strong recommendation that lock sections go at the end of the file. That’s both reasonable and acceptable under the current spec.

My objection is to the idea that “tightening the rules” (in a way that’s not explicitly stated here, we should go to the other thread that @thejcannon started for specifics) will make any real difference. What really matters in this thread are the implications of normalising the idea that Python scripts can contain large blocks of generated data that’s not intended to be read by a human. And that’s something that a PEP proposing a “lock” metadata type needs to discuss.