Embedded pylock.toml trailer for locking PEP 723 scripts?

Packaging
1

This is kind of a follow-up to the older thread about locking PEP 723 single-file scripts, but that topic got a bit off-topic towards the end & has been quiet since last August so I thought it would be better to make a new one than do some forum necromancy.

The thing I keep coming back to is that the “single-file script you can distribute” use-case is one of the things Python is great for (if you can restrict yourself to “things that are in the stdlib on debian stable” until PEP 723 is more widely implemented/available :wink: )

Often I want to write a script that does more complex things than you can reasonably/sanely do in a shellscript, but I also want to be able to share it around with others without having to build a package/make a project directory/publish build artifacts somewhere, etc. - I just want a .py file i can read/edit/copy around/send to friends/put in a single-file github gist/run.

At the same time, if I’m going to be sharing a script with others and I’ve been freed from “stdlib only” by PEP 723, i’d really like to have the dependencies locked, so that when $friend tells me it’s throwing an error on their system I can have some level of confidence that it’s my code which is the problem.

Having a script.py.lock next to script.py does work okay for this, but means the script isn’t really self-contained now - I have to remember to scp/curl/send/whatever both files, and if i don’t - or $friend only downloads the .py because the .lock looks funny or whatever - I don’t get any warning or indication that there should’ve been a pylock present.

Zipapps are neat, but if we’re being honest they’re also kind of cursed. they’ve been known to make security tooling very upset, some multi-user environments block them administratively for security reasons, and a surprisingly large number of users don’t even know they exist. They’re also not directly editable, or particularly convenient to assemble.

The obvious thing would be “oh well just put the lock data in the PEP 723 block”, but that ends up being really ugly. The metadata block is nice because it’s this short, human-readable thing; Python version, dependencies, maybe a few tool option blocks. But a lockfile is a pile of machine-generated/machine-managed stuff full of hashes, wheel names, markers, environment support flags, etc - it’s useful! but it’s also nigh-unreadable and far from concise. I would prefer to not have to scroll past a thousand-plus lines of resolved lockfile just to edit my actual script.

So to get to the point: Would it make sense to support embedding a pylock.toml block as a trailer at the end of a PEP 723 script?

Something like:

#!/usr/bin/env python3 --script
# /// script
# requires-python = ">=3.12"
# dependencies = [
#   "rich",
#   "typer",
# ]
# dependency-lock = "inline"
# ///

# actual script content goes here


# /// pylock
# lock-version = "1.0"
# environments = [
#   "sys_platform == 'linux'",
#   "sys_platform == 'darwin'",
#   "sys_platform == 'win32'",
# ]
# requires-python = ">=3.12"
# ... standard pylock.toml contents ...
# ///

I’m not married to dependency-lock = "inline", especially if extra top-level fields in the PEP 723 block are not acceptable. Maybe the # /// pylock marker is enough and tools can just discover it directly. A related sidecar-lockfile marker like dependency-lock = "file" might also be useful, but that is scope creep and I will probably regret mentioning it.

The exact marker is not the hill I care about, though. The part I care about is the UX - one file, easily human-readable/editable, embedded resolved lockfile at EOF where it’s out of the way, no zipapps, just “you can run this script and have some amount of confidence that it will use these exact dependencies, or throw an error”.

A tool could parse the PEP 723 block, find the trailing pylock/pylock.toml block, check that it still matches the script metadata, create/reuse a cached env, install from the lock, and run the script without doing resolution. That seems like it would preserve the nice properties of PEP 723 single-file scripts while still allowing them to be properly locked. You get one normal editable .py file, not a directory, not a sidecar lockfile, and not a zipapp/build artifact.

Is there a reason that this is a non-starter overall? I’ve done some digging around and I didn’t find it suggested elsewhere - could be a skill issue on my part, apologies if so - or has it just really not been explored?

5 Likes
Pre-PEP: Locking a PEP 723 single-file script
2

This is lot of effort, just to avoid useful version pins (in a comment block few people will read anyway) that you personally find ugly.

Personally, it’s be nice to install PEP723 scripts. But I don’t see how an extra pylock.toml file is substantially prettier than a regular pyproject.toml file, and a simple single file project.

3

I think several of the topics raised in the last thread were never really well answered. We’d have to go back to that to pull them all in, but the one which I recall is that nothing prevents a seemingly innocuous script from having real lines of code in the middle of a multi-MB comment block at the end.

It’s very easy to strip comments and see such things… If you know to look.
I can’t help but remain pessimistically convinced that if embedded lock data became the community practice, it would end badly for some users.

A zipapp is just a opaque, but we know it’s opaque. A script with an enormous blob at the end looks transparent, but isn’t.

TBH, I think this is just too soon after script metadata and lockfiles have been standardized to contemplate the next step. uv uses locks with scripts, but pipx doesn’t yet because there’s no clear tool chain to support it. (We’re getting there!)

If pipx can start having lockfile support, as something alongside a script, I’d say we’re starting to see our tools reach a point where we can start working on the next-step innovations for how we bundle these data together.

4

When we were having this conversation last, I was admittedly not in a place where I had enough in me to really sift through the noise and find my own strong opinions. Anthropic had really sucked the lifeblood out of me.

There’s a chance that’s changed (no way to know for sure and only one way to find out).

I already have newer/better answers to some of the qualms, so maybe it’s time :smiling_face_with_horns:

2 Likes
5

Well, no, it’s not just to avoid ugliness; it’s to avoid having to page through hundreds of lines of mechanically-generated lock content before getting to the actual script contents. This is a real usability/accessibility issue, not just an aesthetic one.

This is a fair point, albeit one I don’t think is particularly likely to be an issue since locked scripts will typically have

if __name__ == "__main__":
    raise SystemExit(main())

or similar as the last statements just before the lock comment block (which should prevent anything past that point from running, AFAIK?), and any such hidden code would be easy to detect and flag in any tool which supported generating/updating it.

It does raise something that I’d considered suggesting - having a sort of “end of script” marker which stops the runtime from parsing beyond it in any given file - but that seems like a fix that could easily be worse than the problem, and would require a PEP/runtime changes/etc anyway so it’s basically a non-starter IMO.

Buuuut all that said:

I’m going to take this as a hint to go back to the “old” thread :slight_smile:

6

Yes, it would execute before anything else below it.