I really love the newest revision of the PEP! Thank you Barry for proposing this and I hope the SC approves it.
I have one minor suggestion which is to specify that the .start file should be utf-8 encoded to simplify parsing. Optionally, this could be utf-8-sig to allow for BOMs, since pth files allow them, but I think .start files should not fallback to the current locale like pth files do. Digging through git history and discussions, it looks like the locale fallback was added because setuptools was writing locale-encoded pth files already. With a new format we can ensure the file must be UTF-8 and make everyone’s lives easier.
Some of these have been mentioned already, but since it’s up for pronouncement:
I don’t understand the motivation for silent error handling. I haven’t ever seen any complaints about pth’s normal error handling (and authors can always try-except if fallibility is okay)
The deprecation period seems a little short
I’m a little skeptical of the security benefits. I would find it useful to see spelled out what makes PEP 829 more constrained and auditable
The PEP says stuff that could maybe unintentionally mislead readers with less context, e.g. “The overall attack surface is not eliminated — a malicious package can still cause arbitrary code execution via entry points”. There are many other ways a malicious package could cause arbitrary code execution. I think the PEP should mention that somewhere in order to avoid giving some future reader the impression that .pth / .start is the only hole
It’s because we were parsing with the current locale, and had to migrate to UTF-8 in a way that wouldn’t break compatibility. setuptools was writing it because we were reading it.
Aside from explainability, the two main concrete benefits are:
the presence of startup code is visible in a wheel’s file listing (right now, pth files might indicate startup code, while in the future, they will necessarily be passive, while start files will be active)
the entry point import events will arguably be easier to identify in the existing audit hook mechanism than the code compilation hooks that currently fire
Is it a massive improvement? No, but that’s also not the claim.
neither of those are security benefits though. Both are beneficial to auditability. For an issue that has gotten as much visibility as this due to a specific abused case in the wild, and that some knee jerk reactions were wanting the functionality dead entirely, why is that distinction still being lost in presenting it?
It’s important that people get accurate information about what is actually improving and that people don’t get the wrong impression that this somehow makes their existing choice to install a package any more or less secure.
Is it even possible to have a .pth file in a wheel that isn’t the code execution type and still makes sense? Sticking an absolute path in a wheel is very unlikely to be portable and relative paths within site-packages sound even less useful.
So for all intents an purposes, couldn’t a .pth in a wheel already be assumed to be the run code on startup type?
I agree that .start files should be utf-8 or probably utf-8-sig. It’s a little tricky in my reference implementation because I’ve refactored the code so that both the .pth and .start files are read using the same logic. So I wonder if we should also deprecate the fallback to current locale for .pth files too? I guess it wouldn’t be that ugly to switch on the suffix to decide whether to fallback or not.
Philosophically, I’d like for both files to use the same rules and don’t want to gate the semantics on the implementation.
You’re assuming usage in publicly released libraries though. Both path extension mechanisms could be useful in non-public code where the execution environment is more tightly constrained.
I agree with both points @ncoghlan, and thanks for clearly expressing it. I think people are getting hung up on claims that the PEP isn’t making, but I take that as my responsibility to clarify the PEP. I might just lift these bullet points (with Acknowledgement of course!).
Generally speaking, we want to avoid annoying end users with errors that they can’t do anything about[1]. I would hope that most developers would catch any problems during testing, long before the code gets into end users hands.
I think a three year deprecation period for the .pth code execution functionality is a good compromise. We really want to get these out of the ecosystem, and with @ncoghlan 's excellent suggestion about an easy migration path and straddling approach, I totally think it’s doable. We’ll add another two years for silently ignoring import lines in .pth files so users of straddling code won’t get overwhelmed with inactionable error messages while Python versions that both support and don’t support 829 are active.
except perhaps report a bug, which can be … unsatisfying ↩︎
The SC is comfortable with the three-year deprecation window as proposed. However, we request the following revisions to the text before the PEP lands:
The current abstract buries the lede a bit. The first paragraph should highlight what is materially changing: <name>.start files with pkg.mod:callable syntax, a three-year .pth import-line deprecation, and retained path-extension .pth.
The security implications should be more specific. The PEP narrows one specific vector: supply-chain attacks via exec() of .pth import lines. The changes do not materially alter the broader threat model. We recommend explicitly acknowledging that the overall “malicious installed package runs code at startup" threat is unchanged. The concrete win is that the code a package intends to run becomes inspectable as a declarative entry point rather than an opaque string passed to exec().
The SC views PEP 829 as a sensible improvement to the interpreter layer, compatible with and not precluding future work in the installer and packaging standards space. We encourage contributors interested in that broader direction to consider proposing complementary standards. Accepting PEP 829 is not a statement that the interpreter layer is the end of this work.
Thank you for your work on this, Barry, and congratulations!
Edit: …and just for the record, as PEP author, Barry had no part in this decision!