More precisely, the use of .pth files to add directories to sys.path is well known and supported by most static analysis tools.
Anything that requires code execution to dynamically affect the import system (import hooks, self-replacing modules, etc) is not amenable to static analysis and therefore makes editable installs distinctly more difficult to work with than “normal” installs.
A standard for statically describing all of the possible semantics of editable installs (which could either incorporate or supersede .pth files, depending on backward compatibility considerations) would be a significant advantage for such tools. We could then have a mechanism for implementing an editable install based on that static description. That mechanism could be a 3rd party library[1] or it could be in the core interpreter or stdlib[2].
Unfortunately, no-one has yet shown any interest in doing this work - neither formally defining what it means to be an “editable install”, nor standardising a static description mechanism. And to be explicit here, it’s not something I want to implement in the editables library - it’s more than I personally have time to develop or support, so I’d rather leave it to someone who does have the necessary time[3].
Full disclosure: This is very close in some ways to the idea proposed in PEP 662. And as such it has the same problems as that PEP - it’s easy to talk in generalities, but to be useful, there’s a whole lot of detail that needs to be thrashed out by someone.
Which would need to be activated early in the interpreter startup process, something that currently requires the “execute arbitrary code at startup” feature of .pth files ↩︎
Taking advantage of the core’s privileged position of having direct access to the interpreter startup machinery ↩︎
I’d be interested in contributing to such an effort, but wouldn’t want to lead it ↩︎
Which is why my position the whole way through was to avoid over-specifying how they should work. There’s absolutely no harm in letting tools provide any option they want here, provided the term “editable install” is defined only well enough to exclude things that are obviously not editable installs (such as system-provided packages or install-on-demand packages or a range of obvious things that we don’t mean). The process was bumpy because people insisted on trying to codify a specific behaviour.
From the POV of running code at startup, I think the way setuptools does it is basically fine (the distutils hack could’ve been pushed into the module rather than checking environment variables in the .pth file, but no big deal), and already consistent enough with the more restricted model that Barry is proposing.[1]
I’m also personally just fine with the current .pth behaviour and don’t see a need to restrict or change it at the core level. I’d be quite okay with limiting or restricting whether packages from online can create/install .pth files, but once they’re on disk, I think they’re already fine. Which means my position is that editable installs should get 100% expressive power, because they are entirely local.[2]
The “laziness” was only really referring to “hey, we can write code in the .pth file instead of putting it in a .py file and installing that as part of setting up the editable install”. Maybe it wasn’t considered, maybe it was, I don’t know. I’m sure every time it came up during the original discussions it was swamped by other threads, and I did follow that discussion closely enough to count myself part of it (though I didn’t contribute much). So I don’t think I meant it in an unfair way, though of course it’s possible to take it in a way that feels unfair. I apologise if that led to any offence being taken. No voluntary contribution can ever be the result of laziness, but perhaps I have that so internalised that I didn’t consider others might read it as “he was so lazy that he made something and gave it away for free”
Though I haven’t seen the update post that he mentioned yet. ↩︎
And online installs should probably get 0% power, since the only choices are 0% or 100% and we don’t really like giving them 100%. ↩︎
