Hi Pradyun, thank you very much for the feedback.
I see things from a slightly different perspective:
We are changing the way setuptools work because of a few motivations: to catch up with PEP 660, to avoid making internal scripts/packages/modues available via sys.path, and to address a series of accumulated issues because of the limitations of the static .pth approach (e.g.pypa/setuptools#230, pypa/setuptools#1801, pypa/setuptools#2662, pypa/setuptools#3399, …).
Between the default editable exerience and the strict mode, setuptools should be able to support all kinds of project layouts. The cost of adding these improvements will be (justifiably) some level of churn until the users understand how to deal with this new dynamic[1].
The only use case identified so far that is not solved by either the default behaviour or the strict mode is the one presented by Stéphane: a flat-layout package using pkgutil namespaces that try to reach for files outside of the package directory with Path(__file__).parent.parent.parent.
If I understood correctly, this use case does not really fit the mindset of PEP 660 (I think that no PyPA standard offers any guarantee about accessing files outside the package directory using the value of __file__).
Therefore, the compat mode is not really a functionality that we are introducing or that we want the users to learn about. Instead, it is a temporary escape hatch (I hope that the documentation makes this clear). This way users have some time to implement any required changes or report use cases that are not covered by the other “blessed” editable installation modes.
I am more than happy to reconsider compat if we identify other use cases requiring it.
But since I personally don’t plan to keep providing support for compat[2], I believe the best is to let it go after the end of the year.
Any sufficiently ambitious/complex change or PEP unfortunately will have this side effect in
setuptools. ↩︎Given the limitations of the static
.pthapproach, fixing issues or adding features would likely require using file links or path hooks, which is already what the default behaviour / “strict” mode do. ↩︎