Do we want to keep the `[build-system]` default for `pyproject.toml`?

Packaging
21

This is where I keep having trouble with the example here. Speaking as a pip maintainer, we have never suggested or supported the practice of using pyproject.toml to store application dependencies. And there are no standards supporting that usage. As far as I can tell, this is an approach that has originated with uv (other managers like poetry or hatch may support it as well - I don’t have enough experience with them to be sure).

Pip has a strong principle of being based on standards[1], and I’m concerned that if we want to adopt this approach of using pyproject.toml for applications that aren’t meant to be built into a wheel, we should make sure it’s standardised. We’ve had attempts to do that, but not been able to get consensus or a clear way forward.

I’m not against making this an accepted practice. I just want it formalised, so we aren’t making one tool’s implementation defined behaviour something that everyone has to track.

This (recording a set of dependencies that are not intended to go into a wheel) is precisely what dependency groups were intended for. So while the UI might not be ideal for this user, I do think we should be considering whether the problem here is that tools aren’t making the dependency group UI sufficiently usable. Maybe uv should be using dependency group “main” rather than project.dependencies for this case?

Which brings us back to the point that maybe we should get community consensus on how non-wheel projects should be handled.

  1. in reaction to being the “only installer” for so long that “what pip does” became a de facto standard ↩︎

4 Likes
22

FWIW, when the PEP517/PEP518 builds were implemented in Fedora, we copied pip’s behavior to only fallback to setuptools when there is a setup.py file. That behavior made sense to me, but it was not explicitly stated that way in the PEPs.

However, recently somebody reported that as a problem, stating that pip no longer requires existing setup.py file to default to setuptools when there is no build backend specified. This has happened in Fix PEP 660 metadata preparation fallback by sbidoul · Pull Request #10577 · pypa/pip · GitHub

I didn’t get back to changing the behavior in Fedora. I don’t think defaulting to setuptools without a setup.py file makes sense.

Context from PEP 517:

If the pyproject.toml file is absent, or the build-backend key is missing, the source tree is not using this specification, and tools should revert to the legacy behaviour of running setup.py (either directly, or by implicitly invoking the setuptools.build_meta:__legacy__ backend).

(Emphasis mine.)

From PEP 518:

If the file exists but is lacking the [build-system] table then the default values[1] as specified above should be used.

  1. requires = ["setuptools"] ↩︎

3 Likes
23

I think the issue may go deeper. Not sure if it does in this case.

If I was new to the Python ecosystem, I’m pretty sure I wouldn’t understand the difference of “installing” a package and just using it where it is without installing it. Rather, I’d see “it doesn’t work, complains that it cannot find something, so probably I should try installing it”. Then maybe Google for “how to install Python package” if you don’t remember from the past that it’s pip install ..

IOW, I’m not sure the typical user who encounters this wanted to install dependencies. Probably they just saw software that didn’t run and tried what seemed logical to them: Install it.

3 Likes
24

To be frank, they are not a Python developer and it’s probably how AI did it for them when they asked for a simple FastAPI example.

Yeah, I don’t think it has permeated into the community yet (I will fully admit it didn’t come to my sleep-deprived brain as a solution either). Heck, I don’t think I thought about how that would work without specifying . until I looked at pip install --help to see about specifying the path to pyproject.toml (answer: just point pip at a file path that ends in pyproject.toml and it picks that up as what to read from instead of treating the directory as a path to a source tree).

25

There was no new discussion for a while here.

Hence a proposal:

If there is no [build-system] defined, only assume the default (setuptools) if setup.py exists.

26

Any reason why it should be only if setup.py exists and not if setup.py or setup.cfg exists?

2 Likes
27

There was no way to execute python setup.py install when there was no setup.py and the default build backend exists to allow transition from that command without all projects needing to get a pyproject.toml with build backend slecificiation. Defaulting to setuptools when there is only setup.cfg makes no sense. Projects that want to have setup.cfg without setup.py need to specify the build backend (they always needed that).

2 Likes
28

Do they?

> cd "$(mktemp -d)"
> touch pyproject.toml
> echo -e '[metadata]\nname = foo' > setup.cfg 
> python -m build --wheel --no-isolation .
Successfully built foo-0.0.0-py3-none-any.whl

I think under the hood it’s doing something akin to:

python -c 'import setuptools; setuptools.setup()' bdist_wheel
29

I suspect they haven’t needed that because of the implicit default.

Setuptools has two build backends:

  • The “normal” one that you have to use explicitly.
  • The "“legacy” one that is used implicitly by default.

At the implementation level, the legacy one wraps the normal one, with the only behavior difference being to modify sys.path prior to the ‘normal” one being used. So anything the “normal” one supports, the “legacy” one also supports.

You get… “something” if you create an empty directory with nothing but an empty pyproject.toml file:

~/projects
❯ mkdir t

~/projects
❯ cd t

~/projects/t
❯ touch pyproject.toml

~/projects/t via 🐍 v3.13.3
❯ pyproject-build .

...

Successfully built unknown-0.0.0.tar.gz and unknown-0.0.0-py3-none-any.whl

A less silly example is filling out some basic information in the pyproject.toml file

~/projects/t via 🐍 v3.13.3
❯ echo '[project]\nname="foo"\nversion="1.0"\n' > pyproject.toml

~/projects/t is 📦 v1.0 via 🐍 v3.13.3
❯ mkdir foo

~/projects/t is 📦 v1.0 via 🐍 v3.13.3
❯ touch foo/__init__.py

~/projects/t is 📦 v1.0 via 🐍 v3.13.3
❯ pyproject-build .

...

Successfully built foo-1.0.tar.gz and foo-1.0-py3-none-any.whl

~/projects/t is 📦 v1.0 via 🐍 v3.13.3 took 8s
❯ unzip -l dist/foo-1.0-py3-none-any.whl
Archive:  dist/foo-1.0-py3-none-any.whl
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  03-14-2026 16:09   foo/__init__.py
       45  03-14-2026 16:10   foo-1.0.dist-info/METADATA
       91  03-14-2026 16:10   foo-1.0.dist-info/WHEEL
        4  03-14-2026 16:10   foo-1.0.dist-info/top_level.txt
      340  03-14-2026 16:10   foo-1.0.dist-info/RECORD
---------                     -------
      480                     5 files

It seems likely to me that there are a non zero number of projects out there implicitly relying on this behavior. Maybe they shouldn’t be, but I am sure that they are. If we want to potentially break those, then someone should probably at least spend some time looking at tarballs on PyPI to figure out what the level of impact is likely to be.

Personally, I think there’s probably a few options to move this forward if someone wants to:

  • Focus on improving the pip install . case specifically, that’s a subset of the general case (and thus is bound to be smaller), but pip would be within their rights to say they don’t want to diverge pip install . and pip install some-sdist.
  • Focus on improving the error message in setuptools, but they’d be within their rights to say that they don’t want to do that (and tbh, it’s unclear to me how they’d differentiate from my foo case and Brett’s case to present a better error message here).
  • Gather information to prove that the impact of removing the fallback (either with or without conditions) is small and unlikely to cause much disruption and write a PEP justifying that.
  • Figure out how to get the ecosystem to transition to new packages being required to declare their build backend, so that hopefully, at some point, the packages that don’t declare their build backend are all old, legacy packages and hopefully are no longer important enough to need to keep working by default.

The first one of those, I think it’s not hard for pip to differentiate internally between pip install . and pip install some-sdist (in a conceptual sense, since they have the information to do that… but it may require plumbing that information through and that may be hard). That does kind of go against what PEP 517 requires though, and given it’s a breaking change, I suspect that pip wouldn’t want to do that “on it’s own”.

I suspect the 2nd one of those is a non-starter because I’m not sure how you’d tell the difference between “I’ve accidentally invoked setuptools” and “I’m relying on the implicit default of setuptools”… except maybe it could fail instead of use an unknown name? But even that feels like it’s kinda fixing one tiny corner of the “problem” and not really fixing the problem at all.

The third one of those I think would be good to do regardless, but I suspect the answer is going to be that more projects than we expect are relying on that implicit default (but I could be wrong! that’s why it’s good to collect data).

I suspect that the fourth option is the actual path forward (but again, that’s just my personal opinion!).

Some ideas for how to do that:

  • Have PyPI enforce that a build-system must be declared for new sdists (probably warning at first, then transition to enforcing).
  • Have build backends rewrite the pyproject.toml file when they produce a sdist to ensure that the [build-system] key is correctly configured.
    • This might only actually affect setuptools given it’s default nature, but it might also effect other tools if they’re primarily being used directly rather than through a generic build tool like python -m build.
    • This concept might be generally useful for other things, for instance cargo rewrites Cargo.toml files to ensure that published artifacts only use TOML 1.0 semantics while developers can use TOML 1.1 (or later hypothetically) semantics.

That doesn’t actually solve the problem Brett had, but it stops the bleeding so maybe some day we could?

Although to be honest, I’m not sure that having a default is a bad thing either. setuptools is a perfectly fine build backend for simple projects. If you’re doing something complex there may be build backends that are better suited, but for a lot of things it’s perfectly fine.

I know I’ve personally omitted the [build-system] table from a package because it was a pure Python thing, so basically every build backend will work fine, and it’s simpler to not fill it out and let the defaults take over than to go look up what the right magic string is to chuck into that field.

Of course that wouldn’t be affected by setuptools rewriting the pyproject.toml to be explicit :slight_smile: and if it did that, then it also wouldn’t be affected by PyPI requiring that key to be filled out… but neither of those actually solve Brett’s problem either, they just make it more likely we can at some point.

2 Likes
30

Well, that only works because the tools default to setuptools even without setup.cfg.

$ mkdir empty
$ cd empty
[empty]$ touch pyproject.toml
[empty]$ python -m build --wheel --no-isolation .
...
Successfully built unknown-0.0.0-py3-none-any.whl

Right. With the implicit default, they don’t need that. I meant “before pyproject build backends were a thing, they needed to have setup.py”.

31

Makes sense, but that was 10 years ago so I think not entirely relevant to today. Particularly if we’re trying to parse out whether the spec intended to only support cases where setup.py existed or not, when the actual rules in the spec are pretty clear:

If the pyproject.toml file is absent, or the build-backend key is missing, the source tree is not using this specification, and tools should revert to the legacy behaviour of running setup.py (either directly, or by implicitly invoking the setuptools.build_meta:__legacy__ backend).

At the time of PEP 517, I don’t think we conceived of the idea that it would be possible to have a shared metadata file that was complete enough for the legacy fallback to be useful on it’s own without a setup.py, since that predates all of that. So I think it’d be reasonable to argue that the call out to setup.py wasn’t being used as the sigil (since the pyproject.toml / build-backend are explicitly called out as such), and instead being used as the description of how to support those old packages.

The truth is probably closer to we just didn’t consider what would happen if setuptools was usable without anything but pyproject.toml, and so it was effectively undefined behavior.

Regardless, we’ve got 10 years of behavior to contend with, so any movement has to take that into account :slight_smile: .

1 Like
32

I think that’s out of date. With current pip and pyproject-build:

❯ pip wheel .
ERROR: Directory '.' is not installable. Neither 'setup.py' nor 'pyproject.toml' found.
❯ uv run --with build  py -m build .
* Creating isolated environment: venv+pip...
ERROR Source . does not appear to be a Python project: no pyproject.toml or setup.py

With a minimal pyproject.toml it does work, though, precisely because of the default for [build-system] which (as I understand it) is what this proposal is suggesting we should remove.

Personally, I’d be against changing pip’s behaviour here without a standard. If a user runs pip install . and the CWD contains a pyproject.toml, then it’s 100% standards compliant that pip should build a wheel from the CWD, and install it. It’s also 100% standards compliant that if there’s no [build-system] section in pyproject.toml, a default of setuptools should be assumed.

If we want to remove that default, that’s a standards change. There are two possible motivating reasons for such a change:

  1. Stop treating setuptools as special. There’s some merit to this, but I find it hard to imagine the benefits would outweigh the breakage.
  2. Allow for “non-installable” projects with a pyproject.toml. This is currently adhoc usage, completely unstandardised. uv seems to be promoting a particular model for how this might work, although I’m not particularly familiar with it so I can’t really comment on whether I’m in favour of their model. But whether we try to standardise what uv does, or we design our own model and standardise that, isn’t the point here - the point is that if we want to standardise “non-installable” projects, that’s a whole PEP in itself. And when that PEP is approved, then pip will be updated to follow it.

My impression is that Brett’s original problem arose from someone having a “non-installable” project, and having (incorrect) expectations on what pip would do based on that. But that’s fundamentally a combination of user error combined with the fact that to setuptools, basically everything is installable, but the way it guesses in the absence of explicit information is confusing to someone who never thought of their directory as installable.

I agree, I don’t see setuptools changing much here. I don’t necessarily like the fact that they go so far in trying to infer a project structure without any explicit information, but it’s part of their design, and I’d imagine they’d be just as reluctant to change that as I am to change what pip install . does.

I agree. Either way, someone needs to write a PEP. Personally, I think that PEP should focus on the “non-installable project” use case, and be about how to adapt pyproject.toml for projects that are not intended to be built as wheels. There’s clearly community interest in that use case, and uv has a bunch of relevant experience so we’d not be designing in a vacuum. But we’ve tried to have that discussion once before, and it’s proved controversial. I imagine that would still be true if we tried again. But I think it’s still a better approach than going down the “setuptools shouldn’t be treated specially” approach.

As a side note, no matter what outcome we end up with, I would strongly oppose ever having pip install . do anything other than install the project in the CWD. If the CWD is a non-installable project, we should fail with an error - we shouldn’t just silently install what’s in project.dependencies. But let’s not have that debate now, we should do it in the context of an actual PEP.

2 Likes
33

You missed the empty pyproject.toml? At least your error seems to suggest you don’t have a pyproject.toml file at all, and my example had a completely empty pyproject.toml.

IOW, “minimal” here means python3 -c ‘open(“pyproject.toml”, “w”).close()’ is enough.

But +1 to everything else :slight_smile:

34

Sorry, yes I did.

1 Like
35

I suppose at the end of the day, it’s perfectly possible to have a functioning setuptools project that only uses the generic [project] section of the pyproject.toml, doesn’t rely on any of setuptools’s (arguably overenthusiastic) core metadata defaults, declares no build backend and has no setup.py/setup.cfg/[tool.setuptools]. So no heuristic will definitively tell you that the user wants their project built with setuptools without false negatives.

Maybe the setup.py or pyproject.toml exists check pip install . does could be setup.py exists or pyproject.toml contains [tool.setuptools] or pyproject.toml contains [project] or pyproject.toml declares a build backend? AFAICT that shouldn’t break anything that currently actually builds minus the naff case of the configurationless unknown-v0.0.0 setuptools project but will catch a non-installable project where the pyproject.toml only exists for some [tool.something] options.

36

Proposal: introduce a new Boolean option project.installable in pyproject.toml. If it’s true, then it’s a no-op. If it’s false, then installers should refuse to install the project (regardless of source) or build a wheel, and package indexes should refuse to accept wheels (similar to NPM’s private option). Installers would be free to eg install dependencies when install --deps-only .. The new option can’t be dynamic.

Worth pursuing? If so, I’ll create a new thread.

1 Like
37

Surely an easy way to do this is to add an empty [build-system] section? That should cause any “normal” build to break, as the section is present so there’s no fallback, but it’s not got a valid backend so it can’t build.