# Pip without setuptools, could the experience be improved?

**URL:** <https://discuss.python.org/t/pip-without-setuptools-could-the-experience-be-improved/11810>\
**Category:** Packaging\
**Created:** [November 8, 2021, 2:26pm UTC](https://discuss.python.org/t/pip-without-setuptools-could-the-experience-be-improved/11810 "2021-11-08T14:26:34Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![hroncok](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/hroncok/32/18696_2.png) [@hroncok](https://discuss.python.org/u/hroncok)\
**Post date:** [November 8, 2021, 2:26pm UTC](https://discuss.python.org/t/pip-without-setuptools-could-the-experience-be-improved/11810/1 "2021-11-08T14:26:34Z")

</div>

Hello.

In Fedora, our pip package _Recommends_ setuptools. Practically that means:

1. Majority of users who install pip will get setuptools by default.
2. Users can explicitly uninstall setuptools after installing pip or exclude setuptools when installing pip.
3. Users might op-out from installing _Recommended_ packages by default to save space (I suppose the majority of such users do it in containers or a similar environment), and such users won’t get setuptools with pip.

I like that our users have a choice not to install setuptools with pip. I also like that by default, they’ll get it. What worries me is the experience for users from (3) who might get pip without setuptools while not explicitly asking for this setup. When pip attempts to install a package from sdist without a PEP 517/518 build backend specified, it’ll fail with:

```auto
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
  ModuleNotFoundError: No module named 'setuptools'

```

This traceback is displayed multiple times, as backtracking causes lots of failed attempts.

Originally, I thought: Right, it’s because the `setup.py` script imports it, but I’ve realized it is actually pip that does that:

> <https://github.com/pypa/pip/blob/21.3.1/src/pip/_internal/utils/setuptools_build.py#L11>

Even packages that don’t use setuptools in their `setup.py` script will fail with the same problem. I’ve successfully tested the assumption with this example:

```python
from distutils.core import setup
setup()

```

After the recent discussions on the Python-Dev mailing list – [Mailman 3 Does ensurepip still have to include a copy of setuptools? - Python-Dev - python.org](https://mail.python.org/archives/list/python-dev@python.org/thread/3BVAUIQOEOXAULHVYQNLLQIZQQETX2EV/), and after the recent discussion with @jaraco in [Declare dependencies in metadata by jaraco · Pull Request #2764 · pypa/setuptools · GitHub](https://github.com/pypa/setuptools/pull/2764), I have mixed expectations whether pip should always _Require_ (rather than _Recommend_) setuptools. In the RPM world, that means:

1. All users who install pip will get setuptools by default.
2. Users cannot explicitly uninstall setuptools after installing pip nor exclude setuptools when installing pip.
3. Users might op-out from installing _Recommended_ packages by default to save space, and such users will still get setuptools with pip.

This feels like a quite inflexible setup. I would rather allow not installing setuptools with pip when users know what they are doing, but users from (3) might not understand this problem, as it might not be from their domain at all. [The Traceback might be interpreted as a problem in pip](https://bugzilla.redhat.com/show_bug.cgi?id=2020635).

Would it make sense to handle missing setuptools from pip’s code in a more explicit way? Ideas:

1. Instead of displaying a Traceback, display an error message explaining that setuptools is not installed and installing \<package\> requires it. Suggest installing setuptools before installing \<package\> or using `--use-pep517` to install \<package\>.
2. Default to `--use-pep517` when setuptools is not installed (see also [Default to --use-pep517 · Issue #9175 · pypa/pip · GitHub](https://github.com/pypa/pip/issues/9175)).

Pip already handles the presence or absence of wheel differently, so there is a precedent.

What do you think?

CC’ing also @pradyunsg and @encukou who might be interested in this topic.

---

<div class="post-metadata">

**Author:** ![pf\_moore](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pf_moore/32/35_2.png) [@pf\_moore](https://discuss.python.org/u/pf_moore)\
**Post date:** [November 8, 2021, 5:02pm UTC](https://discuss.python.org/t/pip-without-setuptools-could-the-experience-be-improved/11810/2 "2021-11-08T17:02:55Z")

</div>

The direct use of setuptools should only be on the “legacy path” for pip. Unfortunately, there’s still a lot of legacy code out there 🙁

The plan is that we switch to PEP 517 behaviour by default, with build isolation and an implied use of setuptools if there’s no `pyproject.toml`. I’m not 100% sure on precisely where we are with that migration, but it’s definitely in progress. (Editable installs is the biggest problem here, as we can’t drop the legacy route until the setuptools PEP 517 backend adds editable install support).

But otherwise, I see no reason (other than “this is going away real soon, so why bother?”) why we wouldn’t be happy to improve the error reporting for the case when setuptools isn’t available by default.

---

<div class="post-metadata">

**Author:** ![jaraco](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/jaraco/32/57_2.png) [@jaraco](https://discuss.python.org/u/jaraco)\
**Post date:** [November 8, 2021, 6:57pm UTC](https://discuss.python.org/t/pip-without-setuptools-could-the-experience-be-improved/11810/3 "2021-11-08T18:57:13Z")

</div>

See [pypa/setuptools#2816](https://github.com/pypa/setuptools/issues/2816) to track Setuptools’ support.

In my workflows, I’ve left Setuptools uninstalled and I rely on `--use-pep517` to force pip to install any package without Setuptools being present. Additionally, I just verified that editable installs work in the same environment, at least with the latest pip:

```auto
draft $ python -m venv env
draft $ env/bin/pip uninstall -y setuptools
Found existing installation: setuptools 57.4.0
Uninstalling setuptools-57.4.0:
  Successfully uninstalled setuptools-57.4.0
draft $ cat > setup.py
__import__ ('setuptools').setup()
draft $ env/bin/pip install --use-pep517 -e .
Obtaining file:///Users/jaraco/draft
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Installing collected packages: UNKNOWN
  Running setup.py develop for UNKNOWN
Successfully installed UNKNOWN-0.0.0

```

That is, the lack of a `pyproject.toml` implies “use setuptools” and pip makes it available, meaning that once pip makes `--use-pep517` the default, Fedora can move from “recommends” to entirely optional, even without PEP 660 support and still supporting legacy code.

Am I mistaken?

---

<div class="post-metadata">

**Author:** ![pf\_moore](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pf_moore/32/35_2.png) [@pf\_moore](https://discuss.python.org/u/pf_moore)\
**Post date:** [November 8, 2021, 7:04pm UTC](https://discuss.python.org/t/pip-without-setuptools-could-the-experience-be-improved/11810/4 "2021-11-08T19:04:59Z")

</div>

> [@jaraco](#):
>
> Additionally, I just verified that editable installs work in the same environment, at least with the latest pip

Oh, that’s cool! Sorry, I should have checked before posting. As I say, I’ve not been keeping up with the state of this work.

---

<div class="post-metadata">

**Author:** ![pradyunsg](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pradyunsg/32/206_2.png) [@pradyunsg](https://discuss.python.org/u/pradyunsg)\
**Post date:** [November 8, 2021, 8:53pm UTC](https://discuss.python.org/t/pip-without-setuptools-could-the-experience-be-improved/11810/5 "2021-11-08T20:53:27Z")

</div>

Improving these error messages is something was working on last Friday.

🙂

As for --use-pep517 by default, I think we’d need to do quite a few band aid removals, before we can get to that. We need to rip out at least the band aids of the legacy versions and legacy setup.py install, for a switch over to PEP 517 by default. It would likely make sense to do so throughout the codebase, rather than just in the PEP 517 code path.

---

<div class="post-metadata">

**Author:** ![hroncok](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/hroncok/32/18696_2.png) [@hroncok](https://discuss.python.org/u/hroncok)\
**Post date:** [November 11, 2021, 12:57pm UTC](https://discuss.python.org/t/pip-without-setuptools-could-the-experience-be-improved/11810/6 "2021-11-11T12:57:30Z")

</div>

> [@pradyunsg](#):
>
> Improving these error messages is something was working on last Friday.

Is there a PR or a draft I can play with? Or is there a way to contribute to help this effort?

---

<div class="post-metadata">

**Author:** ![pitrou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pitrou/32/28_2.png) [@pitrou](https://discuss.python.org/u/pitrou)\
**Post date:** [November 12, 2021, 1:39pm UTC](https://discuss.python.org/t/pip-without-setuptools-could-the-experience-be-improved/11810/7 "2021-11-12T13:39:03Z")

</div>

> [@hroncok](#):
>
> I like that our users have a choice not to install setuptools with pip.

So here is the question: what do you like about it? Is it just aesthetical concerns? Would your users lose anything if pip had setuptools as a hard dependency?

---

<div class="post-metadata">

**Author:** ![pradyunsg](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pradyunsg/32/206_2.png) [@pradyunsg](https://discuss.python.org/u/pradyunsg)\
**Post date:** [November 12, 2021, 5:22pm UTC](https://discuss.python.org/t/pip-without-setuptools-could-the-experience-be-improved/11810/8 "2021-11-12T17:22:04Z")

</div>

I’ll put up a draft PR over the weekend for this. It won’t change the ImportError, but I guess we can improve that as well separately.

---

<div class="post-metadata">

**Author:** ![ncoghlan](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/ncoghlan/32/14266_2.png) [@ncoghlan](https://discuss.python.org/u/ncoghlan)\
**Post date:** [November 17, 2021, 2:52pm UTC](https://discuss.python.org/t/pip-without-setuptools-could-the-experience-be-improved/11810/9 "2021-11-17T14:52:37Z")

</div>

> [@pitrou](#):
>
> So here is the question: what do you like about it? Is it just aesthetical concerns? Would your users lose anything if pip had setuptools as a hard dependency?

It’s a size and attack surface reduction consideration (mostly the former).

In particular, embedded and container use cases often only need wheel installs, where “pip without setuptools” is the most viable current option for both wheel installation and easy environment introspection.

---

<div class="post-metadata">

**Author:** ![hroncok](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/hroncok/32/18696_2.png) [@hroncok](https://discuss.python.org/u/hroncok)\
**Post date:** [November 17, 2021, 3:37pm UTC](https://discuss.python.org/t/pip-without-setuptools-could-the-experience-be-improved/11810/10 "2021-11-17T15:37:47Z")

</div>

What @ncoghlan says + the ability to uninstall setuptools which gives the opportunity for package maintainers to discover packaging bugs (something needs setuptools, but doesn’t declare the dependency).

---

<div class="post-metadata">

**Author:** ![hrnciar](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/hrnciar/32/5648_2.png) [@hrnciar](https://discuss.python.org/u/hrnciar)\
**Post date:** [December 8, 2021, 1:50pm UTC](https://discuss.python.org/t/pip-without-setuptools-could-the-experience-be-improved/11810/11 "2021-12-08T13:50:00Z")

</div>

Hello,

I discussed this issue with @hroncok and I’ll try to send a draft PR where pip will default to `--use-pep517` behaviour in case of setup.py present and missing setuptools. Depending on how it will be received, we can either do it this way or raise `InstallationError` as it is done in other cases.

---

<div class="post-metadata">

**Author:** ![uranusjr](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/uranusjr/32/103_2.png) [@uranusjr](https://discuss.python.org/u/uranusjr)\
**Post date:** [December 8, 2021, 4:53pm UTC](https://discuss.python.org/t/pip-without-setuptools-could-the-experience-be-improved/11810/12 "2021-12-08T16:53:29Z")

</div>

Why the “setup.py present” part? If setup.py is not found, PEP 517 must be enabled anyway for any kind of build to work. So I’d say PEP 517 should be enabled unconditionally if setuptools is not found in the current environment.

---

<div class="post-metadata">

**Author:** ![swills1](https://avatars.discourse-cdn.com/v4/letter/s/e8c25b/32.png) [@swills1](https://discuss.python.org/u/swills1)\
**Post date:** [February 21, 2022, 8:35am UTC](https://discuss.python.org/t/pip-without-setuptools-could-the-experience-be-improved/11810/13 "2022-02-21T08:35:02Z")

</div>

I understand this thread has become a little _stale_, but it seems to discuss exactly what I am looking for and the last reply was only under 2 months ago.

I have a couple of questions, and I hope they don’t skew the scope outlined here.

1. What is the current status of all of this, and is there a good place to track all of it?

> [@jaraco](#):
>
> That is, the lack of a `pyproject.toml` implies “use setuptools” and pip makes it available, meaning that once pip makes `--use-pep517` the default, Fedora can move from “recommends” to entirely optional, even without PEP 660 support and still supporting legacy code.
> 
> Am I mistaken?

1. How exactly is setuptools “being made available”? I followed the test steps outlined by @jaraco and it does in fact work, but I don’t necessarily understand how.

2. What exactly is happening during this step, `Preparing metadata (pyproject.toml) ... done` - especially when pyproject.toml doesn’t exist?

3. Aren’t we currently at a point where we have to rely on a combination of _setup.py_ and _pyproject.toml_? Because aren’t there currently things pyproject.toml cannot do with pip / setuptools such as entry points? Meaning, we would need a setup.py to do the following?

```auto
entry_points={'console_scripts': ['package=package.command:cli']}

```

[https://www.python.org/dev/peps/pep-0621/#entry-points](https://www.python.org/dev/peps/pep-0621/#entry-points)

Thank you.

---

<div class="post-metadata">

**Author:** ![CAM-Gerlach](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/cam-gerlach/32/3688_2.png) [@CAM-Gerlach](https://discuss.python.org/u/CAM-Gerlach)\
**Post date:** [February 21, 2022, 5:51pm UTC](https://discuss.python.org/t/pip-without-setuptools-could-the-experience-be-improved/11810/14 "2022-02-21T17:51:40Z")

</div>

> [@swills1](#):
>
> 1. What is the current status of all of this, and is there a good place to track all of it?

See, e.g., the link in the OP:

> <https://github.com/pypa/pip/issues/9175>
>
> Update on Oct 7 2022: Once \`setup.py install\` and \`setup.py develop\` code paths …are killed, there is no reason for pip to use the legacy setup.py-based builds by default, other than to avoid the additional overhead of the creation of build environments.
> 
> \---
> 
> Currently if you try to \`\`pip install\`\` something that only is provided a "legacy" setuptools only sdist without a \`\`pyproject.toml\`\`, that will be installed using the "legacy" path unless you explicitly invoke \`\`--use-pep517\`\`.
> 
> This is particularly troublesome in cases where you don't have (or want to have) setuptools installed in the environment, since the legacy path depends on having setuptools preinstalled.
> 
> This flag does not affect editable installs in any way, and those still require having setuptools preinstalled in either case.
> 
> This seems like it would be a good stepping stone on the way to https://github.com/pypa/pip/issues/6334, since it narrows the cases where we're using the legacy path to just editable installs.

> [@swills1](#):
>
> 1. How exactly is setuptools “being made available”? I followed the test steps outlined by @jaraco and it does in fact work, but I don’t necessarily understand how.

The initial version of `setuptools` is the one copied from the base Python `site-packages` by `venv` when it creates the environment, which the steps above uninstall. `setuptools` is then downloaded and installed in the isolated PEP 517 build environment by `pip`, because it is listed in `pyproject.toml`, or assumed by default if one is not present, to be a build-time dependency of the package you are installing, just like any other build backend that the package you’re trying to install might use instead–flit, poetry, etc. Then, this isolated set of build-time dependencies is used to build and install the package you’re requesting in your own virtual environment.

> [@swills1](#):
>
> 1. What exactly is happening during this step, `Preparing metadata (pyproject.toml) ... done` - especially when pyproject.toml doesn’t exist?

The `prepare-metadata-for-build-wheel` PEP 517 hook runs:

> **[PEP 517 – A build-system independent format for source trees | peps.python.org](https://peps.python.org/pep-0517/)**
>
> Python Enhancement Proposals (PEPs)

> [@swills1](#):
>
> 1. Aren’t we currently at a point where we have to rely on a combination of _setup.py_ and _pyproject.toml_ ? Because aren’t there currently things pyproject.toml cannot do with pip / setuptools such as entry points? Meaning, we would need a setup.py to do the following?

Per PEP 621, which you linked above, the above can be represented as

```ini
[project.scripts]
package = "package.command:cli"

```

in the `pyproject.toml`, as can the various other options, either metadata under the `[project]` table or tool-specific config under the `[tool]` table.

Most other build backends (flit, pdm, hatch, etc) support this already, and support for all this was just added to Setuptools and should be released soon:

> [@Help testing experimental features in setuptools](https://discuss.python.org/t/help-testing-experimental-features-in-setuptools/13821/):
>
> Hello all. For the past months we have been experimenting with adding some features to setuptools, specially the support for project metadata in pyproject.toml (initially introduced in PEP 621). Other features complement it: automatic layout discovery (for flat, src or single-module layouts), defaulting to include\_package\_data=True, etc… The idea is to make the [tool.setuptools] table in pyproject.toml not necessary that for the simple use cases. If anyone is interested in helping testing the…

However, a `setup.py` has not been required for some time, unless you want to retain support for legacy pre-PEP 517 builds. Instead, you can, and should, [specify your config declaratively in setup.cfg](https://setuptools.pypa.io/en/latest/userguide/declarative_config.html), and only retain a stub setup.py if you need compatibility for legacy tooling. See, also:

> **[Paul Ganssle - Why you shouldn't invoke setup.py directly](https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html)**
>
> The setup.py interface is an old convention from distutils, one that has bugs that cannot be fixed to bring it into line with modern Python packaging approaches. As a result, all direct...

---

<div class="post-metadata">

**Author:** ![swills1](https://avatars.discourse-cdn.com/v4/letter/s/e8c25b/32.png) [@swills1](https://discuss.python.org/u/swills1)\
**Post date:** [February 21, 2022, 6:30pm UTC](https://discuss.python.org/t/pip-without-setuptools-could-the-experience-be-improved/11810/15 "2022-02-21T18:30:11Z")

</div>

Thanks.

I am going to read through that experimental features thread and see if I can test anything and I will make it a point to test the entry points as well. Thanks.

---

<div class="post-metadata">

**Author:** ![pradyunsg](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pradyunsg/32/206_2.png) [@pradyunsg](https://discuss.python.org/u/pradyunsg)\
**Post date:** [February 21, 2022, 8:11pm UTC](https://discuss.python.org/t/pip-without-setuptools-could-the-experience-be-improved/11810/16 "2022-02-21T20:11:23Z")

</div>

The error message when using pip without setuptools is significantly improved on pip 22.0 as well.

(before)

 ![Screenshot 2022-02-21 at 20.10.07](https://us1.discourse-cdn.com/flex002/uploads/python1/original/2X/2/26c792e6f0b7af06efd5ef13212da2b9be6fd7da.jpeg)

(now)

 ![Screenshot 2022-02-21 at 20.10.26](https://us1.discourse-cdn.com/flex002/uploads/python1/original/2X/7/718e8c25ad1d65e55c3dd2965f49f80c3b93e59e.jpeg)

---

<div class="post-metadata">

**Author:** ![swills1](https://avatars.discourse-cdn.com/v4/letter/s/e8c25b/32.png) [@swills1](https://discuss.python.org/u/swills1)\
**Post date:** [February 23, 2022, 3:44am UTC](https://discuss.python.org/t/pip-without-setuptools-could-the-experience-be-improved/11810/17 "2022-02-23T03:44:01Z")

</div>

I’ve re read this thread maybe twelve times now. I misunderstood the scope when I initially read through everything. In regard in how to handle pip with rpm, I don’t think it matters _alot_ in the end.

Eventually `--use-pep517 ` will be the default. If there is not a `pyproject.toml` then it will use setuptools in the isolated environment. If the build-backend specifies setuptools, then setuptools will be grabbed and used. So I don’t think the pip rpm package needs to necessarily come with setuptools. Because I don’t feel like it will matter down the road.

Am I over simplifying the issue? I understand we have legacy paths right now and we need to support them, but once `--use-pep517` becomes the default, and PEP 517 implements editable support I don’t think it will matter.

I do agree with @pf_moore , just because this might eventually be the case doesn’t mean things can’t be made better in the short term. Which I would assume would be as simple as showing a package requires setuptools insted of the tracebck as suggested by @hroncok.

But really, I don’t have any right / place to suggest anything. I am not a maintainer. I am just trying to wrap my head around all of the changes regarding PEP 517 and just here for insight. I have been a Fedora user for maybe 5 years now. Coming with setuptools by default isn’t a huge issue. With the introduction to the isolated path however, I can see myself removing it in the future.

Thanks for all the work and information in this thread and the docs. This page really helped me a lot.

[https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/](https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/)