# Adding a default extra\_require environment **URL:** \ **Category:** Packaging\ **Created:** [August 3, 2020, 6:51pm UTC](https://discuss.python.org/t/adding-a-default-extra-require-environment/4898 "2020-08-03T18:51:56Z")\ **Posts on this page:** 20\ **Page:** 8
**Author:** ![trobitaille](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/trobitaille/32/6922_2.png) [@trobitaille](https://discuss.python.org/u/trobitaille)\ **Post date:** [June 23, 2023, 8:56pm UTC](https://discuss.python.org/t/adding-a-default-extra-require-environment/4898/143 "2023-06-23T20:56:00Z")
For packages I am contributing to, we have an extra called `recommended` which are optional but recommended dependencies. I would like that to be the default extra, but then give users the ability to do minimal installs, which is the “no extras” case (the ability to do minimal installs is something that is important for some of our users, as some of the recommended dependencies are quite hefty and not needed for a reasonable fraction of the package). To be clear, this is different from installing with `--no-deps` since the minimal set of strictly required dependencies is not empty. I’d be happy with either `package[]` or `package[-*]` for this - my point is just that I don’t want users to have to figure out what the name of the default extras are to select them. I personally find `package[]` to be more pleasing and intuitive, as no one is going to bother writing `[]` if it has no effect anyway (and `[-*]` looks like a strange smiley). But both are better options than `package[-recommended]` in my view. ---
**Author:** ![pf\_moore](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pf_moore/32/35_2.png) [@pf\_moore](https://discuss.python.org/u/pf_moore)\ **Post date:** [June 23, 2023, 9:30pm UTC](https://discuss.python.org/t/adding-a-default-extra-require-environment/4898/144 "2023-06-23T21:30:40Z")
Given that users have to find out that in order to not get the recommended extras, they have to type _something_, how much harder is it to explain that they type `[-recommended]` instead of explaining that they write `[]`? I don’t think we want to end up with users routinely using `package[]` “just in case” they need it in order to get a minimal install. And unfortunately that sort of blind copying of inappropriate advice is all too common… ---
**Author:** ![jonathandekhtiar](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/jonathandekhtiar/32/2533_2.png) [@jonathandekhtiar](https://discuss.python.org/u/jonathandekhtiar)\ **Post date:** [June 23, 2023, 9:42pm UTC](https://discuss.python.org/t/adding-a-default-extra-require-environment/4898/145 "2023-06-23T21:42:08Z")
I quite agree with @pf_moore. If we have “default extra”, you could define an extra `"minimal: []` and do `package[minimal]`. Which eventually would deactivate any sort of “default extra”. The advantage of seeing the usecase that way, we don’t have to complexify a feature request that had already been stalling for 2 years … Doing progress with the initial feature scope would be a good thing already 😃 ---
**Author:** ![trobitaille](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/trobitaille/32/6922_2.png) [@trobitaille](https://discuss.python.org/u/trobitaille)\ **Post date:** [June 23, 2023, 10:20pm UTC](https://discuss.python.org/t/adding-a-default-extra-require-environment/4898/146 "2023-06-23T22:20:22Z")
Ok that sounds good with regards to not having `[]` mean “no extras” - I can live with `[-recommended]` and can see how it is probably better to be explicit. From reading the the thread, I wasn’t sure if there was consensus on whether specifying extras explicitly disables the default extras or simply adds to them - the `[minimal]` example would only work in the former case, but I thought there were objections to this? ---
**Author:** ![jonathandekhtiar](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/jonathandekhtiar/32/2533_2.png) [@jonathandekhtiar](https://discuss.python.org/u/jonathandekhtiar)\ **Post date:** [June 24, 2023, 12:24am UTC](https://discuss.python.org/t/adding-a-default-extra-require-environment/4898/147 "2023-06-24T00:24:49Z")
I think that’s the point of “default” anything. You can get the default if you don’t ask for anything or the “not default” if you ask for something. See it as “how do you want your meal ?” - `default=mild_spicy ` but `meal[no_spicy]` and `meal[super_hot]` are available. Quite obviously asking `super_hot` removes the default 😉 It’s the best example I could think of 😃 ---
**Author:** ![effigies](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/effigies/32/35835_2.png) [@effigies](https://discuss.python.org/u/effigies)\ **Post date:** [June 24, 2023, 2:46am UTC](https://discuss.python.org/t/adding-a-default-extra-require-environment/4898/148 "2023-06-24T02:46:56Z")
This expectation really depends on how you’re using extras. Your example supposes they are alternatives, but I often see them used as feature sets. For example, if I see a package with default extra `common`, I would expect `package[extra]` to be equivalent to `package[common,extra]`. ---
**Author:** ![jonathandekhtiar](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/jonathandekhtiar/32/2533_2.png) [@jonathandekhtiar](https://discuss.python.org/u/jonathandekhtiar)\ **Post date:** [June 24, 2023, 4:14am UTC](https://discuss.python.org/t/adding-a-default-extra-require-environment/4898/149 "2023-06-24T04:14:04Z")
Right I was totally expecting this reply 😅 `pizza[tomato_sauce,mozzarella]`. Short coming of my example … But in general a “default” is what you get if you don’t specify anything. Think about a default argument value: ```auto def foo(bar=None): if bar is None: bar = [dep_a, dep_b] print(bar) ``` You can: - not define `bar` - define it and override the default value I gotta say I’m disappointed someone dismantled my awesome & yummy example so fast 😃 ---
**Author:** ![pradyunsg](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pradyunsg/32/206_2.png) [@pradyunsg](https://discuss.python.org/u/pradyunsg)\ **Post date:** [June 24, 2023, 5:05pm UTC](https://discuss.python.org/t/adding-a-default-extra-require-environment/4898/150 "2023-06-24T17:05:33Z")
We’re repeating things that have been discussed in this thread already. > [@Adding a default extra\_require environment](https://discuss.python.org/t/adding-a-default-extra-require-environment/4898/94): > > given pip install D implies the cpu extra should be included, and pip install D[gpu] implies the cpu extra should not be included but gpu should, how do you combine pip install D D[gpu] into a single requirement? Given D == D[cpu], I’d say D D[gpu] == D[cpu] D[gpu] == D[cpu,gpu]. I suggest we hold off on further discussion until someone writes down a design document proposing a specific design and discussing why they didn’t take the alternatives AKA PEP. 🙂 I believe @RonnyPfannschmidt had also spent some time looking at authoring a PEP about this but hasn’t had the time to finish that (please correct me I’m wrong). If someone else wants to pick this up and you haven’t written a PEP before, please feel welcome to reach out to me about sponsoring the PEP. ---
**Author:** ![RonnyPfannschmidt](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/ronnypfannschmidt/32/1714_2.png) [@RonnyPfannschmidt](https://discuss.python.org/u/RonnyPfannschmidt)\ **Post date:** [September 11, 2023, 11:11am UTC](https://discuss.python.org/t/adding-a-default-extra-require-environment/4898/151 "2023-09-11T11:11:59Z")
i have been working on a pep on this, the key problem is how to deal with default vs negatives in the install metadata there basically i need for a canonical and non-canonical extras list for the resolution as well as a - character to note removals i may get back on track with the pep by the end of the month ---
**Author:** ![trobitaille](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/trobitaille/32/6922_2.png) [@trobitaille](https://discuss.python.org/u/trobitaille)\ **Post date:** [September 12, 2024, 9:15am UTC](https://discuss.python.org/t/adding-a-default-extra-require-environment/4898/152 "2024-09-12T09:15:03Z")
@RonnyPfannschmidt - just out of curiosity, did you get a chance to take another stab at the PEP? I am interested in trying to help with this, so let me know if there is anything I can do. If you don’t have time to work on this I’d be happy to try and take the lead on the PEP - in that case feel free to share what you had already, or let me know if I should just start with a blank slate. Thanks! ---
**Author:** ![trobitaille](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/trobitaille/32/6922_2.png) [@trobitaille](https://discuss.python.org/u/trobitaille)\ **Post date:** [September 24, 2024, 11:17am UTC](https://discuss.python.org/t/adding-a-default-extra-require-environment/4898/153 "2024-09-24T11:17:28Z")
@pradyunsg I’d like to try and pick this up and push it forward, and am working on a PEP draft. I’ll reach out to you privately once I have a draft to see if you would be happy to sponsor it (I haven’t written a PEP before). ---
**Author:** ![trobitaille](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/trobitaille/32/6922_2.png) [@trobitaille](https://discuss.python.org/u/trobitaille)\ **Post date:** [September 25, 2024, 10:26am UTC](https://discuss.python.org/t/adding-a-default-extra-require-environment/4898/154 "2024-09-25T10:26:04Z")
With the idea that it’s better to have something to discuss and iterate on, I’ve had a chance to try and write up an initial draft, which I’ve put in my fork here, and have opened up a PR to my own fork to make it easy to review/discuss directly: > > > This is for initial collaboration on the PEP - leave comments/suggestions or ope…n PRs to my branch to suggest changes/additions! > > Preview: https://github.com/astrofrog/peps/blob/default-extras/peps/pep-tbd.rst > > > \---- > 📚 Documentation preview 📚: https://pep-previews--1.org.readthedocs.build/ It’s very much an incomplete draft at this point but I thought it would be a good starting point. I’ve included a link to preview the PEP as-is in that PR. If anyone else would like to join as an author, please just leave a comment in the PR and I can add you! ---
**Author:** ![trobitaille](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/trobitaille/32/6922_2.png) [@trobitaille](https://discuss.python.org/u/trobitaille)\ **Post date:** [October 16, 2024, 10:42am UTC](https://discuss.python.org/t/adding-a-default-extra-require-environment/4898/155 "2024-10-16T10:42:05Z")
Following more discussion over at my pull request, the conclusion for now is that introducing the new -extra syntax is going to be a proposal that is very hard to sell as there is no way to introduce new syntax into PEP 508 specifiers without essentially breaking support for older versions of packaging tools such as pip. Therefore, for now we are instead focusing on the approach of having default extras only being installed by default if no extras are explicitly specified - this was not everyone’s preferred approach (including myself) but it might end up being the best compromise and actually allows for more flexibility and customization than I though might be possible - several scenarios are now mentioned in the PEP to highlight this. > > > This is a PR internal to this fork for initial collaboration on the PEP - leave …comments/suggestions or open PRs to my branch to suggest changes/additions! > > Motivating discussion: https://discuss.python.org/t/adding-a-default-extra-require-environment/4898/152 If you are a package maintainer and interested in potentially making use of this feature, could you take a look at the PEP draft at the above link and confirm if you think this will work for you, or if it still does not address your use case? (even just a 👍 would be useful) I am going to ping @dustin @uranusjr @ofek @steve.dower @matham @sinoroc @dstufft @effigies as people actively involved in the discussion here - some of you have use cases for this, and some of you had opinions, so please take a look if you can! ---
**Author:** ![oscarbenjamin](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/oscarbenjamin/32/1209_2.png) [@oscarbenjamin](https://discuss.python.org/u/oscarbenjamin)\ **Post date:** [October 16, 2024, 11:51am UTC](https://discuss.python.org/t/adding-a-default-extra-require-environment/4898/156 "2024-10-16T11:51:42Z")
> [@trobitaille](#): > > If you are a package maintainer and interested in potentially making use of this feature, could you take a look at the PEP draft at the above link and confirm if you think this will work for you, or if it still does not address your use case? I think that I would like to use this for some packages. My use case is pure Python packages that have optional non-Python accelerator modules. One example is that SymPy can now make use of python-flint as an optional dependency to speed up some operations. Eventually I would like users who do `pip install sympy` to get python-flint by default but there will always be a need to opt out because python-flint includes libraries with a different license and also is much more difficult to build. We might also want to have other optional dependencies like numpy, scipy and matplotlib be part of the default extras. Ideally I would want to have the default requirement conditional on the presence of a wheel for the accelerator module i.e. “install python-flint if there is a wheel but otherwise don’t try to build it”. I guess I could make that effectively happen with environment markers though at least for someone installing from PyPI. Currently the way to achieve this without default-extras is to split a package up so then you have an empty `sympy` package and a `sympy-minimal` package that contains the actual code. The `sympy` package’s requirements would include the default extra requirements but the `sympy-minimal` package would not and then downstream packages can depend on `sympy-minimal` if they want. This is awkward to do mainly just because it is awkward to make multiple packages from a single git repo and coordinate releasing them. An alternative that could achieve what I want then would be if there was better tooling for generating multiple packages from a single git repo although it would still be nicer to use default-extras rather than making another package. (There are other reasons to want multiple packages though like to split out the test suite into a `sympy-test` package.) The separate packages approach is possibly easier to adapt for downstream packagers like distros etc. The PEP seems to describe things in terms of setup.cfg but should these things not be in pyproject.toml? ---
**Author:** ![effigies](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/effigies/32/35835_2.png) [@effigies](https://discuss.python.org/u/effigies)\ **Post date:** [October 16, 2024, 11:56am UTC](https://discuss.python.org/t/adding-a-default-extra-require-environment/4898/157 "2024-10-16T11:56:36Z")
This looks entirely reasonable to me, and I would definitely use this functionality and propose adopting it to upstream dependencies that I would like to include minimal variants of. Thank you for putting the time into this proposal. I had stopped following the discussion after concluding that consensus was unlikely to be reached, so I hope this (or something like it) is broadly satisfactory. ---
**Author:** ![trobitaille](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/trobitaille/32/6922_2.png) [@trobitaille](https://discuss.python.org/u/trobitaille)\ **Post date:** [October 16, 2024, 1:03pm UTC](https://discuss.python.org/t/adding-a-default-extra-require-environment/4898/158 "2024-10-16T13:03:19Z")
@oscarbenjamin @effigies - thanks for the feedback! > [@oscarbenjamin](#): > > The PEP seems to describe things in terms of setup.cfg but should these things not be in pyproject.toml? I’ve now updated the PEP to use pyproject.toml in examples. ---
**Author:** ![bwoodsend](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/bwoodsend/32/4027_2.png) [@bwoodsend](https://discuss.python.org/u/bwoodsend)\ **Post date:** [October 16, 2024, 9:39pm UTC](https://discuss.python.org/t/adding-a-default-extra-require-environment/4898/159 "2024-10-16T21:39:57Z")
How do you expect downstream (non-pip) package managers that have no concept of _recommended_ dependencies to handle the ambiguous dependencies? Should `apk add py3-some-package` unconfigurably install the recommended packages or use the minimal base set? Also, am I really the only person who doesn’t look forward to the idea of, every time I want to install a package throughout the rest of my Python career, having to find the _advanced installation_ documentation section or checking the source code’s pyproject.toml to figure out if/what synonym of `minimal` I need to put into the square brackets in order to not pollute my environment with dead weight? Even if it gets standardized, I’d still need to remember to append `[minimal]` to every package name in every `pip install`. ---
**Author:** ![trobitaille](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/trobitaille/32/6922_2.png) [@trobitaille](https://discuss.python.org/u/trobitaille)\ **Post date:** [October 16, 2024, 10:05pm UTC](https://discuss.python.org/t/adding-a-default-extra-require-environment/4898/160 "2024-10-16T22:05:44Z")
> [@bwoodsend](#): > > How do you expect downstream (non-pip) package managers that have no concept of _recommended_ dependencies to handle the ambiguous dependencies? Should `apk add py3-some-package` unconfigurably install the recommended packages or use the minimal base set? That’s a good question - and interestingly I’ve seen that come up a lot in the context of conda packages, which don’t have the ability to use extras/optional dependency sets. In fact, the situation there is very inconsistent, some conda recipes include all optional dependencies, while some use more minimal sets. As a result, some packages now have multiple recipes, e.g. matplotlib-base and matplotlib. So this is to say that there is already a problem anyway even with extras as they are. Packaging systems such as apt do actually include the concept of recommended but not required dependencies so in that case the concept here would map well onto it. > [@bwoodsend](#): > > Even if it gets standardized, I’d still need to remember to append `[minimal]` to every package name in every `pip install` . What you might want in your case is for pip to support a command-line flag such as `--no-default-extras` that automatically disables/ignores default extras - the PEP draft above mentions this and doesn’t specifically recommend it, leaving it up to the individual tools to decide whether to support it. I personally would find such a flag useful. ---
**Author:** ![oscarbenjamin](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/oscarbenjamin/32/1209_2.png) [@oscarbenjamin](https://discuss.python.org/u/oscarbenjamin)\ **Post date:** [October 16, 2024, 10:49pm UTC](https://discuss.python.org/t/adding-a-default-extra-require-environment/4898/161 "2024-10-16T22:49:09Z")
> [@bwoodsend](#): > > Also, am I really the only person who doesn’t look forward to the idea of, every time I want to install a package throughout the rest of my Python career, having to find the _advanced installation_ documentation section or checking the source code’s pyproject.toml to figure out if/what synonym of `minimal` I need to put into the square brackets in order to not pollute my environment with dead weight? The flipside of this is needing to read the installation instructions to figure out which version of `package[featureA,featureB]` actually provides the features that you expected the package was supposed to provide or not realising that `package[accelerator]` makes your code run 1000x faster. I do agree with the concern though. I am generally amazed by how much space a venv takes up these days and how many dependencies get pulled in for what seem like simple packages. That happens already now when there are no default extras and we don’t have any way of opting out. Maybe with default extras it would be more common that packages would provide a better minimal install since it can be something distinct from the default install. ---
**Author:** ![bwoodsend](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/bwoodsend/32/4027_2.png) [@bwoodsend](https://discuss.python.org/u/bwoodsend)\ **Post date:** [October 17, 2024, 10:41am UTC](https://discuss.python.org/t/adding-a-default-extra-require-environment/4898/162 "2024-10-17T10:41:27Z")
> [@trobitaille](#): > > What you might want in your case is for pip to support a command-line flag such as `--no-default-extras` that automatically disables/ignores default extras Well I do, and I like the idea of something that’s `pip.conf`-able but… I think it’ll make it even worse. If package `foo` thinks it depends on package `bar` but doesn’t realise that it needs `bar[thedefaultgroup]` then `pip install foo` would be broken with the flag. Similarly, if `bar` had no recommended dependencies then decided to move a mandatory to a recommended one, thinking this would not be an incompatible change, then anyone who uses the flag again receives a potentially broken package. To mitigate this, I think we’d end up having to standardise the _recommended_ name then get into the habbit of always writing `foo[recommended]` when specifiying dependencies. It’s puts quite a burden on package developers to always test a `--no-default-extras` built setup. But if we’re going into `pip` details then does anyone have a plan for how ```python pip install package[minimal] pip freeze > requirements.txt # In a new environment pip install -r requirements.txt ``` is going to avoid pulling in `package`’s recommended dependencies? > [@oscarbenjamin](#): > > The flipside of this is needing to read the installation instructions to figure out which version of `package[featureA,featureB]` actually provides the features that you expected the package was supposed to provide or not realising that `package[accelerator]` makes your code run 1000x faster. Yeah, you can argue it either way. Either a getting started page can say run `pip install package[accelerator]` then explain what benefits you’d get with the `accelerator` bit or it can say run `pip install package` or `pip install package[minimal]` and again explain the difference. (I’d prefer the explicitness of the first but I know that’s just a preferences thing.) A lot of packages that have optional cython/rust extensions tend to treat the opt out users as second class citizens (in both documentation and testing). I suspect that `pip install package[minimal]` users will end up the same. [Previous page](https://discuss.python.org/t/adding-a-default-extra-require-environment/4898.md?page=7) [Next page](https://discuss.python.org/t/adding-a-default-extra-require-environment/4898.md?page=9)