Hi everyone,
Along with @jonathandekhtiar we have written PEP 771: Default Extras for Python Software Packages which is now published as a draft!
Short summary
For anyone who hasn’t seen a previous version of this PEP, in short the aim of this PEP is to propose a way for Python packages to define one or more extras that get installed by default if no extras are explicitly specified. The two main cases this aims to solve are:
- packages that have recommended but not required dependencies that should ideally be installed for most users while allowing advanced users to do minimal installs
- packages that need at least one frontend/backend to be installed in order to work and for which it would be ideal for a ‘default’ to be installed for users if no other is requested explicitly
Fundamentally the goal here is to improve end user experience by avoiding having to specify extras for the most common use cases. The approach described in the PEP would allow for example cases like:
pip install package # installs recommended dependencies
pip install package[minimal] # installs only required dependencies
pip install package[additional] # installs recommended and additional dependencies
I encourage you to take a look at the PEP for full details about the different motivating use cases and the different ways that the approach in the PEP can be used to solve these use cases, and for links to a full working implementation of pip and setuptools that you can try out.
Changes since the last draft
A previous draft of this PEP was discussed in this thread. We have made a number of changes based on points raised in that discussion. Here is a summary of the changes:
-
A note was added in the Specification to indicate that
Metadata-Version
will need to be bumped to the next minor version (originally requested by @dustin, @pf_moore and @konstin) -
The
Default-Extra
field is now desribed as being ‘multiple use’ in the Specification section and it is explicitly stated that only entried already present in aProvides-Extra
can be used in aDefault-Extra
(originally requested by @pf_moore) -
Most of the former How to teach this section has been moved to a new Examples sub-section of the Specification section and the How to teach this section is a significant new section (originally requesetd by @pf_moore), and covers things such as:
- supporting older versions of package installers and the various aspects related to this that package authors need to consider
- avoiding bloating package dependencies
- when to have extras inherit default extras
- how to deal with the cases of incompatible and circular dependencies
- considerations for package repository maintainers
-
The Motivation section now includes three real-world examples for each general use case (originally requested by several people including @konstin @pf_moore). In addition, the new Examples sub-section includes one concrete example in each sub-section. I’d be happy to add or replace any of the examples if anyone would like to suggest more.
-
The
default-optional-dependencies
key inpyproject.toml
has been renamed todefault-optional-dependency-keys
to make it clearer it is not a list of dependencies but of keys. If anyone has a more concise suggestion, let me know! -
@pf_moore suggested adding a Transition section, but I found while trying to write it that there is a lot of overlap with How to teach this, so I have decided to just keep the latter for now unless we feel it is necessary to split the transition-related items out or repeat them in a separate section.
-
The Backward compatibility section has been significantly expanded and makes it clearer in which cases things may not be backward-compatible. In particular, this expliclty mentions the
pip freeze
problem raised by @bwoodsend and @pf_moore. -
The PEP now mentions explicitly what should happen if an invalid extra is specified for a package, in the case of tools that do not error when invalid extras are encountered (originally requested by @notatallshaw).
Open questions
The PEP explicitly mentions an open question that was raised and unresolved in the previous discussion thread. The PEP currently states that package[]
is equivalent to package
, but we could also potentially choose package[]
to mean ‘no default extras’. It’s not clear yet whether this would be technically feasible, but it would be worth discussing whether it would be desirable if it was possible.
Thanks, and looking forward to the discussion!
Tom