How to detect which extras were installed at runtime?

I do have an use case where I do want to know if a package was installed with extras or not, but not by checking if all the dependencies from those extras are present as this would be extremely slow.

Does pip store this information anywhere?

1 Like

I’m not sure if this answers your question but what my code does is to wrap an import in a try block. It sets a flag depending on whether the import succeeds or not.

Short answer: no.

Longer answer (AFAIK): Storing "installed extras" metadata in installations · Issue #215 · pypa/packaging-problems · GitHub for the current status & possible resolutions.

1 Like

Notably, this comment with a summary of the discussions from PyCon 2019 has all the details:

It’s also worth noting that pip’s resolver effectively treats package[extra] as a different “candidate”, which depends on package (with extra constraints to make sure the versions match).

That was one of the “how we could do this” options in the discussions. So we (sort of) have a proof of concept of how that approach might work. Which is good, in that it’s not just theory. But it’s also bad, in that the extra handling in the resolver is a bit of a mess, and there are a lot of corner cases. So there’s a good chance that using that approach here would also be a bit less straightforward than we might initially hope.

Import check is bad once your extras have more than one dependency and especially once you start adding version ranges. It would be close to impossible to check that you have all dependencies and within accepted ranges. An import can succeed, even if the version is not compatible.

Sadly that makes the import check quite impractical to use.

1 Like

A while ago there was a discussion to use the currently unimplemented REQUESTED metadata to record this, but it didn’t go far because it’s surprisingly difficult to keep track of installed extras (well maybe not if you know how bad extras are). For example, say I do

pip install package[tests]  # Pulls in pytest
pip uninstall pytest

Should the tests extra still be considered installed? If not, how are we supposed to know to remove it (without a full db scan which is too costly). What if tests pulls in two dependencies and only one is uninstalled? Ultimately the distribution metadata is not up to this task as of now, a lot need to happen before this can be viable.

1 Like

The problem is that people think of this in terms of “record what pip commands were executed”, because that’s what it feels like in their minds. So it’s easy to think the problem is simpler than it is. But that’s not the hard part, the hard bit is maintaining a database of “what is present on the system” dynamically, as things change (i.e., identifying the implications of each pip command).

If people can get what they want from the simpler “what pip commands were issued” data, then it’s probably sufficient to install a wrapper that records sys.argv and then forwards the work onto pip. I don’t think that’s sufficient, but it might cover the requirement as originally stated:

FYI, this was discussed at the PyCon Packaging Summit 2022 and further progress has been made; see: