The fastest way to make a list of installed packages?

Python Help
1

Hi there

I want to make a list of all installed packages (no matter whether importable or not).
So I tried two methods as follows:

def walk_packages():
    def _error(modname):
        print("  Failed: {}".format(modname))
    yield from pkgutil.walk_packages(onerror=_error)

a = list(walk_packages())

def walk_packages_no_import(path=None, prefix=''):
    for info in pkgutil.iter_modules(path, prefix):
        yield info
        if info.ispkg:
            root = info.module_finder.path
            name = info.name.rpartition('.')[2]
            path = [os.path.join(root, name)]
            yield from walk_packages_no_import(path, info.name+'.')

b = list(walk_packages_no_import())

The first method took 8 seconds and produced a complete list of successfully imported packages. It’s a bit slow as it performs import internally.

The second method took 3 seconds and was faster than the first method. It seems good except it didn’t generate a complete list:

>>> set(a) - set(b)

This will display some packages that it is importable but failed to get such as win32comext.
I suspect that the second method fails to visit packages that have more than one path.

>>> import win32com
>>> win32com.__path__
['C:\\Python310\\lib\\site-packages\\win32com', 'C:\\Python310\\lib\\site-packages\\win32comext']

So, what I want to achieve is to get the list as complete as the first method and as fast as the second method.
Any idea?
Thanks in advance!

2

I do that kind of list from a terminal, rather than in Python:

pip3 list --format=columns
3

Importable packages or installed distribution packages?

For the latter:

importlib.metadata.distributions()
4

Thank you for your suggestion, Rob.
Unfortunately that’s not what I intended.
What I want is a list of all packages and modules:

>>> pprint(sorted((x.name for x in a), key=str.upper))
['abc',
 'adodbapi',
 'adodbapi.adodbapi',
 'adodbapi.ado_consts',
 'adodbapi.apibase',
 'adodbapi.is64bit',
 'adodbapi.process_connect_string',
 'adodbapi.remote',
 'adodbapi.schema_table',
 'adodbapi.setup',
...
5

Thanks, but ditto… :point_up::slightly_smiling_face:

>>> import importlib.metadata
>>> sorted(x.name for x in importlib.metadata.distributions())
['Babel', 'Cython', 'Flask', 'Jinja2', 'MarkupSafe', 'Pillow', ...(snip)...]
1 Like