By the hub model, I mean a package structure in which hub modules deliberately collect and re-export names defined elsewhere in the package. These hub modules form a tree rooted at the package’s top-level __init__.py. They are intended as public import locations: users import names from them without needing to know which implementation module actually defines those names.
Implementation modules, by contrast, import names as dependencies needed for their own implementation. They are typically placed under a private package such as _src to discourage users from importing from them directly.
Hub modules always define __all__. However, implementers are free to either:
- Define
__all__ lists in their implementation modules as well, and then star-import in their hub modules, or else
- Use explicit imports in their hub modules.
With PEP 843, either way, the public interface is provided once. Approach #1 has it in the __all__ lists; Approach #2 has it in the hub module explicit exports.
My guess is that you’re imagining approach #1 above.
Approach #2 is attractive because it eliminates most uses of __all__. However, with PEP 844, approach #1 also does the same. Approach #2 is also more flexible because multiple names in a single implementation module could be exposed in different public modules.
I think that since approach #2 is more flexible, it’s probably the approach to encourage. You wouldn’t want to start with approach #1 and then have to switch.
Pandas is a classic example:
Imports like this:
from pandas.core.api import (
# dtype
ArrowDtype,
Int8Dtype,
Int16Dtype,
Int32Dtype,
become exports, and the __all__ declaration is deleted:
__all__ = [
"NA",
"ArrowDtype",
"BooleanDtype",
"Categorical",
"CategoricalDtype",
"CategoricalIndex",
"DataFrame",
True imports (there are two of them) remain:
import pandas.core.config_init # pyright: ignore[reportUnusedImport] # noqa: F401
Numpy does the above, but also benefits from replacing:
__numpy_submodules__ = {
"linalg", "fft", "dtypes", "random", "polynomial", "ma",
"exceptions", "lib", "ctypeslib", "testing", "typing",
"f2py", "test", "rec", "char", "core", "strings",
}
__all__ = list(
__numpy_submodules__ | ...
def __dir__():
public_symbols = (
globals().keys() | __numpy_submodules__
)
...
def __getattr__(attr):
if attr == "linalg":
import numpy.linalg as linalg
return linalg
elif attr == "fft":
import numpy.fft as fft
return fft
elif attr == "dtypes":
import numpy.dtypes as dtypes
return dtypes
...
with simply
lazy from . export linalg
lazy from . export fft
...
) - and it’s worth bearing that in mind.
For example: