If we’re already considering a built-in, it wouldn’t really be important. You can simply use PySequence_InPlaceConcat for concatenating what needs to be concatenated in this case. No need to type check anything here. It won’t really be more expensive IMO compared to the rest of the module’s execution, though you’ll need to replace the reference in globals() by the new tuple.
I also don’t think it’s that insurmountable of a constraint, especially since you can always turn it into a tuple at the bottom of the module and gain all the same benefits.
Yes but it defeats the purpose for me of “simplifying your code”. Writing __all__ = tuple(__all__) makes it harder for static analyzers as well (and I don’t know how mypy would infer the type in this case for instance).
Now, I think this won’t really be a nuisance. But I’d like the limitation’s rationale to be clearly spelled in the “rejected ideas” section (I know it’s in the “Restrictions” but it’s only because we need to mutate it; however there is nothing wrong with updating the __all__'s value in globals() with a replaced one; references to it would be lost though).
Now, I’d like to nitpick on some parts:
There is no complicated introspection, no allocation or work proportional to module size
Strictly speaking, there is a very small allocation for the list and its elements. But I agree that it’s negligible.
Code that already uses public or private as a variable or parameter name will begin to trip linters that flag shadowed builtins, such as flake8-builtins and the equivalent ruff rule. This is a diagnostic change rather than a behavioral one, and the same has been true of every builtin added to Python. How much existing code this affects has not been measured.
Can we have a quick GH search result just to have an idea (saying “as of TIMESTAMP, there are N usages of public/private”?
In the “Acknowledgments” section, the link to PEP-843 is missing.
It follows that @private alone does not exclude a name from from spam import *. Excluding names is the job of @public: as soon as any name in the module is marked public, __all__ exists, and everything not marked public is excluded automatically. @private records the author’s intent; @public is what makes that intent observable.
As an author, I wished I could also have this but I know it will be hard to do because this can’t be achieved solely with a decorator (but could be achieved with additional syntax). That is, it may be simply easier to write
<no __all__>
<lots of public functions>
@private
def my_private_function(): ...
Instead of having a leading _ or adding @public to EVERY other public function, it’d be easier for an author to write __private__ and consider __all__ = [<all names except the private one>]. Instead of creating an empty __all__, one should create an __all__ that contains all current global names in the module that do not start with an underscore.
Since the specs say
If __all__ is not defined, the set of public names includes all names found in the module’s namespace which do not begin with an underscore character ('_').
I consider that not specifying __all__ means that it’s equivalent to have this variable implicitly defined as above. In particular, using @private should alter that set of variable at import time.
# <no __all__>
def f1(): ...
...
def f999(): ...
@private
def g(): ...
# __all__ is implicitly set to ["f1", ..., "f999"]
For backwards compatibility, and to avoid breaking possible tests, __all__ is created by @private here and not automatically for all modules. By default, __all__ is not created (when neither @public/@private are used), but __all__ is always created when using that in a module (whether by creating it and extending it through @public calls, or by creating the implicit __all__ when @private is called first).
I’m unsure however if this is possible because I don’t well in which order the names are created when importing the module. If we follow regular execution, a public name defined after the call to @private would not be added in the initial list of __all__ because the function would not exist already in the module’s globals:
# __all__ is not specified
def f(): ...
@private
# Here, we need for @private to know that "h" exists later
# which needs the symtable as "h" is not in globals() here.
def g(): ...
# Here, __all__ should be ["f", "h"] even if "h" will be seen later.
def h(): ...
It’s not impossible to do, but it will definitely add an overhead as I don’t think we are keeping in memory the top-level names for the module being executed. Now, this can be easily achieved if populate_all() were exposed. I think there should be no heuristic, and clear specifications, so I would definitely prefer having something where I can explicitly say “this is private, the rest is public” rather than having to deal with “I add @public everywhere AND in addition I add @private where I want to be more expressive”.
Should public() and private() diagnose being called outside module scope? Neither inspects its calling scope today
I’d say yes. We want to be nice to users, not possibly trip them. I can definitely see people thinking that @public or @private could be used for more things (despite the docs saying something different) and I can definitely see a usage of @public/@private being extended to methods if we once decide to add __all__ to classes as well. So I would rather having this failing loudly. We should first determine how costly the check is.
FTR, a new keyword could also solve those issues and I personally would prefer a keyword but as the transition from @coroutine to async def, I think it’s also good to check if this is adopted enough first.
