Builtin "export" decorator/function

It would be nice to not have to reiterate names when using __all__, and instead do something like

@export
def my_function():
    pass

@export
class MyClass:
    pass

which would append "my_function" and "MyClass" to __all__, creating it if not already present. This avoids reiterating exported names, and keeps the exportation near objects’ definitions. This is already possible in pure python, but having to import something to export something (cleanly) is strange and clunky. Additionally, it may be desirable to have something like

my_variable = 1
export(my_variable)

which would require reiterating my_variable but would keep its exportation near its definition. It might be possible to do this (and keep the decorator functionality) without making it an intrinsic function by calling various inspect functions and parsing the relative source to get the variable name and whether export is being used as a decorator or not, but that seems very overkill when the interpreter should be able to get all that information much more simply.

2 Likes

That’s an interesting idea. Of course, it (I think) would only work for classes and functions (stuff you can decorate). Something like this:

% python
Python 3.9.1 (default, Dec 11 2020, 14:32:07) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from export_example import export
>>> __all__
autoloading __all__
Error in sys.excepthook:
Traceback (most recent call last):
  File "/home/skip/misc/python/python3/autoload.py", line 35, in _exec
    exec(import_stmt, f_locals, f_globals)
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named '__all__'

Original exception was:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name '__all__' is not defined
>>> @export
... def f(a):
...   return a
... 
>>> __all__
['f']
>>> 
>>> @export
... class X:
...   def __init__(self):
...     pass
... 
>>> __all__
['f', 'X']

Here’s the barebones export_example module. I’m sure the export decorator doesn’t adhere to recommended practice and doesn’t trap any user mistakes, but it demonstrates that it can do what you requested.

import inspect

def export(func_or_class):
    name = func_or_class.__name__
    caller = inspect.stack()[1]
    globals = caller.frame.f_globals
    if "__all__" not in globals:
        globals["__all__"] = []
    if name not in globals["__all__"]:
        globals["__all__"].append(name)
    return func_or_class

if all you care about is decorators, I believe it should be as simple as

import sys

def export(cls_or_func):
    module = sys.modules[cls_or_func.__module__]

    if not hasattr(module, "__all__"):
        module.__all__ = []

    module.__all__.append(cls_or_func.__name__)

    return cls_or_func

I do think it would be nice to be able to do export(my_variable) as well, but even just getting the decorator version as a builtin would be very nice.

1 Like

I think you’re looking for something like atpublic · PyPI .

1 Like

Ah, that is actually really clever using keyword arguments to mimic assignments. I’d still be interested in getting something like this builtin though, as having to import something to neatly export something is cumbersome, not to mention needing a dependency for what I feel should be a pretty basic thing for the language.

Even better. Just toss out my little hack.

Which brings up a corollary question. How are people supposed to find out what’s “good” in PyPI? There are so many packages…

1 Like

Bumping this because I still think it would be good to have an export facility builtin. I am currently writing code which discovers subclasses of a parent class through the __subclasses__ method, and so if I create a subclass of the parent class, that class automatically gets “proper” functionality without me needing to export it, which makes it much harder to remember to add it to the __all__ variable, because everything except actually naming the class works fine without doing so.

And I still very much maintain that

@export
class MyClass:
    pass

and

export(MY_VARIABLE=1)

are not only much easier to write, but also much easier to read.

Have you checked out the atpublic package? I realize it’s technically not built-in, but if anything, you’d need to start with a PEP and/or PyPI package to demonstrate its usefulness and iron out any wrinkles. That package offers an install function you can use to stuff the decorators/functions into the builtin module saving the effort of requiring import everywhere.

Also, @barry opened a ticket for inclusion years ago which was “closed – won’t fix”.

At minimum, you’d have to understand and address the issues which kept the concept out of Python at that time.

Well, as author of atpublic I agree! :wink: It even works exactly the way you want and can even be injected in builtins.

I opened that issue years ago and talked to a bunch of core developers at the time about adding it to the stdlib, but it didn’t get much of a warm welcome, so I dropped it. It’s just as easy to install it from PyPI. The library and its API has been incredibly stable, so it would make a perfect candidate for the stdlib. Just sayin’ :smile:

5 Likes

Yeah, for now I’ll use atpublic, thank you. I investigated injecting it into the builtins but since my project is a library I would be concerned about it leaking into users’ builtins as well. But yes, the library is workable, though I don’t think it’s at the ideal state, which would be making it a standard builtin.

I think a better name would be export (especially when it comes to putting variable names in __all__, a verb makes more conceptual sense I think), but that’s semantics at that point. In the github issue linked I didn’t see too many reasons to oppose it that seemed substantive to me, perhaps it just needed time to ferment? I’d be interested in whatever help I could give to making it a standard feature, though I haven’t really dabbled in PEPs at all. Maybe people nowadays would be more open to it.

And again, thank you for your work on this.

I should probably add that to the documentation as a recommendation against installing in built-ins for a general purpose library.

It’s bikeshedding of course, but to me, @public works well when you add the “opposite” @private [1].


  1. although the use case for even having @private isn’t that strong, IMHO ↩︎

1 Like

Hmmm… Maybe an install/remove pair? Your __init__.py could look like (spitballing, not bikeshedding):

from atpublic import install, remove
install()
blah blah blah
remove()

OR (now maybe I’m bikeshedding):

from atpublic import public_manager

with public_manager:
    blah blah blah

public_manager would take care of properly calling install and remove.

The only open issue on the project tracker is about providing a context manager, although with different semantics as what you’re suggesting. OTOH, I really wonder how useful the inject-into-builtins feature really is.

I think only for library authors to make it easier to get the benefit without having to import it in every module of a (potentially large) package.

That issue actually brings up a decent argument for standardization too. If there were a standard public then linters would be able to see public(a=1) and correctly assume that a is now a variable that could be named. Not something that a non-standard library can really offer I think.

Though I do find myself quite fond of that

with public:
    a = 1

syntax, though am unsure how it would work implementation-wise.

1 Like

I would suggest engaging in the atpublic issue if you want to discuss further.

I’d advise against this. I have been doing builtin injections with mxTools back in the days with many new builtins and esp. in larger projects this became a maintenance pain: figuring out whether a module used one of the many new builtins was getting increasingly difficult. It’s better to be explicit and import whatever you need at the top of the module.

4 Likes

As a late arriver to the thread - the o.p. asking for an “export” just looks like some, with no ofense needed, a “javascriptcism”.
Python does not need it because it has module namespaces - while Javascript, without such mechanisms, would have all declared functions, classes, etc…in all modules of a project being in a common global namespace.

That said, it is normal to try ways to avoid “WET” stuff and having to repeat things in __all__ - but __all__ itself is not that a useful language feature. As a package author, ok, I can do "from subpackage import *` into a package root, but it is about it - (and personally I avoid that too).

Sorry, just my 2 cents - as for the atpublic that does builtin injection - I’d also say it “smells better” not to be a standard language feature - one missing ZEN is that Python invites for good coding practices. So whlle atpubic is great, and I might even end using it in some projects, having this functionality by default might deteriorate coding practices long term.
(for one, it would be very hard for a developer, unaided by a specialized IDE, i.e. with just a common code editor, to find out where did a name came from if it was not explicitly imported)

If you think __all__ is useful, then atpublic can help you keep it in sync. More than that, it is good documentation when reading the code that a symbol is intended to be publicly exported. It doesn’t bother me at all if you find no value in @public.

As for builtins injection, it’s pretty trivial to add yourself if you really want it. So I’ve opened a ticket for its removal.

Just of itself, not of what it decorates.

Edit: It seemed like the rest of the paragraph was aimed at the assumption that @public could inject the things it decorates into builtins, but another reading makes me question that interpretation. So, basically, ignore this post :slight_smile: