Add the export keyword to Python

[pf_moore (Paul Moore)]

@Ovsyanka

But what do you think are the alternatives in this scenario?

Do nothing.

I still don’t see what’s wrong with __all__ apart from a certain amount of repetition, which I don’t consider critical. As written, your proposal defines export as being exactly equivalent to import followed by setting __all__. So literally the only benefit it provides, based on how you’ve stated your proposal, is to save manually writing the assignment to __all__.

Your proposal is clearly defined, and straightforward. I don’t have a problem with it. But as far as I’m concerned, you haven’t made the case that there’s a problem to be solved.

The side discussion triggered by @NeilGirdhar suggesting that there’s a bigger problem to solve is where decorators came up, but that’s not relevant to your proposal, because what you propose doesn’t solve that bigger problem anyway. As a result, we have two different discussions going on here.

1. Your original proposal for replacing __all__ with export, which simply reduces typing/repetition.
2. Neil’s discussion of having a way to define an “explicit restricted API”, which doesn’t yet have a clear problem statement or proposed solution, but which is independent of (but an extension of) your proposal.

I think wide-ranging discussions are generally healthy, and I don’t like to claim things are “off-topic” too quickly, but maybe there’s a case for Neil’s proposal (and the discussions it triggered) to be split into a separate topic here?

[pf_moore (Paul Moore)]

@mdrissi

Being part of standard library is generally expected though.

Precisely. I wish there were a reasonable way of 3rd party solutions being “first class citizens” for typing/static analysis. It would give us a much better way of innovating without having to make the difficult decision between “freeze the solution in the stdlib/language” (usually involving a PEP and choosing a single “blessed” solution) and “accept limited static analysis support”.

Maybe that would be a useful direction for the typing community to work on?

[NeilGirdhar]

@pf_moore

Neil’s discussion of having a way to define an “explicit restricted API”, which doesn’t yet have a clear problem statement or proposed solution, but which is independent of (but an extension of) your proposal.

I feel like I made the problem statement clear in this comment? And then I proposed a solution in this comment.

I feel like you responded to the problem statement when you argued that enforced privacy mechanisms

• creates an “adversarial relationship” between library authors and users,
• is contrary to the “consenting adults” principle, and
• is unnecessarily “formal”.

Is that a good summary of your counterarguments?

We also came up with three definitions about what it means to “expose” something:

• it’s in the documentation,
• it’s in __all__ and thus accessible in dir, and import *, or
• it’s in the package namespace and thus accessible through an import.

I think we both agree that some large libraries make an effort to “prune namespaces”. So the question comes down

• whether we want to make it easier for these libraries to do that, and
• do we think they would even use any mechanism that we might add.

What do you think of this summary?

@pf_moore

I think wide-ranging discussions are generally healthy, and I don’t like to claim things are “off-topic” too quickly, but maybe there’s a case for Neil’s proposal (and the discussions it triggered) to be split into a separate topic here?

I’m happy to split off this discussion. I had just assumed that we were brainstorming about exports here and that Stanislav was okay with that. I agree with you that adding new syntax (as per the original post) is probably not justified by just saving a small amount of typing, which is why I was looking at the larger issue.

@pf_moore

The reason for pip’s _internal module is because we were being particularly heavy-handed about getting our message across

Right. And if you had some machinery to prune namespaces then it would be easier for every library author to be heavy-handed in this way.

@pf_moore

My only point here is that the burdens aren’t sufficient to warrant a new language construct, that’s all. And how much burden something is, is inherently subjective.

Yes.

@pf_moore

Phew! I’m glad that at least I don’t come across as a complete PITA

Same! I hope I’m not upsetting you with my philosophy. I was worried you would lump me in with the people who are pestering you to add type annotations

@pf_moore

as such I tend to be rather defensive of the principle that people shouldn’t assume commitments that don’t exist -

Yeah, that’s an important point to keep in mind.

[pf_moore (Paul Moore)]

@NeilGirdhar

What do you think of this summary?

Sorry, yes. I didn’t check that closely as it wasn’t particularly relevant to my main point which was that it’s a different discussion than the one about @Ovsyanka’s proposal. My bad, I was trying not to spend too much time on what I intended to be a quick reply.

@NeilGirdhar

Right. And if you had some machinery to prune namespaces then it would be easier for every library author to be heavy-handed in this way.

I don’t think that pip’s use of _internal is particularly difficult. And it’s very effective at making it explicit that our API is internal. The complexity is in managing user expectations, dealing with people who need access to information our supported interface (the CLI) doesn’t provide, and generally making our support position clear. All of which are people problems, not technical issues.

And I guess that’s the core of my point - I see “what is your supported API” as a people question, not a technical one. So solving it with people-oriented tools like naming conventions, documentation and user education seems more appropriate to me than adding technical machinery that can at best support those social solutions, but can’t replace them^[1].

---

1. If we ever reach the point where even a determined user can’t access your implementation details, I don’t think we’ll be writing Python any more… ↩︎

[Rosuav (Chris Angelico)]

@Ovsyanka

As I mentioned: a decorator is not enough to handle __init__.py files. Sadly, we do need something more.

I understand that a decorator isn’t, but what if something could be done in importlib? Something like:

importlib.reexport_all(".submodule1", ".submodule2", ".submodule3", globals=globals())

Passing globals is a bit clunky but that would allow the actual imports to be done in a non-magical way, while never duplicating any names. This would go through all its arguments, import them, calculate the public symbols (either __all__ or those without underscores, as per a star import), and imports them all into the specified dictionary; and also updates your own __all__ to include them (creating it if it doesn’t exist).

No syntax needed.