PEP 842: Module Exports

[ZeroIntensity (Peter Bierma)]

@steve.dower

This flow is the critical thing. Python is popular as a prototyping language. Why? Because it’s great for one person to flow through the design of their program very quickly and fluently, and (here’s where this starts becoming directly relevant) part of that flow comes from the lack of boilerplate decision making.

Try and flow the same way in Java, C#, Rust, C++, or virtually any other significantly structured language and you will find yourself interrupted every other line with a decision to make. Should this be public? Private? Protected? Friend? Static? Global? RAII? GC? Which type? Will it convert? Will it cast? etc. You simply can’t flow.

I wholeheartedly agree with you that this is one of the many things that makes Python awesome. I just think we don’t see eye-to-eye yet on how this will be used.

See my previous comment for my own personal motivation. Based on the original discussion thread, there are definitely libraries that want this; JAX has been doing some nasty things to try to prevent people from touching internals. My problem with the existing solutions are:

1. __all__ doesn’t have any intrinsic relation to __dir__. “Unexported” (as in, not in __all__) names are still listed. And anyway, if you access a variable not in __all__, there’s no way to realize that you actually did that. I often use __all__ to remind myself what things are for use in my module, and I can’t count how many times I’ve realized later that something meant to be exported wasn’t listed in __all__. Since its conception, __all__ has been a setting to control namespace pollution with wildcard imports, and it just doesn’t fit the role of __export__ very well in practice.
2. del is painful and separates what should be public/exported from a module. From a maintainer POV, you have to search through a list of everything that isn’t exported, rather than a list of what is, and I think that’s much less useful.

Anyways, with __export__, you do about as much thinking as you would with __all__ (in fact, I expect many users to quite literally do __export__ = __all__). I’m not completely sure how to describe this, but I would argue that you’re not really making a decision every time you add a name. You’re just writing code, and if you decide that another module needs something, you can export it (or not care about exporting at all, if you’re just prototyping). This proposal is also trying to decrease the amount of thinking, since you don’t have to think about “should I prefix this with an underscore”?

In fact, I considered just making this PEP into a flag that made __all__ strict; see this section. If you interpret this as an extension to __all__ and not “access modifiers in Python”, does that make the proposal easier to swallow conceptually? If not, please offer some guidance on where you would like to see this proposal go. I get that we’re basically in the realm of philosophical discussion about Python, but I hope you understand that this proposal does offer real benefit to users (and us as core devs, since it improves stdlib maintenance), and we should investigate ways to achieve that without completely breaking Python’s conventions.

@pf_moore

As a counterpoint, I regularly use dir to find details of the internals of a module, with the explicit intent of either understanding details about the implementation, or using those internals . This PEP would encourage module authors to prevent me from doing just that.

Hm, okay, I hadn’t considered this use case. Though I don’t find this convincing for not having this feature at all – it just seems like another base to cover when designing. I suspect that the number of people who want to prevent users from relying on private details far outweighs the number of people who want to understand internals with dir. We have to make tradeoffs, and there’s basically always going to be some group of people inconvenienced by adding a new feature. For your case, you’d be a del module.__exports__ away from continuing your workflow. Another option is to add an unexported argument to dir.