Could you explain how this is an “unreliable runtime warning”? What makes it unreliable?
Sure there is – you just wrap the call in with warnings.catch_warnings, or use the __dict__ if you really want. I’m having a really difficult time grasping why you’re a fan of the exception but so opposed to a warning.
Neat, but keep in mind that this solution is somewhat insane. It’s on the same level as writing a custom codec to implement new syntax. I don’t find this example convincing for the idea that this PEP “isn’t offering anything new”.
Try to think about this in context a little more.
A user has spent a few months developing a small app, and in it, they’ve used the internals of your library. They weren’t necessarily trying to; they typed something.name_they_wanted and their editor found something._name_they_wanted. They’re already using sys._current_exceptions (which is a documented, public API) in their codebase, so a leading underscore doesn’t mean anything special to them. Their code works!
Then, say a few months later, they upgrade your library. All of a sudden, their code is broken. They track it down to a change in something._name_they_wanted, and file a report upstream. Instead of being told that their code will be fixed, they’re instead told that their code was never right in the first place (citing the leading underscore in _name_they_wanted, which they already find confusing due to the leading underscore in plenty of public APIs) and the only fix is to switch to a different public API that requires a lot of refactoring across their codebase.
Had they known _name_they_wanted was private (or seemingly didn’t exist, since the editor doesn’t suggest it) when they first added it, they could have made a quick search and found the public API when the code was much smaller. But now, they have to either stay on an older version of your library, or put in a lot of effort to migrate their code to the public API.
I hope you realize that this isn’t hypothetical. There’s a reason CPython has so many public APIs that were once private – people found them, used them, and we decided that the best course of action for users was to simply maintain them as public instead of breaking user code.
This isn’t supposed to be secure. People like to use internals for all sorts of monkeypatching. We still want to support that.
The point is to make it explicit that something is private, because I don’t think underscored names do that job perfectly.
Sure, and this is close to JAX’s internal_do_not_use naming, but that’s not nice to the maintainer at all.
And this is still possible. This PEP doesn’t prevent that. Again, this is not “access modifiers in Python”. You’re still free to explore a module’s internals if you want; you’re just not prone to making the mistake of reaching for an internal name without realizing that it’s internal.
This proposal isn’t supposed to remove the underscored names convention. It’s supposed to clear up the cases where users either aren’t completely acknowledging them or their meaning it ambiguous. I wrote a paragraph about this in the PEP:
even if this PEP is accepted, it’s expected that “underscored” names (names prefixed with a leading
_) will remain a staple of Python for years to come. The purpose of this PEP is not to eliminate the need for_in module-level names, but instead to clear up corner cases where a private name is ambiguous or tempting. In other words, this PEP is intended to improve expressiveness and clarity with private APIs, not to add brand new functionality.
I also wrote a section about this. I strongly believe that assuming every developer reads the complete reference documentation for something before using it goes against Python’s “flow” that others have described in this thread.
Another problem with the “read the docs” argument is that private things usually aren’t documented. The user has to just know. There’s nothing telling you not to use the standard _colorize module – you just have to guess because it’s prefixed with an underscore. You might say that Python should clearly document that modules prefixed with _ are private, but oh wait, _thread is public!
Agreed, I was kind of thinking out loud. The more general issue here is whether we want private names to be local to a module. For example, if I have a _utils module, I only want the contents of it to be accessible from inside my package. That’s not solvable with the current PEP, and I’m trying to come up with solutions.