PEP 726: Module __setattr__ and __delattr__

PEPs
21

I haven’t yet read this PEP but it sounds like my idea for using the module object as the global namespace would give the desired behaviour. I had prototype that was passing nearly all tests so I think the backwards compatibility issues are not too horrible. A high level summary of my idea is that global lookups do getattr() on a module object, rather than __getitem__() on a dictionary-like object. The import machinery would pass around module objects rather than the module globals dict, like it currently does. Maybe my idea is not very good or not explained very well. As Guido suggested, I should write a PEP for it.

3 Likes
22

The sys module API is designed around the idea of setting its attribute, but there are other stdlib modules with some attributes which can be overriden (and it’s explained in their doc, it’s the way to go).

Some more examples:

  • threading.excepthook doc: can be overridden to control how uncaught exceptions raised by Thread.run() are handled.

  • warnings

    • warnings.showwarning() doc: You may replace this function with any callable by assigning to warnings.showwarning
    • warnings.filters is not documented (!), but it’s commonly modified. Its own warnings.catch_warnings() functions does self._module.filters = self._filters to restore the attribute to its previous value.

There are some constants which may be interesting to implement as constants: either deny modifying them, or at least disallow removing them. Examples:

I’m not declaring a war to monkey patching (even if I not really like that). If the PEP is approved, each of these attributes should be discussed on a case by case basis.

On the other side, it seems like io.DEFAULT_BUFFER_SIZE can be modified on purpose. Just don’t assign it to a string, right? :wink:

23

@skirpichev @AA-Turner: Would you mind to update the PEP to mention that the API is limited on purpose and list limitations? In short, summarize what has been discussed here. You might also add a few more examples in the PEP, look at examples that I gave.

24

I was hoping the major limitation is mentioned in the Specification section:

Do you suggest to rephrase this or to extend with an explanation for this limitation (which is very clear, I believe)?

Maybe it’s better to replace an artificial example (inspired by the mpmath) in the Motivation section? It was here just to illustrate all use cases for the proposed interface (read-only attributes, input validation, monitoring attribute change) in one shot.

Or we can add a new section (Discussion?) and add real-world examples from the discussion thread. BTW maybe this is a suitable place also to mention reasons for limitations of the API.

25

No, I think that it should be elaborated. See previous messages.

26

FYI: PEP was updated to address intentional limitations of the API and to include few more examples from the discussion thread.

1 Like
27

So, what’s the next step? Ask the Steering Council to take a decision on that PEP?

Or are there still open questions or points to discuss?

28

So, what’s the next step? Ask the Steering Council to take a decision on that PEP?

I think it’s ready. I would appreciate @AA-Turner opinion on the PEP state, so lets wait few days. Then I’ll open an issue.

Edit:
JFR, the issue thread has additional example of using module’s __setattr__ to handle deprecation warning (actually, something like this was mentioned before in the discussion).

29

Hi.
I’m following up on the comment by Sergey.

There is an open issue regarding the warnings module not being thread safe, namely the bit regarding catch_warnings() and monkeypatching of the showwarning function. I have commented on that issue and came up with a very rought draft branch. [1]

This is unfortunatelly an anti-pattern with some bits of the Python stdlib, where specific behavior is controlled by monkeypatching global symbols. This is unsafe.

Compare for instance sys.excepthook vs atexit.register(callback). The latter is a good example of how to setup hooks or callbacks. The first one not so much.

Back to warnings, fixing that module would require forbidding all monkeypatching (and many other implementation details with a stack of catch_warnings objects, not relevant for this discussion).

However, that module is as you well know quite a central part of the python stdlib, and any change in its behavior needs 1. to go through a deprecation cycle, plus, such changes should not introduce any 2. performance penalties.

Therefore, with these concerns, 1. requires a __setattr__ hook to warn that monkeypatching that function will no longer be supported, then in a latter version, raise an Exception. The performance part 2. kind of negates the choice of subclassing the warnings module with a class with a setter, because of the perormance penalty when accessing attributes.

To conclude, the proposal of a __setattr__ in modules is something that would not have that wide use, but there are at least some niche use cases, including these problems with warnings that I mentioned which do not have any other good options. Hopefully, people should not be designing APIs that require patching global functions, but that ship has sailed long ago.
Moreover, I personally like when APIs are complete, symmetric and consistent, so it’s only natural that if a module object has a __getattr__ that it would too have a __setattr__.

Thank you.

PS: I think that there should be a PEP about removing all monkeypatching from stdlib and change those APIs to use register callback functions. It would not be much work implementation wise, but it would be a bit bureocratic.

[1] warnings.catch_warnings is async-unsafe · Issue #91505 · python/cpython · GitHub

1 Like