Extend PEP 562 with __setattr__ for modules?

Ideas
21

There are two separate potential uses of a module __setattr__:

  1. To be able to intercept setting an attribute and do something as a result (e.g. update some other state).
  2. To prevent setting attributes.

I wonder if there are as many uses for case 1 as for case 2. In terms of the problem referenced in the OP case 1 would be something like saying that it is okay for a user to set mpmath.dps = 100 and the mpmath module will have a __setattr__ method that can intercept that and then update the dps attribute on whatever other object and/or update other things as needed. Case 2 is the motivation given in the OP though: we just want to prevent setting attributes and raise an error at runtime if a user of the module attempts that.

Case 2 resembles somewhat typing.Final which makes me wonder if it couldn’t have a simpler solution which is just to set an attribute in the module like:

__final__ = True

Then the modules attributes would become read-only after its code finishes executing (I guess the module’s __setattr__ could handle this). A potential advantage of this approach is that it could also be understandable by type checkers and other static analysis.

22

So, can I try to start with an ordinary issue (and a pr)?

I’ve prepared a draft version of the PEP. There are some doubts if this should be a separate PEP (as now) or replacing the PEP 562.

Let me know if you can sponsor this PEP.

@oscarbenjamin , I appreciate your help with the original issue and related discussions. Let me know if you would like be a co-author.

23

I read it and I think it provides a clear explanation. Thanks.

Regarding the PEP fragment:

Defining module __setattr__ only affect lookups made using the attribute access syntax — directly accessing the module globals is unaffected.

Does that mean (assuming the PEP’s mplib.py) the following?

>>> from mplib import CONSTANT
>>> CONSTANT = 42
>>> import mplib
>>> mplib.CONSTANT
42

If so, how to teach (without explaining __setattr__ at the stage people learn import)?

24

No. Rather you could use globals() to access CONSTANT. See examples in the mplib.__setattr__ for dps/prec. Or just directly access mplib.__dict__:

>>> import mplib
>>> mplib.CONSTANT
3.14
>>> mplib.__dict__['CONSTANT'] = 42
>>> mplib.CONSTANT
42
25

Happy to read that.

That’s pretty advanced, so I’d assume (risky :wink: ) people doing that to understand they override CONSTANT’s protection, and not worry.

26

Let me know if I should somehow reword this sentence in the PEP (it almost literally follows to the note in the current docs for customizing module attribute access). I was thinking that using globals() in the mplib.py example is clear enough.

That works more or less in the same way as for customizing class attributes. Аn example:

>>> class Foo:
>>>    spam = 111
>>>    def __setattr__(self, name, value):
>>>        super().__setattr__(name, 42)
>>>
>>> foo = Foo()
>>> foo.spam
111
>>> foo.spam = 'bacon'
>>> foo.spam
42
>>> foo.__dict__['spam'] = 'bacon'
>>> foo.spam
'bacon'
27

My mental, and technically not fully correct, model of getattr() is that it implements the “.” delimiter-acting-as-kind-of-operator. Then setattr() is the corresponding assignment variant “.=”.
Therefore, I mentally connect __getattr__() and __setattr__() methods to tailor the dotted member read and write access.

Indeed, assuming PEP 562’s first motivating example’s lib.py, and then disassembling:

>>> import dis, lib
>>> dis.dis('lib.old_function')

shows the same bytecode:

  0           0 RESUME                   0

  1           2 LOAD_NAME                0 (lib)
              4 LOAD_ATTR                1 (old_function)
             14 RETURN_VALUE

as:

>>> class X:
...     def __getattr__(self, name): 
...         return 1
>>> x = X()
>>> dis.dis('x.field')

i.e.:

  0           0 RESUME                   0

  1           2 LOAD_NAME                0 (x)
              4 LOAD_ATTR                1 (field)
             14 RETURN_VALUE

The compiler generates the LOAD_ATTR instruction for both module and class instance member access. Then, the bytecode interpreter/virtual machine apparently calls the module’s or class’s __getattr__() method with the expected behaviour.

If we don’t use dotted access (lib.old_function), but rather import the member, it still works (which is specified in PEP 562, but nonetheless surprised me somewhat because of above mental model).
Disassembling:

>>> dis.dis('from lib import old_function')

yields:

  0           0 RESUME                   0

  1           2 LOAD_CONST               0 (0)
              4 LOAD_CONST               1 (('old_function',))
              6 IMPORT_NAME              0 (lib)
              8 IMPORT_FROM              1 (old_function)
             10 STORE_NAME               1 (old_function)
             12 POP_TOP
             14 LOAD_CONST               2 (None)
             16 RETURN_VALUE

Apparently, the bytecode interpreter/virtual machine executes IMPORT_FROM equivalently as LOAD_ATTR, i.e. calls the __getattr__() as needed.

For the proposed PEP 562 extension, and with lib enhanced with CONSTANT = 3.14, the dotted access can work similarly for module as for class instance member, i.e.:

>>> dis.dis('x.field = 10')
>>> dis.dis('lib.CONSTANT = 42')

show:

  0           0 RESUME                   0

  1           2 LOAD_CONST               0 (10)
              4 LOAD_NAME                0 (x)
              6 STORE_ATTR               1 (field)
             16 LOAD_CONST               1 (None)
             18 RETURN_VALUE

  0           0 RESUME                   0

  1           2 LOAD_CONST               0 (42)
              4 LOAD_NAME                0 (lib)
              6 STORE_ATTR               1 (CONSTANT)
             16 LOAD_CONST               1 (None)
             18 RETURN_VALUE

Similar to for the class instance, the bytecode interpreter/virtual machine could be changed to call the module’s __setattr__() method with the expected behaviour.

However, without dots:

>>> dis.dis('from lib import CONSTANT; CONSTANT = 42')

disassembles into:

  0           0 RESUME                   0

  1           2 LOAD_CONST               0 (0)
              4 LOAD_CONST               1 (('CONSTANT',))
              6 IMPORT_NAME              0 (lib)
              8 IMPORT_FROM              1 (CONSTANT)
             10 STORE_NAME               1 (CONSTANT)
             12 POP_TOP
             14 LOAD_CONST               2 (42)
             16 STORE_NAME               1 (CONSTANT)
             18 LOAD_CONST               3 (None)
             20 RETURN_VALUE

I fail to see how STORE_NAME 1 (CONSTANT) could possibly “know” it must call __setattr__() (while it must not do so for STORE_NAME 2 (other_global)).

Very possibly, this is just my limited understanding. But I’d like to understand, and therefore indeed suggest documenting it in the extension PEP.

28

It can’t know. In this case you just add new binding to the current module namespace, which is not the lib’s module namespace.

29

Hi, I created Support defining __setattr__() and __delattr__() in a module · Issue #106016 · python/cpython · GitHub to propose the same idea.

30

Updating this thread with a reference to GH-108261, a proposed reference implementation.

As I commented on the PR, we already have a PEP for adding only one attribute (__call__; PEP-713) – this is a much larger proposal and as such I believe should go through the PEP process, given this precedent.

A

1 Like
31

Follow-up: PEP 726 – Module __setattr__ and __delattr__ was written and is being discussed in the PEP section.

1 Like