In Python, what's the best way to implement a `@cached_module_property`?

I regularly need module attributes (not a function) that are lazily computed on first usage (my_module.MY_VALUE).

I’m trying to implement some @cached_module_property that:

  • extract the globals of the caller (inspect.currentframe().f_back.f_globals) to
  • add a module-level __getattr__ such as
  • calling MY_VALUE trigger the my_module.__getattr__('MY_VALUE') in which I inject the caching logic.

However my_module.__getattr__ is only called if the my_module.MY_VALUE isn’t present in the module. So I’m forced to explicitly remove MY_VALUE, like:

@cached_module_property
def MY_VALUE():
  return ...  # Expensive computation


if not typing.TYPE_CHECKING:
  del MY_VALUE  # Required to trigger the `__getattr__`

Having a cached module property is a quite common feature. Is there a cleaner way to have this ?

Alternatives considered:

Properties (because they are implemented with the descriptor protocol) only trigger the call to the getter because you access the property via an attribute lookup.

For module-level constants, there is no such attribute lookup to trigger the protocol. MY_VALUE is just an ordinary name lookup.

Also, the descriptor protocol is triggered by accessing a class attribute via an instance. MY_VALUE, viewed as an attribute, is an instance attribute on your particular instance of types.ModuleType, so my_module.MY_VALUE would not trigger the descriptor protocol, either.

1 Like

AFAIK there is no clear solution to this problem.

I would add

setup_module_properties(globals())

at the bottom of module sources. setup_module_properties() would simply remove all all items marked as cached module properties. It can also move them to special dict and add a module-level __getattr__, this may simplify the code of cached_module_property.

It was a simple way. The complicated way – cached_module_property can get the current module from sys.modules and replace it with a wrapper with overridden __getattr__ or __getattribute__. It does not require all functions to have a self argument. But it affects performance.

1 Like

Could you use lazy-object-proxy? Your global data class with cached properties seems pretty reasonable if you didn’t want to use a library.

There are a lot of things that can be done with classes that don’t work directly with modules.

One common hack is to have the module replace itself with a class instance in sys.modules on import. Something like (I didn’t test this, but I know I’ve seen similar before):

from functools import cache
from time import sleep

class _implementation:
    @cache
    @property
    def value(self):
        time.sleep(10)
        return 'value'

if __name__ != '__main__' and not isinstance(sys.modules[__name__], _implementation):
    sys.modules[__name__] = _implementation()

What do you need this to do that’s more complicated (with inspect etc.) than what’s already possible with: a singleton pattern, even with a factory. Or howabout just a class that is not instantiated, or has no instance variables, but can just be used as a class. Or a class that checks to see if it the stuff that needs computing has been computed yet before each call of its methods. Or that is setup with wrapper methods, that do the lazy computation on the first call, then afterwards reassign their own name to the wrapped method, and then call that method.