Add a supported read-only classproperty decorator in the stdlib

Ideas
1

In 3.9 and 3.10, we were allowed to compose classmethod and property like so:

class A:
    @classmethod
    @property
    def x(cls):
        return "o hi"

print(A.x)

In 3.11 this was removed, and I am NOT proposing that it comes back.
I’m having trouble finding the bug reports right now, but there were issues with this composition.

I would suggest that most users of classproperties are satisfied for them to be read-only.

You can implement it yourself with a descriptor like so:

from typing import Any, Callable, Generic, TypeVar

T = TypeVar("T")
R = TypeVar("R")

class classproperty(Generic[T, R]):
    def __init__(self, func: Callable[[type[T]], R]) -> None:
        self.func = func

    def __get__(self, obj: Any, cls: type[T]) -> R:
        return self.func(cls)

Why is this solution insufficient?

The main issue is that this is not in any way standardized, so the seams will tend to show between various tools.

Would there be a home for a classproperty descriptor in the stdlib? If so, where?

6 Likes
2

Could you provide some examples motivating classproperties? I’m sure they are useful but I cannot see why at the moment.

3

Where was this documented to being allowed? I don’t see it in the What’s New python 3.9

4

Is this the bug report you are thinking of?

Raymond says that the proposed fix self.f.__get__(cls, cls) “looks weird and worriesome”. Can somebody explain what is weird and worrisome about it? Yes, it is calling a dunder directly, but that is allowed when necessary.

(To be absolutely pedantic, perhaps the call should be type(f:=self.f).__get__(f, cls, cls) but I haven’t tried it.)

3 Likes
5

Yes, that was the main bug report! Thanks for linking it.

It seems like the fix there is concerned with the general case, letting classmethod take other descriptors as its argument. But I’m not sure that’s necessary or even important to most users – class properties are the only case in which I’ve wanted this, and the only one which I’ve seen others ask about.

I think the best case I can make for it is “defining on a base class a computed property from other classvars”. Which is sort of generic / circular as a justification, since I’m just saying “a property, but of the class”.

Here’s a quickly cooked up example where it makes sense though

class HttpsClient:
    domain: ClassVar[str | None] = None
    base_path: ClassVar[str | None] = None

    @classproperty
    def base_url(cls) -> str | None:
        if not cls.domain:
            return None
        if cls.base_path:
            return f"https://{cls.domain}{cls.base_path}"
        return f"https://{cls.domain}/"

class FooApiClient(HttpsClient):
    domain = "foo.example.org"
    base_url = "/api/v1"

I also do not see it in the What’s New page, but it is mentioned on the docstring for classmethod in 3.9 ( Built-in Functions — Python 3.9.17 documentation ) and it’s definitely in there.

4 Likes
6

Seems interesting, but I’m confused that the “class property” can be overwritten and I still don’t understand in which case I should use it.

>>> class A:
...     @classmethod
...     @property
...     def x(cls):
...         return "o hi"
...         
>>> A.x
'o hi'
>>> A.x = 0
>>> A.x
0
>>> a = A()
>>> a.x
0
>>> a.x = 1
>>> a.x
1
7

I believe that you’re just seeing the normal behavior of classmethod there. You’ll see the same behavior with classmethod without a property.

I’m not sure if it has bearing on the utility of classproperties, other than that a stdlib-provided classproperty could define __set__ to raise an AttributeError.

8

Use case: simulate constant class-instance valued class variables, i.e.


class Vector3D:
    def __init__(self, x, y, z):
        self.x, self.y, self.z = x, y, z
        
    def __repr__(self):
        return f'{type(self).__qualname__}({self.x}, {self.y}, {self.z})'
    
    if 'this' is 'possible':
        e1 = Vector3D(1, 0, 0) # preferred way, but impossible as class is not defined yet
    else:  # fallback to simulated class variable
        @classmethod
        @property
        def e1(cls):
            return Vector3D(1, 0, 0)

print(Vector3D.e1)
2 Likes
9

I was going to put together a proof-of-concept implementation, to suggest either a new module or inclusion in functools, and had trouble finding an available and obvious package name on pypi.

I think this speaks to the desirability of this feature.

I found these four, and there are probably others:

10

There is a classproperty in our codebase. I copied one of the StackOverflow solutions and use it in three places.

One of them it to prevent a class atttibute from being picked up as a Enum member. Once 3.11, comes out, we will switch to the new nonmember decorator.

The other two let our base classes implement the Pydantic interface (which requires class attributes to be assigned Pydantic types) without the library having a hard dependency on Pydantic. The class properties let pydantic be imported inside the property body rather than in the global scope so that it is only imported when needed.

I don’t use classproperty much, but it is an annoying little hole that I need to fill myself ever since the death of @classmethod @property.

4 Likes
11

To add to the examples, astropy has its own definition of a classproperty. It is used a reasonable amount throughout the codebase (32 times). As noted on top, it is easy enough to implement read-only (we only added a lazy evaluation option)

1 Like
12

Perhaps open an issue on the tracker so that comments can be collected in one place.

This time we should be extra careful. Approving the current version, bpo-19072, was one of the biggest mistakes I’ve ever made as a core developer. It was plausible enough on the surface but wasn’t well thought-out and led to several bug reports that made clear that arbitrary composition of descriptor decorators didn’t tend to work out well.

It is clear though that people want some way to spell classproperty(). But be advised, it is a thorny subject.

AFAICT, nothing short of a metaclass can make it readonly. Using Stephen’s example:

from typing import Any, Callable, Generic, TypeVar

T = TypeVar("T")
R = TypeVar("R")

class classproperty(Generic[T, R]):
    def __init__(self, func: Callable[[type[T]], R]) -> None:
        self.func = func

    def __get__(self, obj: Any, cls: type[T]) -> R:
        return self.func(cls)

class A:
    @classproperty
    def x(cls):
        return "o hi"

print(A.x)       # Displays "o hi"
A.x = 10         # Easily overwritten
print(A.x)       # Prints 10

Another thing to worry about is that calling help() on the class will trigger the classproperty():


class A:
    @classproperty
    def x(cls):
        print('!')
        return 12

Here is a sample shell session:

>>> help(A)
!
!
!
!
Help on class A in module __main__:

class A(builtins.object)
 |  Data descriptors defined here:
 |  
 |  __dict__
 |      dictionary for instance variables (if defined)
 |  
 |  __weakref__
 |      list of weak references to the object (if defined)
 |  
 |  ----------------------------------------------------------------------
 |  Data and other attributes defined here:
 |  
 |  x = 12

Note that ! was printed four times.

We could hardwire help() to avoid this but help() is likely the canary in a coal mine — I suspect that the whole Python ecosystem is wired with a presumption that accessing a class variable doesn’t trigger arbitrary code. Even a hasattr() call will run the code.

We don’t have that problem with the other descriptors because they just return self when accessed from a class.

This is just a sampling of the kind of problems that arose. So this time, we should be careful and really think it through.

Raymond

1 Like
13

I might not be able to act on this much for a few weeks, but I want to say thanks for the recent additions to this conversation! It gives me confidence that I should open this up on the tracker when I am better able to follow-up on it.

The astropy implementation looks quite sophisticated, and will take some time to digest. But that will definitely be useful in exploring what’s possible.

The behavior of help and the underlying problem it points at, that the introduction of class properties may break existing code’s expectations, strikes me as unavoidable at some level. But help also opens up related and interesting questions – how can you access the classproperty descriptor and methods, rather than just the values? (e.g. how do you get the __doc__ of the getter?)

14

Thanks for putting effort into this. Just don’t expect this to be an easy problem. I had already looked at several published implementations. If there had been easy and clean solution (just do what Project X does), I would have already added it.

My suggestion is to pursue some implementation strategy that doesn’t rely on the existing descriptor protocol which is insufficient for this task. Consider proposing modifications to type.__getattribute__ to get this to work cleanly.¹ ²

The standard for adding this to our standard distribution is much higher than it is for a third-party project. A good classproperty() would need to have a read-only option, be cacheable, be threadsafe, be fast, have an accessible docstring, work with existing introspection tools, and not break other code.³ Ideally, it should be easy to type annotate, be usable with async code, and have a corresponding abstract_class_property decorator. There need to be mechanisms for recognizing a classproperty instance and for accessing it without triggering it.

Along the way, it may be reasonable to decide that it just isn’t worth it. The only benefit over a classmethod is an implicit call, the ability to write solar_system_ephemeris.bodies rather than solar_system_ephemeris.bodies().

Raymond

¹ The comments in Astropy point out that what they want can’t be achieved without a metaclass. However if support were built directly into Objects/typeobject.c, they wouldn’t need the metaclass.

² One idea to explore would be a new class level descriptor protocol with __class_get__, __class_set__, and __class_delete__.

³ The descriptor protocol has been around for 20 years (since Python 2.2). Expect that a substantial fraction of the Python ecosystem assumes that it is safe to inspect or run hasattr() on a class variable and not trigger a method call. Breaking their code is likely to be a non-starter.

1 Like
15

I would very much like to have a classproperty. I’ve had a use for it several times and maintain this implementation, but I’d love to see something like what Raymond proposes.

16

Thumbs up. I have found class property defined in several of my modules. It doesn’t feel right for it to just be added to its own module, then making all the others depend on it because of 7 lines of code. Currently just makes more sense to have it at the top of the module whenever needed.

All was good until I started standardising decoration frameworks. So I need to take classproperties into account for correct unwrapping etc. I could just have 1 implementation, but this doesn’t solve the issue of all the classproperties of external packages.

Long story short - I think it would be very useful to have it in std lib. Fairly common use case and somewhat missing piece for basic completeness.

17

Given that cached_property and property are separate decorators, classproperty does not necessarily need to support caching. In fact, there is already a cached_classproperty/cached_staticproperty implementation on PyPI:

Given that the concerns of the behaviors in __set__ and help() do not apply to cached_classproperty/cached_staticproperty. I wonder if it makes sense to add cached_classproperty/cached_staticproperty to the standard library.