Add public functions to inspect dataclass parameters and methods

[DavidCEllis (David Ellis)]

dataclasses were originally intended as boilerplate writers but due to their structure a number of tools special case them and in doing so currently rely on either internal attributes or external observations.

I think we should acknowledge this and provide correct public functions to access the relevant parts of dataclasses. The idea is to expose the relevant details without requiring the use of internal names or really specific internal details (see the first example).

---

Proposal:

1. Add is_dataclass_type and is_dataclass_instance to detect if a type or instance is a dataclass or instance of a dataclass.
   • These should have a strict/exclude_subclasses^[1] parameter which would restrict detection to actual dataclass types or instances and not things that inherit from dataclasses.
     • Currently you have to check type(obj).__dict__.get("__dataclass_fields__") directly.
   • This strict parameter should also be added to is_dataclass
   • These should check for the presence of both _FIELDS and _PARAMS
   • Arguably we can just let users work out if they have a type or instance and just add strict, I think adding the specific methods is nicer though. dataclasses itself already has _is_dataclass_instance for this purpose.
2. Add params(cls_or_instance) to retrieve construction parameters from __dataclass_params__ in the same way fields(...) retrieves relevant fields
   • This will also require making _DataclassParams public
   • Unlike fields(cls) this function should probably fail for non-dataclass subclasses in order to prevent params(cls).frozen from being misleading. Non-dataclass subclasses of frozen dataclasses are not themselves frozen outside of the dataclass fields they inherited.
3. Add is_dataclass_method to detect methods generated by dataclasses
   • This would work by adding a dunder attribute to the methods when they are generated, unwrapping them to check if necessary.
   • This is already done for internal reasons to identify dataclasses’ __init__.__annotate__ function.

---

My examples here are largely from modules attempting to work out if __repr__ is the dataclasses generated __repr__ or if the class was instructed to create a __repr__ whether or not it actually did and doing so in different ways all of which use different internal details.

Two of these are from the stdlib but my view is that if unrelated stdlib modules need to poke around in dataclass internals then unrelated non-stdlib modules may also need to do the same but aren’t in the same privileged position. Also if we use these methods then people (or potentially their LLMs) may copy how we do it.

(Also I broke the pprint example when working on lazy dataclass methods because nobody should be relying on those details.)

---

Example 1:

pprint tries to detect if a __repr__ function was generated by dataclasses using this logic:

elif (is_dataclass(object) and
      not isinstance(object, type) and
      object.__dataclass_params__.repr and
      # Check dataclass has generated repr method.
      hasattr(object.__repr__, "__wrapped__") and
      "__create_fn__" in object.__repr__.__wrapped__.__qualname__):

This relies on dataclasses __repr__ being generated by a function called __create_fn__ which only exists inside the generation code. It then also relies on the fact that the generated __repr__ isn’t renamed until after it has been decorated with @recursive_repr, so the original __qualname__ still exists. This has already resulted in dataclasses not changing the name of its __create_fn__ function when calls were batched.

This would be replaced with:

elif (
    is_dataclass_instance(object) and 
    params(object).repr and
    is_dataclass_method(object.__repr__)
): ...

---

Example 2:

enum checks if a class is a dataclass and if repr=True was used when a class was created

if (
        '__dataclass_fields__' in base.__dict__
        and '__dataclass_params__' in base.__dict__
        and base.__dict__['__dataclass_params__'].repr
    ):
    return _dataclass_repr

This directly checks if __dataclass_fields__ is in the class dict rather than using is_dataclass, possibly due to is_dataclass detecting subclasses of dataclasses as dataclasses even if they are not dataclasses themselves. It also uses __dataclass_params__ to find out the parameters the class was defined with.

The equivalent to this would then be:

if is_dataclass_type(base, strict=True) and params(base).repr: ...

If it actually wants to check if the __repr__ hasn’t been overridden while repr=True it can also use is_dataclass_method(base.__repr__).

---

Example 3:

For a non-stdlib example rich also tries to detect if a __repr__ function was generated by dataclasses, but with different - flawed - logic

def _is_dataclass_repr(obj: object) -> bool:
    # Digging in to a lot of internals here
    # Catching all exceptions in case something is missing on a non CPython implementation
    try:
        return obj.__repr__.__code__.co_filename in (
            dataclasses.__file__,
            reprlib.__file__,
        )
    except Exception:  # pragma: no coverage
        return False

The actual logic rich’s pprint uses does also use is_dataclass but due to the check for reprlib.__file__ will incorrectly identify any __repr__ function on a dataclass that is wrapped in @recursive_repr as being a default dataclasses repr that it can replace.

This method would (eventually) be eliminated and replaced with is_dataclass_method(obj.__repr__).

---

I also see a number of uses of object.__dataclass_params__.frozen on Github^[2] to check for frozen dataclasses. Note that as-is this can also detect non-dataclass subclasses of frozen dataclasses as being frozen, which as mentioned before isn’t really accurate.

There are also cases of users trying to type hint __dataclass_params__ and needing to ignore errors or use Any because _DataclassParams isn’t public and as such isn’t in typeshed so type checkers don’t believe it exists.

---

1. or some other name ↩︎

2. at least, before Github went down again ↩︎

[NeilGirdhar]

@DavidCEllis

1. Add is_dataclass_type and is_dataclass_instance to detect if a type or instance is a dataclass or instance of a dataclass.

This seems like a good idea to me. It definitely comes up.

@DavidCEllis

• Currently you have to check type(obj).__dict__.get("__dataclass_fields__") directly.

You can just use

def is_dataclass_instance(obj):
    return dataclasses.is_dataclass(obj) and not isinstance(obj, type)

And vice versa for the type.

@DavidCEllis

• to actual dataclass types or instances and not things that inherit from dataclasses.

I see what you’re trying to do, but this is almost always indicative of poor design. You should not have a non-dataclass inherit from a dataclass (or vice versa) because the generated dataclass members will inconsistently work with non-dataclass members.

Points 2 and 3 seem reasonable to me.

[DavidCEllis (David Ellis)]

@NeilGirdhar

I see what you’re trying to do, but this is almost always indicative of poor design. You should not have a non-dataclass inherit from a dataclass (or vice versa) because the generated dataclass members will inconsistently work with non-dataclass members.

You can argue this, but frozen dataclasses’ __setattr__ explicitly allows setting attributes for subclasses that inherit from a dataclass that are not themselves dataclasses. I think I’ve only seen this used in pylint(?) though.

---

I think you get what I’m implying with this in your later paragraph but to be clear about the difference between your is_dataclass_instance function and what I suggested though:

>>> import dataclasses
>>> def is_dataclass_instance(obj):
...     return dataclasses.is_dataclass(obj) and not isinstance(obj, type)
...
>>> @dataclasses.dataclass
... class A:
...     pass
...
>>> class B(A):
...     pass
...
>>> b = B()
>>> is_dataclass_instance(b)
True
>>> type(b).__dict__.get("__dataclass_fields__", False)
False

I agree that inheriting from a dataclass outside of a dataclass is generally a bad idea, but it’s still useful to identify when other people have bad ideas .

[duncathan (Dunc Graham)]

@NeilGirdhar

I see what you’re trying to do, but this is almost always indicative of poor design. You should not have a non-dataclass inherit from a dataclass (or vice versa) because the generated dataclass members will inconsistently work with non-dataclass members.

It’s still useful to be able to inspect these sorts of things. If you’re writing a library that does introspection it’s perfectly good design to account for these cases (if only so you can raise an appropriate error, or find some way to handle the issue gracefully) when provided by users of the library.

[NeilGirdhar]

@DavidCEllis

I think you get what I’m implying with this in your later paragraph but to be clear about the difference between your is_dataclass_instance function and what I suggested though:

What are you trying to do? If your definition deviates from the is_dataclass_instance function that most people define, then you probably should call it something else.

@duncathan

It’s still useful to be able to inspect these sorts of things. If you’re writing a library that does introspection it’s perfectly good design to account for these cases (if only so you can raise an appropriate error, or find some way to handle the issue gracefully) when provided by users of the library.

Great point!

[DavidCEllis (David Ellis)]

@NeilGirdhar

@DavidCEllis

I think you get what I’m implying with this in your later paragraph but to be clear about the difference between your is_dataclass_instance function and what I suggested though:

What are you trying to do? If your definition deviates from the is_dataclass_instance function that most people define, then you probably should call it something else.

It’s about being able to distinguish between something that inherits from a dataclass and something that has actually been decorated with @dataclass. If you look at the second example it’s specifically checking the __dict__, which is the case that isn’t currently covered by is_dataclass. Without this you can’t replace the use of internals without changing the behaviour in that example.

Edit: Note that this is what the strict/exclude_subclasses parameter would be for, not a change to defaults.

[NeilGirdhar]

Right, that makes sense, but I don’t think you should call it is_dataclass_instance because that’s an extremely common name for anything that is an instance of a dataclass.

[DavidCEllis (David Ellis)]

I think perhaps decorated or decorated_class might be a better name. But the logic would be this.

Following the previous example

>>> is_dataclass_instance(b)
True
>>> is_dataclass_instance(b, decorated_class=True)
False

This could be a function like is_decorated_class instead of a parameter if necessary.

I actually think that saying b is a dataclass is misleading, but that’s what is_dataclass does.

>>> from dataclasses import dataclass, is_dataclass
...
... @dataclass(frozen=True)
... class A:
...     a: int
...
... class B(A):
...     pass
...
>>> b = B(42)
>>> is_dataclass(b)
True
>>> b.__dataclass_params__.frozen
True
>>> b.b = "this is writable"  # works, would not if B was actually decorated

[NeilGirdhar]

@DavidCEllis

>>> is_dataclass_instance(b)
True
>>> is_dataclass_instance(b, decorated_class=True)
False

Makes sense.

@DavidCEllis

I actually think that saying b is a dataclass is misleading, but that’s what is_dataclass does.

b is an instance of a dataclass. A is a dataclass, and b is an instance of A.

Honestly, I think that it was a mistake to make dataclass a decorator rather than a base class.

[DavidCEllis (David Ellis)]

@NeilGirdhar

Honestly, I think that it was a mistake to make dataclass a decorator rather than a base class.

It would make handling __slots__ easier.

[UnknownUser03393 (Neat Avocado)]

i have asked this (or alike?) Standardized Reflection Objects for Python Callables - #7 by jonathandung
i’m not saying something like you should not ask this, but instead i wnt to ask you for some suggestions…?

[DavidCEllis (David Ellis)]

So, I’ve done some thinking on this and have some modifications to my original proposal:

1. Add a require_decorator^[1] parameter to is_dataclass (default as False), this will check that __dataclass_fields__ is in the class dict and not just inherited.

2. Make _is_dataclass_instance public and add is_dataclass_type.

   • I think these are just useful helpers, I don’t see why _is_dataclass_instance needs to be private, I’ve seen it duplicated enough and it’s not something we’re likely to change.

3. Add a __generated_for_dataclass__ attribute to dataclass methods that is a reference to the class the method was actually created for, potentially along with a function get_method_class^[2] to retrieve this that returns None if the attribute doesn’t exist.

4. Add an is_frozen function to check if a class/instance is actually itself frozen, meaning you can’t set arbitrary attributes.

   • Essentially shorthand for checking the class __setattr__ and __delattr__ were generated for the exact class being examined.
   • Potentially we could also have has_frozen_fields to indicate the presence of some inherited frozen fields, but I’m not sure this is that useful and it can get pretty messy if you start to consider multiple inheritance cases (like frozen ‘fields’ that aren’t in fields(cls)).

5. Don’t provide params to get the parameters

   • I’ve changed my mind here after changing the attribute attached to the method to give the class the method was generated for which covers the original use case.
   • Many of the parameters will be ignored^[3] depending on other class features, so the features should be checked directly instead of relying on parameters that may not reflect the state of the actual class.

---

With these changes:

1. Checking or replacing methods

To check if a __repr__ has been generated for a specific class you can use:

get_method_class(t.__repr__) is t

To check if you can replace a __repr__ you’d now use something like:

if (repr_cls := get_method_class(cls.__repr__))): ...

As repr_cls (if it is not None) is the class used to generate the original __repr__ that would otherwise be used, dataclasses.fields(repr_cls) can be used to construct a replacement without needing any other requirements.

The behaviour of enum __repr__ replacement would change slightly so that if __repr__ has been replaced it would no longer make a custom enum repr, rather than doing so based solely on whether repr=True is declared in the dataclass params. The dataclasses documentation states that if the __repr__ is defined, the repr parameter should be ignored and the enum behaviour should really respect this.

2. Distinguishing decorated and undecorated classes

>>> from dataclasses import _FIELDS, dataclass
>>> def is_dataclass(obj, *, require_decorator=False):
...     cls = obj if isinstance(obj, type) else type(obj)
...     if require_decorator:
...         return _FIELDS in cls.__dict__
...     return hasattr(cls, _FIELDS)
...
>>> @dataclass
... class A:
...     a: int = 1
...
>>> @dataclass
... class B:
...     b: int = 2
...
>>> @dataclass
... class C(B, A):
...     pass
...
>>> class D(B, A):
...     pass
...
>>> is_dataclass(C), is_dataclass(D)
(True, True)
>>> is_dataclass(C, require_decorator=True), is_dataclass(D, require_decorator=True)
(True, False)

---

1. decorated_class or is_decorated seem nicer but imply that setting it to False should exclude decorated classes which isn’t the intention. Open to other possible names though. ↩︎

2. open to suggestions for better names here ↩︎

3. in practice this isn’t completely accurate currently, but it should be ↩︎

[NeilGirdhar]

Very nice iteration! You don’t need has_frozen_fields since you can just iterate over the fields and check.

@DavidCEllis

• but I’m not sure this is that useful and it can get pretty messy if you start to consider multiple inheritance cases (like frozen ‘fields’ that aren’t in fields(cls)).

Mixing dataclasses and non-dataclasses is essentially broken. I wouldn’t even try to make it work.