# PEP 712: Adding a "converter" parameter to dataclasses.field

**URL:** <https://discuss.python.org/t/pep-712-adding-a-converter-parameter-to-dataclasses-field/26126>\
**Category:** PEPs\
**Created:** [April 23, 2023, 8:04pm UTC](https://discuss.python.org/t/pep-712-adding-a-converter-parameter-to-dataclasses-field/26126 "2023-04-23T20:04:55Z")\
**Posts on this page:** 20\
**Page:** 3

<div class="post-metadata">

**Author:** ![hlovatt](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/hlovatt/32/36838_2.png) [@hlovatt](https://discuss.python.org/u/hlovatt)\
**Post date:** [August 30, 2023, 10:33pm UTC](https://discuss.python.org/t/pep-712-adding-a-converter-parameter-to-dataclasses-field/26126/41 "2023-08-30T22:33:14Z")

</div>

I would find this useful. I have an application that takes JSON data and converts them to data classes and I do type conversions from JSON representations to more specific Python representations in ` __post_init__ `, e.g. ISO date string to `datetime` and literal 0 which comes through as an `int` to `float`.

---

<div class="post-metadata">

**Author:** ![ericvsmith](https://avatars.discourse-cdn.com/v4/letter/e/34f0e0/32.png) [@ericvsmith](https://discuss.python.org/u/ericvsmith)\
**Post date:** [October 9, 2023, 11:57am UTC](https://discuss.python.org/t/pep-712-adding-a-converter-parameter-to-dataclasses-field/26126/42 "2023-10-09T11:57:58Z")

</div>

Maybe I’m mis-understanding things, but I don’t think attrs has the behavior of “converts on attribute setting”:

```pycon
>>> @attr.s
... class C:
... x: int = attr.ib(converter=int)
...
>>> c = C("10")
>>> c
C(x=10)
>>> c.x = "20"
>>> c
C(x='20')

```

I couldn’t find this referenced in the attrs documentation, but maybe I wasn’t looking in the right place.

---

<div class="post-metadata">

**Author:** ![Tinche](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/tinche/32/12507_2.png) [@Tinche](https://discuss.python.org/u/Tinche)\
**Post date:** [October 9, 2023, 12:12pm UTC](https://discuss.python.org/t/pep-712-adding-a-converter-parameter-to-dataclasses-field/26126/43 "2023-10-09T12:12:07Z")

</div>

> [@ericvsmith](#):
>
> Maybe I’m mis-understanding things, but I don’t think attrs has the behavior of “converts on attribute setting”

It does for the new-style APIs (`define`, `frozen`). I remember this being pointed out in the docs but they must’ve deteriorated. We strongly encourage new-style APIs.

---

<div class="post-metadata">

**Author:** ![ericvsmith](https://avatars.discourse-cdn.com/v4/letter/e/34f0e0/32.png) [@ericvsmith](https://discuss.python.org/u/ericvsmith)\
**Post date:** [October 9, 2023, 12:15pm UTC](https://discuss.python.org/t/pep-712-adding-a-converter-parameter-to-dataclasses-field/26126/44 "2023-10-09T12:15:36Z")

</div>

Ah, thanks @Tinche. Here’s an example:

```pycon
>>> from attrs import define, field
>>> @define
... class C:
... x: int = field(converter=int)
...
>>> c = C("10")
>>> c
C(x=10)
>>> c.x = "20"
>>> c
C(x=20)

```

I think the PEP should be updated to note this, and save someone who’s not as familiar with attrs (like me!) the time to research it.

---

<div class="post-metadata">

**Author:** ![thejcannon](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/thejcannon/32/23759_2.png) [@thejcannon](https://discuss.python.org/u/thejcannon)\
**Post date:** [October 9, 2023, 1:01pm UTC](https://discuss.python.org/t/pep-712-adding-a-converter-parameter-to-dataclasses-field/26126/45 "2023-10-09T13:01:56Z")

</div>

Nothing like a miniature panic attack early on the morning 😂 I knew I tested it!

I’m happy to edit the PEP (assuming that’s kosher)

---

<div class="post-metadata">

**Author:** ![ericvsmith](https://avatars.discourse-cdn.com/v4/letter/e/34f0e0/32.png) [@ericvsmith](https://discuss.python.org/u/ericvsmith)\
**Post date:** [October 11, 2023, 11:15am UTC](https://discuss.python.org/t/pep-712-adding-a-converter-parameter-to-dataclasses-field/26126/46 "2023-10-11T11:15:40Z")

</div>

Another thing that occurs to me is interactions with pattern matching. What happens here?:

```python
@dataclass
class Point:
    x: int = field(converter=int)
    y: int

match Point(x="0", y=0):
    case Point(x="0", y=0):
        print("Origin")
    case Point():
        print("Somewhere else")
    case _:
        print("Not a point")

```

Naively, I’d expect it to print “Origin”, not “Somewhere else”. I realize the `Point` in the case statement isn’t creating an object, but clearly it’s meant to parallel object creation, at least visually.

This should be mentioned in the PEP: the match statement ignores any `converter`. But it does, for example, respect `init=False`.

---

<div class="post-metadata">

**Author:** ![fonini](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/fonini/32/20258_2.png) [@fonini](https://discuss.python.org/u/fonini)\
**Post date:** [October 18, 2023, 2:12am UTC](https://discuss.python.org/t/pep-712-adding-a-converter-parameter-to-dataclasses-field/26126/47 "2023-10-18T02:12:52Z")

</div>

> [@ericvsmith](#):
>
> Naively, I’d expect it to print “Origin”, not “Somewhere else”. I realize the `Point` in the case statement isn’t creating an object, but clearly it’s meant to parallel object creation, at least visually.

I think class patterns have a pretty clear rule: they match _attributes_ of the subject. Dataclasses, with or without converters, are not special here – any more than custom user classes in which ` __init__ ` parameters don’t correlate with instance attributes \[1\].

As I user, whenever I see a class pattern in a match statement, I mentally translate it to a sequence of checks like “isinstance + hasattr + the attr value matches the subpattern”. In your example, I believe that `case Point(x="0"): ...` in general should be read as a shorthand for:

```auto
elif (
    isinstance(subject, Point)
    and hasattr(subject, 'x')
    and subject.x == "0"
):
    ...

```

This means that the behavior for dataclasses with converters should be clear enough: the result of accessing the attribute `x` on the instance `Point(x="0")` will be matched to the string `"0"`. Since accessing the attribute returns an integer and `0!='0'`, the pattern will not match.

* * *

1. As far as I understand, dataclasses are only special in that they automatically generate a ` __match_args__ `, which I don’t think is relevant for this discussion

---

<div class="post-metadata">

**Author:** ![pf\_moore](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pf_moore/32/35_2.png) [@pf\_moore](https://discuss.python.org/u/pf_moore)\
**Post date:** [October 18, 2023, 8:11am UTC](https://discuss.python.org/t/pep-712-adding-a-converter-parameter-to-dataclasses-field/26126/48 "2023-10-18T08:11:24Z")

</div>

I disagree. The fact that the match statement uses syntax that looks like the initialiser is the important thing for me (and I believe it was a deliberate design choice as well). The discrepancy here wouldn’t be a complete disaster, but I’m pretty sure it would be a source of confusion and possibly bugs, so I think @ericvsmith is right and this should be explicitly discussed in the PEP.

---

<div class="post-metadata">

**Author:** ![jamestwebber](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/jamestwebber/32/12799_2.png) [@jamestwebber](https://discuss.python.org/u/jamestwebber)\
**Post date:** [October 18, 2023, 5:50pm UTC](https://discuss.python.org/t/pep-712-adding-a-converter-parameter-to-dataclasses-field/26126/49 "2023-10-18T17:50:37Z")

</div>

Although this trip-up can happen with any class/object that converts inputs during initialization. e.g. a contrived example:

```auto
match int('0'):
    case int(real='0'):
        print("I'm zero")
    case int(real=0):
        print("No, I'M zero")
    case _:
        print("another int")

```

It isn’t ideal but it’s also not really new behavior, but maybe this PEP would make such bugs more common by making this type of code more tempting to write.

---

<div class="post-metadata">

**Author:** ![fonini](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/fonini/32/20258_2.png) [@fonini](https://discuss.python.org/u/fonini)\
**Post date:** [October 18, 2023, 6:05pm UTC](https://discuss.python.org/t/pep-712-adding-a-converter-parameter-to-dataclasses-field/26126/50 "2023-10-18T18:05:49Z")

</div>

I believe this is a less contrived example that still exhibits the same unwanted behavior:

```auto
match int('0'):
    case int('0'): # will not match
        print("I'm zero")
    case _:
        print("Another int") # will be printed

```

Specifically, the `case int('0')` does not raise an exception: it’s just a pattern that won’t match, because what is _means_ is the check `isinstance(subject, int) and subject == '0'`. I believe that this behavior of the class pattern is a ship that has already sailed.

I think what’s at stake here is: should Python start avoiding ` __init__ ` arguments that don’t correlate with instance attributes _just because_ `match` is now part of the language? I don’t think so, but that’s definitely just my opinion.

---

<div class="post-metadata">

**Author:** ![pf\_moore](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pf_moore/32/35_2.png) [@pf\_moore](https://discuss.python.org/u/pf_moore)\
**Post date:** [October 18, 2023, 6:46pm UTC](https://discuss.python.org/t/pep-712-adding-a-converter-parameter-to-dataclasses-field/26126/51 "2023-10-18T18:46:22Z")

</div>

All anyone is asking for is for it to be discussed in the PEP. It’s a question for the “how do we teach this” section, at a minimum, as that is precisely where non-intuitive behaviour should be called out explicitly.

---

<div class="post-metadata">

**Author:** ![NeilGirdhar](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/neilgirdhar/32/5776_2.png) [@NeilGirdhar](https://discuss.python.org/u/NeilGirdhar)\
**Post date:** [October 18, 2023, 8:20pm UTC](https://discuss.python.org/t/pep-712-adding-a-converter-parameter-to-dataclasses-field/26126/52 "2023-10-18T20:20:30Z")

</div>

In the PEP discussion, it might be worth pointing out that most type checkers should be able to catch this kind of error. For example,

```auto
match int('0'):
    case int('0'): # PyRight says: pattern will never be matched for subject type "int"
        print("I'm zero")   
    case _:
        pass

```

---

<div class="post-metadata">

**Author:** ![thejcannon](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/thejcannon/32/23759_2.png) [@thejcannon](https://discuss.python.org/u/thejcannon)\
**Post date:** [October 18, 2023, 8:59pm UTC](https://discuss.python.org/t/pep-712-adding-a-converter-parameter-to-dataclasses-field/26126/53 "2023-10-18T20:59:56Z")

</div>

Updated: [PEP 712: Update with suggestions/clarifications from discussion by thejcannon · Pull Request #3496 · python/peps · GitHub](https://github.com/python/peps/pull/3496)

---

<div class="post-metadata">

**Author:** ![patrick-kidger](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/patrick-kidger/32/15089_2.png) [@patrick-kidger](https://discuss.python.org/u/patrick-kidger)\
**Post date:** [November 8, 2023, 9:01pm UTC](https://discuss.python.org/t/pep-712-adding-a-converter-parameter-to-dataclasses-field/26126/54 "2023-11-08T21:01:37Z")

</div>

I’ve just come across this PEP.  
Overall I like it! I have two very nitty comments.

First of all, for reference: we use frozen dataclasses basically everywhere in [Equinox](https://github.com/patrick-kidger/equinox), and we already have an extension to `field` that [adds a `converter` argument](https://docs.kidger.site/equinox/api/module/advanced_fields/#equinox.field). I think we’re basically doing the same thing as this PEP in all cases.`*`

- The interaction with ` __post_init__ ` isn’t specified. From experience we’ve found that conversion before ` __post_init__ ` is most useful.
- Whether to have it run inside `cls. __init__ ` or `type(cls). __call__ ` is not discussed. From experience having it run inside `cls. __init__ ` is most useful, as this makes it substantially easier for runtime type checking libraries – we’ve developed [jaxtyping](https://github.com/google/jaxtyping) – to perform their checks.

(`*` Actually, ever-so-technically there is one discrepancy: in custom ` __init__ ` methods on frozen dataclasses, we allow the `self.foo = bar` syntax, and in this case and unlike this PEP, we do perform conversion. We’ve found this an important for usability, and is our sole divergence from standard dataclasses, which normally mandate the use of `object. __setattr__ `. I don’t think this discrepancy really counts here, as this is already somewhere we’ve made a concious choice to deviate from standard dataclasses.)

---

<div class="post-metadata">

**Author:** ![thejcannon](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/thejcannon/32/23759_2.png) [@thejcannon](https://discuss.python.org/u/thejcannon)\
**Post date:** [November 13, 2023, 12:10pm UTC](https://discuss.python.org/t/pep-712-adding-a-converter-parameter-to-dataclasses-field/26126/55 "2023-11-13T12:10:35Z")

</div>

So, specifically the PEP says it runs during attribute assignment (or in \_\_init\_\_ for frozen data classes).

I think it’s safe to assume attribute assignment happens in \_\_init\_\_ for non-frozen dataclasses. If it isn’t, I don’t think the PEP regarding value conversion would be the right place to specifically call out when it happens, since that’s a more generic behavior.

Glad you like the PEP though, and thanks for the suggestions!

---

<div class="post-metadata">

**Author:** ![thejcannon](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/thejcannon/32/23759_2.png) [@thejcannon](https://discuss.python.org/u/thejcannon)\
**Post date:** [November 13, 2023, 12:13pm UTC](https://discuss.python.org/t/pep-712-adding-a-converter-parameter-to-dataclasses-field/26126/56 "2023-11-13T12:13:58Z")

</div>

> [@patrick-kidger](#):
>
> We’ve found this an important for usability, and is our sole divergence from standard dataclasses

In Pantsbuild,we used to have something similar (still using standard dataclasses), but I switched us to using it the way the docs suggest.

I miss the ergonomics of normal attribute assignment. Now you got me pondering a PEP for `frozen_after_init`. I’ll probably run it in a new thread once this PEP is done. 👍

---

<div class="post-metadata">

**Author:** ![gpshead](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/gpshead/32/54_2.png) [@gpshead](https://discuss.python.org/u/gpshead)\
**Post date:** [November 14, 2023, 2:38am UTC](https://discuss.python.org/t/pep-712-adding-a-converter-parameter-to-dataclasses-field/26126/57 "2023-11-14T02:38:48Z")

</div>

Python Steering Council hat: Thanks for the well written PEP and thorough discussion here. We have reviewed and discussed this PEP and we are unfortunately _not_ finding ourselves leaning towards accepting it today. (The question the SC would like to see answered to change our future selves mind is at the end)

Reasoning:

- A `dataclasses.field` converter adds complexity (additional spooky action at a distance and a concept going further than a “just a struct” mental model).
- There are already multiple ways to do this even if they involve more lines (` __init__ `, alternate constructors, etc).
- For users who really want converters rather than the additional lines of code: They can already use third party dataclass-like libraries (presumably [`attrs`](https://pypi.org/project/attrs/)) providing the feature today instead of waiting for CPython 3.13+.

One of our guiding general themes is that there is less reason for every feature to be done in the standard library now than in years past. Virtually all Python applications are built upon many third party packages from our ecosystem today.

**Q:** Is there a compelling reason for dataclass field converters to be _in the standard library_ that we’re just missing?

It is good to see people piping up on this thread who do want the feature. It would be interesting to know how important that is and if you already do, or why you don’t, use something like `attrs` today just to have it.

We’ll keep [the steering-council pep-712 issue](https://github.com/python/steering-council/issues/206) open for a while as a reminder to observe any further discussion (and let the next elected SC make the final decision).

-gps for the 2023 Python Steering Council

---

<div class="post-metadata">

**Author:** ![thejcannon](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/thejcannon/32/23759_2.png) [@thejcannon](https://discuss.python.org/u/thejcannon)\
**Post date:** [November 14, 2023, 4:04pm UTC](https://discuss.python.org/t/pep-712-adding-a-converter-parameter-to-dataclasses-field/26126/58 "2023-11-14T16:04:40Z")

</div>

(Where’s the broken heart emoji reaction?)

Thanks for the response, and especially the explanation. Although it’s a bummer, I think the answer is very fair and understandable.

I hope others _do_ chime on on their specific use-case, especially since the ones that said they’d benefit already have chosen standard library `dataclass`es over `attrs` _despite_ not having this feature (even though that means more hurdles and pain). I’ll chime in on ours (I don’t remember if I have already) in a separate comment.

* * *

> They can already use third party dataclass-like libraries (presumably [`attrs`](https://pypi.org/project/attrs/)) providing the feature today instead of waiting for CPython 3.13+.

I’ll say that since this PEP augments `dataclass_transform`, I don’t think the choice is binary. On the project that motivated this PEP we’d likely define a dataclass transform that simply handles conversion semantics and then forwards to `dataclass`, with the expectation that once we upgrade to Python 3.13 we’ll replace usage of the decorator with `dataclass`. The ability to do so lies in the fact that type-checkers support `typing_extensions` backports 😉 This was intentional on my part (maybe I should’ve made it explicit on the PEP?)

* * *

And speaking of type-checking…

One gap this decision remains open and unsolved is the type-semantics of _any_ dataclass transform providing conversion semantics. This escapade started off as a question of augmenting `dataclass_transform` to support this feature. Since type-checkers, without special-casing, don’t support `attrs`’s conversion semantics when defined as a `dataclass_transform`, it seems to me that if the answer to this PEP is “you should probably just use `attrs`”, that we ought to plug the gap. Otherwise, we didn’t push the needle on _any_ of the problems outlined in the “Motivation” section 🥲 and told the community to use a sub-optimal solution.

So, can I ask for a gut-feel strawman from the (current) Steering Council on, if this PEP is likely to be rejected, a similar PEP scoped just to `dataclass_transform`?

* * *

(Regardless, saying “no” isn’t fun, but can certainly be necessary. I’m just glad the community is the way it is, from the strangers on these threads all the way up to the big bad Steering Council 😉 . Thanks for considering the PEP)

---

<div class="post-metadata">

**Author:** ![thejcannon](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/thejcannon/32/23759_2.png) [@thejcannon](https://discuss.python.org/u/thejcannon)\
**Post date:** [November 14, 2023, 4:12pm UTC](https://discuss.python.org/t/pep-712-adding-a-converter-parameter-to-dataclasses-field/26126/59 "2023-11-14T16:12:41Z")

</div>

> [@gpshead](#):
>
> **Q:** Is there a compelling reason for dataclass field converters to be _in the standard library_ that we’re just missing?

The use-case that motivated this PEP, pantsbuild, uses frozen dataclasses _everywhere_ (current count is ~1200 instances). \[1\].

Although our application already has third-party dependencies, I’ve been arguing **hard** for whittling them away (ideally towards 0) for two reasons:

- We’re a build tool, so security is objectively more important than other libraries/applications. Infect a build tool, and you can infect everything it builds.
- Every time our users want a new version, they are forced to download and install every one of our dependencies. That’s wasteful

So the standard library is our friend.

* * *

1. As a side-note we use frozen dataclasses because we cache these objects to be re-used from multiple threads. PEP 703 is very very exciting to us!

---

<div class="post-metadata">

**Author:** ![thejcannon](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/thejcannon/32/23759_2.png) [@thejcannon](https://discuss.python.org/u/thejcannon)\
**Post date:** [November 14, 2023, 4:20pm UTC](https://discuss.python.org/t/pep-712-adding-a-converter-parameter-to-dataclasses-field/26126/60 "2023-11-14T16:20:46Z")

</div>

Last thing I’ll say (for now, I promise 😅). Any rich type like this in the standard library quickly becomes a vocabulary type. `pathlib.Path` is my favorite example. Instead of APIs declaring they take paths as strings, and having to document _this_ string represents a path, we now have a type that expresses that intent.

`dataclass`es are such a powerful vocabulary type in themselves, _and_ they allow you to further define very straightforward vocabulary types. Win-win. `attrs` has the same semantics, but it being third-party means it isn’t in everyone’s vocabulary. As an API author, if I’m going to define an API, I want the most users to be able to easily understand and easily use, it doesn’t get better than standard library types.

[Previous page](https://discuss.python.org/t/pep-712-adding-a-converter-parameter-to-dataclasses-field/26126.md?page=2)

[Next page](https://discuss.python.org/t/pep-712-adding-a-converter-parameter-to-dataclasses-field/26126.md?page=4)