If dataclass field converters are accepted, how would we annotate custom field specifiers?

Ideas
1

I want illustrate a problem with type checking custom field specifiers that generate their own converters.

In my Efax library, I have classes like this one. And it has fields like

    negative_rate: JaxRealArray = distribution_parameter(ScalarSupport())
    shape_minus_one: JaxRealArray = distribution_parameter(ScalarSupport())

where distribution_parameter is a PEP 681 field specifier. The JaxRealArray type is just a jax.Array, which is similar to a Numpy array. By having such a narrow type, I can be sure that the parameters have shape and dtype.

However, it would be very convenient for calling code to be able to pass in Python floats and integers. So, this is the perfect use case for a converter, but it’s going to be a bit wordy:

type RealNumeric = JaxRealArray | NumpyRealArray | float | int
def narrow_to_float(x: RealNumeric, /):
    return jnp.asarray(x)

# and then
    negative_rate: JaxRealArray = distribution_parameter(ScalarSupport(), converter=narrow_to_float)
    shape_minus_one: JaxRealArray = distribution_parameter(ScalarSupport(), converter=narrow_to_float)

I’d like it to avoid having this converter specified this way. I’d like it to be able to see the support type. If the support is a symmetric, it could check that a symmetric matrix is being passed in. If the support is a complex type, it could ensure that it produces a complex array. If the support is real type, it could ensure that the array passed in is a real array.

There’s no problem with distribution_parameter generating that kind of converter. The problem is that if the converter isn’t specified where the field specifier is used, then type checkers won’t know what types are allowed to be passed in for that parameter. The simplest solution I can think of would add a whole new FieldSpecifier base class to dataclasses (which is probably well out of scope for this PEP).

Anyway, I would definitely use converters, and I hope that they are accepted. There does seem to be some practical issues with type checking custom field specifiers that generate their own converters (since the type checker can’t see them). Although I think that’s a problem that can be solved in a separate PEP.

2

It’s not clear to me what ScalarSupport() does? Don’t arguments to the field specifier have to be keyword-only?

EDIT: okay, re-reading PEP 681 again, it says that the field specifier may have positional arguments that aren’t relevant to typing. Then can’t you solve it this way:

type RealNumeric = JaxRealArray | NumpyRealArray | float | int
@dataclass
class NarrowToFloat:
    support: SupportBase
    def __call__(self, x: RealNumeric, /):
        # do something with `self.support`
        return jnp.asarray(x)

# and then
    negative_rate: JaxRealArray = distribution_parameter(converter=NarrowToFloat(ScalarSupport()))
    shape_minus_one: JaxRealArray = distribution_parameter(converter=NarrowToFloat(ScalarSupport()))
1 Like