Ok, I’ve made some updates. Updated branch here.
Changes:
- The descriptor now locks around code generation and class mutation to prevent generating the method and updating the class multiple times
- There are now a number of tests around both laziness and the locking behaviour
- These currently live in a separate
test_dataclasses/test_laziness.pysubmodule
- These currently live in a separate
- Descriptor instances are now callable and calling them will also generate and replace the method
- This is so that getting the ‘function’ from the class
__dict__and calling it still works. I don’t think this is a good pattern to use, but technically it works with existing dataclasses so there’s probably code somewhere that does this.
- This is so that getting the ‘function’ from the class
- Methods generated by dataclasses gain a
__generated_by_dataclasses__attribute that is set toTrue.- A new
is_dataclass_methodfunction can be used to detect both this and if the object is an_AutoMethodinstance. (This may not be the final name of the function). - This is now used for the
pprintspecial casing of dataclasses__repr__
- A new
With regard to comments on pre-compiling or using some alternate syntax that have popped up here or in other conversations I’ve had, it’s not actually something you can do easily while matching the complete behaviour of dataclasses.
The simplest example of this is if a dataclass inherits from a base in another file. The compiled .pyc files only depend on changes to individual files but the methods dataclasses would generate for the subclass depend on both files. Without significant changes to how Python handles .pyc compilation you can’t make something that compiles methods to .pyc files and supports this feature.
One of the side-effects of the current laziness design is that generating methods is now handled in one consistent way. Every method generating function has this signature:
def generator(name: str, cls: type) -> types.FunctionType: ...
This was done partly to keep the _AutoMethod logic simple but it also made it easy to replace the templating logic with faster method generation where possible.
For example this is the new __setattr__ generator, which doesn’t use template execution and this is the __repr__ generator which does.
As mentioned before, the purpose in making dataclasses itself faster rather than promoting my own alternate implementation[1] is that along with being common in external libraries, dataclasses is the tool the standard library has and can use internally - so improving it improves the performance of anything that uses _colorize for instance.
I’ll also note that the “exec magic” is essentially a trade off. It’s slower to create the method originally due to the exec overhead, but the resulting methods are faster than what you get by trying to write a function that covers all dataclasses[2].