Why modifying tp_dict is unsafe?

Python Help
,
1

Dear community:

I was trying to find ways to monkey-patch built-in methods by modifying tp_dict. However, “Type Objects — Python 3.12.2 documentation” says “It is not safe to use PyDict_SetItem() on or otherwise modify tp_dict with the dictionary C-API”.

Are there any hints/pointers to why this is the case?

For example, why is this unsafe? When is this unsafe?

2

I believe it is because there are other (fully internal and intentionally undocumented) data structures that need to be kept in sync with the type’s __dict__, and modifying tp_dict directly defeats the synchronization. Instead, you should use PyObject_SetAttr() on the class – this will invoke any synchronization necessary.

4 Likes
4

Will PyType_Modified and PyType_ClearCache work? Some built-in methods seem to be readonly and does not allow setattr in python scripts. Will PyObject_SetAttr still work in this case?

5

If you can’t set the attribute from Python you can’t set it from C using PyObject_SetAttr.

2 Likes
6

Got it. Thanks.

7

Note that you can clear the immutable flag from tp_flags and then call PyObject_SetAttr and generally this works, even for builtins, even for magic methods. But ofcourse, the behavior of this is unsupported, and it might break in various ways, for various reasons, from micropatch to micropatch, and it probably requires you to do unsafe casts somewhere along the line. (this is for example what the forbiddenfruit library does Actually, that library does more work than required, I guess I never looked at their source code before.)