PEP 820 – PySlot: Unified slot system for the C API

PEPs
1

Hello,
I’ve written PEP 820, slightly simplified from last year’s “pre-PEP” thread, a proposal to unify PyType_Slot and PyModuleDef_Slot, and fix some of their shortcomings.
The abstract:

Replace type and module slots with a new, more type-safe structure that allows adding new slots in a more forward-compatible way.

The existing slot structures and related API is soft-deprecated. (That is: it will continue to work without warnings, and it’ll be fully documented and supported, but we plan to not add any new features to it.)

It’s a C API change that isn’t urgent/necessary, but if we want it, it would be good to get in 3.15 so the new PyModExport (PEP 793) can switch to it, and we don’t need a new module export hook.

Another way to look at it: it’s similar in spirit to @steve.dower’s Interfaces API from PEP 809, but for defining things.

What do you think?

5 Likes
Pre-PEP: Unified slot system for the C API
2

I suspect Cython wouldn’t immediately use it (at least for classes… obviously if it goes into PEP 793 then we would for that). Just because it’s mostly targeted at future improvements rather than an immediate new feature. It looks usable though.

I had one thought while reading it. I’m not yet sure if it’s a useful thought. In Steve’s PEP 809 he tried to make interfaces extensible with “namespaces”. Could that be useful here? I could see something like GraalPy wanting some custom type slots (e.g. an “inherits from this Java class” slot[1]). Similarly, at one point PyPy had an extra tp_pypy_flags in their typeobject struct.

  1. I don’t know anything about Java so this is a totally arbitrary example that might have no link to reality ↩︎

1 Like
3

That’s the intention. As the PEP says, users can reuse code they already have written without rewriting/reformatting, and only use the “new” slots if they need any new features.
If this goes to PyModExport, you can still use an array of “old-style” slots to be compatible with older Python versions. You’d need about 4 extra lines:

PyMODEXPORT_FUNC my_modexport(void) {
    static PySlot wrapped_slotswrapped[] = {
        PySlot_STATIC_DATA(Py_mod_slots, the_old_slots_array),
        PySlot_END,
    };
    return wrapped_slots;
}

Not really.
With this proposal, Python won’t “save” the slots array anywhere; it won’t let users retrieve it from the created object.
This is important for forward compatibility: if some of the input needs to be transformed in some way, Python can keep the result, not the original input. For example: “wrapped” old-style slots arrays don’t need to be stored as-is. The Py_mod_gil/Py_mod_multiple_interpreters flags can be forgotten after they’re checked/applied at creation.
Or – very theoretically, if API stability wasn’t a concern – we could turn any Py_nb_invert into an __invert__ method, and remove PyNumberMethods.nb_invert. (Practically this is for any newly added slots: “C function pointer directly on the class object” would be a possible optimization that could be removed at any time as it wouldn’t affect API.)

So, back to the subject of tp_pypy_flags: if you want to save something to retrieve it later, you’ll need to use a different mechanism. Currently, we have module state for modules, and metaclass & PyObject_GetTypeData for types; Steve’s PEP 809 Interfaces API provides “names”/“namespaces” to easily expose the info stored there.