Fully support using the tp_finalize slot of a PyObject in the limited API

I want to avoid the XY problem here, so I’ll start by saying I’m opening up this discussion to follow up from a Cython issue I opened recently. That issue demonstrates the Cython’s current implementation does not support implementing __del__ on Cython’s cdef classes when compiling under the limited API. From inspecting Cython’s source, the issue appears to be that Cython handles the calling of __del__ by populating the tp_finalize slot of the PyObject with a function that calls __del__, but this slot cannot be safely used in the limited API due to some missing functionality as mentioned here by @da-woods.

I’m not sure whether enabling usage of the tp_finalize slot by adding extra functions to the limited API is the right solution (some of the missing functionality was enumerated in this message), or if a different approach entirely is called for, but based on Cython’s internal approach I’m assuming this slot is the best option here and so figuring out how to use it safely under the limited API is the right next step. Based on @da-woods comments linked above (and his PR here) it seems like the main missing piece is PyObject_CallFinalizerFromDealloc, so perhaps adding that to the limited API would be an acceptable and sufficient solution. Please let me know if there is a different alternative.

@vstinner @encukou question above might be something you can comment on

(@vyasr I promise that nobody will read this sort of question in the “help” category… I’d have posted it in “core development”)

Ah whoops sorry. Should I repost? I don’t think DPO has any way to move a discussion, does it?

Moved to Core Development.