I wish this actually allowed to be able to use PyHeapTypeObject
as a casting type when calling PyType_FromSpec
within C code using the limited API for 3.14.0a3. It would also allow one to be able to also access ht_type
on it as well and also allow one to store the PyHeapTypeObject
pointers using extern PyHeapTypeObject *TypeObj;
to be used within their C code even if it spans multiple files. This is practically the only thing on 1 of my C extensions keeping me from using the limited API targeting 3.14 specifically.
Could you explain this in terms of what you need to achieve, as opposed to how it can technically be currently done?
For example, if there was a single (or set of) API to do it, what would the Py_...
functions be called?
We are more than willing to add new APIs to enable people. But breaking the abstraction is literally the definition of “use the non-limited API”, so if that’s what you need, then you should be using the normal API.
Type objects are Python objects, so:
- to get a
PyObject*
from aPyTypeObject*
, cast. - to get a
PyTypeObject*
from aPyObject*
, check that it’s a type, and then cast. (You can skip the check if know it’s a type, for example if you got it fromPyType_FromSpec
.)
PyHeapTypeObject
works the same way. To get the PyTypeObject*
, cast. You don’t need to go through ht_type
.
If you have control over, the extern
pointers, I’d recommend using PyObject *
or PyTypeObject*
, rather than PyHeapTypeObject
.
What is the best practice for checking that the object is actually a heap type? PyType_Check
is not enough, and there is no convenient PyHeapType_Check
function.
PyType_HasFeature(type, Py_TPFLAGS_HEAPTYPE)
But it should be relatively rare to need that in extensions. (Or anywhere outside CPython’s typeobject.c
, for that matter.)
Yet this is what @AraHaan is asking for. Is the answer “Yes, you can safely cast to PyHeapTypeObject
after calling PyType_FromSpec
using the limited API in 3.13.0a3”?
PyHeapTypeObject
isn’t in the Limited API so I don’t see how you can cast to it at all.
No, you cannot do that. But if I understood correctly, @AraHaan ultimately wants:
- access to
ht_type
(cast toPyTypeObject*
instead) - store in an
extern
pointer (that pointer should bePyObject*
orPyTypeObject*
instead)
@AraHaan, is that right?
To clarify why avoiding extern PyHeapTypeObject *
is a good idea: The PyHeapTypeObject
struct itself is not very interesting, and certainly not a candidate for stable ABI. Aside from ht_type
, it has:
- slot tables (whose contents are, with a few exceptions, accessible in the limited API using
PyType_GetSlot
; and in the non-limited API they’re all exposed viaPyTypeObject.tp_as_*
pointers) - several members with dedicated accessors (like
PyType_GetName
forht_name
) - some details that aren’t useful in extensions and probably should have been fully private (like
ht_cached_keys
)
And no API functions take PyHeapTypeObject*
, so you’d need to cast to PyObject*
or PyTypeObject*
for any use other than accessing the members.
Dear moderators: IMO, the conversation starting with AraHaan’s post should be moved to a new topic in Ideas, titled e.g. “Exposing PyHeapTypeObject
in limited API.”
That is correct. However I do also wish that there could be a new type flag that can be used within a specific PyType_Spec
that can opt-out of the part of the code in PyType_FromSpec()
that creates a type as a heap type (for those who want to create a traditional type that is a “static-like” type that cannot directly be created in the limited API currently (without a spec) and also for when they want the created type to be immortal (after creation) until when said module is unloaded (I seen cleanup like this done from within the old audioop module). I also see that some parts of the cpython codebase also suffers from being unable to use spec’s and such a change could allow them to migrate to using specs, while also not losing the benefits of not being heap types in certain edge cases.
And looking back now, I probably should ideally be using multi-phase and storing all of this information within a structure for which is used when I get the module’s state so it can support multiple interpreters (and could also support free-threaded builds for free).