# Adding C API for use in tp\_traverse **URL:** \ **Category:** C API\ **Created:** [December 17, 2025, 1:48pm UTC](https://discuss.python.org/t/adding-c-api-for-use-in-tp-traverse/105331 "2025-12-17T13:48:16Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![encukou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/encukou/32/2461_2.png) [@encukou](https://discuss.python.org/u/encukou)\ **Post date:** [December 17, 2025, 1:48pm UTC](https://discuss.python.org/t/adding-c-api-for-use-in-tp-traverse/105331/1 "2025-12-17T13:48:16Z")
There are newly documented restrictions on `tp_traverse`: [gh-123241: Document restrictions for `tp_traverse` implementations by colesbury · Pull Request #142272 · python/cpython · GitHub](https://github.com/python/cpython/pull/142272) > The traversal function must not have any side effects. It must not modify the reference counts of any Python objects nor create or destroy any Python objects. This primarily means you can’t use functions that return new references, or ones that raise (new) exceptions. (Basically, the kind of functions we’re trying to add nowadays.) But you also can’t use any C API function, unless it guarantees that it has no side effects. I don’t think we guarantee that for anything. You typically don’t need to do much in `tp_traverse`. But, both current PEPs for a free-threading stable ABI (PEP 803 & PEP 809) require some newer APIs for module state and [PEP 697](https://peps.python.org/pep-0697/) layouts (instance structs that don’t include `PyObject`). So, I’d like to add _variants_ for existing functions that are usable in this context – they’d: - never set an exception (on failure, only return `NULL`) - return borrowed references - do not have “internal” side effects (for example, call a “normal C API function and then DECREF the result) For the first two I’d normally name the functions with `_NoError` and `_Borrow` suffixes, but, perhaps that’s included in “safe to use in GC callbacks”. Would `_GCSafe` work? - `PyObject_GetTypeData_GCSafe` (doesn’t set exceptions) - `PyObject_GetItemData_GCSafe` (ditto) - `PyType_GetModuleState_GCSafe` (ditto) - `PyModule_GetState_GCSafe` (ditto) - `PyModule_GetToken_GCSafe` (ditto) - `PyType_GetBaseByToken_GCSafe` (doesn’t set exceptions & returns borrowed ref) - `PyType_GetModule_GCSafe` (ditto) - `PyType_GetModuleByToken_GCSafe` (ditto) Additionally, document `Py_TYPE` & `Py_SIZE` [edit: and `PyObject_VisitManagedDict`] as GC-safe. Does that make sense? ---
**Author:** ![davidhewitt](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/davidhewitt/32/3502_2.png) [@davidhewitt](https://discuss.python.org/u/davidhewitt)\ **Post date:** [December 19, 2025, 2:49pm UTC](https://discuss.python.org/t/adding-c-api-for-use-in-tp-traverse/105331/2 "2025-12-19T14:49:57Z")
I guess that these new variants should be discouraged outside of use during GC because the borrowed references (and lack of exceptions) are generally worse for UX? Should the name be even stronger, maybe a `TpTraverse` suffix or prefix, e.g. `PyObject_GetTypeData_TpTraverse`, or `PyTpTraverse_Object_GetTypeData`? This way it’s more clear (to me) these functions are only for that use case, because they’re so specialized it probably doesn’t matter if their names are long. ---
**Author:** ![da-woods](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/da-woods/32/7507_2.png) [@da-woods](https://discuss.python.org/u/da-woods)\ **Post date:** [December 19, 2025, 4:26pm UTC](https://discuss.python.org/t/adding-c-api-for-use-in-tp-traverse/105331/3 "2025-12-19T16:26:27Z")
At the risk of pointing out the something that’s absolutely obvious and trivial: `PyObject_VisitManagedDict` is hopefully GC-safe (and looks to be documented as such). ---
**Author:** ![encukou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/encukou/32/2461_2.png) [@encukou](https://discuss.python.org/u/encukou)\ **Post date:** [December 19, 2025, 4:27pm UTC](https://discuss.python.org/t/adding-c-api-for-use-in-tp-traverse/105331/4 "2025-12-19T16:27:03Z")
> [@davidhewitt](#): > > I guess that these new variants should be discouraged outside of use during GC because the borrowed references (and lack of exceptions) are generally worse for UX? Good question. As for UX: I expect users to define their own helpers on top of these (most importantly, to cast the result `PyModule_GetState`). Defining two variants is, unfortunately, worse for UX than borrowed refs & lack of exceptions. What I’m more worried about is borrowed references themselves. Without GIL, if another thread changes a class’s MRO, the `PyType_GetBase*` could become invalid. This currently can’t happen: both GC and MRO changes are stop-the-world operations. But that doesn’t seem like something we want to guarantee in the long term. ---
**Author:** ![ngoldbaum](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/ngoldbaum/32/12276_2.png) [@ngoldbaum](https://discuss.python.org/u/ngoldbaum)\ **Post date:** [December 19, 2025, 8:39pm UTC](https://discuss.python.org/t/adding-c-api-for-use-in-tp-traverse/105331/5 "2025-12-19T20:39:34Z")
> [@encukou](#): > > This currently can’t happen: both GC and MRO changes are stop-the-world operations. But that doesn’t seem like something we want to guarantee in the long term. Maybe the stop-the-world part is the key aspect and the part that should be highlighted in the name? I’m not sure if the restrictions you’re thinking of are exactly the same as the ones for code happening during stop-the-world events, but it sounds similar to what @nas described to me when he was doing work to make mutating type objects thread-safe. This is a teeny bit off-topic for this thread, but I want to bring up here that I bet that a future version of Python will expose a C API to trigger a stop-the-world pause. Memray needs it, for one: [memray/src/memray/\_memray/compat.h at 64c0e81806ba01aaa11d8d98e05c479c30982fae · bloomberg/memray · GitHub](https://github.com/bloomberg/memray/blob/64c0e81806ba01aaa11d8d98e05c479c30982fae/src/memray/_memray/compat.h#L104-L121) I suspect other profilers will end up using the private API for stopping the world since it’s needed to do prevent races while observing the Python stack. Better to come up with defined and documented semantics for doing this stuff. ---
**Author:** ![emmatyping](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/emmatyping/32/36516_2.png) [@emmatyping](https://discuss.python.org/u/emmatyping)\ **Post date:** [December 19, 2025, 9:01pm UTC](https://discuss.python.org/t/adding-c-api-for-use-in-tp-traverse/105331/6 "2025-12-19T21:01:50Z")
> [@ngoldbaum](#): > > > [@encukou](#): > > > > This currently can’t happen: both GC and MRO changes are stop-the-world operations. But that doesn’t seem like something we want to guarantee in the long term. > > Maybe the stop-the-world part is the key aspect and the part that should be highlighted in the name? As @encukou says at the end there, we probably don’t want to guarantee stop-the-world in the long-term as it’s really an implementation detail that could change. I’d like to keep the door open for a different design for the GC that may not be stop-the-world. ---
**Author:** ![encukou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/encukou/32/2461_2.png) [@encukou](https://discuss.python.org/u/encukou)\ **Post date:** [December 22, 2025, 3:16pm UTC](https://discuss.python.org/t/adding-c-api-for-use-in-tp-traverse/105331/7 "2025-12-22T15:16:38Z")
In [#142376](https://github.com/python/cpython/issues/142376) we have a request for allowing `PyObject_GetTypeData` _without an attached thread state_. It looks like the use case would apply for all the API listed here, which suggests that it might make sense to include that in the design. Let’s say that for the new `_GCSafe` \[1\] API: CPython **guarantee** that the APIs are strictly read-only; they can’t change refcounts or create new objects. This makes them safe to use in `tp_traverse`. CPython **requires** callers to ensure that the data the API “touches” isn’t concurrently modified. That data being: - the type of any `PyObject*` argument - the MRO of `PyTypeObject*` argument - for completeness: type layout data – flags, `basicsize`, `itemsize`, and similar. (Ideally we’d tell people to _always_ treat those as immutable, but I don’t think we currently do…) And we can let users ensure that by not changing it themselves and: - Call the functions from `tp_traverse`, or - Have an attached thread state (which locks CPython into changing type/bases being a stop-the-world operation), or - (to address [#142376](https://github.com/python/cpython/issues/142376):‍) Use some new API to acquire a “reader lock on types & memory layouts” – instead of an attached thread state, you’d get a lock that prevents stop-the-world operations \[2\], but while you hold it, you’re only allowed to use the “`_GCSafe`” API. (I’m assuming that this could be significantly cheaper than attaching a thread state.) We also need to outlaw directly setting `ob_type`, `tp_bases`, `tp_basicsize` etc., but I assume that’s already part of making an extension free-threading-compatible. Does that sound like a reasonable direction? * * * 1. it needs a better name; let me stick with this for now 2. Or only type/MRO reassignments? ---
**Author:** ![colesbury](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/colesbury/32/28031_2.png) [@colesbury](https://discuss.python.org/u/colesbury)\ **Post date:** [December 22, 2025, 3:47pm UTC](https://discuss.python.org/t/adding-c-api-for-use-in-tp-traverse/105331/8 "2025-12-22T15:47:15Z")
I’m not convinced these are worth the complexity of adding new functions, but I don’t feel super strongly. I guess the idea is that they’re not safe if they are called incorrectly and set an exception, but: 1. Just don’t call them incorrectly. These exceptions are all of the “programmer error” type, not runtime error. 2. Pretty much nobody checks the exception anyways. They’re useless. We’ve made our lives more difficult by making these simple functions capable of failing. I think the idea is that validating arguments and raising a Python error is somehow nicer for C API users than a C assertion or fatal error, but that totally ignores how people actually use these APIs. ---
**Author:** ![encukou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/encukou/32/2461_2.png) [@encukou](https://discuss.python.org/u/encukou)\ **Post date:** [December 22, 2025, 4:24pm UTC](https://discuss.python.org/t/adding-c-api-for-use-in-tp-traverse/105331/9 "2025-12-22T16:24:42Z")
Fair, I partially agree. There was a long discussion on fallible APIs; I don’t think we need to rehash it at _this_ point. If fallibility was the _only_ issue, I think we could make things work with the current API. But the main issue is the borrowed references. In “normal” code, another thread can invalidate borrowed refs; in `tp_traverse` we can’t create strong refs. ---
**Author:** ![da-woods](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/da-woods/32/7507_2.png) [@da-woods](https://discuss.python.org/u/da-woods)\ **Post date:** [December 22, 2025, 9:16pm UTC](https://discuss.python.org/t/adding-c-api-for-use-in-tp-traverse/105331/10 "2025-12-22T21:16:34Z")
As the requester of the thread-state-less `PyObject_GetTypeData` this is going to sound a bit hypocritical. However: What I’m a bit worried about is that the rules are very tied to “these things are done with stop the world” (which @emmatyping says, does feel like a real implementation detail). The opaque type data is mainly useful for the Limited API, where ideally we want to stay away from implementation details. * * * Personally, I could easily live with restricting access to immutable types only. For those, the idea that the type and layout shouldn’t change really isn’t an implementation detail. ---
**Author:** ![colesbury](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/colesbury/32/28031_2.png) [@colesbury](https://discuss.python.org/u/colesbury)\ **Post date:** [December 22, 2025, 9:35pm UTC](https://discuss.python.org/t/adding-c-api-for-use-in-tp-traverse/105331/11 "2025-12-22T21:35:30Z")
> [@encukou](#): > > But the main issue is the borrowed references. I think that’s just `PyType_GetBaseByToken` and `PyType_GetModuleByToken`. It seems pretty rare to need to call them from a `tp_traverse` handler. If really necessary, I think you can work around that in extensions by calling other APIs. If there were a few extensions that would make use of it, I’d say to add those two with `_Borrow` or `_GCSafe` suffixes, but I’m not sure it’s worth it right now. ---
**Author:** ![philthompson](https://avatars.discourse-cdn.com/v4/letter/p/ed8c4c/32.png) [@philthompson](https://discuss.python.org/u/philthompson)\ **Post date:** [December 23, 2025, 10:45am UTC](https://discuss.python.org/t/adding-c-api-for-use-in-tp-traverse/105331/12 "2025-12-23T10:45:23Z")
I strongly agree with this. The only call mentioned that affects me is `PyModule_GetState()` which can only fail if the argument is invalid. The fact that I “should” be checking the result is a pain. I would much rather have something that causes an abort as it’s a sign of a serious programming error on my part. For those cases where code genuinely doesn’t know if there should be state or not then add `PyModule_HasState()`. ---
**Author:** ![davidhewitt](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/davidhewitt/32/3502_2.png) [@davidhewitt](https://discuss.python.org/u/davidhewitt)\ **Post date:** [December 27, 2025, 10:55pm UTC](https://discuss.python.org/t/adding-c-api-for-use-in-tp-traverse/105331/13 "2025-12-27T22:55:29Z")
It sounds to me that for most of these with the error handling an `Unchecked` variant might solve the issue without needing a new convention. ---
**Author:** ![encukou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/encukou/32/2461_2.png) [@encukou](https://discuss.python.org/u/encukou)\ **Post date:** [January 6, 2026, 1:05pm UTC](https://discuss.python.org/t/adding-c-api-for-use-in-tp-traverse/105331/14 "2026-01-06T13:05:10Z")
> [@colesbury](#): > > I think that’s just `PyType_GetBaseByToken` and `PyType_GetModuleByToken`. It seems pretty rare to need to call them from a `tp_traverse` handler. If really necessary, I think you can work around that in extensions by calling other APIs. I don’t think you can, if targetting ABIs with different `sizeof(PyObject)` (i.e. both free-threaded & GIL-ful). For a concrete example, see [here](https://github.com/python/cpython/pull/142827/files#r2664758320). I’d be interested in how you’d write design this. > [@davidhewitt](#): > > It sounds to me that for most of these with the error handling an `Unchecked` variant might solve the issue without needing a new convention. IMO, “GC safe” _needs_ to be a new concept/convention. It is a much stricter limitation than “cannot fail” or “unchecked”. For example, “GCSafe” functions can’t call “normal” functions and clear any exceptions. And they need to be _guaranteed_ to never poke at refcounts, even in future CPython versions. If nothing else, a `_GCSafe` suffix is a good way to remind future core devs of this limitation. > [@colesbury](#): > > I’m not convinced these are worth the complexity of adding new functions, but I don’t feel super strongly. Thanks for your input! > [@colesbury](#): > > We’ve made our lives more difficult by making these simple functions capable of failing. I think the idea is that validating arguments and raising a Python error is somehow nicer for C API users than a C assertion or fatal error, but that totally ignores how people actually use these APIs. Agree that our lives are more difficult, but, adding new API is a solution :‍) The main general argument for making long-term-supported functions capable of failing is to enable them to emit runtime deprecation warnings (which `-Werror` turns into exceptions). The C API WG policy is\*\[1\] that new functions should be regular/predictable (be fallible, return new references, etc.), and we add a special variant (`Unchecked`, `Borrow`, etc.) if needed for performance, ergonomics, etc. In my (obviously biased) opinion, this is a _great_ way to design the API. While it does makes the individual functions more cumbersome, the API as a whole will be much more usable. (Especially if/when we can optionally [hide the irregularities added in the past](https://peps.python.org/pep-0743/).) * * * 1. I’m only a single member of the WG, so I shouldn’t really speak for all of it. I’m the one that’s [writing the rules down](https://github.com/capi-workgroup/api-evolution/blob/main/guidelines.rst) though. ---
**Author:** ![encukou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/encukou/32/2461_2.png) [@encukou](https://discuss.python.org/u/encukou)\ **Post date:** [January 23, 2026, 5:22pm UTC](https://discuss.python.org/t/adding-c-api-for-use-in-tp-traverse/105331/15 "2026-01-23T17:22:30Z")
I opened a C API WG issue: [GCSafe functions · Issue #95 · capi-workgroup/decisions · GitHub](https://github.com/capi-workgroup/decisions/issues/95) ---
**Author:** ![steve.dower](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/steve.dower/32/56_2.png) [@steve.dower](https://discuss.python.org/u/steve.dower)\ **Post date:** [January 27, 2026, 2:53pm UTC](https://discuss.python.org/t/adding-c-api-for-use-in-tp-traverse/105331/16 "2026-01-27T14:53:24Z")
> [@encukou](#): > > You typically don’t need to do much in `tp_traverse`. What _do_ you need to do with these? I’ve written my fair share of `tp_traverse` and `tp_clear` functions and have never needed to find anything that isn’t directly available from the C struct of my own type. If anyone is using these functions today, aren’t they incref-ing objects and potentially upsetting the cycle detector already? > [@encukou](#): > > Let’s say that for the new `_GCSafe` API Agreed it needs a better name, “unchecked” or “unsafe” feel like better starting points to me. We don’t _want_ users to use these unless they’re sure, so a negative connotation will help with that, and it sounds like with a stop-the-world lock in place they’re also safe (enough) to use, so it’s not just about GC. ---
**Author:** ![encukou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/encukou/32/2461_2.png) [@encukou](https://discuss.python.org/u/encukou)\ **Post date:** [January 27, 2026, 3:21pm UTC](https://discuss.python.org/t/adding-c-api-for-use-in-tp-traverse/105331/17 "2026-01-27T15:21:58Z")
> [@steve.dower](#): > > What _do_ you need to do with these? I’ve written my fair share of `tp_traverse` and `tp_clear` functions and have never needed to find anything that isn’t directly available from the C struct of my own type. You need to get the struct for your type. Traditionally this struct includes `PyObject` (or the \*base class struct in general), but if that struct is opaque (i.e. has an unknown size), you need `PyObject_GetTypeData` to get at the data that belongs to _your_ class. And the rest to get the type itself, if it’s not in a global pointer. > [@steve.dower](#): > > If anyone is using these functions today, aren’t they incref-ing objects and potentially upsetting the cycle detector already? Yes, and while CPython has been lenient in practice, it breaks [free-threading GC](https://github.com/python/cpython/issues/123241) and [Cinder](https://github.com/python/cpython/pull/142232). ---
**Author:** ![encukou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/encukou/32/2461_2.png) [@encukou](https://discuss.python.org/u/encukou)\ **Post date:** [January 30, 2026, 1:41pm UTC](https://discuss.python.org/t/adding-c-api-for-use-in-tp-traverse/105331/18 "2026-01-30T13:41:05Z")
> [@steve.dower](#): > > Agreed it needs a better name, “unchecked” or “unsafe” feel like better starting points to me. We don’t _want_ users to use these unless they’re sure, so a negative connotation will help with that, and it sounds like with a stop-the-world lock in place they’re also safe (enough) to use, so it’s not just about GC. I can’t seem to think of a better name :‍/ Just `_GC`? ---
**Author:** ![steve.dower](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/steve.dower/32/56_2.png) [@steve.dower](https://discuss.python.org/u/steve.dower)\ **Post date:** [January 30, 2026, 1:45pm UTC](https://discuss.python.org/t/adding-c-api-for-use-in-tp-traverse/105331/19 "2026-01-30T13:45:35Z")
Maybe something like `_AssumeLocked` or `_AssumeWorldStopped`? ---
**Author:** ![encukou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/encukou/32/2461_2.png) [@encukou](https://discuss.python.org/u/encukou)\ **Post date:** [January 30, 2026, 6:03pm UTC](https://discuss.python.org/t/adding-c-api-for-use-in-tp-traverse/105331/20 "2026-01-30T18:03:05Z")
- `_AssumeLocked` is not clear about what is being locked. - `_AssumeWorldStopped`: This references an implementation detail - As [Emma mentioned above](https://discuss.python.org/t/adding-c-api-for-use-in-tp-traverse/105331/6) we might want to _avoid_ stopping the world for the GC in the future. - Changing an object’s type is currently another stop-the-world event, and it must change refcouns, so it doesn’t have the same requirements as tp\_traverse. “Strictly for use in a GC callback”? Or with the Memray use case in mind – is there a good word for “side effect free”? “Pure”, but with negative connotations? > [@steve.dower](#): > > We don’t _want_ users to use these unless they’re sure Yes, but :‍) With these functions, users don’t get error handling, and possible future runtime warnings (e.g. deprecation). That’s pretty much it. And that’s pretty much exactly what [some users want anyway](https://discuss.python.org/t/adding-c-api-for-use-in-tp-traverse/105331/8)! I think that slapping on a suffix _should_ be enough to get people to notice something is different and go read the docs\[1\]. We don’t necessarily need the negative connotations, * * * 1. At least the ones doing careful reviews, which is the target here. [Next page](https://discuss.python.org/t/adding-c-api-for-use-in-tp-traverse/105331.md?page=2)