# C API: How much private is the private \_Py\_IDENTIFIER() API? **URL:** \ **Category:** C API\ **Created:** [July 7, 2023, 4:26pm UTC](https://discuss.python.org/t/c-api-how-much-private-is-the-private-py-identifier-api/29190 "2023-07-07T16:26:36Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\ **Post date:** [July 7, 2023, 4:26pm UTC](https://discuss.python.org/t/c-api-how-much-private-is-the-private-py-identifier-api/29190/1 "2023-07-07T16:26:36Z")
Hi, I removed 181 privates functions in Python 3.13 C API. A code search on PyPI top 5,000 projects shows that the **private** `_Py_Identifier` API is widely used. Examples of lines using removed functions: - **\_PyObject\_CallMethodIdObjArgs** (35) - **\_PyUnicode\_FromId** (34) - **\_PyObject\_CallMethodId** (14) - **\_PyUnicode\_EqualToASCIIId** (2) I didn’t look for `_Py_IDENTIFIER()` usage. Python 3.12 made strings “immortal” and in CPython code base, `_Py_IDENTIFIER()` was replaced with statically initialized objects: [GH-90699](https://github.com/python/cpython/issues/90699). What is the status of the `_Py_Identifier` API in Python 3.13? Should we help C extensions to move away this API? Or do we want to make some of these APIs public? Currently, PyPy doesn’t support the private `_Py_Identifier` C API. Python 3.12 provides the following provide API: - `_Py_static_string()` - `_Py_static_string_init()` - `_Py_IDENTIFIER()` - `_PyDict_ContainsId()` - `_PyDict_DelItemId()` - `_PyDict_SetItemId()` - `_PyEval_GetBuiltinId()` - `_PyImport_GetModuleId()` - `_PyObject_CallMethodId()` - `_PyObject_GetAttrId()` - `_PyObject_LookupAttrId()` - `_PyObject_LookupSpecialId()` - `_PyObject_SetAttrId()` - `_PyType_LookupId()` - `_PyUnicode_EqualToASCIIId()` - `_PyUnicode_FromId()` Victor ---
**Author:** ![guido](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/guido/32/21_2.png) [@guido](https://discuss.python.org/u/guido)\ **Post date:** [July 7, 2023, 7:03pm UTC](https://discuss.python.org/t/c-api-how-much-private-is-the-private-py-identifier-api/29190/2 "2023-07-07T19:03:09Z")
This seems a question for @eric.snow (who is still on vacation for another week IIRC). ---
**Author:** ![eric.snow](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/eric.snow/32/31_2.png) [@eric.snow](https://discuss.python.org/u/eric.snow)\ **Post date:** [July 14, 2023, 10:27am UTC](https://discuss.python.org/t/c-api-how-much-private-is-the-private-py-identifier-api/29190/3 "2023-07-14T10:27:51Z")
We should definitely work toward getting rid of `_Py_IDENTIFIER`, etc. The fact that the community uses it, which I had not anticipated, is the only reason I didn’t remove it at the time. That said, clearly there is a demand for an efficient API along those lines. If there isn’t already an issue open to pursue that then we should open one. Especially, we still need to reach out to users of the private API to find the best way forward. (My availability is still limited so I won’t be able to follow up right away.) ---
**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\ **Post date:** [July 17, 2023, 2:47pm UTC](https://discuss.python.org/t/c-api-how-much-private-is-the-private-py-identifier-api/29190/4 "2023-07-17T14:47:12Z")
> [@eric.snow](#): > > We should definitely work toward getting rid of `_Py_IDENTIFIER`, etc. The fact that the community uses it, which I had not anticipated, is the only reason I didn’t remove it at the time. Ok, but I did remove them in Python 3.13 C API since they were marked as private, and I removed as many private functions as possible. So what should we do for Python 3.13? Force people to use slower “allocate string, use the string, delete the string” pattern? Suggest a replacement: which one? Expose the API to make it public (and so endorse and support it)? Revert my change and go back to the status quo (not public, you should not use it, but oops, people actually use it): I really want to avoid this situation. ---
**Author:** ![eric.snow](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/eric.snow/32/31_2.png) [@eric.snow](https://discuss.python.org/u/eric.snow)\ **Post date:** [July 18, 2023, 2:21pm UTC](https://discuss.python.org/t/c-api-how-much-private-is-the-private-py-identifier-api/29190/5 "2023-07-18T14:21:30Z")
> [@vstinner](#): > > So what should we do for Python 3.13? Yeah, we definitely kicked this can down the road when we dropped internal usage of `_Py_IDENTIFIER()` last year. At the time, I mentioned how we could [open a new issue](https://github.com/python/cpython/issues/90699#issuecomment-1309055139) to deal with public usage of the API but I don’t think that ever happened. When we started working on dropping `_Py_IDENTIFIER()`, [you pointed out](https://mail.python.org/archives/list/python-dev@python.org/message/7RMLIJHUWVBZFV747TFEHOE6LNBVQSMM/) the projects you found that are using the API, and [I took a closer look](https://github.com/python/cpython/issues/90699#issuecomment-1093943451). My assessment at the time was: “All of them should be trivial to drop \_Py\_IDENTIFIER() without any real performance impact or mess.” Hence, we might just need to work with the handful of projects to get them off that API. That said, a public API may still be worth adding. For example, it sounds like [numpy would use it](https://mail.python.org/archives/list/python-dev@python.org/message/PBBH2RXKZO2HYWYLNZU5VJ6B7BJSABKZ/). Either way, it would probably be worth opening a new issue. FWIW, the [python-dev thread from Feb 2022](https://mail.python.org/archives/list/python-dev@python.org/thread/DNMZAMB4M6RVR76RDZMUK2WRLI6KAAYS/) has a lot of great additional ideas and commentary. > [@vstinner](#): > > Force people to use slower “allocate string, use the string, delete the string” pattern? This works fine for performance insensitive cases. That might cover all the public usage, but we’d need to make sure. However, there might be broader needs that would be make it worth doing more. > [@vstinner](#): > > Suggest a replacement: which one? We’d need to figure that out, based on feedback from the existing `_Py_IDENTIFIER()` users and other projects like numpy. Again, there’s some good discussion about it in [the python-dev thread](https://mail.python.org/archives/list/python-dev@python.org/thread/DNMZAMB4M6RVR76RDZMUK2WRLI6KAAYS/), including your suggestion to [perhaps make argument clinic public](https://mail.python.org/archives/list/python-dev@python.org/message/QNP7RDER74XU3DNHJP5EAZY6G6N4VYE7/). > [@vstinner](#): > > Expose the API to make it public (and so endorse and support it)? I’m pretty sure the existing API isn’t quite what we’d want users using long-term. > [@vstinner](#): > > Revert my change and go back to the status quo (not public, you should not use it, but oops, people actually use it): I really want to avoid this situation. Yeah, I’d like to avoid this too. We may want to revert until we have a better plan though. ---
**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\ **Post date:** [August 29, 2023, 12:06am UTC](https://discuss.python.org/t/c-api-how-much-private-is-the-private-py-identifier-api/29190/6 "2023-08-29T00:06:28Z")
> [@eric.snow](#): > > including your suggestion to [perhaps make argument clinic public](https://mail.python.org/archives/list/python-dev@python.org/message/QNP7RDER74XU3DNHJP5EAZY6G6N4VYE7/). Update in this area: **Argument Clinic now supports the limited C API** , see: - Issue: [https://github.com/python/cpython/issues/108494](https://github.com/python/cpython/issues/108494) - Issue: [https://github.com/python/cpython/issues/85283](https://github.com/python/cpython/issues/85283) - Discussion: [Use the limited C API for some of our stdlib C extensions](https://discuss.python.org/t/use-the-limited-c-api-for-some-of-our-stdlib-c-extensions/32465) The C code generated by Argument Clinic is currently less efficient than the code targeting the internal C API (faster APIs), but there is room for improvement (since we know how to make it faster with the internal C API). ---
**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\ **Post date:** [August 29, 2023, 12:28am UTC](https://discuss.python.org/t/c-api-how-much-private-is-the-private-py-identifier-api/29190/7 "2023-08-29T00:28:48Z")
I wrote [gh-106320: Remove private \_Py\_Identifier API by vstinner · Pull Request #108593 · python/cpython · GitHub](https://github.com/python/cpython/pull/108593) to remove the private `_Py_identifier` from the public C API (move it to the internal C API). This change is part of my larger plan on removing private API: [C API: My plan to clarify private vs public functions in Python 3.13](https://discuss.python.org/t/c-api-my-plan-to-clarify-private-vs-public-functions-in-python-3-13/30131). If the number of affected projects is low, the solution is to guide them towards static/global variables to cache strings (call `PyUnicode_FromUnicode()` only once), and then use this string. Or just use the bytes string functions, variants with the `String` suffix like `PyDict_GetItemStringRef()`. If there are many affected projects, another option is to expose the bare minimum to the public C API: - `Py_IDENTIFIER(name)` macro - `PyUnicode_FromId(&PyId_name)` function - `Py_Identifier` structure with its members (needed by `Py_IDENTIFIER()` macro) - and maybe also the `Py_static_string(name, "...")` macro for strings which are not valid C identifiers. I dislike exposing the `Py_Identifier` structure ☹ Its members already changed when I added support for sub-interpreters, so it’s not implementation-agnostic ☹ As I wrote before, if possible, I would prefer to **not** add this API to the **public** C API. Maybe a brand new API should be design? Or PyUnicode\_FromString() should be optimized? Would it be possible to design a LRU cache on PyUnicode\_FromString() which would be more efficient… than not using a cache? Computing a cache key requires to hash the byte string which is not free in terms of performance. ---
**Author:** ![encukou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/encukou/32/2461_2.png) [@encukou](https://discuss.python.org/u/encukou)\ **Post date:** [August 31, 2023, 2:17pm UTC](https://discuss.python.org/t/c-api-how-much-private-is-the-private-py-identifier-api/29190/8 "2023-08-31T14:17:27Z")
> [@vstinner](#): > > If the number of affected projects is low, the solution is to guide them towards static/global variables to cache strings (call `PyUnicode_FromUnicode()` only once) That would be incompatible with multiple interpreters. Could you guide them to use module state instead of C statics/globals? Maybe we do need an API to atomically create/get an immortal string that’s interned across all interpreters. ---
**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\ **Post date:** [December 16, 2025, 11:53am UTC](https://discuss.python.org/t/c-api-how-much-private-is-the-private-py-identifier-api/29190/9 "2025-12-16T11:53:08Z")
**Update:** `_PyUnicode_FromId()`, `_PyObject_GetAttrId()` and `_PyObject_CallMethodId()` functions (the last 3 functions using `_Py_Identifier`) are now deprecated in Python 3.15 and will be removed in Python 3.20. Instead, use `PyUnicode_InternFromString()` and cache the result in the module state, then call `PyObject_CallMethod()` or `PyObject_GetAttr()`.