# Limited C API: implement Py\_INCREF() and Py\_DECREF() as function calls

**URL:** <https://discuss.python.org/t/limited-c-api-implement-py-incref-and-py-decref-as-function-calls/27592>\
**Category:** C API\
**Created:** [June 9, 2023, 10:26am UTC](https://discuss.python.org/t/limited-c-api-implement-py-incref-and-py-decref-as-function-calls/27592 "2023-06-09T10:26:16Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\
**Post date:** [June 9, 2023, 10:26am UTC](https://discuss.python.org/t/limited-c-api-implement-py-incref-and-py-decref-as-function-calls/27592/1 "2023-06-09T10:26:16Z")

</div>

Hi,

I modified Python 3.8 to make the `PyObject` structure ABI compatible between Python release and debug builds. You can just use python-debug (debug build) as a drop-in replacement of python (release build), it “just works” (on most platforms, ex: Linux).

I fixed Python 3.10 to make add support of the limited C API to Python debug build (Py\_REF\_DEBUG): Py\_INCREF() and Py\_DECREF() are implemented as opaque function calls in this case. Moreover, PyObject\_INIT() is now implemented with an opaque function call: \_Py\_NewReference() is no longer called directly as the ABI level.

Previously, the C API exposed many implementation details at the ABI level (through inlined functions and macros):

- private `_Py_RefTotal` variable (`Py_REF_DEBUG` macro of debug build)
- private `_Py_NewReference()` and `_Py_ForgetReference()`
- private `_Py_tracemalloc_config` variable
- private `_Py_inc_count()` and `_Py_dec_count()` functions of `COUNT_ALLOC` special build (removed in Python 3.9)

In Python 3.12, with the implementation of immortal objects (PEP 683), Py\_INCREF() and Py\_DECREF() implementation became more complex than `PyObject.ob_refcnt++` and `PyObject.ob_refcnt--`.

Extract of Py\_INCREF() implementation (simplified code):

```auto
struct _object {
    _PyObject_HEAD_EXTRA
    union {
       Py_ssize_t ob_refcnt;
#if SIZEOF_VOID_P > 4
       PY_UINT32_T ob_refcnt_split[2];
#endif
    };
    PyTypeObject *ob_type;
};

static inline int _Py_IsImmortal(PyObject *op)
{
#if SIZEOF_VOID_P > 4
    return _Py_CAST(PY_INT32_T, op->ob_refcnt) < 0;
#else
    return op->ob_refcnt == _Py_IMMORTAL_REFCNT;
#endif
}

static inline void Py_INCREF(PyObject *op)
{
#if SIZEOF_VOID_P > 4
    // Portable saturated add, branching on the carry flag and set low bits
    PY_UINT32_T cur_refcnt = op->ob_refcnt_split[PY_BIG_ENDIAN];
    PY_UINT32_T new_refcnt = cur_refcnt + 1;
    if (new_refcnt == 0) {
        return;
    }
    op->ob_refcnt_split[PY_BIG_ENDIAN] = new_refcnt;
#else
    // Explicitly check immortality against the immortal value
    if (_Py_IsImmortal(op)) {
        return;
    }
    op->ob_refcnt++;
#endif
    _Py_INCREF_STAT_INC();
#ifdef Py_REF_DEBUG
    _Py_INC_REFTOTAL();
#endif
}

```

Link to full implementation:

- [Py\_INCREF](https://github.com/python/cpython/blob/58e4b69f698e6fd0694a58f18679bbe0e7e50e91/Include/object.h#L611)
- [Py\_DECREF](https://github.com/python/cpython/blob/58e4b69f698e6fd0694a58f18679bbe0e7e50e91/Include/object.h#L650)

By the way, the current implementation causes [new C compiler warnings/errors](https://github.com/python/cpython/issues/105059).

While PEP 683 says that [the stable ABI is not affected](https://peps.python.org/pep-0683/#stable-abi) (C extensions built with Python 3.11 continue to work), IMO it’s now time to consider **converting Py\_INCREF() and Py\_DECREF() to opaque function calls in limited C API version 3.12** and newer. I propose: [issue](https://github.com/python/cpython/issues/105387) and [PR](https://github.com/python/cpython/pull/105388).

Guido opened a similar discussion at [capi-workgroup/problems](https://github.com/capi-workgroup/problems/issues/45).

In the long term, I even consider doing a similar change for the regular C API, as HPy does, to hide even more implementation details and slowly bend the regular C API to the limited C API and the stable ABI. But that’s out of my short term scope. Now I only consider changing the **limited C API** version 3.12.

Victor

---

<div class="post-metadata">

**Author:** ![steve.dower](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/steve.dower/32/56_2.png) [@steve.dower](https://discuss.python.org/u/steve.dower)\
**Post date:** [June 9, 2023, 10:58am UTC](https://discuss.python.org/t/limited-c-api-implement-py-incref-and-py-decref-as-function-calls/27592/2 "2023-06-09T10:58:51Z")

</div>

I agree, but I don’t think we can make the change in 3.12.

Even more than a regular breaking change, if we introduce “ABI3 built on Python 3.12 isn’t really ABI3 and won’t work on 3.11 or earlier” then we break a lot of promises.

Perhaps this is the point where we introduce ABI4? There are plenty of cleanups we’ve been avoiding doing, so this would be the opportunity.

I don’t think there’s any way we can modify INCREF/DECREF without inherently breaking the stable ABI, the current changes for immortal objects included.

---

<div class="post-metadata">

**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\
**Post date:** [June 9, 2023, 11:09am UTC](https://discuss.python.org/t/limited-c-api-implement-py-incref-and-py-decref-as-function-calls/27592/3 "2023-06-09T11:09:21Z")

</div>

> [@steve.dower](#):
>
> Even more than a regular breaking change, if we introduce “ABI3 built on Python 3.12 isn’t really ABI3” then we break a lot of promises.

My change is backward **compatible** , it doesn’t break the backward compatibility (according to the limited C API specification). Please correct me if I’m wrong. (Check also my PR implementation.)

For the Python **debug build** , I already modified `Py_INCREF()` and `Py_DECREF()` to always use an opaque function call in Python 3.12: it uses `Py_IncRef()` and `Py_DecRef()` which exist since Python 3.0, or `_Py_IncRef()` and `_Py_DecRef()` when requesting the limited API version 3.10 or newer. So the implementation **works on Python 3.2** , as request by [PEP 384: Stable ABI](https://peps.python.org/pep-0384/).

Extract of Py\_INCREF() (simplified code):

```auto
tatic inline void Py_INCREF(PyObject *op)
{
#if defined(Py_REF_DEBUG) && defined(Py_LIMITED_API)
    // Stable ABI for Python built in debug mode. _Py_IncRef() was added to
    // Python 3.10.0a7, use Py_IncRef() on older Python versions. Py_IncRef()
    // accepts NULL whereas _Py_IncRef() doesn't.
# if Py_LIMITED_API+0 >= 0x030a00A7
    _Py_IncRef(op);
# else
    Py_IncRef(op);
# endif
#else
    ...
#endif
}

```

With my proposed change, **if you target limited C API version 3.12** , the wheel binary will use \_Py\_IncRef() and \_Py\_DecRef() which were added to Python 3.10. So the **wheel binary should only be used with Python 3.12** and newer, no?

Request limited C API version 3.2 if you care about maximum compatibility (support the maximum number of Python versions): my change would not be used in this case, the code would access directly `PyObject.ob_refcnt` as before.

---

<div class="post-metadata">

**Author:** ![steve.dower](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/steve.dower/32/56_2.png) [@steve.dower](https://discuss.python.org/u/steve.dower)\
**Post date:** [June 9, 2023, 11:28am UTC](https://discuss.python.org/t/limited-c-api-implement-py-incref-and-py-decref-as-function-calls/27592/4 "2023-06-09T11:28:58Z")

</div>

In the most trivial case, yes, your proposed change is compatible (your existing change is fine, debug builds don’t matter here, it’s the new proposal that concerns me).

The problem is back-and-forth compatibility. ABI3 built on 3.11 will have a different INCREF than 3.12 and so is not technically usable. A change in 3.12’s normal INCREF behaviour means that an ABI3 built on 3.12 targeting 3.11 or earlier will have incorrect INCREF on earlier versions. A difference between ABI3 INCREF and non-ABI3 INCREF within 3.12 builds will lead to issues that remove the value proposition of making the change in the first place.

What this means is that we’re effectively telling users to hold their noses and ignore known compatibility issues, or to avoid ABI3 builds, when we could do neither and instead just _not_ deliver a new feature of copy-on-write being triggered less often (as long as you don’t have ABI3 libraries in your system). The tradeoffs seem all wrong.

> [@vstinner](#):
>
> Request limited C API version 3.2 if you care about maximum compatibility

You know full well this doesn’t work 😉 The limited API has changed enough over time that promise is well broken, and it’s somewhere around 3.6-3.7 that things start becoming usable again (if you’re careful, and test thoroughly on multiple versions on multiple platforms, by which time you may as well have just built without it).

We’d be better off deprecating the limited API entirely and adopting HPy as the stable interface. (I assume they’re in a position to handle these INCREF changes safely, but haven’t checked for sure.)

---

<div class="post-metadata">

**Author:** ![steve.dower](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/steve.dower/32/56_2.png) [@steve.dower](https://discuss.python.org/u/steve.dower)\
**Post date:** [June 9, 2023, 11:34am UTC](https://discuss.python.org/t/limited-c-api-implement-py-incref-and-py-decref-as-function-calls/27592/5 "2023-06-09T11:34:48Z")

</div>

Oh, and you can only call old functions under limited API if they were part of the limited API at that time. Functions on Windows are only exported explicitly, and if they’re weren’t listed as limited API, extensions will simply break.

---

<div class="post-metadata">

**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\
**Post date:** [June 9, 2023, 12:30pm UTC](https://discuss.python.org/t/limited-c-api-implement-py-incref-and-py-decref-as-function-calls/27592/6 "2023-06-09T12:30:31Z")

</div>

It seems like Python 3.10 added Py\_IncRef() and Py\_DecRef to PC/python3dll.c, it wasn’t listed there before.

```auto
$ git show 3.9:PC/python3dll.c|grep -E 'Py_(Inc|Dec)Ref'
# no results

$ git show 3.10:PC/python3dll.c|grep -E 'Py_(Inc|Dec)Ref'
EXPORT_FUNC(_Py_DecRef)
EXPORT_FUNC(_Py_IncRef)
EXPORT_FUNC(Py_DecRef)
EXPORT_FUNC(Py_IncRef)

```

---

<div class="post-metadata">

**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\
**Post date:** [June 9, 2023, 12:43pm UTC](https://discuss.python.org/t/limited-c-api-implement-py-incref-and-py-decref-as-function-calls/27592/7 "2023-06-09T12:43:51Z")

</div>

> [@steve.dower](#):
>
> The problem is back-and-forth compatibility. ABI3 built on 3.11 will have a different INCREF than 3.12 and so is not technically usable.

The SC approved PEP 683 knowing that it impacts the stable ABI, it’s documented in PEP 683. I’m not sure that I get your rationale. If you build an extension for the stable ABI with 3.11, you get a buggy Py\_INCREF, but if you build it with 3.12 (with my change), you get the correct Py\_INCREF. Your point is that it’s better that everybody gets the same buggy Py\_INCREF on Python 3.12 and newer forever?

> [@steve.dower](#):
>
> A change in 3.12’s normal INCREF behaviour means that an ABI3 built on 3.12 targeting 3.11 or earlier will have incorrect INCREF on earlier versions.

Right, my change has no impact on limited API 3.11. Now I’m no longer sure if your point is about PEP 683 or my proposed change. PEP 683 got approved and is now implemented. My proposition is only to make the limited C API better for PEP 683.

> [@steve.dower](#):
>
> A difference between ABI3 INCREF and non-ABI3 INCREF within 3.12 builds will lead to issues that remove the value proposition of making the change in the first place.

PEP 683 states that using Py\_INCREF of Python 3.11 or older can have a negative impact on performance in some cases:

> This will invalidate all the performance benefits of immortal objects.

It doesn’t crash Python or change the object lifetime.

> [@steve.dower](#):
>
> > [@vstinner](#):
> >
> > Request limited C API version 3.2 if you care about maximum compatibility
> 
> You know full well this doesn’t work 😉 The limited API has changed enough over time that promise is well broken, and it’s somewhere around 3.6-3.7 that things start becoming usable again (if you’re careful, and test thoroughly on multiple versions on multiple platforms, by which time you may as well have just built without it).
> 
> We’d be better off deprecating the limited API entirely and adopting HPy as the stable interface. (I assume they’re in a position to handle these INCREF changes safely, but haven’t checked for sure.)

Is your point that the stable ABI was never usable and so it’s better to remove it rather than fixing it? It’s currently used by cryptography and PySide projects which seem to be happy about it.

Yes, there were implementation issues with the stable ABI. It’s getting better. The whole ABI (not only the stable ABI) is now carefully tested once Python 3.x.0 final is released (in stable branches). There are now better tests for the limited C API: [PEP 652: Maintaining the Stable ABI](https://peps.python.org/pep-0652/), accepted and implemented in Python 3.10. Implementation issues are being fixed over time.

---

<div class="post-metadata">

**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\
**Post date:** [June 9, 2023, 12:45pm UTC](https://discuss.python.org/t/limited-c-api-implement-py-incref-and-py-decref-as-function-calls/27592/8 "2023-06-09T12:45:51Z")

</div>

> [@vstinner](#):
>
> It seems like Python 3.10 added Py\_IncRef() and Py\_DecRef to PC/python3dll.c

But I don’t think that it’s relevant here. As I wrote before, my change only concerns the limited C API version 3.12 and newer. If you target older limited C API, these functions are simply not used.

---

<div class="post-metadata">

**Author:** ![ben.boeckel](https://avatars.discourse-cdn.com/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discuss.python.org/u/ben.boeckel)\
**Post date:** [June 9, 2023, 2:41pm UTC](https://discuss.python.org/t/limited-c-api-implement-py-incref-and-py-decref-as-function-calls/27592/9 "2023-06-09T14:41:09Z")

</div>

> [@steve.dower](#):
>
> Even more than a regular breaking change, if we introduce “ABI3 built on Python 3.12 isn’t really ABI3 and won’t work on 3.11 or earlier” then we break a lot of promises.

AFAIK, this has always been the case. If I build with 3.11’s stable ABI and use the buffer API, it won’t (necessarily) work on 3.10. Here, if I use the 3.12 stable ABI, I expect it to work with 3.13 and later, but 3.11 and older have no expectations.

---

<div class="post-metadata">

**Author:** ![steve.dower](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/steve.dower/32/56_2.png) [@steve.dower](https://discuss.python.org/u/steve.dower)\
**Post date:** [June 9, 2023, 3:54pm UTC](https://discuss.python.org/t/limited-c-api-implement-py-incref-and-py-decref-as-function-calls/27592/10 "2023-06-09T15:54:59Z")

</div>

> [@ben.boeckel](#):
>
> AFAIK, this has always been the case.

It has, but unless Victor’s next change brings back the old `Py_INCREF` in 3.12 when compiling with `Py_LIMITED_API < 0x030C0000`, keeps the current `Py_INCREF` for `!defined(Py_LIMITED_API)` and adds the new call for `Py_LIMITED_API >= 0x030C0000` then we’ll introduce new breakage (on top of the breakage that already exists for people who thought that `Py_LIMITED_API < 0x030C0000` would be compatible with Python 3.12…)

---

<div class="post-metadata">

**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\
**Post date:** [June 13, 2023, 1:33pm UTC](https://discuss.python.org/t/limited-c-api-implement-py-incref-and-py-decref-as-function-calls/27592/11 "2023-06-13T13:33:12Z")

</div>

@steve.dower IMO you misunderstood the specification of the limited C API and the stable ABI. Maybe the documentation is unclear about backward and forward compatibility at the API and ABI level. But I don’t see how my proposed change is different than other limited C API / ABI changes done in the change. It respects the specification.

Yes, for each Python version, we have different API and ABI depending the request limited C API version, and the testing matrix is quite big. But I don’t see why Py\_INCREF/DECREF is different and how my change breaks any “promise”.

It seems like you’re thinking about hypothetical correctness issues, but there is no known correctness issue: [https://github.com/python/cpython/issues/105387#issuecomment-1584643774](https://github.com/python/cpython/issues/105387#issuecomment-1584643774) I am unable to reason about hypothetical issues. The discussion would be more constructive if you come with a concrete issue, or example of past mistakes related to limited C API / stable ABI which might happen with my proposed change.

I plan to merge my PR soon. It got approved by @gpshead.

---

<div class="post-metadata">

**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\
**Post date:** [June 13, 2023, 6:30pm UTC](https://discuss.python.org/t/limited-c-api-implement-py-incref-and-py-decref-as-function-calls/27592/12 "2023-06-13T18:30:01Z")

</div>

_Hum, the discussion is now splitted between the issue, the PR and here. Copy of [my message on my PR](https://github.com/python/cpython/pull/105388#issuecomment-1589820822)._

If I understood correctly @steve.dower rationale, he wants to build a C extension with (the limited C API of) the **new** Python 3.(N+1), run it with an **old** Python 3.N and have the **exact same behavior** _as if_ the C extension was built with an **old** Python 3.N. The **current implementation** of the limited C API **DOES NOT provide such ABI warranty.** There are many corner cases like the function being discussed here: Py\_INCREF().

There are around 315 functions in Python 3.12 which are either implemented as a macro or as a static inline function and so is likely to access structure members or relying on another implementation details: [Statistics on the Python C API — pythoncapi 0.1 documentation](https://pythoncapi.readthedocs.io/stats.html#functions-defined-as-macros-and-static-inline-functions)

The intent of moving the limited C API towards “opaque function calls” is to build a C extension with Python 3.N and have exactly the Python 3.x behavior on Python 3.x since the executed code is no longer hardcoded in the extension binary, but instead we call call in the Python executable (libpython).

For example, if this change lands in Python 3.12 beta3, a C extension is **built with** Python 3.12 **beta3** , but Python 3.12 **beta4** changes again the Py\_INCREF() implementation: the C extension will execute the exact **new beta4** implementation, rather than using the copy of **old beta3** implementation.

Without opaque function calls, the promise of a “stable ABI” is **weak**. This change moves the limited C API / stable ABI **one step** towards a better (more stable) stable ABI.

@steve.dower rationale is that the implementation of the limited C API and the stable ABI is broken is should be abandoned. I disagree here. IMO not only the current half-baken implementation is useful to many people, but also it is fixable in an increment way. Moreover, my long term goal is to bend the whole (regular) Python C API towards to limited C API / stable ABI.

---

<div class="post-metadata">

**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\
**Post date:** [June 14, 2023, 3:00am UTC](https://discuss.python.org/t/limited-c-api-implement-py-incref-and-py-decref-as-function-calls/27592/13 "2023-06-14T03:00:27Z")

</div>

I merged my PR: see [PEP 683 (Immortal Objects): Implement Py\_INCREF() as function call in limited C API 3.12 · Issue #105387 · python/cpython · GitHub](https://github.com/python/cpython/issues/105387) for details.

The new compiler warning is not fixed yet: [object.h uses an anonymous union in a struct (older C incompatible) · Issue #105059 · python/cpython · GitHub](https://github.com/python/cpython/issues/105059)

---

<div class="post-metadata">

**Author:** ![encukou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/encukou/32/2461_2.png) [@encukou](https://discuss.python.org/u/encukou)\
**Post date:** [June 14, 2023, 7:58am UTC](https://discuss.python.org/t/limited-c-api-implement-py-incref-and-py-decref-as-function-calls/27592/14 "2023-06-14T07:58:01Z")

</div>

> [@vstinner](#):
>
> It seems like Python 3.10 added Py\_IncRef() and Py\_DecRef to PC/python3dll.c, it wasn’t listed there before.

You’re looking at the wrong place. 3.9 used a different file to list the functions.

```console
$ git show 3.9:PC/python3dll.c | wc --lines
8

$ git grep -E 'Py_(Inc|Dec)Ref' 3.9:PC/python3.def
3.9:PC/python3.def:731: Py_DecRef=python39.Py_DecRef
3.9:PC/python3.def:758: Py_IncRef=python39.Py_IncRef

$ git grep -E 'Py_(Inc|Dec)Ref' v3.2.1:PC/python3.def
v3.2.1:PC/python3.def:627: Py_DecRef=python32.Py_DecRef
v3.2.1:PC/python3.def:646: Py_IncRef=python32.Py_IncRef

```

They were there since the very first stable ABI version. AFAICS, it’s safe to use them.

The only reason against them that I can see is performance – but it’s becoming clear that, long-term, any flavor of stable ABI we come up with will need C function calls even for such frequent operations.  
If C function calls are too slow for some extension, it should build against the version-specific ABI. (Preferably _in addition_ to a stable ABI build, so it works with older/newer interpreters too.)

So let’s switch to function calls now. I wouldn’t do it in 3.12, as it could have performance implications that affect people who test with betas, but that’s the release manager’s call.  
AFAIC, in 3.13 we could switch to function calls in _all_ limited API versions.

---

<div class="post-metadata">

**Author:** ![malemburg](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/malemburg/32/50_2.png) [@malemburg](https://discuss.python.org/u/malemburg)\
**Post date:** [June 14, 2023, 9:36am UTC](https://discuss.python.org/t/limited-c-api-implement-py-incref-and-py-decref-as-function-calls/27592/15 "2023-06-14T09:36:34Z")

</div>

> [@vstinner](#):
>
> Moreover, my long term goal is to bend the whole (regular) Python C API towards to limited C API / stable ABI.

I think you will need to get SC buy-in for such a plan first.

Right now, every single change in that direction appears to create long discussions, so at least part of the core devs are not aligned with your idea. This causes frustration on both sides and is not a healthy approach to implementing change.

Once we have a clear set of goals, everyone can focus on these goals and discussions would then focus more on implementation details and making change less painful.

FWIW, I have never been a fan of the cross-version ABI stability idea. Many of the top used Python extensions want performance and thus need to be compiled per version. The added confusion around the various API subsets doesn’t really pay off, if they are only used by a handful of less used extensions. With today’s CI/CD pipelines, creating distributions for the common set of targets is not hard anymore (as it was when Martin kicked off the idea) and tooling in that area is getting better all the time as well.

---

<div class="post-metadata">

**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\
**Post date:** [June 14, 2023, 10:50am UTC](https://discuss.python.org/t/limited-c-api-implement-py-incref-and-py-decref-as-function-calls/27592/16 "2023-06-14T10:50:14Z")

</div>

> [@malemburg](#):
>
> Right now, every single change in that direction appears to create long discussions, so at least part of the core devs are not aligned with your idea. This causes frustration on both sides and is not a healthy approach to implementing change.

I agree and it was discussed in length during the Language Summit: [Python Software Foundation News: The Python Language Summit 2023: Three Talks on the C API](https://pyfound.blogspot.com/2023/05/the-python-language-summit-2023-three.html)

The outcome is the creation of the creation of this project which will become an informal PEP listing C API “issues” without proposing solutions: [GitHub - capi-workgroup/problems: Discussions about problems with the current C Api](https://github.com/capi-workgroup/problems/)

But here the scope is only: the limited C API, Python 3.12 and PEP 683.

---

<div class="post-metadata">

**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\
**Post date:** [June 14, 2023, 11:02am UTC](https://discuss.python.org/t/limited-c-api-implement-py-incref-and-py-decref-as-function-calls/27592/17 "2023-06-14T11:02:13Z")

</div>

> [@encukou](#):
>
> So let’s switch to function calls now. I wouldn’t do it in 3.12, as it could have performance implications that affect people who test with betas, but that’s the release manager’s call.

In terms of development cycle, IMO it’s better to use function calls right now, spend time to measure the performance impact in length, and keep in mind that we _can_ revert to use again inlined in the limited C API in Python 3.12.

Making such change _late_ in the development cycle with a potential negative impact on performance sounds risky. Eddie Elizondo spent long time to investigate different implementations and measure their cost on performance. I don’t get expect someone to come up with a “magic” clever optimization trick next months.

Moreover, IMO the urgency is to fix the compiler warning about the usage of PyObject anoymous union when using C99, especially strict C99 (“pedantic”).

---

<div class="post-metadata">

**Author:** ![h-vetinari](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/h-vetinari/32/1220_2.png) [@h-vetinari](https://discuss.python.org/u/h-vetinari)\
**Post date:** [June 14, 2023, 11:36am UTC](https://discuss.python.org/t/limited-c-api-implement-py-incref-and-py-decref-as-function-calls/27592/18 "2023-06-14T11:36:19Z")

</div>

> [@malemburg](#):
>
> The added confusion around the various API subsets doesn’t really pay off, if they are only used by a handful of less used extensions.

As just one example, `cryptography` can hardly count as a “less used extension” and its devs are quite [adamant](https://discuss.python.org/t/lets-get-rid-of-the-stable-abi-but-keep-the-limited-api/18458/10) that without it, their support for new python versions would be reduced/delayed. Seeing how `cryptography` underlies `requests` and everything web-related on top of that, this would be a huge barrier for the roll-out of any new CPython version to the broader ecosystem.

---

<div class="post-metadata">

**Author:** ![malemburg](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/malemburg/32/50_2.png) [@malemburg](https://discuss.python.org/u/malemburg)\
**Post date:** [June 22, 2023, 6:34pm UTC](https://discuss.python.org/t/limited-c-api-implement-py-incref-and-py-decref-as-function-calls/27592/19 "2023-06-22T18:34:21Z")

</div>

Fair enough, but is this really such a good example ?

The project is using a home grown solution to building wheels instead of building on top of the officially supported [cibuildwheel](https://github.com/pypa/cibuildwheel), which really makes the process a lot easier.

cibuildwheel supports all Python versions currently supported by `cryptography`. Most of the heavy lifting is taken care off by this project, so the added overhead of building per version wheels is not that significant: [example config](https://github.com/pypa/cibuildwheel#example-setup).

Quite a few _big_ projects use it successfully: [some projects using cibuildwheel](https://github.com/pypa/cibuildwheel#working-examples), including e.g. NumPy and matplotlib.

Anyway, this is getting off-topic and just my personal opinion on the stable ABI idea and the added complexity that it introduces seen from today’s POV (it was a good idea at the time, esp. for Windows builds).

---

<div class="post-metadata">

**Author:** ![MartinBonner](https://avatars.discourse-cdn.com/v4/letter/m/51bf81/32.png) [@MartinBonner](https://discuss.python.org/u/MartinBonner)\
**Post date:** [August 10, 2023, 8:49am UTC](https://discuss.python.org/t/limited-c-api-implement-py-incref-and-py-decref-as-function-calls/27592/20 "2023-08-10T08:49:21Z")

</div>

> FWIW, I have never been a fan of the cross-version ABI stability idea. Many of the top used Python extensions want performance and thus need to be compiled per version. The added confusion around the various API subsets doesn’t really pay off, if they are only used by a handful of less used extensions. With today’s CI/CD pipelines, creating distributions for the common set of targets is not hard anymore (as it was when Martin kicked off the idea) and tooling in that area is getting better all the time as well.

I very much want to use a stable ABI for my (proprietary) extension. Building it for two versions of python was extremely painful in the past, and extending that to 3.8, 3.9, 3.10, and 3.11 is the sort of pain I _really_ don’t want. There may be perfectly good solutions for building multiple wheels for different versions when what you what you are producing is “an extension”, but when you have an existing build system for an existing (fairly large) product incorporating those is likely to be hard.

Using the stable ABI makes my life very much simpler (and the performance difference between the stable ABI and the version-specific ABI is trivial for us.)

[Next page](https://discuss.python.org/t/limited-c-api-implement-py-incref-and-py-decref-as-function-calls/27592.md?page=2)