# Propose that Argument Clinic only uses public Python APIs to make it easier to use outside CPython

**URL:** <https://discuss.python.org/t/propose-that-argument-clinic-only-uses-public-python-apis-to-make-it-easier-to-use-outside-cpython/16557>\
**Category:** Python Help\
**Created:** [June 16, 2022, 9:02am UTC](https://discuss.python.org/t/propose-that-argument-clinic-only-uses-public-python-apis-to-make-it-easier-to-use-outside-cpython/16557 "2022-06-16T09:02:47Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![oraluben](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/oraluben/32/6306_2.png) [@oraluben](https://discuss.python.org/u/oraluben)\
**Post date:** [June 16, 2022, 9:02am UTC](https://discuss.python.org/t/propose-that-argument-clinic-only-uses-public-python-apis-to-make-it-easier-to-use-outside-cpython/16557/1 "2022-06-16T09:02:47Z")

</div>

Argument Clinic is an internal tool in CPython that handle argument parsing and calling convention for builtin modules.  
Even “Its use is not supported for files outside CPython” ([Argument Clinic How-To — Python 3.10.5 documentation](https://docs.python.org/3/howto/clinic.html)), I found it very helpful when working with third-party modules outside CPython, e.g. [code-data-share-for-python/\_cdsmodule.c.h at main · alibaba/code-data-share-for-python · GitHub](https://github.com/alibaba/code-data-share-for-python/blob/main/src/_cds/clinic/_cdsmodule.c.h). This file works perfectly for 3.8 \<= python \<= 3.10, while the initial setup is also quite simple.

While Argument Clinic itself is changing with CPython, it’s fine for CPython as the generated code can always access the latest codebase of CPython. But for outside users, when Argument Clinic uses symbols in a new version of CPython, these change needs additional effort, which could be saved if AC only uses public Python APIs, by putting those new public APIs in [GitHub - python/pythoncapi-compat: The pythoncapi-compat project can be used to write a C extension supporting a wide range of Python versions with a single code base.](https://github.com/python/pythoncapi-compat).

cc @vstinner

PS: I’m not sure how this works for packages that support PyPy, or it’s totally irrevelant?

---

<div class="post-metadata">

**Author:** ![erlendaasland](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/erlendaasland/32/4378_2.png) [@erlendaasland](https://discuss.python.org/u/erlendaasland)\
**Post date:** [June 16, 2022, 9:15am UTC](https://discuss.python.org/t/propose-that-argument-clinic-only-uses-public-python-apis-to-make-it-easier-to-use-outside-cpython/16557/2 "2022-06-16T09:15:05Z")

</div>

See also [Victor’s comment in gh-85283](https://github.com/python/cpython/issues/85283#issuecomment-1093875734):

> Argument Clinic could be modified to have an option to only use C API of the limited C API. Cython is working on a similar option (restraint emitted code to the limited C API).

---

<div class="post-metadata">

**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\
**Post date:** [June 16, 2022, 1:09pm UTC](https://discuss.python.org/t/propose-that-argument-clinic-only-uses-public-python-apis-to-make-it-easier-to-use-outside-cpython/16557/3 "2022-06-16T13:09:08Z")

</div>

IMO we should first modify AC to avoid the internal C API and only use the public C API. Targeting the limited C API would be even better.

Recently, I added \_PyCFunction\_CAST() to fix a C++ compiler warning, but right now the function is not part of the public C API. AC should either avoid it or we should make this macro public: [Consider adding `_PyCFunction_CAST` · Issue #43 · python/pythoncapi-compat · GitHub](https://github.com/python/pythoncapi-compat/issues/43)

I hesitated to make the macro public since… it’s a macro and I am trying to move the Python C API avoid from macro, I mean, I wrote PEP 670 with Erlend 😃 [PEP 670 – Convert macros to functions in the Python C API | peps.python.org](https://peps.python.org/pep-0670/) But well, maybe such macro is better than the previous cast which emitted a C++ compiler warning (“old-style cast”).

---

<div class="post-metadata">

**Author:** ![encukou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/encukou/32/2461_2.png) [@encukou](https://discuss.python.org/u/encukou)\
**Post date:** [June 16, 2022, 1:45pm UTC](https://discuss.python.org/t/propose-that-argument-clinic-only-uses-public-python-apis-to-make-it-easier-to-use-outside-cpython/16557/4 "2022-06-16T13:45:47Z")

</div>

> [@vstinner](#):
>
> IMO we should first modify AC to avoid the internal C API and only use the public C API. Targeting the limited C API would be even better.

No, we should not. AC is internal and CPython-only, so it can use CPython-specific API.  
If AC is needed outside CPython, IMO it should be forked (and probably improved), like [pegen](https://pypi.org/project/pegen/).

---

<div class="post-metadata">

**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\
**Post date:** [June 16, 2022, 2:09pm UTC](https://discuss.python.org/t/propose-that-argument-clinic-only-uses-public-python-apis-to-make-it-easier-to-use-outside-cpython/16557/5 "2022-06-16T14:09:27Z")

</div>

> [@encukou](#):
>
> No, we should not. AC is internal and CPython-only, so it can use CPython-specific API.

It can be modified to have two build modes: one using the internal API, one using public or limited API (or we can have a 3rd mode for the limited API). CPython itself would benefit, I would like to be able to build a stdlib extension module with the limited API _and_ use AC (since AC is great for docstring and inspect.signature()). See: [https://github.com/python/cpython/issues/85283](https://github.com/python/cpython/issues/85283)

Maintaining a fork outside Python sounds counter productive to me.

---

<div class="post-metadata">

**Author:** ![ferdnyc](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/ferdnyc/32/6325_2.png) [@ferdnyc](https://discuss.python.org/u/ferdnyc)\
**Post date:** [April 25, 2024, 12:39am UTC](https://discuss.python.org/t/propose-that-argument-clinic-only-uses-public-python-apis-to-make-it-easier-to-use-outside-cpython/16557/6 "2024-04-25T00:39:43Z")

</div>

So, seems like this sort of went in the other direction, huh?

With the Argument Clinic implementation in current CPython’s `Tools/clinic/`, even if I call `clinic.py --limited` it will emit includes of internal headers, which unless I’m missing something makes its code unusable outside of the CPython tree?

(Partial results of a `clinic.py --limited` run just now, using the `main` branch of [GitHub - python/cpython: The Python programming language](http://github.com/python/cpython/)):

```c
/*[clinic input]
preserve
[clinic start generated code]*/

#include "pycore_long.h" // _PyLong_UnsignedInt_Converter()

PyDoc_STRVAR(search_create_index __doc__ ,
"create_index($module, files, /, n_threads, search_dir, fragments_dir,\n"
" html_dir, private_dir, stopwords_path)\n"
// ...

```

---

<div class="post-metadata">

**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\
**Post date:** [April 29, 2024, 2:02pm UTC](https://discuss.python.org/t/propose-that-argument-clinic-only-uses-public-python-apis-to-make-it-easier-to-use-outside-cpython/16557/7 "2024-04-29T14:02:58Z")

</div>

> [@ferdnyc](#):
>
> With the Argument Clinic implementation in current CPython’s `Tools/clinic/`, even if I call `clinic.py --limited` it will emit includes of internal headers, which unless I’m missing something makes its code unusable outside of the CPython tree?

The limited C API is not fully supported. Some kind of arguments still use the internal C API. It’s a work in progress. Which arguments use the internal C API for you?

---

<div class="post-metadata">

**Author:** ![storchaka](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/storchaka/32/217_2.png) [@storchaka](https://discuss.python.org/u/storchaka)\
**Post date:** [April 30, 2024, 12:30pm UTC](https://discuss.python.org/t/propose-that-argument-clinic-only-uses-public-python-apis-to-make-it-easier-to-use-outside-cpython/16557/8 "2024-04-30T12:30:20Z")

</div>

This option is experimental. It helps us to see what happens when try to use the limited C API globally, what should be done to make it usable. If some private C API is often used, we should consider adding the corresponding public C API. If some private C API is rarely used, we can consider making Argument Clinic not using it with the limited C API.

If you want to use Argument Clinic in your project, consider writing custom converters.

---

<div class="post-metadata">

**Author:** ![ferdnyc](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/ferdnyc/32/6325_2.png) [@ferdnyc](https://discuss.python.org/u/ferdnyc)\
**Post date:** [May 18, 2024, 9:49am UTC](https://discuss.python.org/t/propose-that-argument-clinic-only-uses-public-python-apis-to-make-it-easier-to-use-outside-cpython/16557/9 "2024-05-18T09:49:32Z")

</div>

> [@vstinner](#):
>
> Which arguments use the internal C API for you?

Well, integers, for one! That `pycore_long.h` header being included is located at `/usr/include/python3.12/internal/pycore_long.h` on my system, so I was taking it to be an internal header.