Propose that Argument Clinic only uses public Python APIs to make it easier to use outside CPython

Argument Clinic is an internal tool in CPython that handle argument parsing and calling convention for builtin modules.
Even “Its use is not supported for files outside CPython” (Argument Clinic How-To — Python 3.10.5 documentation), I found it very helpful when working with third-party modules outside CPython, e.g. code-data-share-for-python/_cdsmodule.c.h at main · alibaba/code-data-share-for-python · GitHub. This file works perfectly for 3.8 <= python <= 3.10, while the initial setup is also quite simple.

While Argument Clinic itself is changing with CPython, it’s fine for CPython as the generated code can always access the latest codebase of CPython. But for outside users, when Argument Clinic uses symbols in a new version of CPython, these change needs additional effort, which could be saved if AC only uses public Python APIs, by putting those new public APIs in GitHub - python/pythoncapi-compat: The pythoncapi-compat project can be used to write a C extension supporting a wide range of Python versions with a single code base..

cc @vstinner

PS: I’m not sure how this works for packages that support PyPy, or it’s totally irrevelant?

1 Like

See also Victor’s comment in gh-85283:

Argument Clinic could be modified to have an option to only use C API of the limited C API. Cython is working on a similar option (restraint emitted code to the limited C API).

1 Like

IMO we should first modify AC to avoid the internal C API and only use the public C API. Targeting the limited C API would be even better.

Recently, I added _PyCFunction_CAST() to fix a C++ compiler warning, but right now the function is not part of the public C API. AC should either avoid it or we should make this macro public: Consider adding `_PyCFunction_CAST` · Issue #43 · python/pythoncapi-compat · GitHub

I hesitated to make the macro public since… it’s a macro and I am trying to move the Python C API avoid from macro, I mean, I wrote PEP 670 with Erlend :smiley: PEP 670 – Convert macros to functions in the Python C API | But well, maybe such macro is better than the previous cast which emitted a C++ compiler warning (“old-style cast”).

No, we should not. AC is internal and CPython-only, so it can use CPython-specific API.
If AC is needed outside CPython, IMO it should be forked (and probably improved), like pegen.

It can be modified to have two build modes: one using the internal API, one using public or limited API (or we can have a 3rd mode for the limited API). CPython itself would benefit, I would like to be able to build a stdlib extension module with the limited API and use AC (since AC is great for docstring and inspect.signature()). See: [C API] Convert a few stdlib extensions to the limited C API (PEP 384) · Issue #85283 · python/cpython · GitHub

Maintaining a fork outside Python sounds counter productive to me.