Hiding deprecated stable ABI

In PEP 623, I planned to remove PyUnicode_GetSize() in Python 3.12 like many other functions using Py_UNICODE.

This function has been deprecated since Python 3.3. But now I find that this API is listed as stable ABI.

PEP 384 excluded Py_UNICODE from the stable abi. PyUnicode_GetSize() is tightly coupled with Py_UNICODE, but it doesn’t include Py_UNICODE in its signature. I think that’s why this function is listed as stable abi.

I will make the API return error always instead of removing the API.
This is not the first time that we abandon the stable ABI. For example:

// We keep this around for (accidental) stable ABI compatibility.
// Realisically, no extensions are using it.
void
_PyThreadState_Init(PyThreadState *tstate)
{
    Py_FatalError("_PyThreadState_Init() is for internal use only");
}

Currently, we do not have any plan to remove stable ABI (a.k.a. Python 4).
But can we prevent they are used/linked with new binaries?

  • Remove them from public header files.
    • PyAPI_FUNC macros can be used in source file, not header file.
  • Remove them from “import library” (e.g. python3.lib).
    • Is this possible? I am not Windows expert.

10+ years later, we can discuss about breaking compatibility with binary compiled with Python <3.12 if we don’t have plan for Python 4 yet.

This might be fine, but it’s probably best to undefine it when a certain limited API version is requested. So anyone trying to build an older package will see it, but if they say that they want the 3.12 version of the API then it’ll be missing.

No, this one has to stay. It can return an error when called, but if it’s missing completely then the module might fail to load, regardless of whether it ever calls it.

Hi,
I plan to write the process for removing items from the limited API as a PEP for 3.12, but I won’t have time to draft it for a month or two.
Could you open an issue for this & ping me on it? In the mean time just make it always raise an error.

2 Likes

I wrote gh-85858: Remove PyUnicode_InternImmortal() function by vstinner · Pull Request #92579 · python/cpython · GitHub which removes PyUnicode_InternImmortal() from the API but keeps it in the ABI.

2 Likes

Thanks! I forgot that we have abi_only flag already.

Is it impossible to build an import library that linking only supported limited ABI without removing dead APIs from DLL to avoid link (load) error?

Not quite sure I followed the question, so I’ll just expand on how it works and hopefully that’ll cover it.

The python3.dll has to export all the functions that any extension built with any version of python3.lib might export. It may export more, but removing any functions will break extensions built with that earlier version when they try and load against the newer python3.dll.

I believe with the default linker settings, any completely unreferenced function will usually be completely dropped and won’t be checked. But if it’s referenced (even if never called), it’ll need to be there at load time.

Now, because it technically only has to be in python3.dll, we could turn it into a real DLL and put a stub implementation in there. Right now, it just forwards all names to its matching python3x.dll. But adding code here is a big change to how we build things - it might be worth doing if we were patching something that really needed to change, but it seems like overkill just to remove a deprecated item rather than making a simple implementation.

2 Likes

In order to create .lib file for limited functions, write .def file and compile it with lib.exe.

; python3.def
LIBRARY python3
EXPORTS
  Py_Main
  ; ...
> lib.exe /def:python3.def /out:python3.lib

> dumpbin /exports python3.lib
Microsoft (R) COFF/PE Dumper Version 14.34.31933.0
Copyright (C) Microsoft Corporation.  All rights reserved.


Dump of file python3.lib

File Type: LIBRARY

     Exports

       ordinal    name

                  Py_Main

  Summary

          C3 .debug$S
          14 .idata$2
          14 .idata$3
           8 .idata$4
           8 .idata$5
           C .idata$6