Ongoing packages' rebuild with Python 3.13 in Fedora

Core Development
1

Hello,
In Fedora, we’re rebuilding a set of around 4000 packages with Python 3.13.0 alpha 1.
Here’s the current status.

  • ~2600 packages were rebuilt successfully so far.
  • ~1400 packages were not rebuilt yet, because their dependencies didn’t build.
    • current biggest blocker: numpy (blocks ~500 packages)
    • previous biggest blocker: Cython (fixed in Cython and CPython – we’ve backported both changes)
  • ~220 failed to build. This includes test failures, for packages where we run the test suite.
    • Some failures were already fixed; 158 still fail to build

Here is a rough breakdown of the failures we encountered.

Removal of C APIs with names prefixed by _Py or _PY

5 fail with: implicit declaration of function ‘_Py_IDENTIFIER‘: python-zodbpickle, python-drgn, btrfs-progs, python3-typed_ast, python-multidict

5 fail with: ‘_PyThreadState_UncheckedGet’ was not declared in this scope: python-wsaccel, pybind11, pythia8, HepMC3, python-cffi

4 fail with: implicit declaration of function ‘_PyUnicode_AsString: python-gstreamer1, kernel-tools, python-cffi, python-cvxopt

3 fail with: ‘_PyLong_Copy’ was not declared in this scope; did you mean ‘PyLong_Type’?: python-pyroaring, fonttools, python-cradox

3 fail with: ‘_PyObject_EXTRA_INIT’ undeclared here (not in a function): python-dukpy, pysvn, python-cbor2

2 fail with: implicit declaration of function ‘_PyObject_CallMethodId’: python-multidict, python3-typed_ast

2 fail with: implicit declaration of function ‘_PyStack_AsDict’: setools, python-asyncmy

2 fail with: implicit declaration of functions ‘_PyUnicodeWriter_Init’, ‘_PyUnicodeWriter_WriteASCIIString’, ‘_PyUnicodeWriter_Finish’, ‘_PyUnicodeWriter_Dealloc’, : python-recordclass, python-immutables

2 fail with: implicit declaration of function ‘_PyLong_FromByteArray’, ‘_PyLong_AsByteArray’: python-wsaccel, python-drgn

2 fail with: implicit declaration of function ‘_PyLong_New’: python-crypto, python-gmpy2

2 fail with: implicit declaration of function ‘_PyObject_GetAttrId’: python-drgn, python3-typed_ast

1 fails with: implicit declaration of functions ‘_PyDict_GetItem_KnownHash’, ‘_PyVectorcall_Function’: python-wsaccel

2 fail with: implicit declaration of function ‘_PyDict_NewPresized’: python-wsaccel, mercurial

1 fails with: implicit declaration of function ‘_PyList_Extend’: python-pyrsistent

1 fails with: implicit declaration of function ‘_PyUnicodeWriter_WriteChar’: python-recordclass

1 fails with: implicit declaration of function ‘_PyThread_CurrentFrames’: mod_wsgi

1 fails with: implicit declaration of function ‘_PyLong_Sign’, ‘_Py_HashPointer’, ‘_PyErr_WriteUnraisable’: python-cffi

Removals of deprecated C APIs

21 fail with: implicit declaration of function ‘PyEval_InitThreads’: grpc,
gnumeric, gnofract4d, collectd, python-cradox, libsigrokdecode, lldb, profanity, python-subvertpy, pyotherside, sigil, python-simpleaudio, OpenIPMI, libesedb, libcec, python-gphoto2, python-bsddb3, python-confluent-kafka, python-alsaaudio, freeradius, pyliblo

17 fail with: missing <unistd.h> declaration: onboard, dbus-python, opae, borgbackup, python-schedutils, mercurial, python-maxminddb, rdiff-backup, python-zstandard, python-pywayland, python-kerberos, pyicu, python-apt, py-spidev, python-netifaces, i2c-tools, python-yara

13 fail with: implicit declaration of function ‘PyEval_CallObject’; did you mean ‘PyObject_CallObject’?: dionaea, audit, gdcm, jsonnet, med, newt, pcapy, python-pivy, recoll, python-cups, m2crypto, libsolv, libcec

8 fail with: implicit declaration of function ‘PySys_SetArgv’: python-caja, nemo-extensions, gnumeric, nautilus-python, collectd, remmina, kernel-tools, pyotherside

8 fail with: missing <ctype.h> declaration: pyodbc, python-cx-oracle, proxmark3, python-gammu, python-igraph, sudo, python-cffi, python-bitstruct

4 fail with: implicit declaration of function Py_SetProgramName: glade, dionaea, fontforge, perl-Inline-Python

2 fail with: implicit declaration of function PySys_SetArgvEx: glade, pyotherside

2 fail with: implicit declaration of function ‘Py_TRASHCAN_SAFE_BEGIN’, ‘Py_TRASHCAN_SAFE_END’: python-multidict, python-pyrsistent

2 fail with: ‘PyEval_CallObject’ was not declared in this scope; did you mean ‘PyObject_CallObject’?: vrpn, libprelude

1 fails with: implicit declaration of function ‘PyObject_AsWriteBuffer’, ‘PyObject_AsCharBuffer’: cryptlib

1 fails with: implicit declaration of function ‘PyUnicode_AS_DATA’: python-alsaaudio

1 fails with: implicit declaration of function ‘PyEval_AcquireLock’, ‘PyEval_ReleaseLock’: freeradius

1 fails with: implicit declaration of function ‘PyUnicode_GetSize’, ‘PyUnicode_AS_UNICODE’: python-scss

1 fails with: #error “this header requires Py_BUILD_CORE define”: python-greenlet

1 fails with: ‘PyCFunction_Call’ was not declared in this scope: PyQt4

1 fails with: implicit declaration of function ‘PyObject_AsReadBuffer’: python-intbitset

1 fails with: ‘PySys_SetPath’ was not declared in this scope: insight

Optimization changes impact

4 fail with docstring indentation not matching the expected: python-parameterized, python-oauthlib, python-zope-interface, python-toolz

Removal of Python modules and functions

24 fail with 2to3 removal: swig, uwsgi, wavextract, workrave, tuxpaint-stamps, python-yaql, libsbml, mftrace, b43-tools, TeXmacs, fail2ban, flow-tools, gnu-free-fonts, octave-miscellaneous, python-MultipartPostHandler2, python-amico, python-demjson, python-mglob, python-minimock, python-nose, python-openoffice, python-openopt, python-straight-plugin, python-x2go

10 fail with unittest-related removals: python-lxml, python-ptyprocess, python-random2, python-transaction, python-genshi, python-zope-i18nmessageid, pyserial, python-jsonpointer, python-iniparse, python-defusedxml

5 fail with: AttributeError: module ‘importlib.resources’ has no attribute ‘read_text’, ‘path’, ‘open_binary’ or ‘read_binary’: pre-commit, numpy, python-netaddr, meson, python-installer

5 fail with: ‘Logger’ object has no attribute ‘warn’: python-grabbit, python-etcd, python-idstools, pyzor, mom

2 fail with: ImportError: cannot import name ‘resetlocale’ from ‘locale’: python-fdb, pykickstart

1 fails with: ImportError: cannot import name ‘re’ from ‘typing’: python-ndjson-testrunner

1 fails with implicit declaration of function ‘getpid’ / ‘gettimeofday’: qpid-dispatch

15 Likes
2

@vstinner ^^

(IIRC the SC recommended somewhere to just restore any removed “private” API that is still being used.)

3 Likes
3

Those two look like bugs in the CPython include headers?

4

… or packages incorrectly assuming that the Python headers always included certain standard headers.

5 Likes
5

This is in fact a feature documented in What’s new in Python 3.13:

Python.h no longer includes the <unistd.h> standard header file. If needed, it should now be included explicitly. For example, it provides the functions: read(), write(), close(), isatty(), lseek(), getpid(), getcwd(), sysconf(), getpagesize(), alarm() and pause(). As a consequence, _POSIX_SEMAPHORES and _POSIX_THREADS macros are no longer defined by Python.h. The HAVE_UNISTD_H and HAVE_PTHREAD_H macros defined by Python.h can be used to decide if <unistd.h> and <pthread.h> header files can be included. (Contributed by Victor Stinner in gh-108765.)

Python.h no longer includes the <ctype.h> standard header file. If needed, it should now be included explicitly. For example, it provides isalpha() and tolower() functions which are locale dependent. Python provides locale independent functions, like Py_ISALPHA() and Py_TOLOWER(). (Contributed by Victor Stinner in gh-108765.)

3 Likes
6

I created gh-112026: Add again <unistd.h> include in Python.h by vstinner · Pull Request #112046 · python/cpython · GitHub to add again <ctype.h> and <unistd.h> to Python.h since their removal impacted many projects (8 and 17 projects).

7

Is it possible for distros to use the -include option to force these into packages that are not updated?

I’d rather not make the Python headers be responsible for including other system headers, and there isn’t really any way to warn people that they’re relying on it. Eventually they’ll be removed in another alpha 1 to provide the maximum time to catch up.

4 Likes
8

Agreed. The Python headers should only include those system headers that are necessary for the headers to work properly. Including Python.h is not an excuse for not including the system headers that you use in your own code.

2 Likes