Bump SQLite minimum version requirement

Core Development
1

In January 2021 (mid 3.10 development), Sergey Fodoseev and I raised the compile time (and runtime) SQLite version requirements for the sqlite3 standard library extension module to SQLite 3.7.15 (2012-12-12) or newer. That enabled us to clean up the code by getting rid of a lot of preprocessor conditionals and other version specific workarounds.

For Python 3.13 (to be released October 2024), I’m considering to raise the version requirements to either SQLite 3.8.11.1 (2015-07-29), 3.14.2 (2016-09-12), or 3.15.2 (2016-11-28). Preferably the latter.

AFAICS on DistroWatch, we should be fine with any of those SQLite versions, for all major distros[1].

Major clean-ups:

  • For SQlite 3.8.11.1, I can remove a lot of conditional code related to CTEs and deterministic functions.
  • For SQLite 3.14.2 I no longer need to implement two different variants of the trace callback.
  • For SQLite 3.15.2, I can remove special handling for legacy deterministic user function behaviour.

The diff stats look approx. like this:

commit 3c1c9cf589903a70348758cde48f9446e05ae2b4 (HEAD -> sqlite-raise-the-bar)
Date:   Fri Jun 16 22:33:55 2023 +0200

    Require SQLite 3.15.2

 Doc/library/sqlite3.rst                     |  2 +-
 Lib/test/test_sqlite3/test_userfunctions.py | 27 +++++++++------------------
 Modules/_sqlite/module.c                    | 14 ++++----------
 configure                                   | 20 ++++++++++----------
 configure.ac                                |  6 +++---
 5 files changed, 27 insertions(+), 42 deletions(-)

commit 561f826bb31582d4a15ce30be46d474bdd77e4a4
Date:   Fri Jun 16 22:28:52 2023 +0200

    Require SQLite 3.14.2

 Doc/library/sqlite3.rst             |  2 +-
 Lib/test/test_sqlite3/test_dbapi.py |  9 +++------
 Lib/test/test_sqlite3/test_hooks.py |  4 +---
 Modules/_sqlite/connection.c        | 39 ---------------------------------------
 Modules/_sqlite/module.c            | 18 ++++--------------
 configure                           | 20 ++++++++++----------
 configure.ac                        |  6 +++---
 7 files changed, 22 insertions(+), 76 deletions(-)

commit 69bc871a4f6b7a5a0c1aab9c5b99d0608fd1906f
Date:   Fri Jun 16 22:16:54 2023 +0200

    Require SQLite 3.8.11

 Doc/library/sqlite3.rst                     |  5 +----
 Lib/test/test_sqlite3/test_backup.py        |  2 --
 Lib/test/test_sqlite3/test_dbapi.py         | 66 +++++++++++++++++++++++++-----------------------------------------
 Lib/test/test_sqlite3/test_types.py         |  1 -
 Lib/test/test_sqlite3/test_userfunctions.py |  7 -------
 Modules/_sqlite/blob.c                      |  8 --------
 Modules/_sqlite/connection.c                | 20 --------------------
 Modules/_sqlite/module.c                    | 28 ++++------------------------
 configure                                   | 20 ++++++++++----------
 configure.ac                                |  6 +++---
 10 files changed, 43 insertions(+), 120 deletions(-)

Comments?

  1. RHEL-7.9 ships with SQLite 3.7.17 but is EOL in June 2024. ↩︎

5 Likes
2

Do it (with at least 10 characters to express that sentiment).

7 Likes
3

Now, also on GitHub:

4

SQLite 3.15.2 was released in Nov 2016, so I think that’s definitely a minimum requirement you can add.

BTW: Is there a general agreement on how old supported systems can / should be for CPython ? The manylinux project currently uses a 2014 baseline (well, most wheels built today are using this version), but that appears to be a bit far in the past.

1 Like
5

See here for some explanation & cross-references. In short: there’s a pretty long gap between RHEL 7 (that the manylinux2014==manylinux_2_17 is based on) and RHEL 8, which forms the basis of the next manylinux after that[1].

This got exacerbated by the death of CentOS-as-we-knew-it happening before manylinux had updated, leading to a long discussion of what the new baseline would be, implementing it, and projects generally hesitating to adopt it, not least because there’s still about 30% of overall package downloads (on non-EOL python versions!) that have a too-old glibc to move to manylinux_2_28 (though the EOL of Python 3.7 this month will substantially reduce that number).

  1. see that link on why I’m ignoring manylinux_2_24 here ↩︎

2 Likes
6

Thanks for the added context, @h-vetinari

Looking at DistroWatch, it seems that switching to glibc 2.28 is still going to take some time, esp. on embedded systems. But that’s just the glibc, which distros upgrade in a much more conservative way than other libs such as the SQLite ones… too bad there doesn’t appear to be a system using gblic 2.24 or 2.26 with long term support. That would provide a good midterm solution.

7

That’s what the Debian-based manylinux_2_24 was intended to be, but it turned out to be unusuable for many projects, because people were really unhappy with being forced to use GCC 6, which doesn’t support C++17, etc. At the time the situation looked like this:

manylinux version GCC version based on
2010 8.3 CentOS 6
2014 10.2 CentOS 7
2_24 6.3 :scream: Debian 9
2_28 11.1 AlmaLinux 8

All but 2_24 were based on RHEL-derivatives, where there are ABI-compatible(!) backports of newer compiler versions over the lifetime of the LTS distro. This is the reason why no other distribution is really suitable as a base for manylinux.

2 Likes