Lets make `zlib` required rather than optional to build CPython

Does anyone have strong objections to this? It simplifies our codebase to get rid of the conditionals and extra code we carry for builds in the absence of zlib. The Windows build has treated it as required since forever. The zlib is always available on every supported platform otherwise given how fundamental it is.

Tracked in Make zlib required on all platforms (simplifies code) · Issue #91246 · python/cpython · GitHub with a draft PR demonstrating the cleanup.

The PR drops 300 lines of code.

12 Likes

FWICS Gentoo has always enabled zlib, so no opposition here. I can imagine quite a few things unexpectedly breaking should someone build Python without zlib.

It seems to mostly be a hold over from the post-autoconf 90s when building on unix systems without third party libraries available. I assume the only way it happens now is likely accidental: When someone trys to build python and hasn’t installed their distro’s -dev headers for the common libraries they already have installed. We don’t need to support that.

3 Likes

In Nixpkgs python3 is a relatively early needed package for bootstrapping, however, zlib already seems to be included then so it should be fine.

1 Like

+1. zlib is ubiquitous nowadays.
Note that in this could enable, the future, things like compressed .pyc files.

1 Like

WebAssembly builds doesn’t have it yet, but it should be doable with the right amount of time and effort (according to No zlib in WASI · Issue #93819 · python/cpython · GitHub).

1 Like

One thing to keep in mind here is that requiring zlib means we’re also requiring that zlib be buildable without Python. Considering how fundamental zlib is, and how simple its build is, this is probably fine… but it does make it potentially harder to bootstrap new systems, especially ones that rely on Python for its build system.

6 Likes

You certainly would like to use something more efficient than zlib for that.

3 Likes

Is the expectation that the platform provide the zlib code, or would we ship it for any platform that lacks it?

We’d ship zlib for any platform that lacks it, as we already do for Windows.

3 Likes

In that case I don’t see WebAssembly being a blocker here as it would just be another extension module that got statically linked in.

1 Like

And WebAssembly is definitely not a blocker for this as shown by Make zlib required on all platforms (simplifies code) · Issue #91246 · python/cpython · GitHub , so I think the only blocker on this is someone doing the work.

2 Likes

I suspect that zlib is already effectively required anyways.

If you don’t have zlib the make fails by default (due to ensurepip), and a number of modules (like binascii) rely on zlib being present. I don’t think I’ve seen people guard against those modules being missing in my entire Python programming career.

2 Likes

If your platform supports pip, then yes. :wink:

Feel free to look in the stdlib then as we do in some places for WASI support.

1 Like

zlib has an as-of-now unresolved critical vulnerability. While that specific vulnerability will get fixed eventually, it’s not the first vulnerability in zlib and probably won’t be the last. It would be nice to maintain the ability to build Python without zlib for such circumstances.

Just for reference, the mentioned vulnerability (CVE-2023-45853) affects the MiniZip ‘contrib’ package, not zlib itself. As noted on the NVD page, NOTE: MiniZip is not a supported part of the zlib product. Python is not affected by it, and most other users of zlib probably aren’t either.

Security vulnerabilities should be assumed inevitable in every dependency (including Python); zlib is not special here.

11 Likes