Why python in Debian Docker image is faster than official Docker's python image?

Python Help
1

Hi,
Debian’s python installed inside a docker container is faster than the official python from docker. How is it possible ?

# Dockerfile-debian
FROM debian:sid
RUN apt update && apt install -y python3.12

# Python code for benchmark : test.py
import sys
import timeit
def test_function():
    numbers = list(range(1000))
    [num ** 2 for num in numbers]
benchmark_result = timeit.timeit(test_function, number=int(sys.argv[1]) )
print(benchmark_result)

Results

$ docker build -f ./Dockerfile-debian -t python_debian .
$ docker run -v .:/app python_debian python3.12 /app/test.py
# 0.7484053150001273

$ docker run -v .:/app python:3.12 python3.12 test.py
# 1.1680918899983226

How does debian manage to have better performances than the official python docker image ?

I’ve tried to build python using the same build args than debian, but I’ve ended with an even slower binary.

Debian build args 
'--enable-shared'
'--prefix=/usr'
'--libdir=/usr/lib/x86_64-linux-gnu'
'--enable-ipv6'
'--enable-loadable-sqlite-extensions'
'--with-dbmliborder=bdb:gdbm'
'--with-computed-gotos'
'--without-ensurepip'
'--with-system-expat'
'--with-dtrace'
'--with-ssl-default-suites=openssl'
'--with-wheel-pkg-dir=/usr/share/python-wheels/'
'MKDIR_P=/bin/mkdir -p'
'CC=x86_64-linux-gnu-gcc'
2

Are you referring to https://hub.docker.com/_/python? These are “Docker Official Images”, not official Python images. The images are provided by Docker Community, not by the PSF or Python community.

1 Like
3

A while ago debian started to build python without using a libpython.
They claimed that that change made starting python faster.
May be that is the difference?

Otherwise it could be differences in compiler optimisation options used.

4

Yes, I’m referring to the Python image from the Docker Hub. Sorry, I thought it was made by PSF or the Python community.

5

The “benchmark” I’ve used does not measure python boot time, only code execution. The difference is not here.
I’ve checked python build args using sysconfig.get_config_vars('CONFIG_ARGS')
Debian ones :

'--enable-shared'
'--prefix=/usr'
'--libdir=/usr/lib/x86_64-linux-gnu'
'--enable-ipv6'
'--enable-loadable-sqlite-extensions'
'--with-dbmliborder=bdb:gdbm'
'--with-computed-gotos'
'--without-ensurepip'
'--with-system-expat'
'--with-dtrace'
'--with-ssl-default-suites=openssl'
'--with-wheel-pkg-dir=/usr/share/python-wheels/'
'MKDIR_P=/bin/mkdir -p'
'CC=x86_64-linux-gnu-gcc'

Official docker’s python:

'--build=x86_64-linux-gnu'
'--enable-loadable-sqlite-extensions'
'--enable-optimizations'
'--enable-option-checking=fatal'
'--enable-shared'
'--with-lto'
'--with-system-expat'
'--without-ensurepip'
'build_alias=x86_64-linux-gnu'

Note that debian does not use --enable-optimizations

6

I’ve published the full benchmark I’ve made here : https://github.com/fabien-michel/compare-python

It compare python from different Linux distributions using docker and local Python.
Here are the results on my machine which run Arch Linux: (percent are diff with official Docker’s Python)

debian-on-docker    	0.7246799930017005	-34.072282393951454%
ubuntu-on-docker    	0.7611594549998699	-30.75356559169216%
arch-on-docker      	0.9717710979984986	-11.593184377570633%
local               	1.035610535998785	-5.785395448310301%
fedora-on-docker    	1.035977895000542	-5.751974985814081%
official-docker     	1.099203824000142	0.0%
7

What matters are the C compiler used, gcc I would expect, and the C compiler options.

Even if --enable-optimisations is not passed to python configure script that does mean there are no optimisations. Its typical to pass them in env vars like CFLAGS and LDFLAGS.

Do you know what they where when python was compiled?

1 Like
8

Do you know what they where when python was compiled?

This is the makefile used by Debian to build the Python 3.11 package
in their bookworm (current stable) release:

I assume that’s what the OP installed in their container anyway.

1 Like
9

And this is the key line

DPKG_CFLAGS  := $(shell $(dpkg_buildflags) --get CFLAGS)

you will need to know what that evaluates as.

10

Ok thanks for your search. Python seems to store all config variables. It cam be retrieved using sysconfig.get_config_vars().

Here CFLAGS et LDFLAGS from debian’s python:

    "CFLAGS": "-fno-strict-overflow -Wsign-compare -DNDEBUG -g -O2 -Wall",
    "LDFLAGS": " -Wl,-z,relro -g -fwrapv -O2   ",

And CFLAGS and LDFLAGS from docker’s python:

    "CFLAGS": "-fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall",
    "LDFLAGS": "",

I’m really surprised there is so much difference in build options across distributions.

11

No, your answer wrong. --enable-optimization is used for PGO builds, which makes Python about 10% faster. PGO cannot be enabled by passing different CFLAGS and LDFLAGS alone. In PGO mode, make first compiles an instrumented build of Python, then runs some tests to create optimization profiles, and finally compiles Python a second time. You trade a slower build for a faster Python.

You can get the actual flags for Python interpreter core from sysconfig.get_config_vars("PY_CORE_CFLAGS") and sysconfig.get_config_vars("PY_CORE_LDFLAGS"). Fedora and RHEL use additional flags like -fno-semantic-interposition, which can increase performance by up to 30% with Python 3.8. AFAIK speedups are less impressive in 3.12.

1 Like
12

Thanks for the correction, I’d forgotten about the PGO stuff.