…ah, I understand. The python:3.X-alpine have Python built by Python and have a -gnu suffix. The Python obtained by alpine:latest via apk add python3 has a -musl suffix and indeed the binary package doesn’t work:
/ # python3
Python 3.9.5 (default, May 12 2021, 20:44:22)
[GCC 10.3.1 20210424] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import psycopg
Traceback (most recent call last):
...
ImportError: no pq wrapper available.
Attempts made:
- couldn't import psycopg 'c' implementation: No module named 'psycopg_c'
- couldn't import psycopg 'binary' implementation: cannot import name 'pq' from 'psycopg_binary' (/usr/lib/python3.9/site-packages/psycopg_binary/__init__.py)
- couldn't import psycopg 'python' implementation: libpq library not found
Now I know better what’s going on, but I’m properly confused about how to proceed in regards to which package to distribute. I understand that
- the packages currently released now work on current python-alpine docker images but will break as soon as BPO 43112 will be fixed.
- after it’s fixed, it will be possible to build new packages using newly released cibuildwheel and related images, but the new packages will not work on python:3.10.0-alpine, python:3.9.9-alpine and previous ones
- we can’t release both the images because they have exactly the same tag.
Uhm… what’s the best course of action?
It is worth noticing that the problem is only in the name of the library: creating a symlink to the existing library is enough to import the module (and I tested it works):
(.venv) piro@baloo:~/dev/psycopg3$ d run --rm -ti --volume `pwd`:/src --workdir /src -e PSYCOPG_TEST_DSN -e PGHOST=172.17.0.1 -e PGUSER=`whoami` alpine sh
/src # apk update
...
OK: 14942 distinct packages available
/src # apk add python3 py3-pip tzdata
...
OK: 80 MiB in 53 packages
/src # pip3 install psycopg[binary]
/src # pip3 install -U pip
...
Successfully installed pip-21.3.1
/src # pip install psycopg[binary]
...
Successfully installed psycopg-3.0.4 psycopg-binary-3.0.4
/src # cd /usr/lib/python3.9/site-packages/psycopg_binary
/usr/lib/python3.9/site-packages/psycopg_binary # ln -s pq.cpython-39-x86_64-linux-gnu.so pq.cpython-39-x86_64-linux-musl.so
/usr/lib/python3.9/site-packages/psycopg_binary # ln -s _psycopg.cpython-39-x86_64-linux-gnu.so _psycopg.cpython-39-x86_64-linux-mus
l.so
/usr/lib/python3.9/site-packages/psycopg_binary # ls -l
total 1584
-rw-r--r-- 1 root root 421 Nov 23 00:11 __init__.py
drwxr-xr-x 2 root root 4096 Nov 23 00:11 __pycache__
-rw-r--r-- 1 root root 1267896 Nov 23 00:11 _psycopg.cpython-39-x86_64-linux-gnu.so
lrwxrwxrwx 1 root root 39 Nov 23 00:13 _psycopg.cpython-39-x86_64-linux-musl.so -> _psycopg.cpython-39-x86_64-linux-gnu.so
-rw-r--r-- 1 root root 2596 Nov 23 00:11 _psycopg.pyi
-rw-r--r-- 1 root root 332640 Nov 23 00:11 pq.cpython-39-x86_64-linux-gnu.so
lrwxrwxrwx 1 root root 33 Nov 23 00:13 pq.cpython-39-x86_64-linux-musl.so -> pq.cpython-39-x86_64-linux-gnu.so
-rw-r--r-- 1 root root 0 Nov 23 00:11 py.typed
-rw-r--r-- 1 root root 251 Nov 23 00:11 version.py
/usr/lib/python3.9/site-packages/psycopg_binary # cd -
/src # python3
Python 3.9.5 (default, May 12 2021, 20:44:22)
[GCC 10.3.1 20210424] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import psycopg
>>> psycopg._cmodule._psycopg
<module 'psycopg_binary._psycopg' from '/usr/lib/python3.9/site-packages/psycopg_binary/_psycopg.cpython-39-x86_64-linux-musl.so'>
/src # pip install -e ./psycopg[test]
Obtaining file:///src/psycopg
...
Successfully installed attrs-21.2.0 coverage-6.1.2 importlib-metadata-4.8.2 iniconfig-1.1.1 mypy-0.910 mypy-extensions-0.4.3 pluggy-1.0.0 pproxy-2.7.8 psycopg-3.0.5.dev0 py-1.11.0 pytest-6.2.5 pytest-asyncio-0.16.0 pytest-cov-3.0.0 pytest-randomly-3.10.2 tenacity-8.0.1 tomli-1.2.2 typing-extensions-4.0.0 zipp-3.6.0
/src # pytest -m not\ slow
======================================================= test session starts ========================================================
platform linux -- Python 3.9.5, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
Using --randomly-seed=4128270608
libpq wrapper implementation: binary
libpq used: 120009
libpq compiled: 120009
rootdir: /src, configfile: pyproject.toml, testpaths: tests
plugins: randomly-3.10.2, asyncio-0.16.0, cov-3.0.0
collected 3376 items / 347 deselected / 1 skipped / 3028 selected
tests/types/test_composite.py ................................................. [ 1%]
tests/types/test_bool.py ............... [ 2%]
...
maybe we should fix manylinux/cibuildwheel to create a symlink for “the other tag”, whichever it is?