In aiohttp and family, we provide optional C Extensions with Pure Python fallback.
Now we have a setuptools custom
build_ext command which detects compilation errors and silently switched to pure-python installation.
The problem is:
- Performance without C Extensions is significantly worse than in normal case (e.g. C written HTTP parse is much faster than pure-python one).
- We provide MacOSX, Windows, and manylinux binary wheels but many people use Alpine docker images to install aiohttp. manylinux is not Alpine compatible because Alpine uses musl instead of glibc.
- People do
pip install aiohttpon a platform that has no compatible binary wheel published on PyPI and Compilation Toolset is not available (gcc is missing,
Python.his absent etc.)
- aiohttp install succeeds but it switches to pure-python fallback version.
- it can hurt the performance of the production server easy without any notice, people should pay for more powerful servers but they can avoid it if notified.
I’d like to fail loudly if installation on CPython implementation, PyPy works fine with pure python.
On the other hand, I’d like to provide a way to explicitly say: “Ok, I know that my environment cannot compile C Extension but I can live with it”.
For aiohttp, C Extension compilation should be default but opt-out.
--install-option <options> for
pip install command, I can utilize it and parse in
I can raise an error with the suggestion to use
--install-option --no-extensions to enforce pure python version.
The problem is:
--install-option is not supported by *requirements files`.
Would pip maintainers consider adding it?
Another solution is using environment variables but it looks ugly: currently, we have
multidict libraries in our stack, all three have optional C Accellerators:
NO_AIOHTTP_EXTENSIONS=1 NO_MULTIDICT_EXTENSIONS=1 NO_YARL_EXTENSIONS=1 pip install -r requirement.txt
Sorry, too many variables and again: no way to pin this information inside requirements file.
Providing two packages like
aiohttp-slow is not an option too:
a user should decide what version to use, not an author or third-party library.
The third-party should have just
aiohttp in dependencies. Recall
psycopg2-binary famous problem.
It would be nice if pip has an easy and obvious solution to this.
Did I miss something?