Is there an intermediate solution? For example, keep the full warnings in for editable install and/or local directories, but display only a summary on installs from PyPI? Brainstorming:
No wheels available for package
ab. Built fromsdistwith 4 warnings (see/tmp/something/ab-logs.txtfor details).
The advantage of reaching the end user is that they can be advised in the case they are relying on an abandoned project. But yeah there is cost-benefit trade-off to be analysed here.
I think there are 3 or 4 easy to seperate categories where these warnings might be present:
pip install -e {local directory})pip install {local directory})pip install {vcs backed url})Sdists could also be broken down into where it was collected from (remote index, local index, directly specified) but that would get more complex and as I donāt imagine that information is currently available when pip calls the build backend.
Pip could have some suppression/summary logic for some of these categories, but not others, immediately it would seem to show full warnings on sourcetrees (1-3) but not sdists (4), with a message to say enable verbose mode to see full output from build backend.
I have two mild concerns, firstly this is more complex than passing on the warnings, someone will need to make the PR to implement this logic, secondly there could be warnings from a built sdist that are not present in the source tree, depending on how the final sdist is built or modified before publishing and Iām not sure if this is something we should worry about or not.
Ok, so this sounds complex and I donāt want to push this into frontends.
In my mind verbose warnings is better than no warnings. I would still take it over nothing.
Everything I discussed above is on the realm of brainstorm.
Rather than @notatallshawās idea of classifying warnings by the type of source, maybe we could only display warnings for top-level requirements by default? This is on the assumption that dependencies are less likely to be something the user can fix. We could add a message āSome of your dependencies issued warnings when building - to see those warnings, use the -v optionā.
While the idea of using the -v option to see warnings is good at first glance, there might be a practical issue. If users simply repeat the pip install ... command with the -v flag, they may not see the warnings if the package is already installed, as the installer wonāt go through the process again. Possibly, the message should include additional flags to force reinstalling, such as --force-reinstall?
In my opinion, the simplest approach would be to show all warnings by default. I know this is far from ideal, but it has potential to be effective, as at least it ensures that end-users at least have information available about any issues with their dependencies. This is crucial because dependencies built from sdists can break the installation process, and at that point it does not matter if the dependency is direct or transitive[1]. So increasing awareness before things start to break is good.
I believe that there are 3 main actions that end-users can take when facing a warning[2]:
PIP_CONSTRAINT or similar methods; cached local wheelhouse, etc).Options 1 and 2 are equally applicable regardless of whether the dependency with warnings is direct or transitive. Option 3 is a bit harder to do for transitive dependencies, as the end user would need something like pipdeptree to find the root of the particular dependency chain and then analyze if that one is replaceable.
Hopefully, by showing all the warnings we can (try to) minimize the feeling ābeing caught by surpriseā when if the userās pipelines fail[3].
I have discussed in this thread other options as brainstorm, but I think that they are much more complicated to implement and add more maintenance cost to frontends. So overall, showing all warnings looks like a good compromise to me. ā©ļø
Potentially moreā¦ ā©ļø
We are never going to be able to eliminate it because it is impossible to guarantee all the users read the CI logs, but we can do our best to communicate. ā©ļø
Iāve also come around to this thinking because of the recent issues with wheel.
Specifically because the problem can be related to the user environment, and not the dependency itself. For example if you have some older build backend that is cached or provided by your Python distribution, but it has a dependency, and that dependency isnāt cached, you can end up with build backend dependencies that are not matching, and that can cause build failure.
That kind of scenario would be specific to a users environments, and independant to building a dependency or a transitive dependency.
Just in case anyone tries this itās PIP_CONSTRAINT not PIP_CONSTRAINTS, Iāve made this typo before and been confused.
Thank you very much Damian (and thank you for dedicating time on this). I have fixed the name of the env var on my message to avoid misleading readers.
While I donāt completely disagree, I think users will want to be able to suppress the warnings (ideally, once they have reported the problem, but we know life isnāt always that good 
). Iām not sure how such a suppression should work - would front ends just suppress all backend warnings? I donāt think we want to get into the situation where frontends need a complex warning selection UIā¦
By the way, whatās uvās view on this? Ping @charliermarsh. I donāt want this to be just something pip doesā¦
I think itās hard to speculate right now because we donāt know which users and why. Most users and packages should be using wheels and not encounter this, there are some legitimate use cases for requiring sdists, but perhaps this will encourage many packages that just havenāt got around to building wheels to start build wheels.
Advanced users will be able to suppress build warnings without any need to additional UX from pip, with something like setting PYTHONWARNINGS=ignore::BuildBackendWarning or calling pip like python -W ignore::BuildBackendWarning -m pip ... (it might need the fully qualified location of the warning?). I would be in favor of including this in the release notes and/or docs.
This is all speculation, I agree. But we need to be cautious with how we use our āchurn budgetā, and a bad design choice right on the tail of recent issues might be too much for our end users.
I was assuming this might not work, because the build backend is run in a separate process, so the -W flag wonāt get passed through (Iām less sure about the environment variable, is it one we let through to the child process?) But regardless, Iām not convinced thatās sufficient for the āIāve reported the problem back to the dependency thatās triggering the warning, now I just want to get on with my life until they fix itā scenario.
Of course, if weāre only displaying warnings for top-level requirements, as I suggested above, that might not be an issue, as the user should be in a position to fix the problem themselves (unless they are doing something like installing a project from PyPI that doesnāt publish wheels).
Iām less sure about the environment variable, is it one we let through to the child process?
It has worked in the past, and in one of my projects I used to maintain a mile-long PYTHONWARNINGS export in my test environments so I could hide specific known DeprecationWarnings from sdist building of dependencies by matching on their message strings and then abort on unrecognized warnings so they didnāt slip by unnoticed in CI jobs.
Unfortunately, the way these have to be serialized causes some to just be impossible to filter out, like if they contained : or , as those are unconditionally treated as first and second order field separators in the envvar. Dealing with warnings that use exceptions private to some module is also challenging, though sometimes filtering on the parent exception class is good enough.
Granted itās been a year or two since I was trying to keep that up, maybe the situation has changed in the meantime.
I havenāt had a chance to read the entire thread (I read the first few dozen posts), but my concern with showing build backend warnings to users is that the warningsā¦ arenāt really actionable? In other words, as a consumer of a package, thereās nothing I can do to resolve the warning (apart from notify the package author ā but then I go back, run my command again, and see the warning again). So it does seem likely that users would view this as a nuisance and look to turn them off. Using the setuptools case as an example, if we showed a warning every time users attempted to build requests from source, I think users would find that fairly tedious. My general philosophy on has been that we should only show warnings when theyāre somehow actionable, for this reason. (I donāt have any good answers here and am not saying anything that hasnāt already been expressed; just sharing my reaction.)
(Separately, we do plan to add support for locking build dependencies this year.)
I can imagine a few situations where itās actionable:
build-backend v100ā users can constrain (or lock) build-backend<100 until the package author fixes it.Presumably if those warnings had been displayed, requests would have received enough reports to fix it, or setuptools enough reports to delay switching over, which would have prevented the outage and got rid of the warnings.
I understand some/many/most users would turn them off, but thatās making an active choice to ignore the risks, and of course it is a risk to be building sdists instead of installing wheels, as shown by this outage. And users have very little way to know that right now.
All that said, I appreciate spewing a bunch of warnings is not a good experience for new users. But I do wonder how many new users are installing sdists?
How often do you build third-party software by hand? Is it really a ānuisanceā if you get a warning from time to time?
How often do people build requests from source exactly? What is the use case?
I understand your perspective but isnāt the fact that the setuptools change was so massively disruptive a sign that users are building source distributions? If this was rare, it wouldnāt have caused breakages for so many users. You can skim through any of the linked issues and find packages that caused users problems: ansible-vault, gputil, aiosonic (0.15.1), json-merge-patch (0.2.0), direct-sdk-python3, etc. These are all packages that donāt ship wheels at all. (The other use-cases we see for building from source, e.g., companies that want to build everything themselves from source, are rarer.) I assume theyāre building (in some cases) on every CI run, and then whenever users run on a new instance, clear their build cache, etc.
Yeah, this is pretty reasonable. I donāt know, there are definitely tradeoffs here, but Iām coming around to the idea that itās preferable to show these. (If we did show them, weād probably expose a single environment variable to turn off all build backend warnings.)
Regarding the warnings, PYWARNINGS are not supressed by pip when calling the build backend, testing on this branch I am able to suppress all warnings:
$ PYTHONWARNINGS="ignore" pip install --dry-run --no-cache --no-deps alembic-utils==0.8.6
Collecting alembic-utils==0.8.6
Downloading alembic_utils-0.8.6.tar.gz (22 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Would install alembic_utils-0.8.6
However, I am a bit confused how to supress this specific warning, I tried:
$ PYTHONWARNINGS="ignore::pip._vendor.pyproject_hooks.BuildBackendWarning" pip install --dry-run --no-cache --no-deps alembic-utils==0.8.6
Invalid -W option ignored: invalid module name: 'pip._vendor.pyproject_hooks'
...
And then all the warnings display, when I look at the warnings code Iām not able to reproduce the error in the REPL, the line m = __import__(module, None, None, [klass]) works fine in the REPL and imports pip._vendor.pyproject_hooks. Someone who has more understanding of the warning system would have to explain if itās possible to suppress this specific warning.
Itās definitely trade offs, when this is first turned on there will certainly be users who receive a lot of warnings who can not do anything about it other than suppress them.
I would be happy to start limited and then if there isnāt significant push back turn it on for all builds, as a hypothetical timeline for pip: turn on warnings for building source trees this year, then any direct user requested package next year, then all dependencies the year after that.
My thinking is that by the time most users might see this, hopefully the common warnings have been fixed by the package maintainer, and users are more likely to see warnings only if they truly requires user action (such as adding appropriate build constraints).
Very strong +100 on this one. I donāt think frontend has any business to censor backendās communication with user, especially if -v is used.
Discussion and work is being done to enable this, see: Enable forwarding warnings from subprocess to the caller Ā· Issue #157 Ā· pypa/pyproject-hooks Ā· GitHub
Iām fairly sure every installer tool shows you the full output of the build backend if you pass -v, pip certainly does, e.g. pip install -v --dry-run alembic-utils==0.8.6.
Just as a brainstorm/out of curiosity, I did a little experiment to see what is necessary to forward the -W options to the subprocess:
# > docker run --rm -it python:3.10-bookworm /bin/bash
mkdir /tmp/test && cd /tmp/test
cat <<EOF > wrapper.py
import sys
import subprocess
from itertools import chain
from pprint import pprint
print("\n\n--- wrapper ---")
pprint(sys.warnoptions)
args = [
sys.executable,
*chain.from_iterable(("-W", opt) for opt in sys.warnoptions),
*("-W", "default:not given by user"),
"_subprocess.py"
]
print(subprocess.check_output(args, text=True))
EOF
cat <<EOF > _subprocess.py
import sys
from pprint import pprint
print("\n\n--- _subprocess ---")
pprint(sys.warnoptions)
EOF
PYTHONWARNINGS='once:via env var' python -W 'error:hello' -W 'always:world' wrapper.py
This is the output:
--- wrapper ---
['once:via env var', 'error:hello', 'always:world']
--- _subprocess ---
['once:via env var', 'error:hello', 'always:world', 'default:not given by user']
So it seems that doing the following may be enough:
subprocess.run([
sys.executable,
*chain.from_iterable(("-W", opt) for opt in sys.warnoptions),
...
])I am sorry, but you are wrong. See How to get C compiler stderr when building C extension via pip wheel or via build?
and https://paste.opensuse.org/pastes/fac9775b86fa (code available on ~mcepl/m2crypto - crypto and SSL toolkit for Python - sourcehut git).
Making to see C compiler warnings and messages is close to impossible. python3 -mpip wheel --verbose --no-cache-dir --no-clean --no-build-isolation --wheel-dir dist/ --editable . is still the best what I managed to do, but when you look at build #1495354 - failed, I still do not see much. The only way how to see all messages is to use python3 setup.py build -v, but that we are not supposed to use any more.
Best,
MatÄj