On behalf of the PyPA, I am pleased to announce that pip 19.0 has just been released. This release brings support for multiple new packaging standards.
The highlights of this release are:
Python 3.4 support is deprecated.
PEP 517 support is now implemented.
manylinux2010 wheels are now supported.
Dependency links support has been removed.
Many bug fixes and lots of minor improvements.
In addition, when run on Python 2.7, pip 19.0 prints a warning regarding the upcoming 2020 EOL of Python 2.7. A future version of pip will drop support for Python 2.7.
To install pip 19.0, you can use get-pip (as described in 1) or run:
python -m pip install --upgrade pip
Note that if you are using a version of pip supplied by your distribution vendor, vendor-supplied upgrades will be available in due course.
The pip development team is extremely grateful to everyone in the community for their contributions. Thanks to everyone who put so much effort into the new release. Many of the contributions came from community members, whether in the form of code, participation in design discussions and/or bug reports.
I think I just saw another message indicating there was something dramatically wrong with this release.
I also strongly believe that pip should not be the stick to encourage people to leave Python 3.4 or 2.7 behind. To be clear, I am fine with newer versions of pip not supporting older versions of Python – however, pip tends to aggressively encourage upgrading, and this can leave users with two bad options: they can either not upgrade pip and live forever with its misguided warnings, or they can upgrade pip and risk being broken, with no easy way to back out of this situation.
I think it would be better if for each Python version that reaches its EOL, there was a final pip release supporting that Python version that did not emit any warnings suggesting an upgrade (neither of pip nor of Python).
Presumably users who don’t upgrade their Python version have made that choice understanding the consequences – and if they don’t, I doubt a warning from pip will sway them. This is especially true for Python 2.7: few folks are willingly staying there, but for some of us the migration will still take years. I don’t mind not being able to upgrade pip or not being able to use newer pip features – but I do mind getting non-actionable warnings every time I use pip, and I am not looking forward to the day when pip stops working because some part of the PyPI infrastructure stops supporting pip running on Python 2.7.
There is a regression happening for some projects along with workarounds to resolve it, it’s currently being discussed what the right short and long term fixes are. That discussion is trying to determine ways to ensure we catch those regressions before a release in the future.
Generally speaking, modern versions of pip will not let you upgrade to a version of a package that does not claim support for the version of Python you’re trying to install into. This is an opt-in feature on the package author’s side, but pip itself does use this feature. The practical impact of this means that even if you try to upgrade to a version of pip that does not support your version of Python, pip itself won’t allow it (unless you’re on a sufficiently old version of pip). This means the upgrade warning for pip itself is, at worst, a misleading nuisance. I’ve gone ahead and opened pypa/pip#6211 which covers making sure that the prompt to upgrade pip only fires if the newer version of pip actually supports the current version of Python being ran.
With regards to the warning about 2.7 itself, that’s a bit in the air currently. We have some people arguing that pip should be aggressive in dropping support for 2.7 as quickly as possible, while others have argued that pip should remain conservative. That discussion is ongoing in the previously linked thread but the warning added to pip 19 represents a sort of middle ground for both of those factions. It doesn’t drop support for 2.7 (yet), but it indicates to people that dropping support for 2.7 is on the horizon for pip, we just don’t know exactly when yet. I wrote that message, so it’s entirely possible the wording is poor and could use improvement. Certainly there is the case of PyPy which has no plans to end-of-life their version of 2.7 where this message is being emitted and causing confusion (see pypa/pip#6207).
Note that in both of these cases, you can personally disable the warning. The first way is to use --disable-pip-version-check (which like all pip options can be specified on the CLI, as an environment variable, or through several config files) which will prevent pip from checking if there is a newer version of itself. The version warning uses the Python warning module, and can be disabled using an environment variable like PYTHONWARNINGS=ignore:Please.upgrade::pip._internal.cli.base_command. That’s not really a great experience for disabling that message, largely because historically we’ve not considered dropping support for versions of Python until they were well on their way to dead so it never really came up as an issue before. There is pypa/pip#6119 about unifying the way our warnings are emitted and providing a generic way to silence specific warnings (ala flake8 style IDs).
It’s incredibly unlikely that 2.7 stops working as you’ve described. For the client side of the equation (pip, setuptools, etc) you’ll be stuck on whatever the last version of those tools to support 2.7 was, but as the end user you have complete control over those. The only part you don’t really have complete control over is PyPI itself, which will effectively never drop support for 2.7, because PyPI doesn’t treat 2.7 any differently than 3.7, it’s all just opaque files with some metadata attached to it as far as PyPI is concerned. You can still upload Python 2.4 supporting libraries to PyPI and, assuming your toolchain supports it, it’ll still work completely fine.
Just for kicks, I installed the absolute oldest version of pip that was still on PyPI (from 2008!) and it worked just fine on 2.7, with only needing to change the URL it accessed (because we switched URLs during the migration to Warehouse). I don’t see any reason why in 2029 pip 19 wouldn’t still continue to work with PyPI just fine.
The majority of the issues we’re currently seeing is with users who are simply getting the latest version of pip (typically in their CI), and so being immediately hit by any issues when pip does a new release. I’d argue that such users might be better taking a more measured approach, and explicitly stating which version of pip they are using and upgrading to their own timescales - just like with any other component of their development stack.
But yes, the discussion Guido pointed to (which I wasn’t aware of - I’m not sure yet how to get notified of new threads on Discourse without simply switching on emails for everything) is what we want to look at here. We’ve traditionally had no success getting any community testing of pip pre-releases (to the point where we no longer even bother with them typically) and so we’re at a huge risk of getting caught by scenarios our test suite doesn’t cover. Better integration testing would be a major help here.
If you go to a category page, like Packaging - Discussions on Python.org, in the top right, near the “New Topic” button lets you choose your notification level for that whole category. I have the packaging category set to “Watching the first post”, which will send me an email for each new topic, but subsequent replies to that topic will follow the typical pattern of depending on what level I set notification to for each topic.
Have you see the pip’s own advice on every installation using pip? It suggests us to do
pip install --upgrade pip
The default is for the project to encourage the upgraded version of pip.
@dstufft is only making a point that people can choose to ignore the defaults, and customize the behavior of pip.
I think, encouraging users to go with the upgraded version is a good thing.
Many users, tried to use the updated version of pip (by doing pip install --upgrade pip) in CI, and encountered failures, when they discovered pip 19.0 breaking compatibility with the older version.
Ways to solve would be to have
Deprecation/Warning for the features that were going away. There were user complaints that some features that were removed in 19.0 did not have adequate deprecation warning (like --follow-dependency-links?)
Having a --compat mode with a old version of pip, and still encouraging the upgrade.
Hm. I recall a recent event where a switch in PyPI to mandate a newer version of TLS required upgrades to pip, and another time when the infrastructure was renamed (IIRC from pypi.python.org to pypi.org). In the future such changes will have to take the last pip version that works on Python 2.7 into account.
But for many users even changing that URL in the configuration will cost them a trip to Google. It would be much better if the defaults kept working without undue warnings.
I do understand where you’re coming from, though, and I don’t feel the need to keep arguing. I just want to make sure you’re aware of the effort it takes for folks like myself: I don’t have much understanding of how pip works and I would probably ending up on StackOverflow looking for how to change its configuration – if it even occurred to me that certain things can be configured.
I almost added a bit in there about TLS, and ended up removing it. The thing you’re thinking about was actually an issue with the incredibly ancient version of the OpenSSL library that macOS shipped on certain versions of their OS (at the time, every version but the absolute latest) and upgrading pip “fixed” it because we added a fallback mechanism that would use SecureTransport (a macOS system library with the same purpose as OpenSSL).
It is possible some other situation like that would arise of course, I suspect if it did and 2.7 still had significant usage and the latest version of pip no longer supported 2.7, we’d release a new version of whatever the last 2.7 supporting version was that had a similar fallback mechanism in it if possible.
Yea. The old URL sort of still works, it’s just if you get extremely old versions of pip they used a http:// URL instead of a https:// URL, which we could not securely redirect to the new, proper location so we purposely broke that. If I had used pip 1.3+ (circa 2013) instead of pip 0.2 (circa 2008) then it wouldn’t have required that configuration either, even though pip 1.3 still used https://pypi.python.org/simple/.
Oh I don’t consider it arguing! I’m just trying to make sure that it’s clear that regardless of what happens, people are unlikely going to be locked out of the wider packaging ecosystem due the version of Python they choose to operate. The worst that should happen is you will be stuck only using the tools/libraries/etc that support that particular Python.
Yes, I have. I’m ambivalent about it - I’m a pip developer and I’m well aware of both the problems caused by people not upgrading, and the problems caused by people blindly upgrading. Neither is ideal
We didn’t break compatibility. we simply had a bug. The intention was 100% that the new code was compatible.
Dependency links have been deprecated for literally years. So long that the deprecation process was a mess, precisely because we’d tried to remove the feature (after a deprecation period) multiple times, and had to revert, so people became immune to the warning.
For the PEP 517 change, --no-use-pep517 is the compat mode. Sadly, the fact that there was an unexpected bug means it’s needed too often to be a viable workaround, and that’s why we’re looking at some additional mitigation.
The problems we’re seeing are (apparently) not covered by our tests. That’s a problem, certainly. We’d be more than happy to see additional tests being contributed. The release (and every PR that was part of the release) passed the tests. But I doubt our test coverage is even close to the huge number of usage scenarios we see. The proposed integration test suite would be a big help here. As would users doing pre-release testing, but that’s always hard to organise.
The situation with this release isn’t ideal, certainly. But we’re doing what we can…
Sorry, I wasn’t clear. That’s about what emails I get. I have all emails switched off, I check for new discussions from within Discourse itself (I don’t particularly see a point in relying on emails to get notifications from a tool that’s meant to replace email, but that’s just my personal preference). What I meant was that when I checked “New” and “Unread” in Discourse, I didn’t see this topic in either of them, and there’s no clear “summary list of items in date order with unread stuff highlighted” (or if there is, it doesn’t “feel” like that - a lot of the Discourse UI is very dynamic, which is good in one way, but not so much when looking for a “home base” sort of view). But this is off topic, it belongs in one of the “How to use Discourse” threads, so I’ll leave it at that.
