# Packaging and Python 2 **URL:** \ **Category:** Packaging\ **Created:** [January 11, 2019, 4:07am UTC](https://discuss.python.org/t/packaging-and-python-2/662 "2019-01-11T04:07:43Z")\ **Posts on this page:** 20\ **Page:** 7
**Author:** ![methane](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/methane/32/43_2.png) [@methane](https://discuss.python.org/u/methane)\ **Post date:** [June 14, 2019, 6:42am UTC](https://discuss.python.org/t/packaging-and-python-2/662/121 "2019-06-14T06:42:19Z")
I tried this query: ```auto SELECT details.system.name, details.distro.name, details.distro.version, details.implementation.version, details.installer.name, COUNT(*) AS count FROM [the-psf:pypi.downloads20190612] WHERE file.project = "awscli" GROUP BY details.system.name, details.distro.name, details.distro.version, details.implementation.version, details.installer.name, ORDER BY count DESC ``` The result is here: > **[awscli downloads (20190612)](https://docs.google.com/spreadsheets/d/1QSTwYYtTc4qxw9MPEYteyiSxxSGpU2v9PXkcECDSUHM/edit?usp=sharing)** > > results-20190614-152520.csv > > details\_system\_name,details\_distro\_name,details\_distro\_version,details\_implementation\_version,details\_installer\_name,count > Linux,,,2.7.16,pip,210247 > Linux,Ubuntu,16.04,2.7.12,pip,177231 > Linux,Ubuntu,16.04,3.6.8,pip,74850 > ... Oh, “null” is the most major distro! I don’t know what environment are included in “null”. Ubuntu 16.04 is the second most major distro. But I believe Ubuntu 18.04 will overtake it at some point. ---
**Author:** ![methane](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/methane/32/43_2.png) [@methane](https://discuss.python.org/u/methane)\ **Post date:** [June 14, 2019, 10:30am UTC](https://discuss.python.org/t/packaging-and-python-2/662/122 "2019-06-14T10:30:45Z")
I found very interesting comment. > For CLI v2 we’re planning on distributing standalone installers for all platforms to avoid these and other issues. > [Do not recommend `sudo pip` · Issue #4175 · aws/aws-cli · GitHub](https://github.com/aws/aws-cli/issues/4175#issuecomment-497881116) ---
**Author:** ![pganssle](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pganssle/32/245_2.png) [@pganssle](https://discuss.python.org/u/pganssle)\ **Post date:** [June 14, 2019, 11:03am UTC](https://discuss.python.org/t/packaging-and-python-2/662/123 "2019-06-14T11:03:40Z")
> [@njs](#): > > So uh… anything you can share about how AWS is going to get people off py2? It’s kind of single-handedly holding volunteers hostage for supporting py2. Not that I don’t share this goal, but IIRC Donald has 20% of his time allocated to OSS, so Amazon _is_ one of the few companies paying people to support packaging. 😉 Of course, the burden spreads further than Donald, but I like to take every opportunity I can to recognize companies that are actually doing The Right Thing™ and supporting OSS. 😄 ---
**Author:** ![nad](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/nad/32/51_2.png) [@nad](https://discuss.python.org/u/nad)\ **Post date:** [June 14, 2019, 5:41pm UTC](https://discuss.python.org/t/packaging-and-python-2/662/124 "2019-06-14T17:41:44Z")
> (Especially since py2 doesn’t ship with pip.) ( ?? → [27.2. ensurepip — Bootstrapping the pip installer — Python 2.7.18 documentation](https://docs.python.org/2/library/ensurepip.html)) ---
**Author:** ![njs](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/njs/32/204_2.png) [@njs](https://discuss.python.org/u/njs)\ **Post date:** [June 14, 2019, 8:44pm UTC](https://discuss.python.org/t/packaging-and-python-2/662/125 "2019-06-14T20:44:51Z")
> [@pganssle](#): > > Not that I don’t share this goal, but IIRC Donald has 20% of his time allocated to OSS, so Amazon _is_ one of the few companies paying people to support packaging Oh yeah, for sure. I think it’s more than 20% even. I’m not saying we should get out the pitchforks :-). It’s much appreciated. But, just to overstate the obvious, it doesn’t mean they can stay on py2 forever while volunteers like Paul and Pradyun support them. So they still need some kind of plan. > [@nad](#): > > ( ?? → [https://docs.python.org/2/library/ensurepip.html](https://docs.python.org/2/library/ensurepip.html) Huh, whoops. Today I learned! ---
**Author:** ![nad](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/nad/32/51_2.png) [@nad](https://discuss.python.org/u/nad)\ **Post date:** [June 14, 2019, 9:28pm UTC](https://discuss.python.org/t/packaging-and-python-2/662/126 "2019-06-14T21:28:26Z")
> Huh, whoops. Today I learned! You’re forgiven 🙂 I forgot that there is at least one good reason why ensurepip isn’t as well known in Python 2.7. Besides the fact that is was added late in 2.7’s life, we also elected to make its installation opt-in so, unlike with Python 3, you need to explicitly add `--with-ensurepip` to `./configure`. . ---
**Author:** ![pradyunsg](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pradyunsg/32/206_2.png) [@pradyunsg](https://discuss.python.org/u/pradyunsg)\ **Post date:** [June 15, 2019, 10:27pm UTC](https://discuss.python.org/t/packaging-and-python-2/662/127 "2019-06-15T22:27:18Z")
> [@nad](#): > > you need to explicitly add `--with-ensurepip` to `./configure` . Is it included by default on the [python.org](http://python.org) installer? * * * Edit: Yes. [What’s New in Python 2.7 — Python 3.12.1 documentation](https://docs.python.org/3/whatsnew/2.7.html#bootstrapping-pip-by-default) says: > On Windows and Mac OS X, the CPython installers now default to installing `pip` along with CPython itself (users may opt out of installing it during the installation process). ---
**Author:** ![pradyunsg](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pradyunsg/32/206_2.png) [@pradyunsg](https://discuss.python.org/u/pradyunsg)\ **Post date:** [June 16, 2019, 8:48am UTC](https://discuss.python.org/t/packaging-and-python-2/662/128 "2019-06-16T08:48:44Z")
I’m pretty sure it’s possible to craft a query that shows what proportion of Py2 downloads come from what packages. If we see a substantial amount of downloads for a certain package, that’ll be an indicator here if there’s other places to look. (It’s basically the same analysis as @methane and @njs applied to aws-cli but more generally applied) ---
**Author:** ![pradyunsg](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pradyunsg/32/206_2.png) [@pradyunsg](https://discuss.python.org/u/pradyunsg)\ **Post date:** [June 21, 2019, 4:05pm UTC](https://discuss.python.org/t/packaging-and-python-2/662/129 "2019-06-21T16:05:43Z")
I ran a query (group by project-python\_version-week-year, top 16000 by count, for past 6 months) on the downloads table in BigQuery ([context](https://packaging.python.org/guides/analyzing-pypi-package-downloads/#background)), and dumped the data into a GitHub Gist. The `simple` API records, are too many to query from it without hitting my puny 1TB free tier limit. Be warned, opening the Gist will result in a _huge_ HTML page being served to you (JSON is ~2.5 MB and GitHub serves all of it highlighted – the loaded DOM is ~500 MB). The GitHub Gist: [https://gist.github.com/pradyunsg/f6ebfd39a5b21a097179f8b5f8b5fcb9](https://gist.github.com/pradyunsg/f6ebfd39a5b21a097179f8b5f8b5fcb9) * * * If someone knows of a cool way to generate insights from this, that’d be nice. In the interest of maintaining everyone’s sanity, here’s the raw (plain text) dump of the Gist files: The raw query: [https://gist.githubusercontent.com/pradyunsg/f6ebfd39a5b21a097179f8b5f8b5fcb9/raw/abb15158674b576c896814e69a8ffe1b9f9dbf22/query.sql](https://gist.githubusercontent.com/pradyunsg/f6ebfd39a5b21a097179f8b5f8b5fcb9/raw/abb15158674b576c896814e69a8ffe1b9f9dbf22/query.sql) The raw resulting JSON: [https://gist.githubusercontent.com/pradyunsg/f6ebfd39a5b21a097179f8b5f8b5fcb9/raw/abb15158674b576c896814e69a8ffe1b9f9dbf22/results-20190621-195401.json](https://gist.githubusercontent.com/pradyunsg/f6ebfd39a5b21a097179f8b5f8b5fcb9/raw/abb15158674b576c896814e69a8ffe1b9f9dbf22/results-20190621-195401.json) * * * And, with that I hit the “3 consecutive posts only” limit of Discourse. ---
**Author:** ![methane](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/methane/32/43_2.png) [@methane](https://discuss.python.org/u/methane)\ **Post date:** [June 26, 2019, 12:06pm UTC](https://discuss.python.org/t/packaging-and-python-2/662/130 "2019-06-26T12:06:54Z")
I wrote an article to advocate using Python 3 for awscli. [https://dev.to/methane/use-pip3-to-install-awscli-44pk](https://dev.to/methane/use-pip3-to-install-awscli-44pk) And I made a pull request to Travis document. > Let’s advocate Python 3 more! ---
**Author:** ![methane](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/methane/32/43_2.png) [@methane](https://discuss.python.org/u/methane)\ **Post date:** [July 1, 2019, 10:37am UTC](https://discuss.python.org/t/packaging-and-python-2/662/131 "2019-07-01T10:37:36Z")
I think about 14 (awscli and it’s dependencies) \* 200k DL/day are from not normal users. I suspect AWS downloads them. I filed an issue about it. > ---
**Author:** ![pradyunsg](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pradyunsg/32/206_2.png) [@pradyunsg](https://discuss.python.org/u/pradyunsg)\ **Post date:** [July 2, 2019, 10:12am UTC](https://discuss.python.org/t/packaging-and-python-2/662/132 "2019-07-02T10:12:42Z")
Thanks @methane for the investigation, the blog post and for filing the issue w/ AWS. Much appreciated. =D ---
**Author:** ![methane](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/methane/32/43_2.png) [@methane](https://discuss.python.org/u/methane)\ **Post date:** [July 10, 2019, 7:05am UTC](https://discuss.python.org/t/packaging-and-python-2/662/133 "2019-07-10T07:05:56Z")
I found [awslogs-agent-setup.py](https://s3.amazonaws.com/aws-cloudwatch/downloads/latest/awslogs-agent-setup.py) installs awscli-cwlogs after installing `pip<7.0.0`. I reported it to GitHub already. > BTW, awscli-dwlogs is downloaded about 330k/day, and `awslogs-agent-setup.py` downloads 17 packages (pip, virtualenv, awscli-cwlogs, and it’s dependencies). 330k \* 17 is about 5.6M DL/day. If awslogs-agent-setup.py is updated to download from S3 instead of PyPI, 5.6M DL/day will be disappeared from PyPI download stats! ---
**Author:** ![sumanah](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/sumanah/32/1722_2.png) [@sumanah](https://discuss.python.org/u/sumanah)\ **Post date:** [December 21, 2019, 12:50am UTC](https://discuss.python.org/t/packaging-and-python-2/662/134 "2019-12-21T00:50:50Z")
Update regarding the general Python 2 sunset: [Here’s the simple FAQ to circulate widely, originally published in early September](https://www.python.org/doc/sunset-python-2/) (focusing on the fact that updates to Python 2 will stop on Jan. 1, 2020), and [here’s the press release that just went out](https://www.python.org/psf/press-release/pr20191220/) (the last minor release of Python 2.7 will be released in April 2020). I’d be curious as to whether the projects listed in [the Python 3 statement](https://python3statement.org/) have seen declines in the proportions of Python 2-related downloads from PyPI since we published the FAQ in September. ---
**Author:** ![hugovk](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/hugovk/32/14505_2.png) [@hugovk](https://discuss.python.org/u/hugovk)\ **Post date:** [December 22, 2019, 10:38am UTC](https://discuss.python.org/t/packaging-and-python-2/662/135 "2019-12-22T10:38:53Z")
> [@sumanah](#): > > I’d be curious as to whether the projects listed in [the Python 3 statement](https://python3statement.org/) have seen declines in the proportions of Python 2-related downloads from PyPI since we published the FAQ in September. I made charts for some projects, between January 2016 and November 2019 (pip, six, coverage.py, Django, Flake8, pylint haven’t signed it): > **[GitHub - hugovk/pypi-tools: Command-line Python scripts to do things with PyPI](https://github.com/hugovk/pypi-tools)** > > Command-line Python scripts to do things with PyPI - GitHub - hugovk/pypi-tools: Command-line Python scripts to do things with PyPI (Jan 2016 - Oct 2019: [Python version share over time, 5 - DEV Community](https://dev.to/hugovk/python-version-share-over-time-5-7ik)) ---
**Author:** ![pganssle](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pganssle/32/245_2.png) [@pganssle](https://discuss.python.org/u/pganssle)\ **Post date:** [January 12, 2020, 2:29pm UTC](https://discuss.python.org/t/packaging-and-python-2/662/136 "2020-01-12T14:29:43Z")
@jaraco has merged and released a PR dropping Python 2 support for `setuptools` [here](https://github.com/pypa/setuptools/pull/1955). We still have the compatibility shims in place, so it shouldn’t _stop working_ if people use it even in Python 2, but we are still discussing whether or not to revert this PR. If other PyPA projects have strong feelings one way or the other, now is a good time to comment. ---
**Author:** ![pf\_moore](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pf_moore/32/35_2.png) [@pf\_moore](https://discuss.python.org/u/pf_moore)\ **Post date:** [January 12, 2020, 3:28pm UTC](https://discuss.python.org/t/packaging-and-python-2/662/137 "2020-01-12T15:28:44Z")
I’m on record as wanting pip to drop Python 2 support sooner rather than later, so please take this comment with that in mind. As pip vendors `pkg_resources` (which is part of `setuptools`), we have to pick a version to vendor (I suppose we could in theory vendor two versions, and pick one at runtime, or something like that, but that’s a lot messier). Am I correct in that assumption, or is there some clever way of keeping up to date while still supporting Python 2 that I’m missing? If we do have to (in effect) pin `pkg_resources`, what will be the impact on interoperability standards development? Do such standards typically affect `pkg_resources`? I’m not so worried about build processes, as we install an independent copy of `setuptools` in the build environment. My feeling is that pip _will_ need to pin `pkg_resources`, but this won’t be as significant a problem as it would be if we relied on the broader `setuptools` package. I do think that we should have a discussion, and projects should be careful about unilaterally dropping support for Python 2, so thanks for raising this, @pganssle. I think the key thing we need to consider here is moving the ecosystem forward. If all that mattered was support and bug fixes, I think that telling Python 2 users to just use the last version of the packaging tools with Python 2 support would be fine. But the real question is how do we drive adoption of new standards like `pyproject.toml` and later versions of `manylinux` if Python 2 projects are required to pin their packaging tools. I _don’t_ think it’s impossible (for example, pure Python projects can just build universal wheels using Python 3 - they might need to change their workflow, but I’m OK with that) but if we could offer transition advice then that would likely smooth the process. This is definitely something we should discuss at the packaging summit, IMO. ---
**Author:** ![pf\_moore](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pf_moore/32/35_2.png) [@pf\_moore](https://discuss.python.org/u/pf_moore)\ **Post date:** [January 12, 2020, 8:08pm UTC](https://discuss.python.org/t/packaging-and-python-2/662/138 "2020-01-12T20:08:17Z")
[https://github.com/pypa/virtualenv/issues/1493](https://github.com/pypa/virtualenv/issues/1493) appears relevant as well. Edit: Apparently (see [this pip issue](https://github.com/pypa/pip/issues/7586)) some people build their own wheel mirrors, and don’t necessarily remember to extract the `Requires-Python` metadata and expose it via the index. Unfortunately, this seems like it would be a common mistake to make, which somewhat reduces the effectiveness of the `data-requires-python` tag 🙁 ---
**Author:** ![chrahunt](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/chrahunt/32/947_2.png) [@chrahunt](https://discuss.python.org/u/chrahunt)\ **Post date:** [January 13, 2020, 12:00am UTC](https://discuss.python.org/t/packaging-and-python-2/662/139 "2020-01-13T00:00:48Z")
We have an [open issue](https://github.com/pypa/pip/issues/7413) in pip to transition from `pkg_resources` to `importlib_metadata`. It doesn’t address the larger issue, but would make `setuptools` dropping support for Python 2 less of a risk to pip specifically. ---
**Author:** ![xafer](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/xafer/32/248_2.png) [@xafer](https://discuss.python.org/u/xafer)\ **Post date:** [January 13, 2020, 12:44pm UTC](https://discuss.python.org/t/packaging-and-python-2/662/140 "2020-01-13T12:44:10Z")
> [@pganssle](#): > > If other PyPA projects have strong feelings one way or the other, now is a good time to comment. As much as I would love to drop Python 2 on pip, it seems that a significant number of our users aren’t there yet. Concerning setuptools, I think a deprecation notice of a few months would have made sense and feel that the drop was quite sudden. All the libraries using setuptools and still supporting Python 2 are now likely in an uncomfortable place of choosing between pinning setuptools or having a different version (and thus different options) depending on the interpreter version… > [@chrahunt](#): > > We have an [open issue](https://github.com/pypa/pip/issues/7413) in pip to transition from `pkg_resources` to `importlib_metadata` . It doesn’t address the larger issue, but would make `setuptools` dropping support for Python 2 less of a risk to pip specifically. I wouldn’t be surprised if `importlib_metadata` were to also drop support for Python 2 in a near future 😉 [Previous page](https://discuss.python.org/t/packaging-and-python-2/662.md?page=6) [Next page](https://discuss.python.org/t/packaging-and-python-2/662.md?page=8)