# PEP 777: How to Re-invent the Wheel **URL:** \ **Category:** Standards\ **Created:** [October 10, 2024, 7:09pm UTC](https://discuss.python.org/t/pep-777-how-to-re-invent-the-wheel/67484 "2024-10-10T19:09:27Z")\ **Posts on this page:** 1\ **Showing post:** 243
**Author:** ![cosmicexplorer](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/cosmicexplorer/32/31291_2.png) [@cosmicexplorer](https://discuss.python.org/u/cosmicexplorer)\ **Post date:** [September 29, 2025, 5:24pm UTC](https://discuss.python.org/t/pep-777-how-to-re-invent-the-wheel/67484/243 "2025-09-29T17:24:37Z")
Wanted to say this is a really cool and exciting PEP! `^_^` I do not have answers for the compatibility questions raised upthread, but wanted to note a few points about zips and wheels I’ve found from personal research. I was particularly impressed at the care taken to define forwards and backwards compatibility, and I believe the stability requirements defined in the current draft would be more than sufficient for me to perform some of the experimentation I describe below. # HTTP Range Requests in the Wild First, I have some very positive and lengthy comments regarding this section: > This PEP relies on resolvers being able to efficiently acquire package metadata, usually through [PEP 658](https://peps.python.org/pep-0658/). This might present a problem for users of package indices that do not serve [PEP 658](https://peps.python.org/pep-0658/) metadata. However, today most installers fall back on using HTTP range requests to efficiently acquire only the part of a wheel needed to read the metadata, a feature most storage providers and servers include. Furthermore, future improvements to wheels such as compression will make up performance losses due to inspecting files in the wheel. I’m super glad this practice has been noted in a PEP! It turns out doing this robustly is complex ([https://github.com/pypa/pip/pull/12208](https://github.com/pypa/pip/pull/12208)), and pip currently performs a naive version, but the implementation in that PR was improved based on feedback from the poetry maintainer. It turns out there is actually some additional standardization that could be useful here–fastly’s PyPI CDN doesn’t support negative range requests, which requires performing an additional request. But pip needs to support the whole range of possible backends beyond PyPI anyway, so supporting this quirk is not an additional burden except in PyPI’s bandwidth usage. It may be worth mentioning how the range request approach needs to read from the end of the file, so supporting negative range requests can be an optimization for the backend. I’ll note that I’ve seen some wheels put out by google put the METADATA file at the front ([perform 1-3 HTTP requests for each wheel using fast-deps by cosmicexplorer · Pull Request #12208 · pypa/pip · GitHub](https://github.com/pypa/pip/pull/12208#issuecomment-1667384444)), which is noncompliant and required further workarounds to make the pip implementation of range requests work against all of PyPI. This practice appears to be very uncommon, and is already covered by the existing wheel standard, so no change is needed, but it may be another reason to mention the negative range request optimization. You’ve very effectively described it here already, but I also wanted to note how HTTP range requests are specifically useful for achieving metadata-only resolves against a remote `--find-links` repo (i.e. the simplest possible HTTP server, that just serves a folder of wheels). This is the approach Twitter employed for several years, and it’s extremely convenient for maintenance. It can particularly be used in tandem with a standard simple repository API as an additional index for testing or staging versions of a specific package, so it’s an excellent component of internal developer tooling. I think packaging standards should support self-hosted indexes, so I think it’s great that this practice is finally codified in a PEP. Because the range request approach takes advantage of existing HTTP features (as you’ve noted already), and standardized features of the zip file format (the index at the end), I don’t think it deserves more treatment than you’ve given it here. But I’m very glad to see it finally identified as a valid approach to resolve against wheel repos. # Complex Zip Functionality These are some mechanisms I’ve identified which leverage standardized features of the zip file format to achieve greater performance, space/bandwidth usage, or both. I am mentioning them here to motivate further progress in this area and to describe specific alternate wheel formats I would like to generate from a build system, if that capability were available. 1. Zip extraction can be parallelized. 2. zstd dictionaries can be employed to reduce the size of a wheel archive more than using zstd alone. 3. Wheels can be used to generate zipapps without decompression. > Finally, future wheel revisions **MUST NOT** use any compression formats not in the CPython standard library of at least the latest release. All three of these designs are still in the prototype phase, but the PEP as it stands seems to enforce sufficient compatibility guarantees for them to be employed. I have spent many many hours with the zip file format over the past few years and I would love for this PEP to reach standardization so I can make use of these techniques to make packages smaller and faster. I understand the above was a lot of information, but I hope it provides useful food for thought as to what this PEP would enable. Sorry if not! --- _[View the full topic](https://discuss.python.org/t/pep-777-how-to-re-invent-the-wheel/67484)._