I think Upload API 2.0 is only tangentially related.
Right now releases can’t be closed without some sort of delay or explicit mechanism to signal that files are “done” being uploaded, because the upload API’s unit of atomicity is per file and releases commonly span multiple files.
Upload API 2.0 provides a mechanism that could be used as that signal, but it’s unclear to me if it should (at least by default) be used for that, and the Upload API 2.0 PEP as written does not affect the open nature of releases on PyPI.
For practical purposes, I don’t think Upload API 2.0 can help much here, because anything that treats multiple files as an atomic unit needs some coordination to know when the files are done being uploaded so it can “commit” the upload.
I expect that tools like twine will by default, in the hypothetical Upload API 2.0 future, open a single upload session per invocation, upload all of the files given in that invocation within that session, and then complete it.
However, if your upload process is setup such that you fan out (say for example, you produce binary wheels targeting different platforms) and each of those machines are independently setup to upload to PyPI, then tools like twine won’t be able to automatically extend the upload session across them, and thus will likely grow some sort of flag to not complete the upload.
So treating an Upload API 2.0 upload session as implicitly “closing” the release would be backwards incompatible with any release process that releases multiple artifacts and doesn’t collect all of the artifacts and upload them in a single twine (or whatever tool) invocation. That likely makes it a non-starter.
Not to mention the fact that Upload API 2.0 likely isn’t going to cause us to shut down the legacy upload api anytime soon, because that would also be hugely backwards incompatible.
Also, there are multiple different approaches we could take for restricting “open ended” releases on PyPI, but personally I think getting into the weeds about implementation details is premature, because I don’t believe we’ve yet answered the question about whether or not it’s a thing we should do.
There’s no inherently correct answer between open and closed releases, there’s just different trade offs. Given a change like this is inherently breaking, there needs to be strong justification that the breakage is worth it or that the number of projects that would actually be affected are low, otherwise I think the status quo likely has to win.
Certainly related, but as PEP 694 is currently written[1], stages are an optional feature for an index implementing 694. I would expect PyPI to implement stages though.
Yes, it could. There’s a step in the protocol where a publishing session is … published. Artifacts inside an unpublished session are still mutable, which allows the uploader to fix any problems before the public sees the new release. After publishing, the artifacts become immutable, but there’s no change to the mutability of a “release”, partly because PyPI kind of only sorta has a notion of a release[2]. 694 gives us a place in the new upload protocol to declare whether a release is closed or not upon publishing, but whether indexes support closing a release is a separate matter, and I’d argue out of scope for 694.
I may misunderstand the comment, but 694 has exactly that step. I won’t link to the text because the details are going to change, but there is an explicit publish step that serves this purpose.
That’s what I’d expect, i.e. upload tools would by default publish at the end of a successful multi-upload session, but would have an option to keep the publishing session open, be it for staged testing or fanned-out subsequent artifact uploads.
Given that 694 / Upload 2.0 is a completely new feature, there really isn’t any backwards compatibility with legacy uploads to consider. We could say that publishing a session implicitly closes the release by default (assuming indexes support such things, as described above) and then and an optional keep-open flag in the publish action. It would be quite easy to later explicitly close the release even without uploading any new artifacts[3].
That is very true.
What I’d recommend is that we flesh out the semantics for what “closing a release” means (index independent, does it need a PEP?), and add the ability for PyPI to close a release (also presumably without the ability to reopen it?). I think it would be trivially easy to add a keep-open flag to the session publishing request.
and yes, I owe an update based on @dustin 's last round of feedback ↩︎
694 has a step that competes/publishes the session but a tool can’t generically call that step without knowledge about whether there could be more files to upload if we decide that step implicitly “closes” the release.
It requires knowledge of the release process of the individual project to know the answer.
There’s not directly a backwards compatibility for 694, but if we want tools like twine to opportunistically use upload 2.0, then the implication of using 2.0 does become a backwards compatibility issue.
IOW, upload 2.0 could decide that publishing a session defaults to closing the release, but then I expect tools like twine to default to the legacy API and require opting in to upload 2.0, because if twine upload started “closing” your releases by default, that would break a fan out release process.
Agreed with those points. It’s premature anyway to consider 694’s role in closing a release, until we actually decide that’s something we want and what the semantics are.
Hi! Just shortly throwing a few words to the discussion.
I think it’s important to mention that PyPI already had a precedent when the “mutable” version was abused, thankfully without strong malicious intentions. Three months ago, someone took over the name of the removed-by-author “umap” package and published it again with telemetry beacons. They have also re-released the same version as the previously published one by uploading a different distribution file. So the problem is not just theoretical, it was already explored. (Another side finding was that it’s possible to keep installing removed versions, but it’s a different story not to discuss in this thread.)
I think we probably need someone to run a query for 14 days and 7 days to see how much of an impact it would be (and I would be curious about 3, 2, and 1 day as well).
Okay, I have some data from @miketheman (thanks Mike!) on how often this behavior is used:
warehouse=> SELECT
COUNT(DISTINCT p.id) FILTER (WHERE f.upload_time - r.created >= INTERVAL '14 days') AS days_14,
COUNT(DISTINCT p.id) FILTER (WHERE f.upload_time - r.created >= INTERVAL '7 days') AS days_7,
COUNT(DISTINCT p.id) FILTER (WHERE f.upload_time - r.created >= INTERVAL '3 days') AS days_3,
COUNT(DISTINCT p.id) FILTER (WHERE f.upload_time - r.created >= INTERVAL '2 days') AS days_2,
COUNT(DISTINCT p.id) FILTER (WHERE f.upload_time - r.created >= INTERVAL '1 day') AS days_1
FROM
projects p
INNER JOIN releases r ON r.project_id = p.id
INNER JOIN release_files f ON f.release_id = r.id
WHERE
DATE_PART('year', r.created) >= 2025;
days_14 | days_7 | days_3 | days_2 | days_1
---------+--------+--------+--------+--------
1295 | 1611 | 2008 | 2188 | 2478
So in the past ~year, there have been 1295 projects that have published an artifact 14+ days after the initial release and many more happening in the 1-3 day span. So if we were to implement the 14 day policy there would be 2-3 projects per day that would see the new error until the community “routed around” the error by creating new releases, instead.
Mike also provided a few example projects where this happened over the last ~5 years:
warehouse=> WITH classified AS (
SELECT
p.normalized_name,
r.version,
f.upload_time - r.created AS lag,
CASE
WHEN f.upload_time - r.created >= INTERVAL '14 days' THEN '14+'
WHEN f.upload_time - r.created >= INTERVAL '7 days' THEN '7-13'
WHEN f.upload_time - r.created >= INTERVAL '3 days' THEN '3-6'
WHEN f.upload_time - r.created >= INTERVAL '2 days' THEN '2'
WHEN f.upload_time - r.created >= INTERVAL '1 day' THEN '1'
END AS bucket
FROM
projects p
INNER JOIN releases r ON r.project_id = p.id
INNER JOIN release_files f ON f.release_id = r.id
WHERE
DATE_PART('year', r.created) >= 2021
AND f.upload_time - r.created >= INTERVAL '1 day'
),
ranked AS (
SELECT
*,
ROW_NUMBER() OVER (PARTITION BY bucket ORDER BY random()) AS rn
FROM classified
)
SELECT bucket, normalized_name, version, lag
FROM ranked
WHERE rn <= 10
ORDER BY bucket, rn;
bucket | normalized_name | version | lag
--------+------------------------------------------+-------------+--------------------------
1 | ensmallen | 0.8.76 | 1 day 00:10:45.922215
1 | cmarkgfm | 2025.10.20 | 1 day 01:27:05.343397
1 | libfsapfs-python | 20220501 | 1 day 14:05:15.12519
1 | passagemath-graphs | 10.6.42 | 1 day 07:17:19.996306
1 | fast-bencode | 1.1.6 | 1 day 14:58:30.727354
1 | ros-sensor-msgs | 5.3.6 | 1 day 20:32:41.263156
1 | sccache | 0.8.2 | 1 day 10:40:01.020694
1 | thirdai | 0.9.33 | 1 day 09:56:05.610839
1 | rscheduler | 0.1.0 | 1 day 07:11:43.045616
1 | signalflow | 0.5.3 | 1 day 02:42:15.365097
2 | g2o-python | 0.0.3 | 2 days 01:12:39.599578
2 | libfvde-python | 20240502 | 2 days 12:57:30.880789
2 | passagemath-rankwidth | 10.6.30 | 2 days 06:26:43.426063
2 | netgen-mesher | 6.2.2306 | 2 days 02:08:19.66846
2 | poselib | 2.0.0 | 2 days 00:07:12.73351
2 | passagemath-meataxe | 10.6.31rc3 | 2 days 22:00:14.091101
2 | llama-cpp-cffi | 0.1.2 | 2 days 16:37:21.36003
2 | pymatio | 0.1.0 | 2 days 12:32:45.487768
2 | kratosgeomechanicsapplication | 9.4.3 | 2 days 20:02:06.617887
2 | note-scoring | 0.1.0 | 2 days 09:54:10.648191
3-6 | libevtx-python | 20240504 | 3 days 02:38:41.990573
3-6 | bosa-connectors-binary | 0.0.10 | 4 days 07:02:42.279023
3-6 | nessie-py | 0.1.7 | 3 days 06:05:31.521771
3-6 | graph4nlp-cu110 | 0.2a4 | 3 days 22:24:18.7649
3-6 | mandelbrot-implementations-cython-direct | 0.1.0 | 6 days 20:47:03.090528
3-6 | pillow | 8.3.1 | 4 days 22:10:28.756944
3-6 | gamsapi | 49.6.1 | 3 days 06:32:15.66552
3-6 | rocksdict | 0.3.24 | 3 days 03:44:11.150153
3-6 | fast-dep | 0.0.1 | 4 days 12:16:08.354373
3-6 | trust-free | 2.1.4 | 3 days 20:34:26.858133
7-13 | python-sat | 0.1.8.dev13 | 10 days 05:04:46.601852
7-13 | ros-unique-identifier-msgs | 2.5.0 | 8 days 19:42:51.64209
7-13 | mlpack | 4.7.0 | 9 days 03:24:38.198048
7-13 | ytsaurus-yson | 0.4.5 | 7 days 18:37:11.562206
7-13 | google-re2 | 1.0 | 12 days 00:05:13.312822
7-13 | instanttensor | 0.1.6 | 12 days 20:42:33.076557
7-13 | opencv-contrib-python-headless | 3.4.14.51 | 12 days 19:21:55.810543
7-13 | google-re2 | 1.0 | 12 days 00:05:19.726237
7-13 | dlblas | 0.0.2 | 7 days 08:48:55.218678
7-13 | pyhyperscan | 0.1.9 | 10 days 22:54:47.973345
14+ | xmlib-to-git | 1.0.0 | 474 days 18:57:09.411265
14+ | tensorflow-metal | 0.8.0 | 45 days 14:21:08.78746
14+ | rs-audio-stats | 1.3.9 | 134 days 17:27:47.292922
14+ | dlib-bin | 19.24.6 | 147 days 10:42:54.269826
14+ | abstract-utilities | 0.2.0.58 | 40 days 20:39:22.917332
14+ | pyzstd | 0.15.0 | 106 days 17:25:03.286604
14+ | opening-hours-py | 0.9.1 | 29 days 19:33:15.477529
14+ | sbank | 1.0.2 | 726 days 14:19:53.907989
14+ | pr2codon | 1.1.18 | 722 days 10:33:47.026539
14+ | hmpty | 1.5.10 | 571 days 23:20:35.227465
(50 rows)
I do see a few names I recognize in this list: Pillow, google-re2, pyzstd. Looking at a few of the examples, I see many of the longer durations being “adding wheels after only having sdists” and the medium length durations being “adding wheels with different platforms”.
The cutoff year is by release date, so it may exclude some cases where someone doesn’t make new release for years and just adds new wheels for previously unsupported platforms, e.g. Brotli 1.0.9 has been released on Aug 27, 2020 but the last wheel was published on Nov 16, 2022:
In this case, Brotli did have a newer release since (in 2023) but I do feel it’s worth noting that some projects could stay without releases for a very long time, with only new wheels made when support for new platform tag is needed.
Checking this: the original 64-bit Windows wheels (Pillow-8.3.1-cp*-win_amd64.whl) were mispackaged, so we uploaded fixed ones five days later with an extra build tag (Pillow-8.3.1-1-*-win_amd64.whl) (for reference: python-pillow/Pillow#5573 / pillow · PyPI).
We could also consider deploying it as a warning first. Presumably “late releases” aren’t uploaded by CD workflows, so I suppose people would notice and report here if they find it really problematic.
And even if the change is implemented, there’s always the option of reverting it or increasing the period if people truly have problems with that. I don’t think we need to be overly cautious here.
Sending an email notification to maintainers would probably be a good start. At least then (assuming it’s only a stolen publishing key) some people will get an immediate notification if it happens and they weren’t expecting it.
If the email templates are easy enough, you could even include a message that this kind of functionality will be blocked in the future (maybe pick a date?) and request feedback if it’s going to be a major issue.
Given that a restriction has not been implemented, my plan is to warn package consumers that a new file has been added to an old release. Here is a mockup for context:
My hesitation to this is I am not sure who that warning is for, we’re saying it’s for users but in my mind there are two types of users:
Users who aren’t locking or using any non-default security features with their installers. No hash pins, at best using version pinning. These users would not notice if their installer suddenly started selecting a “build number=1” wheel. They would not see this warning either.
Users who are locking or are using security features with their installers. They will never see this warning either, because their installer is not considering the new files.
The scenario I could see a user finding this warning is if somehow the newly installed file caused new issues that didn’t exist in the previously installed files and this is some way of clarifying to the user that something went wrong. Maybe this is a case to cover? But I suspect it would be a rare one.
The only users that /would/ see this warning are ones selecting a package to use, and by the time that a “new file on old release” is relevant it’s likely that whatever release was “fixed” in the past is no longer relevant. That user is deciding which project to use, not whether they can continue to trust a past release.
Some of this can be addressed with PEP 694’s stages, but of course it’s still possible to miss some wheels. (Yes, I intend to get back to 694, likely during PyCon.)
