Pre-PEP Discussion: RFC 9457 Error Responses for Package registries

Packaging Standards
1

Abstract

This proposal recommends that Python package registries adopt RFC 9457 (Problem Details for HTTP APIs) as a standard format for error responses in PEP 503 (Simple Repository API) and PEP 691 (JSON-based Simple API). Currently, when pip encounters HTTP errors from package registries, it displays the status code and reason phrase from the HTTP response. While HTTP/1.1 includes both fields, HTTP/2 removed the reason phrase entirely, leaving developers with only numeric status codes (e.g., “403”) and no context about why a request failed. This creates a significant user experience problem as HTTP/2 adoption accelerates. This proposal enables registries to provide structured, human-readable error information through RFC 9457’s standardized JSON format, which works consistently across both HTTP/1.1 and HTTP/2. The implementation would be optional and backwards-compatible, with clients falling back to existing behavior for non-RFC 9457 responses.

Motivation

When developers retrieve Python packages from external clients (such as UV), some registries provide error context by embedding messages in the HTTP/1.1 reason phrase field. However, HTTP/2 removed this field entirely, leaving developers with only numeric status codes under modern infrastructure. Without understanding why a package was blocked, developers must contact administrators to diagnose issues, creating unnecessary friction.

Real-World Scenario (Nexus Repository Manager):

The following is how the developer sees a custom error message when trying to download component urllib3 via pip (and using HTTP/1.1 reason phrase to display a custom message):

pip download urllib3
Looking in indexes: http://<hostName>/repository/pypi-proxy/simple
Collecting urllib3
ERROR: HTTP error 403 while getting http://<hostName>/repository/pypi-proxy/packages/urllib3/...
ERROR: Could not install requirement urllib3 because of HTTP error 403 Client Error: Requested item is quarantined ...

Technical Problems with This Approach:

  1. HTTP/2 Incompatibility: HTTP/2 removed the reason phrase field entirely (RFC 9113). Only the numeric status code is transmitted, meaning custom messages are lost when registries use HTTP/2.
  2. Not Standardized: The reason phrase format is not structured, making it impossible for clients to parse and display intelligently.
  3. Limited Length: Reason phrases have practical length limitations and cannot convey complex structured information (e.g. multiple errors, links, metadata).

With RFC 9457, this becomes:

{
  “status”: 403,
  “title”: “Package quarantined by security policy”,
  “detail”: “urllib3 version 2.6.2 has been quarantined due to security policy   violations.”
}

Ecosystem Context

RFC 9457 is already mandated in:

  • PEP 807 Index support for Trusted Publishing
  • PEP 694 Upload 2.0 API for Python Package Indexes

Rationale

Why RFC 9457?

  1. RFC 9457 is a well-established IETF standard (published July 2023) used by major APIs.
  2. Provides machine-readable + human-readable error information.
  3. Works consistently across HTTP/1.1 and HTTP/2
  4. Allows registries to add custom fields for domain-specific context

Why Optional/Recommended?

Making this recommended rather than mandatory allows:

  • Gradual adoption across diverse registry implementations.
  • No breaking changes for existing indices.
  • Progressive enhancement: clients can improve UX when RFC 9457 is detected.

Another option to support custom error messages from registries is to add a new custom header in the pip response. Similar work was done in the HuggingFace ecosystem where the header X-Error-Message is used. However, custom headers are non-standard and don’t provide structured, machine-readable error details like RFC 9457.

Specification (High-Level)

Scope: PEP 503 (Simple Repository API) and PEP 691 (JSON-based Simple API)

When Applied: HTTP error responses (4xx and 5xx status codes) from registry endpoints when pip interacts with the registry.

Minimum Required Fields:

  • status (integer): HTTP status code.
  • title (string): Short, human-readable summary.
  • detail (string): Detailed explanation of the error.

HTTP Content-Type: application/problem+json

Example Response:
{
“status”: 403,
“title”: “Package blocked by security policy”,
“detail”: “packageABC version 1.19.0 contains CVE-2021-12345 (Critical severity). Please upgrade to version 1.21.0 or later, for more details visit:``http://host.example.com``”
}

Client Behavior:

  • Clients like pip may parse and display RFC 9457 responses.
  • If Content-Type:application/problem+json is detected, clients can extract structured error details.
  • If not present, clients continue using existing error handling (fully backwards-compatible)

Related work:

What does the community think about this?. We are maintainers of a private registry and have experienced these error reporting limitations firsthand. We recognize we may not have full visibility into ecosystem-wide impacts, so we’re seeking community feedback before proposing a formal PEP.

I’m happy to collaborate with @woodruffw and others interested in moving this forward. Looking for feedback and buy-in before drafting a formal PEP.

7 Likes
2

Hi Luis, thank you for proposing this! I think it’s a wonderful idea. I’ve stumbled over this a number of times. I’ve moved this topic to the Packaging > Standards topic as I think this will reach the right audience here.

Should this information also be returned even if the return is 200 to maintain consistency? I think consistency in content-type between failure and success would be nice. Additionally, there are tools other than pip which interact with package indexes, so it’d be good to make this more generic to “packaging tools which interact with Simple API servers.”

I think this a valuable building block that would be good to unify across the simple API and other APIs from indexes such as uploading.

I expect folks will want to see an index implement the standard, so that they can understand how much work implementing the PEP would be.

1 Like
3

This seems like a great idea. +1 from me.

I’d hope that an implementation PR for PyPI would be developed as part of the PEP. I’d certainly strongly prefer that we don’t get in a situation where the PEP is approved but PyPI hasn’t yet implemented it, giving clients no motivation to add support (something which has happened in the past with other PEPs).

It would also be interesting to know how likely it is that other index providers will implement this. My gut feeling is that most simply won’t bother, at least initially. But it would be nice to be proved wrong :slightly_smiling_face:

2 Likes
4

Hi Emma, thank you for changing the tags to the correct topic.

My initial focus was on error responses because that’s where the immediate pain point exists (HTTP/2 eliminating reason phrases), but you raise a good point about consistency.

For success responses, package registries already return various content types (package files, JSON for metadata endpoints, etc.). Would you envision RFC 9457 applying to JSON responses specifically, or are you thinking more broadly? I want to make sure I understand the scope you’re suggesting before expanding the proposal.

5

@pf_moore could you please point me to the github project where this PR should be implemented?

6

I assume it would be PyPA/warehouse

7

Thinking about it more, it doesn’t really make a lot of sense to modify JSON returned by indexes (some of which is already standardized and has an expected layout) if the success information is just going to be “it worked” and duplicate the return code, so I’d say ignore me :slight_smile:

I would say that the PEP should clarify that the RFC 9457 information should only be returned in failure cases, and clients should check the status code to determine how to read the payload contents.

Not Paul, but this is the repository that backs pypi.org which Paul is talking about: GitHub - pypi/warehouse: The Python Package Index

2 Likes
8

Thanks for opening this discussion @lgonzalez! I also want to register my hearty support for this idea; I think structured error messages on index responses would be a significant boon to the ecosystem.

I concur with Emma and Paul about the value of having this implemented within PyPI (i.e. Warehouse) from the get-go, since that’s where it’ll have the most immediate impact. The links those two shared are correct, and I’d be happy to point you at the right parts of the code within PyPI for where this would go – if I’m right, it should be a pretty local change since we’d only need to update the error handling in the index responses :slightly_smiling_face:

2 Likes
9

IWBNI there were internal warehouse APIs that eventually 694 and 807 can just call for their own (eventual) use of RFC 9457 error responses.

2 Likes
10

Hi @woodruffw ,

Thank you for the support and for offering to help identify the specific code sections.

I’ve been examining the Warehouse codebase, and I want to confirm which “index responses” need RFC 9457 support. I’ve identified two potential locations in warehouse/views.py:

Option 1: /simple/ endpoint (lines 97-101):

  • Currently returns plain text: “404 Not Found”.
  • However, this endpoint is text/plain, and RFC 9457 is specifically for JSON and XML APIs

Option 2: JSON API endpoint (lines 102-107) (I think this is the right target):

  • Handles /pypi/{package}/json requests.
  • Already returns JSON: {“message”: “Not Found”}.
  • This seems like the natural fit for RFC 9457’s application/problem+json format.

My understanding is that RFC 9457 should apply to JSON APIs in this case, not plain text endpoints. Therefore, I think we should focus on upgrading the JSON API response (line 102).

I’m still getting familiar with this codebase, so please let me know if I’m on the right track or if there are other areas I should be looking at.

I also agree with @barry’s point about creating reusable internal APIs. I plan to create a warehouse/rfc9457.py helper module to avoid code duplication when we eventually expand RFC 9457 support to other endpoints.

Thank you for your guidance.

11

Thanks for looking into this @lgonzalez !

I was thinking about adding the functionality to warehouse/errors.py which doesn’t have much there now and seems like a good place for creating RFC 9457 error response bodies. It should definitely accept $3.2 Extension Members.

I would need to do a deeper reading of RFC 9110, $12.1 Proactive Negotiation, but I think you technically need to check the Accept of the request[1]. You can see the example in $3 has an Accept: application/json, application/problem+json header, which I know isn’t a specification.

It might be expedient to assume that if it encounters an error when it would normally return an application/json body, that an application/problem+json is acceptable, but I don’t know if that’s strictly legal. :winking_face_with_tongue:

  1. RFC 9457 Appendix B defines an alternative application/problem+xml format, but that’s likely overkill for warehouse ↩︎

1 Like
12

I think this is the main one!

The nuance here is that the /simple/ endpoints return two different content-types, based on Accept: header – there’s the simple HTML response (originally defined in PEP 503) and the simple JSON response (originally defined in PEP 691).

The /simple/ endpoints on PyPI currently return text/plain for error responses, but this wasn’t standardized anywhere. IMO a PEP here should propose returning application/problem+json instead, for both HTML and JSON requests :slightly_smiling_face:

Counterintuitively, this one probably isn’t the right target for a PEP – the “JSON APIs” on PyPI are actually unstandardized and PyPI-specific :sweat_smile:. One of the consequences of that is that standards-conforming clients (like pip and uv) don’t actually use those JSON APIs for resolution, so adding RFC 9457 compliant errors to them won’t yield as much benefit.

Oh huh, I hadn’t considered this – my intuition is that it’s always “OK” (but perhaps not nice) for an HTTP server to respond with whatever content type it pleases for error responses, since Accept is always a hint and not a proscription. Given that I think it would be fine for PyPI to use RFC 9457 for error responses even when the user has application/vnd.pypi.simple.v1+html or text/html as their Accept. But it’s possible I’m under thinking that :sweat_smile:

2 Likes
13

I’ve often found[1] that RFC tend to be an odd mix between extremely specific and full of holes.

  1. based on my experience with email related RFCs! ↩︎

3 Likes
14

Hello team,

I just pushed this draft PR as an initial attempt of implementing RFC9457 in Warehouse. Please let me know what you think.

@woodruffw @barry

2 Likes
15

Thanks! I commented with some thoughts and questions.

1 Like
16

Hi all,

I wanted to give a quick update and gently bump this thread. I addressed Barry’s feedback on the draft implementation in this PR.

If anyone has a chance to take another look at the PR or has further thoughts on the approach, I’d really appreciate the feedback. Happy to make any additional adjustments as needed.

Thanks for your time!

@woodruffw @barry would love your thoughts when you get a chance.

3 Likes
17

It’s on my list. I just haven’t had the time yet. Thanks for the ping!

2 Likes