Pre-PEP: Python Security Response Team Membership and Operations

Core Workflow
1

Hello all,

I’m starting the process of defining the Python Security Response Team (PSRT) membership and operations in such a way that allows for more, rather than fewer, core team members to participate in the vulnerability triage and remediation process.

Sponsor cc: @gpshead

Problem statement

The Python Security Response Team is described as “a highly trusted cabal of Python developers”. Historically, this team has been informally defined as members on a private mailing list. In practice this means that core team members interested in vulnerability triage or coordination (either personally or through their employer) have no documented pathway to join or contribute to the PSRT or knowledge about what the role requires.

The PSRT today uses email threads instead of a ticketing system. Being a mailing list means that there is never a clear “assignee” for who is pushing a report forward towards completion. Without this designation, it can be unclear who is the decision-maker for a particular report. If there are longer periods of inactivity on a thread, the report can sometimes run out of steam or be forgotten.

Collaboration between PSRT members and other core team members that are experts in the code with the reported vulnerability is similarly ad-hoc, either requiring the manual addition of an email address to an email thread or to a private GitHub repository and team. This friction means it’s tougher to involve non-PSRT members in determination and remediation efforts. This also results in lots of menial work managing individual accounts within GitHub repositories and teams to maintain “expiring” access.

Proposal

Responsibilities of PSRT members

Define the responsibilities of PSRT members, so that potentially interested contributors know what the role looks like ahead of time before joining the PSRT. Responsibilities include:

  • Being knowledgeable about typical vulnerability report handling processes.

  • Being assigned as a “coordinator” on vulnerability reports, and responsible for moving the report through the PSRT process to a “finished” state (either rejected or publishing an advisory) within the industry standard timeline of 90 days.

  • Coordinating the remediation of vulnerabilities you’re assigned to, by involving relevant core team members or triagers where necessary.

  • Authoring and publishing advisories to be shared on security-announce@python.org. These advisories are used for CVE records by the PSF CVE Numbering Authority.

  • Not sharing or acting on embargoed information publicly, such as applying unpublished mitigations ahead of the publication date.

  • PSRT members that are staff of the Python Software Foundation, as an “Open Source Steward” defined in Article 24 of the Cyber Resilience Act, have additional responsibilities such as reporting actively exploited vulnerabilities to ENISA/CSIRTs.

PSRT coordinators and remediation developers are credited publicly for their contributions in CVE records.

PSRT membership

Create an explicit process so that any core team members interested in vulnerability triage are able to join the PSRT for as long as they’re active as coordinators of vulnerabilities. Members can voluntarily leave the PSRT or will be rotated out after a period of inactivity.

Any core team member or triager may join the PSRT with approval from the Steering Council. Existing members of the PSRT that are not core team members or triagers will be recommended to be added as triagers through the typical process to maintain their role in the PSRT.

Once per year, PSRT admins will query the PSRT list and remove members who have been inactive, defined as not being an assigned coordinator or commenting on a vulnerability report in the past year. Upon PEP acceptance, the existing PSRT list will have inactive members removed.

Active Release Managers and Steering Council members may be members of the PSRT without needing to actively participate in vulnerability coordination, for the purposes of preparing for security releases and oversight.

PSRT shall maintain at least 2 of its members as “admins” of the mailing list.

Steering Council has final say on membership and admins of the PSRT.

Process for adding and removing PSRT members from relevant lists and permissions will be documented in the Python Developer Guide.

Adopting GitHub Security Advisories on projects

Adopt GitHub Security Advisories (GHSA) as the preferred reporting and ticket-tracking mechanism for vulnerability reports to specific projects (such as “CPython”, “pip”, etc). Reports sent to security@python.org intended for specific projects will be redirected to submit via GHSA (or submitted on behalf of the reporter if they don’t want to use GitHub).

Using a ticketing system like GHSA allows for:

  • Associating a report with other information, such as the assigned coordinator, CVE ID and CVSS severity. This information can be used to automatically “generate” a CVE record.

  • Automating aspects of vulnerability reports and remediation with a bot, similar to Bedevere, which can assist coordinators. Currently such a bot exists that assigns CVE IDs to accepted reports, but will be extended further to aid coordinators.

  • Managing access per-report for collaborators outside of the PSRT. This allows non-PSRT members to collaborate in a pull-request UI without tons of manual access management.

  • Tracking the status and timelines associated with each report so no report goes beyond the timeline expected for vulnerability report accidentally.

Projects that are covered by the PSRT that don’t already have GHSA enabled will have the feature enabled on the repositories and security policies updated to point to the GHSA submission form for the project.

15 Likes
2

PSRT members that are staff of the Python Software Foundation, as an “Open Source Steward” defined in Article 24 of the Cyber Resilience Act, have additional responsibilities such as reporting actively exploited vulnerabilities to ENISA/CSIRTs.
[…]
Any core team member or triager may join the PSRT with approval from the Steering Council. Existing members of the PSRT that are not core team members or triagers will be recommended to be added as triagers through the typical process to maintain their role in the PSRT.

Reading between the lines, in order to fulfil steward responsibilities PSF will need to always employ one or more core team members or triagers serving in that role? (And the SC will need to approve them.)

This has been coming up quite a bit lately in ORC discussions, that for foundations to actually qualify as stewards they’ll need to have some means of overseeing/enforcing parts of the vulnerability management process rather than relying solely on community volunteers. While I’m not sure I agree with the interpretation, putting that responsibility on designated PSF staff makes sense to cover all bases.

1 Like
3

IANAL, but the latest knowledge that I have regarding stewards providing engineering resources would mean that indeed, there is some additional obligation on the steward regarding security vulnerabilities. This table (click through for references) that Tobie from ORCWG created was really helpful for understanding exactly what this means in practice:

Steward provides project with… Notify CSIRT/ENISA of Vulnerabilities? Notify CSIRT/ENISA of Incidents? General Advisories?
Non-technical support No No Yes
IT infrastructure No Yes Yes
Engineering resources Yes Yes Yes

I’ve defined the “additional obligations for CRA” this way because if the PSF isn’t providing engineering staffing for whatever reason in the future to a project, there aren’t additional obligations from the CRA.

If anyone is interested in the CRA and what obligations might affect their own projects, please check out the ORCWG “CRA Hub”, it’s a great resource.

1 Like
4

I hope this doesn’t get “too bureaucratic”. I’ve been on the PSRT for many years, and, believe ir or not :wink:, routinely go for over a year without saying anything. I’m not a “security geek”, but do have uncommon tech knowledge, and sometimes reply with semi-authoritative knowledge of what the possible tradeoffs are. Better for me to be “in the loop” from the start, rather than people hope to attract my attention (like everyone else, I’m also very busy, despite being retired). PSRT email gets priority in my inbox.

So I would hope instead that, rather than establish mechanical rules for removing people, you ask someone seemingly inactive whether they’d like to remain (assuming, of course, that they’re still trusted in this area).

2 Likes
5

IANAL, but the latest knowledge that I have regarding stewards providing engineering resources would mean that indeed, there is some additional obligation on the steward regarding security vulnerabilities. This table (click through for references) that Tobie from ORCWG created was really helpful for understanding exactly what this means in practice:

I too am not a lawyer (though I am a steward representative on the ORC Spec Committee and participate in the Vulnerability Handling Task Force), and I haven’t seen a ton of agreement from actual lawyers on this yet. In particular, one interpretation is that if the PSF doesn’t have authority to override the Python SC in these matters and so can’t enforce compliance themselves, then they’re not capable of acting as a steward.

To me, that’s an extreme position which would exclude foundations for a majority of non-corporate-owned free/libre open source projects I’m familiar with, and given there are no actual penalties for stewards that fail to comply I’m not convinced it’s an important distinction. For now, I’m operating under the assumption that foundations like PSF can effectively serve as Open Source Stewards as defined in the CRA in cooperation with community-run governing bodies who actually control the projects being stewarded.

Anyway, all that’s to say I think the plan you’ve outlined is reasonable, regardless of whether PSF staff are actually doing the reporting tasks as a “steward” (in CRA terms) or technically just as SC-delegated participants in the community who conveniently happen to also be employed by PSF. Further, it helps make a good justification for additional funding from stakeholders who want to ensure those specific tasks get done.

3 Likes
6

To be clear, I am defining “participating” to avoid the auto-removal to be as little as “commenting on a report”, not necessarily coordinating a report end-to-end. I thought that participating on a report once in a year was a fairly conservative timeline, considering our volume of reports.

The security-sensitivity aspect of this group means we do probably want peoples’ access to expire if they’re no longer contributing. It’s nothing personal, just keeping the embargoed information safe from account take-over/revival shenanigans. It was important to me to make the criteria/timeline explicit so there’s no confusion or hard feelings about whether or not to remove an inactive member.

I think this is especially important if we’re expecting more PSRT members by making joining the team more accessible and defined. There’s also no perclusion to a “removed” PSRT member from joining again once they have more time to contribute.

I do think this proposed framing of the PSRT pushes it more explicitly towards a group of folks who are interested in participating with triaging and remediating vulnerabilities, rather than a more abstract “group of trustworthy experts”. Adopting GHSA should mean that any expert, whether they are on the PSRT or not, can get tagged in to specific areas much more easily by the PSRT member coordinating the vulnerability.

8 Likes
7

+1, I think formalising how security reports are triaged from a people perspective is a useful thing to do. I presume that much of this PEP is reflecting the status quo.

I would also potentially be interested in helping triage/volunteer with security reports, so having a defined process for core devs to volunteer here (with appropriate approvals/SC sign-off, etc) I think would be a good step.

This is also likely a good thing to have should funding for a dedicated security role change at any point in the future, in the vein of planning for a rainy day etc (hopefully this won’t be the case, of course!).

A

2 Likes
8

I’ve also been on the PSRT since its inception (IIRC, I created the mailing list and alias), and I also rarely chime in, mostly because I think Seth and others do a fantastic job and much better than I can do at this point. FOMO alone makes me want to stay in the loop, but honestly I’m fine if I get booted for non-participation, though I plan on continuing to lurk as long as I’m on the SC.

Great pre-PEP, and I’m glad to see the PSRT get just enough more formalism, but not too much, to serve the needs of Python today.

1 Like
9

Sure, that was clear. I just don’t believe anything is a pure win in real life. Any change from the status quo will have pluses and minuses. As Python grows, I too expect the benefits outweigh the drawbacks, from opening participation to adopting more formal and defined procedures. Part of growing pains. But things will be lost too.

Tagging would be likely to get ignored by me. Too many sources clamor for attention. Building an easy GMail rule to prioritize email from a specific email address (like the PSRT’s) works. The vast bulk of email I get is deleted without even being opened. Not just spam.

There’s also the “Many Eyes Make All Bugs Shallow” fiction, which is nevertheless often applicable. Someone may well think to tag me on a floating-point related vulnerability, but what about a DOS attack based on catastrophic backtracking in regexps? I’m “an expert” on that too, but possibly not known for that outside of long-term PSRT members who’ve seen me address those.

I expect many people are experts in areas others don’t know about. On a “everything everywhere all at once” mailing list, the very lack of specialization and formal structure enables “many eyes” serendipity.

But, ya, I’m just an old man yelling at clouds :laughing: Carry on! I’ll adapt as best I can, and grateful too for the benefits.

2 Likes
10

Could you please elaborate on this part a little more. I am a bit worried that we’re digging ourselves too much into the Github platform by promoting such a choice in a policy document.

IMO, the choice of platform should be abstract in the policy and the SC and PSRT members can then decide on which tool to use.

While it’s not likely that we’ll switch away from Github any time soon, please do remember that we have made such significant changes in the past.

Recent corporate politics around Github no longer give you a warm fuzzy feeling and given the sensitive nature of PSRT reports, special care has to be taken when it comes to choosing a platform for managing such reports.

Are there alternative platforms you could envision for the task ?

1 Like
11

Not good ones that I’m aware of. The big advantage of using the tools built into the repository host is that the advisories get associated with private branches, even in a public repo.

In a hypothetical future where we migrated away from GitHub hosting, we’d ideally choose a destination that offers similar capabilities.

(For example, ForgeJo has an open feature request for advisory handling features that may get developed further now that Fedora are actively looking to migrate their repository hosting: #142 - [FEAT] Secure reporting of security issues - forgejo/forgejo - Codeberg.org)

I agree the exact choice of platform should be separate from the policy PEP defining the PSRT itself. It’s potentially still a PEP level decision in its own right, though (in the sense that the rationale for the choice should be clearly recorded so any future migrations can take those concerns into account).

3 Likes
12

As @ncoghlan mentioned, there aren’t many tools that tightly integrate with GitHub to allow private “pull request”-like UI, vulnerability-specific tooling (such as CVSS, CVE ID, and CVE-specific crediting), already hooked up to our existing GitHub teams and admin, and for no cost. I couldn’t find any alternative that met all those criteria.

I can certainly abstract this out so that PSRT is able to choose its tool based on where the projects are hosted and what is available there.

2 Likes
13

Could we also not update the PEP, if at some point it is warranted, and keep the language in there for now? It seems overly bureaucratic (as is the PEP way, though.. hah!) to not allow specifics in the PEP that could easily be altered in the future if we change platforms but for now at least GitHub is showing us tons of support and we show no sign of moving platforms ever/anytime soon… so it would be nice to codify that as is

2 Likes
14

I think it is simpler to just leave GHSA in this PEP and just make sure it gets described as “because we use GitHub for source control, issues, and PRs already” with a note that “if cpython moves off of GitHub, we’ll include considering what to move to for security coordination purposes as part of that”. Listing the beneficial features it gives us. Feel free to list things it doesn’t as well (keeping us honest and lighting some fire under GH, as if they’ll ever read it), just noting that things we find lacking are not critical enough for us to spend lots of resources on other services today.

6 Likes
15

Well, to be fair, the list does seem to be inspired a lot by what GHSA has to offer :slight_smile:

infobyte/faraday: Open Source Vulnerability Management Platform is an alternative platform, which is open source, can be self-hosted, is written in Python and at least appears to provide similar features and integrations (it’s a bit hard to say because their marketing website is difficult to navigate).

It’s one of the tools listed on the Free for Open Source Application Security Tools | OWASP Foundation page.

That would be better, yes. And as @ncoghlan mentioned, the platform decision itself should most likely go into a separate PEP, since it deserves its own discussion.

There’s another aspect which needs to be addressed and should be mentioned in the policy PEP: Trust in the reporting platform.

Given that Python is one of the most used languages in the world today, vulnerabilities can have a significant value to different players, so the chosen reporting platform provides an excellent target in its own right.

Up until now you have managed these on a private mailing list which is hosted on PSF infrastructure, as I read your intro to the PEP, so there is a high level of trust built into the “platform” you’re currently using.

If you now switch to a different platform that you no longer control, you open up security reporting to a new set of possible loop holes which may even be hidden from your sight. E.g. GHSA runs on Github infrastructure to which you don’t have access.

There should at least be some guidelines in the PEP addressing the trust requirements for the platform decisions, IMO.

2 Likes