CLA-assistant is No-Go đźš«

You might remember a previous discussion about using cla-assistant for Python and retiring the-knights-who-say-ni.

Unfortunately, after further discussions with The PSF, it was determined that the out-of-the-box CLA-assistant is not sufficient for Python/The PSF’s needs.
The main issue is needing to support (and enforce) CLA signed on behalf of organization, and checking if an individual making a PR is covered by an organization’s CLA. cla-assistant does not handle this. We might be able to modify cla-assistant to do what we want it to do, however the point of using cla-assistant is so we don’t have to maintain it ourselves.

I don’t have time available to research alternative CLA system out there. If anyone knows a better system, please do share. This has dragged for so long… :pensive: if I don’t hear anything about alternative solutions, then maybe we want to start looking into what it takes to build our own the very best Python CLA host and enforcement system™️

Thanks so much @EWDurbin @ewa.jodlowska @brettcannon @VanL for bearing with me this past year. I’m sorry to say that for the time being we still need to live with our current workflow.
I know it is disappointing, and I especially feel bad for PSF team members who need to do manual work of checking newly signed CLA. :worried:

9 Likes

Eh, it is what it is. We did at least try and now we have a better understanding of what exactly we need a CLA system to handle. We can also continue to try and make the-knights-who-say-ni usable for more projects since we probably aren’t the only project in this position.

And thanks for trying to make this work.

3 Likes

Thanks for giving it your best effort @Mariatta. :snake: :sunny:

2 Likes

Sorry to hear it, but this is understandable. Maybe the PSF should just try to fund development of a fork of CLA Assistant that works for us, or maybe there are other tools around for purchase that can do what we need (it’s not just the PSF that has special rules for organizations – corporations like Google and Dropbox also do this, e.g. https://opensource.dropbox.com/cla/).

I actually looked around before I created the-knights-who-say-ni and I didn’t find one that quite did we wanted. Ironically a lot of them are not open source.

Just to clarify, what is the exact requirement here? Is it that there must be some sort of interface on the CLA signing page that lets you sign one as part of an organization? (Like from the example Guido linked above)

Presumably it’s this: when a person (a lawyer, typically) for an org/corp signs the CLA on that org’s behalf, we should be able to automatically set the “CLA accepted” flag for all users in that org (e.g. recognized by the domain in their email address, if that org/corp has a sufficiently strong policy for handing out email addresses).

There could be all sorts of technical and legal issues with this sort of thing, but without it, we’d effectively be encouraging org/corp employees to sign an individual CLA (rather than going through legal), and then the PSF won’t be covered (because the org/corp’s lawyers could later just claim “but that employee wasn’t supposed to do that” and sue the PSF anyway – the very thing that the CLA was supposed to protect the PSF from).

I have no idea how this is done today, but clearly there’s a manual part of the process that might involve looking for some kind of indication that an individual is or isn’t an employee of an org/corp that did or didn’t sign the CLA.

I don’t know how watertight the current process is, e.g. what do we do about contractors and ex-employees. But with a human in the loop presumably the PSF legal department (i.e. Van Lindberg) is happy.

Thanks for the clarification, it helps a bunch. I think it might be useful to know how many people contribute individually or as part of a organization CLA.

I’m certainly not a lawyer but would this be a possible solution? Use cla-assistant’s custom field features, to ask if they’re signing on behalf of their employer:

    "category": {
        "title": "How do you sign?",
        "type": {
            "enum": [
                "I am signing on behalf of myself.",
                "I am signing on behalf of my employer."
            ]
        },
        "required": true
    }

During a PR, the-knights-who-say-ni checks if the CLA was signed on behalf of an organization. If so, it marks the status as CLA not signed until it can be manually reviewed like usual. If most contributors submit as part of an individual CLA, this might streamline the process for most contributors while still allowing some safety in the organizational case.

Additionally, it looks like cla-assistant recently (2 weeks ago) got a feature to check if a person belongs to a particular organization on Github: Corporate CLA (membership check) functionality by ibakshay · Pull Request #462 · cla-assistant/cla-assistant · GitHub which might cover most corporate contributors.

I encourage you to contact Van Lindberg directly about this.

I followed that thread, and what I understand is that PR caused a bug with their whitelisting feature, and therefore the commit was reverted. The issue about “can CLA signed by org” is still open. (Can the CLA assistant handle CLA signed by organisations · Issue #414 · cla-assistant/cla-assistant · GitHub)

“checking if CLA was signed on behalf of an org” is not currently supported by cla-assistant or the-knights-who-say-ni. I don’t know how to modify cla-assistant to support this (java code, not my forte).
I would say it is possible to make the-knights-who-say-ni reads the CLA hosted by CLA-assistant, I just don’t know how much effort would that be. I think it would require understanding CLA-assistant’s architecture, something I’m not actually interested in, and I don’t have time to do it.

We can just wait until cla-assistant is fixed, but we don’t know how long it will take.

Summary:

  1. We could wait for cla-assistant (unsure for how long)
  2. Someone could tweak cla-assistant/the-knights-who-say-ni
  3. While we wait for one of the above, perhaps someone knows of a better solution?
  4. Since we’ve spent a year to get this far, and I don’t want to have to wait another year, perhaps it’s just time to build our own open source solution.

It’s that with the added wrinkle that the org-signed CLA only covers contributions from a person made on the behalf of that org, but not personal contributions. This is why the solution was devised that when an org signs a CLA we could then accept contributions under that CLA only if the PR originates from a fork from that org’s GitHub organization which is a clear delineation of where the work is coming from.

Unfortunately not due to the above point plus I suspect we want org’s to control who signs the CLA versus letting any old person claim it’s signed on behalf of an organization.

Yeah, I think this is quickly converging on “make the-knights-who-say-ni nicer and then expand it for corporate CLA signing”. The bot is already designed to abstract out key parts like CLA storage so other projects could easily fork it and use it for their custom needs. We could continue with keeping it flexible and easy for other projects to pick up and use so we are the last major project that has to solve this problem.

1 Like

Could we use CLA-Assistant for individual contributions and a separate solution for org contributions? Surely it’s more important to lower the bar for individual contributors.

2 Likes

Technically yes. As long as we don’t make the status checks required and we’re willing to ignore failing checks no matter what, otherwise we are going to have two status checks that regularly act as XORs of each other, making all PRs get flagged as not passing in some regard.

Shouldn’t it be pretty simple to have a wrapper check that OR’s the two conditions (or maybe even a separate check suite)? I’ll admit that I’m not familiar enough with github CI to create this myself, but it sounds tractable.

Not necessarily as most status checks use webhooks to call back into the GitHub API to do the updates, so you would need to somehow convince both solutions to not tell GitHub what the status is, else we would have to write out own webhook to detect the status checks to then merge them into a single status check.

IOW it isn’t exactly “pretty simple”. :wink:

Combining these ideas, would it be feasible for the PSF to fund the development of writing that webhook? If doing so could potentially fix the CLA issues, it would definitely be worthwhile.

And while not necessarily simpler, but another approach would be have a check for orgs and then have a webhook that will override either the org or cla-assistant check when one of them succeeds so that the PR is considered passing.

Building on the “Supplement cla-assistant, don’t replace it” idea, would an approach along the following lines be plausible?:

  1. Use cla-assistant by default, and have it always be a required check. This means that folks wishing to contribute on behalf of their organisations will also need to sign on as individual contributors - we wouldn’t accept their PR just because they pushed it from their org repo. (Variant if that isn’t acceptable: have the cla-assistant check be advisory, and let the knights-who-say-ni determine if it is needed)
  2. The knights-who-say-ni check status would still control setting the “CLA Signed” label. If the PR comes from a registered org repo, then check for org CLA approval (however that is done), while if the PR comes from any other repo, just echo the cla-assistant result.

Edit: actually phrase my question in the form of a question.

Based on my experience with legal departments, this won’t work.

FYI if we keep the-knights-who-say-ni around we will move away from the label and to a proper status check.

But then what’s the point from our perspective about corporate CLAs? If everyone has already signed a personal one then it’s taken care of from our perspective and it’s between the contributor and their company about whether they can contribute on work time.

If three categories are used for CLA -assistance
i) Individual
ii) Organization
iii) Combined (individual and organization combined)

in third category individual share rights with organization