Official GitHub Action for publishing to PyPI

Packaging
1

Hi,

So I’ve been exploring GitHub’s integration possibilities deeply during about a year now. I’ve even started a framework for writing GitHub Apps (and Actions actually, but that’s not the point) and consider myself knowing a lot in this area.

GitHub Actions feature is in a limited public beta mode now, but a lot of people already have access to it. Some of its uses have already been discussed at New GitHub feature: Actions.
In short, it allows building a graph (called workflow) of actions which can depend on each other triggered by certain events. And those actions (GitHub Actions) are just docker containers with customizable env vars, CMD and ENTRYPOINT in runtime. The execution of actions is orchestrated by GitHub platform as defined in the Workflow and they are run on the GitHub side.

It’s possible to create, share and versions reusable parts of workflows as GitHub Actions. If you store their declarations in separate repos you can publish them in the GitHub Marketplace (example page: Ansible Lint Action) so that others could easily locate and use them.
In a nutshell, Action would look like a repo with Dockerfile and README, metadata would be in dockerfile, readme would contain usage instructions.
Users can refer to Actions using GitHub repo addresses, in general.

Now, the reason I’m raising this discussion is that anyone is able to create Actions and publish them on Marketplace. This doesn’t include a review of any kind by GitHub, unlike typical GitHub Apps.
Also, it’s confusing for users when they’ll start seeing gazillions of Actions: which one is okay to use? is this one secure? etc.

That said, I suggest that PyPA should have an official curated Action published to Marketplace.
What would this Action do? Well, at first just twine upload. Going forward, it should be configurable with env vars and maybe have some built-in toggle for switching to Test PyPI.

How would this work? From the UX perspective, users would have to build their wheels in the previous Action and then use this one. Actions have a shared file system which means that artifacts stored by one Action are accessible by others.

Why would users have a separate Action to “just” publish dists to PyPI? Because the publish action will need a secret password set up in the env and from a security perspective it’s better to limit access to the secret as much as possible.

P.S. I was also thinking about having Actions for building dists (PEP517 and stuff) but that has to be thought through more carefully.

UPD:

Based on the feedback, a few things have been clarified:

  • The license should be BSD 3-clause (@brettcannon)
  • Actions are to be hosted under PyPA org on GitHub (@uranusjr’s concern is about polluting the org, but others seem to still want it under PyPA)
  • An Action is a low-level building block component; end-users would “play Lego” with it.
  • Action repos must have gh-action- prefix (@cjerdonek)
  • The repo name will be gh-action-pypi-publish (@pf_moore)
  • It should be one Action per repo because of the Marketplace requirements and maintenance simplicity + versioning is Git tag based which wouldn’t work for a monorepo (@cjerdonek)
5 Likes
2

(Some additional conversation about this idea has happened on Twitter.)

2 Likes
3

I doubt anyone would object to the idea of having a good Action for running twine, but what are you asking for exactly? For “the PyPA” to write this for you? For the PyPA to officially endorse an action that you already wrote? For the PyPA to officially endorse one that you haven’t written yet but plan to?

3 Likes
4

Yeah, the idea is to have a repo under PyPA and then I’d be installable as pypa/pypi-action, for example, rather than <some-experiments-org>/pypi-action. I’d like to write it: it’d literally take maybe ten lines in dockerfile in the very basic implementation.

5

full disclosure: I’d pointed @webknjaz to post here, since this is likely the best place to discuss this.

I like this idea and, as long as we have someone to maintain it (@webknjaz is willing, I understand), it should be fine to bring it under the PyPA name.

Since this is the case, I think the first actionable bit here is to actually write the action and then ping here again once that’s ready. A decent implementation that works for 80-90% of the cases should be enough to get going here (%-age pulled out of thin air).

Given that no one objects, I think it’ll then just be a matter of transferring the repo to be under pypa.

1 Like
6

Very initial Action (didn’t test it yet, so might contain typos): https://github.com/re-actors/pypi-action
And I’m not going to publish it to the Marketplace because it’s unclear how repo transfer would affect it. But feel free to play with it or provide some feedback :slight_smile:

7

Any specific reason for the GPL license?

8

I think it’s my current default. I don’t really see any reason to have it different

9

So I don’t know what PyPA’s policy is, but for instance the Python org on GitHub doesn’t have any GPL code on purpose to keep with the community norms of permissive licensing.

1 Like
10

Let’s wait for someone from PyPA to make a proper decision, then. If there’s no policy, I’d like to keep a GPL-compatible license.
Currently, I can’t even come up with a use case for vendoring this thing.

11

I’m not sure there is a PyPA policy, but as an individual PyPA member, I’d prefer a non-GPL license.

2 Likes
12

Any specific suggestions?

13

As a pip maintainer, I’d say do what pip does (MIT - which is also what I prefer for my projects, FWIW)

14

MIT is too “I don’t care” kind of thing. I’d like something better

15

It’s more than just vendoring. Technically I shouldn’t even look at your repo’s files it risks me taking anything you have done and incorporating it into any non-GPL code I work on which goes against the GPL.

Could you provide more guidance then on what’s acceptable? Like is BSD 3-clause acceptable?

16

Yeah, BSD 3-clause would work. @pradyunsg told me that the policy is to have anything OSI-approved.

As for GPL concerns, I don’t think it applies unless you were to fork an Action. Technically, you just point to it and GitHub runs it. So AFAICS it shouldn’t affect your code since you don’t really include it into your project. It’s like if you were to use some SaaS/PaaS you wouldn’t be required to relicense your own work.

17

Great!

Correct if you are just using this as an action. But the repo itself is being made publicly available, so if anyone chose to copy something from it, fork it, etc. that makes the license an important thing to consider.

3 Likes
18

So I’ve finished making it compliant with the requests above. What are the next steps?

19

Sorry for entering the discussion late. In the original post, you mentioned the aspiration of having multiple officially blessed Actions. Assuming each Action needs its own repo, would it be a good idea to group them in their own org, instead of adding a bunch of repositories under pypa?

Also, forgive my ignorance, but how would I be able to use a GitHub Action? I casually searched a bit and couldn’t find any obvious documentation from GitHub (there are several pages on how to make one, interestingly). Is it triggered automatically like CI, or do I get a button to press? Personally I prefer to treat releases as a conscious action; the process can be automated, but it should be an explicit “I want to release to PyPI now” action, instead of, say, done automatically when I push a tag.

20

Hi @uranusjr,

So the original post currently suggest only one Action: for publishing dists to PyPI. There may be another Action for building dists but I’m not sure yet about the details and whether it makes sense to make one. This will need another discussion. So, as you can see, it’s just one which doesn’t deserve it’s own org. If there will be more than 2‒3 we can resume this conversation and get back to this concern.

Since it’s in limited public beta, first go to https://github.com/features/actions/signup and add yourself to the waitlist. You can also do this for any organizations where you’re an admin. It takes quite a while so I recommend everyone to do it ASAP. (Or you can nicely ask https://twitter.com/AngieRvra — she helped me to enable Actions for the workshop participants)
If you are already in an organization having such access, you should be able to try it out on the existing repo or create a test repo there for yourself. If you want it right now — you can ping me and I’ll create a repo for you + grant access there.

There’s also this repo consolidating Action-related projects/info: GitHub - sdras/awesome-actions: A curated list of awesome actions to use on GitHub.

Back to using them. First, as a user, you create a file .github/main.workflow using GitHub UI and it drops you into a graphical interface for editing directed graphs (example screenshot: Feature Flag & Toggle Management Blog | LaunchDarkly).
So from there, you can create multiple workflows, which start with a trigger — it’s an event happening in GitHub platform which kicks off the flow of actions. Actions will have dependencies defining the order of execution, think of it as a pipeline.
Interesting fact: Actions are docker containers and they have a shared file system in runtime. So you can put some artifacts there and reuse them later.
So when adding actions via this UI, for each Aciton you’ll specify where to get it (image on docker hub, repo on GitHub or a relative path in the same repo).
You’ll also be able to override ENTRYPOINT and CMD on per action basis. And you’ll need to set normal env vars or secrets for specific actions (secrets are exposed as env vars to Actions but are non-readable from the GitHub UI, their names are unique per repo).

Now, there’s of course events happening all the time (like push or pull_request). But there’s also others, one of which is deployment (which is for some reason not very widely known, yet very useful). You can punch GitHub saying “hey, I want this commit deployed” (via API) and then it will broadcast that event to all GitHub Apps listening to this event. Actions are actually implemented on top of GitHub Apps so they’ll receive this event as well.

You came to the right address! I’ve actually prepared a demo using buttons.
Basically, using Checks API (available to GitHub Apps and Actions) you can add up to 3 buttons. They are accessible to folks with commit access to repo only. When user clicks that, your App/Action also receives an event (check_run with rerequested action and the button identifier). Guess what? You can react to that from GitHub Action as well.
The only problem is that I’m not sure whether you can build this completely on Actions because they have a restriction that events triggered via Actions don’t trigger other Actions. So I’ve just built a separate GitHub App which adds a Deploy button on Checks page for each commit received from push event and then it “consumes” clicks with a handler triggering deployment (code is here GitHub - sanitizers/diactoros-github-app: Diactoros `/dɪʌktɒrɒs/`: another name for Hermes, which literally means "the messenger." This is a GitHub App implementing a "Deploy!" button using Checks API.). The button is on this page https://github.com/sanitizers/octomachinery/runs/91107618 but you will not see it because of no write access to that repo. But here’s a screenshot for you:

GitHub_Actions_and_Checks_API_button
GitHub_Actions_and_Checks_API_button809×532 51.6 KB

Flow demo: https://github.com/sanitizers/octomachinery/actions/workflow-runs/MDEwOkNoZWNrU3VpdGU5MjI2Njk1OQ== / https://github.com/sanitizers/octomachinery/runs/92415775

Another way to trigger the same flow would by to hit API by yourself, I was imagining that I’d do this from Travis in the last stage but it’s only limited with your imagination.

I hope this answers your curiosity a little bit :slight_smile:

Ref: How to start using GitHub Actions for CI | Jason Etcovitch