As you might know,
the Steering Council is working on migrating the data that is currently residing in Roundup at https://bugs.python.org/ (BPO) into the GitHub issues of the CPython repository hosted there. The ultimate goal is to move user- and core developer-provided issue-reporting entirely to Github. We will leave BPO running in a read-only state to ensure existing URLs online continue working. Each issue that currently exists on BPO will include metadata indicating where it was moved on Github. New issues will only exist on Github.
We hope that this will lower the bar for newer contributors and allow for a much smoother user experience than we’re currently having. More details in the accepted PEP 581.
Unfortunately, this is not an easy task technically, procedurally, or legally, as it involves coordinating with several external actors and solving technical challenges mostly unique to our current circumstances. As a result, while progress was steady, it took a long while to get to this point. I was asked by the Steering Council to take over project management on the migration. Since January I’ve been working with @ezio-melotti and our friends on the Github side to push the transition to completion.
At the current stage, we’re asking you to take a look at the links and important dates below, and share any feedback you might have. To help us keep this process effective, please report concrete issues on https://github.com/psf/gh-migration/issues/. You can treat it as exercise in using Github issues Questions and general discussions of course welcome on https://discuss.python.org/ and python-dev.
Multiple dry-runs of the technical side of things were already completed, please take a look at some example migrated issues:
We are also reworking the Issue Tracker documentation in the Developer’s Guide, including a new FAQ section that is intended to answer any questions BPO users may have with regards to differences in the workflow when working with Github issues. See the rendered PR here:
Look for the “Issue Tracking” section which now includes two sub-pages: Github labels and “Github issues for BPO users”. If you’d like to see the raw PR, here it is:
We now propose the following migration roadmap:
- Friday, February 18th 2022: public feedback gathering period begins.
- Friday, March 18th 2022: final end-to-end test migration executed with Github’s help to gather timings and ensure no blockers. We will be using 10% of issues for that test.
- Friday, April 8th 2022 (sic) migration begins. BPO is put in read-only mode at 6pm UTC / 2pm ET / 9pm IDT. Data from BPO is exported and put in a temporary repository (this takes around ~22 hours with our current timings) on Github.
- Saturday, April 9th 2022: Github starts transfer of the issues in the temporary repository to github.com/python/cpython/.
The migration is estimated to take anywhere from 1 to 3 days, depending on the load on Github.com. This is why we will be performing the bulk of it during the weekend to speed things up. While the migration is happening:
- creating new issues WILL NOT be possible either on Github or on BPO;
- creating new PRs and interacting with existing PRs will be possible on Github without interruption;
- interaction with already migrated issues on Github is possible but destructive actions (changing issue titles, editing comment content, deleting comments, removal of labels) are HIGHLY DISCOURAGED as it will make it harder for us to audit whether the migration fully succeeded. In case we are unable to make the in-flight issues invisible until the migration is done, we will create pinned issues and bug tracker templates that explain the situation clearly to users.
Once the migration is over, we will notify everybody that they can interact freely with Github issues. In the unlikely case that the migration cannot be completed in 7 days, the Steering Council decided that we would abort it and re-enable BPO again.
Details of the plan along with some risk mitigation strategies are described in Migration and risk management plans · Issue #13 · psf/gh-migration · GitHub. That plan overrides what is currently written in PEP 588 at this point, we will merge the contents next week so that the PEP is up-to-date.
The LLVM project made a similar migration from Bugzilla in November 2021 and it took them 21 days to complete. We are lucky to be able to use their experience on the matter. We also have time estimates based on existing test runs and communication with Github employees. We are looking into ways of accelerating the process
but it looks like it might not be feasible to move our database of over 50,000 issues any faster than in 4 - 7 days. Github managed to figure out a way to speed up the migration so it takes less than 4 days in total.
Lastly, several legal concerns with respect to the procedure, process, and content were raised. In particular, the question of whether the Python Software Foundation should be able to move user-generated content as well as potentially personally-identifiable information from BPO to Github.com. The Steering Council together with Python Software Foundation lawyers resolved this issue with the following conclusion: the migration of BPO to Github is a ministerial, internal issue, and therefore one that doesn’t require user consent. Both BPO and Github are public-facing systems. Users actively placed their information (including PII) in the BPO system, which actively grants consent for that information to be stored, publicly accessible, and distributed on-demand. Changing our backend to Github does not revoke that permission. At the same time, the migration will not be surfacing any new user information that wasn’t previously publicly accessible in the BPO system.
Summing up, the migration is underway. We’re doing everything we can so that by PyCon US it will already be old news.
In the mean time, please look at the test runs to see if everything is clear to you. If you have any questions, please look at the devguide PR to see if they are already answered in the FAQ. If not, let us know to add the question.