# Pinning GNU Autotools using the GitHub container registry **URL:** \ **Category:** Core Workflow\ **Tags:** github, ci\ **Created:** [March 3, 2024, 12:30am UTC](https://discuss.python.org/t/pinning-gnu-autotools-using-the-github-container-registry/47437 "2024-03-03T00:30:32Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![erlendaasland](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/erlendaasland/32/4378_2.png) [@erlendaasland](https://discuss.python.org/u/erlendaasland)\ **Post date:** [March 3, 2024, 12:30am UTC](https://discuss.python.org/t/pinning-gnu-autotools-using-the-github-container-registry/47437/1 "2024-03-03T00:30:32Z")
## TL;DR @corona10 and I want to manage our GNU Autotools dependency using a custom container hosted by GitHub’s container registry. ## Background Until recently, we used [Christian’s Podman/Docker image](https://github.com/tiran/cpython_autoconf) to regenerate our configure script. The container image was hosted on quay.io. The reason for this, was that we had standardised on a particular distribution of GNU Autotools\[1\]. After we migrated from GNU Autoconf 2.69 to 2.72\[2\], we decided to use the stock Ubuntu 22.04 GNU Autotools, mostly in order to reduce our CI dependency on an external web service (quay.io). @vstinner created a wrapper script\[3\] (based on Christan’s container), which is what we’re currently using. ## Limitations with status quo Currently, we fire up a stock Ubuntu 22.04 container, run some `apt` commands to install GNU Autotools, and then we finally regenerate the configure file **every time we run the regen script**. If we want to upgrade the GNU Autoconf or `autoconf-archive` version, we have to fetch sources, configure, build, and install the packages, and at last regenerate the configure file **every time we run the regen script**. In practice: if we want a more recent GNU Autoconf, our regen script becomes larger and slower, as we have to do more things for each invocation. ## Proposal Donghee and I propose to go back to Christian’s approach of using a custom container, but instead of quay.io, host the repo on the `python` GitHub org\[4\] and use GitHub’s container registry for hosting the container. This should make configure regeneration faster, and it will be easier to pin and upgrade the various parts of the GNU Autotools suite. Donghee has a [proof-of-concept image](https://github.com/corona10/cpython_autoconf/pkgs/container/cpython_autoconf), based off of Christian’s container image. [Try it out](https://github.com/corona10/cpython_autoconf/pkgs/container/cpython_autoconf#cpython-autoconf) now: ```console $ podman run --rm -v$(pwd):/src ghcr.io/corona10/cpython_autoconf:272-final ``` * * * 1. earlier, the `configure` script would be generated by whatever people had installed in their development environments 2. in the 3.12 alpha phase IIRC 3. [cpython/Tools/build/regen-configure.sh at main · python/cpython · GitHub](https://github.com/python/cpython/blob/main/Tools/build/regen-configure.sh) 4. `python/autotools`? `python/buildtools`? ---
**Author:** ![brettcannon](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/brettcannon/32/34895_2.png) [@brettcannon](https://discuss.python.org/u/brettcannon)\ **Post date:** [March 4, 2024, 8:53pm UTC](https://discuss.python.org/t/pinning-gnu-autotools-using-the-github-container-registry/47437/2 "2024-03-04T20:53:13Z")
> [@erlendaasland](#): > > Donghee and I propose to go back to Christian’s approach of using a custom container, but instead of [quay.io](http://quay.io), host the repo on the `python` GitHub org[[4]](#footnote-150354-4) and use GitHub’s container registry for hosting the container. You beat me to this idea! 🙂 I was going to propose we start having dev-oriented containers under some repo so we could have them for general development of CPython (i.e. [cpython/.devcontainer/Dockerfile at main · python/cpython · GitHub](https://github.com/python/cpython/blob/main/.devcontainer/Dockerfile) as its own container). And then we could use that container in [cpython/.devcontainer/devcontainer.json at main · python/cpython · GitHub](https://github.com/python/cpython/blob/main/.devcontainer/devcontainer.json) so it wasn’t in any way confined to GitHub codespaces. We could even go so far as to use the container in CI or anywhere else where we needed an easy way to make sure we had (at least) a container with all the necessary bits to build and test everything under Linux. And as part of this I wanted to do the autotools container as well. So I’m totally supportive of this and having other containers that are dev-specific in some repo where we automatically publish the images for people to use. ---
**Author:** ![erlendaasland](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/erlendaasland/32/4378_2.png) [@erlendaasland](https://discuss.python.org/u/erlendaasland)\ **Post date:** [March 4, 2024, 9:46pm UTC](https://discuss.python.org/t/pinning-gnu-autotools-using-the-github-container-registry/47437/3 "2024-03-04T21:46:51Z")
You’re taking it to the next level: devcontainers 😄 > [@brettcannon](#): > > I was going to propose we start having dev-oriented containers under some repo so we could have them for general development of CPython A general container repo sounds like a good idea. ---
**Author:** ![erlendaasland](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/erlendaasland/32/4378_2.png) [@erlendaasland](https://discuss.python.org/u/erlendaasland)\ **Post date:** [March 5, 2024, 10:42am UTC](https://discuss.python.org/t/pinning-gnu-autotools-using-the-github-container-registry/47437/4 "2024-03-05T10:42:24Z")
> [@erlendaasland](#): > > A general container repo sounds like a good idea. Bikeshedding 🎨 🚲 : - `python/containers` - `python/devcontainers` (I think I prefer this one) Any better? ---
**Author:** ![brettcannon](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/brettcannon/32/34895_2.png) [@brettcannon](https://discuss.python.org/u/brettcannon)\ **Post date:** [March 5, 2024, 11:22pm UTC](https://discuss.python.org/t/pinning-gnu-autotools-using-the-github-container-registry/47437/5 "2024-03-05T23:22:58Z")
> [@erlendaasland](#): > > Any better? We could resurrect the “devinabox” name. “cpython-dev”, “cpython-devcontainers” also work. My concern of the more generic names is it won’t be self-describing for your fork (e.g. what’s `brettcannon/devcontainers`?). ---
**Author:** ![erlendaasland](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/erlendaasland/32/4378_2.png) [@erlendaasland](https://discuss.python.org/u/erlendaasland)\ **Post date:** [March 5, 2024, 11:35pm UTC](https://discuss.python.org/t/pinning-gnu-autotools-using-the-github-container-registry/47437/6 "2024-03-05T23:35:50Z")
`cpython-devcontainers` sounds good to me. ---
**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\ **Post date:** [March 6, 2024, 9:15am UTC](https://discuss.python.org/t/pinning-gnu-autotools-using-the-github-container-registry/47437/7 "2024-03-06T09:15:17Z")
A container to run Python tests in a CI and a container to develop Python sound like two very different use cases, no? ---
**Author:** ![brettcannon](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/brettcannon/32/34895_2.png) [@brettcannon](https://discuss.python.org/u/brettcannon)\ **Post date:** [March 6, 2024, 10:39pm UTC](https://discuss.python.org/t/pinning-gnu-autotools-using-the-github-container-registry/47437/8 "2024-03-06T22:39:42Z")
And through the magic of containers they can stack on top of each other! 😉 Seriously, it really depends on what you put in the containers as to whether they are massively different or one just adds extras on to the other. I mean how many times do we need to specify a C compiler and `make`? ---
**Author:** ![erlendaasland](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/erlendaasland/32/4378_2.png) [@erlendaasland](https://discuss.python.org/u/erlendaasland)\ **Post date:** [March 6, 2024, 11:26pm UTC](https://discuss.python.org/t/pinning-gnu-autotools-using-the-github-container-registry/47437/9 "2024-03-06T23:26:16Z")
> [@vstinner](#): > > A container to run Python tests in a CI and a container to develop Python sound like two very different use cases, no? It is fine to use a container with the correctly pinned GNU Autotools for multiple use cases 😉 ---
**Author:** ![erlendaasland](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/erlendaasland/32/4378_2.png) [@erlendaasland](https://discuss.python.org/u/erlendaasland)\ **Post date:** [March 10, 2024, 11:05am UTC](https://discuss.python.org/t/pinning-gnu-autotools-using-the-github-container-registry/47437/10 "2024-03-10T11:05:57Z")
It seems there is no pushback on this, so let’s get it done. IIRC, we’ll need to open an issue on the SC tracker in order to get permission to create a new repo under the `python` GitHub org. > [@erlendaasland](#): > > `cpython-devcontainers` sounds good to me. Any better suggestions? ---
**Author:** ![hugovk](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/hugovk/32/14505_2.png) [@hugovk](https://discuss.python.org/u/hugovk)\ **Post date:** [March 10, 2024, 1:05pm UTC](https://discuss.python.org/t/pinning-gnu-autotools-using-the-github-container-registry/47437/11 "2024-03-10T13:05:24Z")
I’d probably avoid the `cpython-` prefix, many repos under the org relate to CPython but aren’t prefixed, and you can always rename your fork if you like to avoid ambiguity. If prefixing, a hyphen usually works better than an underscore (for ctrl+left and suchlike) and looks like more repos at [Python · GitHub](https://github.com/python) have hyphens than underscores. Might there be other containers in general than devcontainers? ---
**Author:** ![erlendaasland](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/erlendaasland/32/4378_2.png) [@erlendaasland](https://discuss.python.org/u/erlendaasland)\ **Post date:** [March 13, 2024, 10:36pm UTC](https://discuss.python.org/t/pinning-gnu-autotools-using-the-github-container-registry/47437/12 "2024-03-13T22:36:31Z")
> [@hugovk](#): > > If prefixing, a hyphen usually works better than an underscore (for ctrl+left and suchlike) and looks like more repos at [Python · GitHub](https://github.com/python) have hyphens than underscores. I proposed `cpython-devcontainers` on the SC repo ticket. BTW, [the SC approves](https://github.com/python/steering-council/issues/236#issuecomment-1995417319). ---
**Author:** ![gpshead](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/gpshead/32/54_2.png) [@gpshead](https://discuss.python.org/u/gpshead)\ **Post date:** [August 4, 2024, 7:17pm UTC](https://discuss.python.org/t/pinning-gnu-autotools-using-the-github-container-registry/47437/13 "2024-08-04T19:17:21Z")
It doesn’t look like this happened, did something eliminate the current need for it? ---
**Author:** ![erlendaasland](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/erlendaasland/32/4378_2.png) [@erlendaasland](https://discuss.python.org/u/erlendaasland)\ **Post date:** [August 4, 2024, 8:17pm UTC](https://discuss.python.org/t/pinning-gnu-autotools-using-the-github-container-registry/47437/14 "2024-08-04T20:17:21Z")
Other stuff intervened and it got pushed back in my todo-list. IIRC, @corona10 and I made a list of remaining items at PyCon. I’ll get to it within a couple of weeks. ---
**Author:** ![corona10](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/corona10/32/9943_2.png) [@corona10](https://discuss.python.org/u/corona10)\ **Post date:** [August 4, 2024, 10:39pm UTC](https://discuss.python.org/t/pinning-gnu-autotools-using-the-github-container-registry/47437/15 "2024-08-04T22:39:28Z")
As I left a message at Discord, how about working this task during the core sprint? Core sprint is only a month left. (Last few weeks were too busy with my personal issue) ---
**Author:** ![erlendaasland](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/erlendaasland/32/4378_2.png) [@erlendaasland](https://discuss.python.org/u/erlendaasland)\ **Post date:** [August 4, 2024, 10:55pm UTC](https://discuss.python.org/t/pinning-gnu-autotools-using-the-github-container-registry/47437/16 "2024-08-04T22:55:25Z")
> [@corona10](#): > > As I left a message at Discord, how about working this task during the core sprint? Core sprint is only a month left. Sounds good to me! I can’t be there, but I’ll try to keep up with you from CET 🙂 ---
**Author:** ![brettcannon](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/brettcannon/32/34895_2.png) [@brettcannon](https://discuss.python.org/u/brettcannon)\ **Post date:** [August 7, 2024, 9:56pm UTC](https://discuss.python.org/t/pinning-gnu-autotools-using-the-github-container-registry/47437/17 "2024-08-07T21:56:53Z")
> [@erlendaasland](#): > > > [@corona10](#): > > > > As I left a message at Discord, how about working this task during the core sprint? Core sprint is only a month left. > > Sounds good to me! I can’t be there, but I’ll try to keep up with you from CET 🙂 I was also planning/hoping to work on this at the core sprints. ---
**Author:** ![corona10](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/corona10/32/9943_2.png) [@corona10](https://discuss.python.org/u/corona10)\ **Post date:** [September 6, 2024, 12:51pm UTC](https://discuss.python.org/t/pinning-gnu-autotools-using-the-github-container-registry/47437/18 "2024-09-06T12:51:15Z")
IIRC, @erlendaasland and I discussed the ruff idea with following topics - Version semantics: Because of a minor change in the Dockerfile itself, the autoconf image should be able to be released on the same day with the same version, so we preferred the combination of Samver and Calver. - Repository structure: We should care about multiple container image in a same repo. - Deployment process to the GH image registry: We already have a demo at [GitHub - corona10/cpython\_autoconf: Fork from https://github.com/tiran/cpython\_autoconf](https://github.com/corona10/cpython_autoconf) But I hope we can discuss it in more detail at the sprint. @erlendaasland, do you have any dumps on our discussion? ---
**Author:** ![corona10](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/corona10/32/9943_2.png) [@corona10](https://discuss.python.org/u/corona10)\ **Post date:** [September 24, 2024, 3:32am UTC](https://discuss.python.org/t/pinning-gnu-autotools-using-the-github-container-registry/47437/19 "2024-09-24T03:32:28Z")
Working on [GitHub - python/cpython-devcontainers: Repository for devcontainers of CPython](https://github.com/python/cpython-devcontainers) ---
**Author:** ![corona10](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/corona10/32/9943_2.png) [@corona10](https://discuss.python.org/u/corona10)\ **Post date:** [September 24, 2024, 5:46pm UTC](https://discuss.python.org/t/pinning-gnu-autotools-using-the-github-container-registry/47437/20 "2024-09-24T17:46:08Z")
@EWDurbin cc @brettcannon @erlendaasland I would like to request PAT token for Python Github org, because we need to publish packages to the [Packages · Python · GitHub](https://github.com/orgs/python/packages) through [cpython-devcontainers/.github/workflows/release.yml at 8cc887d0c233566fb57dceea141f4ca30b08e8a8 · python/cpython-devcontainers · GitHub](https://github.com/python/cpython-devcontainers/blob/8cc887d0c233566fb57dceea141f4ca30b08e8a8/.github/workflows/release.yml#L35) [Next page](https://discuss.python.org/t/pinning-gnu-autotools-using-the-github-container-registry/47437.md?page=2)