Private packages on github/gitlab - Inspiration from `nim`s `nimble`

Packaging
1

As organisations on gitlab/github often manage access using “access groups” that determine access to repos (collaborator/maintainer/owner vs reader), combining install rights with access controls seems highly beneficial to a vast range of organisations.

To benefit from github/gitlab for package management without the need for additional servers and any subsequently access management, I found a beautiful example from the nim-language, and the way it manages packages. Their workflow is:

  1. Write code
  2. Commit & push
  3. Run tests.
  4. Write release notes and “tag” the commit (more about releases here: Releases | GitLab )
  5. Hit publish.
  6. Done.

When the package manager for nim (called “nimble”) searches for packages, it needs only the URL (https:/github/{organisation}/{repo}) and, for private repos, a valid https access path. To resolve versions nimble seems to sorts the commit tags in chronological order using, e.g.: git tag --sort=committerdate and roll forward.

Back to python…

For our private python packages (our use case) my thoughts were if I (1) build container, run test suite, etc. as usual, (2) then create a release with .workflows / pipelines such that I first build the pypi package then create a release and upload the pypi package to the release described here, (3) then I have the .whl package in the release archive in github/gitlab.

For a non-maintainer (read-only) to be able to use the package to person will be able to use git by using a git authenticator using browser. After the git authenticator is setup (one time only), the person can:

[1] install manually by pointing the pip installer to the URL with pip install https://github.com/{organisation}/{repo}/archive/refs/tags/major.minor.patch.whl

[2] OR use a requirements.txt file and leave the bare URL as: git+https://github.com/{organisation}/{repo}/archive/refs/tags/major.minor.patch.whl

The only issue is that pip doesn’t sort the tags to find newer versions like nimble does.

Question: Is there a hidden feature (index server) where pip could resolve the version conflicts under these circumstances? Perhaps like nimble does it?

1 Like