tldr; we at Red Hat have developed a server-side resolver for Python packages and would like to offer it to the community. See our project page for more info.
Our mission is to offer an alternative to resolvers implemented in pip/pip-tools/Pipenv/Poetry and enhance the resolution process with “guidance” on which Python packages should be installed when developing Python applications. The guidance is done based on knowledge of packages - one can think of various issues that can arise when using packages, generally installation time issues or runtime issues (e.g. performance, security-based issues but also fatal runtime issues, …). Latest packages are not always the greatest choice and having some smart mechanism to guide the resolution process can save time-consuming debugging, project maintenance, or looking for the perfect set of packages to be installed for a specific application.
The whole resolution is based on reinforcement-learning techniques and has an extensible interface - the actual resolution process is seen as a resolution pipeline made out of pipeline units. These units can be written as Python classes. Alternatively, we also offer so-called “prescriptions” that declaratively (YAML) state how the resolver should behave in specific cases. The actual resolver implementation is available in the thoth-station/adviser repository.
We welcome the Python community and Python package maintainers in contributing to the open knowledge about open-source Python packages - either by reporting issues spotted or directly by opening a pull request to thoth-station/prescriptions or thoth-station/adviser repositories. We hope that we can create a better and more sustainable software that is powered by the Python programming language together.
Using this type of resolution process, we are able to resolve Python software stacks that require a certain set of ABI symbols being available in the runtime environment, fixing overpinning/underpinning issues, issues spotted after a release (less strict yanking that can be done just in some cases), security checks or perform cross-package index resolution. We also tried to address the limited express power of wheel tags (using a configuration file maintained by the user). As the resolution process requires pre-aggregated data, we run a background package analysis for packages hosted on PyPI or our optimized builds of TensorFlow.
The resolver is hosted on a publicly available Massachusetts Open Cloud so anyone can try it using Thamos CLI, jupyterlab-requirements extension or OpenShift S2I container image build process. We are already using this type of resolution process at Red Hat and we were able to resolve some Python software stacks for which Pipenv resolution failed; moreover, we try to give a better selection of packages to Python developers. On the other hand, we still have things to improve - note we are still in the development phase, fixing issues on multiple layers so please bear with us. Nevertheless, we are open to any feedback from the Python community and expertise possibly exchanged (any direction). If you have any questions or would like to participate feel free to reach out to us.
On behalf Thoth team,