Check if an environment's packages conflict

csm10495 · October 11, 2024, 12:34am

Hello folks,

I’m trying to see if there is a way to see if the current environment’s dependencies appear compatible with each other. I also recognize this makes more sense at install time than runtime, though I don’t have a way to modify the install-time process for my work projects. That brought me to pkg_resources.require.

Is there a modern equivalent to pkg_resources.require in importlib (or somewhere else)?

There is importlib.metadata.requires but that just gives the requirements, but doesn’t tell if the requirements are met/violated in the current environment.

If I ignore the pkg_resources deprecation, this pytest seems to catch environment compatibility issues:

from importlib.metadata import packages_distributions
from pkg_resources import require


def test_package_sanity():
    """
    This test is a simple sanity check. It will fail if the dependencies are incompatible.
    """
    issues = set()
    for packages in packages_distributions().values():
        for package in packages:
            try:
                require(package)
            except Exception as e:
                issues.add(repr(e))

    if issues:
        raise ValueError("Issues with dependency compatibility:\n" + '\n'.join(issues))

Is there a cli or third party package that basically does ^ already?

fungi · October 11, 2024, 1:08am

Is there a cli or third party package that basically does ^ already?

Usage:
  pip check [options]

Description:
  Verify installed packages have compatible dependencies.

sinoroc · October 11, 2024, 5:14pm

Cross-link:

github.com/pypa/packaging-problems

Solution to check if a requirement is met without pkg_resources

opened 01:07PM - 18 Apr 23 UTC

HansBug

### Problem description In the previous `pkg_resources` package, this issue can… be solved with the following code ([link](https://stackoverflow.com/questions/16294819/check-if-my-python-has-all-required-packages/16298328#16298328)): ```python import pkg_resources from pkg_resources import DistributionNotFound, VersionConflict # dependencies can be any iterable with strings, # e.g. file line-by-line iterator dependencies = [ 'Werkzeug>=0.6.1', 'Flask>=0.9', ] # here, if a dependency is not met, a DistributionNotFound or VersionConflict # exception is thrown. pkg_resources.require(dependencies) ``` However, `pkg_resources` has been deprecated, so this method is not recommended for continued use. In the [migration guide](https://importlib-metadata.readthedocs.io/en/latest/migration.html#pkg-resources-require) provided in the `importlib_metadata` documentation, there is no new interface that is completely equivalent to `pkg_resources.require`. Therefore, we need to use the `import_metadata` library (native in Python 3.8 or higher) and the `packaging` library to achieve this functionality. I have currently implemented a version, the code is here: https://github.com/HansBug/hbutils/blob/main/hbutils/system/python/package.py#L202, where the most critical part is the `_yield_reqs_to_install` function, which checks a requirement and its sub-requirements included in its extra one by one, and enumerates the unsatisfied requirements. By calling this iterator function, the check of whether a requirement is satisfied can be achieved. This is the core code of the function: ```python def _yield_reqs_to_install(req: Requirement, current_extra: str = ''): if req.marker and not req.marker.evaluate({'extra': current_extra}): return try: version = importlib_metadata.distribution(req.name).version except importlib_metadata.PackageNotFoundError: # req not installed yield req else: if req.specifier.contains(version): for child_req in (importlib_metadata.metadata(req.name).get_all('Requires-Dist') or []): child_req_obj = Requirement(child_req) need_check, ext = False, None for extra in req.extras: if child_req_obj.marker and child_req_obj.marker.evaluate({'extra': extra}): need_check = True ext = extra break if need_check: # check for extra reqs yield from _yield_reqs_to_install(child_req_obj, ext) else: # main version not match yield req def check_reqs(reqs: List[str]) -> bool: """ Overview: Check if the given requirements are all satisfied. :param reqs: List of requirements. :return satisfied: All the requirements in ``reqs`` satisfied or not. Examples:: >>> from hbutils.system import check_reqs >>> check_reqs(['pip>=20.0']) True >>> check_reqs(['pip~=19.2']) False >>> check_reqs(['pip>=20.0', 'setuptools>=50.0']) True .. note:: If a requirement's marker is not satisfied in this environment, **it will be ignored** instead of return ``False``. """ return all(map(lambda x: _check_req(Requirement(x)), reqs)) ``` Of course, if you need this function immediately, you can simply ```shell pip install hbutils>=0.9.0 ``` and then ```python from hbutils.system import check_reqs print(check_reqs(['pip>=20.0'])) print(check_reqs(['pip~=19.2'])) print(check_reqs(['pip>=20.0', 'setuptools>=50.0'])) ``` `hbutils` is a universal toolkit library containing various common functions, and the project address is https://github.com/HansBug/hbutils. It will be maintained for a long time.

mwr · October 17, 2024, 2:02pm

importlib.metadata.requires
packaging.requirements.Requirement
In general can build a package management system without pip.