Programmatically getting non-optional requirements of current directory

Packaging
1

My goal is to programmatically extract the requirements of the current directory without the optional ones. So far, this is the closest I got:

from build.util import project_wheel_metadata
from pathlib import Path
package_metadata = project_wheel_metadata(Path(".").resolve())

print(package_metadata.get_all("Requires-Dist"))

However, this does include everything under the optional-dependencies subtable, which I don’t want.

As a result, I’m considering manually loading pyproject.toml and extracting the dependencies list from it:

try:
    import tomllib
except ImportError:
    import tomli as tomllib

with open("pyproject.toml", "rb") as fh:
    # Assume PEP 621
    reqs = tomllib.load(fh)["project"]["dependencies"]

but I’m not sure if I’m doing something terribly wrong here.

<rant>If I’m allowed to rant just a little bit, the only reason it only took me half an hour to get to this point is because I’m a Python packaging nerd already - but I’m not even nerd enough, so I’m not sure if there’s a function that perfectly does what I need, or if I’m about to hack something horrible to remove whatever contains the extra == string.

If someone could illuminate me, I’ll try to find some time to improve the documentation wherever possible so that nobody else has to go through this rabbit hole, but for that I’d also need to understand which docs to update (official Python docs? packaging user guide? pypa/build? importlib_metadata? something else?). </rant>

In the meantime, I opened an issue on pypa/build about the docs of this function Confusing return type of `build.util.project_wheel_metadata` · Issue #619 · pypa/build · GitHub and any help in achieving my original goal (if you already forgot what it was because you’re crafting a response to my rant, please scroll back to the top of my post again) is more than welcome.

2 
import packaging.requirements

[
    r
    for d in m.get_all('Requires-Dist')
    for r in (packaging.requirements.Requirement(d),)
    if not r.marker or r.marker.evaluate()
]

Simple, right? :stuck_out_tongue_winking_eye:

(I expect this might filter out dependencies based on other environment conditions as well, e.g. the platform.)

3

I agree it’s short. “Simple” is in the eye of the beholder :wink:

Relevant docs: Requirements - Packaging, Markers - Packaging

Thank you!

(I expect this might filter out dependencies based on other environment conditions as well, e.g. the platform.)

Marker objects have no properties, how should we check whether it’s an extras == marker or something else?

4

It’s quite possible the Marker API could be improved - as far as I’m aware it is mostly focused on the use case of answering the question “does this apply in the given environment?” For the use case of filtering requirements, where you want to specify some parameters and return the resulting (partially resolved) marker, there doesn’t seem to be a good API yet.

2 Likes
5

Something as simple as addding a partial_match flag to evaluate might work - best to open a ticket on the packaging tracker I think.

6

I think there are a few issues open already about having more flexible APIs for Markers, see for example No way to do operations on marker · Issue #448 · pypa/packaging · GitHub and Expose Marker tree structure in the API · Issue #496 · pypa/packaging · GitHub. I added a comment to the former.

7

Weird that nobody recommended pyproject-metadata!

from pyproject_metadata import StandardMetadata

parsed_pyproject = { ... }  # you can use parsers like `tomli` to obtain this dict
metadata = StandardMetadata.from_pyproject(parsed_pyproject)

print(metadata.dependencies)
8

I didn’t see this topic until now, but yea, pyproject-metadata is basically meant for what you want to be doing here. :slight_smile:

1 Like