Thanks for the insight. I had not realized that backtracking from the resolver to the selection process was not an option.
I’ll try to come up with a different way to do things. My first gut feeling is to try a scheme that filters packages based on variants, then passes that into the finder. I recognize that all of this is speculation and none of it means anything without a prototype to study behavior, but I’m mostly posting this to iterate on ideas out in the open.
- Installer recurses the dependency chain, downloading variant info for each package. I believe that a global approach to the variants will be necessary for ensuring alignment, given what Paul has said about how the finder works. Variant info looks like:
{
"variant_name": {
"most_preferred_value":
"excludes": ["another_variant_name==somevalue"]
"compatibility_range": ""
"less_preferred_value": {}],
}
This is Oscar’s design, more or less, but with some additions to express compatibility or incompatibility with other variants
- The user specifies which variants to prioritize by listing them in order in some configuration. Prioritization can be done as:
- maximization of number of packages in the graph that have a given feature with a given value (maximize usage of the feature)
- maximize number of packages in the graph that have the latest value of a given feature (optimize for most preferred variant value over broadest availability variant)
-
The installer optimizes the collection of variants (possibly formulated as a resolver problem). This is close to how Conda used to operate, and I think the problem space of just variant names and values (NOT packages and dependencies!) will be small enough to be fast with a pure python implementation. Again, speculation, and I know this needs to be proven.
-
The installer uses the collection of variant values to select a subset of the available wheels for each package in the recursed, exploded package graph. This subset goes into the finder as it exists today.