Looks like everyone is doing this — AMA!
7 Likes
Hi Lucas.
What are your thoughts on reducing the confusion between conda (the open source tool), Anaconda.org channel for packages (which is often referred to as conda), and conda-forge? I think that there is still quite a bit of confusion in the Packaging Ecosystem about these terms.
With the growth of data science, scientific python, and use of Python in HPC (high-performance computing), how do you envision the PPC making progress to improve the end user experience when using GPUs?
1 Like
Hi Carol!
I think the Python Packaging User Guide would be a good place to add some documentation! There are already docs scattered around, including Conda pypi - Pixi, A brief introduction | conda-forge | community-driven packaging for conda, and Frequently asked questions | conda.org, but distilling them down into one document targeted at Python users and packagers should help reduce confusion.
I think the wheel variants PEPs are the highest priority here, as they are probably the most impactful prospective change for the average user trying to use a GPU.
I’m also very keen to see PEP 725 – Specifying external dependencies in pyproject.toml | peps.python.org moved forwards — one of the reasons conda-forge has been so dominant in this space is the prevalence of external dependencies in such workflows. We discussed this PEP at the EuroPython 2026 Packaging Summit, and the key blocker is figuring out the governance/authority over the proposed mappings of PEP 804 – An external dependency registry and name mapping mechanism | peps.python.org. I think the PPC should prioritise providing a solution here, maintaining a structure which houses the source of truth for the mappings.
Not only will PEP 725 make the user story much nicer (with much improved error messages at the least), but it should also (once adopted widely) provide a massive help to package distributors. In particular, having an upstream source of truth for external dependencies should help reduce distribution-specific dependency whack-a-mole and workarounds, which is one of the key headaches for improving UX of Conda-PyPI interoperability identified at User Experience Notes for Conda+PyPI Interop for Pure-Python Wheels - HackMD.
Zooming out a bit, I think that the most important thing the PPC can do is stay constantly in touch with the tools people are using to solve these problems (like Pixi), and the distributors who are making sure the packages are available to these communities. I’m pleased that there are more than a few other candidates who are also well-placed for such a task!
3 Likes
Having this as a focus is very valuable IMO. Having one source of truth where all kowledgeable people in the various ecosystems can pool what tools/OSs they encounter and how to deal with them would be very helpful. I co-maintain applications that get deployed on various systems and have
With PyPA/CPython packaging[1] and Conda, Python has two separate packaging ecosystems with different but often overlapping communities and capabilities. As a pixi maintainer, you’re right at the intersection of both. Beyond PEP 804 and PEP 725, how do you think the two will evolve wrt to each other and where do you see them in the future, both in terms of capabilities and in terms of interoperability?
Do we have a name for the non-conda, PEP-based packaging ecosystem? ↩︎
2 Likes
Hey Konsti! I’ll try to answer in three sections.
Coexistence and interoperability
Distinct ecosystems
I see both remaining as distinct ecosystems long term — as @rgommers mentioned recently at PEP 825: Wheel Variants, Package Format (split from PEP 817) - #198 by rgommers, there are limitations of PyPI’s author-led social model that can be avoided by system distributions with a more centralised social model. At the same time, it would be too disruptive and restrictive to give up on that author-led model entirely. Of the potential solutions discussed at PyPI's author-led social model and its limitations - pypackaging-native, I see
Improve interoperability with system package managers, so that the strengths and weaknesses of language and system package managers can be combined by users.
as something for the PPC to prioritise, and
Separating source and binary distribution on PyPI better (as discussed here) and/or building a build farm (as discussed here) may mitigate some of the issues.
as ideas which are worth exploring (but perhaps are more for the long-term, after the terms of the roles currently being elected).
With that said, I see the possibility of installing packages from both ecosystems via a single tool as an important point of interoperability that we should keep alive.
Interoperability
Focusing in on interoperability efforts: beyond PEP 804 and PEP 725, I think further work on PURLs will be valuable (see GitHub - conda-incubator/sig-purls: Special Interest Group to explore PURL integration and usage in the conda ecosystem · GitHub for early work towards what we need from the conda side.). As mentioned at User Experience Notes for Conda+PyPI Interop for Pure-Python Wheels - HackMD, being able to identify a conda package’s dependencies, the upstream source it is packaging, and the package itself via PURLs will go a long way to enabling smoother mappings (more robustly than how we are currently achieving Parselmouth: Conda to PyPI Map) between PyPI and conda-forge.
One line of work I would really like to see pursued is making it easier for maintainers of PyPI packages to be the maintainers of their corresponding conda packages, with very little extra effort. I think it is a big source of friction between the ecosystems that these maintainer teams often have no overlap in individuals, and that is likely rooted in it not being easy (enough) for maintainers of PyPI packages to contribute to conda-forge. There are a few possible avenues we could pursue (post PEP 725 and further work, perhaps including https://discuss.python.org/t/pre-pep-standardizing-test-dependency-and-command-specification), including automatic generation of recipes in a format already accepted by conda-forge’s tooling from project-level metadata, or adding new tooling to conda-forge which can cut out the recipe middleman and build packages directly from a format that can be specified in pyproject.toml.
In other words, I think we should work to keep the most useful parts of the centralised social model which system distributions like conda-forge have, while maximising the ability for author-led participation and ownership. Our work in Pixi on pixi-build-python - Pixi is a pretty solid base for future interoperability work here.
For what it’s worth, I think I’m less convinced than Dan of the prospect of work on the PyPA ecosystem’s tools gaining the ability to install conda packages (ref. PPC nomination AMA for Dan Yeaw - #3 by danyeaw). Especially once we have wheel variants, I think it is fair to say that a large majority of Python users will be served well by tools which stick to the author-led social model, while other users who are hitting things like ABI-related problems are probably better off using a tool native to the conda ecosystem (for the projects where those problems apply). Overall, I think such work makes sense as something to aim for eventually, but is not something valuable to prioritise in the near future.
Pixi’s place
I’ll say a bit with my Pixi maintainer hat on. Regarding work on these interoperability challenges, I think it makes sense for Pixi to serve a similar role as I see uv has for new standards and features for the PyPA ecosystem: providing a space for fast prototyping and implementation of standards, meeting users with new and outstanding needs where they are. For example, I think Pixi is the perfect place to (as an ambitious goal!) prototype and perhaps implement a unified solver for packages from the two ecosystems (ref. Conda pypi - Pixi).
To speak briefly on the commercial tooling aspect, I echo almost exactly the words of @woodruffw at PPC nomination AMA for William Woodruff - #16 by woodruffw — were the funding situation for Pixi maintainership change significantly, a community fork seems like a realistic and not so bad outcome. But, even if that didn’t transpire, the value provided by such fast prototyping in unlocking better user experiences and providing practical experience to guide standards like PEPs and CEPs will persist.
Where I see the PPC fitting in
Aside from the existing discussions and prioritisations already mentioned, at least for the term of those being elected to the council now, I don’t think there is much else to say regarding significant changes to the relationship between the two ecosystems. Instead, for the following 1 to 2 years, I think the PPC should focus on establishing solid cross-participation in standards between the ecosystems. Giving someone the bandwidth to stay on top of the relevant PEPs and CEPs (as well as related standards discussions in other ecosystems) and provide informed feedback on both sides would be very useful, IMO. Also, gathering input from different system distribution maintainers as a matter of procedure in Python discussions, and vice versa, seems like something valuable for the PPC to mediate.
2 Likes
Hey Lucas, thanks for bringing me in to your AMA 
. Tell me more, why do you think Pixi’s method of having to do a dual solve (once with resolvo and once with uv) and users having to pass in --pypi is a better solution? Rattler already has support for doing a single solve with wheels, so it wouldn’t be that much work for Pixi to support a conda-pypi channel.
As a Pixi maintainer, how will you stay neutral to ensure that standards are good for the whole ecosystem and not just a single tool?
Ah sorry, sounds like there is a misunderstanding, I don’t think having the dual solve is a better solution than having a single solve, hence my mention of the ambitious goal of a unified solver. I wasn’t thinking about the conda-pypi strategy of making wheels available via a conda channel when I was writing the post, but I agree that it makes sense as an avenue to explore. I think the key questions for that are around who maintains the channel infrastructure and how to keep that sustainable long term. An advantage of being able to install directly from PyPI without the channel in the middle is that one can piggyback directly on what the PyPA ecosystem is already maintaining.
For standards that are developed as part of the CEP process, I’m keen to advocate for the conda steering council maintaining a varied composition that is representative of community interests. For standards that are developed as part of the PEP process, I’m keen to advocate for involvement of the conda ecosystem as a whole into discussions, as well as involvement of representatives from other distribution communities. From a PPC perspective, I would plan to be conscious of when I am representing a community as a whole (rather than a subset around a specific tool) to a different community, and aim to gather and present a balanced view in those cases.
1 Like
Thank you for stepping up to serve on the PPC. Your background with SciPy brings a crucial perspective. Given your love for shiny new tools like uv and Pixi, can you also commit to actively advocating for the needs of pip users? Since pip users represent the vast majority of the Python user base, can you promise to keep them and their need for a stable continuity without breakage in mind as well?
The PPC will take on an important large amount of work. How much weekly time can you commit to for the in-depth review of proposed PEPs that don’t touch your preferred interests?
Yes. I love the shiny tools for how they can help expand the horizons of what we think possible to offer across the ecosystem, but in an important sense the real work comes in the PEP process, ensuring those new ideas can outlive the tools of the day in standards. Given the vast amount of pip users out there, I think it is important that these standards always work to bring pip along for the ride (if the pip maintainers so choose), and the most important part of that is crafting standards in a way which does not imply unacceptable breakage.
I would be keen to speak to pip users to find out what the biggest community requests are, and what is blocking them from being addressed.
I think anything up to 5 hours would be feasible. I’m optimistic that the council will have a varied composition, where there are likely others with expertise in areas not of my preferred interests, so in practice this may be less of a concern. However, I do think it would be healthy for members of the council to deliberately learn from each other and expand their own horizons of expertise — co-reviewing PEPs could be a nice way for that to happen.
1 Like