PyPA governance
Future grants and funding priorities – what are our biggest pain points now?
Binary packaging
Expression of dependencies on externally provided software (things that pip does not/should not provide). Metadata encompassing binary compatibility that may be required for these expressions.
With the advent of pip’s solver, the quality of metadata will be much more heavily tested. Pip currently warns (?) but allows creation of inconsistent environments, where one or more constraints don’t agree. There’s two topics here that might need to be considered separately:
- Can/should pip be disallowing installation of packages that break consistency by default?
- How can PyPI deal with bad metadata? From Conda’s experience, once it is in the index, things will be badly disrupted until either the offending package is removed, or the metadata is altered. Conda alters the index metadata, but with the distributed ownership of packages of PyPI, I’m not sure that’s an option.
Have pyproject.toml
standardize on wheel-based/common metdata for that tools are expected to use if (optionally) provided by the user?
E.g. we recognize when version
is set in some [metadata]
section, but tools are allowed to provide their own way of specifying the details as well. Basically help build tools to not re-invent how to specify metadata that every wheel needs unless they are providing a value-add that warrants it (e.g. flit pulling the version number from the source directly). This is motivated by:
- Making it easier to change your build tool and not have to do tremendous updates to your
pyproject.toml
as part of the transition simply because a tool chose different naming - Make it easier to do tool-agnostic analysis, e.g. know what dependencies a wheel end up specifying (if specified statically)
Future of python_requires
, fixing bad uploaded versions and making it mandatory.
https://github.com/pypa/warehouse/issues/3889 is relevant.
PyPA communication (external and internal)
“end to end” pipeline testing for packaging tooling
Updating PyPA Roadmap
This one’s sort of long term and loosely tied to the communication discussion – it’d be nice if we’re able to start a conversation about some long-ish term planning too.
@njs could you please clarify what about binary packaging you’d like to talk about? Perhaps editing that post and adding some background about it?
Packaging and Python 2
Coming up with an action plan and discussing what exact metrics would be used for depreciation/dropping support along with a cut off.
Having a mechanism to upload wheels / prebuilt binaries to the warehouse for packages you don’t own.
For instance, I’d love for conda-forge to be the way that trusted wheels get built and uploaded. We have already done a lot of the hard work to build out the infrastructure for building. It would be great if Python packages could opt-in to building wheels with us, and so no one would have to duplicate this effort.
With apologies in advance as I’m a total n00b to PyPA (but decided to participate in the summit today during my first PyCon Sprint)…have we ever considered supporting namespaces in PyPi? I feel like this (simple??) change could greatly improve security and resistance to typosquatting-type attacks.
Depending on how far we took this concept, this could allow organizations better control over publishing policies (e.g. groups like pallets could require 2FA for publishing any projects under their umbrella).
Some context for why I’m suggesting this: Inspired by the datasettes presentation this weekend, I spent some time poking around the “top 5000” projects downloaded in the last 30 days. I noted some organizations publish a large number of packages on this list (e.g. a ton of azure-related packages), and there’s no obvious way to identify which of these are “official” packages from MS, and which are community developed. Similarly, the aws
package showed up on the list, but it’s a project that hasn’t been updated in 6 years.
I think this addition could benefit both users and package maintainers. I just don’t have an appreciation for the “lift” required to implement, or if this idea has been considered previously and dismissed for some reason (despite the fact that “Namespaces are one honkin’ great idea - let’s do more of those”).
See Manage projects with namespaces · Issue #2589 · pypi/warehouse · GitHub. I think we’re amenable to namespaces, but it would require a significant amount of work across the ecosystem.
Based on the instructions in the topic header, I’ve created a new topic here on discourse where we can flesh out this idea.
If the preference is have this discussion on github instead, I’m happy to shift the discussion over there. (I’m not sure yet on what conventions PyPA has around discussing general improvement ideas like this. From the github issue it sounds like using the mailing list wasn’t particularly useful when this idea previously came up).