Packaging Mini Summit (PyCon US 2019): Topic Suggestions

PyPA governance

6 Likes

Future grants and funding priorities – what are our biggest pain points now?

3 Likes

Binary packaging

3 Likes

Expression of dependencies on externally provided software (things that pip does not/should not provide). Metadata encompassing binary compatibility that may be required for these expressions.

(Drawing a line to the scope of Python packaging)

13 Likes

With the advent of pip’s solver, the quality of metadata will be much more heavily tested. Pip currently warns (?) but allows creation of inconsistent environments, where one or more constraints don’t agree. There’s two topics here that might need to be considered separately:

  1. Can/should pip be disallowing installation of packages that break consistency by default?
  2. How can PyPI deal with bad metadata? From Conda’s experience, once it is in the index, things will be badly disrupted until either the offending package is removed, or the metadata is altered. Conda alters the index metadata, but with the distributed ownership of packages of PyPI, I’m not sure that’s an option.
8 Likes

Structured, Exchangeable lock file format (requirements.txt 2.0?)

6 Likes

Have pyproject.toml standardize on wheel-based/common metdata for that tools are expected to use if (optionally) provided by the user?

E.g. we recognize when version is set in some [metadata] section, but tools are allowed to provide their own way of specifying the details as well. Basically help build tools to not re-invent how to specify metadata that every wheel needs unless they are providing a value-add that warrants it (e.g. flit pulling the version number from the source directly). This is motivated by:

  1. Making it easier to change your build tool and not have to do tremendous updates to your pyproject.toml as part of the transition simply because a tool chose different naming
  2. Make it easier to do tool-agnostic analysis, e.g. know what dependencies a wheel end up specifying (if specified statically)
2 Likes

Future of python_requires, fixing bad uploaded versions and making it mandatory.
https://github.com/pypa/warehouse/issues/3889 is relevant.

5 Likes

PyPA communication (external and internal)

2 Likes

“end to end” pipeline testing for packaging tooling

2 Likes

Updating PyPA Roadmap

This one’s sort of long term and loosely tied to the communication discussion – it’d be nice if we’re able to start a conversation about some long-ish term planning too.

3 Likes

@njs could you please clarify what about binary packaging you’d like to talk about? Perhaps editing that post and adding some background about it?

1 Like

Packaging and Python 2

Coming up with an action plan and discussing what exact metrics would be used for depreciation/dropping support along with a cut off.

2 Likes

PyPI + TUF

1 Like

Having a mechanism to upload wheels / prebuilt binaries to the warehouse for packages you don’t own.

For instance, I’d love for conda-forge to be the way that trusted wheels get built and uploaded. We have already done a lot of the hard work to build out the infrastructure for building. It would be great if Python packages could opt-in to building wheels with us, and so no one would have to duplicate this effort.

5 Likes

With apologies in advance as I’m a total n00b to PyPA (but decided to participate in the summit today during my first PyCon Sprint)…have we ever considered supporting namespaces in PyPi? I feel like this (simple??) change could greatly improve security and resistance to typosquatting-type attacks.

Depending on how far we took this concept, this could allow organizations better control over publishing policies (e.g. groups like pallets could require 2FA for publishing any projects under their umbrella).

Some context for why I’m suggesting this: Inspired by the datasettes presentation this weekend, I spent some time poking around the “top 5000” projects downloaded in the last 30 days. I noted some organizations publish a large number of packages on this list (e.g. a ton of azure-related packages), and there’s no obvious way to identify which of these are “official” packages from MS, and which are community developed. Similarly, the aws package showed up on the list, but it’s a project that hasn’t been updated in 6 years.

I think this addition could benefit both users and package maintainers. I just don’t have an appreciation for the “lift” required to implement, or if this idea has been considered previously and dismissed for some reason (despite the fact that “Namespaces are one honkin’ great idea - let’s do more of those”).

See Manage projects with namespaces · Issue #2589 · pypi/warehouse · GitHub. I think we’re amenable to namespaces, but it would require a significant amount of work across the ecosystem.

1 Like

Based on the instructions in the topic header, I’ve created a new topic here on discourse where we can flesh out this idea.

If the preference is have this discussion on github instead, I’m happy to shift the discussion over there. (I’m not sure yet on what conventions PyPA has around discussing general improvement ideas like this. From the github issue it sounds like using the mailing list wasn’t particularly useful when this idea previously came up).

Public API for the wheel package (#262)

1 Like