Packaging Mini Summit (PyCon US 2019): Topic Suggestions

Hi everyone!

Following the discussion in the thread for PyCon US Packaging Mini-Summit 2019, this thread is created for collecting topic suggestions, in one place, for discussion at the Mini-Summit.

If you’re wondering what the plans are for the summit, the thread linked above is where we’re planning it out.

Please make separate replies for each suggestion. Any discussions (other than requesting clarifications on suggestions) will be broken out into a separated topic.

To aid moderation, instead of replying here and requesting a moderator to move to a new topic, please create a new topic quoting the suggested post.

2 Likes
  • More dependency categories (and properly supporting ones we actually have): Recommends (optional, installed by default), default extras dependencies, Provides, Obsoletes

Background:

Right now we can only say that one thing depends on another or it doesn’t. We cannot specify many “softer” dependencies such as “this is not critical but it should be provided if possible” or “this can be satisfied with any of (X, Y or Z)”. I would like to discuss a taxonomy of dependencies and get oriented. A few options brought up so far:

  • Obsoletes is not used in pip (or by any tool that I know of) (Provides has related issues)
  • Opportunistic dependencies are ones where if you fail to install them the install succeeds with just a warning
  • Recommends: Optional dependencies, installed by default but can be removed with an extra (also opportunistic)
  • default extras dependencies - dependencies used only if no extras are specified.
4 Likes

A post was split to a new topic: Possible default extras/dependency categories?

FYI, to help keep this topic focused and to keep the noise down, I am planning to aggressively break replies out into separate topics if they are not specifically making topic suggestions or asking for clarifications (e.g. riffing on details is considered off-topic).

I will also suggest people make separate replies for each topic as it could allow for voting for topics by liking each one.

3 Likes
  • Recommendations around the src/ layout

Background: This issue is still unresolved, but a lot of people are converting to the src/ layout. The canonical reference for why is Hynek’s post, though I will refrain from at-messaging him because I don’t think he wants to be the poster-boy for src/ layout.

8 Likes
  • The future of editable installs

Background: PEP 517 does not support editable installs, so pip install -e will not work for any projects that rely on PEP 517 (flit projects, setuptools projects with no setup.py).

12 Likes

Somehow related to Paul’s suggestion of additional dependency categories:

Both are strongly linked.
(Link was easily found thanks to https://github.com/pypa/packaging-problems/issues/215)

6 Likes

Project workflow (pipenv, tox, dolly, PEP 582, …)

1 Like

Warehouse security work grant – what’s happening, what’s desired?

4 Likes

Whatever @msarahan and the conda team want to talk about

7 Likes

PyPA governance

6 Likes

Future grants and funding priorities – what are our biggest pain points now?

3 Likes

Binary packaging

3 Likes

Expression of dependencies on externally provided software (things that pip does not/should not provide). Metadata encompassing binary compatibility that may be required for these expressions.

(Drawing a line to the scope of Python packaging)

13 Likes

With the advent of pip’s solver, the quality of metadata will be much more heavily tested. Pip currently warns (?) but allows creation of inconsistent environments, where one or more constraints don’t agree. There’s two topics here that might need to be considered separately:

  1. Can/should pip be disallowing installation of packages that break consistency by default?
  2. How can PyPI deal with bad metadata? From Conda’s experience, once it is in the index, things will be badly disrupted until either the offending package is removed, or the metadata is altered. Conda alters the index metadata, but with the distributed ownership of packages of PyPI, I’m not sure that’s an option.
8 Likes

Structured, Exchangeable lock file format (requirements.txt 2.0?)

6 Likes

Have pyproject.toml standardize on wheel-based/common metdata for that tools are expected to use if (optionally) provided by the user?

E.g. we recognize when version is set in some [metadata] section, but tools are allowed to provide their own way of specifying the details as well. Basically help build tools to not re-invent how to specify metadata that every wheel needs unless they are providing a value-add that warrants it (e.g. flit pulling the version number from the source directly). This is motivated by:

  1. Making it easier to change your build tool and not have to do tremendous updates to your pyproject.toml as part of the transition simply because a tool chose different naming
  2. Make it easier to do tool-agnostic analysis, e.g. know what dependencies a wheel end up specifying (if specified statically)
2 Likes

Future of python_requires, fixing bad uploaded versions and making it mandatory.
https://github.com/pypa/warehouse/issues/3889 is relevant.

5 Likes

PyPA communication (external and internal)

2 Likes

“end to end” pipeline testing for packaging tooling

2 Likes