Packaging Mini Summit (PyCon US 2019): Topic Suggestions

(Pradyun Gedam) #1

Hi everyone!

Following the discussion in the thread for PyCon US Packaging Mini-Summit 2019, this thread is created for collecting topic suggestions, in one place, for discussion at the Mini-Summit.

If you’re wondering what the plans are for the summit, the thread linked above is where we’re planning it out.

Please make separate replies for each suggestion. Any discussions (other than requesting clarifications on suggestions) will be broken out into a separated topic.

To aid moderation, instead of replying here and requesting a moderator to move to a new topic, please create a new topic quoting the suggested post.

2 Likes
PyCon US Packaging Mini-Summit 2019
PyCon US Packaging Mini-Summit 2019
(Paul Ganssle) #2
  • More dependency categories (and properly supporting ones we actually have): Recommends (optional, installed by default), default extras dependencies, Provides, Obsoletes

Background:

Right now we can only say that one thing depends on another or it doesn’t. We cannot specify many “softer” dependencies such as “this is not critical but it should be provided if possible” or “this can be satisfied with any of (X, Y or Z)”. I would like to discuss a taxonomy of dependencies and get oriented. A few options brought up so far:

  • Obsoletes is not used in pip (or by any tool that I know of) (Provides has related issues)
  • Opportunistic dependencies are ones where if you fail to install them the install succeeds with just a warning
  • Recommends: Optional dependencies, installed by default but can be removed with an extra (also opportunistic)
  • default extras dependencies - dependencies used only if no extras are specified.
4 Likes
Optional C Extension handling
(Brett Cannon) split this topic #3

A post was split to a new topic: Possible default extras/dependency categories?

(Brett Cannon) #4

FYI, to help keep this topic focused and to keep the noise down, I am planning to aggressively break replies out into separate topics if they are not specifically making topic suggestions or asking for clarifications (e.g. riffing on details is considered off-topic).

I will also suggest people make separate replies for each topic as it could allow for voting for topics by liking each one.

3 Likes
(Paul Ganssle) #5
  • Recommendations around the src/ layout

Background: This issue is still unresolved, but a lot of people are converting to the src/ layout. The canonical reference for why is Hynek’s post, though I will refrain from at-messaging him because I don’t think he wants to be the poster-boy for src/ layout.

9 Likes
(Paul Ganssle) #6
  • The future of editable installs

Background: PEP 517 does not support editable installs, so pip install -e will not work for any projects that rely on PEP 517 (flit projects, setuptools projects with no setup.py).

12 Likes
Pip freeze, vcs urls and pep 517 (feat. editable installs)
(Xavier Fernandez) #7

Somehow related to Paul’s suggestion of additional dependency categories:

Both are strongly linked.
(Link was easily found thanks to https://github.com/pypa/packaging-problems/issues/215)

6 Likes
(Nathaniel J. Smith) #8

Project workflow (pipenv, tox, dolly, PEP 582, …)

1 Like
Possible default extras/dependency categories?
PyCon US Packaging Mini-Summit 2019
(Nathaniel J. Smith) #9

Warehouse security work grant – what’s happening, what’s desired?

4 Likes
(Nathaniel J. Smith) #10

Whatever @msarahan and the conda team want to talk about

7 Likes
(Nathaniel J. Smith) #11

PyPA governance

6 Likes
Pip 19.1 and installing in editable mode with pyproject.toml
(Nathaniel J. Smith) #12

Future grants and funding priorities – what are our biggest pain points now?

3 Likes
PyCon US Packaging Mini-Summit 2019
(Nathaniel J. Smith) #13

Binary packaging

3 Likes
(Mike Sarahan) #14

Expression of dependencies on externally provided software (things that pip does not/should not provide). Metadata encompassing binary compatibility that may be required for these expressions.

(Drawing a line to the scope of Python packaging)

14 Likes
(Mike Sarahan) #15

With the advent of pip’s solver, the quality of metadata will be much more heavily tested. Pip currently warns (?) but allows creation of inconsistent environments, where one or more constraints don’t agree. There’s two topics here that might need to be considered separately:

  1. Can/should pip be disallowing installation of packages that break consistency by default?
  2. How can PyPI deal with bad metadata? From Conda’s experience, once it is in the index, things will be badly disrupted until either the offending package is removed, or the metadata is altered. Conda alters the index metadata, but with the distributed ownership of packages of PyPI, I’m not sure that’s an option.
8 Likes
(Mike Sarahan) #16

Structured, Exchangeable lock file format (requirements.txt 2.0?)

6 Likes
(Brett Cannon) #17

Have pyproject.toml standardize on wheel-based/common metdata for that tools are expected to use if (optionally) provided by the user?

E.g. we recognize when version is set in some [metadata] section, but tools are allowed to provide their own way of specifying the details as well. Basically help build tools to not re-invent how to specify metadata that every wheel needs unless they are providing a value-add that warrants it (e.g. flit pulling the version number from the source directly). This is motivated by:

  1. Making it easier to change your build tool and not have to do tremendous updates to your pyproject.toml as part of the transition simply because a tool chose different naming
  2. Make it easier to do tool-agnostic analysis, e.g. know what dependencies a wheel end up specifying (if specified statically)
2 Likes
(Matthias Bussonnier) #18

Future of python_requires, fixing bad uploaded versions and making it mandatory.
https://github.com/pypa/warehouse/issues/3889 is relevant.

5 Likes
(Pradyun Gedam) #19

PyPA communication (external and internal)

2 Likes
(Pradyun Gedam) #20

“end to end” pipeline testing for packaging tooling

2 Likes