Pre-PEP: Dependency sources in pyproject.toml

Packaging
21

I have asked @barry to sponsor this PEP to which he kindly agreed. I am hoping to have more updates out this weekend.

3 Likes
22

FYI, I prototyped adding develop.sources into nab, replacing all the old direct sources mechanisms nab had: Implement dependency sources in pyproject.toml by notatallshaw · Pull Request #1319 · notatallshaw/nab · GitHub

It overlaps with workspaces, and explicitly doesn’t work as a workspace replacement due to having no mechanisms to search up and down directory tree (in fact it’s explicitly forbidden to look up the directory tree), but they’re not entirely mutually exclusive.

1 Like
23

I am open to suggestions on how we can enable searching up and down directory trees where it would not cause problems when something like [develop.sources] can be used in a top level pyproject.toml that does not contain a [project]

24

Putting on my pip and packaging maintainers hat for a second, I also prototyped adding this feature to those projects.

The PEP is definitely complex enough that it needs a well defined interface in packaging, and there are some important design questions, e.g. how and when to expose or evaluate markers.

For pip this feature is somewhat awkward, because it continues this series of features that I would call “project options”, like dependency groups, that pip does not have a clearly defined user experience for, so we will have to pin down exactly when pip should read a pyproject.toml and include it’s defined sources. But this is a pip problem, not a PEP problem.

I don’t have a good answer for you, and I think it probably should be avoided due to complexity and different tool designs there will be a lot more disagreement, and this will be harder to get over the line. With the understanding that tools will continue to support the somewhat overlapping feature of workspaces, and that for development of tightly coupled projects (usually in a monorepo) workspaces will probably be the more appropriate choice.

1 Like
25

This is one of the things I have been trying to come up with for the PEP to have rules for when defined sources should be included, as I recognize it’s one of the stickier parts for implementation.

1 Like
26

After thinking more about this, I am considering a root key that would accept a relative path to the root. So that at least searching up the directory tree has a stopping point of the relative path in [develop.sources.root] or [develop.root]. @notatallshaw @konstin what would your thoughts be on that? Or would it make sense to expand the scope here and include workspaces in the scope?

27

Here are the ones I found most relevant.

Earlier proposals that overlap with this one:

* [Pre-PEP discussion: Relative Path Dependencies](Pre-PEP discussion: Relative Path Dependencies) (Stephen Rosen, 2025)

* [Pre PEP discussion: Workspace standardization in pyproject.toml](Pre PEP discussion: Workspace standardization in pyproject.toml) (mine, 2026)

* [Proposal: overrides for installers](Proposal: overrides for installers) (sinoroc, 2023)

* [Dependency notation including the index URL](Dependency notation including the index URL) (sinoroc, 2020)

* [Adding a global config to specify package indexes](Adding a global config to specify package indexes) (Filipe Laíns, 2021)

* [Another idea for additional package repositories](Another idea for additional package repositories) (sinoroc, 2024)

Questions from users looking for this:

* [Is there a pyproject standard for sources specification of packages?](Is there a pyproject standard for sources specification of packages?)

* [Packages installed from PyPI cannot depend on packages which are not also hosted on PyPI](Packages installed from PyPI cannot depend on packages which are not also hosted on PyPI)

* [Dependency management with many private / local packages](Dependency management with many private / local packages)

2 Likes
28

Updates have been made to address more feedback. Thank you everyone so far for the valuable feedback!

1 Like
29

Thanks for the overview of previous threads @cjames23, that is very useful. I read through the various proposals, then did some iterative analysis of their scope and the differences between them (with some LLM assistance to make the task tractable), and ended up with this scope comparison document - that might be helpful for others as well.

I believe your current proposal is scoped quite well and already large enough; having the goal of being able to layer a future workspaces proposal on top of it seems better to me than expanding the scope now.

2 Likes