Pre PEP discussion: Workspace standardization in pyproject.toml

Packaging Standards
1

Now that there are multiple tools that have implemented the concept of workspace dependencies heavily inspired by the same concept in Cargo. I think it is time for some discussion around what standardization would look like. I think having a standard spec for a workspace table in pyproject.toml would avoid drift between tools and avoid user confusion to have to remember which style to use based on which tool is being used.

My thoughts for what would be under the workspace table are

[workspace]
members = ["member1", "path/to/member2"]
exclude = ["path/to/other"]

[workspace.project]
version = "1.2.3"
authors = ["Nice Person"]
license-files = ["SomeLicenseFile", "SomeOtherLicenseFile"]

I think having shared project versioning for workspaces can simplify releasing all projects in a monorepo using the same versioning together. The same holds true for licenses when workspace members are all a part of the same monorepo. Any fields allowed in workspace.project should be considered as inherited from the root pyproject.toml of a monorepo.

I would like to get input from others on what they would like to see as a part of a spec. As a note, both tools that have implemented workspaces both use the members and exclude as a part of their implementations already. The concept of inheriting certain metadata from a root to all children is new and build backends would need to support this when packaging.

edit: What are workspaces? Workspaces in tools like hatch and uv allow you to manage multiple related Python packages within a single repository (a “monorepo”). Instead of treating each package as a completely separate project, workspaces let you define a root project that contains several sub-packages, all sharing a common configuration, virtual environment, or dependency resolution. This is particularly useful when you’re developing libraries that depend on each other—for example, a core library and several plugins—because you can make changes across packages simultaneously, run tests together, and ensure dependencies stay in sync without publishing intermediate versions.

4 Likes
2

For the rest of us, please cover the concept.

4 Likes
3

Workspaces were inspired on the Rust implementation in Cargo, and are the core feature behind monorepos. There’s a nice Talk Python podcast on the subject, Modern Python monorepo with uv and prek - episode 540, which covers how Apache Airflow uses workspaces.

The core idea is that you have two or more packages that might depend on each other. The key feature is that one library can depend on another; if you are developing, it’s a local dependency, while it’s a public dependency when you publish it. An example is you could have a pkg written in Python and a pkg_core written in a compiled language, and pkg could depend on pkg_core. pkg_core gets installed as well, and you can edit both locally.

I’ve not used the uv or hatch implementation (which are similar from what I understand), but I have used it in Cargo for Rust for the Advent of Code 2024, and it’s really nice. Here’s an abridged look at that:

# Top level Cargo.toml
[workspace]
members = ["crates/*"]
default-members = ["crates/aoc", "crates/year_2024"]

[workspace.package]
version = "0.1.0"

[workspace.lints.clippy]
all = "warn"
pedantic = "warn"
nursery = "warn"

[workspace.dependencies]
clap = { version = "4.5.22", features = ["derive"] }
regex = { version = "1.11.1", default-features = false }
thiserror = "2.0.8"

getdata = { path = "crates/getdata" }
aoc = { path = "crates/aoc", default-features = false }
year_2024 = { path = "crates/year_2024", default-features = false }

This declares that there will be packages inside the crates folder. Two of them are “default”, meaning that if I build at the top level, those get built. I also have shared dependencies; any of the projects can use these dependencies and get the versions/options on that dependency. Most important is the last part - those are local dependencies; meaning that I can depend on those in any of the local packages, and I’ll get the local version.

Here’s an example of one of the projects:

# crates/year_2024/Cargo.toml
[package]
name = "year_2024"
version.workspace = true
publish = false

[dependencies]
aoc = {workspace = true, default-features = false}
clap.workspace = true
regex.workspace = true

[lints]
workspace = true

Here you can see we are using the the shared parent version, getting some of the dependencies, and the lint rules all defined in the version. And the doc depencency is a local one, from one of the other crates! We could also define any of this without pulling from the workspace (like version, or a package only needed in this project, etc).

For Python, one of the key things you can do with a workspace is uv build from the top level will build all the packages at once. We don’t support this in cibuildwheel yet, but I plan to work on that soon, which would mean that you could build multiple packages with one cibuildwheel call.

3 Likes
4

This is missing from hatch and hatchling right now but is a gap that we do intend to address. Right now the workspace implementation more handles getting version resolution consistency via hatch environments and easier testing of impact of changes during the local development cycle because of members being editable installs in hatch environments.

I have been thinking through this part in terms of what some of these concepts might look like in pyproject.toml for things such as the dependency inheritance from root to members. I do not know yet if it makes sense to enable that.

5

I haven’t had the time or need these days to try this out, but I’m very happy to see this feature in hatch. I remember a project from $job-1 where we had to do exactly this, and I had a set of fragile hatch plugins to make it “work”[1]. To see this supported natively in hatch now is really fantastic!

  1. FSVO ↩︎

1 Like