Adding a non-metadata installer-only `dev-dependencies` table to pyproject.toml

Ditto for Hatch. As I’ve expressed before, these non-runtime extras are attempts at describing separate environments without an actual environment manager.

1 Like

This fundamentally is a duplicate of Providing a way to specify how to run tests (and docs?)

My response therein: Providing a way to specify how to run tests (and docs?) - #49 by ofek

I don’t think it is, or at least what’s being discussed here was just part of that other topic.

As an example, let’s take packaging (because I bet Expose test dependencies outside of `noxfile.py` · Issue #601 · pypa/packaging · GitHub sparked this idea for Pradyun :grin:). It uses nox, so it has its requirements to run its tests embedded in its noxfile.py. Having those dependencies reachable by nox is important for running the test suite against multiple versions of Python.

The problem is that I also want to be able to run the tests while I’m doing development in VS Code using its test explorer (it’s way faster to run a single test under my virtual environment when trying to get that one test to pass). Right now, though, I have to manually extract the requirements strings from noxfile.py and run the pip install command manually in order to get the appropriate projects installed to run the tests (on top of doing an editable install thanks to the src layout, and if this was some other project probably to install normal dependencies). That is not a great development workflow to ask of contributors to packaging to go through if they want their editors or other tooling to somehow participate in the development process.

The options I’m seeing listed here are:

  1. Standardize in a new way, although it sounds like Hatch and Poetry don’t love that.
  2. Leave it to the tools, which may or may not work depending on whether tools want to support this concept.
  3. Use extras as-is, which some people don’t like as they view extras as part of the API of their package in a way.
  4. Use requirements files, which isn’t a standard and leans into pip pretty hard for this.

But I don’t see this as something we can just leave up to task runners as that cuts out a legitimate workflow IMO.

1 Like

I’m surprised you’re advocating for this instead of your other idea Support a way for other tools to assist in environment/interpreter discovery · Discussion #168 · brettcannon/python-launcher · GitHub

If that’s the only concern for extras can we hide some extras under some dev flag? I don’t personally mind having these in extras.

I get why @pradyunsg thinks using optional-dependencies for this is “dumb” – it feels weird that a user could install tests as an 'extra" – but my impression from the maybe-not-a-duplicate-topic was that a lot of people maintaining downstream package managers actually use this to run tests (edit: without building and inspecting an sdist)?

Like @CAM-Gerlach was saying here:

Would they lose this ability if there was a dev-dependencies table then?
Sorry if this is just revealing my ignorance about what one can get out of a distrib package

As I said earlier:

Sure thing – there’s people who don’t share that view (established by this being implemented in multiple tools, and the list of feature requests in pip). “I don’t see it that way” isn’t a strong argument for “There is no need for this”. :slight_smile:

Agreed; and that’s a good point.

@ofek’s mention of Hatch’s environment management mechanisms is also a good pointer – Hatch and tox 4 have significantly more complex mechanisms for environment management, that covers a super-set of the needs satisfied by the mechanisms proposed here. I don’t see that as particularly problematic personally.

Having this won’t mean that those approaches will somehow become unavailable for these tools, or for users who want it. This isn’t a “tool has to implement this” but rather “here’s a shared design, for tools that want to use it” – with the primary motivation being, well, implementing support for this in pip.

Per the spec, [tool.shared] belongs to the shared project on PyPI. Wouldn’t it be better to collaborate elsewhere?
(IMO it’s best to do this in the tools’ individual namespaces. The fact that something has a shared spec doesn’t really need to be exposed to users.)

Fair point. I’d misremembered that we made this a rule, rather than just a recommendation.

We could use tool._shared (which isn’t a valid project name). I’m assuming the advantage of using a common namespace is that people don’t have to change their config when switching tools, otherwise a tool-specific name is indeed perfectly fine. The original post here didn’t offer any particular reason why we should standardise this (rather than leave it tool-specific) so I’m guessing about the motivation, though…

You can do this right now by using a requirements-test.txt file. I get that you don’t want to support the (not standardised) requirements file format, and you don’t want to pick a naming convention that not everyone follows. That’s fine. But if we look at the proposal from that angle, what you’re saying is that for you, dev-dependencies is a standardised replacement for requirements.txt.

I can accept that (I think standardising a replacement for requirements.txt would be a worthwhile thing to do) but that’s a much broader proposal than what @pradyunsg suggested.

And this also identifies my main reservation with supporting this in pip. We already have a way of doing this in pip, it’s requirements.txt files. It doesn’t align with the mechanisms used by hatch, PDM or Poetry, but it’s addressing the same problem. So just like we’re asking those other tools to drop their existing approaches in favour of a standard one, we should be expecting pip to drop requirements.txt in favour of this new approach[1]. I’m -1 on having pip retain both approaches, because they address the same underlying use case.


  1. Which is obviously unreasonable, as requirements files are far more powerful than the proposed mechanism… ↩︎

1 Like

I view that as orthogonal. My “other idea” helps you find your environment, not create it and populate it. Unless you’re pushing the idea that users run nox to create their test environment and then use my “other idea” to select it. In which case it does work and help solve this (it’s actually one of the original motivators for the idea), but I don’t think it needs to be the solution for how folks get test dependencies into an environment.

Perhaps. I don’t think we ever came to an agreement as to whether we should update Source distribution format - Python Packaging User Guide to suggest people include tests in their sdists.

Right, but then …

Although I might be misunderstanding what “general dependency group mechanism” means here.

Correct, and Pradyun is actually suggesting something along these lines for packaging and it’s specific needs.

I personally don’t care; this is Pradyun’s idea, not mine. :grin: I like the idea of using extras for this, then I’m okay with using requirements files.

One of my points is we don’t seem to have a defined cultural/community practice of where development dependencies should go right now. My other point is putting the dev dependencies only in a place that tox/nox can get to for their environments is limiting when you may want to use a different environment that they don’t control. Now you could argue folks should use the environments that tox and nox create instead (which is what I think Ofek hinted at earlier), but I also don’t know if tox and nox actually want users doing that or even document where those environments live.

I agree as I see requirements files used for things like lock files, a way to specify alternative requirements, etc. so fully supplanting them is a big task.

Yes I think that’s the correct approach:

Hatch can already do this Reference - Hatch

1 Like

@bernatgabor Would you be willing to add such a command to tox?

Seems like folks have moved on and convinced of this. I don’t see it.

What’s the harm in that? How would a user end up pip install package[test]? Where did they end up finding the extras? Can we not invoke “we are all consenting adults here” for this?

My perspective is slightly different. There was a guy recently raising issues on a bunch of projects because he was having trouble running tests but he wasn’t using the project’s documented method of running the tests. That makes me feel that projects have a right to say “this is how you run the tests”, and shouldn’t be expected to support people who don’t follow those instructions. So making the test dependencies accessible from anywher other than the supported test runner isn’t something I particularly want to support.

The VS Code test runner sounds cool (I must try it out) but I fear similar “your project’s tests don’t work in VS Code” complaints (which may actually be unrelated to the availability of dev dependencies) if we standardise a way of setting dev dependencies, and VS Code (or any other tool) uses it.

Overall, though, I guess I don’t care much. Pradyun’s original question was “is there any appetite for trying to consolidate this concept” and my answer is basically “not from me, no”…

2 Likes

I noticed this as well. As far as I know extras are not discoverable by end users, short of looking at the pyproject.toml or setup.cfg or whatever. So unless it is clearly documented somewhere that end users should install package[test] there is no reason why they would. And, for me, it completely nullifies the following:

If I am not mistaken, I would say that nothing here is exposed to end users (via the metadata). As the user of a library I do not go look at how it is packaged, I reach for its documentation, if the extra is not documented I do not know it exists.

So we can see it as: if the extra is not documented then it is a group of development dependencies, if we want.

Testing Python in Visual Studio Code :slightly_smiling_face:

All you should need to run tests in VS Code is your test dependencies installed in your environment and you have configured VS Code to run your tests via whatever flags you need set; that’s it. Anything else is VS Code’s fault for not supporting it. But it’s that first step that can be a bit of a burden when things get embedded inside the test runners. We have multiple options on how to expose test dependencies in a more modular way outside of the test runners while still giving the runners access to the dependencies, so that’s what I’m trying to promote here: don’t embed details inside of test runners that don’t have to be embedded.

That’s entirely dependent on what tooling you are using.

What if your tests need certain environment variables set, or some weird config set up? I’m not saying that’s a good way to set up your test suite, just that some projects may use a test runner like nox precisely because they have complex setup that can’t be distilled down to just “set up these dependencies”.

Most projects won’t need this, agreed. And many that do probably shouldn’t. But that’s the project’s choice, and I’m not sure we should have “a defined cultural/community practice of where development dependencies should go” precisely because I feel it should remain the project’s choice.

Having (yet another) way to specify dev dependencies is fine, as long as it’s not presented as the “one true way” (IMO, at least).

One other thought I had, though. For a “docs” dev-dependency, you might want to install Sphinx and furo. It’s quite possible you don’t need the project itself installed. Extras don’t allow you to do this - they always require the project (and its dependencies) installed. In certain cases, it may even be true that the project depends at runtime on a version of (for example) Sphinx that is different than the one you want to use to build the docs…

Requirements files don’t have this issue. Nor do the proposed dev-dependencies. FWIW.

Not being able to list extras is not a feature - it’s an omission that I hope we will address. We should not rely on an extra being undocumented to signify that it’s for internal use/development purposes only. But in the same “consenting adults” spirit, I don’t see why we’d want to hide development extras anyway…

1 Like