Adding a default extra_require environment

sinoroc · August 14, 2020, 12:42pm

My overall impression (and others seem to somewhat concur) is that a large portion of the use cases for a default extra can be solved with better documentation, and better error messages when dependencies are missing. I don’t see a way to add such a default extra while still having the pip freeze > req.txt; pip install -r req.txt workflow behave as expected.

In order to provide the nice out of the box experience for first-time users, projects might want to have a top level project MyThing that is basically just an empty thing (no code) that depends on MyThingCore[backend_default] and nothing more (not even a version range). And returning users, who are willing to put the effort and actually read installation instructions would naturally move to installing MyThingCore[backend_gpu_accelerated,bells,whistles]>=1.2.3, once they feel the need to go beyond what the default installation provides.

For more advanced needs (which I agree they might be legitimate), seems like it has to be something else than extras and for a possible solution (or at least inspiration) I would like to draw attention again on the ongoing work being done in poetry (they seem to be seriously going for it since it’s on their roadmap):

github.com/python-poetry/poetry

Support dependencies groups

opened 10:36AM - 28 Nov 19 UTC

closed 09:42AM - 23 Jul 21 UTC

pawamoy

area/solver area/installer kind/feature

- [x] I have searched the [issues](https://github.com/sdispater/poetry/issues) of this repo and believe that this is not a duplicate. - [x] I have searched the [documentation](https://poetry.eustace.io/docs/) and believe that my question is not covered. ## Feature Request As @mozartilize stated in [this comment](https://github.com/sdispater/poetry/issues/1007#issuecomment-496779710), [groups](https://bundler.io/guides/groups.html) from Ruby's gemfile could be a great addition and a solution to our complex workflows and different environments (local, production, ci, docs, etc.). Instead of supporting more `tool.poetry.*-dependencies` sections, we could add support for a `groups` property on dependencies. It would be more flexible than rigid sections. I also understand there is the `extras` functionality, but using extras changes the behavior of `poetry install`, as one needs to now specify the extras to install all the dependencies. Also, extras are opt-in only, and often times we need opt-out. ```toml [tool.poetry.dependencies] python = "^3.6" requests = "*" [tool.poetry.dev-dependencies] bandit = { version = "^1.5", groups = ["ci"] } black = { version = "*", allows-prereleases = true, groups = ["ci"] } coverage = { version = "=5.0a8", allows-prereleases = true, groups = ["ci"] } flake8 = { version = "^3.6", groups = ["ci"] } ipython = { version = "^7.2", groups = ["local"] } isort = { version = "^4.3", extras = ["pyproject"], groups = ["ci"] } jinja2-cli = { version = "^0.7.0", groups = ["local"] } pylint = { git = "https://github.com/PyCQA/pylint", groups = ["ci"] } pytest = { version = "^4.3", groups = ["ci"] } pytest-cov = { version = "^2.8", groups = ["ci"] } pytest-sugar = { version = "^0.9.2", groups = ["ci"] } pytest-xdist = { version = "^1.26", groups = ["ci"] } recommonmark = { version = "^0.6.0", groups = ["docs"] } safety = { version = "^1.8", groups = ["ci"] } sphinx = { version = "^1.8", groups = ["docs"] } sphinx-rtd-theme = { version = "^0.4.2", groups = ["docs"] } toml = { version = "^0.10.0", groups = ["docs"] } ``` Also: - `default` group would be attributed to dependencies without groups (`requests` in the above example) - each dependencies would be attributed its own group as well, something like `/bandit` for `bandit`, etc. Then we would have the following options for the `poetry install` command: - `--groups`: explicitely specify which groups to install (these and only these groups, no implicit default group) - `--with-groups` or `--with`: default group **plus** the specified groups - `--without-groups` or `--without`: all groups **minus** the specified groups Usage would be: ```bash # in CI poetry install --groups ci # or in specific CI jobs poetry install --groups /bandit # bandit reads the code, we don't need anything else poetry install --with /safety # safety needs production dependencies to be installed # locally, all three lines are equivalent poetry install # installs everything, as usual poetry install --with local,ci,docs poetry install --groups default,local,ci,docs # on readthedocs poetry install --with docs # sphinx-autodoc needs production dependencies to be installed ``` #1007 could then probably be closed in favor of this issue.

Aside:

True. That could be partly covered by this proposition (which could also reduce – at least slightly – the urgency of specifying source distribution file names, and maybe more):