I’ve been working a lot on how to optimise my workflow (and my teammates’) and each time I’ve been trying a solution I’ve found myself stuck by the lack of tooling standardisation on the development process (where are the scripts, what should be installed first, launched fron the virtualenv or not…) and lately, instead of trying to properly integrate everything my way, I concluded that I should contribute some PEPs but I’m wondering where to start
So, here I am, proposing a first idea. This is a first time to me, any help, any constructive comment… is welcome. Don’t hesitate to redirect me to the proper communication channel if I’m posting at the wrong place.
A proposal for exposing development workflow and housekeeping scripts in
As soon as a project is growing, there is always the need for:
- some housekeeping scripts
- some specific launch scripts to run tests, lints…
- some helpers for common tasks
pyproject.toml is becoming the new standard to expose metadata, dependencies… (See PEP 621 and other PEPs related to
pyproject.toml, thanks @brettcannon ), it seems to be a logical place to also expose those scripts.
Having a standard way of exposing development scripts/tasks would allow:
- developpers to knew where to look when onboarding on a project (easy discovery)
- tools to be able to integrate those script easily
- language-agnostic tools to easily discover and expose some tasks (like
- optionaly being to expose the required dependencies to run those
Given some tools build dynamically there task list (
nox…), the exposition mecanism should allow dynamic tasks/scripts providers.
Note: this is not meant to replace the packaging entrypoints (ie.
project.guiscripts…) which are here to describe your package delivery and target product customers while this specification try to address the developpers needs on the product.
The specification does not make hypothesis on the launcher itself, it cvan be a dedicated tool or your package manager like
flit… It will be represented as
$launcher in the command line examples.
This mostly copy the
[scripts] section as it is well known and simple.
Manual scripts are exposed as key-value pairs under the
The key is the command line argument expected by wahtever launcher that would use the section.
The value represent the actual command line executed.
npm provides an interesting mecanism allowing to reference
node_modules binaries without the
node_modules/.bin prefix. Given most tools relying on
pyproject.toml also provide
venv integration and management, I believe the virtualenv bin path should be added as first
$PATH search entry to easily pick installed dependencies executables
[scripts] lint = "flake8" test = "pytest tests/"
Launchers should allow extra command arguments to pass through, either as added parameter and/or by using the double-dash arguments.
So, in our case, invoking:
$launcher test -k my-slection would resolve as
pytest tests/ -k my-selection and
$launcher test -- -k my-selection into
pytest /tests -- -k my-selection
When command list is growing, sometimes namespacing command can help the discovery.
Namespcaing is done by declaring a dictionnary instead of a string.
[scripts] root-cmd = "a root command" [scripts.doc] build = "my doc build command" publish = "my publish build command" [script.test] unit = "my unit testing command" integration = "my integration testing command"
This would provide the following commands:
$launcher root-cmd $launcher doc:build $launcher doc:publish $launcher test:unit $launcher test:integration
note: namespace separator to define, I arbitrary chose
: as I like it.
When using script provider, the provider entrypoint should be exposed under the
[scripts.providers] invoke = "invoke.program:PyprojectDiscovery" another = "path.to.another:EntryPoint"
This syntax allow multiple providers. The
key is non-significant, only here for documentation.
Each provider is allowed to provide namespaces.
This is the base idea. I volontarily not been into details of the endpoint specification for providers as there is some questions to answer before:
- is it meant to be integrated by package managers or not ?
- what would be the data needed to be exposed ?
- multiple providers or just one ?
- do we specify the provider dependencies here like it’s done for the
requireor do we simply expect to have the dependencies installed before and the provider beeing one of them ?
What do you think ? Should I continue or should I stop right now because this don’t have any chance to be accepted ?
What would be the next steps to continue this work ?