This is a super important use case to address. But I think it’s asking too much of pip to expect pip install -e address it – it’s just not high enough level to handle the whole job in the long run (though I see why it’s one of the better options right now). Some things it can’t do:
- Create/manage the venv itself
- Distinguish between unlocked and locked dependency ranges. When creating a package, we use unlocked ranges. In development, we use locked ranges.
pip install -eis an extension to the “creating a package” part of the system, so it uses the wrong dependencies for development. - Make sure your environment is actually up to date before running tests
A pipenv-like tool can take care of all of this seamlessly for you.
And if you have a pipenv-like tool, editable installs are actually less compelling in general… if pipenv test always does a quick incremental rebuild before invoking the test tool, then that’s just as easy to use as an editable install, and far more friendly to beginning users who aren’t equipped to keep track of which kinds of edits are ok with an editable install, and which ones force you to re-run pip (e.g. compiled code, version bumps, dependency bumps). The tradeoffs here are tricky, because speed matters too. But if I were building a tool like this, I’d probably support editable installs, but I’d leave then disabled by default, so experts could get the speed boost of skipping rebuilds, but only if they’re prepared to track cache consistency in their head.
(Note: a combined build-and-test command is actually the workflow that I’ve always used for projects with native code. E.g. numpy and scipy have a runtests.py tool that takes care of this. So this isn’t something I’m just inventing out of thin air.)
Hmm, so here’s a potentially interesting feature for a future version of editable installs. This suggests editable installs are really just a optimization to make reinstalling really fast, and they sacrifice some accuracy to do that. Maybe we can make them a faster or more accurate optimization. For example, what if the backend could provide some extra metadata to hint to the frontend when a rebuild is required ("if any of these files change, do a full reinstall: setup.py, lib/foo.c, …). Then a pipenv-like tool could stay fast when only .py files are updated, but also take on some of the cache consistency tracking that we currently have to do in our heads.