I think we may want to start with something generic and use Pipelines for coordinating the environment itself.
Ideally we’ll have some collection of scripts that orchestrate the integration tests, and then the configuration files for Azure that specifies which of them gets run on what platforms.
I do agree that we almost certainly need to be closer to the metal than tox will give us, though. For one thing I kinda think
tox should be one of the things under test, but also I predict we’re going to have a lot of trouble making sure that we’re actually using the right versions of everything here. Ideally, we’re going to want a
pyproject.toml that says
["setuptools"] in it to pull the master version of
setuptools, for example. Same with
virtualenv, etc. It will probably be harder to make sure that’s happening with more layers of indirection around the build environments.