Feedback on tox 4

[jaraco (Jason R. Coombs)]

I’ve been using tox 4 for about a week now, and I have some feedback/questions on the new behavior. These are only my impressions and not intended to be hard critiques and I’m interested in learning more about how I maybe should adjust my perspective to accommodate.

1. The new output is somewhat confusing. Maybe I’ll get used to it, but the report scope names seem very technical and not user-friendly. With names like .pkg: and python:, but also exposing the internals of pep 517 APIs (install_requires_for_build_editable). For user-facing content, I’d expect tox to provide more friendly details (without leading dots and underscores and internal names).
2. Tox used to emit all of the packages installed into the environment. Now it only emits the names of the packages declared to be installed. The former was more useful especially when diagnosing errors. There have been many times when I’ve gone back to a CI run to ascertain what package versions that were installed at the time to verify a suspicion that an upgrade of a particular package was implicated in a failure. Is there any way this behavior could be brought back?
3. Previously, tox --help would emit the environment variables that tox supported. Because they’re irregular (sometimes underscore-separated, other times not, sometimes prefixed by TOX sometimes not), it’s nice to be able to query for them rather than track down the online docs. Actually, I just checked and I can’t even find TOXENV in the docs. Is the environment variable behavior no longer documented?

Thanks for tox, and let me know if you’d like me to file any of these as issues with the project.

[bernatgabor (Bernát Gábor)]

Probably tox-dev/tox · Discussions · GitHub would have served a better forum for this, but thanks for the feedback.

@jaraco

The new output is somewhat confusing. Maybe I’ll get used to it, but the report scope names seem very technical and not user-friendly. With names like .pkg: and python:, but also exposing the internals of pep 517 APIs (install_requires_for_build_editable). For user-facing content, I’d expect tox to provide more friendly details (without leading dots and underscores and internal names).

This is on purpose. My experience with tox 3 is that when things go wrong, people don’t understand what’s the problem. Exposing this underlying behavior in hopes of helping people in the long term. It exposes the user concepts like the separation of the build environment and the run environment. Anyone doing python packaging should know this. Otherwise, they’ll struggle when things eventually go wrong.

@jaraco

Tox used to emit all of the packages installed into the environment. Now it only emits the names of the packages declared to be installed. The former was more useful especially when diagnosing errors. There have been many times when I’ve gone back to a CI run to ascertain what package versions that were installed at the time to verify a suspicion that an upgrade of a particular package was implicated in a failure. Is there any way this behavior could be brought back?

Again, done with intent. This long list slows down the run and pollutes the output with uninteresting details. I’m happy to accept a PR to enable it within CIs, see Listing dependencies · Issue #2685 · tox-dev/tox · GitHub.

@jaraco

Previously, tox --help would emit the environment variables that tox supported. Because they’re irregular (sometimes underscore-separated, other times not, sometimes prefixed by TOX sometimes not), it’s nice to be able to query for them rather than track down the online docs. Actually, I just checked and I can’t even find TOXENV in the docs. Is the environment variable behavior no longer documented?

Done with purpose. Now all CLI arguments can be overwritten by TOX_<flag_name>, similar to how pip and pytest do. As we no longer have irregular names, the list was removed. A PR documenting this is welcome.