If you work on Python packaging tools, I want you to know about newly available research reports and design documentation that may be applicable to your work.
More details:
Back in 2020, during grant-funded work on the next-generation pip resolver, the consultancy Superbloom (previously Simply Secure) did fascinating user experience research and design work. They wrote several useful documents that took a while to get merged, but now live in pip’s documentation! Like:
- how users thought pip should react to dependency conflicts
- what users used
pip searchfor - how pip users thought about and dealt with package security
- how to design a survey for pip
It was so valuable to get to work with UX experts on pip. The more we knew about users’ experience, the better decisions we could make. I recently talked with David Lord, a Pallets maintainer, who wants to know for his projects:
- He puts effort into consistent deprecation warnings. Are they effective? Are people seeing those warnings early enough to avoid unpleasant surprises when functionality changes? If not, what could he do to communicate this information to them more effectively?
- Who’s using this software? That is: do their needs cluster around a few major use cases/workflows? If so, what are they?
- Which, if any, of the new features from the past few years do they use?
- How do they seek help? Do they make an effort to start with the official docs, or just search the web and often find old blog posts written by enthusiasts?
- What level of detail do they need in an error message spouted to the command line, and what would be so long that they’d be more likely to skip reading it?
And that’s the kind of stuff we got in this work in 2020. Example: their research findings helped us improve the format and content of the ResolutionImpossible error message, so users could actually work out what went wrong and how to fix it. UX analyses and training have helped massively improve pip’s user experience design, including error messages and docs, as well as the direct ergonomics of invoking the right commands and having them do what users expect.
We didn’t limit ourselves to only researching the UX of pip, because that wouldn’t make sense; pip is an interdependent part of the Python package creation and distribution ecosystem, and we needed to understand how users reasoned about, researched, and learned about packaging broadly. And – I believe – learning these tools and facts has been, and can be, helpful to other packaging tools maintainers beyond pip.
(Side note: I see this as a model that we can replicate in other ecologies, especially groups of related command-line tools. Pool funds and invest in UX research for a suite of tools that people often use together, and learn surprising and helpful things that help you all improve your projects’ UX. I hope this work can inspire maintainers elsewhere to do likewise.)
And last year, Superbloom published more resources to help you start to do user testing and usability testing in your projects, including:
- a user testing HOWTO, also available as an interactive guide on
itch.io - a HOWTO for synthesis (“stage of research in which you reorganize information to make sense of what you observed and heard”)
Hope this is helpful!