UX Research Results

Over the course of 2020, the pip team worked on improving pip’s user experience, developing a better understanding of pip’s UX challenges and opportunities, with a particular focus on pip’s new dependency resolver. The Simply Secure team focused on 4 key areas:

• Understanding who uses pip

• Understanding how pip compares to other package managers, and supports other Python packaging tools

• Understanding how pip’s functionality is used could be improved, and

• Understanding how pip’s documentation is used, and how it could be improved

Some key outcomes from the 2020 work are:

• This documentation & resource section!

• A pip UX research panel (Sign up here!)

• New and expanded GitHub issues

• UX improvements in 2020

  • UX work supporting the dependency resolver

  • Improved error messaging

  • Supporting Documentation

• UX Training for the Pypa + pip maintainers

This work was made possible through the pip donor funded roadmap.

Outreach

We recruited participants for a user research panel that we could contact when we wanted to run surveys and interviews about pip. In total 472 people signed up to the panel, although some unsubscribed during the research period.

At the end of the 2020 research, we asked users to opt-in to a long-term panel, where they can be contacted for future UX studies. Should the pip team wish to continue to build this panel, we recommend translating the sign-up form into multiple languages and better leveraging local communities and outreach groups (e.g. PyLadies) to increase the diversity of the participants.

User Interviews

In total, we interviewed 48 pip users, recruited from the user panel, and through social media channels.

During the interviews, we asked users about:

• How they use Python

• How long they have been using pip

• Whether or not they use a virtual environment

• If and how they address security issues associated with pip

• Which pip commands they regularly use

• How they install packages with pip

• Their experience using pip list, pip show and pip freeze

• Their experience using pip wheel

• Whether or not they use other package managers, and how pip compares to their experience with these other tools

• What the pip team could do to improve pip

• Problems they have experienced while using pip, and how they solved these problems

• Their perception and use of the pip documentation

• What other technical documentation they value, and how the pip docs could take inspiration from these

• What other resources the pip team could provide to help pip users solve their problems

Surveys

We published 10 surveys to gather feedback about pip’s users and their preferences:

Title	Purpose	Results
Pip research panel survey	Recruit pip users to participate in user research, user tests and participate in future surveys. See associated blog post for more information.	472 full sign-ups
Feedback for testing the new pip resolver	Understand use cases where the new resolver fails due to dependency conflicts. See associated blog post for more information.	459 responses via the feedback form, approx. 8 issues transferred to issue tracker
How should pip handle conflicts with already installed packages when updating other packages?	Determine if the way that pip handles package upgrades is in-line with user's expectations/needs. See related blog post and GitHub issue for more information.	See write up, including recommendations
Learning about our users	Learn about pip's users, including: their usage of Python and pip why and how they started using Python if they are living with any disabilities, and if so what effect (if any) this has on their usage of Python and pip if they use assistive technologies when using Python and pip and how this work for them where they get support when you have issues with pip	See write up
Buy a pip feature	Establish which features are most important to pip's users	See write up
Should pip install conflicting dependencies?	Establish whether pip should provide an override that allows users to install packages with conflicting dependencies	See write up
How should pip force reinstall work?	Establish whether or not pip force reinstall should continue to behave the way it currently does, if the functionality should be changed, or if the option should be removed	See write up
Feedback on pip search	To establish whether or not to remove or redesign pip search. See this GitHub issue for more information.	See write up
Feedback on pip's docs	To gather feedback on pip's docs, supplementing feedback gathered in user interviews	See write up

All Results

• About pip’s Users
• How Users Understand pip
• How pip users think about security
• How pip is used in interactive environments (i.e. CI, CD)
• pip Personas
• Prioritizing pip Features
• Providing an override to install packages with conflicting dependencies
• pip --force-reinstall
• pip search
• pip Upgrade Conflict
• Improving pip’s Documentation

Read More

• Pip team midyear report (blog, July 2020)

• Creating rapid CLI prototypes with cli-output (blog, Oct 2020)

• Changes are coming to pip (video)

• How should pip handle dependency conflicts when updating already installed packages? (blog, July 2020)

• Test pip’s alpha resolver and help us document dependency conflicts (blog, May 2020)

• How do you deal with conflicting dependencies caused by pip installs? (blog, April 2020)

• pip UX studies: response data (blog, March 2020)

Other PyPA UX work:

• PyPI User Research (blog, July 2018)

• Warehouse - The Future of PyPI

• Accessibility on Warehouse (PyPI) (blog, May 2018)

• User Testing Warehouse (blog, Mar 2018)

• Designing Warehouse - An Overview (blog, Dec 2015)