Summary
I was on a team that recruited and led twelve participants through an hour-long usability study of the Harvard Business Review website, HBR.org. The study included tasks on both desktop and mobile. We presented a summary of 23 usability issues, with recommendations, to the HBR.org product team.
Case Study
Bentley University's HF 750 course (User Testing and Assessment) assigns students into groups with a sponsor project. My group's sponsor was the Harvard Business Review, which wanted us to perform usability testing on their newly-redesigned website, HBR.org. The research questions we sought to answer centered around navigation, search, the article library, and checkout. An example research question was, "How easily and successfully can users locate and download purchased digital products?"
Method
We recruited 11 participants, including frequent users, infrequent users, business professionals, and graduate business students. Each participant was given a set of tasks to perform, asked to think aloud while executing the tasks, and then rated their experience. Each user test took about one hour. Tasks included:
Start on desktop
- Find an interesting article, start reading it
- Save the article for later
Switch to mobile
- Continue reading the article found earlier
- Find an e-book or case study you would like to buy
- Buy the product
Findings
We categorized usability issues according to Jakob Nielsen's usability severity ratings, from cosmetic to catastrophic. We reported 5 catastrophic, 6 major, 6 minor, and 6 cosmetic usability issues. In future studies I would prefer to use the Dumas and Redish severity ratings because of their specific definitions tied to task success.
The study confirmed that the site's responsive design made transitioning from desktop to mobile an easy task. Participants were pleased with the design and most of the interactions, including the checkout process. Participants had some difficulty with search and filtering, as well as recovering from errors in the checkout process.
The HBR product manager who sponsored this study has cleared us to share our report, so if you would like to see the raw data or finished report, please feel free to ask me. One example slide is shown below to give you an idea of what information the presentation includes. I contributed to designing the study, moderating participants, note-taking, analyzing results, and writing content in the presentation. I did not contribute to the design of the slides, which is too bad, because this is such a great design, right?