by Chris Myhill
Director of Experience
Picture the scene. You’ve just completed a lengthy user testing study of a website or app. Moderating and analysing those sessions took some time, but you’ve gotten some great findings. Armed with this research, you know exactly how to improve the design. Great!
The thing is, you can’t jump straight into making these fixes.
Assuming you work as part of a larger team, you’ll probably need those changes approved. This means sharing your findings. Whether it’s to your teammates, your boss or your client – you’ll need them to understand (and agree) what you’ve learned.
But what happens if they don’t agree?
What if they don’t engage with the results, or even believe your suggestions?
If user testing findings are presented poorly it can be a disaster. Taking the right steps to create an effective report can make all the difference. Without further ado, here are 7 steps for communicating your user testing results more effectively.
If user testing findings are presented poorly it can be a disaster.
During my studies, I was taught very specific methods for conducting and reporting on user testing. Given the nature of academia these techniques were what you’d expect. Overly formal and bit long-winded (sorry, academic friends!).
Looking back at the user testing reports I wrote in the early days of my career, I feel a bit embarrassed.
They included all of the nitty-gritty technical details I’d been taught to think about. I produced 100-page monstrosities stuffed full of complicated tables, graphs and waffly write-ups. Yuck.
I was naive. I thought I was being thorough. I was actually boring my readers to tears. Because the reports were so hefty, people were put off from reading them. They created the exact opposite effect that I wanted.
In a previous post I discussed the importance of keeping your user testing casual. The same thing applies to reporting, too. William Shakespeare once said “Brevity is the soul of wit”. By this he meant that to engage your audience, you’ve got to keep things punchy.
The faster you can communicate your insights, the less of your reader’s busy day you’ll be taking up. Trust me, they’ll thank you for it.
Here’s what I’ll include in a user testing report these days :
Ideally your report should be no more than 20 pages long. Any longer, and you’re at risk of losing the reader’s attention.
There will always be those who just won’t read your full report. No matter how punchy you make it, they won’t take the time.
Unfortunately, these are often the senior level stakeholders who will ultimately approve your design decisions. So we need to cater for these people. It’s irritating, but that’s life. Including a brief summary at the very front of your report is the best way to approach this.
I tend to split the summary into three separate pages.
These summary pages should contain really simple descriptions of the findings. Single bullet points if possible. A simple table can work well, too. For each of these points, give a note of the issue severity. Low, medium or high.
The summary doesn’t need to be any more complicated. If the reader wants more detail, then they’ve always got the full report.
Most UX designers will know how tricky it is to engage audiences with text alone.
Wherever possible, share your insights using visuals. Rather than writing long descriptions of the situations that occurred, provide an annotated screenshot instead. When discussing a particular insight, I’ll always show the particular screen or state in question.
Even better than screenshots, video clips from your session recordings can help to stress a point. You can easily share specific clips using the ‘start at time’ feature of YouTube or similar players.
If you really want to go the extra mile, video editing tools like Camtasia or Adobe Premier can be used to make montage videos for each issue.
Structure for presenting user testing insights
When producing the ‘results in detail’ section of my reports, I’ll only ever include one insight per page. This is to keep the report easily scannable.
Dedicating a whole page to each issue ensures we keep the report focused. It also gives room to present issues in a more impactful way.
Aim to include :
That’s all you need. If it takes more than one page to articulate the research finding, then you’re overcomplicating it!
Speaking of quotes. When it comes to stressing a point, using direct quotes from your participants is incredibly powerful.
A recommendation or insight can be debated, even when supplied by an expert. But nobody can argue with the user themselves. The customer is always right.
Copy down some key quotes for each insight you’re including in the report. These should be short phrases that perfectly sum up the feedback being given, or the issue experienced.
Quotes add character to your report. They help make the results more engaging. The reader will feel like they’ve got more of a direct line to the user themselves, creating a sense of empathy. This will really help them to understand the user’s frustrations, and hopefully agree with your recommendation.
I try to use one or two quotes for each insight in a report. Don’t overdo it though, or they’ll lose their impact. Stick to quality over quantity.
Every issue or insight you include should have a priority rating of low, medium or high.
The priority you give an issue should be determined by its impact on the user experience.
If the issue posed a mild inconvenience, then it’s probably low priority. If it outright caused a task failure, or it visibly angered them, you’re probably dealing with a high priority issue. Highlighting the priority of an issue helps the team understand the importance of resolving it.
It’s likely the product you’re testing already has a development backlog. High priority issues need to be addressed as soon as possible, but the lower priority ones could potentially wait for a little while.
It’s also worth indicating whether or not your proposed solution is a ‘quick win’. Sometimes a high priority issue can be resolved with some small minimal tweaks to the design. It’s certainly worth calling them out.
A good report should be actionable. We should also close the report with a summary of the next steps.
These are the actions that the whole team need to take following the study. A lot of these will be design or implementation activities to develop your recommendations, but that’s not all. There may also be other requirements.
It’s important to break this out as a separate section in your report. It makes your team aware of the process, and what’s expected of them. Your project manager may also need to coordinate these future activities, so it helps make their requirements extra clear.
It should be clear by now that your reports should be kept simple.
That said, sometimes there might be a requirement to add some extra details. If you really must share your workings – such as scripts, transcripts or tables – put them in an appendix at the end.
I’ll tend not to do this unless my team or client specifically wants to see this stuff.
Again. It’s all about keeping the report focused for the majority of readers. They don’t need to see what magic happens behind the curtain!
Just like the products we produce for our end-users, our documentation needs to be useful and usable.
If we’re not engaging our teammates and making their lives easier, we’re not doing our jobs properly. Follow these steps, and your user testing results will go a lot further.
And remember. If in doubt, just keep it simple!
15th September 2020
If we want our projects to run smoothly, we need to involve our clients, teams and users throughout the design process.
2nd September 2020
There’s a lot more to design than just the user interface. We need to think holistically to create the best experiences.
14th August 2020
Affordances are characteristics that suggest interactivity. They provide hints that help users to understand your UI faster.