Design iterations: the proof of concept.
Experience design is not science. There are no text-book rules backed by tons of research conducted over a hundred years. Moreover, the profession as we know it only exists for the past few decades. Which means, all the accumulated knowledge we have now is empiric rather than scientific, and it still has its voids. Common sense is not the most solid turf to build on as well, given mental models of different people may vary a lot depending on the profession, interest in tech, and life experiences. That’s why we should be ready to test our ideas regularly and get rid of them as soon as we find an opportunity to work on the best ones.
Conducting this experiment, the researcher observes the behavior of users while they complete specific tasks using the product in question.
When it works best
It comes in handy when you aim to find valuable insights on your product weaknesses by gathering qualitative data. As professionals, we are interested in tech, which leads to a good understanding of it. Most people prefer other areas of human knowledge, so they use tech for work or everyday routine rather than for pleasure. Hence, some of our assumptions as designers may be incorrect. Observing the way people interact with our product is a great way to find out whether we understand their needs right.
How to prepare
Define the key scenarios. You may want to run every possible situation through your respondents, but that won’t be effective, because people tend to get tired. It’s better to test 2–3 scenarios at a comfortable pace than try to cover all of them skin-deep.
Learn how to experiment
Check out the guideline for usability testing by Google.
Write a script
Moderating the experiment may be an overwhelming task. Spare yourself some unnecessary stress: prepare a script to rely on. Learn it by heart as well — that would help you navigate through the experiment.
Make a test run
Everything may go not as planned — double-check if there are any issues before recruiting respondents. Ask your teammates to assist.
Fix the issues
The first trial will likely be imperfect. Take your time to improve it. Schedule some breaks to reflect on the test run process and fine-tune the script. Take a deep breath.
Which tools to use
- Mobile/action camera — you are not going to film an action movie, so professional gear is not essential.
- Camera holder — the cheapest table mount would do the trick.
- An empty room or any other quiet place — it’s great if you have a meeting room, but a quiet corner in the local cafe is ok too. Don’t forget to ask your respondents if they feel comfortable about it.
- Laptop or mobile phone with your prototype ready to go.
- Tea, coffee, snacks — a friendly and informal environment would help respondents relax and behave as usual.
How to conduct
Design the experiment
Define your key questions and put together 2–3 tasks which would help you to gather most answers.
Example: you are testing an online apparel store.
Bad task: Buy something pretty.
Another bad task: Search jeans, then select the color blue and medium-size and hit “Add to cart” button…
Good task: Are you going to shop for clothes sometime soon? What do you think about buying? Buy this or similar item on our site using this test credit card.
In most cases don’t have to find people who exactly match your target audience to test interaction patterns, since those patterns are usually universal. Ask your friends or colleagues from non-techie departments to participate in testing. Don’t hesitate to reach out to your co-workers to help you with recruitment. Remember to give credit to people who helped and to share your results with them.
Make sure your recruits aren’t tech-savvy. Otherwise, it may appear as the interface performs well while, in fact, it’s respondents who are unusually good with tech.
Here’s a little anecdote about that: I’ve been conducting an extensive usability testing on somewhat complicated features. My respondent was a gym receptionist, and she adored all the features immediately — there wasn’t any learning curve at all — she dug right in. I grew suspicious and started to ask her about her interests, hobbies, and previous experiences. Turned out, she studied to be a personal trainer at the moment, but she still had strong ties to her previous career — a software engineer. The agency we hired to recruit her only asked about her job, but not about her education. It was an hour well spent, all right, but, unfortunately, I had to exclude her results from the set.
Prepare the equipment
Start with the product you are going to test: confirm that everything works well. If you’re testing a prototype, double-check all the links. If you work for a foreign client remotely, translate UI copy for your respondents. Set the camera before your first respondent shows up, make sure that the screen of the device you use for testing is visible on the video. Check if all the devices are charged and keep the aUXiliary power source ready. Don’t forget about test accounts and credit cards if needed — people usually uncomfortable sharing their personal information, especially when the software may be unstable.
Moderate the experiment
This part may be difficult if you’re testing your work because outcome not always (pretty seldom, actually) matches our expectations. Remember the ultimate goal — to create a better product and to become a better professional, not to appear right in a particular situation. So, don’t lead participants in any way and don’t give them hints unless they are completely stuck and frustrated. The only reason to give a tip is not to let your experiment end prematurely. Don’t cheat 😉
Process the results
The approach to processing is the same as for interview and field observation.
How to deliver results
The ultimate advantage of this experiment is that you get immediate insights. For your deck, you may group issues by kind (software bug, logical errors, visual design slips, unclear UI copy, and so on) or by severity from critical bugs to minor improvements. You also may want to cut out the most insightful moments of your experiment and show them during your in-person presentation to the team. Avoid embedding videos to a deck though — it may cause technical problems.
I suggest the following structure for the slides:
- Bug description (+ kind);
- The reason to fix it;
- Possible solutions.
If the number of issues is extensive, you may add photos and some quotes from your respondents to build empathy in your team and to enliven the deck a little bit.
When it fails
Researcher jumps to conclusions during the experiment
Detach yourself from the prototype in question while experimenting. Don’t think of the solution immediately after you see the issue. The goal is to find as many bugs as possible rather than hot-fix a few interaction issues. Also, by occupying yourself with finding solutions, you risk drawing your attention from something important.
The task is very detailed; the researcher asks leading questions
If you give step-to-step directions to your users, you will never know whether they would find the way on their own.
The task is too vague; questions are also common
General queries lead to general answers. The goal is to learn about weak links of interaction, not to gather opinions.
Respondents are overly prepared
The majority of people in the world are neither tech-savvy not interested in tech much. As UX designers, we communicate mostly with people who know a little something about software. So sometimes we fall under the impression the world is way more advanced than it is. To avoid the confirmation bias, we should aim to recruit respondents who don’t have any tech aspirations.
Respondents don’t understand their task
In most cases, it doesn’t matter whether respondents match your target audience exactly or not. Though if you are going to test a niche software for skilled professionals, you should search for users, who have a solid understanding of a subject area. Though, if you are going to test a site selling toys and your marketing agency team believes the target audience is 30 to 40 y.o. middle-class women, it’s still ok to test your prototype on 23 y.o. male student.