A Few Discoveries About Remote Usability Testing
I just completed some remote usability testing (3 on usertesting.com, 3 on youeye.com) for a new tool I am working on at my company, Smart Destinations. Sure, I learned about some flaws in the interface, but I also learned a few things about setting up remote usability testing:
1. Be explicit in your instructions; avoid pronouns.
"What can you do on this page? How do you expect it to work?"
In normal writing or speech, "it" would clearly refer to "this page." However, almost all participants assumed "it" referred to the product discussed on the page, the Go Chicago Card. This led to some confusion and frustration for them, and little useful feedback for me.
2. Provide enough information
It's hard to strike the right balance between giving testers sufficient context and bogging them down with details or worse, providing them information you actually want them to get from the page(s) you're testing. However, if your test is focused, especially on a subpage or specific feature, it's better to err on the side of too much information, so the tester doesn't get distracted and veer off course.
3. Make clear what portion of the site the person will be testing.
Most people assume they are starting on (what they refer to as) the "homepage." This struck me as odd, given how few of our site's visitors actually do start on the homepage. Do they assume this only in the context of usability testing? Or whenever people click through to a website from a search engine result page, do they generally assume they're landing on the "homepage"? Perhaps people misunderstand what a homepage is? I'm not sure, but the upshot is you may want to make clear in the introduction where in the overall site hierarchy the tested page sits.
4. Steve Krug is right, 3 is enough.
Obviously, this is not my own discovery. I chose 6 participants for this test, but learned 95% of what I learned overall after the first three participants. It would far more productive to do frequent tests with fewer participants.
5. Seeing people's faces may not be all that helpful
So, when I first tried youeye.com, I thought actually seeing the participants in their webcams was pretty compelling. However, having run this second test on both usability.com (no webcam), and youeye.com, I wonder how much more you learn from their facial expressions than just their tone of voice. In Rocket Surgery Made Easy, Steve Krug says don't worry about recording user's faces. He may be right.