UX Dave

In his book Selling Usability, John Rhodes suggests your email signature should include UX tips, advice and links.  He even gives us good examples to get started: 

John Doe UX King User interface is 47% - 60% of the lines of system or application code.

Some of it feels like shameless self-promotion but I think its importance goes beyond marketing.  After all user experience is not the result of one person’s job (unless you are a 1 person business).  It is up to everyone involved in business decisions to contribute to the user experience of your customers.  

Nielsen and Pernice discuss eye tracking as an input in their book Eyetracking Web Usability. This has been a long-standing research topic without any practical products up to the date of publication.  The company Tobii introduced such a product last year at the 2014 International Consumer Electronics Show (CES).   We are now at the point where sensors are so affordable it won’t be long before more products incorporate this technology.

The Tobii EyeX is currently available as a dev kit for only $135.  This article from Gizmag on Tobii’s eye-tracking technology describes a demo from the CES show floor and recounts the experience in detail.  Parts of its potential uses include OS commands, software and even games.  This eye tracking tool is being marketed as a complimentary input technology to the mouse and keyboard not as a replacement which is not surprising since it is at such an early commercial stage.  

I looked through the Touch Gesture Reference Guide table and I had some thoughts on the challenges you would encounter if you tried to develop 1 universal gesture framework for a given mobile app:

You need to account for somewhere between 7 (Windows 7 phones) - 12 (Surface tablets) possible gestures depending on devices.

Different names for the same gestures on different platforms (Drag -> Pan Windows 7 phone)

I imagine it would take a large amount of time to optimize interactions for each platform.  Not to mention if we tried to include gestures into usability testing it would take even longer.  I’m a definitely for gesture testing but I can understand why it may be overlooked or ranked as a lower priority when building/testing an application.  

According to the Pew Research Center, mobile device ownership is increasing at an accelerated rate.   Smartphone’s in particular have had a steep increase since 2012 and “as of October 2014, 64% of American adults have a smartphone.”

In the train, bus, and almost every time I see somebody using their phone they are accessing the Internet.  This brings me to my reflection for the week: “Why do people want to access my site from a mobile device?” If we can answer that question we will be on our way to greatly improving our mobile sites.  

Even if Google Analytics shows a moderate use of mobile devices it doesn’t necessarily mean that people do not want to access your site from their mobile devices.  It may just mean the current experience is not good enough for them yet.

The article titled “Make your own iPhone usability testing sled” provides instructions on building an easy affordable option.  If you look closely at the comments section you’ll see at least 7 other links on what other people use to test their mobile sites.   One of my favorite findings: 3d printing + GoPro + phone case = Mobile usability sled. 

I recently missed an opportunity in an asynchronous test...The opportunity to ask a follow-up question after each task.  I say it’s a missed opportunity because an important difference between analytics and usability testing is the chance to find out why people do the things they do.

This week’s assignment was an interesting experiment using Loop11.  I didn't have a very positive experience because unfortunately it wouldn't play nice with yelp.com but I can see it has some uses.  As with any test, I started the planning offline and built the test separately from the platform.  Once I made to Loop11, I liked that I was able to choose between questions and tasks.  Loop11 had a comprehensive list of question types which was useful in setting up diverse questions. So far I like the reporting interface; it’s very easy to understand.  I do miss not being able to have open ended task questions to observe behavior but I think that is more of an issue of unmoderated vs. moderated testing.  For example, one of my original tasks started as: “You are considering moving to the outskirts of Seattle but aren't quite where to move.  Use yelp.com to find an area with restaurants that offer your favorite food/dishes within a walking distance.”

I tried to add a little humor as I wouldn't expect restaurants to be one of the primary reasons to move somewhere for most people but I could be wrong.  The expectation here was the people would have to browse through several reviews and find additional information on each place which I hypothesize is closer to what people normally do.  In addition to that, I wanted people to have some involvement in the task.  Instead, I had to turn the task into:

“You are considering moving to Southeast Portland so you would like to find more information on family friendly restaurants in the area. Use yelp.com to find the highest rated family friendly Mexican restaurant in that area.”

It’s interesting to see how the landscape of remote usability testing continues to grow.  A lot of the tools discussed this week have been significantly refined since I last evaluated them.  Loop11 seems to be relative stable but the licensing cost can be relatively high for some.  One of the posts from this week really caught my attention: “Six Benefits of Remote Usability”.  When I think of remote testing for some reason I directly think of self-guided/un-moderated tools but it doesn’t have to be that way.

It’s very interesting how when you search the web for usability report templates you find very different documents.  Almost every report and example I looked at contained different sections, different content and simply had a different approach to making the deliverable.  I think the pointer “know your audience” is very important to make sure you get the most out of your report.  A 10-page report will probably be unnecessary and not as effective when done internally in a fast pace agile environment, as it would be in a more traditional environment and coming from a consultant.  However for UX documentation purposes, I think having comprehensive reports can be very useful for on-boarding new staff and to get people up to speed with past usability studies and as means to inform people of the challenges that a product has to deal with.

Making recommendation is tricky business as well.  Often times there are proven interaction patterns that may apply to a general type of application but may not be as effective if your use case or application differs in an important point. Measuring the impact of design changes is just as important as providing recommendations but can be often overlooked.  

What happened during your session that surprised you?

The participant was a little nervous at the beginning.  She was surprised at one of the pre-task questions: “What did you order?”

She later shared it felt like a personal question and did not understand the relevance of that question.  It would have helped to know why that question was important but I think I could have also interjected and said “It’s ok if you rather not say” or something along those lines to let the participant know it was ok to skip a question if they were not comfortable with it.

Were you better or worse than you thought you would be?

I did perhaps a little worse than I expected.  The participant was insightful and the tasks were clear-cut but I do wonder if there were a few points where I could’ve prompted more discussion for further insights since this test was very qualitative.

Were you able to remain unbiased?

I think I remained unbiased a good 90% of the time.  There were some instances where the participant needed clarification on the prompt and I felt I could have helped clarify a little more without crossing the line.  I think pausing and allowing the participant to try a while longer worked out in this case.

Did you let the participant speak?

I am likely biased because I enjoy qualitative work such as ethnography, interviews and plain conversations more than planning structured usability tests with the intent to use any type of rates as benchmarks.  Often times, this is not an issue of recruiting participants but that of keeping them for long enough to ask and measure everything you would like to. Unavoidably, people higher up always ask for numbers and measurements so even after interviews measurements and numbers are nearly impossible to avoid.  As an alternative, I heavily rely on analytics to add a quantitative perspective. However, I am the first to admit this is not great and a very difficult trade off.  

For one, my analytics aren’t automated enough or looked at enough to make an impact.    Furthermore, one has to be careful when using analytics versus study metrics because there is no way to guarantee correlation to a task or a particular website change…unless you do the statistical work which takes a specific set of knowledge and additional time (neither are available to me right now).  

Participant screeners are an important part of every usability study. Screeners can be easily overlooked but can help provide a better testing experience for everyone involved. It is important to isolate a group of people that represent your website users closely. Demographics are not as important as behavior when it comes to usability testing.

How do you make sure you recruit participants that represent different behavioral spectrums? I like the idea of using Internet knowledge (user knowledge and experience of the web) and task knowledge (user knowledge of particular domain) as two starting points to identify user groups as recommend by Steve Krug and David Travis. While it largely depends on the product you are testing, I have also found that Domain Knowledge can also make a difference when it comes to behavior in completing certain tasks.

The first thing that I thought after completing the recommended readings this week was that there is a disagreement in the intent or goals of each methodology (specifically formative research quoted below) by various experts:

“Formative evaluations involve evaluating a product or service during development, often iteratively, with the goal of detecting and eliminating usability problems.” - Usability body of knowledge

“The usability testing I was doing…was formative testing. By this, I mean usability testing that occurs early in the product design process, whose goal is not primarily to find usability problems that need to be fixed, but to assess the overall user experience and understand users’ reactions to different ideas.” - Michael Hawley

“The main objective of the exploratory study is to examine the effectiveness of preliminary design concepts. If one thinks of a user interface or a document as being divided into a high-level aspect and a more detailed aspect, the exploratory study is concerned with the former.” – Handbook of Usability Testing

So which one is it?  I think it’s a little bit of all the ideas above.  In fact I would side with the following quote strongly:

“I've come to think of formative and summative as a simplified way to characterize usability activities” – Jeff Sauro

I think maybe the root of the problem is that the distinction is largely theoretical.  It’s based on the assumption that design will follow the same path instructional design:

“The terms come from Educational Theory… formative—providing immediate feedback to improve leaning vs. summative—evaluating what was learned” – Jeff Sauro

This also implies a cyclical method of testing to fit x model and that companies have the budget to follow several stages of testing as proposed in the Handbook of Usability Testing:

Prototyping design and testing methods > final testing and product launch > Need analysis and requirement gathering > lather, rinse, repeat

This lacks practicality:

“In theory there is no difference between theory and practice. In practice there is.” – Yogi Berra

I would argue that other UX research such as Contextual Inquiries or other type of Ethnographic studies coupled with other research could replace the need to have the combination of formative and summative usability testing.  There’s a larger toolbox of UX research that usability tests are a part to pick from…however I understand it probably comes down to personal preference.