#ethnographic interviews

We talked about the need for rest, the negative impacts of 'grind-culture' and America's obsession with productivity. We talked about how much better things could be if social safety nets worked better, and properly supported people all around.

I'm excited to talk to my participants in real time about their experiences.

Stanford University sociologist Asad L. Asad

“Ask attitudinal questions last.” - Asking interviewees about their feelings on a subject may taint the rest of their answers. Best to hold these sort of questions until the end. 

In an industry devoted to the people who use our products, services, and applications, research is paramount. We ask questions. We take notes. We learn everything we can about the target audience, and then iteratively test our work throughout the design process. UX research—or as it’s sometimes called, design research—serves many purposes throughout the design process. It helps us identify and prove or disprove our assumptions, find commonalities across our target audience members, and recognize their needs, goals, and mental models. Overall, research informs our work, improves our understanding, and validates our decisions. In this Complete Beginner’s Guide, we’ll look at the many elements of design research, from interviews and observations, to usability testing and A/B testing. Readers will get a head start on how to use these design research techniques in their work, and improve experiences for all users. Table of Contents * What is UX Research? * Common Methodologies * Daily Tasks and Deliverables * People to Follow * Tools of the Trade * Associations and Conferences * UX Research Books What is UX research? UX research encompasses a variety of investigative methods used to add context and insight to the design process. Unlike other sub-fields of UX, research did not develop out of some other field or fields. It merely translated from other forms of research. In other words, UX practitioners have borrowed many techniques from academics, scientists, market researchers, and others. However, there are still types of research that are fairly unique to the UX world. The main goal of design research is to inform the design process from the perspective of the end user. It is research that prevents us from designing for one user: ourselves. It’s fairly well accepted that the purpose of UX or user-centered design is to design with the end-user in mind, and it’s research that tells us who that person is, in what context they’ll use this product or service, and what they need from us. With that in mind, research has two parts: gathering data, and synthesizing that data in order to improve usability. At the start of the project, design research is focused on learning about project requirements from stakeholders, and learning about the needs and goals of the end users. Researchers will conduct interviews, collect surveys, observe prospects or current users, and review existing literature, data, or analytics. Then, iteratively throughout the design process, the research focus shifts to usability and sentiment. Researchers may conduct usability tests or A/B tests, interview users about the process, and generally test assumptions that will improve the designs. Young, Indi. 2008. Mental Models: Aligning Design Strategy with Human Behavior. New York: Rosenfeld Media. We can also divide UX research methods into two camps: quantitative and qualitative. * Quantitative research is any research that can be measured numerically. It answers questions such as “how many people clicked here” or “what percentage of users are able to find the call to action?” It’s valuable in understanding statistical likelihoods and what is happening on a site or in an app. * Qualitative research is sometimes called “soft” research. It helps us understand why people do the things they do, and often takes the form of interviews or conversations. Common questions include “why didn’t people see the call to action” and “what else did people notice on the page?” Though researchers may specialize in specific types of interviews or tests, most are capable of conducting a wide variety of techniques. All researchers collect the valuable information that allows us to design in an informed, contextual, user-centered manner. Common Methodologies The various types of UX research range from in-person interviews to unmoderated A/B tests (and everything in between), though they are consistent in that they all stem from the same key methodologies: observation, understanding, and analysis. Observation The first step to conducting research is learning to observe the world around us. Much like beginning photographers, beginning researchers need to learn how to see. They need to notice nervous tics that may signal that their interviewees are stressed or uncertain, and pick up on seemingly minor references that may reflect long-held beliefs or thoughts that should be further probed. Observation may seem like a simple skill, but it can be clouded by unconscious biases—which everyone has. Design researchers train themselves to observe and take notes so that they can later find patterns across seemingly diverse groups of people. Understanding Much like observation, understanding is something we do all the time in our daily lives. We strive to understand our coworkers, our families, and our friends, often trying to grasp a point of contention or an unfamiliar concept. But for UX researchers, understanding has less to do with disagreements, and more to do with mental models. A mental model is the image that someone has in their mind when they think of a particular phrase or situation. For example, if someone owns an SUV, their mental model of “car” will likely differ from the mental model of the owner of a smart car. The mental model informs the decisions we make; in the case of the car owners, when asked “how long does it take to drive to Winnipeg,” their answers will vary based on the gas mileage their vehicles get, among other things. Design researchers need to understand the mental models of the people they interview or test, for two reasons. First, we all speak in shorthand at times. Researchers must recognize that shorthand based on the mental model of the speaker. Second, if the researcher can accurately identify the user’s mental model, he or she can share this information with the design team, and design to accommodate the model. Analysis Research on its own can be valuable, but in order to use the insights to inform design, it needs to be analyzed and ultimately presented to a larger team. Analysis is the process by which the researcher identifies patterns in the research, proposes possible rationale or solutions, and makes recommendations. Some analysis techniques include creating personas or scenarios, describing mental models, or providing charts and graphs that represent statistics and user behaviors. Although the techniques described here are focused predominantly on conducting research, it’s important to remember that research is only valuable if it is shared. It does no one any good when it’s locked away in a cabinet, or forgotten in the excitement of design. Daily Tasks and Deliverables Every UX project is different, and the tasks that one researcher takes on will differ from those appropriate in another setting. Some of the most popular forms of research are interviews, surveys and questionnaires, card sorts, usability tests, tree tests, and A/B tests. Interviews One-on-one interviews are a tried and true method of communication between a researcher and a user or stakeholder. There are three main types of interviews, each of which is used in a different context and with different goals. Directed interviews are the most common sort. These are typical question-and-answer interviews, where a researcher asks specific questions. This can be useful when conducting interviews with a large number of users, or when looking to compare and contrast answers from various users. Non-directed interviews are the best way to learn about touchier subjects, where users or stakeholders may be put off by direct questions. With a non-directed interview, the interviewer sets up some rough guidelines and opens a conversation with the interviewee. The interviewer will mostly listen during this “conversation,” speaking only to prompt the user or stakeholder to provide additional detail or explain concepts. Ethnographic interviews involve observing what people do as they go about their days in their “natural habitats.” In this sort of interview, the user shows the interviewee how they accomplish certain tasks, essentially immersing the interviewer in their work or home culture. This can help researchers understand the gaps between what people actually do, and what they say they do. It can also shed light on things that users do when they are feeling most comfortable. Surveys and Questionnaires Questionnaires and surveys are an easy way to gather a large amount of information about a group, while spending minimal time. These are a great research choice for projects that have a large and diverse group of users, or a group that is concerned with anonymity. A researcher can create a survey using tools like Wufoo or Google Docs, email it out, and receive hundreds of responses in just minutes. There are downsides to surveys and questionnaires though. The researcher can’t interact directly with the respondent, and therefore can’t help with interpreting questions or framing them if the wording isn’t quite perfect, and they typically have a limited ability for follow up. Surveys see a far higher response rate when they do not require a login or contact information, and this anonymity makes it impossible to ask for clarification or further details. Card Sorts Card sorts are sometimes done as part of either an interview or a usability test. In a card sort, a user is provided with a set of terms, and asked to categorize them. In a closed card sort, the user is also given the category names; in an open card sort the user creates whatever categories he or she feels are most appropriate. The goal of a card sort is to explore relationships between content, and better understand the hierarchies that a user perceives. Many content strategists and information architects rely on card sorts to test out hierarchy theories, or kickstart work on a site map. Usability Tests Usability testing involves asking potential or current users of a product or service to complete a set of tasks, and observing their behavior to determine the usability of the product or service. This can be done using a live version of a site or app, a prototype or work-in-progress, or even using clickable wireframes or paper and pencil. While there are many variations and styles of usability tests, there are three that are commonly used: moderated, unmoderated, and guerrilla. Moderated usability tests are the most traditional type of test. They can happen in person, or via screenshare and video. Whole usability labs are set up, complete with one-way mirrors for stakeholders to observe, for the purpose of conducting moderated usability tests. In a moderated test an unbiased facilitator sits talks with the user, reading aloud the task and prompting the user to think aloud as he or she accomplishes the task. The facilitator’s role is to act as a conduit between stakeholders and the user, phrasing questions to evaluate the effectiveness of a design and test assumptions while helping the user feel comfortable with the process. Unmoderated usability tests, sometimes also known as asynchronous research, is conducted online, at the user’s convenience. The tasks and instructions are delivered via video or recorded audio, and the user clicks a button to begin the test and record his or her screen and audio. Just like in the moderated test, users are encouraged to speak their thoughts aloud, though there is no facilitator to ask follow up questions. Unmoderated tests are available through numerous online sites, and can be significantly cheaper than moderated tests. Guerrilla testing is a modern, lightweight take on traditional tests. Instead of renting a lab, guerrilla research is typically done out in the community; users are found at coffee shops or subway stations and asked to complete basic tasks with a website or service, in exchange for a few dollars, a coffee, or just out of the goodness of their hearts. While guerrilla testing is a great option, particularly on a budget, it is best used only for products or services with a large user base. More niche products will struggle to find reliable information from the random selection acquired in guerrilla testing. Tree Tests Just as card sorts are a great way to gather information before website’s architecture has been created, tree tests are helpful in validating that architecture. In a tree test, users are given a task, and shown the top level of a site map. Then, much like in a usability test, they are asked to talk through where they would go to accomplish the task. However, unlike in a usability test, the user doesn’t see a screen when they choose a site section. Instead, they will see the next level of the architecture. The goal is to identify whether information is categorized correctly, and how appropriately the nomenclature reflects the sections of the site. A/B Tests A/B testing is another way of learning what actions users take. An A/B test is typically chosen as the appropriate research form when designers are struggling to choose between two competing elements. Whether the options are two styles of content, a button vs. a link, or two approaches to a home screen, an A/B test requires randomly showing each version to an equal number of users, and then reviewing analytics on which version better accomplished a specific goal. A/B testing is particularly valuable when comparing a revised screen to an older version, or when collecting data to prove an assumption. People to follow Many people have contributed greatly to the field of experience design research. Here are just a few that readers may choose to follow, to learn more. Jennifer Romano Bergstorm Twitter As a UX Researcher at Facebook, Jennifer spends her time improving privacy settings and the UX of Facebook in emerging markets. She specializes in eye tracking, usability testing, survey design, and UX for older adults. Jennifer is the president of the International UXPA, and is the co-author and editor of Eye Tracking in User Experience Design. Find her online at http://www.romanocog.com/. Gregg Bernstein Twitter Gregg works as both a user and product researcher. He built the research practice at MailChimp, and is now the senior researcher at Vox Media. Gregg speaks around the world about his work at conferences and for companies including Etsy, EA Games, and Fullstory. He formerly taught design, branding, and typography at Georgia State University and the Savannah College of Art and Design, and is now an occasional guest lecturer. Nate Bolt Twitter Nate Bolt has held numerous research positions, often with a focus on remote research. He ran the research and design firm Bolt | Peters, which focused on remote work and was acquired by Facebook. Nate then became a design research manager at Facebook and Instagram, and now works on Ethnio, a project begun at Bolt | Peters which became the first moderated remote user research software. Nate keeps an online blog, Boltron, and in 2010 authored Remote Research: Real Users, Real Time, Real Research. Dana Chisnell Twitter As the sole proprietor of Usability Works, Dana Chisnell has helped hundreds of people make better design decisions by giving them the skills to gain knowledge about users. Dana also runs Civic Design, where she researches and publishes ways to improve ballot usability and design. She co-wrote Handbook of Usability Testing, Second Edition with Jeff Rubin, and blogs online at her site, usabilityworks.net. Erika Hall Twitter In 2001, Erika co-founded Mule Design Studio with Mike Monteiro, where she directs the research, interaction design, and strategy practices. She speaks at conferences and writes on Medium about cross-disciplinary collaboration and the importance of natural language in user interfaces. Erika is the author of Just Enough Research, and the creator of Unsuck-It, a site dedicated to removing and replacing useless jargon. Steve Krug Twitter When usability testing comes up, there’s one name that people think of: Steve Krug. Steve has been working as a usability consultant for over 20 years, starting before most people thought of user experience or usability as worthwhile concerns. What makes Steve’s work stand out is the way he has championed UX practices and low-tech usability testing in his books Don’t Make Me Think: A Common Sense Approach to Web Usability, and Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability Problems. Steve Portigal Twitter Steve Portigal is a speaker, author, and consultant in UX design and development. His book Interviewing Users: How to Uncover Compelling Insights is one of the most clear and illuminating books available for anyone learning to truly listen to users. He also produces a monthly podcast called Dollars to Donuts, where he speaks with user researchers at a variety of organizations, and blogs regularly. Kyle Soucy Twitter Kyle has spent the last 15 years researching and designing usable interfaces for projects. She is the founding principle of Usable Interface, writes for industry trade magazines, and has spoken at Big Design, CHI, IA Summit, SxSW, UXPA, WritersUA, and Web 2.0 Expo, among other conferences. Kyle is the Founder and Past President of the New Hampshire Chapter of the User Experience Professionals’ Association (NH UXPA), she has served as the Chair of PhillyCHI, the Philadelphia Chapter of the Association for Computing Machinery’s Special Interest Group on Computer-Human Interaction (ACM SIGCHI), and she served as the New Hampshire Ambassador for the User Experience Network (UXnet). Jared Spool Twitter Jared Spool is the founding principle of User Interface Engineering, which he started in 1988. He has more than 20 years of experience conducting usability evaluations on a variety of products, and is an expert in low-fidelity prototyping techniques. Jared is the author of Web Usability: A Designer’s Guide and co-author of Web Anatomy: Interaction Design Frameworks that Work with Robert Hoekman Jr. He is a recognized authority on user interface design and human factors, and gives the keynote annually at UIE’s UI Conference and UX Immersion Conference. Indi Young Twitter In 2001, Indi Young co-founded the user experience agency Adaptive Path, which has provided leadership for the experience and strategy design field. She now teaches for Rosenfeld Media, with a focus on problem space research. Indi is constantly finding new ways to understand customers and employees in order to support them better. She is the author of Practical Empathy and blogs at www.indiyoung.com. Tools of the trade User Research has the potential to be a sizable undertaking, sometimes to the point that budgetary and scheduling concerns scare people away. Fortunately, today we often see a more casual, habitual approach. The tools we have available are responsible for much of that shift. Ethnio Ethnio was the first moderated remote research software when it launched, and it’s still going strong. Ethnio finds users who are currently using a site or app, and (with their permission) allows interviewers to ask them questions about their experience as they go. It automates many elements of the typical in-person test, including real-time notifications, and paying participants with Amazon gift cards. Ethnio has a fourteen day free trial, and four pricing options, to accommodate businesses of every size. Learn more about Ethnio Optimal Workshop Optimal Workshop has everything! The full Workshop is a bundle of four research tools, all of which are also available sold separately (and very affordably). Treejack is great for remotely testing information architecture, either to test the nomenclature or the hierarchies themselves. Optimal Sort provides online card sorting, to see how users choose to organize content. Chalkmark offers heat maps of click patterns across a site, and Reframer is a tool for taking notes and identifying themes easily. All come highly recommended. Learn more about Optimal Workshop SurveyMonkey Surveys and questionnaires are great ways to gather information, but they’re most useful when hundreds of responses can be seen at once. Enter SurveyMonkey, an online survey-creation and reporting tool, which allows people to customize and brand their own surveys and then send them out via social media, embed them into websites, or integrate with mass mailings. SurveyMonkey also allows for easy analysis and reporting when the results come in. It’s available as a free Basic version, or for a monthly fee with additional features. Learn more about SurveyMonkey UserTesting When it’s not possible to schedule a real-time test with users, UserTesting is a great way to see how people use a site. Researchers can create a series of tasks, and then receive videos from participants—either pre-chosen, or randomly selected. Researchers are able to see a video of the participant using the site, and speaking aloud to explain what they’re doing. UserTesting offers Basic and Pro options, and prices accordingly. Learn more about UserTesting UserZoom The good news is, whatever you need, UserZoom has it. Usability testing, both moderated and unmoderated, remote testing for mobile and desktop, benchmarking, card sorting, tree testing, surveys, and rankings: they’ve got it! The bad news, as can be expected with any product this robust, is that it can be overwhelming to learn, and it is expensive. Still, for organizations with the budget to handle it, UserZoom is a solid, effective choice. Learn more about UserZoom Associations and Conferences In a way, all usability conferences are design research conferences. While the other areas of UX tend to have conferences that cater to their factions, trends, or best practices, researchers are by nature generalists, who seek out ways to learn more about humans and usability. To that end, we recommend a little bit of everything, to keep researchers up to speed. UI Conference UIE puts on an annual conference, bringing together thought leaders and tried-and-true concepts for UX practitioners to learn from as much as they learn from one another. One great thing about this conference is UIE’s commitment to helping people attend, whether in person or virtually. More information available at: https://ui20.uie.com World Usability Conference At World Usability Conference, the talks cover all areas of usability, user experience, and customer experience management. People come from around the world, across all industries, to find where we have more similarities than differences. At the conference one project will also be given the World Experience Award! More information available at: http://www.worldusabilitycongress.com Design + Research + Society Held in Brighton, England, the DRS Conference responds to the questions, how can design research help frame and address the societal problems that face us?; how can design research be a creative and active force for rethinking ideas about Design?; and how can design research shape our lives in more responsible, meaningful, and open ways? By focusing on how design research impacts people in today’s society, they turn a light on the human side of our designs. More information available at: http://www.drs2016.org/programme UXPA The User Experience Professionals Association is open to everyone working in UX. They provide networking and professional development opportunities for UX practitioners around the world, by way of local chapters. If you live in a city, there’s a good chance there’s a UXPA chapter to join! Local chapters often host conferences, sponsor events, and create a network for local UX professionals. More information available at: https://uxpa.org UX Research books There are many books that focus on interviews, usability testing, A/B testing, and other specific areas of design research. These are just a smattering of books that cover the big picture. Observing the User Experience Elizabeth Goodman, Mike Kuniavsky, and Andrea Moed Who we think our users are is not always the same as who they actually are. In Observing the User Experience, the authors explain not only the difference, but why it matters, and how to find the actual users. Once we understand who our users truly are, we can design for what they need. Design Research Through Practice: From the Lab, Field, and Showroom Ilpo Koskinen, John Zimmerman, Thomas Binder, Johan Redstrom, and Stephan Wensveen Constructive design research is a contemporary research style derived from both industrial and interaction design. The book gathers information from all sorts of research experts: those in traditional lab science, social science, art, industrial design, UX and HCI, and provides teachable techniques for researchers in design. Rocket Surgery Made Easy Steve Krug The best part of Rocket Surgery Made Easy isn’t that you can finish it on a short plane ride (though you can). It isn’t the comics, or the easily understood tips (though they’re great). It’s the resources! Included in the book is a 6 page Sample Usability Test Script that walks proctors through administering a hypothetical user test, and a Usability Testing Checklist that explains what to do in the weeks, days, and hours leading up to guerilla usability tests. Understanding Your Users: A Practical Guide to User Requirements Methods, Tools, and Techniques Kathy Baxter, Catherine Courage, Kelly Caine We first recommended this book back in 2010, but now there’s a new, revised second edition, and we’re happy to recommend it again! Understanding Your Users shows real-world examples, and tried and true methods of gathering information from users. It’s great for new UX practitioners and for experts looking for a fresh opinion. Validating Product Ideas: Through Lean User Research Tomer Sharon We all want to learn what users are thinking quickly and efficiently, but it seems an impossible task. Tomer Sharon proves it isn’t, in his book, a combination of theory, explanation, and practical advice for product managers and developers who want to test their products with real users. http://dlvr.it/LGv1gP www.regulardomainname.com