User Experience (UX) Research for PhDs

Simulation author – Eric Failes, PhD candidate in Psychological and Brain Sciences at Washington University in St. Louis.

Simulation vetted by the CEO of a start-up in the Greater St. Louis Area

Simulation Objective:

Conduct a User Experience Study

Associated Simulation Library:

Science Sim

Humanities Sim

Background

A user experience (UX) researcher’s role is to help a company gain a better understanding of what customers like and dislike about the company’s product. To learn about a customer’s experience with a given product, UX researchers work with customers and use different data collection methods (e.g., online surveys, interviews, beta testing, etc.) to evaluate the customers’ experiences. They then translate these experiences into actionable insights to present to the company’s management team or product developers. These insights can then be used to refine future iterations of the product to improve user satisfaction and increase sales.

This role is typically completed by a researcher or team of researchers, often with backgrounds in the behavioral sciences. UX researchers tend to have a strong background in qualitative data analysis, and are able to understand and interpret quantitative analyses.

The Process

Identify questions that you want answered about the customer’s experience.
Choose the best methods to collect data to answer your questions. The best method will depend on the questions you want to answer. UX researchers use a variety of research methods, including surveys, interviews, and observational/behavioral studies (e.g., usability testing).
Identify the participants for your research. You want the findings of your research to reflect the sentiments of all users of a given product, so it is important to choose a representative sample of users. You will want participants who have different degrees of experience with similar products to that being tested. It may be especially useful to recruit current users of a competing company’s product.
Collect data using the method(s) identified above in the chosen sample. UX researchers most often rely on behavioral studies, observing users interacting with the product to see if they have any difficulties. Then, the UX researcher will collect qualitative data to help clarify any issues that the user had while interacting with the product.
Analyze the data. Once data is collected, you must choose the right way to analyze the data to answer your questions. Qualitative data (e.g., free response questions) are not numerical in nature, so statistical analyses cannot be used. However, there are several qualitative data analysis techniques that can be used to identify themes in written or verbal responses across participants. Some UX researchers may conduct quantitative data analyses themselves, but at many companies this will be done by a separate group of data scientists; UX researchers should be able to understand and communicate the results of quantitative data analyses.
Present the results of your research. Depending on the size of the company, you may report data to a supervising UX researcher, product designers, or the company’s CEO. UX researchers should present the findings in a clear, jargon-free manner that all people present can understand. UX researchers also give actionable insights based on their findings to guide future iterations of the product. For example, if participants often had trouble finding the “quit” button on a product, this should be communicated along with ideas for how to make this process easier for users.

Resources

UX Research: What it is and Why You Should Do It (Interaction-Design blog)
Essential UX Methods (MeasuringU article)
When To Use Which UX Research Methods (NN group article)
It is also useful to learn programming languages like R (a free, open-source software) to aid in data analysis and visualization

Project SMART Goals

Self Reflection Guide

Informational Interview Guide

The Exercise

For this simulation, you are a UX researcher at a small company developing a new video game controller. You report to the design team and the company’s CEO. They are interested in determining whether users prefer their controller over those developed by other companies.

Task 1 - Determine How You Will Address the Company’s Questions About their Video Game Controller.

You might ask general questions that are informative regardless of the product (e.g., experience or aesthetics). You may collect information on more specific questions that the development team would like answered (e.g., use or functioning of different parts of the controller). In this case, it seems like both observational and subjective data could be useful. Consider what parts of the product you are interested in knowing if they like or dislike.

Task 2 - Choose the Best Methods to Collect Your Data Based on the Questions You Want to Answer

You can choose to use multiple research methods. There is rarely a single question that a company would like answered about a given product, so it may sometimes be best to use multiple methods to get both qualitative and quantitative insights. In choosing your method of study it is also important to ensure that the manner in which you are measuring outcomes is reliable and valid. See the Resources section for UX Research Methods.

For this task, use your questions from Task 1 and consider these factors as you choose your methods:

How will the participants engage with the product?
How long will the activity take place?
How will you collect the data?

Keep in mind, if you are curious about how much users are enjoying a particular product, then a simple survey may be most appropriate. Questions in surveys and interviews should be clear and easy to answer, especially if data is being collected in a setting where the researcher is unavailable to answer questions (e.g., online surveys). Questions should be posed in such a way that they are unbiased and do not compel users to respond in a certain way. Similarly, if conducting an observational study, make sure that there is a clear system for how you will be judging the participants’ interaction with the product. For example, you could use a stopwatch to determine the time it takes users to perform a particular function with the product as an objective measure of the intuitiveness of the product.

Task 3 - Determine Who Will Participate in the Study

You would want to recruit people with different degrees of familiarity with existing gaming controllers. It may be particularly useful to recruit people who currently use a competitor’s gaming controller. Call or send an email to a mix of experienced and inexperienced gamers to see who would be willing to participate in the study.

Task 4 - Collect Data

[Advanced]

This is a main role of a UX researcher. Imagine how this step would play out since testing video game controllers is outside the scope of the present exercise. How would you deliver the surveys (verbally in-person, online in-person, or online out-of-lab) and what factors might you want to consider if using observational measures (measurement tools, reliability/validity of measures)? It may also be helpful to take video recordings of participants using the new controller to see what difficulties they had.

Task 5 - Results

[Advanced]

Communicate the results of the study to the design team and the CEO. This may involve making a poster with graphs to visualize the outcomes of your study, or presenting videos of common issues that participants had.

Deliverables

Task 1

Questions the company may want answered- How much do you enjoy using this product on a scale of 1 to 10? Did you have any trouble finding a particular button when using the controller? Did the user like the feel of the controller?

Task 2

Multiple methods – Usability study and survey. If the company is interested in comparing their new controller to those made by competitors, you may wish to observe users interacting with different controllers to see which they find more intuitive, a method known as A/B testing. Therefore, to assess the controller’s ease-of-use, you could have participants play a simple video game that requires participants to quickly press the different buttons on the controller and compare the number of mistakes made using the new controller and controllers made by other companies.

Then you could conduct a survey or interview to ask participants which controller they preferred and why. Prepare for data collection by gathering the necessary materials (the new controller, examples of alternative controllers, a gaming console compatible with all controllers, and a game) and designing interview/survey questions to ask all participants.

Task 3

Call or send an email to a mix of experienced and inexperienced gamers to see who would be willing to participate in the study. You would want to recruit people with different degrees of familiarity with existing gaming controllers, especially those who currently use a competitor’s controller.

Task 4-5

Collect the data. The main role of a UX researcher is to conduct the study (though it is out of the scope of this job sim). Record their responses and compile data from all participants into a common data sheet.

Analyze the data. Observe users interacting with the controller to see what kinds of issues arise (e.g., trouble finding certain buttons, discomfort in the hand, etc.). Use qualitative data analysis techniques to determine what aspects of the new controller participants liked and which they disliked. Use statistical methods to determine whether participants made fewer errors in the video game using the new controller relative to the alternative controllers.

Communicate the results of the study to the design team and the CEO. This may involve making a poster with graphs to visualize the outcomes of your study. For example, a bar chart could be used to show differences in the number of errors made in the video game across the different controller types. Then based on the responses in the survey, communicate to the team what aspects of the new controller people liked and how it could be improved.

Additional Tasks

Writing papers for publication

Presenting research at conferences
Contributing directly to design of the product