fbpx
Uxlicious logo

Usability test checklist for mobile apps & websites + how to do it [Free Download]

December 28, 2022 „ÉĽ8 min read

What is a usability test?

A usability test entails having representative users interact with a website, mobile app, or other product and service for research and evaluation purposes. During a usability test, participants typically perform tasks while observers record the whole process, mainly their behavior and reactions to the products. 

As a popular UX research method, usability testing aims to identify any usability issues in a product or service, collect qualitative and quantitative data, and evaluate user satisfaction. Successful usability testing will eventually help the product design and development team to build a better product with joyful experiences.

How to conduct usability tests for mobile apps and websites?

Each usability test has unique settings and insights depending on the specific objective and the project constraints. But at the same time, they all follow the basic steps and pre-requisitions. If you kickstart a usability test without fulfilling the requirements and completing the necessary preparation, you will not be able to get the high return you are looking for, and even worse, your research insights may cause you to take the wrong action, which can lead to a negative impact on your product.

To help you get meaningful insights from your usability test for mobile and website, UXlicious have prepared two downloadable usability test checklists of activities to consider when planning and conducting your research.

Usability test for mobile apps

7 key activities to ensure a successful mobile usability test

1) Get clarity on the test goals

At the beginning of the mobile usability test, you shall meet the key stakeholders to determine what they want to learn about the users, the KPI they wish to improve, and identify the questions and concerns. Usability tests are best to gather behavioral data and answer design-related questions (e.g., can users navigate quickly to the section they want? Can they complete a task successfully?). If your goal is to collect attitudinal feedback, consider alternative research methods better suited for those purposes. Learn more about how to determine which UX research method to choose.

2) Decide on the test setting

Before starting the mobile usability test, you need to think about the following: 

  • Will the test be conducted in a lab or field environment?¬†Most face-to-face usability tests are conducted in-house in a lab setting for convenience. However, if the users’ actual environment is critical to getting the real insight or it is too difficult for them to travel to your facility, you should consider traveling to their locations.¬†
  • Is the test going to be moderated or unmoderated?¬†Unmoderated studies are usually cheaper and quicker to complete. It also allows you to get better access to testers that can be hard to recruit (e.g., in a different time zone). In contrast, moderated studies will enable you to probe and ask for clarification and thus can provide richer design insights.¬†
  • Is the test going to be face-to-face or remote?¬†Remote tests are becoming popular because of no travel budget, more flexibility for testers and researchers, and automated recording and analytics tools. But you lose many subtle cues, such as body language, which can only be detected when you are in the same room as the testers.

 

3) Determine who are the right participants

Not all users are the same, so you want to ensure you get representative study participants. The best insights are obtained from gathering feedback from real users. Before starting the recruitment, you would like to find out if you have any specific requirements for the test participants.

  • Demographics and profession:¬†Are you looking for testers from a specific age, gender, marital status, or work in a specific industry or position? Is your product designed for targeted persona(s), and do you want to get testers that match these criteria?
  • Behavioral traits:¬†Are you looking for people who demonstrate certain behavior (e.g., cook regularly, go to the gym at least 3 times a week, play mobile games for at least 5 hours a week), or lack a specific behavior (e.g., have never purchased a property, have never had a pet)?¬†
  • Attitude:¬†Are you looking for testers who hold a certain attitude on a subject or topic (e.g., people who are against abortion, people who prefer ESG investment products)?¬†

Using a filtering questionnaire, you can collect information to identify the right participants for your mobile usability test. Sometimes, you need validation questions to reduce communication bias.

usability test

4) Identify the right tasks that match the usability test goals

In mobile usability testing, researchers ask users to complete activities while interacting with a prototype or the actual product interface. The tasks written in the form of scenarios can be general (e.g., have a look at the homepage of a website and tell me what you see) or specific (e.g., can you find and purchase a red dress, price between GBP20 and GBP50, when you have completed the task, tell me how you think about it). In general, the tasks come in two categories: 

  • Exploratory tasks: these open-ended tasks answer board goals and do not usually have a correct answer. They are best used to understand the product information architecture, navigation, and other more visually related designs.¬†
  • Specific tasks:¬†these tasks have a definite user flow and clear, correct answers or endpoints. They are mostly used to test the key functionalities (aligned with the user goals) on a particular user journey, and you can obtain actionable insights to improve the product design.¬†

 

5) Decide what metrics to collect

In a mobile usability test, usability metrics allow you to objectively evaluate a task in a quantitative study and compare the performance between you and your competitors’ experience. Some commonly used metrics are “time on task”, “success rate”, “error rate” and you can find out how to use these metrics in usability tasks that focus on specific tasks.

6) Do a dry run for the test

A pilot study is essential to test the quality of the task instruction, anticipate the number of tasks you can cover in each session, refine your recruitment criteria and determine the order of the presentation. You can catch the problems that may jeopardize the success of a mobile usability test.

7) Create a test plan that works

Once you have figured out how you will conduct the usability test for your mobile app, document the approach and activities and share it with your team. The document doesn’t need to be lengthy but should contain the following key information:¬†

  • Name of the product and reference info
  • The study objectives
  • The study logistics (e.g., date and time, locations, format, the device needed)¬†
  • Tester profile criteria and reference (e.g., filtering questionnaire link)
  • List of tasks to be tested
  • Metrics and questions to be asked¬†

Mobile usability test checklist - Download it for free!

UXlicious has created a usability test checklist for mobile apps. This checklist offers a framework for a mobile usability test and a comprehensive list of evaluation points to consider based on the areas of the mobile application you are assessing. Some considerations and items might or might not apply to a given usability test, depending on the participants, the product, and other parameters. You can select the relevant questions based on your goal, what tasks you are evaluating, and the key metrics you want to focus on.

Usability test for websites

8 key activities to ensure a successful website usability test

1) Understand the why of the test

A website usability test is best used to collect actionable insights on a design-related question, but if you are exploring a new product idea or identifying the key value proposition that your product marketing shall focus on, then a desirability test or user interview may be a better methodology to use. Making sure the objective matches the methodology we use ensures our insights best fit the purpose.

UX researchers use a wide spectrum of UX research methods based on the objective. Stand down with your key stakeholders to understand what they want to find out from the test, what is not working, and what areas/metrics they want to improve. You can use Hypothesis Mapping to prioritize the questions to focus your test on getting high-quality feedback for the most critical questions.

It often happens that your team wants to get too much out of the test, but over-committing to an over-ambitious objective would either mean you do not get deep insights from any of the questions or that your test is too long for users to complete and you end up rushing through the test. 3-5 primary test objective is often a good amount for a 45-60 minutes usability test.

2) Determine the right participant profile and number

You need representative participants to get the right feedback from real users. There are a few steps you can use to consider whom to recruit: 

  • Is it an existing product and service?¬†For products and services that already have a user base, tap into them as users for the test. If you have an existing user database, you can communicate directly with them after obtaining the appropriate compliance approval.¬†
  • Do the users need to fulfill any behavioral traits?¬†Are you looking for people who are currently or previously engaged in certain activities (e.g., play computer games at least 5 hours a week, use a laptop to create presentations regularly) or have never done so? Or are you looking for people using competitor products to complete the same task?¬†
  • Do the users need to fall into certain demographics?¬†Are you looking for people of a particular age range, gender, profession, family status, language skills, and migration history? The participant profile affects how you want to recruit users. For example, if you tap into the community formed based on their behavior, you may want to turn to respective organizations, forums, social groups, etc., which gives you direct access to the target users. Sometimes the users are not easily accessible to you, and you may want to use professional user recruiting platforms (e.g., UXarmy, UserTesting) or companies to help.¬†
  • For qualitative studies, 5 participants will usually give you the best return on investment.¬†If your research involves more than one target user group, you can multiply the number of users accordingly, depending on the experience level and attitudinal overlap.¬†
  • Quantitative studies and eye-tracking require a larger sample size to obtain meaningful conclusions. You may need at least 20‚Äď30 participants in each target user group.

3) Determine the test format

There are 3 questions you need to answer when determining the format of the website usability test: 

  • Do you need to conduct the test in the users’ natural environment in which the product or service will be used?¬†A website usability test can be conducted in a usability lab or in the field, the former is usually preferred because it is easier to control the environment and set up the test, but when the users are not easily accessible in the lab environment, or the test result could benefit from having it done in the natural environment, you will find the travel time and additional effort worthwhile.¬†
  • Do you need to moderate the test?¬†Moderated tests provide richer design insights and opportunities to ask for clarification. They are also a better choice for collecting open-ended comments from the participants. At the same time, unmoderated studies are cheaper and quicker and may provide better access to hard-to-recruit participants.¬†
  • Do you need to see the users in person?¬†You will benefit from being able to detect subtle cues, such as body language in a face-to-face setting. So you should always choose to see the users in person if possible. But sometimes, this is not possible, for instance, when you are not in the exact location, when you have budget constraints, etc.¬†
usability test4

4) Understand the timeline

A website usability test usually happens in a product development cycle, and its input and output are closely connected to the overall product iteration process. You need to understand when your deliverables are needed and work backward to plan for your usability test. If required, you may also need to identify additional supporting team members or testing tools to speed up the process.

Once you have figured out all the dependencies and the deliverable needs, document your approach in a test plan and share it with the team. You and all involved in the test would need to work again on the time plan to make sure things are delivered on time and in good quality. The document does not need to be lengthy, but it shall include the following components:

  1. Name of the product or service to be tested
  2. Test objectives (primary and secondary)
  3. Test logistics (date and time, location, format, devices and tools used) 
  4. Participant profile information 
  5. Test tasks, metrics, and questions 
  6. Supporting team members and their roles (if applicable) 
  7. Test project plan (activities, date and time, dependencies) 
  8. Reference (links to the recruit fitting questionnaire, prototype URL, login credentials, etc.) 

5) Write solid test scenarios

For a test focusing on behavior, it is crucial to ensure the test task scenarios are clear, achievable, and aligned with the test objective. There are two types of tasks: exploratory tasks, which are open-ended tasks that answer broad goals and may or may not have a correct answer/ metrics (e.g., You are interested in booking a hotel for your parents. 

See if the site offers anything that you might suit your needs) and specific tasks which focus on a particular user journey and usually have a distinct right or wrong answer (e.g., find out the customer service contact email on this website) 

 

6) Refine your test with a dry run

Running a pilot study can help you fine-tune the wording, figure out the number of tasks you can give in a usability test session, and determine the best sequence to present. It can also help you refine your recruiting criteria to ensure you are testing with the right participants. And pilot studies are even more critical when conducting online unmoderated studies because you are not present to offer any clarification or make corrections if study participants misinterpret the instructions or task descriptions.

7) Determine the test metrics

In quantitative usability studies, measuring UX metrics helps you to get the objective result of the task you are analyzing, see how you perform among your competitor experience, and identify the most important areas of improvement. Some common usability metrics are time on task, satisfaction ratings, success rate, and error rate.

If you decide to collect subjective measurements (e.g., ease of use or satisfaction questions about the task, satisfaction about the system), you should decide when you will give the questionnaires: after each task, at the end of the session, or both.

8) Determine the result analysis format

A website usability test first serves the product and business team to work out the best iteration of the product and service, a well-formatted website usability test report makes it easy to identify the most critical issues and areas and what had gone wrong. You should discuss with the team and agree on the best format to present the result.

Website usability test checklist - Download it for free

There are a few useful details that can help you run a smooth website usability test. UXlicious have consolidated them into a checklist. You can apply it to your website usability test according to the stage where your test is.



FAQ usability test for websites and mobile apps

A usability test entails having representative users interact with a website, mobile app, or other product and service for research and evaluation purposes. During a usability test, participants typically perform tasks while observers record the whole process, mainly their behavior and reactions to the products. 

As a popular UX research method, usability testing aims to identify any usability issues in a product or service, collect qualitative and quantitative data, and evaluate user satisfaction. Successful usability testing will eventually help the product design and development team to build a better product with joyful experiences.

In a usability test, usability metrics allow you to objectively evaluate a task in a quantitative study and compare the performance between you and your competitors' experience. Some commonly used metrics are "time on task", "success rate", "error rate" and you can find out how to use these metrics in usability tasks that focus on specific tasks.

Get in touch

Whether you want to learn about UX Design or need product design services, feel free to get in touch. We love a good challenge.

Get in touch