Usability Testing Essentials

Usability Testing Essentials provides readers with the tools and techniques needed to begin usability testing or to advance their knowledge in this area.

The book begins by presenting the essentials of usability testing, which include focusing on the user and not the product; knowing when to conduct small or large studies; and thinking of usability as hill climbing. It then reviews testing options and places usability testing into the context of a user-centered design (UCD). It goes on to discuss the planning, preparation, and implementation of a usability test. The remaining chapters cover the analysis and reporting of usability test findings, and the unique aspects of international usability testing.

This book will be useful to anyone else involved in the development or support of any type of product, such as software or web developers, engineers, interaction designers, information architects, technical communicators, visual or graphic designers, trainers, user-assistance specialists, and instructional technologists.

• Provides a comprehensive, step-by-step guide to usability testing, a crucial part of every product’s development
• The fully updated four-color edition now features important usability issues such as international testing, persona creation, remote testing, and accessibility
• Follow-up to Usability Testing and Research (9780205315192, Longman, 2001), winner of the highest-level award from the Society for Technical Communication

Usability specialists, user experience researchers, usability analysts, information architects, interaction designers, human factors engineers (analyst, specialist, manager), software/web developers, quality assurance personnel, students in industrial design, HCI, information technology, psychology, computer science, technical communication.

Foreword

Acknowledgments

About the author

Image credits and permissions

Introduction: Getting started guide

Usability is invisible

U R usability

How to use this book

Special features you can use or skip

A few words about words

But wait, there’s more on the companion website

1 Establishing the essentials

Focus on the user, not the product

Start with some essential definitions

Defining usability

Defining usability testing

Know when to conduct small studies

Know how to conduct small studies

Define the user profile

Create task-based scenarios

Use a think-aloud process

Make changes and test again

Know when to conduct large studies

Think of usability testing as hill climbing

2 Testing here, there, everywhere

Testing in a lab offers some benefits

The bare essentials for testing in a lab

Other equipment that’s nice to have

Specialized equipment you might need in certain situations

Formal labs can cost a lot, or not

Informal labs can be set up anywhere at very little cost

Field testing gets you into the world of your users

Advantages of field testing

Disadvantages of field testing

Remote testing extends your reach to your users

Moderated remote testing is synchronous

Unmoderated remote testing is asynchronous

New methods push the envelope on remote testing

Choosing the right method is a balancing act

3 Big U and little u usability

Introducing big U and little u usability

Using a user-centered design process

Opening your toolkit and seeing what’s there

Analysis tools

Development tools

Post-release tools

Choosing heuristic evaluation from the toolkit

Conducting a heuristic evaluation

Conducting a formal evaluation

Conducting an expert review

Conducting an informal evaluation

Comparing the results from heuristic evaluation and usability testing

Putting both methods together: The 1–2 punch

Cost-justifying usability

Case Study: Heuristic evaluation of Holiday Inn China website

4 Understanding users and their goals

People are goal-oriented

When people use the web, they bring their experience and expectations

People expect web objects to be in specific places

People don’t want to read — they want to act

Make a good first impression — you might not get a second chance

Generational differences matter when it comes to the web

Personas help you get to know your users

Personas are based on real information about real users

Personas are a creative activity, but don’t get carried away

Personas should be a manageable number

Personas need to be visible

Scenarios tell the story of your users’ goals

Start by knowing the difference between a task and a goal

Tell stories about your personas in a compelling way

5 Planning for usability testing

Scheduling the planning meeting

Establish test goals

Determine how to test the product

Agree on user subgroups

Determine participant incentive

Draft the screener for recruiting participants

Create scenarios based on tasks that match test goals

Determine quantitative and qualitative feedback methods

Set dates for testing and deliverables

Writing the test plan

Writing an informal test plan

Writing a formal test plan

Case Study: Test plan for Holiday Inn China website usability study

6 Preparing for usability testing

Recruiting participants

How to do the recruiting yourself

How to recruit through an agency

How to plan for no-shows

Assigning team roles and responsibilities

Developing team checklists

Writing the moderator’s script

Preparing or using other forms

Preparing a video consent form

Preparing a special consent form for testing with minors

Using a non-disclosure agreement

Preparing an observer form

Creating questionnaires

Creating a pre-test questionnaire

Creating post-task questionnaires

Creating a post-test questionnaire

Using standard post-test questionnaires

Using the SUS

Using the CSUQ

Creating or using qualitative feedback methods

Using product reaction cards

Ending with an interview

Testing the test

Conducting the walkthrough

Conducting the pilot

Case Study: Sample test materials for Holiday Inn China website usability study

7 Conducting a usability test

Setting up for testing

Meeting, greeting, briefing

Meeting and greeting the participant

Conducting the pre-test briefing

Preparing the participant to think out loud

Being an effective and unbiased moderator

Monitor your body language

Balance your praise

Ask “good” questions

Know how and when to intervene

Administer post-test feedback mechanisms

Managing variations on the theme of testing

Testing with two or more participants

Testing with two or more moderators

Testing remotely with a moderator

Providing help or customer support during testing

Logging observations

Handling observers and visitors

Observers with you and the participant

Visitors in the executive viewing room

Remote observers

Working solo

Case Study: Session log from Holiday Inn China website usability study

8 Analyzing the findings

What did we see?

Gather input from everyone

Collect the top fi ndings and surprises

Choose your organizational method

What does it mean?

Determining who should do the analysis

Collating the findings

Presenting quantitative data

Working with statistics

Analyzing questionnaires

Using qualitative feedback from the think-aloud process

Collating responses from the product reaction cards

What should we do about it?

Triangulating the data from findings

Characterizing fi ndings by scope and severity

Making recommendations

Case Study: Findings analysis from Holiday Inn China website usability study

9 Reporting the findings

Following Aristotle’s advice

Preparing the message for the medium

Writing an informal memo report

Writing a formal report

Preparing the parts of a formal report

Writing the executive summary

Organizing the rest of the report to match your audience needs

Presenting the findings

Using tables to summarize the findings

Illustrating the findings

Ordering the findings

Presenting post-task and post-test results

Presenting survey responses

Presenting SUS results

Presenting qualitative responses

Making recommendations

Presenting an oral report

Plan your presentation

Prepare video clips

Practice, practice, practice

Deliver your presentation

Know how and when to ask for questions

Advocating for more UCD

Case Study: Report of Holiday Inn China website usability study

10 International usability testing

Learning about your international users

Some international users are here

Other international users are “there”

Understanding cultural differences

Books to learn more

Articles to learn more

Applying the work of Hall and Hofstede to understand international users

Hall’s concept of high-context and low-context cultures

Hofstede’s concept of five cultural dimensions

Planning for international testing

Where to test

How to test

Structuring the test protocol

Localizing the scenarios

Localizing the questionnaires

Scheduling single sessions or co-discovery sessions

Choosing think-aloud or retrospective recall

Selecting the moderator

Anticipating other aspects of international testing

What if the participant arrives with someone else?

Should more time be set aside for meeting and greeting?

Should breaks be longer between sessions?

Can you interpret nonverbal communication cues?

Case Study: Analysis of the UPS Costa Rican website

References

Index