Quantifying the User Experience

Quantifying the User Experience: Practical Statistics for User Research offers a practical guide for using statistics to solve quantitative problems in user research. Many designers and researchers view usability and design as qualitative activities, which do not require attention to formulas and numbers. However, usability practitioners and user researchers are increasingly expected to quantify the benefits of their efforts. The impact of good and bad designs can be quantified in terms of conversions, completion rates, completion times, perceived satisfaction, recommendations, and sales. The book discusses ways to quantify user research; summarize data and compute margins of error; determine appropriate samples sizes; standardize usability questionnaires; and settle controversies in measurement and statistics. Each chapter concludes with a list of key points and references. Most chapters also include a set of problems and answers that enable readers to test their understanding of the material. This book is a valuable resource for those engaged in measuring the behavior and attitudes of people during their interaction with interfaces.

• Provides practical guidance on solving usability testing problems with statistics for any project, including those using Six Sigma practices
• Show practitioners which test to use, why they work, best practices in application, along with easy-to-use excel formulas and web-calculators for analyzing data
• Recommends ways for practitioners to communicate results to stakeholders in plain English

• Resources and tools available at the authors’ site: http://www.measuringu.com/

 

 

Usability and user experience practitioners, software and web-development professionals, marketers, information architects, interaction designers, business analysts, market researchers, and students in these and related fields

Dedication

Acknowledgments

About the Authors

Chapter 1. Introduction and How to Use This Book

Introduction

The Organization of This Book

How to Use This Book

Key Points from the Chapter

Chapter Review Questions

References

Chapter 2. Quantifying User Research

What is User Research?

Data from User Research

Usability Testing

A/B Testing

Survey Data

Requirements Gathering

Key Points from the Chapter

References

Chapter 3. How Precise Are Our Estimates? Confidence Intervals

Introduction

Confidence Interval for a Completion Rate

Confidence Interval for Rating Scales and Other Continuous Data

Key Points from the Chapter

Chapter Review Questions

References

Chapter 4. Did We Meet or Exceed Our Goal?

Introduction

One-Tailed and Two-Tailed Tests

Comparing a Completion Rate to a Benchmark

Comparing a Satisfaction Score to a Benchmark

Comparing a Task Time to a Benchmark

Key Points from the Chapter

Chapter Review Questions

References

Chapter 5. Is There a Statistical Difference between Designs?

Introduction

Comparing Two Means (Rating Scales and Task Times)

Comparing Completion Rates, Conversion Rates, and A/B Testing

Key Points from the Chapter

Chapter Review Questions

References

Chapter 6. What Sample Sizes Do We Need?

Introduction

Estimating Values

Comparing Values

What can I Do to Control Variability?

Sample Size Estimation for Binomial Confidence Intervals

Sample Size Estimation for Chi-Square Tests (Independent Proportions)

Sample Size Estimation for McNemar Exact Tests (Matched Proportions)

Key Points from the Chapter

Chapter Review Questions

References

Chapter 7. What Sample Sizes Do We Need?

Introduction

Using a Probabilistic Model of Problem Discovery to Estimate Sample Sizes for Formative User Research

Assumptions of the Binomial Probability Model

Additional Applications of the Model

What affects the Value of p?

What Is a Reasonable Problem Discovery Goal?

Reconciling the “Magic Number 5” with “Eight Is Not Enough”

More about the Binomial Probability Formula and Its Small Sample Adjustment

Other Statistical Models for Problem Discovery

Key Points from the Chapter

Chapter Review Questions

References

Chapter 8. Standardized Usability Questionnaires

Introduction

Poststudy Questionnaires

Post-task Questionnaires

Questionnaires for Assessing Perceived Usability of Websites

Other Questionnaires of Interest

Key Points from the Chapter

Chapter Review Questions

References

Chapter 9. Six Enduring Controversies in Measurement and Statistics

Introduction

Is It Okay to Average Data from Multipoint Scales?

Do You Need to Test at Least 30 Users?

Should You Always Conduct a Two-Tailed Test?

Can You Reject the Null Hypothesis When p > 0.05?

Can You Combine Usability Metrics into Single Scores?

What If You Need to Run More Than One Test?

Key Points from the Chapter

Chapter Review Questions

References

Chapter 10. Wrapping Up

Introduction

Getting More Information

Good Luck!

Key Points from the Chapter

References

APPENDIX. A Crash Course in Fundamental Statistical Concepts

Introduction

Types Of Data

Populations and Samples

Measuring Central Tendency

Standard Deviation and Variance

The Normal Distribution

Area Under the Normal Curve

Applying the Normal Curve to User Research Data

Central Limit Theorem

Standard Error of the Mean

Margin of Error

t-Distribution

Significance Testing and p-Values

The Logic of Hypothesis Testing

Errors in Statistics

Key Points from the Appendix

Index