COVID-19 Update: We are currently shipping orders daily. However, due to transit disruptions in some geographies, deliveries may be delayed. To provide all customers with timely access to content, we are offering 50% off Science and Technology Print & eBook bundle options. Terms & conditions.
Quantifying the User Experience - 1st Edition - ISBN: 9780123849687, 9780123849694

Quantifying the User Experience

1st Edition

Practical Statistics for User Research

Authors: Jeff Sauro James Lewis
Paperback ISBN: 9780123849687
eBook ISBN: 9780123849694
Imprint: Morgan Kaufmann
Published Date: 16th March 2012
Page Count: 312


This product is currently not available for sale.

Secure Checkout

Personal information is secured with SSL technology.

Free Shipping

Free global shipping
No minimum order.


Quantifying the User Experience: Practical Statistics for User Research offers a practical guide for using statistics to solve quantitative problems in user research. Many designers and researchers view usability and design as qualitative activities, which do not require attention to formulas and numbers. However, usability practitioners and user researchers are increasingly expected to quantify the benefits of their efforts. The impact of good and bad designs can be quantified in terms of conversions, completion rates, completion times, perceived satisfaction, recommendations, and sales. The book discusses ways to quantify user research; summarize data and compute margins of error; determine appropriate samples sizes; standardize usability questionnaires; and settle controversies in measurement and statistics. Each chapter concludes with a list of key points and references. Most chapters also include a set of problems and answers that enable readers to test their understanding of the material. This book is a valuable resource for those engaged in measuring the behavior and attitudes of people during their interaction with interfaces.

Key Features

  • Provides practical guidance on solving usability testing problems with statistics for any project, including those using Six Sigma practices
  • Show practitioners which test to use, why they work, best practices in application, along with easy-to-use excel formulas and web-calculators for analyzing data
  • Recommends ways for practitioners to communicate results to stakeholders in plain English
  • Resources and tools available at the authors’ site:




Usability and user experience practitioners, software and web-development professionals, marketers, information architects, interaction designers, business analysts, market researchers, and students in these and related fields

Table of Contents



About the Authors

Chapter 1. Introduction and How to Use This Book


The Organization of This Book

How to Use This Book

Key Points from the Chapter

Chapter Review Questions


Chapter 2. Quantifying User Research

What is User Research?

Data from User Research

Usability Testing

A/B Testing

Survey Data

Requirements Gathering

Key Points from the Chapter


Chapter 3. How Precise Are Our Estimates? Confidence Intervals


Confidence Interval for a Completion Rate

Confidence Interval for Rating Scales and Other Continuous Data

Key Points from the Chapter

Chapter Review Questions


Chapter 4. Did We Meet or Exceed Our Goal?


One-Tailed and Two-Tailed Tests

Comparing a Completion Rate to a Benchmark

Comparing a Satisfaction Score to a Benchmark

Comparing a Task Time to a Benchmark

Key Points from the Chapter

Chapter Review Questions


Chapter 5. Is There a Statistical Difference between Designs?


Comparing Two Means (Rating Scales and Task Times)

Comparing Completion Rates, Conversion Rates, and A/B Testing

Key Points from the Chapter

Chapter Review Questions


Chapter 6. What Sample Sizes Do We Need?


Estimating Values

Comparing Values

What can I Do to Control Variability?

Sample Size Estimation for Binomial Confidence Intervals

Sample Size Estimation for Chi-Square Tests (Independent Proportions)

Sample Size Estimation for McNemar Exact Tests (Matched Proportions)

Key Points from the Chapter

Chapter Review Questions


Chapter 7. What Sample Sizes Do We Need?


Using a Probabilistic Model of Problem Discovery to Estimate Sample Sizes for Formative User Research

Assumptions of the Binomial Probability Model

Additional Applications of the Model

What affects the Value of p?

What Is a Reasonable Problem Discovery Goal?

Reconciling the “Magic Number 5” with “Eight Is Not Enough”

More about the Binomial Probability Formula and Its Small Sample Adjustment

Other Statistical Models for Problem Discovery

Key Points from the Chapter

Chapter Review Questions


Chapter 8. Standardized Usability Questionnaires


Poststudy Questionnaires

Post-task Questionnaires

Questionnaires for Assessing Perceived Usability of Websites

Other Questionnaires of Interest

Key Points from the Chapter

Chapter Review Questions


Chapter 9. Six Enduring Controversies in Measurement and Statistics


Is It Okay to Average Data from Multipoint Scales?

Do You Need to Test at Least 30 Users?

Should You Always Conduct a Two-Tailed Test?

Can You Reject the Null Hypothesis When p > 0.05?

Can You Combine Usability Metrics into Single Scores?

What If You Need to Run More Than One Test?

Key Points from the Chapter

Chapter Review Questions


Chapter 10. Wrapping Up


Getting More Information

Good Luck!

Key Points from the Chapter


APPENDIX. A Crash Course in Fundamental Statistical Concepts


Types Of Data

Populations and Samples

Measuring Central Tendency

Standard Deviation and Variance

The Normal Distribution

Area Under the Normal Curve

Applying the Normal Curve to User Research Data

Central Limit Theorem

Standard Error of the Mean

Margin of Error


Significance Testing and p-Values

The Logic of Hypothesis Testing

Errors in Statistics

Key Points from the Appendix



No. of pages:
© Morgan Kaufmann 2012
16th March 2012
Morgan Kaufmann
Paperback ISBN:
eBook ISBN:

About the Authors

Jeff Sauro

Dr. Jeff Sauro is a six-sigma trained statistical analyst and founding principal of MeasuringU, a customer experience research firm based in Denver. For over fifteen years he’s been conducting usability and statistical analysis for companies such as Google, eBay, Walmart, Autodesk, Lenovo and Drobox or working for companies such as Oracle, Intuit and General Electric.

Jeff has published over twenty peer-reviewed research articles and five books, including Customer Analytics for Dummies. He publishes a weekly article on user experience and measurement online at

Jeff received his Ph.D in Research Methods and Statistics from the University of Denver, his Masters in Learning, Design and Technology from Stanford University, and B.S. in Information Management & Technology and B.S. in Television, Radio and Film from Syracuse University. He lives with his wife and three children in Denver, CO.

Affiliations and Expertise

Usability Metrics and Statistical Analyst, Measuring Usability LLC, CO, USA

James Lewis

Dr. James R. (Jim) Lewis is a senior human factors engineer (at IBM since 1981) with a current focus on the measurement and evaluation of the user experience. He is a Certified Human Factors Professional with a Ph.D. in Experimental Psychology (Psycholinguistics), an M.A. in Engineering Psychology, and an M.M. in Music Theory and Composition. Jim is an internationally recognized expert in usability testing and measurement, contributing (by invitation) the chapter on usability testing for the 3rd and 4th editions of the Handbook of Human Factors and Ergonomics, presenting tutorials on usability testing and metrics at various professional conferences, and serving as the keynote speaker at HCII 2014. He was the lead interaction designer for the product now regarded as the first smart phone, the Simon, and is the author of Practical Speech User Interface Design.

Jim is an IBM Master Inventor Emeritus with 88 patents issued to date by the US Patent Office. He serves on the editorial board of the International Journal of Human-Computer Interaction, is co-editor in chief of the Journal of Usability Studies, and is on the scientific advisory board of the Center for Research and Education on Aging and Technology Enhancement (CREATE). He is a member of the Usability Professionals Association (UPA), the Human Factors and Ergonomics Society (HFES), the ACM Special Interest Group in Computer-Human Interaction (SIGCHI), past-president of the Association for Voice Interaction Design (AVIxD), and is a 5th degree black belt and certified instructor with the American Taekwondo Association (ATA).

Affiliations and Expertise

Senior Human Factors Engineer, IBM, FL, USA


"...the definitive book on inferential statistics for usability researchers. The authors present the equations, discussions, and examples for and in the context of usability studies, primarily usability testing." --Technical Communication

"Quantifying the User Experience will make a terrific textbook for any series of UX research courses…I highly recommend this book to anyone who wants to integrate quantitative data into their UX practice."--Technical Communication, May 2013
"…as a whole, it provides a pragmatic approach to quantifying UX, without oversimplifying or claiming too much. It delivers what it promises. This book is valuable for both practitioners and students, in virtually any discipline. It can help psychologists transfer their statistical knowledge to UX practice, practitioners quickly assess their envisioned design and analysis, engineers demystify UX, and students appreciate UX’s merits.", March 19, 2013
"The most unique contributions of this book are the logic and practicality used to describe the appropriate application of those measures…Sauro and Lewis strike a perfect balance between the complexity of statistical theory and the simplicity of applying statistics practically. Whether you wish to delve deeper into the enduring controversies in statistics, or simply wish to understand the difference between a t-test and Chi-square, you will find your answer in this book. Quantifying the User Experience is an invaluable resource for those who are conducting user research in industry."--
User Experience, Vol. 13, Issue 1, 1st Quarter
"Written in a conversational style for those who measure behavior and attitudes of people as they interact with technology interfaces, this guide walks readers through common questions and problems encountered when conducting, analyzing, and reporting on user research projects using statistics, such as problems related to estimates and confidence intervals, sample sizes, and standardized usability questionnaires. For readers with varied backgrounds in statistics, the book includes discussion of concepts as necessary and gives examples from real user research studies. The book begins with a background chapter overviewing common ways to quantify user research and a review of fundamental statistical concepts. The material provides enough detail in its formulas and examples to let readers do all computations in Excel, and a website offers an Excel calculator for purchase created by the authors, which performs all the computations covered in the book. An appendix offers a crash course on fundamental statistical concepts."--Reference and Research Book News, August 2012, page 186-7

Ratings and Reviews