Beyond the Usability Lab

1st Edition

Conducting Large-scale Online User Experience Studies

Print ISBN: 9780123748928
eBook ISBN: 9780080953854
Imprint: Morgan Kaufmann
Published Date: 12th January 2010
Page Count: 328
51.95 + applicable tax
40.95 + applicable tax
32.99 + applicable tax
53.95 + applicable tax
Compatible Not compatible
VitalSource PC, Mac, iPhone & iPad Amazon Kindle eReader
ePub & PDF Apple & PC desktop. Mobile devices (Apple & Android) Amazon Kindle eReader
Mobi Amazon Kindle eReader Anything else

Institutional Access


Usability testing and user experience research typically take place in a controlled lab with small groups. While this type of testing is essential to user experience design, more companies are also looking to test large sample sizes to be able compare data according to specific user populations and see how their experiences differ across user groups. But few usability professionals have experience in setting up these studies, analyzing the data, and presenting it in effective ways.  Online usability testing offers the solution by allowing testers to elicit feedback simultaneously from 1,000s of users. Beyond the Usability Lab offers tried and tested methodologies for conducting online usability studies. It gives practitioners the guidance they need to collect a wealth of data through cost-effective, efficient, and reliable practices. The reader will develop a solid understanding of the capabilities of online usability testing, when it’s appropriate to use and not use, and will learn about the various types of online usability testing techniques.

Key Features

    The first guide for conducting large-scale user experience research using the internet
    Presents how-to conduct online tests with 1000s of participants – from start to finish
    *Outlines essential tips for online studies to ensure cost-efficient and reliable results


    This book is written for usability specialists, user experience researchers, market researchers, information architects, interaction designers, business analysts, and managers who are looking to learn about the capabilities of, and gain experience with, online usability testing.

    Table of Contents

    1. Introduction to online usability methods
    a. What is online usability, and how it differs from traditional usability methods
    b. Examples of different types of online usability studies
    c. Pros and cons of online and non-online methods
    d. When to use (and not use) online methods
    e. Combining online studies with lab testing

    Chapter 1 provides an overview to online usability testing. Special attention will be paid to how it differs from traditional usability methods (including remote testing). There will be an in-depth discussion of the pros and cons of online testing, and when to use and not use online methods. We will provide real-world examples to highlight the value of this method. We will also discuss ways to complement traditional usability testing with online testing. Our intention is that the reader will be in the position to determine if an online usability study is appropriate for their organization.

    2. Planning your study
    a. Study goals
    b. Budgets and timeline
    c. Technology options
    d. Participant recruiting and panels
    e. Sample size
    f. Panel options
    g. Sampling strategy
    h. Study duration
    i. Participant compensation

    Chapter 2 focuses on all the activities and decisions that need to be made place prior to actually putting the survey together. The first three activities (goals, budgets/timelines, and technology options) are all essential to accurately scope an online study. The next part of this chapter focuses on finding the right number of targeted participants. This includes a discussion of research panels, sample size determination, and sampling strategies. The chapter will conclude with a discussion of estimating study duration and participant compensation.

    3. Designing your study
    a. Introducing the survey
    b. Screener questions
    c. Starter questions
    d. Constructing Tasks
    e. Post-task questions and metrics
    f. P


    No. of pages:
    © Morgan Kaufmann 2010
    Morgan Kaufmann
    eBook ISBN:
    Paperback ISBN:


    "If you need to reach a large audience using online tools, this book will guide you, inform you, and help you with practical tips from a team that has a wealth of practical and academic experience. They explain how to plan, organise, conduct, analyse, and present your study, and they aren’t afraid to tackle tricky topics like dealing with possibly fraudulent participants. Highly recommended."--Caroline Jarrett, Effortmark, Ltd, author of Forms that Work

    "Beyond the Usability Lab will become the gold standard for how to conduct online usability studies.  Albert, Tedesco and Tullis masterfully combine strategic design related issues with tactical and practical step-by-step instructions on how to implement an online usability study."--Jim Berling, Managing Director, Burke Institute

    "Beyond the Usability Lab is intended to help you move your usability testing out of the lab environment and into the online world as a cost-effective way to reach more participants. Albert and colleagues write from their own experience conducting online studies and describe situations when online studies are most and least effective, employing real-world examples with abundant black-and-white screen shots and other graphics. A quest to share knowledge seems to be a driving force behind the book, and I am particularly impressed with how often the authors recommend other, potentially rival texts as references for additional information…. Beyond the Usability Lab also has a companion Web site,, to which you’re often referred for support material. Though simple in design, the site is a helpful auxiliary source."--Devor Barton, Technical Communication, Vol 58, Number 1, February 2011

    "The book is well-structured for the practitioner…. It is a real pleasure to encounter a book that not only takes the reader  on a journey through the rich possi