|
Tenopir: Top Tips on User Surveys
 |
| Carol Tenopir |
Professor Carol Tenopir, with the University of Tennessee’s School of Information Sciences, is a leading figure in end-user research and has published books on the topic. Elsevier Account Development and Channel Marketing Manager Alex Lankester recently asked for Professor Tenopir’s thoughts on how libraries can most effectively implement end-user surveys.
Library Connect: Why are user surveys important?
Carol Tenopir: Usage reports are helpful for understanding use of your e-resources, but they don't give you all of the information about your users that you need. Surveys can provide additional information such as the use of both print and electronic resources, as well as the use of alternatives to library collections, and demographic data. We have observed through surveys, for example, that articles read from library collections are judged by users to be more valuable to research than those from other sources and that medical faculty, on average, read two to three times more journal articles than humanities or engineering faculty.
LC: How frequently should libraries conduct surveys and how long should a survey run for?
Tenopir: Surveys are time-consuming for respondents and librarians, so I recommend using a survey to supplement usage reports only every three to five years. Regarding the actual running time of a survey, I suggest a maximum of six weeks. The large majority of responses occur in the first week. After that the response rate decreases dramatically. Reminders do help but you can only send so many of these.
LC: How should people manage survey logistics?
Tenopir: Web-based surveys are definitely less expensive and easier to run, but response rates are invariably lower than with paper surveys. We usually start with an email message, including a link to a Web questionnaire, which we accompany with a PDF version for those who would rather print and send the survey back to us. For some groups, paper should come first, followed by email.
LC: How can we ensure a reasonable response rate to a survey?
Tenopir: Survey fatigue is now common and response rates are noticeably lower than in the past. However some things do ensure a higher response such as offering Web and paper options, sending the email (with a link) from a recognized name, restricting the survey length to 10-12 minutes, and offering an incentive such as a chance to win a prize. Faculty tend to be better responders than students, and you need to consider other ways of reaching students such as passing out surveys in seminars or lectures.
LC: What are key types of questions to include in a survey and what percent should be qualitative versus quantitative?
Tenopir: With surveys there should be several open-ended qualitative questions, but most should be quantifiable. Several types of typical survey questions are: demographic, so you can understand your specific users better; recollection, but only for a short time period (e.g., how many articles have you read in the last four weeks?); opinion and preferences; and critical incident (focusing on specific incidents, so outcomes, value and other details can be obtained).
LC: Can you expand on “critical incident”?
Tenopir: Donald W. King and I use a variation of critical incident called “the last incident of reading.” We focus on this because people remember a recent reading more accurately than general recollections and this allows us to ask very specific questions about that reading such as how an article was located, the format of reading, the time of reading, and the purpose and value of reading. The last incident gives us a random sample of readings rather than just a random sample of respondents.
LC: You once wrote, “A survey is only as good as the wording.” What do you advise regarding the format of questions in surveys?
Tenopir: It is important to define terms so respondents have clear understanding of what is being asked. For example, we define reading as “going beyond the table of contents, title and author into the body of an article.” It is also advisable to run a small pre-test to make sure respondents understand your wording and meaning. Likewise to ensure they are sensible, you should read questions aloud to other researchers. Finally be careful with complex, multi-part questions and put questions in logical order.
LC: To be sufficiently representative, how comprehensive in its coverage of different groups (subject or research level) should a survey be?
Tenopir: There are big differences among subject disciplines, so these should be segmented. Sometimes the response rate from a particular group will be too small so you have to either omit such a group or post-categorize into broader groups such as "humanities" rather than history, arts or literature.
LC: How can we avoid misinterpreting survey results?
Tenopir: You shouldn’t over-conclude or try to answer questions not in your survey. Likewise be careful of generalizing if your response rate is low or your population is unique, and ensure you use only appropriate statistical tests. Sometimes just straight reporting of numbers, means, medians, modes and standard deviations is sufficient and there is no need for elaboration.
|
