The continued objective of the research was to obtain specific feedback about TULIP from end-users to guide future developments of delivering journal information to the desktop, to get insights on the requirements for electronic services to be attractive and valuable, both from the content provider and infrastructure provider’s side. Several areas of user behavior and perceptions were investigated:
1. Generic and basic understanding of information behavior of scientists and librarians.
2. What gets (and keeps) users interested in electronic products in general? What are the “pull” and what are the “push” factors?
pull: what is perceived added value? Why do they want desktop access?
push: what is unattractive in the current situation and can be improved on in a “full text” electronic environment?
3. Which requirements should electronic products meet to be attractive and valuable?
4. How should we bring electronic (full text) information to the desktop?
5. How should we promote?
These general research areas were translated to a number of specific topics:
Current information needs, including various professional roles which require respondents to seek information, primary information needs and sources used.
Background information including problems encountered in the daily search for information or data, changes in respondents’ method of reading or searching for information over the past year, expected changes in the future and typical pattern used to review journals.
Specific questions about electronic information sources, including the amount of time spent in front of a computer each day and how that time is allocated, the amount of time spent using WWW browsers and usage of specific databases, perceived benefits and drawbacks of electronic information, desired improvement with regard to desktop access, perceived role of the library as electronic information services become more widespread, and payment for use of electronic information sources.
Usage and adoption of the TULIP files, penetration of the potential user group.
Evaluation of TULIP including sources of awareness and information about the program, initial expectations and whether or not these expectations were met, frequency of use, reasons for use, effect of TULIP on readership of paper journals, missing items, evaluation of specific features of the program (i.e. journal coverage, search features, printing facilities etc.) and suggestions for improvement.
Reasons for not using TULIP among respondents who have either used it in the past and stopped using it or do so only occasionally or who have never tried it.
The actual research done as part of the TULIP project can be divided in two categories, quantitative and qualitative.
IV 2. Quantitative Research
IV 2.1. Research methods used
The major part of the quantitative research is based on the analysis of computer generated files in which the actions of end users are recorded (logfiles). The main elements of these logfiles are:
type (faculty, student), department and location of user;
actions like reading the abstract, viewing full text, printing etc.;
journal article, page(s);
date and time of (beginning and end of) usage, per action.
The logfile analysis is based on the actions users carry out, the most important ones being browsing of abstracts, browsing of page images (per page) and printing (one action per abstract or article printed). The relation between these three major types of actions differs from TULIP site to site because of the differences in infrastructure. As an example: at the University of California, access to the TULIP page images is through a search action in the secondary databases Inspec and Current Contents on the central MELVYL library system; the actions in MELVYL, including viewing the abstract in Inspec, are not captured in the TULIP logfile for this university. On the other hand, at the University of Michigan, the TULIP implementation itself incorporates all searches, therefore each abstract or page viewed as well as each abstract or article printed is registered as an action.
Repeat users and penetration:
The primary question to be answered was how the acceptance of TULIP at the various sites has developed, which was translated as: How many users in the primary user groups (faculty and graduate students) use TULIP repeatedly?
Information from the focus groups and interviews shows that graduate students and faculty visit the library on average every two to three months. Analysis of the logfiles showed a similar frequency emerging, therefore we have defined a repeat user as follows:
A repeat user is a user (faculty or graduate) who used TULIP in at least two (not necessarily consecutive) months within a six months period.
The degree of penetration, which tells us the relative importance of the number of repeat users, is defined as follows: the number of repeat users (faculty or graduate students), divided by the potential user group of TULIP at a site.
So the degree of penetration is dependent both on the actual number of repeat users and on the size of the potential user group. As the number of repeat users grows or declines, the penetration curve indicates the acceptance of TULIP by the potential user group.
As Materials Science is a rather interdisciplinary subject area, the potential user group not only consists of "core user groups” in the Materials Science departments. Faculty and graduate students from, for instance, the general physics and chemistry departments, can also be part of the potential user group. For calculation of the size of the potential user groups we relied on information provided by the TULIP contact persons at the various sites.
This analysis was carried out for all sites which provided usable logfiles and allowed for comparing the degree of acceptance of TULIP. This comparison led to further questions about the reasons behind a certain degree of penetration. These questions were then asked during the focus group sessions and interviews in order to generate insights into why the acceptance of TULIP differed strongly from site to site.
Further questions that were answered by logfile analysis are:
What is the share of usage of the various user groups (faculty, graduate students, staff, undergraduates, others)?
What is the relation between different types of usage, mainly browsing abstracts/page images and printing?
Eventually, four types of analysis were carried out for all sites:
an overview of the usage by type (browsing abstracts or page images/printing/other) per site per month for all users (graph & table 1); for the University of California this analysis was done separately for the large, medium and small sites;.
an overview of usage by all user types per month (faculty, graduate students, undergraduates, library staff, staff, others, and unknown) (graph & table 2);
the same analysis, but only of the usage of the core user groups: faculty and graduate students (graph & table 3);
a month by month report on the development of the number of users and repeat users for the faculty & graduate students (Graph & Table 4).
Analysis of the following logfiles has been carried out:
Carnegie Mellon University (September ’94 to October ’95)
Cornell University (October ’94 to October ’95)
Georgia Tech (September ’94 to October ’95)
MIT (July ’94 to October ’95)
University of California (January ’94 to October ’95) for all nine campuses
University of Michigan (January ’94 to October ’95) Integrated logfile-analysis of the three environments (MASC, TULIPView and WWW)
University of Tennessee.
At the moment the logfile analysis started, the October 1995 logfiles were the latest available for all universities, therefore no later logfiles were included in the analysis.
We have to repeat here that there are several different types of TULIP implementations, which has consequences for the interpretation of the graphs:
The implementations at the University of Michigan (TULIPView and WWW) are stand alone TULIP implications in which the use of the abstracts is registered, as well as the use of the page images for viewing or printing.
In the implementations at the University of California and MIT, only the viewing and printing of the page images is registered as an action. The actions of the users on the abstracts are not logged.
At Georgia Tech, only usage of the abstracts and printing are logged, as there is no viewing of page images available in this service.
For Carnegie Mellon’s system we can not differentiate between any kind of actions.
Additional quantitative research by two universities consists of “reshelving studies”, a method to calculate the usage of the paper-versions of the journals in the library by counting the number of times (unbound) journal issues are used. The question behind these studies is whether the TULIP project increased or decreased the usage of the paper versions of the journals in the project in the library. Studies of this kind were carried out by the library staff at the University of Michigan and Tennessee, results are discussed below in IV 2.2.3.
IV 2.2. Results of the logfile-analysis per TULIP-site
IV 2.2.1 Carnegie Mellon University
Figure 1: Logfile analysis: CMU total usage per month
Figure 2: Logfile analysis: CMU users and repeat users per month
Usage
Usage only exceeded 1,000 actions per month in July 1995 (which peak was mainly caused by usage of a few library staff for testing reasons), which in comparison to other sites is rather low. Important factors were:
only 1992 data was available up until early 1995;
TULIP was not actively introduced until Spring 1995;
when there was access to the images, it was with an interface previously rejected in user tests.
Printing
No data available, as printing was impossible throughout the project.
Users and repeat users
Despite the rather low usage figures more than 100 users used the system at least once, of which a high percentage returned. The number of repeat users peaked in March/April and July/August 1995. The logfiles did not differentiate usage between the various user types. The potential user group is about 650, there are between 80 and 50 repeat users, which means a penetration of between 12% in the middle of 1995 and 8% in October 1995. However, it is not clear how usage by development and library staff distorts this picture.
IV 2.2.2 Cornell University
Usage
There is no data available on the usage of the X-Windows based system which has been available to end users with limited files for about a year and a half at Cornell. This system was based on a book paradigm, developed for a book preservation project. Usage of this implementation was very low. The connection to the Web implementation at the University of Michigan is “live” since March 1995. Usage, however, does not exceed a few hundred hits, of which 80% are from one user.
Printing
No printing is registered.
Users and repeat users
In total one graduate student and 23 “unknown” users have used the system. Only two users can be classified as repeat users. A concluding remark is, therefore, that TULIP has no substantial user base at Cornell.
IV 2.2.3 Georgia Institute of Technology
Figure 1: Logfile analysis: GT usage per month
Figure 2: Logfile analysis: GT usage per usertype per month
Figure 3: Logfile analysis: GT usage of graduate and faculty per month
Figure 4: Logfile analysis: GT users and repeat users per month
Usage
At this site, users can either search the TULIP citations and abstracts file connected to the library’s OPAC, or select journal titles and browse their table of contents. Desktop viewing of articles is not possible.
Both usage-peaks and lows are more frequent here than at other TULIP sites. Remarkable is also that these fluctuations remained within certain boundaries, with peaks of around 4,000 actions per month, and lows of about 1,500 user actions per month. This pattern seemed to continue in the remaining months of 1995.
Printing
At Georgia Tech, printing was done on a central campus facility, then the printed articles were delivered to the requester using an existing campus-courier service, a nice combination of new and old technology. Compared with other sites, printing remained at a rather low level of about 100 prints per month.
Users and repeat users
In total, some 440 people used the system at least once. The number of users per month is now stable around 50 to 55 graduate students per month, and five to eight faculty. These numbers were already reached a few months after the launch of TULIP at Georgia Tech. A plausible explanation is, that Georgia Tech already had an on-line library service in place which gave access to bibliographic data. TULIP added the availability of abstracts and direct ordering of the articles, although before that, the service to deliver articles identified otherwise was already in place. TULIP was, therefore, a logical extension of this service, with better access to the bibliographic data, including abstracts, and faster delivery of the actual articles. Adapting this new service did not require much training or explanation, while offering direct benefits.
Repeat users developed rapidly among graduate students, reaching a peak in May 1995 and stabilizing since then. With 220 estimated potential users among the faculty and graduate students, Georgia Tech reached a penetration of over 50%.
IV 2.2.4 Massachusetts Institute of Technology
Figure 1: Logfile analysis: MIT usage per month
Figure 2: Logfile analysis: MIT usage per usertype per month
Figure 3: Logfile analysis: MIT usage of graduate and faculty per month
Figure 4: Logfile analysis: MIT users and repeat users per month
Usage
Usage at MIT, which was rather low in the beginning, increased gradually to a peak in April 1995, with a sharp spike in June (probably due to the fact that graduate students were being asked to participate in focus groups), and then flattened out on a lower level.
Printing
Unlike, for instance, the University of Michigan, where both browsing in the citation file and viewing/printing the page images are logged, MIT logfiles only registered actions on the image files. Selecting articles for printing, which was carried out in either a Willow or a Web table of contents browser, were not registered as actions in the logfiles. Therefore, the number of printing actions are close to the browsing actions, as all other actions concerning the citation file remain outside the logfiles.
Users and repeat users
At MIT, some 314 people viewed and/or printed TULIP images at least once. Users at MIT had the option to log in either with their user type code (faculty, graduate student, etc.) or as “unknown”; this makes it rather difficult to identify the various user groups and thus to calculate their size and growth. To obtain a good estimate of the real number of faculty and graduate students using TULIP, a proportional share of the unknown category is assumed to be faculty and graduate students, and is added on a monthly basis.
The number of users per month at MIT developed from an initial two to a peak of 27 graduate students; by October 1995, this had gone down to 12. Faculty remained stable around two users. Penetration, with a potential user group of Materials Researchers of 450, developed from 2% in November 1994, to 8-9% in September/October 1995. TULIP was also used by faculty and graduate students from nine other departments.
4.2.2.5 University of California (all campuses)
Figure 1: Logfile analysis: UC (all campuses) usage per month
Figure 1a: Logfile analysis: 3 UC sites with highest total usage per month
Figure 1b: Logfile analysis: 3 UC sites with medium usage per month
Figure 1c: Logfile analysis: 4 UC sites with low usage per month
Figure 2: Logfile analysis: UC usage per usertype per month
Figure 3: Logfile analysis: UC usage of faculty and graduates per month
Figure 4: Logfile analysis: UC users and repeat users per month (all UC sites)
Usage
If one compares the months with the highest and lowest usage in 1994 and 1995, it is clear that at the University of California the usage of TULIP only recently increased, but remains at a rather low level overall. October ’95, has the highest usage of the whole project period. This recent increase however is almost completely due to the activity of a few users at UCLA. Three campuses (Berkeley, Los Angeles and San Diego), account for about two-thirds of the total usage at the University of California. Usage is almost zero at four other campuses (Santa Barbara, Riverside, San Francisco and Santa Cruz). These campuses are not very strong in Materials Research, which is not only reflected in the low number of potential users, but also in the fact that there are few subscriptions to the paper journals at these campuses.
Printing
Because the TULIP application at the University of California is linked with the MELVYL library system, providing direct access to Inspec and Current Contents, searching and selecting relevant items is not recorded in the TULIP logfiles. Only once people decide to look at the page images, are subsequent actions logged. This situation, similar to the MIT logfiles, leads to an almost equal number of printing and browsing actions.
Users and repeat users
There have been 253 people who have used the University of California TULIP images at least once. The number of users has been declining since May 1995, and since then is lower than in the same period of 1994. The number of repeat users increased somewhat in October ’95, after remaining stable for a while, at 13 faculty and 30 graduate students, probably because of the announcement of focus group research at the University of California campuses in that month. With a total potential user group at the University of California of almost 3,400, penetration does not grow above 1-2%. However, this potential user group seems to include many “non-core” potential users. Certainly, at those sites where the usage is heaviest, penetration will be relatively higher.
IV 2.2.6 University of Michigan
Figure 1: Logfile analysis: UM usage per month
Figure 2: Logfile analysis: UM usage per usertype per month
Figure 3: Logfile analysis: UM usage of graduate and faculty and unknowns per month
Figure 4: Logfile analysis: UM users and repeat users per month
Usage
The University of Michigan’s usage statistics show that TULIP usage in 1995 is higher overall than in 1994. If one compares the month of June, which shows lowest usage for both years, usage in June ’95 is almost four times as high as in June ’94. The same holds true if one compares the months with peak usage, which in both years occurred in March. March ’95 usage is almost twice as high as in the same month of the previous year. The trend, therefore, indicates a steady increase in usage in the second year of the TULIP project at the University of Michigan. This trend is especially clear in the development of page image viewing and printing (see graph 1). This trend is probably partly influenced by the expansion of the number of TULIP journals included in the project from 43 to 82 per January 1, 1995.
Printing
Printing articles stays at a low level, but since the introduction of Web access to TULIP (in March ’95), printing has increased from 185 to 570 articles per month. As the Web version will probably be more frequently used, it can be expected that this trend will continue.
Users and repeat users
At the University of Michigan, 437 people have used the TULIPView, MASC or Web implementation at least once. Due to changes in the logfile procedures (since March ’95), the number of faculty and graduate students are no longer registered by user type. Distinguishing between faculty and graduate students was no longer possible, and for continuity reasons we have had to combine the user types in the analysis.
The number of users per month fluctuates between 50 and 80 to 90. The share of Web users is steadily increasing. The number of repeat users developed significantly, from 30 in August ’94 to 90 in October ’95. Since June ’95, we have seen a stabilization occurring. Penetration, with an estimated potential user group of 360, is 25% as of October ’95.
IV 2.2.7 University of Tennessee
Usage
The logfiles of the University of Tennessee did not meet the agreed standards and can, therefore, not be analyzed here. The number of actions however, could be counted and did not exceed an average of 100 per month by all user types. This is also confirmed by analysis carried out at the University of Tennessee, which mentions the lowest usage in December 1994 (four actions), to a maximum of 812 actions in January 1995.
Printing
The number of requests for full text articles was no more than ten throughout 1994 and 1995.
Users and repeat users
As repeat usage could not be traced, there is no information about number of repeat users or penetration. The core user group consists of 15 faculty and about 100 graduate students, of whom 38 registered for a TULIP password. Conclusion: there was only moderate usage of TULIP at the University of Tennessee.
IV 2.3. Overall results and comparison of sites
Notwithstanding the diversity of the TULIP implementations and its consequences for the logfiles, we feel that we can compare the results of the University of Michigan, the University of California and MIT, because all three logged the usage of page images for viewing on screen and printing. Although the University of Michigan logfiles have the added registration of the use of the abstracts, this has proven not to be of influence on the number of repeat users. Analysis of the logfiles showed that the same users are browsing and printing both abstracts and image files on a regular basis, and thus the number of repeat users is not influenced. This means that the results of the University of Michigan, the University of California and MIT can be compared.
The University of Michigan, which had a 25% penetration, invested a lot of time and effort in providing good and useful interfaces to the material, as well as in communicating the benefits of TULIP to the end users, training, and in the technical improvement of especially the printing facilities.
At MIT, a penetration of about 10% overall was reached, however the number of faculty using the system remained rather low. There was high usage intensity, that is, the usage of the system, as measured by the number of pages viewed and printed, clearly demonstrates a strong interest in using the service. There are questions about the percentage of repeat usage the system received, however. We attribute the lack of repeat usage to a mismatch between the content and the needs of the users rather than in dissatisfaction with how the service was implemented.
At the University of California (1-2% penetration), the technical infrastructure seems to be the major roadblock. Especially extreme difficulties with printing, sometimes excruciatingly slow response from the system, and the lack of computers running X-windows had a negative effect on repeat usage.
At Georgia Tech, more than 50% of the potential users regularly made use of the system. This service, which provided browsing of abstracts only and printing of the articles, can be considered highly succesful when compared to the other sites which had comparable systems (the University of Tennessee and Carnegie Mellon for much of the project lifetime). The reason for this high user acceptance is most probably that their TULIP implementation was a logical extension of an existing service with better access to bibliographical data (including abstracts) and fast delivery of actual articles.
IV 2.4. Results of the Reshelving Studies
IV 2.4.1 University of Michigan
The results of the reshelving statistics at the University of Michigan indicate that “TULIP publicity efforts increased awareness (...) of the TULIP journals in the first year of the experiment and thus the journals were used more in 1993 than in 1992”. The continued promotion and the publicity around focus group research in early 1995 may have caused an increase in use in 1995 over 1994. However, the library staff at the University of Michigan, knowing the pitfalls of this type of usage studies, are rather cautious to draw too strong conclusions.
IV 2.4.2 University of Tennessee
The statistics kept in the Current Periodicals room of the library indicated that usage of (unbound) TULIP journal-issues was low and did not change very much during the project. This was also confirmed by lending statistics of bound volumes and the proportion of TULIP titles in interlibrary loan requests.
IV 2.4.3 Conclusion
Promotion of electronic access to the journals makes people aware of the availability of these journals on campus, which could lead to increased usage in the library.
IV 3. Qualitative Research
IV 3.1. Research methods used
Two types of qualitative research have been used, structured individual interviews (with faculty) and focus groups (with graduate students). Although focus groups would have been preferable for both groups, structured interviews were used for faculty members, because focus groups proved not feasible for this group of respondents.
Focus groups were considered the preferred method of research, because they are an even better tool for getting an in-depth insight into the considerations, the motivations, the needs and other driving forces behind actual behavior than one-on-one interviews, because of the interaction between the participants.
The focus groups as well as the interviews were moderated by an external professional researcher and were based on an identical interview guide. The interview guide has been adapted for each site and included questions and discussion points regarding expectations, satisfaction and unfulfilled needs. In addition, preliminary conclusions from the other (quantitative) research activities have also been an important input for the formulation of the interview guides. The following qualitative research activities have been carried out:
University of Michigan (January, February 1995)
three personal pilot interviews among faculty members
one focus group with nine graduate students
ten interviews with faculty and graduate students
15 telephone interviews with faculty and students who had either only tried to use TULIP once or who heard of TULIP but never tried to use it.
MIT (July 1995)
one focus group session with nine graduate students.
University of California (October 1995) at Santa Barbara:
one focus group session with ten graduate students
five interviews with faculty.
at Berkeley:
one focus group with nine graduate students
five interviews with faculty.
at Davis:
six interviews with faculty.
All focus groups lasted approximately two hours and the personal interviews approximately one hour each. The respondents of the focus groups and personal interviews were all TULIP users. The telephone interviews at the University of Michigan (of approximately 20 minutes each), were focused on getting information about the reasons of non-use. Respondents who participated in the focus groups and personal interviews were paid an honorarium for their participation in the research.
Carnegie Mellon University and the University of Tennessee have conducted other qualitative research, the results of which can be found in their reports (appendices I and VII, respectively). To the extent that their research objectives were comparable to those of the research described above, their results are included in the analysis below.
IV 3.2. Major findings
The interview guides used for the focus groups and the one-on-one interviews have been adapted for each site in order to fit the particular implementations of TULIP and/or the specific situations on each campus (see appendix XIII for the interview guide used at MIT). Nevertheless, all interview guides covered the same issues so we were able to compare the results.
A summary of the findings is presented below. The detailed results of the qualitative research, following the sequence of the interview guides, are given in appendix XIV.
IV 3.2.1 Summary of the results
The results of the study are evaluated in the light of the five research objectives described in chapter IV 1.
Re 1. Generic and basic understanding of information behavior of scientists.
The major elements that have been learned or confirmed are:
When searching for information a rather severe selection takes place based on the title, name of the author, abstract, pictures and tables. Therefore, in-depth reading of articles is relatively rare.
Graduate students have a more active information seeking behavior than faculty members. Their focus is also broader.
Graduate students use a larger number of journals with a lower frequency and in a more ad hoc manner.
On the one hand, searching behavior of faculty is more focused, because their network of personal contacts, etc. indicates the relevant sources. On the other hand, they appreciate (more than graduate students do) the possibilities of browsing and the serendipity of paper.
Faculty seem to be more limited by time constraints, they therefore rely heavily on reference librarians and graduate students to search for information.
Faculty members in particular, also read journals at home.
Library visits are often perceived as cumbersome: distance, parking problems, issues not available, etc. On the other hand, the library has some very appreciated elements such as social contacts and “pleasure reading”. Many users have a kind of emotional relationship with the library.
Re 2. “Pull” and “push” factors regarding electronic products
The reasons for accepting or rejecting innovative products like electronic information can be divided in “pull” factors, the attractive elements of the new product, and “push” factors, those elements that are unattractive in the current situation and will therefore motivate to look for other solutions.
There is no doubt that one of the major “pull” factors for electronic products in general, and for desk top access to full text/image articles in particular, is CONVENIENCE. Respondents like the idea of being able to search and select information and to print it out without leaving their desk. Therefore, the concept of TULIP as it is perceived by the end users (which is desktop access to full text/image information), is very well received. Another “pull” factor of electronic products is the better search capabilities, mentioned in particular by those who were able to use INSPEC and Current Contents.
The major “push” factors are the inconveniences of the current situation with a reliance on paper copies which are kept at the library. This method is cumbersome, time consuming, the relevant issues are not available, etc.
TULIP has made clear that the basic pull factor for full text/image products is convenience. The usage statistics make clear that this is fairly difficult to achieve.
Re 3. Requirements for electronic products
Users seem to have the following requirements regarding functionality, ease of use and critical mass:
ease of use: as intuitive as possible, and preferably using a familiar interface
access to all information from one source
effective search capabilities
high processing speed (downloading and printing)
high publishing speed (timeliness of the information)
good image text quality
sufficient journal and time coverage
linking of information
This is the end user’s definition of convenience.
Re 4. Distribution of electronic (full text) information to the desktop
There is no doubt that electronic distribution of full text/image information is very much appreciated, under the condition that convenience (as defined above) is really improved. Georgia Tech seems to be successful in implementing a non-electronic element in the distribution (delivery of the documents by special campus courier), assuring a fast distribution of good quality print. This illustrates clearly that not the medium is important, but the final results. The experience on several other sites demonstrates that the missing of only one link in the total distribution of the information, in particular smooth printing capabilities, can seriously limit the usage of electronic information. The electronic distribution seems to be more convenient for searching, while the easy-to-browse aspect of paper makes it more convenient for serendipity. That is also the reason why reading a paper version of a journal takes more time than using the electronic version: browsing gives greater likelihood of being “sidetracked” by other information. Finally, most users prefer reading a hard copy to viewing information on the computer screen. Printing is therefore an important feature. It should be easy, fast and good quality.
Re 5. Promotion
Expectations regarding desk top access to full articles are in general very positive: once aware of the launch of a new product, many people (in particular graduate students), will give it a try. We see this relative ease of gaining interest in TULIP in the usage statistics: many new users are trying TULIP at each promotional activity.
However, it appears to be extremely difficult to influence decisions about repeat use with communication and/or promotional activities when the first experiences are not meeting the initial expectations, which is the case in many TULIP sites. In such a situation, there needs to be an on-going promotional effort to attract previous users and alert potential new users about improvements and updates made to the system.
IV 4. Conclusions
The conclusions we can draw from the user studies are the following:
The general concept underlying the TULIP project is very well received by students and faculty. This concept consists of desktop access to full text/image articles: fast and easy to search, to read, and to print.
Hardware and software are serious obstacles for convenient use of the TULIP information at most of the sites. That is certainly a reason why it appears to be difficult at most TULIP sites to increase the user base.
Most users consider the coverage (in number of titles and in time) of the journals in the TULIP project to be insufficient. This insufficient coverage (not all core journals --those not published by Elsevier-- are available in the database) requires end users to search additional information elsewhere, which is considered time-consuming and redundant, not contributing to an increased convenience. However, we have no definite answer concerning the relative importance of coverage versus convenience. If a system is able to offer satisfactory convenience (more than TULIP did), is limited coverage still a major handicap for frequent use?
For the users of TULIP there has been an accumulation of discouraging factors in many cases; limited coverage, slow response times, difficulties in accessing the system, printing problems, limited computer literacy, delays in adding new journal issues, etc. Some of these elements, the “objective” ones, can be measured, such as response time and the number of issues that are delayed. However, there are also much more subjective elements determining (repeat) usage of the TULIP implementation on a campus, for instance the initial expectations users had of the convenience and performance of TULIP. These expectations influence greatly whether a user is satisfied with the service offered or not. Because of these subjective factors, it is impossible to isolate the impact of improvements of the objective factors. In other words, we are unable to indicate to what extent the acceptance and usage of TULIP will be influenced by improving each of the above mentioned factors. Their weight in the users’ trade off of the value of TULIP compared with their existing “research tool kit”, remains unknown.
Graduate students are using TULIP more frequently than faculty. This is probably due to a more active information seeking behavior of graduate students and their broader orientation. Also, faculty need a compelling reason to change - either it is so much more convenient or so much more complete. They are both more sceptical and less willing to invest in learning to use an electronic information tool due to severe time constraints and to a lack of computer literacy.
Although the number of users might seem to be relatively low at some sites, it is difficult to evaluate these figures, because we lack reference points. For innovative products in consumer markets, a market penetration of 10 to 25% after one year is considered extremely high. However, for the type of market we are looking at here, where end users get the products and services free of charge, there are no comparable figures.
The steadily declining number of users in many cases is cause for concern, because it is a clear indication of dissatisfaction.
There is enthusiasm about the concept of desktop access to electronic information, but the end of paper products still seems to be far away. Besides some practical benefits of paper products, there also seem to be “emotional” ties with paper and the library.
Home | Elsevier Sites | Privacy Policy | Terms and Conditions | Feedback | Site Map | A Reed Elsevier Company