A Model for Journal Selection

in a Multiprogram National Laboratory Library

William F. Myers, A. A. S., B. A., M. S. L. S.

Chemistry and Physics Librarian

Deborah J. Cole, B. A., M. S. L. S.

Biosciences Librarian

Oak Ridge National Laboratory

Oak Ridge, Tennessee

Oak Ridge National Laboratory (ORNL) conducts basic and applied research and development to create scientific knowledge and technological solutions that strengthen the nation's leadership in key areas of science; increase the availability of clean, abundant energy; restore and protect the environment; and contribute to national security. ORNL also performs other work for the Department of Energy including isotope production, information management, and technical program management. As one of the nation’s premier research organizations focusing on a wide range of science and engineering disciplines, ORNL provides research and technical assistance to other organizations in the private and public sectors.

ORNL differs significantly from most industrial research institutions in that it is focused on a wider range of science and technology than is typical in industry. Unlike most universities, there is virtually no research at ORNL in the humanities and liberal arts other than a group that is active in the economics of developing countries. The Laboratory's structure consists of sixteen major divisions separated into 120 research groups, each focusing on a general research area such as geochemistry, electron microscopy, carbon materials, protein structure, or mammalian genetics.

The library at ORNL has active subscriptions to more than 1000 unique journal titles, including print, electronic, and print-with-electronic formats, as well as an extensive collection of books, microforms, technical reports, and other documents. The library serves a permanent total staff of 3800, including 1500 research scientists and engineers, as well as an additional 3000 guest researchers each year. The library is responsible for supporting a wide variety of research efforts conducted in ORNL's major program disciplines in the physical and chemical sciences, materials science, energy and environmental sciences, biosciences, and high-performance computing. The High-Flux Isotope Reactor and the Spallation Neutron Source currently under construction at Oak Ridge are responsible for expanding research opportunities in the neutron sciences.

Because the library at ORNL is supported largely by overhead funding, there is competition with several other functions for limited resources. Thus, the library budget is inherently not capable of adjusting easily to the high inflation rates in the costs of library materials, especially journal subscriptions. In fact, throughout the 1990s and into the current decade, the budgets for all library operations and collections have either remained constant or fallen. Since 1991 the effective loss of funding has required reductions in the size of the library staff from 42 to 16 professionals and paraprofessionals. The following table illustrates the trend for the journal collection alone.

Table: Journal Subscriptions 1991-2002

	1991	1992	1993	1994	1995	1996	1997	1998	1999	2000	2001	2002
Journal Subscriptions	2672	2668	2640	2247	1630	1236	1287	1127	974	1030	940	1027
Journal Budget ($) (x 1,000)	965	1,048	1,213	1,102	1,007	1,061	1,212	1,274	1,183	1,350	1,361	1,506

To accomplish the reductions in journal subscriptions, the librarians at ORNL have in recent years used a system of weighted criteria for journal selection and deselection. These criteria have been applied to the bulk of the journals in our collections, but not to certain periodicals that we have designated as "reference" titles, including the Wall Street Journal, New York Times, Washington Post, local newspapers, and trade and professional magazines such as Chemical & Engineering News, New Scientist, Nuclear News, and Water & Waste Treatment. These are periodicals that researchers and librarians at ORNL frequently use, but rarely cite in scholarly papers or technical reports.

Each year the librarians compile information to use in journal evaluation. In the early 1990s we asked the researchers to identify the "core" journals in the collection. We found that they often included titles we did not have, so in more recent years we have asked researchers to name the ten journals most critical to their current and anticipated research, whether or not the titles were already part of the library's collection. We did not ask the respondents to rank, nor did we try to estimate the relative importance of, the titles listed. We simply counted the number of times a title was recommended and ranked the results. The reference librarians used e-mail, paper flyers, announcements in the laboratory's daily electronic bulletin board, and personal contacts to publicize the survey, which could be filled out using paper or electronic forms available on the library's web site. The survey requested sufficient identification to allow us to eliminate multiple responses from one person, but some analysis was still required as part of the collation process. For instance, some respondents named fewer than ten favorite titles, some more than ten, and others named superseded, non-existent, incomplete, or extinct titles. None of these irregular responses, however, would likely disqualify a respondent's input. If, for example, he or she identified a title such as Nuclear Instruments and Methods without specifying either Section A or Section B of Nuclear Instruments and Methods in Physics Research, we credited both sections of the obviously intended title.

In 2000 we searched Science Citation Index using the statement ((ZP=37831) OR (CS=((OAK()RIDGE) OR (OAKRIDGE))) AND PY>1995) then, using the RANK JN command, found the journals in which ORNL authors had published most frequently in the immediately preceding five years. In 2001 we used the same search statement, but used the RANK CW command to identify the journals most frequently cited by ORNL authors. We recognized that this approach would include in the results a few authors not affiliated with ORNL, but because the Laboratory is overwhelmingly the largest scientific research institution in the town of Oak Ridge, we believed we could tolerate the relatively small number of false positives. We also recognized that it was unnecessary to repeat these searches every year at journal renewal time because trends in these types of data change very slowly. We have not succeeded in finding a satisfactory way to identify those journals in which ORNL authors are most frequently cited by others. Such a criterion could be very useful in making selection decisions about journals specializing in research areas important to our patrons, and we continue to explore the practicality of this measure.

Each year at journal renewal time we consider the data collected in recent years to help us decide whether to add or drop subscriptions. We score the data from the user surveys and citation analyses by giving a "10" to those few titles that occurred most frequently in each category. For example, Physical Review Letters, Physical Review B, and Science, each cited over 1100 times in the period 1994 to 2001 by ORNL authors, received a "10." The next most frequently occurring titles got a "9," the next an "8," and so on. The titles that occurred the fewest times got a "1." We arranged the titles so that the lower the score, the greater the population of titles for each score. For example, in scoring the journals in which ORNL authors had published there were ten journals with a "10," 22 with a "9," and so on until there were 457 with a "1." The distribution of the data was such that, using some degree of judgment, we could position the boundaries between the rankings at natural gaps in the distribution.

In 2002 we obtained a cumulative score for each of four categories: journals in which ORNL authors published from 1994 to 2000, journals that ORNL authors cited from 1995 to 2001, journals that ORNL researchers identified as "core" journals in 1999, and journals that the researchers identified as "core" in 2002. We then selected multipliers to weight the score in each category for each journal title in our collection. The sum of the multipliers was equal to one, so that the cumulative scores, like the scores of the individual categories, would range from 1 to 10. Because we wanted to give more weight to the more recent data for the "core" journal evaluation in 2002 than we gave to the data collected in 1999, we multiplied the score for the 2002 data by 0.3, and the score for the 1999 data by 0.2. Recognizing that the ORNL library functions more as a resource than as a repository, we weighted the score for the category of journals that ORNL authors had cited by using a multiplier of 0.3, and a multiplier of 0.2 for the category of journals in which ORNL authors had published. We then took the sum of the products from each category to use as a single numerical value assigned to each journal title.

After we had obtained a quantitative value for the journals using the described process, the journal team of four reference librarians went through the list title by title to decide which journal subscriptions would continue for the coming year. We used a spreadsheet that included the journal title, its holding library within the ORNL library system, its cost for each of the most recent three years, the name of the publisher, electronic availability, and the scores for each category measured in recent surveys and citation analyses. With the spreadsheet information we could conveniently consider factors such as the occurrence of multiple subscriptions of the same title in different branches of the library system. In recent years we have eliminated all multiple subscriptions, now keeping a single subscription only at the branch library where the title is most in demand. Cost differentials among journals of similar ranking provided another criterion to be used in meeting the yearly target budget. We also made a list of high-scoring titles not in our collection for consideration as new subscriptions.

Although the librarians at ORNL recognize various problems as well as advantages associated with electronic journals, we have favored moving toward electronic subscriptions whenever it has seemed reasonable. An important consideration in moving toward "electronic only" has been our clientele's overwhelming preference for the electronic medium. Even so, many of our users say they would also like to have access to the most recent issues in paper because the paper format is judged to be easier to browse than the electronic one. The electronic versions are usually preferred in searching for articles in older issues.

With the advent of electronic journals we have dropped some criteria we once used for journal selection. Until the middle of the last decade we counted journal issues one month each year as they were returned to the shelves or were circulated to the different branches in the library system. As the number of electronic subscriptions grew, these surveys became correspondingly less compelling since they did not reflect electronic usage. Instead, we prefer to use publishers' counts of the number of times their electronic journal sites have been visited by ORNL users, or to combine these with the circulation statistics. The methods used by different vendors to measure site activity are, however, inconsistent, and we can therefore only confidently compare the usage of titles from the same publisher.

Cost has inevitably become an important consideration in journal selection in this era of declining budgets and inflation rates exceeding ten percent. Nonetheless, we attempt to minimize our consideration of cost on the basis that, if a journal is regarded as important to our clientele's research, the library should make every reasonable attempt to offer access to that journal. On the other hand, we recognize that often the price of a single journal is enough to pay the cost of many document delivery transactions and that such transactions are usually paid from the researchers' own project accounts rather than by the ORNL library. When we find that there are large numbers of document delivery orders for particular journals, we can consider purchasing new subscriptions to those titles most requested.

In recent years “Big Deal” publisher discounts have become yet another factor to be evaluated in journal selection. Purchasing these offers is tempting when expensive journals can be acquired as components of a journal package. It is only as the total cost of desired subscriptions from a publisher approaches the cost of a package deal the publisher offers have we bought the package. Our philosophy continues to be that the most effective way to control costs is to remain focused on the criteria we have determined to be most appropriate for journal selection and user service at ORNL.

We continue to modify our approach to journal selection as new criteria become important. For instance, in the life sciences the journal evaluation process revealed a high user preference for several titles, including Trends in Genetics, Genes and Development, Nature Genetics, and Nature Biotechnology. Besides retaining these titles, the librarians recognized the need to pursue expansion of the collection in genetics, genomics and proteomics, biotechnology, and computational biology. Several recent large grants to ORNL scientists in these research areas led us to consider the sources of available funding as an additional weighting factor. Future versions of the model may encode the funding levels for research initiatives related to ORNL's new Joint Institute for Biological Sciences and the Laboratory for Comparative and Functional Genomics, called the "Mouse House," or other fields. The evolving combination of quantitative and qualitative data, assigned weights, and value-ranking of each title provide a systematic method for making decisions about the current and future status of the journal collection, and a method to evaluate the needs of individual research areas, including life sciences.

Our method of journal selection has several advantages. While no approach can be completely objective, we believe that our method substantially reduces the subjectivity of the decision-making process. Any decision to cancel or not purchase a given journal has the potential to disappoint some part of our clientele, and usually does. While budgetary constraints make some negative reaction unavoidable, the availability of quantified data that can be discussed with a concerned customer provides the library staff with a consistent method to explain and defend our decisions if necessary.

BIBLIOGRAPHY

1. Christie, A., Kristick, L. (Spring 2001). Developing an Online Science Journal Collection: A Quick Tool for Assigning Priorities. Issues in Science and Technology Librarianship, http://www.istl.org/istl/01-spring/article2.html.

2. Frazier, K. (March 2001). Contemplating the Costs of the "Big Deal." D-Lib Magazine, 7(3), http://www.dlib.org/dlib/march01/frazier/03frazier.html.

3. Lynch, P. M. (1997). Partnering to Provide Electronic Access to Life-Sciences Serials: The Experience of Journal Selection for the Electronic Library Project. The Serials Librarian, 31(1-2), 227-233.

4. Robb, D. J., McCormick, A. (1997). Decision Support for Serials Deselection and Acquisition: A Case Study. Journal of the American Society for Information Science, 48(3), 270-273.

5. Sidney, S. A., Diodato, V. (1993). Do Developing Technologies Help Predict Science Journal Selection? Science & Technology Libraries, 14(1), 15-28.