Information Outlook, Vol. 5, no. 3, March 2001
Information Professionals: Changing Tools, Changing Roles
by Nils C. Newman,
Alan L. Porter,
and Julie Yang
Nils Newman is the president and founder of IISC, a company dedicated to the exploration of text mining and bibliometrics for the management of technology. He may be reached at newman@iisco.com.
Alan Porter directs Georgia Tech's Technology Policy and Assessment Center. He may be reached at alan.porter@isye.gatech.edu.
Julie Yang is an Information Consultant, Library and Information Center, Georgia Tech. She may be reached at Julie.yang@library.gatech.edu.
No profession will undergo more radical change between 2000 and 2010 than will the Information Professional.
This presumptuous prediction is based on a set of convergent trends that, taken together, imply a coming new world for information professionals. We associate these trends with a series of studies we have performed for industry, government, and academia to draw implications for information professionals. We make the case for dramatic changes, then recommend particular actions to the profession and to its individual members. Within the scope of this short article, we don't try to distinguish implications specifically for various information professionals (e.g., information managers, database searchers, marketing research supporters). We address these generally to "you" to determine how they come to bear personally.
Trends
The trends coming together to change your lives reflect the emergence of the Information Economy. Increasing availability of informationand our nascent efforts to more effectively use that informationdrive this dominant societal transformation of our era. Without undue hand waving about exponential information growth and the Internet, let's look at four discrete trends.
1. Where once successful information retrieval might have yielded, say, 10 "good hits" (articles or whatever), today it might yield 1,000, and in coming years, perhaps, 10,000 relevant records.
2. Information staff resources will remain roughly constant.
3. To bridge the gap between good hits and what we can read, software tools will organize and process the additional good hits.
4. The product of this software processing will not be fewer hits. It will be new forms of information that add intrinsic value for users.
Figure 1 suggests the "paradigm shifting" potential of these trends. We need to take advantage of better tools to make use of this outpouring of information. We need to have the computer "read" for us, if you will. That is, we need to analyze text not just to point us to precious nuggets (to extract a few really good articles), but to illuminate patterns in the full body of information. The issue here is not information retrievalthere is too much good information for that to suffice. The issue is to perceive the patternse.g., associations among particular concepts within the domain, producer emphases changing over time, new techniques entering the "fringe" of the domain with promise to change it greatly.
Figure 1
Let's track what's driving each trend.
Trend One
Search potential is being driven upward by a) enhanced fingertip access to b) increasing numbers of databases with c) unlimited access licenses, using more comfortable d) search engines. Retrieval potential is bolstered by e) broadband media and f) more intuitive electronic downloading. Such enhanced search potential makes for a qualitative change in the nature of the resulting information. Previously you might have handed over a few well-targeted items for the requesting user to read. Now you and the requester must decide how to get maximum value from thousands of items. (Literally, we have found that our studies on particular scientific or technological topics often generate this order of magnitude of relevant information.)
There is no way that better search tools, better indexing, better information categorization, or better anything is going to reduce the number of good hits to a digestible number. Something new is needed in how we treat this pertinent information for our research reviews, competitive intelligence activities, or marketing assessments.
Trend Two
We may be mistaken, but our parochial sense suggests no increase in information staff resources. Multiple forces interact. Some promote increased support for information professionals due to demand for services driven by the increasing information resources, perceived value in accessing those resources, and changing technologies. But other forces press against increasing supporteasier access encourages a do-it-yourself mindset; new graduates who have been trained on the far side of the "digital divide" are willing to perform their own information retrieval; and short-term financial payback considerations demand pressure "information services" to pay their full way. Taken together, we don't think these forces sum to increasing support for information professionals.
So, combining trends one and two, we find inexorably increasing textual (and other forms of) information with no increase in human capacity to absorb it. Also, information staff resources who could potentially prefilter the excessive information are relatively fixed. (Under trend one, argue that this is hopeless, even were that increasing information professionals at hand.) The gap between information availability and usability will keep increasing.
Trend Three
To bridge the gap between good hits and user capacity to digest, we need new knowledge perspectives. We need to extract insights from the set of information per se, not just from individual items within it. Software can help.
New information exploitation tools are becoming available. Enormous data compilations and the Internet provide a wealth of numeric, text, and graphical resources. Exponentially improving computer processing and memory capabilities reinforce corresponding telecommunications capabilities that enable us to process those information resources in real time.
Our group has been working to develop and apply one such suite of text mining tools since 1993. Called "Technology Opportunities Analysis" (for details and 'projects & papers' see http://tpac.gatech.edu), we have focused on exploiting searches of bibliographic abstract databases to profile research and development activities on particular topics. This approach is presently encapsulated in VantagePoint software (http://theVantagePoint.com) and associated analyses and scripts. Taken together these enable rapid analysis of thousands or tens-of-thousands of abstract records on particular search topics to see "who's doing what?" Such analyses can help identify vital trends and topical interrelationships. In turn, these can help researchers locate others with shared interests, aid R&D managers in evaluating programs, alert lawyers to intellectual property issues, yield competitive intelligence on what other organizations are emphasizing, and even inform national policymaking.
We are not alone in advancing such tools. Related work appears under a confusing potpourri of terminology. Professional concentrations have emerged in Computational Linguistics, Natural Language Processing, Knowledge Discovery in Databases (KDD), Data Mining, Text Mining, Bibliometrics, and Information Visualization. Much work has centered on finding related texts Message Understanding Conferences (MUC) and Text Retrieval Evaluation Conferences (TREK). Most notably, the software powerhouses are getting involved. IBM's "Intelligent Miner" suite provides a platform for other tool developers. In the late 1990s, we reviewed activity in KDD. The most intriguing finding was that the core members of the world's leading research group had just moved from Caltech to Microsoft. We believe that the commitment of IBM and Microsoft signals that development of information exploitation capabilities will accelerate strongly in this decade.
Trend Four
The product of this enhanced processing of information resources will not be fewer good hits for the user to read. It will be new forms of information. New visualization capabilities can show, at a glance, "Mount O.J." peaking above a background of news articles in the year of his trial. Text summarization capabilities seek to convey the sense of extensive sets of text items in a quick read. Charts show that the trend of activity bearing on Topic A is dying off, but that addressing Topic B is exploding.
User demand for enhanced information exploitation is poised to explode. This can be taken two ways! The confluence of these increasing information resources, together with a globally competitive climate foreordain a coming burst of enthusiasm for gaining a competitive edge. External intelligence is vital to public and private sector organizations in prioritizing investments, taking advantage of intellectual property, and moving rapidly to capture new markets.
On the other hand, our studies have hit lots of resistance to information exploitation. Decision-makers remain widely unaware of information resources, easily intimidated by fancy-sounding tools and novel knowledge forms, and moronically convinced that their intuitive judgments suffice. As a result, demand for better intelligence to inform decision making remains very modest.
That will change (we assert) as particular successes emerge thanks to effective information mining. We can imagine word of such successes spreading like wildfire to generate escalating demand by "all" professionals and managers for the edge that comes from better intelligence. We envision this in the form of an S-curve such as that prevalent in many natural and technological growth processes. That is, very slow growth for a considerable period of time, followed by rapid escalation, then slowing as a limit of opportunity is approached. We anticipate that rapid escalation this decade (2000-2010).
Taken together, trends one, three, and four force each other to support a forecast of greatly increasing demand for information and its analysis, representation, and interpretation. We see this as a rousing vote of support for expanded information professional needs. Oh, but meshing that with trend two is left for you (information professionals) to work out. And that's a real challenge.
Tools
Here's our "equation":
More information + new tools + heightened demand = challenges and opportunities!
More information calls for new ways to exploit it. One needs to profile 1,000-article sets, not search for a few needles in such haystacks. We see potential in changing mores regarding appropriate background for articles, theses, and reports. Instead of selective spotlighting of a theory here and an empirical finding there, it's more effective to first provide a figurative profile of who is active, what issues receive high levels of attention, how those issues interlink, and what are the emerging hot topics. This "research profiling" can provide better context. We need a new form of information analysis that looks at the forest, not just those few trees that happen to lie directly in front of our eyes.
Put more generally, new knowledge forms are required, e.g., research profiles complementing traditional style literature reviews; inductive categorizations displacing deductive, fixed hierarchies; and "just-in-time" elucidation of relationships to fulfill specific demands. The new information exploitation tools enable creation of such new knowledge forms.
For example, our "Technology Opportunities Analyses" usually took a lot of timemost of our applications took months. A good search (typically in multiple bibliographic abstract databases) required iterations. Then analyses took us "forever" as we tried myriad alternatives on an ad hoc basis. Then we had to develop effective figures and charts with interpretive text. However, in the past year, we have developed the capability to "script" such analytical steps. We first formulate an effective search and download the records (usually abstracts from databases such as INSPEC, MEDLINE,and/or U.S. Patents). We now have automated routines to clean up the data (i.e., apply fuzzy matching routines and thesauri to consolidate like terms). We discern user preferences (e.g., for PowerPoint slides in a style that a particular manager likes). Then, most of the analytical steps, through to generation of attractive tables and charts, are done automatically. Results need be interpreted, of course, with fine-tuning, but the process is both tailored to the individual user's preferences and fast.
We are working to integrate the entire process from search through representation. Smarter "front-ends" can help adapt search strategies across multiple information resources. Experience can be captured by continually improving thesauri (e.g., building ever-better collections of synonymous terms for a given institution). In our work, directed mainly at emerging technologies, we try to consolidate particularly informative analyses as "innovation indicators." For instance, we think that tabulating how much R&D on an emerging technology is being conducted by industry (versus government and academia) is an indicator of possible commercialization approaching.
However, the success story is not yet complete. The tools don't automatically fulfill the end-users' needs. Managers and subject experts probably don't want to learn to use information search, analysis, and representation tools themselves. They do want to understand the representations, be comfortable that they are well founded, and know how to interpret them. None of these usability steps comes easily. Accomplishing these efforts to train end-users to take advantage may present the greatest challenge of all to information professionals. That implies that you need to learn how to use these tools, how to interpret the resulting analyses, and how to convey this understanding to intimidated, but needy, "users."
Roles
Our experiences with adoption of information exploitation include two major types. For one, we have conducted roughly one-hundred analyses on particular topics for clients in both public and private sectors. We have learned a little about "what works," and a lot about what doesn't. Those experiences have been sufficiently painful that we successfully made a case for National Science Foundation support to figure out "Why Managers Don't Want Our Knowledge" (and what can we do about it). That research has put us onto several promising leads:
· Managers often prefer nicely packaged recommendations on what action to take, in preference to "do-it-yourself" bits and pieces
· Would-be users need to understand the bases of those recommendations and have confidence in them
· Speed is often of the essence; decision-makers need analyses fast or not at all
· Empirical compilations of information have a difficult time competing with expert opinion for the decision-maker's ear.
These user demands suggest that "someone" needs to perform multiple information functions. The demand for integrated analyses argues against separating functions into information management, information analysis, subject matter knowledge, and information presentation. That is too complex to build robust, close relationships between users and information providers. Furthermore, it's too slow. We ran a small survey that asked when users needed technology analyses to be completed to be useful twenty-one percent reported within that day, with another forty-five percent within a week. Thus, delays as information requests are relayed from decision-maker to analyst to information search & retrieval specialist, and back, with side-nods to graphics experts (visualizers) don't cut it.
The need for speed and comprehension of the full information exploitation process argues against separation into piecework. The jobs of information professionals must expand beyond traditional information management and services. Roles are fusing with those of analysts. Information professionals will increasingly diagnose problems, "discover" relationships (e.g., KDD Knowledge Discovery in Databases), network with topical experts, and make action recommendations. Instead of being information collectors, information professionals must take on analytical roleseven as analysts come to perform information search and retrieval. To take on these challenges, information professionals need to be path-breakers in the derivation of new knowledge forms (e.g., literature profiling), and be adept at using and training others to use the tools to generate that knowledge (e.g., text mining, data visualization).
Easy rhetoric, hard action. We suggest that the information profession needs to reach out forcefully to assess the plethora of emerging information tools and ascertain what each has to offer in various applications. Sharing such assessments with SLA members can serve to alert them to emerging capabilities. A crucial next step is to provide training to SLA members in how to use these tools and interpret the results. In essence, you need to enhance the profession's real and perceived capabilities.
"Information Services" needs to power up to become each organization's key information analysis and intelligence resource. Certainly the specifics will vary greatly from one organizational context and a given information job to another. But, to meekly reside within once accepted limits of just providing raw information would be suicidal to the profession. Were that the case, then those who have been providing analyses will enhance their scope to obtain direct access to information resources, obviating the need for Information Services. The premise here is that having separate information management and analysis functions is counterproductive. If you don't do it, "they" will.
This message ought to feel downright threatening. Most of us have comfort zones that are being sorely pressed by change in this most rapidly innovating domain of all information in the Information Economy/Age/Society. This article says, "that's right, and you ought to be seriously scared." But not intimidatedthe information resources and the information exploitation tools are becoming easier to use. As mentioned, aids such as "scripting" (e.g., "macros" written in VisualBasic to sequence a number of actions in multiple programs) can make life quite pleasant. Based on our experiences, librarians need to expand their toolkits to become adept at a suitable, large set of complementary search, analysis, and representation tools.
But we're not done. We advise you to also boost your subject expertise. By working together with those knowledgeable about particular topics vital to your organization, you can become adept yourself.
And there's more. You also need to become the communications master. You want to understand the prime user (manager, professional, etc.) and her/his needs. You want to be able to translate those needs into information management and analyses. You want to be able to communicate those analytical processes and the findings back to that user. And, you want to be able to convince that user of their value and train her/him to understand them. A good model might be the ultimate consultant who understands problems, interprets those, finds the critical data, and sells the solution.
The aim is for you personally to bring to bear "how to" skills, "what" topical knowledge, and "who" sense of needs, with attendant communication skills. That will enable you to generate effective analyses rather than provide raw information. We see this combination of skills as your competitive edge making you invaluable to your organization. In the process we're proposing that you stretch well beyond your likely comfort zones to move strenuously into analytical, communicating, and problem-solving arenas. 
RSS
Feedback form
