Benchmarking Measuring and Comparing for Continuous Improvement
by Sue Henczel
Sue Henczel is the manager of training, cataloging and consortia at CAVAL Collaborative Solutions, Australia.
Benchmarking for Success
When organizations want to improve their performance, they benchmark.
They compare and measure their policies, practices, philosophies and performance against high-performing organizations anywhere in the world. The process of benchmarking is used to identify useful business practices, innovative ideas, effective operating procedures and winning strategies that can be adopted by an organization to accelerate its own progress by ensuring quality, productivity and cost improvements. In other words, benchmarking involves investigating how things are done elsewhere and where they are done differently or better, to see whether a group could adapt the processes of another organization to improve their own processes.
Libraries have traditionally used external comparative benchmarking studies to measure themselves against others in order to justify their existence or prove their value and support their case for maintaining existing levels of staffing or funding. These studies were invariably based on statistics gathered and shared for the purposes of measuring how they rank with other libraries (Gohlke, 1998). Libraries have also employed internal benchmarking methodologies to measure the "value" they contribute to their organization and compare this against what is contributed by other departments, divisions or information providers.
One of the primary aims of a special librarian is to ensure the service they offer contributes significantly to the success of the organization and is as good as it can be. To do this, a librarian must utilize internal and external benchmarking processes to measure performance and identify possible areas of improvement.
Defining Benchmarking
There are many similar definitions of benchmarking, most of which fit neatly in two groupsthose that include implementation of the findings and those that do not. Jane Foot's definition is widely cited in library literature, yet it's a definition that does not incorporate the implementation of the identified best practices. On reflection, I realized many of the benchmarking case studies I had read were focused on the identification of the benchmarks and the measurement and comparison process, while very few followed through to the process of adapting best practice to improve a process.
Foot (1998) defines benchmarking as the process of comparing yourself with othersmeasuring your service's processes and performance and systematically comparing them to the performance of others in order to seek best practice. It enables the identification of areas where improvement is possible, how it might be achieved and what benefit it might deliver. Other definitions include the implementation of best practice to improve a process beyond the benchmark performance such as that used by Partnership Sourcing (1997).
Christopher Bogan and Michael English (1994) and Gerald Balm (1992) clarify some of the associated terms:
· A benchmark is a fixed point, target or standard against which you can be measured.
· A benchmarking partner is any group or organization that is used for comparison.
· The highest performer in a benchmarking partnership is considered the best practices organization (i.e., the one that has the most efficient and effective practices in place). The goal of benchmarking is to improve performance by adopting the best practices of benchmarking partners.
· Performance indicators are used for measuring performance and monitoring progress against set targets.
Types of Benchmarking
Benchmarking can be done within your organization or externally, with other organizations. Internal benchmarking is a comparison of similar operations within your organization, while external forms of benchmarking include competitive benchmarking (a comparison with your competitors) and functional benchmarking (a comparison of methods with organizations who have similar processes in a different industry) (Hinton, Francis and Holloway, 2000).
Data benchmarking measures and compares inputs and outputs of a process against a benchmark to assess performance. Process benchmarking analyzes a sequence of activities and compares them with similar functions in best practice organizations.
Historical Development
Although benchmarking has been used as a management tool for many years, it experienced resurgence in the early 1980s due to the Total Quality Management (TQM) movement, in which benchmarking was inherent as a means of ensuring quality improvements. Its focus was on accountability, performance measures, best practice and the rational use of resources all issues of interest to librarians and the organizations that fund and support the services.
Since the mid-1980s librarians have seen benchmarking as having two distinct purposes: to demonstrate how their services and overall performance rank against those of other similar libraries (external benchmarking); and to measure the value of their contribution within their organization, as compared with that of other internal divisions and information services (internal benchmarking). In Australia, special libraries (particularly those in the government sector) were the earliest exponents of external benchmarking methodologies, using their informal networks to identify suitable benchmarking partners.
They shared statistics and details of processes and services with each other to demonstrate to their organizations that their performance is as good or better than similar libraries, or conversely that they require a higher level of support from their organization to perform as well as others. This type of comparison was often used to highlight inadequate resources (usually staffing), technology and/or funding.
The early 1990s saw libraries refocusing on their responsibilities to their organizations and becoming more concerned with internal benchmarking processes that enabled them to measure their performance in comparison with other parts of the organization. This was (and still is) done to justify the existence of the library when faced with threats such as downsizing or outsourcing, and to demonstrate the value of the contribution made by the library to the organization's success. Internal benchmarking can demonstrate how the library's performance ranks against that of other departments or divisions, and against competing internal information services.
Toward the mid-1990s librarians underwent major changes in their service focus as they struggled to deal with the incorporation of electronic content and the associated social, legal and technological issues related to access, delivery, user expectations and licensing. This led to a renewed emphasis on benchmarking and performance measures as librarians were being asked to provide additional services with the same or minimally increased resources.
In 1996 the Special Library Association's special report Competencies for Special Librarians in the 21st Century was released, which incorporated knowledge and application of benchmarking in a number of the competencies. The fact that benchmarking is inherent in the SLA competencies resulted in a renewed interest in benchmarking and its uses and benefits. Major issues became "how to benchmark?," "what to benchmark?" and "who to benchmark with?"
It wasn't until 1998, with the development and release of ISO 11620 Library Performance Indicators, that libraries had a list of performance measures that they could apply to their benchmarking processes to maintain the consistency previously missing from the majority of benchmarking projects. The performance indicators are grouped into user satisfaction, public services and technical services. A revised ISO 2789 Library Statistics was released in 1999, with the revision of the 1991 standard being closely coordinated with the development of ISO 11620. ISO 2789 covers how to count libraries, librarians and items for inclusion in statistics related to library provision and services.
Benefits of Benchmarking
Those who have participated in benchmarking projects have identified a significant number of benefits for both the library and the library staff. These include:
· More efficient and effective processes
· Improved performance and customer service
· Increased competitiveness
· Increased utilization of resources
· Improved levels of management support
· Better and faster decision-making
· More efficient and effective marketing
· Accelerated change and facilitated change management
· Improved professional relationships
· Support for strategic goals of the library, which in turn support the goals of the organization
· Proof of library's value to the organization
Pitfalls and Problems
· Collaboration Versus Competition - Benchmarking requires collaboration, either with other groups within the organization in the case of internal benchmarking or within other organizations in the case of external benchmarking. This is often difficult when potential external benchmarking partners are also competitors, as "commercial sensitivity" often prevents them from revealing details of their processes.
· Non-Standard Data Collection Methods - Methods for collecting data are not consistent or standardized, therefore the comparisons made may not be as valid as they may appear to be. Also the boundaries around the processes being benchmarked must be clearly defined.
· Changing Environment - Continuous measurement and comparison does not easily reflect changes such as new competition, new technology and inflation rates, etc., unless these are incorporated into the data collection processes when they become apparent.
· Reliance - There is a danger of becoming reliant on benchmarking rather than seeking inventive or innovative process improvements. When over-used, it can perpetuate a culture of "sameness" and stifle creative thought that is needed for the development of new ways of doing things.
· Resources - Benchmarking requires a significant commitment of resources such as time, people, money, etc., without any guarantee that there will be a cost benefit. The costs may outweigh the tangible benefits, as many of the benefits achieved through benchmarking are intangible.
· Identifying Partners - Identifying potential benchmarking partners is difficult. Sufficient information must be known about the processes used by each partner to be sure that there will be benefits in measuring them and comparing them.
· The People Factor - Often the adaptation of a process is not successful, as its success was dependent on the skills and expertise of those using it in the initiating environment. It is important to recognize and understand where processes are successful due to the synergies of the group or team using them as against where quality is inherent in the process itself.
· Inappropriate Adaptation - Benchmarking the processes that you have ascertained as being strategically important to your organization is important. Beware, however, of benchmarking processes that are not strategically important just because you think that someone else may be doing them better than you.
· Innovative and Efficient Processes - Benchmarking is less useful to those who have established innovative and efficient processes that have been developed for their unique environment. It can, however, be very useful for those who are struggling with inefficient and uneconomical processes and who are looking for better ways of doing things.
· Best Practice - Best practice is not always appropriate. Best practice can be unique to an environment or situation and will not adapt successfully to a different environment. Take into account the people using the process and the relationship between that process and others before making a decision to change.
Developing A Benchmarking Process
There are many books, articles and papers about the development of a benchmarking process, and the most consistent message appears to be that the process must be rigorous and well planned. Suggested stages include:
· Plan what you are going to benchmark, how, with whom and for what purpose. As with any other project, planning is the most critical stage because inadequate planning and preparation can cause a project to fail. Plan your resource allocation (who will do what, how and who will pay for it) and decide which processes will be benchmarked and why.
· Define your process and identify all of the inputs to the process, the outputs and the customers. Define the boundaries of your process.
· Flowchart your process - include inputs, activities, flow indicators, decisions and outputs.
· Identify problems in the process and their possible causes. Use the expertise of the staff who actually perform the activities, as they have the most in-depth understanding of related issues.
· Develop your metrics (how you will measure) and identify significant qualitative data that is important to the process.
· Collect data - measure your process.
· Compare collected data with benchmark partner.
· Identify areas for improvement.
· Formulate recommendations and an implementation plan.
· Implement improvements.
Choosing Processes to Benchmark
Choosing what to benchmark can be as important, if not more so than choosing a benchmarking process or partner, because it will impact how useful the results are to the organization. A useful list have been developed by Catherine Cassell, Sara Nadin and Melanie Older Gray (2001) who suggest that the processes chosen for benchmarking must be:
· related to critical success factors for success;
· processes that are currently causing trouble;
· processes that are important to your customers and which are not performing up to expectations;
· in areas where the competitive pressures are impacting the most (e.g., price, flexibility, products);
· processes that have the greatest potential for differentiating you from the competition; and
· processes that are not in transition.
Successful Benchmarking
Some tips for successful benchmarking include:
· Aligning benchmarking projects with strategic objectives (critical business issues that have high pay-offs and are aligned with organizational values and strategy)
· Following a rigorous process (planning, analysis, implementation and review)
· Benchmarking the processes and not just the outputs
· Choosing an optimal benchmarking partner
· Committing to implementing the changes required
· Understanding your organization and its culture so that you are adopting the most appropriate changes in the most appropriate way
· Understanding the limitations of benchmarking
Current Trends and Projects
Current benchmarking literature focuses on the electronic environment and how libraries and information services are delivering e-content; how they are managing user access, licensing and authentication; and how they are viewing performance standards and quality management issues in the electronic environment. All have government support funding and are based in the academic and public library sectors.
One major benchmarking effort is the EQUINOX PROJECT (http://equinox.dcu.ie). EQUINOX is a project funded under the Telematics for Libraries Programme of the European Commission. This project addresses the need of all libraries to develop and use methods for measuring performance in the networked, electronic environment, alongside the traditional performance measurement, and to operate these methods within a framework of quality management.
Equinox has resulted in a number of 'spin-off' projects designed to measure and monitor service quality. For example, in January 2000 the London Business School Library commenced a program of internal benchmarking as a means of monitoring service quality. Performance indicators used were chosen from those emanating from the Equinox Project.
Spin-off projects include:
ARL (Academic and Research Libraries) E-Metrics Project
Twenty-three ARL member libraries, plus the Library of Congress, are collaborating in this study to determine how to develop statistics and performance measures that address the delivery of networked information resources and services. The primary objective of this project is to find ways to gather consistent and comparable data to evaluate electronic information services (http://www.arl.org/stats/newmeas/).
The McClure/Bertot Project
The McClure/Bertot Project (sponsored by the U.S. Department of Education) involves the development of U.S.-wide performance measures and national core statistics for electronic networked-based services offered by public libraries.
The JUSTEIS Project
The U.K.-based JUSTEIS Project aims to provide the information and tools to identify the current use of electronic information services and to forecast areas of growth in British higher education. This would enable JISC (Joint Information Systems Committee) to plan for new services, network enhancements and other areas of investment.
Although these projects are focused on academic and public library sectors, there are elements of their outcomes, for example the processes (planning, data collection, analysis or the metrics) that can be used successfully within special libraries.
Benchmarking in Special Libraries
The greatest benefit of benchmarking for the special library is to discover what others are doing that you could use to add value to your services and strategic processes. Librarians in corporate, government, health and other special libraries (including academic faculty libraries) can use internal and external benchmarking to improve their processes and prove their "value" to their organization.
Before a special library embarks on a benchmarking project, it is critical that its strategic processes are identifiedthose processes that actually contribute to the achievement of its goals and objectives. The boundaries of those processes must be clearly defined so that accurate comparisons can be made with benchmarking partners.
To demonstrate how important the boundaries are, take for example, a library that defines its acquisitions process as inclusive of resources acquired for the organization but not retained within the library. A potential partner may define its acquisition process as exclusive of those resources. Difficulties arise when comparing these processes, as one has a much broader scope than the other and is considerably more complex.
Once the processes and their boundaries have been defined and you are investigating what others are doing and how they are doing it, it is important to understand that not all "best practice" is appropriate for your situation. Best practice in one environment is not always best practice in another, as the situation is different, the people are different and most importantly the relationship with other processes may be different. Take care that you don't adapt processes that are not entirely suitable for you situation.
Is anything that you are currently doing innovative and unique? If so, you will be unlikely to find benchmarking partners for these processes. It will, however, demonstrate these processes as being unique.
The key to benchmarking is to see what is being done differently and to have the ability to assess the value of those differences to your library.
The pitfalls and problems associated with benchmarking are many and varied, and most will impact the success of a benchmarking project if not fully understood. If used correctly and for the right reasons, benchmarking can result in improved productivity impacting competitiveness, internal and external relationships, support networks and resources. 
| References and Further Reading: |
| In addition to the following references, a bibliography is available on the SLA Web site at http://www.sla.org/content/membersonly/electrinfo/qa.cfm.
Balm, Gerald J. Benchmarking: A Practitioner's Guide for Becoming and Staying Best of the Best. Schaumburg, Ill.: QPMA, 1992.
Bogan, Christopher E. and English, Michael J. Benchmarking for Best Practices: Winning Through Innovative Adaptation. New York: McGraw Hill, 1994.
Cassell, Catherine; Nadin, Sara and Older Gray, Melanie. "The Use and Effectiveness of Benchmarking in SMEs," Benchmarking: An International Journal, (8) 3 2001; 212-222.
Corrall, Sheila. Strategic Management of Information Services: A Planning Handbook. London: Aslib, 2000.
Foot, Jane. How to do Benchmarking: A Practitioner's Guide. London: Inter-Authorities Group, 1998.
Gohlke, Annette. Law Library Resource Exchange, 1998. http://www.llrx.com/features/bench.htm [accessed 25 March 02]
Hinton, Matthew; Francis, Graham and Holloway, Jackie. "Best Practice Benchmarking in the UK," Benchmarking: An International Journal, (7) 1 2000; 52-61.
Partnership Sourcing. Benchmarking the Supply Chain: First Cycle of Surveys. London: Partnership Sourcing Ltd., 1997 |
RSS
Feedback form
