A Roadmap for the Successful Implementation of Competitive Intelligence Systems
by Katherine Shelfer and June Verner
Katherine Shelfer is associate professor at the College of Information Science and Technology, Drexel University, Philadelphia, PA. She may be reached at: kathy.shelfer@cis.drexel.edu.
June Verner is professor at the College of Information Science and Technology, Drexel University, Philadelphia, PA. She may be reached at: june.verner@cis.drexel.edu.
Regardless of format or location, an organization's knowledge is generally filtered through both a cognitive dimension and a relationship dimension.
The successful design,development and deployment of a successful
CI system requires a good project plan. Much like a roadmap, this plan serves to identify important milestones and provide information about alternative routes that can help the project team(s) avoid delays. According to a survey by the Delphi Group, 58% of the useful knowledge of an organization is recorded information (documents and databases) and 42% resides in employee brains (Hickens 1999). Integrating knowledge management and competitive intelligence encourages their use, improves their quality and allows the firm to respond more rapidly to changing business conditions (Senge 1994), so the best CI system uses what is already inside the organization. One of the first decisions is whether to improve access to the organization's recorded information or elicit knowledge that currently resides in employee brains. Regardless of format or location, an organization's knowledge is generally filtered through both a cognitive dimension and a relationship dimension.
The cognitive dimension focuses on the "stuff," but to identify the important attributes of the relevant "stuff," it is important to know how it is filtered through the relationship dimension. The relationship dimension has the following characteristics:
Purpose the organization's business purpose, its vision, mission, goals and objectives
Process the means by which strategic initiatives are moved from "clean sheet" to launch
People the "four - ics"
· Demographics personal characteristics of current and potential users (e.g., position, education and training, learning style)
· Psychographics personal belief systems that impact action/reaction/interaction
· Geographics factors of culture, distance and time
· Politics formal/informal lines of authority, innovation and trust (Shelfer and Goodrum 1999)
The planning process and the project itself must take these characteristics into consideration in order to be successful. In fact, the successful CI system might also be likened to the steps involved in successful community gardening: (1) seed the ground, (2) water and fertilize what you plant, (3) weed the garden, (4) reward the gardeners, (5) discourage the predators, and (6) harvest the value.
Indicators of Project Failure or Project Success
Experienced consultants have identified the following critical failure indicators: (1) lack of informed consensus; (2) acceptance of the status quo; (3) unwarranted trust in the vendor; (4) failure to support the business purpose; (5) a short term, internal, myopic approach; (6) paralysis by analysis; (7) sabotage by external predators; (8) suicide through ignoring project constraints; and (9) failure to consider business, human or technology limitations imposed on the project (Tyson, 1998). Careful planning is the best form of failure prevention. There are both management constraints and technical constraints to be considered. Management constraints involve three key problem areastime, money and scope. The flexibility needed to deliver a quality project is severely hampered if any one or two of these three are fixed. For example, project constraints impact deadline constraints. A fixed budget with deadline constraints generally kills any chance of success. Regarding technical constraints, is there any flexibility in terms of the tools available? It is imperative to avoid getting caught up in the "trade rag" hype, so a warning is appropriate here: NEVER buy off vendor presentations! Other key factors to consider include experience, whether legacy systems are involved and whether the system will be "bleeding edge" or a patch. It helps to know if this system will be a pilot for knowledge-sharing in the organization.
Unlike failure, success can't be guaranteed, but it is much more likely if the project includes: (1) flexible design; (2) willingness to implement a mechanized "less than ideal" system; (3) use of an evolutionary approach with prototyping; (4) giving users substantial (to total) control; (5) coordination by individual business units; and (6) active networking. Though there are many factors contributing to software project success, the presence of a committed project sponsor is one of the most important early success factors (Proccacino and Verner 2001). A committed sponsor has a significant impact on many of the project phases and project functions, including the (1) schedule estimates, (2) quality of the project team members, and (3) degree of interaction with other stakeholders.
Choosing Between MethodologiesWaterfall Lifecycle or Prototyping Lifecycle
Without early defined and agreed-upon acceptance criteria, there is no way to recognize when the project is completed or if it has been successful. Most project managers would prefer to use a waterfall lifecycle for project development as this ensures better control of the project and its schedule (Verner and Cerpa, 1997). A waterfall development methodology must begin with good requirements in order to ensure project success, as poor requirements are a major cause of project failure. These days, projects are notorious for beginning (and sometimes even ending) with inadequate requirements (Proccacino and Verner 2001).
An iterative prototyping methodology, such as the prototyping lifecycle, is preferable when it is difficult to obtain adequate requirements at the start of a project, as is often the case with a CI system. Iterative prototyping tends to be more verbal and to involve the developers in many more interactions with the customers and users than is required for a waterfall methodology (Verner and Cerpa 1999). With iterative prototyping, project managers and developers are often uncomfortable with the lack the documentation that would be part of a waterfall process. As a result, development personnel tend to dislike using iterative prototyping. They also feel that they have little project control and that they do not really get a good understanding of the requirements.
Since this method normally involves many iterations, it is important to limit the number of incoming requests for additional functions. Having early agreement concerning acceptance criteria can ensure that the prototyping iterations do not continue endlessly. One way that project managers control iterative prototyping in practice is to make it clear from the beginning that there will be a maximum number of iterations (e.g., three). If stakeholders are warned in advance of exactly how many iterations will be permitted, they are more likely to make a genuine effort to complete requirements within this time frame. The project manager will also be more content, since there is little chance that the project will continue iterating endlessly.
Mapping System Requirements to User Needs
There are several useful techniques that can be applied to understand user needs in order to define the requirements of the CI system. These include (1) stakeholder analysis, (2) needs analysis, (3) gap analysis and (4) cost-benefit analysis. Even if we have done a good job of gathering the software requirements, these cannot be frozen at the start of a project. Inevitably, there will be changes that must be made. A project can become hopelessly bogged down without a change control process. All members of the team are vulnerable to approaches for undocumented changes, which can spin out of control. Uncontrolled changes can escalate the costs of a project and lead to huge cost overruns, so it is really important to have clearly defined roles for the change process, and for the users to have clear understanding and expectations.
Given that we must expect change and plan for it, there should be an agreed-upon change process, with clearly defined roles and processes to deal with change requests. This allows us to minimize changes to those parts of the project that are critical to its overall success. The two-level control scheme is generally effective in controlling changes. That is, a senior review board oversees the scope of the project and a change control board is responsible for managing the "tweaks." It is generally best if decision-level representatives from ALL stakeholders are represented on these boards. These boards rule on the change requests as appropriate, and monitor their impact on the project itself. The discussion below is an overview of the planning process that leads to a good project plan.
TRIP PLANNER
1 Identify WHO - Stakeholders
1.1 Sponsor
1.2 Other Stakeholders
1.3 Project Manager
1.4 Team Leader
1.5 Initial Team
1.6 Risk Management (RM) Plan
1. Gap Analysis
2. Identify WHAT
2.1 Create a K-Map
2.2. Needs Analysis
3. Identify HOW MUCH
3.1 High-Level Cost-Benefit Estimates
4. Identify HOW
4.0 Options Trading
4.1 Implementation Plan
4.2 Plan to Action
5. Identify HOW WELL
5.1 Feedback
5.2 Post Mortem Review
Five-Step Checklist
In order to craft a successful CI system, we use a trip planner to identify (1) who is involved with the system, (2) what they actually require, (3) how much they are prepared to pay and (4) how many benefits they expect to obtain from the system. We then must discover (4) how we will go about developing the system and, at completion, (5) determine how well we met the project goals.
Step 1. Identify WHO
The members of the CI unit whose viewpoints are critical to the success of the project are the CI Unit Coordinator, the Industry Watcher (Researcher) and the Issues Management Analyst. In a small CI unit, these are often overlapping roles. In addition, there are other Stakeholders who should be included. These are the sponsor as well as representatives from operations management, strategic business units and various functional areas. It might be a good idea to include someone who interfaces with all these units, such as the manager of quality assurance.
Stakeholders should be identified in the very earliest stages of the project. To identify stakeholders, you need to identify those who USE the system, but it is also important to find out more about those who IGNORE/BYPASS formal systems. Two methods include gap analysis and user requirements analysis. These are formal methodologies about which much has been written. This is the point at which you also need to select a Project Manager and a Team Leader. In selecting members of the Initial and Subsequent Team(s), be sure you know who you need to please and who will give you the necessary time. It is important to integrate different forms of sharing and to plan for organic evolution and growth. Imperfection will happen, so it helps to be prepared to deal with it.
A caution is relevant hereguard against your own subconscious filters! It is important to listen carefully and obtain verification that you have understood the intended input. In the example below, what is the topic of discussion? Would you guess the topic of this example is (I)NSECTS and (Y)ELLOW STUFF? (Shelfer 1998).
1.1 Sponsor -What's in it for ME?
With a visible, politically powerful, committed project sponsor, things are easier for the project manager and the project team. Customers and users are more likely to accept the project as a whole, to be cooperative and to be
· Who is attracted to Y is it I ?
· How might I be attracted to Y?
· Where might I be attracted to Y?
available when required. They are also more likely to provide adequate time for the development team. If the project begins without a sponsor, or the sponsor is not visibly committed, the seeds of failure have been already sown (Proccacino and Verner 2001). Because of the huge effect that a sponsor has on a project, it is important at the beginning of the project for the project manager to find out as much as possible about the project sponsor.
· How much political power does the sponsor have?
· How interested is the sponsor is in the project?
· How visible is the sponsor likely to be?
The project needs a senior management sponsor. The more politically adept the sponsor, the more likely the project will be able to avoid road blocks. The sponsor must be visible and continue to show interest and commitment throughout the project. Other stakeholders are unlikely to feel that they must cooperate if the sponsor is seen to lose interest. If the project begins without a sponsor or the sponsor shows little commitment, then the project is unlikely to succeed. It is better for the project to begin without a sponsor (and for a committed sponsor to be found later) than it is for the project to continue with an invisible uncommitted sponsor (Proccacino and Verner 2001).
1.2 Other Stakeholders
Anyone who is impacted in some way by the system should be considered, but the majority of stakeholders are users. It is important to consider all operational or functional areas that are impacted by the proposed system. All of these groups have the ability to impede the development of the system in some way. If senior management shows interest and commitment to the project, other stakeholders are much more likely to cooperate. If senior management is not supportive, then why should the other stakeholders make time to work with the project teams? At this point, it helps to prioritize. Which matters FIRST? Generally, this would be the capabilities, priorities and politics of the technical staff who are charged with the design, development and/or deployment of the system.
1.3 Project Manager
Any project is more likely to be viewed as successful if the project manager is viewed as credible by the stakeholders. Other factors leading to successful project outcomes include having a project manager who (1) has a clear vision of the project, (2) is respected by team members, (3) does not play favorites, and (4) is able to delegate tasks (Proccacino and Verner 2001). At the beginning of the project, the project manager is responsible for generating a project plan that is monitored and modified as the project progresses.
1.4 Project Team Leader(s)
The project leader should be both a visionary and a pragmatist. This individual will serve as a communicator, bridge, translator, facilitator, champion, cheerleader, nurse, and traffic cop.
1.5 Team(s)
How do you identify likely members of the initial/subsequent team(s)? The objective of The INITIAL team is to assess and plan. The objectives of SUBSEQUENT teams are to carry out, evaluate and iterate. These teams require different knowledge, skills and abilities. However, continuity is important, so there should be core members who carry forward as part of the succession plan. No plan is perfect. Bad things can happen to good teams, and bad teams can implement successful projects. However, careful selection and management of the project team provides a greater likelihood of project success.
1.6 Risk Management Plan
· Product size risks
· Business impact risks
· Customer related risks
· Process risks
· Process issues
· Technical issues
· Technology risks
· Development environment risks
· Staffing risks (Pressman 1997).
Once a list of potential risks has been identified and listed, the impact of each of the risks needs to be assessed. After the highest risk items are identified, a risk management plan can be developed for each of these risks. The risk management plan may include risk avoidance, risk mitigation, risk management and contingency planning. The project manager may decide that some risks are so unlikely they can be ignored. The manager may also choose to transfer the risk to others. As the project proceeds, risk monitoring activities commence. To do this monitoring the project manager must ensure that the project management process includes risk tracking. This might take the form of developing a project risk table and reviewing the status of each risk at weekly or biweekly meetings. At each of these meetings, identified risks may be dropped from the list or new risks added. The effort put into the risk management plan should correspond to the probability or importance of the risk; that is, the greater the risk, the more developed should be the risk management plan.
Step 2. Identify the WHAT
Management literature identifies four basic kinds of knowledge problems: Ambiguity; Equivocality; Complexity; and Uncertainty (Zack 1999). When dealing with ambiguity and equivocality, optimal systems support face time. That is, they provide fast access to experts and link the key decision makers. Systems designed to minimize complexity are those which can manage and analyze complex interrelated inputs, variables, processes and outputs. Those designed to resolve uncertainties will increase connectivity through space and time. The following checklists help determine relevant system features (Zack 1999).
2.1 Create a K(nowledge)-MAP
A Knowledge Map (or K-map) is defined as a collective view of the Knowledge and skills required to successfully perform each step in delivering a solution (Fulmer, Gibbs and Keys, 1998). A K-map is predicated on the following assumptions:
1. Knowledge is composed of data to which insight has been added. It is derived from processes taken in context.
2. Processes involve players who engage in rules-based activities
3. Rules have definitions
4. Activities can be monitored, managed and mapped
The process of creating a K-map begins with mapping the existing terrain. The processes used to identify relevant knowledge include asset-mapping and user needs analysis. In developing an asset-map, the current available corporate knowledge assets are identified, as are the processes by which these are obtained and used. The next step is to engage in user needs analysis. Once these have been completed, knowledge assets are compared to user knowledge requirements. It may be that the existing knowledge could provide additional value if it were more
Complexity Checklist
· Handle complex interrelated inputs, variables, processes & outputs
· Include searchable online repositories
· Provide access to experts who can handle complex situations
· Coordination of complex tasks
· Coordinated integration of diverse expertise
· Support decentralized Decision-making
Uncertainty Checklist
· Data: Locate, Gather data; Cue to gaps
· Connectivity : Across time, across distance
· Communications Configuration: Flexible; Handle sudden, unpredictable events; Broadcast at-large RFI; Generate pointers
· Support Learning: Estimation, Inference, Prediction; Automated management of central knowledge repositories; Feedback effectively managed, so the CI system project might be directed toward enhancing internal use of existing information. At this point, however, gaps between available and needed knowledge generally remain. This is the knowledge that must be acquired from others (external to the organization). At this point, the K-map should be refined so that it becomes project-specific. This refined version should be used to establish the project's direction.
2.2 Needs Analysis
Needs analysis takes two forms"road maintenance" and "traffic control." For each category, the relevant tools are listed in Table I.
There are several tools that can also be used to plan for future needs, including Delphi, content analysis, scenario analysis, impact analysis and K-maps (Fulmer, Gibbs and Key 1998). The K-map is the tool emphasized in this paper.
Step 3. Identify the HOW MUCH
The goal of any CI system is to enhance decision support through knowledge transfer. At this stage, it is not yet known which components of the CI systems project will actually provide the most useful enhancements to the CI function at the most affordable cost. For this reason, a high level cost-benefit analysis is used to support the choice between two options: (1) the one that shows the greatest likelihood of a quick success; and (2) the one that caters to a specific stakeholder/champion/unit who is most likely to protect Phase Two. Such priorities need to be determined before a more detailed project plan can be developed. Regardless of the option(s) chosen, the objective at this stage is to buy time to protect the next phase of the project.
3.1 High-Level Cost-Benefit
The initial project cost estimation process requires a mental review of the project. For example, estimate the start-up tasks involved in terms of people or space. How are such costs estimated for other projects? It helps to know what happened to other projects when costs were more (or the benefits less) than initially projected. Throughout the life of the project, it is important to keep cost data. Comparing projected costs to actual costs is an important component of the post-mortem review & will benefit future planning. This is a difficult process, so experienced project planners should be involved.
Step 4. Identify the HOW
As the initial project starts to take shape, it helps to take a few days just to think about it For example, can you define the CI system? What are the mission/objectives? How will this impact network management? What new resources might be required to maintain it? How will it support industry watch or issues management?
4.1 Options Trading
It is now time to trade information systems/technology implementation options. There are three possible categories from which to choose: (1) commercial "plug and play"
| Table I. NEEDS ANALYSIS TOOLS AND TECHNIQUES | |||||||||||
| Road Maintenance · employee suggestions · consensus-building · self-directed teams · Statistical process control (SPC) · benchmarking · workout programs | Traffic Control · transfer innovation · effective intervention · business process reengineering (BPR) · task forces · ad hoc groups · internal management development | Utilities (multipurpose) Basic: Customer surveys External advisory groups Consultants Content analysis Advanced: · Dialog · Scenario Planning · Merlin Exercise · Practice Fields · Action learning 1. Six Sigma Quality 2. CAP - Change Acceleration Process · Knowledge Management (KM) · K-maps | |||||||||
| Source: Fulmer, Gibbs and Key 1998. | |||||||||||
software; (2) customizable off-the-shelf (COTS) packages; and (3) in-house development. There are three myths/legends that need to be debunked. No option is the "magic bullet" solution.
1. Commercial plug & play is [all/not at all] what we need.
In opting for commercial software, the cost of development is shared with others and there is the added benefit of the supplier's development expertise and insight into CI systems. However, the system might also include unwanted features or lack important ones. The vendor's promises might fall short of reality.
2. Customizable off the shelf [gives/costs us] WHAT?
Choosing COTS can provide a clear advantage in terms of creating a system that matches user requirements without the cost of creating a unique system. However, going "outside" for the customization delivers core knowledge about an organization's business processes to outsiders, the vendor may not actually have the skilled employees required for high-level customization, and the process might be far more expensive than initially anticipated. Access to (and ownership of) the code can also be an issue. The organization can be held hostage to costly technical support of the final hybrid product.
3. In-house is [ cheaper/more expensive]
than other options.
In-house is not automatically cheaper/more expensive than other options. You might get a considerably better product without the risk of outsiders getting access to sensitive information. Your project might be "piggy-backed" onto an existing project at little additional cost. However you might also find that project resources have been "hijacked" or that other projects continually have a higher priority. You might have a great price from the in-house team, yet never get the system built.
In evaluating system options, it is a good idea to develop a features "beat sheet" or features comparison checklist. Examples of such features are:
· Natural Language v. Structured Queries
· Jargon-mapping. The following list illustrates the different terms used for similar concepts (Pearson 1999).
· Locals - Plan, Do, Study, Act
· Explorers - Explore, quantify, analyze, verify
· Scientists- Problem-solve, teamwork, new ideas, create value
· Systems - Chaos, simulation, team learning, feedback
· KM - Communicate, virtual consults, knowledge base. lessons learned
· K-building - Psychology, statistical methods, systems theory, Knowledge theory
· Content Mining (good for e-mail)
· Statistical Process Control (SPC)
· Constant Processing
· Taxonomy Templates
· Flat and stays close to the business process
· Visualization Software
To properly organize a project plan, the project manager must break the project down into its parts using hierarchical decomposition that includes the following: (1) work breakdown structure; (2) PERT chart; (3) GANTT chart; and (4) resource table. A work breakdown structure is an extension of an organizational chart that is used to show the essential components of the system. Because a work breakdown structure is used to break projects into manageable pieces, it is possible to work on and monitor one phase at a time. Once this is completed, a task network or PERT chart is used to identify tasks with parallel, or sequential, execution. After the PERT chart has been developed and a critical path defined, a Gantt chart is developed to show:
· What tasks must be done
· When they should start
· When they should finish
· How long each should take
Gantt charts are very useful, because they provide good visibility of major tasks and allow for comparison of tasks with estimates and actual project status. Along with this, a resource table should be developed that shows who is available to work on the project and what, exactly, is their availability.
4.2 Plan to Action!
User feedback is essential to the identification of system requirements and good system requirements are essential to project success. If a customer is unsure of what exactly is needed then an iterative prototyping approach can be helpful in more accurately pinning down requirements. After each prototyping iteration user feedback is used to modify the prototype. This process continue until maximum benefit for minimum cost has been achieved.
New conditions are documentedsuccess metrics are generated
4.21 Managing the Team
The fatal error that some teams make is to develop a "throw it over the wall and see what sticks" mentality. This happens when a team defines a set of system requirements, develops a system and then just throws it at the users. In this situation, there is very little end-user testing. Follow-up is restricted to those features that "stick." In reality, useful features may have unrelated problems such as dated or poorly formatted content. Once the system has been defined, it is important for the team to keep stakeholders involved in the development process, since there are sure to be technical and functional trade-offs.
4.22 Management Reporting
For a project to be considered a success, it must show steady progress. In the case of systems development, no news is almost always considered bad news, so the stakeholders should never have unpleasant surprises during development. Weekly team meetings that monitor and track progress should be held. From the very beginning of the project, of course, it should be clear to all parties as to the type and frequency of progress reporting that is expected. It is essential that milestones be monitored and management and other stakeholders be kept informed of progress.
4.23 Quality Assurance, Control
Quality is not something that can be added at the completion of a projectit needs to be included from the very beginning of the project. During the development process, it is important to review quality early and often. An excellent method for improving quality is to use peer reviews to identify and remove errors. Used properly, peer reviews can (1) save time, (2) improve the quality of the product, and (3) be more efficient than testing for identifying faults (McConnell 1997). Peer reviews can involve both team members as well as selected clients. Team members can work through various scenarios with the client. As a result of this stakeholder interaction project results are found more credible by the stakeholders.
Step 5: Identify the HOW WELL
5.1 Feedback
Once the system has been opened to the users, there are several specific passive and unobtrusive methodologies that can be used to obtain feedback. For example, transaction log analysis (TLA) can be used to identify where (and in what direction) patterns of usage may have changed. (TLA requires that a baseline be established at the inception of the project.) E-mail response can also be integrated into the system. Ethnography, in the form of analysis of user feedback messages, can be used to direct future enhancements.
5.2 Port-Mortem Review
Many useful lessons can also be learned from a post-mortem review of the project. The port-mortem review should not be used as a performance review of individuals, but as a vehicle to discover what went right/wrong in the project. What went right should be institutionalized. For example, how good was the risk analysis? What risks did or did not happen? What else should be considered, or managed more carefully, next time? The post-mortem review can also be used as a vehicle for quality control. The information acquired can be used to improve all phases of future projects. It is particularly useful to have ideas, issues and problems documented throughout the project and examined during the post mortem review. The whole contribution to date can then be evaluated. While it might not be advisable to share all the information from the post mortem review, it will be helpful if an expunged version is made available within the organization.
Endnotes
· Hickens, Michael (2000). "Xerox Shares its Knowledge," Management Review September 1999, in Knowledge Management Yearbook 2000-2001, ed. By James W. Cortada and John A. Woods. Butterworth-Heineman 2000, p. 100.
· Senge, Peter, et al (1994). The Fifth Discipline Fieldbook: Strategies and Tools for Building a Learning Organization. NY: Doubleday, 1994, p. 10.
· Tyson, Kirk (1998). Complete Guide to Competitive Intelligence. Lisle, ILL: K. Tyson International.
· Verner, J. M. and N. Cerpa (1997). "Prototyping-Does your Perception Depend on Your Job?" Journal of Systems and Software, January, No. 36, pp. 3-16.
· Proccacino, D. and J. Verner (2001). "Early Risk Factors for Software Development" Proceeding of the European Cost and Metrics Conference, London, April 2001 pp. 107-116.
· Pressman, R. (1997). Software Engineering A Practitioner's Approach, McGraw Hill.
· McConnell, S. (1997). Rapid Development Microsoft Press.
· Shelfer, Katherine and Abby Goodrum, "Competitive Intelligence Education as an Extension of Library Education," JELIS: Journal of Education for Library and Information Science. Fall 2000.
· Shelfer, Katherine (1998). "Understanding Science And Technology Research Needs." Florida Libraries 41.6 (September/October , p. 123-7.
· Thomas A. Pearson, "Measurements and the Knowledge Revolution," Quality Progress September 1999. In Knowledge Management Yearbook, 2000-2001. ed. By James W. Cortada and John A. Woods. Butterworth-Heineman, 2000, p. 411.
· Michael H. Zack, "Managing Organizational Ignorance," Knowledge Directions: The Journal of the Institute for Knowledge Management, Spring 1999. In Knowledge Management Yearbook 2000-2001, ed. By James W. Cortada and John A. Woods. Butterworth-Heineman, 2000, p. 353-373.
· Robert M. Fulmer, Philip Gibbs and J. Bernard Keys, "New Tools for Sustaining Competitive Advantage," Organizational Dynamics, 1998. In Knowledge Management Yearbook 2000-2001, ed. By James W. Cortada and John A. Woods. Butterworth-Heineman, 2000, p. 336-352. 
RSS
Feedback form
