Evaluation of knowledge management tools using AHP E.W.T. Ngai*, E.W.C. Chan Department of Management and Marketing, The Hong Kong Polytechnic University, Kowloon, Hong Kong, People’s Republic of China Abstract This paper presents an application of the analytic hierarchy process (AHP) used to select the most appropriate tool to support knowledge management (KM). This method adopts a multi-criteria approach that can be used to analyse and compare KM tools in the software market. The method is based on pairwise comparisons between several factors that affect the selection of the most appropriate KM tool. An AHP model is formulated and applied to a real case of assisting decision-makers in a leading communications company in Hong Kong to evaluate a suitable KM tool. We believe that the application shown can be of use to managers and that, because of its ease of implementation, others can benefit from this approach. q 2005 Elsevier Ltd. All rights reserved. Keywords: Analytic hierarchy process; Knowledge management, knowledge management tools 1. Introduction Although knowledge management (KM) is not a new idea, it is the management buzzword of recent years. KM is widely discussed in many studies (Kakabadse, Kakabadse, & Kouzmin, 2003; Liao, 2003). Interest in KM from both industry and academia has been growing rapidly. There is no doubt that KM has come to play an important role in organizations and is a key issue for them. KM solutions make it possible to deliver knowledge to all departments with an organization. According to a survey conducted by Hackett (2000) some kind of KM effort is underway in 80% of companies. Six percent use KM enterprise-wide and this figure will grow to 60% in the next 5 years. Twenty-five percent of organizations have a chief knowledge officer (CKO) or chief learning officer (CLO). Another survey (Murray, 1999) reports that 50% of Fortune 500 companies plan to invest in KM systems (KMS), with the figure being much higher in companies with over 500 employees. These figures indicate that most organizations are rapidly moving to have a communicated KM strategy in part of their business. Generally speaking, KMS integrates various knowledge processes to solve one or more business problems as an organizational information system (IS) (Mattison, 1999). Vast numbers of tools are available in the software market to support KM; however, no framework currently exists to aid in the evaluation and selection of KM tools. This paper is primarily concerned with providing such a framework. The analytic hierarchy process (AHP) to aid decision-making in evaluating KM tools is implemented via a case study. We believe that this evaluation framework will be extremely useful to many organizations. The paper is organized as follows. Section two discusses the literature on KM, KMS, KM tools and AHP applications. Section three presents the background of the company that wants to purchase a KM tool. Section four describes the development of the AHP model for evaluating KM tools. Section five depicts the procedure of the AHP model as applied to a company and reports the findings. Section six concludes the paper. 2. Literature review 2.1. Knowledge management In order to understand what KMS is and what KM tools are, it is necessary to have a clear view of knowledge and KM. Knowledge is based on data and information. Data represents the raw facts without meaning; information is what is obtained when data is organized in a Expert Systems with Applications 29 (2005) 889–899 www.elsevier.com/locate/eswa 0957-4174/$ - see front matter q 2005 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2005.06.025 * Corresponding author. Tel.: C852 2766 7296; fax: C852 2765 0611. E-mail address: [email protected] (E.W.T. Ngai).meaningful context, while knowledge is characterized as the meaningfully organized accumulation of information (Zack, 1999). Nonaka (1994) points out that there are two different types of knowledge in an organization: explicit and tacit knowledge. Explicit knowledge is formal and systemic, while tacit knowledge is highly personal and difficult to formalize. These two types of knowledge are both essential to the organization and must be captured and shared for others to benefit. Thus, knowledge in the organization should be managed properly and carefully. A successful KM will obviously provide sustainable competitive advantage. Various potential benefits for KM have been listed in (Giannetto & Wheeler, 2002). There is no universal definition of KM. KM refers to the set processes or practice of developing in an organization the ability to create, acquire, capture, store, maintain and disseminate the organization’s knowledge. A number of researchers have proposed KM frameworks that contribute to an understanding of KM. For example, Nonaka (1991) suggested four basic processes (socialization, externalization, combination and internationalization) for creating knowledge in any organization, based on the conversion from tacit to explicit knowledge. Holsapple and Joshi (2002) identified four major knowledge manipulation activities: acquiring, selecting, internalizing and using knowledge. A comparative examination of other KM frameworks can be found in (Holsapple & Joshi, 1999; Rubenstein-Montano, Liebowitz, Bchwalter, McCaw, Newman, & Rebeck, 2001) 2.2. Knowledge management systems and tools KMS are the IT-based systems developed to support and enhance the organizational processes of knowledge creation, storage/retrieval, transfer, and application (Alavi and Leidner, 2001). As organizations come to see the importance of KM, many are developing KMS that offer various benefits to facilitate KM activities (Alavi & Leidner, 1999). During the development of KMS, attention should be paid to various issues and challenges related to using IT to support KM (Hahn & Subramani, 2000). Prior studies have suggested some ideas for designing and developing KMS. Bowman (2002) described the structure of KMS and identified the features that are expected in a comprehensive KMS. These features include text and multimedia search and retrieval, knowledge mapping, personalization, collaboration, messaging, etc. Salisbury (2003) proposed the collaborative cognition model and successfully built a KMS based on this theoretical foundation. There are many KM tools in the software market with functionalities well suited to develop a KMS. Ruggles (1997) (defined KM tools as technologies to enhance and enable the generation, codification and transfer of knowledge. Gallupe (2001) stated that “KM tools are the basic technological building blocks of any specific KMS. Individual tools can be combined or integrated to form a specific KMS with particular functions such as knowledge storage and retrieval. Another specific KMS may comprise tools to generate ideas and share those ideas among a work group.” KM tools are also referred to as an enabler of business processes that create, store, maintain and disseminate knowledge (Tsui, 2003). For a KM tool to be considered effective, it should perform each part of the KM process. Therefore, the tools should be able to capture, store, organize and index information, ensure that the information is secure, establish a workflow process and distribute information (English, 2003). ICASIT KM Central (http:// www.icasit.org/km/tools/index.htm) divided KM tools into four essential functions, namely capturing and codifying knowledge; collaborating, sharing and leveraging acquired knowledge; and creating knowledge. With the aid of KM tools, the organization can develop their KMS more easily. Technologies are the key enabler of the implementation of KM (McCampbell, Clare, & Gitters, 1999; Nonaka, Umemoto, & Senoo, 1996). The most frequently utilized types of technology in KM tools are: Intranets, content management systems, document management systems, relational and object databases, groupware and workflow systems, data warehousing systems and data mining systems (Duffy, 2001; Lee & Hong, 2002). A vast number of tools have been deemed to be KM tools. Some are established ITbased tools borrowed from other disciplines that have entered into our KM arena as IT tools with extended functionality (Tyndale, 2002). In our study, we only focus on IT-based tools that have been designed specifically as KM tools from their inception. 2.3. The AHP The AHP, developed by Saaty (1980) is designed to solve complex multi-criteria decision problems. It is a flexible and powerful tool for handling both qualitative and quantitative multi-criteria problems. The AHP is aimed at integrating different measures into a single overall score for ranking decision alternatives. Its main characteristic is that it is based on pairwise comparison judgements. AHP has been applied to a wide variety of decisions such as car purchasing (Byun, 2001), vendor selection (Tam & Tummala, 2001), IS project selection (Muralidar & Santhanam, 1990; Schniedejans & Wilson, 1991), and software selection (Kim & Yoon, 1992; Mamaghani, 2002). Although there have been some studies on using AHP for software selection, each of the studies has focused on software with a different nature and function, such as antivirus and content filtering software, executive IS, simulation software, expert systems, multimedia authoring systems, logistics IS and AHP software. It is necessary to design and develop a generic AHP model to help KM practitioners select a KM tool to meet the KM objectives or requirements of their organization. 890 E.W.T. Ngai, E.W.C. Chan / Expert Systems with Applications 29 (2005) 889–8993. Background ABC Co. Ltd (the name has been changed to provide anonymity) is one of leading integrated communications companies in Asia. ABC Co. Ltd has been facing fierce competition after the deregulation of the telecommunications market in Hong Kong in 1995. In order to adapt to the market and to sustain a competitive advantage, ABC Co. Ltd restructures its organization almost every year. To gain more business opportunities by providing customers with total and end-to-end solutions, ABC Co. Ltd is transforming itself into an Information Technology and Telecommunications (IT&T) company. Currently, they are encountering several problems. First, most of the projects start from zero, without leveraging the knowledge/learning from previous, similar projects. As a result, the same failures in projects may occur again and again. Second, with restructuring every year, staff redundancies or layoffs are inevitable. This results in gaps in knowledge. Third, although ABC Co. Ltd is a leading telecommunications service provider, its services are less competitive (in terms of cost) than others in the market. Yet, ABC Co. Ltd is managing to increase the value of its services to customers and enhance its competitive advantage over competitors. Fourth, to position itself as an IT&T company, ABC Co. Ltd must evolve into an innovative company. However, although has many database systems and information systems throughout its offices in different locations in the territory, effective knowledge-sharing among staff has yet to be achieved, to say nothing of knowledge creation or innovation. Hence, ABC Co. Ltd has decided to develop and implement a KMS by using a KM tool on a KM pilot project in the Solution Architect and Bid Management (SAAB) system under its marketing branch, and then introduce the project to other branches. It is expected that KMS can help the company resolve the above-mentioned problems. The SAAB department mainly consists of two functional teams, a bid management and a solution specialist team. The role of the bid management team is to develop bid strategies and solutions (commercial and technical) and to bid on tenders. The role of the solution specialist team is to act as a system integrator by developing a total technical solution (including network services, networking equipment, server/PC, application software and application development, and so forth). The solution specialist team is also expected to assist the sales force in selling the proposed solution, thus helping to generate revenue for the company. Both teams have been confronted with the fact that no lesson learned from each bid is documented and that no repository has even been designed for centralizing each bid proposal. The bid team, including the bid manager, account manager and solution manager, often reinvents the wheel when a tender similar to a previous one comes in. They repeat the bid management process and are unable to leverage knowledge about the key to the success or failure of a previous, similar tender. This results in an ineffective use of the company’s resources and also allows failures to be repeated. Therefore, the purpose of the KM pilot system is for the SAAB department to retain, share, leverage and create knowledge. In the first phase, a KM pilot system will be implemented in the bid management team. 3.1. Company expectations ABC Co. Ltd expects to reap the following potential benefits with the help of the KMS. First, the KMS can improve the bid management process. As ‘re-inventing the wheel’ occurs with nearly each new project-bidding exercise the company hopes that the KMS will support the bid management process by providing the manager and officers with related information and knowledge for preparing documents to submit a tender. Second, the KMS retains previous knowledge and experience learned from past bidding. When a solution specialist in SAAB moves to other position/department or even leaves the company, his knowledge can still be retained in the KMS. Also, newly recruited staff can learn the knowledge that has been stored in the system. Third, the KMS creates and shares knowledge. Solution specialists can share and learn information about the product and knowledge about solutions from the knowledge discussion space in the system. The knowledge manager can harvest and create new knowledge in this collaborative environment. Fourth, the KMS enhances competitive advantage. Leveraging knowledge through a KM-enabled bid management process can lead to higher productivity and better bid management decisions. 3.2. User requirements The purpose of the KM pilot system is for the SAAB department to retain, share, leverage and create knowledge. Since it is much less expensive, time-consuming and can be integrated with a variety of off-the-self software products (Thierauf, 1999), ABC Co. Ltd decides to select a suitable KM tool that should be able to provide the context and facilities for an effective KM. In order to fulfil the purpose and the expected outcome of the KM pilot system, five requirements, based on (Tiwana & Ramesh, 2001) are used to select the proper KM tool: (i) Web-based system: The expected KMS should be able to run on the Web/Internet/Intranet. Users such as knowledge workers and knowledge managers can communicate, coordinate and collaborate with each other using a Web browser. (ii) High scalability: The KM system should be designed to be fully scaleable, such that the KMS should be able to support an increasing number of users and a higher load of transactions. Although the KMS is a pilot project in the SAAB department, there is an E.W.T. Ngai, E.W.C. Chan / Expert Systems with Applications 29 (2005) 889–899 891opportunity for the company to extend KM to other departments. With more than 10,000 staff in ABC Co. Ltd, the scalability of the KMS is an important requirement and cannot be ignored. (iii) Security: The security of a KMS is an important issue, particularly for a system running on the Internet. A Web-based KMS is accessible from any location via the Internet. Appropriate validations for access levels and security control, comprehensive audit logging, system security and data security are required. (iv) Flexibility and customizability: The choice of KM tool should allow for a certain degree of customization, particularly in the user interface aspect. (v) Integration with legacy and existing systems: The selected KM tool should be able to be integrated with other tools and applications. After an initial deliberation, three well-known KM tools that met with the company’s requirement were selected. The SAAB department decided to evaluate the KM tools to develop a KMS. Software products are often evaluated based only two or three criteria; as a result, it is easy for an organization to end up owning software that does not meet its needs or to have more than one KM tool. Therefore, there is a need to develop a systematic evaluation model for KM tools. A detailed description of how to evaluate KM tools is given in the following section. 4. The development of the AHP model The first step in developing the AHP model is to come up with a hierarchical structure of the problem. This classifies the goal and all decision criteria and variables into three major levels. The highest level of the hierarchy is the overall goal, i.e. to select the best KM tool for KM in the SAAB department. Level 2 represents the criteria and sub-criteria used in selecting the KM tools. Level 3 contains the decision alternatives that affect the ultimate selection of the KM tools. Fig. 1 depicts the hierarchy of the AHP model for evaluating the KM tools. 4.1. Identification of the criteria for evaluation In order to formulate the AHP model, it is necessary to identify the factors that influence KM practitioners’ choice of KM tools. After discussions with four KM consultants and the operations manager, we studied the features of the KM tools provided by vendors, reviewed the literature for selecting software, and identified three essential evaluation criteria to use in selecting the best KM tools: cost, functionality and vendors with sub-criteria and their attributes. The identified criteria were validated by the CKO responsible for the firm’s KM programme. AHP decomposes the overall decision objective into a hierarchic structure of criteria, sub-criteria and alternatives. 4.1.1. Cost Cost is a common factor influencing the purchaser to choose the software (Davis & Williams, 1994). It is simply the expenditure associated with KMS and includes product, license, training, maintenance and software subscription costs. Technically, these costs can be grouped under two major criteria, namely, capital expenditures and operating expenditures. Capital expenditures are the non-recurring costs involved in setting up the KMS. They include product costs (the basic cost of the KM tool), license costs (the cost of the KM tools in terms of number of users) and training costs (the cost of training provided to customers). Operating expenditures are the recurring costs involved in operating the KMS, which include maintenance costs and software subscription costs (the annual, pre-paid cost of upgrading the product to a major software release when it is launched). 4.1.2. Functionality Functionality refers to those features that the KM tool performs and, generally, to how well the software can meet the user’s needs and requirements. Thus, functionality is usually considered when selecting software, such as in (Lai, Trueblood, & Wong, 1999; Ossadnik & Lange, 1999). Since the objectives of KM can vary in different organizations, different functions provided by the KM tool can help the organization to achieve their KM goals. Based on a review of the literature and on consultations with KM practitioners, we identified six key functional elements of a KM tool: document management, collaboration, communication, measurement, workflow management and scalability. 4.1.2.1. Document management. Knowledge is embedded in documents in many organizations (Sahasrabudhe, 1999). Users become unproductive if the documents are managed inefficiently. Document management, which mainly involves searching for and organizing knowledge, consists of the following six basic features: storage, publishing, subscription, reuse, collaboration and communication (Conway & Sligar, 2002) 4.1.2.2. Collaboration. Collaboration is one of the key aspects of KM, since collaborative problem solving, conversation and teamwork generate a significant proportion of knowledge assets. Collaboration, mainly for generating and sharing knowledge, consists of the discussion, workspace and voting features. 4.1.2.3. Communication. The communication function provided in a KM tool helps users to work together and share knowledge. The function should capture and manage all forms of information exchange, and consist of the realtime chat and paging (instant messaging) features. 4.1.2.4. Measurement. ‘Measurement’ is the keeping of records on activities and changes in managed knowledge. It consists of the number of views feature that is able to 892 E.W.T. Ngai, E.W.C. Chan / Expert Systems with Applications 29 (2005) 889–899Fig. 1. The hierarchy of the AHP model for selecting KM tools. E.W.T. Ngai, E.W.C. Chan / Expert Systems with Applications 29 (2005) 889–899 893capture, calculate and record the activities of users in terms of number of views of particular contents or documents, and to generate regular reports of such activities. Another feature measures the number of entries in terms of the number of times some contents or documents were accessed, and generates regular reports of such activities. 4.1.2.5. Workflow management. Workflow management allows the movement of documents in information processes among individuals and applications to be specified according to a predefined process (Wensley, 2000). Workflow management may be used in communicating, cooperating, coordinating, solving problems or negotiating. Workflow technology delivers work items to appropriate users and helps the users by selecting appropriate applications and utilities. It mainly consists of the document approval and progress tracking features. 4.1.2.6. Scalability. Scalability refers to the ability to scale up without degradation in performance when the number of workspaces, knowledge bases and users grows (Tiwana, 2002). A system that performs well within a work group of limited size might not perform well when it is extended to an organization-wide level. Scalability can make the KMS a victim of its own success if it comes as an after thought. Scalability consists of features that include the (i) number of workspaces (ii) number of knowledge bases, and (iii) number of users. The system should at least support an unlimited number of these features. If not, the number of these features should be expanded without replacing the existing system. 4.1.3. Vendor The quality of vendor support and its characteristics are of major importance in the selection of software, such as in (Byun & Suh, 1996; Min, 1992). It is also critical for the successful installation and maintenance of the software. The important factors affecting the decision to select a KM tool are vendor reputation, the training provided, the implementation vendor, KM consulting services and support, maintenance, upgrades and integration. (i) Vendor reputation: When a vendor is selected for a KM tool, he becomes the business partner of the company. The vendor is expected to be committed to offering quality support services and to care about the business needs of the company. The reputation of the vendor can reflect the quality of the services and support given to the customer. Related to vendor reputation is: (a) the vendor’s expertise and (b) the vendor’s experience in the area of KM. These elements give the customer more confidence in the quality of the KM tool. (ii) Training: There are two kinds of training for users: KM training and product-specific user training (i.e. training in KM tools for users). The availability of training courses offered by vendor companies was considered along with the actual effectiveness of these courses. (iii) Implementation partner: Most companies overlook the importance of finding the right implementation partner. Even though a software product may be a perfect match, if the right implementation partner is not selected, there is a greater danger that the implementation will fail. (iv) KM consulting services: KM consulting services are important and required if a company does not have experience in KM. KM consulting services can offer help/advice to the company before, during and after the implementation of the KM tool. (v) Support, maintenance, upgrade and integration: A vendor does not only sell a KM tool but is also responsible for the ongoing support, maintenance, upgrading and integration of the tool. The services of a vendor generally include responding to technical queries, reporting and resolving faults, fixing bugs in the software, assisting in system design, conducting online tutorials, providing online documentation and product trials, and so forth. The actual services provided will depend on the kind of agreement reached with the individual vendor. 4.2. Alternatives In order to develop a KMS, the first thing we had to consider is the KM tool. In this case study, only IT-based tools that had been designed as KM tools from their inception were considered. We analysed three alternatives in KM tools—the Knowledger system, eRoom and the Microsoft SharePoint Portal Server. Not only are these the most commonly used KM tools, but demonstration software is also available during the evaluation and testing period. Alternative 1. Knowledger (http://www.knowledgeassociates.com). Knowledge Associates Ltd is a technology and consulting organization that provides KM solutions consisting of KM education, KM consulting, KM software systems (e.g. Knowledger) the use of the Internet and groupware technologies. Knowledger consists of components that support personal KM, team KM, and organizational KM. The benefit of these components is that, through the knowledge portal, it is possible to manage, collaborate, capture and convey information and so forth to the teams or organization. It integrates KM solutions with a high-level framework, methodologies, systems and tools to optimize working with knowledge at all levels. Alternative 2. eRoom (http://www.eroom.com). eRoom Technology focuses exclusively on providing Internet collaboration solutions to the extended enterprise. The eRoom software is a digital workplace that allows organizations to quickly assemble a project team, wherever people are located and to manage the collaborative activities 894 E.W.T. Ngai, E.W.C. Chan / Expert Systems with Applications 29 (2005) 889–899that drive the design, development and delivery of their products and services. In addition, it is a secure extranet or Intranet which, by enabling teams to discuss ideas, share information and make decisions all within a central location, also provides a valuable KM solution. Alternative 3. Microsoft SharePoint Portal Server (http:// www.microsoft.com/sharepoint/). Microsoft offers a wide range of products and services designed to empower people through software at any time, any place and on any device. It is currently the worldwide leader in software, services and Internet technologies for personal and business computing. SharePoint Portal Server software is a KM tool that is an end-to-end solution for managing documents, developing custom portals and aggregating content from multiple sources into a single location. 5. A case study using AHP In our case study, AHP is applied to the ABC Co. Ltd, which wanted to use KM tools to develop a KMS to improve its bid management process, retain knowledge, share and create knowledge, and enhance its competitive advantages. 5.1. Applying the AHP method The AHP is a powerful and flexible decision-making process. It can help people set priorities and make the best decision when both qualitative and quantitative aspects of a decision need to be considered. The Expert Choice (http:// www.expertchoice.com), a computer software package, is used to structure the decision into criteria and alternatives, measure the criteria and alternatives using pairwise comparisons, synthesize criteria and subjective inputs to arrive at a prioritized list of alternatives, and report on the sensitivity analysis. A four-step decision-making process based on (Expert Choice 2000) in using AHP is presented, as follows. 5.1.1. Breaking down the problem The first step in AHP is to develop a hierarchical structure to define a single pre-defined goal, the decision criteria supporting that goal, and potential sub-criteria supporting each criterion. As discussed in Section 4, the AHP model is then formulated (see Fig. 1). All criteria and sub-criteria ultimately contribute to the goal. A list of alternatives provides the decision points that are evaluated against this hierarchy. Upon completion of the evaluation, priorities will be derived for each alternative reflecting the degree to which the alternative satisfies the goal. 5.1.2. Comparative judgements to establish priorities Once the AHP model is set up, the priorities need to be developed. Weights are assigned to each criterion and subcriterion. These weights are assigned through a process called pairwise comparison. In pairwise comparison, each objective is compared at a peer level in terms of importance. Judgements were elicited from the CKO in ABC Co. Ltd and from the four KM consultants, and the matrices were formulated. For example, in order to determine the relative importance of the three major criteria, a 3!3 matrix was formed. Expert Choice provides ratings to facilitate comparison. These then need to be incorporated into the decision-making process. One of the comparisons can be done by verbal comparison. Decision-makers compare criteria for their relative importance and alternatives for their relative preferences, using words such as Equal, Moderate, Strong, Very Strong and Extreme. Thus, one of the questions that one may ask when using pairwise comparison is ‘How important is the cost factor to the selection of KM tools?’ The answer can be ‘Equal’, ‘Moderate’, etc. The verbal responses are then quantified and translated in a score through the nine-point scale shown in Table 1. For instance, if three major criteria, namely cost, functionality and vendor, are equally important in the selection of a KM tool and are inputted into Expert Choice, then the priorities from each set of judgements are found and recorded in Fig. 2. Table 1 Judgement scores for the importance/preference of criteria using AHP Verbal Judgement Numerical rating Extremely important/preferred 9 Very strongly to Extremely important/preferred 8 Very strongly important/preferred 7 Strongly to very strongly important/preferred 6 Strongly important/preferred 5 Moderately to strongly important/preferred 4 Moderately important/preferred 3 Equally to moderately important/preferred 2 Equally important/preferred 1 Fig. 2. Priorities. E.W.T. Ngai, E.W.C. Chan / Expert Systems with Applications 29 (2005) 889–899 8955.1.3. Evaluation When the priority of each criterion and sub-criterion is developed in the hierarchy, the actual evaluation of alternatives takes place. This involves yet another set of pairwise comparisons, this time between each alternative, evaluated against each criterion and sub-criterion. When these comparisons are finished, each alternative will have a derived priority, representing how well that alternative satisfies the pre-defined goal. The overall priority of a KM tool alternative is calculated by multiplying its local priority with its corresponding weight along the hierarchy. When we synthesize all elements using Expert Choice, we obtain the results shown in Fig. 3. The result of the calculation gives Knowledger, eRoom and Share Portal priority values of 0.391, 0.286 and 0.323, respectively. This indicates that among the three alternatives, Knowledger, with the highest priority value, is the most suitable KM tool for the company. Moreover, the overall consistency of the input judgements at all levels is within the acceptable ratio of 0.1. 5.1.4. Sensitivity analysis to obtain overall priorities Sensitivity analysis attempts to check the impact of change in the input data or the parameters of the proposed solutions. Relatively small changes in the hierarchy or judgement may lead to different outcomes (Mustafa & Al-Bahar, 1991). In order to examine the response of the overall priority of alternatives to changes in the relative importance of each criterion, three gradient sensitivity analyses are performed. Fig. 4 indicates that the priority of the cost criteria is adjusted from 0.333 upwards, and that the SharePoint Portal dominates the others. This reflects the substantial price difference between SharePoint Portal and Knowledger. Fig. 5 depicts that when the priority of the functionality criteria is adjusted upwards from 0.4 to 0.75, the optimal choice is Knowledger. However, when the weight of functionality is larger than 0.75, eRoom becomes the most favourable alternative. This reflects the fact that if the weight of the functionality factor is very high, eRoom will be the choice. Fig. 6 shows that the ranking of the alternatives is stable with respect to changes in the importance of vendor-specific factors because eRoom and SharePoint remain the same in weight, ranging from 0.125 to 1.0. 6. Conclusions and implications The development of a KMS is still relatively new to many organizations. With the rise of the organization came a strong interest in KM, and KM tools assumed an important Fig. 3. Synthesis for the problem of selecting KM tools. Fig. 4. Gradient sensitivity analysis—Cost. 896 E.W.T. Ngai, E.W.C. Chan / Expert Systems with Applications 29 (2005) 889–899role in supporting KM. KM tools can capture, organize, share and leverage knowledge elements, along with the necessary support and training to insure a successful launch of KM solutions within an organization. In this paper, a systemic approach is proposed using AHP to evaluate an appropriate KM tool for the organization. The model was developed and implemented for a real problem situation at a leading IT&T company in Hong Kong. The usefulness of the model was examined through observing its effect on the decision-making process in selecting an appropriate KM tool. ABC Co. Ltd has employed Knowledger to provide the company with a total KM solution. Initially, the company, with the help of Knowledger to implement the KMS, started a pilot project for the bid management team to enable to team to manage their knowledge to improve the bid management process. The management and the team members of the SAAB department have responded positively to the findings of this research. They agreed that it was easy to use the KMS to search for knowledge and that the KMS provided them with a collaborative environment and up-to-date knowledge. In addition, they perceived that the KMS could assist Fig. 5. Gradient sensitivity analysis—Functionality. Fig. 6. Gradient sensitivity analysis—Vendor. E.W.T. Ngai, E.W.C. Chan / Expert Systems with Applications 29 (2005) 889–899 897the company to gain competitive advantages and to retain and leverage valuable knowledge assets that are valuable. The management and team members then began to move on to the second stage. They extended the pilot project to the solution specialist team using the KMS to build up a knowledge community for the sharing, creation and collaboration of knowledge within the team. This study has several implications for KM practitioners who intend to evaluate KM tools to build a KMS. The main contribution of the paper is the AHP model presented in Fig. 1. This model provides a useful guideline as a structured and logical means of synthesizing judgements for evaluating appropriate KM tools for an organization. It helps structure a difficult and often emotion-burdened decision. The second implication is the functionalities of the KM tools listed in the model. Based on a comprehensive review, the features of KM tools have been examined and identified. These give an overview structure for companies without much knowledge of KM. Such companies can better understand the evaluation criteria in term of the functions in the KM tools. Third, decision-makers can compare different scenarios and possibilities with respect to appropriate criteria and sub-criteria through the sensitivity analysis of AHP, which provides a real time, interactive and graphical display of the overall priorities (Udo, 2000). Thus, these decision-makers can examine the strengths and weaknesses of each KM tool. Finally, with the aid of the computer tool, Expert Choice, it was shown that AHP can be easily applied to evaluate KM tools. The AHP methodology is particularly useful for decisionmaking in a multi-criteria context. 6.1. Limitations and future research In this paper, we integrate costs into the model through ‘absolute’ measures. Their integration with budgetary constraints may not be sufficient. There is no budgeting limitation with AHP, and this could be a robust approach to the effort of modeling. In addition, the utility values may be integrated into a capital rationing model. The AHP model provides comprehensive evaluation criteria, although it is not exhaustive, to evaluate KM tools that are rapidly changing (Marwick, 2001). The checklist of the criteria may have to be updated. The evaluation criteria can be further refined so that KM tools can be evaluated for many different environments. Acknowledgements This research was supported in part by The Hong Kong Polytechnic University under grant number G-T739. References Alavi, M., & Leidner, D. E. (1999). Knowledge management systems: issues, challenges, and benefits. Communication of AIS, 1(7), 21–41. Alavi, M., & Leidner, D. E. (2001). Review: Knowledge management and knowledge management:Conceptual foundations and research issues. MIS Quartely, 25(1), 107–136. Bowman, B. J. (2002). Building knowledge management systems. Information Systems Management, 19(3), 32–40. Byun, D. H. (2001). The AHP approach of selecting an automobile purchase model. Information and Management, 38, 289–297. Byun, D. H., & Suh, E. H. (1996). A methodology for evaluation EIS software packages. Journal of End User Computing, 8(21), 31. Conway, S., & Sligar, C. (2002). Unlocking knowledge assets: Knowledge management solutions from microsoft. Redmond: Microsoft Press. Davis, L., & Williams, G. (1994). Evaluating ansd selecting simulation software using the analytic hierarachy process. Integrated Manufactur Systems, 5(1), 23–32. Duffy, J. (2001). The tools and technologies needed for knowledge management. Information Management Journal, 35(1), 64–67. English, B. (2003). The administrator’s guide to sharePoint portal server 2001. Boston: Addison Wesley. eRoom, http://www.eroom.com. Expert Choice, 2000 Product Brief, http://www.expertchoice.com/productbrief/. Gallupe, B. (2001). Knowledge management systems: surveying the landscape. International Journal of Management Review, 3(1), 61–77. Giannetto, K., & Wheeler, A. (2002). Knowledge management toolkit: A resource for creating policy and strategy, with practical guidance for managing knowledge at all levels within the organization. Hampshire England: Gower. Hackett, B. (2000). Beyond knowledge management: new ways to work and learn. New York: The Conference Board. Hahn, J., Subramani, M. R. (2000). A framework of knowledge management systems: Issues and challenges for theory and practice. (pp. 302–312) Proceedings of the twenty first international conference on information systems. Holsapple, C. W., Joshi, K. D. (1999). Description and analysis of existing knowledge management frameworks. Proceedings of the Hawaiian international conference on system sciences. Holsapple, C. W., & Joshi, K. D. (2002). Knowledge manipulation activities: Results of a Delphi study. Information & Management, 39(6), 477–490. ICASIT KM Central, http://www.icasit.org/km/tools/index.htm. Kakabadse, N. K., Kakabadse, A., & Kouzmin, A. (2003). Reviewing the knowledge management literature: towards a taxonomy. Journal of Knowledge Management, 7(4), 75–91. Kim, C. S., & Yoon, Y. (1992). Selection of a good expert system shell for instructional purposes in business. Information & Management, 23(5), 249–262. Knowledger, http://www.knowledgeassociates.com. Lai, V. S., Trueblood, R. P., & Wong, B. K. (1999). Software selection: a case study of the application of the analytical hierarchical process to the selection of a multimedia authoring system. Information & Management, 36(3), 221–232. Lee, S. M., & Hong, S. (2002). An enterprise-wide knowledge management system infrastructure. Industrial Management & Data Systems, 102(1), 17–25. Liao, S. H. (2003). Knowledge management technologies and applications—literature review from 1995 to 2002. Expert Systems with Application, 25, 155–164. Mamaghani, F. (2002). Evaluation and selection of an antivirus and content filtering software. Information Management & Computer Security, 10(1), 28–32. Marwick, A. D. (2001). Knowledge management technology. IBM Systems Journal, 40(4), 814–830. 898 E.W.T. Ngai, E.W.C. Chan / Expert Systems with Applications 29 (2005) 889–899Mattison, R. (1999). Web warehousing and knowledge management. New York: McGraw-Hill. McCampbell, A. S., Clare, L. M., & Gitters, S. H. (1999). Knowledge management: The new challenge for the 21st century. Journal of Knowledge Management, 3(3), 172–179. Microsoft Sharepoint Portal Server, http://www.microsoft.com/sharepoint/. Min, H. (1992). Selection of software: The analytic hierarchy process. International Journal of Physical Distribution & Logistics Management, 22, 42–52. Muralidar, K., & Santhanam, R. (1990). Using the analytic hierarchy process for information system project selection. Information & Management, 18(1), 87–95. Murray G. (1999). The knowledge management factbook. International data corporation report. Mustafa, M. A., & Al-Bahar, J. F. (1991). Project risk assessment using the analytic hierarchy process. IEEE Transactions on Engineering Management, 38(1), 46–52. Nonaka, I. (1991). The knowledge creating company. Harvard Business Review, 69(6), 96–104. Nonaka, I. (1994). A dynamic theory of organization knowledge creation. Organization Science, 5(1), 14–37. Nonaka, I., Umemoto, K., & Senoo, D. (1996). From information processing to knowledge creation: A paradigm shift in business management. Technology in Society, 18(2), 203–218. Ossadnik, W., & Lange, O. (1999). AHP-based evaluation of AHP software. European Journal of Operational Research, 118(3), 578–588. Rubenstein-Montano, B., Liebowitz, J., Bchwalter, J., McCaw, D., Newman, B., & Rebeck, B. (2001). The knowledge management team, A systems thinking framework for knowledge management. Decision Support Systems, 31, 5–16. Ruggles, R. (1997). Knowledge management tools. Oxford: ButterworthHeinemann. Saaty, T. L. (1980). The analytic hierarchy process. New York: McGraw-Hill. Sahasrabudhe, V. (1999). Information technology in support of knowledge management. In T. K. Srikantaiah, & M. E. D. Koenig, Knowledge management for the information professional. Information today, 269–276. Salisbury, M. W. (2003). Putting theory into practice to build knowledge management systems. Journal of Knowledge Management, 7(2), 128–141. Schniedejans, M. J., & Wilson, R. L. (1991). Using the analytic hierarchy process and goal programming for information system project selection. Information & Management, 20(5), 333–342. Tam, M. C. Y., & Tummala, V. M. R. (2001). An application of the AHP in vendor selection of a telecomuunications system. Omega, 29(2), 171–182. Thierauf, R. (1999). Knowledge management systems for business. London: Quorum Books. Tiwana, A. (2002). The knowledge management toolkit. Upper Saddle River, NJ: Prentice Hall PTR. Tiwana, A., & Ramesh, B. (2001). Integrating knowledge on the Web. IEEE Internet Computing, 5(3), 2–9. Tsui, E. (2003). Tracking the role and evolution of commercial knowledge management software. In C. W. Holsapple (Ed.), Handbook on knowledge management 2: Knowledge directions (pp. 5–28). New York: Springer, 5–28. Tyndale, P. (2002). A taxonomy of knowledge management software tools: origins and applications. Evaluation and Program Planning, 25, 183–190. Udo, G. G. (2000). Using analytic hierarchy process to analyze the information technology outsourcing decision. Industrial Management & Data Systems, 100(9), 421–429. Wensley, A. K. P. (2000). Tools for knowledge management, http://www. icasit.org/km/resources/toolsforkm.htm. Zack, M. (1999). Managing codified knowledge. Sloan Management Review, 40(4), 45–58. E.W.T. Ngai, E.W.C. Chan / Expert Systems with Applications 29 (2005) 889–899 899