Technology and Investment
Vol.2 No.1(2011), Article ID:3987,9 pages DOI:10.4236/ti.2011.21005

Evaluation on Information Technology Service Management Process with AHP

Jiangping Wan1,2, Hui Zhang1, Dan Wan1

1School of Business Administration, South China University of Technology, Guangzhou, China

2Institute of Emerging Industrialization Development South China University of Technology, Guangzhou, China


Received November 9, 2010; revised January 10, 2011; accepted January 28, 2011

Keywords: Information Technology Infrastructure Library, Information Technology Service Management Process, Analytical Hierarchy Process, Customer Satisfaction, Knowledge Support Structure


Though analyzing the IT service management framework and associated processes of KM company, this paper determines various indicators of IT service management process. The weights of each indicator are calculated with analytical hierarchy process (AHP), and the evaluation was executed in KM company. And the suggestions are proposed with consideration of both the evaluation results and the real situation of KM company’s IT service management. The case is deeply considered in the views of IT service management objectives and misunderstandings, customer satisfaction model of IT service management, knowledge supporting structure of IT service management. It is concluded that if the organization wishes to successfully implement IT service management’s customer satisfaction model, it should eliminate the misunderstandings of IT service objectives first, while the adoption of knowledge supporting structure of IT service management can play a multiplier effect.

1. Introduction

Information Technology Service Management (ITSM) based on Information Technology Infrastructure Library (ITIL), which integrates the best practices of global IT management and forms the normative truth standard to reduce effectively cost and improve the quality of service, is applied widely in the world [1]. ITIL, which was developed by the Central Computer and Telecommunications Agency (CCTA, merged with the OGC) of the UK government in the middle of 1980s, is a set of service management standard library which focuses on IT industry. In the past 20 years, the content of ITIL has been updated and renewed. Today the OGC has enacted the ITIL 3.0.

ITIL emphasizes on managing IT service from organization business and customer desires, and realizes the integration of service management processes and organization business. The module of service management plays a role of “glue” which makes an organic connection between the business perspective and the technology. The core module of ITIL, service management, consists of ten core processes and an ITSM function. Service support and service delivery are considered to be at the heart of the ITIL framework for ITSM. Service delivery describes the services that the customer needs to support their business, and what is needed to provide these services; service support describes how a customer can get access to the appropriate services to support their business.

This paper is organized as follows: Firstly, IT service management of KM company is introduced. Secondly, its IT service management processes are deeply analyzed, then IT service management process assessment model is proposed based on Analytical Hierarchy Process (AHP) and assessment is made. Finally, conclusions are put forward according to the results and the thinking on case study.

2. KM Company and Its IT Service Management

KM is a professional service company with a global network, specifically providing financial audit tax and consulting services. KM company has more than a dozen subsidiaries in Beijing, Tianjin, Daian, Shanghai, Nanjing, Chengdu, Hangzhou, Guangzhou, Changsha, Shenzhen, Hong Kong SAR and Macao SAR, with over 8,000 professionals. KM’ IT department is an independent department, constituting the company’s basic structure with personnel, administration, marketing, training, and finance department. IT department’s main responsibilities include planning, constructing and maintaining company’s hardware and software of computer system, developing IT system’s security policies, monitoring the company’s network applications, backup company’s data and planning and procuring computers, printers and other consumable goods for the company.

Based on the ITIL model, through studying and understanding ITIL, and enterprise’s own practice for IT service management, KM company develops processes completely on its own and continuously improves them. KM company’s IT service management implementation is divided into three phases: 1) To implement IT helpdesk and service level management. 2) To implement emergency management, problem management and change management. 3) To implement configuration management, cost management, availability management and IT service continuity management.

The scope of IT application management service that KM company provides, covers the user’s application system, database performance and monitoring, data backup and so on, such as: 1) Application service management: Incident Management, Problem Management, Configuration Management, Change Management and Knowledge Management. 2) Technical support: Configuration Installation, Service Upgrading, Application Incident Management, Ordinary Onsite Maintenance and Operation Supporting Service. 3) Monitoring management: Monitoring Service and Application Performance Management Service.

3. Analysis on KM Company’s IT Service Management Processes

This section analyzes KM company’s IT service management process in the views of customer satisfaction management, operation maintenance service process, IT service financial management, service level management and IT service implementation management process. It is believed that all IT service management processes should be aimed at enhancing customer satisfaction.

3.1. Customer Satisfaction Management of KM Company

Customer satisfaction management is an organizational behavior that enterprises try to understand customer satisfaction through investigating, analyzing and researching, based on this, we identify the effect factors and continuously improve it. KM’s solutions are: 1) Developing the principles of customer satisfaction management from the concept level. 2) Formulating strategies from the technical aspect to improve customer satisfaction. 3) Focusing on customer value, and promoting satisfactory experience delivery. 4) Shortening service-response time, and providing continuous service. 5) Optimizing service cycle and saving cost.

3.2. Operation Maintenance Service Process of KM Company

The key roles of KM company’s application system operation maintenance service team are as follows: 1) Operation director, who is responsible for controlling the quality of services, maintaining customer satisfaction, timely responding to users’ new needs and reporting the situation of operation maintenance to the management. 2) Operation manager, who is responsible for managing the operation maintenance services team, keeping in touch with every department’s heads, and ensuring to complete the operation maintenance support service with highquality. 3) Operation maintenance service support team, including on-site operation maintenance service and support team and remote support team, who grantee the stable operation of users’ application system together. The operation maintenance service processes mainly include service team, service process, management tool, fault prevention, emergency response system and so on.

3.3. IT Service Financial Management of KM Company

KM company’s IT service financial work includes defining the cost elements, and further subdividing the cost project. Define cost element structure in terms of service, that is, identify the cost based on determining the service element structure, distribute them to every service and sum up respectively. After determining the relationship between cost elements structure and service elements structure, service-based cost model would be established.

3.4. IT Service Level Management of KM Company

According to customer’s requirements and actual situation, KM company develops the service level indicators together with the customers. After reaching consent on the service level, it will become one part of the contract, and is the basis of measuring the quality of operation maintenance outsourcing services.

3.5.  IT Service Implementation Management Process of KM Company

KM company will gradually track and handle all customer service content electronically with special tools and customers’ information will be stored in the form of database or table. The service processes will be regularly checked and tracked, and continuously optimized in order to improve service efficiency and quality, including incident management, problem management, change management, configuration management, helpdesk management, knowledge management and other processes.

4. IT Service Management Process Assessment System of KM Company

4.1. Analytical Hierarchy Process

AHP was proposed in the mid 1970’s by T. L. Saaty, a U.S. operational research expert and professor at the University of Pittsburgh. Its feature is based on deeply analyzing the essence of complex decision-making problem, influencing factors and their intrinsic relationship, using little quantitative information to achieve mathematics of decision-making process to provide simple method for complex decision-making problems with multiobjective, multi-standard or non-structural properties [2].

There are five steps in AHP: 1) To clarify the scope, policies to be adopted, guidelines for achieving objective, strategy and various constrains involved in decision planning, and determine the general goal by deeply understanding the system. 2) To establish a multi-level hierarchical structure, and to divide the system into several hierarchical levels in terms of different objectives and different functions. 3) To define the correlation degree between the elements of adjacent-level in the hierarchical structure by constructing the comparison matrix and matrix operation of mathematical method. 4) To calculate the elements’ synthesis weight in each level for system target, and then sort totally, in order to determine the importance of the various elements at the bottom of the hierarchical structure to the general goal. 5) To consider the corresponding decision-making, according to the analysis results.

4.2. Evaluation Model

According to the service level agreement of KM company’s IT service management, this paper uses the method of two-way communication with the experts. Firstly, the indicator system is designed with through repeatedly consulting experts, the key performance indicators of KM company’s IT service management process framework are developed at last. There are 7 first-grade indicators (Table 1), namely configuration management (configuration database update frequency, the percentage of unauthorized use facilities and the percentage of correctly updated configuration items), service level management (the service percentage involved in service level agreement, the monitoring and return frequency of service level agreement, the user’s satisfaction degree with the agreement involved in service level agreement, the monitoring and return frequency of service level agreement, the user’s satisfaction with the agreement, the percentage of service reaching defined service level, the percentage of approving a new service with less than 30 days and the percentage of updating the service level agreement with less than 30 days), help desk management (help desk staff’s training time, the number of customer complaints, the percentage of delayed response to service request, the percentage of service request records being correctly described, the percentage of the service request status being updated timely and the percentage of correct record of help desk personnel), emergency management (the percentage of emergency incident being resolved within the time defined by service level, the percentage of problem management processes triggered by emergency incident, the percentage of the emergency incident recorded by help desk, the percentage of emergency incident being resolved within the time defined by service level, the percentage of problem management processes triggered by emergency incident, the percentage of the emergency incident recorded by help desk and the percentage of emergency incident recurring), change management(improving the efficiency of dealing with change requests, the number of emergency caused by change, the percentage of successful implementation of change requests, user testing passing rate within the specified time, the delayed time of change implementation and the percentage of complete change document), problem management (improving problem solving efficiency, the percentage of known problems, the number of repeating problems and the increasing percentage of problem numbers after emergency incident occurred) and IT services financial management (reasonable cost analysis, the accuracy of measuring IT service delivery costs and the timeliness of payments), and there are 32 secondary indicators.

4.3. Determine the Weight

First, this study invites 10 experts, including the IT director, technical manager, IT information technologyspecialists, system pre-designer and users, to score the judgment matrix, and then calculates the average value, constructs judgment matrix. For the convenience of calculation, the average value is round number.

Table 1. KM Company’s IT service management process assessment indicator system.

The judgment matrix of first-grade indicators is illustrated in Table 2" target="_self"> Table 2. The Expert Choice software is used to calculate the weight, getting the Results Configuration Management (U1, 0.137), Service Level Management (U2, 0.244), Help Desk Management (U3, 0.334), Emergency Management (U4, 0.075), Change Management (U5, 0.061), Problem Management (U6, 0.050) and IT Service Financial Management (U7, 0.099). Finally, the consistency test is done, the CR = 0.07 < 0.1, therefore, it can be concluded that the matrix is acceptable. The weights of 32 secondary indicators are worked out by calculating their corresponding weight respectively (Table 3).

5. KM Company’s IT Service Management Process Assessment

5.1 Assessment Implementation

Invite the KM company’s core staff to assess the configuration management, service level management, service desk management, emergency management, change management, problem management and configuration management, and finally obtain the evaluation value (Table 4). It is found that the most concerned areas in the IT service management are help desk management, configuration management and service level management, so that we can determine priorities for project implementation.

5.2 Analysis of Assessment Results

With assessing IT service management and comparing traditional IT work pattern with new ITIL management style, we can reach the following suggestions for improvement: 1) IT work pattern will become active mode from the traditional passive mode: The computer problematic events are presented by the user, and help desk

Table 2. First-grade indicators judgment matrix.

Table 3. The result of IT service management process assessment.

allocates tasks according to each IT worker’s ability, establishing resolution with IT. The original unordered passive mode was gradually transformed to ordered active way. 2) To avoid malicious use of IT resources: Traditional IT work pattern allows for user’s malicious use of IT resources. After the implementation of ITIL, it will help to avoid malicious use of IT resources. 3) To reduce the loss of events: Under the traditional mode, the event would be recorded by each IT staff member, which easily causes the loss of incidents. Implementation of ITIL will let help desk track and record the entire event process, ensuring practical resolution of users’ problems. 4) To share IT resources: Some staffs are scattered, which leads to ineffective improve efficiency and reduce costs: After the situation, thus companies have a wide scope of IT management, and share of resources among IT staff and finally causes that some IT staff are busy chronically, while other IT staff cannot play their role. Once ITIL is implemented, help desk will allocate computer events to every area’s IT staff according to actual the IT resources can be shared. 5) To implementation of ITIL, IT resources will be fully shared, so the existing IT resources can cover a larger area, at the same time realizing the goal of improving efficiency and reducing IT costs. 6) To maintain a stable system and provide the necessary platform for KM business: The implementation of ITIL will enable the KM company to plan the network platform from the perspective of management, and to reach a stable state.

6. Thinking on IT Service Management

6.1. The Goals of IT Service Management

From the perspective of marketing development, the goal of IT service management is to change the past random behavior completely and achieve the service process standardization with ITIL methods (Table 5).

IT service companies in China have many misunderstandings on IT service management assessment, such as: 1) They believe that the company’s IT service management is just customer relationship management, and all problems can be handled if relationships with customers are improved. IT service business is closely related to the quality of services, however many project managers and ordinary employees mistaken service management as public relation management. The biggest risk of IT service implementation is the assessment of service quality. If an enterprise fails to establish service evaluation system, thus there is no corresponding service assessing standards, which will eventually lead to loss of customers. 2) IT service management assessment is the customers’ duty, and IT service management assessment costs are

Table 4. Weight of all indicators.

Table 5. The goals of IT service management.

high. IT service providers should take the initiative to set up their own service evaluation system, evaluating the IT service management process provided from the customer’s perspective, in this way, the company can remain invincible position in the marketplace. 3) IT service management assessment is needed when IT service maturity is evaluated. The ability of IT service to provide service management for enterprise is not formed after IT external evaluation, but improved during the processes of enterprise continuously improving and restraining itself.

6.2. The Customer Satisfaction Model of IT Service Management

The ITSM architecture model can classified to six different levels, each considering one or two the most concern problems for services and determine the priorities of ITIL modules basing on the analysis of the actual management situation. It can be illustrated in Table 6.

With case study and experience learnt from information systems [3-5], and Chinese ITIL best practices (e.g., Guangzhou Earth Village Corp. and YuWei Corp., and CEPREI certification body etc.), three processes are extracted in the following: namely “Service Level Management”, “Event Management” and “Problems Management”, and a function module, namely “ Help Desk Management”, integrating the service concept of ITIL V3 and clarifying that the “Customer Satisfaction Management” guide the overall specific operations. Note that the synergy and interdependence of the above five basic elements collectively figure out the key to the ITSM processes. These five major processes are interrelated and have influence on each other, if the company only focuses on a certain level of the five basic processes, while ignoring the impact of other processes, then the efficiency of such service management implementation will often be compromised. On the other hand, any one business process design can not be finished in one step. For this reason the implementation process of service management process can embody the principles, which include to be controlled, managed, assessed and implemented, and these processes indeed penetrate into the actual service process, “Self-Improving Process” and “Knowledge Management” which are two sets of control functions should be added to the model to ensure the sustainable development of the management model. It can be illustrated in Figure 1.

6.3. ITSM Knowledge Support Structure

In our understanding, ITSM solution implementation is essentially complicated knowledge transfer, and the successful knowledge management experiences of software process improvement (SPI) [6,7] could be applied to the operation and maintenance management. For examples, integrated operation and maintenance system of state tax e-Government system of ShenZhen City implements the transition from “technology-focused” to “process-focused”, and from the role of a passive “emergency treatment doctor” to the role of an active “health protection doctor”. It also disentangles IT administrators from frequent, repetitive maintenance jobs to improve their work efficiency, and makes them have time to reflect how to use IT to improve service. SLA defines the responsibility, rights and duties of both sides. When asking for service, SLA can responses in accordance to service priority and submits it to the corresponding administrator to dispose.

ITSM knowledge support structure is put forward in Figure 2 [8]. ITSM administrators need merge orient management philosophy with west management philosophy, in accordance with Chinese situation, and learn the lessons from the best practices to design the suitable ITSM solution. When the implemental standards and methods are established, managing three elements of ITSM, “Technology, Process and People”, just accords with the unification of three parts, “Sky, Earth and People”, and obeys to the systemic methodology thoughts of “Wu-Li Shi-Li Ren-Li”. Rule is “Sky”, to obey the requirement of theory of ICT; real world is “Earth”, to accord with the logic of management of ICT; people desire is “RenLi”, to deal with the relationship between application of ICT with the benefit-related people in the organization.

The theory includes service support, service delivery, business perspective, ICT infrastructure management and applications management. IT governance has five critical decisions [9]: IT principle decision, IT framework decision, IT infrastructure decision, IT investment and priority order, and business application decision. ITSM theory involves the three phases of business values creation by Son Pham, IBM Corp. [10]: component management, application management and ITSM; and the four phases of IT values creation by Kelbrick, HP Corp. [11]: technology-centric, service-centric, business-centric and customer-centric.

Table 6. Process priorities checklist.

Figure 1. ITSM architecture model for customer satifaction.

Six key successful factors in ITIL implementation in China by Mr. Zhao Guoxiang are in the following [12]: 1) Top leaders. 2) Customers. 3) Achievement. 4) Cognition. 5) Quickly Win. 6) Tools. Methodology includes several kinds of models and standards, such as BS15000, ISO20000, COBIT, ITSM Maturity Model and eSCM-SP (eSourcing Capability Model Service Providers by IT Sqc of CMU). Method involves ITIL, CIOBOK and SWEBOK, such as the guidelines of CIO knowledge body and CIO Wisdom [13-15].

Although some standards have been formed based on the theory related to IT management and the best practices, ITIL is just explains what to do to manage optimally IT operation and maintenance, and don’t tell you how to do it. Some companies develop the ITSM implemental solution, such as ESM of CA, ITPM of IBM; and explore the relative software products, such as Unicenter of CA, Tivoli of IBM. In the real world, enterprise should select the implemental modules and the implemental order that are suitable for itself in accordance with the situation of enterprise and real world. In the first step, enterprise could realize service support on the operation layer, and realizes service delivery on the tactics layer, firstly, the enterprise should analyze ITSM requirement, including technology, process and people. Secondly, IT infrastructure requirement, which mainly is the systemic integration of ICT, is refined. Finally, the ITSM implementation is essentially ITSM knowledge, including explicit knowledge and implicit knowledge, transformation to reach the objective of improving the organizational ITSM capability.

Operation and maintenance management of e-Government based on ITIL would largely reduce the complexity, and help government to avoid the repeated investing on IT management resource, and improve the quality of service, and promote the satisfaction degree of customers. In the future, ITSM researchers need further practice to improve the ITSM knowledge support structure for both e-Government and e-Business, even all the operation and maintenance management of information system. In additional, when defining software requirement, managers should consider both their values, and the cost of operation and maintenance [16].

Figure 2. ITSM knowledge support structure.

7. Conclusions

With analyzing the IT service management framework and associated processes of KM company, this paper determines various indicators of IT service management process. The weights of each indicator are calculated with AHP, and KM company is evaluated. And some reasonable suggestions are proposed with consideration of both the evaluation results and the actual situation of KM company’s IT service management. In the end, the case is deeply considered in the views of IT service management objectives and misunderstandings, customer satisfaction model of IT service management, knowledge supporting structure of IT service management. It is concluded that if the organization wishes to successfully implement IT service management’s customer satisfaction model, it should eliminate the misunderstandings of IT service objectives first, while the adoption of knowledge supporting structure of IT service management can play a multiplier effect. For example, operation maintenance management should be considered during the design stage of IT service management, which requires clear and profoundly understanding of the four development stages of IT service management: technology-centric, service-centric, business-centric and customer-centric.

8. Acknowledgements

Thank for helpful discussion with Mr. Zhuo Zhixin, Mr. Cai hua, Mr. Chen Ji, Mr. Zhao Guoxiang, and Zhao Xi etc..

9. References

[1]       Zaltbommel, “Foundations of IT Service Management based on ITIL,” Van Haren Publishing, 2005.

[2]       T. L. Saaty, “The Analytic Hierarchy Process,” New York: McGraw-Hill International, 1980.

[3]       Dean Lane, “CIO Wisdom: Best Practices from Silicon Valley’s Leading IT Experts,” Prentice-Hall, Inc, 2004.

[4]       C. C. Chen, C. Law and S. C. Yang, “Managing ERP Implementation Failure: A Project Management Perspective,” IEEE Transactions on Engineering Management, Vol. 56, No. 1, January, 2009, pp.157-170. doi:10.1109/TEM.2008.2009802

[5]       D. Severance and J. Passino, “Make It Work,” John Wiley & Sons, 2002.

[6]       Brooks and P. Frederick, “No Silver Bullet: Essence and Accidents of Software Engineering,” IEEE Computer, Vol. 20, No. 4, April, 1987, pp.10-19.

[7]       J. P. Wan, “Research on Software Product Support Structure,” Journal of Software Engineering and Applications, Vol. 2, No. 3, October, 2009, pp.174-194. doi:10.4236/jsea.2009.23025

[8]       J. P. Wan, Y. F. Wang and C. W. Zheng, “Research on Knowledge Support Structure of IT Service Management,” IEEE WiCom 2007, Shanghai, China, September, 2007, pp. 6729-6733.

[9]       P. Weill, “IT Governance: How Top Performers Manage IT Decision Rights for Superior Results,” Harvard Business School Press, 2004.

[10]    Son Pham, “Pulling it All Together: How to Implement ITSM,” ITSM.ppt, 2007.

[11]    C. Kelbrick, “ITSM Road Map, services/downloads/hp_itsm_work-shop.pdf,” 2007.

[12]    G. X. Zhao, “ITSM Improvement and Promotion, http: //,” 2009.

[13]    D. Lane, “CIO Wisdom: Best Practices from Silicon Valley’s Leading IT Experts,” Prentice-Hall, Inc, 2004.

[14]    M. Y. Zuo, “The Guidelines of CIO Knowledge Body,” Beijing Chian, Publishing House of Electronics, 2004, pp.301-425.

[15]    P. Bourque, R. Dupuis, A. Abran, James W. Moore and L. Tripp, “The guide to the Software Engineering Body of Knowledge,” IEEE Software, Vol. 16, No. 6, November/December,1999, pp.35-44

[16]    J. Azar, R. K. Smith and D. Cordes, “Value-Oriented Requirements Prioritization in a Small Development Organization,” IEEE Software, Vol.24, No.1, January/February, 2007, pp. 32-37. doi:10.1109/MS.2007.30


This research was supported by Key Project of Guangdong Province Education Office (06JDXM63002), NSF of China (70471091), and QualiPSo (IST-FP6-IP-034763)