Journal of Geographic Information System
Vol.07 No.04(2015), Article ID:58359,19 pages
10.4236/jgis.2015.74026

A Model for Measuring Geographic Information Systems Success

Khalid A. Eldrandaly, Soaad M. Naguib*, Mohammed M. Hassan

Information Systems Department, Faculty of Computers and Informatics, Zagazig University, Al-Sharqiyah, Egypt

Email: Khalid_eldrandaly@zu.edu.eg, *so3ad_mn@zu.edu,eg, monirhm2002@yahoo.com

Copyright © 2015 by authors and Scientific Research Publishing Inc.

This work is licensed under the Creative Commons Attribution International License (CC BY).

http://creativecommons.org/licenses/by/4.0/

Received 21 June 2015; accepted 23 July 2015; published 28 July 2015

ABSTRACT

Geographic Information Systems (GIS) have become a fact of our life as they are being used by more people and organizations for more complex decision problems than ever before. The use of GIS can achieve valuable benefits for individuals, organizations and society; however, the achieve- ment of these benefits depends on the success of GIS. While information systems (IS) success models have received much attention among researchers, there is a general scarcity of research conducted to measure the GIS success. This paper proposes a success model for measuring GIS success by extending and modifying previous IS success models. The developed success model consists of two main levels: GIS project diffusion success, and GIS post-implementation success. The first level identifies the critical success factors (CSFs) that influence the success of GIS adoption at each stage of the diffusion process. The second level of the proposed model identifies and organizes the success dimensions (outcome measures) of GIS in temporal and causal relationships. In order to assess the relationships among the success dimensions, 11 hypotheses were tested. Data were collected through a questionnaire that was distributed to 252 GIS users/managers in Egypt and abroad. The empirical results support 6 hypotheses and reject 5 hypotheses.

Keywords:

IS Success Models, GIS Success Model, Critical Success Factor, Model Validation

1. Introduction

Geographic Information Systems (GIS) are a mainstream technology with a vital and growing use across all industries [1] . A GIS is a computer-based information system that enables capture, modeling, storage, retrieval, sharing, manipulation, analysis, and presentation of geographically referenced data [2] . A working GIS integrates five key components: hardware (the equipment needed to support the many activities of GIS ranging from data collection to data analysis and sharing), software (different GIS software packages for creating, editing and analyzing data), data (the core of any GIS, categorized as spatial and non-spatial data), organizational structure and people (well-trained and skilled people to use and maintain the GIS) and methods (well-designed plan and business rules that are the models and operating practices unique to each organization) [3] . GIS are well established as giving competitive advantage and enhancing organizational decision-making in a wide array of functions, including improved information sharing and flows, better informed decision making, stronger competitive ability, greater analysis and understanding of problems, justification for decision made, improved visualization of data, cost saving, increased effectiveness, and better quality output [4] . However, the achievement of these valuable benefits depends on the success of GIS. Although measuring traditional IS success has been a major topic in IS research, there is a scarce of research addressing GIS success. This paper accordingly attempts to propose a comprehensive multidimensional success model for GIS and to empirically investigate the multidimensional relationships among the success measures. The validated GIS success model can provide GIS managers with a useful framework for evaluating GIS success.

This paper is structured as follows. First, we review the development of IS success models, and consider the challenges and difficulties facing these IS success models. Second, based on prior studies, a GIS success model and a comprehensive set of hypotheses are proposed. Third, the methods, measures and results of the study are presented. And, finally, the results are discussed.

2. IS Success Models

Information system (IS) success is one of the most researched topics in IS literature [5] . Measuring IS success is believed to be a critical issue in the IS field. Several studies have been conducted, and considerable attention was paid to this issue; due to the amounts of money, time and efforts spent on IT/IS projects.

The IS literature provides several definitions and measures of IS success. As [6] stated that, although there are nearly as many measures as there are studies; obviously, there is no ultimate definition of IS success. The definition of IS success may vary in respect of different types of IS that yield different benefits for individuals, workgroups and organizations [7] .

2.1. The Delone and Mclean IS Success Model (1992)

A wide range of research has proposed IS success models [6] [8] -[10] . These models postulate their own definitions of IS success and factors that affect the defined IS success, however, a first synthesis of the manifold perspectives on IS success and its underlying antecedents was achieved by DeLone and McLean (1992) [6] , who developed an IS success model. This model has received much attention from IS scholars and can be regarded as one of the most prominent and influential models in IS research [11] . DeLone and McLean (1992) [6] made a major breakthrough. They conducted a comprehensive review of IS success literature published between 1981- 1987 in seven journals in the IS field, and proposed a model of IS success shown in Figure 1.

2.2. The 3-D Model of IS Success

Since 1992, a number of studies have undertaken empirical investigations of the multidimensional relationships among the measures of IS success such as [12] [13] . In addition to studies that have tested and validated the

Figure 1. DeLone and McLean model (1992).

DeLone and McLean IS success model (1992), several studies have been published that challenge, critique, or extend the model itself. On balance, these articles have contributed to a better understanding of IS success such as Ballantine et al. (1996) [18] .

Ballantine et al. (1996) [18] assessed the DeLone and McLean model (1992) from a number of different viewpoints and a new model, the 3-D model, has been proposed. First, the independent variable or the critical success factors, factors that may cause success rather than being part of success, have been incorporated to the 3-D model. Second, the causal relationship that exist between the success dimensions in DeLone and McLean model (1992) has been criticized arguing that, success in one dimension does not lead directly to success at the next dimension. If this were so, then one need only be successful at system quality to ensure the success of the whole IS. Finally, Ballantine et al. (1996) [18] mentioned that DeLone and McLean (1992) [6] provide a unidirectional model which moves from system and information quality to organizational impact of IS, the impact upon organizational learning is not discussed. The implication in the DeLone and McLean model (1992) is that success is necessary at each stage otherwise the next stage will not be successful. It is not sufficient to move only in one direction. Unless individuals and the organization can learn from experience and develop better systems and recognize better information quality then it is unlikely that the measurement of IS outcome will serve any useful purpose. Thus, Ballantine et al. (1996) [8] depict the importance of organization learning through time and experience in their model through the learning feedback loop as shown in Figure 2. The 3-D model separates success into three fundamental levels: the technical development level, the deployment to the user, and the delivery of business benefits. Hence it is termed the 3-D model as shown in Figure 2. In the 3-D model, filters act between the levels of IS success and contains influences which inhabit or encourage the adoption of the IS at the next higher level. Success at each level is influenced by a number of different independent variables and the outcomes of each level (development, deployment, and delivery) are the closest equivalent to dependent variable (outcome measures of IS success).

Figure 2. The 3-D model of IS success.

2.3. The Update DeLone and McLean Model (2003)

Based on suggestions offered by researchers, and criticisms directed to the DeLone and McLean original model, DeLone and McLean (2003) [9] proposed an updated version of their IS success model (1992) and evaluated its usefulness in light of the dramatic changes in IS practice, especially the advent and explosive growth of e- commerce. The updated model is shown in Figure 3. The primary differences between the original and updated models include:

1) The addition of service quality to reflect the importance of service and support in successful e-commerce systems;

2) The addition of intention to use to measure user attitude;

3) The collapsing of individual impact and organizational impact into the net benefits construct.

The categories of the updated taxonomy were system, information, and service quality, intention to use, use, user satisfaction, and net benefits. DeLone and McLean models (1992, 2003) could serve as a basis for the selection of appropriate IS measures. Researchers had to choose several appropriate success measures based on the objectives and the phenomena under investigation, as well as consider possible relationships among the success dimensions when constructing the research model [14] .

2.4. Challenges and Difficulties Facing the IS Success Models

While the updated DeLone and McLean model (2003) is a comprehensive IS success model, it suffers from certain difficulties. First, the Net Benefit measure in the model is conceptually too broad to define. As [15] suggests, “The new net benefits construct immediately raises three issues that must be addressed: what qualifies as a benefit? for whom? and at what level of analysis” (p. 32). Thus, when using the updated DeLone and McLean model, researchers need to clearly and carefully define the stakeholders and the context in which Net Benefits are to be measured.

Second, as DeLone and McLean (1992) [6] stated “The selection of IS success measures should also consider the contingency variable such as the independent variable being researched: the organizational strategy, structure, size, and environment of the organization being studied; the technology being employed; and the task and individual characteristics of the system under investigation” (p.88). Thus, DeLone and McLean (1992) [6] recognized the limited perspective of their model. They identify only the dependent variables (outcome measures) of IS success.

While the 3-D IS success model represents a holistic view of the concept of IS success. This model did not focus on the outcome measures of IS as DeLone and McLean (1992, 2003) did. Although Ballantine et al. (1996) [8] incorporated the critical success factors (independent variables) to their success model, the outcome measures of IS success are unclear and misspecified.

Based on the above mentioned literature, this study proposes a new GIS success model by extending and respecifying both the 3-D model of IS success and the updated DeLone and McLean model (2003) in the context of GIS.

Figure 3. DeLone and McLean model (2003).

3. Research Model and Hypothesis

The literature search indicated that, there is a general scarcity of models and frameworks for measuring GIS success. However, there are some frameworks that were developed for evaluating the contributions of GIS to efficiency, effectiveness, and societal well being (see e.g., [16] - [19] ).

By extending and respecifying both the 3-D model and the updated DeLone and McLean model (2003) of IS success, we proposes a GIS success model that provides a holistic view of GIS success concept via defining GIS success as a cumulative process that starts from initiating successful GIS projects and ending with the success of GIS in delivering their business objectives. Thus, we divide success into two levels: (1) GIS diffusion success, and (2) GIS post-implementation success, as shown in Figure 4.

Figure 4. The proposed GIS success model.

3.1. Level One: GIS Diffusion Success

This level extends and respecifies the 3-D model of IS success developed by Ballantine et al. (1996) [8] to GIS context by identifying the CSFs in each stage of GIS diffusion process (instead of development, deployment, and delivery levels in the 3-D model) that influences the success of GIS project adoption at each stage. The concept of CSFs was popularized in the context of IS and project management (PM) by [20] as factors affecting the success of activities and projects. CSF is defined by [21] as “the limited number of areas in which satisfactory results will ensure successful competitive performance for the individual, department, or organization. CSFs are the few key areas where ‘things must go right’ for the business to flourish and for the managers goal to be attained” (p. 385). CSFs make it easier for managers to prioritize vital aspects of a project [22] .

According to [23] diffusion is “the fundamental process that is responsible for the transfer of innovations from the workshops of their inventors to becoming a daily part of the lives of a large section of society”. GIS project diffusion can be classified into three main stages [24] - [27] : (1) initiation (pre-implementation), (2) acquisition (implementation), and (3) incorporation (post-implementation). Detailed descriptions of these stages and the steps carried out in each one are reported elsewhere (see, e.g., [28] [29] , for an excellent overview). The main objective of the first level of the proposed GIS success model is to identify the CSFs in each stage of the GIS project diffusion that influence the success of GIS project adoption at each stage as shown in Table 1. Some CSFs may work at more than one stage. The CSFs identified in the three stages of GIS diffusion process were extracted from:

1) GIS success research cited in the literature which is mostly based on case studies or observations of GIS projects and practices, such as [30] - [32] ;

2) GIS failure research which is based on lessons learned from certain types of GIS projects, but they are mostly similar enough to be generalized, such as [33] - [35] ;

3) Researches about GIS implementation that mentioned CSFs briefly such as [36] - [38] .

Table 1. GIS CSFs according to their occurrence in GIS diffusion stages.

3.1.1. The Initiation Stage (GIS Concept Introduction and Funding Commitment)

The objective of this stage is to learn about GIS technology, explore the appropriateness of GIS for the organization, and to gain official sanction for the next stage (acquisition). The result from this stage reveals if the organization is ready to accept GIS. This stage includes the following CSFs: organization culture, organization structure, clear goal and vision, top management support/awareness, and the external environment [24] .

3.1.2. The Acquisition Stage (GIS Needs Analysis, Design, and Installation)

The acquisition stage begins when the organization becomes aware of the GIS and decides to adopt it, and when GIS advocates have confidence that financial and management support exist to establish a budget. In this stage, the organization engages in the activities necessary to put the GIS into practice [26] [29] . Planning, system design, development, installation and operating the system are the objectives of this stage. This stage includes the following CSFs: strategic planning, skilled staff, communication channels, user participation, user experience, education and training, process re-engineering, hardware and software selection, data issues, software customization, and top management support.

3.1.3. The Incorporation Stage (GIS Acceptance and Use)

The incorporation stage focuses on the acceptance of the technology by members of organization and its utilization over time [39] . All operational GIS installations must move into this third stage if the promises of GIS technology are to be realized. The objective of this stage is to use GIS technology for the daily tasks and decision making required by the organization [24] . This stage includes the following CSFs: perceived usefulness, vendor support, user skills and computer experience, task characteristics and top management support.

The stages of GIS diffusion process are represented in the proposed success model with respect to their occurrence. Hence, the arrows between stages of GIS diffusion process represent temporal relationship.

During the achievement of GIS diffusion success, as shown in Figure 4, there is an issue of learning through time and experience which is suggested by Ballantine et al. (1996) [8] in his IS success model. Thus, many factors which have an effect on the success of the three stages of GIS diffusion, will be probably affected by previous experience and the maturity of the organization in developing IS projects. Furthermore, if the organization is a learning organization, it will have in place procedures and people who monitor the success and adapt changes in the diffusion processes in order to achieve success. This is represented by the learning feedback loop in Figure 4.

3.2. Level Two: GIS Post-Implementation Success

The second level of the proposed model extends and respecifies the updated DeLone and McLean model (2003) to the GIS context via the following steps:

1) Replacing service quality dimension (that reflect the importance of service and support in successful e-com- merce systems) with user quality. As GIS is not solely technical in nature, adding user quality dimension as a part of the process of producing geographic information is very important. GIS user quality is considered an important human factor in a successful GIS as mentioned in many GIS researches [62] ;

2) According to [15] , researchers need to clearly and carefully define the stakeholders and the context in which Net Benefits are to be measured. Therefore, we split the net benefits dimension to net benefits to individuals, organization, and society;

3) DeLone and McLean IS success models (1992, 2003) are built upon the taxonomy developed by both Shannon and Weaver (1949) [64] and Mason (1978) [65] , which considered that a message in a communication system can be measured at different levels including: the production level, the product level, and the influence level. The production level is the accuracy and efficiency of the system which produces the information. The product level is the success of the information in conveying the intended meaning. The influence level is the effect of the information on the receiver. The influence level is presented as a hierarchy of events taking place at the receiving end of an information system; these events are receipt of the information; influence of the information on the receiver; and the influence of the information on the performance of the system. The concept of levels of output from communication theory demonstrates the serial (process) nature of information (i.e., a form of communication). The IS creates information which is communicated to the recipient who is then influenced (or not) by the information. In this sense, information flows through a series of stages from its production through its use or consumption to its influence on individual and/or organizational performance [6] . Therefore, to reflect the interdependent and the process nature of the success dimensions that was suggested by DeLone and Mclean in their models (1992, 2003), we believe that system quality (which measures the success of the production level) and information quality (which measures the success of the product level) should not happen in parallel as DeLone and McLean (1992, 2003) did in their models. Instead, information quality should come after the system quality dimension to reflect the interdependent and the process nature of the success dimensions;

4) To avoid model complexity and to reflect the cross-sectional nature of this study, the feedback links from net benefits to both use and user satisfaction in the updated DeLone and McLean model (2003) were excluded.

The success dimensions of the proposed GIS success model are system quality, user quality, information quality, information use, user satisfaction, net benefits to individuals, net benefits to organization, and net benefits to society as shown in Figure 4. The arrows between success dimensions represent temporal (process) and causal relationships. The following section will discuss the GIS success dimensions.

The second level of the proposed model is concerned with measuring the GIS success after incorporating and adapting the GIS into the organization's operations. The GIS success should be measured after a wide spread of use to allow the members of the organization to arrive at informed opinions about the success of their GIS [29] . Hence it is termed GIS post-implementation success.

Model Construct

In this paper, according to previous researches on GIS, measures of the proposed model have been determined (see Table 2).

Ÿ System quality: system quality dimension measures the success of the technical aspects of GIS. System quality has been represented in many GIS studies by functionality, response time, system reliability, user friendless, error recovery, database content.

Ÿ User quality: user quality dimension represents the quality of GIS users in terms of spatial abilities and self efficacy. In IS field, Bonner (1995) [66] revised the DeLone and McLean model and introduced user quality in terms of knowledge skills and abilities. His recognition of the people element was a welcome addition to the model. Also, the recognition of the importance of human factor in evaluating GIS performance was first initiated by [67] , who stated that people, not the computerized equipment, make a GIS success or fail.

Ÿ Information quality: information quality dimension is the quality of information provided to the organization using GIS, in the form of maps, tables, charts, and reports. The information quality dimension measured by accuracy, completeness, ease of interpretation, relevancy, reliability, timeliness, and clarity.

Ÿ Information use: information use is a broad construct that is frequently used in measuring the utilization of IS. Information use dimension measures to what extent the GIS output is being used in the decision making process. Clapp et al. (1989) [18] mentioned that, the system can provide the capability to obtain all desired information, but for some reasons, the information is not used in the decision process whether private or public. In this case, the GIS will fail due to lack of utilization. Use can be based on objective measures such, number of functions used, frequency of access, and amount of connecting time [68] . Questions about who uses the system, levels of use, motivations for and voluntariness of use, and the purpose and nature of system use are also relevant [67] .

Ÿ User satisfaction: this dimension measures GIS user’s level of satisfaction with the system. User satisfaction was traditionally employed as the most common measure of IS success. The most widely used user satisfaction instruments are End User Computing Support (EUCS) [69] instrument and User Information Satisfaction (UIS) [70] . Both the EUCS and the UIS instruments contain items related to system quality, information quality, rather than only measuring overall user satisfaction with the system. Because of this, some researchers have chosen to parse out the various quality dimensions from these instruments, and use a single item to measure overall satisfaction with an IS [71] .

Ÿ Net benefits to individuals: this dimension summarizes benefits that can be gained by users when using GIS such as enhanced decision making, time saving, increase the understanding and awareness of problems [72] .

Ÿ Net benefits to organization: this dimension summarizes the benefits that organization derives from using GIS, which refers to efficiency and effectiveness criteria. Efficiency is the degree to how GIS operates with minimum waste, duplication, and expenditure of resources, and can be expressed as cost savings, cost avoidance, or productivity gains. Efficiency may also result in the generation of revenue. Effectiveness involves generating a product of better quality or accomplishing an intended purpose [67] .

Table 2. Measures of the model constructs.

Ÿ Net benefits to society: based on the study of “The impact of GIS technology” conducted by [67] , “societal impact” dimension has been proposed as a further variable to the lists of six DeLone and McLean’s IS success dimensions. Societal impact is important to be considered in the evaluation of GIS success because the ultimate goal of all technologies introduced in the public sector agencies is to benefit society. Many researchers reported the benefits of GIS on broad societal objectives such as, citizen-public sector interactions, individual integrity, economic benefits, distribution of wealth and fulfillment of human aspirations, and equity.

3.3. Model Hypotheses

The first level of the proposed GIS success model acts like a guide or a reference for GIS project managers to concentrate on the most critical success factors of GIS project diffusion. Although, establishing the CSFs of GIS project does not implicate that the whole project will automatically succeed, but it would be erroneously to neglect one of these CSFs.

The second level of the proposed GIS success model is a multidimensional construct, and the dimensions are interrelated. GIS are first implemented and incorporated within the organization and exhibit various degrees of system, and user quality. System quality and user quality affect the quality of the produced information. Managers/decision makers experienced the quality of information by using it for their works. Users and managers/de- cision makers are either satisfied or not satisfied with using the GIS. Finally, the use of information by managers/decision makers and the satisfaction of GIS users trigger influence on net benefits to individuals, organization and society.

The second level of the proposed model suggests that there can be positive influence between the GIS success dimension. Thus, we propose the following 11 hypotheses:

H1. System quality will positively affect Information quality;

H2. User quality will positively affect Information quality;

H3a. Information quality will positively affect user satisfaction;

H3b. Information quality will positively affect information use;

H4. Information use will positively affect User satisfaction;

H5a. Information use will positively affect net benefit to individuals;

H5b. Information use will positively affect net benefit to organization;

H5c. Information use will positively affect net benefit to society;

H6a. User satisfaction will positively affect net benefit to individuals;

H6b. User satisfaction will positively affect net benefit to organization;

H6c. User satisfaction will positively affect net benefit to society.

4. Research Design and Method

4.1. Measures of the Constructs

To ensure the content validity of the scales used in this study, We used the measurement items that were operationalized and tested in previous empirical GIS/IS studies and were found to have demonstrated good psychometric properties. The measuring items for each success dimension are summarized in Table 2.

4.2. Data Collection Procedure

The data used to test the model were obtained from a sample of experienced GIS users and managers. This study developed a questionnaire (see appendix A) using a five-point Likert scale (1 - 5) ranging from “strongly disagree” to “strongly agree.” The questionnaire was sent to 350 GIS users and managers in different GIS organizations in Egypt and abroad to answer the questions by assessing their GIS. For each question, respondents were asked to circle the response which best described their level of agreement. In total, 252 samples were received with an effective ratio of 72%. Detailed descriptive statistics relating to the respondents’ characteristics are shown in Table 3.

Table 3. Respondents characteristics.

5. Analysis and Results

5.1. Reliability Analysis

Reliability refers to the consistency or stability of the questionnaire results. Fewer errors lead to a higher level of reliability. In other words, a better reliability measurement will result from the consistency and stability of results. The present study measured the questionnaire reliability and the consistency of the items using Cronbach’s alpha (SPSS Version 20). Many scholars have suggested that a Cronbach’s alpha coefficient exceeding the 0.7 threshold indicates a high level of consistency among the aspects; a Cronbach’s alpha coefficient exceeding 0.9 indicates a much higher level of consistency among the aspects (e.g., [79] - [83] ). The Cronbach’s alpha coefficients of the eight construct are greater than or equal the recommended value of 0.8 (see Table 4).

5.2. Model Analysis

Structural Equation Modeling technique was used to assess the model fit and show empirical findings and hypotheses results using LISREL 8.8. Seven common model-fit measures were used to assess the model's overall goodness of fit: the ratio of X2 to degrees-of-freedom (df), goodness-of-fit index (GFI), adjusted goodness-of-fit index (AGFI), normalized fit index (NFI), comparative fit index (CFI), root mean square residual (RMSR), and root mean square error of approximation (RMSEA). As shown in Table 5, all the model-fit indices exceeded their respective common acceptance levels suggested by previous research, thus demonstrating that the proposed model exhibited a fairly good fit with the data collected. Thus, we could proceed to examine the path coefficient of the structural model.

Properties of the causal path, including the path coefficient, t-values, and variance explained for each equation in the hypothesized model, are presented in Figure 5.

System quality and user quality had significant positive influences on information quality. Thus, H1 and H2 were supported. The influence of information quality on both user satisfaction and information use were also significant. H3a and H3b were supported. The influence of information use on user satisfaction was not significant. Thus, H4 was rejected. User satisfaction has no significant impact on net benefits to (individuals, organization, and society). H6a, H6b, and H6c were rejected. Information use has a significant positive influence on both net benefits to individuals and net benefits to society, but has no significant effect on net benefits to organization. H5a and H5b were supported, while H5c was rejected. The direct, indirect, and total effects of system quality, user quality, information quality, information use and user satisfaction on net benefits to (individuals, organization, and society) are summarized in Table 6.

With regard to the constructs explained in the variance (R2), 89% of the variance in information quality was explained by system quality and user quality, while 20% of the variance in information use was explained by information quality. 87% of the variance in user satisfaction was explained by information quality and information use. The variance explained by information use and user satisfaction on net benefit to individuals is 34%, net benefit to organization is 8%, and net benefit to society is 13%.

6. Discussion and Conclusions

In this paper, by combining IS success models and previous studies, a model for measuring GIS success is pre-

Figure 5. Hypotheses testing results (note: t-values for standardized path coefficient are described in parentheses). Statistically significant at p < 0.05; **Statistically significant at p < 0.01.

Table 4. Cronpach’s alpha reliability analysis.

Table 5. Fit indices for structural model.

Table 6. The direct, indirect, and total effects.

sented. The proposed GIS success model has comprehensive components. It integrates the CSFs (independent variables) with the outcome measures of GIS (dependent variables) into one model. The first level of the proposed model extends and respecifies the 3-D model of IS success by organizing the CSFs that have been discussed in the GIS literature, according to their occurrence in GIS diffusion stages. The first level of the proposed model can benefit organizations by focusing on the vital aspect of a successful GIS project. The second level of the proposed model is concerned with measuring the post-implementation success of GIS. The second level of the proposed model extends and respecifies the updated DeLone and McLean model (2003) in the context of GIS. Through the above analysis, 8 success dimensions, 49-items instrument were demonstrated to produce acceptable reliability estimates. The second level of the proposed model exhibited reasonable fit with the collected data. Six of the eleven hypotheses were found to be significant. The empirical results showed that the system quality and user quality had a significant positive influence on information quality. In addition, information quality had a positive influence on both information use and user satisfaction. It can be interpreted as a response to high system and user quality; a high information quality is produced which in turn causes satisfaction to GIS users and causes more use to these valuable information. The finding that information use did not have a significantly direct influence on user satisfaction was inconsistent with most prior IS studies. Thus, information use is necessary but not sufficient to cause user satisfaction. The results showed that information use had a significant influence on both net benefits to individuals and society, and that information use had no significantly positive effect on net benefits to organization. Also, user satisfaction had no significantly positive effect on net benefits to individuals, organization, and society. This explains why there are many different stakeholders whose satisfaction needs to be considered. The user satisfaction measures the technological satisfaction and the overall satisfaction of the GIS users (direct users) with their software, while the net benefits to individuals measure the satisfaction of the indirect users (like decision makers who make use of the technology by relying on other members of the organization) with GIS in enhancing the process of the decision making. The net benefits to organization are directed to GIS managers to measure the efficiency and the effectiveness of GIS in their organizations, while net benefits to society are directed to citizens to measure the contribution of GIS to societal well being especially in governmental organizations. This may explain why there is no direct causal relationship among these constructs. For example, the GIS users may be satisfied by using their GIS, but for some reasons the organization cannot achieve benefits due to administrative or economic problems. Therefore, user satisfaction should precede net benefits dimensions, but it is not sufficient to cause them.

This study is regarded as the first step in the long term research agenda of the researcher to develop and improve a model for measuring the GIS success. Therefore, the validity of a GIS success model cannot be truly established on the basis of a single study. Thus, caution needs to be taken when generalizing these findings. Validation of measurement requires the assessment of measurement properties over a variety of samples in similar and different contexts.

Cite this paper

Khalid A.Eldrandaly,Soaad M.Naguib,Mohammed M.Hassan, (2015) A Model for Measuring Geographic Information Systems Success. Journal of Geographic Information System,07,328-347. doi: 10.4236/jgis.2015.74026

References

  1. 1. Douglas, B. (2008) Achieving Business Success with GIS. John Wiley & Sons, London.
    http://dx.doi.org/10.1002/9780470985595

  2. 2. Worboys, M. and Duckham, M. (2004) GIS: A Computing Perspective. CRC Press, Florida.

  3. 3. Longley, P., Goodchild, M., Maguire, D. and Rhind, D. (2011) Geographic Information Systems and Science. John Wiley & Sons, Hoboken.

  4. 4. Heywood, I., Corneliues, S. and Carver, S. (2011) An Introduction to Geographical Information Systems. Pearson, London.

  5. 5. Roldan, J. and Leal, A. (2003) Executive Information Systems in Spain: A Study of Current Practices and Comparative Znalysis. In: Mora, M., Forgionne, G. and Gupta, J., Eds., Decision Making Support Systems: Achievements, Trends, and Challenges for the New Decade, Idea Group Publishing, London, 272-286. http://dx.doi.org/10.4018/978-1-59140-045-5.ch018

  6. 6. DeLone, W.H. and McLean, E.R. (1992) Information Systems Success: The Quest for the Dependent Variable. Journal of Information Systems Research, 3, 60-95. http://dx.doi.org/10.1287/isre.3.1.60

  7. 7. Seddon, P.B., Staples, D.S., Patnayakuni, R. and Bowtell, M.J. (1999) Dimensions of Information Systems Success. Journal of Communications of the Association for Information Systems, 2, 1-61.

  8. 8. Ballantine, J., Bonner, M., Levy, M., Martin, A., Munro, I. and Powell, P. (1996) The 3-D Model of Information Systems Success: The Search of the Dependent Variable Continuous. Journal of Information Resources Management, 9, 5-14. http://dx.doi.org/10.4018/irmj.1996100101

  9. 9. DeLone, W.H. and McLean, E.R. (2003) The DeLone and McLean Model of Information Systems Success: A Ten-Year Update. Journal of Management Information Systems, 19, 9-30.

  10. 10. Seddon, P.B. (1997) A Respecification and Extension of the DeLone and McLean Model of IS Success. Journal of Information Systems Research, 8, 240-253. http://dx.doi.org/10.1287/isre.8.3.240

  11. 11. Kaiser, M.G. and Ahlemann, F. (2010) Measuring Project Management Information Systems Success—Towards a Conceptual Model and Survey Instrument. Proceedings of the 18th European Conference on Information Systems, Pretoria, 1 January 2010, 1-12.

  12. 12. Rai, A., Lang, S.S. and Welker, R.B. (2002) Assessing the Validity of IS Success Models: An Empirical Test and Theoretical Analysis. Journal of Information Systems Research, 13, 50-69.
    http://dx.doi.org/10.1287/isre.13.1.50.96

  13. 13. Seddon, P.B. and Kiew, M.Y. (1994) A Partial Test and Development of the DeLone and McLean model of IS Success. Australasian Journal of Information Systems, 4, 90-109.

  14. 14. Wu, J.H. and Wang, Y.M. (2006) Measuring KMS Success: A Respecification of the DeLone and McLean’s Model. Journal of Information & Management, 43, 728-739. http://dx.doi.org/10.1016/j.im.2006.05.002

  15. 15. DeLone, W.H. and McLean, E.R. (2004) Measuring e-Commerce Success: Applying the DeLone and McLean Information Systems Success Model. International Journal of Electronic Commerce, 9, 31-47.

  16. 16. Antenucci, J.C., Brown, K., Crosswell, P.L., Kevany, M.J. and Archer, H. (1991) Geographic Information Systems: A Guide to the Technology. Springer, New York.

  17. 17. Calkins, W. and Obermeyer, N. (1991) Taxonomy for Surveying the Use and Value of Geographical Information. International Journal of Geographical Information Systems, 5, 341-351.
    http://dx.doi.org/10.1080/02693799108927859

  18. 18. Clapp, J., McLaughlin, J., Sullivan, J. and Vonderohe, A. (1989) Toward a Method for the Evaluation of Multipurpose Land Information Systems. Journal of the URISA, 1, 39-45.

  19. 19. Tulloch, D.L. (1999) Theoretical Model of Multipurpose Land Information Systems Development. Transaction in GIS, 3, 259-283. http://dx.doi.org/10.1111/1467-9671.00021

  20. 20. Rockart, J.F. (1979) Chief Executives Define Their Own Data Needs. Harvard Business Review, 57, 81-93.

  21. 21. Bullen, C.V. and Rockart, J.F. (1981) A Primer on Critical Success Factors. Massachusetts Institute of Technology, Sloan School of Management, Massachusetts, USA.

  22. 22. Yeoh, W., Koronios, A. and Goa, J. (2006) Critical Success Factors for the Implementation of Business Intelligence System in Engineering Asset Management Organizations. Decision Support for Global Enterprise, 2, 34-51.

  23. 23. Campbell, H. and Masser, I. (1995) GIS and Organizations. Taylor and Francis, London.

  24. 24. Anderson, C.S. (1996) GIS Development Process: A Framework for Considering the Initiation, Acquisition, and Incorporation of GIS Technology. Journal of URISA, 8, 10-26.

  25. 25. Campbell, H. (1992) Organizational Issues and the Implementation of GIS in Massachusetts and Vermont: Some Lessons for the United Kingdom. Environmental and Planning B: Planning and Design, 19, 85-95. http://dx.doi.org/10.1068/b190085

  26. 26. Onsrud, H.J. and Pinto, J.K. (1993) Evaluating Correlates of GIS Adoption Success and the Decision Process of GIS Acquisition. Journal of URISA, 5, 18-39.

  27. 27. Rogers, E.M. (1995) Diffusion of Innovations. The Free Press, New York.

  28. 28. Huxhold, W.E. and Levinsohn, A.G. (1995) Managing Geographic Information System Projects. Oxford University Press, New York.

  29. 29. Obermeyer, N.J. and Pinto, J.K. (2008) Managing Geographic Information Systems. Guilford Press, New York.

  30. 30. Biehl, M. (2007) Success Factors for Implementing Global Information Systems. Communications of The ACM, 50, 53-58. http://dx.doi.org/10.1145/1188913.1188917

  31. 31. Kohan, M., Yusoff, W. and Asadi, A. (2011) KSFs in GIS Adoption during Crisis Management. Proceedings of the International Conference on Sociality and Economics Development, Kuala Lumpur, 15 May 2011, 331-335.

  32. 32. Nasirin, S. and Birks, D. (1998) Geographical Information Systems (GIS) Success Factors amongst UK Food Retailers: Comparisons between Market Leaders and Followers. Proceedings of the 10th Annual Colloquium of the Spatial Information Research Center, Dunedin, 16 November 1998, 235-242.

  33. 33. Alwaraqi, G. and Zahary, A. (2012) Critical Factors of GIS Projects Failure in Yemeni Governmental Agencies. Proceedings of the 13th International Arab Conference on Information Technology, Jordan, 10-13 December 2012, 53-65.

  34. 34. Birks, D., Nasirin, S. and Zailani, S. (2003) Factors Influencing GIS Project Implementation Failure in the UK Retailing Industry. International Journal of Information Management, 23, 73-82.
    http://dx.doi.org/10.1016/S0268-4012(02)00036-1

  35. 35. Openshaw, S., Cross, A., Charlton, M. and Brunsdon, C. (1990) Lessons Learnt from a Post Mortem of Failed GIS. Proceedings of the 2nd National Conference and Exhibition of the AGI, Brighton, 1 January 1990, 231-235.

  36. 36. Clarke, A. (1991) GIS Specification, Evaluation and Implementation. In: Goodchild, M.F., Maguire, D.J. and Rhind, D.W., Eds., Geographic Information Systems and Sciences, John Wiley & Sons, Hoboken, 477-448.

  37. 37. Somers, R. (1998) Developing GIS Management Strategies for an Organization. Journal of Housing Research, 9, 157-178.

  38. 38. Sieber, R.E. (2000) GIS Implementation in the Grassroots. Journal of URISA, 12, 15-29.

  39. 39. Velibeyoglu, K. (2004) Institutional Use of Information Technologies in City Planning Agencies: Implications from Turkish Metropolitan Municipalities. PhD Dissertation, Izmir Institute of Technology, Izmir.

  40. 40. Croswell, P.L. (1991) Obstacles to GIS Implementation and Guidelines to Increase the Opportunities for Success. Journal of URISA, 3, 43-56.

  41. 41. Man, W.H. and Toorn, W.H. (2002) Culture and the Adoption and Use of GIS within Organizations. International Journal of Applied Earth Observation and Geoinformation, 4, 51-63.
    http://dx.doi.org/10.1016/S0303-2434(02)00013-2

  42. 42. Rumor, M. (1993) The Use of Geographic Information Technology in the City of Paova. In: Masser, I. and Onsrud, H.J., Eds., Diffusion and Use of Geographic Information Technologies, Kluwer Academic Publishers, Dordrecht, 229-243.

  43. 43. Campbell, H. (1994) How Effective Are GIS in Practice? A Case Study of British Local Government. International Journal of Geographical Information Systems, 8, 309-325.
    http://dx.doi.org/10.1080/02693799408902002

  44. 44. Cavric, B. (2002) Human and Organizational Aspects of GIS Development in Botswana. Proceedings of the 6th GSDI Conference, Pudabest, 10 October 2002, 1-18.

  45. 45. Hussain, M. and Johar, F. (2010) The Socio-Technical Factors in the Use of GIS at the Planning Departments of the Kuala Lumpur City Hall Planning Malaysia. Journal of the Malaysian Institute of Planners, 8, 69-103.

  46. 46. Dekolo, S.O. (2001) Implementing GIS for Land Use Planning and Management in Lagos State. Proceedings of the URISA Conference, California, 20 October 2001, 173-182.

  47. 47. Higgs, G., Smith, D.P., Myles, I. and Gould, M.I. (2005) Findings from a Survey on GIS Use in the UK National Health Service: Organizational Challenges and Opportunities. Health Policy, 72, 105-117.
    http://dx.doi.org/10.1016/j.healthpol.2004.06.011

  48. 48. Lamer, N. and Disera D. (1997) System Success Factors and the Top Ten Reasons AM/FM/GIS Projects Falter. Proceedings of the Geospatial Information and Technology Association Conference, Nashvill, 401-404.

  49. 49. Davis, J. (1999) Six Keys to Gaining Executive Commitment to GIS. Proceedings of the URISA Conference, Chicago, 11 September 1990, 566-570.

  50. 50. Demens, M. (2005) Fundamentals of Geographic Information Systems. John Wiley & Son, Hoboken.

  51. 51. Gallaher, D. (1999) Three Leading Killers of GIS. Proceedings of the URISA Conference, Chicago, 21 August 1999, 585-586.

  52. 52. Klosterman, R.E. (1995) The Appropriateness of Geographic Information Systems for Regional Planning in the Developing World. Computers, Environment and Urban Systems, 19, l-13.
    http://dx.doi.org/10.1016/0198-9715(94)00028-X

  53. 53. Nasirin, S. and Birks, D. (2003) DSS Implementation in the UK Retail Organizations: A GIS Perspective. Journal of Information and Management, 40, 325-336.

  54. 54. Otawa, T. (2004) Benefits and Obstacles of GIS Implementation: Recent Perceptual Shifts and Implications for City and Regional Planning Organizations. Proceedings of the Geospatial Information and Technology Association Conference, Seattle, April 2004, 1-9.

  55. 55. Somers, R. (2001) Quick Guide to GIS Implementation and Management. URISA, Chicago.

  56. 56. Nedovic-Budic, Z. and Godschalk, D. (1996) Human Factors in Adoption of Geographic Information Systems: A Local Government Case Study. Public Administration Review, 56, 554-67.
    http://dx.doi.org/10.2307/977254

  57. 57. Taleai, M., Mansourian, A. and Sharifi, A. (2009) Surveying General Prospects and Challenges of GIS Implementation in Developing Countries: A SWOT-AHP Approach. Journal of Geographical Systems, 11, 291-310. http://dx.doi.org/10.1007/s10109-009-0089-5

  58. 58. Beck, D. (1997) Avoid Top 10 Mistakes GIS Project Mistakes. GIS World, 10, 48-51.

  59. 59. Brown, M. (1996) An Empirical Assessment of the Hurdles to Geographic Information System Success in Local Government. State and Local Government Review, 28, 193-204.

  60. 60. Gillespie, S.R. (2000) An Empirical Approach to Estimating GIS Benefits. Journal of URISA, 12, 7-13.

  61. 61. Aldaijy, E.Y. (2004) The Multidimensional Measurements of Geographic Information Systems (GIS) Effectiveness in Crisis Management. Proceedings of the 2004 Command and Control Research and Technology Symposium, Monterey, June 2004, 1-22.

  62. 62. Bone, T. and Johnson, D. (2007) Human Factors in GIS Use: A Review and Suggestions for Research. Proceedings of the Information Systems Education Conference, Pittsburgh, 3 November 2007, 1-15.

  63. 63. Jarupathirun, S. and Zahedi, F. (2003) Exploring the Influence of Perceptual Factors in the Success of Web-Based Spatial DSS. Decision Support Systems, 43, 933-951.
    http://dx.doi.org/10.1016/j.dss.2005.05.024

  64. 64. Shannon, C. and Weaver, W. (1949) The Mathematical Theory of Communication. University of Illinois Press, Chicago.

  65. 65. Mason, R. (1978) Measuring Information Output: A Communication Systems Approach. Information and Management, 1, 219-234. http://dx.doi.org/10.1016/0378-7206(78)90028-9

  66. 66. Bonner, M. (1995) DeLone and McLean’s Model for Judging Information Systems Success—A Retrospective Application in Manufacturing. Proceedings of the European Conference on IT Investment, Henley Management College, Berkshire, 11 July1995, 218-226.

  67. 67. Nedovic-Budic, Z. (1999) Evaluating the Effects of GIS Technology: Review of Methods. Journal of Planning Literature, 13, 284-295. http://dx.doi.org/10.1177/08854129922092405

  68. 68. Chein, S. and Tsaur, S. (2007) Investigating the Success of ERP Systems: Case Studies in Three Taiwanese High-Tech Industries. Journal of Computers in Industry, 58, 783-793.
    http://dx.doi.org/10.1016/j.compind.2007.02.001

  69. 69. Doll, W.J. and Torkzadeh, G. (1988) The Measurement of End-User Computing Satisfaction. MIS Quarterly, 12, 259-272. http://dx.doi.org/10.2307/248851

  70. 70. Ives, B., Olson, M. and Baroudi, J.J. (1983) The Measurement of User Information Satisfaction. Communications of the ACM, 26, 785-793. http://dx.doi.org/10.1145/358413.358430

  71. 71. Petter, S., DeLone, W.H. and McLean, E.R. (2008) Measuring Information Systems Success: Models, Dimensions, Measures, and Interrelationships. European Journal of Information Systems, 17, 236-263. http://dx.doi.org/10.1057/ejis.2008.15

  72. 72. Akingbade, A., Navarra, D. and Georgiadou, Y. (2009) A 10 Years Review and Classification of the Geographic Information Systems Impact Literature 1998-2008. Nordic Journal of Surveying and Real Estate Research, 4, 84-116.

  73. 73. Witkowski, M.S., Rich, P.M. and Keating, G.N. (2007) Metrics of Success for Enterprise GIS. Journal of Map and Geography Libraries, 4, 59-82. http://dx.doi.org/10.1300/J230v04n01_04

  74. 74. Nedovic-Budic, Z., Pinto, J.K. and Budhathoki, N.R. (2008) SDI Effectiveness from the User Perspective. In: Crompvoets, J., Rajabifard, A., Loenen, B. and Fernández, T.D., Eds., A Multi-View Framework to Assess Spatial Data Infrastructures, The Melbourne University Press, Melbourne, 273-303.

  75. 75. Lee, J. and Bednarz, R.S. (2012) Components of Spatial Thinking: Evidence from a Spatial Thinking Ability Test. Journal of Geography, 111, 15-26. http://dx.doi.org/10.1080/00221341.2011.583262

  76. 76. Franz, C.R. and Robey, D. (1986) Organizational Context, User Involvement, and the Usefulness of Information Systems. Decision Sciences, 17, 329-356.
    http://dx.doi.org/10.1111/j.1540-5915.1986.tb00230.x

  77. 77. Zwart, P. (1991) Some Indicators to Measure the Impact of Land Information Systems in Decision Making. Proceedings of the URISA Conference, Washington DC, 11 August 1991, 68-79.

  78. 78. Sanders, G.L. and Coutney, J.F. (1985) A Field Study of Organizational Factors Influencing DSS Success. MIS Quarterly, 9, 77-93. http://dx.doi.org/10.2307/249275

  79. 79. Ganapati, S. (2010) Using GIS to Increase Citizen Engagement. IBM Center for the Business of Government, Washington DC.

  80. 80. Chang, L.M., Chang, S.L., Ho, C.T., Yen, D.C. and Chiang, M.C. (2011) Effects of IS Characteristics on e-Business Success Factors of Small- and Medium-Sized Enterprises. Computers in Human Behavior, 27, 2129-2140. http://dx.doi.org/10.1016/j.chb.2011.06.007

  81. 81. Cooper, D.R. and Schindler, P.S. (2001) Business Research Methods. McGraw-Hill Higher Education, London.

  82. 82. Hair, J.F., Anderson, R.E., Tatham, R.L. and Black, W.C. (2006) Multivariate Data Analysis with Reading. Prentice Hall, Upper Saddle River.

  83. 83. Wang, H.C. and Chiu, Y.F. (2009) Assessing e-Learning 2.0 System Success. Computers and Education, 57, 1790-1800. http://dx.doi.org/10.1016/j.compedu.2011.03.009

Appendix A. Survey Items Used in This Study

1) System quality

SQ1: The GIS software contains all the features and functions required to perform the required tasks

SQ2: The hardware and operating system response time are acceptable

SQ3: The server downtime typically 8 hours or less per year

SQ4: All failures (including server, network, and software) are less than 40 hours downtime per year

SQ5: The GIS software is user-friendly

SQ6: It is easy to recover from errors encountered while using GIS software

SQ7: The database content is secured

SQ8: Data backup is maintained throughout the organization regularly

SQ9: The database content is regularly updated throughout the organization

SQ10: The database contains accurate data

SQ11: The database contains all the needed data for related tasks

2) User quality

UQ1: You feel comfort while using the GIS

UQ2: You are capable to do the required task

UQ3: You understand what you do

UQ4: You feel confidence while using the GIS software

UQ5: Spatial ability test (20 questions developed by Lee and Bednarz (2012))

3) Information quality

IQ1: The GIS provides the accurate information you need

IQ2: The GIS provides sufficient information

IQ3: The information on the map is easy to understand

IQ4: The information provided meet your needs regarding your questions or problems

IQ5: The GIS provides reliable information

IQ6: The GIS provide up to date information

IQ7: The information on digital or hardcopy maps are clear

4) User satisfaction

US1: You are pleased with the GIS

US2: You like to use the GIS

US3: You are willing to use the GIS

US4: Overall, how would you rate your satisfaction with the GIS?

5) Information use

IU1: To what extent do you actually use the reports or the output generated by the GIS?

IU2: To what extent could you get along without the use of the GIS?

IU3: What is the level of importance of decisions affected by the generated information?

6) Net benefits to individuals

IND1: Using GIS save time required for making decisions

IND2: As a result of GIS, I am better able to set my priorities in decision making

IND3: GIS has improved the quality of decisions I make in this organization

IND4: As a result of GIS, the speed at which I analyze decisions has increased

IND5: GIS enables timely problem recognition

IND6: GIS enhances the understanding of the problems

7) Net benefits to organization

ORG1: The GIS helps the organization save cost in information production and provision

ORG2: The GIS increases the organization profitability

ORG3: The GIS improves the organization’s competitive position

ORG4: The GIS helps the organization to achieve its goal

ORG5: The GIS enables a new range of output (maps, tables, lists, etc.)

ORG6: The GIS provides the organization with better motivated workforce

ORG7: The GIS improves information sharing and flows to management and between departments.

ORG8: The GIS reduces risk in the decision making process

8) Net benefits to society

SOC1: The GIS provide equal availability of information to citizens when needed and equal ease of access

SOC2: The GIS enables participation by public in decision process (enhancement of principles of a democratic society)

SOC3: Using the GIS improves the standard of health and safety in the society

SOC4: Using the GIS increases the economic benefits to the society

SOC5: The GIS provides better service to public/citizens

NOTES

*Corresponding author.