American Journal of Industrial and Business Management
Vol.3 No.2(2013), Article ID:30246,10 pages DOI:10.4236/ajibm.2013.32030

A Neural Network and Expert Systems Based Model for Measuring Business Effectiveness of Information Technology Investment*

Mahmud Mavaahebi, Ken Nagasaka

Department of Electrical and Electronic Engineering, Tokyo University of Agriculture and Technology, Tokyo, Japan.

Email: 50010834701@st.tuat.ac.jp, mavaahebi_m@yahoo.com, bahman@cc.tuat.ac.jp

Copyright © 2013 Mahmud Mavaahebi, Ken Nagasaka. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Received October 11th, 2012; revised November 20th, 2012; accepted December 17th, 2012

Keywords: Artificial Neural Network; Expert System; Information Technology; Technology Investment; Business Effectiveness; Counter propagation; Business Objectives

ABSTRACT

Today’s indispensable bound between Information Technology (IT) and Business bears heavy expectations on IT to enable firms achieve their strategic business goals and drive competitiveness [1] and revenue growth via offered Technology Solutions and Services. To assess achievement level associated with such expectations, mechanisms must exist for determining the relationship between Organization’s Technology investments and services provided by IT, enabling quantification of effectiveness. In spite of many methods and tools on the market for measuring the Return On Investment (ROI) [2], Net Present Values (NPV), etc., and various studies that have been conducted toward measuring IT’s Business effectiveness, the result has been mostly qualitative, speculative and hypothetical. A mechanism does not seem to exist for measuring [3] quantitatively the effectiveness of incurred technology investment in an organization by leveraging such concepts as Neural Nets or Fuzzy Logic. While Neural Network has been providing possibilities for solving problems in various fields such as Medicine, Engineering, Finance, Economics, etc., [4] its capabilities do not appear to have been explored adequately in the field of Information Technology. Hence, a research is being conducted to develop a Neural Nets/Expert Systems model that can identify within a firm the correlation between IT cost factors, IT services, percentage of utilized services by Business Functions and their associative technology costs inline with the percentage of contributions made by each Function toward achieving Business Objectives. Once developed, the model can calculate Yielded Unit Costs of IT Services and Business Objectives for comparison with their respective optimized unit costs to determine effectiveness and impact that Technology investment has caused on achieving Objectives during a given fiscal period. Neural Network’s modeling is used for developing patterns and quantifying correlations between various layers based on past experiences. Additionally, the model can more accurately forecast required Technology investment for an upcoming fiscal period.

1. Introduction

As ordinarily Technology investment in an organization is channeled through and managed by the IT Department [2], it is imperative for IT to fully understand the Business expectations and to make utmost effort toward effectively fulfilling them. Generally firms define the effectiveness of their IT organizations in terms of improving efficiencies, enhancing productivity, enabling delivery of high quality products and services to customers, maintaining cost competitiveness, minimizing information security risks and contributing to customer retention and growth as well as revenue increase [5]. In brief addressing such expectations comes at a cost that afirm bears through the investment it makes in its technology solutions and services delivery. As the number and degree of contributing parameters and factors to the IT services cost calculation and Business objectives vary over time with complex relationships, a set of neural network based models that are capable of learning the relationships from the past and applying the same to the future scenarios could potentially simplify the method of determining the impact of technology on the business. Such model not only can provide a tool to a firm for making judgment on the effectiveness of its IT services and the invested costbut also can be used for projecting the technology cost for a given period. In this paper a high-level overview of a research model with the characteristics stated above is being provided. The foundation of the model is based on building a relationship between the invested IT cost and the business goals to the extent that it would be possible to measure IT services consumed by each business function, their associative cost, and the total yieldedcost related to each business objective.

In Figure 1, the flow of IT resources, generated services and their utilization by business functions to achieve certain objectives are illustrated.

The lowest layer 1) consists of IT Supply or IT resources that are cost bearing and requiring Technology Investment. This layer generates Solutions and Services 2) to be used by Business Functions, which in turn make certain contributions 3) toward achieving each Business Objective. In Figure 1 the components appearing in each layer as well as the numbers are all illustrative and can vary in each organization. Figure 1 is the foundation of the model that will be built to link the components of each layer to the next ones above. The relationship between layers and the transition from one layer to the next will be described via Neural Network based mathematical models and Expert Systems.

2. Methodology

2.1. Current Methods

Lack of a standard methodologyin an organization often leaves the measurement of IT effectiveness to become a subjective judgment particularly when intangible factors such as service quality, enhanced productivity [6] growth in customer satisfaction, increase in efficiency and so on are to be taken in to account. Such methods leave the interpretation window wide open creating potential inconsistencies leading in to less reliable results. In order to address questions relevant to validity and consistency of such subjective measurements, there have been many studies and attempts to develop models that are more logic based enabling quantification of tangible and intangible factors for measuring the impact and values that information technology brings to an organization.

However, what the existing methods [7] may lack include:

• Elaboration on how to measure realized values and benefits post technology investment and implementation;

• Being able to calculate the variance between projected/expected returns and realized values;

• Being mindful of a need for analyzing variance and providing recommendations, course of actions, and the highlighting lessons learnt;

• Clarity on quantitatively mapping the technology investment with major Business goals [3];

• Seeing a need to develop an optimization model for future benchmarking and analysis.

To address such limitations, a new model that intends to exploit a quantitative approach is being introduced.

2.2. A New Method

The underlying building block of this new method, to be called ANN-ES-IEC (Artificial Neural Network Expert System IT Effectiveness Calculation) Model, is based on establishing a relationship between the Key Success Factors (KSF), in other words key Business Objectives (BO), and the Total Cost of Technology Ownership (TCT), including infrastructure, operations, processes, governance, manpower, etc. Such relationship should be able to provide significant insight on the degree of contribution that technology investment makes to the business in fulfilling its goals [5]. In Figure 2, the high-level flow that links IT Cost Factors all the way through to Business Objectives Unit Costs, Effectiveness Measurement and Projection of Next Period TCT is depicted.

The foundation of this method is based on the premise that in an organization, IT function that is operated at certain cost, provides technology solutions and services to various Business operations and functions to achieve certain Objectives, as illustrated in Figure 1. Hence, it should be possible to establish a relationship between the Objectives that each Business Function in the organization pursuesand the incurred technology cost by that function which in turn is a fraction of the Total Cost of Technology.

Some of the characteristics of ANN-ES-IEC Model are listed below:

• Allowing quantification of cost associated with each service type produced by IT;

• Allowing quantification of the percentage of services utilized by each Business Function that contributes to the business objectives achievement;

• Enabling calculation of cost associated with utilized IT services by each Business Function;

• Being capable of calculating the degree of effectiveness associated with IT cost;

• Allowing calculation of incurred cost per Unit of IT service;

• Being capable of providing optimized cost per Unit of IT service;

• Being capable of providing recommendation for enhancing IT services and their associative cost relevant to areas that have been less effective or identifying areas where the cost has relatively decreased;

• Enabling projection of cost per IT Unit Service relevant to next fiscal period.

Figure 1. Correlation between IT cost factors, services, business functions and objectives.

Figure 2. Process flow—IT cost factors through unit cost optimization.

2.3. Approach

The ANN-ES-IEC Model is based on clarify, measure and optimize analogy. It begins with the assumption that IT’s cost factors have already been identified and their associative costs calculated. Next the model tries to build a relationship between the cost factors and generated services, calculating percentage of leveraged services based on the percentages of contribution made by each Business Function toward achieving each BO. It also calculates each Business Objective’s Yielded Unit Cost with respect to the overall invested technology cost for a defined period. Additionally the model contains an optimizer that takes in various parameters and outputs optimized Unit Cost relevant to each BO, as well as a Projection engine to project the Technology cost for next Period.

In Figure 3 the correlation between the IT’s cost generating factors, services and incurred costs by BOs are illustrated. The provided services to Business Functions enable them to contribute to the achievement of firm’s Business Objectives at a certain technology cost [1] that can be further ramified for the calculation of each Business Objective’s total incurred cost as well as its cost per unit. Such unit cost is a useful parameter that can be used for determining whether or not the invested Information Technology cost during a given period has been effective enough or not. In Figure 3, the last step is either to use the Yielded Unit cost for calculating a new Optimized Unit cost, or to provide a recommendation depending on

Figure 3. Technology effectiveness measurement process flow.

the level of effectiveness recognition.

The model’s measurement of IT investment effectiveness is in terms of relativity rather than absoluteness, as it compares already incurred costs (of a specific period) with the optimized (standard) values relevant toeach BO and subsequently draws conclusion on the degree of effectiveness or ineffectiveness that the incurred IT investment has caused the Business during that specific fiscal period.

Access to each BO’s unit cost, will also enable development of an optimized parameters set over a period of time that could be defined as the firm’s standards and to be used for benchmarking in future analysis. Moreover, availability of such granular level information will allow an organization to drill down to the areas relevant to the IT’s services and technologies that have been performing effectively and those areas that have not been doing so. Aligned with such results, plans can be developed for the next course of action. Additional features of this model include provision of functionality for projection of re quired IT investment in relevance to the desired BOs for a specific period and based on the Optimized unit costs. In various areas within the model, Artificial Neural Netmethodology has been exploited for certain calculations that have been described in details in the following sections.

2.4. New Method’s Modeling

The ANN-ES-IEC Model consists of 3 interrelated tiers [4] as illustrated in Figure 4.

• 1st Tier: a 3-layer Counterpropagation neural network for determining a correlation between IT Cost Factors and IT Services. The Competitive (hidden) layer is Consumed IT Resources Cost Contributions.

• 2nd Tier: a 3-layer Back Error Propagation neural network for establishing a correlation between IT Services and Business Functions. The hidden layer is Consumed IT Services Cost Contributions.

• 3rd Tier: a 3-layer Back Error Propagation neural network for establishing a correlation between Business Functions and Business Objectives. The hidden layer is Contribution to Business Objectives Cost Contributions.

In the 1st and 2nd Tiers, the Input and Output Layers are fully connected, whereas in the 3rd Tier, partial connectivity would exist since not every Business Function may contribute to every Business Objectives.

This model will have the following characteristics:

• The model uses data from previous fiscal periods to develop patterns based on a correlation between the lowest Layer of IT Cost Factors through the highest Layer of Business Objectives [4].

• There will be multiple other Expert Systems and ANN based models in the system for various purposes such as BO unit cost calculation, optimization of BO unit cost, projection of BO unit cost, determination of effectiveness level, etc.

• As the model will be accepting external factors carrying some level of uncertainty, the usage of fuzzy logic shall be taken in to consideration.

• To fine-tune the model, certainly significant amount of data from the real world would be essential for conducting various simulations using MATLAB software package.

The correlation between the input and output Layers associated with each Tier, is formulated in a few mathematical models described at the high-level below.

Figure 4. Multi-tier neural network model.

2.5. Incurred Technology Cost Calculations for Services & Business Objectives

Tier 1: Cost of Services—based on correlation between IT Cost Factors and IT Services:

where:

ITC: IT Total Cost (=TCT: Total Cost of Technology)

TCFi: Technology Cost Factor i SjC = Service type j’s Cost Ci = Cost of IT Cost Factor i wji = Multiplier (weight of connection from TCFi to Service type j)

Tier 2: Cost incurred by Business Functions—based on correlation between IT Services and Business Functions:

where:

: Services Utilization by Business Function i

: Business Function i Technology Cost

: Service type j’s Cost

: Multiplier (% of Service type j used by Business Function i)

Tier 3: Cost of Business Objectives—base on correlation between Business Functions and Business Objectives:

where:

: Business Objective i’s Achievement

: Business Objective i’s Cost

: Business Function j’s Technology Cost

: Business Function i

: Multiplier (% of contribution by Business Function j to Business Objective i)

3. Results

3.1. Projection of Technology Services Cost

The ANN-ES-IEC Model allows projection of required Technology investment for a specific period leveraging Optimized Costs Pool and various other factors as illustrated in Figure 5.

Projection Engine, which is an Artificial Neural Network based mathematical model, accepts various parameters such as Optimized Business Objectives Unit Costs, next period’s Projected Business Objectives, External and Internal factors. Post processing, based on previous learning, the model supplies Projected Cost associated with each Business Objective. The Model can also predict the percentages of required services to be provided by each Business Function and consequently the percentage of utilized IT Services. With such information at hand, the overall Technology investment for the next period can be projected. In Figure 6 anticipated

Figure 5. Reverse engineering process—projection engine.

IT services required for each Business Objective with their associative percentages are illustrated. Subsequently, the percentages of utilized IT services by the Business Function-i from the pool of services are depicted as well.

In summary, post stabilization, the model would be able to accept a specific Period’s Projected Business Objectives along with Optimized Business Objectives Unit Costs and certain Factors to generate Projected Technology Cost numbers for the next Period. The concept at the high-level has been depicted in Figure 7.

3.2. IT Effectiveness Trend Analysis

Tracking the behavior of both Yielded and Optimized Unit Cost for each Business Objective over a period of time, provides a useful means for determining the trend of such costs and accordingly taking corrective actions as and when needed. A few illustrative examples are provided in Figure 8 for clarifying the concept at a highlevel.

Having access to such kind of data would enable a firm to easily track the cost trend of the factors that are most significant for its business and take action accordingly.

3.3. BOs Yielded Unit Costs (Model-1)

At the high-level, the ANN-ES-IEC Model is based on IPO (Input, Process, and Output) analogy. As its Input it accepts IT Cost Factors, Business Objectives and Functions that directly contribute to the achievement of Business Objectives. The Process layer consists of various mathematical and neural network operations that convert the Input in to certain Output results that is the Yielded Unit Cost of BOs. Hence the core of the model is its Process layer that is mathematically modeled, a glimpse of which was presented in the previous section. Figure 9 illustrates the IPO flow.

The 3 Tiers described earlier are all clubbed together and embedded in the Process layer consisting of various Expert Systems and ANN based equations and formulas. It is in this core component that the correlations between various factors are established via mathematical modeling.

The learning that the model goes through using different sets of real data feeds enables the discovery of patterns based on relationships between various factors which can be refined through a large number of runs. The Model exploits 2 types of learning methods, Supervised (Back Error Propagation) and Unsupervised (Competitive Learning), in order to develop the patterns. Post Convergence, only Inputs will be provided to the model for which the Output could be produced based on the previously determined relationship (Hidden Layer) in the patterns [4].

3.4. BOs Optimized Unit Costs (Model-2)

Another by-product of ANN-ES-IEC Model is an Optimizer Engine that takes in various parameters from the previous fiscal period such as, Business Objectives and their associated incurred costs, Business Functions contributions, and IT’s total cost. Also, various Internal Factors such as expense reduction/cost saving initiatives, common IT trends, Business Objectives impacted by Merger & Acquisition, etc. as well as External Factors such as price of Market parameters on IT Cost Factors, Customers behavior change due to economical circumstances, etc., will be provided as inputs to the Optimizer Engine. The generated output will be Optimized Unit Cost of each Business Objective. The Optimizer Engine, its Inputs and Outputs are illustrated in Figure 10.

The Optimizer Engine is an Expert System that has matured over a period of time by taking in Input parameters, described earlier, analyzing them, determining their

Figure 6. Projection engine’s output.

Figure 7. Projection of next period’s technology cost.

relationships and patterns and eventually calculating new optimized unit costs. New patterns are discovered as sets of new inputs go into the model, which result in building of new relationship among various factors and outputting new costs for the units of various parameters. The Optimized Unit Cost associated with each Business Objective, generated by the Optimizer Engine, are stored in the system to be used for comparison with the Yielded Unit Cost for a specific period to determine the degree of Business effectiveness as a result of IT investment in a given period. It is noteworthy to know that the Optimized Unit Costs are recalculated afresh for every period once the Yielded Unit Costs for that period become available.

3.5. Effectiveness Assessment: Case Study 1— Effectiveness Contribution

Calculating the Variance between Optimized (Standard) Unit Cost and Yielded Unit Cost for BOi.

where:

: Business Objective i’s Unit Cost Variance

: Business Objective i’s Optimized Unit Cost

: Business Objective i’s Yielded Unit Cost

Figure 8. Unit costs trend analysis; IUC: Yielded Unit Cost; OUC: Optimized Unit Cost.

Figure 9. IPO flow as model foundation.

Figure 10. Optimization process flow.

Example 1. BOi (5% Revenue Growth).

where:

: Business Objective i’s Technology Cost

: Business Objective i’s Yielded Unit Cost

: Business Objective i’s Optimized Unit Cost

Last Period’s Technology Investment has effectively contributed in achieving Business Objective i (e.g., Customer Satisfaction)

Areas with Lower Costs:

4.8% S3: IT Service Type 3 (e.g., System Availability).

3.6. Effectiveness Assessment: Case Study 2— Less than Effective Contribution

Calculating the Variance between Optimized (Standard) Unit Cost and Yielded Unit Cost for BOj.

Example 2. BOj (7% Customer Growth).

Last Period’s Technology Investment was NOT adequately effective for Business Objective 2 (e.g., Customer Growth)

Areas with Higher Costs:

2.7% S1: IT Service Type 1 (e.g., System Development).

3.7. Practical Application of the Model

This model will be built with adequate level of flexibility to allow it to be used in any organization that does investment in Information technology and wishes to assess the values that such investment bears to its business.

The model has the following features:

• Can calculate previous period’s incurred technology cost associated with one unit of BO;

• Can compare BO’s incurred cost with the optimized/ standard cost to identify gap and provide recommendation for enhancement, where optimized cost is below incurred cost;

• Can be used for projection of desired BOs costs relevant to the next period;

• Will be able to provide what-if functionality for simulation;

• Will have capability to project the cost of IT services associated with the next period;

• Also allows projection of BOs should the next period’s IT investment remains unchanged.

3.8. Example

Firm A has been allocating millions of dollars every year on its Information Technology for the purpose of automating various Business processes, enhancing productivity of its Business Operations, lowering expenses and risks associated with the firm’s information security toward achieving its Business Objectives including Revenue increase by 15%, Customer increase by 10% and Cost reduction by 11%. Firm A is facing two major issues with respect to its annual IT investments—the 1st issue is inability to quantitatively relate the IT investment with each Business Objective in order to measure the Business effectiveness of its IT [5] and the 2nd issue is not being able to project how much IT investment would be required in a particular fiscal year for enabling it achieve that year’s Business Objectives. ANN-ES-IEC Model, highlighted in this paper, would enable Firm A to address both issues by calculating Yielded Unit Cost of each Business Objective, compare it with its associative Optimized Unit Cost and determining the level of IT investment effectiveness as well as leveraging Optimized Unit Cost associated with each Business Objective for projecting next fiscal year’s required total IT investment. Having these two issues out of its bucket, Firm A will have a clear picture of its IT investment and the ability to enhance it whenever required, thanks to the Neural Network based model that has made this possible.

4. Conclusions

As the Yielded and Optimized costs per unit of each Business Objective becomeavailable, via the model introduced in this paper, next would be just a matter of comparing two numbers for arriving to a conclusion whether the budget spent on IT during a specific period enabled Business to achieve its Objectives effectively or not. The most challenging task in this model for arriving to the Unit Cost values is the establishment of correlation between various layers that exist between the lowest layer, IT cost factors, all the way up to the Business Objectives. That is where Neural Network can contribute to overcoming the challenges by exploiting thereal world data from the past for training and discovering the patterns and relations [4]. Once the relationships between various layers are quantified, that becomes the foundation for the model, taking in certain inputs and producing the Unit Cost values.

Post comparison of Yielded and Optimized Unit Costs for arriving to a conclusion on IT effectiveness, the Optimized Unit Cost will be adjusted as some of the contributing factors normally change over time. This newly adjusted Optimized Unit Cost can be used further for projection of required IT investment for the coming fiscal period based on certain input parameters fed in to the model.

Since data from previous fiscal periods are required for developing patterns and determining the relationship between various IT cost factors, services and the Business Objectives, Neural Network based models appear to be the best fit for such purpose. These models leverage historical data for conducting supervised and unsupervised learning [4] to establish relationship between IT cost factors, IT provided services, IT consumed services by various Business Operations to achieve certain Business Objectives. Once patterns are established, the Neural Network model will allow the collected data for a particular fiscal year to be matched against such patterns in order to determine the effectiveness of IT investment toward achieving that period’s Business Objectives.

As it has been demonstrated in this paper, building and utilizing an Artificial Neural Network and Expert Systems based model enables establishing of relationships between IT cost bearing factors and the Organization’s Business Objectives that can result in having access to a mechanism capable of measuring the technology cost associated with those objectives and determining the level of effectiveness caused by the investment.

REFERENCES

  1. L. Worrall, D. Remenyi and A. Money, “Measuring the Effectiveness of Information Technology Management: A Comparative Study of Six UK Local Authorities,” Working Paper Series, No. WP 012/98, 1998.
  2. K. Morton, “Running IT as a Business Seven Steps to Aligning IT with the Business,” SAS (Institution) Australia. http://resources.idgenterprise.com/original/AST-0030398_Running_IT_as_a_Business_105043_0211.pdf
  3. D. Sward, “Measuring the Business Value of Information Technology: Practical Strategies for IT and Business Managers,” Intel—Information Technology, 2006.
  4. J. Dayhoff, “Neural Network Architectures—An Introduction,” VNR—Van Nostrand Reinhold, New York, 1990.
  5. P. P. Tallon, K. L. Kraemer and V. Gurbaxani, “Executives’ Perceptions of the Business Value of Information Technology: A Process-Oriented Approach,” University of California, Irvine, 2001.
  6. E. Ugboma, “Assuring Information Systems’ Effectiveness through Data Integrity: Essential Guidelines for Information Systems Databases,” Florida Memorial College, Miami.
  7. C. Symons, L. M. Orlov and L. Sessions, “Measuring the Business Value of IT, a Survey of IT Value Methodologies,” Forrester Research, 2006.