Technology and Investment, 2011, 2, 142-153
doi:10.4236/ti.2011.22015 Published Online May 2011 (http://www.SciRP.org/journal/ti)
An Integrated Approach for Selecting Information Systems:
A Case Study
Konstantinos Viglas1, Panos Fitsilis2, Achille s Kameas1
1Hellenic Open University, Patras , Greece
2Technological Educational Institute of Larissa, Larissa, Greece
Received March 22, 2011; revised April 28, 2011; accepted May 4, 2011
Abstract
There is agreement within the academia and practitioners that IT investments should be evaluated in order to
be in agreement with the overall strategic objectives of an organization. Moving toward to this direction, the
aim of this paper is to present a model that combines Balanced ScoreCard (BSC) methodology and a deci-
sion support method such as Analytic Network Process (ANP) for assisting the selection of an IT system.
The proposed model provides a simple, flexible and easy to use approach that can be applied by organiza-
tions to support their investment decisions. The proposed approach is presented through a case study for se-
lecting a Quality Management Information System for a large Greek retailer.
Keywords: Multi-Criteria Decision making, Balanced Scorecard, Analytic Network Process
1. Introduction
Information system selection plays an important role in all
modern organizations since their smooth and efficient op-
eration depends heavily on Information Systems (IS). Fur-
thermore, large software systems are built by using com-
ponents developed by others (commercial or open source),
therefore an inc reasing need ap pears to se lect the righ t sys-
tem, the appropriate components in a systematic, factual,
objective, and cost efficient manner.
The selection process is far from being trivial since it has
to combine many, complex and in many cases contradict-
ing factors such as: business strategy, numerous functional
and non-functional requirements, operating priorities,
availability of resources etc. [1,2].
Traditional approaches and methods for selecting infor-
mation systems focus on well-known financial measures,
such as the Return On Investment (ROI) [3], Net Present
Value (NPV), the Internal Rate of Return (IRR), Cost/
Benefit Analysis (CBA) and the payback period [4,5].
However, these methods cannot offer the analytical power
needed for today’s complex decisions, since they fail in
quantifying intangible criteria.
Multi-criteria decision making (MCDM) can be quite
useful to support an IT system selection process. Although
there is no generic methodology that can be adopted for
selecting a soft ware packag e o f any type, liter atur e reviews
on evaluating software products suggest that users and de-
cision makers can receive a lot of support, if they decide to
adopt an MCDM method [6]. In particular, the findings of
review studies [6,7] present that the Analytic Hierarchy
Process (AHP) has been widely and successfully used in
evaluating several types of software packages (e.g.,
MRP/ERP systems, simulation software, CAD/CASE sys-
tems, Knowledge Management systems etc.). The AHP
method was introduced by Saaty [8] and its primary objec-
tive is to classify a number of alternatives (e.g., a set of
candidate software packages) b y considering a given set of
qualitative and/or quantitative criteria, according to pair
wise comparisons/judgments provided by the decision
makers. AHP r esults in a hierarch ical lev eling of the se lec-
tion criteria, where the upper hierarchy level is the goal of
the decision process, the next level defines the selection
criteria which can be further subdivided into subcriteria at
lower hierarchy levels and, finally, the bottom level pre-
sents the alternative decisi ons to be evaluat ed.
A newer version of AHP is Analytic Network Process
(ANP) and is considered as a generic form of AHP. The
main difference between AHP and ANP is th at AHP struc-
tures a decision problem into levels forming a hierarchy,
while the ANP is using a network approach. ANP allows
both interaction and feedback within clusters of elements
(inner dependence) and between clusters (outer depend-
ence). Such feedback capture s the co mplex effec ts of inter-

K. VIGLAS ET AL.143
play in co mplex situat ions in a better wa y, esp ecially when
risk and uncertainty are involved [9,10].
Nevertheless, the overall decision process should be fil-
tered in the context of business strategy. This can be ac-
complished with the application of Balanced Scorecard
Method (BSC). Not only being a methodology, BSC is
considered a performance measurement framework that
provides an integrated look at the business performance of
an organization by a set of both financial and non-financial
objectives.
Obviously, the selection of the appropriate information
system can offer strategically, tactical and operational ad-
vantages to an organization. However, this selection is a
complex process that should be in line with the overall
strategy, take into account financial aspects and at the same
time be analytical.
In our paper, through the case study under investigation,
we present a model that starts at the high level with the
strategic objectives of an organization, as they have been
described by the use of BSC, and ends wi th the application
of ANP method, which quantifies and balances the low
level criteria.
The application of this model can greatly assist both the
high and mid level management in approaching the deci-
sion process from a different perspective, while at the same
time this decision is factual, consistent and well docu-
mented.
The structure of the paper is as follows, Section 2 pre-
sents the relevant literature background and an overview of
the employed methodologies. Section 3 presented the pro-
posed approach. In Section 4 we demonstrate the proposed
approach though the presentation of a case study. The case
study is focusing on the selection of a Quality Management
System (QMS) for a multinational food retail organization.
Conclusions and extensions of the research work are ad-
dressed in chapter 5.
2. Background
2.1. Financial Methodologies for IS Selection
Traditionally, investment appraisal was based on finan-
cial accounting methodologies, such as return on invest-
ment and payback period. Their application has been
criticized as biased [11], since they tend to overlook
market status, human capital and process improvement,
growth opportunities etc. As such, they cannot measure
objectively past performance and forecast future out-
comes. However, financial indices are always considered
important since they measure the monetary value of the
IT investment.
Net Present Value (NPV) is defined as the total Pre-
sent Value (PV) of a time series of cash flows. It is a
standard method for using the time value of money to
appraise long-term projects [12]. It is defined with the
formula

011
ttt
C
NPV Cr  
 (1)
where C0 defines the initial investment, Ct is the valua-
tion of the current cash flow and r is the discount rate.
Intuitively, NPV defines what would cost today a cash
flow that will take place in the future. In practical terms,
if NPV is positive then the investment adds value to the
business, the project is profitable and therefore the IT
system should be developed or purchased.
Similar to NPV’s measure is IRR (Internal Rate of Re-
turn), which is defined with the formula

010
1
ttt
C
CIRR



 (2)
Semantically, IRR is the calculation of the rate that
nullifies NPV [13]. In case of selection between mutually
exclusive alternatives and especially when the initial co st
is different, incremental analysis shall be applied in o rder
to evaluate the IRR of the difference between two alter-
natives with the smaller cost [14]. The reason behind the
application of incremental analysis lies to the fact that
IRR is measuring one single alternative.
Return of Investment (ROI), is a popular accounting
method for evaluating investments. ROI defines how
much an organization gets from the spent amount of
money. Therefore, ROI helps an organization to decide
on different investment alternatives. ROI is defined as
cos
investment profit
ROI investment t  (3)
and provides a comparison of the investment result ver-
sus the investment cost [13]. Investment profit is defined
as the ex pec ted inco me min us the inve stmen t cos t, whe re
the investment cost is the initial cost plu s the cost during
the life-cycle of the project.
Finally, Payback Period (PP) is used to evaluate in-
vestments where the payback period of the investment
(the period needed to replenish the initial cost) is com-
pared to a predefined time period, the so-called cut-off
period. It is calculated by deducting the initial cost of an
investment from the financial benefits of the investment
throughout the def ined periods (months, years, etc.). E.g.
if the payback period is three years and the result of the
above mentioned operation on the third year (or earlier)
is bigger than zero, the investment must take place, oth-
erwise it must not.
2.2. Multi-Criteria Decision Making with
Analytic al Pro cesses
An MCDM method (like AHP and ANP) overcomes the

K. VIGLAS ET AL. 144
vantages have caused the wide application of
A

Estimation is executed in every tree level
tie lt points related with the practical
ap
rather a special case (or an extension) of AHP
[9
limitations of the conventional financial methods as it
combines a set of criteria in order to reach to a decision,
handles both quantitative and qualitative criteria and is
applicable to both individual and group-based decision
making.
These ad
HP to multi-criteria decision making problems, in
many different sectors, including software project man-
agement and IT system procurement. Two representative
examples of software engineering project management
problems that gained a lot of attention to be suppo rted by
AHP are: 1) prioritizing software requirements and 2)
selecting Component off the Self systems (COTS). In
both problems AHP has been used to compare software
requirements [15] or COTS products [16,17] by taking
into account the relative importance between value and
cost of each requirement/COTS product, respectively.
AHP is based on three basic conce pts (see Figure 1):
Complexity Analysis: A hierarchical tree is created
with criteria, sub-criteria and alternative solutions as
the leaves.
 Calculation/
based on a 1 to 9 scale in order to measure priorities.
More specifically, a pair wise comparison takes place
in every tree level with regards to the parent node.
The goal node in the hierarchical tree exists only to
highlight the top-down analysis of the methodology.
Synthesis with ultimate goal to extract the fin al priori-
s of the alternatives.
There are two difficu
plication of AHP. Firstly, when determining “crisp”
comparative values, any uncertainties on judgments of
decision makers cannot be easily handled and, secondly,
when there are dependencies among the selection criteria.
In such a case, the Analytic Network Process (ANP) can
be used, an AHP extension that handles both intra- and
inter-dependencies among clusters of selection criteria
[9,18].
ANP is
] and is based on the same principles as AHP. Its ba-
Figure 1. AHP hierarchical tree.
sic differencnstead of a
2.3. Performance Measurement with Balanced
alanced ScoreCard (BSC) [19] is a methodology that
d with four discrete perspec-
tiv
he
dri
e is that a network is created i
hierarchy (see Figure 2), where there is no specific Goal
object but instead the sub-criteria of AHP stand as the
elements of the objects (clusters in ANP terminology).
Still, the main difference is the feedback, where the
evaluation of criteria with regard to alternatives is al-
lowed, against the top -down approach of AHP where the
importance of the alternatives is examined with regards
to criteria. The goal of selecting the best alternative is
utterly produced by the evaluation of the objects/clusters
versus the alternatives and vice versa.
Scorecard
B
has achieved wide publicity among both scientists and
managers. BSC is being widely accepted since it fills the
gap between the development of a strategy and its reali-
zation by supporting and linking critical management
processes [20]. More specifically, it takes conventional
financial measures like ROI and payback period and
complements them with additional ones that reflect cus-
tomer satisfaction, internal business processes, and the
ability to learn and grow.
The above idea is modele
es, which are used to sp lit the overall business strategy
to 1) Financial, 2) Customer, 3) Internal Business Process,
and 4) Learning & Growth dimensi ons (see Figure 3).
1) The Learning & Growth Perspective provides t
vers for achieving the objectives of the other three
areas of the scorecard. The key factors that constitute this
perspective are: employ capabilities, information system
capabilities and employee motivation, empowerment etc.
Figure 2. Connec tions in a network.

K. VIGLAS ET AL.145
Figure 3. Synopsis of BSC perspe c t ive s (Adapte d from the Balanced Scorecard by Kaplan & Norton).
2) The Business Process Perspective refers to internal
bu
at
m
typical fi-
na
ed by a strategy map.
A
. The Proposed Approach
he proposed approach is tackling the problem of strat-
g this approach, where BSC is
used for strategy development, while for the i mplementa-
tio
ples on how you can
tra
l for crafting the strategy of
th
in the case of “learning and growth”
pe
nalytical selection proc-
es
siness processes. Metrics (or measures) based on this
perspective allow the managers to know how well their
business is running, and whether its products and ser-
vices conform to customer requirements (the mission).
3) The Customer Perspective contains indices th
easure customer satisfaction, via analyzing customers
in groups, and via assigning business processes to prod-
ucts and services delivered to these groups.
4) The Financial perspective contains the
ncial performance measures, which are mainly related
to profitability. The measurement criteria are usually
profit, cash flow, ROI, return on invested capital (ROIC),
and economic value added (EVA).
The BSC is usually complement
strategy map is a diagram that connects organization’s
strategic objectives in explicit cause-and-effect relation-
ship and describes the way that value is created within
the organization.
3
T
egy diffusion at di fferent l e vel s wi t hi n the or gani zat i on by
offering different mechanisms at each strategic level in an
integrated fashion.
Figure 4 is illustratin
n of strategic choices traditional decision support
methodologi es are empl oyed.
Even though, this as an idea rather simple, literature
does not offer large number of exam
nsform the BSC objectives and measures identified, to
criteria used in a decision management methodologies for
taking strategic deci si ons.
In the approach used, the first step is the development
of BSC which is fundamenta
e organization. This is considered as complex task since
a strategist has to consider a large number of heterogene-
ous aspects, but on the other hand this is a well docu-
mented process.
The implementation of the identified strategic objec-
tives, especially
rspective, involves select i on of IT sy st ems, able to meet
the performance measures identified. In most cases, this
selection process is done in isolation by the IT department
of the organization and using criteria mostly referring to
the functionality of the system.
In our approach, the selection process is strategic proc-
ess which is composed of 1) an a
s and 2) a financial—investme nt evaluation process. The
analytical selection process is based on the assumption
that the performance measures of the scorecard should be
transformed to selection process criteria, in order to achi-

K. VIGLAS ET AL. 146
Figure 4. Pyramid of decision making levels within the or-
ganization.
m strategic alignment. At the same time, the
SC financial performance measures are used in a typical
y
ncerns a multinational retail organiza-
on, operating in three continents with more than 3000
framework that will optimize its
pe
eve maximu
B
investment evaluation. The end results of these two par-
allel processes are combined in a qu alitative way in order
to conclude with the selection of the IT system and the
final decision.
4. Case Stud
This case study co
ti
Points of Sale (POS).
The problem the organization faces is the integration of
its strategic plan to a
rformance measurement and, as a consequence, will
propose measures (alternatives) to be taken to improve it
in terms of Information Systems.
Having in mind the pyramid of decision making levels
within the org anization (Figure 4), an integrated solution
is obvious to be required to provide added value and re-
usability to the organization. In our approach, in order to
support the decision process at the highest level, the ap-
plication of BSC is suggested, for defining the strategic
objectives and the necessary initiatives that the organiza-
tion has to take. For the middle level decision support,
ANP is used in order to assist the process of selecting the
most beneficial QM S.
To give an insight on the quality management process
within the org anizatio n, it is h an d led manu all y or with th e
use of ad-hoc applications developed locally at each dif-
ferent country of the multi-n ational company. The qu ality
management process includes quality controls, report
creation towards the top management and compliance
control against to quality standards.
The different threads of the quality process are pre-
sented in Figure 5. The complexity of this p rocess is sub-
stantial since it involves a large number of stakeholders, a
large number of quality controls and control points. Some
process statistics taken fo r a 4 years’ period are presented
in Table 1. The need is evident to merge multiple and
interlinked activities under a common IT platform of
managem ent and processing.
4.1. Developing the Scorecard
The first step of our approach was the development of
the strategy map for the organization under study. As
we already mentioned, the strategy map defines the
Figure 5. Quality management system.

K. VIGLAS ET AL.147
Table 1. Quality control in the organization.
No Activity
Volume for a
Period of 4
Years
1 Qu a lity C o nt r o l o f P r o d u c t s ( L a b Te s t s ) 9700
2 Quality Control at the Level of Stores 850
3 Supplier Control (Providers) 150
4 Supplier Control (Agriculture Products) 60
5 Production of Reports 1300
6 Customer Complaints 4500
7 Crisis Management 200
strategic objectives of the organization for every per-
spective of the balanced scorecard and interlinks these
objectives with cause-effect relations. The cause-effect
relations define a finish-to-start relationship between
objectives. Figure 6 presents the strategic map for the
organization. The arrow connections imply cause-
effect relation, e.g. Process Quality Assu
ess Stauality
Improvement or that Service Quality A a
prerisite for Custtisfaction.
The second step of our method is the detefini-
tion e strategic objec-
tive will sed to
measure the performance related with the objective, its
scope, the measurement frequency (yearly, monthly), etc.
Additionally, in order to b e able to compare and quan-
titatively evaluate each objective, we need to assign a
weight to each goal within the perspective. A snapshot of
the detailed definition of some of the strategic ob jectives
is presented in Table 2. As it appears in the table, the
implementation of a QMS is the proposed strategic ini-
tiative for some objectives (CP2, CP3).
In order to calculate the priorities of the organiza-
tion’s strategy regarding the initiatives to take, we add
the products of the weights of every strategic objective
with the weight of the hosting perspective [21]. E.g.
Process standardization, Process Quality Assurance
and Service Quality Assurance suggest QMS as the
preferred initiative to be undertaken. After doing these
calculations, we end up to a score of 39% for QMS as
the suggested strategic initiative. For all calculons,
Figure 7, we have used the tool Bal-
bu
rance and Proc-as presented in
ndardization are prerequisites for Product Q
ssurance is
ailed d
equomer Sa
ofach strategic objective. For each
weneed to define the metrics that be u
ati
anced Scorecard Designer (http://www.strategy2act.com).
Balanced Scorecard Designer is a tool that helps in
ilding balance scorecards.
Having decided that the correct strategic initiative to
be undertaken is the development of QMS, the next
step is to proceed with the evaluation of alternative
Figure 6. Strategic map of the organization.

K. VIGLAS ET AL. 148
Table 2. Snapshot of the analysis of Strategic goals and performance measures.
Strategic
objectives Objective description Performance measures TargetFrequency Weight Strategic Ini- tiatives
Product quality
improvement
(CP2)
The target is to improve product
quality and its maintenance to
high level, allowing the organi-
zation to be considered as “best
in class”
Minimization of the ratio of
defective items versus total
items produced (per prod-
uct)
0.01%Yearly 4
OMS Imple-
mentation
Customer
Satisfaction (CP3)
The target is to maintain cus-
tomer satisfaction in the highest
level
Minimization of the com-
plaints per store ratio 5% Yearly 3 OMS Imple- mentation
Product adjust-
ment to custom-
ers’ needs (CP4)
The target is to adjust products
customer’ needs ,using data like
cultural habits, geographical
position or customer’ habits for
sales maximization
Ratio of special products’
gross profit per their sales 10% Yearly 1 OMS Imple- mentation
QM systems.
As a last comment, what must be clear for BSC is
that it requires the participation of all the organization,
lead by a project team or in other cases the manage-
ment team, for all the steps mentioned above.
4.2. Applying Financial Measures
The organization sent Request for Information (RFI) to
different vendors in a form of questionnaire and received
information from 10 vendors referring to 10 different
QMS systems. The project team evaluated them and
eliminated those with the lower perfor
to three alternatives, t
nted in the case stlecting th
tihe succed of activity by
uina
th2].
itial analys d
period of six yearsost oner-
sh). TCO d
dt related with an IT sys
quince ierating cost
och co
w ve
Table 3 presentsfor a period of
preferred alternatives [21]. A short
ivities of
r producing the
spent on each
ctivity at the company level, number that came as a
The calculations of the NPV, IRR and ROI for the
three alternatives are presented in Table 5.
Consequently, one can progress the alternative B as
the most preferable solution, as it proves to have the
biggest ROI, as well as NPV and ROI. Still, this result
depends only on financial measures and is not taking into
consideration intangible criteria, such as functionality,
that are examined within the ANP framework.
4.3. Applying Analytical Network Process
The next step in our approach was the application ofan
for evaluation and selection. In our
P due to its superior-
ity in defin cri
To setupasvalof thdif-
oducts/vendors a set of criteria was created by
g duriorksho(see Tf-
s, the criteria were sorted in four major categories:
Cost, Functionality, Technology and Supplier. For each
themre w criter(that nd
w). In thst steprithb-
lem and scope definition takes place, as follows:
the criteria anr conntions P
an bFigure 8:
Among others, it is worth to observe 3 points:
1) The element DB connectivity and the relative clus-
is an element in this cluster that is
co
the scale presented, for defining their relative importance.
mance, resulting in analytical method
he systems A, B, C (this process is case, we have decided to follow AN
ot presenudy). Se
ssful ones in their fiele alterna- ves from t
sing the prelim
e decision [2
An in
ry elimination increases the
is of the investments was
calculati
quality of
one for a
f Ow
ferent pr
consultin
terward
ng the Total C
is a financial estimate that
costs
ip (TCOetermines
tem. It is
one of
belo
irect and indirec
te useful, si
f a system whi
ith the initial in
t takes into account the op
in the case of IT system is
stment and significant.
the TCO for the QMS
mparable All
model c
six years for the three
description of the process to end up to the three preferred
alternatives is described in paragraph 3.3.
In order to apply the financial methods we need to
calculate the benefits of installing a QMS system to the
organization by qualifying and valuating the act
Table 1. The key metric that wa s used fo
financial benefits is the averag e man-hours
a
feedback from the HR department of the organization
[21] (see Table 4).
Additionally, we calculate an increase of 10% for each
year, attributed to the organization’s organic growth.
ters/elements it is connected (can be seen in bold squares
for clusters Alternatives and Supplier)
2) Feedback takes place between clusters Alternatives
and Cost
3) Inner-dependence appears in cluster Functionality,
meaning that there
ing relatio
the bnships between is for the e the uation teria. e three
users ng a wp able 6). A
, the
e 1ere sub of the ANP algo
ia can be fou
m, the pro
d theiecof the AN
e seen in
nnected with an element within Functionality.
Like AHP, pair wise comparisons take place between
elements based on the Saaty’s Fundamental Scale of
Absolute Numbers [23] (see Table 7).
Each element in compared to all other elements, using

K. VIGLAS ET AL.149
Table 3. TCO for QMS.
Alternative Alternative
T
A B Alternative
C
Investment
Cost 248,500 134,000 188,000
Operating
Costs 147,125 67,000 106,500
Table 4. Financial benefit realized per activity.
Activities Financial Benefit
Quality control of products (lab tests) 43,650
Quality con trol at the le vel of sto res 61,200
Supp lie r co n t ro l (p ro v id ers ) 5400
Supplier control (agriculture products) 2160
Production of Reports 5850
Customer complaints 20,250
Crisis management 3600
142,110
Table 5. Application of NPV, IRR and ROI for QMS.
Alternative A Alternative B Alternative C
NPV 316753.90 495237.08 409694.67
IRR 45.08% 102.25% 67.42%
ROI 119.30% 331.64% 194.60%
Table 6. QMS Selection cr ite ria.
C1: Cost C11: Implementation cost
C12: License cost
C13: Maintenance cost
C2: Functionality C22: Flexibility
C23: Product audit
C24: Repor
C21: Email notification
ts
ile
C25: Store audit
C26: Supplier audit
C27: Security
C31: Company prof
C3: Supplier C32: Implementation time
C33: International solution
C4: Technology
C41: Databases
C42: DB connectivity
C43: Infrastructure
C44: Migration tools
C45: Reporting tools
C46: Web application
able 7. Saaty’s Fundamental Scale of Absolute Numbers.
Numerical Ratf Preferences ing Verbal Judgments o
9 Exred tremely prefer
8 ry rem
Very streferred
Strongly strongly
Stroerred
4 Moderately to strongly
2 Equally to moderately
1 Equally pr
Vestrongly to extely
7 ongly pr
to very6
5 ngly pref
3 Moderately preferred
eferred
For example, the pair wise comparisonn Li-
c and Main cost
smoderately more imrtant (3
times) than Implementation cost and Mainte cost.
I cost hav same
i the pair wise comparmatrix,
wate the priority of eac
tbution to the overall goallecting
the best system. This step of the process is nthe-
the results are presented in Table 8.
Fon)
vec ri-
ate positions in the supermatrix. In our case study we
hav uDe(h-
persionso operatonstratole
ps (T
ing sub-matr as colurow
“nappears mentionde-
ce of element C24: Reports with regards to all the
ther elements of the values are
zero this imp not depend
to the examined elemehip appears
in the rix havister Alterna-
tives and as row the clhe ele-
ments of Technology were compared with regards to the
Alternatives (in bold tvalues for System B). Finally,
total zero values in theas column the
cluy anster Cost indicate
that there is no connec of Cost with regards
to Technology, as it cin the model (no
arrow from Technology
The last step of syntesis, pair wise comparisons take
placelusterof the Cluster
Weig [24].ing Cluster
Weights Matrix is tha an ele-
ment with the highest priority. That does not necessarily
mean that this element has the highest priority among all
other elements of the other clusters [9]. There comes the
need to compare all the clusters in pairs with regards
betwee
ense cost, Implementation costtenance
hows that License cost is po
enanc
mplementation and Maintenancee the
mportance.
After constructingison
e can now calculh element in
erms of its contri of se
called sy
sis and
llowing the same process, all priority (or Eige
tors are produced and are finally put in the approp
esed Super cisions tool ttp://www.su
-deci
roces .com) t able 9). e and deme the wh
Check on theix havingmn and
Functio
enden lity”, it athe aforeed inner p
o cluster it belong s. When
lies that the specific ele ments do
nt. Feedback relations
e clusub matng as column th
uster Technology, where t
he
sub-matrix having
ster Technologd as row the clu
tion between
an be also seen
to Cost).
h
between c
hts Matrixs for the creation The reason of creat
t in every cluster there is

K. VIGLAS ET AL. 150
Fie 7. BSC mode
gurl of the organization.
Figure 8. ANP model for QMS selection.
Figure 9. Pair wise comparison of cluster Cost with regards
to System B.
Table 8. Pair wise compar is on mat ri x.
C11 C12 C13 Priority vector
C11 1.000 0.333 1.000
C12 3.000 1.000 3.000
C13 1.000 0.333 1.000
to a higher control criterion. These comparisons produce
a priority vector for every cluster with regards to the
others and are used to weigh (multiply) the relative sub
matrices of the unweighted supermatrix. E.g. the first
value of this vector (first column, first row) is multiplied
with all the elements of the relative sub matrix of the
unweighted supermatrix, the second value (first column,
second row) with the sub matrix having as column the
cluster Alternatives and as row the Cost, etc.
The result of this process is the production of the
weighted or stochastic supermatrix (Table 11). The
transformation to a stochastic per column or simpler
stochastic comes out of the fact that the fina l priorities of
the elements have to meet some reduction and cyclicity
needs [9]. As every column’s summation equals to one,
the intuitive reason is to present the priority of every
element throughout the network.
The weighted supermatrix is multiplied by itself until
the supermatrix’s row values converge to the same value
for each column of the matrix. The result is the limiting
supermatrix (Table 12). Its columns (normalized per
cluster) constitute the final priorities of the network, in-
cluding alternatives.
Focusing to the alternatives sub-matrix, the alternative
that has the highest priority shall be chosen and for our
case study, System A shall be the proposed solution
scoring 41.7%. An interesting feature is the (lowest of
all) score of System B, that was progressed by the finan-
cial methodologies. This can be explained by taking th
sum of t0.043
.031 +
0% of the total network dependency, it gives a good
reason why Cost (and in general tangible criteria) does
not hold the major role in IS selection.
5. Conclusions
The purpose of this paper was to present a BSC-ANP
unified model for IS selection. Through its case study,
this model was ex ecuted for the selection of a QMS for a
multinational retail organization. It is the first time tha
such a mzation’s
trategy in total. Previous attempts were using ANP to
“evaluate” the importance of one BSC perspective over
the others for insurance or manufacturing organizations
[25,26], which is far from the main goal of this model.
Its basic principles support the execution of the or-
ganization’s strategy by approaching it in high level
when applying BSC and to a lower level when executing
NP in order to assist the selection problem that comes
ut as the proposed initiative from BSC. This is where
perational performance of the system se-
cted by ANP will then have to b e measured in terms of
BSC.
Thadded og Andl meth-
ods liPVis proven byeeing the results of
those ods syBcost effective) ap-
pears thtablnive (Table 5). The
ct that the Cost cluster reached the third place (0.204,
e
+ he 1 column of the Cost sub-matrix (
0.021 = 0.096). As it only results in less than0
1
t
odel is presented to assist an organi
s
A
o
the strong connection between the two methodologies
lies, as the o
le
e valuef usinNP a not financia
ke N, IRR easily s
methologie wherestem (
to bee mos prefere alterat
fa
see Ta ble 10) compared to Alternatives, behind Function-
ality (0.427) and Technology (0.204), gives a very good
reason why the financial methods fail to quantify intangi-
ble criteria. The incorporation of the process owners to the
evaluation phase is another reason why ANP is a good
choice. In that way, resistance to change is significantly

K. VIGLAS ET AL.151
hteTable 9. Un-Weigd Supermatrix.
Table 10. Cluste
Alternatives C1:Cost
r Weights Matrix.
C2:Functionality C3:Supplier C4:Tcchnology
Alternatives 0.074 0.156 0.109 0.667 0.250
C1:Cost 0.204 0.000
C2:Functionality 0.427 0.659
C3:Supplier 0.091 0.185
C4:Tcchnology 0.204 0.000
Row sum 1.000 1.000
0.309 0.000 0.000
0.582 0.000 0.000
0.000 0.333 0.750
0.000 0.000 0.000
1.000 1.000 1.000
Table 11. Weighted Supermatrix.
lower than the one that could be caused if system B was
selected. This would happen because selection based on
financial methodologies is done by a team of experts and
ot from the operational teams. Seeing this the opposite
way, this incorporation of process owne
creased resources and increased resources mean r
costs. Furthssible the n
criteria coulditional evons (mead-
ditional administrative costs). Nevertheless, selection
systems like the one used in this study (SuperDecisions)
mitigate these risks and lower the calculation/admini-
stration costs, not to mention the cost of a syst that
d based only on cost-effective criteria but
d its
, the ompgards to
Altes is not ava The obvioreasonable
nrs means in-will be selecte
highe
soermore, po
ddch toangeselecti imply aaluatining a
em
will not succee
A AHP goals. feaf cs for ture oarisoth ren wi
ernativilable. us and

K. VIGLAS ET AL. 152
Tablimiting suprix.
e 12. Lermat
comparison e.g. of the elements of cluster Cost with re-
gards to Alternatives is something that cannot happen in
AHP and clearly shows the limitations of this methodol-
gy. Still, its wide expansion in all kinds of problems
A decision criterion that the organization co
cal can be easily added while the selection problem is
ongoing. In that way, due to its general directives, the
use of the model can be validated for all kinds of organi-
zations and IS.
Enhancements can also be made in terms of monitor-
ing the performance of the BSC model. As each strategic
objective is monitored on a certain frequency, it would
be interesting to integrate an iterative process in the
model to measure the added value that came out of the
implementation of the selected QMS (System A) and its
“score” versus the relative strategic objectives (Figure 7).
Tools like the one used in this study can easily integrate
such processes through business intelligence techniques
to give the chance to top management to have a real-time
view on the strategy execution.
ledgements
7. References
[1] Z. Irani and P. Love, “Information Systems Evaluation:
Past, Present and Future,” European Journal of Informa-
tion Systems, Vol. 10, No. 4, 2001, pp. 183-188.
04.1293068
o
through the years lik e project delivery method [27], ERP
selection [28], etc. makes AHP a reference among
MCDM methodologies.
While we believe that the model presented provides
value, there are areas for future enhancements and vali-
dation. It is acknowledged that the decision levels in-
volved in any selection problem can vary, depending on
the organization. Indeed, this is one of the strengths of
ANP: the ability to adap t its framework to custom needs.
[2] Z. Irani and P. Love, “Evaluating Information Systems.
Public and Private Sector,” 1st Edition, Butterworth-
Heinemann, Hungary, 2008.
[3] H. Erdogmus, J. Favaro and W. Strigel, “Return on In-
vestment,” IEEE Software, Vol. 21, No. 3, 2004, pp.
18-22.
nsiders criti-
6. Acknow
The authors would like to thank Mr. Robert Elliott from
AKS-Labs for his kind offer of a full-feature version of
the Balanced Scorecard Designer tool.
doi:10.1109/MS.20
. Westerfield and B. Jordan, “Fundamentals of
Corporate Finance,” Times Mirror Professional Publish-
ing, New York, 1996.
[6] A. S. Jadhav and R. M. Sonar, “Evaluating and Selecting
Software Packages: A Review,” Information and Soft-
ware Technology, Vol. 51, No. 3, 2009, pp. 555-563.
doi:10.1016/j.infsof.2008.09.003
[4] B. Boehm, “Software Engineering Economics,” Prentice
Hall, Upper Saddle River, 1981.
[5] S. Ross, R
[7] O. S. Vaidya and S. Kumar, “Analytic Hierarchy Process:
An Overview of Applications,” European Journal of Op-
erational Research, Vol. 169, No. 1, 1996, pp. 1-29.
doi:10.1016/j.ejor.2004.04.028
[8] T. L. Saaty, “The Analytic Hierarchy Process: Planning,
Priority Setting, Resource Allocation,” McGraw-Hill,
Pittsburgh, 1980.
[9] T. L. Saaty, “Decision Making with Dependence and
Feedback,” RWS Publications, Pittsburgh, 1996.
[10] T. L. Saaty, “Theory and Applications of the Analytic
Network Process: Decision Making with Benefits, Op-
portunities, Costs, and Risks,” RWS Publications, Pitts-
ess,” International Journal of Production Economics, Vol.
burgh, 2005.
[11] K. Hafeez, Y. Zhang and N. Malak, “Determining Key
Capabilities of a Firm Using Analytic Hierarchy Proc-

K. VIGLAS ET AL.153
3(01)00141-4
76, No. 1, 2002, pp. 39-51.
doi:10.1016/S0925-527
[1 2] R. A. Brealey and S. C. Myers, “Capital Investment and
Valuation,” McGraw/Hill, New York, 2002.
[13] M. Schniederjans, J. Hamaker and A. Schniederjans,
“Information Technology Investment: Decision Making
Technology,” World Scientific Publishing Company,
Singapore, 2004.
[14] L. T. Blank and A. J. Tarquin, “Engineering Economy,”
3rd Edition, McGraw-Hill, NewYork, 1989.
[15] J. Karlsson and K. Ryan, “A Cost-Value Approach for
Prioritizing Requirements,” IEEE Software, Vol. 14, No.
5, 1997, pp. 67-74.doi:10.1109/52.605933
[16] J. Kontio, “A Case Study in Applying a Systematic
Method for COTS Selection,” Proceedings of the 18th
International Conference on Software Engineering, IEEE
Press, New York, 1996. doi:10.1109/ICSE.1996.493416
[17] A. Lozano-Tello and A. Gomez Perez, “BAREMO: How
to Choose the Appropriate Software Component Using
the Analytic Hierarchy Process,” Proceedings of the In-
ternational Conference on Software Engineering and
Knowledge Engineering (SEKE), Skokie, 21-23 June,
2002.
[18] V. Gerogiannis, P. Fitsilis, G. Kakarotzas and A. Tzikas,
“Involving Stakeholders in the Selection of a Project and
Portfolio Management Tool,” 21st Greek Operation Re-
28-29 May 2009.
9] R. S. Kaplan and D. P. Norton, “The Balanced Scorecard:
recard Concept: An
port,” Long Range Planning, Vol. 34, No.
search Conference, Athens,
[1
Translating Strategy into Action,” Harvard Business
Press, Massachusetts, 1996.
[20] H. Ahn, “Applying the Balanced Sco
Experience Re
4, 2001, pp. 441-461.
doi:10.1016/S0024-6301(01)00057-7
[21] K. Viglas, “Evaluation/Selection Methodologies of Infor-
Process
apm.2006.10.002
mat i o n Syst e m s ,” Hell e n i c Ope n U n i v er sity, Patr a s, 2009.
[22] C. Gencer and D. Gürpinar, “Analytic Network
in Supplier Selection: A Case Study in an Electronic
Firm,” Applied Mathematical Modelling, Vol. 31, No. 11,
2007, pp. 2475-2486.
doi:10.1016/j.
: An Inter-
, pp. 630-649.
orecard and
a Manufacturing Firm,” Expert Systems
[23] T. L. Saaty, “Fundamentals of the Analytic Network
Process,” ISAHP, Kobe, 1999.
[24] S. Percin, “Using the ANP Approach in Selecting and
Benchmarking ERP Systems,” Benchmarking
national Journal, Vol. 15, No. 5, 2008
[25] R. Ak and B. Oztaysi, “Performance Measurement of
Insurance Companies by Using Balanced Sc
ANP,” Istanbul Technical University, Istanbul, 2009.
[26] I. Yuksel and M. Dagdeviren, “Using the Fuzzy Analytic
Network Process (ANP) for Balanced Scorecard (BSC):
A Case Study for
with Applications, Vol. 37, No. 2, 2010, pp. 1270-1278.
doi:10.1016/j.eswa.2009.06.002
[27] M. I. Al Khalil, “Selecting the Appropriate Project De-
livery Method Using AHP,” International Journal of
Project Management, Vol. 20, No. 6, 2002, pp. 469-474.
doi:10.1016/S0263-7863(01)00032-1
[28] C. C. Wei, C. F. Chien and M. J. J. Wang, “An
AHP-Based Approach to ERP System Selection,” Inter-
national Journal of Production Economics, Vol. 96, No.
1, 2005, pp. 47-62.
doi:10.1016/j.ijpe.2004.03.004