Journal of Software Engineering and Applications, 2011, 4, 181-186
doi:10.4236/jsea.2011.43020 Published Online March 2011 (http://www.SciRP.org/journal/jsea)
181
Software Reliability Early Prediction in
Architectural Design Phase: Overview
and Limitations
Lalit Kumar Singh1, Anil Kumar Tripathi2, Gopika Vinod3
1Department of Atomic Energy, Nuclear Power Corporation of India Limited, Mumbai, India; 2Department of Computer Sci ence and
Engi neeri ng, Institute of Technology -Banaras Hindu University, Varanasi, India; 3Department of Atomic Energy, Bhabha Atomic Re-
search Center, Mumbai, India.
Received January 31st, 2011; revised March 7th, 2011; accepted March 10th, 2011.
ABSTRACT
In recent times, computer based systems are frequently used for protection and control in the various industries viz Nu-
clear, Electrical, Mechanical, Civil, Electronics, Medical, etc. From the operating experience of those computer based
systems, it has been found that the failure of which can lead to the severe damage to equipments or environmental harm.
The culprit of this accident is nobody other than our software, whose reliability has not been ensured in those condi-
tions. Also for real time system, throughput of the system and average response time are very important cons tructs/ me-
trics of reliability. Moreover neither of the so ftware reliability model is availab le which can be fitted generically for all
kinds of software. So, we can ensure reliability at the early stage i.e. during design phase b y arch itectu ring the softwa re
in a better way. The objective of this paper is to provide an overview of the state-of-the-art research in the area of ar-
chitecture-based so ftware reliability analysis. We then describ e the shortcomings and the limiting assump- tions under-
lying the prevalent research. We also propose various approaches which have the potential to address the existing li-
mitations
Keywords: Software Reliability, Software Engineering, DTMC, Early Prediction
1. Introduction
The Control and Instrumentation systems are widely
based on software in today’s era and software is the root
cause of most of today’s system problems. It should be
noticed that failures never occur if the software is not
used but at the same time without software we can’t get
comfort and comfort is the necessity of everybody. The
quality and longevity of a software system is determined
by its ar ch itec tu r e. So f twa r e ar chitecture is a “first cut” at
solving the problem and designing the system. The im-
portance of the software architecture lies in earliest de-
sign decisions, addressing first design artifact & key to
systematic use. Analysis of a system at the architectural
level enables the choice of the right architecture for the
system under consideration, thus saving major potential
modifications later in the development cycle or tuning
the system after deployment.
Reliability of the safety critical & safety related sys-
tems are of prime importances, which are computer based
systems. So, the software that are used in these systems
must be reliable, for which software professionals follow
so many standards and get software verified by Inde-
pendent Verification and Validation team before deploy-
ing it on the site. But during Verification & Validation, it
is impossible to generate all the possible test cases and it
may possible that in real life a test case might has been
executed, which we had not thought during testing. So,
we can never claim that my software is error free—in fact
no software in this world is error free.
For this reason our software experts/scientists starts
thinking to ensure th e reliability of the software especial-
ly when those are going to be deployed in a safety critical
systems. So, they had given/proposed so many software
reliability models to ensure the reliability of the software.
But unfortunately till date no model fits in all kinds of
software, means there is no generic model that can be
used to predict the reliability of all kinds of software.
These models are analytically derived from assumptions.
The emphasis is on developing the model, the interpreta-

Software Reliability Early Prediction in Architectural Design Phase: Overview and Limitations 182
tion of the model assumptions, and the physical meaning
of the parameters. These types of reliability models are
known as “software reliability growth models”. [Fenton
& Pfleeger] These models attempt to statically correlate
defect detection data with known functions such as an
exponential function. If the correlation is good, the known
function can be used to predict the future behavior. The
reasons that these reliability models lack the needed str-
ength to excel in eliminating errors in software environ-
ment are:
1) The misconception of fault & failure phenomena [1]
2) Inaccurate modeling parameters
3) Difficulty in selecting the reliability models
4) Difficulty in building the software operational pro-
file
So, different approach has been started [2-12] i.e. to
predict the reliability based on the design parameters of
software. The models based on this concept are known as
“defect density” models. These models use code charac-
teristics such as lines of code, design characteristics such
as calculate the weight of a class or architectural charac-
teristics.
The layout of the paper is organized as follows: Sec-
tion 2 provides an overview of the existing techniques. It
also describes the similarities and differences among
those approaches. Section 3 summarizes the limitations
of those approaches. Section 4 describes the proposed so-
lutions to address the stated limitations and Section 5
summarizes the paper.
2. Overview
Software systems are developed to achieve the require-
ments in an automated manner. Well, need not to say that
requirements can be functional as well as non-functional.
The critical prerequisite for a system’s success is the ex-
hibited non-functional quality, i.e. how a system does.
Every software is composed of one or more software mo-
dules. Scientists/Researchers proposed many approaches.
These approaches are more or less similar in the sense
that quantitative methods for reliability assessment de-
pend on the availability of system usage information—
i.e. a system’s operational profile. The operational profile
information is combined with the non-probabilistic beha-
vior models in order to obtain prob abilistic models which
can be used for reliability analysis. Error propagation can
also take place so modeling approach to analyze the im-
pact of error prorogation on reliability is also done [2].
So, these approaches have the common model as re-
presented in Figure 1.
But the approach to gen erate Probabilistic Models and
to estimate the software reliability differs from one appr-
oach to other app roach [3]. Probab ilistic behavior o f a so-
ftware system is often modeled and analyzed with dis-
crete-time Markov chains. DTMC [4] consists of a set of
states; each state has corresponding probabilities of tran-
sitioning to other states. DTMCs embody a basic way of
modeling, and reasoning about probabilistic behavior;
further demands, including the need to more faithfully re-
present software systems, enabled a proliferation of pro-
babilistic automat a formalisms [5]. As an example DTMC
modeling of Robot has been shown in Figure 2. Using
DTMC modeling there are two main paradigms for mod-
eling probabilistic behaviors: the generative, which can
generate some actions and the reactive paradigm, which
can react to the external actions. In this case calculating
the system-wide reliability is equivalent to computing the
steady state probabilities of DTMC. To illustrate th is ap-
proach, consider DTMC model of an example Robot,
which is depicted in Figure 2. The architecture consists
of Controller, Sensor, Follower, and GUI components.
Several difficulties arise when deriving transition proba-
Figure 1. Existing model for reliability prediction based on architecture.

Software Reliability Early Prediction in Architectural Design Phase: Overview and Limitations 183
bilities from an operational profile:
1) GUI can invoke different Controller’s operations
depending on the navigation model.
2) GUI and Follower may be concurrent to each other.
Some researchers also proposed Probabilistic model,
based on the control flow graph as shown in Figure 3,
where p1,2 is the probability of going from node 1 to node
2. The enumeration of paths could be conducted algo-
rithmically, experimentally or by simulation. The reli-
ability of each path is obtained as a product of the reli-
abilities of the components along path. For example ap-
plication software shown in Figure 3, possible execution
path is 1, 3, 5, 6 and its reliability is obtained by R1, R3,
R5, R6. The application reliability can be estimated by
averaging path reliabilities. But the main drawback is
that this approach doesn’t give accu rate reliability due to
looping effects, like- in the path 1, 2, 4, 21···*, 6. The sub-
path 2, 4, 2 can occur infinite number of times. The clas-
sification of architecture-based software reliability mod-
els can be understo od by Figure 4 [6].
3. Limitations
In this section we discuss th e limitations of the prev alent
state-based architecture-based analysis techniques. The
limitations of the existing approaches can be classified
into—1) modeling, 2) analysis 3) parameter estimation 4)
validation and 5) optimization. Usually modeling limita-
tions are due to the assumptions we made to ensure mod-
el expansibility, which may l ead to unreliable estimation.
Analysis limitations are due to lacking of analysis tech-
niques. Parameter Estimation limitations are due to non
consideration of different software artifacts. Validation
limitations are due to paying little effort. Optimization
limitations are due to non consideration of complex inte-
ractions between components in the archit ectural design.
Figure 2. DTMC model of an example Robot.
Figure 3. Probabilistic control flow graph of an example
application software.
Figure 4. Classification of architecture-based software reliability models.

Software Reliability Early Prediction in Architectural Design Phase: Overview and Limitations 184
3.1. Modeling Limitations
1) A practice in which an engineer combines an operatio-
nal profile and a non pro babilistic specifications to direc-
tly produce an analysis-enabled generative model is tedi-
ous, non-intuitive and error pro ne.
2) The operational pro file information that the existing
approaches assume available is often just a subset of the
available information.
3) Support for discovery and modeling of error states
is not clear or accurate
4) Existing DTMC based models assume that at a time
the application can be in one state only which is not v alid
for today’s complex systems.
5) Also the failure of one component can pass on its
impact on other components as well which has not been
taken care in anyone of the approaches.
6) None of the approach has taken into consideration,
the nature of the interface between the components, as
there is a possibility that components may be distributed
with the advanced technologies like RMI, RPC, etc.
7) The architectural style of different components of
the same application software may be different, for which
we suspect to fit the common reliability approach on all
the components. Also dynamically i.e. when application
operates its architecture also changes dynamically.
3.2. Analysis Limitations
In some approaches Hidden Markov model has been used
when it is difficult to have the surety of next probabilistic
transition. But in this case the transition matrix and ob-
servation probability matrix (which represents the prob-
ability of observing event in a particular state) has been
initialized randomly, which may not be accurate.
3.3. Parameter Estimation Limitations
The system-level model can be analyzed using tradition al
DTMC analysis [7], whose complexity is O(n3), where n
is the number of states. Generally large complex software
systems have thousands of states, which could be very
expensive to solve the DTMC model.
3.4. Validation Limitations
The less effort has been paid to valid ate the predicted re-
liability, based on arch itectural design with the estimated
reliability, just before product release to ensure the cor-
rectness of the predicted methodology so that, it can be
applied to the future projects.
3.5. Optimization Limitations
Reliability p rediction based on architecture can optimized
if we success to optimize the software architecture. So-
metimes it is noticed that architects design the software
in a complex manner, full of tight coupling and low co-
hesion, which is a poor quality attributes of an architect-
ture [8]. For resource allocation optimization see [9]
So, in this section we have seen the overview of the
existing approaches to predict the software reliability in
architecture phase. We have also compared the approa-
ches, identified the similarities and differences. We also
identified the various limitatio ns which needs to ad dress.
4. Proposed Solutions to Address Limitations
We have seen many limitations of the existing appro ach-
es which are crucial for reliability prediction. We propo-
sed the solutions to ad dress the above limitations.
As we have seen the two major problems when we il-
lustrated DTMC model of an example Robot (Figure 2)
[10,11]. The solution can be sough t in more intu itive way
by:
1) Specifying systems behavior in terms of scenarios
for which use case diagram can help a lot [12].
2) Try to analyze the transition probabilities between
different scenarios. Sequence diagrams, swimlane diagra-
ms, collaboration diagram can help in this regard
3) Specify the component failure probabilities. The
above limitations, based on classification, can be addres-
sed with the help of the following respective approaches:
4.1. Modeling Limitations
1) We feel that an operational profile and non probabili-
stic specifications can be more intuitively combined by
handling output actions and those actions which are con-
trolled by other components or external environments.
Output actions are basically controlled actions. So we can
ease the generation of analysis-enabled generative model
as well as it will be less likely to be error prone
2) Operational profile, a most important construct to
create DTMC model. Hybrid information must be taken
into consideration for deriving the operational profile.
Though, it is impossible to identify accurate operational
profile but by consi dering va rious hy brid sou rces of infor -
mation, more near operational profile can be identified.
The suggested information sources are: i) Domain expert;
ii) Similar components/software that are already being
used somewhere iii) simulation, mandatory in case of new
type of app lication; can be used fo r existing type as well;
iv) Requirements specifications; v) architectural models
of the system at the highest level of d etail; vi) d ata log of
the existing running applications. [13]
3) Support for discovery and modeling of error states
is not clear or accurate. The main cause could be the in-
consistencies in the architectural design. For which,
mapping of interaction protocols, dynamic behavior and
static behavior can serve the purpose [14].
4) The solution is given above in 3 steps at the begin-

Software Reliability Early Prediction in Architectural Design Phase: Overview and Limitations 185
ning of this section.
5) To represent the architecture of concurrent applica-
tions, a high-level specification mechanism, such as a
Stochastic Reward Net [15], may be used.
6) The methodology proposed by Littlewood to use the
constant failure rates to represent interface failures can
be used.
7) There are various architectural styles available viz
client-server, layered, centric, etc. Also a single applica-
tion can have many components which again may have
different architectural style. For this we propose the solu-
tion to convert into layered architectural style because it
has been seen that almost every module can be designed
by layered architecture. Then we can make the use of
Jalote’s method to solve the same.
4.2. Analysis Limitations
Architectural models can be used to derive the transition
matrix and observation probability more intuitively.
4.3. Parameter Estimation Limitations
After generating the DTMC model of the while applica-
tion and if the nu mber of states exceeds 100, we shall try
to break the DTMC model into DTMC submodels and
can solve the individual submodels. The final solution can
be sought by integrating the solution of individual sub-
models. The major issue of breaking can be done in the
following steps: 1) be cautious to decide which states
should be kept in one group. We must try the most cou-
pled (to each other) states in a single group unless the
coupled states are larger than 100; 2) properly define the
interface states which has to be introduced between brea-
ked states; 3) solve each DTMC sub models traditionally;
4) Again be cautious while integrating the solutions, have
to exclude the effect of the interface states, as being in-
troduced in 2)
4.4. Validation Limitations
The results obtained from evaluation process may be va-
lidated conceptually, which is not quantitative approach.
This can be done by consulting with the stake-holders.
4.5. Optimization Limitations
To simplify the architectural design, the architects can
make the use of Data Flow Diag rams through which sim-
ple architectural design can be derived by identifying the
transaction center and transformation center [Pressman].
5. Conclusions
In this paper we first introduced the need of the early
prediction of the software reliability. We then provided
an overview of the existing approaches of software relia-
bility early prediction. We also describe the communali-
ties and differences of those approaches. We also stated
various limitations of the existing approaches. We then
proposed approaches that could help in addressing the
stated limitations. Note that we only proposed the appr-
oaches, while providing the exact solutions using those
approaches may needs separate papers for each limitation;
which we plan to take it as a part of our future work.
REFERENCES
[1] S. Gokhale, W. E. Wong, K. S. Trivedi and J. R. Horgan,
“An Analytic Approach to Architecture-Based Software
Reliability Prediction,” Proceedings of International Per-
formance and Dependability Symposium, Durham, Sep-
tember 1998, pp. 13-22.
[2] V. Cortellessa and V. A. Grassi, “A Modeling Approach
to Analyze the Impa ct of Error Propagation on Relia bility
of Component-Based Systems,” Proceedings of the Com-
ponent-Based Software Engineering, Medford, Vol. 4608,
9-11 July 2007, pp. 140-156.
doi:10.1007/978-3-540-73551-9_10
[3] I. Krka, L. Golubchik and N. Medvidovic, “Probabilistic
Automata for Architecture-Based Reliability Assess-
ment,” QUOVADIS’ 10, Cape Town.
[4] W. J. Stewart, “Numerical Solution of Markov Chains,”
CRC Press, USA, 1991.
[5] L. Cheung et al., “Early Prediction of Software Compo-
nent Reliability,” ACM/IEEE 30th International Confer-
ence on Software Engineering, Leipzig, 10-18 May 2008,
pp. 111-128.
[6] S. Gokhale, “Architecture-Based Software Reliability
Analysis: Overview and Limitations,” IEEE Transactions
on Dependable and Secure Computing, Vol. 4, No. 1, Jan
2007, pp. 32-40. doi:10.1109/TDSC.2007.4
[7] S. S. Gokhale and K. S. Trivedi, “Reliability Prediction
and Sensitivity Analysis Based on Software Architec-
ture,” Proceedings of International Symposium on Soft-
ware Reliability Engineering, Annapolis, November 2002,
pp. 64-75.
[8] Pressman, “Software Engineering, A Practitioner’s Ap-
proach,” 6th Edition, McGraw-Hill International Edition,
New York.
[9] J. Lo, S. Kuo, M. R. Lyu and C. Huang, “Optimal Re-
source Allocation and Reliability Analysis for Compo-
nent-Based Software Applications,” Proceedings of 26th
Annual International Computer Software and Applica-
tions Conference, Oxford, August 2002, pp. 7-12.
[10] B. Boehm, J. Bhuta, D. Garlan, E. Gradman, L. G. Huang,
A. Lam, R. Madachy, N. Medvidovic, K. Meyer, S. Mey-
ers, G. Perez, K. Reinholtz, R. Roshandel and N. Rou-
quette, “Using Empirical Testbeds to Accelerate Tech-
nology Maturity and Transition: The SCRover Experi-
ence,” Proceedings of the 2004 International Symposium
on Empirical Software Engineering, IEEE Computer So-
ciety, Washington DC, 19-20 August 2004, pp. 117-126.
doi:10.1109/ISESE.2004.1334899
[11] B. Littlewood, “A Semi-Markov Model for Markov

Software Reliability Early Prediction in Architectural Design Phase: Overview and Limitations 186
Structured Software,” International Conference on Reli-
able Software, New York, Vol. 10, No. 6, June 1975, pp.
204-207.
[12] S. Yacoub, B. Cukic and H. H. Ammar, “A Scenario
Based Reliability Analysis Approach for Component
Based Software,” IEEE Transactions on Reliability, Vol.
53, No. 4, December 2004, pp. 465-480.
[13] J. D. Musa, “Operational Profiles in Software-Reliability
Engineering,” IEEE Software, Vol. 10, No. 2, 1993.
doi:10.1109/52.199724
[14] K. S. Trivedi, “Analytical Models for Architectural-Based
Software Reliability Prediction: A Unification Frame-
work,” IEEE Transactions on Reliability, Vol. 55, No. 4,
December 2006, pp. 578-590.
[15] A. Puliafito, et al., “The Evolution of Stochastic Petri-
nets,” Proceedings of World Congress Systems Simula-
tion, Singapore, September 1997, pp. 3-15.