Journal of Software Engineering and Applications, 2012, 5, 810-815
http://dx.doi.org/10.4236/jsea.2012.510093 Published Online October 2012 (http://www.SciRP.org/journal/jsea)
Lessons Learned from Practical Independent Verification
and Validation Based on IEEE 1012
Joon Ku Lee1*, Yang Mo Kim2
1Korea Atomic Energy Research Institute, Daejeon, South Korea; 2Department of Electrical Engineering, Chungnam National Uni-
versity, Daejoen, South Korea.
Email: *jklee@kaeri.re.kr, ymkim@cnu.ac. kr
Received July 27th, 2012; revised August 31st, 2012; accepted September 13th, 2012
ABSTRACT
IEEE 1012 [1] describes the SDLC phase activities for software independent verification and validation (IV & V) for
nuclear power plant in truly general and conceptual manner, which requires the upward and/or downward tailoring on
its interpretation for practical IV & V. It contains crucial and encompassing check points and guidelines to analyze the
design integr ity, without addressing the formalized an d the specific criteria for IV & V activities confirming th e techni-
cal integrity. It is necessary to list up the inspection viewpoint via interpretation of the standard that is practical review
points checking design consistency. For fruitful IV & V of Control Element Driving Mechanism Control System
(CEDMCS) softw are for Yonggwang Nuclear Powe r Plant unit 3 & 4, the specific viewpoints and appr oach are neces-
sary based on the guidelines of IEEE 1012 to enhance the system quality by considering the level of implementation of
the theoretical and the practical IV & V. Additionally IV & V guideline of IEEE 1012 does not specifically provide the
concrete measure considering the system characteristics of CEDMCS. This paper provides the seven (7) characteristic
criteria for CEDMCS IV & V, and by applying these viewpoints, the design analysis such as function, performance,
interface and exception, backward and forward requirement traceability analysis has been conducted. The requirement,
design, implementation, and test phase were only considered for IV & V in this project. This article also provides the
translation of code to map theoretical verification and validation into practical verification and validation. This paper
emphasizes the necessity of the intensive design inspection and walkthrough for requirement phase to resolve the design
faults because the IV & V of early phase of SDLC obviously contributes to find out most of critical design inconsis-
tency. Especially for test phase IV & V, it is strongly recommended to prepare the test plan docu ment which is going to
be the basis for the test coverage selection and test strategy. This test plan document should be based on the critical
characteristics of function and performance of CEDMCS. Also to guarantee the independency of V & V organization
participating in this project, and to acquire the full package of design details for IV & V, the systematic approach and
efforts with an aspect of management is highlighted among the participants.
Keywords: Korea Standard Nuclear Plant (KSNP); Instrumentation and Control (I & C); Control Element Drive
Mechanism Control System (CEDMCS); Software Development Life Cycle (SDLC); Independent
Verification and Validation (IV & V); Reactor Regulating System (RRS)
1. Introduction
Due to the hardware aging and obsolescence, the upgrade
of CEDMCS for Yonggwang 3 & 4, and Ulchin 3 & 4
nuclear power plants was brought up as necessary. In the
course of upgrade, IV & V is requested to validate the
design integrity of the system which is classified to safety-
related.
For the transparency of V & V activity, the design
team and review team is officially separated for manag-
ing the independent review of system design, which is
also the requirement from licensing organization of Korea
Institute of Nuclear Safety (KINS) shown i n Figure 1.
However IEEE 1012 code is conceptual that is appli-
cable to all the software of various fields including CE-
DMCS, it was necessary to devise application-specific
review points. This approach is enhancing the reliability
of the CEDMCS software system.
Group Utility Design Group
Coordination
Licensing Group IV & V Group
Figure 1. The organization for CEDMCS IV & V in Yong-
gwang Nuclear Power Plant 3 & 4.
*Corresponding a uthor.
Copyright © 2012 SciRes. JSEA
Lessons Learned from Practical Independent Verification and Validation Based on IEEE 1012 811
2. IV & V of CEDMCS in KSNP
CEDMCS on Korea Standard Nuclear Plant (KSNP) was
independently verified and validated. Based on the con-
ceptual IV & V activities of IEEE 1012, the major view-
point is selected as below through the system function
and performance analysis of CEDMCS.
1) Identification of the critical functional characteris-
tics for the CEDMCS;
2) Identification of the interface between the internal
and external sub-components like the communication and
its transmissi on frequency (i ncluding the serial data links);
3) Identification of the performance characteristics for
the target system;
4) Identification of the appropriateness on the functional
cohesion and coupling for fina l im plem entation [1,2] ;
5) Reliability of the function and performance;
6) Exceptional handling;
7) Identification of the test coverage.
The following section will address the d etail o f the it e m
enumerated above.
2.1. Identification of Critical Characteristics
There are following design factors for CEDMCS, when
an independent review is conducted.
1) Profile to drive the control rod up, down, and hold;
2) Conditions that interlock the rod driving;
3) Engagement condition for the rod driving;
4) Operation mode of the rod driving.
In addition to the above, th ere are many design factors
for controlling rods which are regulated by CEDMCS. These
factors are identified as critical design characteristics
and should be highlighted when an independent review is
conducted.
Figure 2 indicates the interconnection diagram between
CEDMCS and other auxiliary system which provides the
control input and output. The CEDMCS marked with
cloud in Figure 2 is to control the reactivity by insertion
and withdrawal o f control rod. The main inpu t from RRS
is the control signal, and rod is also controlled by operator
intervention. Thus integral IV & V is the critical task to
guarantee the integrity of the system. Otherwise CE-
DMCS causes the plant trip bringing about the financial
damage and public hazard.
2.2. Identification of the Interface
For the modernization or upgrade of CEDMCS, some of
the hardwired interconnections between components are
connected through a communication network. The main
differences between these two configurations are discon-
tinuity and continuity of data. In case of hardwired, the
continuity of data is guaranteed, but when communica-
tion is used for data exchange, there might be a disconti-
nuity of data when a network failure occurs and is recov-
ered soon.
2.3. Identification for Performance
Characteristic
As mentioned above 1) in 2.1, the profile should be im-
plemented according to the tolerable range in time line.
This could results in critical hazard of malfunction in in-
sertion, withdrawal and holding of rod.
Figure 2. CEDMCS in Yonggwang Nuclear Power Plant 3 & 4.
Copyright © 2012 SciRes. JSEA
Lessons Learned from Practical Independent Verification and Validation Based on IEEE 1012
812
2.4. Cohesion and Coupling
It is the design verification to check if the software mo-
dule is constructed well based on the logical function de-
composition, getting rid of implementation complexity
and ambiguity that result from bad software design. It is
a critical measure to judge the testability and maintain-
ability of software [2].
2.5. Reliability of Function and Performance
It is a new aspect of verification and validation to check
if the function to be implemented is implementable as
software or hardware. Recently most of functions, even
implemented with hardware in a legacy plant, are re-
formed as software, targeting a digital system.
2.6. Exception Handling
In any software function, there is an exception of partial
function. This partial function shall be clearly designed
and implemented, which supports a reliable test plan and
procedure in the test phase.
2.7. Test Coverage
Practically exhaustive test coverage is not desirable and
recommended for robust software testing. However, when
the output of the software is actuating the hardware de-
vice that is connected, the maximum test coverage is
recommended in test. For this systematic and concrete
test coverage has been generated by designer as well as
IV & V reviewer based on the 7 criteria. It was very
helpful to remove the delicate failure sources, which was
the solid platform of test procedure.
Based on the fundamental principles mentioned in
IEEE 1012, the IV & V team extracts and summarizes
the non-trivial points for verification and validation as
described in 2.1 through 2.7, which is used for estimating
the design integrity of whole CEDMCS system except
hardware design and assembly part.
3. Statistical Distribution of Anomaly Data
The following is a number of anomaly reports issued at
the each phase of SDLC, excluding the planning, instal-
lation and maintenance for simplicity. Tables 1 and 2
show the anomaly pattern of each design segment thro-
ughout software development life cycle. As shown, most
of design anomaly is identified in the early phase of re-
quirement and design. And even thought that the number
of anomaly is not so significant, most of anomaly topic in
test phase is the coverage and the scope of each testing
such as unit testing, integration testing, Factory Accep-
tance Testing (FAT) and Site Acceptance Testing (SAT).
Figure 3 is indicating the amount of the anomaly for
each design segment in total. There are several issues
that we need to be aware of. The extent of Line of Code
(LOC) and complexity of design is closely related to a
number of anomalies, and especially the design segment
of Man Machine Interface (MMI) that has human inter-
face is relatively high number of Human Factor (HF)
anomaly. Even though HF can be viewed as a different
side of IV & V, ther e is obviou sly a tenden cy that th e HF
design is significantly overlooked in the design process.
Table 1. Anomaly pattern of CEDMCS software upgrade in
Yonggwang Nuclear Power Plant 3 & 4.
Req’t Design Impl.Test
Logic Controller 17 7 5 5
Logic Controller MTP 8 9 5 8
PC and DCHC Controller 2 5 4 3
PC and DCHC Controller FPGA 3 5 2 0
MCB OM 16 3 4 3
Total 71 78 29 23
Table 2. Anomaly pattern of CEDMCS software upgrade in
Ulchin Nucle a r Power Pla nt 3 & 4.
Req’t Design Impl.Test
Logic Controller 10 8 6 5
Logic Controller MTP 12 14 9 4
SSPE Power Controller 5 10 1 2
SSPE Power Controller FPGA 4 5 0 0
Data Process Controller 21 21 5 5
Data Process Controller MTP 19 20 8 7
Total 71 78 29 23
Figure 3. Diagram for AR distribution in Yonggwang Nu-
clear Power Plant 3 & 4.
Copyright © 2012 SciRes. JSEA
Lessons Learned from Practical Independent Verification and Validation Based on IEEE 1012 813
4. Lesson Learned in V & V of KSNP
CEDMCS
The following is the pattern found through the analysis of
anomaly reports in each phase about CEDMCS of Yon-
ggwang unit 3 & 4, and Ulchin unit 3 & 4 nuclear power
plants.
4.1. High Level Design Error
Most of errors were found in the early stage of software
development life cycle [3,4].
Requirement and design phases in SDLC are much
more important in software development as indicated in
Tables 1 and 2, and Figure 3. The fault of early phase
design can cause the wrong implementation which is not
verified because it is well designed based on the wrong
early phase of design. The small number of anomaly in
the phase behind requirement phase is truly based on the
early stage of development phase.
For requirement IV & V, the contract, project planning
documents, user (utility) requirements, technical meeting
minutes, system design documents such as System Re-
quirement (SR), Design Requirement (DR), Design Spe-
cification (DS), and Interface Requirement (IR), Perfo-
rmance Requirement (PR) and other design relevant doc-
uments were used for requirement phase IV & V input.
4.2. The Test Preparation of CEDMCS
With the result of requirement phase IV & V, the design
and implementation phase is conducted without remark-
able controversy. But in test phase, the hot debate for test
coverage and plan has been done. The outstanding devia-
tion between designer and IV & V reviewer was identi-
fied. To resolve these issues, each participating organiza-
tion together had a meeting, and decided to prepare the
test plan document based on IEEE code [5,6].
4.3. The Management of IV & V
According to Annex C “definition of independent V & V
(IV & V)” of IEEE 1012-2004 [1], IV & V is defined by
three parameters: technical independence, managerial in-
dependence, and financial independence as described in
Table 3.
Table 3. Independent verific a tion and validation for m .
IV & V Form Technical Management Financial
Classical I I I
Modified I i I
Integrated i I I
Internal i i i
Embedded e e e
Note: I: Rigorous; i: Conditional Independence; e: M inimal Independence.
4.3.1. Technical Independence
Technical independence requires the V & V effort to util-
ize personnel who are not involved in the development of
the software. The IV & V effort should formulate its own
understanding of the problem and how the proposed sys-
tem is solving the problem. Technical indepen d en c e ( “f r es h
viewpoint”) is an important method to detect subtle errors
overlooked by those too close to t he solution.
For software tools, technical independence means that
the IV & V effort uses or develops its own set of test and
analysis tools separate from the developer’s tools. But
this type of tool independence is overlooked in this pro-
ject for the reason of milestone, budget and suspicion
that diverse and independent tool environment does not
exactly guarantee the correctness and reliability of the
system.
4.3.2. Manage r ial Indepe n dence
Managerial independence requires that the responsibility
for the IV & V effort be vested in an organization sepa-
rate from the development and program management
organizations. Managerial independence also means that
the IV & V effort independently selects the segments of
the software and system to analyze and test, chooses the
IV & V techniques, defines the schedule of IV & V ac-
tivities, and selects the specific technical issues and prob-
lems to act upon. The IV & V effort provides its findings
in a timely fashion simultaneously to both the develop-
ment and program management organizations. The IV &
V effort must be allowed to submit to program manage-
ment the IV & V results, anomalies, and findings without
any restrictions (e.g., without requiring prior approval
from the development group) or adverse pressures, direct
or indirect, from the development group. In this project
this independence is kept almost in perfect manner via
technical meeting and managerial meeting continuously.
4.3.3. Financial Independen ce
Financial Independence requires that control of the IV &
V budget be vested in an organization independent of the
development organization. This independence prevents
situations where the IV & V effort cannot complete its analy-
sis, or test or deliver timely results because funds have
been diverted or adverse financial pressures or influences
have been exerted. This is totally evident and there is no
way out from this dilemma in this project.
4.3.4. Forms of Indep en dence
The extent to which each of the three independence pa-
rameters (technical, managerial, financial) is vested in a
V & V organization determines the degree of indepen-
dence achieved.
Many forms of independence can be adopted for a V
& V organization. The five most prevalent are: 1) Clas-
Copyright © 2012 SciRes. JSEA
Lessons Learned from Practical Independent Verification and Validation Based on IEEE 1012
814
sical; 2) Modified; 3) Integrated; 4) Internal; and 5) Em-
bedded. Table 3 illustrates the degree of independence
achieved by these five forms. The IV & V for CEMDCS
upgrade has been conducted in the combination of “mo-
dified” and “integrated” in Table 3.
1) Classical IV & V
Classical IV & V embodies all three independence pa-
rameters. The IV & V responsibility is vested in an orga-
nization that is separate from the development organiza-
tion. IV & V uses a clo se working relation ship wi th the d e -
velopment organization to ensure that IV & V findings
and recommendations are integrated rapidly back into the
development process. Typically, classical IV & V is per-
formed by one organization (e.g., supplier) and the de-
velopment is performed by a separate organization (i.e.,
another vendor). Classical IV & V is generally required
for software integrity level 4 (i.e., loss of life, loss of
mission, significant social, or financial loss) through re-
gulations and standards imposed on the system develop-
ment.
2) Modified IV & V
Modified IV & V is used in many large programs
where the system prime integrator is selected to manage
the entire system develo pment in clud ing the IV & V. The
prime integrator selects organizations to assist in the de-
velopment of the system and to perform the IV & V. In
the modified IV & V form, the acquirer reduces its own
acquisition time by passing this responsibility to the
prime integrator. Since the prime i ntegrator perform s al l or
some of the development, the managerial independence is
compromised by having the IV & V effort report to the
prime integrator. Technical independence is preserved
since the IV & V effort formulates an unbiased opinion
of the system solution and uses an independent staff to
per form the IV & V. Financial independence is preserved
since a separate budget is set aside for the IV & V effort.
Mod ified IV & V effort would be appropriate for systems
with software integrity level 3 (i.e., an important mission
and purpose).
3) Integrated IV & V
This form is focused on providing rapid feedback of V
& V results in the development process and is perform-
ed by an organization that is financially and managerially
independent of the development organization to minimize
compromises with respect to independence. The rapid
feedback of V & V results in the development process is
facilitated by the integrated IV & V organization: work-
ing side-by-side with the development organization; re-
viewing interim work products; and providing V & V
feedback during inspections, walkthroughs, and reviews
conducted by the development staff (potential impact on
technical independence). Impacts on the technical inde-
pendence are counterbalanced by the benefits associated
with a focus on interdep endence between th e integrated IV
& V organization and the development organization.
4) Internal IV & V
Internal IV & V exists when the developer conducts
the IV & V with personnel from within its own o rganiza-
tion, although preferably not the same personnel in-
volved directly in the development effort. Technical,
managerial, and financial independence are compromised.
Technical independence is compromised because the IV
& V analysis and test is vulnerable to overlooking errors
by using the same assumptions or development environ-
ment that masked the error from the developers. Mana-
g e r i al i n d e pendence is compromised because the internal IV
& V effort uses the same common tools and corporate
analysis procedures as the development group. Peer
pressure from the development group may adversely in-
fluence how aggressively the software is analyzed and
tested by the IV & V effort. Financial independence is
compromised because the development group controls
the IV & V budget. IV & V fu nds, resources, and sch ed-
ules may be reduced as development pressures and needs
redirect the IV & V funds into solving the development
problems. The benefit of an internal IV & V effort is ac-
cess to staff who knows the system and its software. This
form of IV & V is used when the degree of independence
is not explicitly stated and the benefits of preexisting staff
knowledge outweigh t he benefits of objecti vity .
5) Embedded IV & V
This form is similar to an internal IV & V in that it
uses personnel from the development organization who
should not be involved directly in the develop ment effort.
Embedded V & V is focused on ensuring conformance to
the development procedures and processes. The embed-
ded V & V organization works side-by-side with the de-
velopment organization and attends the same inspections,
walkthrough, and reviews as the development staff (i.e.,
compromise of technical independence). Embedded V &
V is not tasked specifically to independently assess the
original solution or conduct independent tests (i.e., com-
promise of managerial independence). Financial indepen-
dence is compromised because the V & V staff resource
assignments are controlled by the development group.
Embedded V & V allows rapid feedback of V & V re-
sults into the development process but compromises the
technical, managerial, and financial independence of the V
& V organization.
Regardless of the independence types and form of the
IV & V, the most difficult thing to handle in the process
of IV & V itself and AR resolution is the “financial in-
dependence” Anything else except this, IV & V activity
has been conducted in reasonable manner to remove the
design fault and to optimize the design products pro-
duced in the development cycle.
After analysis of IV & V form based on IEEE 1012, the
realistic IV & V form will be described comparing to this
project. Ta ble 4 shows the summary of IV & V form that
is achievable and not achievable in the process of IV & V.
Copyright © 2012 SciRes. JSEA
Lessons Learned from Practical Independent Verification and Validation Based on IEEE 1012
Copyright © 2012 SciRes. JSEA
815
Table 4. Comparison between IEEE Std. 1012 and practical
independent verifica tion and validation.
IEEE Std. 1012 Practical IV & V
Technical
independence Achievable Achievable
Managerial
independence Achievable Achievable
Financial
independence Achievable
Achievable, but partly
influenced by adverse
or distorted pressure
Forms of
independence The classic model of
IV & V is desirable. The “integrated model”
of IV & V is realistic.
Financial independence seems to be tough to imple-
ment in realistic project environment. Thus it could be a
concern to regulator for IV & V for safety-related system.
It is complicated issue which might involve legal sup-
port.
5. Conclusions
Yonggwang un it 3 & 4 and Ulchin unit 3 & 4 that is one
of the KSNP are upgraded with new ha rdware where the
CEDMCS software is running. For software reliability,
independent verification and validation has b een condu ct ed
throughout the SDLC. It was important to correctly ana-
lyze and recognize the core function, performance and
interface of CEDMCS to draw the seven (7) criteria for
IV & V view points in starting point, which is used
throughout the IV & V. These items were also prepared
by identifying the hazardous failure event of CEDMCS
[7]. The design result was reviewed to analyze the design
error involved in the design process. They show that
most of the design inconsistency o ccurs in the early stag e
of the design process such as the requirement phase and
the design phase. Thus special care for design inception
phase is required via well-known practices like technical
meeting design inspection and walkthrough, and design
iteration.
Regardless of the type of independency and IV & V,
the sensitive difference between reality and standards is
the aspect that it is difficult to overcome the circum-
stance of financial independency in a sense that the main
contractor provides the money. If it is necessary, the le-
gal support will be an efficient way to overcome finan-
cial independency. However the solution was technical
meeting including licensing body in this project.
Once the completion of IV & V, issuing the anomaly
report, a resolution meeting between IV & Ver and desi-
gner to obtain the optimal solutio n is held in every SDLC
phases. Unfortunately there is a tendency that the desi-
gner will not partly accept the anomaly issued just be-
cause the function is anyway performed well even though
there is room for optimization and documentation. In this
case anomaly resolution process was efficiently and man-
datorily used with the written verification and validation
plan and procedure including anomaly report disposition
procedure for app r oval .
Before commencing the test phase IV & V, it was very
important to create and review the test plan to extract the
essential test coverage and test scope of unit testing, in-
tegration testing, factory acceptance testing, and site ac-
ceptance testing. Likewise the SDLC design process, IV
& V for SDLC design process has been iterated for soft-
ware quality enhancement through technical meeting for
anomaly report resolution. The activities for resolving
these topics are completed successfully, resulting in that
CEDMCS software system is running without errors or
failures so far.
6. Acknowledgements
This work has been conducted with the support of Man-
Machine Interface System (MMIS) group in Korea Ato-
mic Energy Research Institute (KAERI), and reliability
and nuclear engineering group in KOrea Reliability Tec-
hnology System (KORTS).
REFERENCES
[1] IEEE Standard 1012, “IEEE Standard for Software
Verification and Validation,” 2004.
http://standards.ieee.org/findstds/standard/1012-2004.html
[2] R. S. Pressman, “Software Engineering, a Practitioner’s
Approach,” 5th Edition, McGraw-Hill Higher Education,
New York, 2004.
[3] K. H. Cha, K. C. Kwon and C. S. Woo, “The Software
Verification and Validation Tasks for a Safety Critical
System in Nuclear Power Plants,” International Journal
of Safety, Vol. 3, No. 1, 2004, pp. 38-46.
[4] C. Ponsard, P. Massonet, J. F. Molderez, A. Rifaut, A.
van Lamsweerde and H. Tran Van, “Early Verification
and Validation of Mission Critical Systems,” Formal Me-
thods in System Design, Vol. 30, No. 3, 2004, pp. 233-
247.
[5] IEEE Standard 829, “IEEE Standard for Software and
System Test Documentation,” 2008.
http://standards.ieee.org/findstds/standard/829-2008.html
[6] IEEE Standard 1008, “IEEE Standard for Software Unit
Testing,” 1987.
http://standards.ieee.org/findstds/standard/1008-1987.html
[7] NUREG/CR-6430, “Software Safety Hazard Analysis,”
1995.