Journal of Software Engineering and Applications, 20 12, 5, 140-143
doi:10.4236 /js ea.2012.512b027 Published Online December 2012 (http://www.SciRP.org/journal/jsea)
Copyright © 2012 SciR es. JSEA
Analysis of Uncontrollable Variables to the Performance
of Predefi ned Tasks
Eunghyun Lee1, Jinwoo Lee1, Jungwan Hong1, Younghoon Ko1, Yongjin Kwon1, Sang C. Park1,
Jonghoon Lee2, Miseon Yi2
1Department of Industrial Engineering, Ajou University, Suwon, South Korea; 2D Agency for Defense Development, South Korea.
Received 2012
ABSTRACT
The ai m of this res earch was to investigate the impacts of c hanging environ mental fa ctors on pilots by implementing the
effective flight procedure that we found from the previous research. During the experiments, camcorders were used to
monitor and analyze the tasks and physical exhaustion of the pilot. NASA-TLX was also used to collect the data of the
workload. Through this experiment, we were able to approach the experiment from a different angle regarding the op-
timal assigned work of the pilo t, unlike the previo us studies. Also, we were able to find out the impacts o f environ men-
tal factors on the pilot’s workload.
Keywords: Helicopter Mission; Task Performance Time; Tas k Workload; Armed Helicopter
1. Introduction
This research has adopted a new working procedure that
involves a different kind of helicopter: a side-by-side
helicopter. Due to the fact that the optimal weather con-
dition was included, we did not take other external envi-
ronmental factors, such as the change in visual field or
the c han ge in the mana geme nt o f the eq uipme nt, i nto co n-
sideration. T herefore, the main purpose of this research is
to find to what e xtent cap tain pilots/co-pilo ts are a ffected
by the external environmental factors including Daytime
experiment, Nighttime experiment, Bad weather experi-
ment and Solo flight by using the procedure derived from
our previ ous studies.
2. Experimental Configuration
2.1. Applying Our Previo u s Research
The side-by-side helicopter enables the captain pilot/co -
pilot to share the same working area and dualize the
flight instr uments, thus a llo wing the m to share t heir ta sks.
Hence, in order to invent a ne w combination of the task,
we divided the working procedure into four different
procedures for Experiment 1. Also, in Experiment 2, we
developed a new scenario by dividing the working pro-
cedure differently from Experiment 1 and analyzed the
record ed data including the total amount of time spent to
complete the task, lethality, the efficiency of the per for-
mance of the pilots, and more. The optimal working pro-
cedure developed through this method is shown in Table
1 and has been used for this research.
2.2. Experimental Method
Due to the fact that this research is mainly focused on
measuring the change in the wo r kl oa d of the pilot accord-
ing to the change in external environmental factors, we
constructed four different experimental environment
based on the different time of the day and meteorological
conditions that can be manipulated by the flight simulator.
Daytime experiment, which was also used in the previous
studie s, was the most basic expe ri mental en viro n ment wit h
the least amount of restrictions. Nighttime experi- ment
was executed with the use of Night Vision and had more
constraints, such as speed limit, due to the difficulties of
securing a clear view. Also, to apply the difficulties of
securing a clear view because of the fog, we set the
visual
Table 1 . Table of flig ht procedu re.
Pilots Task detail
Captain
Pilot Main flight/hovering & return Operate commun ications
Co-Pilot Operate warning system Detect the threat
Common Evasion flight/Operati on of radio

Analysis of Uncontrollable Variables to the Performance of Predef ined Tasks
Copyright © 2012 SciR es. JSEA
141
range as approximately 3 miles in Bad weather ex-
perimen t. Lastly, Solo flight had the same environment
as daytime experiment; however, we set it as the environ-
ment that captain pilot performs the work by himself/
herse lf witho ut the help of co-p ilo t.
We applied randomization and 5 repetitions for each
experiment, thus performing approximately 20 experi-
ments. Similar to the previous research, the data of the
workload of the pilot was gathered by surveying with
NASA-TLX, and the total amount of time spent for each
task accor ding to the work di stribution was recorded with
a camcorder. The video clips from the camcorder were
also subdivided with the interval of 5 seconds.
The types of experiments are sho wn bel o w in Table 2,
the images of experiments are shown below in Table 3
and the i ma ge of a night vi si on i s s ho wn b el o w i n Figure
1.
Table 2. Ty pes of ex periments.
Type s of
experiments Detail Characteristic
Day Opera tion starts at 12:00 All clear w eather
Flight speed limit 130kts
Night Operation starts at 24:00 All clear w eather
Flight speed limit 110kts
Use Night
Vision
Bad
Weather
Operation star ts at 12:00
Visual range 3 miles
Flight speed limit 100kts
Solo fligh t Sa me as d ay experiment
Table 3. Images of experiment.
Type s of
experiments Images o f exp er imen t
Day
Night
Bad
Weather
Solo fligh t Sa me as day experiment
Figure 1. Image of t he night vision in TADS.
2.3. Scenario Composition
The appropriate scenario has to be composed to increase
the efficiency of measuring the workload varied by en-
vironmental factors. The scenario consists of the rando-
mized location of the enemy encampment which is ac-
companied by 4 targets (3 buildings and 1 tank). The
helicopter that took off at the assigned starting point en-
ters the opera tional ar ea b y contour flying. It the n goes to
the location of reconnaissance, confirms the target, and
completes the change-over. The helicopter moves to the
first enemy base a nd attack. The helicopter operates sur-
vival gears and evasion flight. After completing the task
and retreating from the operational area, as the helicopter
switches back to normal fl yi n g a nd l and s o n the r eturnin g
point, then the scenario ends.
3. Data Analysis
Setting the reliability as 95% and using Minitab 16 and
the ANOVA (Anal ysis o f vari ance) in ever y analysi s, we
examined the significance of the change in the workload
of the pilot according to external environmental factors.
Also, we used the method of the Time & Motion Study
to analyze the video clips recorded by the camcorder.
The camcorder was placed and secured on the location
where it can observe the motion of the captain pilot/co-
pilot and the battlefront at the same time. The recorded
video clips were subdivided into the intervals of 5 sec-
onds and used to analyze the actions of captain pilot/co -
pilot for each task.
3.1. Analysis of Mission Run-Time
Setting the standard for dividing t he amount of time was
quite important to verify the significance of the time re-
quired for each mission according to the different types
of experiments. The missions required to enact the sce-
nario ar e shown bel ow in Table 4.
Inter val s wher e t he pi lot e xp erie nced a huge c hange o f
action were more subdivided and i ntervals where the pilot
experienced a trivial change of action were combined.
The data of captain pilot in the four experiments were

Analysis of Uncontrollable Variables to the Performance of Predef ined Tasks
Copyright © 2012 SciR es. JSEA
142
analyzed and the data of co-pilot was no t ana lyzed in the
last experiment where he/she is assumed to be out. Each
section was carefully numbered and analyzed with the
method of the ANOVA. The first characteristic of this
data is that there is a point that cannot be analyzed with
the significance test. For these unanalyzable points, we
set the duration time. The analysis indicated that the only
difference is the flight time between varying weather cond i-
tions. Other mission tasks appear to be not different, even
thou gh the weather c hanges a ccor ding to our experi men-
tal procedures.
3.2. Mission Success Rate
We did not analyze the mission success rate due to the
fact that all of the cases had 100% of success rate in our
previous studies. In other word s, regardless of the environ-
ment, the pilot is able to complete the mission even
though there might be a difference of the amount of time
spent and workload. We believe that it occurred due to
the difference of the level of difficulty of the missions
caused b y the limitation of the simulation.
3.3. Analysis of Captain Pilot/Co-pilot’s TLX Data
By using the method of ANOVA, we analyzed the TLX
scores of captain pilot/co-pilot’s and investigated which
expe riment has t he lo west wo rkload . T he follo wing ta ble
shows the p-values. The Statistical significance test of
two pilot’s TLX da ta is s hown below in Table 5 .
Table 4. Mis sion procedure.
Mission Detail
Flight to operation area Move to a reconnaissance position
Lodgment of reconnaissance
posi tion Target searching & Hovering
Detect target Detect the target & Identify
Evasion fligh t Conduct evasion flight in the event of
threat
Operate a defensi v e system Work the defensi v e system in t he eve nt
of continuous threat
Operate a co m m. Work the communications system
Ch eck the gauges Check the condition of instrum ents
Return to bas e After completing the task, the helicopter
back to the base
Table 5. Statistical significance test of two pilot’s TLX data.
TLX
subsection
Captain pilot Co-pilot
p-value criteria result p-value criteria result
Mental 0 <= 0.05 ○ 0 <= 0. 05 ○
Physical 0 <= 0.05 ○ 0.216 > 0.05 ×
Temporal 0 <= 0.05 ○ 0 <= 0. 05 ○
Own
Performance 0.004 <= 0.05 ○ 0.002 <= 0. 0 5
○
Effort 0 <= 0.05 ○ 0.001 <= 0.05 ○
Frustration 0 <= 0.05 ○ 0 <= 0. 05 ○
Since the data of captain pilot are all statistically sig-
nificant, we have come to a conclusion that captain pilot
is affected by external environmental factors. The order
in incr easin g worklo ad is dayti me experiment, solo flight,
nighttime experiment, and bad weather experiment. Since
there is no physical category for the data of co-pilot, we
found out that there is the difference of workload ac-
cording to external environmental factors in other cate-
gories. The order in increasing workload is daytime ex-
perimen t, bad weather experiment, and nighttime ex-
periment .
4. Analysis Results
The reason for the manifest significance of the duration
of captain pilot is the speed limit because the helicopter
is flyi ng the same amount of distance with the lo wer ve-
locity. Also, we believe that the reason for the insigni-
ficance is that even though we set the enemy encamp-
ment, it is impossible for the helicopter to stop at the
same exact point and implement the Hovering for every
experiment. Contrary to captain pilot, since it is more
difficult to cle arly dif ferentiate his/her action and there is
less limits of movement than that of captain pilot,
co-pilo t had a te nde ncy to var y mor e i n ter ms o f t he ti me
spend to co mplete the mission. Also, due to the fact that
most of the missions of co-pilot accompanied those of
captain pilot, when the duration was relatively longer,
co-pilot tended to take more time detecting and discern-
ing the target. In terms of the analysis of workload with
the use of TLX, since captain pilot had to complete dif-
ficult tasks at the same point, he/she had to pay more
attention to find the target. We can clearly see that cap-
tain pilot has more workload when piloting in the night
time using Night Vision and having more limited
left-and-right visual field than when piloting after the
rain a nd having a li mited visual r ange. Also, co-pilo t has
more workload after the rain than in the night time be-
cause it is more arduous to detect the enemy and attack
with missiles when he/she has a limited vis ual range than

Analysis of Uncontrollable Variables to the Performance of Predef ined Tasks
Copyright © 2012 SciR es. JSEA
143
when he/she has a narrow angle of visual field.
5. Conclusion
Through the experiments, we could see that both captain
pilot/co-pilot is under more pressure when external envi-
ronmental factors are involved. Considering the fact that
the mission capability of flight is important even with
external environmental factors, such as after the rain or
in the night time, it is necessary to install attacking and
defensing system based on automation and radio system
based on network in the newly developed helicopters to
increase the mission capability. In order to enhance the
strengths and improve the weaknesses, more research on
the development of tactical air employment is recom-
mended.
6. Acknowledgement s
This work was supported by the Basic Science Research
Program through the National Research Foundation of
Korea (NRF) funded by the Ministry of Education,
Science and Technology (Grant No. 2010-0012517).
This work was supported by the Agency for Defense
Development (ADD). This paper was also partially sup-
ported by the Ajou University Research Fund . The au-
thors wish to express sincere gratitude for the financial
support. We also appreciate Ms. Jeongwon Lee for her
help with the E nglish translatio n as well as the editin g of
ou r man usc rip t.
REFERENCES
[1] Kim Ki-Choel, Lee Jong-Sun, “A Study on the Pilot
Workload Analysis in Flight Phases using the Flight Si-
mulator,” Journal of Korean Institute of industrial engi-
neerings, V ol. 11, 2001, p p. 22 7-230.
[2] Woo-Sub Oh, Sung-Woo Kim, Choi Ie-Na, Jin -Suk Jang,
“A Study on the System Concept of KHP Mission
Equipment Package,” Journal of The Korean Society for
Aeronautical and Space Sciences, Vol. 2, 2005, pp.
1100-1103.
[3] Suk-Joon Yoon, “Airplane R&D simulator,” Journal of
The Korean Society for Aeronautical and Space Sciences,
Vol. 30, 20 02, pp. 15 0-162.
[4] Jae-Moon Lee, Chi-Young Jung, Jae-Yeong Lee, “The
Combat Effectiveness Analysis of Attack Helicopter Us-
ing Simulation and AHP,” Journal of The Korea Society
for Simulation, V ol . 19, 2010, pp. 63-70.
[5] Jung Hoon Lee, Jun Ho Kim, Jung Yong Park, Iee Ki An,
“Current Status of Civil-Military Dual Use Helicopter
Development Focusing on Conversion Case from Civil to
Military Helicopter,” Journal of The Korean Society for
Aeronautical and Space Sciences Conference,2011, pp.
1783-7886.
[6] Jun -Woo Lee, Sang-Won Chae, Chil-Gee Lee, “Devel-
opment of Panel Part in Flight Simulator based on PC,”
Journal of The Korea Society for Simulation, Vol. 10,
2001, pp . 31-39.
[7] Sunh-Yun Choi, Sang-won Chae, Young-Sin Han, Chil
Gee Lee, “3D Flight Simulator for Education of Flying
Tactics,” Journal of The Korea Society for Simulation,
Vol. 12,2002, pp. 1-11.
[8] Tomas Schnell, Yong-Jin Kwon, Sohel Merchant, “Im-
proved flight technical performance in flight decks
equipped with synthetic vision information system dis-
plays”, The international Journal of Aviation psychology,
Vol. 14, No. 1, 2004, p. 79-102.
[9] Jung-Hoon Lee, Byeong-Hee Chang, Hwang In-Hee,
“Current Status of Civil-Military Dual Use Helicopter
Development - Focusing on Conversion Case from Mili-
tary to Civil Helicopter”, Current industrial and Tech-
nological Trends in Aerospace, Vol. 7, 2009, pp 32-42.
[10] Kang Byeong-Kil, “Development of a Pilot Workload
Prediction Program using TAWL (Task Analysis /
Workload) Methodology,” Master's Thesis, Gyeongsang
National University, Jinju, 2010.
[11] Sung-Ho Cho i, Ye on-Chul Choi , Yoeng-Hoek Lee(2007),
A Study on Situation Awareness of Helicopter Pilot, Ph.D.
thesis, Korea Aeros pace Univers ity, Seoul, 2007.