Engineering, 2013, 5, 171-175
http://dx.doi.org/10.4236/eng.2013.510B037 Published Online October 2013 (http://www.scirp.org/journal/eng)
Combination of the Kinect with Virtual Reality in
Balance Training for the Elderly
Wei-Min Hsieh1,2, Chih-Chen Chen3, Shih-Chuan Wang4, Yu-Luen Chen4,5*,
Yuh-Shyan Hwang6, Jin-Shin Lai7
1Graduate Institute of Computer and Communication Engineering, National Taipei University
of Technology, Taipei, Chinese Taipei
2Department of Electronic Engineering, Hwa Hsia Institute of Technology, Taipei, Chinese Taipei
3Management Information System, Hwa Hsia Institute of Technology, Taipei, Chinese Taipei
4Department of Computer Science, National Taipei University of Education, Taipei, Chinese Taipei
5Department of Information Management, St. Mary ’s Medicine, Nursing and Management College, Yilan, Chinese Taipei
6Department of Electronic Engineering & Institute of Computer and Communication,
National Taipei University of Technology, Taipei, Chinese Taipei
7Department of Physical Medicine and Rehabilitation, National Taiwan University Hospital,
Taipei, Chinese Taipei
Received May 2013
ABSTRACT
Daily life movements such as standing, walking, and jumping need balance ability to achieve. Good balance control is
closely related to the body stability and its development. According to medical research, people showing dizziness after
taking drugs may have their balance ability substantially affected, and are more likely to fall, especially true to the el-
derly. This study proposes the combination of Kinect with vir tual reality to build an information platform of interactive
scenarios, for practices and evaluation on balance ability. Based on the indicators of balance ability, this platform sets
out various training activities to improve the balance ability, making the supposedly boring process fun and vivid for a
much better training purposes. Also, according to literatures, training with gaming patterns results in 30% reduction of
falls. What’s more, this type of training can have the data, like time of use, scores, and joint postures, recorded and sent
through network to databases on remote computer servers. The data collected from this platform can be sorted and an a-
lyzed, and the results can then be used to evaluate the performance of the balance training, and referenced for follow-up
planning in the future.
Keywords: Balance; Kinect; Fall; Virtual Reality
1. Introduction
Balancing means the ability to maintain stability in all
kinds of movements and postures. Most human move-
ments rely on the foundation of balance, and keeping
body balanced is critical to human movements. Body
movements are pulled by gravity; a movement can be
done only after the balancing point is achieved. Although
normal people have decent balance ability to deal with
daily needs, dizziness caused by diseases or drugs—es-
pecially taking multiple drugs, such as sleeping pills and
tranquilizers, can bring about fall incidents.
Bureau of Health Promotion, Department of Health,
points out in its Elderly Health Promotion Plan (2009-
2012) that falling is one of the fetal causes of the elderly
illness and death (51.0%). Elderly people over the age of
65 are the group with highest risk of fall death. People
prone to fall are handicapped in activities of daily living
(ADL) and instrumental activities o f daily livin g (IADL).
Researches also point out that 42.3% of the elderly
people who suffer fall injuries need hospitalization, and
37.0% of them need to stay at nursing homes after the
clinic therapies. Also, statistics show that in 2008 there
are about 125,000 patients hospitalized for fall injuries.
All the evidences tell the tru th that aging does make sub -
stantial impact on the ability of balance control [1]. This
study proposes the combination of Kinect with virtual
reality to build an information platform of interactive
scenarios, for practices and evaluation on balance ability.
Based on the indicators of balance ability, this platform
sets out various training activities to improve the balance
ability, making the supposedly boring process fun and
vivid for a much better training purpose.
*Corresponding a uthor.

W.-M. HSIEH ET AL.
172
In his studies on older people’s falling, Robbin et al.
(1989) argues that abnormal gaits are in connection with
the falls [2].
Daubney et al. (1999) studied 50 elderly people over
the age of 65, and analyzed the relationship between their
lower limb muscle strengths and balances, and the results
showed that the ones who fell without obvious reasons
obviously had less balance ability than the ones who
never fell, and the lower limb muscle strength was
closely related to the balance ability; therefore, the mus-
cle strength measurement index provides a very high
prediction performance [3].
Robertson et al. (2002)’s research for a community
showed that the elderly an d people who fell before could
go through exercise training to reduce the incidents of
fall and alleviate fall injuries as well [4]. Ledin et al.
(1990)’s research also showed that the elderly undertak-
ing a period of time of balance training could have their
balance ability improved to prevent fall and bone fracture
[5].
In his study, Belgen et al. (2006) argues that individual
body’s balance ability is related to the probability of fall
incidence, and people who suffer more fall incidents
have less balance abilities [6].
D. Taylor et al. (2004) point out certain programmes
of exercise such as the Falls Management Exercise
(FaME) and Otago Exercise (OEP) programmes are ef-
fective in returning falls patients to normal functional
movement and can reduce the risk of falling by as much
as 30% [7].
R. Y. Wang et al. point out in their research projects
(NSC100-2314-B010-022-MY2): falls among the elderly
often lead to injury and thus lead to other complications.
Exercise can prevent falls in older people. Previous stu-
dies also suggested integrated exercise training program
was better than a single exercise. For healthy older adults,
combining virtual reality (VR) with integrated exercise
training program can enhance their motivation [8].
Y. R. Yang et al. point out in their research projects
(NSC100-2314-B010-021-MY2): Exercise has beneficial
physiologic effects in older adults, including effects on
strength, aerobic capacity, and balance. These effects can
reduce the risk and rate of falling among older adults. To
effectively improve physical activities and to reduce risk
of fall, appropriate exercise programs for specific fallers
are needed [9].
2. Material and Method
Figure 1 is an overall schematic illu s tration, where Unity-
3D and Windows SDK software tools in combination
with Kinect sensors are used to show body movements
on a notebook computer at the other side.
Figure 2 shows the system structure of this study,
Figure 1. Overall schematic diag ra m.
Figure 2. System structure diagram.
where the development process includes the hardware
connections, signals receiving and analyses, movement
identifying, etc. The hardware used is Kinect (for Win-
dows) developed by Microsoft. The software application
is developed with the Microsoft officially released Win-
dows SDK software tool along with Unity3D, a 3D-
enabled game development software tool. The Windows
SDK tool includes all the needed documents and API
libraries and header files, while Unity3D is a compre-
hensive, 3D-game and 3D interactive tool for creativity
development.
Kinect is used to detect human skeleton, and the de-
tected 3D in-depth images are converted to the skeleton
tracing system, which is capable of tracing up to 20 ske-
leton joints of the human body. The coordinates of the
detected skeleton joints are then reflected in the 3D
scenes developed with Windows SDK programs using
Unity3D’s libraries to visualize the body movements and
present the visual reality simulation.
This study mainly focuses on fall prevention, and the
proposed training and test items are designed by taking
references to the Four Square Step Test (FSST) method
often used for examining balance ability; one of the de-
signed activities is using the hand or foot or the combina-
tion of foot and hand to touch the random balls for the
balance training or test. The lead preparation for the

W.-M. HSIEH ET AL.
173
training and test activities is shown in Figure 3, where
the medical personnel needs to pre-select the difficulty
level, applicable module, evaluation method, and suitable
postures.
The degree of difficulty of the stretching movements is
defined by the participant’s skeleton. To take into ac-
count safety during the activities, Figure 4 demonstrates
the scope of the stretching movements, where the 50%,
75%, and 100% degree between SHOULDER CENTER
and WRIST are easy, medium, and difficult stretching,
respectively, which are corresponded with the random
balls appearing at specific spots designed by the software
programs.
The program needs to be initialized before the training
or test starts, i.e. benchmarking the participant’s position
and skeleton, followed by limb stretching warm-up be-
fore entering the training procedure. Figure 5 is a block
diagram showing the initialization, warm-up and training.
The activities are conducted in a game-like manner in
order to increase the participant’s interest in doing the
activities. This system also provides the training data that
can be used for evaluation on the training effectiveness.
Figure 6 is a block diagram of the test, which r equires
the lead process and initialization before being com-
menced. The result can be based on fixed time to calcu-
late the scores or on fixed scores to calculate the time.
3. Results
This study takes an empirical test on Mr. A of 67 years of
age, who is 178 cm high, and weighted 83 kg. With the
developed virtual-reality training program, the tested
male uses his hand or foot to touch the random balls, and
the time used to finish the given number of touches on
the balls is measured for the balance training.
Figure 3. Lead preparation for the training and test activi-
ties.
Figure 4. Th e scope of the st ret ch i n g m o v em ents .
Figure 5. The initialization, warm-up and training block
diagram.
Figure 6. Test block diagram.
Figure 7 shows the screen of the hand touching the
ball, where the ball appears on the upper right corner,
and the tester sc ore s with his ri g ht hand touc hi ng the ball .
On this screen, (a) shows the right hand has not yet
touched the ball, and (b) shows the right hand are touch-
ing the bal l.
A training program was set as a six-week session, and
the difficulty level was easy palm or foot exercise. Each
of the exercises was given 3 games, with a 2-minute in-
terval between games. There was a 5-minute break be-
fore switching to a different touch method. In each game,
the random ball showed up 5 times, and each of which
was to be touched by a specific limb. The time used for
the games was calculated. And the times used were av-
eraged by each different touch method. Table 1 lists the
results from the elderly Mr. A’s training, which shows
the times used for these two training methods, and the

W.-M. HSIEH ET AL.
174
Table 7. The screen of the hand touching the ball.
Table 1. Lists of the training results.
Week/Limb Week 1 Week 2 Week 3 Week 4 Week 5 Week 6
Hand 35 sec 34 sec 32 sec 32 s ec 31 sec 29 s ec
Foot 37 sec 36 sec 36 s ec 35 sec 33 sec 32 sec
results from adjacent weeks show either improved or
unchanged. Also, Mr. A’s training records of the times
used by his palm and foot show that his foot reaction is
weaker than his palm. An interview with elderly Mr. A
reveals that his foot received medical treatment because
of sports injuries, and that’s why his foot is weaker in
reaction. Figure 8 shows the averaged weekly time used
for his hand or foot to touch the random balls.
4. Conclusion
This study proposes the combination of Kinect with vir-
tual reality to build an information platform of interactive
scenarios, for practices and evaluation on balance ability.
A clinical assessment test was given to another elderly
person, and it was also proved that after the 6-week
training, the tester obviously made progress in reactions.
From the training results of the participants, we know
that the information platform developed for enhancing
the balance ability is in line with the practical needs, and
the fun and interesting game-like exercises it introduces
are much helpful to the improvement of balance ability,
and certainly to the prevention of falling. In the future,
more tests w ill be given on the elderly in order to collect
(a)
(b)
Figure 8. The averaged weekly time used for hand or foot.
(a) theaveraged weekly time of hand; (b) the averaged
weekly time of foot.
more effective and comprehensive quantitative data for
the system’s fine tuning and modifications.
5. Acknowledgements
The authors wish to thank the National Science Council
(NSC) in Taiwan (Grant Number: NSC 101-2221-E-146
-001-, NSC101-2221-E-152-001-, NSC101-2627-E-002-
006-) for support this research.
REFERENCES
[1] Bureau of Health Promotion, Department of Health, Chi-
nese Taipei.
http://www.bhp.doh.gov.tw/bhpnet/portal/PressShow.asp
x?No=200911060002
[2] A. S. Robbin and L. Z. Rubenstein, “Predictors of Fall
among Elderly People,” Archives of Internal Medicine,
Vol. 49, 1989, pp. 1628-1633.
[3] M. E. Daubney and E. G. Culham, “Lower-Extremity
Muscle Fore and Balance Performance in Adults Aged 65
Years and Older”, Physical Therapy, Vol. 79, No. 12,
1999, pp. 1177-1185.
[4] M. C. Robertson, A. J. Campbell, M. M. Gardner and N.
Devlin, “Preventing Injuries in Older People by Prevent-
ing Falls: A Meta-Analysis of Individual-Level Data,”
Journal of the American Geriatrics Society, Vol. 50, 2002,
pp. 905-911.
[5] T. Ledin, A. C. Kronhed, C. Moller, M. Moller, L. M.
Odkvist and B. Olsson, “Effects of Balance Training in
Elderly Evaluated by Clinical Tests and Dynamic Petro-
graphy,” Journal of Vestibular Research, Vol. 1, No. 2,
1990, pp. 129-138.
[6] B. Belgen, M. Beninato, P. E. Sullivan and K. Nariel w al-
la, “The Association of Balance Capacity and Fall s Self- -
efficacy with History of Falling in Community-Dwelling
People with Chronic Stroke,” The American Congress of
Rehabilitation Medicine and the American Academy of

W.-M. HSIEH ET AL.
175
Physical Medicine and Rehabilitation, Vol. 87, 2006, pp.
554-561.
[7] D. Taylor and C. Stretton, “The Otago Exercise Program,
An Evidence-Based Approach to Falls Prevention for
Older Adults Living in the Community,” Journal of Pri-
mary Health Care, Vol. 31, No. 6, 2004.
[8] R. Y. Wang, “Virt ual Reali ty-Enhanced Optimal Exercise
Training Model for Falls Prevention and Its Remote Ap-
plications in Elderly with Different Fall Risks,” NSC
100-2314-B010-022-MY2, 2011.
[9] Y. R. Yang, “Optimized Exercise Model and Remote
Training for Fall Prevention in Elderly (I),” NSC100-
2314-B010-021-MY2, 2011.