A Journal of Software Engineering and Applications, 2012, 5, 113-116
doi:10.4236/jsea.2012.512b022 Published Online December 2012 (http://www.scirp.org/journal/jsea)
Copyright © 2012SciRes. JSEA
Multi-Sensor Ensemble Classifier for Activit y Recognitio n
Lingfei Mo1, Shaopeng Liu2, Robert X. Gao2*, Patty S. Freedson3
1School of Instrument Science and Engineering, Southeast University, Nanjing, China; 2Department of Mechanical Engineering,
University of Connecticut, Storrs, CT, USA; 3Department of Kinesio l ogy, University of Massachusetts, Amherst, MA, USA.
Email: rgao@engr.uconn.edu
Received 2012.
This pap er presents a multi-sensor ensemb le classifier (MSEC) for phys ical activity (P A) pattern recognition of human
subjects. The MSEC, developed for a wearable multi-sensor integrate measurement system (IMS),combines multiple
classifiers based on different sensor feature sets to improve the accuracy of P A type id ent ificatio n.Expe rimenta l e va lua-
tion of 56 subjects has sho wn that the MSECi s mo re effectivein a ssessing activities o f varying intensitiestha n the tradi-
tional homogeneous classifiers. It is able to correctly recognize 6 PA types with an accuracy of 93.50%, which is 7%
higher tha n the non-ensemble support vector machine method . Furt hermo re, the MSECi s effe ctive in red ucing the s ub-
ject-to-subject variabilityin activity recognition.
Keywords:Physical Activit y Assessment;Multi-Sensor Ensemble;Support Vector Machine.
1. Introduction
Physical activity ( PA), d efined as bodi ly mo vement gen-
erated by skeletal muscles [1]such as walki ng, jog gingor
sport activities, is important for maintaining health and
preventing cardiovasc ular diseases, diabetes, and obesity.
Accurate monitoring and assessment of PA under free-
living conditions provides information on the type and
intensity of activitie s that the pe rson has been engage d in,
thus is of significant interest to the research community
[1] and commercial co mpanies.
The goal of PA assessment is to recognize the type,
dura tio n, and i nte ns i t y of a b roa d ra nge o f physical activ-
ities and quantify the energy expenditureof the test sub-
jectdur ing his/her d aily life, as illustrated in Figure 1. In
recent years, multi-sensor systems have been increasing-
ly inve sti gated for PA assessment. For example, multiple
accelerometers have been placed at different loc ations on
the bodyof test subjects [3]or combined with other types
of sensors, such as respiratory sensor or GPS [4] for PA
measurement. Combining advanced computational tech-
niques such as machine learning and sensor fusi o n [3,4],
differentiation of various activities has shown to be im-
proved. For example, a multi-sensor integrate measure-
ment system (IMS) was developed with two accelero-
meters and one ventilation sensor to measure and assess
the physical act ivity [5].
Rec en tly , ens em bl e lea rn ing has been in cre as ing ly inve st-
tigated for pattern recognition [6]. An ensemble learning
method co mbines multiple individual clas sifier s to ob tain
better predictive performance than that obtained by any
of the constituent classifiers [7]. T his is a technique that
usually combines a number of weak classifiers together
to produce a strong classifiers. Through combination of
decisions from multiple classifiers or multiple sensors,
recognition accuracy has shown to be improved effect-
tively [6]. For example, Ravi et al. combined multiple
different clas sifiers to identify eight common activities of
two subjects with a single tri-axial accelerometer [8].
These classifiers used the same four-feature (Mean,
Standard Deviation, Energy and Correlation) datasets,
and had obtained good recognition accuracies, especially
by means ofMajority Voting. Lester et al. used AdaBoost
[9] to select features and combined multiple weak classi-
fiers, each of which accepting only a single feature as
Figure 1. Physical activit y assessment.
Multi-Sensor Ensemble System for Human Physical Activity Recognition
Copyright © 2012SciR es. JSEA
input, and obtained good classification result from a
weighted combination of the weak classifiers[10]. The
eight sensors,inlcuding accelerometer, audio sensor, IR/
visible light, high frequency light, barometric pressure,
humidity, temperature and compass, were integrated in a
unit a t ta c hed on t he shoulde r of the test subject, a nd te sted
by two subjects. For multi-sensor measurement system,
combining different sensors at different body locations
would reveal different characteristics of body movement
and have different statistical distribution. Therefore,
combining different classifiers based on the datasets of
different sensors would have better identification results
than a homogeneous classifier.
In this paper , a multi -senso r en se mble classifier (MSEC)
for PA type identification is presentedfor a multi-sensor
integrated measurement system (IMS) [5]. It combines
multiple classifiers of SVM, based on different sensor
datasets of the IMS. Due to different PA type identifica-
tion accuracies of the different classifiers, an instance
specific weight majority voting is proposed for the clas-
sifier combination. The performance of the MSECis ex-
perimentally evaluated by 56 human subjects performing
free-living acti vities.
2. Ensemble Learning
2.1. System Design
The architecture of the multi-sensor ensemble classifieris
sho wn in Figure 2. The sensor sets for ense mble are gen-
erated by choosing different sensors of the multi-sensor
measurement system. Features corresponding to these
sensor datasets are then extracted and selected. Multiple
classifiers with different feature selection can be derived
from each sensor set, and the diversity of the classifiers
can be achieved by choosing the sensor set and feature
combinations. For each classifier, a learning model is
first selected and trained, with a part of the sensor data as
testing dataset to evaluate the classifier. Each classifier
has a decision result, and the final decision is thus ob-
tained b y combining the classification results from all the
classifiers by the weight majority voting.
2.2. Individual Classifiers
Seven clas sifiers with di f f e r en t sens o r d a tasets were devised
for the e nsemble system based on the three sensors in the
IMS. Each sensor data set consists of a cluster of classi fiers,
including 1) C1 (classifier cluster from the wrist
accelerometer dataset), 2) C2 (classifier cluster fro m the
hip accele ro meter da ta set), 3) C 3 (clas sifier cluster fro m the
abdominal ventilation sensor dataset), 4) C4 (classifier
cluster from the hip accelerometer and the wrist accelero-
meter datasets), 5) C5 (classifier cluster from the hip
accelerometer and the abdominal ventilation sensor
datasets), 6) C6 (classifier cluster fro m the wrist accelero -
meter and the abdominal ventilation sensor datasets), and
7) C7 (classifier cluster from the hip and wrist accelero-
meter and the ab d ominal ve ntilation sensor datasets). Each
classifier cluster consists of multiple (n) classifiers by
different feature selection (random selection). As a result,
a total of 7 × n classifiers and 7 × n testing res ult s can b e
obtained. The final ensemble decision is obtained based
on these 7 × n classifiers by instance specific weighted
majo r it y voti ng.
Figure 2. Multi -sensor ensemble system block diagram.
Multi-Sensor Ensemble System for Human Physical Activity Recognition
Copyright © 2012SciR es. JESA
2.3. Activity Recognition
For each single sensor, 7 time-domain features, namely
the 10th, 25th, 50th (median), 75th, and 90th per centiles,
the mean value, and the standard deviation, were ex-
tracted. In additio n, a correlation feature between the hip
accelerometer and the wrist accelerometer was also ex-
tracted, providing a measure for the coordination or vari-
ation between the upper limb and the body during an
activity. For each accelerometer, two frequency domain
features, energy and entrop y were extracted. For the ven-
tilation senso r, the dominant f requency of the resp iratory
signal obtained from a spectral analysis was extracted as
the breathing frequency. These features were computed
for every 30-second data segment, and linear scaling was
then applied to the extracted features in the range of [0,
1], to avoid that features of greater numeric values would
overwhelm those in the smaller numeric ranges.As a re-
sult, a total of 63 features (50 time-domain and 13 fre-
quency-domain features) were extracted. To achieve the
diversity of the input of each classifier, 70% of the fea-
tures were selected randomly from the overall feature
sets for training classifiers.
The SVM algorithm was chosen as the base classifier
of the ensemble system, and the selected features were
used as inputs to the SVM classifier. A two-step proce-
dure was taken for pr edicting the types of physical activ-
ity. First, a training data set that consists of the selected
features from all the 56 subjects but one was constructed
for building the SVM model and selecting the penalty
parameter and Gaussian kernel parameter. The model
parameters were selected through a “grid-search” with
5-fold cross validation. The parameters that yielded the
highest recognition rate were chosen during the process.
Second, upon completion of the training, the SVM model
was applied to the feature set of the subject that was left
out in the training process, to predict the activity type
reflected in the 30-second data segments. Such a two-
step procedure constitutes a “leave-one -subject-out” cross
validation, and was executed on each subject data.
3. Experimental Evaluation
3.1. Design of Experiments
A total of 56 subjects (26 male and 30 female) were re-
cruited forphysical activity assessment, with the follo w -
ing characteristics, expressed in terms of the mean ±
standard de viation:
1) age = 38.7±11.6 years,
2) mass = 71.1±14.5 kg,
3) height = 169.3±9.1 cm and
4) body mass index = 24.7±4.2 kg/m2.
Each subject performed 6 types of activities of varying
intensities, which are commonly seen in daily lives as
illustratedin Table 1. For each subject, the actual PA
types and times performed by the subjects were recorded,
and sensor data whendifferent PA types were performed
were collected by the IMS (as shown in Figure 2) cor-
respondingly [7]. Each PA type was performed for 7
minutes, followed b y a 2-minute rest period.
3.2. Individual Classifier Results
In order to ensemble different classifiers by the instance
specific weight majority voting method,it is necessary to
first investigatethe accuracies of the different classifier
clusters. Furthermore, since these classifiers use features
from different sensor datasets, they yield different accu-
racies and confidences on identifying the PA types. The
average accuracies of the classifiers on the different PA
types are first calculated, as s hown in Table 2. It i s seen
that these classifiers have yielded different accuracies on
PA type identifications, which is due to the fact that dif-
ferent sensor combinations are sensitive to different PA
t yp e s.
3.3. Ensemble Results
The performance of the ensemble system is based on the
number and accuracies of the classifiers integrated for
the ensemble learning. Various ensemble classifiers had
been evaluated. T he definitions of the different ensemble
classifiers are: E1E7 integrate the classifiers within
each sensor cluster C1-C7, respectively, E8 integrates
classifiers of C1, C2 and C3, E9 integrates classifiers of
C4, C5 , and C6, and E10 integrates classifiers C1-C7.
Table 1. Physical Activities types for testing.
Activities Category Abbr.
Computer work Sedentary activity CW
Moving boxes Household and other MB
Cycling with 1-kp resistance C1
Treadmill a t 3.0 mph Moderate locomotion T3
Treadmill at 4.0 mph 5% grade Vigorous activity T4-5
Table 2. Classification accuracies of different sensor
classifiers on differ ent PA types
PA T ypes
CW MB C1 T3 T4-5 TE
Classifier Cluster
C2 88.2% 71.7% 86.2% 91.7% 86.0% 71.5%
C4 91.2% 87.1% 89.5% 91.4% 86.2% 73.4%
C6 91.2% 76.3% 81.3% 82.7% 66.5% 96.7%
C7 92.9% 72.9% 91.0% 94.0% 92.4% 85.4%
Multi-Sensor Ensemble System for Human Physical Activity Recognition
Copyright © 2012SciR es. JSEA
Figure 3. E nsemble classifier result comparison.
Figure 3 illustrates t he P A classi ficatio n results of the
various ensemble classifiers E1-E10. It is seen that in
general, the more classifiers and sensor datasets are in-
cluded in the ense mble classifier, the better the result has
been. For example, classifier E10 integrates all the clas-
sifiers (total 21), and hasyielded the best classification
mean accuracy of 93.5% with the smallest standard devi-
ation of 11.6%.
4. Conclusions
In this study, a multi-sensor ensemble classi fier is designed
for physical activity recognition, as a critical component
of a multi-sensor integrated measurement system. Spe-
cifically, MSEC integrates data measured by three sen-
sors, and fuses the multiple classifiers by performing
instance specific weight majority voting. Compared with
non-ensemble classifier, the MSECmethod has shown
higher mean accuracies and lower standard deviations,
thus d emonstrateing better ge ner a liz a tion capabilit y.
Altho ugh promising, the MSEC is generally computa-
tional ly int ensi ve, r eq uir ing more computational resourc es
than non-ensemble single classifier to achieve good per-
formance. Furthermore, there are still questions that re-
maine d unans wered , e.g. , 1) ho w many se nsor s and wha t
type of sensors (including the locations where the sen-
sors are attached) are required for PA assessment, and 2)
what type of features and classifiers are suitable and op-
timal for the e nsemble system. Research is being contin-
ued to systematically investigate the developed MSEC
algorithm in terms of effectiveness and computational
efficiency for improved PA classification.
5. Acknowledgement
The aut hor s gr at ef ull y ackno wl ed ge fu nd in g p r ovi de d for
this research by the National Institutes of Health under
Grant UO1 A130783.
[1] C. Caspersen, K. Powell, and G. Christenson, “Physical
activity, exercise, and physical fitness: definitions and
distinctions for health-related research”, Public health
reports, Vol. 100, No. 2, pp. 126-131, 1985.
[2] D. Hendelman, K. Miller, C. Bagget, E. Debold, and
P. Freedson, “Validity of accelerometry for the assess-
ment of moderate intensity physical activity in the field”,
Medicine and Science in Sports and Exercise, Vol. 32, No.
9, pp. 442-449, 2000 .
[3] L. Bao, and S. Intille, “Activity recognition from us-
er-annotated acceleration data”, Pervasive Computing,
Vol. 3001, pp. 1-17, 2004.
[4] M. Ermes, J. Parkka, J. Mantyjarvi, and I. Korhonen,
“Detection of daily activities and sports with wearable
sensors in controlled and uncontrolled conditions”, IEEE
Transa ctions on Information Technology in Biomedicine,
Vol. 12, No. 1, pp. 20-26, 2008.
[5] S. Liu, R. Gao, and P. Freedson, “Design of a wearable
multi-sensor system for physical activity assessment”,
Proc. IEEE/ASME Int. Conf. on Advanced Intelligent
Mechatronics, pp. 254-259, 20 10.
[6] J. Lester, T. Choudhury, and G. Borriello, “A practical
approach to recognizing physical activities”, Pervasive
Computing, pp. 1-16, 2006.
[7] L. Breiman, “Bagging predictors”, Machine learning,
Vol.2 4, N o. 2, pp. 123-140, 19 96.
[8] N. Ravi, N. Dandekar, P. Mysore, and M. Littman, “Ac-
tivity recognition from accelerometer data”, Proceedings
of the Seventeenth Conference on Innovative Applications
of Artificial Intelligence(IAAI 2005), pp. 1541-1546,
[9] M. Gashler, C. Giraud-Carrier, and T. Martinez, “Deci-
sion tree ensemble: small heterogeneous is better than
large homogeneous”, IEEE Seventh International Confe-
rence on Machine Learning and Applications, San Diego ,
CA, USA, pp. 900-905, 2008.
[10] D. Opitz, and R. Macli n, “Popular ensemble metho ds: An
empirical study”, Journal of Artificial Intelligence Re-
search, Vol. 11, No. 1, pp. 169-198, 19 99.