Journal of Signal and Information Processing, 2013, 4, 66-71
doi:10.4236 /jsip.2013.43B012 Published Online August 2013 (http://www.scirp.org/journal/jsip)
Copyright © 2013 SciRes. JSIP
Fuzzy C Mean Thresholding based Level Set for
Automated Segmentation of Skin Lesions
Ammara Masood, Adel Ali Al-Jumaily
Scho ol of Electrical, Mechanical and Mechatronic Engineer ing, Universi ty of Techn ology, Sydney, Austral ia.
Email: ammara.masoo d@student.uts. edu.au, Adel.Al-Jumaily@uts.edu.au
Received April, 2013.
ABSTRACT
Accurate se gmentation is an importa nt and challe nging task in any comp uter vision s ystem. It also plays a vital role in
computerized analysis of skin lesion images. This paper presents a new segmentation method that combines the advan-
tages of fuzzy C mean algorithm, thresholding and level set method. 3-class Fuzzy C mean thresholding is applied to
initialize level set automatically and also for estimating controlling parameters for level set evolution. Parameters for
performance evaluation are presented and segmentation results are compared with some other state-of-the-art segmenta-
tion methods. Increased true detection rate and reduced false positive and false negative errors confirm the effectiveness
of proposed method for skin cancer detection.
Keywords: Skin Cancer; Segmentation; Diagnosis; Fuzzy; T hresholding; Level Se ts
1. Introduction
The incidence of skin cancer is rapidly increasing
thro ugh-out the world [1]. An estimated 76,250 new cas-
es of invasive melanoma were diagnosed in the US in
2012, with an estimated 9,180 resulting in death [2].
Australia has one of the highest rates of skin cancer in
the world. Over 1890 Australians die from skin cancer
each year [3]. Melanoma is capable of deep invasion.
The most dangerous characteristic of melanoma is that it
can spread widely over the body via the lymphatic ves-
sels and blood vessels. Thus, early diagnosis of mela-
noma is a key factor for prognosis of the disease.
Analysis of skin lesion images taken using dermo-
scopic imaging technique [4] is a commonly used me-
thod for diagnosis of skin cancer but, this method re-
quires great deal of experience [5]. Due to the lack of
reproducibility and subjectivity of human interpretation,
the development of computerized image analysis tech-
niques is of paramount importance. Segmentation is one
of the most important and difficult task in computerized
image analysis process. The accuracy of the subsequent
steps highly depends on the success of image segmenta-
tion technique.
Segmentation o f ski n lesi ons is d iffic ult because of the
great variety of lesion shapes, sizes, and colors along
with different types and textures of human skin. Other
difficulties that make it a challenging task include, low
contrast between the lesion and the surrounding skin,
smooth transitio n between the lesion and the skin, reflec-
tions d ue to wrong i llu mina tio n and art ifacts suc h as s ki n
texture, air bubbles and hair.
Image segmentation algorithms available in literature
can be broadly classified into two categories: 1) discon-
tinuity based segmentatio n 2) Similar ity based seg menta-
tion, which c a n further be classified into thresholding [6],
clustering [7], and region based approach [8]. Compara-
tive analysis of these approaches in the area of skin le-
sion segmentation can be found in literature [9-11].
Active contour is a popular approach used to estimate
boundaries in medical images. Two types of algorithms
lie under this category: 1) parametric active contours [12]
which adapt a deformable curve until it fits the object
boundary. 2) Geometric active contours based on level
set theory. Some of the active contour models need user
interventio n for initialization. Thus auto matic approaches
like gradient vector flow algorithm [13, 14] based on
anisotropic diffusion [15] and robust algorithms like
adaptive snakes and shape probability association model
[16] are ta king important plac e in liter a ture.
Level set (LS) methods, as one of the automatic proc-
ess approach have shown effective results for medical
image segmentation. However, intensive computational
requirements and regulation of controlling parameters
make it a complex and time consuming method. To
overcome such shortcomings, fuzzy clustering has been
used to facilitate the LS segmentation for automatic
segmentation of ultrasound, computed tomography and
Fuzzy C Mean Thresholding based Level Set for Automated Segmentation of Skin Lesions
Copyright © 2013 SciRes. JSIP
67
magnetic resonance imaging [17-19]. Fuzzy C mean
(FCM) based thresholding [20] and LS algorithm are
different computational models that have been applied
individuall y for segme ntation of der moscopic images [21,
22]. Similarly, some work is done on using clustering for
initial segmentation followed by further edge preserving
and refining steps for border tracing of skin lesions [23].
We analyzed the advantages and limitations of these
methods for skin lesion segmentation. The observations
showed that FCM works well for making rough cluster-
ing of pixels which can further be used to get better
threshold values for image segmentation. In addition to
this, if the thresholded image is used as initial estimate
for level set evolution, it can result in more accurate
segmentation as compared to standard level set method
[24,25] and region based active contours [26,27]. We,
hereby presented a technique, based on combining these
methods to achieve higher accuracy in segmentation of
skin lesion images .
This paper introduces 3-class FCM based thresholding
to be used for ini tializi n g the LS evolu tion a nd re gulati ng
the controlling parameters. Performance evaluation is
done by comparing the diagnosis results of this method
with three other segmentation methods i.e. FCM cluster-
ing [28], region based active contours [26, 29] and adap-
tive thresholding [11]. Results show that the proposed
approach performs reasonably well for the segmentation
of ski n lesion i mages. The pap er is or ganized as follows:
Section 2 provides details of the proposed segmentation
method. Section 3 discusses experimental results and
Section 4 provides performance evaluation. Concluding
remarks ar e given in Sect ion 5.
2. Methodology
This section describes our technique for segmentation of
skin cancer images. The main parts of our proposed al-
gorithm are:
2.1. Image Pre-Processing
For skin lesion images there are certain extraneous arti-
facts such as skin texture, air bubbles, dermoscopic gel,
presence of ruler markings and hair that make border de-
tection a bit difficult. In order to reduce the effect of these
artifacts on segmentation results, the images need to be
pre-processed with a smoothing filter. We found that the
best segmentation results were obtained using median
filter with 7x7 mask to smooth the images before seg-
mentation. Thus, the first step of the overall process is to
get a filtered image.
2.2. Fuzzy C-Mean Thresholding
FCM clustering is used to partition N objects into C
classes. In our method, N is equal to the number of pixels
in the image i.e. N=Nx x Ny and C=3 for 3-class FCM
clustering. The FCM algorithm uses iterative optimization
of an objective function based on a weighted similarity
measure between the pixels in the image and each of the
c-clus ter cen ters . A local extre mism of the objecti ve func-
tion indicates an op timal clustering o f the input d ata. The
objective function that is minimized is given by (1)
(1)
where and &
where . ‖*‖ is a norm expressing the simi-
larity between any measured data value and the cluster
centre; m [1, ∞] is a weighting exponent and can be any
real number greater than 1.
Calculations suggest that best choice of m is in the in-
terval [1.5, 2.5], so m=2 is used here as it is widely ac-
cepted as a good choice of fuzzification parameter. The
fuzzy c-partition of given data set is the fuzzy partition
matrix U= [ with i =1 , 2 …. C a nd j =1, 2, 3…N , where
indicate the degree of membership of jth pixel to ith
cluster. The membership functions are subject to satisfy
the following conditions.
for j=1,2,3,….N ; for i=
1,2….C;
The aim of FCM algorith m is to find an optimal fuzz y
c-partition by evolving the fuzzy partition matrix U=
[ iteratively and computing the cluster centres. In
order to achieve this, the algorithm tries to minimize the
objective function Q (1) b y iteratively updating the clus-
ter centres and the membership functions using the fol-
lowing equations.
(2)
(3)
After performing FCM clustering, finally each pixel is
assigned to the cluster for which its membership value is
maximum. Based on the intensity distribution obtained
using histogram of the image, the threshold value is
calculated by taking mean of maximum of cluster 1 and
minimum of cluster 2 or maximum of cluster 2 and
minimum of cluster 3. This method of threshold selection
takes into acco unt the intensity distrib ution i n the ima ge.
This choice helps in obtaining opti mum threshold values
for different images obtained under different conditions.
The overall FCM thresholding algorithm is presented in
Figure 1. The output of this stage is a binary image (Bi)
which has been used in the followed steps.
2.3. Fuzzy C-mean Thresholding based Level Set
Segmentation
Segmentation of images by means of active contours is a
well-established approach. Active contours [12] are used
Fuzzy C Mean Thresholding based Level Set for Automated Segmentation of Skin Lesions
Copyright © 2013 SciRes. JSIP
68
to detect objects in a given image using techniques of
curve evolution. The basic idea is to evolve a curve, sub-
ject to constraints from a given image, for detecting ob-
jects in that image.
Figure 1. Algorith m for FCM thresholding.
In pr o b lems o f curve evo l uti o n, le ve l set methods have
been extensively used. LS methods are established on
dynamic implicit interfaces and partial differential equa-
tions (PDEs). In traditional LS formulation [25], the
contours denoted by C, are represented by the zero level
set C(t)= {(x,y)| (t,x,y)=0)of a level set function
(t, x,y). T he evolving eq uation of the level set functio n
can b e written in the following general form (4)
(4)
which is called levels set equation. The function F is the
speed function that represents the comprehensive forces,
including the internal force from the interface geometry
and the external force from image gradient or/and artifi-
cial mome ntums .
In order to stop the level set e volutio n near the opti ma l
solution, the advancing force has to be regularized by an
edge indication function g. The edge indication function
used here is given by (5)
(5)
where is the filtered image. T raditional LS method is
computationall y intens ive and has certain limi tations like
need of re-initialization of level set function to signed
distance function for stable curve evolution [26]. There-
fore, in this paper fast leve l set for mulation has been used
which was proposed by Li. et al. [24]. This method is
computatio nally more efficient and can be implemented
by using simple finite difference scheme. In order to
segment the skin lesion image the overall iterative proc-
ess for levels se t e volution is given by (6).
(6)
where is the term
for attracting towards the variational boundary and
is the penalty term that forces to
approach the genuine signed distance function automati-
cally.
This fast level set formulation proposed in [24] pro-
vides a benefit of flexible initialization where roughly
obtained region from thresholding can be used to con-
struct initial level set function. Taking advantage of this
facility binary image (Bi) obtained from FCM threshold-
ing algorithm, discussed in the previous section is used
here for automatic initializatio n of the level set fu nction.
The initial level se t function is given as [18] .
(7)
where is the regulator for dirac function [25] defined
as follows:
(8)
Controlling parameters are also adaptively determined
fro m the bina ry i mage (Bi) fo r re gular izi ng the LS evo lu-
tion process automatically. The weighting coefficient
of penalty term is taken as the ratio of area of on
pixels in the binary image (Bi) to its perimeter pixels.
The time step is take n a s 0 .2 / so that ( remains
s maller than 0.25 which is necessary to ensure stable
evolution as found in [24]. is the coefficient of con-
tour length for smoothness regulation and its value is
taken here as 0.1/. The val ue of can be increased for
accelerating the evolution process but it leads to
smoother contours. Thus, care must be taken especially
for skin lesion images, where over smoothened images
may lose significant details about boundary of lesion,
which is an important feature for correct diagnosis. The
balloon force which determines the advancing direction
Fuzzy C Mean Thresholding based Level Set for Automated Segmentation of Skin Lesions
Copyright © 2013 SciRes. JSIP
69
and speed of the evolvi ng curve is given as ( 9)
(9)
where is the modulating argument which i s t aken her e
as 0.5 through experimental analysis. Fig ure 2 shows a
systematic diagram of the proposed method. Each block
represents different steps of the algorithm.
Figure 2. Flowchart of proposed algorithm.
3. Experimental Results
Segmentation results obtained using the proposed algo-
rithm, FCM clustering, Adaptive Thresholding (AT) and
Region based Active Contours (RBACs) are presented in
Figures 3-7 for some of the skin lesion images. While
presenting results, we tried to present images having dif-
ferent common problems of dermoscopic images which
can badly affect the segmentation process. Figure 3
shows an image of dysphasic nevi ha ving poor illumina-
tion and uneven boundary of lesion. Fig ure 4 represents
melanoma lesio n with uneven border and hair on it. Fig-
ure 5 is melanoma lesion with smooth transaction be-
tween lesion and skin. Figure 6 is a melanoma lesion
present on a skin with spots and redness effect. Figure 7
is a benign lesion surrounded by a lot of hair.
FCM clustering and AT provide rough segmentation
but cannot track the boundary exactly. Region based ac-
tive contours provided results comparable to proposed
method in some cases but this method cannot track the
border in the presence of many hair (see Figure 7(f)) or
spotty skin (see Figure 6(f)). Similarly, it missed the
exact boundary in the case where there is a smooth
transaction between lesion and skin (see Figure 5(f)).
Figure 3. Se g menta ti o n res ul t s of dy s pla s ti c ne vi ( a) or ig i nal
image (b) Ground truth by expert (c) Proposed method (d)
FCM clustering (e) AT (f) RBACs.
Figure 4. Segmentation results of melanoma (a) original
image (b) Ground truth by expert (c) Proposed method (d)
FCM clustering (e) AT (f) RBACs.
Figure 5. Segmentation results of melanoma (a) original
image (b) Ground truth by expert (c) Proposed method (d)
FCM clustering (e) AT (f) RBACs.
Figure 6. Segmentation results of melanoma (a) original
image (b) Ground truth by expert (c) Proposed method (d)
FCM clustering (e) AT (f) RBACs.
Figure 7. Segmentation results of benign lesion (a) original
image (b) Ground truth by expert (c) Proposed method (d)
FCM clustering (e) AT (f) RBACs.
Analysis of segmentation results showed accuracy of
proposed method even in the presence of all these
artifacts.
4. Evaluation of Segmentation Resu l t s
The o bj ec tive eva lua ti o n o f se g me ntatio n al go ri t hms on a
Fuzzy C Mean Thresholding based Level Set for Automated Segmentation of Skin Lesions
Copyright © 2013 SciRes. JSIP
70
large set of clinical data is one of the important steps
toward establishing validity and clinical applicability of
an al gorith m. T hus, in or der to evaluate the ef ficienc y of
the proposed method a performance comparison is pro-
vided with three of the well-known segmentation meth-
ods used here for segmentation of same collection of skin
lesion images. The metric used here for measurements is
based on pixel-by-pixel comparison of pixels enclosed in
the segmented result (SR) and the ground truth result
(GT) from expert. First, binary images are constructed
for each boundary, where a pixel is considered non-zero
if it lies inside the boundary and zero otherwise. Error
value s hows t he ro ot mean sq uare e rror b y taking gr ound
truth image as standard segmentation result. Other
evaluation pa ra meters are calculated as follows:
Hammoude distance (HM): This metric makes a
pixel by pixel comparison of the pixels enclosed by the
two boundaries:
(10)
This well-known metric takes into account two types
of error; p ixels classified a s lesion by au tomatic seg men-
tation that were not classified as such by medical expert
and pixels classified as lesion by medical expert that
were not classified as such by automatic segmentation.
The Hammoude distance gives equal importance to both
types of errors. However, from a clinical point of view,
the 2nd type of error is more important since the lesion
pixels should never be missed by the automatic diagnos-
tic system. Therefore, separate metrics should be used to
take into account the two types of error.
True detection rate (TDR): This metric measure the
rate of pixels classified as lesion by both the automatic
and the medical expert segmentation. Higher TD R sho ws
better performance of segmentation method.
(11)
False Positive Error (FPE): This metric determines
the rate of pixels assigned as lesions by the segmentation
method that were not assigned as lesion by the medical
expert. Lower the value of FPE better is the performance
of r espe c tive segmentatio n method.
(12)
False Negative Error (FNE): It determines the rate of
pixels categorized as lesions by the medical expert that
were not assigned as lesion by the automatic segmenta-
tion:
(13)
Table 1 shows the comparative results. The database
used for analysis comprised of 238 dermoscopic and
clinical view lesion images which were collected from
various sources but most images were obtained from
Sydney Melanoma Diagnostic Centre, Royal Prince Al-
fred Hospital. The segmentation results were compared
with the reference images (ground truth) and average of
segmentation scores are presented here for each method.
Table 1. Results of Segmentation Methods. The values in
bold correspond to t he b est performance.
Method Evaluated Parameters
FNE (%)
FPE
(%)
TDR
(%)
HM
(%)
Error
%)
FCM
Clustering
13.75 6.52 86.25 34.8
34.31
RBACs 11.72 9.08 88.28 37.1 36.47
AT 13.14 6.95 86.86 35.2 34.94
FCM
thresholding
based LS 7.34 4.66 92.66 11.3 15.19
It is evident from the results that the proposed method
has shown reasonably better performance as compared to
other methods. Region based active contour method has
shown low FNE and higher TDR as compared to FCM
cluste ri n g, and ada ptive thre sho lding but it has quite hig h
false positive error, which makes it susceptible of de-
claring benign lesions as melanoma. On the basis of our
analysis, we believe that the proposed method can show
promising results for lesion segmentation in a computer
aided d ia gnosis system.
5. Conclusions
In this paper, a segmentation algorithm is presented for
skin lesion detection. It combines the advantages of
clustering, thresholding and level set method, for getting
more accurate segmentation results. The proposed me-
thod showed reasonably good accuracy for segmenta-
tion of skin lesion images with an average true detection
rate of 92.6% and quite reduced false positive and false
negative error i.e. 4.66% and 7.34% respectively. Com-
parative analysis proved that it works well even in the
presence of different artifacts present in skin images.
Keeping an eye on the importance of tissue and cell level
diagnosis of skin cancer, this method can provide a basis
for segmenti ng histo -patho logica l images as well.
REFERENCES
[1] R. Siegel, et al., “Cancer statistics, 2011”, CA: A Cancer
Journal for Clinicians, Vol. 61, No. 4, 2011, pp. 21 2-236.
doi:10.3322/caac.20121
[2] Society, A.C., Cancer F acts & Figur es 2012,
http://www.cancer.org/acs/groups/content/epidemiologys
urveilance/documents/document/acspc-031941.pdf 2012.
[3] Causes of Death 2010,C.W.O. Australia, Editor, Aus-
Fuzzy C Mean Thresholding based Level Set for Automated Segmentation of Skin Lesions
Copyright © 2013 SciRes. JSIP
71
tralian Bureau of Statistics, Canberra, Austr alia.
[4] G. Argenziano and H.P. Soyer, Dermoscopy of pig-
mented skin lesions, a valuable tool for early diagnosis of
Melanoma,The Lancet Oncology, Vol. 2, No. 7, 2001,
pp. 443-449. doi:10.1016/S1470-2045(00)00422-8
[5] D. Piccolo, et al., Dermoscopic Diagnosis by A Trained
Clinician vs. A Clinician with Minimal Dermoscopy
Trainin g vs. Computer-aided Diagnosis of 341 Pigmented
Skin Lesions: A Comparative Study,” British Journal of
Dermatology, Vol. 147, No. 3, 2002, pp. 481-486.
doi:10.1046/j.1365-2133.2002. 04978.x
[6] S. Ben Chaabane, et al., “Color Image Segmentation Us-
ing Automatic Thresholding and the Fuzzy C-means
Techniques”, in Proceedings 14th IEEE Mediterranean
Electro techn ical Conference, 2008, pp. 857 -861.
[7] L. Dongju and Y. Jian . , Otsu Me thod and K-means,” in
Proceedings Ninth International Conference on Hybrid
Intelligent Systems, China , 2009, pp. 344-349.
[8] M. Emre Celebi, et al., “Bo rder Detection in Dermoscopy
Images Using Statistical Region Merging”, Skin Resea rch
and Technology, V ol. 14, No. 3, 2008, pp. 347-353.
doi:10.1111/j.1600-0846.2008.00301.x
[9] M. E. Celebi, G. S. H. Iyatomi and W. V. Stoecker, “Le-
sion Border Detection in Dermoscopy Images,” Co mp u-
terized Medical Imaging & Graphics, Vol. 33, 2009, pp.
148-153.
[10] T. Mendonca, et al., “Comparison of Segmentation Me-
thods for Automatic Diagnosis of Dermoscopy Images”,
in Proceedings of 29th Annual International Conference
of the IEEE Engineering in Medicine and Biology Society,
2007, pp . 6572-6575.
[11] M. Silveira, et al., Comparison of Segmentation Me-
thods for Melanoma Diagnosis in Dermoscopy Images,
IEEE Journal of Selected Topics in Signal Processing,
Vol. 3, No. 1, 2009, pp. 35-45.
doi:10.1109/JS TSP.2008.20111 19
[12] Isard, A. B. A. M., Active Contours 1998: Springer Ver-
lag.
[13] Mahmoud, M. K. A. and A. Al-Jumaily, “Segmentation
of Skin Cancer Images Based on Gradient Vector Flow
Snake,” in Proceedings of 2011 International Conference
on Mechatronics and Automation, 2011, pp.
216-220. doi:10.1109/IC MA.2011.59 85659
[14] Bulent Erkol, R. H. M., R. Joe Stanley, William V.
Stoecker and Erik Hvatum, “Automatic Lesion Boundary
Detection in Dermoscopy Images Using Gradient Vector
Flow Sn akes,” Skin Research and Technology, Vol. 11,
No. 1, 2005, pp. 17-26.
doi:10.1111/j.1600-0846.2005.00092.x
[15] P. Perona, an d J. Mali k, Scale-sp ace and Edge Detection
Using Anisotropic Diffusion,” IEEE Transactions on
Pattern Analysis and Machine Intelligence, Vol. 12, No.
7, 1990, pp. 629-639. doi:10.1109/34.56205
[16] J. C. Nascimento , et al., “Adaptive Snakes Using the EM
Algorithm,” IEEE Transactions on Image Processing,
Vol. 14, N o. 11, 2005, pp.1678-1686.
doi:10.1109/TIP.2005.85 7252
[17] M. N. M. Babu, V. K. Hanmandlu, M. Vasikarla, S.,
“Histo-pathological Image Analysis Using OS-FCM and
Level Sets”, in Proceedings of IEEE 39th Applied Im-
agery Pattern Recognition Workshop 201 0, pp. 1-10.
[18] Li, B . N. et al., Integrating Spatial Fuzzy Clustering with
Level Set Methods for Automated Medical Image Seg-
mentati o n”, Computers in Biology and Medicine, Vol. 41,
pp. 1-10, 2011. doi:10.1016/j.compbiomed.2010.10.007
[19] B. N. Li , et al., “Integrati ng FCM and Level Sets for Liv-
er Tumor Segmentation,” in Proceedings 13th Int. Con-
ference on Biomedical Engineering, Singapore, 2009 , pp.
202-205.
[20] Aja-Fernandez et al., Soft Thresholding for Medical
Image Segmentation,” in Proceedings 32nd Annual In-
ternational Conference of the IEEE Engineering in Medi-
cine and Biology Society, Argentina, 2010, pp.
4752-4755.
[21] S. Sookpotharom, “Border Detection of Skin Lesion Im-
ages Based on Fuzzy C-Means Thresholding,” in Pro-
ceedings of 3rd Int. Conference on Genetic and Evolu-
tionary Computing, Chi na , 2009, pp. 777-780.
[22] M. Silveira and J. S. Marques, Level Set Segmentation
of Dermoscopy Images,presented at 5th IEEE Interna-
tional Symposium on Biomedical Imaging: From Nano to
Macro, May 14-1 7, 2008.
[23] M. Kamali and G. Samei, “Border Preserving Skin Lesion
Segmentation,” in Proceedings of SPIE 6915, Medical
Imagi ng 2008: Comp ut e r -A ided D iagnosis , 20 08.
[24] L. Chunming, et al., Level Set Evolution without
Re-initialization: A New Variational Formulation,” in
Proceedings of IEEE Computer Society Conference on
Computer Vision and Pattern Recognition, San Diego,
2005, pp . 430-436.
[25] S. Osher and R. Fedkiw, Level Set Methods and Dy-
namic Implicit Surfaces2002,New York: Sprin-
ger-Ver l ag.
[26] T. F. Chan and L. A. Vese, Active contours without
edges,IEEE Transactions on Image Processing, Vol. 2,
pp.266-277, 2001. doi:10.1109/83.902291
[27] V. Caselles, R. Kimmel, and G. Sapiro , “Geodesi c Active
Contours”, International Journal of Computer Vision,
Vol. 22, No. 1, 1997, pp. 61-79.
doi:10.1023/A:1007979827043
[28] Huiyu, Z., et al., Anisotropic Mean Shift Based Fuzzy
C-Means Segmentation of Dermoscopy Images,IEEE
Journal of Selected Topics in Signal Processing, Vol. 3,
No. 1, 2009 , pp. 26-34. doi:10.1109/JSTSP.2008.2010631
[29] Q. Abbas, I. Fondón and M. Rashid, Unsupervised Skin
Lesions Border Detection via Two-dimensional Image
Analysis,Computer Methods and Programs in Biomedi-
cine, Vol. 104, No. 3, 2011, pp. 1-15.
doi:10.1016/j.cmpb.2010.06.016