In this work, we propose an original approach of semi-vectorial hybrid morphological segmentation for multicomponent images or multidimensional data by analyzing compact multidimensional histograms based on different orders. Its principle consists first of segment marginally each component of the multicomponent image into different numbers of classes fixed at K. The segmentation of each component of the image uses a scalar segmentation strategy by histogram analysis; we mainly count the methods by searching for peaks or modes of the histogram and those based on a multi-thresholding of the histogram. It is the latter that we have used in this paper, it relies particularly on the multi-thresholding method of OTSU. Then, in the case where i) each component of the image admits exactly K classes, K vector thresholds are constructed by an optimal pairing of which each component of the vector thresholds are those resulting from the marginal segmentations. In addition, the multidimensional compact histogram of the multicomponent image is computed and the attribute tuples or ‘colors’ of the histogram are ordered relative to the threshold vectors to produce ( K + 1) intervals in the partial order giving rise to a segmentation of the multidimensional histogram into K classes. The remaining colors of the histogram are assigned to the closest class relative to their center of gravity. ii) In the contrary case, a vectorial spatial matching between the classes of the scalar components of the image is produced to obtain an over-segmentation, then an interclass fusion is performed to obtain a maximum of K classes. Indeed, the relevance of our segmentation method has been highlighted in relation to other methods, such as K-means, using unsupervised and supervised quantitative segmentation evaluation criteria. So the robustness of our method relatively to noise has been tested.
With the advancement of electronics and computing, image has become a fundamental vector of computing and communication [
In view of the technological advances of the sensors and the storage memory capacity, multicomponent images (color, multispectral, hyperspectral and multimodal) are more and more solicited in relation to scalar images (gray levels) because of their richness to characterize a given scene [
The new challenge facing experts in various fields is the processing of this mass of images in order to extract useful information for the purpose of making decisions [
In the field of image processing, segmentation plays a leading role because it is undoubtedly the task which mobilizes more effort [
Conceptually, segmentation consists of extracting from the image primitives, either of contour type or of region type [
There are several types of approaches to segmentation. These approaches can be grouped into two large families [
Classification segmentation involves partitioning a set of pixels of vector attributes into k disjoint classes. Each class groups pixels with characteristics as similar as possible. Obviously the pixels of two distinct classes have very different attributes. In contrast to the previous approaches, this approach only considers the characteristic vectors. To conclude, segmentation into regions requires a pixel connectivity analysis in the labeled image.
In addition, those based on the thresholding of monodimensional (1D) histograms have shown their efficiency in terms of speed of execution and segmentation quality [
Unfortunately, the vectorial aspect of multicomponent images makes it difficult to apply directly to scalar methods, particularly those based on multivariate histogram analysis for the following reasons: i) order on vectors and ii) topology of connectivity in a discrete multidimensional space.
Indeed, the approach of vectorial morphological segmentation by analysis of multidimensional (nD) compact histograms [
To facilitate the manipulation of nD histograms, the nD compact histogram [
The application of a complete lattice to the attributes vectors or “colors” to the compact histogram makes it possible to order the vectors and render the histogram nD compact scalar by substituting the tuples for their order number [
Thus, this paper presents an original method of semi-vectorial hybrid morphological segmentation of multicomponent images based on multi-thresholding analysis of compact multidimensional histograms.
For the understanding of this work, the rest of the document is organized as follows: in section 2, we present the principle and the algorithm of the proposed approach, then in section 3, a theoretical and experimental validation of our approach is presented. We conclude with some prospects for future work.
Let I, a P-component image (P ≥ 2). The principle is declined in several steps (see the flowchart in
a) We first construct the compact histogram related to the image I in order to solve the problem of coding and manipulation of nD histograms. Indeed, the compact histogram stores only the E cells actually occupied in clear it removes the empty cells. For an image with P components, a table of size E × P for storing the tuples in the lexicographic morphological order and a table of size E × 1 corresponding to the number of associated pixels (the number of tuples).
Let Hc be the nD compact histogram of the image I containing only the tuples
and i a fixed row of Hc.
Let j describing the number of planes of the image varying from 1 to P. A tuple in the nD compact histogram can be described by the triplet ( i , H c i , E i ) where E i = c a r d ( H c i ) represents the cardinal or the number of a tuple at the row i in H c .
Let I = [ I 1 , ⋅ ⋅ ⋅ , I j , ⋅ ⋅ ⋅ , I P ] and H c j i the jth multimodal component of the tuple located at row i of Hc, with Ij the jth component of the image.
b) Then we segment each component (Ij) of the image I by multi-thresholding into a fixed number of classes K (K ≥ 2).
Let K j = K max j be the number of classes obtained for the component Ij of the image I. Let be Hj the marginal monodimensionnal compact histogram of the jth component of the image I(Ij) and Ni the number of pixels of a gray level of Hj at the row i.
To generalize, let us call H j i the gray level at the row i of the jth 1D compact histogram of the image I(Ij) and by N j i the number of pixels associated.
The multi-thresholding used here is inspired by the Otsu method applied to the compact marginal 1D histograms Hj. Thus, for a given component j, it produces K j − 1 thresholds denoted [ t 1 j , t 2 j , … … . , t K m a x − 1 j ] = T j ordered in ascending order for a partition of the image into K classes [ C 1 j , C 2 j , ⋅ ⋅ ⋅ , C K j ] where K = K max = K j . They take their values in Hj. Here, we denotes T j the vector of the segmentation thresholds of the component Ij.
The thresholds are obtained by the optimization of Equation (1) in which σ_Intra defines the standard deviation intra-classes whose expression is explicitly given by Equation (2) and other parameters defined by Equations ((3) and (4)).
{ t 1 j , t 2 j , ⋅ ⋅ ⋅ , t K − 1 j } = arg { max t ∈ H j σ I n t r a 2 { t 1 j , t 2 j , ⋅ ⋅ ⋅ , t K − 1 j } } (1)
σ I n t r a 2 = ∑ n = 1 K w n ( μ n − μ T ) (2)
w n = ∑ m = t n − 1 j t n j P m (3)
μ n = ∑ m = t n − 1 j t n j m . P m w n (4)
where Pm and µT are respectively the probability of a gray level pixel m and the total average of the gray levels of the jth component of the image I.
c) On the basis of the thresholds calculated on each component of the image, if K 1 = K 2 = ⋅ ⋅ ⋅ = K j = ⋅ ⋅ ⋅ . = K − 1 then we construct K − 1 threshold vectors Sk with k varying from 1 to K − 1, such that the threshold vector of rank k (Sk) has the scalar thresholds of rank k ( t k j ) from each component of the image I, with S k = ( t k 1 , t k 2 , ⋅ ⋅ ⋅ t k P ) .
In the contrary case, a cross product combination of the partitions of the components of the image is realized.
d) In the first case where the number of classes is fixed and equal to K, the tuples of the nD compact histogram (Hc) are first ordered with respect to the vector thresholds Sk according to a vectorial order of choice and then the K partitions are created. In the case of a complete order, all tuples are immediately assigned to the different classes, otherwise the remaining pixels are assigned to the nearest class. In the second case where the number of classes is variable by class, an over-segmentation is performed and then an interclass fusion is applied to obtain at a maximum the K desired classes.
Note that in this paper, we used the marginal vectorial order, the lexicographic vectorial order and the hybrid vectorial order.
[Iseg]=Fonction FonctSegMorphologiqSemiVectorial (I, K)
// I : Multicomponent image
// K : Number of classes
// Iseg : Segmented image into K classes
P = FonctNberComponents (I) // Return the number of components of I
Hc_nD = FonctHistoCompactnD (I) // Return the nD compact histogram of I
For j =1 to P
H(j).Histo_1D = HistoCompactnD (Ij) // return the 1D compact histogram of components of I
T(j).T = FonctSegMultiThresold (H(j).Histo_1D) // Multi-thresholding of Histo_1D
K(j).K = FonctNberComponents (T(j).T) // Return the number of threshold of Hj
End
If ( K(1).K = K(2).K =….= K − 1 ) then
For k =1 to K − 1
S(k).S = FonctVectorsThresold(T(j).T) // Construction of threshold vectors (K-uplets)
End
H = [Hc_nD ; S(1 à K − 1).S] // concatenation of Hc_nD tuples and threshold vectors
H_ord =FonctOrdonnement (H, ChoiceOrder) // Order tuples according to ChoiceOrder
Classes.C=FonctConstructionClasses (H_ord, S(1 à K − 1).S ) // Classification into K classes
Iseg=FonctImageSegmentée (I, Classes.C) // Image segmentée
Else
ClassesSurSegmente.C=FonctCombinatorial (H_ord, S(1 à K − 1).S ) // over-segmentation
Classes.C=FonctFusionInterClasses (ClassesSurSegmente.C) // Fusion interclasses into K classes
Iseg=FonctImageSegmentée (I, Classes.C) // Segmented image
End
In this section, we first present some segmentation results on synthetic and real images from our proposed segmentation method using some vectorial orders and compared with those of the K-means method [
In order to highlight the relevance of the proposed approach, segmentations have been performed on natural images using marginal order, lexicographic order and hybrid order with absolute referent. Some original images and their segmentation results by the proposed method and the K-means method are presented in Figures 2-4 by varying the number of classes from 2 to 4. Moreover, a series of images of forsythia of which we have the ground truths (reference
expert segmentations from [
These segmentation results clearly show the visual relevance of our approach through its various variants. However, a quantitative evaluation is required to validate our method, hence the following section.
Different assessment methods exist in the literature and can be grouped into two classes: supervised methods and unsupervised methods. In this work, on the one hand for the supervised evaluation, we used Vinet’s criterion [
The Levine-Nazif Intra-region criterion favors segmentation with homogeneous classes and relies on intra-class energy. However, Zeboudj’s criterion takes into account intra-class energy and inter-class energy, thus favoring segmentations with homogeneous and well-separated classes.
The latter two criteria give the best score to the best segmentation. The evaluation results of our segmentation approach and those of K-means are presented in the tables below with an accuracy of 10−4.
In
Consequently, it could be said that the relevance of the segmentation methods is related to the content of the image and the criterion used.
In order to demonstrate the robustness to the noise of our approach, we first added Gaussian noise of different average powers Pb to the synthetic image Savoise [
Images | Proposed semi-vectorial morphological segmentation and K-means | |||
---|---|---|---|---|
Seg Vect Marginal | Seg Vect Lexico | Seg Vect Hybrid | K-means | |
Img 01 | 0.0931 | 0.0820 | 0.0931 | 0.0843 |
Img 02 | 0.1074 | 0.1196 | 0.1019 | 0.1022 |
Img 03 | 0.0953 | 0.0997 | 0.0893 | 0.0898 |
Img 05 | 0.1057 | 0.0981 | 0.0994 | 0.0984 |
Img 06 | 0.1033 | 0.0955 | 0.0860 | 0.0936 |
Img 07 | 0.1475 | 0.1678 | 0.1568 | 0.1406 |
Img 18 | 0.1348 | 0.1494 | 0.1428 | 0.1322 |
Img 20 | 0.0921 | 0.0885 | 0.0785 | 0.0844 |
Number of Classes: k | Image: House | |||
---|---|---|---|---|
Seg Vect Marginal | Seg Vect Lexico | Seg Vect Hybrid | K-means | |
2 | 0.9675 | 0.9154 | 0.9416 | 0.9675 |
3 | 0.9732 | 0.9046 | 0.9460 | 0.9732 |
4 | 0.9640 | 0.9640 | 0.9640 | 0.9640 |
5 | 0.9638 | 0.9638 | 0.9638 | 0.9638 |
Image: Mandrill | ||||
2 | 0.9040 | 0.8756 | 0.8928 | 0.9040 |
3 | 0.9145 | 0.8585 | 0.8642 | 0.9145 |
4 | 0.8880 | 0.8880 | 0.8880 | 0.8880 |
5 | 0.9007 | 0.9007 | 0.9007 | 0.9007 |
Image: Lena | ||||
2 | 0.9547 | 0.9516 | 0.9497 | 0.9547 |
3 | 0.9636 | 0.9526 | 0.9529 | 0.9636 |
4 | 0.9542 | 0.9542 | 0.9542 | 0.9542 |
5 | 0.9685 | 0.9685 | 0.9685 | 0.9685 |
Number of Classes: k | Image: House | |||
---|---|---|---|---|
Seg Vect Marginal | Seg Vect Lexico | Seg Vect Hybrid | K-means | |
2 | 0.7906 | 0.6749 | 0.2160 | 0.7951 |
3 | 0.6967 | 0.6502 | 0.3081 | 0.7925 |
4 | 0.6928 | 0.6928 | 0.6928 | 0.6881 |
5 | 0.5612 | 0.5612 | 0.5612 | 0.6759 |
Image: Mandrill | ||||
2 | 0.4376 | 0.3024 | 0.3758 | 0.4577 |
3 | 0.4547 | 0.2878 | 0.2517 | 0.3902 |
4 | 0.3039 | 0.3039 | 0.3039 | 0.4023 |
5 | 0.3251 | 0.3251 | 0.3251 | 0.3967 |
Image: Lena | ||||
2 | 0.6748 | 0.6588 | 0.4480 | 0.6748 |
3 | 0.6565 | 0.6364 | 0.3991 | 0.6204 |
4 | 0.6318 | 0.6318 | 0.6318 | 0.6200 |
5 | 0.6033 | 0.6033 | 0.6033 | 0.5995 |
Noisy Savoise image (Pb = 0.02) | Segmentation by lexicographic order | Segmentation by hybrid order | Segmentation by K-means | ||
---|---|---|---|---|---|
Noisy Savoise image (Pb = 0.03) | Segmentation by marginal order | Segmentation by lexicographic order | Segmentation by hybrid order | Segmentation by K-means |
Gaussian Noise Power (Pb) | Image Savoise: Performance to noise for a segmentation into 2 classes | |||
---|---|---|---|---|
Seg Vect Marginal | Seg Vect Lexico | Seg Vect Hybrid | K-means | |
Pb = 0.01 | 0 | 0 | 0 | 0 |
Pb = 0.02 | 0 | 3.05 × 10−5 | 0 | 15.3 × 10−5 |
Pb = 0.03 | 2.59 × 10−4 | 7.93 × 10−4 | 4.88 × 10−4 | 891 × 10−5 |
Image Savoise: Performance to noise for a segmentation into 3 classes | ||||
Pb = 0.01 | 375 × 10−5 | 158 × 10−3 | 466 × 10−3 | 0 |
Pb = 0.02 | 218 × 10−3 | 183 × 10−3 | 111 × 10−3 | 793 × 10−6 |
Pb = 0.03 | 514 × 10−3 | 307 × 10−3 | 175 × 10−3 | 129 × 10−3 |
Image Savoise: Performance to noise for a segmentation into 4 classes | ||||
Pb = 0.01 | 824 × 10−6 | 824 × 10−6 | 824 × 10−6 | 894 × 10−4 |
Pb = 0.02 | 421 × 10−4 | 421 × 10−4 | 421 × 10−4 | 885 × 10−4 |
Pb = 0.03 | 140 × 10−3 | 140 × 10−3 | 140 × 10−3 | 121 × 10−4 |
from 2 to 5, an illustration of which is given in
Indeed, an evaluation of the relevance of the segmented images is presented in
In this paper, we have developed a semi-vectorial morphological segmentation strategy with different variants capable of segmenting color images and more generally multicomponent images. The performance of our approach was evaluated and demonstrated its relevance to K-means through quantitative criteria for supervised and unsupervised evaluation of segmentation. On the other hand, the robustness to the Gaussian noise of the proposed segmentation method was highlighted. We also find that the relevance of the quality of a segmentation is on the one hand to the evaluation criterion used and on the other hand to the content of the image. In perspective, we plan to develop a purely vectorial morphological segmentation method.
Kouassi, A., Ouattara, S., Okaingni, J.-C., Vangah, W.J. and Clement, A. (2017) A Semi-Vectorial Hybrid Morphological Segmentation of Multicomponent Images Based on Multithreshold Analysis of Multidimensional Compact Histogram. Open Journal of Applied Sciences, 7, 597-610. https://doi.org/10.4236/ojapps.2017.711043