In this paper, we consider a fuzzy c-means (FCM) clustering algorithm combined with the deterministic annealing method and the Tsallis entropy maximization. The Tsallis entropy is a q-parameter extension of the Shannon entropy. By maximizing the Tsallis entropy within the framework of FCM, membership functions similar to statistical mechanical distribution functions can be derived. One of the major considerations when using this method is how to determine appropriate q values and the highest annealing temperature, T high , for a given data set. Accordingly, in this paper, a method for determining these values simultaneously without introducing any additional parameters is presented. In our approach, the membership function is approximated by a series of expansion methods and the K-means clustering algorithm is utilized as a preprocessing step to estimate a radius of each data distribution. The results of experiments indicate that the proposed method is effective and both q and T high can be determined automatically and algebraically from a given data set.
Techniques from statistical mechanics can be used for the investigation of the macroscopic properties of a physical system consisting of many elements. Recently, research activities utilizing statistical mechanical models or techniques for information processing have become increasingly popular.
Rose et al. [
Tsallis [
One of the major challenges with using Tsallis-DAFCM is the determination of an appropriate value for
Accordingly, we presented a method that can determine both
To overcome this difficulty, in this study, we propose a method that utilizes K-means [
Experiments are performed on numerical data and the Iris Data Set [
Let
be the objective function of FCM, where
On the other hand, the Tsallis entropy is defined as
where
Next, we apply the Tsallis entropy maximization method to FCM [
Then, the objective function in Equation (1) is rewritten as
Under the normalization constraint of
the Tsallis entropy functional becomes
where
where
From Equation (7), the expression for
The performance of Tsallis-DAFCM is superior to those of other entropy-based- FCM methods [
When the temperature is high enough, if the series expansion up to the third order terms is used, Equation (10) becomes
where
Based on the results in Section 3.1, we propose a method for determining both
First, to ensure the convergence of Equation (10), we use the following expression for
where
Then, setting
where
From this equation,
Furthermore, by assuming that the radius
It should be noted that in this equation, for simplicity,
because Equation (7) tends to
By combining the method presented in the previous section with Tsallis- DAFCM, we proposed the following fuzzy c-means clustering algorithm [
In the first algorithm shown in
The second algorithm is the conventional Tsallis-DAFCM algorithm [
1) Set the temperature reduction rate
2) Generate c initial clusters at random locations. Set the current temperature
3) Calculate
4) Calculate the cluster centers using Equation (9).
5) Compare the difference between the current centers and the centers of the previous iteration obtained using the same temperature
turn to Step 2.3.
6) Compare the difference between the current centers and the centers of the previous iteration obtained using a lower temperature
The experimental results in [
To examine the effectiveness of the proposed algorithm, we conducted two experiments.
The first experiment examined whether appropriate
In this experiment, data sets containing (a) three clusters and (b) five clusters were used, as shown in
Dependencies of the maximum, minimum, mean and standard deviation of
In
N | Maximum | Minimum | Mean | Std. deviation |
---|---|---|---|---|
10 | 5.624e−06 | 3.877e−06 | 5.147e−06 | 6.568e−07 |
100 | 6.098e−06 | 3.446e−06 | 5.350e−06 | 6.024e−07 |
1000 | 6.121e−06 | 2.793e−06 | 5.353e−06 | 5.775e−07 |
10,000 | 6.207e−06 | 2.497e−06 | 5.351e−06 | 5.914e−07 |
N | Maximum | Minimum | Mean | Std. deviation | |
---|---|---|---|---|---|
5 | 200.4 | 196.8 | 198.8 | 1.7 | |
10 | 204.6 | 196.8 | 200.1 | 2.2 | |
100 | 203.8 | 195.7 | 198.8 | 1.7 | |
1000 | 305.9 | 195.0 | 199.3 | 6.2 | |
10,000 | 313.4 | 194.8 | 199.6 | 8.1 | |
5 | 197.5 | 190.3 | 192.0 | 2.8 | |
10 | 193.3 | 190.7 | 193.8 | 2.7 | |
100 | 198.3 | 189.5 | 193.1 | 3.0 | |
1000 | 292.8 | 188.1 | 194.1 | 8.8 | |
10,000 | 298.0 | 188.2 | 194.0 | 8.6 |
fixing
From
From
Comparing the results in
Accordingly,
Dependencies of the maximum, minimum, mean and standard deviation of
N | Maximum | Minimum | Mean | Std. deviation |
---|---|---|---|---|
10 | 5.878e−06 | 2.686e−06 | 4.030e−06 | 9.264e−07 |
100 | 5.088e−06 | 2.466e−06 | 3.618e−06 | 5.801e−07 |
1000 | 6.738e−06 | 2.316e−06 | 3.608e−06 | 6.117e−07 |
10,000 | 7.060e−06 | 2.118e−06 | 3.608e−06 | 6.320e−07 |
N | Maximum | Minimum | Mean | Std. deviation | |
---|---|---|---|---|---|
5 | 343.2 | 116.1 | 236.0 | 99.0 | |
10 | 353.4 | 116.1 | 229.4 | 93.7 | |
100 | 356.4 | 116.0 | 193.8 | 91.0 | |
1000 | 394.2 | 115.7 | 203.8 | 92.3 | |
10,000 | 395.0 | 115.7 | 198.4 | 91.6 | |
5 | 189.6 | 115.3 | 150.4 | 33.2 | |
10 | 151.7 | 115.1 | 121.9 | 13.1 | |
100 | 190.2 | 115.2 | 134.7 | 24.5 | |
1000 | 249.5 | 115.1 | 138.3 | 30.0 | |
10,000 | 279.5 | 115.0 | 134.4 | 29.5 |
Comparing these results with those in
In
Substituting the values of
with
for
The figures show no significant difference between
Compared with the clusters in
From these results, it can be confirmed that
In this experiment, the Iris Data Set [
The maximum, minimum, mean, and standard deviation of
From
N | Maximum | Minimum | Mean | Std. deviation |
---|---|---|---|---|
10 | 1.455e−01 | 8.040e−02 | 1.097e−01 | 1.554e−02 |
100 | 1.455e−01 | 7.409e−02 | 1.081e−01 | 1.765e−02 |
1000 | 1.810e−01 | 5.978e−02 | 1.075e−01 | 1.872e−02 |
10,000 | 1.857e−01 | 5.893e−02 | 1.076e−01 | 1.949e−02 |
N | Maximum | Minimum | Mean | Std. deviation | |
---|---|---|---|---|---|
5 | 3.855 | 3.855 | 3.855 | 0.000 | |
10 | 3.855 | 3.855 | 3.855 | 0.000 | |
100 | 4.935 | 3.855 | 3.866 | 0.107 | |
1000 | 4.935 | 3.855 | 3.861 | 0.083 | |
10,000 | 4.935 | 3.855 | 3.862 | 0.085 | |
5 | 2.066 | 2.066 | 2.066 | 0.000 | |
10 | 2.066 | 2.066 | 2.066 | 0.000 | |
100 | 2.066 | 1.849 | 2.064 | 0.022 | |
1000 | 2.066 | 1.849 | 2.063 | 0.026 | |
10,000 | 2.066 | 1.849 | 2.063 | 0.027 |
can be calculated regardless of the value of
It can be found that these tables show that the proposed method gives similar results to those in the Section 5.1, and
It is also found that not only the estimations of the radius are important to improve the accuracy because
The maximum and mean number of data points misclassified by the previous method [
Even though the experiment was repeated 1000 times, the results obtained with the proposed method were almost identical.
By comparing the mean number of misclassified data points of the proposed method with those of the previous method, it can be confirmed the results from both methods are not significantly different when
By comparing the mean number of misclassified data points of the proposed method with those of Tsallis-DAFCM, it can be confirmed the proposed method can get slightly better results. By examining the maximum number of misclassified, we see that Tsallis-DAFCM misclassifies data more often than does the proposed method.
These results confirm that appropriate values of
tering in 1000 trials (Executions were conducted on an Intel(R) Core(TM)2 Duo CPU E6550 @ 2.33 GHz).
From the experimental results in 5.1 and 5.2, the effectiveness of the proposed algorithm using K-means can be evaluated as follows:
1)
2)
3) Much computational time;
4) The numerical data sets and the Iris Data Set can be clustered desirably using
The Tsallis entropy is a q-parameter extension of the Shannon entropy. FCM with the Tsallis entropy maximization has a proper characteristic for clustering, especially when it is combined with DA as Tsallis-DAFCM. The extent of its membership function strongly depends on the parameter
In this study, we proposed a method for approximating the membership function of Tsallis-DAFCM which, by using the K-means method as a preprocessing step, determines
Experiments were performed on the numerical data sets and the Iris Data Set, and showed that the proposed method can more accurately and stably determine
In the future, as described in 5.1, we first intend to explore ways to improve the accuracy of the estimates for
Yasuda, M. (2017) On Utilization of K-Means for Determination of q-Parameter for Tsallis-Entropy-Maximized- FCM. Journal of Software Engineering and Applications, 10, 605-624. https://doi.org/10.4236/jsea.2017.107033
N | Maximum | Minimum | Mean | Std. deviation | |
---|---|---|---|---|---|
5 | 1.825 | 1.165 | 1.437 | 0.225 | |
10 | 2.089 | 1.290 | 1.688 | 0.249 | |
100 | 2.823 | 1.029 | 1.667 | 0.320 | |
1000 | 5.699 | 1.006 | 1.631 | 0.374 | |
10,000 | 9.884 | 1.001 | 1.632 | 0.412 | |
5 | 2.256 | 1.820 | 2.043 | 0.160 | |
10 | 2.279 | 1.959 | 2.038 | 0.106 | |
100 | 2.612 | 1.501 | 2.072 | 0.235 | |
1000 | 2.749 | 1.255 | 2.050 | 0.247 | |
10,000 | 2.889 | 1.198 | 2.060 | 0.244 | |
5 | 2.415 | 2.394 | 2.405 | 0.006 | |
10 | 2.415 | 2.372 | 2.397 | 0.014 | |
100 | 2.421 | 2.374 | 2.404 | 0.010 | |
1000 | 2.427 | 1.888 | 2.401 | 0.031 | |
10,000 | 2.430 | 1.827 | 2.400 | 0.040 | |
5 | 2.461 | 2.410 | 2.449 | 0.019 | |
10 | 2.459 | 2.404 | 2.438 | 0.020 | |
100 | 2.461 | 2.404 | 2.440 | 0.021 | |
1000 | 2.476 | 1.909 | 2.435 | 0.046 | |
10,000 | 2.474 | 1.913 | 2.436 | 0.046 |
N | Maximum | Minimum | Mean | Std. deviation | |
---|---|---|---|---|---|
5 | 2.266 | 1.042 | 1.616 | 0.498 | |
10 | 2.106 | 1.122 | 1.650 | 0.330 | |
100 | 2.596 | 1.018 | 1.784 | 0.281 | |
1000 | 2.400 | 1.020 | 1.781 | 0.238 | |
10,000 | 8.514 | 1.002 | 1.789 | 0.259 | |
5 | 2.884 | 2.308 | 2.562 | 0.184 | |
10 | 2.816 | 1.218 | 2.514 | 0.181 | |
100 | 3.125 | 1.890 | 2.489 | 0.227 | |
1000 | 3.381 | 1.855 | 2.490 | 0.213 | |
10,000 | 4.355 | 1.730 | 2.482 | 0.220 | |
5 | 3.452 | 2.245 | 2.790 | 0.543 | |
10 | 3.452 | 1.860 | 2.786 | 0.567 | |
100 | 3.453 | 1.841 | 3.005 | 0.525 | |
1000 | 3.454 | 1.765 | 2.946 | 0.535 | |
10,000 | 3.454 | 1.760 | 2.979 | 0.529 | |
5 | 3.456 | 2.966 | 3.240 | 0.223 | |
10 | 3.457 | 3.205 | 3.421 | 0.077 | |
100 | 3.488 | 2.962 | 3.361 | 0.151 | |
1000 | 3.492 | 2.630 | 3.341 | 0.183 | |
10,000 | 3.496 | 2.455 | 3.344 | 0.181 |
N | Maximum | Minimum | Mean | Std. deviation | |
---|---|---|---|---|---|
5 | 1.606 | 1.314 | 1.515 | 0.110 | |
10 | 1.829 | 1.057 | 1.412 | 0.268 | |
100 | 1.952 | 1.011 | 1.382 | 0.218 | |
1000 | 1.993 | 1.011 | 1.408 | 0.209 | |
10,000 | 2.000 | 1.010 | 1.410 | 0.205 | |
5 | 1.920 | 1.649 | 1.821 | 0.093 | |
10 | 1.953 | 1.628 | 1.767 | 0.099 | |
100 | 1.984 | 1.270 | 1.730 | 0.160 | |
1000 | 1.998 | 1.201 | 1.744 | 0.164 | |
10,000 | 2.000 | 1.170 | 1.746 | 0.163 | |
5 | 1.069 | 1.013 | 1.044 | 0.022 | |
10 | 1.105 | 1.013 | 1.050 | 0.030 | |
100 | 1.962 | 1.013 | 1.062 | 0.095 | |
1000 | 1.962 | 1.013 | 1.057 | 0.056 | |
10,000 | 1.962 | 1.013 | 1.056 | 0.055 | |
5 | 1.876 | 1.876 | 1.876 | 0.000 | |
10 | 1.876 | 1.876 | 1.876 | 0.000 | |
100 | 1.993 | 1.876 | 1.877 | 0.012 | |
1000 | 1.993 | 1.876 | 1.878 | 0.012 | |
10,000 | 1.993 | 1.876 | 1.877 | 0.012 |
Method | Maximum | Minimum | Mean | ||
---|---|---|---|---|---|
Previous method ( | 5 | 1.515 | 14 | 14 | 14.00 |
10 | 1.412 | 16 | 15 | 16.00 | |
100 | 1.382 | 16 | 16 | 16.00 | |
1000 | 1.408 | 16 | 14 | 16.00 | |
10,000 | 1.410 | 17 | 16 | 16.00 | |
Previous method ( | 5 | 1.821 | 13 | 13 | 13.00 |
10 | 1.767 | 13 | 13 | 13.00 | |
100 | 1.730 | 14 | 13 | 13.05 | |
1000 | 1.744 | 13 | 13 | 13.00 | |
10,000 | 1.746 | 13 | 13 | 13.00 |
Proposed method ( | 5 | 1.044 | 17 | 17 | 17.00 |
---|---|---|---|---|---|
10 | 1.050 | 17 | 17 | 17.00 | |
100 | 1.062 | 17 | 17 | 17.00 | |
1000 | 1.057 | 17 | 17 | 17.00 | |
10,000 | 1.056 | 17 | 17 | 17.00 | |
Proposed method ( | 5 | 1.876 | 16 | 13 | 13.00 |
10 | 1.876 | 13 | 13 | 13.00 | |
100 | 1.877 | 13 | 13 | 13.00 | |
1000 | 1.877 | 13 | 13 | 13.00 | |
10,000 | 1.877 | 13 | 13 | 13.00 | |
Tsallis-DAFCM | 1.2 | 16 | 16 | 16.00 | |
1.6 | 16 | 14 | 15.91 | ||
2.0 | 14 | 14 | 14.00 | ||
2.4 | 27 | 13 | 13.01 | ||
2.8 | 26 | 13 | 13.01 |
Clustering | ||||
---|---|---|---|---|
5 | 0.000 | 0.000 | 0.049 | |
10 | 0.000 | 0.000 | 0.059 | |
100 | 0.000 | 0.000 | 0.060 | |
1000 | 0.016 | 0.023 | 0.059 | |
10,000 | 0.156 | 0.242 | 0.059 | |
5 | 0.000 | 0.000 | 0.047 | |
10 | 0.000 | 0.000 | 0.047 | |
100 | 0.000 | 0.000 | 0.047 | |
1000 | 0.016 | 0.023 | 0.047 | |
10,000 | 0.156 | 0.219 | 0.047 | |
5 | 0.000 | 0.000 | 0.068 | |
10 | 0.000 | 0.023 | 0.067 | |
100 | 0.000 | 0.188 | 0.068 | |
1000 | 0.016 | 1.695 | 0.068 | |
10,000 | 0.156 | 17.586 | 0.068 | |
5 | 0.000 | 0.000 | 0.047 | |
10 | 0.000 | 0.000 | 0.047 | |
100 | 0.000 | 0.000 | 0.047 | |
1000 | 0.016 | 0.039 | 0.047 | |
10,000 | 0.156 | 0.430 | 0.047 |