Soft spectral clustering ensemble applied to image segmentation

doi:10.1007/s11704-010-0161-9

Front Comput Sci Chin

0, Vol.

Issue () : 66-78 https://doi.org/10.1007/s11704-010-0161-9

RESEARCH ARTICLE

Soft spectral clustering ensemble applied to image segmentation

Jianhua JIA^1,2(

), Bingxiang LIU¹, Licheng JIAO²

1. School of Information Engineering, Jingdezhen Ceramic Institute, Jingdezhen 333002, China; 2. Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education of China and Institute of Intelligent Information Processing, Xidian University, Xi'an 710071, China

Download: PDF(677 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

An unsupervised learning algorithm, named soft spectral clustering ensemble (SSCE), is proposed in this paper. Until now many proposed ensemble algorithms cannot be used on image data, even images of a mere 256 × 256 pixels are too expensive in computational cost and storage. The proposed method is suitable for performing image segmentation and can, to some degree, solve some open problems of spectral clustering (SC). In this paper, a random scaling parameter and Nystr?m approximation are applied to generate the individual spectral clusters for ensemble learning. We slightly modify the standard SC algorithm to aquire a soft partition and then map it via a centralized logcontrast transform to relax the constraint of probability data, the sum of which is one. All mapped data are concatenated to form the new features for each instance. Principal component analysis (PCA) is used to reduce the dimension of the new features. The final aggregated result can be achieved by clustering dimension-reduced data. Experimental results, on UCI data and different image types, show that the proposed algorithm is more efficient compared with some existing consensus functions.

Keywords spectral clustering (SC) Nystr?m approximation centralized logcontrast transform principal component analysis (PCA) ensemble learning

Corresponding Author(s): JIA Jianhua,Email:jjh163yx@163.com

Issue Date: 05 March 2011

Cite this article:

Jianhua JIA,Bingxiang LIU,Licheng JIAO. Soft spectral clustering ensemble applied to image segmentation[J]. Front Comput Sci Chin, 0, (): 66-78.

URL:

https://academic.hep.com.cn/fcs/EN/10.1007/s11704-010-0161-9
https://academic.hep.com.cn/fcs/EN/Y0/V/I/66

Fig.1 Framework of spectral clustering ensemble

Tab.1 Properties of Selected UCI Data Sets

Fig.2 Error rate of data sets of UCI data sets using different clustering algorithm Chess (ks-vs-kp). (a) Iris data set, (b) Sonar data set, (c) Wine data set, (d) Breast_cancer data set, (e) Chess (ks-vs-kp)

Fig.3 Error rate of large sample data set (Chess (ks-vs-kp) Data sets) of UCI data sets. (a) Average value and standard deviation of error rates by SC_Nys with the changing of the scaling parameters over Chess (ks-vs-kp) data set; (b) average value of SC_Nys and SSCE with different intervals of scaling parameter. The dashed line denotes the standard deviation of error rate by SSCE

Tab.2 Comparison of average error rate

Tab.3 CPU time of single runs of different consensus functions on the UCI data sets (s)

Fig.4 Performance of SSCE while varying the number of constituent clusterings. (a) Iris data; (b) Chess (ks-vs-kp) data

Fig.5 Results of 4-class synthesis texture image segmentation. (a) Original image; (b) real segmentation; (c) result of SC_Nys (92.32%); (d )result of MSCE (94.99%); (e) result of SSCE (97.10%)

Fig.6 Results of some synthesis texture images segmentation. (a) Original images; (b) results of SC_Nys; (c) results of MSCE; (d) results of SSCE

Fig.7 Results of Ku-band SAR image segmentation. (a) Original image; (b) result of SC_Nys; (c) result of MSCE; (d) result of SSCE

Fig.8 Results of X-band SAR image segmentation. (a) Original image; (b) result of SC_Nys; (c) result of MSCE; (d) result of the proposed algorithm

Fig.9 (a) SAR image-A (mountain/plain); (b )result of SC_Nys; (c) result of MSCE; (d) result of SSCE; (e) SAR image-B (arass/forest); (f) result of SC_Nys; (g) result of MSCE; (h) result of SSCE

1	Duda R O, Hart P E, Stork D G. Pattern classification. New York: Wiley Interscience, 2001
2	Dietterich T G. Machine-learning research: four current directions. AI Magazine , 1997, 18(4): 97–136
3	Zhou Z H, Wu J, Tang W. Ensembling neural networks: many could be better than all. Artificial Intelligence , 2002, 137(1-2): 239–263 doi: 10.1016/S0004-3702(02)00190-X
4	Tang W, Zhou Z. Bagging-based selective clustering ensemble. Journal of Software , 2005, 34(12): 496–502 doi: 10.1360/jos160496
5	Strehl A, Ghosh J. Cluster ensembles—a knowledge reuse framework for combining multiple partitions. Journal of Machine Learning Research , 2002, 3(12): 583–617
6	Liu X, Wang Z, Chen W, Li X. Remote sensing images hierarchical clustering using Markov random field and generalized Gaussian mixture models. Journal of Remote Sensing , 2007, 11(6): 838–844
7	Fan G, Xia X. A joint multicontext and multiscale approach to Bayesian image segmentation. IEEE Transactions on Geoscience and Remote Sensing , 2001, 39(12): 2680–2688 doi: 10.1109/36.975002
8	Topchy A P, Law M H C, Jain A K, Fred A L. Analysis of consensus partition in cluster ensemble. In: Proceedings of the 4th IEEE International Conference on Data Mining . 2004, 225–232
9	Munkres J. Algorithms for the assignment and transportation problems. Journal of the Society for Industrial and Applied Mathematics , 1957, 5(1): 32–38 doi: 10.1137/0105003
10	MacQueen J B. Some methods for classification and analysis of multivariate observation. In: Proceeding of the 5th Berkeley Symp. on Mathematical Statistics and Probability . Berkeley: University of California Press, 1967, 281–297
11	Ding C H Q, He X, Zha H, Gu M, Simon H D. A min-max cut algorithm for graph partitioning and data clustering. In: Proceedings of IEEE International Conference on Data Mining . 2001, 107–114
12	Shi J, Malik J. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2000, 22(8): 888–905 doi: 10.1109/34.868688
13	Ng A, Jordan M, Weiss Y. On spectral clustering: analysis and an algorithm. Advance of Neural Information Processing System . Cambridge: MIT Press, 2002, 849–856
14	Wang S, Siskind J M. Image segmentation with ratio cut. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2003, 25(6): 675–690 doi: 10.1109/TPAMI.2003.1201819
15	Bach F R, Jordan M I. Blind one-microphone speech separation: a spectral learning approach. In: Proceedings of Advance of Neural Information Processing System (NIPS) . Cambridge: MIT Press, 2005
16	Odobez J M, Gatica-Perez D, Guillemot M. Video shot clustering using spectral methods. In: Proceedings of International Workshop on Content-based Multimedia Indexing . Rennes, 2003, 94–102
17	Zhang X, Jiao L, Liu F, Bo L, Gong M. Spectral clustering ensemble applied to SAR image segmentation. IEEE Transactions on Geoscience and Remote Sensing , 2008, 46(7): 2126–2136 doi: 10.1109/TGRS.2008.918647
18	Dhillon I S. Co-clustering documents and words using bipartite spectral graph parititioning. In: Proceedings of Knowledge Discovery and Data Mine . 2001, 269–274
19	Topchy A, Jain A K, Punch W. Clustering ensembles: models of consensus and weak partitions. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2005, 27(12): 1866–1881
20	Topchy A, Jain A K, Punch W. Combining multiple weak clusterings. In: Proceedings of IEEE International Conference on Data Mining. Melbourne , 2003, 331–338
21	Fred A L N, Jain A K. Combining multiple clusterings using evidence accumulation. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2005, 27(6): 835–850 doi: 10.1109/TPAMI.2005.113
22	Fred A L N. Data clustering using evidence accumulation. In: Proceeding of the 16th International Conference on Pattern Recognition . Quebec, 2002, 276–280
23	Fred A L N. Finding consistent clusters in data partitions. In: Proceeding of the 3d International Workshop on Multiple Classifier System . Roli J K F, ed. LNCS 2364, 2001, 309–318
24	Fern X Z, Brodley C E. Random projection for high dimensional data clustering: a cluster ensemble approach. In: Proceedings of the 20th International Conference on Machine Learning (ICML). Washington DC , 2003, 186–193
25	Fischer B, Buhmann J M. Path-based clustering for grouping of smooth curves and texture segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2003, 25(4): 513–518 doi: 10.1109/TPAMI.2003.1190577
26	Minaei-Bidgoli B, Topchy A, Punch W F. Ensembles of partitions via data resampling. In: Proceedings of International Conference on Information Technology, ITCC. LasVegas , 2004, 188–192
27	Fred A L N. Finding consistent clusters in data partitions. In: Proceedings of the 3d International Workshop on Multiple Classifier System . Roli J K F (ed.), LNCS 2364, 2001, 309–318
28	Wang X, Yang C Y, Zhou J. Clustering aggregation by probability accumulation. Pattern Recognition , 2009, 42(5): 668–675 doi: 10.1016/j.patcog.2008.09.013
29	Karypis G, Kumar V. A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM Journal on Scientific Computing , 1995, 20(1): 359–392 doi: 10.1137/S1064827595287997
30	Karypis G, Aggarwal R, Kumar V, Shekhar S. Multilevel hypergraph partitioning: application in VLSI domain. In: Proceedings of ACM/IEEE Design Automation Conference , 1997, 526–529
31	Fern X Z, Brodley C E. Solving cluster ensemble problems by bipartite graph partitioning. In: Proceedings of the 21st International Conference on Machine Learning (ICML) . Canada, 2004
32	Dempster A P, Laird N M, Rubin D B. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series A (General) , 1977, 39(1): 1–38
33	Punera K, Ghosh J. Soft cluster ensembles. In: Oliveira J V, Pedrycz W, eds. Advances in Fuzzy Clustering and its Applications . Wiley, 2007, 69–90 doi: 10.1002/9780470061190.ch4
34	Gao Y, Gu S, Xia L, . Fuzzy clustering ensemble based on mutual information. In: Proceedings of the 6th WSEAS International Conference on Simulation, Modelling and Optimization, Lisbon, Portugal , 2006, 476–481 ().
35	Yu Z, Deng Z, Wong H, . Fuzzy cluster ensemble and its application on 3d head model classification. In: Proceedings of International Joint Conference on Neural Networks (IJCNN) . 2008, 569–576
36	Zhai S, Luo B, Guo Y. Fuzzy clustering ensemble based on dual boosting. In: Proceedings of 4th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD) . 2007 doi: 10.1109/FSKD.2007.316
37	Fowlkes C, Belongie S, Chung F, Malik J. Spectral grouping using the Nystr?m method. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2004, 26(2): 214–225 doi: 10.1109/TPAMI.2004.1262185
38	Zhang X. Classification and Segmentation of SAR Images Based on Selective Feature Fusion and Ensemble Learning. Xi'an: Xidian University, 2006
39	Barthelemy J P, Leclerc B. The median procedure for partitions. In: Cox I J, Hansen P, Julesz B, eds. Partitioning Data Sets. American Mathematical Society, Providence , 1995, 3–34
40	Aitchison J. The statistical analysis of compositional data. Journal of the Royal Statistical Society, Series B. Methodological , 1982, 44(2): 139–177
41	Billheimer D, Guttorp P, Fagan W F. Statistical analysis and interpretation of discrete compositional data. National Center for Statistics and the Environment (NRCSE) Technical Report NRCSE-TRS, 11, 1998
42	Blake C L, Merz C J. UCI repository of machine learning databases.1998, available from: http://www.ics.uci.edu/mlearn/MLRepository
43	Hansen L, Salamon P. Neural network ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence , 1990, 12(10): 993–1001 doi: 10.1109/34.58871
44	Melville P, Mooney R J. Constructing diverse classifier ensembles using artificial training examples. In: Proceedings of the 18th International Joint Conference on Artificial Intelligence . 2003, 505–510
45	Opitz D, Maclin R. Popular ensemble methods: an empirical study. Journal of Artificial Intelligence Research , 1999, 11(8): 169–198

[1]	Baiyou QIAO, Zhongqiang WU, Ling MA, Yicheng Zhou, Yunjiao SUN. Effective ensemble learning approach for SST field prediction using attention-based PredRNN[J]. Front. Comput. Sci., 2023, 17(1): 171601-.
[2]	Juntao CHEN, Quan ZOU, Jing LI. DeepM6ASeq-EL: prediction of human N6-methyladenosine (m ⁶A) sites with LSTM and ensemble learning[J]. Front. Comput. Sci., 2022, 16(2): 162302-.
[3]	Xibin DONG, Zhiwen YU, Wenming CAO, Yifan SHI, Qianli MA. A survey on ensemble learning[J]. Front. Comput. Sci., 2020, 14(2): 241-258.
[4]	Tao SUN, Zhi-Hua ZHOU. Structural diversity for decision tree ensemble learning[J]. Front. Comput. Sci., 2018, 12(3): 560-570.
[5]	Bo SUN, Haiyan CHEN, Jiandong WANG, Hua XIE. Evolutionary under-sampling based bagging ensemble method for imbalanced data classification[J]. Front. Comput. Sci., 2018, 12(2): 331-350.
[6]	Qian LI, Gang LI, Wenjia NIU, Yanan CAO, Liang CHANG, Jianlong TAN, Li GUO. Boosting imbalanced data learning with Wiener process oversampling[J]. Front. Comput. Sci., 2017, 11(5): 836-851.
[7]	Lean YU, Shouyang WANG, Kin Keung LAI. Developing an SVM-based ensemble learning system for customer risk identification collaborating with customer relationship management[J]. Front Comput Sci Chin, 2010, 4(2): 196-203.

Viewed

Full text

Abstract

Cited

Shared

Discussed