Image categorization using a semantic hierarchy model with sparse set of salient regions

doi:10.1007/s11704-013-2410-1

Front Comput Sci

2013, Vol. 7

Issue (6) : 838-851 https://doi.org/10.1007/s11704-013-2410-1

RESEARCH ARTICLE

Image categorization using a semantic hierarchy model with sparse set of salient regions

Chunping LIU¹(

), Yang ZHENG², Shengrong GONG¹

1. School of Computer Science and Technology, Soochow University, Suzhou 215006, China; 2. The Second Hospital of Nanjing, Nanjing 210003, China

Download: PDF(810 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

Image categorization in massive image database is an important problem. This paper proposes an approach for image categorization, using sparse set of salient semantic information and hierarchy semantic label tree (HSLT) model. First, to provide more critical image semantics, the proposed sparse set of salient regions only at the focuses of visual attention instead of the entire scene was formed by our proposed saliency detection model with incorporating low and high level feature and Shotton’s semantic texton forests (STFs) method. Second, we also propose a new HSLT model in terms of the sparse regional semantic information to automatically build a semantic image hierarchy, which explicitly encodes a general to specific image relationship. And last, we archived image dataset using image hierarchical semantic, which is help to improve the performance of image organizing and browsing. Extension experimental results showed that the use of semantic hierarchies as a hierarchical organizing framework provides a better image annotation and organization, improves the accuracy and reduces human’s effort.

Keywords salient region sparse set semantic hierarchy image annotation image categorization

Corresponding Author(s): LIU Chunping,Email:cpliu@suda.edu.cn

Issue Date: 01 December 2013

Cite this article:

Chunping LIU,Yang ZHENG,Shengrong GONG. Image categorization using a semantic hierarchy model with sparse set of salient regions[J]. Front Comput Sci, 2013, 7(6): 838-851.

URL:

https://academic.hep.com.cn/fcs/EN/10.1007/s11704-013-2410-1
https://academic.hep.com.cn/fcs/EN/Y2013/V7/I6/838

1	Grifflths T, Jordan M, Tenenbaum J, Blei D M. Hierarchical topic models and the nested chinese restaurant process. Advances in Neural Information Processing Systems , 2004, 16: 106-114
2	Bannour H, Hudelot C. Towards ontologies for image interpretation and annotation. In: Proceedings of the 9th International Workshop on Content-Based Multimedia Indexing (CBMI) . 2011, 211-216
3	Tousch A M, Herbin S, Audibert J Y. Semantic hierarchies for image annotation: a survey. Pattern Recognition , 2012, 45(1): 333-345 doi: 10.1016/j.patcog.2011.05.017
4	Marszalek M, Schmid C. Semantic hierarchies for visual object recognition. In: Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2007, 1-7 doi: 10.1109/CVPR.2007.383272
5	Wei X Y, Ngo C W. Ontology-enriched semantic space for video search. In: Proceedings of the 15th International Conference on Multimedia . 2007, 981-990
6	Deng J, Dong W, Socher R, Li L J, Li K, Fei-Fei L. Imagenet: a largescale hierarchical image database. In: Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2009, 248-255 doi: 10.1109/CVPR.2009.5206848
7	Snow R, Jurafsky D, Ng A Y. Semantic taxonomy induction from heterogenous evidence. In: Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics . 2006, 801-808
8	Miller G A. Wordnet: a lexical database for English. Communications of the ACM , 1995, 38(11): 39-41 doi: 10.1145/219717.219748
9	Jin Y, Khan L, Wang L, Awad M. Image annotations by combining multiple evidence & wordnet. In: Proceedings of the 13th Annual ACM International Conference on Multimedia . 2005, 706-715
10	Joshi D, Datta R, Zhuang Z, Weiss W, Friedenberg M, Li J, Wang J Z. Paragrab: a comprehensive architecture for web image management and multimodal querying. In: Proceedings of the 32nd International Conference on Very Large Data Bases . 2006, 1163-1166
11	Datta R, Ge W, Li J, Wang J Z. Toward bridging the annotationretrieval gap in image search. IEEE MultiMedia , 2007, 14(3): 24-35 doi: 10.1109/MMUL.2007.67
12	Torralba A, Fergus R, Freeman W T. 80 million tiny images: a large data set for nonparametric object and scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2008, 30(11): 1958-1970 doi: 10.1109/TPAMI.2008.128
13	Sivic J, Russell B C, Zisserman A, Freeman W T, Efros A A. Unsupervised discovery of visual object class hierarchies. In: Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2008, 1-8 doi: 10.1109/CVPR.2008.4587622
14	Bart E, Porteous I, Perona P, Welling M. Unsupervised learning of visual taxonomies. In: Proceedings of the 2008 IEEE onference on Computer Vision and Pattern Recognition (CVPR) . 2008, 1-8
15	Yao B Z, Yang X, Lin L, Lee M W, Zhu S C. I2t: image parsing to text description. Proceedings of the IEEE , 2010, 98(8): 1485-1508 doi: 10.1109/JPROC.2010.2050411
16	Griffln G, Perona P. Learning and using taxonomies for fast visual categorization. In: Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2008, 1-8 doi: 10.1109/CVPR.2008.4587410
17	Marszaek M, Schmid C. Constructing category hierarchies for visual recognition. In: Proceedings of the 10th European Conference on Computer Vision . 2008, 479-491
18	Ahuja N, Todorovic S. Learning the taxonomy and models of categories present in arbitrary images. In: Proceedings of 11th IEEE International Conference on Computer Vision (ICCV) . 2007, 1-8
19	Li L J, Wang C, Lim Y, Blei D M, Fei-Fei L. Building and using a semantivisual image hierarchy. In: Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2010, 3336-3343 doi: 10.1109/CVPR.2010.5540027
20	Fan J, Gao Y, Luo H. Hierarchical classification for automatic image annotation. In: Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval . 2007, 111-118 doi: 10.1145/1277741.1277763
21	Fan J, Gao Y, Luo H. Integrating concept ontology and multitask learning to achieve more effective classifier training for multilevel image annotation. IEEE Transactions on Image Processing , 2008, 17(3): 407-426 doi: 10.1109/TIP.2008.916999
22	Fan J, Gao Y, Luo H, Jain R. Mining multilevel image semantics via hierarchical classification. IEEE Transactions on Multimedia , 2008, 10(2): 167-187 doi: 10.1109/TMM.2007.911775
23	Wu L, Hua X S, Yu N, Ma W Y, Li S. Flickr distance: a relationship measure for visual concepts. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2012, 34(5): 863-875 doi: 10.1109/TPAMI.2011.195
24	Bannour H, Hudelot C. Building semantic hierarchies faithful to image semantics. In: Proceedings of the 18th International Conference on Advances in Multimedia Modeling . 2012, 4-15
25	Moosmann F, Nowak E, Jurie F. Randomized clustering forests for image classification. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2008, 30(9): 1632-1646 doi: 10.1109/TPAMI.2007.70822
26	Lowe D G. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision , 2004, 60(2): 91-110 doi: 10.1023/B:VISI.0000029664.99615.94
27	Wu L, Hu Y, Li M, Yu N, Hua X S. Scale-invariant visual language modeling for object categorization. IEEE Transactions on Multimedia , 2009, 11(2): 286-294 doi: 10.1109/TMM.2008.2009692
28	Bannour H, Hudelot C. Hierarchical image annotation using semantic hierarchies. In: Proceedings of the 21st ACM International Conference on Information and Knowledge Management . 2012, 2431-2434
29	Deng J, Berg A C, Li K, Fei-Fei L. What does classifying more than 10 000 image categories tell us? In: Proceedings of the 11th European Conference on Computer Vision . 2010, 71-84
30	Theeuwes J. Top–down and bottom–up control of visual selection. Acta Psychologica , 2010, 135(2): 77-99 doi: 10.1016/j.actpsy.2010.02.006
31	Hou X, Zhang L. Saliency detection: a spectral residual approach. In: Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2007, 1-8 doi: 10.1109/CVPR.2007.383267
32	Harel J, Koch C, Perona P. Graph-based visual saliency. Advances in Neural Information Processing Systems , 2006, 545-552
33	Achanta R, Hemami S, Estrada F, Susstrunk S. Frequency-tuned salient region detection. In: Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2009, 1597-1604 doi: 10.1109/CVPR.2009.5206596
34	Goferman S, Zelnik-Manor L, Tal A. Context-aware saliency detection. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2012, 34(10): 1915-1926 doi: 10.1109/TPAMI.2011.272
35	Zheng Y, Liu C p, Liu G, Wang Z h. Saliency detection based on inhibition of blur regions. Microelectronics & Computer , 2012, 29(3): 84-88
36	Yang Z, Chunping L, Zhaohui W, Yi J, Shengrong G. A saliency detection model based on multi-feature fusion. In: Proceedings of the 7th International Conference on Computational Intelligence and Security (CIS) . 2011, 1062-1066
37	Sivic J, Russell B C, Efros A A, Zisserman A, Freeman W T. Discovering objects and their location in images. In: Proceedings of the 10th IEEE International Conference on Computer Vision (ICCV) . 2005, 370-377
38	Li Z, Wang Y, Chen J, Xu J, Larid J. Image topic discovery with saliency detection. Journal of Machine Learning Research , 2003, 3: 993-1022
39	Wu L, Hoi S C. Enhancing bag-of-words models with semanticspreserving metric learning. IEEE Multimedia Magazine , 2011, 18(1): 24-37 doi: 10.1109/MMUL.2011.7
40	Wu L, Hoi S C, Yu N. Semantics-preserving bag-of-words models and applications. IEEE Transactions on Image Processing , 2010, 19(7): 1908-1920 doi: 10.1109/TIP.2010.2045169
41	Shotton J, Johnson M, Cipolla R. Semantic texton forests for image categorization and segmentation. In: Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2008, 1-8 doi: 10.1109/CVPR.2008.4587503
42	Friston K, Kiebel S. Cortical circuits for perceptual inference. Neural Networks , 2009, 22(8): 1093-1104 doi: 10.1016/j.neunet.2009.07.023
43	Shotton J, Winn J, Rother C, Criminisi A. The MSRC 21-class object recognition database, 2006
44	Everingham M, Van Gool L, Williams C K, Winn J, Zisserman A. The pascal visual object classes (VOC) challenge. International Journal of Computer Vision , 2010, 88(2): 303-338 doi: 10.1007/s11263-009-0275-4
45	Shotton J, Winn J, Rother C, Criminisi A. Textonboost for image understanding: multi-class object recognition and segmentation by jointly modeling texture, layout, and context. International Journal of Computer Vision , 2009, 81(1): 2-23 doi: 10.1007/s11263-007-0109-1

[1]	Rizwan Ahmed KHAN, Alexandre MEYER, Hubert KONIK, Saida BOUAKAZ. Saliency-based framework for facial expression recognition[J]. Front. Comput. Sci., 2019, 13(1): 183-198.
[2]	Jian-Hao LUO,Wang ZHOU,Jianxin WU. Image categorization with resource constraints: introduction, challenges and advances[J]. Front. Comput. Sci., 2017, 11(1): 13-26.

Viewed

Full text

Abstract

Cited

Shared

Discussed