Please wait a minute...
Frontiers of Computer Science

ISSN 2095-2228

ISSN 2095-2236(Online)

CN 10-1014/TP

Postal Subscription Code 80-970

2018 Impact Factor: 1.129

Front Comput Sci    2013, Vol. 7 Issue (6) : 838-851    https://doi.org/10.1007/s11704-013-2410-1
RESEARCH ARTICLE
Image categorization using a semantic hierarchy model with sparse set of salient regions
Chunping LIU1(), Yang ZHENG2, Shengrong GONG1
1. School of Computer Science and Technology, Soochow University, Suzhou 215006, China; 2. The Second Hospital of Nanjing, Nanjing 210003, China
 Download: PDF(810 KB)   HTML
 Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks
Abstract

Image categorization in massive image database is an important problem. This paper proposes an approach for image categorization, using sparse set of salient semantic information and hierarchy semantic label tree (HSLT) model. First, to provide more critical image semantics, the proposed sparse set of salient regions only at the focuses of visual attention instead of the entire scene was formed by our proposed saliency detection model with incorporating low and high level feature and Shotton’s semantic texton forests (STFs) method. Second, we also propose a new HSLT model in terms of the sparse regional semantic information to automatically build a semantic image hierarchy, which explicitly encodes a general to specific image relationship. And last, we archived image dataset using image hierarchical semantic, which is help to improve the performance of image organizing and browsing. Extension experimental results showed that the use of semantic hierarchies as a hierarchical organizing framework provides a better image annotation and organization, improves the accuracy and reduces human’s effort.

Keywords salient region      sparse set      semantic hierarchy      image annotation      image categorization     
Corresponding Author(s): LIU Chunping,Email:cpliu@suda.edu.cn   
Issue Date: 01 December 2013
 Cite this article:   
Chunping LIU,Yang ZHENG,Shengrong GONG. Image categorization using a semantic hierarchy model with sparse set of salient regions[J]. Front Comput Sci, 2013, 7(6): 838-851.
 URL:  
https://academic.hep.com.cn/fcs/EN/10.1007/s11704-013-2410-1
https://academic.hep.com.cn/fcs/EN/Y2013/V7/I6/838
1 Grifflths T, Jordan M, Tenenbaum J, Blei D M. Hierarchical topic models and the nested chinese restaurant process. Advances in Neural Information Processing Systems , 2004, 16: 106-114
2 Bannour H, Hudelot C. Towards ontologies for image interpretation and annotation. In: Proceedings of the 9th International Workshop on Content-Based Multimedia Indexing (CBMI) . 2011, 211-216
3 Tousch A M, Herbin S, Audibert J Y. Semantic hierarchies for image annotation: a survey. Pattern Recognition , 2012, 45(1): 333-345
doi: 10.1016/j.patcog.2011.05.017
4 Marszalek M, Schmid C. Semantic hierarchies for visual object recognition. In: Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2007, 1-7
doi: 10.1109/CVPR.2007.383272
5 Wei X Y, Ngo C W. Ontology-enriched semantic space for video search. In: Proceedings of the 15th International Conference on Multimedia . 2007, 981-990
6 Deng J, Dong W, Socher R, Li L J, Li K, Fei-Fei L. Imagenet: a largescale hierarchical image database. In: Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2009, 248-255
doi: 10.1109/CVPR.2009.5206848
7 Snow R, Jurafsky D, Ng A Y. Semantic taxonomy induction from heterogenous evidence. In: Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics . 2006, 801-808
8 Miller G A. Wordnet: a lexical database for English. Communications of the ACM , 1995, 38(11): 39-41
doi: 10.1145/219717.219748
9 Jin Y, Khan L, Wang L, Awad M. Image annotations by combining multiple evidence & wordnet. In: Proceedings of the 13th Annual ACM International Conference on Multimedia . 2005, 706-715
10 Joshi D, Datta R, Zhuang Z, Weiss W, Friedenberg M, Li J, Wang J Z. Paragrab: a comprehensive architecture for web image management and multimodal querying. In: Proceedings of the 32nd International Conference on Very Large Data Bases . 2006, 1163-1166
11 Datta R, Ge W, Li J, Wang J Z. Toward bridging the annotationretrieval gap in image search. IEEE MultiMedia , 2007, 14(3): 24-35
doi: 10.1109/MMUL.2007.67
12 Torralba A, Fergus R, Freeman W T. 80 million tiny images: a large data set for nonparametric object and scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2008, 30(11): 1958-1970
doi: 10.1109/TPAMI.2008.128
13 Sivic J, Russell B C, Zisserman A, Freeman W T, Efros A A. Unsupervised discovery of visual object class hierarchies. In: Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2008, 1-8
doi: 10.1109/CVPR.2008.4587622
14 Bart E, Porteous I, Perona P, Welling M. Unsupervised learning of visual taxonomies. In: Proceedings of the 2008 IEEE onference on Computer Vision and Pattern Recognition (CVPR) . 2008, 1-8
15 Yao B Z, Yang X, Lin L, Lee M W, Zhu S C. I2t: image parsing to text description. Proceedings of the IEEE , 2010, 98(8): 1485-1508
doi: 10.1109/JPROC.2010.2050411
16 Griffln G, Perona P. Learning and using taxonomies for fast visual categorization. In: Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2008, 1-8
doi: 10.1109/CVPR.2008.4587410
17 Marszaek M, Schmid C. Constructing category hierarchies for visual recognition. In: Proceedings of the 10th European Conference on Computer Vision . 2008, 479-491
18 Ahuja N, Todorovic S. Learning the taxonomy and models of categories present in arbitrary images. In: Proceedings of 11th IEEE International Conference on Computer Vision (ICCV) . 2007, 1-8
19 Li L J, Wang C, Lim Y, Blei D M, Fei-Fei L. Building and using a semantivisual image hierarchy. In: Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2010, 3336-3343
doi: 10.1109/CVPR.2010.5540027
20 Fan J, Gao Y, Luo H. Hierarchical classification for automatic image annotation. In: Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval . 2007, 111-118
doi: 10.1145/1277741.1277763
21 Fan J, Gao Y, Luo H. Integrating concept ontology and multitask learning to achieve more effective classifier training for multilevel image annotation. IEEE Transactions on Image Processing , 2008, 17(3): 407-426
doi: 10.1109/TIP.2008.916999
22 Fan J, Gao Y, Luo H, Jain R. Mining multilevel image semantics via hierarchical classification. IEEE Transactions on Multimedia , 2008, 10(2): 167-187
doi: 10.1109/TMM.2007.911775
23 Wu L, Hua X S, Yu N, Ma W Y, Li S. Flickr distance: a relationship measure for visual concepts. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2012, 34(5): 863-875
doi: 10.1109/TPAMI.2011.195
24 Bannour H, Hudelot C. Building semantic hierarchies faithful to image semantics. In: Proceedings of the 18th International Conference on Advances in Multimedia Modeling . 2012, 4-15
25 Moosmann F, Nowak E, Jurie F. Randomized clustering forests for image classification. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2008, 30(9): 1632-1646
doi: 10.1109/TPAMI.2007.70822
26 Lowe D G. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision , 2004, 60(2): 91-110
doi: 10.1023/B:VISI.0000029664.99615.94
27 Wu L, Hu Y, Li M, Yu N, Hua X S. Scale-invariant visual language modeling for object categorization. IEEE Transactions on Multimedia , 2009, 11(2): 286-294
doi: 10.1109/TMM.2008.2009692
28 Bannour H, Hudelot C. Hierarchical image annotation using semantic hierarchies. In: Proceedings of the 21st ACM International Conference on Information and Knowledge Management . 2012, 2431-2434
29 Deng J, Berg A C, Li K, Fei-Fei L. What does classifying more than 10 000 image categories tell us? In: Proceedings of the 11th European Conference on Computer Vision . 2010, 71-84
30 Theeuwes J. Top–down and bottom–up control of visual selection. Acta Psychologica , 2010, 135(2): 77-99
doi: 10.1016/j.actpsy.2010.02.006
31 Hou X, Zhang L. Saliency detection: a spectral residual approach. In: Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2007, 1-8
doi: 10.1109/CVPR.2007.383267
32 Harel J, Koch C, Perona P. Graph-based visual saliency. Advances in Neural Information Processing Systems , 2006, 545-552
33 Achanta R, Hemami S, Estrada F, Susstrunk S. Frequency-tuned salient region detection. In: Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2009, 1597-1604
doi: 10.1109/CVPR.2009.5206596
34 Goferman S, Zelnik-Manor L, Tal A. Context-aware saliency detection. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2012, 34(10): 1915-1926
doi: 10.1109/TPAMI.2011.272
35 Zheng Y, Liu C p, Liu G, Wang Z h. Saliency detection based on inhibition of blur regions. Microelectronics & Computer , 2012, 29(3): 84-88
36 Yang Z, Chunping L, Zhaohui W, Yi J, Shengrong G. A saliency detection model based on multi-feature fusion. In: Proceedings of the 7th International Conference on Computational Intelligence and Security (CIS) . 2011, 1062-1066
37 Sivic J, Russell B C, Efros A A, Zisserman A, Freeman W T. Discovering objects and their location in images. In: Proceedings of the 10th IEEE International Conference on Computer Vision (ICCV) . 2005, 370-377
38 Li Z, Wang Y, Chen J, Xu J, Larid J. Image topic discovery with saliency detection. Journal of Machine Learning Research , 2003, 3: 993-1022
39 Wu L, Hoi S C. Enhancing bag-of-words models with semanticspreserving metric learning. IEEE Multimedia Magazine , 2011, 18(1): 24-37
doi: 10.1109/MMUL.2011.7
40 Wu L, Hoi S C, Yu N. Semantics-preserving bag-of-words models and applications. IEEE Transactions on Image Processing , 2010, 19(7): 1908-1920
doi: 10.1109/TIP.2010.2045169
41 Shotton J, Johnson M, Cipolla R. Semantic texton forests for image categorization and segmentation. In: Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2008, 1-8
doi: 10.1109/CVPR.2008.4587503
42 Friston K, Kiebel S. Cortical circuits for perceptual inference. Neural Networks , 2009, 22(8): 1093-1104
doi: 10.1016/j.neunet.2009.07.023
43 Shotton J, Winn J, Rother C, Criminisi A. The MSRC 21-class object recognition database, 2006
44 Everingham M, Van Gool L, Williams C K, Winn J, Zisserman A. The pascal visual object classes (VOC) challenge. International Journal of Computer Vision , 2010, 88(2): 303-338
doi: 10.1007/s11263-009-0275-4
45 Shotton J, Winn J, Rother C, Criminisi A. Textonboost for image understanding: multi-class object recognition and segmentation by jointly modeling texture, layout, and context. International Journal of Computer Vision , 2009, 81(1): 2-23
doi: 10.1007/s11263-007-0109-1
[1] Rizwan Ahmed KHAN, Alexandre MEYER, Hubert KONIK, Saida BOUAKAZ. Saliency-based framework for facial expression recognition[J]. Front. Comput. Sci., 2019, 13(1): 183-198.
[2] Jian-Hao LUO,Wang ZHOU,Jianxin WU. Image categorization with resource constraints: introduction, challenges and advances[J]. Front. Comput. Sci., 2017, 11(1): 13-26.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed