A comprehensive review of significant researches on content based indexing and retrieval of visual information

doi:10.1007/s11704-013-1276-6

Front Comput Sci

2013, Vol. 7

Issue (5) : 782-799 https://doi.org/10.1007/s11704-013-1276-6

REVIEW ARTICLE

A comprehensive review of significant researches on content based indexing and retrieval of visual information

R PRIYA(

), T. N SHANMUGAM

Department of Mathematics, Anna University-Chennai, Chennai 600 025, India

Download: PDF(474 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

Developments in multimedia technologies have paved way for the storage of huge collections of video documents on computer systems. It is essential to design tools for content-based access to the documents, so as to allow an efficient exploitation of these collections. Content based analysis provides a flexible and powerfulway to access video data when compared with the other traditional video analysis techniques. The area of content based video indexing and retrieval (CBVIR), focusing on automating the indexing, retrieval and management of video, has attracted extensive research in the last decade. CBVIR is a lively area of research with enduring acknowledgments from several domains. Herein a vital assessment of contemporary researches associated with the content-based indexing and retrieval of visual information. In this paper, we present an extensive review of significant researches on CBVIR. Concise description of content based video analysis along with the techniques associated with the content based video indexing and retrieval is presented.

Keywords nultimedia information content based video retrieval (CBVR) content based video indexing and retrieval (CBVIR) shot segmentation object segmentation feature extraction indexing motion estimation querying key frame retrieval and indexing

Corresponding Author(s): PRIYA R,Email:priyarajendranphd@gmail.coma

Issue Date: 01 October 2013

Cite this article:

R PRIYA,T. N SHANMUGAM. A comprehensive review of significant researches on content based indexing and retrieval of visual information[J]. Front Comput Sci, 2013, 7(5): 782-799.

URL:

https://academic.hep.com.cn/fcs/EN/10.1007/s11704-013-1276-6
https://academic.hep.com.cn/fcs/EN/Y2013/V7/I5/782

1	Hanis A, Sziranyi T. Measuring the motion similarity in video indexing. In: Proceedings of the 4th EURASIP Conference Focused on Video/Image Processing andMultimedia Communications . 2003, 507-512
2	Calic J, Izuierdo E. Efficient key-frame extraction and video analysis. In: Proceedings of the 2002 International Conference on Information Technology: Coding and Computing . 2002, 28-33
3	Carbonaro A. Ontology-based video retrieval in a semantic-based learning environment. Journal of e-Learning and Knowledge Society , 2009, 4(3): 203-212
4	George A, Rajakumar B, Suresh B. Markov random field based image restoration with aid of local and global features. International Journal of Computer Applications , 2012, 48(8): 23-28
5	Kundra E H, Verma E M, Aashima E. Filter for removal of impulse noise by using fuzzy logic. International Journal of Image Processing (IJIP) , 2011, 3(5): 195-202
6	Umamakeswari A, Rajaraman A. Object based video analysis, interpretation and tracking. Journal of Computer Science , 2007, 3(10): 818-822 doi: 10.3844/jcssp.2007.818.822
7	Amer A. Object-based video retrieval based on motion analysis and description. Technical Report, University du Québec , 1999
8	Javed O, Shah M, Comaniciu D. A probabilistic framework for object recognition in video. In: Proceedings of the 2004 International Conference on Image Processing . 2004, 2713-2716
9	Radhakrishnan R, Divakaran A, Xiong Z, Otsuka I. A content-adaptive analysis and representation framework for audio event discovery from unscripted multimedia. EURASIP Journal on Applied Signal Processing , 2006: 1-24 doi: 10.1155/ASP/2006/89013
10	Schnettler B, Raab J. Interpretative visual analysis developments: state of the art and pending problems. Historical Social Research/Historische Sozialforschung , 2009, 265-295
11	Ramoser H, Schlogl T, Beleznai C, Winter M, Bischof H. Shape-based detection of humans for video surveillance applications. In: Proceedings of the 2003 IEEE International Conference on Image Processing . 2003, 3: 1013-1016
12	Ahmad A M, Lee S Y. Fast and robust object-extraction framework for object-based streaming system. International Journal of Virtual Technology and Multimedia , 2008, 1(1): 39-60 doi: 10.1504/IJVTM.2008.017109
13	Ngo C W, Pong T C, Zhang H J. Motion-based video representation for scene change detection. International Journal of Computer Vision , 2002, 50(2): 127-142 doi: 10.1023/A:1020341931699
14	Zelnik-Manor L, Irani M. Event-based analysis of video. In: Proceedings of the 2001 IEEE Conference on Computer Vision and Pattern Recognition . 2001, II-123-II-130
15	Ding Y, Fan G Camera view-based american football video analysis . In: Proceedings of the 8th IEEE International Symposium on Multimedia . 2006, 317-322 doi: 10.1109/ISM.2006.42
16	Mohan C K, Dhananjaya N, Yegnanarayana B. Video shot segmentation using late fusion technique. In: Proceedings of the 7th International Conference on Machine Learning and Applications . 2008, 267-270
17	Affendey L S, Mamat A, Ibrahim H, Ahmad F. Video data modeling to support hybrid query. International Journal of Computer Science and Network Security , 2007, 7(9): 53-61
18	Aytar Y, Shah M, Luo J. Utilizing semantic word similarity measures for video retrieval. In: Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition . 2008, 1-8 doi: 10.1109/CVPR.2008.4587822
19	Dimitrova N, Zhang H J, Shahraray B, Sezan I, Huang T, Zakhor A. Applications of video-content analysis and retrieval. IEEE MultiMedia , 2002, 9(3): 42-55 doi: 10.1109/MMUL.2002.1022858
20	Bergman L D, Castelli V, Li C. Progressive content-based retrieval from satellite image archives. Technical Report, D-LibMagazine , 1997 doi: 10.1045/october97-li
21	Chang S F, Smith J R, Meng H J, Wang H, Zhong D. Finding images/video in large archives. Technical Report, D-Lib Magazine , 1997
22	Gupta A, Jain R. Visual information retrieval. Communications of the ACM , 1997, 40(5): 70-79 doi: 10.1145/253769.253798
23	Papadias D, Mantzourogiannis M, Kalnis P, Mamoulis N, Ahmad I. Content-based retrieval using heuristic search. In: Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval . 1999, 168-175
24	Koprinska I, Carrato S. Temporal video segmentation: a survey. Signal Processing: Image Communication , 2001, 16(5): 477-500 doi: 10.1016/S0923-5965(00)00011-4
25	Hanjalic A. Shot-boundary detection: unraveled and resolved? IEEE Transactions on Circuits and Systems for Video Technology , 2002, 12(2): 90-105 doi: 10.1109/76.988656
26	Girgensohn A, Boreczky J. Time-constrained keyframe selection technique. In: Proceedings of the 1999 IEEE International Conference on Multimedia Computing and Systems . 1999, 756-761 doi: 10.1109/MMCS.1999.779294
27	Liu T, Kender J ROptimization algorithms for the selection of key frame sequences of variable length. In: Proceedings of the 2002 European Conference on Computer Vision . 2002, 403-417 doi: 10.1007/3-540-47979-1_27
28	Aslandogan Y A, Yu C T. Techniques and systems for image and video retrieval. IEEE Transactions on Knowledge and Data Engineering , 1999, 11(1): 56-63 doi: 10.1109/69.755615
29	Lu G. Techniques and data structures for efficient multimedia retrieval based on similarity. IEEE Transactions on Multimedia , 2002, 4(3): 372-384 doi: 10.1109/TMM.2002.802831
30	Lelescu D, Schonfeld D. Video skimming and summarization based on principal component analysis. In: Proceedings of the 4th IFIP/IEEE International Conference on Management of Multimedia on the Internet . 2001, 128-141 doi: 10.1007/3-540-45508-6_10
31	Gargi U, Kasturi R, Strayer S H. Performance characterization of video-shot-change detection methods. IEEE Transactions on Circuits and Systems for Video Technology , 2000, 10(1): 1-13 doi: 10.1109/76.825852
32	Huang Y, Liu Q, Metaxas D. Video object segmentation by hypergraph cut. In: Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition . 2009, 1738-1745 doi: 10.1109/CVPR.2009.5206795
33	Aydm Alatan A, Tuncel E, Onural L. A rule-based method for object segmentation in video sequences. In: Proceedings of the 1997 International Conference on Image Processing . 1997, 522-525
34	Otoom A F, Gunes H, Piccardi M. Feature extraction techniques for abandoned object classification in video surveillance. In: Proceedings of the 15th IEEE International Conference on Image Processing . 2008, 1368-1371
35	Van Cauwelaert D. Generic models for adaptive robust feature extraction in video. In: Proceedings of the 9th FirW PhD Symposium . 2008, 148-149
36	Zhong D, Chang S F. An integrated approach for content-based video object segmentation and retrieval. IEEE Transactions on Circuits and Systems for Video Technology , 1999, 9(8): 1259-1268 doi: 10.1109/76.809160
37	Avrithis Y S, Doulamis A D, Doulamis N D, Kollias S D. An adaptive approach to video indexing and retrieval using fuzzy classification. In: Proceedings of VLBV . 1998
38	Idris F, Panchanathan S. Review of image and video indexing techniques. Journal of Visual Communication and Image Representation , 1997, 8(2): 146-166 doi: 10.1006/jvci.1997.0355
39	Christel M G, Smith M A, Taylor C R, Winkler D BEvolving video skims into useful multimedia abstractions. In: Proceedings of the 1998 SIGCHI Conference on Human factors in Computing Systems . 1998, 171-178
40	Marchand-Maillet S. Content-based video retrieval: an overview. 2000
41	Jawahar C, Chennupati B, Paluri B, Jammalamadaka N. Video Retrieval Based on Textual Queries. Technical Report , 2005
42	Zhang X P, Chen Z. An automated video object extraction system based on spatiotemporal independent component analysis and multiscale segmentation. EURASIP Journal on Applied Signal Processing , 2006, 2006: 184
43	Chang Y, Lee D J, Hong Y, Archibald J. Unsupervised video shot detection using clustering ensemble with a color global scale-invariant feature transform descriptor. Journal on Image and Video Processing , 2008, 9
44	Kolekar M H, Palaniappan K, Sengupta S, Seetharaman G. Semantic concept mining based on hierarchical event detection for soccer video indexing. Journal of Multimedia , 2009, 4(5): 298-312 doi: 10.4304/jmm.4.5.298-312
45	Jiang R, Crookes D. Approach to automatic video motion segmentation. Electronics Letters , 2007, 43(18): 968-970 doi: 10.1049/el:20070663
46	Basharat A, Zhai Y, Shah M. Content based video matching using spatiotemporal volumes. Computer Vision and Image Understanding , 2008, 110(3): 360-377 doi: 10.1016/j.cviu.2007.09.016
47	Kuo T C, Chen A L. A maskmatching approach for video segmentation on compressed data. Information Sciences , 2002, 141(1): 169-191 doi: 10.1016/S0020-0255(01)00196-7
48	Chen D Y, Hsiao MH, Lee S Y. Automatic closed caption detection and filtering in mpeg videos for video structuring. Journal of Information Science and Engineering , 2006, 22(5): 1145-1162
49	Duan L Y, Xu M, Tian Q, Xu C S, Jin J S. A unified framework for semantic shot classification in sports video. IEEE Transactions on Multimedia , 2005, 7(6): 1066-1083 doi: 10.1109/TMM.2005.858395
50	Liu S, Xu M,Yi H, Chia L T, Rajan D. Multimodal semantic analysis and annotation for basketball video. EURASIP Journal on Applied Signal Processing , 2006: 182
51	Han B, Gao X, Ji H. A shot boundary detection method for news video based on rough-fuzzy sets. International Journal of Information Technology , 2005, 11(7): 101-111
52	Gao X, Tang X. Unsupervised video-shot segmentation and model-free anchorperson detection for news video story parsing. IEEE Transactions on Circuits and Systems for Video Technology , 2002, 12(9): 765-776 doi: 10.1109/TCSVT.2002.800510
53	Zhai Y, Shah M Video scene segmentation using markov chain monte carlo . IEEE Transactions on Multimedia , 2006, 8(4): 686-697 doi: 10.1109/TMM.2006.876299
54	Fan J, Aref W G, Elmagarmid A K, Hacid M S, Marzouk M S, Zhu X. Multiview: multilevel video content representation and retrieval. Journal of Electronic Imaging , 2001, 10(4): 895-908 doi: 10.1117/1.1406944
55	D?nderler M E, Ulusoy ?, Güdükbay U. Rule-based spatiotemporal query processing for video databases. The VLDB Journal , 2004, 13(1): 86-103 doi: 10.1007/s00778-003-0114-0
56	Erozel G, Cicekli N K, Cicekli I. Natural language querying for video databases. Information Sciences , 2008, 178(12): 2534-2552 doi: 10.1016/j.ins.2008.02.001
57	Smeaton A F, Wilkins P, Worring M, De Rooij O, Chua T S, Luan H. Content-based video retrieval: three example systems from trecvid. International Journal of Imaging Systems and Technology , 2008, 18(2-3): 195-201 doi: 10.1002/ima.20150
58	Schonfeld D, Lelescu D. Vortex: video retrieval and tracking from compressed multimedia databases—multiple object tracking from mpeg-2 bit stream. Journal of Visual Communication and Image Representation , 2000, 11(2): 154-182 doi: 10.1006/jvci.1999.0432
59	Vrochidis S, Doulaverakis C, Gounaris A, Nidelkou E, Makris L, Kompatsiaris I. A hybrid ontology and visual-based retrieval model for cultural heritage multimedia collections. International Journal of Metadata, Semantics and Ontologies (IJMSO) , 2008, 3(3): 167-182 doi: 10.1504/IJMSO.2008.023566
60	Wen C Y, Chang L F, Li H H. Content based video retrieval with motion vectors and the rgb color model. Forensic Science Journal , 2007, 6(2): 1-36
61	Lili N, Noah S, Khalid F. Extracting and integrating multimodality features via multidimensional approach for video retrieval. International Journal of Computer Science and Network Security , 2009, 9(2): 252
62	Hoi S C, Lyu M R. A multimodal and multilevel ranking scheme for large-scale video retrieval. IEEE Transactions on Multimedia , 2008, 10(4): 607-619 doi: 10.1109/TMM.2008.921735
63	Tjondronegoro D, Chen Y P PContent-based indexing and retrieval using mpeg-7 and x-query in video data management systems. World Wide Web , 2002, 5(3): 207-227 doi: 10.1023/A:1020988713878
64	Ma Y F, Zhang H J. Motion pattern-based video classification and retrieval. EURASIP Journal on Applied Signal Processing , 2003: 199-208 doi: 10.1155/S1110865703211021
65	DeMenthon D, Doermann D. Video retrieval of near-duplicates using κ-nearest neighbor retrieval of spatio-temporal descriptors. Multimedia Tools and Applications , 2006, 30(3): 229-253 doi: 10.1007/s11042-006-0029-z
66	Yang J, Li Q, Wenyin L, Zhuang Y. Searching for flash movies on the web: a content and context based framework. World Wide Web , 2005, 8(4): 495-517 doi: 10.1007/s11280-005-0905-x
67	Dagtas S, Al-Khatib W, Ghafoor A, Kashyap R L. Models for motionbased video indexing and retrieval. IEEE Transactions on Image Processing , 2000, 9(1): 88-101 doi: 10.1109/83.817601
68	Lu H, Ooi B C, Shen H T, Xue X. Hierarchical indexing structure for efficient similarity search in video retrieval. IEEE Transactions on Knowledge and Data Engineering , 2006, 18(11): 1544-1559 doi: 10.1109/TKDE.2006.174
69	Zhang D, Nunamaker J FA natural language approach to contentbased video indexing and retrieval for interactive e-learning. IEEE Transactions on Multimedia , 2004, 6(3): 450-458 doi: 10.1109/TMM.2004.827505
70	Amir A, Srinivasan S, Efrat A. Search the audio, browse the video—a generic paradigm for video collections. EURASIP Journal on Advances in Signal Processing , 1900, 2003(2): 209-222 doi: 10.1155/S111086570321012X
71	Chiu C Y, Chao S P, Wu M Y, Yang S N, Lin H C. Content-based retrieval for human motion data. Journal of Visual Communication and Image Representation , 2004, 15(3): 446-466 doi: 10.1016/j.jvcir.2004.04.004
72	Munesawang P, Guan L. Adaptive video indexing and automatic/semiautomatic relevance feedback. IEEE Transactions on Circuits and Systems for Video Technology , 2005, 15(8): 1032-1046 doi: 10.1109/TCSVT.2005.852412
73	Fablet R, Bouthemy P, Pérez P. Nonparametric motion characterization using causal probabilistic models for video indexing and retrieval. IEEE Transactions on Image Processing , 2002, 11(4): 393-407 doi: 10.1109/TIP.2002.999674
74	Yi H, Rajan D, Chia L T. A new motion histogram to index motion content in video segments. Pattern Recognition Letters , 2005, 26(9): 1221-1231 doi: 10.1016/j.patrec.2004.11.011
75	Babu R V, Ramakrishnan K. Compressed domain video retrieval using object and global motion descriptors. Multimedia Tools and Applications , 2007, 32(1): 93-113 doi: 10.1007/s11042-006-0048-9
76	Snoek C G, Worring M, Koelma D C, Smeulders A W. A learned lexicon-driven paradigm for interactive video retrieval. IEEE Transactions on Multimedia , 2007, 9(2): 280-292 doi: 10.1109/TMM.2006.886275
77	Abdelali A B, Mtibaa A, Bourennane E, Abid M. Design of hardware accelerators for content based video indexing. Asian Journal of Information Technology , 2006, 5(9): 976-984
78	Doulamis A D, Doulamis N D, Kollias S D. A fuzzy video content representation for video summarization and content-based retrieval. Signal Processing , 2000, 80(6): 1049-1067 doi: 10.1016/S0165-1684(00)00019-0
79	Lee J, Dickinson B WHierarchical video indexing and retrieval for subband-coded video. IEEE Transactions on Circuits and Systems for Video Technology , 2000, 10(5): 824-829 doi: 10.1109/76.856461
80	Fan J, Zhu X, Hacid M S, Elmagarmid A KModel-based video classification toward hierarchical representation, indexing and access. Multimedia Tools and Applications , 2002, 17(1): 97-120 doi: 10.1023/A:1014635823052
81	Albanese M, Chianese A, Moscato V, Sansone L. A formal model for video shot segmentation and its application via animate vision. Multimedia Tools and Applications , 2004, 24(3): 253-272 doi: 10.1023/B:MTAP.0000039421.91449.10
82	Cheung R. Indexing an intelligent video database using evolutionary control. Journal of Digital Information Management , 2003, 1: 8-19
83	Erol B, Kossentini F. Retrieval by local motion. EURASIP Journal on Advances in Signal Processing , 1900, 2003(1): 41-47 doi: 10.1155/S1110865703210076
84	Xu C, Cheng J, Zhang Y, Zhang Y, Lu H. Sports video analysis: semantics extraction, editorial content creation and adaptation. Journal of Multimedia , 2009, 4(2): 69-79 doi: 10.4304/jmm.4.2.69-79
85	Lee S, Yoo C D. Robust video fingerprinting for content-based video identification. IEEE Transactions on Circuits and Systems for Video Technology , 2008, 18(7): 983-988 doi: 10.1109/TCSVT.2008.920739
86	Fan J, Elmagarmid A K, Zhu X, Aref W G, Wu L. Classview: hierarchical video shot classification, indexing, and accessing. IEEE Transactions on Multimedia , 2004, 6(1): 70-86 doi: 10.1109/TMM.2003.819583
87	Fan J, Luo H, Elmagarmid A K. Concept-oriented indexing of video databases: toward semantic sensitive retrieval and browsing. IEEE Transactions on Image Processing , 2004, 13(7): 974-992 doi: 10.1109/TIP.2004.827232
88	Khan A, Sun L, Ifeachor E. Content-based video quality prediction for mpeg4 video streaming over wireless networks. Journal ofMultimedia , 2009, 4(4): 228-239
89	Dumont E, Merialdo B. Rushes video summarization and evaluation. Multimedia Tools and Applications , 2010, 48(1): 51-68 doi: 10.1007/s11042-009-0374-9
90	Thyagharajan K, Ramachandran V. An effective transmission and browsing methodology for streaming video. Journal of Computer Science , 2006, 2(4): 326-332 doi: 10.3844/jcssp.2006.326.332

[1]	Xia-an BI, Yiming XIE, Hao WU, Luyun XU. Identification of differential brain regions in MCI progression via clustering-evolutionary weighted SVM ensemble algorithm[J]. Front. Comput. Sci., 2021, 15(6): 156903-.
[2]	Zhumin CHEN, Xueqi CHENG, Shoubin DONG, Zhicheng DOU, Jiafeng GUO, Xuanjing HUANG, Yanyan LAN, Chenliang LI, Ru LI, Tie-Yan LIU, Yiqun LIU, Jun MA, Bing QIN, Mingwen WANG, Jirong WEN, Jun XU, Min ZHANG, Peng ZHANG, Qi ZHANG. Information retrieval: a view from the Chinese IR community[J]. Front. Comput. Sci., 2021, 15(1): 151601-.
[3]	Ling SHEN, Richang HONG, Yanbin HAO. Advance on large scale near-duplicate video retrieval[J]. Front. Comput. Sci., 2020, 14(5): 145702-.
[4]	Fei WANG, Tieyun QIAN, Bin LIU, Zhiyong PENG. Patent expanded retrieval via word embedding under composite-domain perspectives[J]. Front. Comput. Sci., 2019, 13(5): 1048-1061.
[5]	Ying LI, Xiangwei KONG, Haiyan FU, Qi TIAN. Contextual modeling on auxiliary points for robust image reranking[J]. Front. Comput. Sci., 2019, 13(5): 1010-1022.
[6]	Kun SU, Gongping YANG, Lu YANG, Peng SU, Yilong YIN. Non-negative locality-constrained vocabulary tree for finger vein image retrieval[J]. Front. Comput. Sci., 2019, 13(2): 318-332.
[7]	Yaru XIAN, Jun XIAO, Ying WANG. A fast registration algorithm of rock point cloud based on spherical projection and feature extraction[J]. Front. Comput. Sci., 2019, 13(1): 170-182.
[8]	Xiaohui TAN, Yachun FAN, Ruiliang GUO. Local features and manifold ranking coupled method for sketch-based 3D model retrieval[J]. Front. Comput. Sci., 2018, 12(5): 1000-1012.
[9]	Zhixing LI, Gang YIN, Tao WANG, Yang ZHANG, Yue YU, Huaimin WANG. Correlation-based software search by leveraging software term database[J]. Front. Comput. Sci., 2018, 12(5): 923-938.
[10]	Jingjing WEI, Xiangwen LIAO, Houdong ZHENG, Guolong CHEN, Xueqi CHENG. Learning from context: a mutual reinforcement model for Chinese microblog opinion retrieval[J]. Front. Comput. Sci., 2018, 12(4): 714-724.
[11]	Xueming WANG, Zechao LI, Jinhui TANG. Visual understanding by mining social media: recent advances and challenges[J]. Front. Comput. Sci., 2018, 12(3): 406-422.
[12]	Xiaoye MIAO, Yunjun GAO, Su GUO, Wanqi LIU. Incomplete data management: a survey[J]. Front. Comput. Sci., 2018, 12(1): 4-25.
[13]	Ilyes KHENNAK, Habiba DRIAS. Strength Pareto fitness assignment for pseudo-relevance feedback: application to MEDLINE[J]. Front. Comput. Sci., 2018, 12(1): 163-176.
[14]	Le DONG, Wenpu DONG, Ning FENG, Mengdie MAO, Long CHEN, Gaipeng KONG. Color space quantization-based clustering for image retrieval[J]. Front. Comput. Sci., 2017, 11(6): 1023-1035.
[15]	Wei SHAO,Yi DING,Hong-Bin SHEN,Daoqiang ZHANG. Deep model-based feature extraction for predicting protein subcellular localizations from bio-images[J]. Front. Comput. Sci., 2017, 11(2): 243-252.

Viewed

Full text

Abstract

Cited

Shared

Discussed