Developments in multimedia technologies have paved way for the storage of huge collections of video documents on computer systems. It is essential to design tools for content-based access to the documents, so as to allow an efficient exploitation of these collections. Content based analysis provides a flexible and powerfulway to access video data when compared with the other traditional video analysis techniques. The area of content based video indexing and retrieval (CBVIR), focusing on automating the indexing, retrieval and management of video, has attracted extensive research in the last decade. CBVIR is a lively area of research with enduring acknowledgments from several domains. Herein a vital assessment of contemporary researches associated with the content-based indexing and retrieval of visual information. In this paper, we present an extensive review of significant researches on CBVIR. Concise description of content based video analysis along with the techniques associated with the content based video indexing and retrieval is presented.
. A comprehensive review of significant researches on content based indexing and retrieval of visual information[J]. Frontiers of Computer Science, 2013, 7(5): 782-799.
R PRIYA, T. N SHANMUGAM. A comprehensive review of significant researches on content based indexing and retrieval of visual information. Front Comput Sci, 2013, 7(5): 782-799.
Hanis A, Sziranyi T. Measuring the motion similarity in video indexing. In: Proceedings of the 4th EURASIP Conference Focused on Video/Image Processing andMultimedia Communications . 2003, 507-512
2
Calic J, Izuierdo E. Efficient key-frame extraction and video analysis. In: Proceedings of the 2002 International Conference on Information Technology: Coding and Computing . 2002, 28-33
3
Carbonaro A. Ontology-based video retrieval in a semantic-based learning environment. Journal of e-Learning and Knowledge Society , 2009, 4(3): 203-212
4
George A, Rajakumar B, Suresh B. Markov random field based image restoration with aid of local and global features. International Journal of Computer Applications , 2012, 48(8): 23-28
5
Kundra E H, Verma E M, Aashima E. Filter for removal of impulse noise by using fuzzy logic. International Journal of Image Processing (IJIP) , 2011, 3(5): 195-202
6
Umamakeswari A, Rajaraman A. Object based video analysis, interpretation and tracking. Journal of Computer Science , 2007, 3(10): 818-822 doi: 10.3844/jcssp.2007.818.822
7
Amer A. Object-based video retrieval based on motion analysis and description. Technical Report, University du Québec , 1999
8
Javed O, Shah M, Comaniciu D. A probabilistic framework for object recognition in video. In: Proceedings of the 2004 International Conference on Image Processing . 2004, 2713-2716
9
Radhakrishnan R, Divakaran A, Xiong Z, Otsuka I. A content-adaptive analysis and representation framework for audio event discovery from unscripted multimedia. EURASIP Journal on Applied Signal Processing , 2006: 1-24 doi: 10.1155/ASP/2006/89013
10
Schnettler B, Raab J. Interpretative visual analysis developments: state of the art and pending problems. Historical Social Research/Historische Sozialforschung , 2009, 265-295
11
Ramoser H, Schlogl T, Beleznai C, Winter M, Bischof H. Shape-based detection of humans for video surveillance applications. In: Proceedings of the 2003 IEEE International Conference on Image Processing . 2003, 3: 1013-1016
12
Ahmad A M, Lee S Y. Fast and robust object-extraction framework for object-based streaming system. International Journal of Virtual Technology and Multimedia , 2008, 1(1): 39-60 doi: 10.1504/IJVTM.2008.017109
13
Ngo C W, Pong T C, Zhang H J. Motion-based video representation for scene change detection. International Journal of Computer Vision , 2002, 50(2): 127-142 doi: 10.1023/A:1020341931699
14
Zelnik-Manor L, Irani M. Event-based analysis of video. In: Proceedings of the 2001 IEEE Conference on Computer Vision and Pattern Recognition . 2001, II-123-II-130
15
Ding Y, Fan G Camera view-based american football video analysis . In: Proceedings of the 8th IEEE International Symposium on Multimedia . 2006, 317-322 doi: 10.1109/ISM.2006.42
16
Mohan C K, Dhananjaya N, Yegnanarayana B. Video shot segmentation using late fusion technique. In: Proceedings of the 7th International Conference on Machine Learning and Applications . 2008, 267-270
17
Affendey L S, Mamat A, Ibrahim H, Ahmad F. Video data modeling to support hybrid query. International Journal of Computer Science and Network Security , 2007, 7(9): 53-61
18
Aytar Y, Shah M, Luo J. Utilizing semantic word similarity measures for video retrieval. In: Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition . 2008, 1-8 doi: 10.1109/CVPR.2008.4587822
19
Dimitrova N, Zhang H J, Shahraray B, Sezan I, Huang T, Zakhor A. Applications of video-content analysis and retrieval. IEEE MultiMedia , 2002, 9(3): 42-55 doi: 10.1109/MMUL.2002.1022858
20
Bergman L D, Castelli V, Li C. Progressive content-based retrieval from satellite image archives. Technical Report, D-LibMagazine , 1997 doi: 10.1045/october97-li
21
Chang S F, Smith J R, Meng H J, Wang H, Zhong D. Finding images/video in large archives. Technical Report, D-Lib Magazine , 1997
22
Gupta A, Jain R. Visual information retrieval. Communications of the ACM , 1997, 40(5): 70-79 doi: 10.1145/253769.253798
23
Papadias D, Mantzourogiannis M, Kalnis P, Mamoulis N, Ahmad I. Content-based retrieval using heuristic search. In: Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval . 1999, 168-175
24
Koprinska I, Carrato S. Temporal video segmentation: a survey. Signal Processing: Image Communication , 2001, 16(5): 477-500 doi: 10.1016/S0923-5965(00)00011-4
25
Hanjalic A. Shot-boundary detection: unraveled and resolved? IEEE Transactions on Circuits and Systems for Video Technology , 2002, 12(2): 90-105 doi: 10.1109/76.988656
26
Girgensohn A, Boreczky J. Time-constrained keyframe selection technique. In: Proceedings of the 1999 IEEE International Conference on Multimedia Computing and Systems . 1999, 756-761 doi: 10.1109/MMCS.1999.779294
27
Liu T, Kender J ROptimization algorithms for the selection of key frame sequences of variable length. In: Proceedings of the 2002 European Conference on Computer Vision . 2002, 403-417 doi: 10.1007/3-540-47979-1_27
28
Aslandogan Y A, Yu C T. Techniques and systems for image and video retrieval. IEEE Transactions on Knowledge and Data Engineering , 1999, 11(1): 56-63 doi: 10.1109/69.755615
29
Lu G. Techniques and data structures for efficient multimedia retrieval based on similarity. IEEE Transactions on Multimedia , 2002, 4(3): 372-384 doi: 10.1109/TMM.2002.802831
30
Lelescu D, Schonfeld D. Video skimming and summarization based on principal component analysis. In: Proceedings of the 4th IFIP/IEEE International Conference on Management of Multimedia on the Internet . 2001, 128-141 doi: 10.1007/3-540-45508-6_10
31
Gargi U, Kasturi R, Strayer S H. Performance characterization of video-shot-change detection methods. IEEE Transactions on Circuits and Systems for Video Technology , 2000, 10(1): 1-13 doi: 10.1109/76.825852
32
Huang Y, Liu Q, Metaxas D. Video object segmentation by hypergraph cut. In: Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition . 2009, 1738-1745 doi: 10.1109/CVPR.2009.5206795
33
Aydm Alatan A, Tuncel E, Onural L. A rule-based method for object segmentation in video sequences. In: Proceedings of the 1997 International Conference on Image Processing . 1997, 522-525
34
Otoom A F, Gunes H, Piccardi M. Feature extraction techniques for abandoned object classification in video surveillance. In: Proceedings of the 15th IEEE International Conference on Image Processing . 2008, 1368-1371
35
Van Cauwelaert D. Generic models for adaptive robust feature extraction in video. In: Proceedings of the 9th FirW PhD Symposium . 2008, 148-149
36
Zhong D, Chang S F. An integrated approach for content-based video object segmentation and retrieval. IEEE Transactions on Circuits and Systems for Video Technology , 1999, 9(8): 1259-1268 doi: 10.1109/76.809160
37
Avrithis Y S, Doulamis A D, Doulamis N D, Kollias S D. An adaptive approach to video indexing and retrieval using fuzzy classification. In: Proceedings of VLBV . 1998
38
Idris F, Panchanathan S. Review of image and video indexing techniques. Journal of Visual Communication and Image Representation , 1997, 8(2): 146-166 doi: 10.1006/jvci.1997.0355
39
Christel M G, Smith M A, Taylor C R, Winkler D BEvolving video skims into useful multimedia abstractions. In: Proceedings of the 1998 SIGCHI Conference on Human factors in Computing Systems . 1998, 171-178
40
Marchand-Maillet S. Content-based video retrieval: an overview. 2000
41
Jawahar C, Chennupati B, Paluri B, Jammalamadaka N. Video Retrieval Based on Textual Queries. Technical Report , 2005
42
Zhang X P, Chen Z. An automated video object extraction system based on spatiotemporal independent component analysis and multiscale segmentation. EURASIP Journal on Applied Signal Processing , 2006, 2006: 184
43
Chang Y, Lee D J, Hong Y, Archibald J. Unsupervised video shot detection using clustering ensemble with a color global scale-invariant feature transform descriptor. Journal on Image and Video Processing , 2008, 9
44
Kolekar M H, Palaniappan K, Sengupta S, Seetharaman G. Semantic concept mining based on hierarchical event detection for soccer video indexing. Journal of Multimedia , 2009, 4(5): 298-312 doi: 10.4304/jmm.4.5.298-312
45
Jiang R, Crookes D. Approach to automatic video motion segmentation. Electronics Letters , 2007, 43(18): 968-970 doi: 10.1049/el:20070663
46
Basharat A, Zhai Y, Shah M. Content based video matching using spatiotemporal volumes. Computer Vision and Image Understanding , 2008, 110(3): 360-377 doi: 10.1016/j.cviu.2007.09.016
47
Kuo T C, Chen A L. A maskmatching approach for video segmentation on compressed data. Information Sciences , 2002, 141(1): 169-191 doi: 10.1016/S0020-0255(01)00196-7
48
Chen D Y, Hsiao MH, Lee S Y. Automatic closed caption detection and filtering in mpeg videos for video structuring. Journal of Information Science and Engineering , 2006, 22(5): 1145-1162
49
Duan L Y, Xu M, Tian Q, Xu C S, Jin J S. A unified framework for semantic shot classification in sports video. IEEE Transactions on Multimedia , 2005, 7(6): 1066-1083 doi: 10.1109/TMM.2005.858395
50
Liu S, Xu M,Yi H, Chia L T, Rajan D. Multimodal semantic analysis and annotation for basketball video. EURASIP Journal on Applied Signal Processing , 2006: 182
51
Han B, Gao X, Ji H. A shot boundary detection method for news video based on rough-fuzzy sets. International Journal of Information Technology , 2005, 11(7): 101-111
52
Gao X, Tang X. Unsupervised video-shot segmentation and model-free anchorperson detection for news video story parsing. IEEE Transactions on Circuits and Systems for Video Technology , 2002, 12(9): 765-776 doi: 10.1109/TCSVT.2002.800510
53
Zhai Y, Shah M Video scene segmentation using markov chain monte carlo . IEEE Transactions on Multimedia , 2006, 8(4): 686-697 doi: 10.1109/TMM.2006.876299
54
Fan J, Aref W G, Elmagarmid A K, Hacid M S, Marzouk M S, Zhu X. Multiview: multilevel video content representation and retrieval. Journal of Electronic Imaging , 2001, 10(4): 895-908 doi: 10.1117/1.1406944
55
D?nderler M E, Ulusoy ?, Güdükbay U. Rule-based spatiotemporal query processing for video databases. The VLDB Journal , 2004, 13(1): 86-103 doi: 10.1007/s00778-003-0114-0
56
Erozel G, Cicekli N K, Cicekli I. Natural language querying for video databases. Information Sciences , 2008, 178(12): 2534-2552 doi: 10.1016/j.ins.2008.02.001
57
Smeaton A F, Wilkins P, Worring M, De Rooij O, Chua T S, Luan H. Content-based video retrieval: three example systems from trecvid. International Journal of Imaging Systems and Technology , 2008, 18(2-3): 195-201 doi: 10.1002/ima.20150
58
Schonfeld D, Lelescu D. Vortex: video retrieval and tracking from compressed multimedia databases—multiple object tracking from mpeg-2 bit stream. Journal of Visual Communication and Image Representation , 2000, 11(2): 154-182 doi: 10.1006/jvci.1999.0432
59
Vrochidis S, Doulaverakis C, Gounaris A, Nidelkou E, Makris L, Kompatsiaris I. A hybrid ontology and visual-based retrieval model for cultural heritage multimedia collections. International Journal of Metadata, Semantics and Ontologies (IJMSO) , 2008, 3(3): 167-182 doi: 10.1504/IJMSO.2008.023566
60
Wen C Y, Chang L F, Li H H. Content based video retrieval with motion vectors and the rgb color model. Forensic Science Journal , 2007, 6(2): 1-36
61
Lili N, Noah S, Khalid F. Extracting and integrating multimodality features via multidimensional approach for video retrieval. International Journal of Computer Science and Network Security , 2009, 9(2): 252
62
Hoi S C, Lyu M R. A multimodal and multilevel ranking scheme for large-scale video retrieval. IEEE Transactions on Multimedia , 2008, 10(4): 607-619 doi: 10.1109/TMM.2008.921735
63
Tjondronegoro D, Chen Y P PContent-based indexing and retrieval using mpeg-7 and x-query in video data management systems. World Wide Web , 2002, 5(3): 207-227 doi: 10.1023/A:1020988713878
64
Ma Y F, Zhang H J. Motion pattern-based video classification and retrieval. EURASIP Journal on Applied Signal Processing , 2003: 199-208 doi: 10.1155/S1110865703211021
65
DeMenthon D, Doermann D. Video retrieval of near-duplicates using κ-nearest neighbor retrieval of spatio-temporal descriptors. Multimedia Tools and Applications , 2006, 30(3): 229-253 doi: 10.1007/s11042-006-0029-z
66
Yang J, Li Q, Wenyin L, Zhuang Y. Searching for flash movies on the web: a content and context based framework. World Wide Web , 2005, 8(4): 495-517 doi: 10.1007/s11280-005-0905-x
67
Dagtas S, Al-Khatib W, Ghafoor A, Kashyap R L. Models for motionbased video indexing and retrieval. IEEE Transactions on Image Processing , 2000, 9(1): 88-101 doi: 10.1109/83.817601
68
Lu H, Ooi B C, Shen H T, Xue X. Hierarchical indexing structure for efficient similarity search in video retrieval. IEEE Transactions on Knowledge and Data Engineering , 2006, 18(11): 1544-1559 doi: 10.1109/TKDE.2006.174
69
Zhang D, Nunamaker J FA natural language approach to contentbased video indexing and retrieval for interactive e-learning. IEEE Transactions on Multimedia , 2004, 6(3): 450-458 doi: 10.1109/TMM.2004.827505
70
Amir A, Srinivasan S, Efrat A. Search the audio, browse the video—a generic paradigm for video collections. EURASIP Journal on Advances in Signal Processing , 1900, 2003(2): 209-222 doi: 10.1155/S111086570321012X
71
Chiu C Y, Chao S P, Wu M Y, Yang S N, Lin H C. Content-based retrieval for human motion data. Journal of Visual Communication and Image Representation , 2004, 15(3): 446-466 doi: 10.1016/j.jvcir.2004.04.004
72
Munesawang P, Guan L. Adaptive video indexing and automatic/semiautomatic relevance feedback. IEEE Transactions on Circuits and Systems for Video Technology , 2005, 15(8): 1032-1046 doi: 10.1109/TCSVT.2005.852412
73
Fablet R, Bouthemy P, Pérez P. Nonparametric motion characterization using causal probabilistic models for video indexing and retrieval. IEEE Transactions on Image Processing , 2002, 11(4): 393-407 doi: 10.1109/TIP.2002.999674
74
Yi H, Rajan D, Chia L T. A new motion histogram to index motion content in video segments. Pattern Recognition Letters , 2005, 26(9): 1221-1231 doi: 10.1016/j.patrec.2004.11.011
75
Babu R V, Ramakrishnan K. Compressed domain video retrieval using object and global motion descriptors. Multimedia Tools and Applications , 2007, 32(1): 93-113 doi: 10.1007/s11042-006-0048-9
76
Snoek C G, Worring M, Koelma D C, Smeulders A W. A learned lexicon-driven paradigm for interactive video retrieval. IEEE Transactions on Multimedia , 2007, 9(2): 280-292 doi: 10.1109/TMM.2006.886275
77
Abdelali A B, Mtibaa A, Bourennane E, Abid M. Design of hardware accelerators for content based video indexing. Asian Journal of Information Technology , 2006, 5(9): 976-984
78
Doulamis A D, Doulamis N D, Kollias S D. A fuzzy video content representation for video summarization and content-based retrieval. Signal Processing , 2000, 80(6): 1049-1067 doi: 10.1016/S0165-1684(00)00019-0
79
Lee J, Dickinson B WHierarchical video indexing and retrieval for subband-coded video. IEEE Transactions on Circuits and Systems for Video Technology , 2000, 10(5): 824-829 doi: 10.1109/76.856461
80
Fan J, Zhu X, Hacid M S, Elmagarmid A KModel-based video classification toward hierarchical representation, indexing and access. Multimedia Tools and Applications , 2002, 17(1): 97-120 doi: 10.1023/A:1014635823052
81
Albanese M, Chianese A, Moscato V, Sansone L. A formal model for video shot segmentation and its application via animate vision. Multimedia Tools and Applications , 2004, 24(3): 253-272 doi: 10.1023/B:MTAP.0000039421.91449.10
82
Cheung R. Indexing an intelligent video database using evolutionary control. Journal of Digital Information Management , 2003, 1: 8-19
83
Erol B, Kossentini F. Retrieval by local motion. EURASIP Journal on Advances in Signal Processing , 1900, 2003(1): 41-47 doi: 10.1155/S1110865703210076
84
Xu C, Cheng J, Zhang Y, Zhang Y, Lu H. Sports video analysis: semantics extraction, editorial content creation and adaptation. Journal of Multimedia , 2009, 4(2): 69-79 doi: 10.4304/jmm.4.2.69-79
85
Lee S, Yoo C D. Robust video fingerprinting for content-based video identification. IEEE Transactions on Circuits and Systems for Video Technology , 2008, 18(7): 983-988 doi: 10.1109/TCSVT.2008.920739
86
Fan J, Elmagarmid A K, Zhu X, Aref W G, Wu L. Classview: hierarchical video shot classification, indexing, and accessing. IEEE Transactions on Multimedia , 2004, 6(1): 70-86 doi: 10.1109/TMM.2003.819583
87
Fan J, Luo H, Elmagarmid A K. Concept-oriented indexing of video databases: toward semantic sensitive retrieval and browsing. IEEE Transactions on Image Processing , 2004, 13(7): 974-992 doi: 10.1109/TIP.2004.827232
88
Khan A, Sun L, Ifeachor E. Content-based video quality prediction for mpeg4 video streaming over wireless networks. Journal ofMultimedia , 2009, 4(4): 228-239
89
Dumont E, Merialdo B. Rushes video summarization and evaluation. Multimedia Tools and Applications , 2010, 48(1): 51-68 doi: 10.1007/s11042-009-0374-9
90
Thyagharajan K, Ramachandran V. An effective transmission and browsing methodology for streaming video. Journal of Computer Science , 2006, 2(4): 326-332 doi: 10.3844/jcssp.2006.326.332
