Please wait a minute...
Frontiers of Computer Science

ISSN 2095-2228

ISSN 2095-2236(Online)

CN 10-1014/TP

Postal Subscription Code 80-970

2018 Impact Factor: 1.129

Front Comput Sci    2013, Vol. 7 Issue (5) : 782-799    https://doi.org/10.1007/s11704-013-1276-6
REVIEW ARTICLE
A comprehensive review of significant researches on content based indexing and retrieval of visual information
R PRIYA(), T. N SHANMUGAM
Department of Mathematics, Anna University-Chennai, Chennai 600 025, India
 Download: PDF(474 KB)   HTML
 Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks
Abstract

Developments in multimedia technologies have paved way for the storage of huge collections of video documents on computer systems. It is essential to design tools for content-based access to the documents, so as to allow an efficient exploitation of these collections. Content based analysis provides a flexible and powerfulway to access video data when compared with the other traditional video analysis techniques. The area of content based video indexing and retrieval (CBVIR), focusing on automating the indexing, retrieval and management of video, has attracted extensive research in the last decade. CBVIR is a lively area of research with enduring acknowledgments from several domains. Herein a vital assessment of contemporary researches associated with the content-based indexing and retrieval of visual information. In this paper, we present an extensive review of significant researches on CBVIR. Concise description of content based video analysis along with the techniques associated with the content based video indexing and retrieval is presented.

Keywords nultimedia information      content based video retrieval (CBVR)      content based video indexing and retrieval (CBVIR)      shot segmentation      object segmentation      feature extraction      indexing      motion estimation      querying      key frame      retrieval      and indexing     
Corresponding Author(s): PRIYA R,Email:priyarajendranphd@gmail.coma   
Issue Date: 01 October 2013
 Cite this article:   
R PRIYA,T. N SHANMUGAM. A comprehensive review of significant researches on content based indexing and retrieval of visual information[J]. Front Comput Sci, 2013, 7(5): 782-799.
 URL:  
https://academic.hep.com.cn/fcs/EN/10.1007/s11704-013-1276-6
https://academic.hep.com.cn/fcs/EN/Y2013/V7/I5/782
1 Hanis A, Sziranyi T. Measuring the motion similarity in video indexing. In: Proceedings of the 4th EURASIP Conference Focused on Video/Image Processing andMultimedia Communications . 2003, 507-512
2 Calic J, Izuierdo E. Efficient key-frame extraction and video analysis. In: Proceedings of the 2002 International Conference on Information Technology: Coding and Computing . 2002, 28-33
3 Carbonaro A. Ontology-based video retrieval in a semantic-based learning environment. Journal of e-Learning and Knowledge Society , 2009, 4(3): 203-212
4 George A, Rajakumar B, Suresh B. Markov random field based image restoration with aid of local and global features. International Journal of Computer Applications , 2012, 48(8): 23-28
5 Kundra E H, Verma E M, Aashima E. Filter for removal of impulse noise by using fuzzy logic. International Journal of Image Processing (IJIP) , 2011, 3(5): 195-202
6 Umamakeswari A, Rajaraman A. Object based video analysis, interpretation and tracking. Journal of Computer Science , 2007, 3(10): 818-822
doi: 10.3844/jcssp.2007.818.822
7 Amer A. Object-based video retrieval based on motion analysis and description. Technical Report, University du Québec , 1999
8 Javed O, Shah M, Comaniciu D. A probabilistic framework for object recognition in video. In: Proceedings of the 2004 International Conference on Image Processing . 2004, 2713-2716
9 Radhakrishnan R, Divakaran A, Xiong Z, Otsuka I. A content-adaptive analysis and representation framework for audio event discovery from unscripted multimedia. EURASIP Journal on Applied Signal Processing , 2006: 1-24
doi: 10.1155/ASP/2006/89013
10 Schnettler B, Raab J. Interpretative visual analysis developments: state of the art and pending problems. Historical Social Research/Historische Sozialforschung , 2009, 265-295
11 Ramoser H, Schlogl T, Beleznai C, Winter M, Bischof H. Shape-based detection of humans for video surveillance applications. In: Proceedings of the 2003 IEEE International Conference on Image Processing . 2003, 3: 1013-1016
12 Ahmad A M, Lee S Y. Fast and robust object-extraction framework for object-based streaming system. International Journal of Virtual Technology and Multimedia , 2008, 1(1): 39-60
doi: 10.1504/IJVTM.2008.017109
13 Ngo C W, Pong T C, Zhang H J. Motion-based video representation for scene change detection. International Journal of Computer Vision , 2002, 50(2): 127-142
doi: 10.1023/A:1020341931699
14 Zelnik-Manor L, Irani M. Event-based analysis of video. In: Proceedings of the 2001 IEEE Conference on Computer Vision and Pattern Recognition . 2001, II-123-II-130
15 Ding Y, Fan G Camera view-based american football video analysis . In: Proceedings of the 8th IEEE International Symposium on Multimedia . 2006, 317-322
doi: 10.1109/ISM.2006.42
16 Mohan C K, Dhananjaya N, Yegnanarayana B. Video shot segmentation using late fusion technique. In: Proceedings of the 7th International Conference on Machine Learning and Applications . 2008, 267-270
17 Affendey L S, Mamat A, Ibrahim H, Ahmad F. Video data modeling to support hybrid query. International Journal of Computer Science and Network Security , 2007, 7(9): 53-61
18 Aytar Y, Shah M, Luo J. Utilizing semantic word similarity measures for video retrieval. In: Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition . 2008, 1-8
doi: 10.1109/CVPR.2008.4587822
19 Dimitrova N, Zhang H J, Shahraray B, Sezan I, Huang T, Zakhor A. Applications of video-content analysis and retrieval. IEEE MultiMedia , 2002, 9(3): 42-55
doi: 10.1109/MMUL.2002.1022858
20 Bergman L D, Castelli V, Li C. Progressive content-based retrieval from satellite image archives. Technical Report, D-LibMagazine , 1997
doi: 10.1045/october97-li
21 Chang S F, Smith J R, Meng H J, Wang H, Zhong D. Finding images/video in large archives. Technical Report, D-Lib Magazine , 1997
22 Gupta A, Jain R. Visual information retrieval. Communications of the ACM , 1997, 40(5): 70-79
doi: 10.1145/253769.253798
23 Papadias D, Mantzourogiannis M, Kalnis P, Mamoulis N, Ahmad I. Content-based retrieval using heuristic search. In: Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval . 1999, 168-175
24 Koprinska I, Carrato S. Temporal video segmentation: a survey. Signal Processing: Image Communication , 2001, 16(5): 477-500
doi: 10.1016/S0923-5965(00)00011-4
25 Hanjalic A. Shot-boundary detection: unraveled and resolved? IEEE Transactions on Circuits and Systems for Video Technology , 2002, 12(2): 90-105
doi: 10.1109/76.988656
26 Girgensohn A, Boreczky J. Time-constrained keyframe selection technique. In: Proceedings of the 1999 IEEE International Conference on Multimedia Computing and Systems . 1999, 756-761
doi: 10.1109/MMCS.1999.779294
27 Liu T, Kender J ROptimization algorithms for the selection of key frame sequences of variable length. In: Proceedings of the 2002 European Conference on Computer Vision . 2002, 403-417
doi: 10.1007/3-540-47979-1_27
28 Aslandogan Y A, Yu C T. Techniques and systems for image and video retrieval. IEEE Transactions on Knowledge and Data Engineering , 1999, 11(1): 56-63
doi: 10.1109/69.755615
29 Lu G. Techniques and data structures for efficient multimedia retrieval based on similarity. IEEE Transactions on Multimedia , 2002, 4(3): 372-384
doi: 10.1109/TMM.2002.802831
30 Lelescu D, Schonfeld D. Video skimming and summarization based on principal component analysis. In: Proceedings of the 4th IFIP/IEEE International Conference on Management of Multimedia on the Internet . 2001, 128-141
doi: 10.1007/3-540-45508-6_10
31 Gargi U, Kasturi R, Strayer S H. Performance characterization of video-shot-change detection methods. IEEE Transactions on Circuits and Systems for Video Technology , 2000, 10(1): 1-13
doi: 10.1109/76.825852
32 Huang Y, Liu Q, Metaxas D. Video object segmentation by hypergraph cut. In: Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition . 2009, 1738-1745
doi: 10.1109/CVPR.2009.5206795
33 Aydm Alatan A, Tuncel E, Onural L. A rule-based method for object segmentation in video sequences. In: Proceedings of the 1997 International Conference on Image Processing . 1997, 522-525
34 Otoom A F, Gunes H, Piccardi M. Feature extraction techniques for abandoned object classification in video surveillance. In: Proceedings of the 15th IEEE International Conference on Image Processing . 2008, 1368-1371
35 Van Cauwelaert D. Generic models for adaptive robust feature extraction in video. In: Proceedings of the 9th FirW PhD Symposium . 2008, 148-149
36 Zhong D, Chang S F. An integrated approach for content-based video object segmentation and retrieval. IEEE Transactions on Circuits and Systems for Video Technology , 1999, 9(8): 1259-1268
doi: 10.1109/76.809160
37 Avrithis Y S, Doulamis A D, Doulamis N D, Kollias S D. An adaptive approach to video indexing and retrieval using fuzzy classification. In: Proceedings of VLBV . 1998
38 Idris F, Panchanathan S. Review of image and video indexing techniques. Journal of Visual Communication and Image Representation , 1997, 8(2): 146-166
doi: 10.1006/jvci.1997.0355
39 Christel M G, Smith M A, Taylor C R, Winkler D BEvolving video skims into useful multimedia abstractions. In: Proceedings of the 1998 SIGCHI Conference on Human factors in Computing Systems . 1998, 171-178
40 Marchand-Maillet S. Content-based video retrieval: an overview. 2000
41 Jawahar C, Chennupati B, Paluri B, Jammalamadaka N. Video Retrieval Based on Textual Queries. Technical Report , 2005
42 Zhang X P, Chen Z. An automated video object extraction system based on spatiotemporal independent component analysis and multiscale segmentation. EURASIP Journal on Applied Signal Processing , 2006, 2006: 184
43 Chang Y, Lee D J, Hong Y, Archibald J. Unsupervised video shot detection using clustering ensemble with a color global scale-invariant feature transform descriptor. Journal on Image and Video Processing , 2008, 9
44 Kolekar M H, Palaniappan K, Sengupta S, Seetharaman G. Semantic concept mining based on hierarchical event detection for soccer video indexing. Journal of Multimedia , 2009, 4(5): 298-312
doi: 10.4304/jmm.4.5.298-312
45 Jiang R, Crookes D. Approach to automatic video motion segmentation. Electronics Letters , 2007, 43(18): 968-970
doi: 10.1049/el:20070663
46 Basharat A, Zhai Y, Shah M. Content based video matching using spatiotemporal volumes. Computer Vision and Image Understanding , 2008, 110(3): 360-377
doi: 10.1016/j.cviu.2007.09.016
47 Kuo T C, Chen A L. A maskmatching approach for video segmentation on compressed data. Information Sciences , 2002, 141(1): 169-191
doi: 10.1016/S0020-0255(01)00196-7
48 Chen D Y, Hsiao MH, Lee S Y. Automatic closed caption detection and filtering in mpeg videos for video structuring. Journal of Information Science and Engineering , 2006, 22(5): 1145-1162
49 Duan L Y, Xu M, Tian Q, Xu C S, Jin J S. A unified framework for semantic shot classification in sports video. IEEE Transactions on Multimedia , 2005, 7(6): 1066-1083
doi: 10.1109/TMM.2005.858395
50 Liu S, Xu M,Yi H, Chia L T, Rajan D. Multimodal semantic analysis and annotation for basketball video. EURASIP Journal on Applied Signal Processing , 2006: 182
51 Han B, Gao X, Ji H. A shot boundary detection method for news video based on rough-fuzzy sets. International Journal of Information Technology , 2005, 11(7): 101-111
52 Gao X, Tang X. Unsupervised video-shot segmentation and model-free anchorperson detection for news video story parsing. IEEE Transactions on Circuits and Systems for Video Technology , 2002, 12(9): 765-776
doi: 10.1109/TCSVT.2002.800510
53 Zhai Y, Shah M Video scene segmentation using markov chain monte carlo . IEEE Transactions on Multimedia , 2006, 8(4): 686-697
doi: 10.1109/TMM.2006.876299
54 Fan J, Aref W G, Elmagarmid A K, Hacid M S, Marzouk M S, Zhu X. Multiview: multilevel video content representation and retrieval. Journal of Electronic Imaging , 2001, 10(4): 895-908
doi: 10.1117/1.1406944
55 D?nderler M E, Ulusoy ?, Güdükbay U. Rule-based spatiotemporal query processing for video databases. The VLDB Journal , 2004, 13(1): 86-103
doi: 10.1007/s00778-003-0114-0
56 Erozel G, Cicekli N K, Cicekli I. Natural language querying for video databases. Information Sciences , 2008, 178(12): 2534-2552
doi: 10.1016/j.ins.2008.02.001
57 Smeaton A F, Wilkins P, Worring M, De Rooij O, Chua T S, Luan H. Content-based video retrieval: three example systems from trecvid. International Journal of Imaging Systems and Technology , 2008, 18(2-3): 195-201
doi: 10.1002/ima.20150
58 Schonfeld D, Lelescu D. Vortex: video retrieval and tracking from compressed multimedia databases—multiple object tracking from mpeg-2 bit stream. Journal of Visual Communication and Image Representation , 2000, 11(2): 154-182
doi: 10.1006/jvci.1999.0432
59 Vrochidis S, Doulaverakis C, Gounaris A, Nidelkou E, Makris L, Kompatsiaris I. A hybrid ontology and visual-based retrieval model for cultural heritage multimedia collections. International Journal of Metadata, Semantics and Ontologies (IJMSO) , 2008, 3(3): 167-182
doi: 10.1504/IJMSO.2008.023566
60 Wen C Y, Chang L F, Li H H. Content based video retrieval with motion vectors and the rgb color model. Forensic Science Journal , 2007, 6(2): 1-36
61 Lili N, Noah S, Khalid F. Extracting and integrating multimodality features via multidimensional approach for video retrieval. International Journal of Computer Science and Network Security , 2009, 9(2): 252
62 Hoi S C, Lyu M R. A multimodal and multilevel ranking scheme for large-scale video retrieval. IEEE Transactions on Multimedia , 2008, 10(4): 607-619
doi: 10.1109/TMM.2008.921735
63 Tjondronegoro D, Chen Y P PContent-based indexing and retrieval using mpeg-7 and x-query in video data management systems. World Wide Web , 2002, 5(3): 207-227
doi: 10.1023/A:1020988713878
64 Ma Y F, Zhang H J. Motion pattern-based video classification and retrieval. EURASIP Journal on Applied Signal Processing , 2003: 199-208
doi: 10.1155/S1110865703211021
65 DeMenthon D, Doermann D. Video retrieval of near-duplicates using κ-nearest neighbor retrieval of spatio-temporal descriptors. Multimedia Tools and Applications , 2006, 30(3): 229-253
doi: 10.1007/s11042-006-0029-z
66 Yang J, Li Q, Wenyin L, Zhuang Y. Searching for flash movies on the web: a content and context based framework. World Wide Web , 2005, 8(4): 495-517
doi: 10.1007/s11280-005-0905-x
67 Dagtas S, Al-Khatib W, Ghafoor A, Kashyap R L. Models for motionbased video indexing and retrieval. IEEE Transactions on Image Processing , 2000, 9(1): 88-101
doi: 10.1109/83.817601
68 Lu H, Ooi B C, Shen H T, Xue X. Hierarchical indexing structure for efficient similarity search in video retrieval. IEEE Transactions on Knowledge and Data Engineering , 2006, 18(11): 1544-1559
doi: 10.1109/TKDE.2006.174
69 Zhang D, Nunamaker J FA natural language approach to contentbased video indexing and retrieval for interactive e-learning. IEEE Transactions on Multimedia , 2004, 6(3): 450-458
doi: 10.1109/TMM.2004.827505
70 Amir A, Srinivasan S, Efrat A. Search the audio, browse the video—a generic paradigm for video collections. EURASIP Journal on Advances in Signal Processing , 1900, 2003(2): 209-222
doi: 10.1155/S111086570321012X
71 Chiu C Y, Chao S P, Wu M Y, Yang S N, Lin H C. Content-based retrieval for human motion data. Journal of Visual Communication and Image Representation , 2004, 15(3): 446-466
doi: 10.1016/j.jvcir.2004.04.004
72 Munesawang P, Guan L. Adaptive video indexing and automatic/semiautomatic relevance feedback. IEEE Transactions on Circuits and Systems for Video Technology , 2005, 15(8): 1032-1046
doi: 10.1109/TCSVT.2005.852412
73 Fablet R, Bouthemy P, Pérez P. Nonparametric motion characterization using causal probabilistic models for video indexing and retrieval. IEEE Transactions on Image Processing , 2002, 11(4): 393-407
doi: 10.1109/TIP.2002.999674
74 Yi H, Rajan D, Chia L T. A new motion histogram to index motion content in video segments. Pattern Recognition Letters , 2005, 26(9): 1221-1231
doi: 10.1016/j.patrec.2004.11.011
75 Babu R V, Ramakrishnan K. Compressed domain video retrieval using object and global motion descriptors. Multimedia Tools and Applications , 2007, 32(1): 93-113
doi: 10.1007/s11042-006-0048-9
76 Snoek C G, Worring M, Koelma D C, Smeulders A W. A learned lexicon-driven paradigm for interactive video retrieval. IEEE Transactions on Multimedia , 2007, 9(2): 280-292
doi: 10.1109/TMM.2006.886275
77 Abdelali A B, Mtibaa A, Bourennane E, Abid M. Design of hardware accelerators for content based video indexing. Asian Journal of Information Technology , 2006, 5(9): 976-984
78 Doulamis A D, Doulamis N D, Kollias S D. A fuzzy video content representation for video summarization and content-based retrieval. Signal Processing , 2000, 80(6): 1049-1067
doi: 10.1016/S0165-1684(00)00019-0
79 Lee J, Dickinson B WHierarchical video indexing and retrieval for subband-coded video. IEEE Transactions on Circuits and Systems for Video Technology , 2000, 10(5): 824-829
doi: 10.1109/76.856461
80 Fan J, Zhu X, Hacid M S, Elmagarmid A KModel-based video classification toward hierarchical representation, indexing and access. Multimedia Tools and Applications , 2002, 17(1): 97-120
doi: 10.1023/A:1014635823052
81 Albanese M, Chianese A, Moscato V, Sansone L. A formal model for video shot segmentation and its application via animate vision. Multimedia Tools and Applications , 2004, 24(3): 253-272
doi: 10.1023/B:MTAP.0000039421.91449.10
82 Cheung R. Indexing an intelligent video database using evolutionary control. Journal of Digital Information Management , 2003, 1: 8-19
83 Erol B, Kossentini F. Retrieval by local motion. EURASIP Journal on Advances in Signal Processing , 1900, 2003(1): 41-47
doi: 10.1155/S1110865703210076
84 Xu C, Cheng J, Zhang Y, Zhang Y, Lu H. Sports video analysis: semantics extraction, editorial content creation and adaptation. Journal of Multimedia , 2009, 4(2): 69-79
doi: 10.4304/jmm.4.2.69-79
85 Lee S, Yoo C D. Robust video fingerprinting for content-based video identification. IEEE Transactions on Circuits and Systems for Video Technology , 2008, 18(7): 983-988
doi: 10.1109/TCSVT.2008.920739
86 Fan J, Elmagarmid A K, Zhu X, Aref W G, Wu L. Classview: hierarchical video shot classification, indexing, and accessing. IEEE Transactions on Multimedia , 2004, 6(1): 70-86
doi: 10.1109/TMM.2003.819583
87 Fan J, Luo H, Elmagarmid A K. Concept-oriented indexing of video databases: toward semantic sensitive retrieval and browsing. IEEE Transactions on Image Processing , 2004, 13(7): 974-992
doi: 10.1109/TIP.2004.827232
88 Khan A, Sun L, Ifeachor E. Content-based video quality prediction for mpeg4 video streaming over wireless networks. Journal ofMultimedia , 2009, 4(4): 228-239
89 Dumont E, Merialdo B. Rushes video summarization and evaluation. Multimedia Tools and Applications , 2010, 48(1): 51-68
doi: 10.1007/s11042-009-0374-9
90 Thyagharajan K, Ramachandran V. An effective transmission and browsing methodology for streaming video. Journal of Computer Science , 2006, 2(4): 326-332
doi: 10.3844/jcssp.2006.326.332
[1] Xia-an BI, Yiming XIE, Hao WU, Luyun XU. Identification of differential brain regions in MCI progression via clustering-evolutionary weighted SVM ensemble algorithm[J]. Front. Comput. Sci., 2021, 15(6): 156903-.
[2] Zhumin CHEN, Xueqi CHENG, Shoubin DONG, Zhicheng DOU, Jiafeng GUO, Xuanjing HUANG, Yanyan LAN, Chenliang LI, Ru LI, Tie-Yan LIU, Yiqun LIU, Jun MA, Bing QIN, Mingwen WANG, Jirong WEN, Jun XU, Min ZHANG, Peng ZHANG, Qi ZHANG. Information retrieval: a view from the Chinese IR community[J]. Front. Comput. Sci., 2021, 15(1): 151601-.
[3] Ling SHEN, Richang HONG, Yanbin HAO. Advance on large scale near-duplicate video retrieval[J]. Front. Comput. Sci., 2020, 14(5): 145702-.
[4] Fei WANG, Tieyun QIAN, Bin LIU, Zhiyong PENG. Patent expanded retrieval via word embedding under composite-domain perspectives[J]. Front. Comput. Sci., 2019, 13(5): 1048-1061.
[5] Ying LI, Xiangwei KONG, Haiyan FU, Qi TIAN. Contextual modeling on auxiliary points for robust image reranking[J]. Front. Comput. Sci., 2019, 13(5): 1010-1022.
[6] Kun SU, Gongping YANG, Lu YANG, Peng SU, Yilong YIN. Non-negative locality-constrained vocabulary tree for finger vein image retrieval[J]. Front. Comput. Sci., 2019, 13(2): 318-332.
[7] Yaru XIAN, Jun XIAO, Ying WANG. A fast registration algorithm of rock point cloud based on spherical projection and feature extraction[J]. Front. Comput. Sci., 2019, 13(1): 170-182.
[8] Xiaohui TAN, Yachun FAN, Ruiliang GUO. Local features and manifold ranking coupled method for sketch-based 3D model retrieval[J]. Front. Comput. Sci., 2018, 12(5): 1000-1012.
[9] Zhixing LI, Gang YIN, Tao WANG, Yang ZHANG, Yue YU, Huaimin WANG. Correlation-based software search by leveraging software term database[J]. Front. Comput. Sci., 2018, 12(5): 923-938.
[10] Jingjing WEI, Xiangwen LIAO, Houdong ZHENG, Guolong CHEN, Xueqi CHENG. Learning from context: a mutual reinforcement model for Chinese microblog opinion retrieval[J]. Front. Comput. Sci., 2018, 12(4): 714-724.
[11] Xueming WANG, Zechao LI, Jinhui TANG. Visual understanding by mining social media: recent advances and challenges[J]. Front. Comput. Sci., 2018, 12(3): 406-422.
[12] Xiaoye MIAO, Yunjun GAO, Su GUO, Wanqi LIU. Incomplete data management: a survey[J]. Front. Comput. Sci., 2018, 12(1): 4-25.
[13] Ilyes KHENNAK, Habiba DRIAS. Strength Pareto fitness assignment for pseudo-relevance feedback: application to MEDLINE[J]. Front. Comput. Sci., 2018, 12(1): 163-176.
[14] Le DONG, Wenpu DONG, Ning FENG, Mengdie MAO, Long CHEN, Gaipeng KONG. Color space quantization-based clustering for image retrieval[J]. Front. Comput. Sci., 2017, 11(6): 1023-1035.
[15] Wei SHAO,Yi DING,Hong-Bin SHEN,Daoqiang ZHANG. Deep model-based feature extraction for predicting protein subcellular localizations from bio-images[J]. Front. Comput. Sci., 2017, 11(2): 243-252.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed