Hierarchical deep hashing for image retrieval

doi:10.1007/s11704-017-6537-3

Front. Comput. Sci.

2017, Vol. 11

Issue (2) : 253-265 https://doi.org/10.1007/s11704-017-6537-3

RESEARCH ARTICLE

Hierarchical deep hashing for image retrieval

Ge SONG^1,²,Xiaoyang TAN^1,²(

)

¹. College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, China
². Collaborative Innovation Center of Novel Software Technology and Industrialization, Nanjing 211106, China

Download: PDF(1096 KB)
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

We present a new method to generate efficient multi-level hashing codes for image retrieval based on the deep siamese convolutional neural network (DSCNN). Conventional deep hashing methods trade off the capability of capturing highly complex and nonlinear semantic information of images against very compact hash codes, usually leading to high retrieval efficiency but with deteriorated accuracy. We alleviate the restrictive compactness requirement of hash codes by extending them to a two-level hierarchical coding scheme, in which the first level aims to capture the high-level semantic information extracted by the deep network using a rich encoding strategy, while the subsequent level squeezes them to more global and compact codes. At running time, we adopt an attention-based mechanism to select some of its most essential bits specific to each query image for retrieval instead of using the full hash codes of the first level. The attention-based mechanism is based on the guides of hash codes generated by the second level, taking advantage of both local and global properties of deep features. Experimental results on various popular datasets demonstrate the advantages of the proposed method compared to several state-of-the-art methods.

Keywords image retrieval deep hashing hierarchical deep hashing

Corresponding Author(s): Xiaoyang TAN

Just Accepted Date: 09 February 2017 Online First Date: 23 March 2017 Issue Date: 06 April 2017

Cite this article:

Ge SONG,Xiaoyang TAN. Hierarchical deep hashing for image retrieval[J]. Front. Comput. Sci., 2017, 11(2): 253-265.

URL:

https://academic.hep.com.cn/fcs/EN/10.1007/s11704-017-6537-3
https://academic.hep.com.cn/fcs/EN/Y2017/V11/I2/253

1	Smeulders A W M, Worring M, Santini S, Gupta A, Jain R. Contentbased image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(12): 1349–1380 https://doi.org/10.1109/34.895972
2	Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 2012, 25(2): 2012
3	Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014, 580–587 https://doi.org/10.1109/cvpr.2014.81
4	Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 1337–1342 https://doi.org/10.1109/cvpr.2015.7298965
5	Zheng L, Yang Y, Tian Q. SIFT Meets CNN: a decade survey of instance retrieval. 2016, arXiv preprint arXiv:1608.01807
6	Babenko A, Slesarev A, Chigorin A, Lempitsky V. Neural codes for image retrieval. In: Proceedings of European Conference on Computer Vision. 2014, 584–599 https://doi.org/10.1007/978-3-319-10590-1_38
7	Razavian A S, Azizpour H, Sullivan J, Carlsson S. CNN features offthe- shelf: an astounding baseline for recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2014, 512–519
8	Babenko A, Lempitsky V. Aggregating deep convolutional features for image retrieval. In: Proceedings of the IEEE Conference on Computer Vision. 2015, 1269–1277
9	Tolias G, Sicre R, Jégou H. Particular object retrieval with integral max-pooling of CNN activations. Computer Science, 2015
10	Ng Y H, Yang F, Davis L S. Exploiting local features from deep networks for image retrieval. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 53–61 https://doi.org/10.1109/cvprw.2015.7301272
11	Zheng L, Zhao Y L, Wang S J, Wang J D, Tian Q. Good practice in CNN feature transfer. 2016, arXiv preprint arXiv:1604, 00133
12	Zheng L, Wang S J, Wang J D, Tian Q. Accurate image search with multi-scale contextual evidences. International Journal of Computer Vision, 2016(1): 1–13 https://doi.org/10.1007/s11263-016-0889-2
13	Zhou B L, Khosla A, Lapedriza A, Oliva A, Torralba A. Learning deep features for discriminative localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 2921–2929 https://doi.org/10.1109/cvpr.2016.319
14	Andoni A, Indyk P. Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions. In: Proceedings of the 47th Annual IEEE Symposium on Foundations of Computer Science. 2006, 459–468 https://doi.org/10.1109/focs.2006.49
15	Liong V E, Lu J W, Wang G, Moulin P, Zhou J. Deep hashing for compact binary codes learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 2475–2483 https://doi.org/10.1109/cvpr.2015.7298862
16	Zhao F, Huang Y Z, Wang L, Tan T N. Deep semantic ranking based hashing for multi-label image retrieval. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 1556–1564
17	Chang S F. Supervised hashing with kernels. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2012, 2074–2081
18	Gong Y C, Pawlowski M, Yang F, Brandy L, Bourdev L, Fergus R. Web scale photo hash clustering on a single machine. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 19–27
19	Xia R K, Pan Y, Lai H J, Liu C, Yan S C. Supervised Hashing for Image Retrieval via Image Representation Learning. In: Proceedings of the 28th AAAI Conference on Artificial Intelligence. 2014
20	Li W J, Wang S, Kang W C. Feature learning based deep supervised hashing with pairwise labels. Computer Science, 2015
21	Liu H M, Wang R P, Shan S G, Chen X L. Deep supervised hashing for fast image retrieval. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 2064–2072 https://doi.org/10.1109/cvpr.2016.227
22	Paulin M, Douze M, Harchaoui Z, Mairal J, Perronin F, Schmid C. Local convolutional features with unsupervised training for image retrieval. In: Proceedings of the IEEE International Conference on Computer Vision. 2015, 91–99 https://doi.org/10.1109/iccv.2015.19
23	Kalantidis Y, Mellina C, Osindero S. Cross-dimensional weighting for aggregated deep convolutional features. In: Proceedings of European Conference on Computer Vision. 2016, 685–701 https://doi.org/10.1007/978-3-319-46604-0_48
24	Salvador A, Giroinieto X, Marques F, Satoh S I. Faster R-CNN features for instance search. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2016, 9–16 https://doi.org/10.1109/cvprw.2016.56
25	Lin K, Yang H F, Hsiao J H, Chen C S. Deep learning of binary hash codes for fast image retrieval. In: Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2015, 27–35 https://doi.org/10.1109/cvprw.2015.7301269
26	Lai H J, Pan Y, Liu Y, Yan S C. Simultaneous feature learning and hash coding with deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 3270–3278 https://doi.org/10.1109/cvpr.2015.7298947
27	Zeiler M D, Fergus R. Visualizing and understanding convolutional networks. In: Proceedings of European Conference on Computer Vision. 2013, 818–833
28	Mahendran A, Vedaldi A. Understanding deep image representations by inverting them. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 5188–5196 https://doi.org/10.1109/cvpr.2015.7299155
29	Krizhevsky A. Learning multiple layers of features from tiny images. Technical Report. 2012
30	Jegou H, Douze M, Schmid C. Hamming embedding and weak geometric consistency for large scale image search. In: Proceedings of European conference on computer vision. 2008, 304–317 https://doi.org/10.1007/978-3-540-88682-2_24
31	Philbin J, Chum O, Isard M, Sivic J, Zisserman A. Object retrieval with large vocabularies and fast spatial matching. In: Proceedings of the IEEE International Conference on Computer Vision. 2007, 1–8 https://doi.org/10.1109/cvpr.2007.383172
32	Philbin J, Chum O, Isard M, Sivic J, Zisserman A. Lost in quantization: improving particular object retrieval in large scale image databases. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2008, 1–8 https://doi.org/10.1109/cvpr.2008.4587635
33	Nister D, Stewenius H. Scalable recognition with a vocabulary tree. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2006, 2161–2168 https://doi.org/10.1109/cvpr.2006.264
34	Jia Y Q, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T. Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM international conference on Multimedia. 2014, 675–678 https://doi.org/10.1145/2647868.2654889
35	Gong Y C, Lazebnik S. Iterative quantization: A procrustean approach to learning binary codes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2011, 817–824 https://doi.org/10.1109/cvpr.2011.5995432
36	Weiss Y, Torralba A, Fergus R. Spectral hashing. In: Proceedings of the Neural Information Processing Systems Conference. 2008, 1753–1760
37	Heo J P, Lee Y, He J, Chang S F, Yoon S E. Spherical hashing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2012, 2957–2964
38	Jiang Q Y, Li W J. Scalable graph hashing with feature transformation. In: Proceedings of the International Conference on Artificial Intelligence. 2015, 331–337
39	Lin G S, Shen C H, Shi Q F, van den Hengel A, Suter D. Fast supervised hashing with decision trees for high-dimensional data. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014, 1971–1978 https://doi.org/10.1109/cvpr.2014.253
40	Shen F M, Shen C H, Liu W, Shen H T. Supervised discrete hashing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 37–45 https://doi.org/10.1109/cvpr.2015.7298598
41	Zhang P C, Zhang W, Li W J, Guo M Y. Supervised hashing with latent factor models. In: Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2014, 173–182 https://doi.org/10.1145/2600428.2609600
42	Kang W C, Li W J, Zhou Z H. Column sampling based discrete supervised hashing. In: Proceedings of the 30th AAAI Conference on Artificial Intelligence. 2016
43	Zhang R M, Lin L, Zhang R, Zuo W M, Zhang L. Bit-scalable deep hashing with regularized similarity learning for image retrieval and person re-identification. IEEE Transactions on Image Processing, 2015, 24(12): 4766–4779 https://doi.org/10.1109/TIP.2015.2467315
44	Arandjelovic R, Zisserman A. All about VLAD. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2013, 1578–1585 https://doi.org/10.1109/cvpr.2013.207
45	Jégou H, Zisserman A. Triangulation embedding and democratic aggregation for image search. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014, 3310–3317 https://doi.org/10.1109/cvpr.2014.417
46	Razavian A S, Sullivan J, Carlsson S, Maki A. Visual instance retrieval with deep convolutional networks. 2014, arXiv preprint arXiv:1412.6574

[1]	Ying LI, Xiangwei KONG, Haiyan FU, Qi TIAN. Contextual modeling on auxiliary points for robust image reranking[J]. Front. Comput. Sci., 2019, 13(5): 1010-1022.
[2]	Kun SU, Gongping YANG, Lu YANG, Peng SU, Yilong YIN. Non-negative locality-constrained vocabulary tree for finger vein image retrieval[J]. Front. Comput. Sci., 2019, 13(2): 318-332.
[3]	Xueming WANG, Zechao LI, Jinhui TANG. Visual understanding by mining social media: recent advances and challenges[J]. Front. Comput. Sci., 2018, 12(3): 406-422.
[4]	Le DONG, Wenpu DONG, Ning FENG, Mengdie MAO, Long CHEN, Gaipeng KONG. Color space quantization-based clustering for image retrieval[J]. Front. Comput. Sci., 2017, 11(6): 1023-1035.
[5]	Weimin TAN,Bo YAN. A survey on high coherence visual media retargeting: recent advances and applications[J]. Front. Comput. Sci., 2016, 10(5): 778-796.

Viewed

Full text

Abstract

Cited

Shared

Discussed