|
|
Contextual modeling on auxiliary points for robust image reranking |
Ying LI1, Xiangwei KONG1( ), Haiyan FU1, Qi TIAN2 |
1. School of Information and Communication Engineering, Dalian University of Technology, Dalian 116024, China 2. Department of Computer Science, University of Texas at San Antonio, Texas 78249, USA |
|
|
Abstract Image reranking is an effective post-processing step to adjust the similarity order in image retrieval. As key components of initialized ranking lists, top-ranked neighborhoods of a given query usually play important roles in constructing dissimilarity measure. However, the number of pertinent candidates varies with respect to different queries. Thus the images with short lists of ground truth suffer from insufficient contextual information. It consequently introduces noises when using k-nearest neighbor rule to define the context. In order to alleviate this problem, this paper proposes auxiliary points which are added as assistant neighbors in an unsupervised manner. These extra points act on revealing implicit similarity in the metric space and clustering matched image pairs. By isometrically embedding each constructed metric space into the Euclidean space, the image relationships on underlying topological manifolds are locally represented by distance descriptions. Furthermore, by combining Jaccard index with our auxiliary points, we present a contextual modeling on auxiliary points (CMAP) method for image reranking.With richer contextual activations, the Jaccard similarity coefficient defined by local distribution achieves more reliable outputs as well as more stable parameters. Extensive experiments demonstrate the robustness and effectiveness of the proposed method.
|
Keywords
image retrieval
unsupervised reranking
context construction
Jaccard distance
query expansion
|
Corresponding Author(s):
Xiangwei KONG
|
Just Accepted Date: 28 May 2018
Online First Date: 04 September 2018
Issue Date: 25 June 2019
|
|
1 |
H Jégou, M Douze, C Schmid. Improving bag-of-features for large scale image search. International Journal of Computer Vision, 2010, 87(3): 316–336
https://doi.org/10.1007/s11263-009-0285-2
|
2 |
G Song, X Tan. Hierarchical deep hashing for image retrieval. Frontiers of Computer Science, 2017, 11(2): 253–265
https://doi.org/10.1007/s11704-017-6537-3
|
3 |
D G Lowe. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, 60(2): 91–110
https://doi.org/10.1023/B:VISI.0000029664.99615.94
|
4 |
Y Li, X Kong, L Zheng, Q Tian. Exploiting hierarchical activations of neural network for image retrieval. In: Proceedings of the 24nd ACM International Conference on Multimedia. 2016, 132–136
https://doi.org/10.1145/2964284.2967197
|
5 |
H Jégou, F Perronnin, M Douze, J Sanchez, P Perez, C Schmid. Aggregating local image descriptors into compact codes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(9): 1704–1716
https://doi.org/10.1109/TPAMI.2011.235
|
6 |
Z Liu, S Wang, Q Tian. Fine-residual VLAD for image retrieval. Neurocomputing, 2016, 173: 1183–1191
https://doi.org/10.1016/j.neucom.2015.08.076
|
7 |
L Zheng, S Wang, Z Liu, Q Tian. Packing and padding: coupled multiindex for accurate image retrieval. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014, 1939–1946
https://doi.org/10.1109/CVPR.2014.250
|
8 |
O Chum, A Mikulik, M Perdoch, J Matas. Total recall II: query expansion revisited. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2011, 889–896
https://doi.org/10.1109/CVPR.2011.5995601
|
9 |
S Bai, X Bai, Q Tian, L J Latecki. Regularized diffusion process for visual retrieval. In: Proceedings of the AAAI Conference on Artificial Intelligence. 2017, 3967–3973
|
10 |
S Bai, Z Zhou, J Wang, X Bai, L J Latecki, Q Tian. Ensemble diffusion for retrieval. In: Proceedings of the IEEE Conference on Computer Vision. 2017, 774–783
https://doi.org/10.1109/ICCV.2017.90
|
11 |
S Köknar-Tezel, L J Latecki. Improving SVM classification on imbalanced time series data sets with ghost points. Knowledge and Information Systems, 2011, 28(1): 1–23
https://doi.org/10.1007/s10115-010-0310-3
|
12 |
H Jégou, M Douze, C Schmid. On the burstiness of visual elements. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2009, 1169–1176
https://doi.org/10.1109/CVPR.2009.5206609
|
13 |
Y Zhu, J Jiang, W Han, Y Ding, Q Tian. Interpretation of users’ feedback via swarmed particles for content-based image retrieval. Information Sciences, 2017, 375: 246–257
https://doi.org/10.1016/j.ins.2016.09.021
|
14 |
L Zheng, Y Yang, Q Tian. Sift meets CNN: a decade survey of instance retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 40(5): 1224–1244
https://doi.org/10.1109/TPAMI.2017.2709749
|
15 |
K Chen, G Ding, J Han. Attribute-based supervised deep learning model for action recognition. Frontiers of Computer Science, 2017, 11(2): 219–229
https://doi.org/10.1007/s11704-016-6066-5
|
16 |
Y Gong, L Wang, R Guo, S Lazebnik. Multi-scale orderless pooling of deep convolutional activation features. In: Proceedings of the European Conference on Computer Vision. 2014, 392–407
https://doi.org/10.1007/978-3-319-10584-0_26
|
17 |
A Babenko, A Slesarev, A Chigorin, V Lempitsky. Neural codes for image retrieval. In: Proceedings of the European Conference on Computer Vision. 2014, 584–599
https://doi.org/10.1007/978-3-319-10590-1_38
|
18 |
Y Kalantidis, C Mellina, S Osindero. Cross-dimensional weighting for aggregated deep convolutional features. In: Proceedings of the European Conference on Computer Vision. 2016, 685–701
https://doi.org/10.1007/978-3-319-46604-0_48
|
19 |
J Y Ng, F Yang, L S Davis. Exploiting local features from deep networks for image retrieval. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2015, 53–61
https://doi.org/10.1109/CVPRW.2015.7301272
|
20 |
A Babenko, V Lempitsky. Aggregating local deep features for image retrieval. In: Proceedings of the IEEE International Conference on Computer Vision. 2015, 1269–1277
|
21 |
O Chum, J Philbin, J Sivic, M Isard, A Zisserman. Total recall: automatic query expansion with a generative feature model for object retrieval. In: Proceedings of the IEEE International Conference on Computer Vision. 2007, 1–8
https://doi.org/10.1109/ICCV.2007.4408891
|
22 |
D Qin, S Gammeter, L Bossard, T Quack, L V Gool. Hello neighbor: accurate object retrieval with k-reciprocal nearest neighbors. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2011, 777–784
https://doi.org/10.1109/CVPR.2011.5995373
|
23 |
H Jégou, C Schmid, H Harzallah, J Verbeek. Accurate image search using the contextual dissimilarity measure. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(1): 2–11
https://doi.org/10.1109/TPAMI.2008.285
|
24 |
S Sun, Y Li, W Zhou, Q Tian, H Li. Local residual similarity for image re-ranking. Information Sciences, 2017, 417: 143–153
https://doi.org/10.1016/j.ins.2017.07.004
|
25 |
R Arandjelovi′c, A Zisserman. Three things everyone should know to improve object retrieval. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2012, 2911–2918
|
26 |
X Yang, L Prasad, L J Latecki. Affinity learning with diffusion on tensor product graph. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(1): 28–38
https://doi.org/10.1109/TPAMI.2012.60
|
27 |
M Donoser, H Bischof. Diffusion processes for retrieval revisited. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2013, 1320–1327
https://doi.org/10.1109/CVPR.2013.174
|
28 |
S Bai, X Bai, Q Tian. Scalable person re-identification on supervised smoothed manifold. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017, 3356–3365
https://doi.org/10.1109/CVPR.2017.358
|
29 |
S Bai, X Bai, Q Tian, L J Latecki. Regularized diffusion process on bidirectional context for object retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 99: 1
https://doi.org/10.1109/TPAMI.2018.2828815
|
30 |
S Bai, X Bai. Sparse contextual activation for efficient visual reranking. IEEE Transactions on Image Processing, 2016, 25(3): 1056–1069
https://doi.org/10.1109/TIP.2016.2514498
|
31 |
D Nister, H Stewenius. Scalable recognition with a vocabulary tree. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2006, 2161–2168
https://doi.org/10.1109/CVPR.2006.264
|
32 |
H Jégou, M Douze, C Schmid. Hamming embedding and weak geometric consistency for large scale image search. In: Proceedings of the European Conference on Computer Vision. 2008, 304–317
https://doi.org/10.1007/978-3-540-88682-2_24
|
33 |
V Balntas, K Lenc, A Vedaldi, K Mikolajczyk. Hpatches: a benchmark and evaluation of handcrafted and learned local descriptors. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017, 3852–3861
https://doi.org/10.1109/CVPR.2017.410
|
34 |
J Philbin, O Chum, M Isard, J Sivic, A Zisserman. Object retrieval with large vocabularies and fast spatial matching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2007, 1–8
https://doi.org/10.1109/CVPR.2007.383172
|
35 |
M J Huiskes, M S Lew. The mir flickr retrieval evaluation. In: Proceedings of the ACM International Conference on Multimedia Information Retrieval. 2008, 39–43
https://doi.org/10.1145/1460096.1460104
|
36 |
Y Jia, E Shelhamer, J Donahue, S Karayev, J Long, R Girshick, S Guadarrama, T Darrell. Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM International Conference on Multimedia. 2014, 675–678
https://doi.org/10.1145/2647868.2654889
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|