Local features and manifold ranking coupled method for sketch-based 3D model retrieval

doi:10.1007/s11704-017-6595-6

Front. Comput. Sci.

2018, Vol. 12

Issue (5) : 1000-1012 https://doi.org/10.1007/s11704-017-6595-6

RESEARCH ARTICLE

Local features and manifold ranking coupled method for sketch-based 3D model retrieval

Xiaohui TAN¹, Yachun FAN²(

), Ruiliang GUO³

¹. College of Information Engineering, Capital Normal University, Beijing 100048, China
². College of Information Science and Technology, Beijing Normal University, Beijing 100875, China
³. School of Fashion, Beijing Institute of Fashion Technology, Beijing 100028, China

Download: PDF(539 KB)
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

3D model retrieval can benefit many downstream virtual reality applications. In this paper, we propose a new sketch-based 3D model retrieval framework by coupling local features and manifold ranking. At technical fronts, we exploit spatial pyramids based local structures to facilitate the efficient construction of feature descriptors.Meanwhile, we propose an improved manifold ranking method, wherein all the categories between arbitrary model pairs will be taken into account. Since the smooth and detail-preserving line drawings of 3D model are important for sketch-based 3D model retrieval, the Difference of Gaussians (DoG) method is employed to extract the line drawings over the projected depth images of 3D model, and Bezier Curve is then adopted to further optimize the extracted line drawing. On that basis, we develop a 3D model retrieval engine to verify our method. We have conducted extensive experiments over various public benchmarks, and have made comprehensive comparisons with some state-of-the-art 3D retrieval methods. All the evaluation results based on the widely-used indicators prove the superiority of our method in accuracy, reliability, robustness, and versatility.

Keywords sketch-based retrieval 3D model manifold ranking line drawing local features

Corresponding Author(s): Yachun FAN

Just Accepted Date: 29 September 2017 Online First Date: 30 October 2017 Issue Date: 21 September 2018

Cite this article:

Xiaohui TAN,Yachun FAN,Ruiliang GUO. Local features and manifold ranking coupled method for sketch-based 3D model retrieval[J]. Front. Comput. Sci., 2018, 12(5): 1000-1012.

URL:

https://academic.hep.com.cn/fcs/EN/10.1007/s11704-017-6595-6
https://academic.hep.com.cn/fcs/EN/Y2018/V12/I5/1000

1	Zeleznik R C, Herndon K P, Hughes J F. SKETCH: an interface for sketching 3D scenes. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques. 1996, 163–170 https://doi.org/10.1145/237170.237238
2	Gao Y, Wang M, Tao D C, Ji R R, Dai Q H. 3-D object retrieval and recognition with hypergraph analysis. IEEE Transactions on Image Processing, 2012, 21(9): 4290–4303 https://doi.org/10.1109/TIP.2012.2199502
3	Walther D B, Chai B, Caddigan E, Beck D M, Li F F. Simple line drawings suffice for functional MRI decoding of natural scene categories. Proceedings of the National Academy of Sciences, 2011, 108(23): 9661–9666 https://doi.org/10.1073/pnas.1015666108
4	Eitz M, Hays J, Alexa M. How do humans sketch objects? ACMTransactions on Graphics, 2012, 31(4): 44 https://doi.org/10.1145/2185520.2185540
5	Funkhouser T, Min P, Kazhdan M, Chen J, Halderman A, Dobkin D, Jacobs D. A search engine for 3D models. ACM Transactions on Graphics, 2003, 22(1): 83–105 https://doi.org/10.1145/588272.588279
6	Li B, Lu Y J, Ghumman A, Strylowski B, Gutierrez M, Sadiq S, Forster S, Feola N, Bugerin T. 3D Sketch-Based 3D model retrieval. In: Proceedings of the 5th ACM International Conference on Multimedia Retrieval. 2015, 555–558 https://doi.org/10.1145/2671188.2749349
7	Liu X L, Huang L, Deng C, Tao D C. Query-adaptive hash code ranking for large-scale multi-view visual search. IEEE Transactions on Image Processing, 2016, 25(10): 4514–4524 https://doi.org/10.1109/TIP.2016.2593344
8	Liu X L, Du B W, Deng C, Liu M, Lang B. Structure sensitive hashing with adaptive product quantization. IEEE Transactions on Cybernetics, 2016, 46(10): 2252–2264 https://doi.org/10.1109/TCYB.2015.2474742
9	DeCarlo D, Finkelstein A. Rusinkiewicz S, Santella . ASuggestive contours for conveying shape. ACM Transactions on Graphics, 2003, 22(3): 848–855 https://doi.org/10.1145/882262.882354
10	Saavedra J M, Bustos B, Schreck T, Yoon S M, Scherer M. Sketchbased 3D model retrieval using keyshapes for global and local representation. In: Proceedings of the 5th Eurographics Conference on 3D Object Retrieval. 2012, 47–50
11	Yoon S, Scherer M, Schreck T, Kuijper A. A sketch-based 3D model retrieval using diffusion tensor fields of suggestive contours. In: Proceedings of the 18th ACM International Conference on Multimedia. 2010, 193–200 https://doi.org/10.1145/1873951.1873961
12	Yoon S M, Kuijper A. Sketch-based 3D model retrieval using compressive sensing classification. Electronics Letters, 2011, 47(21): 1181–1183 https://doi.org/10.1049/el.2011.2158
13	Judd T, Durand F, Adelson E. Apparent ridges for line drawing. ACM Transactions on Graphics, 2007, 26(3): 19 https://doi.org/10.1145/1276377.1276401
14	Eitz M, Richter R, Boubekeur T, Hildebrand K, Alexa M. Sketch-based shape retrieval. ACM Transactions on Graphics, 2012, 31(4): 1–10 https://doi.org/10.1145/2185520.2185527
15	Li B, Schreck T, Godil A, Alexa M, Boubekeur T, Bustos B, Chen J, Eitz M, Furuya T, Hildebrand K, Huang S, Johan H, Kuijper A, Ohbuchi R, Richter R, Saavedra J M, Scherer M, Yanagimachi T, Yoon G J, Yoon S M. Track: sketch-based 3D shape retrieval. In: Proceedings of the 5th Eurographics Conference on 3D Object Retrieval. 2012, 109–118
16	Shao T J, Xu W W, Yin K K, Wang J D, Zhou K, Guo B N. Discriminative sketch-based 3D model retrieval via robust shape matching. In: Proceedings of Comput Graph Forum. 2011, 2011–2020 https://doi.org/10.1111/j.1467-8659.2011.02050.x
17	Li B, Lu Y, Li C, Godil A, Schreck T, Aono M, Burtscher M, Fu H, Furuya T, Johan H, Liu J, Ohbuchi R, Tatsuma A, Zou C. SHREC’14 track: extended large scale sketch-based 3D shape retrieval. In: Proceedings of Eurographics Workshop on 3D Object Retrieval. 2014
18	Li B, Lu Y, Godil A, Schreck T, Aono M, Johan H, Saavedra J M, Tashiro S. SHREC’13 track: scale sketch-based 3D shape retrieval. In: Proceedings of the 6th Eurographics Workshop on 3D Object Retrival. 2013, 89–96
19	Yoon S M, Yoon G, Schreck T. User-drawn sketch-based 3D object retrieval using sparse coding. Multimedia Tools and Applications. 2015, 74(13): 4707–4722 https://doi.org/10.1007/s11042-013-1831-z
20	Hu R, Wang T H, Collomosse J. A bag-of-regions approach to sketchbased image retrieval. In: Proceedings of the 18th IEEE International Conference on Image Processing. 2011, 3661–3664
21	Li B, Johan H. Sketch-based 3D model retrieval by incorporating 2D- 3D alignment. Multimedia Tools and Applications, 2013, 65(3): 363–385 https://doi.org/10.1007/s11042-012-1009-0
22	Li B, Johan H. A 3D model feature for retrieval. In: Proceedings of MMM. 2010, 185–195
23	Belongie S, Malik J, Puzicha J. Shape matching and object recognition using shape contexts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(4): 509–522 https://doi.org/10.1109/34.993558
24	Ohbuchi R, Furuya T. Scale-weighted dense bag of visual features for 3D model retrieval from a partial view 3D model. In: Proceedings of the 12th IEEE International Conference on Computer Vision. 2009, 63–70 https://doi.org/10.1109/ICCVW.2009.5457716
25	Lowe D G. Distinctive Image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, 60(2): 91–110 https://doi.org/10.1023/B:VISI.0000029664.99615.94
26	Wang F, Lin L F, Tang M. A new sketch-based 3D model retrieval approach by using global and local features. Graphical Models, 2014, 76(3): 128–139 https://doi.org/10.1016/j.gmod.2013.11.002
27	Wang F, Le K, Li Y. Sketch-based 3D shape retrieval using convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 1875–1883 https://doi.org/10.1109/CVPR.2015.7298797
28	Zhu F, Xie J, Fang Y. Learning cross-domain neural networks for sketch-based 3D shape retrieval. In: Proceedings of the 30th AAAI Conference on Artificial Intelligence. 2016.
29	He J R, Li M J, Zhang H J, Tong H H, Zhang C S. Manifold-ranking based image retrieval. In: Proceedings of the 12th Annual ACM International Conference on Multimedia. 2004, 9–16 https://doi.org/10.1145/1027527.1027531
30	Tong H H, He J R, Li M J, Ma W Y, Zhan g H J, Zhang C S. Manifoldranking- based keyword propagation for image retrieval. EURASIP Journal on Applied Signal Processing, 2006, 2006: 190
31	Guan Z Y, Bu J J, Mei Q Z, Chen C, Wang C. Personalized tag recommendation using graph-based ranking on multi-type interrelated objects. In: Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2009, 540–547 https://doi.org/10.1145/1571941.1572034
32	Agarwal S. Ranking on on graph data. In: Proceedings of the 23rd Internatioal Conference on Machine Learning. 2006, 25–32 https://doi.org/10.1145/1143844.1143848
33	Bu J J, Tan S L, Chen C, Wang C, Wu H, Zhang L J, He X F. Music recommendation by unified hypergraph: combining social media information and music content. In: Proceedings of the 18th ACM International Conference on Multimedia. 2010, 391–400 https://doi.org/10.1145/1873951.1874005
34	Furuya T, Ohbuchi R. Similarity metric learning for sketch-based 3D object retrieval. Multimedia Tools and Applications, 2015, 74(23): 10367–10392 https://doi.org/10.1007/s11042-014-2171-3
35	Liu X L, Lang B, Xu Y, Cheng B. Feature grouping and local soft match for mobile visual search. Pattern Recognition Letters, 2012, 33(3): 239–246 https://doi.org/10.1016/j.patrec.2011.10.002
36	Yang Y1, Nie F P, Xu D B, Luo J, Zhuang Y T, Pan Y H. Multimedia retrieval framework based on semi-supervised ranking and relevance feedback. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(4): 723–742 https://doi.org/10.1109/TPAMI.2011.170
37	Liu X L, Deng C, Lang B, Tao D C, Li X L. Query-adaptive reciprocal hash tables for nearest neighbor search. IEEE Transactions on Image Processing, 2015, 25(2): 907–919 https://doi.org/10.1109/TIP.2015.2505180

[1]

Download

[1]	Ying-Ying XU, Li-Xiu YAO, Hong-Bin SHEN. Bioimage-based protein subcellular location prediction: a comprehensive review[J]. Front. Comput. Sci., 2018, 12(1): 26-39.
[2]	Yongjin LIU, Minjing YU, Qiufang FU, Wenfeng CHEN, Ye LIU, Lexing XIE. Cognitive mechanism related to line drawings and its applications in intelligent process of visual media: a survey[J]. Front. Comput. Sci., 2016, 10(2): 216-232.
[3]	Vahid MEHRDAD,Hossein EBRAHIMNEZHAD. 3D object retrieval based on histogram of local orientation using one-shot score support vector machine[J]. Front. Comput. Sci., 2015, 9(6): 990-1005.
[4]	Song WANG, Seiichi UCHIDA, Marcus LIWICKI, Yaokai FENG. Part-based methods for handwritten digit recognition[J]. Front Comput Sci, 2013, 7(4): 514-525.

Viewed

Full text

Abstract

Cited

Shared

Discussed