|
|
Joint salient object detection and existence prediction |
Huaizu JIANG1, Ming-Ming CHENG2(), Shi-Jie LI1, Ali BORJI3, Jingdong WANG4 |
1. College of Information and Computer Sciences, University of Massachusetts Amherst, Amherst, MA 01003, USA 2. CCCE, Nankai University Jinnan Campus, Tianjin 300353, China 3. Center for Research in Computer Vision, University of Central Florida, Orlando, FL 32816, USA 4. Microsoft Research, Beijing 100080, China |
|
|
Abstract Recent advances in supervised salient object detection modeling has resulted in significant performance improvements on benchmark datasets. However, most of the existing salient object detection models assume that at least one salient object exists in the input image. Such an assumption often leads to less appealing saliencymaps on the background images with no salient object at all. Therefore, handling those cases can reduce the false positive rate of a model. In this paper, we propose a supervised learning approach for jointly addressing the salient object detection and existence prediction problems. Given a set of background-only images and images with salient objects, as well as their salient object annotations, we adopt the structural SVM framework and formulate the two problems jointly in a single integrated objective function: saliency labels of superpixels are involved in a classification term conditioned on the salient object existence variable, which in turn depends on both global image and regional saliency features and saliency labels assignments. The loss function also considers both image-level and regionlevel mis-classifications. Extensive evaluation on benchmark datasets validate the effectiveness of our proposed joint approach compared to the baseline and state-of-the-art models.
|
Keywords
salient object detection
existence prediction
joint inference
saliency detection
|
Corresponding Author(s):
Ming-Ming CHENG
|
Just Accepted Date: 22 August 2017
Online First Date: 25 May 2018
Issue Date: 29 May 2019
|
|
1 |
LItti, CKoch, ENiebur. A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Snalysis and Machine Intelligence, 1998, 20(11): 1254–1259
https://doi.org/10.1109/34.730558
|
2 |
ABorji, LItti. State-of-the-art in visual attention modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(1): 185–207
https://doi.org/10.1109/TPAMI.2012.89
|
3 |
ABorji, D NSihite, LItti. Quantitative analysis of human-model agreement in visual saliency modeling: a comparative study. IEEE Transactions on Image Processing, 2013, 22(1): 55–69
https://doi.org/10.1109/TIP.2012.2210727
|
4 |
TLiu, ZYuan, JSun, J Wang, NZheng, XTang, H YShum. Learning to detect a salient object. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(2): 353–367
https://doi.org/10.1109/TPAMI.2010.70
|
5 |
G XZhang, M MCheng, S MHu, R R Martin. A shape-preserving approach to image resizing. Computer Graphics Forum, 2009, 28(7): 1897–1906
https://doi.org/10.1111/j.1467-8659.2009.01568.x
|
6 |
TChen, M MCheng, PTan, A Shamir, S MHu. Sketch2photo: Internet image montage. ACM Transactions on Graphics (TOG), 2009, 28(5): 124
https://doi.org/10.1145/1618452.1618470
|
7 |
TChen, PTan, L QMa, M M Cheng, AShamir, S MHu. Poseshop: human image database construction and personalized content synthesis. IEEE Transactions on Visualization and Computer Graphics, 2013, 19(5): 824–837
https://doi.org/10.1109/TVCG.2012.148
|
8 |
M MCheng, N JMitra , XHuang, S M Hu. Salientshape: group saliency in image collections. The Visual Computer, 2014, 30(4): 443–453
https://doi.org/10.1007/s00371-013-0867-4
|
9 |
JWang, LQuan, JSun, X Tang, H YShum. Picture collage. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2006, 347–354
|
10 |
AAbdulmunem, Y KLai, XSun. Saliency guided local and global descriptors for effective action recognition. Computational Visual Media, 2016, 2(1): 97–106
https://doi.org/10.1007/s41095-016-0033-9
|
11 |
JZhang, YHan, JJiang. Tucker decomposition-based tensor learning for human action recognition. Multimedia Systems, 2016, 22(3): 343–353
https://doi.org/10.1007/s00530-015-0464-7
|
12 |
S MHu, TChen, KXu, M MCheng, R RMartin. Internet visual media processing: a survey with graphics and vision applications. The Visual Computer, 2013, 29(5): 393–405
https://doi.org/10.1007/s00371-013-0792-6
|
13 |
M MCheng, Q BHou, S HZhang, P L Rosin. Intelligent visual media processing: when graphics meets vision. Journal of Computer Science and Technology, 2017, 32(1): 110–121
https://doi.org/10.1007/s11390-017-1681-7
|
14 |
HJiang, JWang, ZYuan, Y Wu, NZheng, SLi. Salient object detection: a discriminative regional feature integration approach. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2013, 2083–2090
https://doi.org/10.1109/CVPR.2013.271
|
15 |
RZhao , WOuyang, HLi, XWang. Saliency detection by multi-context deep learning. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2015, 1265–1274
https://doi.org/10.1109/CVPR.2015.7298731
|
16 |
GLi, YYu. Visual saliency based on multiscale deep features. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2015, 5455–5463
|
17 |
FPerazzi, P Krähenbühl, YPritch, AHornung. Saliency filters: contrast based filtering for salient region detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2012, 733–740
https://doi.org/10.1109/CVPR.2012.6247743
|
18 |
WZhu, SLiang, YWei , J Sun. Saliency optimization from robust background detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2014, 2814–2821
https://doi.org/10.1109/CVPR.2014.360
|
19 |
XLi, HLu, LZhang, X Ruan, M HYang. Saliency detection via dense and sparse reconstruction. In: Proceedings of IEEE International Conference on Computer Vision. 2013, 2976–2983
https://doi.org/10.1109/ICCV.2013.370
|
20 |
YLeCun, LBottou, YBengio, P Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998, 86(11): 2278–2324
https://doi.org/10.1109/5.726791
|
21 |
AKrizhevsky, I Sutskever, G EHinton. Imagenet classification with deep convolutional neural networks. In: Proceedings of the Neural Information Processing Systems Conference. 2012, 1106–1114
|
22 |
ABorji. What is a salient object? a dataset and a baseline model for salient object detection. IEEE Transactions on Image Processing, 2015, 24(2): 742–756
https://doi.org/10.1109/TIP.2014.2383320
|
23 |
PWang, JWang, GZeng, J Feng, HZha, SLi. Salient object detection for searched Web images via global saliency. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2012, 3194–3201.
https://doi.org/10.1109/CVPR.2012.6248054
|
24 |
YBoykov, V Kolmogorov. An experimental comparison of mincut/ max-flow algorithms for energy minimization in vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(9): 1124–1137
https://doi.org/10.1109/TPAMI.2004.60
|
25 |
ABorji, M MCheng, QHou, H Jiang, JLi. Salient object detection: a survey. 2014, arXiv preprint arXiv:1411.5878
|
26 |
ABorji, M MCheng, HJiang, J Li. Salient object detection: a benchmark. IEEE Transactions on Image Processing, 2015, 24(12): 5706–5722
https://doi.org/10.1109/TIP.2015.2487833
|
27 |
JHan, NLiu, DZhang. Visual saliency detection and applications: a survey. Frontiers of Computer Science, 2017
|
28 |
RAchanta, SHemami, FEstrada, S Süsstrunk. Frequency-tuned salient region detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2009, 1597–1604
https://doi.org/10.1109/CVPR.2009.5206596
|
29 |
SGoferman, L Zelnik-Manor, ATal. Context-aware saliency detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(10): 1915–1926
https://doi.org/10.1109/TPAMI.2011.272
|
30 |
YTian, JLi, SYu, THuang. Learning complementary saliency priors for foreground object segmentation in complex scenes. International Journal of Computer Vision, 2015, 111(2): 153–170
https://doi.org/10.1007/s11263-014-0737-1
|
31 |
SFang, JLi, YTian, T Huang, XChen. Learning discriminative subspaces on random contrasts for image saliency analysis. IEEE Transactions on Neural Networks and Learning Systems, 2017, 28(5): 1095–1108
https://doi.org/10.1109/TNNLS.2016.2522440
|
32 |
RMargolin, ATal, LZelnik-Manor. What makes a patch distinct? In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2013, 1139–1146
https://doi.org/10.1109/CVPR.2013.151
|
33 |
M MCheng, N JMitra, XHuang, P H Torr, S MHu. Global contrast based salient region detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(3): 569–582
https://doi.org/10.1109/TPAMI.2014.2345401
|
34 |
ABorji, LItti. Exploiting local and global patch rarities for saliency detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2012, 478–485
https://doi.org/10.1109/CVPR.2012.6247711
|
35 |
WQi, M MCheng, ABorji, H Lu, L FBai. SaliencyRank: two-stage manifold ranking for salient object detection. Computational Visual Media, 2015, 1(4): 309–320
https://doi.org/10.1007/s41095-015-0028-y
|
36 |
HJiang , JWang, ZYuan, T Liu, NZheng, SLi . Automatic salient object segmentation based on context and shape prior. In: Proceedings of the British Machine Vision Conference (BMVC). 2011
https://doi.org/10.5244/C.25.110
|
37 |
P FFelzenszwalb, D P Huttenlocher. Efficient graph-based image segmentation. International Journal of Computer Vision, 2004, 59(2): 16–181
https://doi.org/10.1023/B:VISI.0000022288.19776.77
|
38 |
M MCheng, YLiu, QHou, J Bian, PTorr, S MHu, ZTu. HFS: hierarchical feature selection for efficient image segmentation. In: Proceedings of European Conference on Computer Vision. 2016, 867–882
https://doi.org/10.1007/978-3-319-46487-9_53
|
39 |
QYan, LXu, JShi, J Jia. Hierarchical saliency detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2013, 1155–1162
https://doi.org/10.1109/CVPR.2013.153
|
40 |
YWei , FWen, WZhu, J Sun. Geodesic saliency using background priors. In: Proceedings of European Conference on Computer Vision. 2012, 29–42
https://doi.org/10.1007/978-3-642-33712-3_3
|
41 |
CYang, LZhang, HLu, XRuan, M HYang. Saliency detection via graph-based manifold ranking. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2013, 3166–3173
https://doi.org/10.1109/CVPR.2013.407
|
42 |
BJiang, LZhang, HLu, CYang, M HYang. Saliency detection via absorbing Markov chain. In: Proceedings of IEEE International Conference on Computer Vision. 2013, 1665–1672
https://doi.org/10.1109/ICCV.2013.209
|
43 |
JZhang, S Sclaroff. Saliency detection: a boolean map approach. In: Proceedings of IEEE International Conference on Computer Vision. 2013, 153–160
https://doi.org/10.1109/ICCV.2013.26
|
44 |
K YChang, T LLiu, H TChen, S H Lai. Fusing generic objectness and visual saliency for salient object detection. In: Proceedings of IEEE International Conference on Computer Vision. 2011, 914–921
|
45 |
PJiang, HLing, JYu, JPeng. Salient region detection by UFO: uniqueness, focusness and objectness. In: Proceedings of IEEE International Conference on Computer Vision. 2013, 1976–1983
https://doi.org/10.1109/ICCV.2013.248
|
46 |
YJia, MHan. Category-independent object-level saliency detection. In: Proceedings of IEEE International Conference on Computer Vision. 2013, 1761–1768
https://doi.org/10.1109/ICCV.2013.221
|
47 |
M MCheng, J Warrell, W YLin, SZheng, VVineet, NCrook. Efficient salient region detection with soft image abstraction. In: Proceedings of IEEE International Conference on Computer Vision. 2013, 1529–1536
https://doi.org/10.1109/ICCV.2013.193
|
48 |
LMai, YNiu, FLiu. Saliency aggregation: a data-driven approach. In: Proceedings of IEEE International Conference on Computer Vision. 2013, 1131–1138
https://doi.org/10.1109/CVPR.2013.150
|
49 |
SLu, V Mahadevan, NVasconcelos. Learning optimal seeds for diffusion-based salient object detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2014, 2790–2797
https://doi.org/10.1109/CVPR.2014.357
|
50 |
PMehrani, O Veksler. Saliency segmentation based on learning and graph cut refinement. In: Proceedings of the British Machine Vision Conference (BMVC). 2010, 1–12
https://doi.org/10.5244/C.24.110
|
51 |
JKim, DHan, Y WTai, J Kim. Salient region detection via highdimensional color transform. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014, 883–890
|
52 |
PKhuwuthyakorn, A Robles-Kelly, JZhou. Object of interest detection by saliency learning. In: Proceedings of European Conference on Computer Vision. 2010
https://doi.org/10.1007/978-3-642-15552-9_46
|
53 |
QHou, M MCheng, XHu, ABorji, ZTu, PTorr. Deeply supervised salient object detection with short connections. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2017, 5300–5309
https://doi.org/10.1109/CVPR.2017.563
|
54 |
JZhang, SMa, MSameki, S Sclaroff, MBetke, ZLin, XShen, BPrice, R Mech. Salient object subitizing. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2015, 4045–4054
https://doi.org/10.1109/CVPR.2015.7299031
|
55 |
JDeng, WDong, RSocher, L J Li, KLi, F FLi. Imagenet: a large-scale hierarchical image database. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2009, 248–255
https://doi.org/10.1109/CVPR.2009.5206848
|
56 |
RGirshick, J Donahue, TDarrell, JMalik. Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2014, 580–587
https://doi.org/10.1109/CVPR.2014.81
|
57 |
SYang, PLuo, C CLoy, X Tang. From facial parts responses to face detection: a deep learning approach. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2015, 3676–3684
https://doi.org/10.1109/ICCV.2015.419
|
58 |
MCimpoi, SMaji, AVedaldi. Deep filter banks for texture recognition and segmentation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2015, 3828–3836
https://doi.org/10.1109/CVPR.2015.7299007
|
59 |
T YLin, A RoyChowdhury, SMaji. Bilinear CNN models for finegrained visual recognition. In: Proceedings of IEEE International Conference on Computer Vision. 2015, 1449–1457
|
60 |
HSu, SMaji, EKalogerakis, ELearned-Miller. Multi-view convolutional neural networks for 3D shape recognition. In: Proceedings of IEEE International Conference on Computer Vision. 2015, 945–953
https://doi.org/10.1109/ICCV.2015.114
|
61 |
M DZeiler, RFergus. Visualizing and understanding convolutional networks. In: Proceedings of European Conference on Computer Vision. 2014, 818–833
https://doi.org/10.1007/978-3-319-10590-1_53
|
62 |
T M TDo, T Artières. Regularized bundle methods for convex and non-convex risks. Journal of Machine Learning Research, 2012, 13: 3539–3583
|
63 |
JXiao, JHays, K AEhinger, A Oliva, ATorralba. SUN database: large-scale scene recognition from abbey to zoo. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2010, 3485–3492
https://doi.org/10.1109/CVPR.2010.5539970
|
64 |
MCimpoi, SMaji, IKokkinos, S Mohamed, AVedaldi. Describing textures in the wild. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2014, 3606–3613
https://doi.org/10.1109/CVPR.2014.461
|
65 |
XShen, YWu. A unified approach to salient object detection via low rank matrix recovery. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2012, 853–860
|
66 |
HHuang, LZhang, H CZhang. Arcimboldo-like collage using internet images. ACM Transactions on Graphics, 2011, 30(6): 155
https://doi.org/10.1145/2070781.2024189
|
67 |
HLiu, LZhang, HHuang. Web-image driven best views of 3D shapes. The Visual Computer, 2012, 28(3): 279–287
https://doi.org/10.1007/s00371-011-0638-z
|
68 |
YWei, XLiang, YChen, X Shen, M MCheng, JFeng, YZhao, SYan. STC: a simple to complex framework for weakly-supervised semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(11): 2314–2320
https://doi.org/10.1109/TPAMI.2016.2636150
|
69 |
A Y SChia, SZhuo, R KGupta, Y W Tai, S YCho, PTan, SLin. Semantic colorization with Internet images. ACM Transactions on Graphics, 2011, 30(6): 156
https://doi.org/10.1145/2070781.2024190
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|