Survey of visual sentiment prediction for social media analysis

doi:10.1007/s11704-016-5453-2

Front. Comput. Sci.

0, Vol.

Issue () : 602-611 https://doi.org/10.1007/s11704-016-5453-2

REVIEW ARTICLE

Survey of visual sentiment prediction for social media analysis

Rongrong JI^1,²,Donglin CAO^1,^2,^*(

),Yiyi ZHOU^1,²,Fuhai CHEN^1,²

¹. Cognitive Science Department, Xiamen University, Xiamen 361005, China
². Fujian Key Laboratory of the Brain-like Intelligent Systems, Xiamen 361005, China

Download: PDF(760 KB)
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

Recent years have witnessed a rapid spread of multi-modality microblogs like Twitter and SinaWeibo composed of image, text and emoticon. Visual sentiment prediction of such microblog based social media has recently attracted ever-increasing research focus with broad application prospect. In this paper, we give a systematic review of the recent advances and cutting-edge techniques for visual sentiment analysis. To this end, in this paper we review the most recent works in this topic, in which detailed comparison as well as experimental evaluation are given over the cuttingedge methods.We further reveal and discuss the future trends and potential directions for visual sentiment prediction.

Keywords visual sentiment analysis sentiment predication human emotion

Corresponding Author(s): Donglin CAO

Just Accepted Date: 31 March 2016 Online First Date: 06 July 2016 Issue Date: 06 July 2016

Cite this article:

Rongrong JI,Donglin CAO,Yiyi ZHOU, et al. Survey of visual sentiment prediction for social media analysis[J]. Front. Comput. Sci., 0, (): 602-611.

URL:

https://academic.hep.com.cn/fcs/EN/10.1007/s11704-016-5453-2
https://academic.hep.com.cn/fcs/EN/Y0/V/I/602

1	Picard R W. Affective Computing. MIT Technical Report. 1995
2	Pang B, Lee L, Vaithyanathan S. Thumbs up?: sentiment classification using machine learning techniques. In: Proceedings of the ACL- 02 Conference on Empirical Methods in Natural Language Processing. 2002, 79–86Wilson T, Wiebe J, Hwa R. Just how mad are you? Finding strong and weak opinion clauses. In: Proceedings of National Conference on Artificial Intelligence. 2004, 761–767
3	Zhou L, Li B, Gao W, Wei Z, Wong K F. Unsupervised discovery of discourse relations for eliminating intra-sentence polarity ambiguities. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2011, 162–171
4	Hu M Q, Liu B. Mining and summarizing customer reviews. In: Proceedings of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2004, 168–177 https://doi.org/10.1145/1014052.1014073
5	Wilson T, Wiebe J, Hoffmann P. Recognizing contextual polarity in phrase-level sentiment analysis. In: Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing. 2005, 347–354 https://doi.org/10.3115/1220575.1220619
6	Liu J J, Cao Y B, Lin C Y, Huang Y L, Zhou M. Low-quality product review detection in sentiment summarization. In: Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. 2007
7	McGlohon M, Glance N S, Reiter Z. Star quality: aggregating reviews to rank products and merchants. In: Proceedings of the International Conference on Weblogs and Social Media. 2010
8	Joshi M, Das D, Gimpel K, Smith N A. Movie reviews and revenues: an experiment in text regression. In: Proceedings of the 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics. 2010, 293–296
9	Feldman R, Rosenfeld B, Bar-Haim R, Fresko M. The stock sonar— sentiment analysis of stocks based on a hybrid approach. In: Proceedings of the 23rd IAAI Conference on Artificial Intelligence. 2011
10	Zhang W B, Skiena S. Trading strategies to exploit blog and news sentiment. In: Proceedings of the International Conference on Weblogs and Social Media. 2010
11	Tumasjan A, Sprenger T O, Sandner P G, Welpe I M. Predicting elections with twitter: what 140 characters reveal about political sentiment. In: Proceedings of the International Conference onWeblogs and Social Media. 2010, 178–185
12	Chen B, Leilei Z, Daniel K, Dongwon L. What is an sentiment about? exploring political standpoints using sentiment scoring model. In: Proceeedings of AAAI Conference on Artificial Intelligence. 2010
13	Yano T, Smith N A. What’s worthy of comment? content and comment volume in political blogs. In: Proceedings of the International AAAI Conference on Weblogs and Social Media. 2010
14	Groh G, Hauffa J. Characterizing social relations via NLP-based sentiment analysis. In: Proceedings of the 5th International AAAI Conference on Weblogs and Social Media. 2011
15	Li B, Feng S, Xiong W, Hu W. Scaring or pleasing: exploit emotional impact of an image. In: Proceedings of the 20th ACM International Conference on Multimedia. 2012, 1365–1366 https://doi.org/10.1145/2393347.2396487
16	Jia J, Wu S, Wang X H, Hu P Y, Cai L H, Tang J. Can we understand van Gogh’s mood?: learning to infer affects from images in social networks. In: Proceedings of the 20th ACM International Conference on Multimedia. 2012, 857–860 https://doi.org/10.1145/2393347.2396330
17	Vonikakis V, Winkler S. Emotion-based sequence of family photos. In: Proceedings of the 20th ACM International Conference onMultimedia. 2012, 1371–1372 https://doi.org/10.1145/2393347.2396490
18	Siersdorfer S, Minack E, Deng F, Hare J. Analyzing and predicting sentiment of images on the social Web. In: Proceedings of the 18th ACM International Conference on Multimedia. 2010, 715–718 https://doi.org/10.1145/1873951.1874060
19	Borth D, Ji R, Chen T, Breuel T, Chang S F. Large-scale visual sentiment ontology and detectors using adjective noun pairs. In: Proceedings of the 21st ACM International Conference on Multimedia. 2013, 223–232 https://doi.org/10.1145/2502081.2502282
20	Yuan J, Mcdonough S, You Q, Luo J. Sentribute: image sentiment analysis from a mid-level perspective. In: Proceedings of the 2nd International Workshop on Issues of Sentiment Discovery and Opinion Mining. 2013 https://doi.org/10.1145/2502069.2502079
21	Cao D L, Ji R R, Lin A Z, Li S Z. Visual sentiment topic model based microblog image sentiment analysis. Multimedia Tools and Applications, 2014: 1–14 https://doi.org/10.1007/s11042-014-2337-z
22	Ji R R, Yao H X, Liu W, Sun X S, Tian Q. Task dependent visual codebook compression, IEEE Transactions on Image Processing, 2012, 21(4): 2282–2293 https://doi.org/10.1109/TIP.2011.2176950
23	Ji R R, Duan L Y, Yao H X, Xie L X, Rui Y, Gao W. Learning to distribute vocabulary indexing for scalable visual search. IEEE Transactions on Multimedia, 2013, 15(1): 153–166 https://doi.org/10.1109/TMM.2012.2225035
24	Ji R R, Gao Y, Liu W, Xie X, Tian Q, Li X L. When location meets social multimedia: a comprehensive survey on location-aware social multimedia. ACM Transactions on Intelligent System and Technology, 2015, 6(1): 1–18 https://doi.org/10.1145/2597181
25	Ji R R, Yao H X, Tian Q, Xu P F, Sun X S, Liu X M, Context-aware semi-local feature detector. ACM Transactions on Intelligent System and Technology, 2012, 3(3): 44–71 https://doi.org/10.1145/2168752.2168758
26	Ji R R, Yu F X, Chang S F. Active query sensing for mobile location search. ACM Transactions on Multimedia Computing, Communications and Applications, 2012, 3S(8): 40–61
27	Ji R R, Gao Y, Zhong B N, Yao H X, Tian Q. Mining flickr landmarks bymodeling reconstruction sparsity. ACM Transactions onMultimedia Computing, Communications and Applications, 2011, 7S(1)
28	Ji R R, Duan L Y, Chen J, Huang T J, Gao W. Mining compact 3D patterns for low bit rate mobile visual search. IEEE Transactions on Image Processing, 2014, 23(7): 3099–3113 https://doi.org/10.1109/TIP.2014.2324291
29	You Q Z, Luo J B, Jin H L, Yang J C. Robust image sentiment analysis using progressively trained and domain transferred deep networks. 2015, arXiv:1509.06041
30	Chen T, Yu F X, Chen J, Cui Y, Chen Y Y, Chang S F. Object-based visual sentiment concept analysis and application. In: Proceedings of the ACM International Conference on Multimedia. 2014, 367–376. https://doi.org/10.1145/2647868.2654935
31	Li L X, Cao D L, Li S Z, Ji R R. Sentiment analysis of Chinese micro-blog based on multi-modal correlation model. In: Proceedings of the 2015 IEEE International Conference on Image Processing. 2015, 4798–4802 https://doi.org/10.1109/ICIP.2015.7351718
32	Chen F H, Gao Y, CAO D L. Multimodal hypergraph learning for microblog sentiment prediction. In: Proceedings of the 2015 IEEE International Conference on Multimedia and Expo. 2015, 1–6 https://doi.org/10.1109/ICME.2015.7177447
33	Wang M, Cao D L, Li L X, Li S Z, Ji R R. Microblog sentiment analysis based on cross-media bag-of-words model. In: Proceedings of International Conference on Internet Multimedia Computing and Service. 2014 https://doi.org/10.1145/2632856.2632912
34	You Q Z, Luo J B, Jin H L, Yang J C. Cross-modality consistent regression for joint visual-textual sentiment analysis. In: Proceedings of the 9th ACM International Conference on Web Search and Data Mining. 2016, 13–22 https://doi.org/10.1145/2835776.2835779
35	Zhong B N, Yao H X, Chen S, Ji R R, Chin T J, Wang H Z. Visual tracking via weakly supervised learning from multiple imperfect oracles. Pattern Recognition, 2014, 47(3): 1395–1410 https://doi.org/10.1016/j.patcog.2013.10.002
36	Gao Y, Wang F L, Luan H B, Chua T S. Brand data gathering from live social media streams. In: Proceedings of ACM International Conference on Multimedia Retrieval. 2014 https://doi.org/10.1145/2578726.2578748
37	Gao Y, Wang M, Zha Z J, Shen J L, Li X L, Wu X D. Visual-textual joint relevance learning for tag-based social image search. IEEE Transactions on Image Processing, 2013, 22(1): 363–376 https://doi.org/10.1109/TIP.2012.2202676
38	Gao Y, Wang M, Zha Z J, Tian Q, Dai Q H, Zhang N Y. Less is more: efficient 3-D object retrieval with query view selection. IEEE Transactions on Multimedia, 2011, 13(5): 1007–1018 https://doi.org/10.1109/TMM.2011.2160619
39	Gao Y, Tang J H, Hong R C, Yan S C, Dai Q H, Zhang N Y, Chua T S. Camera constraint-free view-based 3-D object retrieval. IEEE Transactions on Image Processing, 2012, 21(4): 2269–2281 https://doi.org/10.1109/TIP.2011.2170081
40	Huang Y C, Liu Q S, Zhang S T, Metaxas D N. Image retrieval via probabilistic hypergraph ranking. In: Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition. 2010, 3376–3383 https://doi.org/10.1109/CVPR.2010.5540012
41	Gao Y, Wang M, Zha Z J, Shen J L, Li X L,Wu X D. Visual-textual joint relevance learning for tag-based social image search. IEEE Transactions on Image Processing, 2013, 22(1): 363–376 https://doi.org/10.1109/TIP.2012.2202676
42	Cai Z, Cao D L, Ji R R. Video (GIF) sentiment analysis using largescale mid-level ontology. 2015, arXiv:1506.00765
43	Wang Y L, Wang S H, Tang J L, Huan Liu H, Li B X. Unsupervised sentiment analysis for social media images. In: Proceedings of the 24th International Conference on Artificial Intelligence. 2015, 2378–2379
44	Narihira T, Borth D, Yu S X, Ni K, Darrell T. Mapping images to sentiment adjective noun pairs with factorized neural nets. 2015, arXiv preprint arXiv:1511.06838

[1]

FCS-0602-15453-DLC_suppl_1

Download

Viewed

Full text

Abstract

Cited

Shared

Discussed