How graph convolutions amplify popularity bias for recommendation?

Please wait a minute...

Shopping Cart Email List Chinese

Advanced Search Fig/Tab

Frontiers of Computer Science

ISSN 2095-2228

ISSN 2095-2236(Online)

CN 10-1014/TP

Postal Subscription Code 80-970

2018 Impact Factor: 1.129

Featured Articles | Most Downloaded | Most Reading

Online Submission Free E-mail Alert Subscribe to Our RSS Content Feed

Front. Comput. Sci.

2024, Vol. 18

Issue (5) : 185603 https://doi.org/10.1007/s11704-023-2655-2

RESEARCH ARTICLE

How graph convolutions amplify popularity bias for recommendation?

Jiajia CHEN¹, Jiancan WU¹, Jiawei CHEN², Xin XIN³, Yong LI⁴, Xiangnan HE¹(

)

¹. School of Information Science and Technology, University of Science and Technology of China, Hefei 230026, China
². School of Computer Science and Technology, Zhejiang University, Hangzhou 310058, China
³. School of Computer Science and Technology, Shandong University, Qingdao 250100, China
⁴. Department of Electronic Engineering, Tsinghua University, Beijing 100084, China

Download: PDF(4510 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

Graph convolutional networks (GCNs) have become prevalent in recommender system (RS) due to their superiority in modeling collaborative patterns. Although improving the overall accuracy, GCNs unfortunately amplify popularity bias — tail items are less likely to be recommended. This effect prevents the GCN-based RS from making precise and fair recommendations, decreasing the effectiveness of recommender systems in the long run.

In this paper, we investigate how graph convolutions amplify the popularity bias in RS. Through theoretical analyses, we identify two fundamental factors: (1) with graph convolution (i.e., neighborhood aggregation), popular items exert larger influence than tail items on neighbor users, making the users move towards popular items in the representation space; (2) after multiple times of graph convolution, popular items would affect more high-order neighbors and become more influential. The two points make popular items get closer to almost users and thus being recommended more frequently. To rectify this, we propose to estimate the amplified effect of popular nodes on each node’s representation, and intervene the effect after each graph convolution. Specifically, we adopt clustering to discover highly-influential nodes and estimate the amplification effect of each node, then remove the effect from the node embeddings at each graph convolution layer. Our method is simple and generic — it can be used in the inference stage to correct existing models rather than training a new model from scratch, and can be applied to various GCN models. We demonstrate our method on two representative GCN backbones LightGCN and UltraGCN, verifying its ability in improving the recommendations of tail items without sacrificing the performance of popular items. Codes are open-sourced

See github.com/MEICRS/DAP website.

.

Keywords recommendation graph convolution networks popularity bias

Corresponding Author(s): Xiangnan HE

Just Accepted Date: 30 May 2023 Issue Date: 31 July 2023

Cite this article:

Jiajia CHEN,Jiancan WU,Jiawei CHEN, et al. How graph convolutions amplify popularity bias for recommendation?[J]. Front. Comput. Sci., 2024, 18(5): 185603.

URL:

https://academic.hep.com.cn/fcs/EN/10.1007/s11704-023-2655-2
https://academic.hep.com.cn/fcs/EN/Y2024/V18/I5/185603

Fig.1 Performance change of LightGCN with different graph convolution layers on Gowalla. Recall@20 and TR@20 stand for the overall recall score and the ratio of tail items in the top-20 recommendation list, respectively

$U$ , $I$	User set, item set
$N u$ , $N i$	The one-order neighbors of user $u$ or item $i$
$d u$ , $d i$	The degree of user $u$ or item $i$
$e u (l)$ , $e i (l)$	The embedding of user $u$ or item $i$ at the $l$ th graph convolution layer
$L u i$	The individual loss term of an interaction $(u, i)$
$C p (l)$	A set of nodes in the $p$ -th cluster obtained by using Kmeans given the embeddings $E (l)$
$H v (l)$	A set of nodes ${j ∈ C p (l), d j > d v \| v ∈ C p (l)}$
$L v (l)$	A set of nodes ${j ∈ C p (l), d j < d v \| v ∈ C p (l)}$
$?^H v (l)$	The pooling representations after normalization of $H v (l)$
$?^L v (l)$	The pooling representations after normalization of $L v (l)$

Tab.1 Main notations used in the paper

Fig.2 Average

‖ θ ‖

and

d 32 ‖ θ ‖

in each items group. Items are sorted into groups in ascending order of their degrees. (a) Gowalla; (b) Yelp2018; (c) Amazon-book

Fig.3 The average aggregation weight of one-order neighbor users in each items group. Items are sorted into groups in ascending order of their degrees. (a) Gowalla; (b) Yelp2018; (c) Amazon-book

Tab.2 Dataset description

Tab.3 Performance comparison between our method DAP and other counterparts on the Overall and Tail test sets. The “improve” is the relative improvement of LightGCN-DAP-o over LightGCN

Tab.4 Performance comparison between our method DAP and other counterparts on the Overall and Tail test sets. The “improve” is the relative improvement of UltraGCN-DAP-o over UltraGCN

Fig.4 Performance comparison between LightGCN and LightGCN-DAP with different layers of graph convolution on the Overall test set. (a) Gowalla Recall; (b) Gowalla NDCG; (c) Amazon-book Recall; (d) Amazon-book NDCG

Fig.5 Ablation study of DAP with different hyper-parameters

α

and

β

on the Overall test set. (a) Gowalla Recall; (b) Yelp2018 Recall; (c) Amazon-book Recall; (d) Gowalla NDCG; (e) Yelp2018 NDCG; (f) Amazon-book NDCG

Tab.5 Effect of

P

over DAP for the three datasets on the Overall test set

Fig.6 DAP can effectively alleviate the popularity bias. (a) Gowalla; (b) Yelp2018; (c) Amazon-book

1	L, Wu X, He X, Wang K, Zhang M Wang . A survey on accuracy-oriented neural recommendation: From collaborative filtering to information-rich recommendation. IEEE Transactions on Knowledge and Data Engineering, 2023, 35( 5): 4425–4445
2	J, Chen H, Dong X, Wang F, Feng M, Wang X He . Bias and debias in recommender system: a survey and future directions. ACM Transactions on Information Systems, 2023, 41( 3): 67
3	X, Wang X, He M, Wang F, Feng T S Chua . Neural graph collaborative filtering. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2019, 165–174
4	L, Wu P, Sun Y, Fu R, Hong X, Wang M Wang . A neural influence diffusion model for social recommendation. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2019, 235−244
5	F, Wu Jr A H, Souza T, Zhang C, Fifty T, Yu K Q Weinberger . Simplifying graph convolutional networks. In: Proceedings of the 36th International Conference on Machine Learning. 2019, 6861–6871
6	X, He K, Deng X, Wang Y, Li Y, Zhang M Wang . LightGCN: simplifying and powering graph convolution network for recommendation. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2020, 639–648
7	F, Liu Z, Cheng L, Zhu Z, Gao L Nie . Interest-aware message-passing GCN for recommendation. In: Proceedings of the Web Conference 2021. 2021, 1296−1305
8	Rendle S, Freudenthaler C, Gantner Z, Schmidt-Thieme L. BPR: Bayesian personalized ranking from implicit feedback. In: Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. 2009, 452−461
9	T, Wei F, Feng J, Chen Z, Wu J, Yi X He . Model-agnostic counterfactual reasoning for eliminating popularity bias in recommender system. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2021, 1791−1800
10	Y, Zheng C, Gao X, Li X, He Y, Li D Jin . Disentangling user interest and conformity for recommendation with causal embedding. In: Proceedings of the Web Conference 2021. 2021, 2980−2991
11	Y, Zhang F, Feng X, He T, Wei C, Song G, Ling Y Zhang . Causal intervention for leveraging popularity bias in recommendation. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2021, 11−20
12	Liang D, Charlin L, Blei D M. Causal inference for recommendation. In: Proceedings of the 32nd Conference on Uncertainty in Artificial Intelligence Workshop on Causation: Foundation to Application. 2016
13	Abdollahpouri H, Burke R, Mobasher B. Managing popularity bias in recommender systems with personalized re-ranking. In: Proceedings of the 32nd International Florida Artificial Intelligence Research Society Conference. 2019, 413−418
14	K, Mao J, Zhu X, Xiao B, Lu Z, Wang X He . UltraGCN: ultra simplification of graph convolutional networks for recommendation. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 2021, 1253−1262
15	T N, Kipf M Welling . Semi-supervised classification with graph convolutional networks. In: Proceedings of the 5th International Conference on Learning Representations. 2017
16	Johnson C C. Logistic matrix factorization for implicit feedback data. In: Proceedings of Advances in Neural Information Processing Systems. 2014
17	P W, Koh P Liang . Understanding black-box predictions via influence functions. In: Proceedings of the 34th International Conference on Machine Learning. 2017, 1885−1894
18	K, Xu C, Li Y, Tian T, Sonobe K I, Kawarabayashi S Jegelka . Representation learning on graphs with jumping knowledge networks. In: Proceedings of the 35th International Conference on Machine Learning. 2018, 5453−5462
19	X, Tang H, Yao Y, Sun Y, Wang J, Tang C, Aggarwal P, Mitra S Wang . Investigating and mitigating degree-related biases in graph convoltuional networks. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2020, 1435–1444
20	J, Wu X, Wang X, Gao J, Chen H, Fu T, Qiu X He . On the effectiveness of sampled softmax loss for item recommendation. 2022, arXiv preprint arXiv: 2201.02327
21	Z, Zhao J, Chen S, Zhou X, He X, Cao F, Zhang W Wu . Popularity bias is not always evil: disentangling benign and harmful bias for recommendation. IEEE Transactions on Knowledge and Data Engineering, 2022, doi:
22	M, Zhao Q, Deng K, Wang R, Wu J, Tao C, Fan L, Chen P Cui . Bilateral filtering graph convolutional network for multi-relational social recommendation in the power-law networks. ACM Transactions on Information Systems, 2021, 40( 2): 31
23	Gruson A, Chandar P, Charbuillet C, McInerney J, Hansen S, Tardieu D, Carterette B. Offline evaluation to make decisions about playlistrecommendation algorithms. In: Proceedings of the 12th ACM International Conference on Web Search and Data Mining. 2019, 420−428
24	Schnabel T, Swaminathan A, Singh A, Chandak N, Joachims T. Recommendations as treatments: debiasing learning and evaluation. In: Proceedings of the 33rd International Conference on Machine Learning. 2016, 1670−1679
25	S, Bonner F Vasile . Causal embeddings for recommendation. In: Proceedings of the 12th ACM Conference on Recommender Systems. 2018, 104−112
26	Z, Liu T K, Nguyen Y Fang . Tail-GNN: tail-node graph neural networks. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2021, 1109−1119
27	D P, Kingma J Ba . Adam: a method for stochastic optimization. In: Proceedings of the 3rd International Conference on Learning Representations. 2015
28	Z, Zhu Y, He X, Zhao J Caverlee . Popularity bias in dynamic recommendation. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2021, 2439−2449
29	G, Xv C, Lin H, Li J, Su W, Ye Y Chen . Neutralizing popularity bias in recommendation models. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2022, 2623−2628
30	P, Gupta A, Sharma P, Malhotra L, Vig G Shroff . CauSeR: causal session-based recommendations for handling popularity bias. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 2021, 3048−3052
31	S, Gupta H, Wang Z, Lipton Y Wang . Correcting exposure bias for link recommendation. In: Proceedings of the 38th International Conference on Machine Learning. 2021, 3953−3963
32	H, Guo J, Yu Q, Liu R, Tang Y Zhang . PAL: a position-bias aware learning framework for CTR prediction in live recommender systems. In: Proceedings of the 13th ACM Conference on Recommender Systems. 2019, 452−456
33	M, Dudík J, Langford L Li . Doubly robust policy evaluation and learning. In: Proceedings of the 28th International Conference on Machine Learning. 2011, 1097−1104
34	A, Krishnan A, Sharma A, Sankar H Sundaram . An adversarial approach to improve long-tail performance in neural collaborative filtering. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management. 2018, 1491−1494
35	J, Chen H, Dong Y, Qiu X, He X, Xin L, Chen G, Lin K Yang . AutoDebias: learning to debias for recommendation. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2021, 21−30
36	Y, Zhang D Z, Cheng T, Yao X, Yi L, Hong E H Chi . A model of two tales: dual transfer learning framework for improved long-tail item recommendation. In: Proceedings of the Web Conference 2021. 2021, 2220−2231
37	Abdollahpouri H, Burke R, Mobasher B. Controlling popularity bias in learning-to-rank recommendation. In: Proceedings of the 11th ACM Conference on Recommender Systems. 2017, 42−46
38	Y, Wang D, Liang L, Charlin D M Blei . The deconfounded recommender: a causal inference approach to recommendation. 2019, arXiv preprint arXiv: 1808.06581
39	M, Sato S, Takemori J, Singh T Ohkuma . Unbiased learning for the causal effect of recommendation. In: Proceedings of the 14th ACM Conference on Recommender Systems. 2020, 378−387
40	R, Qiu S, Wang Z, Chen H, Yin Z Huang . CausalRec: causal inference for visual debiasing in visually-aware recommendation. In: Proceedings of the 29th ACM International Conference on Multimedia. 2021, 3844–3852
41	W, Wang F, Feng X, He X, Wang T S Chua . Deconfounded recommendation for alleviating bias amplification. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2021, 1717−1725
42	H, Dai H, Li T, Tian X, Huang L, Wang J, Zhu L Song . Adversarial attack on graph structured data. In: Proceedings of the 35th International Conference on Machine Learning. 2018, 1115−1124
43	K, Xu W, Hu J, Leskovec S Jegelka . How powerful are graph neural networks? In: Proceedings of the 7th International Conference on Learning Representations. 2018
44	Q, Li Z, Han X M Wu . Deeper insights into graph convolutional networks for semi-supervised learning. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018, 3538−3545
45	D, Chen Y, Lin W, Li P, Li J, Zhou X Sun . Measuring and relieving the over-smoothing problem for graph neural networks from the topological view. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020, 3438−3445
46	M, Chen Z, Wei Z, Huang B, Ding Y Li . Simple and deep graph convolutional networks. In: Proceedings of the 37th International Conference on Machine Learning. 2020, 1725−1735
47	M, Liu H, Gao S Ji . Towards deeper graph neural networks. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2020, 338−348
48	K, Sun Z, Lin Z Zhu . Multi-stage self-supervised learning for graph convolutional networks on graphs with few labeled nodes. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020, 5892−5899

[1]

FCS-22655-OF-JC_suppl_1

Download

[1]	Yi ZHU, Yishuai GENG, Yun LI, Jipeng QIANG, Xindong WU. Representation learning: serial-autoencoder for personalized recommendation[J]. Front. Comput. Sci., 2024, 18(4): 184316-.
[2]	Yongquan LIANG, Qiuyu SONG, Zhongying ZHAO, Hui ZHOU, Maoguo GONG. BA-GNN: Behavior-aware graph neural network for session-based recommendation[J]. Front. Comput. Sci., 2023, 17(6): 176613-.
[3]	En XU, Zhiwen YU, Nuo LI, Helei CUI, Lina YAO, Bin GUO. Quantifying predictability of sequential recommendation via logical constraints[J]. Front. Comput. Sci., 2023, 17(5): 175612-.
[4]	Jinwei LUO, Mingkai HE, Weike PAN, Zhong MING. BGNN: Behavior-aware graph neural network for heterogeneous session-based recommendation[J]. Front. Comput. Sci., 2023, 17(5): 175336-.
[5]	Xiao PAN, Lei WU, Fenjie LONG, Ang MA. Exploiting user behavior learning for personalized trajectory recommendations[J]. Front. Comput. Sci., 2022, 16(3): 163610-.
[6]	Ming HE, Hao GUO, Guangyi LV, Le WU, Yong GE, Enhong CHEN, Haiping MA. Leveraging proficiency and preference for online Karaoke recommendation[J]. Front. Comput. Sci., 2020, 14(2): 273-290.
[7]	Zhiqian ZHANG, Chenliang LI, Zhiyong WU, Aixin SUN, Dengpan YE, Xiangyang LUO. NEXT: a neural network framework for next POI recommendation[J]. Front. Comput. Sci., 2020, 14(2): 314-333.
[8]	Jia HE, Fuzhen ZHUANG, Yanchi LIU, Qing HE, Fen LIN. Bayesian dual neural networks for recommendation[J]. Front. Comput. Sci., 2019, 13(6): 1255-1265.
[9]	Min GAO, Bin LING, Linda YANG, Junhao WEN, Qingyu XIONG, Shun LI. From similarity perspective: a robust collaborative filtering approach for service recommendations[J]. Front. Comput. Sci., 2019, 13(2): 231-246.
[10]	Xiaobing SUN, Hui YANG, Hareton LEUNG, Bin LI, Hanchao (Jerry) LI, Lingzhi LIAO. Effectiveness of exploring historical commits for developer recommendation: an empirical study[J]. Front. Comput. Sci., 2018, 12(3): 528-544.
[11]	Yaxin YU, Yuhai ZHAO, Ge YU, Guoren WANG. Mining coterie patterns from Instagram photo trajectories for recommending popular travel routes[J]. Front. Comput. Sci., 2017, 11(6): 1007-1022.
[12]	Rui LIU, Wenge RONG, Yuanxin OUYANG, Zhang XIONG. A hierarchical similarity based job recommendation service framework for university students[J]. Front. Comput. Sci., 2017, 11(5): 912-922.
[13]	Wendi JI, Xiaoling WANG, Feida ZHU. Time-aware conversion prediction[J]. Front. Comput. Sci., 2017, 11(4): 702-716.
[14]	Hua MA,Zhigang HU. Recommend trustworthy services using interval numbers of four parameters via cloud model for potential users[J]. Front. Comput. Sci., 2015, 9(6): 887-903.
[15]	Jianjun YU,Tongyu ZHU. Combining long-term and short-term user interest for personalized hashtag recommendation[J]. Front. Comput. Sci., 2015, 9(4): 608-622.

Viewed

Full text

Abstract

Cited

Shared

Discussed