A general tail item representation enhancement framework for sequential recommendation

doi:10.1007/s11704-023-3112-y

Front. Comput. Sci.

2024, Vol. 18

Issue (6) : 186333 https://doi.org/10.1007/s11704-023-3112-y

RESEARCH ARTICLE

A general tail item representation enhancement framework for sequential recommendation

Mingyue CHENG¹, Qi LIU¹(

), Wenyu ZHANG¹, Zhiding LIU¹, Hongke ZHAO², Enhong CHEN¹

¹. Anhui Province Key Laboratory of Big Data Analysis and Application, University of Science and Technology of China & State Key Laboratory of Cognitive Intelligence, Hefei 230026, China
². College of Management and Economics, Tianjin University, Tianjin 300072, China

Download: PDF(12556 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

Recently advancements in deep learning models have significantly facilitated the development of sequential recommender systems (SRS). However, the current deep model structures are limited in their ability to learn high-quality embeddings with insufficient data. Meanwhile, highly skewed long-tail distribution is very common in recommender systems. Therefore, in this paper, we focus on enhancing the representation of tail items to improve sequential recommendation performance. Through empirical studies on benchmarks, we surprisingly observe that both the ranking performance and training procedure are greatly hindered by the poorly optimized tail item embeddings. To address this issue, we propose a sequential recommendation framework named TailRec that enables contextual information of tail item well-leveraged and greatly improves its corresponding representation. Given the characteristics of the sequential recommendation task, the surrounding interaction records of each tail item are regarded as contextual information without leveraging any additional side information. This approach allows for the mining of contextual information from cross-sequence behaviors to boost the performance of sequential recommendations. Such a light contextual filtering component is plug-and-play for a series of SRS models. To verify the effectiveness of the proposed TailRec, we conduct extensive experiments over several popular benchmark recommenders. The experimental results demonstrate that TailRec can greatly improve the recommendation results and speed up the training process. The codes of our methods have been available

See github.com/Mingyue-Cheng/TailRec website.

Keywords sequential recommendation long-tail distribution training accelerating

Corresponding Author(s): Qi LIU

Just Accepted Date: 04 July 2023 Issue Date: 27 September 2023

Cite this article:

Mingyue CHENG,Qi LIU,Wenyu ZHANG, et al. A general tail item representation enhancement framework for sequential recommendation[J]. Front. Comput. Sci., 2024, 18(6): 186333.

URL:

https://academic.hep.com.cn/fcs/EN/10.1007/s11704-023-3112-y
https://academic.hep.com.cn/fcs/EN/Y2024/V18/I6/186333

Fig.1 Evaluation results of NDCG@10 on three benchmark recommenders over three public datasets. (a) Evaluation on MovieLens; (b) evaluation on Yoochoose; (c) evaluation on Deginetica

Fig.2 Testing loss and NDCG of SASRec on Deginetica. (a) Testing loss on SASRec; (b) Testing NDCG on SASRec

Fig.3 (a) Base sequential recommender; (b) contextual representation enhanced sequential recommender framework

Tab.1 Statics of three public datasets after pre-processing

Tab.2 Performance comparison of all methods on sequential recommendation scenarios, where the best results are in bold and “*” denotes TailRec obtain gains over corresponding baselines

Fig.4 (a) Training curves of TailRec and its baseline in SRS, evaluated on MovieLens data using the metric of Recall@10; (b) training curves of TailRec and its baseline in SRS, evaluated on MovieLens data using the metric of NDCG@10

Tab.3 Comparison of sequential recommendation performance with respect to different evaluation scopes, where the upper and below tables are Recall@10, NDCG@10, respectively. The best results are noted in bold while “*” denotes TailRec can obtain gains on baseline networks

Fig.5 The impacts of updating manner for contextual embedding module. (a) Recall@10; (b) NDCG@10

Fig.6 The performance of GRU4Rec and NextItNet, both with and without enhanced representation on three datasets. (a) Testing Recall@10 on GRU4Rec; (b) Testing NDCG@10 on GRU4Rec; (c) Testing Recall@10 on NextItNet; (d) Testing NDCG@10 on NextItNet

Fig.7 Hyper-parameter sensitivity analysis of the proposed TailRec evaluated on SRS. (a) Trade-off γ; (b) trade-off α; (c) half window size s; (d) scope of enhanced item set

1	Wang S, Hu L, Wang Y, Cao L, Sheng Q Z, Orgun M A. Sequential recommender systems: challenges, progress and prospects. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence. 2019, 6332−6338
2	Wu S, Tang Y, Zhu Y, Wang L, Xie X, Tan T. Session-based recommendation with graph neural networks. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence and 31st Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence. 2019, 43
3	E, Xu Z, Yu N, Li H, Cui L, Yao B Guo . Quantifying predictability of sequential recommendation via logical constraints. Frontiers of Computer Science, 2023, 17( 5): 175612
4	X, Zhaok H, Liu W, Fan H, Liu J, Tang C, Wang M, Chen X, Zheng X, Liu X Yang . AutoEmb: automated embedding dimensionality search in streaming recommendations. In: Proceedings of 2021 IEEE International Conference on Data Mining. 2021, 896−905
5	M, Cheng Q, Liu Z, Liu Z, Li Y, Luo E Chen . FormerTime: hierarchical multi-scale representations for multivariate time series classification. In: Proceedings of the ACM Web Conference. 2023, 1437−1445
6	M, Cheng Q, Liu Z, Liu H, Zhang R, Zhang E Chen . TimeMAE: self-supervised representations of time series with decoupled masked autoencoders. 2023, arXiv preprint arXiv: 2303.00320
7	Y, Sun F, Yuan M, Yang G, Wei Z, Zhao D Liu . A generic network compression framework for sequential recommender systems. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2020, 1299−1308
8	L, Chen F, Yuan J, Yang X, Ao C, Li M Yang . A user-adaptive layer selection framework for very deep sequential recommender models. In: Proceedings of the AAAI Conference on Artificial Intelligence. 2021, 3984−3991
9	S, Zhang D, Yao Z, Zhao T S, Chua F Wu . CauseRec: counterfactual user sequence synthesis for sequential recommendation. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2021, 367−377
10	J, Yin C, Liu W, Wang J, Sun S C H Hoi . Learning transferrable parameters for long-tailed sequential user behavior modeling. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2020, 359−367
11	Y, Kim K, Kim C, Park H Yu . Sequential and diverse recommendation with long tail. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence. 2019, 2740−2746
12	Z, Fan Z, Liu J, Zhang Y, Xiong L, Zheng P S Yu . Continuous-time sequential recommendation with temporal graph collaborative transformer. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 2021, 433−442
13	G K Zipf . Human Behavior and the Principle of Least Effort: An Introduction to Human Ecology. Xue C F, trans. Shanghai: Shanghai People’s Publishing House, 2016
14	H, Yin B, Cui J, Li J, Yao C Chen . Challenging the long tail recommendation. Proceedings of the VLDB Endowment, 2012, 5( 9): 896–907
15	Z, Liu M, Cheng Z, Li Q, Liu E Chen . One person, one model-learning compound router for sequential recommendation. In: Proceedings of IEEE International Conference on Data Mining. 2022, 289−298
16	T, Wei F, Feng J, Chen Z, Wu J, Yi X He . Model-agnostic counterfactual reasoning for eliminating popularity bias in recommender system. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2021, 1791−1800
17	S, Liu Y Zheng . Long-tail session-based recommendation. In: Proceedings of the 14th ACM Conference on Recommender Systems. 2020, 509−514
18	Jang S, Lee H, Cho H, Chung S. CITIES: contextual inference of tail-item embeddings for sequential recommendation. In: Proceedings of the 20th IEEE International Conference on Data Mining. 2020, 202−211
19	W C, Kang J McAuley . Self-attentive sequential recommendation. In: Proceedings of 2018 IEEE International Conference on Data Mining. 2018, 197−206
20	S, Rendle C, Freudenthaler L Schmidt-Thieme . Factorizing personalized Markov chains for next-basket recommendation. In: Proceedings of the 19th International Conference on World Wide Web. 2010, 811−820
21	B, Hidasi A, Karatzoglou L, Baltrunas D Tikk . Session-based recommendations with recurrent neural networks. In: Proceedings of the 4th International Conference on Learning Representations. 2016
22	A, Vaswani N, Shazeer N, Parmar J, Uszkoreit L, Jones A N, Gomez Ł, Kaiser I Polosukhin . Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 6000−6010
23	M, Cheng Z, Liu Q, Liu S, Ge E Chen . Towards automatic discovering of deep hybrid network architecture for sequential recommendation. In: Proceedings of the ACM Web Conference 2022. 2022, 1923−1932
24	F, Sun J, Liu J, Wu C, Pei X, Lin W, Ou P Jiang . BERT4Rec: sequential recommendation with bidirectional encoder representations from transformer. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 2019, 1441−1450
25	M, Cheng F, Yuan Q, Liu S, Ge Z, Li R, Yu D, Lian S, Yuan E Chen . Learning recommender systems with implicit feedback via soft target enhancement. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2021, 575−584
26	F, Xue X, He X, Wang J, Xu K, Liu R Hong . Deep item-based collaborative filtering for top-N recommendation. ACM Transactions on Information Systems, 2019, 37( 3): 33
27	Y, Cai Z, Cui S, Wu Z, Lei X Ma . Represent items by items: An enhanced representation of the target item for recommendation. 2021, arXiv preprint arXiv: 2104.12483
28	K, He H, Fan Y, Wu S, Xie R Girshick . Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020, 9726−9735
29	Q, Liu Y, Zeng R, Mokhosi H Zhang . STAMP: short-term attention/memory priority model for session-based recommendation. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018, 1831−1839
30	Souza Pereira Moreira G, De S, Rabhi J M, Lee R, Ak E Oldridge . Transformers4Rec: bridging the gap between NLP and sequential/session-based recommendation. In: Proceedings of the 15th ACM Conference on Recommender Systems. 2021, 143−153
31	Tang J, Wang K. Personalized top-n sequential recommendation via convolutional sequence embedding. In: Proceedings of the 11th ACM International Conference on Web Search and Data Mining. 2018, 565−573
32	M, Cheng F, Yuan Q, Liu X, Xin E Chen . Learning transferable user representations with sequential behaviors via contrastive pre-training. In: Proceedings of 2021 IEEE International Conference on Data Mining (ICDM). 2021, 51−60
33	Li J, Ren P, Chen Z, Ren Z, Lian T, Ma J. Neural attentive session-based recommendation. In: Proceedings of 2017 ACM on Conference on Information and Knowledge Management. 2017, 1419−1428
34	Yuan F, Karatzoglou A, Arapakis I, Jose J M, He X. A simple convolutional generative network for next item recommendation. In: Proceedings of the 12th ACM International Conference on Web Search and Data Mining. 2019, 582−590
35	Rendle S, Freudenthaler C, Gantner Z, Schmidt-Thieme L. BPR: Bayesian personalized ranking from implicit feedback. In: Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. 2012, 452−461
36	Y, Zhang D Z, Cheng T, Yao X, Yi L, Hong E H Chi . A model of two tales: dual transfer learning framework for improved long-tail item recommendation. In: Proceedings of the Web Conference 2021. 2021, 2220−2231
37	Z, Zhao J, Chen S, Zhou X, He X, Cao F, Zhang W Wu . Popularity bias is not always evil: disentangling benign and harmful bias for recommendation. IEEE Transactions on Knowledge and Data Engineering, 2022, doi:
38	X, He T S Chua . Neural factorization machines for sparse predictive analytics. In: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2017, 355−364
39	Q, Pi G, Zhou Y, Zhang Z, Wang L, Ren Y, Fan X, Zhu K Gai . Search-based user interest modeling with lifelong sequential behavior data for click-through rate prediction. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2020, 2685−2692
40	R, He C, Fang Z, Wang J McAuley . Vista: a visually, socially, and temporally-aware model for artistic recommendation. In: Proceedings of the 10th ACM Conference on Recommender Systems. 2016, 309–316
41	H, Ying F, Zhuang F, Zhang Y, Liu G, Xu X, Xie H, Xiong J Wu . Sequential recommender system based on hierarchical attention network. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence. 2018, 3926–3932
42	Gu Y, Lei T, Barzilay R, Jaakkola T. Learning to refine text based recommendations. In: Proceedings of 2016 Conference on Empirical Methods in Natural Language Processing. 2016, 2103−2108
43	Y K, Tan X, Xu Y Liu . Improved recurrent neural networks for session-based recommendations. In: Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. 2016, 17−22
44	J, Huang W X, Zhao H, Dou J R, Wen E Y Chang . Improving sequential recommendation with knowledge-enhanced memory networks. In: Proceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 2018, 505−514
45	J, Wang F, Yuan J, Chen Q, Wu M, Yang Y, Sun G Zhang . StackRec: efficient training of very deep sequential recommender models by iterative stacking. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2021, 357−366
46	Y, Liang Q, Song Z, Zhao H, Zhou M Gong . BA-GNN: Behavior-aware graph neural network for session-based recommendation. Frontiers of Computer Science, 2023, 17( 6): 176613
47	E, Brynjolfsson Y J, Hu M D Smith . From niches to riches: anatomy of the long tail. Sloan Management Review, 2006, 47( 4): 67–71
48	D, Liang L, Charlin D M Blei . Causal inference for recommendation. In: Proceedings of Causation: Foundation to Application, Workshop at UAI. 2016
49	Abdollahpouri H, Burke R, Mobasher B. Controlling popularity bias in learning-to-rank recommendation. In: Proceedings of the 11th ACM Conference on Recommender Systems. 2017, 42−46
50	G, Adomavicius Y Kwon . Improving aggregate recommendation diversity using ranking-based techniques. IEEE Transactions on Knowledge and Data Engineering, 2012, 24( 5): 896–911
51	B, Bai Y, Fan W, Tan J Zhang . DLTSR: a deep learning framework for recommendations of long-tail web services. IEEE Transactions on Services Computing, 2020, 13( 1): 73–85
52	J, Li K, Lu Z, Huang H T Shen . Two birds one stone: on both cold-start and long-tail recommendation. In: Proceedings of the 25th ACM International Conference on Multimedia. 2017, 898−906

[1]

FCS-23112-OF-MC_suppl_1

Download

[1]	En XU, Zhiwen YU, Nuo LI, Helei CUI, Lina YAO, Bin GUO. Quantifying predictability of sequential recommendation via logical constraints[J]. Front. Comput. Sci., 2023, 17(5): 175612-.
[2]	Mingdi HU, Long BAI, Jiulun FAN, Sirui ZHAO, Enhong CHEN. Vehicle color recognition based on smooth modulation neural network with multi-scale feature fusion[J]. Front. Comput. Sci., 2023, 17(3): 173321-.

Viewed

Full text

Abstract

Cited

Shared

Discussed