Adaptive Knowledge Contrastive Learning with Dynamic Attention for Recommender Systems
Abstract
:1. Introduction
- An adaptive data augmentation strategy is proposed to expand semantic associations and user behaviors in knowledge and interaction graphs by adding hypothetical edges. This strategy effectively leverages sparse information, thereby ensuring contrastive views are more representative of underlying data relationships.
- A personalized dynamic attention network is proposed to bridge the gap between individual interests and general knowledge by dynamically adjusting user attention based on behaviors, alleviating the mismatch found in existing methods.
- Extensive experiments are conducted on the Yelp2018, LastFM, and MovieLens datasets, and the proposed AKCL-DA method shows significant improvements over state-of-the-art recommendation methods.
2. Related Research
2.1. Knowledge-Aware Recommendation
2.2. Contrastive Learning
3. Preliminaries
4. Methodology
4.1. Adaptive Data Augmentation
4.2. Personalized Dynamic Attention Network
4.3. Adaptive Contrastive Learning
4.4. Model Prediction
4.5. Multi-Task Training
5. Experiment
5.1. Experimental Settings
5.1.1. Datasets
- Yelp2018: It mostly makes recommendations for merchants, including eateries, pubs, and their locations. Users and items with fewer than ten interactions were removed from the dataset using a 10-core setting.
- LastFM: These data are collected from the LastFM online music system, containing the listening history of 23,566 users.
- MovieLens: It is a movie recommendation benchmark dataset containing interaction records from 37,385 users.
5.1.2. Baselines
- BPRMF [5]: It enhances pairwise matrix factorization for implicit feedback by optimizing through Bayesian Personalized Ranking (BPR) loss, enabling the learning of implicit feature representations between users and items.
- FM [21]: Every characteristic is represented as a latent vector, and the inner product of the latent vectors is then calculated to determine the interaction between the features
- NFM [42]: It represents features as latent vectors and learns the interactions between these latent vectors through neural networks.
- CKE [12]: It is an embedding-based method that uses TransR to encode item semantic information and combines collaborative filtering and item knowledge embedding.
- KGCN [10]: It is a GCN-based method, which addresses the high-order dependent context encoding of semantic information in KG.
- LightGCN [7]: By simplifying the graph neural network structure, a lightweight recommendation method based on GCN learns the embedding vectors of users and items.
- KGAT [20]: A propagation-based approach builds a collaborative knowledge graph (CKG) using KG and user–item graphs as building blocks and employs an attention mechanism to generate user and item representations.
- CKAN [43]: A propagation-based method combines collaborative filtering representation space with knowledge graph embedding to further expand KGCN by taking into account the user–item graph and utilizing various neighborhood aggregation algorithms.
- KGIN [31]: A cutting-edge GNN-based method that uses auxiliary item information to try and understand the meaning underlying user–item interactions and gives recommendation interpretability.
- KGCL [44]: A state-of-the-art recommendation system based on contrastive learning proposes a knowledge graph augmentation approach to mitigate KG noise during information aggregation, resulting in a more dependable representation of item knowledge awareness.
5.1.3. Evaluation Metrics
5.1.4. Parameter Settings
5.2. Performance Comparison
5.3. Sensitivity Analysis
5.3.1. Effect of Hypothesis Edge Ratio
5.3.2. Effect of Layer Number of DAn
5.3.3. Effect of
5.3.4. Effect of Edge Dropout
5.4. Ablation Studies
- AKCL-DAw/o Assumed edge: In this variant, we do not add hypothetical edges to the interaction graph and keep other parts consistent with the full model, observing the impact of the results on the entire model.
- AKCL-DAw/o Att: This variant does not use dynamic attention to calculate the importance between nodes but uses the original version of GAT to calculate the attention score.
5.5. Benefits of AKCL-DA in Alleviating Knowledge Noise
6. Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Hu, B.; Shi, C.; Zhao, W.X.; Yu, P.S. Leveraging meta-path based context for top-n recommendation with a neural co-attention model. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, London, UK, 19–23 August 2018; pp. 1531–1540. [Google Scholar]
- Wang, C.; Zhu, H.; Zhu, C.; Qin, C.; Xiong, H. Setrank: A setwise bayesian approach for collaborative ranking from implicit feedback. In Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA, 7–12 February 2020; Volume 34, pp. 6127–6136. [Google Scholar]
- Wang, C.D.; Deng, Z.H.; Lai, J.H.; Philip, S.Y. Serendipitous recommendation in e-commerce using innovator-based collaborative filtering. IEEE Trans. Cybern. 2018, 49, 2678–2692. [Google Scholar] [CrossRef] [PubMed]
- He, X.; Liao, L.; Zhang, H.; Nie, L.; Hu, X.; Chua, T.S. Neural collaborative filtering. In Proceedings of the 26th International Conference on World Wide Web, Perth, Australia, 3–7 April 2017; pp. 173–182. [Google Scholar]
- Rendle, S.; Freudenthaler, C.; Gantner, Z.; Schmidt-Thieme, L. BPR: Bayesian personalized ranking from implicit feedback. arXiv 2012, arXiv:1205.2618. [Google Scholar]
- Koren, Y.; Bell, R.; Volinsky, C. Matrix factorization techniques for recommender systems. Computer 2009, 42, 30–37. [Google Scholar] [CrossRef]
- He, X.; Deng, K.; Wang, X.; Li, Y.; Zhang, Y.; Wang, M. Lightgcn: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event, China, 25–30 July 2020; pp. 639–648. [Google Scholar]
- Pan, W.; Wei, W.; Mao, X.L. Context-aware entity typing in knowledge graphs. arXiv 2021, arXiv:2109.07990. [Google Scholar]
- Li, S.; Jia, Y.; Wu, Y.; Wei, N.; Zhang, L.; Guo, J. Knowledge-Aware Graph Self-Supervised Learning for Recommendation. Electronics 2023, 12, 4869. [Google Scholar] [CrossRef]
- Wang, H.; Zhao, M.; Xie, X.; Li, W.; Guo, M. Knowledge graph convolutional networks for recommender systems. In Proceedings of the World Wide Web Conference, San Francisco, CA, USA, 13–17 May 2019; pp. 3307–3313. [Google Scholar]
- Wang, H.; Zhang, F.; Xie, X.; Guo, M. DKN: Deep knowledge-aware network for news recommendation. In Proceedings of the 2018 World Wide Web Conference, Lyon, France, 23–27 April 2018; pp. 1835–1844. [Google Scholar]
- Zhang, F.; Yuan, N.J.; Lian, D.; Xie, X.; Ma, W.Y. Collaborative knowledge base embedding for recommender systems. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 13–17 August 2016; pp. 353–362. [Google Scholar]
- Cao, Y.; Wang, X.; He, X.; Hu, Z.; Chua, T.S. Unifying knowledge graph learning and recommendation: Towards a better understanding of user preferences. In Proceedings of the World Wide Web Conference, San Francisco, CA, USA, 13–17 May 2019; pp. 151–161. [Google Scholar]
- Xie, F.; Zhang, Y.; Przystupa, K.; Kochan, O. A Knowledge Graph Embedding Based Service Recommendation Method for Service-Based System Development. Electronics 2023, 12, 2935. [Google Scholar] [CrossRef]
- Wang, H.; Zhang, F.; Hou, M.; Xie, X.; Guo, M.; Liu, Q. Shine: Signed heterogeneous information network embedding for sentiment link prediction. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining 2018, Los Angeles, CA, USA, 5–9 February 2018; pp. 592–600. [Google Scholar]
- Bordes, A.; Usunier, N.; Garcia-Duran, A.; Weston, J.; Yakhnenko, O. Translating Embeddings for Modeling Multi-Relational data. In Proceedings of the 26th International Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA, 5–10 December 2013; Available online: https://proceedings.neurips.cc/paper/2013/hash/1cecc7a77928ca8133fa24680a88d2f9-Abstract.html (accessed on 21 August 2024).
- Wang, Z.; Zhang, J.; Feng, J.; Chen, Z. Knowledge graph embedding by translating on hyperplanes. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, Québec City, QC, Canada, 27–31 July 2014; Volume 28. [Google Scholar]
- Wang, H.; Zhang, F.; Wang, J.; Zhao, M.; Li, W.; Xie, X.; Guo, M. Ripplenet: Propagating user preferences on the knowledge graph for recommender systems. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management, Torino, Italy, 22–26 October 2018; pp. 417–426. [Google Scholar]
- Zhao, H.; Yao, Q.; Li, J.; Song, Y.; Lee, D.L. Meta-graph based recommendation fusion over heterogeneous information networks. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada, 13–17 August 2017; pp. 635–644. [Google Scholar]
- Wang, X.; He, X.; Cao, Y.; Liu, M.; Chua, T.S. Kgat: Knowledge graph attention network for recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA, 4–8 August 2019; pp. 950–958. [Google Scholar]
- Rendle, S.; Gantner, Z.; Freudenthaler, C.; Schmidt-Thieme, L. Fast context-aware recommendations with factorization machines. In Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, Beijing, China, 25–29 July 2011; pp. 635–644. [Google Scholar]
- Zou, D.; Wei, W.; Wang, Z.; Mao, X.L.; Zhu, F.; Fang, R.; Chen, D. Improving knowledge-aware recommendation with multi-level interactive contrastive learning. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, Atlanta, GA, USA, 17–21 October 2022; pp. 2817–2826. [Google Scholar]
- Jing, M.; Zhu, Y.; Zang, T.; Wang, K. Contrastive self-supervised learning in recommender systems: A survey. ACM Trans. Inf. Syst. 2023, 42, 1–39. [Google Scholar] [CrossRef]
- Brody, S.; Alon, U.; Yahav, E. How attentive are graph attention networks? arXiv 2021, arXiv:2105.14491. [Google Scholar]
- Yang, B.; Yih, W.T.; He, X.; Gao, J.; Deng, L. Embedding entities and relations for learning and inference in knowledge bases. arXiv 2014, arXiv:1412.6575. [Google Scholar]
- Lin, Y.; Liu, Z.; Sun, M.; Liu, Y.; Zhu, X. Learning entity and relation embeddings for knowledge graph completion. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, Austin, TX, USA, 25–30 January 2015; Volume 29. [Google Scholar]
- Rakhlin, A. Convolutional neural networks for sentence classification. GitHub 2016, 6, 25. [Google Scholar]
- Yu, X.; Ren, X.; Gu, Q.; Sun, Y.; Han, J. Collaborative filtering with entity similarity regularization in heterogeneous information networks. IJCAI HINA 2013, 27, 1–6. [Google Scholar]
- Wang, X.; Wang, D.; Xu, C.; He, X.; Cao, Y.; Chua, T.S. Explainable reasoning over knowledge graphs for recommendation. In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence, Honolulu, HI, USA, 27 January–1 February 2019; Volume 33, pp. 5329–5336. [Google Scholar]
- Yu, X.; Ren, X.; Sun, Y.; Sturt, B.; Khandelwal, U.; Gu, Q.; Norick, B.; Han, J. Recommendation in heterogeneous information networks with implicit user feedback. In Proceedings of the 7th ACM Conference on Recommender Systems, Hong Kong, China, 12–16 October 2013; pp. 347–350. [Google Scholar]
- Wang, X.; Huang, T.; Wang, D.; Yuan, Y.; Liu, Z.; He, X.; Chua, T.S. Learning intents behind interactions with knowledge graph for recommendation. In Web Conference; ACM: New York, NY, USA, 2021; pp. 878–887. [Google Scholar]
- Li, Y.; Zhang, Y.; Gao, Y.; Xu, B.; Liu, X. FCL: Pedestrian Re-Identification Algorithm Based on Feature Fusion Contrastive Learning. Electronics 2024, 13, 2368. [Google Scholar] [CrossRef]
- Fu, H.; Zhou, S.; Yang, Q.; Tang, J.; Liu, G.; Liu, K.; Li, X. LRC-BERT: Latent-representation contrastive knowledge distillation for natural language understanding. In Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence, Vancouver Convention Centre, VA, Canada, 2–9 February 2021; Volume 35, pp. 12830–12838. [Google Scholar]
- Wu, J.; Wang, X.; Feng, F.; He, X.; Chen, L.; Lian, J.; Xie, X. Self-supervised graph learning for recommendation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event, Canada, 11–15 July 2021; pp. 726–735. [Google Scholar]
- Wang, L.; Jin, D. A Time-Sensitive Graph Neural Network for Session-Based New Item Recommendation. Electronics 2024, 13, 223. [Google Scholar] [CrossRef]
- Cai, X.; Huang, C.; Xia, L.; Ren, X. LightGCL: Simple yet effective graph contrastive learning for recommendation. arXiv 2023, arXiv:2302.08191. [Google Scholar]
- Zou, D.; Wei, W.; Mao, X.L.; Wang, Z.; Qiu, M.; Zhu, F.; Cao, X. Multi-level cross-view contrastive learning for knowledge-aware recommender system. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain, 11–15 July 2022; pp. 1358–1368. [Google Scholar]
- Gao, B.; Liu, T.Y.; Wei, W.; Wang, T.; Li, H. Semi-supervised ranking on very large graphs with rich metadata. In Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Diego, CA, USA, 21–24 August 2011; pp. 96–104. [Google Scholar]
- Luo, D.; Cheng, W.; Xu, D.; Yu, W.; Zong, B.; Chen, H.; Zhang, X. Parameterized explainer for graph neural network. Adv. Neural Inf. Process. Syst. 2020, 33, 19620–19631. [Google Scholar]
- Liu, Y.; Xu, C.; Chen, L.; Yan, M.; Zhao, W.; Guan, Z. TABLE: Time-aware Balanced Multi-view Learning for stock ranking. In Knowledge-Based Systems; Elsevier: Amsterdam, The Netherlands, 2024; p. 112424. [Google Scholar]
- Chen, T.; Kornblith, S.; Norouzi, M.; Hinton, G. A simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning, Vienna, Austria, 13–18 July 2020; pp. 1597–1607. [Google Scholar]
- He, X.; Chua, T.S. Neural factorization machines for sparse predictive analytics. In Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval, Tokyo, Japan, 7–11 August 2017; pp. 355–364. [Google Scholar]
- Wang, Z.; Lin, G.; Tan, H.; Chen, Q.; Liu, X. CKAN: Collaborative knowledge-aware attentive network for recommender systems. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event, China, 25–30 July 2020; pp. 219–228. [Google Scholar]
- Yang, Y.; Huang, C.; Xia, L.; Li, C. Knowledge graph contrastive learning for recommendation. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain, 11–15 July 2022; pp. 1434–1443. [Google Scholar]
- Kingma, D.P. Adam: A method for stochastic optimization. arXiv 2014, arXiv:1412.6980. [Google Scholar]
- Glorot, X.; Bengio, Y. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, Chia Laguna Resort, Sardinia, Italy, 13–15 May 2010; pp. 249–256. [Google Scholar]
Stats. | # Users | # Items | # Interactions | # Entities | # Relations | # Triplets |
---|---|---|---|---|---|---|
Yelp2018 | 45,919 | 45,538 | 1,183,610 | 47,472 | 42 | 869,603 |
LastFM | 23,566 | 48,123 | 3,034,763 | 106,389 | 9 | 464,567 |
MovieLens | 37,385 | 6182 | 539,300 | 24,536 | 20 | 237,155 |
Model | Yelp2018 | LastFM | Movielens | |||
---|---|---|---|---|---|---|
Recall | NDCG | Recall | NDCG | Recall | NDCG | |
FM | 0.0307 | 0.0206 | 0.0736 | 0.0627 | 0.4175 | 0.2599 |
NFM | 0.0418 | 0.0258 | 0.0673 | 0.0538 | 0.3903 | 0.2357 |
BPR-MF | 0.0499 | 0.0324 | 0.0715 | 0.0618 | 0.4052 | 0.2609 |
CKE | 0.0686 | 0.0431 | 0.0746 | 0.0652 | 0.4106 | 0.2669 |
KGCN | 0.0532 | 0.0338 | 0.0819 | 0.0705 | 0.4237 | 0.2753 |
LightGCN | 0.0682 | 0.0443 | 0.0765 | 0.0686 | 0.4486 | 0.3054 |
KGAT | 0.0653 | 0.0423 | 0.0877 | 0.0749 | 0.4532 | 0.3007 |
CKAN | 0.0698 | 0.0441 | 0.0812 | 0.0690 | 0.4314 | 0.2891 |
KGIN | 0.0712 | 0.0462 | 0.0967 | 0.0847 | 0.4661 | 0.3120 |
KGCL | 0.0736 | 0.0493 | 0.0899 | 0.0793 | 0.4516 | 0.2967 |
AKCL-DA | 0.0762 | 0.0518 | 0.1113 | 0.0981 | 0.4742 | 0.3264 |
Improv% | 3.41% | 4.82% | 13.12% | 13.66% | 1.71% | 4.41% |
Model | Yelp2018 | LastFM | MovieLens | |||
---|---|---|---|---|---|---|
Recall | NDCG | Recall | NDCG | Recall | NDCG | |
w/o Assumed edge | 0.0743 | 0.0487 | 0.1103 | 0.0977 | 0.4733 | 0.3257 |
w/o Att | 0.0712 | 0.0475 | 0.1092 | 0.0948 | 0.4298 | 0.3228 |
AKCL-DA | 0.0762 | 0.0518 | 0.1113 | 0.0981 | 0.4742 | 0.3264 |
Model | Yelp2018 | LastFM | MovieLens | Avg. Dec | |||
---|---|---|---|---|---|---|---|
Recall | NDCG | Recall | NDCG | Recall | NDCG | ||
KGAT | 0.0623 | 0.0406 | 0.0796 | 0.0692 | 0.4489 | 0.2983 | 4.54% |
KGCL | 0.0736 | 0.0472 | 0.0856 | 0.0762 | 0.4503 | 0.2923 | 2.89% |
AKCL-DA | 0.0759 | 0.0512 | 0.1109 | 0.0976 | 0.4726 | 0.3257 | 0.43% |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Li, H.; Zheng, J.; Jin, B.; Zhu, H. Adaptive Knowledge Contrastive Learning with Dynamic Attention for Recommender Systems. Electronics 2024, 13, 3594. https://doi.org/10.3390/electronics13183594
Li H, Zheng J, Jin B, Zhu H. Adaptive Knowledge Contrastive Learning with Dynamic Attention for Recommender Systems. Electronics. 2024; 13(18):3594. https://doi.org/10.3390/electronics13183594
Chicago/Turabian StyleLi, Hongchan, Jinming Zheng, Baohua Jin, and Haodong Zhu. 2024. "Adaptive Knowledge Contrastive Learning with Dynamic Attention for Recommender Systems" Electronics 13, no. 18: 3594. https://doi.org/10.3390/electronics13183594