Content-Aware Few-Shot Meta-Learning for Cold-Start Recommendation on Portable Sensing Devices
Abstract
:1. Introduction
- We design a DT-Net learning system that minimizes the introduction of irrelevant features, which could otherwise degrade model performance.
- We implement a model-agnostic meta-optimization strategy that empowers the CFSM to represent new users and items, thereby addressing the cold-start problem from both the model and auxiliary data perspectives.
- Empirical results on the three datasets demonstrate that CFSM outperforms state-of-the-art models in terms of the AUC metric.
2. Related Works
3. Preliminaries and Definition
4. Proposed Approach
4.1. DT-Net for Representation Learning
4.1.1. Meta-Encoder
4.1.2. Mutual Attention Encoder
4.2. Optimization with Meta-Learning
Algorithm 1 Model training of our proposed CFSM model. |
|
5. Experimental Settings
5.1. Datasets
- ShortVideos is a collection of Chinese short videos, and it contained three tables: interaction data, user information, and video information. The user information included age, gender, and location, and the video information included title, category, text description, and cover image. For each user, we retained K = 36 samples and sorted them by the timestamp. Users whose samples were fewer than K were removed.
- MovieLens-1M (https://grouplens.org/datasets/movielens/1m/) (accessed on 22 August 2024) is one of the most widely used benchmark datasets, and it included three tables: rating, user information, and movie information. It contains 1M rating records as-signed by users. The user information included ID, age, occupation, gender, and zip code. The movie information included title, publication, and genre, and we obtained text descriptions and cover images of the movies from the Internet Movie Database. We binarized the rating scores to zero and one, and a value greater than three was set to one.
- Book-Crossing (https://grouplens.org/datasets/book-crossing/) (accessed on 22 August 2024) is a public dataset containing three tables: rating, user information, and book information. User information included ID, age, and location, and book information included title, author, and publisher. We obtained text descriptions and cover images from Amazon.com. We binarized the rating score to zero and one, and a value larger than five was set to one.
5.2. Evaluation Settings
- PMF: This model obtains feature matrices by exploiting user–item interaction data and predicts unknown values in a rating matrix, which has been shown to be a particularly flexible and effective framework for addressing large, sparse, and very imbalanced datasets.
- FM: This model adopts linear and pairwise feature interactions as the inner products of their respective feature latent vectors. This approach is far more effective than previous approaches, especially with sparse datasets.
- DeepFM: This model combines the power of factorization machines for the deep learning of features in new recommender architectures. Low-order feature interactions are modeled, such as FMs and high-order feature interaction models (e.g., DNNs).
- DropoutNet: This model is based on the observation that cold starting is equivalent to the missing data problem in which preference information is missing. Instead of adding additional objective terms to the model content, this model modifies the learning procedure by applying a dropout to explicitly condition the model for missing input.
- TDNN: This model is the most widely used deep learning recommendation model in the industry, and it applies two parallel networks to learn latent vector representations of users and items.
- MetaCS-DNN: This model learns the representation of users and items using a DNN, and the parameters in the deep learning recommendation framework are updated by applying meta-learning optimization.
5.3. Metrics
6. Results and Discussion
6.1. Comparison of Cold-Start Scenarios
6.2. Comparison with the Normal Scenario
6.3. Ablation Study
6.4. Parameter Analysis
7. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Gao, J.; Lin, Y.; Wang, Y.; Wang, X.; Yang, Z.; He, Y.; Chu, X. Set-sequence-graph: A multi-view approach towards exploiting reviews for recommendation. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, Virtual Event, Ireland, 19–23 October 2020; pp. 395–404. [Google Scholar]
- Ying, H.; Zhuang, F.; Zhang, F.; Liu, Y.; Xu, G.; Xie, X.; Xiong, H.; Wu, J. Sequential recommender system based on hierarchical attention network. In Proceedings of the IJCAI International Joint Conference on Artificial Intelligence, Stockholm, Sweden, 13–19 July 2018. [Google Scholar]
- Liu, T.; Lou, S.; Liao, J.; Feng, H. Dynamic and static representation learning network for recommendation. IEEE Trans. Neural Netw. Learn. Syst. 2022, 35, 831–841. [Google Scholar] [CrossRef] [PubMed]
- Pan, X.; Chen, Y.; Tian, C.; Lin, Z.; Wang, J.; Hu, H.; Zhao, W.X. Multimodal meta-learning for cold-start sequential recommendation. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, Atlanta, GA, USA, 17–21 October 2022; pp. 3421–3430. [Google Scholar]
- Han, J.; Ma, Y.; Mei, Q.; Liu, X. DeepRec: On-device deep learning for privacy-preserving sequential recommendation in mobile commerce. In Proceedings of the Web Conference 2021, Ljubljana, Slovenia, 19–23 April 2021; pp. 900–911. [Google Scholar]
- Liu, Z.; Wang, Y.; Liu, T.; Zhang, L.; Li, W.; Liao, J.; He, D. Semantic-enhanced contrastive learning for session-based recommendation. Knowl.-Based Syst. 2023, 280, 111001. [Google Scholar] [CrossRef]
- Wang, S.; Wang, Y.; Tang, J.; Shu, K.; Ranganath, S.; Liu, H. What your images reveal: Exploiting visual contents for point-of-interest recommendation. In Proceedings of the 26th International Conference on World Wide Web, Perth, Australia, 3–7 April 2017; pp. 391–400. [Google Scholar]
- Zhao, W.X.; Li, S.; He, Y.; Chang, E.Y.; Wen, J.R.; Li, X. Connecting social media to e-commerce: Cold-start product recommendation using microblogging information. IEEE Trans. Knowl. Data Eng. 2015, 28, 1147–1159. [Google Scholar] [CrossRef]
- Liao, J.; Liu, T.; Yin, H.; Chen, T.; Wang, J.; Wang, Y. An integrated model based on deep multimodal and rank learning for point-of-interest recommendation. World Wide Web 2021, 24, 631–655. [Google Scholar] [CrossRef]
- Volkovs, M.; Yu, G.; Poutanen, T. Dropoutnet: Addressing cold start in recommender systems. In Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA, 4–9 December 2017; Volume 30. [Google Scholar]
- Finn, C.; Abbeel, P.; Levine, S. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the International Conference on Machine Learning, Sydney, Australia, 6–11 August 2017; pp. 1126–1135. [Google Scholar]
- Lu, Y.; Fang, Y.; Shi, C. Meta-learning on heterogeneous information networks for cold-start recommendation. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Virtual Event, CA, USA, 6–10 July 2020; pp. 1563–1573. [Google Scholar]
- Bharadhwaj, H. Meta-learning for user cold-start recommendation. In Proceedings of the 2019 International Joint Conference on Neural Networks (IJCNN), Budapest, Hungary, 14–19 July 2019; pp. 1–8. [Google Scholar]
- Tan, H.; Yao, D.; Huang, T.; Wang, B.; Jing, Q.; Bi, J. Meta-learning enhanced neural ODE for citywide next POI recommendation. In Proceedings of the 2021 22nd IEEE International Conference on Mobile Data Management (MDM), Toronto, ON, Canada, 15–18 June 2021; pp. 89–98. [Google Scholar]
- Vartak, M.; Thiagarajan, A.; Miranda, C.; Bratman, J.; Larochelle, H. A meta-learning perspective on cold-start recommendations for items. In Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA, 4–9 December 2017; Volume 30. [Google Scholar]
- Lee, H.; Im, J.; Jang, S.; Cho, H.; Chung, S. Melu: Meta-learned user preference estimator for cold-start recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA, 4–8 August 2019; pp. 1073–1082. [Google Scholar]
- Dong, M.; Yuan, F.; Yao, L.; Xu, X.; Zhu, L. Mamo: Memory-augmented meta-optimization for cold-start recommendation. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Virtual Event, CA, USA, 6–10 July 2020; pp. 688–697. [Google Scholar]
- Zheng, Y.; Liu, S.; Li, Z.; Wu, S. Cold-start sequential recommendation via meta learner. In Proceedings of the AAAI Conference on Artificial Intelligence, Virtually, 2–9 February 2021; Volume 35, pp. 4706–4713. [Google Scholar]
- Du, X.; Yin, H.; Chen, L.; Wang, Y.; Yang, Y.; Zhou, X. Personalized video recommendation using rich contents from videos. IEEE Trans. Knowl. Data Eng. 2018, 32, 492–505. [Google Scholar] [CrossRef]
- Zhang, Y.; Yin, H.; Huang, Z.; Du, X.; Yang, G.; Lian, D. Discrete deep learning for fast content-aware recommendation. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, Los Angeles, CA, USA, 5–9 February 2018; pp. 717–726. [Google Scholar]
- Li, J.; Lu, K.; Huang, Z.; Shen, H.T. Two birds one stone: On both cold-start and long-tail recommendation. In Proceedings of the 25th ACM International Conference on Multimedia, Mountain View, CA, USA, 23–27 October 2017; pp. 898–906. [Google Scholar]
- Xie, M.; Yin, H.; Wang, H.; Xu, F.; Chen, W.; Wang, S. Learning graph-based poi embedding for location-based recommendation. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, Indianapolis, IN, USA, 24–28 October 2016; pp. 15–24. [Google Scholar]
- Wang, H.; Wang, N.; Yeung, D.Y. Collaborative deep learning for recommender systems. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Sydney, NSW, Australia, 10–13 August 2015; pp. 1235–1244. [Google Scholar]
- Kim, D.; Park, C.; Oh, J.; Lee, S.; Yu, H. Convolutional matrix factorization for document context-aware recommendation. In Proceedings of the 10th ACM Conference on Recommender Systems, Boston, MA, USA, 15–19 September 2016; pp. 233–240. [Google Scholar]
- Zhang, F.; Yuan, N.J.; Lian, D.; Xie, X.; Ma, W.Y. Collaborative knowledge base embedding for recommender systems. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 13–17 August 2016; pp. 353–362. [Google Scholar]
- Wang, X.; Peng, Z.; Wang, S.; Yu, P.S.; Fu, W.; Xu, X.; Hong, X. CDLFM: Cross-domain recommendation for cold-start users via latent feature mapping. Knowl. Inf. Syst. 2020, 62, 1723–1750. [Google Scholar] [CrossRef]
- Xu, X.; Dong, H.; Qi, L.; Zhang, X.; Xiang, H.; Xia, X.; Xu, Y.; Dou, W. Cmclrec: Cross-modal contrastive learning for user cold-start sequential recommendation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, Washington, DC, USA, 14–18 July 2024; pp. 1589–1598. [Google Scholar]
- Zhao, C.; Li, C.; Xiao, R.; Deng, H.; Sun, A. CATN: Cross-domain recommendation for cold-start users via aspect transfer network. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, Xi’an, China, 25–30 July 2020; pp. 229–238. [Google Scholar]
- Tay, Y.; Luu, A.T.; Hui, S.C. Multi-pointer co-attention networks for recommendation. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, London, UK, 19–23 August 2018; pp. 2309–2318. [Google Scholar]
- Li, J.; Jing, M.; Lu, K.; Zhu, L.; Yang, Y.; Huang, Z. From zero-shot learning to cold-start recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA, 27 January 2019; Volume 33, pp. 4189–4196. [Google Scholar]
- Wang, C.; Zhu, Y.; Sun, A.; Wang, Z.; Wang, K. A Preference Learning Decoupling Framework for User Cold-Start Recommendation. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, Taipei, Taiwan, 23–27 July 2023; pp. 1168–1177. [Google Scholar]
- Barkan, O.; Koenigstein, N. Item2vec: Neural item embedding for collaborative filtering. In Proceedings of the 2016 IEEE 26th International Workshop on Machine Learning for Signal Processing (MLSP), Salerno, Italy, 13–16 September 2016; pp. 1–6. [Google Scholar]
- Chang, B.; Park, Y.; Park, D.; Kim, S.; Kang, J. Content-aware hierarchical point-of-interest embedding model for successive poi recommendation. In Proceedings of the IJCAI, Stockholm, Sweden, 13–19 July 2018; Volume 20, pp. 3301–3307. [Google Scholar]
- Wang, X.; Wang, Y. Improving content-based and hybrid music recommendation using deep learning. In Proceedings of the 22nd ACM International Conference on Multimedia, Orlando, FL, USA, 7 November 2014; pp. 627–636. [Google Scholar]
- Diederik, P.K. Adam: A method for stochastic optimization. arXiv 2014, arXiv:1412.6980. [Google Scholar]
- Mnih, A.; Salakhutdinov, R.R. Probabilistic matrix factorization. In Proceedings of the 20th International Conference on Neural Information Processing Systems, Vancouver, BC, Canada, 3–6 December 2007; Volume 20. [Google Scholar]
- He, X.; Chua, T.S. Neural factorization machines for sparse predictive analytics. In Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval, Shinjuku, Japan, 7–11 August 2017; pp. 355–364. [Google Scholar]
- Guo, H.; Tang, R.; Ye, Y.; Li, Z.; He, X. DeepFM: A factorization-machine based neural network for CTR prediction. arXiv 2017, arXiv:1703.04247. [Google Scholar]
- Yi, X.; Yang, J.; Hong, L.; Cheng, D.Z.; Heldt, L.; Kumthekar, A.; Zhao, Z.; Wei, L.; Chi, E. Sampling-bias-corrected neural modeling for large corpus item recommendations. In Proceedings of the 13th ACM Conference on Recommender Systems, Copenhagen, Denmark, 16–20 September 2019; pp. 269–277. [Google Scholar]
- Xie, R.; Zhang, S.; Wang, R.; Xia, F.; Lin, L. Hierarchical reinforcement learning for integrated recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence, Virtually, 2–9 February 2021; Volume 35, pp. 4521–4528. [Google Scholar]
- Zhang, K.; Qian, H.; Cui, Q.; Liu, Q.; Li, L.; Zhou, J.; Ma, J.; Chen, E. Multi-interactive attention network for fine-grained feature learning in ctr prediction. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining, Virtual Event, Israel, 8–12 March 2021; pp. 984–992. [Google Scholar]
Variable | Interpretation |
---|---|
Set of users and items | |
Sequence of user–item interaction | |
Interest representation of user | |
Content representation of item | |
Embedding matrix for user’s kth attribution | |
Document and visual feature matrices, respectively | |
Attention network’s query and key, respectively |
Dataset | ShortVideos | MovieLens-1M | Book-Crossing |
---|---|---|---|
#Users | 289,434 | 6040 | 278,858 |
#Items | 19,412 | 3706 | 271,379 |
#Interactions | 10,419,624 | 1,000,209 | 1,149,780 |
User information | ID, age, gender, location | ID, age, gender, occupation, zip-code | ID, age, location |
Item information | title, category, description, cover image | title, category, description, posters | title, publisher, cover image, description |
MovieLens-1M | Book-Crossing | ShortVideos | ||||
---|---|---|---|---|---|---|
AUC | RelaImpr | AUC | RelaImpr | AUC | RelaImpr | |
PMF | 0.7521 | 0.00% | 0.7024 | 0.00% | 0.6041 | 0.00% |
FM | 0.7400 | −1.61% | 0.7132 | 1.54% | 0.6724 | 11.31% |
DeepFM | 0.8517 | 13.24% | 0.7915 | 12.69% | 0.7541 | 24.83% |
TDNN | 0.8247 | 9.65% | 0.7821 | 11.35% | 0.7142 | 18.23% |
DropoutNet | 0.7546 | 0.33% | 0.7831 | 11.49% | 0.7501 | 24.17% |
MetaCS-DNN | 0.8251 | 9.71% | 0.8011 | 14.05% | 0.7497 | 24.10% |
CFSM | 0.8651 | 15.02% | 0.8141 | 15.90% | 0.7704 | 27.53% |
1.55% | 1.34% | 2.42% |
MovieLens-1M | Book-Crossing | ShortVideos | ||||
---|---|---|---|---|---|---|
AUC | RelaImpr | AUC | RelaImpr | AUC | RelaImpr | |
CSFM-naïve | 0.7238 | 0.00% | 0.6754 | 0.00% | 0.6864 | 0.00% |
CSFM | 0.8651 | 19.52% | 0.8141 | 20.54% | 0.7704 | 12.24% |
14.13% | 13.87% | 8.4% |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Lv, X.; Fang, K.; Liu, T. Content-Aware Few-Shot Meta-Learning for Cold-Start Recommendation on Portable Sensing Devices. Sensors 2024, 24, 5510. https://doi.org/10.3390/s24175510
Lv X, Fang K, Liu T. Content-Aware Few-Shot Meta-Learning for Cold-Start Recommendation on Portable Sensing Devices. Sensors. 2024; 24(17):5510. https://doi.org/10.3390/s24175510
Chicago/Turabian StyleLv, Xiaomin, Kai Fang, and Tongcun Liu. 2024. "Content-Aware Few-Shot Meta-Learning for Cold-Start Recommendation on Portable Sensing Devices" Sensors 24, no. 17: 5510. https://doi.org/10.3390/s24175510