A Comprehensive Survey of Recommender Systems Based on Deep Learning

Abstract

With the increasing abundance of information resources and the development of deep learning techniques, recommender systems (RSs) based on deep learning have gradually become a research focus. Although RSs have evolved in recent years, a systematic review of existing RS approaches is still warranted. The main focus of this paper is on recommendation models that incorporate deep learning techniques. The objective is to guide novice researchers interested in this field through the investigation and application of the proposed recommendation models. Specifically, we first categorize existing RS approaches into four types: content-based recommendations, sequence recommendations, cross-domain recommendations, and social recommendation methods. We then introduce the definitions and address the challenges associated with these RS methodologies. Subsequently, we propose a comprehensive categorization framework and novel taxonomies for these methodologies, providing a thorough account of their research advancements. Finally, we discuss future developments regarding this topic.

1. Introduction

With the rapid progression of technology and the development of the Internet, we have transitioned from an era of information scarcity to the age of big data. The growth in the amount of information available has led to the challenge of “information overload” [1]. In the context of the digital age, recommender systems (RSs) have arisen. RSs analyze data to discern user preferences for items and assist users in efficiently sifting through information [2], directing them towards the content that is most relevant and valuable for their interests. Currently, RSs are widely adopted and have provided many economic benefits. Essentially, RSs are founded on the premise that when users exhibit similar item ratings or behaviors, they are likely to demonstrate similar ratings or actions on other items [3].

Lately, due to the continuous improvement in computational capabilities, artificial neural networks (ANNs) have started to garner widespread attention [4]. This has propelled deep learning to emerge as a burgeoning field in computer science. An ANN is composed of layers of nodes. Neural networks acquire preferences by leveraging training data and improving their accuracy over time. Deep learning builds upon the foundation of machine learning, incorporating the concept of neural networks [5]. Using deep learning techniques, we can effectively process complex data and uncover deeply hidden features and the relationships between these features, thereby greatly enhancing data representation. In 2016, after years of continuous research in the video recommendation domain, YouTube integrated deep neural networks into its recommender system and applied them to video recommendations with outstanding results [6]. This achievement served as inspiration for an increasing number of recommendation models to incorporate deep learning techniques [7].

In an RS, ‘item’ denotes a product or service recommended by the system to its users. Recommendation of products to a user requires the analysis of the past preferences of similar users and leveraging item information. At the same time, users’ preferences are dynamically evolving [8], so researchers employ time-based/sequential methods to model users’ dynamic preferences and sequential patterns that change over time [9].

The integration of recommendation techniques with neural networks has paved the way for broader application prospects [10]. Users frequently interact with multiple platforms using various services. By utilizing cross-domain recommendations, insights from different domains can be leveraged to provide more comprehensive and relevant suggestions [11]. This approach effectively tackles data sparsity, mitigates the cold start problem, and captures a more holistic view of a user’s multifaceted preferences. Some studies [12,13,14] have indicated that social relationships between users can be effectively used to alleviate issues such as the cold start problem. People have proposed integrating information-rich social networks with neural networks and incorporating the social–relational attributes of users or items (such as friendships and tag categorizations) into traditional recommendation frameworks, thereby overcoming the limitations of conventional recommendation algorithms. However, many recommendation algorithms still fall in terms of handling data sparsity and processing large-scale data, which to some extent compromises the precision of recommendations produced [15]. RS algorithms must have the ability to deal with highly sparse data and need to scale with increasing numbers of users and items. At the same time, they must provide satisfactory results and address other issues, such as data noise and privacy protection.

In the evolution of recommender systems, various challenges have arisen, such as information overload, cold start problems, long tail effects, and so on. In recent years, deep learning technology has emerged as a pivotal force in the field of recommender systems. Deep learning models, known for their exceptional performance, have become a hot spot in both academia and industry, offering solutions to many of the core challenges currently confronting recommender systems.

What makes deep learning compelling is its effectiveness in addressing the numerous challenges encountered by recommender systems. First of all, deep learning models excel at learning complex user and project features, making them adept at capturing subtle differences in user behavior. Second, deep learning models can handle large-scale data and are therefore able to cope with information overload. In addition, personalized recommendations that take advantage of potential representations of users and items can solve the cold-start problem to some extent. In addition, deep learning can improve the robustness of recommender systems and improve user experience. The reason why deep learning is so captivating lies in its effectiveness in addressing numerous challenges encountered by recommender systems. First, deep learning models can learn intricate features of users and items, allowing for a more precise capture of subtle variations in user behavior. Second, deep learning models can handle large-scale data, thus mitigating the issue of information overload. Furthermore, personalized recommendations based on the latent representations of users and items can, to some extent, alleviate the cold-start problem. Additionally, deep learning can enhance the robustness of recommender systems, improving the user experience.

In this study, we provide a comprehensive overview of recommender systems with a particular focus on their integration with deep learning. We introduce various types of recommendation methods, technological trends, and application domains that leverage deep learning. We offer detailed descriptions from four perspectives: content-based recommendation, sequence recommendation, cross-domain recommendation, and social recommendation. Additionally, we delve into the application of deep learning in recommender systems, including model principles, performance enhancements, and application cases, with the aim of providing readers with a deep understanding of this field. Furthermore, we discuss potential future trends in deep learning-based recommender systems, including model interpretability, multi-modal recommendation, privacy, and fairness considerations. The primary contributions of this paper can be summarized as follows.

• We present a comprehensive examination of recommender systems, with a specific emphasis on their integration with deep learning. We categorize them in terms of their developmental perspective, providing a comprehensive view of the evolution of the recommender systems field.
• We conduct a review of the research progress of recommender systems integrated with deep learning, focusing on methods for applying deep learning to collaborative filtering. Specifically, we perform a comprehensive analysis of four recommendation approaches that incorporate deep learning: content-based recommendation, sequence recommendation, cross-domain recommendation, and social recommendation.
• We identify future research directions in the field of deep learning-based recommender systems, contributing to the advancement of the research community.

Throughout this review study, in comparison to existing literature, we have pinpointed articles that excessively concentrate on content-based recommendation and collaborative filtering, with relatively less attention given to reviews concerning the direction of deep learning. Consequently, the following gaps have been identified: Despite the exceptional performance of deep learning models, their interpretability remains a challenge. This paper aims to explore ways to improve the interpretability of models to meet the requirements of both users and regulatory authorities. Future research could delve deeper into methods for enhancing model interpretability to bolster user trust. With the proliferation of multi-modal data, recommender systems need to better integrate diverse types of information. We believe that future research can explore methods for the more profound fusion of multi-modal data. Recommender systems should also make strides in addressing user privacy concerns and guaranteeing fairness in recommendation results. Future research should focus on how to protect user privacy and ensure fairness while providing personalized recommendations. In summary, this review aims to highlight these research gaps and offer valuable guidance for future research efforts in this field.

2. Related Work

Recommender systems are derived from advances in cognitive science, approximation theory, information extraction, and prediction theory. They have undergone rapid evolution since their birth and have become an important catalyst for the rise of the modern business economy [16]. These systems filter out redundant information from vast amounts of data, selecting items that can fulfill latent user needs. Traditional recommender systems typically focus on whether users have shown interest or rated an item [17]. By analyzing historical interaction data, these systems uncover the underlying demands, thereby achieving efficient recommendations.

Recommender systems are fundamentally based on exploiting binary relationships between users and items. By utilizing historical behaviors or similarity relations, these systems help identify items that might pique users’ interests [18]. A utility function, denoted as “s”, is employed to calculate the recommendation score for item “i” with respect to user “u”. In this context, both users and items are characterized by a collection of distinct attribute features [19]. By calculating recommendation scores, the aim is to find the most interesting i′ ∈ I for every u ∈ U, as shown in Equation (1).

$$\forall \mathrm{u}\in U,i_u^{\prime }=\arg \max s(u,i)$$

(1)

In 2008, Ma et al. from the University of Hong Kong [20] took an innovative step by integrating user social interaction information with historical item rating data. This was a pioneering effort that introduced the SoRec recommendation algorithm, which was built on the foundation of probabilistic matrix factorization (PMF) [21]. Recognizing the increasing significance of social interactions in recommendations, research in this area witnessed exponential growth. In 2010, Jamali [22] delved deeper into the sphere of trust propagation among users and their social friends. They adeptly combined matrix factorization techniques with interactive trust, leading to the conception of the Social MF model. A year later, in 2011, building on the foundation laid by Social MF, Ma et al. utilized social regularization for recommendation [23]. This model was predicated on the assumption that the greater the influence of neighboring friends on a user, the more the user’s latent preferences would converge towards that friend. When it comes to neural networks, Sedhain et al. of Australia’s NICTA research center ingeniously applied an autoencoder model to recommender systems in 2015, giving birth to the single-hidden-layer neural network recommendation model known as AutoRec [24]. Using these operations, AutoRec has generalization and expression capabilities and it is the basic application of deep learning in this field. Pushing the boundaries further, Devooght and Bersini [25] achieved enhanced short-term prediction accuracy in 2017 by redefining collaborative filtering (CF) as a sequence prediction problem using neural networks. In summary, the evolution of recommender systems, as seen over the years, has seamlessly transitioned from leveraging social interactions to harnessing the prowess of deep neural networks. This evolution trend underscores the dynamic nature of research in this domain and the search for more refined and accurate recommendation techniques.

Graph neural networks (GNNs) form a collective term for models that apply neural networks on graphs, which have powerful graph-structured data learning capabilities. Graphs consist of nodes and edges connecting nodes and are typical non-Euclidean spatial data structures. Social networks are quintessential examples of graph data. Using GNNs, social network graph models can be seamlessly integrated into recommender systems. Hence, research on social recommendation has progressively become a focal point. By harnessing deep learning techniques to incorporate information from social networks, one can deeply understand the relevant features of users and items, achieving better accuracy of the recommendation while simultaneously improving user experience [26,27]. Based on this concept, GraphSAGE [28] is an inductive learning framework that efficiently generates embeddings for unknown vertices using their attribute information. This alleviates the issue of models being unable to directly generalize to vertices that have not appeared during the training process. Based on GraphSAGE, Ying et al. introduced an efficient GCN algorithm [29], they combine random walks with graph convolutions to produce node embeddings. GraphSAGE introduced a novel paradigm: inductive learning. Its strength lies in its ability to generalize from specific instances to broader contexts, discerning unknown data on unknown nodes. This advantage enables it to handle various scenarios in the industrial sector, where graph structures undergo dynamic changes.

The primary objective of this article is to provide scholars interested in this topic with an understanding of the main effects of employing deep learning methods in recommender systems. This work particularly delves into the motivations and developments associated with the application of various methods in such systems. Furthermore, it aims to offer solution-oriented perspectives that address the current challenges faced by recommender systems.

3. Overview of the Recommender Systems

Early recommender systems were primarily based on collaborative filtering and content-based recommendation. User-based collaborative filtering methods were introduced in 1992, while item-based collaborative filtering methods emerged in 1999. Content-based recommendation methods also began to appear during this period, utilizing item attributes and user preferences for recommendations. In the 21st century, with the rise of social networks and the development of deep learning technology, social recommendation methods gradually gained attention. Similarly, cross-domain recommendation methods also started to emerge, allowing the application of a user’s historical behavior in one domain to recommendations in other domains. In recent years, with the proliferation of mobile devices and online shopping, sequence recommendation methods have become increasingly important. These methods consider user behavior sequences, such as purchase history or browsing history, to provide more personalized recommendations. Deep learning methods can handle large-scale data and multi-modal information, enhancing the effectiveness of recommendations. Reinforcement learning methods have also been applied to recommender systems, optimizing recommendation strategies via user interactions. The development history of recommender systems can be divided into several key stages, each producing different types of recommendation methods. A summary of some important recommender system methods is provided in the following Figure 1.

New technologies, compared to old ones, are not a complete revolution. New technologies are built on the foundation laid by their predecessors, incorporating improvements rather than being entirely different. The field of recommender systems is continuously evolving, with new methods and technologies emerging to adapt to evolving user demands and data environments. Currently, efforts are underway to refine the integration of deep learning for improved recommendation results. Therefore, the journey of recommender systems is a dynamic and evolving process. We will begin by introducing the most fundamental approaches, combining deep learning with collaborative filtering and content-based recommendation.

Collaborative filtering is one of the first and fundamental recommender systems. In 1992, GlodBerg et al. [47] introduced the concept of collaborative filtering. The key idea lies in finding the nearest neighbors using the interaction matrix of users and items, and then generating various forms of recommendations based on the information from these neighbors, which is essentially a process of matrix completion. The core concept of collaborative filtering algorithms is to provide product recommendations or predictions based on similar users or similar items, encompassing two dimensions: neighborhood-based and model-based approaches. The key to collaborative filtering recommendations is the computation of similarity between different users and different items.

Over the past three decades, collaborative filtering has been widely used by people. However, as time has progressed, its scalability limitations have become apparent, making it less suitable for scenarios with large datasets. People have turned to machine learning and data mining algorithms to learn and recognize complex patterns using training data. Currently, one of the most common paradigms for building deep learning recommendation algorithms is the use of multi-layer perceptions, which can introduce nonlinear transformations into existing collaborative filtering methods.

Since then, the research of recommender systems has shifted towards deep learning-based recommendation [48]. To address the complexities of user behavior, neural network-based models have become increasingly favored by researchers [49]. Among them, neural collaborative filtering (NCF) stands out as a notable attempt [30]; this method enhances the expressiveness of the developed model by expanding the inner product within a multilayer perceptron (MLP). The success of this approach has further motivated researchers to explore the integration of deep learning techniques using traditional recommendation algorithms.

This trend is effectively illustrated in the deep factorization machine (DeepFM) [31], which cleverly integrates a shallow factorization machine (FM) [50] with an MLP. This approach effectively combines the FM and deep components to achieve a unified representation of both low-order and high-order feature interactions, significantly enhancing the model’s expressive power. This approach is similar to traditional machine learning, but its training is divided into two phases. Therefore, it can converge better and faster.

Wu, Y et al. proposed FedDeepFM [32], a model grounded in federated learning, which offers a mechanism that delivers high-quality recommendations while enhancing user privacy. This approach is particularly suitable for scenarios that prioritize privacy preservation or data transmission reduction. It generates training data using real data combined with interaction-based synthetic data. Built upon DeepFM, FedDeepFM offers a mechanism for delivering high-quality recommendations while enhancing user privacy.

The “multi-layer neural network + output layer” allows for a more extensive interaction of user/item vectors, introducing additional non-linear features and enhancing the learning capability for sparse features. By incorporating attention mechanisms and introducing attention layers between the embedding layer and the multi-layer perceptron, it has evolved into deep interest networks (DIN) [51]. Furthermore, there are variations such as deep interest evolution networks (DIEN) [52], which incorporate sequence models to simulate changes in user preferences, and multi-interest networks with dynamic routing (MIND) [53], which utilize capsule networks to extract diverse user interests and introduce tag-based attention mechanisms for dynamic path selection.

As one of the most widely used recommendation algorithms at present, collaborative filtering exhibits strong generality and significant effectiveness. However, collaborative filtering always faces serious issues of data sparsity and cold start. The integration of deep learning with collaborative filtering recommender systems not only simplifies engineering implementation but also alleviates these problems. Content-based recommendation, which coexisted with collaborative filtering, has also evolved in a similar manner.

3.1. Content-Based Recommendation

Content-based recommendation is the earliest and most basic type of recommendation algorithm, and it has played a pivotal role in the entire history of recommender systems [54]. Although its effect may not be as good as that of new recommendation algorithms, it is still very valuable and even essential.

Content-based recommendation algorithms form a relatively intuitive category of algorithms that primarily offer recommendations by comparing item features with users’ historical preferences. In the early days, it relied primarily on specific content, such as the characteristics of text, images, or audio. Their approach entails suggesting items to users that exhibit content similarity to previously preferred items, determined by the historical preferences of these users [55,56]. A content-based recommendation model is constructed based on the target item, user information, and interactions between the user and items. Importantly, it notably neglects the actions of other users. A schematic process is shown in Figure 2 below.

To achieve content-based recommendation, the general steps include (1) constructing an item feature representation based on the content features of each item; (2) building a user feature representation based on user characteristics and behaviors; and (3) generating a recommendation list according to the level of alignment between the features of the target item and those of the user [57]. When integrated with neural networks, the entire recommendation model can be divided into an input layer, a hidden layer, an output layer, and a prediction layer. A content-based recommendation model is shown in Figure 3.

The hidden layer is a fully connected layer consisting of 1–2 sublayers, and the score of the item is output [58]. This model highlights the embedding and transformation of item features. The transfer process from the input layer to the next is represented as shown in Equation (2).

$$h=f ( W_{ ih}x+b_{ h})$$

(2)

The transfer process between hidden layers is depicted as shown in Equation (3) (if multiple hidden layers are present).

$$h_j=f(W_{hj}\hspace{0.17em}{h}_{j-1}+b_j)$$

(3)

The transfer process to the output is shown in Equation (4).

$$y=f(W_{ho}\hspace{0.17em}h+b_o)$$

(4)

where x represents the input and y signifies the output. w and b are parameters, while f is a nonlinear activation function.

The process of representing items and users via tagging is becoming increasingly popular. Tags can serve as content information and be used in content-based recommendation methods [59], or they can be treated as a third entity, utilizing collaborative filtering approaches for recommendation. However, as data representations become more diverse, the preprocessing tasks for items become increasingly challenging. A one-dimensional vector representation can be overly simplistic, thereby neglecting the intrinsic relationships between various attributes of the given content. A tensor is a multidimensional data structure that is capable of representing the co-occurrence relationships of multiple types of attributes. Tensor decomposition is fundamentally a feature extraction method, and many algorithms that leverage tensor decomposition for model construction have been developed in the past. Peng et al. [60] constructed a tensor for user–item–tag triples and employed Tucker decomposition to obtain abstract representations of users. The core tensor obtained from the decomposition procedure encapsulates the associations between different attributes, thereby taking the cooccurrence relationships between items and tags into consideration.

Unlike traditional methods such as prod2vec, which typically only considers user interactions to create embeddings, Meta-prod2vec not only utilizes tags but also incorporates additional information, such as product descriptions and reviews, to generate richer product embeddings [61]. It utilizes product-side information to produce product embeddings. By integrating meta-information with interaction data, new embedding vectors are generated for products. The future of content-based recommender systems is inextricably linked with advancements in deep learning. Furthermore, feedback loops derived from users can enhance the adaptability of the utilizing model, ensuring continuous user profile learning and updating.

When dealing with large and complex user-item interaction matrices, traditional matrix factorization models tend to underperform [62]. The paper titled “Deep learning” [63], published in Nature, serves as a fundamental contribution to the field of deep learning. It presents the concept of deep learning by demonstrating how computational models with multiple processing layers can effectively represent data with varying levels of abstraction. These methods achieve significantly improved performance in various tasks. The paper delineates the fundamental equations of neural networks, backpropagation, CNNs, and RNNs, offering readers a comprehensive overview of the domain’s evolution, key concepts, and challenges.

Currently, the content-based recommendation is extensively employed in industrial-scale recommender systems, as it possesses the following advantages.

• Personalized recommendations: These recommendations are based on the user’s historical interests, ensuring that the recommended content aligns with the user’s preferences.
• Simple principle with strong interpretability: Content-based recommendations can be made based on label dimensions or by embedding items into a vector space using similarity, making this strategy easy to implement. It is also readily accepted and validated by users.
• Addresses the cold-start problem to some extent: As long as sufficient content attributes are available, new items can be effectively handled without relying on other users’ behaviors.

However, some of the drawbacks of content-based recommendation led to limitations in its effectiveness and scope of application. It has a narrow recommendation scope, and its novelty is not pronounced. The results obtained from content-based recommender systems tend to converge on categories of items that the user has previously shown interest in. New users, without sufficient interaction histories, might not receive effective recommendations. Moreover, the comprehensiveness, integrity, and accuracy of content understanding can impact the efficacy of the recommendation process. Additionally, based on practical experience, the recommendation accuracy of this approach is not particularly high.

Overall, content-based recommendation is a fundamental technique within recommender systems. It suggests items by analyzing item feature vectors and user interest vectors. The aim is to propose new items to users that match their preferences. But people’s preferences change over time. So, sequence recommendations came into being.

3.2. Sequential Recommendation

Traditional recommender systems focus on identifying static connections between users and items, often neglecting dynamic shifts in user preferences over time. Sequential recommendation approaches posit that the recommendations given to a user at a specific moment should be determined based on the user’s prior behaviors prior to that moment. Sequential recommendation captures the dynamic nature of user behaviors, revealing certain relationships between items over a given time span. Sequential recommender systems can capture dynamic changes in user interests by modeling their interaction sequences. The issue of sequential recommendation entails time-based learning to anticipate user interactions with items. The aim of sequential recommendation is to explicitly model users’ sequential behaviors. We delve into the intricacies of sequence-based recommendation models. To offer a more intuitive understanding of this concept, Figure 4 presents the architectural diagram of a typical sequence-based recommendation model.

As illustrated in Figure 4, sequence-based recommendation models are capable of capturing the dynamic changes in user behaviors, thereby generating more personalized and timely recommendations. In its input layer, such a model receives a sequence of user behaviors, which commonly represent the user’s historical interactions. These sequential data are subsequently encoded into high-dimensional vectors. Advanced feature extraction is then performed via attention mechanisms or neural networks. Following this step, in the hidden layer, a range of more complex network architectures, such as long short-term memory (LSTM) or transformer models, are employed to capture the temporal dependencies and other latent patterns within the sequence. Finally, the output layer typically comprises a fully connected layer and an activation function, employed to compute scores or probabilities for each potential recommended item.

In summary, the development of deep learning in recommender systems has witnessed an orchestrated blend of innovations, each building upon its predecessor. These advancements, rooted in complex mechanisms and models, have progressively enhanced the precision and context awareness of recommendations. Sequence recommender systems (SRS) can be categorized into three groups: standard sequence recommendation, long-short term sequence recommendation, and multi-objective sequence recommender systems.

3.2.1. Standard Sequence Recommendation

Standard sequence recommendation refers to recommendation algorithms that extract user representations using a single behavior sequence [64]. Traditional sequence recommendation algorithms incorporate various prevalent sequence modelling techniques, such as pooling, RNNs, CNNs, memory networks, and attention mechanisms. This approach has been employed in Google’s recommendation model. Due to its simplicity and efficacy, it stands as one of the most prevalent techniques for leveraging sequence features. Neural networks have also found extensive applications in sequence recommendation tasks.

In the initial stages of sequence-based recommendation models, the primary methods employed are Markov chains [33] and feature-based matrix factorization [65,66]. Compared to deep learning techniques, Markov chains have limitations in terms of handling complex data. Researchers employ personalized Markov chains to model individual users, leading to enhanced efficiency. Some people have pointed out that the Markov chain approach underperforms when modelling joint-level sequence patterns and fails to capture the effects of skip behaviors between users and items on recommendation outcomes. While factorization-based methods might model sequences by summing up sequence item vectors, they neglect the influence of order within the user interaction sequences on the recommendation results. Traditional sequence recommendation methods primarily utilize Markov chains and their derivatives for predicting a user’s next action. They operate under the assumption that a user’s subsequent action is solely dependent on their recent few actions.

Compared to traditional recommendation models, deep learning models have demonstrated superior capabilities with respect to capturing the evolution trends of user sequences. The evolution of deep learning in sequential recommendation can be delineated via a series of transformative advancements. Recurrent neural networks (RNNs) have demonstrated their potent capabilities [67] and have been extensively utilized in sequence-based recommendation. Sequence recommendation models based on RNNs can be represented as Equation (5).

$$h_t=\sigma ( W_{hh}\hspace{0.17em}{h}_{t-1}+W_{hx}{x}_t+b_h )$$

(5)

where x_t represents the input and h_t denotes the hidden state. Finally, a different activation function for output is selected. Some people introduced CNNs to sequence recommendation tasks [68]. They underlined the fact that the existing Markov chain models can only model point-level sequential patterns and fall short when representing set-level patterns. They employ convolutional layers to extract local features from the user’s historical behavior sequence, capturing these features from different embedding perspectives (vertical and horizontal). Different convolutional kernels are utilized to extract features from the input sequence. These features are subsequently used to generate the final recommendation list. An attention mechanism offers a notable solution to this gap.

Based on this foundation, SASRec [34] introduced a sequence recommendation method founded on self-attention to improve its effect. Their approach employs a multihead self-attention mechanism and positional encoding to handle the input user behavior sequence. However, for an extensive sequence, earlier interactions might be overlooked. Consequently, researchers have begun to investigate recommendation models that integrate both long- and short-term interactions.

3.2.2. Long- and Short-Term Sequence Recommendation

The evolution of long- and short-term sequence recommendation reflects the progression of recommender systems from simple collaborative filtering methods to sophisticated deep learning models. With continual advancements in technology, models are becoming increasingly adept at distinguishing and integrating two user preferences, thereby facilitating more accurate and personalized recommendations.

Long-term preferences reflect their relatively stable interests, while their recent preferences indicate their recent interest changes [69]. Although user preferences change dynamically over time [70], some long-term stable preferences that influence user behaviors remain. The existing approaches tend to combine two preferences [71].

Taking into account that interactions between two behaviors may exert diverse effects on the user’s present interests [72]. Compared to recent interaction sequences, long-term interactions tend to evolve more slowly [73], thereby reducing the demand for real-time responsiveness [74]. SHAN [75] categorizes user behaviors into two behaviors, employing a multilayer attention network for modelling purposes. By coupling users’ two preferences, an optimal user representation is generated, enhancing the output recommendation results. This hybrid representation is calculated as shown in Equation (6).

$$u_t^{hybrid}={\mathrm{\beta }}_0{u}_{t-1}^{long}+{\displaystyle \sum _{j\in S_t^u}{\mathrm{\beta }}_j{\mathrm{v}}_j}$$

(6)

where v_j represents the rating embedding of an item, which is indicative of the short-term embedding, while u_t−1 denotes the long-term embedding. β denotes varying attention scores. This formulation captures the dynamic nature and differentiates between the contributions of items towards predicting the next.

RNNs encounter challenges like gradient vanishing and gradient explosion when dealing with lengthy sequences, diminishing their proficiency in handling extended sequences. To tackle this problem, long short-term memory networks (LSTMs) [76] were initially introduced, introducing the fundamental concept of gating mechanisms that effectively addressed the issue of vanishing gradients.

Recommender systems often use LSTM or gate recurrent units (GRUs) for user modelling. LSTM is an enhanced version of an RNN [35]. It has been demonstrated that LSTM outperforms conventional RNNs in sequence recommendation tasks. A GRU, another variant of LSTM, is computationally simpler yet equally effective. Recognizing that the two-term impacts of preferences might differ, attention mechanisms have also been employed. The attention mechanisms used in deep learning stem from the idea that humans are drawn to the significant parts of a target. Their origin can be traced back to the research performed by Bahdanau et al. [77]. They used attention mechanisms to model the importance of the output using different parts of the input sentence. Building upon this foundation, vanilla attention was introduced to function as an RNN decoder, and it has been extensively used in sequence recommendation [78]. However, the self-attention mechanism (originating from the transformer in Google’s 2017 neural machine translation work [79]) has also been deployed in sequence recommendation. Contrary to vanilla attention, self-attention does not involve an RNN structure, but it has achieved better performance in recommender systems than RNN-based models [80].

Building upon this idea, a GRU [81] offered streamlined versions of the LSTM gating processes. Venturing into the realm of natural language processing, the transformer model [82] gained widespread acclaim. It established unparalleled standards in terms of discerning the dependencies between sequences, which paved the way for swift parallel computations and accelerated sequence information extraction [83].

This foundation was ingeniously adopted in Alibaba’s DIN model. The self-attention mechanism of this model was integrated within the recommendation domain, amalgamating both fundamental user details and context information. SDM [36] takes a nuanced approach by segregating user interests. It employs LSTM, complemented by multi-head attention mechanisms, to home in on immediate user inclinations. The culmination of this model witnessed the amalgamation of both fleeting and lasting user interests, encapsulating the user’s essence in a vector representation. Compared to previous models, this approach employs gating mechanisms to act as weights, ultimately outputting a user behavior vector, as shown in Equations (7) and (8).

$$G_u^t=sigmoid\hspace{0.33em}(W^{ 1}{e}_u+W^{ 2}{s}_t^u+W^{ 3}{p}^u+b)$$

(7)

$$o_t^u = (1- G_t^u)\odot p^u+G_t^u\odot \hspace{0.17em}{s}_t^i$$

(8)

Here, e_u is the user embedding, while s_u and q_u are data derived from user embeddings processed via different attention networks. This facilitates a more effective data processing strategy. Unlike prior works that simply used parameters as weights, this current approach enables short-term behaviors to be better integrated.

Finally, the Trans2D model [84] showcased a pioneering expandable attention mechanism. This module adeptly discerns user predilections for specific item attributes, which are gleaned from their behavioral sequence data, incrementally elevating the efficacy of the recommendation process. Compared to recommendation algorithms based on RNNs, those relying on convolutional neural networks (CNNs) present greater challenges. Tuan et al. [85] devised a 3DCNN model for high-dimensional modelling, demonstrating its efficacy when the given user sequences are associated with intricate element features. Yan et al. [86] encoded interaction sequences into three-dimensional vectors, employing 2D convolutional filters to learn local characteristics.

Tang and Wang et al. [87] proposed the Caser model, building upon the foundations of prior convolutional techniques. They employed both horizontal and vertical convolutional filters to capture continuous patterns in pointwise, conjoint, and skip behaviors, achieving promising results. The vertical convolution operation is shown in Equations (9) and (10).

$$c_i^k={\mathrm{\Phi }}_c(E_{i:i+h-1}\odot F^k)$$

(9)

$${\tilde{c}}_i^k={\displaystyle \sum _{l=1}^L{\tilde{F}}_l^k\cdot E_l}$$

(10)

where E serves as a matrix formed by stacking embeddings to facilitate the convolution operations. F^k slides over E from top to bottom during the horizontal convolution process, capturing the user’s behavioral patterns. Behavioral patterns are how users interact with different items at different points in time. The vertical convolution operates in the embedding space, focusing on how to group or categorize different items based on their embedding representations. By combining these two types of convolutions, the model is capable of capturing user behavior patterns in two distinct dimensions. It also adapts the embedding representations according to the dynamics of the user behaviors. However, this approach separately learns embeddings for users and items, failing to capture inherent interaction information and overlooking contextual details.

GRU4Rec [37] integrates an RNN into session-based recommender systems, treating the interactions within a session as the historical context for sequence modelling. The highlight of this work is its rational optimization of sequence recommendation based on an RNN. With continued research advancements, a multitude of improved models have emerged.

These models aim to delve deeper into item information by optimizing their model architectures and training methodologies, thereby enhancing the produced recommendation outcomes. One such enhanced RNN model evolved from GRU4Rec employs data augmentation techniques to bolster the stability of the model training process [88]. Another modified RNN model adjusts for batch generation by considering dwell time to more precisely capture user behaviors, discovering the potential impact of dwell time on the results [89]. However, a significant challenge is that when all candidate items rank above popular items based on a popularity-driven sampling method, the learning speed of the developed model is constrained, particularly when recommending long-tailed items. Addressing this issue, an RNN model with top-k gains [90] introduced a novel sampling strategy for GRU4Rec, blending uniform sampling with popularity-based sampling to markedly improve performance. To enhance the modeling of item information, not only was the item’s ID considered but other features such as text descriptions and images were also incorporated. This was effectively realized using a parallel RNN (p-RNN) architecture for efficient recommendation [91]. In terms of training strategies, the authors innovatively introduced methods such as alternating training, residual training, and interlaced training to optimize the performance of the p-RNN. Furthermore, Dietmar explored multiple approaches to combine a session-based KNN classifier with GRU4Rec, such as switching, cascading, and weighted blending, further enhancing the accuracy of the output recommendations [92].

The RUM model [93] incorporates a user memory module to preserve such interaction information. Utilizing a dual-layer memory mechanism, the model employs an external memory mechanism to simulate the user’s memory process, thereby capturing both the long-term and short-term preferences of the user. Recurrent structures are employed to capture the user’s recent activities, whereas an external memory unit stores the user’s historical behaviors. Attention mechanisms are applied to select the most relevant historical information. A multilayer network architecture is finally used to capture deep interaction information. By dynamically updating the memory unit, the model reflects user preference changes, as indicated in the following Equation (11).

$$ad{d}_i=\tan \mathrm{h}\hspace{0.17em}(A{\hspace{0.17em}}^{T}q{\hspace{0.17em}}_i+b_a),m_k^u\leftarrow m_k^u+z_{i\hspace{0.17em}k}\cdot ad{d}_i$$

(11)

The above equation is utilized to update the user’s preference memory. A and b_a act as trainable additive parameters with the aim of erasing existing information prior to the incorporation of new data. Transformers have achieved significant breakthroughs in NLP tasks, with substantial pretrained models such as BERT leading the way. Bert4Rec [94] adapts this architecture for recommender systems.

To achieve optimal recall results, it is essential for the employed system to account for both the long- and short-term interests of the target user. In practical applications, recommendation models can discern these two preferences from the user’s historical sequence and ultimately merge them using gated unit modules.

3.2.3. Multi-Objective Sequence Recommendation

Researchers have recognized that in addition to relevance, diversity and novelty are also important objectives in recommender systems. This implies that these systems should not only recommend items that users are likely to enjoy but also those they have not previously encountered. In the realm of multiobjective sequence recommendation, understanding and modelling user behaviors is one of the core tasks. Traditional methods often overlook the personalized needs of users when addressing different learning objectives [95]. As time has evolved, researchers have started to realize that user interests are not monolithic and often span multiple domains, prompting the emergence of a series of methods. These methods attempt to capture this diversity by encoding the target user’s sequential behavior into different interest representation vectors [96]. In simple terms, the input is subjected to grouped convolution operations, which are determined by coupling coefficients obtained via an iterative dynamic routing process and subsequent weighted embedding vectors. Through such convolutional operations, each group of outputs can be regarded as an embedding vector corresponding to a specific interest.

Complexity increases when users exhibit a diverse array of behavioral sequences in real-world applications, such as clicks, shares, and purchases. Researchers need to model multiple types of behavioral sequences to uncover users’ genuine preferences more comprehensively [97]. This issue also underscores the critical importance of explicitly considering users’ historical interactions to enhance performance [98]. Further studies have revealed that the actions within a user’s behavioral sequence are often heterogeneous and polysemic [99,100]. To capture the underlying intents of these actions more effectively, researchers have started assigning varying weights to different types of behaviors. For instance, a purchase action is typically considered a stronger indicator of a user’s genuine interest and preference for a product or service compared to a mere click action. Therefore, distinguishing different types of behaviors becomes particularly critical [101]. This concept has been concretely implemented in the CBS model. CBS categorizes the given behavioral sequence into target sequences and supporting sequences based on the types of behaviors observed [102]. This enables the target sequences that are closely related to the most predictive types of behavior (e.g., purchasing) to be highlighted. The authors proposed a novel approach that integrates a basket representation into a recurrent layer to capture sequential effects. The implicit recurrent representation of h_t is presented as shown in Equation (12).

$$h_t=g({\mathrm{\Phi }}_b b_t+{\mathrm{\Phi }}_h{h}_{t-1}+{\mathrm{\Omega }}_h)$$

(12)

This approach takes both temporal ordering and continuity into account. The formula explicitly incorporates basket representation, placing particular emphasis on the influence that user choices at specific time steps exert on the hidden state. This distinguishes it from more traditional sequence recommendation methods.

A similar line of thinking has also been implemented in BINN [103]. BINN aims to capture users’ current interests by utilizing all types of behaviors, such as clicks, purchases, and favorites, while exclusively employing behaviors related to purchases (e.g., buying, adding to cart, and favoriting) to reflect users’ long-term preferences.

In another study [104], the authors attempted to integrate the specific representations of each type of behavior with the corresponding item embedding vectors, aiming to capture users’ interests more comprehensively. The researchers integrated a masked beam search and determinantal point process (DPP) selection to produce a high-quality and diversified bundle list with an appropriate bundle size. The formula is presented in Equation (13).

$$masked\_soft\max {(h_t,E,m_t)}_j=\frac{\exp (h_t^T{e}_j-m_{t,j})}{{\displaystyle {\sum }_{\widehat{j}=1}^N\exp (h_t^T{\widehat{e}}_j-m_{t,\widehat{j}})}}$$

(13)

Specifically, the beam search retains the top k most promising candidate bundle lists in each step and subsequently extends these lists in the next step to progressively construct longer recommended bundle lists. This paper employed a beam search to generate more accurate and relevant bundle recommendation lists while also ensuring the computational efficiency of the model.

After all, with the continual advancements achieved in deep learning and natural language processing technologies, sequential recommender systems are well-primed for further optimization and development. More advanced model architectures, such as transformers and BERT, can be employed to capture complex sequential patterns. Moreover, integrating these techniques with other recommendation approaches, such as those based on knowledge graphs, may enhance both the diversity and accuracy of sequential recommender systems. As the Internet of Things and smart home technologies continue to proliferate, the utilization of sequential recommendation is expected to further broaden in everyday life.

Recent research on sequential recommendation remains highly active, with some researchers considering temporal information in sequential recommendation [105]. They aggregate sequential information and collaborative signals in user behavior sequences, thereby taking a more comprehensive approach to considering the information in sequential behaviors. In order to enhance the transition probabilities between items in sequences, researchers employ contrastive learning for sequential recommendation [106]. They introduce two informative augmentation operators leveraging item correlations to create high-quality views for contrastive learning.

Traditionally, recommender systems have often employed static strategies to characterize the interactions between users and items, relying on long-term historical behaviors to infer user interests. However, the selection of a particular item by a user is not solely based on their long-term stable preferences but is increasingly driven by their fluctuating short-term interests [107]. In fact, user interests dynamically evolve over time. This is precisely where sequential recommendation comes into play. Sequential recommendation models the user-product interaction history as a dynamic sequence and utilizes the temporal dependencies within this sequence to capture the evolving user preferences, leading to more precise and timely recommendations. Deep learning enables the hierarchical treatment of complex problems, swiftly identifying the latent patterns and relationships between different layers of data. The integration of deep learning with a sequential recommendation can also adjust recommendations based on dynamic user preference changes, further enhancing their effectiveness [108].

Sequential recommender systems excel at accurately predicting users’ immediate interests and dynamically adapting to their behavior changes and shifting preferences. This approach not only guarantees content diversity but also furnishes recommendations that are highly pertinent to users’ recent activities, even if these suggestions may not fully correspond with users’ earlier behavioral patterns. While sequential recommendation undoubtedly offers unique advantages, it is important to mention that handling time-series data can introduce additional computational demands on the model [109]. Furthermore, an undue emphasis on users’ short-term actions carries its own set of risks, potentially leading to a neglect of their long-term interests and preferences.

3.3. Cross-Domain Recommendation

In the preceding sections, our discussion was primarily focused on generating recommendations based on users’ historical behaviors and content characteristics, usually within a single application scenario or domain. However, to reap benefits from a variety of services, users frequently engage with multiple social media platforms [110]. Cross-domain recommendation (CDR) has been introduced to utilize richer information from multiple domains, aiming to improve recommendation performance. Studies indicate strong correlations between user-generated data across different domains [111]. Traditional data mining techniques are primarily designed for single-domain analysis, and by overlooking data from other domains, they often encounter issues related to data sparsity [112]. Consequently, addressing the utilization of cross-domain data to enhance the comprehensiveness and accuracy of recommendations has become a prominent research focus. By aggregating data from different domains, CDR methods not only compensate for the missing information in a single domain but also holistically harness the value embedded in social media data. The framework is shown in Figure 5 below.

As depicted in Figure 5, the input layer is designed to accommodate user behavior and item attributes from disparate domains. After performing feature extraction, the data acquired from these diverse domains are aggregated into a shared hidden layer, which is tasked with capturing cross-domain patterns and relationships. Within the hidden layer, neural networks can be employed to integrate information from various domains. By employing this design, the model can generate an output layer that synthesizes information from multiple domains, thereby facilitating more comprehensive and accurate recommendations.

Cross-domain recommender systems aim to improve recommendation performance [113] by utilizing data from various domains, particularly in scenarios with data sparsity or cold start challenges. These systems take users’ behaviors and preferences across different domains into account, harnessing comprehensive multidomain information to provide a more holistic user profile and, in turn, bolster recommendation accuracy. Cross-domain recommender systems exploit multidomain learning and knowledge transfer [114], diversifying their recommended content to satisfy the needs of varied scenarios and domains. However, multidomain data processing and the associated knowledge transfer step may also result in increased computational costs.

Traditional recommender systems grapple with two main challenges: the cold start problem and data sparsity [115]. CDR methods offer an effective remedy for these issues by analyzing user interactions from other domains to capture user preferences from specific perspectives. This can be employed to enhance data in the target domain or supplement information during the initialization process for new users. While traditional recommender systems build and analyze a recommendation model within the current domain, cross-domain recommendations necessitate deciding which information should be transferred between distinct domains and the means of performing such transfers. Cross-domain recommendation approaches can harness user interaction data from other domains for auxiliary analysis purposes, capturing certain user preferences. This enables enhanced recommendations to be produced in the target domain or even across multiple domains and provides additional information for new user initialization, thereby addressing the two challenges that are endemic to traditional recommender systems.

In recent years, researchers have focused on addressing data sparsity and cold-start issues in the target domain for cross-domain recommendation. They have conducted extensive studies on deep learning-based cross-domain recommendation methods. These approaches consider deep auxiliary information, including neighboring domain descriptions and rating information [116,117,118]. For instance, a multilayer perceptron (MLP) utilizes the latent features of common users in the source domain as inputs and generates latent rating features for those users in the target domain as outputs during the integration step. Through this process, the network is trained, obtaining a nonlinear mapping function. Optimization is finally achieved via backpropagation, resulting in improved recommendation outcomes. Later, Natarajan, S et al. proposed CD-SemMF [38], which uses the semantic relevance of a project to obtain better results.

In the realm of CDR, multiple domains are encountered, each with an information volume denoted as IN. Within this framework, Domain A possesses dense data, and a newer Domain B is characterized by data sparsity and suboptimal recommendation performance. The inequality INA ≫ INB suggests that the information volume in Domain A significantly surpasses that of Domain B. This discrepancy prompts the consideration of whether information from Domain A can be transferred for utilization in Domain B, a process termed cross-domain transfer. Domain A is designated as the source or auxiliary domain, symbolized as D_S, whereas Domain B is identified as the target domain, denoted as D_T. The source domain can comprise multiple distinct domains, which are aimed at supplementing and enriching the information within the target domain D_T. The respective user sets in domains S and T are U_S and U_T, the item sets are I_S and I_T, the rating sets are RS and RT, and the comment sets are S and C_T, respectively. Thus, the objective is to transfer the knowledge embedded in the source domain to enhance the recommendation quality of the target domain. The distinction between cross-domain recommendation and other methods lies in its feature input, which consolidates multiple domains. Let V represent the user’s behavioral features; user embedding for cross-domain recommendation is illustrated in Equation (14).

$$u_i=[w_1\cdot f(u_i^s)+w_2\cdot u_i^t\hspace{0.17em},v_1^h\hspace{0.17em},v_2^h\hspace{0.17em},...,\hspace{0.17em}{v}_n^h]$$

(14)

Cross-domain recommender systems comprise three pivotal modules: a user modelling module, an item modelling module, and a recommendation algorithm module, as illustrated in Figure 6.

CDR seeks to merge data from various domains by integrating supplementary information from other domains to improve recommendations within the target domain or even across multiple domains. However, it is not a matter of arbitrarily combining any group of domains for cross-domain recommendation purposes; typically, there should be some overlapping information among the different domains. Compared to conventional recommender systems, CDR necessitates meticulously considering which information should be transferred between domains and the methodology used for such transfers.

CDR tasks are closely linked to user-related aspects, with two fundamental factors driving this connection: the range of items for recommendations and the diversity of target users, giving rise to a variety of recommendation scenarios. CDR algorithms can be classified according to the target domain into three categories: single-domain CDR, cross-domain CDR, and multi-target CDR.

3.3.1. Single-Target CDR

Single-target CDR is the conventional approach in the CDR domain, and most CDR methods primarily focus on this scenario. It involves recommending items to users in the target domain by utilizing information acquired from the source domain. Single-objective CDR methods predominantly focus on transferring valuable knowledge.

There are also various recommendation methods, it initially establishes connections based on common content elements such as user/item attributes [119], ‘like’ data, and browsing or viewing histories [120]. Subsequently, they employ these connections for the cross-domain transfer of user/item data or knowledge. Embedding-based transfer methods often project instances from both the source and target domains into a novel feature space, where instances from both domains display similarities. Other methods often adopt distinct techniques or philosophies, including Bayesian latent factor models, interest drift, ternary relations, and reinforcement learning. Rating pattern-based transfer methods typically begin by learning the distinct rating patterns of users from the source domain. Subsequently, these patterns from the source domain are harnessed to enhance the model’s recommendation performance in the target domain, facilitating knowledge transfer. Shapira et al. and Tiroshi et al. [121] both focused their efforts on leveraging social networks to enhance recommender systems. While the former harnessed Facebook’s friend relationships to reinforce user models in the target domain, the latter delved deeper, employing random walk algorithms to extract latent user information within social networks. These approaches can be perceived as endeavours to uncover both explicit and implicit user relationships within the constructed network.

Subsequently, Jiang et al. [122] implemented a semisupervised transfer learning methodology, exploring the user interest similarities between the source and target domains, thereby indicating the transferability of user interests across different domains. Their research contradicts the traditional belief, suggesting that even with a small number of overlapping users, valuable information can be provided for the entire system. By extracting and utilizing information from overlapping user groups, semisupervised transfer learning approaches can be employed to deal with the issue.

However, these studies also highlighted challenges, emphasizing that the effectiveness of the recommendation process heavily relies on the extent of user overlap across different domains. This identified limitation provides a direction for subsequent research, specifically on how to deliver effective recommendations even when the user intersection is minimal. To exploit these latent and obscured relationships, studies have increasingly pivoted towards multidomain recommendation (MDR). Compared to single-target CDR, the focus of MDR primarily lies in handling recommendations across different domains. Although multidomain methods can be applied within the context of CDR, their scopes in practical applications often face certain constraints. To address data sparsity and the other challenges inherent in multidomain recommendation scenarios, Zhang et al. introduced MCF. Successive studies [123,124,125] integrated more advanced techniques such as feature fusion aiming to further enhance the accuracy of the output recommendations.

In summary, from harnessing both explicit and implicit information within social networks to the transfer of interests across domains and onwards to the exploration of multidomain recommendations, researchers have continuously striven to identify more effective and universal recommendation methods that can cater to user needs across various domains and scenarios.

3.3.2. Dual-Target CDR

Single-target approaches can only utilize auxiliary information from a richer domain to assist a sparser domain. However, the richness of various types of information varies across different domains. If these types of information can be effectively harnessed, it is possible to enhance the recommendation performance in all domains simultaneously, rather than limiting improvements to a single target domain. To address this, dual-target CDR and multi-target CDR have been recently proposed to enhance recommendation performance across dual/multiple domains. To achieve cross-domain recommendations for dual-target domains, researchers have begun to develop various models. These models often incorporate advanced techniques such as graph models and attention mechanisms to facilitate knowledge transfer between different domains.

Research on recommender systems that operate across multiple target domains has been increasingly explored in depth. Zhu et al. [39] were pioneers in that they introduced a dual-target-domain CDR framework called DTCDR. This framework capitalizes on multisource information, ensuring that more detailed embeddings are produced for both users and items. To further integrate this information, the authors employed multitask learning techniques to merge the embeddings of overlapping users.

Building upon this foundation, Liu et al. [126] delved deeper into the embeddings of overlapping users. They created two distinct heterogeneous graphs utilizing ratings and content information from both domains to produce more representative user and item embeddings. Taking hyperparameters and data sparsity into consideration, they achieved more accurate embeddings for these users. Subsequently, the DDTCDR model presented in [40], another dual-target domain CDR framework, took a different approach, emphasizing the bidirectional implicit relationships between users and items. It utilized implicit orthogonal mapping to learn user preferences, allowing for the bidirectional transfer of user embeddings between two domains.

Furthermore, in [127], the authors incorporated graph embedding techniques. They not only utilized connectivity of the user-item graph within a single domain via a novel feature propagation layer but also facilitated bidirectional knowledge transfer between the two domains by employing common users as bridges. Moreover, distinct from prior cross-domain collaborative filtering methods, they integrated both shared user features and domain-specific attributes during the transfer process.

In summary, these studies were dedicated to optimizing the performance of dual-target-domain recommender systems. Their collective efforts have contributed valuably to enhancing the accuracy and robustness of cross-domain recommender systems.

3.3.3. Multi-Target CDR

Multitarget domain CDR is a recommendation to users in either domain for items in both domains. However, its objectives are more intricate, aiming to provide a comprehensive solution to the data sparsity problem. The fundamental concept behind multitarget CDR is to harness additional auxiliary information from multiple domains to enhance recommendation performance. Theoretically, if one can identify and effectively utilize a sufficient number of relevant domains, the longstanding data sparsity issue can be substantially mitigated. Nonetheless, beyond the challenges that are inherent in single- and dual-target-domain CDR scenarios, negative transfer becomes inevitable in practical multitarget-domain CDR contexts. Specifically, with the addition of domains, the recommendation performance of the constructed model might deteriorate, especially in scenarios where the domains are sparse.

The multitarget domain task presents a particularly challenging recommendation scenario, and as of now, no perfect solution tailored to this objective is available. In [128], the authors employed a shared heterogeneous graph to generate richer user and item feature embeddings across domains. The MDCDR [41] utilizes auxiliary data from source domains to improve recommendation quality across multiple domains. However, none of these approaches effectively addresses the negative transfer issue.

As the complexity of multiplatform and multidevice environments intensifies, the demand and significance of cross-domain recommendations are anticipated to escalate further. Advanced models, such as deep learning and graph neural networks, may offer more robust support for cross-domain recommendation tasks. These developments have introduced new challenges such as feature mapping and negative transfer. These emerging research trends inspire us to delve into the challenges within CDR and provide an overview of the ongoing research advancements.

3.4. Social Recommendation

Users’ decision-making processes are influenced by their social circles, and numerous researchers have integrated social relations into recommender systems. The main goal is to integrate users’ social information as auxiliary data into conventional recommendation frameworks, thereby improving the accuracy of the resulting recommendations. Distinct from other recommendation strategies, social recommendation focuses on leveraging users’ social network information to generate more personalized and accurate recommendations. The framework for social recommendation is illustrated in Figure 7.

The primary concept involves aggregating features from both a node itself and its neighboring nodes to generate a consolidated representation. The inputs of this approach are users, items, and the social relationship network among the users. After performing embedding, neural networks are generally employed to handle the relationships between users. The final output consists of item recommendations.

Numerous studies have integrated social relational information into collaborative filtering-based recommendations with the goal of modelling the interrelationships between users. The evolution of social recommendation has spanned three phases. Initially, researchers began to recognize the potential value of trust relationships within a user’s social network for recommendation purposes. Leveraging random walks, a trust-based recommendation method was proposed in [129]. The trust relationships among users were employed as weights to adjust the recommendation outcomes. Subsequently, with the application of matrix factorization techniques, models integrating matrix factorization with social networks gained popularity. Building upon trust relationships, these models endeavored to incorporate both strong and weak social ties into a unified framework, allowing recommendations to benefit from both types of relations [130]. With the help of deep learning, the field has progressed into the current era of graph-based social recommendations. These models usually take data from a user’s social network as input and subsequently utilize neural networks to forecast users’ interests [131].

A graph-based social recommendation has emerged as a focal area of academic interest. The advantages of GNNs in terms of mining graph-structured data have facilitated research on social network recommender systems. GNNs possess the capability to efficiently learn and extract features from graph structures, demonstrating commendable results in user-item representation learning tasks. Consequently, numerous scholars have integrated social networks into recommender system research and employed GNNs to construct recommender system models grounded in graph representation learning. For instance, the GC-MC model [132] and the NGCF model [27] graph convolutions to identify user-item interactions in the original space, yielding enhanced recommendation outcomes in practical applications. Moreover, some researchers have found that adding nonlinear activation functions to GNNs does not significantly improve the accuracy of the resulting recommendations. This observation led to the proposal of a streamlined GNN recommendation model, LightGCN [42], which outperforms NGCF in terms of recommendation performance. LR-GCCF [43] eliminates nonlinear transformations from graph neural networks and substitutes them with linear embedding propagation, thereby reducing unnecessary operations, simplifying the complexity of the model, and still ensuring that satisfactory performance metrics are produced.

Social recommendation algorithms can be broadly classified into three main categories: traditional collaborative filtering methods, deep social recommendation methods relying on graph embeddings, and social recommendation methods based on graph neural networks.

3.4.1. Traditional Collaborative Filtering-Based Social Recommender Systems

In traditional social network-based recommendations, two separate graphs need to be constructed: one representing user-item interactions, and another representing user-user social connections. Therefore, social recommendation algorithms utilizing graph neural networks require knowledge extraction from both graphs simultaneously for accurate inference. They can leverage coarse-grained and sparse user trust relationships to enhance conventional collaborative filtering methods. Essentially, this approach involves the use of similarity metrics to identify users similar to the target user. Subsequently, different weights are assigned to ratings for the target item based on this similarity, facilitating the computation of the user’s rating for that item. The method for determining user-to-user similarity is crucial to this concept. Consequently, existing social recommendation algorithms have explored various designs for graph neural network encoders and adopted different architectures based on their objectives. Upon determining the similarity between two entities, the set of users most similar to the target user can be identified based on their ratings. This, in turn, allows for the recommendation of items that the user may find interesting and ranks them accordingly.

3.4.2. Deep Social Recommendation Based on Graph Embedding

Traditional social recommender systems only integrate explicit trust relationships between users and ignore implicit trust relationships. Graph embedding models represent all entities as nodes and their relationships as edges in a graph. These models transform nodes into vector representations, allowing them to retain their approximate relationships from the graph structure in a Euclidean space. Afterwards, vector distances are computed to predict the scores of candidate items for the target user.

Using the application of graph embedding techniques, it becomes possible to acquire low-dimensional feature representations of users within a social network. These low-dimensional representations enable the inference of detailed trust relationships between users. Consequently, by incorporating the weighted ratings from trusted and similar users for a specific item, predictions regarding the user’s rating for that item can be generated. Figure 8 illustrates the framework of a recommendation algorithm employing graph embedding models.

Prominent graph embedding techniques, such as Deep Walk [133], graph factorization [44], and LINE [134], possess distinct characteristics and limitations. For instance, graph factorization [45] is suited for performing embedding learning in large-scale networks but fails to preserve the network’s global structure and is solely applicable to undirected information networks. In contrast, while Deep Walk reveals certain attributes of an information network using a random walk algorithm, it lacks a comprehensive depiction of the preserved network properties and overlooks the connections between feature vectors. In contrast, LINE effectively maintains both local and global structural aspects of information networks and has found widespread application in recommender systems. Additionally, the prowess of graph embedding techniques in terms of extracting the semantic relationships within structures is evident in applications such as the work performed by Xie et al. [135] on POI recommendations and the research conducted by Yin et al. [136] on low-dimensional vector representations for users and items. Graph embedding representation learning, with its powerful representational capabilities, low memory consumption, and efficient computations, has been widely researched and applied for semantic relationship extraction in network structures [137].

Overall, given the advantages of graph embedding techniques in modelling and optimization tasks, they present broad prospects for various applications and merit further exploration.

3.4.3. Social Recommendation Based on GNN

Over time, neural networks’ applications in the recommendation field have significantly broadened. Graph neural networks (GNNs) represent innovative extensions of neural networks, drawing inspiration from convolutional neural networks (CNNs) and the concept of graph embeddings. They are adept at extracting and representing features in graph-based data. Characterized by their efficiency and scalability, GNNs have demonstrated profound capabilities when handling graph data. Users and items are considered as nodes in the graph, and interactions are represented as edges. This process leads to the transformation of user-item interactions into embedded representations within the graph. By integrating these social relationships into recommender systems and subsequently feeding them into neural networks, more comprehensive and personalized recommendations can be provided.

The goal of a GNN is to acquire a state embedding that captures the neighborhood information of each node. The embedding is a z-dimensional real-valued vector associated with node u that can be used to generate a node label output, representing the vector output of node u at a given network layer. Local transfer functions are shared across all nodes and update node state based on input neighborhood. Finally, the output is produced. Then, based on the characteristics of nodes, the characteristics of adjacent edges, and the state embeddings and features of adjacent fixed points, the social relationship model is further improved, for example, by considering the impact of various social relationships on user behaviors. Attention mechanisms, among other techniques, have been introduced [138]. Broadly speaking, the architecture of a social recommendation model comprises three key components: an encoder, a decoder, and a loss function. The encoder [139] utilizes different graph neural network encoders to represent users and items as low-dimensional vectors (i.e., embeddings). The decoder then predicts each user’s preference for each item by performing different operations on the obtained user and item embeddings. Optimization is finally achieved using various loss functions [140]. The performance of social recommendation methods based on GNN largely depends on their encoders.

Beyond user behaviors, social recommender systems also take various types of information into account, including social interactions and shared interests. They operate on the premise that users are likely to share similar interests with friends or contacts within their social networks, utilizing these social relationships to enhance the accuracy and relevance of recommendations. Additionally, the strength and depth levels of social ties are considered to enhance the produced recommendation outcomes. For new users, the cold-start problem can be mitigated via their social networks. Furthermore, recommendations stemming from friends are more likely to be trusted and accepted by users. However, social recommendations exhibit some limitations. Utilizing users’ social network data may raise privacy and security concerns [141]. Moreover, not all social interactions correlate with genuine preferences, necessitating the filtering of irrelevant or misleading information.

Graph neural networks offer robust mechanisms for recommender systems to capture the intricate interactions and relationships between users and items [142], with several models demonstrating significant influences and innovations. Initially, GraphSAGE [28], a pioneering spatial GNN model, marked an essential milestone. It moved away from relying on the entirety of graph information and instead strategically sampled the neighbors of target nodes and combined their embeddings. This approach achieved efficient target embedding updates and offered a viable tool for providing social recommendations in large-scale networks. Following GraphSAGE, the GAT [143] further refined the spatial GNN concept, addressing the limitations of the previous spectral methods regarding key issues such as model generalization. Notably, the GAT introduced an attention mechanism, enabling the model to allocate different weights based on the importance of neighboring nodes, leading to a more selective neighborhood feature aggregation process. This strategy enhanced the model’s discriminatory capacity, achieving commendable results in social recommendation tasks. For dealing with heterogeneous graph structures, HetGNN [144] emerged as a representative technique. Distinct from traditional GNN models, HetGNN devises specific aggregation strategies for nodes and edges that are present in heterogeneous graphs. Initially, it segregates neighbors based on their types and then employs LSTM and MEAN operations to individually process these subsets, effectively capturing the abundant structural and attribute information inherent in heterogeneous graphs. Furthermore, it is noteworthy that HetGNN implements spectral GNN methods on hypergraph structures, propelling recommender system research in a novel direction. Regarding the exploration of social network structures, DiffNet and DiffNet++ stand as pivotal milestones. Both methods emphasize delving into higher-order social structures to achieve enhanced recommendation accuracy and efficacy. DiffNet accentuates capturing higher-order neighbor information within the network to boost its recommendation results [145], while DiffNet++ further refines and extends this base [46]. Following DiffNet and DiffNet++, GraphRec emerged as an innovative approach, emphasizing not only capturing interactions between users and items but also jointly capturing opinions about both users and items [146]. This dual capture strategy is designed to ensure recommendation accuracy while maintaining system interpretability.

With the continuous evolution of social media and platforms, the significance of social recommender systems is set to expand further [147]. Advanced technologies such as deep learning and graph neural networks have notably enhanced the accuracy and efficiency of social recommendation [148]. Moreover, striking a balance between user privacy and the quality of recommendations will emerge as a pivotal research direction [149]. In the future, we may also witness an increase in cross-platform social recommendation solutions, ensuring that users receive consistent and high-quality recommendation experiences across diverse platforms.

4. Challenges and Developments

Recommender systems are designed to help users discover items that align with their preferences from a wide range of potential recommendations. Deep learning can aid researchers in improving the effectiveness of these recommendations. This paper analyzes four distinct types of recommender systems, including content-based recommendation, sequential recommendation, cross-domain recommendation, and social recommender systems. Although the integration of these recommendation techniques with deep learning has achieved satisfactory results, challenges remain [150]. Future research efforts can be pursued in the following areas.

(1) Security enhancements are needed

With the growth of networking sites, accurately recommending items of interest to users has become one of the key strategies employed by various websites to attract users. Only by digging deep into multidimensional user data can recommendations be identified that truly align with user preferences [151]. In reality, while users expect recommender systems to suggest items of interest, they do not want their other private details to be disclosed [152]. Current research primarily involves distorting and obfuscating user data to ensure privacy [153]. Although this data perturbation strategy does protect the user’s personal information, it can lead to the inaccurate extraction of user data, significantly compromising the resulting recommendation accuracy. Therefore, future research could focus on a methodology that not only safeguards user privacy but also enhances recommendation precision.

(2) Methods for extracting user preference features are lacking

At present, recommendation subjects rely heavily on users’ ratings or feedback on recommended items. Current research lacks adequate modeling methods that can multidimensionally extract user features and recommended items [154], as well as their linear and non-linear relationships [155]. Although neural networks can address this issue to some extent, there remains significant room for improvement. Consequently, forthcoming research should incorporate a more diverse range of methods to extract the features of both users and recommended entities.

(3) Evaluation metrics are singular

When evaluating the performance of recommender systems, the existing studies primarily emphasize the accuracy of results and associated precision rate, considering accuracy as the key metric for determining the effectiveness of recommender systems. However, when users interact with these applications in real-world scenarios, they not only expect the system to provide accurate recommendations for items of interest but also seek a broader and more innovative range of suggestions [156]. Hence, the novelty and diversity of the output recommendations should be considered in the future.

In summary, in addition to the aforementioned aspects, there are several directions for future research. There may be an increased focus on the interpretability of deep learning models. This is because, in practical applications, there is a growing demand from users and regulatory bodies for greater transparency and comprehensibility in recommender system decisions. With the proliferation of multimodal data (text, images, audio, etc.), future recommender systems may integrate these data sources more extensively to provide a richer and more diverse recommendation experience. The application of reinforcement learning in recommender systems may become more widespread, optimizing recommendation strategies via user interactions for higher long-term returns.

Future research may explore how to transfer deep learning models from one domain to another to enhance the generalizability of recommender systems. Federated learning, while protecting user privacy, allows different institutions and platforms to collaboratively train recommendation models and may become an important direction in future recommender system research.

Despite improvements in handling sparse data with deep learning methods, data sparsity and cold-start problems remain challenges. Specifically, providing accurate recommendations for new users and items remains a challenge. Privacy and fairness have long been key challenges in the field of recommender systems. Balancing personalized recommendations with user privacy and ensuring that recommendation system decisions are fair will be a focus of future research. Deep learning models often require substantial computational resources. Building scalable and efficient deep learning recommender systems in practical applications remains a challenge. Deep learning methods typically require a large amount of training data, so encouraging user participation in feedback collection to improve the performance of recommender systems remains a question.

5. Conclusions

As technologies like deep learning, data mining, and predictive algorithms mature, future research will focus on enhancing the accuracy, security, and privacy of recommender systems. This article explores both traditional recommendation methodologies and those incorporating various deep learning models. We compare the differences between conventional recommendation models and deep learning-based approaches, summarizing the prevalent challenges in recommender systems. We also offer insights into future research directions for recommender systems, benefiting researchers who are interested in the fields of recommender systems or deep learning.