A Study of Recommendation Methods Based on Graph Hybrid Neural Networks and Deep Crossing

Abstract

In the face of complex user behavior patterns and massive data, improving the performance of recommender system models is an urgent challenge. Traditional methods often struggle to effectively handle feature interactions and complex user-item relationships. Combining the advantages of graph neural networks and the Deep Crossing network, this paper proposes a recommendation method based on hybrid neural networks with Deep Crossing (Deep Crossing with Graph Convolution and GRU, DCGCN-GRU). First, by constructing the graph structure of users and items, higher-order feature representations are extracted, and node features are updated using a multilayer graph convolution operation. Then, the higher-order features learned by the graph convolution network are spliced and weighted with the original features to form new feature inputs. Next, a Gated Recurrent Unit (GRU) is introduced to capture the inter-feature temporal dynamic relationships and sequence information. Finally, the Deep Crossing model is utilized to learn the interactions between the fused features at multiple levels and enhance the interactions between the features. Comparative experiments on three public datasets, MovieLens-ml-25m, Book-Crossings, and Amazon Reviews’23, show that the model achieves significant improvements in accuracy, mean square error (MSE), and mean absolute error (MAE).

1. Introduction

Recommender systems provide personalized recommendations to users by analyzing their historical behaviors, preferences, and social relationships [1], which not only improves user satisfaction and loyalty, but also increases the user stickiness and profitability of the platform. However, traditional recommendation methods often face challenges such as data sparsity, feature crossover and complexity of user-item relationships [2,3,4], which limit the performance and effectiveness of recommendation systems. These user-item relationships can be naturally represented as a graph structure, where nodes represent users and items, and edges represent user interactions or preferences for items. In this graph structure, Graph Neural Networks (GNN), a powerful machine learning tool, is specialized for processing graph data structures, such as user-item relationships in social networks and recommender systems. GNN significantly improves the performance of recommender systems and personalization modeling. Zhu et al. [5] used the semi-supervised algorithm Node2vec graph embedding technique to map the nodes in the graph into low-dimensional embeddings of users and items. Wang et al. [6] achieved higher-order information extraction from graph-structured data in terms of node features and structural features by introducing a GNN to increase the recommendation performance of the model. It is shown that by applying GNN to the recommendation system, the model’s ability to capture user preferences and item characteristics can be significantly improved, which in turn improves the quality of recommendations. Hochreiter et al. [7] proposed Long Short-Term Memory (LSTM), which introduces the mechanism of “gates” to effectively control the forgetting and retention of information. LSTM, which introduces a “gate” mechanism to effectively control the forgetting and retention of information, and thus is able to memorize longer sequences of information. Subsequently, Chung et al. [8] proposed the Gate Recurrent Unit (GRU) as a simplified version of LSTM, which reduces the number of parameters and makes the model training more efficient. Zhang et al. [9] who combined the advantages of external attention mechanism proposed a session recommendation model (SR-BGEAN) with Bi-GRU and external attention network, which improves the interest representation of the model. In this context, Deep Crossing model has also been widely used as a feature learning method.

Deep Crossing was proposed by the company Microsoft in 2016, which completely solves a series of deep learning re-recommender system application problems such as feature engineering, sparse vector thickening, and multi-layer neural networks for optimal goal fitting [10]. The techniques involved in this model are relatively basic, adding ideas such as embedding, residual connectivity, etc. to traditional neural networks, but its simple architecture of Deep Crossing model is unable to capture these complex interactions when dealing with multiple interactions between complex user behaviors and item attributes [11]. The Wide&Deep model [12] combines the traditional generalized linear models and deep neural networks, which solves the problem of difficulty in retaining both model memory capability and generalization capability to adapt to known data and generalize well in recommender systems. However, its reliance on manual feature selection, the need for domain expertise and high labor costs limit its scalability [13]. To overcome the problem of manual feature selection in the Wide&Deep model, the DeepFM model [14,15] introduces the Factorization Machine (FM) [16] algorithm, which effectively improves the training speed and accuracy in recommender systems and eliminates the need for manual feature selection, thus speeding up the process of the model’s real-world application; On the other hand, the DCN (Deep & Cross Network) model [17] improves upon the Wide & Deep model by introducing a cross network layer, which automatically realizes feature cross-combination and reduces the reliance on manual feature selection. This approach not only enhances the model’s ability to capture feature interactions, but also solves the problems of gradient vanishing and gradient explosion in deep neural networks by fitting the residuals, thus improving the model’s expressive ability and training effect [18]. However, DCNs have high computational complexity when dealing with high dimensional data, which affects the efficiency of real-time recommendation [19]. Although existing recommendation methods have achieved certain results, there are still some shortcomings:

(1)

Traditional recommendation methods mainly focus on dealing with interactions between features, but they cannot capture the interactions of sequential information well, and these problems limit the performance and effectiveness of the recommendation system, leading to poor performance of the model when dealing with sequential-type data [20].

(2)

Traditional recommendation models mainly learn the relationship between users and items through feature interactions, usually relying on simple interactions between features, but they cannot directly model the implicit relationships between users and items, such as the user’s interest preferences, potential attributes of items, etc., which leads to the model’s inability to accurately predict the user’s preferences for items.

In order to solve the above problems, this paper proposes a recommendation method based on hybrid neural network and Deep Crossing. The method firstly constructs a graph structure between users and items based on users’ social information; secondly, designs a Graph Convolutional Neural Network (GCN) structure to transform the feature information of users and items into vector representations, and takes them as the input features of the GCN; then, the constructed graph structure and the features are inputted into the GCN for training to learn the representation vectors of users and items to capture the relationship between them; next, their behavioral sequences are processed by GRU to obtain a sequence vector representation to reflect the temporal dynamic features of users and items; finally, the learned user and item representation vectors are used as the features as well as the sequence features extracted by GRU are spliced together and inputted into Deep Crossing model, the Feature crossing and prediction are performed by deep neural network, combined with multi-layer feature interaction and weighted combination to generate final recommendation scores and provide personalized recommendations for users. The main innovations of this paper are:

(1)

Innovatively considers the higher-order relationship features between users and items, and effectively extracts the higher-order feature representations of users and items through GCN. These higher-order features capture the complex interactions between users and items, thus helping to improve the overall performance of the model.

(2)

Introducing the gated loop unit module, the complex sequence information and intrinsic associations between features are mined, which can effectively capture the changing trends of user interest preferences, thus enhancing the model’s understanding of user behavior and matching of items.

(3)

The high-order features extracted by GCN are input into Deep Crossing model for multi-level feature interaction and deep learning. This mechanism not only enhances the interaction between features and improves the expressive ability of the model, but also ensures the accuracy and diversity of the recommendation results and avoids falling into local optimal solutions.

Section 1 of this paper introduces the related work on recommendation algorithms; Section 2 describes the hybrid neural network and Deep Crossing based recommendation method proposed in this paper; Section 3 verifies the effectiveness of the proposed model through comparative experiments; and finally concludes the paper.

2. Related Work

2.1. Recommendation Methods Based on Deep Crossing

Deep Crossing algorithm is a deep learning model for click-through prediction. It consists of a fully-connected neural network that can handle high-dimensional sparse input features and is able to automatically learn non-linear relationships between features. The network structure of the model consists of multiple fully connected layers, each of which contains a linear transformation and a nonlinear activation function. Between each fully connected layer, the model introduces residual connections [21,22] to avoid the gradient vanishing problem. These residual connections add the output of the previous layer to the input of the subsequent layer, allowing the model to learn combinations of features more deeply. To handle high-dimensional sparse input features, the Deep Crossing model employs an embedding layer. The embedding layer maps each feature to a low-dimensional dense vector representation, which reduces the dimensionality of the features and better captures the relationships between features. The parameters of the embedding layer are also learnable and updated by a back-propagation algorithm. Gao et al. [23] proposed Particle Swarm Optimization Deep Cross Neural Network Recommendation Algorithm (PSO-DCN) for global parameter search optimization and to solve the problem of difficult to set hyper-parameters. Keila et al. [24] proposed a deep cross-training methodology (DCTr) based on the ResNet-18 architecture to develop binary and multi-classification models to improve the accuracy of lesion identification. Pei et al. [19] proposed an innovative width-depth crossover neural network (WDCN) model based on the width and depth learning (WDL) model and the DCN model to adapt to the complex correlation relationship between multiple drilling parameters and Rate of Penetration (ROP). The relationships of linear, nonlinear and cross product, which are difficult to be solved by traditional networks, are addressed.

In Deep Crossing model, feature interactions are modeled through a feature crossing layer. The feature crossing layer crosses the embedding vectors of different features and learns the weights between the crossed features through a fully connected layer. This captures the nonlinear relationships between features and allows for a better representation of the combination of features.

2.2. A Recommendation Method Based on GCN

Graph Convolutional Networks (GCN) is a neural network model specialized in processing graph-structured data. Unlike traditional convolutional neural networks, GCN is able to perform convolutional operations directly on the graph, effectively capturing complex relationships between nodes as well as higher-order connectivity information [25]. This enables the GCN model to take full advantage of the connectivity relationships in the graph structure for effective feature extraction and representation. Fan et al. [26] proposed a neural network model for social recommendation that learns the feature representation of users and items by aggregating neighborhood features through the graph structure. On the other hand, Zhang et al. [27] proposed a node-importance based graph pooling method that aims to optimize the model performance by reducing the size of the graph and retaining the key nodes. Wang et al. [28] proposed a graph convolutional network model called HAN that is specifically designed to deal with heterogeneous graphs, which contains graphs with different types of nodes and edges, allowing GCNs to be applied to recommender systems and knowledge graphs in a several domains. Wu et al. [29]. proposed a social recommendation model Social-GCN, which utilizes the propagation properties of graphs and is based on Graph Convolutional Neural Networks (GCN). By introducing GCN, the model aims to simulate the diffusion process of user preferences in a social network and thus learn feature representations more efficiently. These studies show the effectiveness of using the GNN framework for embedding learning in recommender systems.

In summary, this paper proposes a recommendation method called DCGCN-GRU, which aims to alleviate the complex interaction problem between users and items as well as to enhance the interaction between features. The method works by combining the feature interaction capabilities of Graph Convolutional Networks (GCN), Gated Recurrent Units (GRUs), and Deep Crossing in order to enhance the representation of features. GCN is responsible for capturing the complex relationships in the graph structure, GRUs are used to model the feature sequences and capture the temporal dynamics of the user preferences and item features, and Deep Crossing is responsible for digging deeper into the interactions between features. Through the synergy of multiple modules, the method in this paper is able to better capture the deep user-item relationships and enhance the recommendation accuracy and model performance. The experimental results verify the effectiveness of DCGCN-GRU and significantly improve the performance of the recommendation system.

3. Recommendation Method Based on Hybrid Neural Network with Deep Crossing

3.1. Overall Framework

In this paper, we design a recommendation method based on hybrid neural network with Deep Crossing. The overall architecture of the method is shown in Figure 1, which consists of five main parts: (a) Graph Convolution Module, which processes the graph structure data and learns the representation of nodes. (b) GRU Module, which introduces gated recurrent units to sequence modeling of the embedded features, capturing the temporal dynamics and intrinsic associations among the features, and enhancing the learning capability of the model on the changing trends of user preferences. (c) Embedding Layer Module, which converts the input features into embedding vectors of fixed dimensions. (d) Deep crossover network module, which extracts complex feature interactions through the depth and crossover layers, where the depth layer learns feature representations through the network layer and ReLU activation function, and the crossover layer cross-combines the features. (e) Scoring layer module, which calculates the final scores and prediction values.

The overall flow of the DCGCN-GRU is as follows:

First, the input features are converted into fixed-dimension embedding vectors through the embedding layer. Then, these embedding vectors are fed into the GCN module, GRU module, and deep cross network module, respectively. The GCN module extracts the higher-order features of the nodes in the graph structure, while the GRU module models the feature sequences and captures the temporal dynamics among the features. Next, the Deep Crossing Network module learns the complex interactions of features through the depth and crossing layers. Finally, the output features of GCN, GRU and deep crossover network are integrated into the scoring layer to compute the final prediction score.

3.2. Constructing a Network Diagram of User Items

Given a set $U=\{u_1,u_2,\dots ,u_n\}$ of users and a set $M=\{m_1,m_2,\dots ,m_n\}$ of items, where n denotes the number, a set $R=\left\{r_{um}\right\}$ of user ratings of items. Construct the user-item bipartite graph $G$ based on the given dataset, $A_{\mathrm{ij}}$ is denoted as the adjacency matrix of $G$. The user-item interaction bipartite graph is constructed as follows: from a given set $U$ of users, each user $u_{\mathrm{i}}$ acts as a node in the graph. From a given set of items $M$, each item $m_{\mathrm{j}}$ also acts as a node in the graph; The interaction between users and items is reflected in the edges in the graph, if a user $u_{\mathrm{i}}$ has a rating behavior for an item $m_{\mathrm{j}}$, an edge is connected between node $u_{\mathrm{i}}$ and node $m_{\mathrm{j}}$. The weights of the edges can react to the user’s rating of the item, The maximum rating value is set at 5, and the rating matrix is shown in Figure 2.

Based on the above operations, a bipartite graph of rating interactions between users and items is constructed, denoted as $G_{\mathrm{I}}=(U,M,\epsilon )$. ε denotes the set of edges in the graph $G_{\mathrm{I}}$ $\epsilon$, $\epsilon \in [0,5]$, $\epsilon =R$. The rating interaction graph between users and items is shown in Figure 3.

3.3. GCN-Based User Feature Learning

For each user and item, the feature information needs to be transformed into vector representations, which can be Embedding of users and items. The feature vectors of users and items are combined into a feature matrix, assuming that the user feature dimension is $D_{\mathrm{u}}$ and the item feature dimension is $D_m$, then the size of the user feature matrix $X_{\mathrm{u}}$ is $N_{\mathrm{u}}\times D_{\mathrm{u}}$, and the size of the item feature matrix $X_m$ is $N_m\times D_m$, where $N_u$ is the number of users and $N_m$ is the number of items; The feature matrices of users and items are merged to form the feature matrix $X$ of the whole graph.

$$X=\left[\begin{array}{c}{X}_u\\ X_m\end{array}\right]$$

(1)

Construct the adjacency matrix $A$. The adjacency matrix is an $N_u\times N_m$ matrix where the value of $A_{\mathrm{ij}}$ in the adjacency matrix is the corresponding rating value if user $u_{\mathrm{i}}$ item $m_{\mathrm{j}}$ has a rating behavior, and 0 otherwise. The feature matrix $X$ and the adjacency matrix $A$ are used as input features to the GCN, denoted as ($X$, $A$). GCN aggregates the features of neighboring nodes by means of adjacency matrices and also performs linear transformations on these features. The adjacency matrix is combined with the feature matrix to update the feature representation of each node by graph convolution. GCNs usually consist of multiple layers of graph convolution layers, each of which updates the node’s representation. The forward propagation process is as follows:

$$H^{(l+1)}=\sigma ({\widehat{D}}^{-{\displaystyle \frac{1}{2}}}\widehat{A}{\widehat{D}}^{-{\displaystyle \frac{1}{2}}}{H}^{(l)}{W}^{(l)})$$

(2)

where $H^{(l)}$ is the node feature matrix of layer $l$, $\widehat{A}$ is the normalized adjacency matrix, $\widehat{D}$ is the diagonal degree matrix of $\widehat{A}$, i.e., ${\widehat{D}}_{ii}={{\displaystyle \sum }}_j{\widehat{A}}_{ij}$, $W^{(l)}$ is the weight matrix of layer $l$, the user feature change, and $\sigma$ is the activation function.

3.4. Introduction of GRU for Sequence Modeling

The GRU (Gated Recycling Unit) module is introduced, which is specifically designed to dynamically model users’ time-series behaviors (e.g., historical ratings, browsing, or clicking) and capture the trend of user preferences over time. The process of updating the state of the GRU is as follows:

update gate $z_t$:

$$z_t=\sigma (W_z[h_{t-1},x_t]+b_z)$$

(3)

where $h_{t-1}$ is the hidden state at the previous moment, $x_t$ is the input feature at that time, and $W_z$ is the weight matrix of the update gate.

reset gate ${\mathrm{r}}_t$:

$${\mathrm{r}}_t=\sigma (W_{\mathrm{r}}[h_{t-1},x_t]+b_{\mathrm{r}})$$

(4)

Based on the reset gate, the current candidate state is calculated.

Hide status updates:

$$h_t=z_t\ast h_{t-1}+(1-z_t)\ast {\tilde{h}}_t$$

(5)

The final output $h_t$ is used as a time series feature.

3.5. Feature Combination Based on Deep Crossing Modeling

The model is divided into four structures, which are Embedding layer, Stacking layer, Residual Unit layer and Scoring layer, and the model structure is shown in Figure 4.

The Embedding Layer maps discrete features (User_id, Movies_id, etc.) into a continuous low-dimensional vector space that can be processed by successive neural network layers. The network is defined as follows:

$$X_j^O=\max (0,W_j{X}_j^I+b_j)$$

(6)

where j represents the index of the input feature, $X_j^I\hspace{1em}\in \hspace{1em}{ℝ}^{n_j}$ is the input feature, $W_j$ is the matrix of $m_j\times n_j$, and $b\in {ℝ}^{n_j}$, $X_j^O$ are the embedded features. The Embedding layer performs dimensionality reduction on the input features when $m_j<n_j$. The size of Embedding has an important effect on the size of the whole model. In the model structure diagram, it can be found that Feature#2 does not use Embedding, and for numerical features with dimensions less than 256 are directly connected to the Stacking layer.

The main job of Stacking Layer is to aggregate all the input features obtained to form a feature vector containing all the features. Where K is the number of input features.

$$X^O=[X_0^O,X_1^O,\cdots \hspace{0.17em},X_K^O]$$

(7)

Residual Layer is mainly an MLP structure that uses a stacked structure of a series of residual units to help the model better capture the hierarchical structure and nonlinear relationships of the data [30]. Compared to the ordinary neural network with the perceptron as the basic unit, Deep Crossing model adopts a multilayer residual network as the specific implementation of MLP and is improved in that it does not use the convolution kernel, the original input features are added back after the two-layer ReLU transform, and then the inputs are directly added to the output vectors through a short-circuiting pathway that performs the element summing operation with the output vectors, to form the final output vectors. The formula is as follows. The residual unit is shown in Figure 5.

$$X^{\mathrm{O}}=ℱ(X^{\mathrm{I}},\{W_0,W_1\},\{b_0,b_1\})+X^{\mathrm{I}}$$

(8)

where $\mathbf{W}\left\{0,1\right\}$, $\mathbf{b}\left\{0,1\right\}$ represent the two connectivity layer parameters, $ℱ$ is the function that maps the residual unit $X^I$ to $X^O$, ($I$ represents the input and $O$ represents the output), and $ℱ(.)$ represents the residuals of the fitting $X^O-X^I$.

The deepening of neural networks also brings the phenomenon of overfitting, in the residual unit, because of the existence of short-circuit links in the input vectors can be used across multiple layers of ReLU networks, thus reducing the possibility of overfitting [31]; when the number of layers of neural networks is too many, gradient vanishing becomes a common problem, but the use of the ReLU activation function as well as the short-circuit links of the input vectors in the residual unit can help to alleviate the gradient vanishing problem. This is some of the reasons why residual networks are able to go up to 152 layers and maintain high performance.

The Scoring Layer, as the final output layer, uses logistic regression to make predictions about clicks.

3.6. DCGCN-GRU Personalized Recommendations

Predictions are made based on the last learned feature information to form the final recommendation.

First, feature merging is performed, and the user representation vector $h_u^L$ and item representation vector $h_m^L$ output from the GCN model are effectively fused with the time series features learned from the GRU to combine them into a unified feature vector. As well as the feature $h_{DC}$ output from Deep Crossing model is combined into a composite feature vector.

$$H=[h_u^{(L)},h_v^{(L)},h_t,h_{DC}]$$

(9)

Secondly, the obtained feature vectors are input into the model to get the final recommendation results.

$$\widehat{y}=\sigma (W_{1}H+b_1+W_{2}H+b_2)$$

(10)

4. Experiments and Analysis of Results

4.1. The Experimental Setup as Well as the Dataset

The parameters of the article are set as follows: the number of GCN layers is set to 2, the hidden dimension is 128, the learning rate is set to 1 × 10⁻³, the activation function is ReLU, the GRU sequence length is 20 steps, the number of Embedded layers is set to 256, the batch size is 1798, the epoch is 10 rounds, the optimizer uses Adam, and the L2 regularization coefficient is set to 1 × 10⁻⁵.

The evaluation metrics of the recommended method in this paper include Accuracy, Mean Square Error MSE, and Mean Logarithmic Error MAE. The calculation process of the three evaluation metrics is as follows:

$$\mathrm{ACC}={\displaystyle \frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{FP}+\mathrm{FN}+\mathrm{TN}}}$$

(11)

$$MSE={\displaystyle \frac{1}{N}}{{\displaystyle \sum }}_{t=1}^N{(observe{d}_t-predicte{d}_t)}^2$$

(12)

$$MAE={\displaystyle \frac{1}{N}}{{\displaystyle \sum }}_{i=1}^N\mid (f_i-y_i)\mid$$

(13)

where TP denotes that the sample predicted values match the true values and are all positive, FP denotes that the sample predicted values are positive and the true values are negative, TN denotes that the sample predicted values match the true values and are all negative, FN denotes that the sample predicted values are negative and the true values are positive, and both $observe{d}_t-predicte{d}_t$ and $f_i-y_i$ denote the deviation of the predicted values from the true values.

The experiments in this paper selected three public datasets: the MovieLens movie dataset (ml-25m), the Book-Crossings book rating dataset, and the Amazon Reviews’23 dataset. The datasets were split into training, validation, and testing sets in a ratio of 7:1:2. The rating scale for all datasets ranges from 1 to 5. For the Book-Crossing dataset, which originally had a rating scale from 1 to 10, the ratings were adjusted to a scale of 1 to 5 for this experiment.

4.2. Comparison Models

In order to verify the effectiveness of the recommendation method (DCGCN-GRU) proposed in this paper, four personalized recommendation models are selected as comparison models.

NGCF (Neural Graph Collaborative Filtering) model [32]: this model is a collaborative filtering model based on graph neural networks. Its core idea is to represent the interaction data of users and items as a graph structure, and capture the higher-order relationship between users and items through graph convolution operation, so as to improve the recommendation performance.

LightGCN (Light Graph Convolutional Networks) [33]: this model is a simplified and optimized version of Graph Convolutional Networks (GCN) for recommender systems. It improves the computational efficiency and performance of the model by removing complex nonlinear transformations and feature transformation operations.

WDCN model: this model provides a more accurate ROP prediction method by combining width, depth, and cross-feature learning, especially excelling in complex drilling data environments. It demonstrates significant advantages in improving the stability and generalization ability of the model.

Deep Crossing: the model effectively captures the complex relationships between features by stacking multiple residual network modules. Compared to traditional feature engineering methods, Deep Crossing is able to automatically learn richer feature representations, improving the accuracy and efficiency of click-through rate prediction and recommendation tasks.

4.3. Performance Comparisons

Table 1. Data information for MovieLens, Book-crossing, and Amazon Reviews’23.

Data Set	Users	Items	Rating
MovieLens	162,541	60,000	25,000,095
Book-crossing	278,858	271,679	1,149,780
Amazon Reviews’23	362,000	112,600	701,500

presents the data content of this experiment, detailing the key variables and their respective values used in the analysis.

Table 2. Comparison of accuracy, MSE combined MAE values.

Data Set	MovieLens			Book-Crossings			Amazon Reviews’23
Model	ACC	MSE	MAE	ACC	MSE	MAE	ACC	MAE	MSE
NGCF	0.7343	0.0358	0.1506	0.6985	0.0483	0.1745	0.6721	0.0525	0.1820
LightGCN	0.7611	0.0289	0.1336	0.7201	0.0438	0.1620	0.6923	0.0461	0.1704
WDCN	0.7769	0.0417	0.1628	0.7332	0.0512	0.1832	0.7105	0.0553	0.1908
Deep Crossing	0.7659	0.0400	0.1682	0.7265	0.0465	0.1787	0.7017	0.0500	0.1865
DCGCN-GRU	0.8001	0.0200	0.1289	0.7450	0.0345	0.1456	0.7359	0.0310	0.1400

The data in bold indicate the best performance among all methods.

shows the performance comparison of each model, where higher accuracy, lower mean square error value as well as lower mean square logarithmic error value represent better model performance. The experimental results are shown in Table 2.

Based on the data in Table 2, it can be seen that the DCGCN-GRU model performs better in terms of accuracy (Acc), mean square error value (MSE), and mean logarithmic error value (MAE) compared to the comparison models. After comprehensive testing on three public datasets (MovieLens-ml-25m, Book-Crossings, and Amazon Reviews’23), the DCGCN-GRU model has demonstrated significant advantages across various performance metrics, particularly in the critical indicator of accuracy. Compared to the best baseline model, our model achieved performance improvements of 5.12%, 1.62%, and 3.53% on the three datasets, respectively. This significant performance enhancement fully validates the effectiveness and superiority of the proposed model, attributed to several key factors: (1) DCGCN-GRU effectively extracts high-order features by constructing a graph structure of users and items, enhancing the model’s ability to model complex relationships. (2) The introduction of Gated Recurrent Units (GRU) captures the temporal dynamics between features, thereby improving the accuracy and timeliness of recommendations. (3) The use of the Deep Crossing model strengthens the interactions between features, resulting in superior performance in feature fusion.

In order to compare the convergence speed of each model, this paper compares the log loss function of each model, and the log loss function is calculated as follows:

$$\mathrm{logloss}=-{\displaystyle \frac{1}{N}}{{\displaystyle \sum }}_{i=1}^N\left(y_i\log (p_i)+(1-y_i)\log (1-p_i)\right)$$

(14)

where N is the number of samples, $y_{\mathrm{i}}$ is the label of each sample, and $p_{\mathrm{i}}$ is the output of the Scoring layer.

Cross-entropy loss for each model is shown in Figure 6.

According to the figure, it can be seen that as the number of iterations increases, the change in the loss of each model shows a decreasing trend, and the best result is the DCGCN-GRU, and the data shows that the improved model is better than the base model.

4.4. Ablation Experiment

In this section’s ablation study, we conduct a comparative analysis of two key modules in the DCGCN-GRU model: the high-order feature interaction module (implemented with GCN) and the sequential behavior extraction module (implemented with GRU). By removing each of these modules separately, we observe their impact on model performance to understand their individual contributions. DCGCN-GRU-1: This variant removes the high-order feature interaction module and retains only the sequential behavior extraction module (GRU), capturing temporal dynamics without processing the user-item graph structure. DCGCN-GRU-2: This variant removes the sequential behavior extraction module and retains only the high-order feature interaction module (GCN), relying on high-order relationships in the user-item interaction graph while ignoring temporal sequence information. DCGCN-LSTM: This model retains the high-order feature interaction module (GCN) while replacing GRU with LSTM to handle sequential data. DCGCN-GRU is the complete model capable of simultaneously processing both complex user-item graph relationships and temporal behavior sequences. By comparing the performance of these three variants, we gain further insights into the roles of each module within the overall model. As shown in

Table 3. Performance results of different components in DCGCN-GRU model.

Data Set	MovieLens			Book-Crossings			Amazon Reviews’23
Model	ACC	MSE	MAE	ACC	MSE	MAE	ACC	MAE	MSE
DCGCN-GRU-1	0.7502	0.0355	0.2523	0.7123	0.0415	0.1617	0.6920	0.0398	0.1584
DCGCN-GRU-2	0.7689	0.0200	0.1405	0.7258	0.0340	0.1536	0.7102	0.0364	0.1493
DCGCN-LSTM	0.7803	0.0203	0.1359	0.7400	0.0325	0.1512	0.7256	0.0320	0.1455
DCGCN-GRU	0.8001	0.0200	0.1289	0.7450	0.0345	0.1456	0.7359	0.0310	0.1400

, both the high-order feature interaction module (GCN) and the sequential behavior extraction module (GRU) contribute significantly to the DCGCN-GRU model. The complete DCGCN-GRU model performs best, indicating that the combination of high-order features and sequential information can significantly enhance the performance of the recommendation system.

5. Conclusions

In this paper, we propose a Deep Crossing recommendation method based on hybrid neural networks, which utilizes the relationships between nodes in the graph structure to learn the representations of users and items by introducing a graph convolutional network (GCN) model. The information of neighboring nodes is gradually aggregated through multi-layer convolutional operations to effectively capture the complex graph structure information. Meanwhile, the added gated recurrent unit (GRU) module further enhances the modeling capability of feature sequences and captures the temporal dynamic relationships among features. The algorithm not only better utilizes feature intersection and graph structure information, but also demonstrates stronger generalization ability and robustness. Although the model successfully solves the feature crossover problem, there is still room for improvement in the representation of higher-order interaction information. Future work will try to introduce more complex intersection structures in the intersection layer of the model and adjust the feature weights according to the characteristics of the input data to further improve the model performance.