Matrix Factorization Recommendation Algorithm Based on Attention Interaction

Abstract

Recommender systems are widely used in e-commerce, movies, music, social media, and other fields because of their personalized recommendation functions. The recommendation algorithm is used to capture user preferences, item characteristics, and the items that users are interested in are recommended to users. Matrix factorization is widely used in collaborative filtering algorithms because of its simplicity and efficiency. However, the simple dot-product method cannot establish a nonlinear relationship between user latent features and item latent features or make full use of their personalized information. The model of a neural network combined with an attention mechanism can effectively establish a nonlinear relationship between the potential features of users and items and improve the recommendation accuracy of the model. However, it is difficult for the general attention mechanism algorithm to solve the problem of attention interaction when the number of features between the users and items is not the same. To solve the above problems, this paper proposes an attention interaction matrix factorization (AIMF) model. The AIMF model adopts a symmetric structure using MLP calculation. This structure can simultaneously extract the nonlinear features of user latent features and item latent features, thus reducing the computation time of the model. In addition, an improved attention algorithm named slide-attention is included in the model. The algorithm uses the sliding query method to obtain the user’s attention to the latent features of the item and solves the interaction problem among different dimensions of the user, and the latent features of the item.

1. Introduction

With the development of Internet technology, the amount of network information has increased rapidly, resulting in information overload. The problem of information overload renders users unable to obtain effective information quickly [1,2,3,4,5,6]. A recommendation system [7] has an excellent effect in addressing this problem. A recommender system is an online personalized recommendation system that helps users find and recommend items of interest by searching, and analyzing user preferences [8,9]. Currently, the main methods for recommender systems to learn user preferences fall into three categories [10,11]: content-based methods [12], collaborative filtering methods [13,14,15,16,17,18,19,20,21], and hybrid recommendation methods [22]. In essence, these algorithms solve the common problems of data sparsity, data scalability, and cold starts in recommender systems from different perspectives. For instance, Gulzar et al. [23] proposed a new clustering algorithm based on an ordered clustering algorithm (OCA), which aims to reduce the impact of cold starts, and data sparsity. Wu et al. [24] proposed the sampled SoftMax (SSM) loss as an efficient substitute for SoftMax loss, which can optimize long-tail recommendation.

Collaborative filtering is a popular recommendation algorithm. Based on the user behavior data (such as the ratings of users over items, user comments on items, and so on), it determines the correlation between users and items to provide personalized recommendations for users. The model-based recommendation is the most common collaborative filtering algorithm. Especially in recent years, models integrating an attention mechanism have become a research hotspot in the field of recommendation systems. The basic principle of the attention mechanism imitates the human visual system to assign different weights when processing information at different positions in the input sequence.

More and more researchers have integrated an attention mechanism to different degrees to solve the problems of data sparsity, data scalability, and cold starts.

Wang et al. [25] proposed a multi-attention deep neural network (MADNN) recommendation model based on embedding, and matrix factorization that can effectively alleviate data sparsity and cold start problems. The model enhances the interactivity of user/item embedding through a multi-attention mechanism. However, the dimensions of interactive user/item embeddings must be consistent, which may cause information loss caused by the number of features of different users, and items embedded into the same dimension.

Zhang et al. [26] proposed a probabilistic matrix factorization recommendation for self-attention mechanism convolutional neural networks with item auxiliary information, which can solve the problem of data sparsity in recommendation systems. The model adds a self-attention mechanism to the convolutional layer to establish the interaction between the auxiliary information of different channels. However, there is also the limitation that the dimensions of the auxiliary information of different channels must be consistent.

To verify the existence of the above limitations, we take the features of the movie domain as an example. In the MovieLens100K dataset, the user features include user ID, age, gender, occupation, and zip code, and the movie features include movie ID, movie title, release date, video release date, IMDb URL, unknown, action, and so on. In order to make the user, and item interact, they are usually encoded into a vector with the same dimensions (similar to the two models above), which may lead to a decrease in recommendation accuracy.

To solve the interaction problem, we propose an attention interaction matrix factorization (AIMF) model that integrates the interactions among the users, latent feature relationships of the items, and the users of the original rating matrix.

The main contributions of this paper are as follows:

• We propose a novel collaborative filtering model that leverages the attention mechanism, and matrix factorization for item recommendation. To prevent unreasonable model initialization parameters, which lead to too large or too small model training loss, we used the nonnegative matrix factorization (NMF) technique to improve the information of the two latent features.
• To fully explore users’ preferences, we propose the use of the user–item interactive attention mechanism, and the original rating matrix self-attention mechanism to explore the implicit preferences between the users and item potential features and the explicit preferences among the user ratings, respectively, to improve the recommendation accuracy.
• To effectively carry out the user item interaction, and improve the generalization ability of the model, this study proposes a slide-attention algorithm to establish the global attention of the potential features of users and items and to solve the problem of the attention interaction among the feature vectors of different dimensions.
• To verify the effectiveness and feasibility of the model, this study verifies the impact of the latent feature dimension, implicit preference factor, and explicit preference factor on recommendation performance using a large number of experiments on two public datasets in the movie domain. The experimental results show that the model achieved excellent results for the root mean squared error (RMSE) and mean absolute error (MAE).

The remainder of this paper is organized as follows. The related work is presented in Section 2. Then, in Section 3, the proposed model is depicted. The experiment and analysis of the results are discussed in Section 4. The experimental results are discussed in Section 5. Finally, we conclude this paper with future research directions in Section 6.

2. Related Work

Collaborative filtering algorithms based on matrix factorization (MF) are widely used, owing to their simplicity and ease of implementation. However, the rating matrix obtained by MF generally has the features of highly sparse data and uneven distribution, which lead to problems such as low recommendation performance, cold starts, and long tails [27,28]. To solve these problems, many researchers have proposed improved MF algorithms. Koren et al. [14] proved that a latent factor vector can enhance the ability of a model to deal with coefficient features. Badrul et al. [29] proposed singular value decomposition (SVD) to learn a user–item rating information matrix. However, the MF model learned using SVD was prone to overfitting. Subsequently, Funk [30] proposed the FunkSVD model, which adds a regularizer to the conventional SVD method to avoid overfitting the MF model. Koren et al. [14] Proposed a BiasSVD model with a bias term to solve the problem of large fluctuations in user-rating information. Based on BiasSVD, Koren [31,32] proposed the SVD++ model with implicit information to solve the cold-start problem caused by score sparsity. The abovementioned improved matrix factorization algorithm achieved excellent results in solving the sparsity problem of user–item rating information. Nevertheless, the simple vector dot product cannot establish a nonlinear relationship between the latent features of users and items, and the user and item features may not make full use of the latent space, leading to limited recommendation performance.

Deep learning combined with matrix factorization has gradually become a mainstream research topic because of its nonlinear expression ability to establish a model. Li et al., [33] proposed a POI recommendation method fusing auxiliary attribute information based on the neural matrix factorization, integrating the convolutional neural network and attention mechanism (NueMF-CAA) to alleviate the data-sparsity problem. He et al., [34] introduced neural networks based on generalized matrix factorization (GMF). To express the nonlinear relationship between latent features, Tian et al. [35] proposed a deep matrix factorization (DMF) model that combines deep neural networks and matrix factorization techniques. DMF adds multiple hidden layers after the fully connected layers of the neural network to model higher-order interactions between users and items.

Deep neural networks (DNNs) establish nonlinear relationships using weighted summation and reactivation, which may not highlight the key feature information. Therefore, the fusion of deep neural networks and attention mechanisms [36] has become a popular research topic. Wang et al. [37] proposed a convolutional neural network model based on the attention mechanism for CAPTCHA recognition, and their experimental results showed that the accuracy of CAPTCHA recognition was 93.27%. He et al. [38] proposed the inner attention-based recurrent neural network GATE function (IARNN-GATE), which uses the attention mechanism in the gate function of an RNN to control the information transmission of states between the hidden layers. Zhou et al. [39] proposed a recurrent neural network–attention mechanism model (RNN-AM) for microblog sentiment classification. Zhou et al. [40] proposed an image-denoising algorithm based on an attention mechanism and residual block, which effectively solved the problem of real image noise.

3. Attention Interaction Matrix Factorization Model

In this section, we introduce a method for solving the interaction problem of a user and item with different dimensions and the data sparsity problem. We introduce the definition of the problem in Section 3.1. We summarize all notations in this paper as well as functions in Section 3.2. Then, in Section 3.3, the proposed model is depicted in detail. Finally, we propose the loss function for training in Section 3.4.

3.1. Problem Definition

The main tasks of the recommendation algorithm are to explore user preferences and recommend items to users by processing the feature information of the users and items. Suppose that the user set is  $U=\{u_1,u_2,\dots ,u_M\}$, the item set is $I=\{i_1,i_2,\dots ,i_N\}$, $M$ is the number of users, and $N$ is the number of items. The raw rating matrix $Raw\_Rating\in {ℝ}^{M\times N}$ represents a user’s actual rating of an item. Given user $u\in U$ and item  $i\in I$ , suppose user $u$ has not rated item $i$. We must construct a user preference for item $i$ and predict $r^{ui}$ (rating of $i$ by user $u$) based on the rating information of the actual rating matrix $Rating\in {ℝ}^{M\times N}$ ($R\in {ℝ}^{M\times N}$).

3.2. Related Terminology

We summarize the meaning of each term in

Table 1. Relative terms.

Notation	Description
$Rating/R\in {ℝ}^{M\times N}$	User–item rating matrix
$U$	User set
$I$	Item set
M/N	Number of users/number of items
$d$	Latent feature dimension
$u_o/i_o$	User $o$/item $o$
$UI-att$	User item latent feature attention inter- action output
$R-att$	Inter-users rating attention output
$r^{ui}/r^{\prime }$	Predicted rating of item i by user $u$
$P_u\in {ℝ}^{M\times d}$	User latent feature matrix
$Q_i\in {ℝ}^{d\times N}$	Item latent feature matrix
$P_{u-mlp}\in {ℝ}^{M\times d}$	MLP layer output of user latent features
$Q_{i-mlp}\in {ℝ}^{d\times N}$	MLP layer output of item latent features
$Query=(quer{y}_1,quer{y}_2,\dots ,quer{y}_M)$	Query matrix
$Key=(ke{y}_1,ke{y}_2,\dots ,ke{y}_d)$	Key matrix
$Value=(valu{e}_1,valu{e}_2,\dots ,valu{e}_d)$	Value matrix
q	Query features
$\mathit{k}$	Key features
$\mathit{v}$	Value feature
$SoftMax(·)$	Normalization function
$norm(·)$	Min–max normalization function
$concat(·)$	Concatenates multiple vectors into a matrix

.

3.3. AIMF Model

This section describes the AIMF model in detail. In the UI-attention module, NMF, and slide-attention algorithms are used to capture deeper features of users and construct a potential feature correlation between users and items. Simultaneously, personalized recommendations are performed by combining the correlation among user ratings in the self-attention module. This model is illustrated in Figure 1.

The model consists of four parts: input, attention, feature fusion, and output layers. The model establishes an attention interaction between the potential features of the users and items in the UI-attention module to capture the implicit preferences of the users. In the self-attention module, a rating correlation among the users is established to capture their explicit preferences.

Next, the working principle of AIMF is briefly introduced.

Normalized (min–max normalization) real rating data of users are obtained for the input layer. In the UI-attention module, the input layer data are decomposed into two nonnegative latent feature matrices, $P_u$ and $Q_i$, using the NMF technique. Next, the nonlinear features of the two are extracted using a multilayer perceptron (MLP). Then, the output results are input into slide-attention to establish the correlation between the potential feature relationships between the users and items and to obtain the user item attention matrix $UI-att$. Simultaneously, in the self-attention module, the one-hot encoding dot product of the input layer data and the user ID is used to obtain the rating vector of a single user. Next, these rating vectors are used as the query and key-value for attention weight allocation, and the rating correlation score matrix between each user is obtained. Subsequently, the results are inputted into $add\&norm$ for re-centralization and rescaling adjustment, and the attention scoring matrix $R-att$ is obtained. After data processing in the attention layer, $UI-att$ and $R-att$ are input to the feature fusion layer for data integration. Finally, the user prediction score for the item is obtained in the output layer.

(1).

Input layer

In this study, simple data extraction was carried out on the datasets, and the user’s rating matrix data for the item were obtained. To reduce the loss of model training and improve the prediction ability of the model and the generalization ability of the model to the datasets, we normalized the original user item rating matrix.

$$Rating=norm(Raw\_Rating)$$

(1)

Equation (1) refers to the normalization of the raw rating matrix, where $norm(·)$ represents the min–max normalization function. $Raw\_Rating$ is a raw rating matrix (user ratings of items). $Rating$ is a normalized rating matrix (range 0 to 1).

(2).

Attention layer

(a).

UI-attention module

The main task of this module is to explore the implicit preferences of users. In addition, the UI-attention module uses the slide-attention algorithm to solve the problem of attention interactions where the user feature dimension is inconsistent with the item feature dimension.

The details are as follows:

Step 1: In this step, we perform an NMF operation on $Rating$. Methods based on matrix factorization perform well in sparse matrices. This method also has the advantages of high recommendation accuracy, scalability, and high flexibility [41]. NMF is a variant of the MF technique [42,43,44]. NMF technology is used to reduce the dimensions of the data, and high-dimensional data are mapped to the low-dimensional space while retaining the main information. This operation can reduce the complexity of the data and improve the training efficiency of the machine learning model.

The specific treatment is shown in Equations (2) and (3):

$$E_{err}=min\parallel Rating-P_u·Q_i{\parallel }_2$$

(2)

$$Rating\approx P_u·Q_i$$

(3)

where $E_{err}$ denotes computational loss. $Rating\in {ℝ}^{M\times N}$ represents the rating matrix. $P_u\in {ℝ}^{M\times d}$ ($P_u>0$) represents the latent feature matrix of the user. $Q_i\in {ℝ}^{d\times N}$ ($Q_i>0$) represents the latent feature matrix of an item. $\parallel ·{\parallel }_2$ denotes the 2-Norm. $d$ is the dimension of the latent features of users/items. $P_u$ and $Q_i$ are updated by gradient descent (GD) [45], and when a certain number of iterations is reached, the dot product of the two will approximate $Rating$. In this study, $P_u$ and $Q_i$ were obtained by 100 iterations.

Step 2: In this step, the nonlinear features of $P_u$ and $Q_i$ are extracted using MLP. To have a certain generalization ability of the model, here, we use MLP to raise and then reduce the dimensions of $P_u$ and $Q_i$, respectively, and obtain their implicit nonlinear features. The $P_u$ and $Q_i$ obtained in Step 1 are respectively input into the MLP for calculation to obtain $P_{u-mlp}\in {ℝ}^{M\times d}$ and $Q_{i-mlp}\in {ℝ}^{d\times N}$. Figure 2 shows the MLP design used in this study.

The calculation is as follows:

• Hidden layer-1

$$ou{t}_1=\mathrm{F}(x·W_1+b_1)$$

(4)

where $ou{t}_1$ denotes the output of the first hidden layer. $x$ is the input of the MLP, and the inputs in this study were $P_u$ and $Q_i$, respectively. $W_1$ and $b_1$ are the weight matrix and the bias of the first hidden layer, respectively. F(·) is an activation function (in this study, $ReLU$ was chosen as the activation function in the MLP).

II.

Hidden layer-L

$$ou{t}_L=\mathrm{F}(ou{t}_{L-1}·W_L+b_L)$$

(5)

where ${out}_L$ is the output of the first hidden layer. $W_L$ and $b_L$ are the weight matrix and bias of the $L$th hidden layer, respectively.

III.

Output layer

$$final\_out=F(ou{t}_L·W_o+b_o)$$

(6)

where $final\_out$ is the final output. $W_L$ and $b_L$ are the weight matrix and the bias of the output layer, respectively.

Therefore, $P_{u-mlp}$ and $Q_{i-mlp}$ can be calculated as follows:

$$P_{u-mlp}=MLP(P_u)$$

(7)

$$Q_{i-mlp}=MLP(Q_i)$$

(8)

where $MLP(·)$ denotes the I–III computational processes. $P_{u-mlp}$ and $Q_{i-mlp}$ denote the outputs of the MLP layer.

Step 3: In this step, we apply the attention interaction between $P_{u-mlp}$ and $Q_{i-mlp}$ obtained from Step 2 to obtain $UI-att\in {ℝ}^{M\times N}$. However, the user latent feature matrix represented by $P_{u-mlp}$ and the item latent feature matrix represented by $Q_{i-mlp}$ cannot directly establish the attention relationship by conventional attention mechanism. The reason is that the latent feature dimension of each user representing the item is d, while the number of items is $N$.

In this study, we propose a slide-attention algorithm to establish the relevance among different dimensions of user and item latent features. Based on the attention mechanism, the algorithm adjusts the score calculation to adapt to different numbers of query and key-value attention interactions with different feature dimensions, thereby making attention technology more widely used. In the AIMF model, $P_{u-mlp}$ is used as $Query$, and $Q_{i-mlp}$ as $Key$, $Value$.

Next, the calculation process of the method is introduced. In Step 3-1, we show the general computational process of the attention mechanism. The main difference between slide-attention and the attention mechanism is shown in Step 3-2.

Step 3-1. Attention mechanism

The attention mechanism interacts with the query and key through attention convergence to generate a score for the value [46]. This works for the mechanism in Figure 3.

The similarity between the query and the key is computed as follows:

$$s_i=\mathit{q}·{\mathit{k}}_i$$

(9)

where $s_i$ represents the ${\mathit{v}}_i$ score. $\mathit{q}$ can represent any vector in the set $\{{\mathit{q}}_1,{\mathit{q}}_2,\dots ,{\mathit{q}}_m\}$ (where the value of $m$ is problem-specific). ${\mathit{k}}_i$ is a key feature of ${\mathit{v}}_i$ (${\mathit{v}}_i$ is the $i$th value feature). Equation (9) reflects the process of determining the ${\mathit{v}}_i$ score, and Figure 3 shows the single $\mathit{q}$ feature calculation score process. The full attention process computes each $\mathit{q}$ feature that interacts with the $\mathit{k}-\mathit{v}$ feature.

The SoftMax function is used to numerically convert the attention scores:

$${\alpha }_i=SoftMax(s_i)=\frac{exp(s_i)}{{\displaystyle \sum _{j}exp(s_j)}}$$

(10)

where ${\alpha }_i$ is the score of the ${\mathit{v}}_i$ features obtained after a single q feature query and normalization using the SoftMax function.

The weighted sum of the values based on the score is calculated as follows:

$$Attention((\mathit{k},\mathit{v}),\mathit{q})={\displaystyle \sum _i{\alpha }_i{\mathit{v}}_i}={\displaystyle \sum _i\frac{exp(s({\mathit{k}}_i,\mathit{q}))}{{\displaystyle \sum _{j}exp(s({\mathit{k}}_j,\mathit{q}))}}}{\mathit{v}}_i$$

(11)

Equation (11) calculates the attention of the $\mathit{q}$ feature to the $\mathit{k}-\mathit{v}$ feature, $Attention((\mathit{k},\mathit{v}),\mathit{q})\in {ℝ}^{1\times n}$ (the $n$ value depends on the specific problem). $\mathit{k}$ and $\mathit{v}$ denote the key and value features, respectively.

In summary, the output of the attention mechanism is calculated as follows:

$$att=concat(Attentio{n}_1,Attentio{n}_2,\dots ,Attentio{n}_n)$$

(12)

where $concat$ represents the concatenation operation, which concatenates $attentio{n}_1,attentio{n}_2,\dots ,attentio{n}_n$ into a matrix $att\in {ℝ}^{m\times n}$.

Step 3-2. Slide-attention

Compared with conventional attention, slide-attention achieves the purpose of the attention interaction of different dimensions by adjusting the interaction scores of $\mathit{q}$ and ${\mathit{k}}_i$. Given that ${\mathit{p}}_1\in {ℝ}^{1\times d}$ and ${\mathit{q}}_1\in {ℝ}^{1\times N}$, which represent the latent features rated by user 1 for all items and the latent features rated by all items themselves, our main task is to establish the correlation between them, that is, to query all features in ${\mathit{q}}_1$ using ${\mathit{p}}_1$ and generate an interaction. The calculation principle is as follows.

The $\mathit{q}$, $\mathit{k}$, and $\mathit{v}$ values are calculated as follows:

$${\mathit{q}}_i={\mathit{W}}^q·quer{y}_i$$

(13)

$${\mathit{k}}_j={\mathit{W}}^k·ke{y}_j$$

(14)

$${\mathit{v}}_j={\mathit{W}}^v·valu{e}_j$$

(15)

Equations (13)–(15) comprise the calculation processes of the query feature, key feature, and value feature, respectively. ${\mathit{q}}_i$ represents the $i$th query feature. ${\mathit{W}}^q\in {ℝ}^{d\times d}$ represents the trainable weight parameter matrix of the query. $quer{y}_i$ denotes the $i$th query vector. ${\mathit{k}}_j$ is the $j$th key feature. ${\mathit{W}}^k\in {ℝ}^{d\times d}$ represents the trainable weight parameter matrix of the key. Let $ke{y}_j$ denote the $j$th key vector. ${\mathit{v}}_j$ is the $j$th value feature. ${\mathit{W}}^v\in {ℝ}^{d\times d}$ represents the trainable weight parameter matrix of the value. $valu{e}_j$ is the $j$th value vector. Here, $i=1,2,\dots ,M$, $j=1,2,\dots ,d$.

Calculating the slide-attention score:

Figure 4 shows the score calculation process of query feature $\mathit{q}$ and key feature ${\mathit{k}}_i$ in slide-attention. It works by taking the dot product of the query feature $\mathit{q}$ (with $d$ feature elements) and the corresponding elements of ${\mathit{k}}_i$ and then averaging all the scores to obtain the final score of $\mathit{q}$ against ${\mathit{k}}_i$. Here, $\mathit{q}$ is any vector in the set $\{{\mathit{q}}_1,{\mathit{q}}_2,\dots ,{\mathit{q}}_M\}$ and $i=1,2,\dots ,d$, $M$ is the number of users.

The calculation process of the score of $\mathit{q}$ against ${\mathit{k}}_i$ is as follows:

$\exists \mathit{q},{\mathit{k}}_i$, $\mathit{q}=({\mathit{q}}_1,{\mathit{q}}_2,\dots ,{\mathit{q}}_d)$, ${\mathit{k}}_i=(k_{i,1},\dots ,k_{i,d},\dots ,k_{i,N})$, and d are the feature dimensions. The dot product of $\mathit{q}$ and ${\mathit{k}}_i^1=(k_{1,1},\dots ,k_{1,d})$ yields the $score1$ value. To ensure that $\mathit{q}$ is sufficient for ${\mathit{k}}_i$, in this paper, ${\mathit{k}}_i^1$ is merged into the tail of ${\mathit{k}}_i$ to obtain ${\mathit{k}}_i^{+}$ (${\mathit{k}}_i^{+}=({\mathit{k}}_{i,1},{\mathit{k}}_{i,2},\dots ,{\mathit{k}}_{i,d},\dots ,{\mathit{k}}_{i,N},\dots ,{\mathit{k}}_{i,N+d})$). Thus, after $\mathit{q}$ sufficiently interacts with ${\mathit{k}}_i^{+}$, the score vector $scor{\mathit{e}}_i$ is obtained ($scor{\mathit{e}}_i=(score1,score2,\dots ,scoreN)$). Finally, the score of $\mathit{q}$ over ${\mathit{k}}_i$ was obtained by averaging the sum of all the elements in $scor{\mathit{e}}_i$ and yielding $Scor{\mathit{e}}_i$.

The relevant calculation formula is as follows:

$$scor{\mathit{e}}_i\mathit{\omega }=\mathit{q}·{\mathit{k}}_i^{+\omega }$$

(16)

$$Scor{\mathit{e}}_i=Avg(scor{\mathit{e}}_i)$$

(17)

Equation (16) represents the multiplication of $\mathit{q}$ by the corresponding elements of the $\mathit{\omega }~\mathit{\omega }+d$-elements of ${\mathit{k}}_i^{+}$ ($\mathit{\omega }=1,2,\dots ,N$). Here, $scor{\mathit{e}}_i\mathit{\omega }$ denotes the score obtained by the $\mathit{\omega }$th dot product. Equation (17) represents the calculation of the ${\mathit{v}}_i$ score. $Scor{\mathit{e}}_i$ represents the final interaction score between $\mathit{q}$ and ${\mathit{k}}_i$, that is, all $scor{\mathit{e}}_i\mathit{\omega }$ values calculated by q and ${\mathit{k}}_i^{+}$ are averaged. Here, $\mathit{q}$ represents any member in the $\{{\mathit{q}}_1,{\mathit{q}}_2,\dots ,{\mathit{q}}_M\}$ set. $Avg(·)$ is the mean function.

Combining Equations (11) and (17), $slide-attentio{n}_l$ is obtained as follows:

$$slide-attentio{n}_l((\mathit{k},\mathit{v}),{\mathit{q}}_l)={\displaystyle \sum _{i=1}^N{\alpha }_i{\mathit{v}}_i}={\displaystyle \sum _{i=1}^N\frac{exp(\mathit{S}\mathit{c}\mathit{o}\mathit{r}{\mathit{e}}_i({\mathit{k}}_i,{\mathit{q}}_l))}{{\displaystyle \sum _{j}exp(\mathit{S}\mathit{c}\mathit{o}\mathit{r}{\mathit{e}}_i({\mathit{k}}_j,{\mathit{q}}_l))}}}{\mathit{v}}_i$$

(18)

Substituting Equation (18) into Equation (12), the UI-attention module output $UI-att$ is obtained.

$$UI-att=\mathit{c}\mathit{o}\mathit{n}\mathit{c}\mathit{a}\mathit{t}(slide-attentio{n}_1,\dots ,slide-attentio{n}_N)$$

(19)

where $UI-att\in {ℝ}^{M\times N}$ and $slide-attentio{n}_l\in {ℝ}^{1\times N}$. $UI-att$ is the attention score of the user latent features to the item latent features obtained by $P_{u-mlp}$ and $Q_{i-mlp}$ after the slide-attention operation, which is the implicit preference of the user.

The process of the slide-attention Algorithm 1 is as follows:

Algorithm 1: Slide-attention process

Require: Deep user latent features $P_{u-mlp}\in {ℝ}^{M\times d}$, deep item latent features $Q_{i-mlp}\in {ℝ}^{d\times N}$. Ensure: $UI-att\in {ℝ}^{M\times N}$. Input: $P_{u-mlp}\in {ℝ}^{M\times d}, Q_{i-mlp}\in {ℝ}^{d\times N}$. Step1: Determining query features $\mathit{Q}\in {ℝ}^{M\times d}$, key features $\mathit{K}\in {ℝ}^{d\times N}$, value features $\mathit{V}\in {ℝ}^{d\times N}$. We take the dot product of ${\mathit{P}}_{u-mlp}$ with the query weight matrix ${\mathit{W}}^q\in {ℝ}^{d\times d}$ to obtain the query features (Q). We take the dot product of $Q_{i-mlp}$ with the key weight matrix ${\mathit{W}}^k\in {ℝ}^{d\times d}$ and the value weight matrix ${\mathit{W}}^v\in {ℝ}^{d\times d}$, respectively, to obtain the key and value features (K and V)

$\mathit{Q}\leftarrow {\mathit{W}}^q·P_{u-mlp}, \mathit{K}\leftarrow {\mathit{W}}^k·Q_{i-mlp}, \mathit{V}\leftarrow {\mathit{W}}^v·Q_{i-mlp}$

Step2: We extend $\mathit{K}\in {ℝ}^{d\times N}$ to $\mathit{K}\in {ℝ}^{d\times (N+d)}$ (add the first d column elements of K to the end of K). Step3: Expand the dimensions of $\mathit{Q}\in {ℝ}^{M\times d}, \mathit{K}\in {ℝ}^{d\times N}, \mathrm{and} \mathit{V}\in {ℝ}^{d\times N}$ to $\mathit{Q}\in {ℝ}^{M\times 1\times d}, \mathit{K}\in {ℝ}^{M\times d\times N} \mathrm{and} \mathit{V}\in {ℝ}^{M\times d\times N}$, respectively. The purpose is to let Q, K, and V perform batch dot multiplication to improve computational efficiency. Step4: Calculate the score between the latent features of the user and item.

FOR Count in N

$score\leftarrow \mathbf{B}\mathbf{a}\mathbf{t}\mathbf{c}\mathbf{h}\_\mathbf{m}\mathbf{a}\mathbf{t}\mathbf{r}\mathbf{i}\mathbf{x}\_\mathbf{m}\mathbf{u}\mathbf{l}\mathbf{t}\mathbf{i}\mathbf{p}\mathbf{l}\mathbf{i}\mathbf{c}\mathbf{a}\mathbf{t}\mathbf{i}\mathbf{o}\mathbf{n}(\mathit{Q},{\mathit{k}}_{Count})$

$\mathit{s}\mathit{c}\mathit{o}\mathit{r}\mathit{e}\leftarrow \mathit{c}\mathit{o}\mathit{n}\mathit{c}\mathit{a}\mathit{t}(\mathit{s}\mathit{c}\mathit{o}\mathit{r}\mathit{e},score)$

END FOR

Here, the count goes from 1 to N. ${\mathit{k}}_{Count}\in {ℝ}^{M\times d\times d}$ is the matrix in K consisting of columns Count to Count + d. $score\in {ℝ}^{M\times 1\times d}$ represents the attention score computed by Q on the sub-features of K. $score\in {ℝ}^{M\times N\times d}$ represents the query feature Q operation on the key feature K to obtain the total score matrix. concat(A, B) means to attach B to A in the second dimension. N denotes the number of keys (the number of columns of K). Step5: The masking SoftMax operation is performed on score.

$score\leftarrow SoftMax(score,invlide-data)$

where invlide-data is a two-dimensional matrix used to mask some elements of score. Step6: Averaging $score\in {ℝ}^{M\times N\times d}$ yields $score\in {ℝ}^{M\times d\times 1}$. $score\in {ℝ}^{M\times d\times 1}$ is the final score of the queries for the keys. Step7: The resulting $score\in {ℝ}^{M\times d\times 1}$ is a batch-inner product with the value feature $\mathit{V}\in {ℝ}^{M\times d\times 1}$, and the dimension is reduced to obtain $UI-att\in {ℝ}^{M\times N}$

$UI-att\leftarrow \mathbf{B}\mathbf{a}\mathbf{t}\mathbf{c}\mathbf{h}\_\mathbf{m}\mathbf{a}\mathbf{t}\mathbf{r}\mathbf{i}\mathbf{x}\_\mathbf{m}\mathbf{u}\mathbf{l}\mathbf{t}\mathbf{i}\mathbf{p}\mathbf{l}\mathbf{i}\mathbf{c}\mathbf{a}\mathbf{t}\mathbf{i}\mathbf{o}\mathbf{n}(\mathit{s}\mathit{c}\mathit{o}\mathit{r}\mathit{e},\mathit{V})$

$UI-att\leftarrow \mathbf{d}\mathbf{i}\mathbf{m}\mathbf{e}\mathbf{n}\mathbf{s}\mathbf{i}\mathbf{o}\mathbf{n}\mathbf{a}\mathbf{l}\mathbf{i}\mathbf{t}\mathbf{y}\_\mathbf{r}\mathbf{e}\mathbf{d}\mathbf{u}\mathbf{c}\mathbf{t}\mathbf{i}\mathbf{o}\mathbf{n}(UI-att)$

where $\mathbf{d}\mathbf{i}\mathbf{m}\mathbf{e}\mathbf{n}\mathbf{s}\mathbf{i}\mathbf{o}\mathbf{n}\mathbf{a}\mathbf{l}\mathbf{i}\mathbf{t}\mathbf{y}\_\mathbf{r}\mathbf{e}\mathbf{d}\mathbf{u}\mathbf{c}\mathbf{t}\mathbf{i}\mathbf{o}\mathbf{n}(UI-att)$ is the compressed dimension function. Here, it means that the dimension reduction operation is performed on $UI-att$ in order to reduce unnecessary space loss. Step8: return $UI-att$ Output: $UI-att$

•  (b).

  Self-attention module

Compared with the UI-attention module capturing the implicit preferences of the user, the main task of this module is to establish a rating correlation among users in order to explore their explicit preferences.

The details are as follows:

Step 1: All the user IDs were encoded using one-hot encoding. The main idea is to map each class into a vector, where only one element is one and the rest are zero. The one indicates the class in which the data point is located. For example, to encode the user ID, the code of user 1 is denoted as 100,000…, and the code of user 2 is denoted as 010000…, where the length of the code is the number of users M [39].

Step 2: The one-hot encoding of each user from Step 1 is a dot product with the rating matrix. The purpose of this part of the one-hot operation is to obtain the rating of each user for the item. The calculation is given by Equation (20):

$$use{\mathit{r}}_l=One-ho{t}_l·Rating$$

(20)

where $use{\mathit{r}}_l\in {ℝ}^{1\times N}$ is the rating of the lth user ($l=1,2,\dots ,M$). $One-ho{t}_l\in {ℝ}^{1\times N}$ denotes the one-hot encoding of user l.

Step 3: We assigned the attention score from each user’s rating of the item obtained in Step 2, in order to obtain the similarity relationship among users about the rating of the item. Compared with the attention mechanism, the self-attention mechanism pays more attention to finding connections among the inputs themselves (internal) and is commonly used to calculate [47] the scaled dot-product attention [38].

$$Self-attention(\mathit{Q},\mathit{K},\mathit{V})=SoftMax(\frac{\mathit{Q}·{\mathit{K}}^T}{\sqrt{d_k}})\mathit{V}$$

(21)

Equation (21) provides a brief formula to calculate the self-attention mechanism. $\mathit{Q}$ represents the set of query features. $\mathit{K}$ represents a set of key features. $\mathit{V}$ represents a set of value features. Different from traditional attention, here, $\mathit{Q}$, $\mathit{K}$, and $\mathit{V}$ are obtained by the dot product of the same inputs with ${\mathit{W}}^q$, ${\mathit{W}}^k$, and ${\mathit{W}}^v$, respectively ($\mathit{Q}=\{{\mathit{q}}_1,{\mathit{q}}_2,\dots ,{\mathit{q}}_M\}$, $\mathit{K}=\{{\mathit{k}}_1,{\mathit{k}}_2,\dots ,{\mathit{k}}_N\}$, and $\mathit{V}=\{{\mathit{v}}_1,{\mathit{v}}_2,\dots ,{\mathit{v}}_N\}$).

Thus, similar to Equations (13)–(15), the $\mathit{q}$, $\mathit{k}$, and $\mathit{v}$ of $Self-attention$ are calculated as follows:

$${\mathit{q}}_l={\mathit{W}}^q·\mathit{u}\mathit{s}\mathit{e}{\mathit{r}}_l$$

(22)

$${\mathit{k}}_l={\mathit{W}}^k·\mathit{u}\mathit{s}\mathit{e}{\mathit{r}}_l$$

(23)

$${\mathit{v}}_l={\mathit{W}}^v·\mathit{u}\mathit{s}\mathit{e}{\mathit{r}}_l$$

(24)

where ${\mathit{q}}_l\in {ℝ}^{1\times N}$, ${\mathit{k}}_l\in {ℝ}^{1\times N}$, and ${\mathit{v}}_l\in {ℝ}^{1\times N}$ denote the query, key, and value features of user $l$, respectively.

Next, the rating relevance among users is obtained by substituting Equations (22)–(24) into Equations (11) and (12), respectively.

$$Self-attention=concat(Self-attentio{n}_1,\dots ,Self-attentio{n}_N)$$

(25)

In this study, $Rating\in {ℝ}^{M\times N}$ of the self-attention mechanism was input. To facilitate understanding and distinguish the difference between the attention mechanism and the self-attention mechanism, one-hot coding was used to separate the scores of each user for all items in the rating matrix $Rating$, and an attention calculation was performed.

Step 4: We performed a residual operation on the similarity matrix obtained among the users’ ratings of items in Step 3 ($Self-attention$) and the rating matrix ($Rating$). To preserve the original rating information and improve the structure of the model stability and training efficiency, we normalized the sum of $Self-attention$ and $Rating$. The residual network can maintain the performance of the network model with an increase in the number of network layers and then slow down the degradation of the model performance [48].

$$R-att=norm(Self-attention+Rating)$$

(26)

Equation (26) represents the attention calculation method of the rating matrix, that is, the user display preference is explored through the rating correlation among users. $norm(·)$ represents the min–max normalization operation. $Self-attention$ is the similarity matrix obtained among the users’ ratings of items. $Rating$ represents the rating matrix.

(3).

Feature fusion layer

This layer shows the degree of influence of implicit and explicit user preferences ($UI-att$ and $R-att$) on the accuracy of personalized user recommendations. For a recommendation system, when the user’s preference information is not detailed, mining the implicit information can reduce the difficulty of user decision making and improve the accuracy of recommendation. This study experimentally proved the influence of implicit and explicit preferences on recommendation accuracy.

The formula is as follows:

$$F-att=UI-att+R-att$$

(27)

where $F-att\in {ℝ}^{M\times N}$ represents the output result of the feature fusion layer. $UI-att\in {ℝ}^{M\times N}$ represents the user–item implicit attention. $R-att\in {ℝ}^{M\times N}$ denotes the explicit attention among users.

(4).

Output layer

The output of this layer is the model’s prediction of the user ratings of an item. In this layer, we activate the output result of the feature fusion layer to achieve a result value within a certain range. The parametric rectified linear unit ($PReLU$) function is used as the activation function of the final feature fusion layer output in this study. Compared with $ReLU$, the $PReLU$ function improves model fitting by adding a parameter to adjust the negative value (non-0) in the face of negative inputs, with almost no additional computational cost and little risk of overfitting [49]. The formula is as follows:

$$f(y_i)=\{\begin{array}{ll}{y}_i,& if y_i>0\\ a_i{y}_i,& if y_i\le 0\end{array}$$

(28)

where $y_i$ is the input channel of the $i$th nonlinear activation function $f(·)$. $a_i$ is a coefficient that controls the slope of the curve ($a_i$ is learnable). The subscript $i$ in $a_i$ indicates that nonlinear activation is allowed to vary across channels. $a_i=0$ is a ReLU function, in which $a_i$ was initially set to 0.25.

Therefore, AIMF’s prediction $r^{\prime }$ is calculated as follows:

$$r^{\prime }=PReLU(F-att)$$

(29)

3.4. Loss Function

To optimize the parameter values of the AIMF model better, in this study, we used the latent features of users and items as the mean score difference loss of the L2 regularizer, which is calculated as follows:

$$L=\phi \ast {{\displaystyle \sum (R-r^{\prime })}}^2+\mu \ast {\Vert P_u\Vert }_2+\theta \ast {\Vert Q_i\Vert }_2$$

(30)

where $R$ and $r^{\prime }$ represent the true and predicted scores of the training set, respectively. $P_u$ and $Q_i$ represent the latent user and item features, respectively, where $\phi$ is the error factor, $\phi$=1/$num\_R$, and $num\_R$ is the true number of ratings. $\mu$ represents the user latent feature factor, that is, $\mu$ = 1/num_u, where $num\_u$ is the number of users. $\theta$ represents the latent feature factor of the item, that is, $\theta$=1/$num\_i$, where $num\_i$ represents the number of items. $\parallel ·{\parallel }_2$ denotes the 2-Norm.

4. Experiment and Result Analysis

The main task presented in this section is the training of the model and predicting the results on two public datasets in the movie domain: MovieLens100K and MovieLens1M [50]. In this study, RMSE and MAE were used to verify the feasibility and effectiveness of the model in comparison with other methods. This study focused on the following three aspects:

(1)

The recommendation performance of the attention interaction model.

(2)

The effect of the number of neurons on model performance.

(3)

The effect of the number of hidden layers on model performance.

This section is divided into three subsections. In Section 4.1, we present the setting for the experiment, including the dataset, evaluation criterion, baseline, and parameter settings. Then, the comparison and analysis of the experimental results are presented in Section 4.2. Finally, in Section 4.3, we present an ablation study.

4.1. Experiment Setting

4.1.1. Dataset

This study verifies the feasibility and effectiveness of the AIMF model on the public datasets MovieLens100K and MovieLens1M [50]. MovieLens is a movie dataset widely used in recommendation systems. It is mainly composed of movie rating data, user demographic data, and movie attribute data and contains multiple versions of different data sizes. See

Table 2. Description of the MovieLens data.

Feature	MovieLens
	100 K	1 M
Users	943	6040
Items	1682	3952
Ratings	100,000	1,000,209
Range of ratings	1~5	1~5
Rating sparsity	93.70%	95.81%

for further details.

• The MovieLens100K dataset contains features such as the ID, age, gender, and occupation of 943 users, as well as 100,000 ratings for 1682 items. The ratings range from one to five. Each user rated at least twenty movies. The data were collected through the studio website (movielens.umn.edu) over a seven-month period from 19 September 1997 to 22 April 1998.
• The MovieLens1M dataset contains 10,000,209 ratings from 6040 users for 3952 items. The ratings range from one to five. Each user rated at least twenty movies. One million sampled users joined the system in the year 2000.

We normalized the rating matrix in both datasets to reduce the loss of model training, improve the prediction ability of the model, and improve the generalization ability of the model to the dataset. The rating data have a score of [0, 1], where a score of 0 indicates that the user has not rated the item.

4.1.2. Evaluation Criterion

In this study, two commonly used recommender system indicators were used to verify the effectiveness and feasibility of the AIMF model: MAE and RMSE [51]. Equations (31) and (32) define MAE and RMSE, respectively.

$$MAE=\frac{1}{T}{\displaystyle \sum _{i=1}^T\left|R-r^{\prime }\right|}$$

(31)

$$RMSE=\sqrt{\frac{1}{T}{\displaystyle {\sum }_{i=1}^T{\left|R-r^{\prime }\right|}^2}}$$

(32)

where $R$ and $r^{\prime }$ represent the true and predicted ratings of the test set, respectively. $T$ represents the number of true ratings in the test set.

4.1.3. Comparison of Methods and Parameter Settings

We compared and analyzed the performance of the AIMF model using the following models to verify its feasibility:

• SVD++ [33,34]. SVD++ is an improved SVD algorithm proposed by Koren. It adds user bias, item bias, and overall bias to the SVD algorithm and uses these bias scores to replace the actual score to evaluate the model and then fit the user preferences, item features, and global features.
• NMF [52]. The basic idea is that, given a nonnegative matrix X, the NMF algorithm can find a nonnegative matrix W and a nonnegative matrix H, such that the dot product of matrix W and H is equal to X so that a nonnegative matrix can be decomposed into two nonnegative matrices.
• DeepHAMF [53]. The deep hierarchical attention matrix factorization model can fully explore the user’s preference information by establishing the interactions among the original input ID information (users and items), the self-attention layer information, and the hierarchical attention information. This paper shows that the self-attention layer is used to capture the explicit preference information of the target and that hierarchical attention is used to capture the potential information of the target.
• DNNMF [54]. The deep nonlinear nonnegative matrix factorization model limits the embedding layer data to be greater than 0 by using nonnegative matrix factorization technology and then uses a deep neural network to establish a nonlinear relationship between the potential features of the users and items to solve the problem of data sparsity.

All the models in this paper were implemented using pytorch (vision 1.13.1). The training data were 80% randomly drawn from the user–item rating matrix, and the test data were the remaining 20%. We used the Adam [55] optimizer, with a learning rate $lr=0.5$. MLP used three hidden layers with 150, 100, and 10 neurons in each layer; the hidden layer used $\mathrm{R}\mathrm{e}\mathrm{L}\mathrm{U}$ as the activation function, and the output layer used $\mathrm{P}\mathrm{R}\mathrm{e}\mathrm{L}\mathrm{U}$ as the activation function. For the MLP network parameter settings, we used a normal distribution with a mean of 0 and a variance of 0.01 for initialization. For the slide-attention and self-attention structures, we randomly initialized the model parameters and set the number of attention heads in self-attention to 2. For the output layer activation function $\mathrm{P}\mathrm{R}\mathrm{e}\mathrm{L}\mathrm{U}$, we used $a_i=0.25$ as the initialization parameter. We manually adjusted all latent feature dimensions $d$ = 5 (including the NMF latent feature dimension, ${\mathit{W}}^q$, ${\mathit{W}}^k$, and ${\mathit{W}}^v$ in slide-attention and self-attention). The number of NMF iterations was 100.

4.2. Comparison and Analysis of Experimental Results

The main task presented in this section is the analysis of the AIMF model and its comparison with the selected methods to verify its effectiveness and feasibility.

4.2.1. Comparison of the MAE and RMSE Performance Metrics of the Models

To answer question (1), we compared the performance of the SVD++ algorithm, NMF algorithm, DeepHAMF, and DNNMF algorithms. The results are listed in

Table 3. Comparison of the MAE and RMSE results.

Method	MovieLens100K		MovieLens1M
	MAE	RMSE	MAE	RMSE
SVD++	0.726	0.9224	0.6629	0.8510
NMF	0.766	0.9688	0.7354	0.9180
DeepHAMF	0.701	0.900	0.659	0.851
DNNMF	0.696	0.8434	0.6425	0.8245
AIMF	0.3201	0.4562	0.1847	0.2360

.

We can intuitively see that the AIMF model outperformed the other methods in terms of the MAE and RMSE metrics for both datasets. In Table 3, the recommendation performance of SVD++ with implicit factors was better than that of NMF with a single matrix factorization and reconstruction score. The performance of DeepHAMF with an attention mechanism based on implicit preferences was better than that of SVD++. Compared with DeepHAMF, DNNMF with MLP performed slightly better than DeepHAMF without an attention mechanism. Our model, AIMF, which combines the nonlinear expressive power of MLP with the attention interaction ability of different dimensions, outperformed the baselines above.

4.2.2. Effect of the Number of Neurons on Model Performance

In order to determine the number of neurons, we used group experiments. Under the condition that the number of hidden layers was three, we conducted three sets of experiments as follows:

(1)

We determined the number of neurons in the first hidden layer. The experimental parameters were: 50, 100, 10; 100, 100, 10; and 150, 100, 10.

(2)

We determined the second hidden layer. After determining the first hidden layer by (1), we determined the experimental parameters of this group: 150, 50, 10; 150, 100, 10; and 150, 50, 30.

(3)

We identified the third hidden layer. After determining the number of neurons in the first two hidden layers by (1) and (2), we determined the experimental parameters of this group: 150, 100, 10; 150, 100, 20; and 150, 100, 30.

The experimental results for MovieLens100K and MovieLens1M are shown in Figure 5 and Figure 6.

Figure 5 shows the results of each set of experiments on MovieLens100K. Figure 5a shows the results when the number of neurons in the first hidden layer is 50, 100, and 150, respectively (assuming that the numbers of neurons in the second and third layers are 100 and 10, respectively). It can be clearly seen that when the number of neurons in the first hidden layer is 150, the recommendation performance of the model is relatively excellent. Under the condition that Figure 5a the number of neurons in the first hidden layer is fixed, Figure 5b shows the results when the numbers of neurons in the second hidden layer are 50, 100, and 150, respectively. When the number of neurons in the second hidden layer is 100, the model recommendation performance is excellent. Figure 5c shows the results of the third hidden layer with 10, 20, and 30 neurons. At this point, we determined the number of neurons in the first two layers to be 150 and 100. Finally, according to the results shown in Figure 5c, when the number of neurons in the third hidden layer is 10, the recommendation performance of the model is good. After we performed the experiments on groups (1), (2), and (3) one at a time, the final numbers of neurons in the MLP were 150, 100, and 10.

Figure 6 shows the results of each set of experiments on MovieLens1M. Similar to the steps on MovieLens100K, we again experimented on groups (1), (2), and (3) one at a time. It can clearly be seen that the experimental results of MovieLens1M are basically the same as that of MovieLens100K. The final number of neurons in the hidden layer is 150, 100, and 10, respectively.

4.2.3. Effect of the Number of Hidden Layers on Model Performance

Based on the experiments in Section 4.2.2, we determined the number of neurons in each hidden layer. However, the above results are based on the assumption that the number of hidden layers is three. Therefore, the main task presented in this section is the determination of the specific number of hidden layers.

The experimental setting of this subsection includes the following. The number of digits represents the number of layers, and the value of the number is the number of neurons. The order from left to right represents the first layer, the second layer, … For example, (150, 100) means two hidden layers with 150 neurons in the first hidden layer and 100 neurons in the second hidden layer.

(1)

Number of hidden layers and neurons: (150, 100);

(2)

Number of hidden layers and neurons: (150, 100, 10);

(3)

Number of hidden layers and neurons: (150, 100, 10, 10);

(4)

Number of hidden layers and neurons: (150, 100, 10, 10, 10).

The experimental results for MovieLens100K and MovieLens1M are shown in Figure 7 and Figure 8.

Figure 7 and Figure 8 visually show the results of the four sets of experiments. It can be seen that the results on MovieLens100K are not consistent with those on MovieLens1M. On MovieLens100K, the recommendation performance of the model is better when the number of hidden layers is four, followed by when the number of hidden layers is three. On MovieLens1M, the recommendation performance is better when the number of hidden layers is three, followed by two hidden layers. This indicates that the number of layers of the MLP is an important factor affecting the recommendation performance. In addition, the corresponding MAE and RMSE are better than the other benchmarks regardless of if the number of layers is three or four. Finally, considering the size of the dataset, we chose three hidden layers as the structure of the MLP.

4.3. Ablation Study

In this section, we present the designs of three variants of the model to verify the impact of different modules on the recommendation performance.

Submodules: AIMF-SS, AIMF-SM, and AIMF-MS.

• AIMF-SS: This includes the slide-attention and self-attention components and removes the MLP.
• AIMF-SM: This includes the slide-attention and MLP components and removes the self-attention.
• AIMF-MS: This includes the MLP and self-attention components and removes the slide-attention.

These sub-models were experimented on two datasets, and the experimental results are shown in

Table 4. Comparison of the experimental results.

Method	MovieLens100K		MovieLens1M
	MAE	RMSE	MAE	RMSE
AIMF-SS	0.2841	0.2366	0.3295	0.3851
AIMF-SM	0.7085	0.7433	0.7208	0.7540
AIMF-MS	0.1978	0.2466	0.1860	0.2355
AIMF	0.3201	0.4562	0.1847	0.2360

.

The results in Table 4 show that the overall performance of AIMF was higher than its variant model, and its variant model performed better than the baseline model. This indicates that the components in the model (slide-attention, MLP, self-attention) play an important role in improving the recommendation performance of the model. By comparing AIMF-SS and AIMF, we found that removing MLP made the variant model perform better than AIMF on MovieLens100K and slightly worse than AIMF on MovieLens1M, which may indicate that the size of the dataset has some influence on the role of MLP. By comparing AIMF-MS and AIMF, we found that removing slide-attention made the variant model perform better than AIMF on both datasets. However, compared with MovieLens100K, the performance of AIMF on MovieLens1M is significantly improved. At the same time, it can solve the problem of an attention interaction between users and item latent features of different dimensions. By comparing AIMF-SM and AIMF, we found that the recommendation performance of the variant model with self-attention removed was much worse than that of AIMF. This indicates that mining explicit user preferences is necessary.

5. Discussion

The deep learning model based on MLP has certain advantages for solving nonlinear problems. In particular, the traditional BP neural network is used to fit the model parameters, which makes the prediction of the model more extensive. Deep learning models based on an attention mechanism are gradually becoming widely used in recommendation systems, with the advantages of fewer parameters, easy training, and better establishment of the similarity relationship between two targets. AIMF combines the advantages of both and improves the recommendation performance in both implicit and explicit user preferences. In addition, compared with the existing attention mechanism, slide-attention makes some improvements based on general attention so that users and items with different dimensional features can interact with attention, and a similarity relationship between them is obtained.

However, AIMF has the following limitations:

Firstly, it can be seen from the scale of the MoiveLens100K and MovieLens1M datasets that AIMF is more suitable for scenarios with a relatively large amount of data. Secondly, as the difference between the user feature dimension and the item feature dimension increases, its calculation time also increases. Finally, AIMF is currently suitable for the scoring mechanism in the film domain, but there is still a lot of room for improvement in other domains, such as music and e-commerce.

6. Conclusions

Among the many recommendation algorithms based on deep learning, knowledge graphing, and time decay, the research focus is on enhancing the personalized characteristics of recommendation systems. This study captures the characteristic information of user–item ratings and deeply mines users’ personalized preferences through users’ ratings on one to five items, so as to recommend items of interest to users. In this paper, we propose a novel AIMF model that integrates matrix factorization, MLP, an attention mechanism, and a self-attention mechanism to reflect the user’s overall preference from two aspects: implicit preference and explicit preference. In addition, AIMF incorporates an improved attention algorithm to overcome the problem that the dimension of latent features between users and items is not the same, and the attention interaction cannot be directly established. Experiments were conducted on two movie domain public datasets, and the experimental results show that the proposed method outperformed several baselines by at least 54% in RMSE and by at least 46% in MAE.

With the development of information technology and the increasing needs of users, dynamic recommendation has become a mainstream research topic. Therefore, processing massive information in real time and making recommendations according to the dynamic preferences of users will be our next research topic. At the same time, applying the model to a wider field of recommendation systems is also a difficult problem we will work to overcome.