Personalized Movie Recommendations Based on a Multi-Feature Attention Mechanism with Neural Networks

Abstract

With the rapid growth of the Internet, a wealth of movie resources are readily available on the major search engines. Still, it is unlikely that users will be able to find precisely the movies they are more interested in any time soon. Traditional recommendation algorithms, such as collaborative filtering recommendation algorithms only use the user’s rating information of the movie, without using the attribute information of the user and the movie, which has the problem of inaccurate recommendations. In order to achieve personalized accurate movie recommendations, a movie recommendation algorithm based on a multi-feature attention mechanism with deep neural networks and convolutional neural networks is proposed. In order to make the predicted movie ratings more accurate, user attribute information and movie attribute information are added, user network and movie network are presented to learn user features and movie features, respectively, and a feature attention mechanism is proposed so that different parts contribute differently to movie ratings. Text features are also extracted using convolutional neural networks, in which an attention mechanism is added to make the extracted text features more accurate, and finally, personalized movie accurate recommendations are achieved. The experimental results verify the effectiveness of the algorithm. The user attribute features and movie attribute features have a good effect on the rating, the feature attention mechanism makes the features distinguish the degree of importance to the rating, and the convolutional neural network adding the attention mechanism makes the extracted text features more effective and achieves high accuracy in MSE, MAE, MAPE, R², and RMSE indexes.

1. Introduction

In recent years, with the rapid development of the Internet, mobile smart devices, and other technologies, a large amount of information has flooded the network [1]. Along with the continuous improvement of people’s living standards and the rapid popularization of mobile Internet, the Internet and smart mobile devices have gradually become the main way for people to obtain information resources. Today, with the rapid development of mobile Internet, the information in mobile Internet is increasing geometrically and the number of mobile users is soaring, making the contradiction between limited network resources and users’ increasing demand for network more and more acute [2]. When we enter the era of big data, modern technology has revolutionized the quantity, type, and speed of data generation, and the massive amount of data brings about information overload [3]. Information overload can be defined as a state in which information exceeds what people can receive, process, or use effectively. As human society enters the era of information explosion, it becomes more and more difficult to find the required information from the vast amount of data. Since different users have different hobbies, interests, and personal experiences, it is difficult for them to sift through the vast amount of data to find the information they are interested in. Search and recommendation technologies help people to acquire and process relevant data, and can better address information overload [4,5].

Due to the rapid development of the Internet, the film industry has been riding on the fast train of technology and has proliferated, with the number of films growing significantly, the variety of films becoming more diverse, and the quality of films improving [6]. A wealth of movie resources are readily available on major search engines. However, due to the wide variety of movies, users cannot precisely find the movies they are more interested in in a short period. Therefore, it is essential to explore the potential needs of users based on their hobbies and behaviours, and make personalized movie recommendations according to their needs [7,8,9]. To help users find movies of interest efficiently, recommendation systems have attracted the attention of many researchers. The movie recommendation system can help users overcome the obstacle of finding movies that meet their needs in a large number of movies [10], and can recommend only the movies that are suitable for them according to their needs, which achieves personalized recommendations for users. With the current limited resources, better algorithms need to be explored to accurately capture user features and movie features in order to accurately and efficiently extract user features and movie features, analyse user interests and needs, and achieve targeted personalized movie recommendations [11].

In recent years, there has been a lot of research on recommender systems. For example, the literature [12] investigated a hybrid recommendation system for heart disease diagnosis. The literature [13] studied personalized mobile news recommendation systems. The literature [14] mined practical features through statistical analysis of users’ shopping behaviour in e-commerce in Internet scenarios and predicted users’ shopping behaviour in the coming month by constructing random forest, logistic regression, and support vector machine models. The literature [15] proposed a “state-behaviour” model for describing user information behaviour, systematically describing and accurately modelling users’ information behaviour in e-commerce, which is vital for guiding the formulation of platform recommendation strategies and marketing programs. The literature [16] gave insights into building a microservice recommendation system for online news, which develops a personalized online news recommendation system by providing an initial menu of topics/themes for users to choose from to determine their initial preferences. The literature [17] presented a personalized product recommendation model for automated question-answering bots using deep learning, which is better adapted to the practical needs of automated question-answering bots. The literature [18] proposed a big data music personalized recommendation method based on big data analysis, integrating user behaviour, behavioural context, user information, and music work information, based on traditional music recommendation methods; a collaborative filtering recommendation algorithm was improved based on users’ behaviour; the semantic similarity between lyrics and song co-occurrence similarity was calculated based on users’ music download history. The literature [19] proposed a Chinese news text classification method based on combined convolutional neural network (combined-CNN), and finally, the Chinese news classification algorithm of combined-CNN was applied to implement a personalized recommendation system. The literature [20] analysed the connection between users, items, and tags in music social networks and implemented a personalized music recommendation algorithm. The literature [21] proposed a user-personalized recommendation algorithm based on deep learning networks, using a gated recurrent unit (GRU) network to build the main model of the personalized recommendation algorithm in order to reduce the influence of overfitting of multilayer networks, and introduces an attention mechanism in the GRU network so that the recommendation model can obtain more accurate information about the features of user data, reducing the influence of irrelevant data in the model. The above articles explored some sub-domains of recommendation systems, which can provide reference and help for researching personalized movie recommendation systems.

Traditional recommendation algorithms, such as collaborative filtering recommendation algorithms only use information about users’ ratings of movies, calculate user similarity or movie similarity only based on movie ratings, and then make recommendations without considering user characteristics such as the user’s gender, age, occupation, etc., or movie characteristics such as a movie’s title, title, release time, etc. [4,22]. Both user characteristics and movie characteristics have an impact on users’ interest preferences, which in turn influence users’ ratings of movies. Most of the current improvements to traditional collaborative filtering recommendation algorithms only consider the user rating as a factor in them. Jie Zhang et al. [23] proposed a fusion collaborative filtering recommendation algorithm based on the entropy of rating information, which improved the method of calculating the similarity between users and made the calculated similarity more accurate, thus making the recommendation more effective. Ziyan Zhang et al. [24] proposed an improved collaborative filtering recommendation algorithm by increasing user preferences for categories, and their experimental results showed an enhanced recommendation effect. Honglin Chu et al. [25] proposed the JSD-AC similarity calculation method, which improved the traditional modified cosine similarity and fused the improved JS scatter into the similarity calculation. Jinming Yu et al. [26] proposed a new similarity measure between items, IPSS, which enabled the cold start problem to be solved. Based on the above, traditional methods such as collaborative filtering and matrix decomposition only apply user rating information, and both user and item attribute information are not utilized. User and item feature representation is not accurate enough. Therefore, user and movie features can be fused to predict movie ratings, and then movies can be recommended to users.

To achieve personalized movie recommendations, this paper proposes a movie recommendation algorithm based on a multi-feature attention mechanism with deep neural networks and convolutional neural networks. The innovative points and contributions of this paper are as follows. (1) In order to better learn the user and movie feature representation, this paper adds user and movie features by considering the information of the user’s and movie’s own attributes. (2) In order to make the movie rating prediction more accurate, this paper first obtains the importance of features and then proposes a feature attention mechanism to give different weights to each feature so that different features can contribute differently to the movie rating prediction. (3) In order to make the extracted text features more accurate, this paper proposes a convolutional neural network plus an attention mechanism fusion model to extract text features. (4) Two deep neural networks, one learning user features, named the user network, and one learning movie features, named the movie network, are designed to learn user and movie features, respectively. The experimental results show that the algorithm is effective and achieves good results in terms of accuracy.

The sections of this paper are structured as follows. The Section 2 describes the related work. Section 3 of the paper proposes the designed algorithm. Section 4 reports the experimental design and analysis. Finally, Section 5 presents the conclusions.

2. Related Work

A convolutional neural network, a representative model structure for deep learning, is a deep neural network inspired by the biological brain and simulates the activity of the visual nerve centre, commonly used to process images, text, and some unstructured data, among others [27]. The structural feature of convolutional neural networks with shared weights makes the computational scale during training much lower, and it has been applied to image recognition, semantic segmentation, and non-image aspects, such as natural language processing. Convolutional neural networks have also been applied to tasks such as text classification and language recognition [28]. The convolutional neural network mainly includes an input layer, hidden layer, and an output layer, where the hidden layer is divided into a convolutional layer, pooling layer, and fully connected layer.

The core of a convolutional neural network is the convolutional layer, which operates on the input by convolution, which simulates the response of visually relevant neurons within the brain to visual stimuli, and the computational results are similar to neurotransmitters for information transfer [27]. The convolutional layer consists of a set of convolutional kernels, which are also called filters. These convolutional kernels generally have a small coverage, i.e., the perceptual field size of the convolutional kernels. The concept of receptive fields was originally derived from some visual neurons, which have the property of receiving stimuli only in their control area. The perceptual field is covered by step and fill to cover the entire field of view. The step and fill settings can be overlapped to cover all channels of the input data in depth.

The pooling layer is a sampling-based discretization operation, which is a non-linear downsampling of the input data, which contains information about the features of local regions in order to reduce the dimensionality of the input data, thus reducing the number of parameters and the computational complexity [29]. Pooling operations do not need to be involved in parameter updates, and average pooling and maximum pooling are the two most common types of pooling. The role of the pooling layer is to aggregate information. For example, using a $2\times 2$ convolutional kernel, with a step size of 2, the height and width of the data can be halved to aggregate the values within the coverage of the convolutional kernel, usually with a maximum pooling operation and an average pooling operation.

The fully connected layer dimensionally transforms the data that has passed through the convolutional and pooling layers. The output data of the convolutional and pooling layers represent the high-level features of the original data, and the individual dimensional metrics represent the depth features of the original data. This layer has a large number of parameters to learn the non-linear way of combining these features [28].

The attention mechanism was initially derived from the brain’s attention and is a method based on human intuition [30]. People’s attention tends to focus more on what interests them, and what makes an item memorable is that it has characteristics that set it apart from other items. The attention mechanism is usually used between the input and output layers of a neural network, which simulates the changes in visual attention when humans observe objects. The main idea of the attention mechanism is to filter out useless or unimportant feature information as much as possible when processing many features and focus on some features that are most critical to the target task, and different weights can be assigned to each feature so that the critical feature information can be more prominent [22,29,31,32,33].

Attentional mechanisms have been shown to be powerful in the field of NLP. The role of the attention mechanism is to assign weights to the parts of the input being processed, with the more important parts being assigned greater weights. The attention mechanism, which first appeared in machine translation tasks, functions in the analysis of utterance components, and the output is an estimate of the weight of each element of the utterance [31,34].

In the movie recommendation scenario, different features of users and movies have different effects on the rating results of end users, and certain weights can be assigned to the features of users and movies, and the weights represent the degree of importance, with high weights representing high importance and low weights representing low importance [22]. For example, if age is a characteristic, users in the lower age group may rate animated movies higher, and the age group characteristic is more important to the user’s rating result than the gender characteristic or occupation characteristic. For example, some users like to watch horror movies. At this time, the movie subject matter is more critical to the user’s rating results. Therefore, the introduction of an attention mechanism allows focussing more attention to features with high importance to the end users’ scoring results, assigning higher weights, and giving features with low priority less attention [34].

The collaborative filtering recommendation algorithm is one of the most widely used and successful techniques in recommender systems [35,36,37,38]. The collaborative filtering recommendation algorithm assumes that users with similar preferences will like similar items or similar items will be liked by similar users. The core idea is to make recommendations based on nearest neighbours [39], which uses historical behavioural data between users or between items to calculate similarity and then effectively recommend items to target users. This recommendation technique can be implemented completely independent of the attributes of the items and is a recommendation that belongs to memory based on the data of the user’s historical behaviour and using group intelligence to solve the problem [40]. This can identify potential but undiscovered interests of users and can provide novel items for users, thus enhancing their experience. The collaborative filtering recommendation algorithm is based on the association between each user or item, based on the user’s interest preferences, and then fuses this association information to predict the user’s interaction behaviour. Collaborative filtering recommendation algorithms include user-, item-, and model-based recommendations.

User-based recommendations are based on a fundamental assumption that users in the same user group tend to choose similar items. The similarity between users can be in terms of demographic characteristics, such as age, gender, occupation, etc., or in terms of historical behaviourial records, such as clicked or followed the same type of items, or even giving the same rating for certain items. Item-based recommendations are based on the fundamental assumption that items of the same class tend to be interacted with by similar users, and that similarity between items is calculated not on the basis of the items themselves, but on the basis of users’ historical interactions with the items. Model-based recommendations broadly refer to many recommendation algorithms that use theories and tools from machine learning and data mining that can model a variety of recommendation scenarios [27].

A deep neural network is a feed-forward neural network [41], which is trained based on the error backpropagation algorithm and has a strong fitting and learning capability to fit and learn more complex functions due to its multiple hidden layers and the use of a non-linear activation function. Deep neural networks use the BP algorithm to continuously adjust the weight parameters in the network so that the loss function reaches a minimum value [42]. The training of neural networks includes forward propagation of signal and backpropagation of error. In the process of forward propagation, the input signal passes from the input layer through multiple hidden layers and finally reaches the output layer, and the state of neurons in each layer only affects the state of neurons in the next layer but not the state of neurons in this layer. If the state of the neurons in the output layer is not the desired output, then the neural network performs backpropagation of the error, which propagates the error signal backwards along the neural network and makes the error continuously decrease by modifying the weights between the neurons in each layer [43]. Deep neural networks generally include an input layer, hidden layer, and output layer. Each layer comprises multiple neurons, adjacent layers are fully connected to each other, neurons between the same layer are not connected to each other, and the hidden layer generally has multiple.

3. The Proposed Algorithm in This Paper

3.1. Overall Design of the Algorithm

Traditional collaborative filtering recommendation algorithms make recommendations for users based on their historical behavioural data, which only focuses on single explicit feature information of users’ rating data and cannot learn the deeper implicit non-linear feature information in the data, such as the user’s age, user’s occupation, movie’s subject matter, and other feature information [4]. The collaborative filtering recommendation algorithm has a significant head effect on the recommendation results and is weak in handling sparse vectors [44]. The collaborative filtering recommendation algorithm only utilizes information about the interaction between the user and the movie, i.e., the movie rating, and does not utilize information about the user’s own attributes, such as the user’s age, occupation, etc., or the movie’s own attributes, such as the movie’s title or subject matter, which undoubtedly results in the omission of valid information [22]. Although the function of recommendation can also be achieved by using a single dominant feature information, the result of recommendation is often less accurate and cannot meet the personalized needs of users. Therefore, this paper uses the attribute information of users, movies, and the movie rating information to design two neural networks, one for learning user features, named the user network, and one for learning movie features, named the movie network.

The user network uses a deep neural network to learn the user’s features, the input is each attribute feature of the user, and the output is the user feature vector. In order to focus on the critical user attribute information, the feature attention mechanism is used to reinforce the important feature information. The movie network uses a deep neural network and convolutional neural network to learn movie features, and the convolutional neural network is used to process movie title features. In order to make the extracted movie title features more accurate, an attention mechanism is added to the convolutional neural network, and the input is each attribute feature of the movie, and the output is the movie feature vector. In order to focus on the important movie attribute information, the feature attention mechanism is used to strengthen the important feature information. After obtaining the user and movie feature vectors, the movie rating is fitted using the number product of the vectors, and after training, the predicted rating values are compared with the real user rating values when the errors are backpropagated, and the parameters in the model are continuously adjusted and optimized to finally obtain the user feature vector that can characterize the user and the movie feature vector that characterizes the movie features. After obtaining the user and movie feature vectors that can accurately characterize users and movies, three movie recommendation schemes are given, one is a top-k recommendation based on predicted ratings, and the other two are a user-based collaborative filtering recommendation and a movie-based collaborative filtering recommendation, respectively. The overall design structure of the algorithm is shown in Figure 1.

3.2. User Network Architecture Design

The inputs to the user network are user attribute characteristics such as user ID, gender, age, occupation, and zip code. Each category of input features first passes through the embedding layer to become a dense feature vector, then the features passes through the feature attention layer to dynamically learn the importance of each feature and assign weights to each feature, used to amplify important user features by large weights and suppress invalid user features by small weights, then each passes through the fully connected layer to obtain a highly abstract representation of each feature, each feature is then stitched together to form a feature vector and finally passes through the fully connected layer again to form a user feature vector. The user network structure diagram is shown in Figure 2.

The feature attention layer first finds the weights of each user feature, and then multiplies the found weights to the corresponding feature vectors, thus completing the weighting operation of feature importance. Step one is the compression operation of the feature vectors, where the dimensionality of the feature vectors can be different, and the mean value of each feature vector is found to serve the purpose of data compression and information aggregation for each feature vector, which means that each user feature vector is compressed to a value, and then each value representing the user features is stitched into a vector, and the dimensionality of the vector is the number of user attribute features, which is five dimensions here. The number of neurons in this layer is set to three, and the number of neurons in this layer is set to less than five. This layer is mainly used for feature crossover, so that each user attribute feature fully interacts with each other through the fully connected layer. For user attribute features, they are interrelated to dynamically determine which features are important and which are not. After this, a fully connected layer is added, mainly to ensure that the dimension size of the output and the number of user attribute features are the same, so the number of neurons in this layer is five. After the mapping of the two fully connected layers, a weight vector with the same number of dimensions as the number of user attribute features is generated.

Suppose the user’s ith attribute feature is $u_i$, then the compressed information value of the user’s ith attribute feature is $x_i$, and the formula is shown in Equation (1), where $u_i$ is the user’s ith attribute feature vector, m is the dimension of the user’s ith attribute feature vector, and $x=\left[x_1,x_2,\dots ,x_k\right]$, x is the vector stitched by the compressed information value $x_i$ of the user’s ith attribute feature. The weight vector $\alpha$ of the feature is calculated as shown in Equation (2), $\alpha =\left[{\alpha }_1,{\alpha }_2,\dots ,{\alpha }_k\right]$, ${\alpha }_i$ denotes the weight of the user’s ith attribute feature, and the first fully connected layer uses the ReLU activation function, the second fully connected layer uses the Softmax activation function.

$$x_i=\frac{1}{m}\sum _{j=1}^m{u}_i^j$$

(1)

$$\alpha =\mathrm{Softmax}\left(W_2^T\left(\mathrm{ReLU}\left(W_1^T{x}^T+b_1\right)\right)+b_2\right)$$

(2)

The user feature vector with attention $u_i^{\prime }$ is obtained by multiplying the ith attribute feature weight ${\alpha }_i$ with the ith attribute feature vector $u_i$, and the formula is shown in Equation (3).

$$u_i^{\prime }={\alpha }_i{u}_i$$

(3)

The flow chart of user network feature attention layer processing is shown in Figure 3, and the schematic diagram of user network attention calculation is shown in Figure 4.

3.3. Movie Network Architecture Design

The inputs to the movie network are movie attribute features such as movie ID, title, and release date. Firstly, there are multiple input features which are processed separately. The input category features are transformed into dense feature vectors through the embedding layer, the input text features are transformed into dense feature matrices through word separation and encoding and then passed through the convolutional neural network plus attention mechanism to obtain text feature vectors, and the input multi-category features are transformed into feature vectors through muti-hot encoding. Secondly, the processed features go through the feature attention layer to dynamically learn the importance of each feature and assign weights to each feature, performed by amplifying important movie features with large weights and suppressing invalid movie features with small weights, and then each goes through the fully connected layer to obtain a highly abstract representation of each feature. Finally, each feature is stitched together to form a feature vector to go through the fully connected layer to form a movie feature vector. The structure diagram of the movie network is shown in Figure 5.

The feature attention layer first finds the weights of each feature of the movie, and then multiplies the found weights to the corresponding feature vectors, thus completing the weighting operation of feature importance. Finding the weights can be divided into two steps. Step one is the compression operation of the feature vectors, where the dimensionality of the feature vectors can be different, finding the mean value for each feature vector, which serves the purpose of data compression and information aggregation for each feature vector, that is to say, each movie feature vector is compressed to one value. Each value representing the movie features is then stitched into one vector, and the dimensionality of the vector is the number of movie attribute features, here it is four-dimensional. The number of neurons in this layer is set to two, and the number of neurons in this layer is set to less than four. This layer is mainly used for feature crossover, so that each movie attribute feature can fully interact with each other through the fully connected layer, and for movie attribute features, they can be interrelated to dynamically determine which features are important and which are not. After this, a fully connected layer is added, mainly to ensure that the dimensional size of the output and the number of movie attribute features are the same, so the number of neurons in this layer is four. After the mapping of the two fully connected layers, a weight vector with the same number of dimensions as the number of movie attribute features is generated.

Suppose the ith attribute of the movie is $i_i$, then the compressed information value of the ith attribute of the movie is $x_i$, and the formula is shown in Equation (4), where $i_i$ is the ith attribute vector of the movie, m is the dimension of the ith attribute vector of the movie, $x=\left[x_1,x_2,\dots ,x_k\right]$, and x is the vector formed by the compressed information value $x_i$ of the ith attribute feature of the movie. The formula of the feature weight vector $\beta$ is shown in Equation (5), $\beta =\left[{\beta }_1,{\beta }_2,\dots ,{\beta }_k\right]$, and ${\beta }_i$ denotes the weight of the ith attribute of the movie. The first fully connected layer uses the ReLU activation function and the second fully connected layer uses the Softmax activation function.

$$x_i=\frac{1}{m}\sum _{j=1}^m{i}_i^j$$

(4)

$$\beta =\mathrm{Softmax}\left(W_2^T\left(\mathrm{ReLU}\left(W_1^T{x}^T+b_1\right)\right)+b_2\right)$$

(5)

The movie feature vector with attention $i_i^{\prime }$ is obtained by multiplying the ith attribute feature weight ${\beta }_i$ with the ith attribute feature vector $i_i$, and the formula is shown in Equation (6).

$$i_i^{\prime }={\beta }_i{i}_i$$

(6)

The flowchart of movie network feature attention layer processing is shown in Figure 6, and the schematic diagram of movie network attention calculation is shown in Figure 7.

The movie title features are textual features, and for textual features, this paper uses convolutional neural networks and adds an attention mechanism to extract textual features [45]. The input of this module is the feature matrix of movie titles, each row represents a word vector, and the number of rows of the feature matrix represents the number of words in a movie title. In the data processing, the number of words of movie titles in each sample is unified, and the movie title with the maximum length is used as the benchmark, and the movie titles with less than the maximum length in other samples are used to fill the operation, so that the movie titles of all samples are of uniform length.

In processing image data, the height and width of convolutional kernels used in the convolutional neural networks are consistent, but in the text domain, it is necessary to set the width of convolutional kernels to be consistent with the dimension of the word vector. This is because each row in the feature matrix represents a word vector, and in extracting features the word is used as the minimum granularity of the text, if the width of convolutional kernels is smaller than the dimension of the word vector it is not the minimum granularity of the word, and the height of convolutional kernels can be set by itself. Since a movie title is a sentence and the correlation between adjacent words in a sentence is high, it can be operated with convolution, which takes into account not only the word’s meaning but also the context.

After the size of the convolution kernel is specified, the convolution can be performed. The result of one convolution operation is obtained by multiplying the convolution kernel with the corresponding positions of the corresponding feature matrix and then summing. With the sliding of the convolution kernel window, the feature matrix is continuously convolved, and finally, a feature vector of a certain dimension is obtained. Multiple convolutional kernels of different heights can be set, and multiple convolutional kernels of each different height can be set. Next, the pooling operation is performed, and here the maximum pooling strategy is used, taking the maximum of the values in each dimension of the feature vector obtained by the convolution operation until all pooling operations are completed. Finally, the values obtained by convolution and pooling operations under each convolution kernel are stitched into a vector as the feature representation of the movie title, whose dimension is the number of convolution kernels.

After obtaining the feature representation of the movie title, an attention mechanism layer is added so that the values obtained from operations under different convolution kernels receive different weights, and then the weights are multiplied onto the movie title feature representation vector to obtain the final movie title feature representation vector. The first fully connected layer plays the role of feature crossover, and the feature values obtained under each convolutional kernel are fully crossed over to learn which feature values are more important under the convolutional kernel. The second fully connected layer serves to keep the weight feature and movie title feature representation vector dimensions consistent.

The input to the convolutional layer is a feature matrix representing a sentence of dimension $k\times l$, i.e., each sentence has k words, each word is represented by a l-dimensional word vector, and a sentence of length k words is represented as $x_{1:k}$. A convolution kernel w with h height and l width is used to convolve with h words, i.e., $x_{i:i+h-1}$, and then the feature $f_j$ is obtained by the ReLU activation function, and $f_j$ represents the jth convolution operation, which is shown in Equation (7).

$$f_j=\mathrm{ReLU}\left(w·x_{i:i+h-1}+b\right)$$

(7)

After all the convolution operations under one convolution kernel are completed, a $(k-h+1)$-dimensional vector f is generated, $f=\left[f_1,f_2,\dots ,f_{k-h+1}\right]$, where f is the vector stitched by the eigenvalues obtained from all the convolution operations under one convolution kernel. Next is the pooling operation, where the maximum pooling strategy is used, and the formula is shown in Equation (8), which completes the convolution and pooling operations under one convolution kernel to obtain the feature values $\widehat{f}$ after convolution and pooling operations under one convolution kernel.

$$\widehat{f}=\max \left(f\right)$$

(8)

After the convolution and pooling operations under all convolution kernels are completed, all the obtained features are stitched into a vector $p=\left[{\widehat{f}}_1,{\widehat{f}}_2,\dots ,{\widehat{f}}_n\right]$, where n is the number of convolution kernels. Next is the attention layer, and the weights are calculated as shown in Equation (9), first by passing the obtained vector p through a fully connected layer so that the features obtained under each convolutional kernel are fully crossed, and second by adding another fully connected layer so that the dimensionality of the weight vector is the same as the number of convolutional kernels.

$$\widehat{p}=\mathrm{Sigmoid}\left(w_2^T\left(\mathrm{ReLU}\left(w_1^{T}p+b_1\right)\right)+b_2\right)$$

(9)

Finally, the obtained weight vector $\widehat{p}$ is made a Hadamard product with the feature vector p obtained by convolution and pooling operations, and the formula is shown in Equation (10) to obtain the final movie title feature representation.

$$z=\widehat{p}\circ p$$

(10)

The schematic diagram of convolution, pooling, and attention calculation is shown in Figure 8, and the flow chart of movie title feature processing is shown in Figure 9.

3.4. Movie Rating Prediction and Recommendation

The user and movie feature vectors are obtained after the user and movie networks, respectively, and the rating prediction is performed next. Here, the product of the number of user and movie feature vectors are used to fit the movie ratings, and the formula is shown in Equation (11), where $r_{iv}$ is the fitted movie rating, u is the user feature vector, and v is the movie feature vector.

$$r_{uv}=u·v$$

(11)

After obtaining the user and movie feature vectors that can accurately characterize users and movies, three movie recommendation schemes are given.

Scheme 1 is as follows: firstly, the candidate movies to be recommended to the user are selected, and the movies with no ratings from the user are screened as candidate movies. Secondly, the movie ratings are predicted by using the feature vector of the user and the feature vector of the candidate movies as a quantitative product; subsequently, the predicted ratings of the candidate movies are sorted in descending order. Finally, top-k recommendations are made. The schematic diagram of the recommendation strategy of scheme 1 is shown in Figure 10.

Scheme 2 is as follows: firstly, the similarity between the feature vector of the user and the feature vector of all other users is calculated, with the top-k users who are similar to the user to be recommended as the candidate users are filtered out; subsequently, a random selection of k movies rated by candidate users, but not rated by the targeted user, are recommended. The schematic diagram of the recommendation strategy of scheme 2 is shown in Figure 11.

Scheme 3 is as follows: firstly, the similarity between the movie rated by the user to be recommended and all other movies is calculated. Secondly, we average the similarity between all other movies and the movie rated by the user to be recommended. Then we recommend the top-k movies to the user to be recommended. The schematic diagram of the recommendation strategy of scheme 3 is shown in Figure 12.

The similarity in schemes 2 and 3 is calculated using the cosine similarity, and the formula is shown in Equation (12), where u is the user feature vector and v is the movie feature vector.

$$S_{uv}=\frac{u·v}{\parallel u\parallel \parallel v\parallel }$$

(12)

4. Experimental Design and Analysis

4.1. Dataset Introduction

The MovieLens 1M dataset, collected by the Grouplens team at the University of Minnesota, is used for this experiment and can be found at http://files.grouplens.org/datasets/movielens/ (accessed on 13 September 2022). The MovieLens 1M dataset contains 1,000,209 rating data of 3883 movies by 6040 users, including user attribute, movie attribute, and movie rating information. The user attribute information includes user ID, gender, age, occupation, postal code, etc. The movie attribute information includes movie ID, title, release time, etc. The rating information includes user ID, movie ID, movie rating and movie rating time, etc.

4.2. Experimental Environment Introduction

This experiment uses Python 3.6 as the development language, Jupyter Notebook as the development tool, a deep learning platform with TensorFlow as the basic framework, TensorFlow2 version, and the deep learning library Keras as the development library for the model, using 64-bit Windows 10 laptop with memory is 16.0 GB, and the processor is Intel(R) Core(TM) i5-7200U as the operating system.

4.3. Evaluation Metrics

In order to verify the accuracy and validity of the model predictions, five indicators, $MSE$ (mean square error), $MAE$ (mean absolute error), $MAPE$ (mean absolute percentage error), $R^2$ (coefficient of determination), and $RMSE$ (root mean square error), were selected for evaluation [46]. When the values of MSE, MAE, and RMSE are small, the value of MAPE tends to 0, and the value of $R^2$ tends to 1, the better the model fits the movie ratings and the more accurate the model predicts the movie ratings. $MSE$, $MAE$, $MAPE$, $R^2$, and $RMSE$ evaluation indicators are calculated as shown in Equations (13)–(17), where the predicted value is $\widehat{y}$, the true value is y, and the number of samples is n.

$$MSE=\frac{1}{n}\sum _{i=1}^n{\left({\widehat{y}}_i-y_i\right)}^2$$

(13)

where its value range is $[0,+\infty )$, and the larger the value, the larger the error.

$$MAE=\frac{1}{n}\sum _{i=1}^n\left|{\widehat{y}}_i-y_i\right|$$

(14)

where its value range is $[0,+\infty )$, and the larger the value, the larger the error.

$$MAPE=\frac{1}{n}\sum _{i=1}^n\left|\frac{{\widehat{y}}_i-y_i}{y_i}\right|$$

(15)

where its value range is $[0,+\infty )$, and the larger the value, the larger the error.

$$R^2=1-\frac{{\sum }_{i=1}^n{\left({\widehat{y}}_i-y_i\right)}^2}{{\sum }_{i=1}^n{\left({\overline{y}}_i-y_i\right)}^2}$$

(16)

where its value range is $(-\infty ,1]$, and the closer the value is to 1, the smaller the error.

$$RMSE=\sqrt{\frac{1}{n}\sum _{i=1}^n{\left({\widehat{y}}_i-y_i\right)}^2}$$

(17)

where its value range is $[0,+\infty )$, and the larger the value, the larger the error.

4.4. Performance Comparison with Classical Machine Learning Models

The performance comparison results of the DNN+CNN-attention+feature attention model proposed in this paper with classical machine learning models such as decision tree, random forest, LightGBM, XGBoost, and KNN on evaluation metrics $MSE$, $MAE$, $MAPE$, and $R^2$ are shown in Figure 13.

From the results in Figure 13, it can be seen that the model proposed in this paper achieves the best performance, outperforming models such as decision tree, random forest, LightGBM, XGBoost, KNN, DNN-Our, and DNN+CNN-Our. Machine learning models, such as decision tree, random forest, LightGBM, XGBoost, and KNN, show worse performance than deep learning models in metrics such as $MSE$, $MAE$, $MAPE$, and $R^2$, which indicates that deep learning models are capable of learning deep representations of features and have powerful learning capabilities. Classical machine learning models, such as decision tree, random forest, LightGBM, XGBoost, and KNN, have low performance compared with the model proposed in this paper, for which it is difficult to handle textual features which are poorly processed with label encoding, as is reported in the literature [47,48,49,50] where the data used do not have textual features. In addition, the DNN+CNN model achieves good performance compared to the DNN model, which indicates that the convolutional neural network performs well in learning movie text features, and the convolutional neural network is also able to learn accurate feature representations for text features. The performance of the model proposed in this paper is comparable with all the above models, which shows that the introduction of the attention mechanism in the CNN can assign different weights to different learned features. Meanwhile, the feature attention mechanism proposed in this paper is able to assign different weights to different user and movie features for the input, which achieves the different importance of different features for movie ratings, verifying the effectiveness of our proposed model.

4.5. Performance Comparison with Some of the Major and Recent Models

The model proposed in this paper was compared with some major approaches and some recent ones, and the performance comparison graph in terms of $RMSE$ metrics is shown in Figure 14.

As can be seen from the results in Figure 14, the model proposed in this paper achieves the best performance, outperforming DST-HRS [51], GHRS [52], ISVD [53], GA-GELS [54], NC Based RS [55], RippleNet [56], Robust ConvMF [57], AE COFILS [58], ConvMF [59], ConvMF+ [59], DNN [60], DNN-Our, DNN+CNN-Our, and other models. DNN-Our shows good performance compared to DNN [60], indication that the user and movie networks designed in this paper learn user and movie features separately to obtain the user and movie feature vectors and then uses the number products of these two vectors to effectively fit the movie ratings. DNN+CNN-Our achieves good performance compared to DNN-Our, indicating that the movie title features learned by the convolutional neural network can reflect the movie title information well. The performance of the model proposed in this paper compares well with DST-HRS [51], GHRS [52], ISVD [53], GA-GELS [54], NC Based RS [55], RippleNet [56], Robust ConvMF [57], AE COFILS [58], ConvMF [59], and ConvMF+ [59]. All models have obvious advantages over each other, which fully demonstrates that the user and movie networks proposed in this paper learn user and movie features separately, and it is very effective in fitting movie ratings using the number products of the user and movie feature vectors. In addition, it also shows that the introduction of an attention mechanism in the CNN can assign different weights to different learned features, avoiding the problem that the learned features have the same importance. Meanwhile, the feature attention mechanism proposed in this paper can assign different weights to different user and movie features of the input, and realize the different importance of different features for movie scoring, fully validating the effectiveness of our proposed model.

4.6. Performance Analysis of Text Feature Extraction Based on CNN-Attention

We designed experiments to further analyse the effect of attentional mechanisms on the effectiveness of convolutional neural networks in extracting text features. Figure 15 compares the effects of convolutional neural networks with or without attention mechanisms on the evaluation metrics of movie rating $MSE$, $MAE$, $MAPE$, and $R^2$. From the experimental results in Figure 15, we can see that the convolutional neural network with the attention mechanism outperforms the convolutional neural network in the evaluation metrics of $MSE$, $MAE$, $MAPE$, and $R^2$. This shows that the introduction of the attention mechanism in the convolutional neural network can assign different weights to different learned text features, avoiding the problem that the text features learned by the model have consistent weights, thus making the learned text features more accurate.

4.7. Feature Attention-Based Feature Extraction Performance Analysis

To verify the effectiveness of our proposed feature attention mechanism, we designed an experiment to compare the effects of DNN and DNN+Feature Attention, DNN+CNN and DNN+CNN+Feature Attention on the evaluation metrics of movie rating $MSE$, $MAE$, $MAPE$, and $R^2$, respectively. The feature attention performance results graph is shown in Figure 16. From the experimental results in Figure 16, we can see that compared with DNN, DNN+Feature Attention has obvious advantages, improving by 0.0836, 0.066, 0.0254, 0.0671 in $MSE$, $MAE$, $MAPE$, and $R^2$, respectively, indicating that different user and movie features have different contributions to movie ratings. This indicates that different user and movie features contribute differently to movie ratings, and the feature attention mechanism is used to make each feature have its own specific weight, achieving the different contributions of different features to movie ratings. Compared with DNN+CNN, DNN+CNN+Feature Attention improves by 0.0692, 0.0751, 0.0289, 0.0555 on $MSE$, $MAE$, $MAPE$, $R^2$, respectively, which is equally apparent and shows the superiority of the feature attention mechanism, contributing to the movie scoring of different features with different weights, solving the problem of different features having the same weight, thus affecting the accuracy of movie ratings.

5. Conclusions

Traditional movie recommendation algorithms only utilize movie rating information. Considering that user and movie attribute information can have an impact on movie ratings, this paper considers user and movie attribute information and adds these two types of useful features to the model to participate in movie rating prediction. Considering the different importance of user and movie to movie ratings, this paper designs a feature attention mechanism to focus on features that are important in movie ratings. As can be seen from the experimental results in Figure 16, compared to DNN, DNN+Feature Attention has obvious advantages in $MSE$, $MAE$, $MAPE$, $R^2$, as evaluation is improved by 0.0836, 0.066, 0.0254, 0.0671, respectively. Compared with DNN+CNN, DNN+CNN+Feature Attention has obvious advantages, with the $MSE$, $MAE$, $MAPE$, $R^2$ evaluation indexes are improved by 0.0692, 0.0751, 0.0289, 0.0555, respectively. In this paper, we design user and movie networks to learn user and movie features separately, and each network learns the user and movie feature vectors separately to carry out subsequent recommendation operations. After obtaining the user and movie feature vectors, they can be used flexibly to design a corresponding recommendation strategies. In this paper, three recommendation strategies were designed, for example, the first uses the product of the two to obtain the movie rating for a top-k recommendation; the second uses the user feature vector to calculate the similarity for collaborative filtering recommendation based on users; and the third uses the movie feature vector to calculate the similarity for collaborative filtering recommendation based on movies. In text feature processing, a convolutional neural network is used to process features and an attention mechanism is added to better focus on key features. From the experimental results in Figure 15, we can see that the convolutional neural network with the attention mechanism outperforms the convolutional neural network in $MSE$, $MAE$, $MAPE$, and $R^2$ evaluation metrics. On the MovieLens 1M dataset, using $MSE$, $MAE$, $MAPE$, $R^2$, and $RMSE$ evaluation metrics, a multi-feature attention mechanism movie recommendation algorithm based on deep neural network and a convolutional neural network proposed in this paper were compared with other models, showing that the $MSE$, $MAE$, $MAPE$ and $RMSE$ evaluation metrics were reduced and the $R^2$ was improved to a certain extent in the proposed algorithm. This shows that the movie recommendation algorithm proposed in this paper is effective.

This paper also has some limitations, this paper uses a convolutional neural network plus an attention mechanism to extract text features. In the future, we can use Transformer and other models with more powerful learning abilities to extract text features. The algorithm proposed in this paper can be applied in many fields. When the problem of a certain field is abstracted into a machine learning problem, the multi-feature attention mechanism can be used to extract features, and multiple networks can be abstracted to learn each feature separately to obtain multiple feature vectors according to the problem.