Inferring the Disease-Associated miRNAs Based on Network Representation Learning and Convolutional Neural Networks

Abstract

Identification of disease-associated miRNAs (disease miRNAs) are critical for understanding etiology and pathogenesis. Most previous methods focus on integrating similarities and associating information contained in heterogeneous miRNA-disease networks. However, these methods establish only shallow prediction models that fail to capture complex relationships among miRNA similarities, disease similarities, and miRNA-disease associations. We propose a prediction method on the basis of network representation learning and convolutional neural networks to predict disease miRNAs, called CNNMDA. CNNMDA deeply integrates the similarity information of miRNAs and diseases, miRNA-disease associations, and representations of miRNAs and diseases in low-dimensional feature space. The new framework based on deep learning was built to learn the original and global representation of a miRNA-disease pair. First, diverse biological premises about miRNAs and diseases were combined to construct the embedding layer in the left part of the framework, from a biological perspective. Second, the various connection edges in the miRNA-disease network, such as similarity and association connections, were dependent on each other. Therefore, it was necessary to learn the low-dimensional representations of the miRNA and disease nodes based on the entire network. The right part of the framework learnt the low-dimensional representation of each miRNA and disease node based on non-negative matrix factorization, and these representations were used to establish the corresponding embedding layer. Finally, the left and right embedding layers went through convolutional modules to deeply learn the complex and non-linear relationships among the similarities and associations between miRNAs and diseases. Experimental results based on cross validation indicated that CNNMDA yields superior performance compared to several state-of-the-art methods. Furthermore, case studies on lung, breast, and pancreatic neoplasms demonstrated the powerful ability of CNNMDA to discover potential disease miRNAs.

1. Introduction

MicroRNAs (miRNAs) are a class of endogenous small RNAs of approximately 20–24 nucleotides in length. miRNAs regulate gene expression in plants and animals after transcription [1,2,3]. Accumulating studies indicate that miRNAs are closely related to the development of human diseases [4,5,6,7]. Therefore, it is imperative to explore potential disease-associated miRNAs (disease miRNAs) in order to understand disease etiology and pathogenesis.

Disease miRNAs prediction can provide reliable candidates for experimental research. Several methods have been proposed for predicting potential disease miRNAs. Mainstream methods are roughly grouped into two categories. The first category of methods primarily uses the regulatory relationship between miRNAs and their target mRNA to predict potential miRNA-disease associations [8]. First, target genes related to miRNAs are obtained by analyzing base complementarity between the miRNA sequence and the putative target gene sequence. Then, using the interactions between the target gene and known disease-related genes, the potential disease miRNAs are predicted [9,10,11,12]. However, such methods are difficult to use due to experimentally validated targets being insufficiently described to date. Although more target gene samples were obtained through some experiments [13,14], prediction results from these methods have a high false positive rate.

Methods belonging to the second category are based on prior biological knowledge that miRNAs with similar functions are usually associated with similar diseases [15]. First, network medicine is the mainstream way of defining related diseases [16,17,18], some methods make full use of network topology to identify disease miRNAs [19,20]. Moreover, disease miRNAs are identified by a random walk on a single miRNA similarity network [21,22]. However, these methods rely too much on known disease-associated miRNAs and are ineffective for new diseases that lack associated miRNAs. To address this drawback, disease similarity information and miRNA-disease associations were introduced to form miRNA-disease heterogeneous networks, where random walks on a two-layer network were used to predict candidate miRNA-disease associations [23,24]. In addition, there are other methods available for calculating miRNA-disease correlation scores, several methods use non-negative matrix factorization [25,26,27,28,29]. By applying structural perturbation [30], by using transduction learning [31], by using the induction matrix [32], through the binary network projection [33], and extracting potential features that pertain to positive sample information [34]. However, there are complex and non-linear relationships between miRNA-miRNA, disease-disease, and miRNA-disease, all previous methods struggle to extract such relationships.

In this study, we present a new approach on the basis of convolutional neural networks for predicting miRNA-disease association, called CNNMDA. It contains two parts consisting of a left and a right. CNNMDA’s left part deeply integrates miRNA similarities, disease similarities, and miRNA-disease associations, and uses these prior biological knowledge to construct the left embedding layer of the miRNA-disease node pair. The right part uses network representation learning to obtain a potential low-dimensional representation of the network node while preserving the topology of the network. Integrating the low dimensional features of miRNAs and diseases helps to estimate the likelihood of association between miRNAs and diseases at the global network level. We construct a deep learning framework based on convolutional neural networks (CNN) for the left and right parts, and learn the original representation and global representation of miRNA-disease node pairs. For some high-frequency diseases, CNNMDA can determine them with high accuracy. Moreover, case studies on 3 diseases indicate that CNNMDA is able to discover potential disease associated miRNAs.

2. Results and Discussion

2.1. Evaluation Metrics

To evaluate the performance of our prediction model, we performed a 5-fold cross-validation on CNNMDA. In the miRNA-disease association data set, the known miRNA-disease associations are called positive samples, while the unknown associations are considered negative samples. In the first place, all positive samples were extracted, and were divided into five subsets randomly. The next step was to extract the same number of negative samples as the positive samples, and these negative samples are also divided into five subsets randomly. In each cross-validation, we took four positive and four negative samples from five subsets to train the prediction model, and the remaining one positive sample and one negative sample were used as test data to evaluate the prediction performance.

Given a threshold τ, a positive sample is obtained when the prediction score is higher than τ, otherwise a negative sample is added. Accordingly, TPR and FPR are calculated by the following formula:

$$\mathrm{TPR}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}, \mathrm{FPR}=\frac{\mathrm{FP}}{\mathrm{TN}+\mathrm{FP}},$$

(1)

where TP and TN represent the number of positive and negative samples that are judged correctly, respectively. FN indicates the number of positive samples that are misidentified as negative samples, and FP represents the number of negative samples that are misidentified as positive examples. We can calculate different TPRs and FPRs based on different thresholds. The obtained TPRs and FPRs can be plotted as ROC curves, and the area under the receiver operating characteristic curve (AUC) can be used as a criterion for evaluating prediction performance.

By observing relevant data, we noted that there were only a few known miRNA-disease associations (positive samples), accounting for $\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$31$}\right.$ of all associated data. It is not difficult to surmise that there is a serious imbalance between positive samples and negative samples. In this case, the PR (precision-recall) curve usually reflects more information than the ROC curve [35,36]. Precision indicates the proportion of positive samples that are defined correctly compared to the number of positive samples currently defined as positive examples. Recall indicates the proportion of positive samples that are defined correctly compared to all positive samples. This is calculated as follows:

$$\mathrm{Precision}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}, \mathrm{Recall}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}.$$

(2)

Similarly, precisions and recalls are calculated by different thresholds. Based on these values, the PR curve can be plotted and the area under the precision–recall curve (AUPR) can be calculated to evaluate the prediction performance of the model. In addition, biologists usually choose the top-rank prediction results for experimental validation, so we calculated the average recall value for 15 diseases in the $\mathrm{top}\mathrm{k}\in \left\{30,60,90,\dots ,240\right\}$ as another evaluation method.

2.2. Comparison with Other Method

To evaluate the prediction performance of CNNMDA, we compare it with several methods that are at the forefront of the field. These included DMPred [29], GSTRW [37], BNPMDA [33], and Liu’s method [23], where the parameter settings for each method were set to achieve the best performance. In CNNMDA, the parameters ${\mathrm{w}}_{\mathrm{l}}$, ${\mathrm{w}}_{\mathrm{f}}$, and ${\mathrm{w}}_{\mathrm{p}}$ in the convolution operation were set to 3, 5, and 2, respectively. Thus, the size of the convolution sliding window $\mathrm{J}\in {\mathrm{R}}^{3\times 5}$, and the sliding window $\mathrm{F}\in {\mathrm{R}}^{1\times 2}$ in the pooling operation. The number of filters was set to 30 (n_conv = 30). The parameters $\mathsf{\alpha },\mathsf{\beta },{\mathsf{\lambda }}_{\mathrm{m}},{\mathrm{and}\mathsf{\lambda }}_{\mathrm{d}}$. used in the matrix factorization were all obtained from the set {0.2, 0.5, 0.8, 1, 2, 5, 8} by cross-validating the values of the various parameters. CNNMDA achieved the best performance when $\mathsf{\alpha }=0.2$, $\mathsf{\beta }=0.2$, ${\mathsf{\lambda }}_{\mathrm{m}}=0.2$, and ${\mathsf{\lambda }}_{\mathrm{d}}=0.2$. In addition, the parameter $\mathsf{\lambda }$ in the combination formula for the left part and right part was set to 0.4. In other comparison methods, the parameters are set according to the values given in the original article.

As shown in Figure 1A and

Table 1. Prediction results of CNNMDA and the other four methods for 15 diseases in terms of the area under the receiver operating characteristic curve (AUC).

Diseases Name	AUC CNNMDA	GSTRW	DMPred	BNPMDA	Liu’s Method
Breast neoplasms	0.991	0.822	0.939	0.906	0.896
Hepatocellular carcinoma	0.978	0.770	0.899	0.784	0.846
Renal cell carcinoma	0.960	0.801	0.897	0.830	0.785
Squamous cell carcinoma	0.932	0.821	0.894	0.793	0.897
Colorectal neoplasms	0.924	0.742	0.882	0.724	0.864
Glioblastoma	0.916	0.821	0.906	0.781	0.828
Heart failure	0.986	0.823	0.984	0.929	0.816
Acute myeloid leukemia	0.969	0.817	0.894	0.784	0.924
Lung neoplasms	0.987	0.795	0.941	0.903	0.931
Melanoma	0.994	0.788	0.909	0.909	0.859
Ovarian neoplasms	0.955	0.831	0.934	0.924	0.855
Pancreatic neoplasms	0.971	0.853	0.913	0.725	0.892
Prostatic neoplasms	0.982	0.828	0.947	0.896	0.895
Stomach neoplasms	0.994	0.781	0.922	0.740	0.838
Urinary bladder neoplasms	0.982	0.821	0.921	0.879	0.870

The bold values indicate the higher AUCs.

, CNNMDA achieved the best average performance for 15 diseases (AUC of ROC curve = 0.968). DMPred’s performance was the second best, where the AUC was 5% lower than CNNMDA, reaching 0.918. In addition, the AUC values of BNPMDA and Liu reached 0.838 and 0.870, which were 13% and 9.8% lower than CNNMDA, respectively. GSTRW performed poorly compared with other methods, and its AUC value was only 0.816, 15.2% lower than CNNMDA. Among the methods, GSTRW displayed poor performance since only miRNA and disease similarity information is used in this method. Liu’s method and BNPMDA fully capture the information of the network topology, and DMPred improves performance by integrating multiple sources of effective information. Our method, CNNMDA, through deep learning original representation and global representation of miRNA-disease node pairs, achieved the best prediction performance. CNNMDA also obtained the best results in each disease.

As shown in Figure 1B and

Table 2. Prediction results of CNNMDA and other four methods for 15 diseases in terms of the area under the precision–recall curve (AUPR).

Diseases Name	AUPR CNNMDA	GSTRW	DMPred	BNPMDA	Liu’s Method
Breast neoplasms	0.919	0.261	0.681	0.245	0.378
Hepatocellular carcinoma	0.871	0.234	0.539	0.574	0.335
Renal cell carcinoma	0.549	0.127	0.325	0.328	0.152
Squamous cell carcinoma	0.290	0.104	0.191	0.272	0.170
Colorectal neoplasms	0.425	0.136	0.279	0.177	0.273
Glioblastoma	0.277	0.142	0.270	0.452	0.166
Heart failure	0.874	0.160	0.669	0.451	0.157
Acute myeloid leukemia	0.262	0.118	0.236	0.367	0.207
Lung neoplasms	0.706	0.140	0.481	0.480	0.343
Melanoma	0.896	0.157	0.410	0.477	0.309
Ovarian neoplasms	0.543	0.152	0.453	0.386	0.239
Pancreatic neoplasms	0.593	0.133	0.308	0.136	0.283
Prostatic neoplasms	0.673	0.150	0.414	0.175	0.231
Stomach neoplasms	0.881	0.207	0.503	0.306	0.303
Urinary bladder neoplasms	0.694	0.134	0.331	0.292	0.229

The bold values indicate the higher AUPRs.

, we obtained the average AUPR of all the methods with respect to 15 diseases, and plotted the corresponding PR curves. It is not difficult to surmise that the average AUC-PR area of CNNMDA under 15 diseases was also significantly higher than for other methods. Compared with GSTRW, BNPMDA, Liu’s Method and DMPred, CNNMDA displayed AUC-PR increases of 43.9%, 28.9%, 27.7%, and 24%, respectively. Moreover, in 13 of the 15 diseases, CNNMDA achieved the best performance.

In addition, to further verify the superior performance of our method compared with other methods, we applied a commonly used method called a paired t-test. After calculation, the p-values of all paired t-test results were less than 0.05 (

Table 3. Comparison of different methods based on AUCs with a paired t-test.

p-Value between CNNMDA and Another Method	DMPred	GSTRW	BNPMDA	Liu’s Method
p-values of ROC curves	3.3219 × 10−5	8.5916 × 10−23	5.4483 × 10−10	2.0247 × 10−10
p-values of PR curves	1.4386 × 10−8	2.7951 × 10−13	1.181 × 10−2	2.9012 × 10−8

), indicating that the performance of CNNMDA is significantly better than other methods.

This was accompanied by a higher recall rate, which means that we have successfully identified more positive samples in the top k candidate list, further indication of the superiority of this model’s prediction performance. Therefore, we calculated the average recall rate for all methods in 15 diseases (Figure 2). Our method achieved the highest average recall rate at different thresholds, where the top 30 reached 0.712, the top 60 reached 0.921, and the top 90 reached 0.980. The recall rate of DMPred was the second best at all thresholds, and ranked 0.512 in the top 30, 0.726 in the top 60, and 0.860 in top 90. The recall rate of BNPMDA and Liu was very close. The average recall rates of the top 30, the top 60, the top 90 in the former were 0.459, 0.645, and 0.753, and the latter were 0.411, 0.641, and 0.763, respectively. In contrast, GSTRW exhibited poor performance, and the recall rates in the top 30, top 60 and top 90 were 0.191, 0.469, and 0.661, respectively.

2.3. Case Studies of Lung Neoplasms, Breast Neoplasms, and Pancreatic Neoplasms

To demonstrate CNNMDA’s ability to discover potential candidate disease miRNAs, we carried out our method on case studies of lung, breast, and pancreatic neoplasms. Because of space limitations, here, we focused on analyzing the candidates for lung neoplasms and listed the potential top 50 candidate miRNAs in detail (

Table 4. The top 50 lung neoplasms-related candidates.

Rank	miRNA Name	Evidence
1	hsa-mir-106b	dbDEMC, PhenomiR
2	hsa-mir-15a	Literature [47]
3	hsa-mir-16	dbDEMC, PhenomiR, miRCancer
4	hsa-mir-130a	dbDEMC, PhenomiR
5	hsa-mir-193b	dbDEMC, PhenomiR, TCGA
6	hsa-mir-520d	dbDEMC
7	hsa-mir-429	dbDEMC, miRCancer
8	hsa-mir-122	dbDEMC, PhenomiR, miRCancer
9	hsa-mir-149	dbDEMC, PhenomiR
10	hsa-mir-424	dbDEMC, PhenomiR
11	hsa-mir-451a	dbDEMC
12	hsa-mir-378a	Literature [42]
13	hsa-mir-708	dbDEMC
14	hsa-mir-20b	dbDEMC, PhenomiR, TCGA
15	hsa-mir-15b	dbDEMC, PhenomiR, miRCancer
16	hsa-mir-520a	dbDEMC, TCGA
17	hsa-mir-10a	dbDEMC
18	hsa-mir-520b	dbDEMC
19	hsa-mir-625	dbDEMC
20	hsa-mir-141	dbDEMC, PhenomiR, miRCancer
21	hsa-mir-449a	dbDEMC, PhenomiR, miRCancer
22	hsa-mir-99a	dbDEMC, PhenomiR, TCGA
23	hsa-mir-195	dbDEMC, PhenomiR, miRCancer
24	hsa-mir-151a	Literature [43]
25	hsa-mir-296	Literature [44]
26	hsa-mir-449b	dbDEMC, PhenomiR, miRCancer
27	hsa-mir-28	dbDEMC, PhenomiR
28	hsa-mir-342	dbDEMC, PhenomiR
29	hsa-mir-372	dbDEMC, PhenomiR, TCGA
30	hsa-mir-345	dbDEMC, PhenomiR
31	hsa-mir-92b	dbDEMC, PhenomiR
32	hsa-mir-328	dbDEMC, PhenomiR
33	hsa-mir-367	dbDEMC, PhenomiR
34	hsa-mir-373	dbDEMC, PhenomiR
35	hsa-mir-302b	dbDEMC, PhenomiR, miRCancer
36	hsa-mir-194	dbDEMC, PhenomiR
37	hsa-mir-1258	dbDEMC
38	hsa-mir-320a	dbDEMC, PhenomiR
39	hsa-mir-152	dbDEMC, PhenomiR
40	hsa-mir-302c	dbDEMC, PhenomiR
41	hsa-mir-151b	dbDEMC
42	hsa-mir-204	dbDEMC, PhenomiR
43	hsa-mir-23b	dbDEMC, PhenomiR
44	hsa-mir-129	dbDEMC, PhenomiR, TCGA
45	hsa-mir-451b	Literature [45]
46	hsa-mir-374a	Literature [48]
47	hsa-mir-211	dbDEMC, PhenomiR
48	hsa-mir-208a	Literature [46]
49	hsa-mir-1254	dbDEMC, miRCancer
50	hsa-mir-337	dbDEMC, PhenomiR, TCGA

). For the other two diseases, we briefly analyzed the top 50 candidates, and their candidates are listed separately in Supplementary Table S1 and Supplementary Table S2, respectively. To ensure the reliability of prediction results, we first verified our predictions through four public databases, dbDEMC [38], PhenomiR [39], miRCancer [40], and TCGA [41]. Among them, dbDEMC explored miRNAs with abnormal expression in different cancers, where miRNAs with significantly different expression levels in cancer compared with normal tissues were retrieved and statistically analyzed through a “Significance Analysis of Microarrays” method. Similarly, PhenomiR consisted of dysregulated miRNAs associated with diseases. miRCancer provided a comprehensive collection of miRNA expression profiles in a variety of human cancers that are automatically extracted from published literature. TCGA sequenced the entire genome of some neoplasms, including at least 6000 candidate genes and microRNA sequences. It stored genomic characterization and sequence analysis of different tumor types. Since lung cancer is one of the most frequent cancers at present, we took lung neoplasms as an example and analyzed the top 50 candidate miRNAs in detail (Table 4). Among them, dbDMEC contained 43 candidates, and 32 candidates were verified by PhenomiR, indicating that they have been confirmed to be upregulated or downregulated in lung neoplasms. In addition, 10 candidates are included in the miRCancer, which further confirms their associations with the disease, and 7 miRNAs are contained in TCGA, indicating their different expression levels between cancer and normal tissues. The remaining 7 candidates were verified by the literature, where 5 miRNAs were confirmed to exert dysregulations in lung tissues compared with normal tissue [42,43,44,45,46]. miR-15a is involved in the regulation of non-small cell lung cancer and controls cell cycle progression in a synergistic and Rb-dependent manner [47], while miR-374a was confirmed to have different effects at different stages of lung cancer [48].

Among the top 50 candidates for breast neoplasms (ST1), dbDEMC and PhenomiR included 46 and 33 candidates, respectively, whose expression levels varied significantly in breast tumors compared with the normal tissues. The miRCancer contained 22 candidates indicating their associations with breast neoplasms, and 3 candidates were confirmed by TCGA, which demonstrates their different expression levels in different biological states. The remaining 3 candidates were verified by the literature. Among them, miR-142 is upregulated in human breast cancer stem cells (BCSCs) as compared to the non-tumorigenic breast cancer cells [49]. In addition, miR-542 can be used to predict the prognosis of breast cancer patients based on the mRNA expression of target gene lymphocyte antigen 9 (LY9), resulting in the secretion of frizzled protein-related protein 1 (SFRP1) [50]. miR-30e has separately been identified as an independent subtype-specific prognostic marker in breast cancer [51].

The top 50 pancreatic tumor candidates are listed in ST2, where 45 and 34 candidates are contained in the dbDEMC and PhenomiR, respectively. There are 19 candidates in the miRCancer that are known to be associated with the disease. Moreover, TCGA comprises 3 candidates. Five other candidates were also confirmed by the literature [52,53], where we also confirmed their different regulatory effects on pancreatic tumors. Moreover, the downregulation of the tumor protein UNC51-like kinase 1 (ULK1) by miR-372 inhibits the survival of human pancreatic cancer cells [54]. While miR-483 promotes cell proliferation by down-regulating its target gene Smad4 in pancreatic ductal adenocarcinoma (PDAC) cells. The three case studies provided above demonstrated the strong performance of CNNMDA in discovering potential disease associated miRNAs [55].

Functional enrichment analysis of miRNAs is helpful in understanding the function of disease-related miRNAs. Some tools [56,57,58] can be used to analyze the association between the function of the potential disease-associated miRNAs and disease progression. Among these tools, TAM [57] is a convenient online tool (http://cmbi.bjmu.edu.cn/tam), it integrates miRNAs into different sets according to various rules and provides investigators with the potential biological functions of the list of miRNAs. We performed functional enrichment analysis for the predicted top 50 potential disease-related miRNAs based on TAM. Here, we focused on the analysis of candidate miRNAs related to lung neoplasms (Figure 3). The results of the enrichment analysis of breast neoplasms and pancreatic neoplasms are listed in Supplementary Figures S1 and S2, respectively. Among the top 50 candidate miRNAs that relate to lung neoplasms, 12 miRNAs are involved in cell cycle-related functions, and 13 miRNAs are involved in human embryonic stem cell regulation functions. Furthermore, 9 miRNAs are concerned with apoptosis. In addition, 7, 7, and 6 miRNAs are related to cell proliferation, hormones regulation, and immune response, respectively. All the miRNA-related functions mentioned above have been confirmed to be closely related to the development of diseases. For instance, numerous studies have confirmed that cell cycle changes are closely related to cancer. When the normal cell cycle changes, the changes may lead to the division of some cells in the body and further cause cancer [59,60]. Specifically, it has been confirmed that cell cycle regulators play an important role in lung neoplasms [61]. As for human embryonic stem cell regulation, some research indicates it may be the origin of some solid tumors, including lung neoplasms, stomach neoplasms, and breast neoplasms [62,63]. Moreover, the metastasis of lung cancer may occur due to the dysregulation of some hormones in the human body [64], and the senescence of the immune system is a possible cause of lung cancer [65]. The other enriched functions associated with more miRNAs, such as apoptosis and cell proliferation, are related to the occurrence and development of diseases [66]. The above analysis can provide some insights into the putative roles of these candidates in lung neoplasms.

3. Materials and Methods

3.1. Dataset

We obtained miRNA-disease association data from the human miRNA-disease database (HMDD v2. 0) [67]. The database has collected thousands of miRNA-disease associations that have been experimentally verified. There were 492 miRNAs and 329 diseases in the dataset of our study, which contained 5218 known associations between them. The disease terms we used were derived from the U.S. National Library of Medicine. In terms of diseases, phenotype similarities and the semantic similarities between them were extracted from related literature [68].

3.2. Representation of miRNA and Disease Heterogeneous Data

3.2.1. MiRNA Similarity Measure

miRNAs with approximate function have high probabilities of being associated with similar diseases. Most existing miRNA similarity data are obtained by calculating the similarity of the diseases to which they are associated. For example, miRNA $m_1$ is associated with diseases $d_2$, $d_3$, and $d_4$, miRNA $m_2$ is associated with diseases $d_1$, $d_3$, and $d_4$. By calculating the similarity between disease set {$d_2$, $d_3$, $d_4$} and set {$d_1$, $d_3$, $d_4$} as the similarity between $m_1$ and $m_2$ [69], it can be defined as $M_{12}$. miRNA similarities used in this study were calculated according to the above method. The similarity of $N_m$ miRNAs is represented by matrix $\left[M_{ij}\right]\in R^{N_m\times N_m}$ and each value is between 0 and 1.

3.2.2. Disease Similarity Measure

Similarities between disease pairs can be judged by their semantics and phenotype; under normal conditions, if there are more common semantic terms and phenotypes between disease pairs, then they have a high probability of similarity. Accordingly, previous work calculated disease similarity based on the phenotypic and semantic information of the disease [29]. Disease similarities used in this study were obtained using Xuan’s method. The similarity of N_d diseases are represented by matrix $\left[D_{ij}\right]\in R^{N_d\times N_d}$ and each value is also between 0 and 1.

3.2.3. miRNA-Disease Associations

We used the matrix $A\in R^{N_m\times N_m}$ to represent the associations between $N_m$ miRNAs and $N_d$ diseases. If miRNA $m_i$ is known to be associated with a disease $d_j$, $A_{ij}=1$; contrastingly, $A_{ij}=0$ indicates that their association has not been explored.

3.3. Prediction Model Based on Network Representation Learning and Dual CNN

Here, we developed a novel prediction method based on network representation learning and dual CNN to infer potential miRNA-disease associations. Its prediction model is divided into a left part and a right part (Figure 4). The left part learns feature association representation between a miRNA $m_i$ and a disease $d_j$ through original feature information. The right part projects all miRNA and disease nodes into a low-dimensional space, thereby integrating their global information to obtain representative low-dimensional features of $m_i$ and $d_j$. These two parts use CNN layer deep learning node level representation and global level representation, respectively. Next, the two sides obtain prediction scores for $m_i$ and $d_j$ through the fully connected layer, respectively. Finally, we integrated two scores as a final prediction score between $m_i$ and $d_j$.

3.3.1. Embedding Layer on the Left

The left part integrates original feature information of miRNA and disease pairs. This is performed on the basis that miRNAs may be associated with similar diseases if they have similar functions and vice versa. Therefore, we combined miRNA and disease similarities as well as associations between them to form the feature representation of the left part. As an example, we have described the integration process of miRNA $m_1$ and disease $d_5$ (Figure 5). The first row of M is denoted as $M_1$. It contains similarity information between miRNA $m_1$ and all of the miRNAs. The fifth row of $A^T$ is denoted as $A_5^T$, it consists of the association of disease $d_5$ with all of the miRNAs. miRNA $m_1$ is similar to $m_3$, $m_5$, and $m_6$, and the disease $d_5$ has known association with $m_3$ and $m_5$. Thus $m_1$ and $d_5$ are likely to be associated, as they are all related to $m_3$ and $m_5$. Similarly, we integrate the first row of matrix A ($A_1$) together with the third row of matrix D ($D_5$). miRNA $m_1$ is known to be associated with $d_1$, $d_3,$ and $d_4$, and disease $d_5$ is similar to $d_1$ and $d_3$, since both $m_1$ and $d_5$ are related to $d_1$ and $d_3$. Therefore $m_1$ and $d_5$ may be associated with each other. Finally, we integrated $M_1$, $A_1$, $D_5$, and $A_5^T$ to form the feature matrix $B\in R^{2\times \left(N_m+N_d\right)}$.

3.3.2. Embedding Layer on the Right

In the right part, miRNA (disease) is projected into k-dimensional space to obtain representative low-dimensional features of miRNA and disease pairs, and integrate their global information. Non-negative matrix factorization (NMF) is an effective way to get a low-dimensional representation, and is widely used in data representation [70,71]. It aims to calculate two optimal non-negative matrices such that their product approximates the original matrix. Specifically, for the miRNA similarity matrix $M\in R^{N_m\times N_m}$, each row in it can be considered as a feature vector of a single miRNA, and we need to find non-negative matrices $W\in R^{N_m\times k}$ and $X\in R^{N_m\times k}$ whose products approximate to M, such as $M\approx \mathrm{W}{X}^T$. Therefore, there is an optimization item as follows:

$$\underset{W\ge 0,X\ge 0}{min}{\Vert M-W{X}^T\Vert }_F^2,$$

(3)

where ${\Vert ·\Vert }_F$ is the Frobenius norm of a matrix, X represents a low-dimensional feature matrix of miRNA, and $W$ is the basic matrix which is similar to the parameter matrix. Finally, k represents the target dimension that we reduce to.

Similarly, we also project disease information into k-dimensional space, in terms of disease similarity matrix $D\in R^{N_d\times N_d}$, calculating matrices $V\in R^{N_d\times k}$ and $Y\in R^{N_d\times k}$, and $D\approx \mathrm{V}{Y}^T$. Thus, combined with Equation (3), we obtain the following objective function:

$$\underset{W,X,V,Y\ge 0}{min}{\Vert M-W{X}^T\Vert }_F^2+\alpha {\Vert D-V{Y}^T\Vert }_F^2,$$

(4)

where $\mathsf{\alpha }$ is a parameter for control the contribution of the second item. Y represents a low-dimensional disease feature matrix, and V is a basic matrix.

The i-th row of feature matrix X, $x_i$, which is a row vector, represents the k-dimensional features of miRNA $m_i$. Similary, the j-th row of feature matrix Y, $y_j$, also a row vector, represents the k-dimensional features of disease $d_j$. If the k-dimensional features of $m_i$ and $d_j$ are mostly consistent, there may be potential links between them. The association probability between them is estimated by the formula $\left(x_i\right){\left(y^T\right)}_j={\left(x{y}^T\right)}_{ij}$, and the score should be close to $A_{ij}$, which is the true association probability between $m_i$ and $d_j$. As a result, we extend the objective function to:

$$\underset{W,X,V,Y\ge 0}{min}{\Vert M-W{X}^T\Vert }_F^2+\alpha {\Vert D-V{Y}^T\Vert }_F^2+\beta {\Vert A-X{Y}^T\Vert }_F^2,$$

(5)

where $\mathsf{\beta }$ is a parameter used to adjust the contribution of the third item.

In addition, if miRNA $m_i$ is similar to miRNA $m_j$, $m_i$ is likely related to other miRNAs whose similarity scores are relatively high with $m_j$. To preserve this network topology information, we introduce the graph regular term, which indicates that if the two miRNAs (diseases) $m_i$ and $m_j$ are close in original feature space, these two miRNAs (diseases) should also be closer to each other when their feature dimensions are reduced. However, prior to this, we need to establish a graph model for miRNA and disease feature matrices.

For the miRNA feature matrix, a graph model $S^m$ is constructed. The elements $S_{ij}^m$ are comprised of:

$$S_{ij}^m=\{\begin{array}{ll}1,& if m_i is the k-nearest\\ & neighbor of m_j\\ 0,& otherwise\end{array}$$

(6)

where $m_i$ and $m_j$ represent the i-th miRNA and the j-th miRNA, respectively. The similarity score between them is obtained from matrix M, and similarity scores of the $m_i$ are sorted with the rest of the miRNAs to determine whether $m_j$ belongs to the k-nearest of $m_i$.

For the disease feature matrix, a supplementary graph model $S^d$ is constructed:

$$S_{pq}^d=\{\begin{array}{ll}1,& if d_p is the k-nearest\\ & neighbor of d_q\\ 0,& otherwise\end{array}$$

(7)

where $d_p$ and $d_q$ represent disease p and disease q. The similarity between $d_p$ and $d_q$ are obtained from matrix D.

The graph regular terms for miRNAs and diseases are defined as:

$$\frac{1}{2}{{\displaystyle \sum }}_{i,j=1}^{N_m}{\Vert x_i-x_j\Vert }^2{S}_{ij}^m=tr\left(X^T{L}_{m}X\right),$$

(8)

$$\frac{1}{2}{{\displaystyle \sum }}_{p,q=1}^{N_d}{\Vert y_p-y_q\Vert }^2{S}_{pq}^d=tr\left(Y^T{L}_{d}Y\right),$$

(9)

where tr(.) represents the trace of a matrix, $x_i$ represents the i-th row of the matrix X, and $y_p$ represents the p-th row of the matrix Y. $L_m=D_m-S^m$ and $L_d=D_d-S^d$ are graph Laplacian matrices for $S^m$ and $S^d$, respectively, $D_m$ and $D_d$ are the diagonal matrices and $D_m\left(i,i\right)={{\displaystyle \sum }}_{j=1}^{N_m}{S}^m\left(i,j\right)$, $D_d\left(p,p\right)={{\displaystyle \sum }}_{q=1}^{N_d}{S}^d\left(p,q\right)$. Combining the graph regular terms into the objective function gives:

$$\begin{gathered} \underset{W,X,V,Y\ge 0}{min}{\Vert M-W{X}^T\Vert }_F^2+\alpha {\Vert D-V{Y}^T\Vert }_F^2+\beta {\Vert A-X{Y}^T\Vert }_F^2 \\ +{\lambda }_{m}Tr\left(X^T{L}_{m}X\right)+{\lambda }_{d}Tr\left(Y^T{L}_{d}Y\right), \end{gathered}$$

(10)

where ${\lambda }_m$ and ${\lambda }_d$ are parameters used to adjust the regularization terms.

Since the objective function in Equation (10) is not convex, it is unrealistic to hope to find a global optimal solution. We propose a strategy to find local minima by iteratively updating one item with other items fixed, such as updating X with W, Y, and V fixed. In addition, to constrain the matrix elements that are non-negative ($w_{ij}\ge 0,x_{ij}\ge 0,v_{pq}\ge 0,y_{pq}\ge 0$), we add the corresponding Lagrangian function. Finally, according to the trace and Frobenius norm of a matrix, the objective function L can also be expressed as:

$$\begin{array}{cc}\hfill L=& Tr\left(M{M}^T-W{X}^T{M}^T-MX{W}^T+W{X}^{T}X{W}^T\right)\hfill \\ & +\alpha Tr\left(D{D}^T-V{Y}^T{D}^T-DY{V}^T+V{Y}^{T}Y{V}^T\right)\hfill \\ & +\beta Tr\left(A{A}^T-X{Y}^T{A}^T-AY{X}^T+X{Y}^{T}Y{X}^T\right)\hfill \\ & +{\lambda }_{m}Tr\left(X^T{L}_{m}X\right)+{\lambda }_{d}Tr\left(Y^T{L}_{d}Y\right)\hfill \\ & +Tr\left(\delta W^T\right)+Tr\left(\mu X^T\right)+Tr\left(\varphi V^T\right)+Tr\left(\theta Y^T\right),\hfill \end{array}$$

(11)

where $\delta , \mu , \phi , \theta$ represents a Lagrange multiplier. Then the partial derivatives of X, W, Y, and Z can be calculated through the following function:

$$\frac{\partial L}{\partial X}=-2M^{T}W+2X{W}^{T}W-2\beta AY+2\beta X{Y}^{T}Y+2{\lambda }_m{L}_{m}X+\mu ,$$

(12)

$$\frac{\partial L}{\partial W}=-2MX+2W{X}^{T}X+\delta ,$$

(13)

$$\frac{\partial L}{\partial V}=-2\alpha DY+2\alpha V{Y}^{T}Y+\varphi ,$$

(14)

$$\frac{\partial L}{\partial Y}=-2\alpha D^{T}V+2\alpha Y{V}^{T}V-2\beta A^{T}X+2\beta Y{X}^{T}X+2{\lambda }_d{L}_{d}Y+\theta .$$

(15)

According to Karush–Kuhn–Tucker (KKT) conditions [72], ${\delta }_{ij}{w}_{ij}=0$, ${\mu }_{ij}{x}_{ij}=0,{\phi }_{ij}{v}_{ij}=0,{\theta }_{ij}{y}_{ij}=0$, the following equations are obtained:

$${(-M^{T}W+X{W}^{T}W-\beta AY+\beta X{Y}^{T}Y+{\lambda }_m{L}_{m}X)}_{ij}{x}_{ij}=0,$$

(16)

$${(-MX+W{X}^{T}X)}_{ij}{w}_{ij}=0,$$

(17)

$${(-\alpha DY+\alpha V{Y}^{T}Y)}_{ij}{v}_{ij}=0,$$

(18)

$${(-\alpha D^{T}V+\alpha Y{V}^{T}V-\beta A^{T}X+\beta Y{X}^{T}X+{\lambda }_d{L}_{d}Y)}_{ij}{y}_{ij}=0.$$

(19)

Finally, we obtained the following update rules:

$$x_{ij}\leftarrow x_{ij}\frac{{(M^{T}W+\beta AY+{\lambda }_m{S}^{m}X)}_{ij}}{{(X{W}^{T}W+\beta X{Y}^{T}Y+{\lambda }_m{D}_{m}X)}_{ij}},$$

(20)

$$w_{ij}\leftarrow w_{ij}\frac{{(MX)}_{ij}}{{(W{X}^{T}X)}_{ij}},$$

(21)

$$w_{ij}\leftarrow w_{ij}\frac{{(MX)}_{ij}}{{(W{X}^{T}X)}_{ij}},$$

(22)

$$y_{ij}\leftarrow y_{ij}\frac{{(\alpha D^{T}V+\beta A^{T}X+{\lambda }_d{S}^{d}Y)}_{ij}}{{(\alpha Y{V}^{T}V+\beta Y{X}^{T}X+{\lambda }_d{D}_{d}Y)}_{ij}}.$$

(23)

Here, we iteratively update W, X, V, and Y through the above update formula until convergence. The first row of $X$, $x_1$, is the feature vector of miRNA $m_1$ and the fifth row of $Y$, $y_5$, is the feature vector of disease $d_5$. If the k-dimensional features of $m_1$ and $d_5$ are mostly consistent, there may be potential links between them. Moreover, $x_1$ and $y_5$ are integrated together to form a global feature representation matrix $PϵR^{2\times k}$ (Figure 6).

3.3.3. Convolutional Module on the Left

Feature matrix B, consisting of $m_1$ and $d_5$, is input to the CNN module to learn the original node pair representation between $m_1$ and $d_5$. In the convolutional layer, the convolution filter size is set to $w_l\times w_f$, and the number of filters is $n_{conv}$. Therefore, the convolution filters can be represented as $W_{conv}\in R^{w_l\times w_f\times n_{conv}}$. The output after the convolution operation is expressed as $C_1\in R^{2\times \left(N_m+N_d-w_f+1\right)\times n_{conv}}$. The following formulas represents the convolution process of X:

$$X_{conv,i,j}=(X{(i,j,1)}_{,}X\left(i,j,2),\dots ,X\left(i,j,j+w_f-1\right)\right)\hspace{1em}{X}_{conv,i,j}\in R^{w_l\times w_f},$$

(24)

$$\begin{gathered} C_1\left(i,j,t\right)=g\left(X_{conv,i,j}\ast W_{conv}\left(:,:,t\right)+b_{conv}\left(t\right)\right), \\ i\in \left[1,2\right], j\in \left[1,N_m+N_d-w_f+1\right], t\in \left[1,n_{conv}\right], \end{gathered}$$

(25)

where $\mathrm{X}\left(i,j,1\right)$. indicates the first column vector in the sliding window when the filter moves to the j-th position of the i-th layer, and $C_1\left(i,j,t\right)$ represents the convolution result when the t-th filter slides to the j-th position of the i-th layer. g is a nonlinear activation function and $b_{conv}$ is a bias vector. In the above formula, the stride is set to 1 by default. In the pooling layer, we apply the max-pooling operation to compress the convolution result $C_1$, and get the output $P_1\in R^{\left(N_m+N_d\right)\times n_{conv}}$:

$$P_1\left(i,p,t\right)=max(C_1\left(i,w_p\ast \left(p-1\right)+1,t\right),\dots ,C_1\left(i,w_p\ast p,t\right)),$$

(26)

where $P_1\left(i,p,t\right)$ is the pooling result for the p-th position in the i-th row, and $w_p$ is the width of the sliding window in the pooling operation. Next, $P_1$ is used as the input to enter the second convolution layer after the same convolution and pooling operations as above to get the result $H_1\in R^{\frac{1}{2}\times \left(N_m+N_d\right)\times 2n_{conv}}$. We then flatten $H_1$ to a column vector $\mathrm{c}\in R^{v\times 1}$($\mathrm{v}=\frac{1}{2}\times \left(N_m+N_d\right)\times 2n_{conv}$). Finally, through the fully connected layer $W_L$ and the softmax layer, we obtain the association prediction score between $m_1$ and $d_5$. The score is defined as $scor{e}_1\in R^{2\times 1}$:

$$scor{e}_1=W_L\times c.$$

(27)

3.3.4. Convolutional Module on the Right

The embedding in the right part, $P\in R^{2\times k}$, is used as input to learn global information about miRNA $m_1$ and disease $d_5$ through their representative k-dimensional features. The process of convolution and pooling on the right is similar to the left, and the detailed operation process is defined as follows:

$$Y_{conv,i,j}=(Y{(i,j,1)}_{,}Y\left(i,j,2),\dots ,Y\left(i,j,j+w_f-1\right)\right)\hspace{1em}{Y}_{conv,i,j}\in R^{w_l\times w_f},$$

(28)

$$C_2\left(i,j,t\right)=\mathrm{g}\left(Y_{conv,i,j}\ast W_{conv}\left(:,:,t\right)+b_{conv}\left(t\right)\right),$$

(29)

$$P_2\left(i,p,t\right)=max(C_2\left(i,w_p\ast \left(p-1\right)+1,t\right),\dots ,C_2\left(i,w_p\ast p,t\right)),$$

(30)

where Y indicates the value of the sliding window at different positions. $C_2$ is the feature output after the convolution layer, which then passes through the pooling layer to obtain $P_2$. We also use $P_2$ as the input for the next convolution layer, and obtain the output $H_2\in R^{\frac{1}{2}\times k\times 2n_{conv}}$ through convolution and pooling operations. The next step is to flatten $H_2$ to a column vector $\mathrm{o}\in R^{v\times 1}$($\mathrm{v}=\frac{1}{2}\times k\times 2n_{conv}$). Finally, through the fully connected layer $W_R$ and the softmax layer, we obtain the association prediction score between $m_1$ and $d_5$. The score is defined as $scor{e}_2\in R^{2\times 1}$:

$$scor{e}_2=W_R\times \mathrm{o}.$$

(31)

3.3.5. Combined Strategy

Considering the two parts of the prediction scores between $m_1$ and $d_5$ from different perspectives, the optimal performance of the two parts may be different. Therefore, we integrated $scor{e}_1$ and $scor{e}_2$ as the final association score. It is defined as follows:

$$score=\lambda \times scor{e}_1+\left(1-\lambda \right)\times scor{e}_2,$$

(32)

where $\lambda \in \left(0,1\right)$ is a parameter used to weigh the score contributions of $scor{e}_1$ and $scor{e}_2$. The left and right CNN models all establish a loss function based on cross entropy, defined as $los{s}_1$ and $los{s}_2$, respectively:

$$los{s}_1=-{{\displaystyle \sum }}_{i=1}^T\left[y_{label}\times log\hspace{0.17em}a+\left(1-y_{label}\right)\times log(1-a\right)],$$

(33)

$$a=\frac{e^{scor{e}_1\left(1\right)}}{e^{scor{e}_1\left(0\right)}+e^{scor{e}_1\left(1\right)}},$$

(34)

$$los{s}_2=-{{\displaystyle \sum }}_{i=1}^T\left[y_{label}\times log\hspace{0.17em}b+\left(1-y_{label}\right)\times log(1-b\right)],$$

(35)

$$b=\frac{e^{scor{e}_2\left(1\right)}}{e^{scor{e}_2\left(0\right)}+e^{scor{e}_2\left(1\right)}},$$

(36)

where $y_{lable}$ represents the actual associated label between the miRNA and the disease. If the association between the miRNA and the disease is known, $y_{lable}=1$, otherwise, $y_{lable}=0$. $scor{e}_1\left(0\right)$ and $scor{e}_1\left(1\right)$ represent the association scores of miRNAs and diseases on the left side. It is similar to a binary classification problem, where $scor{e}_1\left(0\right)$ represents the probability that $m_1$ and $d_5$ are not associated, and $scor{e}_1\left(1\right)$ represents the probability of an association. Finally, we used the softmax function to obtain the association probability a. Similarly, for the calculated right path association probability b, $\mathrm{score}\left(1\right)$ indicates the final prediction score between $m_1$ and $d_5$, and T represents the number of training samples.

3.4. Predicting Novel Disease-Related miRNAs

The predictive performance of CNNMDA was evaluated through a cross-validation process and several case studies, and was applied to predict potential candidate miRNAs for all 329 diseases. We used all positive and negative samples to train CNNMDA. The predicted results of 329 diseases are listed in Supplementary Table S3. Moreover, the candidate miRNAs related to 3 diseases are analyzed in case studies and they come from Supplementary Table S3.

4. Conclusions

CNNMDA has been developed as a novel method based on network representation learning and dual convolutional neural networks for predicting potential miRNA-disease associations. CNNMDA captures the internal relationships between miRNAs and diseases, including miRNA similarities and disease similarities. Meanwhile, it also captures the associations between miRNAs and diseases. Moreover, the representations of the miRNA nodes and the disease nodes are learned based on an entire miRNA-disease network, and as such are deeply integrated to enhance logical reasoning. The new framework based on network representation learning and dual convolutional neural networks is able to learn the original and global representations of a miRNA-disease pair. CNNMDA’s performance was verified by cross-validation with 15 common diseases and case studies on 3 diseases. Experimental results indicated that CNNMDA outperforms existing methods in terms of both AUCs and AUPRs. It is able to generate reliable candidate miRNA-disease associations for subsequent validation by biologists.