DRR: Global Context-Aware Neural Network Using Disease Relationship Reasoning and Attention-Based Feature Fusion

Abstract

The prediction of future disease development based on past diagnosis records has gained significant attention due to the growing health awareness among individuals. Recent deep learning-based methods have successfully predicted disease development by establishing relationships for each diagnosis record and extracting features from a patient’s past diagnoses in chronological order. However, most of these models have ignored the connections between identified diseases and low-risk diseases, leading to bottlenecks and limitations. In addition, the extraction of temporal characteristics is also hindered by the problem of global feature forgetting. To address these issues, we propose a global context-aware net using disease relationship reasoning and attention-based feature fusion, abbreviated as DRR. Our model incorporates a disease relationship reasoning module that enhances the model’s attention to the relationship between confirmed diseases and low-risk diseases, thereby alleviating the current model’s bottlenecks. Moreover, we have established a global graph-based feature fusion module that integrates global graph-based features with temporal features, mitigating the issue of global feature forgetting. Extensive experiments were conducted on two publicly available datasets, and the experiments show that our method achieves advanced performance.

1. Introduction

The growing interest in using artificial intelligence (AI) [1] to improve healthcare delivery and patient outcomes through electronic health records (EHRs) reflects a shift towards leveraging technology in medicine, especially in predicting patient health outcomes. A vital component of this progression is the extraction and utilization of patient characteristics from longitudinal EHRs [2,3]. The more detailed the patient data available, the more sophisticated and effective the resulting medical AI systems can be. In recent years, there has been a significant surge in the adoption of Electronic Health Record (EHR) systems globally, leading to the accumulation of substantial amounts of electronic patient data. These data include both structured components, such as disease and medication codes, and unstructured elements, such as clinical narratives and progress notes.

Given that EHRs consist of both structured and unstructured data, the application of deep learning techniques is an excellent approach. Deep learning can effectively process and interpret the varied and complex layers of data in EHRs. This ability allows for the extraction of a wide range of information, which can be used to predict the likelihood of a patient developing certain medical conditions, anticipate reactions to specific medications, and forecast future health conditions. The use of deep learning for disease prediction in the context of EHRs represents a forward-thinking strategy in leveraging the rich, multifaceted data available in these records. In this realm, disease prediction models using deep learning are primarily categorized into three types: models based on the Transformer architecture [4,5,6], time-series models based on recurrent neural networks (RNNs) [7,8], and models based on convolutional neural networks (CNNs) [9] and graph-based [10,11,12,13]. One of the main keys behind these models for disease prediction tasks is to mine information from patients’ previous diagnostic data at each diagnosis and utilize it to forecast patients’ future disease progression. How models learn disease features, especially the contextual relationships between diseases, remains a challenge to this day.

Recently, Lu et al. proposed GCL [10] and Chet [11], which employed graph-based methods to establish relationships between diseases and achieved success in health event prediction tasks. However their methods also face certain bottlenecks:

• The clinical observation that if a patient has had disease A for a prolonged length of time, the probability of the patient developing disease B in the future significantly increases is the primary motivation for employing a graphical approach to establish connections between diseases. Therefore, it is reasonable to draw graphical correlations between diseases with current diagnoses and diseases with high risk. Deep learning models can be used to understand the relationships between high-risk diseases and presently identified diseases, aligning with clinical practice and potentially assisting in disease prediction. However, this clinical experience frequently neglects the potential for future diagnoses of low-risk disorders [14]. While using high-risk diseases to forecast future diseases can improve model performance, ignoring low-risk ones will create a bottleneck in the model. As illustrated in Figure 1, previous methods have focused on the relationship between high-risk diseases and diagnosed diseases; ours looks at not only the relationship between high-risk diseases and diagnosed diseases but also the relationship between low-risk diseases and diagnosed diseases.
• RNNs are frequently used to extract temporal information from a patient’s historical diagnostic records after correlations between currently diagnosed diseases and high-risk or low-risk diseases have been shown graphically. This approach aligns with clinical experience, as it involves predicting disease progression based on a patient’s prior medical history. However, when dealing with long-term sequences, RNN algorithms encounter the issue of forgetting. Depending on the co-occurrence connections between diseases, it is possible to overlook global information and consider certain diseases as low-risk when they seem to have improved. However, these diseases often have a significant likelihood of recurrence, which will create a bottleneck in the model.

We propose two modules to address these problems: the disease relationship reasoning module and the global graph-based feature fusion module. We offer a module for disease relationship reasoning to address the first bottleneck. The aim is to disrupt the co-occurrence associations among specific diseases to increase the number of samples in the training set where low-risk diseases transition to confirmed diseases. This compels the model to focus on the connections between diagnosed diseases and low-risk diseases. Additionally, we introduce a module for global graph-based feature fusion and denoising to tackle the second challenge. This module extracts subgraph-based features for high-risk diseases, presently diagnosed diseases, and their relationships with high-risk diseases at prior time points at various time points. A global subgraph is created by denoising and combining subgraphs from multiple time points. The issue of forgetting global features is solved by the global subgraph-based features, which complement the global characteristics that RNN algorithms overlook.

We conducted experiments with our method on two public datasets. For the health event prediction task, DRR achieved state-of-the-art results, demonstrating improvements of 2.06% and 2.95% over the previous best results on the two datasets, respectively. In summary, the main contributions of this paper are summarized as follows:

• We propose the DRR model, which reconstructs the relationships between diagnosed diseases, high-risk diseases, and low-risk diseases, breaking the model bottleneck caused by existing models’ over-reliance on diagnosed and high-risk diseases.
• In our approach, we mitigate the global feature forgetting issue in disease prediction tasks of the GRU method by de-fusing the features of high-risk diseases at different time nodes with the features of diagnosed diseases.
• DRR was tested on two public datasets, Medical Information Mart for Intensive Care III (MIMIC-III) [15] and Medical Information Mart for Intensive Care IV (MIMIC-IV) [16], and achieved state-of-the-art results, thereby confirming the effectiveness of our method.

2. Related Work

2.1. RNN-Type in Health Event Prediction

Health event prediction is a fundamental aspect of the field of medical informatics, encompassing the prediction of individual medical conditions, diseases, or health-related events. Researchers have made significant strides in this field over the years through various data sources and methodologies. EHR is widely used in studies to forecast health events. Researchers use machine learning and deep learning techniques to mine patient demographics, medical histories, diagnostic records, and other EHR data for insightful information. These models are designed to predict diseases, adverse events, or treatment outcomes based on patients’ historical data. Dipole [8] incorporated three attention mechanisms into the recurrent neural network (RNN) to model connections between different patient visits. Choi et al. proposed RETAIN [7], an improved RNN model that combines clinical interpretability with high accuracy. DRR constructs relationships between diagnosed diseases and high-risk or low-risk diseases in a patient’s single visit using a graph-based method. Subsequently, it utilizes RNN-like methods to extract temporal features from a patient’s entire visit history records. While methods exist that leverage CNN features to address the limitations of RNN-based approaches, they fundamentally differ from DRR.

2.2. Graph Method in Health Event Prediction

Graph-based methods have attracted significant attention and demonstrated remarkable success in the field of health event prediction. These approaches represent relationships among medical entities, such as diseases, symptoms, and drugs, as nodes and edges within a graph structure. This enables the extraction of more intricate features and enhances prediction capabilities [17]. In recent times, substantial efforts have been directed towards two primary objectives: establishing relationships within graphs [18,19] and improving the efficiency of graph neural networks. Notably, GRAM [13] was the pioneering work that successfully integrated graph-based approaches into disease prediction tasks. G-BERT [12] is another notable example, which constructs disease relationships using graphs and subsequently leverages BERT [20] to process these constructed disease relationships. Moreover, CGL [10] delves deep into the exploration of patient–disease interactions and the utilization of medical domain knowledge. In contrast, the Chet method [11] constructs relationships between diseases using graphs and utilizes RNN-like techniques for the extraction of temporal features. This is achieved by incorporating transformation functions that learn patterns in disease progression, ultimately enabling predictions of future disease development. DRR, during the construction of the graph, intentionally disrupts certain co-occurrence relationships to sharpen the model’s attention on connections between diagnosed diseases and low-risk diseases, thus effectively tackling the bottleneck challenges encountered in the aforementioned graph-based methods.

2.3. NLP Method in Health Event Prediction

Recently, researchers have recognized natural language processing (NLP) as a crucial technology for healthcare event prediction. NLP techniques learn from extensive amounts of human text data in a self-supervised manner and have demonstrated significant success when fine-tuned for specific tasks within EHR [5,6]. Nevertheless, such models often demand substantial computational resources. In contrast, DRR, with its modest model parameter size of just 2.3 million, can be deployed at the edge for efficient processing.

3. Method

In this section, we present the details of our proposed method DRR. An overview of DRR is shown in Figure 2.

3.1. Problem Formulation

EHR contains the medical records of patients for each visit to the hospital. During each medical encounter, patients are diagnosed with one or more diseases, which are represented by a series of medical codes. These confirmed diseases are assigned specific codes predefined by modern medical systems such as ICD-9-CM or ICD-10. For instance, “left heart failure, diabetes mellitus” is assigned the code 428.1, 250 in ICD-9-CM. We use the set $U=\left\{u_1,u_2,\dots ,u_d\right\}$ to represent the confirmed diseases, where d is the code number. For each patient $p\in P$, a set of vectors $m^t=\left\{m_{u_1},m_{u_2},\dots ,m_{u_d}\right\}$ is used to represent the diagnosed diseases during the t-th visit, where $m_{u_i}^t\in \left\{0,1\right\}$, and $m_{u_i}^t=1$ indicates that the patient was diagnosed with disease $u_i$ during the t-th visit, and $t=1,2,\dots ,T,i\in \left[1,d\right]$, where T is the visit number of p. For patient p, all of their diagnostic records can be represented as $E_p=\left\{m^1,m^2,\dots ,m^T\right\}$. The EHR dataset is defined as $D=\left\{E_p|p\in P\right\}$.

Health event prediction involves using an EHR dataset D, given a patient p’s entire history of confirmed medical records $E_p$, to predict all potential health events $m^{t+1}$ that p may encounter during their next hospital visit.

Common disease prediction involves the identification of prevalent chronic conditions such as palpitations, hypertension, diabetes, etc. By analyzing a patient’s historical confirmed medical records $E_p$, it aims to predict whether the patient will be diagnosed with a specific disease $u_i$ in the future, $m_{u_i}^{T+1}\in \left\{0,1\right\}$.

3.2. Global Graph Definition

We constructed the global graph G using the method proposed by Chet. In G, each node represents a disease $u_i$ from set U. The edges $<u_i,u_j>$ in G represent the frequency of diseases $u_i$ and $u_j$ co-occurring. It is important to note that the values of $<u_i,u_j>$ and $<u_j,u_i>$ are different because, based on clinical experience, the presence of disease $u_i$ may lead to the presence of disease $u_j$, but the reverse is not necessarily true. We used an adjacency matrix $A\in R^{d\times d}$ to represent the global graph G:

$$A_{ij}=\left\{\begin{array}{cc}0& ifi=jor\frac{f_{ij}}{{\sum }_{j=1}^d{f}_{ij}}<\delta ,\\ \frac{f_{ij}}{q_i}& otherwise.\end{array}\right.,$$

(1)

$${\Delta }_i=\left\{c_j|\frac{f_{ij}}{{\sum }_{j=1}^d{f}_{ij}}\ge \delta \right\}.$$

(2)

$A_{ij}$ represents the edges $<u_i,u_j>$ in the global graph G. $f_{ij}$ is the frequency of disease $u_j$ appearing in the samples when disease $u_i$ is present. We have set a threshold $\delta$, to filter out diseases that have low-frequency occurrences when disease $u_i$ is present while retaining diseases with high-frequency occurrences ${\Delta }_i$, $q_i$ represents the sum of frequencies of all other high-frequency occurring diseases when disease $u_i$ is present, $q_i={\sum }_{c_j\in {\Delta }_i}{f}_{ij}$.

3.3. Disease Relationship Reasoning Module

In this section, we introduce the disease relationship reasoning module in DRR. From the diseases identified in patients at time t, it is possible to deduce high-risk diseases based on clinical experience. In addition, the illnesses that people may be diagnosed with at time $t+1$ frequently develop from the high-risk illnesses noted at time t. However, the low-risk illnesses discovered at time t may also be the source of the illnesses detected at time $t+1$. If the majority of the training samples at time $t+1$ are derived from high-risk diseases identified at time t, the model may not allocate sufficient attention to the relationship between low-risk diseases at time t and the diagnosed diseases at time $t+1$. To tackle this issue, we employed a masking strategy to conceal certain diseases identified at time t, effectively categorizing some high-risk diseases as low-risk diseases. This method increases the number of instances where low-risk diseases progress into diagnosed diseases, as defined in Equations (3) and (4):

$$mas{k}_i=\left\{\begin{array}{cc}0& \alpha \\ 1& 1-\alpha \end{array}\right.,0\le \alpha \le 1,$$

(3)

$$\overline{m^t}=m^t\oplus mask.$$

(4)

We have set a threshold value $\alpha$. For a patient u at a certain time t and all confirmed disease sequences $m^t$ for any disease j, there is a probability $1-\alpha$ that $m_j^t$ transitions from confirmed to unconfirmed, and a probability of $\alpha$ to remain unchanged, $\overline{m^t}$ represents the confirmed sequence that has undergone random masking.

$E_i^t$ represents the disease confirmed in patient i at time t. We need to predict $E_i^{t+1}$ based on $E_i^t$. If patient j is diagnosed with a disease similar to that of patient i and also develops some diseases that patient i does not have, then we consider these newly developed diseases as high-risk diseases $H_i^t$ that patient i may be diagnosed with at time $t+1$. It is important to note that some diseases may not have appeared in patient i’s previous visits or in the list of high-risk diseases that may be confirmed, but they appear in the $t+1$ diagnosis. We refer to such diseases as low-risk diseases $L_i^t$. This assumption is also based on complex clinical experience.

$$H_i^t=\left\{\begin{array}{cc}1& m_j^t=1,m_i^t=0,A_{ij}\ne 0\\ 0& otherwise\end{array}\right.,m_j^t,m_i^t\in \overline{m^t}$$

(5)

$$L^t=\neg \overline{m^t}\cap \neg H^t$$

(6)

We provided the calculation method for hight-risk diseases $H_i^t$ for a patient at time t from Equation (5). A patient’s high-risk diseases at time t are represented as $H^t=\left\{H_1^t,H_2^t,\dots ,H_d^t\right\}$. Low-risk $L^t=\left\{L_1^t,L_2^t,\dots L_d^t\right\}$ are determined using Equation (6).

In order to transform the multi-hot column vectors of $\overline{m_t}$, $H_t$, and $L_t$ into vectors suitable for deep learning model training, we conducted a self-encoding on them in the context of the global graph G. The details are as follows:

$$L_{em}^t=(L^t\odot em),H_{em}^t=(H^t\odot em),{\overline{m^t}}_{em}=(\overline{m^t}\odot em),$$

(7)

$$F_m^t=\partial (W_{\overline{m}}(A\otimes {\overline{m^t}}_{em}+{\overline{m^t}}_{em}+H_{em}^t)+b_{\overline{m}}),$$

(8)

$$F_H^t=\partial (W_H(A\otimes H_{em}^t+H_{em}^t+{\overline{m^t}}_{em})+b_H),$$

(9)

$$F_L^t=\partial (W_L(A\otimes L_{em}+L_{em})+b_L).$$

(10)

where $em$ denotes a set of randomly generated vectors between 0 and 1, and $F_{\left\{m,H,L\right\}}^t$ denotes the feature encoding learned through the global graph G at time t. $W_{\left\{\overline{m},H,L\right\}}$ and $b_{\left\{\overline{m},H,L\right\}}$ are weight and bias matrices, and ∂ refers to the LeakyReLU [21] activation function.

3.4. Global Graph-Based Feature Fusion Module

This section introduces the proposed global graph-based feature extraction module. To improve future disease prediction accuracy, we extract information from the patient’s previous diagnostic records. The patient’s historical diagnostic features are divided into two categories: local subgraph-based features extracted using our global graph-based feature and fusion module from the patient’s subsequent diagnoses, and temporal features extracted using a $GRU$ [22] module from sequential information. To extract global graph-based characteristics representing the patient’s disease progression, we aggregate all feature subgraphs from different time points into a unified representation. These global graph attributes will provide essential decision support for identifying potential future illnesses of the patient. The denoising module seeks to combine these feature subgraphs since local feature subgraphs from various time points may contain information that overlaps. This enhances the accuracy of predictions by allowing the global graph-based feature data to complement the temporal features produced by the $GRU$.

$$F_{GRU}^t,hidde{n}_t=GRU(F_m^t,F_H^t,F_L^t,hidde{n}_{t-1})$$

(11)

$$g_m^t=Covn1d\left(F_m^t\right)$$

(12)

$$g_H^t=Covn1d\left(F_H^t\right)$$

(13)

$$g_m^{t-1}=Covn1d\left(F_m^{t-1}\right)$$

(14)

$$\delta =softmax\left(\frac{g_m^t{\left(g_m^{t-1}\right)}^T}{\sqrt{a}}\right)g_H^t$$

(15)

$$output=softmax\left(\delta W\right)$$

(16)

The confirmed diseases at time t, high-risk diseases, and the high-risk diseases at time $t-1$ are constructed into a local feature subgraph. The Covn1d is used to denoise the local feature subgraph, followed by the $Atten$ module to establish the relationship between high-risk diseases at time $t-1$, confirmed diseases, and high-risk diseases at time t. The method for establishing the relationship between low-risk diseases and diagnosed diseases is the same as mentioned above. Here, a represents the attention size, $g_m^t$ denotes the features of the confirmed subgraph at time t, $g_m^{t-1}$ signifies the features of the high-risk disease subgraph at time $t-1$, $g_H^t$ embodies the features of the high-risk disease subgraph at time t, and $F_{GAU}^t$ represents the temporal features extracted using the $GRU$ module at time t. Specifically, $g_m^t$ is utilized as Q, $g_m^{t-1}$ as K, and $g_H^t$ as V to derive the local feature subgraph $F_G^t$ through the application of the $Atten$ module. The global graph-based features are denoted as $F_G=\left\{F_G^1,F_G^2,\dots ,F_G^t\right\}$, and the temporal features as $F_{GAU}=\left\{F_{GAU}^1,F_{GAU}^2,\dots ,F_{GAU}^t\right\}$. After concatenating these vectors, they are fused using $Atten$ to combine the temporal features and global graph-based features. Finally, the fused features are passed through a classification module to obtain the disease diagnosis prediction results at time $T+1$.

4. Experiments

4.1. Experimental Setups

We use three common tasks to predict health events:

• Disease prediction. This task involves predicting all possible diagnosed diseases for a patient at time $T+1$ based on the patient’s previous T instances of confirmed disease records. It is a multi-label classification.
• Heart failure prediction. This task involves predicting whether a patient will be diagnosed with heart failure at time $T+1$ based on the patient’s previous T instances of confirmed disease records. It is a binary classification.
• Common disease prediction. We have collected data on some common diseases diagnosed in the MIMIC-IV dataset, including hypertension, diabetes, and others. This task involves predicting whether a patient will be diagnosed with these common diseases at time $T+1$ based on the patient’s previous T instances of confirmed disease records. It is a binary classification.

The above three tasks all use $sigmoid$ as the activation function for the classifier and employ binary cross-entropy loss as the loss function.

Evaluation metrics. The evaluation metrics for the health events prediction are weighted $F_1$ score (w-$F_1$) [23] and top k recall ($R@k$) [24]. w-$F_1$ is a performance metric for evaluating medical codes, calculated as the weighted sum of $F_1$ scores. This metric is commonly employed to assess the performance of machine learning models in the medical field. $R@k$ is a metric used to evaluate the performance of predictive models. It assesses the model’s performance by calculating the average ratio of correctly predicted medical codes in the top k predictions for each visit or scenario, compared to the total number of correct medical codes. This metric helps evaluate the model’s ability to prioritize and predict the most relevant medical codes.

Datasets. We used MIMIC-III [15] and MIMIC-IV [16] to validate the predictive power of DRR. MIMIC-III contains 7493 patients with multiple visits (T≥ 2) from 2001 to 2012, while MIMIC-IV includes 85,155 patients with multiple visits from 2008 to 2019. There is a temporal overlap between the two datasets, and we randomly selected 7493 patients from MIMIC-III and 10,000 patients from MIMIC-IV for the years 2013 to 2019.

We divided both the two datasets into training, validation, and test sets based on patient records randomly. In the case of MIMIC-III, this involved 6000/493/1000 patients, while MIMIC-IV included 8000/1000/1000 patients. We used the last visits as labels and considered the rest as features. The global combination graph G was constructed based on feature visits within the training set.

For the common disease prediction task, we identified common diseases among patients in the MIMIC-IV training set. If a patient is diagnosed with a common disease during their last visit, we assign a label of 1; otherwise, the label is set to 0. The heart failure prediction task followed a similar method.

Baseline methods. To compare DRR with the state-of-the-art models, we selected the following methods as baselines:

• RNN/Attention-based model:RETAIN [7], Dipole [8], Timeline [23], and HiTANet [25].
• CNN-based model: Deepr [9].
• Graph-based model: GRAM [13], G-BERT [12], CGL [10], and Chet [11].

The evaluation metrics for the above-mentioned baseline were based on the results reported in Chet’s study.

Parameter settings. In the experiments, we initialized the model’s parameters randomly. Hyperparameters and activation functions were tuned on the validation set. For disease prediction, the masking rate $\alpha$ was set to 0.6 for MIMIC-III and 0.5 for MIMIC-IV. The $GRU$ hidden size was set to 256 for MIMIC-III and 200 for MIMIC-IV. Subgraph-based features $F_G^t$ were the same for both datasets, set at 32. Atten size a was set to 32 for both datasets. For heart failure prediction, the masking rate $\alpha$ was set to 0 for both datasets. The GRU hidden size was set to 32 for MIMIC-III and 64 for MIMIC-IV. Subgraph-based features $F_G^t$ were the same for both datasets, set at 32. Atten size a was set to 32 for both datasets. Common disease prediction and heart failure prediction used the same parameter settings, with experiments conducted only on the MIMIC-IV dataset.

We employed the Adam [26] optimizer with 100 iterations and a learning rate of 0.01. Our experiments were carried out on a GeForce RTX 3090 while using Python 3.10 and PyTorch 2.0.

4.2. Comparative Experiments

Disease prediction. Comparison of diagnosis prediction results on the MIMIC-III and MIMIC-IV datasets using w-$F1$ (%), $R@10$ (%), $R@20$ (%), AUC (%), and $Parame$ (M). On average, each patient in the dataset has around 13 diseases per visit, and we chose $k=10$ and $k=20$ for our $R@K$ metric to evaluate the performance of disease prediction.

In

Table 1. Comparison of diagnosis prediction results on MIMIC-III datasets using w-$F1$ (%), $R@10$ (%), $R@20$ (%), AUC (%), and $Parame$ (M). On average, each patient in the dataset has around 13 diseases per visit, and we chose $k=10$ and $k=20$ for our $R@K$ metric to evaluate the performance of disease prediction.

Diagnosis Prediction	MIMIC-III
Models	$\mathit{w}$-${\mathit{F}}_{\mathbf{1}}$(%)	$\mathit{R}@\mathbf{10}$(%)	$\mathit{R}@\mathbf{20}$(%)	Params (M)
RETAIN	20.69	26.13	35.08	2.90
Deepr	18.87	24.74	33.47	1.16
GRAM	21.52	26.51	35.80	1.59
Dipole	19.35	24.98	34.02	2.18
Timeline	20.46	25.75	34.83	1.23
G-BERT	19.88	25.86	35.31	6.15
HiTANet	21.15	26.02	35.97	3.33
CGL	21.92	26.64	36.72	1.5
Chet	22.63	28.64	37.87	2.12
DRR	24.69	28.31	37.43	2.34

and

Table 2. Comparison of diagnosis prediction results on MIMIC-IV datasets using w-$F1$ (%), $R@10$ (%), $R@20$ (%), AUC (%), and $Parame$ (M). On average, each patient in the dataset has around 13 diseases per visit, and we chose $k=10$ and $k=20$ for our $R@K$ metric to evaluate the performance of disease prediction.

Diagnosis Prediction	MIMIC-IV
Models	$\mathit{w}$-${\mathit{F}}_{\mathbf{1}}$(%)	$\mathit{R}@\mathbf{10}$(%)	$\mathit{R}@\mathbf{20}$(%)	Params (M)
RETAIN	24.71	28.02	34.46	3.56
Deepr	24.08	26.29	33.93	1.44
GRAM	23.50	27.29	36.36	1.67
Dipole	23.69	27.38	35.58	2.51
Timeline	25.26	29.00	37.13	1.52
G-BERT	24.49	27.16	35.86	7.53
HiTANet	24.92	27.45	36.37	3.93
CGL	25.41	28.52	37.15	1.83
Chet	26.35	30.28	38.69	2.59
DRR	29.30	30.73	39.65	2.32

, we showcase DRR’s predictive performance for diseases across both the MIMIC-III and MIMIC-IV datasets. Our model achieves the state-of-the-art performance. Notably, it performs admirably despite having a modest 2.3 million parameters. Furthermore, our model outperforms other models significantly, particularly on the MIMIC-IV dataset. MIMIC-IV is more advanced than MIMIC-III due to its inclusion of a larger volume of historical patient data. This highlights the shortcomings of earlier methodologies in handling extended time series, while our approach successfully augments time-series features by extracting global subgraph-based features.

Heart failure prediction. We showcase the results of our model in the heart failure prediction task in

Table 3. Hear failure prediction results on MIMIC-III and MIMIC-IV using AUC (%) and F1 (%).

Heart Failure	MIMIC-III			MIMIC-IV
Models	$\mathit{AUC}$(%)	${\mathit{F}}_{\mathbf{1}}$(%)	Params (M)	$\mathit{AUC}$(%)	${\mathit{F}}_{\mathbf{1}}$(%)	Params (M)
RETAIN	83.21	71.32	1.67	89.02	67.38	1.99
Deepr	81.36	69.54	0.53	88.43	61.36	0.65
GRAM	83.55	71.78	0.96	89.61	68.94	0.88
Dipole	82.08	70.35	1.41	88.69	68.94	0.88
Timeline	83.34	71.03	0.95	87.53	66.07	0.73
G-BERT	81.50	71.18	3.58	87.26	68.04	3.95
HiTANet	82.77	71.93	2.08	88.10	68.21	3.95
CGL	84.19	71.77	0.55	89.05	69.36	0.60
Chet	86.14	73.08	0.68	90.83	74.14	0.88
DRR	86.33	72.35	0.85	94.30	81.57	1.00

, using $AUC$ (%) and $F_1$ (%) as evaluation metrics. Even in low-parameter situations, our method achieves state-of-the-art performance. It is crucial to notice that we set the masking rate to 0 for the heart failure prediction task. This is primarily due to the importance of disease co-occurrence associations in clinical medical diagnosis. We categorize undiscovered illnesses into high-risk and low-risk categories by considering these co-occurrence associations. Although a patient’s condition might be caused by a low-risk disease, the majority of confirmed diseases should be caused by high-risk diseases. We use masking to disrupt some of the disease co-occurrence relationships in order to force the model to learn the association between detected diseases and low-risk diseases. It is interesting to note that while the disease prediction task benefits from using masking, the heart failure prediction challenge sees the model performing better without it. This discovery is primarily attributed to the fact that the heart failure prediction task focuses on determining whether a patient will be diagnosed with heart failure at $T+1$, making the correlations between diseases’ co-occurrences essential. On the other hand, the association between currently diagnosed diseases and low-risk diseases emerges as the key bottleneck for boosting model performance in the disease prediction task, which aims to predict all potential diseases a patient may be diagnosed with at $T+1$. This outcome also supports the important role played by our disease relationship reasoning module in establishing connections between low-risk diseases and currently identified diseases.

Common disease prediction. As shown in

Table 4. Common diseases prediction results on MIMIC-IV using AUC (%) and F1 (%).

Diseases Prediction	Chet		DRR
Diseases Name	$\mathit{AUC}$(%)	${\mathit{F}}_{\mathbf{1}}$(%)	$\mathit{AUC}$(%)	${\mathit{F}}_{\mathbf{1}}$(%)
Diabetes	83.98	74.55	95.13	87.15
Heart Attack	91.13	61.94	94.11	63.58
Hypertension	84.32	75.22	87.52	77.22
Cardiac Arrhythmia	85.34	32.43	90.03	79.37

, we selected four common disorders to assess the applicability of our methodology from the MIMIC-IV dataset. To predict these four prevalent diseases, we used the same factors as in the heart failure prediction task. Additionally, we used the state-of-the-art Chet method as a benchmark because it also utilizes parameters from the heart failure prediction task.

4.3. Ablation Study

In

Table 5. The results of ablation experiments are conducted on MIMIC-IV for disease prediction tasks, Using w-$F_1$ (%) as the evaluation metric.

Model Name	w-${\mathit{F}}_1$ (%)
${\mathrm{DRR}}_{{rd}^{-}}$	26.35
${\mathrm{DRR}}_{r^{-}}$	28.95
${\mathrm{DRR}}_{d^{-}}$	28.43
DRR	29.30

, we performed ablation tests to confirm the efficacy of our suggested disease relationship reasoning module and global graph-based feature fusion module. In the MIMIC-IV dataset’s disease prediction task, we compared these experiments. ${\mathrm{DRR}}_{{rd}^{-}}$ is the model with both modules removed. ${\mathrm{DRR}}_{r^{-}}$ is the model with the disease relationship reasoning module removed, and ${\mathrm{DRR}}_{d^{-}}$ is the model with the global graph-based feature fusion module removed. DRR represents the model without removing either module.

4.4. Visualization Analysis

We conducted a visualization analysis in the context of heart failure prediction to validate the efficacy of our model. Prior to applying the classifier, we extracted features and reduced the dimensionality using the t-SNE approach [27]. Subsequently, we labeled the data points and applied color coding, with red indicating diagnosed patients and blue indicating undiagnosed patients. Figure 3 presents the visualization results, where Figure 3a represents the outcomes before training, and Figure 3b represents the results after training. In Figure 3a, the features of both diagnosed and undiagnosed patients seem to cluster together. However, after model training, as shown in Figure 3b, we observe that diagnosed and undiagnosed patients are divided into distinct clusters. This demonstrates the strong classification capability of our model.

5. Conclusions

In this paper, we introduce an approach aiming at addressing the challenges associated with health event prediction. This model incorporates a disease relationship reasoning module and a global graph-based feature fusion module. The disease relationship reasoning module enhances the model’s understanding of the relationship between detected diseases and low-risk diseases, thereby overcoming the prediction model’s limitations. It achieves this by randomly masking illness co-occurrence connections. The global graph-based feature fusion module complements the global features that are often neglected in RNN-based methods by integrating local subgraph-based features. Experiments conducted on the EHR dataset verify the effectiveness of our method. Our method achieved 2.06% of w-$F_1$ improvement on MIMIC-III, 2.95% of w-$F_1$ improvement on MIMIC-IV compared to the current state-of-the-art methods. In the future, we plan to incorporate a wider range of clinical data and explore more efficient and interpretable approaches for healthcare event prediction.