Med-Tree: A Medical Ontology Tree Combined with the Graph Attention Networks for Medication Recommendation

Abstract

Medication recommendation based on Electronic Health Records (EHRs) is a significant research direction in the field of intelligent medicine, which aims to recommend personalized medication combinations for patients based on their historical and current physical conditions. However, since the structural and temporal characteristics of medical records are affected by many uncertain factors, there are many limitations in medication recommendation methods based on EHRs. Specifically, most existing works either fail to adequately assess the structural correlation and temporal dependency among various medical entities or ignore existing knowledge of Drug–Drug Interactions (DDI), which could lead to adverse outcomes. These factors contribute to poor recommendation quality. Therefore, we propose a medical ontology tree model combined with the Graph Attention Networks (GAT) for medication recommendations. First, the class hierarchy extracted from the medical ontology and the GAT model is used to learn the ICD-9 codes of diagnoses and procedures, which enriches the semantic representation of medical entities. Secondly, Gate Recurrent Units (GRU) are used to learn the temporal characteristics of medical entities. Finally, memory bank, dynamic memory and DDI graph are used to optimize the hidden layer results, which improve the accuracy of the model. Experimental results show that the proposed model is superior to the previous methods in all evaluation indicators, and the recommended results have a lower DDI rate.

1. Introduction

Electronic Health Records (EHRs) are the original records of the whole process of diagnosis and treatment of patients, which can greatly improve the work efficiency and medical quality of the care process and provide a smoother patient experience [1,2,3,4]. With the deepening of medical information construction, efforts have been devoted to research studies, including medication recommendations, medical knowledge questions and answers, etc. [5,6,7]. Figure 1 illustrates the synergy of the medication recommendation system in the diagnosis and treatment process, which simplifies the medical process and assists doctors in formulating safe and effective prescriptions. Moreover, the goal of medication recommendations is to recommend personalized medication combinations for patients based on their current medical records and historical health conditions. However, most of the early medication recommendation tasks are based on the prior knowledge of experts with rich clinical experience. In recent years, the continuous optimization of the wireless sensor structure has enabled the collection work of EHRs to be fully carried out [8,9]. Meanwhile, a large number of deep learning models are widely used in the field of medication recommendation, which significantly improves the accuracy of recommendation tasks and the feasibility for practical applications [10,11,12,13]. However, although the EHRs are cleaned and organized in pretraining, there are still many uncertainties, including a lack of information and inconsistencies from different sources, highly subjective and imprecise medical history, etc. Among them, obscure structural and temporal features and Drug–Drug Interactions (DDIs) [14,15] are two more critical factors that bring difficulties to the subsequent medication recommendation tasks:

(1)

Obscure structural and temporal features: the EHRs can be considered as a continuous collection of medical entities consisting of diagnoses, procedures and medications, which are full of obscure structure and temporal correlations among medical entities. For example, a peptic ulcer may cause gastric perforation, chickenpox may cause pneumonia, and cerebral infarction often causes high blood pressure. This cryptic medical knowledge affects the accuracy of medication recommendation tasks.

(2)

DDIs: two or more drugs taken at the same time or in sequences of time may result in a compound effect. The effect can enhance or reduce the drugs’ efficacy and reduce or aggravate the side effects. For example, the combination of a cholinesterase reactivator and atropine sulfate can produce complementary effects, which reduces the amount of atropine and adverse effects. The antimalarial drug artemisinin is susceptible to resistance when used alone and can delay resistance when used in combination with sulfamethoxine and pyrimethamine.

To enhance the learning of obscure structures and temporal features and reduce the DDI rate in recommendation results, early studies include k-means clustering [16], the association rule method [17] and the expert system [18]. With the development of deep learning, Graph Neural Networks (GNN) are introduced into medication recommendation. Studies [19,20,21] introduce Graph Convolutional Networks (GCN) to capture the structural and temporal features between medical events that fully improve the recommendation efficiency. However, they ignore the influence of DDIs in medication-recommendation results. In addition, some models [22,23,24] account for DDIs in training, but their abilities to model the structural and temporal properties of EHRs are poor. Therefore, to simultaneously learn the cryptic structural and temporal features and reduce the DDIs in recommendation results, we propose a medical ontology tree model (Med-tree) combined with the Graph Attention Networks (GAT) for medication recommendation. Specifically, the Med-tree model applies the class hierarchy extracted from the medical ontology and the GAT model to learn the representations of diagnoses and procedures, and the model builds a DDI graph structure to reduce the DDI rate. As a result, the Med-tree model significantly improves the recommendation quality.

The contributions of our work can be summarized as follows:

(1)

We treat EHRs as continuous records with structural and temporal properties and propose a medical ontology tree combined with GAT. The proposed model uses the medical hierarchy to model the structural features of medical entities. Meanwhile, the model applies two GRU models with attention mechanisms to capture the temporal characteristics in EHRs.

(2)

We model the medical knowledge database and construct the drug interaction map. Specifically, the drug interaction map is embedded into the memory component using the memory bank and dynamic memory. Moreover, the query generator is applied to realize the memory search based on the attention mechanism, which effectively reduces the DDI rate.

(3)

The proposed model is tested on MIMIC-III datasets, and the performance of the model is superior to all baselines in terms of the Jaccard Similarity Score, Average F1 score and Precision–Recall AUC. In addition, the model achieves a lower DDI rate among recommended medication combinations compared with previous recommendation models.

The rest of the paper is organized as follows: Section 2 introduces the related works, and Section 3 reviews the framework of the proposed model. In Section 4, the predictive performance of the proposed model is evaluated with baselines from MIMIC-III datasets, and several analyses are presented. In the end, Section 5 presents the conclusions and future directions of the research.

2. Related Work

2.1. Graph Attention Networks

Attention mechanisms have been successfully applied to many sequence-based tasks, such as Machine Translation (MT) and Natural Language Understanding (NLU) [25,26]. Different from Graph Convolutional Networks (GCN) that treat all neighbor nodes equally, Graph Attention Networks (GAT) integrated with the attention mechanism can assign different attention weights to each neighbor node so as to identify the relatively important nodes. Moreover, the attention mechanism incorporated into the GCN can make the propagation step more intuitive, and the GAT can also be regarded as a method of the GCN family.

The GAT model has been applied in many fields. To be specific, Fang et al. [27] proposed a new traffic network speed prediction model named L-GAT, which can capture the spatial characteristics and the temporal dynamics of the traffic network. Based on GAT, Cai et al. [28] proposed an unsupervised model named DQ-GAT, which can achieve scalable and proactive autonomous driving. Moreover, DQ-GAT provided a better trade-off between safety and efficiency in both seen and unseen scenarios. Qin et al. [29] proposed a Co-Interactive Graph Attention Network (Co-GAT), and the model can establish connections between the dialog act recognition and the sentiment classification so as to capture speakers’ intentions. Moreover, studies [22,30] applied GAT to model the internal relationship between medical events that bring breakthrough progress in the medical field.

Similar to the GCN model, the GAT model executes the calculation process by calculating the local network of nodes rather than the whole graph structure, and this process improves the calculation efficiency and reduces memory usage. However, differently from the GCN model, the GAT model assigns different weights to different neighbor nodes according to the importance of the current node, which can better deal with the structural problems, so the GAT model is widely used in social prediction, drug discovery and recommendation systems.

2.2. Medication Recommendation

Medication recommendation is one of the significant research directions in the field of intelligent medicine, which can assist doctors in making safe and effective prescriptions and has great significance for drug synergism and safety. Due to the lack of available datasets and the difficulties in information sharing in early medical research work [31,32], early medication recommendation methods are mainly based on expert prior rules and focus on the association and causality between diagnosis, procedure, and drug combinations. Specifically, Chen et al. [33] described a physician-advisory system for Chronic Heart Failure (CHF) management, which encoded the entire set of clinical practice guidelines using answer-set programming and gave patients medical information like a human physician. Slavescu et al. [34] presented a rule-based system that can assist medical doctors in routine tasks, and the system suggested a diagnosis and recommended treatment for patients based on their medical history and current symptoms. Moreover, Ajmi et al. [35] proposed an expert system that could recommend the right medication combinations depending on the location where the patient lives and the symptoms of the patient. Although these rule-based medication recommendation methods can recommend drugs for patients, their recommendation accuracy is limited.

With the security of information collection and sharing improved [36,37], medical data have been widely accumulated. Meanwhile, medication recommendation methods based on deep learning have gradually become the mainstream methods in the field of medication recommendation. These deep learning approaches learn the relationships between medical entities based on statistical regularities of EHRs and apply these relationships to medical consultation and medication recommendation. To be specific, Wang et al. [23] constructed temporal information from medical records to obtain patient representations and build a key-value memory network to recommend medications. Wang et al. [38] obtained the target distribution associated with safe medication combinations from raw patient records, which could shape distributions of patient representations and reduce the DDI rate of the medication recommendation. Furthermore, Shang et al. [22] pretrained the relationship between drugs in advance and constructed a knowledge map. Moreover, studies [39,40] used the GCN model to learn the relationship between medical events and built a medical relationship tree. Although these methods have higher accuracy than the earlier methods, they still have many limitations and defects, such as high complexity and more training parameters.

Based on the above reasons, we propose a medical ontology tree model (Med-tree) that can simultaneously capture the complex correlation and temporal features in EHRs and effectively reduce the DDI rate in real medical datasets.

3. The Proposed Model

In this section, the training process of the proposed model is described in detail. To be specific, the description of Med-tree is divided into three parts. First, the data structures and the medication recommendation tasks are explained. Next, the framework of the proposed model is described in detail. Finally, the optimization of the Med-tree and the training algorithm are presented.

3.1. Problem Formulation

Medication recommendation models based on EHRs need to be trained on datasets with high accuracy. Moreover, to improve the recommendation accuracy, the EHRs need to be standardized and pretrained. Specifically, the standardized EHRs, the medical ontology tree constructed in pretraining, the definition of the EHR graph and DDI graph and the specific medication recommendation task are described in the following parts. Moreover, the notations used in the proposed model are shown in

Table 1. Notations used in Med-tree.

Notation	Description
${\mathit{X}}^n$	the representation of the standardized EHRs
${\mathit{X}}_{1:t-1}$	the historical visit representation of tth visit
${\mathit{x}}_t$	the representation of tth visit
${\mathit{c}}_d^t,{\mathit{c}}_p^t,{\mathit{c}}_m^t$	the diagnosis codes, procedure codes and medication codes of tth visit
$G_e,G_d$	the EHR and DDI graph
${\mathit{A}}_e,{\mathit{A}}_d$	the adjacency matrixes of $G_e$ and $G_d$
${\mathit{c}}_i,{\mathit{c}}_i^{\prime },{\mathit{e}}_i$	the representation of nodes changed in the medical ontology tree
${\mathit{e}}_d^t,{\mathit{e}}_p^t$	the more comprehensive representations for diagnoses and procedures through the Ontology Embedding
${\mathit{q}}_d^t,{\mathit{q}}_p^t$	the representations of diagnoses and procedures through the Temporal Dependency
${\mathit{q}}^t$	the query of the dynamic memory
${\mathit{M}}_b$	the memory bank
${\mathit{M}}_d^t$	the dynamic memory
${\mathit{M}}_{d,k}^t,{\mathit{M}}_{d,v}^t$	the key vector and the value vector of the dynamic memory
${\mathit{o}}_b^t,{\mathit{o}}_d^t$	the memory outputs through the Knowledge Memory
${\hat{\mathit{y}}}_t$	the multi-label medication recommendation of tth visit
$\widehat{\mathit{Y}}$	the recommended medication set
$\mathit{Y}$	the ground truth of the medication set

.

3.1.1. Definition of the Standardized EHRs

To improve the accuracy of medication recommendation, the EHRs are standardized into ICD-9 diagnosis codes, ICD-9 procedure codes and ATC medication codes, as shown in Figure 2. Furthermore, the standardized EHRs can be represented as a sequence of multi-variate observations:${\mathit{X}}^n$=$[{\mathit{x}}_1^1,$${\mathit{x}}_2^1,$${\mathit{x}}_3^1,$${\mathit{x}}_1^2,$$\dots ,{\mathit{x}}_t^n]$, where $n\in [1,N],t\in [1,T],N$ represents the total number of the patients and T represents the maximum number of one’s visits. To avoid confusion and ambiguity, the superscript n is omitted and the training process of the Med-tree is described for a single patient. To be specific, the tth visit ${\mathit{x}}_t$=$\{{\mathit{c}}_d^t,{\mathit{c}}_p^t,{\mathit{c}}_m^t\}$ of a patient contains diagnosis codes ${\mathit{c}}_d^t$, procedure codes ${\mathit{c}}_p^t$ and medication codes ${\mathit{c}}_m^t.$

3.1.2. Definition of the Medical Ontology Tree

The structure of the ICD-9 encoding system is similar to the class hierarchy extracted from the medical ontology, and it can be represented by a directed acyclic graph structure. Specifically, the leaf node of the structure represents an ICD-9 diagnosis code or an ICD-9 procedure code, and the ancestor node represents the medical hierarchy with the specific classification significance of the medicine. Moreover, the structure of the medical ontology tree is shown in Figure 3, and each node in Figure 3 is described in

Table 2. The description of each node in Figure 3.

Node	ICD-9 Code	Description
${\mathit{c}}_0$	∖	Root
${\mathit{c}}_2$	410∼414	Ischemic heart disease
${\mathit{c}}_5$	413	Angina pectoris
${\mathit{c}}_8$	413.0	Angina decubitus
${\mathit{c}}_9$	413.1	Prinzmetal Angina

. It can be seen that $c_8$ and $c_9$ represent the angina decubitus and prinzmetal Angina, respectively, and the nodes belong to the sub-classification of angina pectoris represented by $c_5$ and the general classification of ischemic heart disease represented by $c_2$. In the training of Med-tree, all diagnosis nodes and their ancestors are regarded as tree nodes, and the relationship between each medical entity is regarded as the edge composition between nodes and their neighbors. Based on the edge connection of these medical entities, the medical ontology tree is constructed. Meanwhile, due to the different medical concepts of diagnosis and procedure, the diagnosis ontology tree and the procedure ontology tree are constructed, respectively.

3.1.3. Definition of the EHR Graph and DDI Graph

The EHR graph and the DDI graph can be represented as $G_e$ = $\{\vartheta ,$${\xi }_e\}$ and $G_d$ = $\{\vartheta ,{\xi }_d\}$ respectively, where $\vartheta$ is the set of medications, ${\xi }_e$ is the edge set of the EHR datasets, and the ${\xi }_d$ is the edge set of known DDIs between a pair of drugs. Furthermore, adjacency matrices ${\mathit{A}}_e$, ${\mathit{A}}_d$∈$R^{\left|\vartheta \right|\times \left|\vartheta \right|}$ are constructed to clarify the construction of edge ${\xi }_e,{\xi }_d$. To be specific, ${\mathit{A}}_e[i,j]$ = 1 can be described as drug i and drug j appearing in the same prescription and acting synergistically. Meanwhile, ${\mathit{A}}_d[i,j]$ = 1 indicates that drug i and drug j contradict each other.

3.1.4. Medication Recommendation Task

Given diagnosis codes ${\mathit{c}}_d^t$, procedure codes ${\mathit{c}}_p^t$ of the current tth visit and historical visit representations ${\mathit{X}}_{1:t-1}$ = $[{\mathit{x}}_1,{\mathit{x}}_2,\dots ,$${\mathit{x}}_{t-1}]$, where ${\mathit{x}}_i$ = $\{{\mathit{c}}_d^i,{\mathit{c}}_p^i,{\mathit{c}}_m^i\}$ represents the set of the diagnosis codes, procedure codes and medication codes of the ith visit. The goal of the medication recommendation task is to obtain a medication combination ${\widehat{\mathit{y}}}_t\in {\{0,1\}}^{N_m}$ at the tth visit based on the patient’s current clinical events ${\mathit{c}}_d^t$, ${\mathit{c}}_p^t$ and historical visits ${\mathit{X}}_{1:t-1}$, where $N_m$ represents the total number of the recommended medications. Meanwhile, Since the medication combination ${\hat{\mathit{y}}}_t$ is more than one label, the medication recommendation based on EHRs is considered a multi-label classification task.

3.2. Model Framework

The Med-tree model is divided into three parts: Ontology Embedding, Temporal Dependency and Knowledge Memory. In the following content, the specific structure shown in Figure 4 and the training process of the proposed model is explained in detail.

3.2.1. Ontology Embedding

The EHR of a single admission is considered a set of medical entities consisting of diagnoses, procedures and medications. Moreover, these entities have internal correlations, and this correlation with different meanings and degrees is called structural characteristics. Specific procedures and medications may be needed for certain diagnoses. However, the vector representation of the standardized EHRs is unordered, and it does not conform to the actual situation. Therefore, the medical ontology tree is used to enrich the vector representation of medical entities. As shown in Figure 5, the point of the arrow indicates the direction of the node message delivery in the medical ontology tree, and the tree realizes the convergence and update of messages among medical nodes through a special GAT. Different operations are carried out for the leaf nodes of specific medical entities and the ancestor nodes with medical classification concepts in the medical ontology tree.

For the non-leaf nodes with the concept of medical classification, the correlation of each child node is adaptively learned, and its representation is updated by weighted addition as follows:

$${\mathit{c}}_i^{\prime }=\prod _{k=1}^K\sigma \left(\begin{array}{c}{\displaystyle \sum _{j\in c\left(i\right)\cup i}}{\alpha }_{ij}^k{\mathit{c}}_j\end{array}\right),$$

(1)

where K represents the number of multiple attention; $\sigma$ is the sigmoid function; $c\left(i\right)$ represents all children nodes of node i; ${\mathit{c}}_j$ is the vector representation of node j; ${\alpha }_{ij}^k$ can be interpreted as the weight coefficient of the kth attention between node i and node j. Specifically, the calculation formula of ${\alpha }_{ij}^k$ [41] can be illustrated below:

$${\alpha }_{ij}^k=\frac{\exp \left(\mathrm{LeakyReLU}\right({\mathit{a}}^T[{\mathit{c}}_i\left|\right|{\mathit{c}}_j\left]\right))}{{\sum }_{l\in c\left(i\right)\cup i}\exp \left(\mathrm{LeakyReLU}\right({\mathit{a}}^T[{\mathit{c}}_i\left|\right|{\mathit{c}}_l\left]\right))},$$

(2)

where ${\mathit{a}}^T$ is a single-layer feedforward neural network; $\left|\right|$ is a concatenation operation; the weight coefficient ${\alpha }_{ij}^k$ is obtained through the LeakyReLU activation function.

For the leaf node ${\mathit{e}}_i$, the information of its ancestor nodes can be aggregated into itself, and the embedded representation of the leaf node can be enriched by combining the information of its medical classification. Furthermore, the graph attention mechanisms are used to adaptively learn the correlation coefficients of each classification concept, and the mechanisms update the representation of leaf node ${\mathit{e}}_i$ as follows:

$${\mathit{e}}_i=\prod _{k=1}^K\sigma \left(\begin{array}{c}{\displaystyle \sum _{j\in p\left(i\right)\cup i}}{\alpha }_{ij}^{k^{\prime }}{\mathit{c}}_j^{\prime }\end{array}\right),$$

(3)

where $p\left(i\right)$ represents all ancestor nodes of node i; ${\mathit{c}}_j^{\prime }$ represents the vector representation of node j; the formula of ${\alpha }_{ij}^{k^{\prime }}$ is translated as follows:

$${\alpha }_{ij}^{k^{\prime }}=\frac{\exp \left(\mathrm{LeakyReLU}\right({\mathit{a}}^T[{\mathit{c}}_i^{\prime }\left|\right|{\mathit{c}}_j^{\prime }\left]\right))}{{\sum }_{l\in p\left(i\right)\cup i}\exp \left(\mathrm{LeakyReLU}\right({\mathit{a}}^T[{\mathit{c}}_i^{\prime }\left|\right|{\mathit{c}}_l^{\prime }\left]\right))}.$$

(4)

The establishment of the medical ontology tree uses the GAT model to transfer information from bottom to top and aggregates the transferred results into all leaf nodes from top to bottom, which enriches the vector representation of the medical entities. Therefore, the diagnosis codes ${\mathit{c}}_d^t$ and procedure codes ${\mathit{c}}_p^t$ of the current tth visit are transformed into the more comprehensive representations ${\mathit{e}}_d^t \mathrm{and} {\mathit{e}}_p^t$.

3.2.2. Temporal Dependency

The records of a patient’s multiple visits can be regarded as a collection of time series. Meanwhile, the current diagnosis of patients may be influenced by their previous health conditions. For example, for patients with chronic conditions, such as diabetes or cerebral infarction, their current visits may be very similar to the historical visits. Moreover, stomach perforation and erysipelas may appear in a patient’s later visit when they have a peptic ulcer or chickenpox. Therefore, it is necessary to fully explore the temporal characteristics of the EHRs.

The more comprehensive representations ${\mathit{e}}_d^t$ and ${\mathit{e}}_p^t$ are input into the GRU model to capture the temporal features of the EHRs. To be specific, the training of the diagnosis representation ${\mathit{e}}_d^t$ can be taken as an example. Two GRU models with attention mechanisms are used to calculate the influence of the historical diagnosis sequence on the current diagnosis representation, and the hidden layer representations ${\mathit{h}}_{d,1}^t$ and ${\mathit{h}}_{d,2}^t$ are obtained as follows:

$${\mathit{h}}_{d,1}^t={\mathrm{GRU}}_{{\beta }_{d,1}}({\mathit{e}}_d^1,{\mathit{e}}_d^2,\dots ,{\mathit{e}}_d^t),$$

(5)

$${\mathit{h}}_{d,2}^t={\mathrm{GRU}}_{{\beta }_{d,2}}({\mathit{e}}_d^1,{\mathit{e}}_d^2,\dots ,{\mathit{e}}_d^t),$$

(6)

where ${\beta }_{d,1},{\beta }_{d,2}$ are two weights of the attention mechanisms, and they can be activated using different functions as follows:

$${\beta }_{d,1}=\mathrm{Softmax}({\mathit{h}}_{d,1}^1,{\mathit{h}}_{d,1}^2,\dots ,{\mathit{h}}_{d,1}^t),$$

(7)

$${\beta }_{d,2}=\tanh ({\mathit{h}}_{d,2}^1,{\mathit{h}}_{d,2}^2,\dots ,{\mathit{h}}_{d,2}^t).$$

(8)

The weights of the attention mechanisms ${\beta }_{d,1}$ and ${\beta }_{d,2}$ are applied to obtain the representation with historical diagnostic information, and the formula of the representation ${\mathit{q}}_d^t$ can be expressed as follows:

$${\mathit{q}}_d^t=\sum _{i=1}^t{\beta }_{d,1}\left[i\right]{\beta }_{d,2}\left[i\right]\otimes {\mathit{e}}_d^t,$$

(9)

where ⊗ represents the element multiplication.

Similar to the training of diagnosis representation ${\mathit{e}}_d^t$, the procedure representation ${\mathit{e}}_p^t$ can be input into the GRU model to capture the influence of the patient’s historical process information and obtain the procedure representation with history dependency information. Therefore, the representations ${\mathit{e}}_d^t$ and ${\mathit{e}}_p^t$ are transformed into ${\mathit{q}}_d^t$ and ${\mathit{q}}_p^t$.

3.2.3. Knowledge Memory

The Knowledge Memory module is constructed to make full use of the medication knowledge, and this module not only embeds EHR and DDI graphs as facts in the memory bank but also inserts the patients’ history information into the dynamic memory to fully obtain information from different views. Specifically, inspired by research [22], the EHRs and DDI records in Section 3.1.3 are embedded into the memory bank to further improve the model recommendation accuracy and reduce the DDI rate of recommendation results. To be specific, the Knowledge Memory module can be divided into the following five steps:

• Convert inputs into a representation vector of the tth visit. Here, ${\mathit{q}}_d^t$ and ${\mathit{q}}_p^t$ transformed from the Temporal Dependency module can be generated as a query ${\mathit{q}}^t$ as follows:

  $${\mathit{q}}^t=f\left([{\mathit{q}}_d^t,{\mathit{q}}_p^t]\right),$$

  (10)

  where $f(\ast )$ represents a transformation function that connects the diagnosis representation ${\mathit{q}}_d^t$ and the procedure representation ${\mathit{q}}_p^t$.
• Design a memory bank. Graphically enhanced memory representations are stored in the memory bank with two adjacency matrices ${\mathit{A}}_e$ and ${\mathit{A}}_d$. According to the GCN procedure, each $\widetilde{{\mathit{A}}_{\ast }}$ ($\widetilde{{\mathit{A}}_{\ast }}$ is used for $\widetilde{{\mathit{A}}_e}$ and $\widetilde{{\mathit{A}}_d}$) is preprocessed as follows:

  $${\tilde{\mathit{A}}}_{\ast }={\tilde{\mathit{D}}}^{-\frac{1}{2}}({\mathit{A}}_{\ast }+\mathit{I}){\tilde{\mathit{D}}}^{-\frac{1}{2}}$$

  (11)

  where $\tilde{\mathit{D}}$ is a diagonal matrix such that ${\tilde{\mathit{D}}}_{ii}$ = ${\sum }_j{\mathit{A}}_{ij}$ and $\mathit{I}$ are identity matrices. Moreover, two-layer GCN is used for the EHR graph and DDI graph to capture the medical relationship between drug combination usage and DDIs, respectively, and the output ${\mathit{M}}_b$ is generated by two graph embeddings added as follows:

  $${\mathit{M}}_b={\mathit{Z}}_1-\beta {\mathit{Z}}_2,$$

  (12)

  $${\mathit{Z}}_1={\tilde{\mathit{A}}}_{e}tanh\left({\tilde{\mathit{A}}}_e{\mathit{W}}_{e,1}\right){\mathit{W}}_1,$$

  (13)

  $${\mathit{Z}}_2={\tilde{\mathit{A}}}_{d}tanh\left({\tilde{\mathit{A}}}_d{\mathit{W}}_{e,2}\right){\mathit{W}}_2,$$

  (14)

  where $\beta$ is a training coefficient; ${\mathit{W}}_{e,1}$ and ${\mathit{W}}_{e,2}$ are the medication embedding from the EHR graph and DDI graph; and ${\mathit{W}}_1$ and ${\mathit{W}}_2$ are the weight matrix of the hidden layer.
• Design a dynamic memory. Specifically, patients’ historical information is inserted into the dynamic memory as key-value pairs to fully capture the information from different perspectives. Moreover, based on ${\mathit{q}}^t$ and medication representation ${\mathit{c}}_m^t$, the history cache of the tth visit can be represented in the form of key-value pairs as follows:

  $${\mathit{M}}_d^t={\{{\mathit{q}}^{t^{\prime }}:{\mathit{c}}_m^{t^{\prime }}\}}_1^{t-1},$$

  (15)

  where ${\mathit{M}}_d^t$ is empty when t = 1, and $t^{\prime }$∈$(1,t-1)$ represents the historical visit before the tth visit. For convenience, ${\mathit{M}}_{d,k}^t$:$[{\mathit{q}}^1,{\mathit{q}}^2,\dots ,{\mathit{q}}^{t-1}]$ are denoted as key vectors, and ${\mathit{M}}_{d,v}^t$:$[{\mathit{c}}_m^1,{\mathit{c}}_m^2,\dots ,{\mathit{c}}_m^{t-1}]$ are denoted as value vectors to represent the history cache of the tth visit.
• Output the memory representation. Get the output ${\mathit{o}}_d^t$ and ${\mathit{o}}_b^t$ by the query ${\mathit{q}}^t$, the memory bank ${\mathit{M}}_b$ and the dynamic memory ${\mathit{M}}_d^t$. Here, attention mechanisms are applied to retrieve the most relevant information with the query ${\mathit{q}}^t$ as follows:

  $${\mathit{o}}_b^t={\mathit{M}}_b^T\mathrm{Softmax}\left({\mathit{M}}_b{\mathit{q}}^t\right),$$

  (16)

  $${\mathit{o}}_d^t={\mathit{M}}_b^T\left({\mathit{M}}_{d,v}^t\right)\mathrm{Softmax}\left({\mathit{M}}_{d,k}^t{\mathit{q}}^t\right).$$

  (17)
• Obtain the multi-label recommended medication combination ${\hat{\mathit{y}}}_t$ by activating ${\mathit{q}}^t$, ${\mathit{o}}_b^t$, and ${\mathit{o}}_d^t$, which can be expressed as follows:

  $${\hat{\mathit{y}}}_t=\sigma \left([{\mathit{q}}^t,{\mathit{o}}_b^t,{\mathit{o}}_d^t]\right),$$

  (18)

  where $\sigma$ represents the activation function.

3.3. Optimization

The gap between the recommended medication combination and the real medication recommendation determines the quality of the recommendation model. For this reason, the medication recommendation task can be regarded as a multi-label classification problem. Thus, the multi-label margin loss ${\mathcal{L}}_{multi}$ and the binary cross-entropy loss ${\mathcal{L}}_{bce}$ are combined as the multi-label classification loss ${\mathcal{L}}_{mlp}$ as below:

$${\mathcal{L}}_{mlp}=\alpha \ast {\mathcal{L}}_{bce}+(1-\alpha )\ast {\mathcal{L}}_{multi},$$

(19)

$${\mathcal{L}}_{bce}=-{\sum }_t^T{\sum }_i{\mathit{y}}_i^{t}log\sigma \left({\hat{\mathit{y}}}_i^t\right)+(1-{\mathit{y}}_i^t)log(1-\sigma \left({\hat{\mathit{y}}}_i^t\right)),$$

(20)

$${\mathcal{L}}_{multi}={\sum }_t^T{\sum }_i^{|{\mathit{c}}_m|}{\sum }_j^{{\hat{\mathit{Y}}}^t}\frac{max(0,1-({\hat{\mathit{y}}}_t\left[{\hat{\mathit{Y}}}_j^t\right]-{\hat{\mathit{y}}}_t\left[i\right]))}{L}.$$

(21)

Here, $\alpha$ is the mixture weights; T represents the maximum number of one’s visits; ${\hat{\mathit{y}}}_i^t$ and $\hat{\mathit{y}}\left[i\right]$ are the mean medication i coordinate at the tth visit and its value, respectively; and ${\hat{\mathit{y}}}^t\left[{\hat{\mathit{Y}}}_j^t\right]$ represents the jth classification label indexed by classification label set ${\hat{\mathit{Y}}}^t$.

DDI loss is designed to control the DDI rate in the medication recommendation result, and its formula can be described as follows:

$${\mathcal{L}}_{DDI}={\sum }_t^T\sum _{i,j}({\mathit{A}}_d\odot \left({\hat{\mathit{y}}}_t^T{\hat{\mathit{y}}}_t\right))[i,j],$$

(22)

where ${\hat{\mathit{y}}}_t^T{\hat{\mathit{y}}}_t$ gives the pair-wise probability of the recommendation result. ⊙ is the element-wise product.

To achieve a lower DDI rate for medication recommendation results, the balance between ${\mathcal{L}}_{mlp}$ and ${\mathcal{L}}_{DDI}$ should be achieved. Inspired by study [42], we transform between ${\mathcal{L}}_{mlp}$ and ${\mathcal{L}}_{DDI}$ with a certain probability as follows:

$$\mathcal{L}=\left\{\begin{array}{cc}{\mathcal{L}}_{DDI}\hfill & \mathrm{if}{s}^{\prime }>s\&\&p=\exp (-\frac{s^{\prime }-s}{temp})\hfill \\ {\mathcal{L}}_{mlp}\hfill & \mathrm{else}\hfill \end{array}\right.,$$

(23)

on the one hand, there will be a high probability of using ${\mathcal{L}}_{DDI}$ in the case where the DDI rate $s^{\prime }$ larger than the expected DDI rate s. On the other hand, decay rate $ϵ$ will be applied to the temperature $temp$ when the model becomes stable with training time, and ${\mathcal{L}}_{mlp}$ will be used as the loss function.

In summary, the training algorithm of the Med-tree model is detailed in Algorithm 1.

Algorithm 1: Training Algorithm.

4. Performance Analysis

This section is mainly introduced from three aspects. First, the experimental configurations are explained, which includes the data source, the baselines, and so on. Secondly, the performance of Med-tree for medication recommendation is verified by comparing it with some basic models. Finally, the feasibility of the proposed model in practical application is derived from a case study.

4.1. Experimental Setup

4.1.1. Data Source

The experiments are carried out on the MIMIC-III datasets, and the structure of the MIMIC-III datasets is described in detail as follows:

MIMIC-III are open-source medical datasets based on intensive care unit patient monitoring managed by MIT. It contains more than 50,000 admissions to intensive care units in large tertiary care hospitals between 2001 and 2012 and 7870 newborns admitted between 2001 and 2008. Specifically, the MIMIC-III data components include vital signs, diagnoses, procedures, medications, and so on. Furthermore, to improve EHRs standardization and availability, the data of the MIMIC-III datasets are transformed into a time-series list of diagnoses, procedures, and medications that are easy to be trained.

In addition, the representations of medical entities on MIMIC-III datasets are converted according to medical standards. To be specific, the medications using NDC codes are converted to ATC codes, and the diagnoses and procedures are integrated with the ICD-9 codes. Moreover, the statistical results are listed in

Table 3. Statistical results of MIMIC-III datasets.

Type			MIMIC-III
patients			35,886
- single-visit			28,936
- multi-visit			6950
clinical entities			3529
- diagnosis			1958
- procedure			1426
- medication			145
max # of visits			29
avg # of visits			2.36
avg # of diagnosis			10.51
avg # of procedure			3.84
avg # of medication			8.80

to further illustrate the characteristics of the MIMIC-III datasets.

4.1.2. Baselines

Several baselines are considered for comparisons as follows:

• LR (Logistic regression) is a classical machine learning method that uses L2 regularization logistic regression and multi-heat vector to represent the multi-label output.
• Leap [43] is a method to predict future events through an additional set of tensors that contain temporal information, and the model performs well in medical data, recommendation engine, and many multiple scenarios.
• RETAIN [21] provides medication recommendations by establishing a two-layer attention-based RNN that considers the influence of temporal factors.
• GAMENet [22] establishes a Drug–Drug Interaction graph to strengthen the relationship between medications and employs a memory enhancement network to learn the temporal dependency of medication recommendation tasks.
• PREMIER [44] adds two attention mechanisms for learning the medical history information, and it combines the GAT model to learn the DDIs, which enhances the unlockability of the recommendation.

4.1.3. Metrics

To measure the accuracy of experimental results, the Jaccard Similarity Score (Jaccard), Average F1 (F1) and Precision–Recall AUC (PRAUC) are considered as the scoring functions. The formula of Jaccard can be written below:

$$\mathrm{Jaccard}=\frac{1}{{\sum }_k^N{\sum }_t^{T_k}1}\sum _k^N\sum _t^{T_k}\frac{|{\mathit{Y}}_t^{\left(k\right)}\bigcap {\hat{\mathit{Y}}}_t^{\left(k\right)}|}{|{\mathit{Y}}_t^{\left(k\right)}\bigcup {\hat{\mathit{Y}}}_t^{\left(k\right)}|},$$

(24)

where N means the total number of patients and $T_k$ can be interpreted as the maximum number of visits of the kth patient.

The PRAUC is calculated by the trapezoidal integral for the area under the PR curve, which can be applied to datasets with imbalanced positive and negative sample numbers.

The F1 score can transform the multi-classification problem into n bipartitions and calculate the average score of the bipartition to obtain the final evaluation index. Moreover, the F1 score can be described as follows:

$$Avg(P_t^{\left(k\right)})=\frac{|{\mathit{Y}}_t^{\left(k\right)}\bigcap {\hat{\mathit{Y}}}_t^{\left(k\right)}|}{|{\mathit{Y}}_t^{\left(k\right)}|},Avg(R_t^{\left(k\right)})=\frac{|{\mathit{Y}}_t^{\left(k\right)}\bigcap {\hat{\mathit{Y}}}_t^{\left(k\right)}|}{|{\hat{\mathit{Y}}}_t^{\left(k\right)}|},$$

(25)

$$\mathrm{F}1=\frac{1}{{\sum }_k^N{\sum }_t^{T_k}1}\sum _k^N\sum _t^{T_k}\frac{2\times Avg(P_t^{\left(k\right)})\times Avg(R_t^{\left(k\right)})}{Avg(P_t^{\left(k\right)})+Avg(R_t^{\left(k\right)})},$$

(26)

where t means the tth visit, and k is the kth patient in the test set.

To measure the medication safety, the DDI rate is defined as the percentage of medication recommendations containing DDIs, and the calculation formula of the DDI rate is as follows:

$$\mathrm{DDI} \mathrm{rate}=\frac{{\sum }_k^N{\sum }_t^{T_k}{\sum }_{i,j}|\{(c_i,c_j)\in {\mathit{Y}}_t^{\left(k\right)}|(c_i,c_j)\in {\epsilon }_d\}}{{\sum }_k^N{\sum }_t^{T_k}{\sum }_{i,j}1}.$$

(27)

Here, the DDI rate will count each medication pair $(c_i,c_j)$ in a recommendation set ${\mathit{Y}}_t^{\left(k\right)}$ if pair $(c_i,c_j)$ belongs to an edge set ${\xi }_d$ of the DDI graph. Furthermore, the $\Delta \mathrm{DDI} \mathrm{rate}\%$ is defined as the percentage of the DDI rate compared to the DDI rate in EHR test datasets, and its formula is defined below:

$$\Delta \mathrm{DDI} \mathrm{rate}\%=\frac{\mathrm{DDI} \mathrm{rate}-\mathrm{DDI} \mathrm{rate}\left(\mathrm{EHRs}\right)}{\mathrm{DDI} \mathrm{rate}\left(\mathrm{EHRs}\right)}.$$

(28)

4.1.4. Evaluation Strategies

The datasets are randomly divided into training, validation, and test sets at a ratio of $2/3:1/6:1/6$. To be specific, the EHR graph $G_e$ and the DDI graph $G_d$ are constructed in pretraining, and the Adam is applied as an optimizer with an initial learning rate of 0.001. The best recommendation result can be fixed on the evaluation set in 40 epochs. All the methods are implemented in PyTorch 1.7.0 and trained on Ubuntu 18.04 with 12 GB memory and Nvidia 3090 GPU.

4.2. Experimental Results

The validity of the proposed model is proven in the following four parts. To be specific, the first part compares the recommendation accuracy of the proposed model with baselines. In the second part, the DDI rate between Med-tree and baselines is compared. Next, in the third part, the performance of Med-tree is compared with the baseline model when the frequency of medication occurrence is different. In the end, the performance of Med-tree with baselines is compared in medical records with varying lengths.

4.2.1. Prediction Performance

Table 4. Collaborative medication recommendation performance of different methods obtained on MIMIC-III datasets. The average number of medicines on the test set is 14.61.

Methods	Jaccard	PR-AUC	F1	${\mathit{R}}_{\mathit{DDI}}\mathbf{Top}40$	Avg # of Med
LR	0.4087	0.6739	0.5669	0.0782	11.37
Leap [43]	0.3911	0.5699	0.5486	0.0633	15.96
RETAIN [21]	0.4140	0.6612	0.5746	0.0863	18.46
GAMENet [22]	0.4479	0.6883	0.6053	0.0780	13.93
PREMIER [44]	0.4641	0.7055	0.6186	0.0754	13.96
Med-tree (w/o DDI)	0.4738	0.7157	0.6306	0.0767	15.03
Med-tree (w/o ontology)	0.4691	0.7098	0.6220	0.0726	14.34
Med-tree	0.4743	0.7164	0.6309	0.0705	14.98

shows the performance comparison of the Jaccard Similarity Score, Average F1 score, Precision–Recall AUC, and DDI rate between Med-tree and baselines on MIMIC-III datasets. It can be seen from the experimental results that the proposed model achieves the best results among all indicators. In terms of Jaccard Similarity Score, Average F1 score, and Precision–Recall AUC, the proposed model is 1.02%, 1.23%, and 1.09% higher than that of the latest method (PREMIER), respectively. Moreover, in terms of the number of medication recommendation outcomes, the average number of drugs recommended by Med-tree is 14.98, which has the smallest gap from the true value of 14.68. The several indicators above show that the proposed model can effectively improve the accuracy of medication recommendations. Meanwhile, Med-tree takes into account drug safety. To be specific, LR and LEAP are machine learning-based methods, which are inferior to RETAIN and other deep learning methods in all indicators. Moreover, GAMENet and PREMIER are the latest methods that introduce DDI knowledge and reduce the DDI rate in the medication recommendation combination. However, the DDI rate of these methods is all lower than the proposed model, which verifies that the proposed model combined with the medical ontology tree and learning medical feature representations can greatly improve the medication recommendation result.

4.2.2. Evaluation of the DDIs

To verify the ability of the model to reduce the DDI rate in medication recommendation combinations, the experiments are classified with Top 40, Top 60, Top 80, and Top 100 to study the comparisons between Med-tree and the baselines, and

Table 5. The comparisons of the DDI rate between Med-tree and baselines are divided into Top 40, Top 60, Top 80, and Top 100 types.

Top K	DDI	Leap [43]	RETAIN [21]	GAMENet [22]	PREMIER [44]	Med-Tree
40	$R_{DDI}$	0.0633	0.0863	0.0780	0.0754	0.0705
	${\Delta }_{DDI}$	−26.81%	−0.21%	−9.81%	−12.82%	−18.48%
60	$R_{DDI}$	0.1923	0.2065	0.1985	0.1989	0.1869
	${\Delta }_{DDI}$	−3.15%	4.00%	−0.03%	0.17%	−5.87%
80	$R_{DDI}$	0.2931	0.2985	0.3000	0.3095	0.2873
	${\Delta }_{DDI}$	−0.08%	8.77%	2.28%	5.52%	−2.05%
100	$R_{DDI}$	0.3964	0.3979	0.4035	0.4201	0.3880
	${\Delta }_{DDI}$	1.90%	12.29%	3.73%	7.99%	−0.26%

lists the comparison results.

Leap is a rule-based approach where medication recommendation combinations are selected from the prescribed drugs of previous doctors, so the DDI rate recommended by Leap is lower than other baselines. However, the Leap model requires a large number of manual participations, which is not suitable for complex datasets. Moreover, the RETAIN model does not incorporate the knowledge of DDIs, which leads to a higher DDI rate for medication recommendation. The GAMENet model and the PREMIER model learn the knowledge of DDIs by establishing the EHR graph and the DDI graph so that their Top 40 and Top 60 indicators still have a low DDI rate. However, their DDI rates are inevitably increased when the number of indicators increases. Furthermore, the Med-tree model is superior to all previous deep learning methods. Even though the ${\Delta }_{DDI}$ increases from −18.48% to −0.26%, the Med-tree model can still keep the DDI rate below zero. These characteristics further indicate that the Med-tree can still take into account drug safety in some complex environments and situations. It also shows that the introduction of the medical ontology tree can enrich the representations of medical entities and improve the quality of downstream medication recommendation tasks.

4.2.3. Evaluation for Unbalanced Medications

Due to the particularity of EHRs, the occurrence frequency of drugs may be different, so it is difficult to recommend drugs with low frequency during medication recommendation. Moreover, the proposed model could mitigate the influence of different frequencies of medications by constructing the medical ontology tree and the memory bank. Figure 6a counts the number of medications in different frequency bands, and it can be seen that 58 of the 145 medication types appear less than 100 times, while nearly 40 types are recommended more than 1000 times. The average F1 scores of medication recommendation results are calculated in different frequency ranges, as shown in Figure 6b, which indicates that the proposed model significantly improves the recommendation of less frequent medications compared with other methods. In addition, several frequency bands have higher average F1 scores because they contain more dedicated medications for specific diseases, such as hypertension drugs, diabetes drugs, etc.

4.2.4. Evaluation for EHRs of Different Length

As can be seen from Table 3, the maximum number of medical visits of patients on MIMIC-III datasets is 29. Since each patient has a different number of admissions, the influence of temporal series length should be considered. Figure 7 shows the evaluations for EHRs with different temporal lengths on MIMIC-III datasets. To be specific, the Leap model is a rule-based model, and the length of the EHRs has no effect on its recommendation results, which results in low F1 indicators. Moreover, the GAMENet model uses the GRU to learn the temporal characteristics of a patient’s medical records, which greatly improves the F1 indicators compared with the LEAP model. However, its F1 indicators gradually decrease with the increase in the sequence length of the EHRs. The PREMIER model improves the GAMENet model by adding two attention mechanisms to prompt the learning of temporal information of medical records, which results in relatively high F1 indicators. Furthermore, Med-tree has the best performance among all methods, and its F1 value can still maintain a high level with the increase in medical record sequence length, which indicates that the proposed model can better learn the temporal dependency of a patient’s medical records.

4.3. Case Study

To specifically and intuitively observe the ability of the proposed model in medication recommendation, the performance of Med-tree is compared with other baselines on the case samples randomly selected from MIMIC-III datasets. A typical case is shown in

Table 6. Results of the last recommended medications for a patient with a total of five visits on MIMIC-III datasets. Here, “unseen” indicates the drugs are not appearing in the actual recommendation results, while “missed” refers to the drugs that should be recommended in the actual situation but are not recommended.

Methods	Recommended Medication Combination (the Last Visit)
Leap [43]	8 correct + 2 unseen + 7 missed (Antigout, Anxiolytics, Cardiac glycosides, …)
RETAIN [21]	10 correct + 4 unseen + 5 missed (Anxiolytics, Cardiac glycosides, Potassium, …)
GAMENet [22]	12 correct + 2 unseen + 3 missed (Antigout, Anxiolytics, Dopaminergic agents, …)
PREMIER [44]	13 correct + 4 unseen + 2 missed (Anxiolytics, Cardiac glycosides, …)
Med-tree	14 correct + 4 unseen + 1 missed (Anxiolytics)

, which indicates the recommendation result of a patient’s last visit. Moreover, there are 15 drugs actually recommended in the last visit, and the recommended medication combinations from Med-tree and baseline methods are listed in Table 6. It is seen that Med-tree has the best performance. To be specific, Med-tree correctly recommends 14 of the 15 drugs, only 1 medication is missed and 3 types are mispredicted. In contrast, the previous best method is PREMIER, which recommends one less correct drug than Med-tree. Furthermore, it can be found that none of the models successfully hit the medication “Anxiolytics”, which may due to the specific habits of certain doctors for this drug. In summary, compared with other recommended methods, Med-tree has more accurate recommendation effects on MIMIC-III datasets, and the comparison results of the case fully prove the excellent performance of the proposed model in the actual medication recommendation process.

5. Conclusion and Future Work

In this work, we propose a novel method for medication recommendation that can adequately enhance the learning of obscure structural and temporal features and reduce the DDIs in recommendation results. Moreover, in terms of structural correlation, we construct the medical ontology tree and apply the GAT model to learn the target features, which effectively learns the internal correlation between medical events. Meanwhile, the GRU model combined with the attention mechanisms is applied to capture the temporal characteristics, which improves the accuracy of recommendation. Finally, the memory bank and dynamic memory mechanism are introduced to reduce the DDI rate of the recommendation result. Experimental results show that the proposed model effectively captures the correlation and temporal features and further has a better performance than the existing methods. Furthermore, the case study also shows that the proposed model can make more accurate and reasonable prescriptions for patients in practical applications.

In fact, EHRs are inadequate for the task of personalized and accurate drug recommendation. Moreover, due to a lack of information, inaccurate information, contradictory information, contradictions, and other reasons, there are many uncertainties in EHRs for medication recommendation. However, we only consider the influences of diagnoses and procedures, and there is still great research potential for improvement in the feature mining of EHRs. In the future, we will consider incorporating the original textual information of the EHRs and focus on how to effectively model the fine-grained temporal evolution in EHRs. Meanwhile, medical artificial intelligence studies based on EHRs can further expand the application scope. In addition to medication recommendation, we will further apply EHRs to medical knowledge question answering, disease prediction, and other issues in the future.