MARIE: A Context-Aware Term Mapping with String Matching and Embedding Vectors

Featured Application

With its term mapping capability, MARIE can be used to improve data interoperability between different biomedical institutions. It can also be applied to text data pre-processing or normalization in non-biomedical domains.

Abstract

With growing interest in machine learning, text standardization is becoming an increasingly important aspect of data pre-processing within biomedical communities. As performances of machine learning algorithms are affected by both the amount and the quality of their training data, effective data standardization is needed to guarantee consistent data integrity. Furthermore, biomedical organizations, depending on their geographical locations or affiliations, rely on different sets of text standardization in practice. To facilitate easier machine learning-related collaborations between these organizations, an effective yet practical text data standardization method is needed. In this paper, we introduce MARIE (a context-aware term mapping method with string matching and embedding vectors), an unsupervised learning-based tool, to find standardized clinical terminologies for queries, such as a hospital’s own codes. By incorporating both string matching methods and term embedding vectors generated by BioBERT (bidirectional encoder representations from transformers for biomedical text mining), it utilizes both structural and contextual information to calculate similarity measures between source and target terms. Compared to previous term mapping methods, MARIE shows improved mapping accuracy. Furthermore, it can be easily expanded to incorporate any string matching or term embedding methods. Without requiring any additional model training, it is not only effective, but also a practical term mapping method for text data standardization and pre-processing.

1. Introduction

Due to the growing interest in text mining and natural language processing (NLP) in the biomedical field [1,2,3], data pre-processing is becoming an increasingly crucial issue for many biomedical practitioners. As datasets used in research and practice are often different, it is challenging to directly apply recent advancements in biomedical NLP to existing IT systems without effective data pre-processing. One of the key issues addressed during pre-processing is concept normalization, which refers to the task of aligning different text datasets or corpora into a common standard.

Previous works on concept normalization have relied on either extracting meaningful features from standardized ontologies [4,5,6] or training machine learning models [7,8]. Recently, deep learning models [9,10,11] have further improved normalization accuracy. However, these previous works are limited to a supervised learning setting, in which correctly aligned and standardized training datasets are required. If these manually aligned training datasets are not readily available, the normalization accuracy of these supervised approaches cannot be guaranteed. Without any additional annotated data, previously trained and supervised models cannot be fine-tuned for new datasets that exhibit completely different properties from their annotated counterparts. When the models fail to capture changes in the data distribution, their modeling capabilities cannot be transferred to the new datasets [12,13,14]. Therefore, these previous supervised learning models that were trained from standardized datasets cannot be easily applied in practice, in which mapping between non-standardized datasets is prevalent.

However, creating new annotated or standardized datasets is extremely challenging. Due to geographical, organizational or operational factors, biomedical organizations have their own unique sets of standards and conventions for processing their text data. For example, medical institutions in the United Kingdom and the United States use a systematized nomenclature of medicine, using clinical terms (SNOMED CT) [15] to standardize their clinical data. In South Korea, on the other hand, medical institutions rely on completely different sets of clinical terminologies issued by the Korean Health Insurance Review & Assessment Service, due to its fee-for-service model. Therefore, creating a highly standardized annotated dataset for every mapping need is simply too costly.

Despite its practical importance, unsupervised biomedical term mapping has not been actively explored. Previously, unsupervised learning-based mapping methods have relied on string matching-based approaches [16,17]. Without any model training, they utilize the structural information of the terms to define a similarity score, thereby providing greater applicability in mapping non-standardized or small datasets. However, a similarity measure between terms is often defined beyond their structural similarity. Using word embedding methods [18,19], we can also define a similarity measure between the terms based on their contextual similarity. As shown in the tasks of text classification [20,21], sentiment analysis [22,23] and machine translation [24], the contextual information captured by the embedding methods provides finer granularity to defining term similarities.

In this paper, we introduce MARIE (Python implementation of MARIE is available online at: https://github.com/hank110/MARIE), a context-aware mapping method with string matching and embedding vectors. It is a new, unsupervised concept normalization method that maps biomedical terms from different datasets, based on both their structural and contextual similarities. Its approach for capturing structural similarity is identical to previous string matching methods. However, the term similarity calculated from these sting matching methods is fine-tuned by corresponding term similarity measures, calculated from the word embedding vectors of BERT [25]. By incorporating both structural and contextual similarities, our proposed method improves upon the existing unsupervised term mapping methods, thereby providing an effective tool for aligning and standardizing biomedical text data in practice.

2. Methods

Since MARIE utilizes both string matching methods and BERT, we will briefly discuss these methods. Subsequently, we will provide details of our proposed method.

2.1. String Matching Methods

String matching methods define a similarity score between terms, based on the number of commonly shared letters or substrings. Among various string matching methods, edit distance [26,27], the Jaccard index [28,29] and Ratcliff/Obershelp similarity [30] are some of the most commonly used string matching methods. Given two terms, ${\mathrm{T}}_1$ and ${\mathrm{T}}_2$, edit distance calculates their distance based on the number of insertions, deletions and substitutions of characters required in transforming ${\mathrm{T}}_1$ into ${\mathrm{T}}_2$. While edit distance captures the structural information of terms on a character level, the Jaccard index utilizes the token-level, structural information. It computes the similarity between ${\mathrm{T}}_1$ and ${\mathrm{T}}_2$ as the length of the longest common substring in proportion to the overall length of ${\mathrm{T}}_1$ and ${\mathrm{T}}_2$. Ratcliff/Obershelp (R/O) similarity is regarded as an extension of the Jaccard index. Initially, it finds the longest common substring between ${\mathrm{T}}_1$ and ${\mathrm{T}}_2$. Subsequently, it finds the next longest common substring from the non-matching regions of ${\mathrm{T}}_1$ and ${\mathrm{T}}_2$. This process of finding the longest common substring from the non-matching regions is repeated until there is no common substring between ${\mathrm{T}}_1$ and ${\mathrm{T}}_2$. Finally, it uses the length of all common substrings in proportion to the overall length of ${\mathrm{T}}_1$ and ${\mathrm{T}}_2$ to calculate their similarity.

2.2. BERT

As a word embedding model, BERT learns to generate continuous numerical vector representations of words from text data. To generate these embedding vectors, it relies on a bi-directional transformer architecture [31]. Although Embeddings from Language Model (ELMo) [32] similarly utilizes bi-directional contextual information to train its word vectors, BERT is one of the first embedding methods that simultaneously captures bi-directional contextual information during its training. Furthermore, its multi-head attention mechanisms serve as a computationally efficient solution for retaining long-range contextual information, thereby improving the quality of the embedding vectors, compared with those generated from the recurrent neural network-based models [33]. Additionally, it uses WordPiece embeddings [34] to tokenize the input words during training. Subsequently, a word vector is generated by aggregating these tokenized embedding vectors, thereby addressing the issue of out-of-vocabulary words. Recently, BioBERT [35] has been introduced in the field of biomedical NLP. Despite its near-identical architecture to BERT, it has been trained from biomedical corpora such as PubMed abstracts and PubMed Central full-text articles. Due to its domain-specific training data, it has improved the performance of named entity recognition, relation extraction and question answering within the biomedical domain. Based on its trained word embedding vectors, the similarity between terms ${\mathrm{T}}_1$ and ${\mathrm{T}}_2$ is measured by calculating the cosine distance between their respective embedding vectors.

2.3. MARIE

As described in Equation (1), MARIE calculates a term similarity between terms ${\mathrm{T}}_1$ and ${\mathrm{T}}_2$ as a weighted average of string matching methods and the cosine similarity of their embedding vectors from BioBERT. As long as the range of the resulting similarity score is between 0 and 1, any type of string matching methods can be incorporated into MARIE. To generate word embedding vectors, MARIE extracts the activations from the last few layers of BioBERT. When a term is composed of multiple words or tokens, MARIE uses the average of the constituting word or token embedding vectors to generate a final term vector. As both similarity measures are ranged between 0 and 1, the final similarity score computed from MARIE also lies between 0 and 1. Furthermore, a hyperparameter α enables users to adjust the relative importance of the similarity scores calculated from each of the string matching methods and the BioBERT embedding vectors. Although α is a user-defined hyperparameter, it enables MARIE to flexibly adjust its mapping capability to different datasets. To summarize the entire mapping process of MARIE, Figure 1 provides its overall flow diagram. The MARIE calculation method is as follows:

$${\mathrm{S}}_{\mathrm{MET}}\left(T_1,T_2\right)=\left(1-\alpha \right)S_{str}\left(T_1,T_2\right)+\alpha S_{BB\left(E_1,E_2\right)}$$

(1)

$$E_1,E_2={\mathrm{Embedding}\mathrm{vectors}\mathrm{of}\mathrm{T}}_1{\mathrm{and}\mathrm{T}}_2,\mathrm{respectively}$$

$$S_{MET}\left(T_1,T_2\right)=\mathrm{Similarity}\mathrm{score}\mathrm{from}\mathrm{MARIE}$$

$$S_{str}\left(T_1,T_2\right)=\mathrm{Similarity}\mathrm{score}\mathrm{from}\mathrm{a}\mathrm{string}\mathrm{matching}\mathrm{method}$$

$$S_{BB}\left(E_1,E_2\right)={\mathrm{Cos}\mathrm{ine}\mathrm{similarity}\mathrm{between}\mathrm{E}}_1{\mathrm{and}\mathrm{E}}_2$$

The string matching methods and BioBERT use distinctively different criteria when calculating a similarity score between terms ${\mathrm{T}}_1$ and ${\mathrm{T}}_2$. The string matching methods purely rely on the structural similarities between the terms. On the other hand, BioBERT uses the distance between the embedding vectors ${\mathrm{E}}_1$ and ${\mathrm{E}}_2$. As these embedding vectors are trained to capture contextual dependencies between the terms, the distance between ${\mathrm{E}}_1$ and ${\mathrm{E}}_2$ is determined by the contextual similarities between ${\mathrm{T}}_1$ and ${\mathrm{T}}_2$. By combining these two approaches, MARIE utilizes both structural and contextual dependencies, thereby employing a richer set of criteria to define a similarity between ${\mathrm{T}}_1$ and ${\mathrm{T}}_2$.

Previously, word embedding-based edit distance [36] had similarly attempted to incorporate contextual information in edit distance. However, MARIE is a generalized improvement upon this previous work. Besides edit distance, MARIE is capable of incorporating a wider selection of string matching methods, thereby further expanding its applicability in practice. Furthermore, the biggest improvement of MARIE is its choice of term embedding methods. Among various term embedding methods, the previous work relies on word2vec [37] to generate term embedding vectors. However, word2vec does not capture richer bidirectional contextual information between words. Furthermore, it does not employ WordPiece embedding, thereby suffering from an out-of-vocabulary issue. When the embedding vector of a term ${\mathrm{T}}_1$ has not been trained, word2vec is not capable of inferring its embedding vector. For this type of out-of-vocabulary term, word embedding-based edit distance simply ignores the contextual information and reverts back to a string matching method. However, BioBERT relies on WordPiece embedding to overcome this weakness of word2vec. During training, BioBERT learns the embedding vectors of both input words and their subwords. Thus, when it needs to infer the embedding vector of ${\mathrm{T}}_1$, it breaks ${\mathrm{T}}_1$ into known subwords and estimates its embedding vector by adding up its subword embedding vectors. By using BioBERT instead of other term embedding methods such as word2vec or GloVe [38], MARIE utilizes richer contextual dependencies between terms and is robust to new words. Therefore, MARIE is capable of leveraging contextual information, even when the resources for training term embedding vectors are limited.

3. Results

3.1. Dataset and Experiment Setups

To validate the performance of MARIE, we tested it by mapping 3489 medical terms currently used in Seoul National University Hospital (SNUH corpus) with the Observational Medical Outcomes Partnership Common Data Model (OMOP CDM). Among various code standards available in the CDM, we limited our mapping to SNOMED CT, Logical Observation Identifiers Names and Codes (LOINC), RxNorm and RxNorm Extension [39,40]. With Usagi software and manual reviews from six medical professionals, we created a true mapping between SNUH corpus and the CDM. Given the terms from SNUH corpus as inputs, the performance of MARIE was evaluated on how accurately it could reconstruct this true mapping between SNUH corpus and the CDM. For example, given a term ${\mathrm{T}}_1$ from SNUH corpus, we calculated its similarity scores with all of the terms in a target dataset and selected the term with the highest similarity score ${\mathrm{T}}_1^{’}$ as its predicted mapped term. The mapping between ${\mathrm{T}}_1$ and ${\mathrm{T}}_1^{’}$ was considered correct if it corresponded to the true mapping between SNUH corpus and the CDM.

For robust evaluation, we tested MARIE on three different artificial target datasets. Each of these target datasets contained 3489 correctly mapped terms, as expressed in the CDM. In addition to these correctly mapped terms, each of these target datasets also contained 5000, 10,000 and 50,000 randomly sampled terms from the CDM, respectively. These random samples included only the terms that were excluded or ignored during the true mapping between SNUH corpus and the CDM. The sampled target datasets increased the number of possible mapping candidates that MARIE needed to explore, thereby introducing additional complexity to accurately mapping the terms in SNUH corpus.

As for BioBERT, we used the BioBERT-Large v1.1 model from its official Github repository. Without additional model training, we extracted the term embedding vectors from its pre-trained weights.

3.2. Experiment Results

Table 1. Experiment results in biomedical mapping terminologies between Seoul National University Hospital (SNUH corpus) and three artificial target datasets. Compared with other existing mapping methods, MARIE showed superior mapping accuracy, which was further improved by changing the value of α.

	Method	Jaccard Index	Edit Distance	R/O	MARIE(BioBERT + Edit Distance)			MARIE(BioBERT + R/O)			Embedding Vector Similarity
Metric					$\mathit{\alpha }=\mathbf{0.2}$	$\mathit{\alpha }=\mathbf{0.5}$	$\mathit{\alpha }=\mathbf{0.8}$	$\mathit{\alpha }=\mathbf{0.2}$	$\mathit{\alpha }=\mathbf{0.5}$	$\mathit{\alpha }=\mathbf{0.8}$
Top 1 accuracy		33.68%	44.71%	49.44%	46.49%	48.29%	51.16%	51.25%	53.48%	54.66%	35.83%
Top 3 accuracy		40.64%	55.49%	60.05%	58.56%	62.77%	67.58%	63.37%	66.12%	69.30%	55.69%
Top 5 accuracy		42.65%	59.50%	63.92%	63.26%	67.73%	72.43%	66.72%	70.71%	74.63%	61.45%
Top 10 accuracy		46.09%	64.35%	69.07%	68.64%	73.40%	78.48%	71.80%	75.67%	79.02%	67.04%
(a) Target dataset with 5000 random samples
Top 1 accuracy		33.02%	41.93%	47.38%	43.57%	45.69%	48.15%	49.33%	50.47%	51.22%	33.51%
Top 3 accuracy		40.04%	53.71%	58.24%	56.75%	60.33%	64.66%	61.48%	64.12%	67.27%	52.97%
Top 5 accuracy		41.90%	58.13%	62.85%	61.22%	65.84%	70.39%	65.63%	69.02%	72.14%	58.30%
Top 10 accuracy		45.34%	63.54%	67.96%	67.10%	71.74%	76.33%	70.34%	73.77%	77.27%	64.26%
(b) Target dataset with 10,000 random samples
Top 1 accuracy		28.58%	34.80%	40.76%	35.31%	37.63%	38.32%	41.82%	43.31%	41.99%	26.37%
Top 3 accuracy		37.17%	47.49%	52.59%	49.21%	52.02%	55.69%	54.54%	56.78%	58.01%	43.85%
Top 5 accuracy		39.18%	51.76%	57.12%	53.48%	57.38%	61.31%	59.16%	62.28%	64.37%	49.47%
Top 10 accuracy		42.19%	57.55%	63.00%	60.22%	64.40%	68.24%	65.12%	68.16%	70.31%	56.32%
(c) Target dataset with 50,000 random samples

shows the top 1, 3, 5 and 10 mapping accuracies of MARIE in each of three target datasets. To provide objective comparisons, we also reported the mapping accuracy of the string matching methods and an embedding vector-based mapping method. For string matching methods, we used the Jaccard index, edit distance and R/O similarity. For the embedding vector-based mapping method, we calculated the similarity score between terms ${\mathrm{T}}_1$ and ${\mathrm{T}}_2$, based on the cosine distance of their respective embedding vectors ${\mathrm{E}}_1$ and ${\mathrm{E}}_2$. This embedding vector-based mapping is a variant of the concept normalization method investigated by Karadeniz et al. [41]. The only difference is that the embedding vectors were generated from BioBERT instead of word2vec.

The interplay of structural and contextual information within terms enabled MARIE to outperform other mapping methods across all target datasets. Considering the process involved in mapping terms, this outcome is not surprising. Let us assume that a correct mapping between datasets A and B resulted in a term ${\mathrm{T}}_1\in \mathrm{A}$ to be mapped to a term ${\mathrm{T}}_2\in \mathrm{B}$. Unless ${\mathrm{T}}_1$ and ${\mathrm{T}}_2$ are identical strings, two types of discrepancies occur between these two terms. Given ${\mathrm{T}}_1$, either the order of words in ${\mathrm{T}}_2$ will be different, or ${\mathrm{T}}_2$ will be missing some of the words or subwords in ${\mathrm{T}}_1$. The term embedding vectors and string matching methods are effective in addressing each of these discrepancies, respectively. As an embedding vector of a term is defined by the average of its word vectors, the similarity score computed from the embedding vectors of BioBERT is invariant to the changes in the word orders within a term. Therefore, the contextual information of an entire term is preserved, regardless of the changes in its word order. On the other hand, similarity scores from string matching methods depend on the length of common characters or substrings. Consequently, as long as the lengths of missing words are short, compared with the overall lengths of the terms, string matching methods are effective in overlooking these missing words. As MARIE simultaneously resolves these two discrepancies, it achieves effective mapping performance. This characteristic is also evident in

Table 2. Comparison of the top three mapped outcomes from MARIE, the R/O string matching method and embedding vector-based mapping. A bold and underlined term represents a correctly mapped term. (a) MARIE and the embedding vector-based method found the correct mapping for the input word to a target term in RxNorm, even though there was a word order discrepancy. (b) MARIE and the R/O string matching method found the correct mapping for the input word to a target term in LOINC, even when there were missing words between the mapping. (c) MARIE was even capable of finding a mapping missed by the other two methods, thereby correctly mapping to a term in LOINC.

Mapping Methods	Rank 1	Rank 2	Rank 3
MARIE	cobicistat 150 mg/elvitegravir 150 mg/emtricitabine 200 mg/tenofovir alafenamide 10 mg oral tablet [RxNorm 1721612) (0.8538)	tenofovir alafenamide 25 mg oral tablet (RxNorm 1858261) (0.8536)	emtricitabine 200 mg/tenofovir disoproxil fumarate 300 mg oral tablet (RxNorm 476445) (0.8433)
R/O	atazanavir 300 mg/cobicistat 150 mg oral tablet (RxNorm 1601654) (0.4371)	amylases 10 mg/betaine 300 mg/bromelains 10 mg/papain 100 mg oral tablet (RxNorm Extension OMOP1092499) (0.4333)	tenofovir alafenamide 25 mg oral tablet (RxNorm 1858261) (0.4255)
BioBERT Embedding Vectors	cobicistat 150 mg/elvitegravir 150 mg/emtricitabine 200 mg/tenofovir alafenamide 10 mg oral tablet (RxNorm 1721612) (0.9895)	emtricitabine 200 mg/tenofovir disoproxil fumarate 300 mg oral tablet (RxNorm 476445) (0.9616)	tenofovir alafenamide 25 mg oral tablet (RxNorm 1858261) (0.9606)
(a) tenofovir alafenamide/emtricitabine/elvitegravir/cobicistat 10 mg/200 mg/150 mg/150 mg tab (local concept)
Mapping Methods	Rank 1	Rank 2	Rank 3
MARIE	cortisone (mass/time) in 24 h urine (LOINC Lab Test 14044-2) (0.8700)	color of urine (LOINC Lab Test 5778-6) (0.8599)	glucose (mass/time) in 24 h urine (LOINC Lab Test 2351-5) (0.8475)
R/O	color of urine (LOINC Lab Test 5778-6) (0.5882)	cortisone (mass/time) in 24 h urine (LOINC Lab Test 14044-2] (0.5862)	creatinine (mass/time] in 24 h urine (LOINC Lab Test 2162-6] (0.5574)
BioBERT Embedding Vectors	somatotropin^15th specimen post xxx challenge (LOINC component) (0.9509)	insulin ab (titer) in serum (LOINC Lab Test 11087-4) (0.9484)	glucose (presence) in urine by test strip (LOINC 25428-4) (0.9481)
(b) cortisol (24 h urine) (local concept)
Mapping Methods	Rank 1	Rank 2	Rank 3
MARIE	does manage ileostomy (SNOMED CT Clinical Finding 1073731000000109) (0.8203)	bone marrow pathology biopsy report narrative (LOINC 66119-9) (0.8143)	dome osteotomy (SNOMED CT Procedure 447761008) (0.8133)
R/O	pilopos (RxNorm Extension OMOP2011960) (0.5)	minor blood groups (SNOMED CT Procedure 143157006) (0.4440)	dome osteotomy (SNOMED CT Procedure 447761008) (0.4348)
BioBERT Embedding Vectors	her3 ag|tissue and smears (LOINC Hierarchy LP132424-5) (0.9449)	procedure on head (SNOMED CT Procedure 118690002) (0.9435)	cells.estrogen receptor|tissue and smears (LOINC Hierarchy LP262344-7) (0.9397)
(c) bm biopsy (local concept)

, which compares the actual mapping outcomes between SNUH local concepts and Observational Health Data Sciences and Informatics (OHDSI) Athena (https://athena.ohdsi.org/) from MARIE, R/O similarity and the embedding vector-based mapping.

4. Discussion

4.1. Impact of α

Hyperparameter $\mathsf{\alpha }$ balances the effect of structural and contextual information on calculating the overall similarity score with MARIE, thereby altering the mapping accuracy, as shown in Table 1. Therefore, we additionally analyzed the changes in the accuracy, with respect to various $\mathsf{\alpha }$, in the target dataset with 50,000 random samples. Within the range between 0.2 and 0.9, we changed the value of $\mathsf{\alpha }$ in increments of 0.1.

As shown in Figure 2, there is an optimal cutoff value of $\mathsf{\alpha }$ that maximizes the mapping accuracy, thereby suggesting that solely relying on either structural or contextual information is not sufficient for accurate mapping. However, the optimal value of $\mathsf{\alpha }$ will vary, depending on the input and target datasets, and should be determined based on the type of discrepancy that is prevalent in them. If the word order discrepancy is a major difference between the input and target datasets, an $\mathsf{\alpha }$ value greater than 0.5 should be used for better mapping accuracy. On the other hand, if the words missing from the corresponding terms are prevalent, an $\mathsf{\alpha }$ value less than 0.5 is recommended.

4.2. Impact of BioBERT Layers

Depending on the number of BioBERT layers used to represent a word embedding vector, the amount of contextual information captured by the vector can vary. When we utilized multiple layers, the final embedding vector was generated by taking an average of the embedding vectors from each layer.

Table 3. Changes in the mapping accuracy of MARIE (BioBERT + R/O with α = 0.8) with respect to the number of BioBERT layers used to represent an embedding vector.

Number of Layers	Top 1 Accuracy	Top 3 Accuracy	Top 5 Accuracy	Top 10 Accuracy
1	54.66%	69.30%	74.63%	79.02%
2	54.71%	69.27%	74.23%	79.34%
3	55.09%	69.39%	74.29%	79.48%
4	55.35%	69.99%	74.75%	79.79%
(a) Target dataset with 5000 random samples
1	51.22%	67.27%	72.14%	77.27%
2	51.19%	67.13%	72.03%	77.30%
3	51.68%	67.33%	72.17%	77.59%
4	52.05%	67.64%	72.48%	78.07%
(b) Target dataset with 10,000 random samples
1	41.99%	58.01%	64.37%	70.31%
2	42.08%	58.01%	64.12%	70.31%
3	42.33%	58.41%	64.17%	70.48%
4	43.31%	58.87%	64.55%	71.02%
(c) Target dataset with 50,000 random samples

summarizes the changes in the mapping accuracy of MARIE when we used the last 2~4 layers of BioBERT to generate a word embedding vector. Although the improvement was not significant, using more layers increased the overall mapping accuracy.

4.3. Limitations of MARIE

Despite its superior mapping capability, MARIE does suffer from a few limitations. One of its limitations is its inability to handle biomedical terms that contain both English and foreign languages, such as Korean. In order to map these terms possessing multiple languages, we will need a BioBERT model that has been simultaneously trained for these different languages. However, such a heterogeneous model or training corpus for biomedical applications does not yet exist, to the best of our knowledge.

Furthermore, the computation speed of MARIE is bounded by the computation speed of its string matching methods. As MARIE extracts pre-trained BioBERT vectors, the majority of its computation time is spent on applying string matching methods. However, this computation time varies, based on the choice of string matching methods and their implementations.

This computation time issue becomes especially critical when mapping between large datasets. Therefore, we highly recommend using string matching methods with low computational complexities if fast computation time is critical.

5. Conclusions

In this paper, we introduced a new biomedical term mapping method that incorporates both structural and contextual information to calculate a term similarity score. As pre-processing and standardizing datasets are crucial in applying the latest text mining and machine learning models in practice, MARIE will serve as an essential tool for many biomedical professionals. As a generalized mapping method, it does not require any additional model training and can incorporate other string matching and embedding methods not discussed in this paper.

Furthermore, MARIE will make significant contributions to the process of mapping local clinical concept codes to standardized codes. Specifically, this mapping generated from MARIE will be useful for deploying the OMOP CDM outside of the United States or the United Kingdom. Based on the data standardization achieved by MARIE, applications related to health information exchange and personal health records will also benefit from improved interoperability.

For the direction of future research, it will be interesting to incorporate recent advances in machine translation or transfer learning to MARIE, thereby enabling it to map biomedical terms across datasets recorded in different languages. Furthermore, we will also explore ways to apply MARIE to create new standardized biomedical text datasets. By aligning and aggregating biomedical text datasets from numerous organizations, we will be able to create a large, yet highly standardized, training dataset for machine learning applications.