Entity Alignment Method Based on Joint Learning of Entity and Attribute Representations

Featured Application

In practical application, there are different knowledge graphs in different fields, such as financial graph, commodity graph, medical graph, and so on. Entity alignment technique can be applied to the fusion of multiple knowledge graphs in different domains or even across domains.

Abstract

Entity alignment helps discover and link entities from different knowledge graphs (KGs) that refer to the same real-world entity, making it a critical technique for KG fusion. Most entity alignment methods are based on knowledge representation learning, which uses a mapping function to project entities from different KGs into a unified vector space and align them based on calculated similarities. However, this process requires sufficient pre-aligned entity pairs. To address this problem, this study proposes an entity alignment method based on joint learning of entity and attribute representations. Structural embeddings are learned using the triples modeling method based on TransE and PTransE and extracted from the embedding vector space utilizing semantic information from direct and multi-step relation paths. Simultaneously, attribute character embeddings are learned using the N-gram-based compositional function to encode a character sequence for the attribute values, followed by TransE to model attribute triples in the embedding vector space to obtain attribute character embedding vectors. By learning the structural and attribute character embeddings simultaneously, the structural embeddings of entities from different KGs can be transferred into a unified vector space. Lastly, the similarities in the structural embedding of different entities were calculated to perform entity alignment. The experimental results showed that the proposed method performed well on the DBP15K and DWK100K datasets, and it outperformed currently available entity alignment methods by 16.8, 27.5, and 24.0% in precision, recall, and F1 measure, respectively.

1. Introduction

Knowledge graphs (KGs) are structured representations of different real-world entities and their relationships [1,2]. In essence, KG converts unstructured knowledge into a simple and well-defined collection of triples comprising a head entity, relation, and tail entity. KGs are widely used in many areas, such as recommendation systems [3], text classification [4], question-answering systems [5], semantic search [4], and knowledge-based reasoning [5].

Rapid developments of KG techniques have led to the emergence of numerous large-scale KGs; this results in problems such as knowledge overlap between different KGs and ambiguous relationships, which are obstacles to KG integration on a semantic level. Multilingual KGs such as DBpedia [6], YAGO [7], and Freebase [8] contain large quantities of knowledge; however, their heterogeneous data sources make it difficult to fuse KGs into a comprehensive KG. Knowledge fusion aligns and merges heterogeneous and redundant knowledge in KGs, while entity alignment is used to discover entities in different KGs that refer to the same real-world entity. As each KG is developed independently, different KGs will have different knowledge sources, resulting in different surface forms, posing a significant problem when fusing different KGs. Therefore, knowledge source fusion, integration of existing knowledge resources, and large-scale knowledge mapping are crucial endeavors.

Entity alignment can be used to link knowledge from different KGs and fuse KGs into larger, authoritative domain KGs, which can serve as a knowledge base for downstream applications. However, entity alignment requires complex calculations, and the conventional approach involves using entity-based labeling and handcrafted features, both of which are laborious and difficult to scale in terms of practical applications. Translation-based representation learning is a method for learning vector representations for entities in a dense low-dimensional space. TransE [9], a classic example of this approach, is the first method to utilize the structural embeddings of entities in a KG. Extensions of TransE, such as TransR [10], TransH [11], and TransD [12], are used to learn entity embedding representations and perform entity alignment or entity inference in the unified vector space, where the likelihood of alignment between any pair of entities is proportional to their semantic similarity, i.e., closeness in the embedding space. The most recent advances in KG entity alignment are embedding-based techniques, such as MTransE [13]—a translation-based model for multilingual KG embeddings in which existing multilingual KGs are used for cross-lingual entity alignment. Herein, the entities and relations of each language are encoded in separated embedding spaces, whereas an alignment model embeds the multilingual vectors into an independent space where they are aligned by translation and linear transformations. MTRansE aims to facilitate the construction of a coherent multilingual knowledge base and assist machines in dealing with expressions of entity relationships across different human languages. In IPTransE [14], entity alignment is performed using an iterative- and parameter-sharing method for joint knowledge embeddings. As this method requires manual labeling to produce a seed set of pre-aligned entities, whose structure is projected to a low-dimensional vector space, it is highly dependent on the accuracy of the manually labeled seed set. If the labeling is incorrect, the iteration of the seed entity set will be extremely noisy, thereby decreasing the alignment accuracy. The JAPE model [15] jointly embeds the relationship structure of two KGs into a unified vector space and uses attribute correlations to cluster entities. Unlike other models, JAPE embeds attributes while learning entity embeddings and relationships between different KGs in a unified embedding space and uses attribute correlations to optimize entity embeddings. In other words, it is a joint attribute-preserving embedding model for cross-lingual entity alignment. However, if the attributes are heterogeneous and the KGs are ambiguously correlated, the effectiveness of attribute embedding can be significantly reduced. BootEA [16] adds bootstrapping to JAPE to reduce error accumulation in iteratively grown seed sets. Furthermore, it treats entity alignment as a classification problem and maximizes alignment likelihood over all labeled and unlabeled entities based on KG embeddings while using ε-truncated uniform negative sampling to improve the alignment performance. Additionally, labeled entities are edited or removed to reduce error accumulation during iterations. MultiKE [17] divides KG characteristics into multiple subsets, or views and embeds each entity into specific views for complementary learning from multiple views, enhancing alignment accuracy. KDCoE [18] uses the co-training approach to iteratively train two embedding models: one model is trained on the KG structure; the other is trained on entity descriptions for refining entity embeddings.

All embedding-based KG entity alignment methods utilize the TransE family of models to learn the structural embeddings of entities and relations. However, these models can only embed the entity and relations of a single KG into the unified vector space. As the entities in each KG are named using different naming schemes, their structural embeddings fall in different vector spaces, making it impossible to directly calculate the entity similarity, highlighting one of the limitations of entity alignment. The current solution to this problem is to design a mapping function that projects entity embeddings from different KGs into a unified vector space. This mapping function can only be designed if some number of pre-aligned entities are already present; however, these pre-aligned entities are collected manually, which can be problematic. Furthermore, it is difficult to determine whether the entity pairs are aligned accurately, considering that any inaccuracy can profoundly affect the entity alignment process.

To address the above issues, this study proposes to learn structural embeddings in parallel with attribute character embeddings. First, we calculate the similarities in the relations and attributes of the semantic embeddings. Relationships and attributes with the same meaning are then renamed with the same names, with manual inspection and amendment, followed by the joint learning of structural embeddings and attribute character embeddings from the relation and attribute triples. Finally, the structural embeddings of the entities are transferred to a unified vector space, and entity alignment is performed by calculating the similarities of the structural embeddings.

The contributions of this study are summarized as follows:

(1)

We proposed a relation and attribute alignment method based on cosine similarity. The cosine similarities of the semantic embeddings of relations and attributes are calculated, followed by the manual inspection and amendment of the results. Relation–attribute alignment is then performed by renaming relations and attributes with the same meaning to the same name.

(2)

A learning model is proposed to learn the structural and attribute character embeddings from different KGs using their relation and attribute triples. Structural embeddings are learned via a triples modeling method using TransE and PTransE, which use the semantic information in direct and multi-step relation paths to extract structural embedding vectors from the embedding vector space. Attribute character embeddings are learned using the N-gram-based compositional function to encode a character sequence for the attribute values. Then, by using TransE to model attribute triples in the embedding vector space, we obtain the attribute character embedding vectors. Finally, the structural embeddings and attribute character embeddings are jointly learned to transfer the structural embedding vectors of entities from different KGs into a unified vector space.

(3)

We introduce a limit-based loss function that assigns absolutely low and high scores to positive and negative triples, respectively, to improve the loss function for embedding learning and prevent drift while mapping structural embeddings into the unified vector space.

2. Materials and Methods

2.1. Problem Formulation

In a knowledge graph, knowledge is represented as triples, which are either relation or attribute triples [19,20,21]. Relation and attribute triples are expressed as:

$$T_R=\left\{\left(h,r,t\right)|h,t\in E,r\in R\right\}$$

(1)

$$T_A=\left\{\left(e,a,v\right)|e\in E,a\in A,v\in V\right\}$$

(2)

where E is the set of entities, R is the set of relations, A is the set of attributes, V is the set of attribute values, $T_R\subseteq E\times R\times E$ is the set of relation triples, and $T_A\subseteq E\times A\times V$ is the set of attribute triples. Therefore, a KG can be expressed as $KG=\left(E,R,A,V,T_R,T_A\right)$.

Based on this expression for KGs, entity alignment can be described as follows: given a pair of KGs—KG₁ and KG₂—the aim of entity alignment is to search for an entity pair $\langle e_1,e_2\rangle$, where e₁ and e₂ refer to the same real-world entity, with $e_1\in K{G}_1$ and $e_2\in K{G}_2$.

2.2. Relation and Attribute Alignment

If pre-aligned entity pairs are scarce, using pre-aligned entity pairs in the relation alignment module for transferring embeddings from different KGs into the vector space is not recommended. Instead, it is recommended to directly extract the relation and attribute triples of these KGs for relation and attribute alignment. In this study, this extraction was performed by calculating the similarities of the semantic embeddings of the relations and attributes, followed by manually inspecting and amending the results. Finally, the relations and attributes with the same meaning were renamed to the same name, thereby allowing two different KGs to be embedded in the same vector space and merging the KGs.

Here, similarity refers to cosine similarity [22], given as:

$$\cos \left(r_1,r_2\right)=\frac{r_1\cdot r_2}{\Vert r_1\Vert \Vert r_2\Vert }$$

(3)

$$\cos \left(a_1,a_2\right)=\frac{a_1\cdot a_2}{\Vert a_1\Vert \Vert a_2\Vert }$$

(4)

2.3. Embedding Learning

Embedding learning includes the learning of structural embeddings and attribute character embeddings. Structural embeddings are learned from relation triples, $T_R=T_R^{K{G}_1}\cup T_R^{K{G}_2}$, whereas attribute character embeddings are learned from attribute triples, $T_A=T_A^{K{G}_1}\cup T_A^{K{G}_2}$.

2.3.1. Structural Embedding Learning

Structural embedding learning is performed using the TransE model, i.e., by minimizing the loss function L_SE (Equation (5)):

$$L_{SE-direct}={\displaystyle \sum _{\left(h,r,t\right)\in T_R}{\displaystyle \sum _{\left(h^{\prime },r^{\prime },t^{\prime }\right)\in {T^{\prime }}_R}{\left[\gamma +{\Vert \mathit{h}+\mathit{r}-\mathit{t}\Vert }_{L_2}-{\Vert {\mathit{h}}^{\prime }+{\mathit{r}}^{\prime }-{\mathit{t}}^{\prime }\Vert }_{L_2}\right]}_{+}}}$$

(5)

where T_R is the set of valid relation triples; T^′_R is the set of corrupted relation triples, where ${T^{\prime }}_R=\left\{\left(h^{\prime },r,t\right)|h^{\prime }\in E\right\}\cup \left\{\left(h,r,t^{\prime }\right)|t^{\prime }\in E\right\}\cup \left\{\left(h,r^{\prime },t\right)|r^{\prime }\in R\right\}$; and γ is the margin hyperparameter.

The TransE model exhibits fewer parameters, low complexity and simplicity, and good efficiency in the construction of large KGs. However, its simplicity limits its applications to complex relation modeling, multisource information fusion, and relation path modeling. To address the inadequacies of TransE in handling complex KG relations, we embedded entities and relations into two separate spaces: entity space ${ℝ}^m$ and relation space ${ℝ}^n$ (m and n are the dimensionalities of these spaces, and m and n can be equal to each other). Each triple has an entity vector $\mathit{h},\mathit{t}\in {ℝ}^m$ and relation vector $\mathit{r}\in {ℝ}^n$. First, the head and tail entities of the entity space ${ℝ}^m$ are mapped to the relation space ${ℝ}^n$ according to the mapping matrix ${\mathit{M}}_r\in {ℝ}^{m\times n}$, which gives the head and tail entities of the relation space, h_r and t_r, expressed as:

$${\mathit{h}}_r=\mathit{h}{\mathit{M}}_r,{\mathit{t}}_r=\mathit{t}{\mathit{M}}_r$$

(6)

With this, the loss function in Equation (5) can be replaced by the loss function in Equation (7) as:

$$L_{SE-direct}={\displaystyle \sum _{\left(h,r,t\right)\in T_R}{\displaystyle \sum _{\left(h^{\prime },r^{\prime },t^{\prime }\right)\in {T^{\prime }}_R}{\left[\gamma +{\Vert \mathit{h}{\mathit{M}}_r+\mathit{r}-\mathit{t}{\mathit{M}}_r\Vert }_{L_2}-{\Vert {\mathit{h}}^{\prime }{\mathit{M}}_r+{\mathit{r}}^{\prime }-{\mathit{t}}^{\prime }{\mathit{M}}_r\Vert }_{L_2}\right]}_{+}}}$$

(7)

When modeling triples, the aforementioned embedding learning method only considers direct entity relations. However, entities in large KGs often have multi-step relation paths that cannot be modeled by TransE. Therefore, the semantic information contained by these relation paths will be excluded, inevitably reducing the alignment performance. To address this problem, we use the PTransE [23] model to improve our method for embedding learning.

Equation (7) can be used to learn the direct relation r between the entity pair (h, t) of some triple (h, r, t). However, in the case of (h, P, t), where P represents the set of multi-path relation between a pair of entities ($P\left(h,t\right)=\left\{p_1,p_2,\cdots ,p_N\right\}$, with $p=r_1\circ r_2\circ r_3\cdots \circ r_l$ being the path between the pair of entities), the loss function shown in Equation (8) can be used to learn the triple of the entity pair as:

$$L_{SE-multistep}={\displaystyle \sum _{\left(h,r,t\right)\in T_R}\left\{\frac{1}{Z}{\displaystyle \sum _{p\in P\left(h,t\right)}R\left(p|h,t\right){\displaystyle \sum _{\left(h,r^{\prime },t\right)\in {T^{\prime }}_R}{\left[\gamma +{\Vert \mathit{p}-\mathit{r}\Vert }_{L_2}-{\Vert \mathit{p}-{\mathit{r}}^{\prime }\Vert }_{L_2}\right]}_{+}}}\right\}}$$

(8)

where $R\left(p|h,t\right)$ is the plausibility of relation path p, $Z={\displaystyle \sum _{p\in P\left(h,t\right)}R\left(p|h,t\right)}$ is the normalization factor, and p is the embedding vector representation of the relation path, defined as $p=r_1+r_2+\cdots +r_l$ in this study.

Combining Equations (7) and (8), the loss function for structural embedding learning can be expressed as:

$$L_{SE}=L_{SE-direct}+L_{SE-multistep}$$

(9)

2.3.2. Attribute Character Embedding Learning

For learning attribute character embeddings, we first define a character sequence for attribute values, $v=\left\{v_1,v_2,\cdots ,v_t\right\}$. The N-gram-based compositional function is then used to encode the character sequence of the attribute values.

$$f_a\left(\mathit{v}\right)={\displaystyle \sum _{n=1}^N\left(\frac{{\displaystyle {\sum }_{i=1}^t{\displaystyle {\sum }_{j=i}^n{\mathit{v}}_j}}}{t-i-1}\right)}$$

(10)

where ${\mathit{v}}_1,{\mathit{v}}_2,\cdots ,{\mathit{v}}_t$ are the character embeddings of the attribute values.

Based on the TransE model, the attribute a in (e, a, v) is the entity-to-attribute value mapping. In other words, adding attribute vector a to entity vector e yields the attribute-value vector f_a (v). The learning of attribute character embeddings is performed by minimizing the loss function L_CE, given as:

$$L_{CE}={\displaystyle \sum _{\left(e,a,v\right)\in T_A}{\displaystyle \sum _{\left(e^{\prime },a^{\prime },v^{\prime }\right)\in {T^{\prime }}_A}{\left[\gamma +{\Vert \mathit{e}+\mathit{a}-f_a\left(\mathit{v}\right)\Vert }_{L_2}-{\Vert {\mathit{e}}^{\prime }+{\mathit{a}}^{\prime }-f_a\left({\mathit{v}}^{\prime }\right)\Vert }_{L_2}\right]}_{+}}}$$

(11)

where T_A is the set of valid attribute triples and T′_A is the set of corrupted attribute triples, where ${T^{\prime }}_A=\left\{\left(e^{\prime },a,v\right)|e^{\prime }\in E\right\}\cup \left\{\left(e,a,v^{\prime }\right)|v^{\prime }\in V\right\}\cup \left\{\left(e,a^{\prime },v\right)|a^{\prime }\in A\right\}$.

2.3.3. Improved Loss Function for Embedded Learning

The loss functions shown in Equations (5), (7), (8) and (11) use the margin-based ranking criterion to learn embeddings. The goal of the learning process is to score positive triples lower than the negative triples. However, the aforementioned loss functions make it impossible for positive triples to obtain absolutely low scores, resulting in drifts during the mapping of entity embeddings to the unified vector space. To address this problem, we propose the limit-based loss function. Modifying the loss function in Equation (5) to the limit-based criterion, we obtain:

$$L_{SE-direct}={\displaystyle \sum _{\left(h,r,t\right)\in T_R}{\left[{\Vert \mathit{h}+\mathit{r}-\mathit{t}\Vert }_{L_2}-{\gamma }_1\right]}_{+}}+\mu {\displaystyle \sum _{\left(h^{\prime },r^{\prime },t^{\prime }\right)\in {T^{\prime }}_R}{\left[{\gamma }_2-{\Vert {\mathit{h}}^{\prime }+{\mathit{r}}^{\prime }-{\mathit{t}}^{\prime }\Vert }_{L_2}\right]}_{+}}$$

(12)

where γ₁ and γ₂ are the margin hyperparameters that satisfy γ₁ > 0 and γ₂ > 0, respectively, and µ is the balancing factor. The loss function in Equation (12) ensures that positive triples are scored lower than negative triples and provides absolutely low and high scores to positive and negative triples, i.e., ${\Vert \mathit{h}+\mathit{r}-\mathit{t}\Vert }_{L_2}\le {\gamma }_1$ and ${\Vert {\mathit{h}}^{\prime }+{\mathit{r}}^{\prime }-{\mathit{t}}^{\prime }\Vert }_{L_2}\ge {\gamma }_2$, respectively.

The modified versions of the loss functions in Equations (7), (8) and (11) are shown in Equations (13), (14) and (16), respectively.

$$L_{SE-direct}={\displaystyle \sum _{\left(h,r,t\right)\in T_R}{\left[{\Vert \mathit{h}{\mathit{M}}_r+\mathit{r}-\mathit{t}{\mathit{M}}_r\Vert }_{L_2}-{\gamma }_1\right]}_{+}}+\mu {\displaystyle \sum _{\left(h^{\prime },r^{\prime },t^{\prime }\right)\in {T^{\prime }}_R}{\left[{\gamma }_2-{\Vert {\mathit{h}}^{\prime }{\mathit{M}}_r+{\mathit{r}}^{\prime }-{\mathit{t}}^{\prime }{\mathit{M}}_r\Vert }_{L_2}\right]}_{+}}$$

(13)

$$\begin{array}{l}{L}_{SE-multistep}={\displaystyle \sum _{\left(h,r,t\right)\in T_R}\left\{\frac{1}{Z}{\displaystyle \sum _{p\in P\left(h,t\right)}R\left(p|h,t\right)}{\left[{\Vert \mathit{p}-\mathit{r}\Vert }_{L_2}-{\gamma }_1\right]}_{+}\right\}}\hspace{1em}\hspace{1em}\hspace{1em}\\ \hfill +\mu {\displaystyle \sum _{\left(h,r^{\prime },t\right)\in {T^{\prime }}_R}\left\{\frac{1}{Z}{\displaystyle \sum _{p\in P\left(h,t\right)}R\left(p|h,t\right)}{\left[{\gamma }_2-{\Vert \mathit{p}-{\mathit{r}}^{\prime }\Vert }_{L_2}\right]}_{+}\right\}}\end{array}$$

(14)

$$L_{SE}=L_{SE-direct}+L_{SE-multistep}$$

(15)

$$L_{CE}={\displaystyle \sum _{\left(e,a,v\right)\in T_A}{\left[{\Vert \mathit{e}+\mathit{a}-f_a\left(\mathit{v}\right)\Vert }_{L_2}-{\gamma }_1\right]}_{+}}+\mu {\displaystyle \sum _{\left(e^{\prime },a^{\prime },v^{\prime }\right)\in {T^{\prime }}_A}{\left[{\gamma }_2-{\Vert {\mathit{e}}^{\prime }+{\mathit{a}}^{\prime }-f_a\left({\mathit{v}}^{\prime }\right)\Vert }_{L_2}\right]}_{+}}$$

(16)

The learning of structural embeddings and attribute character embeddings yields e_SE and e_CE, which are the structural embeddings and attribute character embeddings of the entities in KG₁ and KG₂, respectively. In general, e_SE tends to fall in different vector spaces, considering the entities in KG₁ and KG₂ are usually named using different naming schemes. In contrast, the vector representations of the attribute values in KG₁ and KG₂ always fall into the same vector space, considering the character embeddings are learned from attribute value strings. Consequently, e_CE falls into a unified vector space. Therefore, the learned attribute character embeddings can be used to transfer the structural embeddings of entities into a unified vector space. The joint learning of structural embeddings and attribute character embeddings was performed by minimizing L_SIM (Equation (17)).

$$L_{SIM}={\displaystyle \sum _{e\in K{G}_1\cup K{G}_2}\left[1-\frac{{\mathit{e}}_{SE}\cdot {\mathit{e}}_{CE}}{\Vert {\mathit{e}}_{SE}\Vert \Vert {\mathit{e}}_{CE}\Vert }\right]}$$

(17)

Finally, Equations (15)–(17) are combined to obtain the overall training loss function L (Equation (18)).

$$L=L_{SE}+L_{CE}+L_{SIM}$$

(18)

2.4. Entity Alignment

By jointly learning the structural embeddings and attribute character embeddings, the entity embeddings of KG₁ and KG₂ can be placed into a unified vector space. Additionally, similar entities will have short semantic distances, defined as:

$$E\left({\mathit{e}}_1,{\mathit{e}}_2\right)={\Vert {\mathit{e}}_1-{\mathit{e}}_2\Vert }_{L_2},\forall e_1\in E_1,\forall e_2\in E_2$$

(19)

Given some unaligned entity ${\widehat{\mathit{e}}}_1$ in KG₁, the aim of entity alignment is to identify the unaligned entity ${\widehat{\mathit{e}}}_2$ in KG₂ with the minimum semantic distance to ${\widehat{\mathit{e}}}_1$, i.e., ${\widehat{\mathit{e}}}_2=\underset{e_2}{\arg \min }{\Vert {\widehat{\mathit{e}}}_1-{\mathit{e}}_2\Vert }_{L_2}$. Furthermore, we define a semantic distance threshold hyperparameter θ, such that ${\Vert {\widehat{\mathit{e}}}_1-{\widehat{\mathit{e}}}_2\Vert }_{L_2}<\theta$, ${\widehat{\mathit{e}}}_1$ and ${\widehat{\mathit{e}}}_2$ are deemed to refer to the same real-world entity. These considerations form the basis by which the entities in KG₁ and KG₂ are aligned.

To perform entity alignment, we first construct an aligned entity set $M=\left\{\left({\widehat{e}}_1,{\widehat{e}}_2\right)\right\}$ and minimize the loss function shown in Equation (20):

$$L_{EA}={\displaystyle \sum _{\left({\widehat{e}}_1,{\widehat{e}}_2\right)\in M}\left(L_{EA-SE}\left({\widehat{e}}_1,{\widehat{e}}_2\right)+L_{EA-SE}\left({\widehat{e}}_2,{\widehat{e}}_1\right)+L_{EA-CE}\left({\widehat{e}}_1,{\widehat{e}}_2\right)+L_{EA-CE}\left({\widehat{e}}_2,{\widehat{e}}_1\right)\right)}$$

(20)

where the expressions of $L_{EA-SE}\left({\widehat{e}}_1,{\widehat{e}}_2\right)$, $L_{EA-SE}\left({\widehat{e}}_2,{\widehat{e}}_1\right)$, $L_{EA-CE}\left({\widehat{e}}_1,{\widehat{e}}_2\right)$ and $L_{EA-CE}\left({\widehat{e}}_2,{\widehat{e}}_1\right)$ are shown in Equations (21)–(24), respectively.

$$\begin{array}{l}{L}_{EA-SE}\left({\widehat{e}}_1,{\widehat{e}}_2\right)={\displaystyle \sum _{\left({\widehat{e}}_1,r,t\right)}{\left[{\Vert {\widehat{\mathit{e}}}_2{\mathit{M}}_r+\mathit{r}-\mathit{t}{\mathit{M}}_r\Vert }_{L_2}-{\gamma }_1\right]}_{+}}+{\displaystyle \sum _{\left({\widehat{e}}_1,r,t\right)}\left\{\frac{1}{Z}{\displaystyle \sum _{p\in P\left({\widehat{e}}_2,t\right)}R\left(p|{\widehat{e}}_2,t\right)}{\left[{\Vert {\widehat{\mathit{e}}}_2+\mathit{p}-\mathit{t}\Vert }_{L_2}-{\gamma }_1\right]}_{+}\right\}}\\ \hfill +{\displaystyle \sum _{\left(h,r,{\widehat{e}}_1\right)}{\left[{\Vert \mathit{h}{\mathit{M}}_r+\mathit{r}-{\widehat{\mathit{e}}}_2{\mathit{M}}_r\Vert }_{L_2}-{\gamma }_1\right]}_{+}}+{\displaystyle \sum _{\left(h,r,{\widehat{e}}_1\right)}\left\{\frac{1}{Z}{\displaystyle \sum _{p\in P\left(h,{\widehat{e}}_2\right)}R\left(p|h,{\widehat{e}}_2\right)}{\left[{\Vert \mathit{h}+\mathit{p}-{\widehat{\mathit{e}}}_2\Vert }_{L_2}-{\gamma }_1\right]}_{+}\right\}}\end{array}$$

(21)

$$\begin{array}{l}{L}_{EA-SE}\left({\widehat{e}}_2,{\widehat{e}}_1\right)={\displaystyle \sum _{\left({\widehat{e}}_2,r,t\right)}{\left[{\Vert {\widehat{\mathit{e}}}_1{\mathit{M}}_r+\mathit{r}-\mathit{t}{\mathit{M}}_r\Vert }_{L_2}-{\gamma }_1\right]}_{+}}+{\displaystyle \sum _{\left({\widehat{e}}_2,r,t\right)}\left\{\frac{1}{Z}{\displaystyle \sum _{p\in P\left({\widehat{e}}_1,t\right)}R\left(p|{\widehat{e}}_1,t\right)}{\left[{\Vert {\widehat{\mathit{e}}}_1+\mathit{p}-\mathit{t}\Vert }_{L_2}-{\gamma }_1\right]}_{+}\right\}}\\ \hfill +{\displaystyle \sum _{\left(h,r,{\widehat{e}}_2\right)}{\left[{\Vert \mathit{h}{\mathit{M}}_r+\mathit{r}-{\widehat{\mathit{e}}}_1{\mathit{M}}_r\Vert }_{L_2}-{\gamma }_1\right]}_{+}}+{\displaystyle \sum _{\left(h,r,{\widehat{e}}_2\right)}\left\{\frac{1}{Z}{\displaystyle \sum _{p\in P\left(h,{\widehat{e}}_1\right)}R\left(p|h,{\widehat{e}}_1\right)}{\left[{\Vert \mathit{h}+\mathit{p}-{\widehat{\mathit{e}}}_1\Vert }_{L_2}-{\gamma }_1\right]}_{+}\right\}}\end{array}$$

(22)

$$L_{EA-CE}\left({\widehat{e}}_1,{\widehat{e}}_2\right)={\displaystyle \sum _{\left(e_1,a,v\right)}{\left[{\Vert {\mathit{e}}_2+\mathit{a}-f_a\left(\mathit{v}\right)\Vert }_{L_2}-{\gamma }_1\right]}_{+}}$$

(23)

$$L_{EA-CE}\left({\widehat{e}}_2,{\widehat{e}}_1\right)={\displaystyle \sum _{\left({\widehat{e}}_2,a,v\right)}{\left[{\Vert {\widehat{\mathit{e}}}_1+\mathit{a}-f_a\left(\mathit{v}\right)\Vert }_{L_2}-{\gamma }_1\right]}_{+}}$$

(24)

2.5. Datasets

Three datasets were used to evaluate the entity alignment methods:

DBP15K: This dataset comprises cross-lingual data extracted from DBpedia, including Chinese-to-English (DBP15K_ZH-EN), Japanese-to-English (DBP15K_JA-EN), and French-to-English (DBP15K_FR-EN). Each subset comprises 15,000 aligned entity pairs.

DWY100K: This dataset comprises monolingual data extracted from Dbpedia, Wikidata, and Yago3, and its subsets are DWY100K_DBP-WD and DWY100K_DBP-YG. Each subset comprises 100,000 aligned entity pairs.

A statistical summary of the DBP15K and DWK100K datasets is presented in

Table 1. DBP15K and DWY100K datasets.

Dataset			Entities	Relations	Attributes	Relation Triples	Attribute Triples
DBP15K	ZH-EN	Chinese	66,469	2830	8113	153,929	379,684
		English	98,125	2317	7173	237,674	567,755
	JA-EN	Japanese	65,744	2043	5882	164,373	354,619
		English	95,680	2096	6066	233,319	497,230
	FR-EN	French	66,858	1379	4547	192,191	528,665
		English	105,889	2209	6422	278,590	576,543
DWY100K	DBP-WD	DBpedia	100,000	330	351	463,294	381,166
		Wikidata	100,000	220	729	448,774	789,815
	DBP-YG	DBpedia	100,000	302	334	428,952	451,646
		Yago3	100,000	31	23	502,563	118,376

.

3. Results

3.1. Evaluation Indicators and Experimental Setting

Hits@k, mean rank (MR), and mean reciprocal rank (MRR) were used to evaluate the link prediction performance while comparing the methods.

Hits@k is the ratio of correctly aligned entities ranked in the top k. The larger the Hits@k value, the better is the performance of the model.

$$Hits@k=\frac{1}{\left|S\right|}{\displaystyle \sum _{i=1}^{\left|S\right|}{\displaystyle \prod \left(ran{k}_i\le k\right)}}$$

(25)

where S is a set of triples, |S| is the number of triples, rank_i is the link prediction ranking of the i-th triple, and ${\displaystyle \prod (\cdot )}$ is the indicator function, where 1 denotes that the condition is true and 0 denotes otherwise.

MR is the mean rank of correctly aligned entities; smaller values are indicative of better performance.

$$MR=\frac{1}{\left|S\right|}{\displaystyle \sum _{i=1}^{\left|S\right|}ran{k}_i}=\frac{1}{\left|S\right|}\left(ran{k}_1+ran{k}_2+\cdots ran{k}_{\left|S\right|}\right)$$

(26)

MRR is the mean of the reciprocal rank of the correctly aligned entities; larger values indicate better performance.

$$MRR=\frac{1}{\left|S\right|}{\displaystyle \sum _{i=1}^{\left|S\right|}\frac{1}{ran{k}_i}}=\frac{1}{\left|S\right|}\left(\frac{1}{ran{k}_1}+\frac{1}{ran{k}_2}+\cdots \frac{1}{ran{k}_{\left|S\right|}}\right)$$

(27)

The metrics used to evaluate entity alignment were precision, recall, and F1 measure.

Precision is the ratio of correctly aligned entities to all aligned entities, i.e., the accuracy of entity alignment, expressed as:

$$Precision=\frac{T_N}{T_N+F_N}$$

(28)

where T_N is the number of correctly aligned entities in the experimental results and F_N is the number of incorrectly aligned entities. Higher precision values indicate better performance.

Recall is the ratio of correctly aligned entities to all aligned entities in the dataset, given as:

$$Recall=\frac{T_N}{A_N}$$

(29)

where A_N is the total number of aligned entity pairs in the dataset. Larger recall values indicate better performance.

F1 measure is a combined measure of precision and recall, as shown in Equation (30):

$$F1=\frac{2\times Recall\times Prcision}{Recall+Prcision}$$

(30)

In the proposed entity alignment model, optimization was performed using stochastic gradient descent (SGD). The model parameters were configured as follows. Margin hyperparameters: γ₁ = 0.01 and γ₂ = 2; balancing factor: µ = 0.2; learning rate = 0.001; number of iterations: {500, 1500, 2500, 3500, 4500}; θ = 1.0; search step length: k = 0.2; dimensionality of the entity, relation, attribute, and attribute value embeddings = 100.

3.2. Validation of the Proposed Entity Alignment Method

In this section, we investigated the effect of (1) including semantic information from multi-step relation paths in structural embedding learning (Section 3.2) and (2) using improved loss function for embedding learning (Section 3.2) on the entity alignment performance of the proposed model. Furthermore, the proposed method will be compared to the following models to verify its effectiveness in improving entity alignment performance.

Model I: The semantic information in multi-step relation paths is not considered when learning structural embeddings, and the conventional loss function (margin-based ranking criterion) is used for embedding learning.

Model II: The semantic information in multi-step relation paths is considered when learning structural embeddings, and the conventional loss function (margin-based ranking criterion) is used for embedding learning.

Model III: The semantic information in multi-step relation paths is not considered when learning structural embeddings, and the loss function proposed in Section 3.2 (limit-based loss function) is used for embedding learning.

Table 2. Entity alignment performance of our method and Models I, II, and III on the DBP15K dataset, in terms of Hits@10, MR, and MRR.

Method	ZH-EN			JA-EN			FR-EN
	Hits@10	MR	MRR	Hits@10	MR	MRR	Hits@10	MR	MRR
Model I	0.781	179.4	0.777	0.793	188.9	0.763	0.728	191.3	0.796
Model II	0.868	142.3	0.837	0.854	98.5	0.862	0.874	102.7	0.884
Model III	0.847	114.7	0.855	0.867	107.9	0.834	0.827	88.5	0.867
Our Method	0.956	61.4	0.916	0.980	56.0	0.922	0.966	47.3	0.935

and

Table 3. Entity alignment performance of our method and Models I, II, and III on the DWY100K dataset, in terms of Hits@10, MR, and MRR.

Method	DBP-WD			DBP-YG
	Hits@10	MR	MRR	Hits@10	MR	MRR
Model I	0.843	147.7	0.785	0.822	133.7	0.804
Model II	0.909	66.5	0.856	0.932	76.0	0.911
Model III	0.877	92.1	0.872	0.914	68.3	0.925
Our Method	0.971	30.2	0.939	0.982	27.8	0.980

show the entity alignment performances of the proposed method and Models I, II, and II on the DBP15K and DWY100K datasets, in terms of Hits@10, MR, and MRR.

As shown in Table 2 and Table 3, the proposed method outperforms Models I–III by varying degrees on the DBP15K and DWY100K datasets in all metrics. Furthermore, models II and III outperform Model I on both datasets. Therefore, including semantic information from multi-step relation paths when learning structural embeddings and using the improved loss function can benefit the entity alignment performance, thereby proving the efficacy of the proposed method.

Figure 1 presents the data in Table 2 and Table 3 as bar charts to provide a better understanding of the experimental results. Figure 1a–c compare the proposed method to six other methods in terms of Hits@10, MR, and MRR on the DBP15K and DWY100K datasets. As shown in Figure 1a,c, the proposed method exhibited significantly higher Hits@10 and MRR scores compared to models I, II, and III. In addition, Figure 1b indicates that our method exhibited a significantly lower MR than models I, II, and III, which provides further evidence of the efficacy of our method in improving entity alignment performance.

Table 4. Entity alignment performance of our method and Models I, II, and III on the DBP15K dataset, in terms of precision, recall, and F1 measure.

Method	ZH-EN			JA-EN			FR-EN
	Precision	Recall	F1	Precision	Recall	F1	Precision	Recall	F1
Model I	0.866	0.807	0.834	0.835	0.819	0.821	0.837	0.803	0.842
Model II	0.906	0.887	0.910	0.903	0.868	0.893	0.900	0.870	0.907
Model III	0.915	0.854	0.892	0.917	0.877	0.909	0.911	0.882	0.923
Our Method	0.951	0.912	0.932	0.963	0.928	0.936	0.968	0.916	0.941

and

Table 5. Entity alignment performance of our method and Models I, II, and III on the DWY100K dataset, in terms of precision, recall, and F1 measure.

Method	DBP-WD			DBP-YG
	Precision	Recall	F1	Precision	Recall	F1
Model I	0.876	0.881	0.882	0.853	0.816	0.882
Model II	0.935	0.914	0.942	0.947	0.879	0.951
Model III	0.944	0.922	0.956	0.931	0.855	0.936
Our Method	0.998	0.956	0.988	0.987	0.962	0.991

show the entity alignment performances of our method and Models I–III in terms of precision, recall, and F1 measure on the DBP15K and DWY100K datasets.

According to the experimental results, on the ZH-EN dataset, the proposed method improved precision, recall, and F1 measure by 3.6–8.5%, 2.5–10.5%, and 2.2–9.8%, respectively, compared to the ZH-EN dataset. On the JA-EN dataset, our method improved precision, recall, and F1 measure by 4.6–12.8%, 5.1–10.9%, and 2.7–11.5%, respectively. On the FR-EN dataset, our method improved precision, recall, and F1 measure by 5.7–13.1%, 3.4–11.3%, and 1.8–9.9%, respectively. On the DBP-WD dataset, our method improved precision, recall, and F1 measure by 5.4–12.2%, 3.4–7.5%, and 3.2–10.6%, respectively. On the DBP-YG dataset, our method improved precision, recall, and F1 measure by 4.0–13.4%, 8.3–14.6%, and 4.0–10.9%, respectively. These results indicate that our model improves the performance of all aforementioned datasets in all metrics, and hence, outperforms Models I–III.

The data in Table 4 and Table 5 are shown as bar charts in Figure 2 to facilitate a more intuitive understanding of the experimental results. Figure 2a–e compare our method to Models I–III on the ZH-EN, JA-EN, FR-EN, DBP-WD, and DBP-YG datasets, in terms of precision, recall, and F1 measure. The comparison results verify that our method significantly outperforms Models I–III on all three metrics, highlighting the effectiveness of our model in improving entity alignment performance.

The epoch-wise changes of our method and Models I–III on the ZH-EN, JA-EN, FR-EN, and DBP-WD and DBP-YG in terms of the F1 measure are shown in Figure 3. The F1 measure gradually improves and stabilizes with each epoch in all methods. Furthermore, our method exhibited the highest F1 measure of the aforementioned methods, highlighting its effectiveness in improving entity alignment performance.

3.3. Validating the Superiority of Our Method in Entity Alignment

In this section, we compare the proposed method to six other entity alignment methods: MTransE [13], IPTransE [14], JAPE [15], BootEA [16], MultiKE [17], and KDCoe [18].

Table 6. Comparison between our method and other methods in entity alignment performance on the DBP15K dataset, using Hits@10, MR, and MRR.

Method	ZH-EN			JA-EN			FR-EN
	Hits@10	MR	MRR	Hits@10	MR	MRR	Hits@10	MR	MRR
MTransE	0.435	356.7	0.334	0.578	384.5	0.496	0.562	289.5	0.451
IPTransE	0.559	208.2	0.508	0.452	329.6	0.384	0.441	224.3	0.343
JAPE	0.662	127.4	0.600	0.449	240.2	0.408	0.689	191.2	0.635
BootEA	0.629	109.8	0.589	0.661	109.4	0.623	0.762	117.6	0.719
MultiKE	0.701	66.2	0.627	0.702	78.9	0.653	0.812	101.8	0.746
KDCoE	0.668	78.0	0.611	0.652	91.1	0.611	0.817	92.5	0.802
Our Method	0.956	61.4	0.916	0.980	56.0	0.922	0.966	47.3	0.935

and

Table 7. Comparison between our method and other methods in entity alignment performance on the DWY100K dataset, using Hits@10, MR, and MRR.

Method	DBP-WD			DBP-YG
	Hits@10	MR	MRR	Hits@10	MR	MRR
MTransE	0.610	212.3	0.555	0.513	202.7	0.320
IPTransE	0.651	199.9	0.504	0.798	114.9	0.760
JAPE	0.792	108.4	0.686	0.413	99.4	0.508
BootEA	0.721	97.0	0.590	0.625	71.3	0.430
MultiKE	0.633	74.5	0.507	0.725	83.7	0.654
KDCoE	0.707	86.7	0.771	0.738	55.0	0.811
Our Method	0.971	30.2	0.939	0.982	27.8	0.980

show the entity alignment performances of these methods on the DBP15K and DWY100K datasets, in terms of Hits@10, MR, and MRR.

According to the experimental results, the proposed method outperformed all other methods in all metrics on the DBP15K and DWY100K datasets. As expected, MTransE performed poorly in the entity alignment task, considering it loses information when learning the mapping between different embedding spaces. Consequently, it could not compute the similarities between entities from different KGs. IPTransE and JAPE outperformed MTransEE, considering they can utilize relation paths and entity attributes. However, MTransE, IPTransE, and JAPE are reliant on the number of seed alignments, which constrains their ability to improve the entity alignment performance. Although KDCoE uses entity descriptions for entity alignment, the semantic information in these descriptions has limited applications. Furthermore, KDCoE adds newly labeled entities to the seed set during co-training, making it susceptible to error accumulation. As a result, the entity alignment performance of this model is inferior to that of the proposed model. Furthermore, in comparison to the proposed model to the MultiKE model, which utilizes the entity structure, names, and descriptions for entity alignment, our model exhibited higher Hits@10 and MRR scores and lower MR scores, indicating that our model better uses the entity, relation, and attribute information, and hence, performs better in entity alignment.

The experimental results in Table 6 and Table 7 are shown as bar charts in Figure 4. The Hits@10, MR, and MRR of the aforementioned methods on the DBP15K and DWY100K datasets are shown in Figure 4a–c. As shown in Figure 4a,c, our method exhibited higher Hits@10 and MRR scores compared to the six other methods. Figure 4b indicates that our method exhibited the lowest MR values. Therefore, our method is superior to all other methods in terms of entity alignment performance.

Figure 4 contains bar charts comparing the entity alignment performance of our method with that of the six other methods, in terms of Hits@10, MR, and MRR.

Table 8. Comparison between our method and six other methods in terms of entity alignment performance (precision, recall, and F1 measure) on the DBP15K dataset.

Method	ZH-EN			JA-EN			FR-EN
	Precision	Recall	F1	Precision	Recall	F1	Precision	Recall	F1
MTransE	0.876	0.637	0.736	0.795	0.661	0.696	0.833	0.733	0.766
IPTransE	0.892	0.646	0.750	0.872	0.726	0.793	0.894	0.749	0.834
JAPE	0.924	0.659	0.766	0.911	0.809	0.817	0.851	0.821	0.857
BootEA	0.928	0.803	0.862	0.921	0.815	0.857	0.921	0.847	0.861
MultiKE	0.935	0.775	0.849	0.905	0.889	0.869	0.911	0.855	0.870
KDCoE	0.937	0.861	0.872	0.933	0.874	0.901	0.930	0.879	0.871
Our Method	0.951	0.912	0.932	0.963	0.928	0.936	0.968	0.916	0.941

and

Table 9. Comparison between our method and six other methods in terms of entity alignment performance (precision, recall, and F1 measure) on the DWY100K dataset.

Method	DBP-WD			DBP-YG
	Precision	Recall	F1	Precision	Recall	F1
MTransE	0.901	0.885	0.885	0.913	0.837	0.895
IPTransE	0.923	0.886	0.894	0.918	0.864	0.910
JAPE	0.952	0.892	0.919	0.933	0.910	0.922
BootEA	0.955	0.907	0.933	0.955	0.937	0.954
MultiKE	0.967	0.921	0.934	0.968	0.939	0.964
KDCoE	0.977	0.934	0.952	0.977	0.945	0.967
Our Method	0.998	0.956	0.988	0.987	0.962	0.991

compare our method to other methods in terms of precision, recall, and F1 measure on the DBP15K and DWY100K datasets, respectively.

According to the experimental results, on the ZH-EN dataset, our model improved precision, recall, and F1 measure by 1.4–7.5%, 5.1–27.5%, and 6.0–19.6%, respectively. On the JA-EN dataset, our model improved precision, recall, and F1 measure by 3.0–16.8%, 3.9–26.7%, and 3.5–24.0%, respectively. On the FR-EN dataset, our model improved precision, recall, and F1 measure by 3.8–13.5%, 3.7–18.3%, and 7.0–17.5%, respectively. On the DBP-WD dataset, our model improved precision, recall, and F1 measure by 2.1–9.7%, 2.2–7.1%, and 3.6–10.3%, respectively. On the DBP-YG dataset, our model improved precision, recall, and F1 measure by 1.0–7.4%, 1.7–12.5%, and 2.4–9.6%, respectively. Therefore, our model significantly improved all three metrics on all datasets, thereby confirming that our model is superior to the other entity alignment models in terms of entity alignment performance.

In Figure 5, the data in Table 8 and Table 9 are shown in the form of bar charts. Figure 5a–e compare our method to six other methods in terms of precision, recall, and F1 measure on the ZH-EN, JA-EN, FR-EN, DBP-WD, and DBP-YG datasets. These figures clearly indicate that our method exhibited higher precision, recall, and F1 measure scores compared to all other methods, suggesting that our method is superior in terms of entity alignment performance.

4. Discussion

Currently, entity alignment is generally performed based on knowledge representation learning, which requires an adequate number of pre-aligned entity pairs to be collected in advance. Considering these pre-aligned entity pairs must be manually collected, the collection process may be plagued by various difficulties. In addition, it is difficult to ensure that the pre-aligned entity pairs are aligned accurately, considering that any inaccuracy can negatively affect the subsequent entity alignment process. Therefore, to address these issues, this study proposed an entity alignment method based on the joint learning of entity and attribute representations. We proposed a method for relation and attribute alignment based on cosine similarity by calculating the cosine similarities between the semantic embeddings of the relations and attributes, followed by manual inspection and amendments. The relations and attributes were then aligned by renaming the relations and attributes with the same meaning to the same name. Then, we proposed a learning model for the structural embeddings and attribute character embeddings of relation and attribute triples from different KGs. The structural embedding learning was performed via a triple modeling method using TransE and PTransE, which incorporated the semantic information contained by direct and multi-step relation paths, thereby extracting the structural embedding vectors from the embedding vector space. The attribute character embeddings were learned using the N-gram-based compositional function to encode a character sequence for the attribute values, with TransE used to model attribute triples in the embedding vector space to obtain attribute character embedding vectors. Additionally, the structural embeddings and attribute character embeddings were jointly learned to enable transferring structural embeddings from different KGs into a unified vector space. Entity alignment was then performed by calculating the similarities of the structural embeddings of entities. Moreover, a limit-based loss function was proposed to replace the conventional margin-based ranking criterion for learning structural and attribute character embeddings, which ensured that positive and negative triples received significantly low and high scores, respectively, thereby preventing drifts during the projection of structural embeddings into the unified vector space. Finally, we experimentally demonstrated that the proposed entity alignment method performed well on the DBP15K and DWK100K datasets and outperformed all entity alignment methods of the same type by a maximum of 16.8, 27.5, and 24.0% in precision, recall, and F1 measure, respectively.