ERDERP: Entity and Relation Double Embedding on Relation Hyperplanes and Relation Projection Hyperplanes

Abstract

Since data are gradually enriched over time, knowledge graphs are inherently imperfect. Thus, knowledge graph completion is proposed to perfect knowledge graph by completing triples. Currently, a family of translation models has become the most effective method for knowledge graph completion. These translation models are modeled to solve the complexity and diversity of entities, such as one-to-many, many-to-one, and many-to-many, which ignores the diversity of relations themselves, such as multiple relations between a pair of entities. As a result, with current translation models, it is difficult to effectively extract the semantic information of entities and relations. To effectively extract the semantic information of the knowledge graph, this paper fundamentally analyzes the complex relationships of the knowledge graph. Then, considering the diversity of relations themselves, the complex relationships are refined as one-to-one-to-many, many-to-one-to-one, one-to-many-to-one, many-to-one-to-many, many-to-many-to-one, one-to-many-to-many, and many-to-many-to-many. By analyzing the complex relationships, a novel knowledge graph completion model, entity and relation double embedding on relation hyperplanes and relation projection hyperplanes (ERDERP), is proposed to extract the semantic information of entities and relations. First, ERDERP establishes a relation hyperplane for each relation and projects the relation embedding into the relation hyperplane. Thus, the semantic information of the relations is extracted effectively. Second, ERDERP establishes a relation projection hyperplane for each relation projection and projects entities into relation projection hyperplane. Thus, the semantic information of the entities is extracted effectively. Moreover, it is theoretically proved that ERDERP can solve antisymmetric problems. Finally, the proposed ERDERP are compared with several typical knowledge graph completion models. The experimental results show that ERDERP is significantly effective in link prediction, especially in relation prediction. For instance, on FB15k and FB15k-237, Hits@1 of ERDERP outperforms TransH at least 30%.

1. Introduction

The knowledge graph was first proposed in 2012 [1]. The goal of knowledge graphs is to build a structured knowledge base to serve artificial intelligence [2,3,4], such as intelligent recommendation systems [5,6], community discovery [7,8], and so on. A large number of knowledge entities exist in real life and the network, and the relations among them are diverse. Knowledge graphs can connect the entities through relations to form a huge semantic network to effectively realize semantic expression intuitively, naturally, and efficiently. Knowledge in a knowledge graph is expressed in the form of head entity, relation, and tail entity (denoted as (h, r, t)). The simple and efficient expression has attracted extensive attention from all walks of life. Thus, knowledge graphs have developed rapidly in recent years. Many classic knowledge graphs are established, such as ConceptNet of MIT Media Lab [9], the world knowledge base Freebase [10], WordNet of Princeton University Cognitive Science Lab [11], and so on.

The knowledge graph has become an important information source in academia and industry. Knowledge graphs have attracted much attention because they can accurately reflect the real world by expressing abstract knowledge well, such as concepts and levels. Therefore, they have been widely used in many fields and extensively and deeply studied. For example, in biomedicine, as an important tool for mining future associations between biomedical concepts, the knowledge graph has attracted much attention [12]. In traditional Chinese medicine, knowledge graphs are used to provide a variety of knowledge services, such as knowledge visualization, knowledge retrieval, and knowledge recommendation [13]. In metallic materials, knowledge graphs based on Wikipedia and DBpedia have become a key tool to provide guidance for material researchers [14]. In geoscience, a geological knowledge graph constructed by combining general terms in the geological dictionary with geological terms improves the performance of the CRF-based Chinese word segmentation model [15].

A complete knowledge graph is of great significance for promoting the development of artificial intelligence and supporting the application of intelligent information services, such as intelligent search [16], intelligent answer system [17], personalized recommendation [18], and so on. However, the knowledge graph is usually incomplete in the construction process, for example, missing entities or relations. To effectively complete a knowledge graph, knowledge representation learning methods are developed. From the perspective of knowledge representation methods, knowledge representation learning can be divided into two stages. At the initial stage of knowledge representation learning, some typical knowledge representation models are established, such as structured embedding (SE) model [19], single-layer model (SLM), neural tensor network [20], semantic matching energy (SME) model [21,22], latent factor model (LFM) [23,24] and matrix factorization model (RESCAL) [25,26]. Facts show that SE, SLM, SME, LFM, neural tensor network and matrix factorization model are weak in accurately learning the semantic information of entities and relations, but the computational complexity is high. Therefore, these typical knowledge representation models are difficult to apply to some practical application scenarios. Therefore, researchers began to explore new knowledge representation learning models. After practical tests, translation models, as typified by TransE [27], have gradually become the mainstream of knowledge representation learning.

The family of translation models is effective for a few optimization parameters, simple model structure, and efficient knowledge graph completion. Since TransE was developed and verified the effectiveness of knowledge graph completion, a large number of researchers focused on translation models to effectively extract the semantic features of entities and relations to solve the complex relationships problems of knowledge graphs, such as one-to-many, many-to-one, and many-to-many. TransE has been continuously improved and many effective translation models have been established, such as TransH [28], TransR [29], TransD [30], TransA [31], TransAt [32], TransMS [33], TransG [34], KG2E [35], PTransE [36], PaSKoGE [37], Boxe [38], and so on. Among them, TransH, TransR and TransD adopt different strategies to project the head entity and tail entity. TransA changes the distance measure function in the loss function to Mahalanobis distance, and learns different weights for each dimension of (h + r − t). TransAt conducts presentation learning by introducing an attention mechanism. TransMS applies nonlinear functions to propagate multidirectional semantics. TransG and KG2E use Gaussian distribution to describe entities and relations. PTransE and PaSKoGE consider the semantic information of the path in the translation process. BoxE embeds entities as points, and relations as a set of hyperrectangles (or boxes), which spatially characterize basic logical properties. In addition, some researches focus on representation space. For example, ComplEx models entity and relation embedding from complex space [39], RotatE regards the relation as the rotation from head entity to tail entity in complex space [40], and HAKE maps entities to polar coordinates to learn entity and relation embedding [41].

However, with the current translation models, it is difficult to extract semantic features of relations, and the representation ability of entities and relations by these models needs to be further improved. There may be two reasons. On the one hand, the current translation models are generally applied in entity prediction, usually ignoring the importance of relation prediction. On the other hand, most current translation models are designed to address the complexity and diversity of entities to solve the complex relationships, but the diversity of relations themselves is ignored in the complex relationships. Therefore, the diversity of relations themselves is introduced into the complex relationships of knowledge graphs to highlight the complex relationships. Then, the complex relationships are refined into one-to-one-to-many, many-to-one-to-one, one-to-many-to-one, many-to-one-to-many, many-to-many-to-one, one-to-many-to-many, and many-to-many-to-many, as shown in Figure 1. In reality, the complex relationship proposed in this paper is ubiquitous in real datasets. For example, there are 485,661 triples of 1-1-N, 520,476 triples of N-1-1, 211,457 triples of 1-M-1, and 26,943 triples of N-1-N, and the number of head entities with 1-M-N in FB15K is 10,143. It is intuitively obtained that multiple relations exist between a pair of entities from the refined complex relationship, as shown in Figure 2. However, by presenting the diversity of the relations themselves independently, it is found that using traditional models without considering the diversity of relations themselves makes it difficult to distinguish multiple relations between a pair of entities.

To solve the above issues, this paper proposes an entity and relation double embedding on relation hyperplanes and relation projection hyperplanes model (ERDERP) to effectively extract the semantic information of entities and relations and improve the accuracy of entity prediction and relation prediction. In a knowledge graph, each relation has its own characteristics. First, to solve the diversity of relations, ERDERP establishes a relation hyperplane for each relation and projects the relation embedding into the relation hyperplane to increase the distinguishability of multiple relations. Then, ERDERP can effectively capture the semantic characteristics of relations. Second, to solve the complexity and diversity of entities, ERDERP establishes a relation projection hyperplane for each relation projection (i.e., the projection of relation on the relation hyperplane) and projects entities into relation projection hyperplane to increase the distinguishability of multiple entities. Then, ERDERP can effectively capture the semantic characteristics of entities. Finally, the relation projection is regarded as the translation vector, and the translation process from head entity to tail entity is realized on the relation projection hyperplane. Then, the entity and relation embedding vectors with rich semantic information can be obtained to improve the accuracy of link prediction. Moreover, it is theoretically proved that ERDERP can solve the inference patterns of antisymmetry.

The rest of this paper is organized as follows. Section 2 introduces several typical translation models. The ERDERP model is proposed and detailed described in Section 3. In Section 4, the proposed ERDERP is compared with several typical knowledge representation models in entity prediction and relation prediction. Section 5 gives the conclusion of this paper.

2. Related Work

The family of translation models is widely used for a few optimization parameters, simple model structure and efficient knowledge graph completion. Mikolov et al. proposed word2vec to learn word representation, and found that there exists translation invariance in word vector space [42]. Inspired by word2vec, through deeply studying the method of embedding entities and relations of multiple relational data into low-dimensional space, Borders et al. proposed the first translation model—TransE [27]. They regarded the triples in knowledge graph as the translation process from head entity into tail entity, and the relation was regarded as the translation between two entities, i.e., $h+r=t$. However, there are complex relationships between entities, such as one-to-many, many-to-one, and many-to-many. Theoretical analysis and experimental results show that it is difficult to solve complex relationships with TransE. Thus, translation models need to be improved for extracting the semantic features of entities and relations.

To solve the complex relationships of entities and enhance the representation ability of low-dimensional embedding vectors, researchers have made a series of improvements to TransE. Several classical translation models have been proposed. By projecting the head entity and tail entity to the hyperplane of the relation to alleviate the problem of complex relationships, TransH realizes that the same entity has different representations on different relation hyperplanes. However, TransH assumes that relations and entities are in the same semantic space, which limits its representation ability. To solve the problem of TransH, TransR was proposed. It respectively establishes an entity space and a relation space for entities and relation to enhance the representation ability of low-dimensional embedding vectors. Then, a mapping matrix is established for each relation to obtain the projection vectors of head entity and tail entity. Finally, the translation process is realized in the relation space. However, the computational complexity of TransR is increased because of the mapping matrix. Meanwhile, the representation ability of the TransR is limited by using the same mapping matrix in head entity and tail entity. Furthermore, TransD is proposed to solve these problems of TransR. It constructs two different mapping matrices through three special projection vectors to reduce model parameters, which solves the problem of the head entity and tail entity sharing one projection matrix in TransR. Through analysis, the above translation models solve the complex relationships of entities by projecting entities.

However, the above models take the complex relationships of entities as the starting point of research, ignoring the diversity of the relations themselves, which results in difficulty distinguishing multiple relations between a pair of entities for traditional models. In fact, there may be multiple relations between a pair of entities from the refined complex relationships. For example, there are 211,457 triples of 1-M-1. In addition, the above models are applied in entity prediction. However, the application and performance in relation prediction are ignored. Therefore, this paper fundamentally studies translation models, i.e., refining the complex relationships of knowledge graphs to highlight the diversity of the relations themselves and modeling translation models to solve the refined complex relationships to effectively extract the semantic features of entities and relationships. Specifically, the complex relationships are refined into one-to-one-to-many (1-1-N), many-to-one-to-one (N-1-1), one-to-many-to-one (N-1-N), many-to-one-to-many (1-M-1), many-to-many-to-one (N-M-1), one-to-many-to-many (1-M-N), and many-to-many-to-many (N-M-N), as shown in Figure 1. The definitions of the complex relationships are as follows.

• 1-1-N: Only one relation exists between head entity and tail entity, and one head entity corresponds to multiple tail entities.
• N-1-1: Only one relation exists between head entity and tail entity, and one tail entity corresponds to multiple head entities.
• N-1-N: Only one relation exists between head entity and tail entity, and the relation connects multiple head entities and multiple tail entities.
• 1-M-1: Multiple relations exist between two entities.
• N-M-1: For tail entity, there are multiple head entities and multiple relations.
• 1-M-N: For head entity, there are multiple tail entities and multiple relations.
• N-M-N: For tail entity, there are multiple head entities and multiple relations, and for head entity, there are multiple tail entities and multiple relations.

By definition, the complex relationships are not disjointed. The refined complex relationships reflect the complexity of knowledge graphs. For example, N-M-1 can be regarded as a combination of 1-M-1 and N-1-1. In N-M-1 of Figure 1, for the tail entity “Snake,” using 1-M-1 and N-1-1 alone does not fully describe the complexity relationships of “Snake.” If the head entity “Eagle” of “Snake” is solely focused on, the relationship is 1-M-1. If the relation “IsCompetitorof” is solely focused on, the relationship is N-1-1. Therefore, the combination of 1-M-1 and N-1-1 can fully describe the complex relationship of the tail entity “Snake.” Similarly, 1-M-N can be regarded as a combination of 1-M-1 and 1-1-N, and N-M-N can be regarded as a combination of 1-M-N and N-M-1.

An entity and relation double embedding on relation hyperplanes and relation projection hyperplanes model (ERDERP) is proposed to improve the accuracy of entity prediction and relation prediction. ERDERP first establishes a relation hyperplane for each relation and projects the relation embedding into the relation hyperplane to increase the distance of the multiple relation embedding. Thus, the diversity of relations can be solved and the distinguishability of multiple relations can be enhanced. Second, ERDERP establishes a relation projection hyperplane for each relation projection (i.e., the projection of relation on the relation hyperplane) and projects entities into relation projection hyperplane to increase the distinguishability of multiple entities. Thus, the complexity and diversity of entities can be solved. Finally, the relation projection is regarded as the translation vector, and the translation process from head entity to tail entity is realized on the relation projection hyperplane. Then, the entity and relation embedding vectors with rich semantic information can be obtained. Figure 3 shows the schematic diagram of ERDERP. In Figure 3, $r_1$ and $r_2$ are two different relations of a pair of entities. ERDERP first establishes two relation hyperplanes (i.e., ${\alpha }_1$ for $r_1$ and ${\beta }_1$ for $r_2$) and projects the relation to its relation hyperplane to enhance the distinguishability between the two relations. Then ERDERP establishes a relation projections hyperplane (i.e., ${\alpha }_2$ or ${\beta }_2$) for each relation projection (i.e., $r_{1\perp }$ or $r_{2\perp }$). Thus, the embedding of the two entities can be distinguished.

Table 1. Score functions and analysis of the parameter complexity of several translation models ($n_e$ represents the number of entities. $n_r$ represents the number of relations. $k$ and $n$ are the dimensions of the embedding space for entities and relations, respectively. $d$ represents the number of relation clusters).

Model	$\mathbf{Score} \mathbf{Function} {\mathit{f}}_{\mathit{r}}\left(\mathit{h},\mathit{t}\right)$	Parameters
TransE	${\Vert h+r-t\Vert }_{L_1/L_2},h,t,r\in {ℝ}^k$	$O\left(n_{e}k+n_{r}k\right)$
TransH	$\Vert (h-w_r^{T}h{w}_r)+d_r-(t-w_r^{T}t{w}_r)\Vert$ $h,t,d_r,w_r\in {ℝ}^k$	$O\left(n_{e}k+2n_{r}k\right)$
TransR	$\Vert (h{\mathrm{M}}_r)+r-(t{\mathrm{M}}_r)\Vert$ $h,t\in {ℝ}^k,r\in {ℝ}^n{,\mathrm{M}}_r\in {ℝ}^{k\times n}$	$O\left(n_{e}k+n_r(k+1)n\right)$
CTransR	${\Vert h{\mathrm{M}}_r+r_c-h{\mathrm{M}}_r\Vert }_2^2+\alpha {\Vert r_c-r\Vert }_2^2$ $h,t\in {ℝ}^k,r,r_c\in {ℝ}^n{,\mathrm{M}}_r\in {ℝ}^{k\times n}$	$O\left(n_{e}k+n_r(k+d)n\right)$
TransD	${\Vert h{\mathrm{M}}_{r,h}+r-t{\mathrm{M}}_{r,t}\Vert }_2^2$ $h,t\in {ℝ}^k,r\in {ℝ}^n{,\mathrm{M}}_{r,h},{\mathrm{M}}_{r,t}\in {ℝ}^{k\times n}$	$O\left(2n_{e}k+2n_{r}n\right)$
ERDERP (our)	$\Vert (h-w_r^{T}h{w}_r)+(r-s_r^{T}r{s}_r)-(t-w_r^{T}t{w}_r)\Vert$ $h,t,r,w_r,s_r\in {ℝ}^k$	$O\left(n_{e}k+3n_{r}k\right)$

compares the above translation models and the proposed ERDERP. Some information of several typical translation models and the proposed ERDERP in this paper, such as the score function and the parameters complexity, are described in Table 1. From a problem-oriented view, the typical translation models are used to solve the complexity and diversity of entities, such as one-to-many and many-to-one. ERDERP aims to solve the complexity and diversity of entities of entities and the diversity of the relations themselves. The distinguishing feature among translation models is that each has its own score function that shows the transformation method of head entity and tail entity.

3. An Entity and Relation Double Embedding on Relation Hyperplanes and Relation Projection Hyperplanes Model for Knowledge Graph Completion

The refined complex relationships show that the relation is also diversified. However, traditional translation models are modeled to solve the complexity and diversity of entities and do not directly take the diversity of relations themselves into account, resulting in the learned embedding vector of entities and relations not fully containing the semantic information of relations. Thus, the accuracy of link prediction of the translation model needs to be improved. To effectively extract the semantic information of entities and relations and solve the diversity of the relations themselves and the complexity and diversity of entities, a novel knowledge graph completion model, the entity and relation double embedding on relation hyperplanes and relation projection hyperplanes model (ERDERP), is proposed in this section. ERDERP is applied to link prediction, including entity prediction and relation prediction. First, the general concepts and symbols are described, as shown in

Table 2. The general concepts and symbols in this paper.

Concept or Symbol	Meaning
(h, r, t)	Triple in knowledge graph.
$(h, r, t)$	The embedding vector of (h, r, t).
h, r, t	Head entity, relation and tail entity in (h, r, t), respectively.
$h$,$r$,$t$	The embedding vector of h, r, t, respectively.
$E$	The entity set of knowledge graph.
$\mathsf{\Delta }$	The triple set of knowledge graph.
$(h,r,t)\in \mathsf{\Delta }$	(h, r, t) is a triple in $\mathsf{\Delta }$.
${\mathsf{\Delta }}_{\left(h,r,t\right)}^{\prime }$	The incorrect triple set generated by (h, r, t).
$(h^{\prime },r,t^{\prime })\in {\mathsf{\Delta }}_{\left(h,r,t\right)}^{\prime }$	$(h^{\prime },r,t^{\prime })$ is a triple in ${\mathsf{\Delta }}_{\left(h,r,t\right)}^{\prime }$.

.

3.1. An Entity and Relation Double Embedding on Relation Hyperplanes and Relation Projection Hyperplanes Model

The current translation models were proposed to solve the complexity and diversity of entities, such as one-to-many, many-to-one and many-to-many, so as to capture semantic information of entities and relations and improve the accuracy of link prediction. However, the current translation models cannot extract the different features of relations well, which makes it difficult to distinguish relations through the learned relation embedding, and further affects the extraction of entity semantic information. To effectively capture the semantic information of entities and relations, this paper fundamentally studies the translation model, which is different from the starting point of other translation models, i.e., analyzing the TransE model and the complexity and diversity of the entities and the diversity of relations. Then, an entity and relation double embedding on relation hyperplanes and relation projection hyperplanes model (ERDERP) is proposed. The detailed modeling process of ERDERP model is as follows.

For a correct triple (h, r, t), the object of TransE is to minimize the score function ${\Vert h+r-t\Vert }_{L_1/L_2}$. Results show that TransE significantly improves the performance of link prediction compared with the traditional non-translation models. Under TransE, $h+r=t$ is workable for the triple (h, r, t). Considering the 1-M-1 of complex relationships, the following result can be directly obtained by TransE.

• $\forall i\in \left\{1,2,3,\cdots ,m\right\}$ and $\left(h,r_i,t\right)\in \mathsf{\Delta }$, then the relation embedding vectors of $r_i$ satisfy $r_1=r_2=r_3=\cdots =r_m$.

The above result shows that through TransE, the embedding vectors of relations between a pair of entities are the same as each other. The result indicates that it is difficult to distinguish the multiple relations between a pair of entities through TransE. Then, RTransE is proposed to solve the above problem (i.e., correctly distinguishing the relations). RTransE enhances the embedding representation of relations by establishing a relation hyperplane for each relation, so as to enhance the distinguishability of multiple relations between a pair of entities. Specifically, the RTransE modeling process is as follows.

In RTransE, for a triple (h, r, t), the embedding vector $r$ is projected into its relation hyperplane, $S_r$, and $r_{\perp }$ is the projected vector of $r$, where $r_{\perp }=r-s_r^{T}r{s}_r$ and $s_r$ is the unit normal vector of $S_r$. Then, RTransE makes the triplet (h, r, t) satisfy $h+r_{\perp }\approx t$ on the hyperplane $S_r$. Thus, each embedding vector of relation can be the same on the hyperplane and be different on the whole, so as to ensure that the model fully extracts the different features and the same features between relations. Then the multiple relations between a pair of entities can be distinguished, i.e., RTransE can solve 1-M-1. However, RTransE is weak in solving the problems of many-to-one-to-one and one-to-one-many, which is the same as TransE. The following results are directly obtained by using RTransE and TransE.

• For N-1-1 of complex relationships, i.e., if $\forall i\in \left\{1,2,3,\cdots ,m\right\}$ and $\left(h_i,r,t\right)\in \mathsf{\Delta }$, then the tail entity embedding vectors of $h_i$ satisfy $h_1=h_2=h_3=\cdots =h_m$.
• For 1-1-N of complex relationships, i.e., if $\forall i\in \left\{1,2,3,\cdots ,m\right\}$ and $\left(h,r,t_i\right)\in \mathsf{\Delta }$, then the tail entity embedding vectors of $t_i$ satisfy $t_1=t_2=t_3=\cdots =t_m$.$\forall i\in \left\{1,2,3,\cdots ,m\right\}$ and $\left(h,r,t_i\right)\in \mathsf{\Delta }$, then the tail entity embedding vectors of $t_i$ satisfy $t_1=t_2=t_3=\cdots =t_m$.

The above result shows that RTransE is weak in solving the complexity and diversity of the entities. To solve the above problems, ERDERP is proposed based on RTransE. ERDERP establishes a relation projection hyperplane for each relation projection (i.e., the projection of relation on the relation hyperplane) and projects entities into the relation projection hyperplane to increase the distinguishability between multiple entities and capture the semantic characteristics of entities. Finally, the relation projection is regarded as the translation vector, and the translation process from head entity to tail entity is realized on the relation projection hyperplane. Thus, ERDERP can solve the problems of N-1-1 and 1-1-N while guaranteeing the distinguishability of multiple relations between a pair of entities. Therefore, ERDERP can effectively extract semantic information of entities and relations.

In detail, for a triple (h, r, t), ERDERP first establishes a relation hyperplane for relation r and projects the relation embedding representation ($r$) into its relation hyperplane ($S_r$) to increase the distinguishability of the relations. Then, the head entity and tail entity are projected into the relation projection hyperplane ($W_r$) to which the relation projection ($r_{\perp }$) belongs to complete the entity translation process on the relation projection hyperplane ($W_r$). Thus, Equation (1) is obtained.

$$h_{\perp }=h-w_r^{T}h{w}_r, t_{\perp }=t-w_r^{T}t{w}_r, r_{\perp }=r-s_r^{T}r{s}_r$$

(1)

where $s_r$ is the unit normal vector of $S_r$, and $w_r$ is the unit normal vector of $W_r$. $r_{\perp }$ is the projection vector of the relation on $S_r$, $h_{\perp }$ is the projection vector of the head entity on $W_r$, and $t_{\perp }$ is the projection vector of the tail entity on $W_r$.

Finally, the score function of ERDERP is proposed, as shown in Equation (2).

$$f_r\left(h,t\right)={\Vert \left(h-w_r^{T}h{w}_r\right)+\left(r-s_r^{T}r{s}_r\right)-\left(t-w_r^{T}t{w}_r\right)\Vert }_{L_1/L_2}$$

(2)

Equation (2) shows that the score function is a distance function. For correct triples, the score function is expected to be as small as possible to ensure the correct translation. For incorrect triples, the score function is expected to be larger to increase the irrationality of the translation process.

To sum up, the ERDERP captures the complex relationships through two steps. First, it develops a relation hyperplane for each relation. Then, multiple relations between a pair of entities are projected into the relation hyperplane to ensure that the projection of the relations can be the same as each other and remain different as a whole. Therefore, ERDERP can effectively extract the same features and different features of the multiple relations. Moreover, the embedding representations for multiple relations between a pair of entities through ERDERP are different. ERDERP increases the distinguishability of relations and solves the diversity of relations. Second, drawing on the projection mode of TransH, ERDERP develops a relation projection hyperplane for each relation projection. Then, the head entity and tail entity are projected into the relation projection hyperplane, and the translation process is completed on the relation projection hyperplane with the relation projection as the translation. Therefore, ERDERP can effectively extract the same features and different features of the multiple entities. Moreover, the embedding representations for multiple head entities of N-1-1 and multiple tail entities of 1-1-N through ERDERP are different. ERDERP projects entities into the relation projection hyperplane. ERDERP increases the distinguishability of entities and solves the complexity and diversity of entities. Therefore, ERDERP can effectively solve complex relationships in a knowledge graph.

The traditional translation models solved the complexity and diversity of entities by projecting head entities and tail entities, but did not directly solve the diversity of relations themselves, making it difficult to distinguish multiple relationships in the learned embedded representation. The diversity of unsolved relations makes it difficult to improve the accuracy of relation prediction, which further affects the improvement of the accuracy of entity prediction. Compared the traditional translation models, ERDERP can effectively solve the diversity of relations themselves. Theoretically, ERDERP can effectively solve complex relationships in a knowledge graph. Section 3.2 theoretically verifies that ERDERP can effectively solve antisymmetry.

3.2. Theoretical Justifications of the Inference Patterns

In this section, the performance of ERDERP in solving the antisymmetric relationship is verified in theory. The theoretical proof of the antisymmetry of ERDERP is as follows.

Let $E$ represents the entity set, $h,t$ represents the embedding representation of the head entity and tail entity, respectively, and $h$ and $t$ are n-dimensional row vectors by ERDERP. For relation $r$, if there exits $r\left(h,t\right)$, then $(h-w_r^{T}h{w}_r)+(r-s_r^{T}r{s}_r)=(t-w_r^{T}t{w}_r)$ can be obtained by the score function of ERDERP.

Further, the objective is to prove that $(t-w_r^{T}t{w}_r)+(r-s_r^{T}r{s}_r)\ne (h-w_r^{T}h{w}_r)$ when $h\ne t$ if there exits $r\left(h,t\right)$. According to $(h-w_r^{T}h{w}_r)+(r-s_r^{T}r{s}_r)=(t-w_r^{T}t{w}_r)$, Equation (3) can be obtained.

$$\begin{array}{c}(h-w_r^{T}h{w}_r)+(r-s_r^{T}r{s}_r)=(t-w_r^{T}t{w}_r)\\ \Updownarrow \\ (h-w_r^{T}h{w}_r)+(r-s_r^{T}r{s}_r)+(t-w_r^{T}t{w}_r)-(h-w_r^{T}h{w}_r)=(t-w_r^{T}t{w}_r)+(t-w_r^{T}t{w}_r)-(h-w_r^{T}h{w}_r)\\ \Updownarrow \\ (t-w_r^{T}t{w}_r)+(r-s_r^{T}r{s}_r)=2(t-w_r^{T}t{w}_r)-(h-w_r^{T}h{w}_r)\end{array}$$

(3)

Moreover, assume equation $(t-w_r^{T}t{w}_r)+(r-s_r^{T}r{s}_r)=(h-w_r^{T}h{w}_r)$ holds. Then, Equation (4) can be obtained.

$$\begin{array}{c}(t-w_r^{T}t{w}_r)=(h-w_r^{T}h{w}_r)\\ \Updownarrow \\ t-h=w_r^{T}t{w}_r-w_r^{T}h{w}_r\\ \Updownarrow \\ t-h=w_r^T\left(t-h\right)t{w}_r\\ \Updownarrow \\ w_r{w}_r^T\left(t-h\right)=0\end{array}$$

(4)

Let us assume that $w_r={\left(a_1,a_2,\cdots ,a_n\right)}^T$, then $w_r{w}_r^T$ can be expressed as follows.

$$\begin{array}{ccccc}{a}_1^2& a_1{a}_2& \cdots & a_1{a}_{n-1}& a_1{a}_n\\ a_2{a}_1& a_2^2& \cdots & a_2{a}_{n-1}& a_2{a}_n\\ ⋮& ⋮& \ddots & ⋮& ⋮\\ a_{n-1}{a}_1& a_{n-1}{a}_2& \cdots & a_{n-1}^2& a_{n-1}{a}_n\\ a_n{a}_1& a_n{a}_2& \cdots & a_n{a}_{n-1}& a_n^2\end{array}$$

Equation (4) indicates that $w_r{w}_r^T\left(t-h\right)=0$ holds for $\forall t,h$, i.e., $\forall t-h$, $w_r{w}_r^T\left(t-h\right)=0$. Then, $w_r={\left(a_1,a_2,\cdots ,a_n\right)}^T={\left(0,0,\cdots ,0\right)}^T$ is easily verified. Since $w_r$ is the normal vector of the hyperplane corresponding to the projection of relation r, $w_r\ne {\left(0,0,\cdots ,0\right)}^T$ holds. Therefore, the assumption of $(t-w_r^{T}t{w}_r)+(r-s_r^{T}r{s}_r)=(h-w_r^{T}h{w}_r)$ does not hold. Then, if $(h-w_r^{T}h{w}_r)+(r-s_r^{T}r{s}_r)=(t-w_r^{T}t{w}_r)$ and $h\ne t$, $(t-w_r^{T}t{w}_r)+(r-s_r^{T}r{s}_r)\ne (h-w_r^{T}h{w}_r)$ holds, i.e., ERDERP can effectively solve the antisymmetric relationship.

3.3. Model Training

In the training of ERDERP, a margin-based objective function is proposed by introducing the maximum interval method to accurately extract semantic information of entities and relations, as shown in Equation (5).

$$L={\displaystyle \sum _{\left(h,r,t\right)\in \mathsf{\Delta }}{\displaystyle \sum _{\left(h^{\prime },r,t^{\prime }\right)\in {\mathsf{\Delta }}_{\left(h,r,t\right)}^{\prime }}\max \left(f_r\left(h,t\right)+\gamma -f_r\left(h^{\prime },t^{\prime }\right),0\right)}}$$

(5)

where $\max \left(x,0\right)$ denotes the maximum between $x$ and 0, $f_r\left(h,t\right)$ denotes the value of the correct triplet (h, r, t) using the score function, $f_r\left(h^{\prime },t^{\prime }\right)$ denotes the value of the incorrect triplet $(h^{\prime },r,t^{\prime })$ using the score function, $\gamma$ is a margin hyperparameter and $\gamma >0$, $\mathsf{\Delta }$ is the set of correct triples, and ${\mathsf{\Delta }}_{\left(h,r,t\right)}^{\prime }$ is the set of incorrect triples, which is generated by $\left(h,r,t\right)$. ${\mathsf{\Delta }}_{\left(h,r,t\right)}^{\prime }$ is shown in Equation (6).

$${\mathsf{\Delta }}_{\left(h,r,t\right)}^{\prime }=\left\{\left(h^{\prime },r,t\right)|h^{\prime }\in E\right\}\cup \left\{\left(h,r,t^{\prime }\right)|t^{\prime }\in E\right\}$$

(6)

For a correct triplet, the corresponding incorrect triplets set is generated in two steps. (1) Randomly select an entity from the entity set to replace the head entity of the correct triplet. (2) Randomly select an entity from the entity set to replace the tail entity of the correct triplet. In fact, the head entity and tail entity should be replaced with different probabilities to avoid the situation that the generated incorrect triples are the correct triples in the knowledge graph. On the one hand, one relation generally corresponds to multiple head entities and multiple tail entities, and on the other hand, the number of head entities and tail entities corresponding to a relation are generally different. TransH shows that the head entity tended to give more chance to be replaced if the relation is one-to-many and the tail entity is given more chance to be replaced if the relation is many-to-one.

Therefore, Bernoulli distribution in TransH is adopted to generate the incorrect triples. For a correct triple (h, r, t), the incorrect triples are generated in three steps. (1) Count all triples containing r and calculate the average number of tail entities per head entity, denoted as tph. (2) Count all triples containing r and calculate the average number of head entities per tail entity, denoted as hpt. (3) Replace the head entity in the triplet (h, r, t) with a probability of $\frac{tph}{tph+hpt}$ or replace the tail entity in the triplet (h, r, t) with a probability of $\frac{hpt}{tph+hpt}$. This method of generating incorrect triples improves the quality of incorrect triples and provides support for improving the knowledge representation ability of the translation model.

Through the above analysis, the training process of ERDERP is shown in Figure 4.

4. Experiments

In this section, the proposed ERDERP model is compared with several comparison models in link prediction (i.e., entity prediction and relation prediction). In the experiments, the PyTorch framework and Adam optimization method are adopted to train the translation models. Section 4.1 introduces the datasets used in the experiments. Section 4.2 develops the model evaluation. Section 4.3 details the experimental design and analysis of experimental results.

4.1. Experimental Datasets

Four public datasets of knowledge graph completion were used in the experiment, including two subsets of WordNet, WN18 [27] and WN18RR [43], and two subsets of Freebase—FB15K [27] and FB15K-237 [44]. Detailed information of the five datasets is shown in

Table 3. Details of the experimental data.

Dataset	Relation	Entity	Triples
			Train	Valid	Test
WN18	18	40,943	141,442	5000	5000
WN18RR	11	40,943	86,835	3034	3134
FB15K	1345	14,951	483,142	50,000	59,071
FB15K-237	237	14,541	271,115	17,535	20,466

, including the size of relations, the size of entities, the size of training samples, the size of validation samples, and the size of test samples.

4.2. Model Evaluation

Two aspects are considered to evaluate the presentation capability of the models. One is to evaluate the entity prediction accuracy, i.e., predict tail entity or head entity for a given triplet (h, r, ?) or (?, r, t). The other is to evaluate the relation prediction accuracy, i.e., predict relation for a given triplet (h, ?, t). Mean Rank and Hits@10 are adopted as the metrics of entity prediction accuracy. Mean Rank and Hits@N are adopted as the metrics of relation prediction accuracy. Note that Hits@10 is a special case of Hits@N, where N = 10.

Mean Rank: The following describes the calculation process of Mean Rank in detail with the prediction tail entity as an example. First, for a test triple $\left(h,r,t\right)$, an incorrect triplet set is obtained by replacing tail entity with all entities in the knowledge graph. Second, the score of each incorrect triplet is obtained by the score function $f_r\left(h,t\right)$. Third, the score sequence is obtained by sorting the scores in ascending order. Finally, the position rank of the tail entity in $\left(h,r,t\right)$ in the score sequence is extracted. In the same way, the position rank of head entity in $\left(h,r,t\right)$ is obtained. Then, the average rank of the head entity and tail entity of all test triples is taken as the Entity Mean Rank, as shown in Equation (7). Similarly, the average rank of the relation of all test triples is taken as the Relation Mean Rank, as shown in Equation (8).

$$\mathit{Entity} Mean Rank=\frac{{\displaystyle \sum _{\left(h,r,t\right)\in Tt}Rank\left(\left(t,f_r\left(h,t\right)\right),S_t\right)}+{\displaystyle \sum _{\left(h,r,t\right)\in Tt}Rank\left(\left(h,f_r\left(h,t\right)\right),S_h\right)}}{2\cdot \left|Tt\right|}$$

(7)

$$Relation Mean Rank=\frac{{\displaystyle \sum _{\left(h,r,t\right)\in Tt}Rank\left(\left(r,f_r\left(h,t\right)\right),S_r\right)}}{\left|Tt\right|}$$

(8)

where $Rank\left(x,X\right)$ denotes the position of the element $x$ in $X$. $S_t$ is the score sequence of the tail entity in $\left(h,r,t\right)$, $S_h$ is the score sequence of the head entity in $\left(h,r,t\right)$, and $S_r$ is the score sequence of the relation in $\left(h,r,t\right)$, which are shown in Equation (9). $Tt$ denotes the test triple set. $\left|Tt\right|$ is the size of the test triple set.

$$S_t=\mathrm{sort}\left\{\left(t^{\prime },f_r\left(h,t^{\prime }\right)\right)|t^{\prime }\in E\right\}, S_h=\mathrm{sort}\left\{\left(h^{\prime },f_r\left(h^{\prime },t\right)\right)|h^{\prime }\in E\right\}, S_r=\mathrm{sort}\left\{\left(r^{\prime },f_{r^{‘}}\left(h,t\right)\right)|r^{\prime }\in R\right\}$$

(9)

where $\mathrm{sort}\left\{{\left(x,y\right)}_n\right\}$ illustrates that the set is sorted in ascending order by the value of $y$. $E$ is the entity set of the knowledge graph, and $R$ is the relation set of the knowledge graph.

Hits@N: For the above ascending sequence, the count has 1 added if the correct tail entity/head entity/relation of the test triplet is in the top N. Then, Hits@N is given by averaging the counts of all test triplets, which is shown in Equations (10) and (11).

$$Entity Hits@N=\frac{{\displaystyle \sum _{\left(h,r,t\right)\in Tt}{s}_N\left(Rank\left(\left(t,f_r\left(h,t\right)\right),S_t\right)\right)}+{\displaystyle \sum _{\left(h,r,t\right)\in Tt}{s}_N\left(Rank\left(\left(h,f_r\left(h,t\right)\right),S_h\right)\right)}}{2\cdot \left|Tt\right|}$$

(10)

$$\mathit{Relation} Hits@N=\frac{{\displaystyle \sum _{\left(h,r,t\right)\in Tt}{s}_N\left(Rank\left(\left(r,f_r\left(h,t\right)\right),S_r\right)\right)}}{\left|Tt\right|}$$

(11)

where $s_N$ is a counting function, as shown in Equation (12). In addition, the other parameters refer to Equations (8) and (9).

$$s_N\left(Rank\left(x,X\right)\right)=\{\begin{array}{c}\begin{array}{cc}1, & if Rank\left(x,X\right)\le N\end{array}\\ \begin{array}{cc}0, & if Rank\left(x,X\right)>N\end{array}\end{array}$$

(12)

where $Rank\left(x,X\right)$ denotes the position of the element $x$ in $X$, and N is a positive integer.

The above calculation method of Mean Rank and Hits@N is called the “raw” setting. In fact, an incorrect triplet generated may be a triplet in the knowledge graph, so it is not wrong to rank it before the original correct triplet. To eliminate this factor, those incorrect triplets, which exist in either training, validation, or test sets, are removed before obtaining the rank of each test triple. The above setting is called “Filt.” The score function $f_r\left(h,t\right)$ is essentially a distance function. Thus, a lower $f_r\left(h,t\right)$ illustrates that the translation model is more effective, and vice versa, i.e., the translation model with lower Mean Rank and higher Hits@N is more effective.

4.3. Link Prediction Experiments

The link prediction experiments consist of three parts. The first part is the parameter range and optimal parameter selection for the proposed model, the second part is the analysis of the experimental results of entity prediction, and the last part is the analysis of the experimental results of relation prediction.

4.3.1. Optimal Parameter Selection

The optimal optimization selection is divided two parts. The first part is the optimal parameter selection for ERDERP in the entity prediction based on WN18 and FB15K. The second part is the optimal parameter selection for ERDERP, TransD and TransH in the relation prediction based on WN18RR, FB15K and FB15K-237. The two parts are described in detail below.

In the optimal parameter selection of entity prediction, the learning rate ($lr$), the margin ($\gamma$), the embedding dimension ($k$), the batch size ($B$), and the epoch ($e$) are the hyperparameters that ERDERP needs to be optimized. The hyperparameter optimization spaces of ERDERP on WN18 and FB15K are shown in

Table 4. The hyperparameter optimization spaces of ERDERP on WN18 and FB15K.

Hyperparameters	$\mathit{l}\mathit{r}$	$\mathit{\gamma }$	$\mathit{k}$	$\mathit{B}$	$\mathit{e}$
WN18	{0.01, 0.001}	{1, 2, 4, 6}	{50, 100}	{1200, 2400, 4800}	500
FB15K	{0.01, 0.001}	{1, 1.5, 2, 2.5}	{50, 100}	{1200, 2400, 4800}	500

.

Through multiple experiments, the optimal parameters of ERDERP on WN18 and FB15K are shown in

Table 5. The optimal hyperparameters of ERDERP on WN18 and FB15K.

Hyperparameters	$\mathit{l}\mathit{r}$	$\mathit{\gamma }$	$\mathit{k}$	$\mathit{B}$	$\mathit{e}$
WN18	0.001	2	100	1200	500
FB15K	0.0001	2.5	100	2400	500

.

In the optimal parameter selection of relation prediction, the relation prediction accuracy of TransH and ERDERP are mainly compared in relation prediction, and the PyTorch framework and Adam algorithm are adopted to train ERDERP, TransD and TransH. The datasets of WN18RR, FB15K and FB15K-237 are adopted as the experimental data. The learning rate ($lr$), the margin ($\gamma$), the embedding dimension ($k$), the weight ($C$), a hyperparameter to measure the importance of the soft constraints in TransH), the batch size ($B$) and the epoch ($e$) are the hyperparameters that TransH, TransD and ERDERP needs to be optimized. The hyperparameter optimization spaces of the three datasets are shown in

Table 6. Hyperparameter optimization spaces of ERDERP and TransH of the four datasets.

Hyperparameters	$\mathit{l}\mathit{r}$	$\mathit{\gamma }$	$\mathit{k}$	$\mathit{C}$	$\mathit{B}$	$\mathit{e}$
WN18RR	{0.01, 0.001}	{0.5, 1, 2, 4}	{50, 100}	{0.25, 0.5, 1}	{1200, 2400, 4800}	500
FB15K	{0.01, 0.001}	{1, 1.5, 2, 2.5}	{50, 100}	{0.25, 0.5, 1}	{1200, 2400, 4800}	500
FB15K-237	{0.01, 0.001}	{0.5, 1, 2, 4}	{50, 100}	{0.25, 0.5, 1}	{1200, 2400, 4800}	500

.

Through multiple experiments, the optimal parameters of ERDERP, TransD and TransH of the three datasets are shown in

Table 7. Optimal configurations of ERDERP, TransD and TransH of each dataset.

Hyperparameters		$\mathit{l}\mathit{r}$	$\mathit{\gamma }$	$\mathit{k}$	$\mathit{C}$	$\mathit{B}$	$\mathit{e}$
WN18RR	TransH	0.001	1	50	0.5	2400	500
	TransD	0.01	4	100	-	2400	500
	ERDERP	0.001	4	100	-	4800	500
FB15K	TransH	0.001	2	50	0.25	4800	500
	TransD	0.01	2.5	100	-	2400	500
	ERDERP	0.001	2.5	100	-	1200	500
FB15K-237	TransH	0.001	2	50	0.25	4800	500
	TransD	0.001	4	100	-	4800	500
	ERDERP	0.01	4	100	-	4800	500

.

4.3.2. Analysis of the Experimental Results of Entity Prediction

WN18 and FB15K are adopted as the experimental data in the entity prediction experiments. To validate the link prediction performance of the proposed ERDERP, unstructured [21], RESCAL [25], SE [19], SME (Linear) [21], SME (Bilinear) [21], LFM [23], TransE [27], TransH [28], TransR [29], CTransR [29] and TransD [30] models are adopted as the comparison models. To quantify the accuracy of link prediction, the Raw Entity Mean Rank, Filt Entity Mean Rank, Raw Entity Hits@10 and Filt Entity Hits@10 are adopted in the entity prediction experiments.

The optimal hyperparameters in Table 5 are adopted to train ERDERP. The entity prediction experimental results of the ERDERP and the comparison models on WN18 and FB15K are shown in

Table 8. Entity prediction results.

Dataset	WN18				FB15k
Metric	Mean Rank		Hits@10		Mean Rank		Hits@10
	Raw.	Filt	Raw	Filt.	Raw.	Filt	Raw.	Filt
Unstructured	315	304	35.3	38.2	1074	979	4.5	6.3
RESCAL	1180	1163	37.2	52.8	828	683	28.4	44.1
SE	1011	985	68.5	80.5	273	162	28.8	39.8
SME	545	533	65.1	74.1	274	154	30.7	40.8
SME	526	509	54.7	61.3	284	158	31.3	41.3
LFM	469	456	71.4	81.6	283	164	26.0	33.1
TransE	263	251	75.4	89.2	243	125	34.9	47.1
TransH	400.8	388	73	82.3	212	87	45.7	64.4
TransR	238	225	79.8	92.0	198	77	48.2	68.7
CTransR	231	218	79.4	92.3	199	75	48.4	70.2
TransD	224	212	79.6	92.2	194	91	53.4	77.3
ERDERP	258	246	79.9	93.2	189	54	49.1	71.1

.

As shown in Table 8, the experimental results show that compared with the unstructured, RESCAL, SE, SME (linear), SME (bilinear), LFM, TransE, TransH, TransR, and CTransR models, the proposed ERDERP achieves the best entity prediction accuracy overall on WN18 and FB15K and its performance is equivalent to that of TransD. Moreover, the parameter complexity of TransD is much higher than that of ERDERP, as shown in Table 1. Therefore, the proposed ERDERP outperforms TransD in entity prediction of WN18 and FB15K, i.e., the proposed ERDERP is more effective in entity prediction of WN18 and FB15K.

On WN18, the entity prediction accuracy of ERDERP is significantly better than that of TransH and other the non-translation models. Compared with TransR, CTransR, and TransD, ERDERP achieves the best results at Hits@10, which indicates that the head/tail entity of most test triplets are in the top 10 in the head/tail entity scoring sequence. However, ERDERP performs poorly at Mean Rank, which may be because the head/tail entities of a few test triplets are in a larger position in the head/tail entity scoring sequence. ERDERP is improved on TransH. Moreover, the experimental results illustrate that compared with TransH, ERDERP can effectively realize entity prediction in the dataset with sample relations. For example, compared with TransH, Raw Mean Rank and Filt Mean Rank increased by 142.8 and 122. Raw Hits@10 and Filt Hits@10 increased by 6.9% and 10.9%. The results show that by considering the diversity of the relations, ERDERP can effectively handle simple relations, so as to effectively complete entity prediction.

On FB15K, ERDERP outperforms unstructured, RESCAL, SE, SME (linear), SME (bilinear), LFM, TransE, TransH, TransR and CtransR in entity prediction accuracy and its entity prediction accuracy is equivalent to that of TransD. ERDERP outperforms TransD at Mean Rank and TransD outperforms ERDERP at Hits@10. Overall, ERDERP achieves the same performance as TransD on a few parameters. Compared with TransH, TransR and CtransR, ERDERP achieves great improvement in all aspects, especially compared with TransH. Raw Mean Rank and Filt Mean Rank increased by 23 and 33. Raw Hits@10 and Filt Hits@10 increased by 3.4% and 6.7%. The experimental results show that the proposed ERDERP model can effectively realize entity prediction on the dataset with complex relationships, i.e., ERDERP is effective in solving the complex relationship problem of the knowledge graph.

On the one hand, there is a lack of effective methods to completely separate datasets based on the refined complex relations in this paper. On the other hand, the traditional complex relationships can be regarded as the complexity and diversity of entities. Therefore, 1-N, N-1, N-N are mainly adopted to verify the ability of ERDERP in extracting the features of the complexity and diversity of entities. Then, ERDERP and the comparison models are respectively adopted to predict head entity and tail entity on 1-to-1, 1-to-N, N-to-1 and N-to-N of FB15K. The experimental results are shown in

Table 9. Decomposition experimental results for complex relationships.

Tasks	Predict Head (Filt Hits@10)				Predict Tail (Filt Hits@10)
Relation Category	1-to-1	1-to-N	N-to-1	N-to-N	1-to-1	1-to-N	N-to-1	N-to-N
Unstructured	34.5	2.5	6.1	6.6	34.3	4.2	1.9	6.6
SE	35.6	62.6	17.2	37.5	34.9	14.6	68.3	41.3
SME(Linear)	35.1	53.7	19.0	40.3	32.7	14.9	61.6	43.3
SME(Bilinear)	30.9	69.6	19.9	38.6	28.2	13.1	76.0	41.8
TransE	43.7	65.7	18.2	47.2	43.7	19.7	66.7	50.0
TransH	66.8	87.6	28.7	64.5	65.5	39.8	83.3	67.2
TransR	78.8	89.2	34.1	69.2	79.2	37.4	90.4	72.1
ERDERP	78.1	86.3	49.5	70.3	79.1	51.7	85.6	73.3

.

Table 9 illustrates that compared with unstructured, SE, SME (linear), SME (bilinear), TransE, and TransH, ERDERP achieves the best prediction accuracy of head entity and tail entity in the complex relationships of a knowledge graph. Compared with TransR, ERDERP is more effective on N-to-1 of the head entity prediction and 1-to-N of the tail entity prediction, with a significant increase of 15.4% and 11.9%, respectively. In other aspects, the effects of TransR and ERDERP are similar. Therefore, ERDERP outperforms TransR in entity prediction, as shown in Table 8. Moreover, compared with TransH, the head entity prediction accuracy of ERDERP on 1-to-1, 1-to-N, N-to-1, and N-to-N increased by 11.3%, −1.3%, 20.8%, and 5.8%, respectively, while the tail entity prediction accuracy of ERDERP on 1-to-1, 1-to-N, N-to-1, and N-to-N was 13.6%, 11.9%, 10.5%, 2.3%, and 6.1%. The experimental results show that ERDERP can effectively improve entity prediction in datasets with simple relationships and solve the complex relationships of knowledge graphs.

The experimental results of entity prediction illustrate that it is believed that the entities and relations embedding vectors of ERDERP contain rich semantic information by considering the diversity of the relations themselves and that ERDERP has a significant effect on improving entity prediction accuracy. The results show that ERDERP can effectively solve the simple relationships and complex relationships of knowledge graphs. On the other hand, to verify that the relation embedding vector of ERDERP contains the diversity information of the relation, ERDERP, TransD and TransH are used in the relation prediction experiments on various datasets.

4.3.3. Analysis of the Experimental Results of Relation Prediction

The hyperparameters in Table 7 were respectively used to train TransH, TransD and ERDERP to carry out comparative experiments of relation prediction on WN18RR, FB15K and FB15K-237. In addition, Filt Mean Rank, Filt Hits@1, Filt Hits@3 and Filt Hits@10 were adopted as the evaluation indices for the accuracy of relation prediction. The experimental results of relation prediction are shown in

Table 10. Relation prediction results of TransH, TransD, and ERDERP on three datasets.

Dataset		WN18RR	FB15K	FB15K-237
TransH	Mean Rank	5	111	28
	Hits@1	43.2	40.7	43.4
	Hits@3	54.0	55.6	50.5
	Hits@10	63.8	66.4	57.0
TransD	Mean Rank	3	5.4	2.3
	Hits@1	38.2	72.8	84.9
	Hits@3	69.8	86.4	93.5
	Hits@10	99.9	93.6	96.6
ERDERP	Mean Rank	2	4.6	2.1
	Hits@1	43.7	74.9	86.9
	Hits@3	78.3	87.8	94.2
	Hits@10	99.9	94.3	96.7

.

Table 10 shows that in terms of relation prediction, compared with TransH and TransD, ERDERP achieves the best results in all evaluation indicators of the three datasets. On the three datasets, WN18RR contains 11 relations, FB15K 1345 relations and FB15K-237 237 relations. Compared with TransH, the more relations the dataset contains, the more the relation prediction accuracy is improved by ERDERP at Mean Rank. For example, Mean Rank increases by 3 on WN18RR, by 26 on FB15K-237, and 106 on FB15K. Compared with TransD, ERDERP improves the Mean Rank by 1/0.2/0.8 on WN18RR/FB15K-237/FB15K.

In terms of Hits@N, the relation prediction accuracy of ERDERP on WN18RR, FB15K and FB15K-237 is significantly higher than that of TransH and TransD, which may be because TransH and TransD did not directly solve the complexity of the relations, which makes it difficult to extract the semantic features of the relations. Compared with TransH, Hits@1/Hits@3/Hits@10 of ERDERP respectively increased by 0.7%/24.3%/36.1% on WN18RR, 34.2%/32.2%/27.9% on FB15K, and 43.5%/43.7%/39.7 on FB15K-237. Compared with TransD, Hits@1/Hits@3/Hits@10 of ERDERP respectively increased by 5.5%/8.5%/0% on WN18RR, 2.1%/1.4%/0.7% on FB15K, and 2%/0.7%/0.1% on FB15K-237.

The experimental results of relation prediction show that compared with TransH and TransD, ERDERP shows excellent performance on datasets with fewer and more relations. Therefore, ERDERP can effectively realize relation prediction because it can directly handle the diversity of relations.

The above experimental results of link prediction show that ERDERP significantly improves relation prediction accuracy and entity prediction accuracy by considering the diversity of the relations and allocating a relation subspace to each relation. ERDERP can effectively realize the link prediction of datasets with fewer and more relations, i.e., ERDERP can effectively solve the complex relationships of knowledge graphs. The relation prediction accuracy of ERDERP is significantly higher than that of TransH on datasets with more relations. In real life, the relations between entities are more complex and diverse. Therefore, the proposed ERDERP can provide technical support for actual link prediction.

5. Conclusions

Currently, TransE and other translation models are widely used in knowledge graph completion technology. To improve the accuracy of entity prediction, the traditional translation models are oriented to solve the complex relationships of knowledge graphs, such as one-to-many, many-to-one, and many-to-many. However, few researchers consider the diversity of the relations and study relation prediction. Therefore, we studied the diversity of the relations, and the diversity of the relations was integrated into the complex relationships of the knowledge graph. Then, the complex relationships of the knowledge graph were refined as one-to-one-to-many, many-to-one-to-one, one-to-many-to-one, many-to-one-to-many, many-to-many-to-one, one-to-many-to-many, and many-to-many-to-many. By presenting the diversity of the relations themselves independently, it was found that with traditional models, without considering the diversity of relations themselves, it is difficult to effectively extract the semantic features of the relations. Then, to effectively extract the semantic information of entities and relations and improve the accuracy of link prediction, we studied the translation model, i.e., analyzing the complex relationships of the knowledge graph and TransE. Then, an entity and relation double embedding on relation hyperplanes and relation projection hyperplanes (ERDERP) model was proposed. First, ERDERP establishes a relation hyperplane for each relation to solve the diversity of relations and increase the distinguishability between multirelations. Second, ERDERP establishes a relation projection hyperplane for each relation projection to solve the complexity and diversity of entities and increase the distinguishability between multiple entities. Thus, the entity and relation embedding vectors with rich semantic information can be obtained. Moreover, it is theoretically proved that ERDERP can solve antisymmetry. Finally, ERDERP is compared with several typical knowledge graph completion models on several typical datasets. The experimental results show that the proposed ERDERP significantly improves the link prediction accuracy, especially in relation prediction. For example, compared with TransH, Hits@1 of ERDERP increased by at least 30% on FB15K and FB15K-237. The results verify that considering the diversity of the relations has a significant effect in link prediction.

ERDERP has strong extensibility. Other translation models can also solve the diversity of the relations by allocating a subspace to each relation. Therefore, combining the ideas proposed in this paper with other translation models to improve entity prediction accuracy and relation prediction accuracy is our future research direction. In addition, providing theoretical justifications of other inference patterns for ERDERP would also be worth researching in the future.