Geometry Interaction Embeddings for Interpolation Temporal Knowledge Graph Completion

Abstract

Knowledge graphs (KGs) have become a cornerstone for structuring vast amounts of information, enabling sophisticated AI applications across domains. The progression to temporal knowledge graphs (TKGs) introduces time as an essential dimension, allowing for a dynamic representation of entity relationships. Despite their potential, TKGs often suffer from incompleteness, necessitating the development of temporal knowledge graph completion (TKGC) techniques. These methods, particularly focusing on interpolation within the known timeframe, aim to infer missing temporal facts and enhance the predictive capabilities of TKGs. The prevalent reliance on Euclidean space modeling in TKGC methods presents challenges in capturing the complex, hierarchical, and time-varying nature of TKGs. To overcome these limitations, we introduced the attention-based geometry interaction embedding (ATGIE) method, a novel approach that leverages the strengths of multiple geometric spaces, i.e., Euclidean, hyperbolic, and hypersphere, to model the intricacies of TKGs more effectively. ATGIE employs an attention mechanism to dynamically weigh the contributions of different geometric spaces, allowing it to adaptively form reliable spatial structures based on interactive geometric information. This multi-space modeling not only captures the diverse relationships within TKGs but also facilitates a nuanced understanding of how entities and their relationships evolve over time. Through extensive experiments, we demonstrate ATGIE’s superiority in TKGC tasks, showcasing its improvement over existing methods, robustness to noise, and sensitivity to temporal dynamics. The results highlight ATGIE’s potential to advance the state-of-the-art in TKGC, offering a promising direction for research and application in the field.

1. Introduction

Knowledge graphs (KGs) are a revolutionary framework for the systematic structuring and utilization of information, where data are encapsulated within a network of nodes representing entities and edges symbolizing the relationships between them [1,2,3]. These graphs empower Artificial Intelligence (AI) systems to comprehend and infer the intricate connections spanning a multitude of fields, ranging from healthcare and financial services to social media platforms and further afield. The graph-based representation of KGs allows for the development of AI applications that are more context-sensitive and sophisticated [4,5], leading to advancements in areas such as natural language understanding and personalized recommendation algorithms. Expanding on the groundbreaking work of KGs, temporal knowledge graphs (TKGs) incorporate the critical element of time [6]. In TKGs, data are articulated as a quadruple, denoted as $(s,r,o,t)$, indicating that a relationship r exists between a subject entity s and an object entity o at a specific time point t. This temporal component transcends the limitations of static KGs by embedding time-varying interactions among entities, thereby equipping AI systems with the ability to perceive the evolution of relationships over time. The integration of temporality is indispensable for applications that necessitate a dynamic representation of knowledge, such as forecasting future occurrences, examining patterns over time, and tracing the progression of ideas. TKGs, by capturing the temporal flux of entities and their relationships, mark a significant advancement towards crafting AI systems that are more anticipatory, agile, and perceptive.

Although TKGs are prevalent, their frequent incompleteness poses a considerable challenge, primarily due to the constraints of human understanding. This limitation has motivated the advancement of temporal knowledge graph completion (TKGC) techniques, which are designed to deduce missing factual connections from the existing corpus of knowledge. The body of work on TKGC is rich with strategies, predominantly categorized into interpolation and extrapolation methods, as elaborated in various scholarly articles for interpolation [7,8,9,10] and extrapolation [11,12,13]. In the context of a TKG that includes data from time $t_0$ to time $t_{\mathrm{T}}$, the extrapolation method endeavors to foresee occurrences that could arise after $t_{\mathrm{T}}$, in contrast to interpolation, which aims to uncover omissions within this period. However, the reliability and applicability of extrapolation forecasts in practical scenarios are often questionable [14]. In light of these issues, our research has concentrated on the interpolation facet of TKGC. This targeted effort is intended to fill the voids in the current timeline of TKGs, with the goal of augmenting their precision and thoroughness to provide more reliable and utilizable knowledge. Our specific objective is to address queries of the form $(s,r,?,t)$ and $(?,r,o,t)$ within a TKG that is timestamped from $t_0$ to $t_{\mathrm{T}}$. In these queries, the placeholder “?” denotes an entity whose identity is to be ascertained, predicated on its association with another entity and the particular time instant within the stipulated temporal frame.

Most existing methods for TKGC rely primarily on Euclidean space modeling for knowledge representation and reasoning. However, the unique characteristics of temporal knowledge graphs pose challenges that can limit the effectiveness of Euclidean space modeling [15,16]. The interconnectedness and evolving nature of TKGs often involve complex hierarchical structures and long-range dependencies. Euclidean space modeling struggles to effectively capture these intricate relationships [17,18]. Additionally, Euclidean spaces may not be well-suited for representing the underlying geometry of the data in TKGs, further hindering their ability to capture the dynamics of these graphs [15]. To overcome these limitations, researchers have explored the use of non-Euclidean spaces, such as hyperbolic and hypersphere spaces [16,19]. By leveraging these alternative geometries, these approaches offer potential solutions to the challenges posed by Euclidean space modeling. Non-Euclidean spaces can better capture hierarchical structures, handle long-range dependencies, and represent the intrinsic geometry of the data more effectively [18,20]. However, most of these approaches focus on modeling a single geometric space and do not fully address the heterogeneity that exists among different spaces within a temporal knowledge graph [18,21]. This limitation restricts their ability to perform pattern reasoning across multiple spaces. In other words, they may not adequately capture the complex interactions and dependencies that span different types of entities or relationships in a heterogeneous TKG. Therefore, although non-Euclidean space modeling presents a promising solution to the limitations of Euclidean space modeling, there is still ongoing research to explore the full potential of multiple geometries in TKGs.

To this end, we introduce a novel attention-based geometry interaction embedding (ATGIE) method for temporal knowledge graph completion. Our ATGIE focuses on developing a geometric interaction embedding model specifically tailored for TKGC. Unlike previous approaches that primarily targeted a single geometric space, this novel method aims to obtain interactive embeddings across multiple geometric spaces, thereby addressing the heterogeneity and complexity inherent in TKGs. One key aspect of this approach is the utilization of various geometric spaces, including euclidean, hyperbolic, and spherical spaces, to model the structural and temporal dependencies within the graphs. By leveraging multiple geometric spaces, the approach can capture the diverse and intricate relationships present in temporal knowledge graphs more effectively. By combining the strengths of different geometric spaces, the approach enhances its ability to capture long-range dependencies, hierarchical structures, and temporal dynamics within the graph, leading to more robust and informed predictions and inferences. Furthermore, by incorporating interactive embeddings across multiple geometric spaces, the approach facilitates a more nuanced understanding of how entities and relationships evolve over time. This comprehensive modeling framework has the potential to significantly advance the field of temporal knowledge graph completion by enabling more accurate predictions, improved analysis of temporal dependencies, and enhanced capabilities for reasoning across heterogeneous spaces.

We summarize our main contributions as follows:

• We introduce a novel attention-based geometry interaction embedding (ATGIE) method for temporal knowledge graph completion, which leverages multiple geometric spaces to capture the intricate structural and temporal dependencies present in temporal knowledge graphs.
• We take analysis on the interactive nature between different geometric spaces in order to tackle the inherent heterogeneity and complexity within TKGs.
• Extensive experiments demonstrated the promising capacity of our ATGIE from four aspects, i.e., superiority, improvement, robustness, and sensitivity.

2. Related Work

2.1. Static KG Completion

In the realm of static knowledge graph (KG) completion, a plethora of methods has emerged, categorized into three distinct types. The initial type encompasses translational models that employ vector addition to represent entity relationships, with TransE [22] and its derivatives [23,24,25] being prominent examples. The second type involves semantic matching models, which utilize triangular norms to evaluate the likelihood of relationships; DistMult [26] and ComPlEx [27] are notable for their use of bilinear and complex space asymmetry functions, respectively. The third type focuses on embedding-based reasoning, with models referenced in [28,29] striving to learn distributed representations for KG entities and relations. Extensions to graph convolutional networks (GCNs), such as R-GCN [30] and CompGCN [31], have also been developed for enhanced relation-awareness in KGs. Despite their success, these models predominantly address static reasoning, often overlooking the temporal dynamics present in knowledge graphs.

2.2. Extrapolation TKG Completion

Extrapolation methods in TKG reasoning, aimed at forecasting future facts, have gained traction in recent years. RE-GCN [32] models TKGs by considering structural dependencies and temporal features alongside static attributes without separate query encoding. RE-NET [11] and CyGNeT [33] utilize subgraph aggregators and sequential copy networks, respectively, to model sequences of facts. TANGO-TuckER [34] and TANGO-DistMult [34] introduce the use of neural ODEs in GCNs to encode continuous-time dynamics for improved extrapolation. TLogic [13] offers an interpretable framework based on temporary logical rules extracted through random walks, while xERTE [35] employs a temporal association attention mechanism for causality preservation. TITer [12] employs a reinforcement learning strategy, defining a relative time coding function and a Dirichlet distribution-based reward to enhance learning. CEN [36] addresses evolutionary patterns using a length-aware CNN and curriculum learning, adapting to pattern changes over time. TiRGN [37] models historical dependencies with local and global recurrent graph encoders, providing deeper historical insights for extrapolation tasks.

2.3. Interpolation TKG Completion

Interpolation methods in temporal knowledge graph (TKG) completion extend static KG models by integrating temporal features. TTransE [38], an extension of TransE, incorporates temporal order constraints to enhance the modeling of time-sensitive relationships. HyTE [39] directly incorporates time by associating timestamps with hyperplanes, merging temporal aspects within the entity-relationship framework. TA-DistMult [40] further extends this concept by integrating relations and timestamps via a character LSTM, addressing the variability in time expressions. TComplEx [8], expanding on ComPlEx, uses tensor decomposition to generate timestamp embeddings, improving the model’s temporal adaptability. ATiSE [41] introduces time series decomposition to embed time information, mapping TKG representations into multidimensional Gaussian spaces for uncertainty. TeRo [7] proposes dual complex embeddings for relations to manage the start and end of temporal intervals. TIMEPLEX-base [42] unifies entity, relation, and time embeddings, capitalizing on recurrent facts and temporal interactions for enhanced link and time prediction. RotateQVS [10] and T-GAP [43] prioritize interpretability, with the former using quaternion space for temporal evolution and the latter focusing on temporal displacement for path-based inference. TeLM [9] examines the impact of time granularity through tensor decomposition, furthering the understanding of temporal knowledge graphs.

3. Preliminaries on Non-Euclidean Geometry

The exploration of temporal knowledge graphs is often enriched by the integration of non-Euclidean geometries, which provide a sophisticated framework for representing intricate data structures [15,18,20]. Within this context, Riemannian manifolds serve as a mathematical cornerstone for comprehending spaces with constant curvature. These manifolds are distinguished by the presence of a tangent space at every point, enabling the definition of fundamental operations such as addition and multiplication, which are essential for TKG embeddings.

A pivotal attribute that differentiates these spaces is their curvature. While Euclidean spaces are characterized by zero curvature, furnishing a flat geometrical plane, non-Euclidean spaces present distinct curvatures, positive or negative, that offer varied geometrical properties. Hypersphere spaces, exemplified by positive curvature (denoted as $c>0$), and hyperbolic spaces, marked by negative curvature ($-c$), are especially pertinent for encapsulating specific relational constructs within TKGs.

In the realm of hyperbolic geometry, various models facilitate mathematical operations. These operations include vector addition and multiplication, which are integral to the representation of geometric structures. For instance, the Möbius addition operation, fundamental to hyperbolic space, is articulated by the equation:

$$x{\oplus }_{c}y=\frac{{(1+2c\langle x,y\rangle +c\parallel y\parallel }^2{)x+(1-c\parallel x\parallel }^2)y}{1+2c\langle x,y\rangle +c^2{\parallel x\parallel }^2{\parallel y\parallel }^2},$$

(1)

where $x,y\in H_c^d$ represent points within the hyperbolic space, and $\langle ·,·\rangle$ signifies the inner product. The geodesic distance, a natural metric within hyperbolic space, is articulated as follows:

$$d^c(x,y)=\frac{2}{\sqrt{c}}\mathrm{arctanh}\left(\sqrt{c}\parallel -x{\oplus }_{c}y\parallel \right).$$

(2)

For computational efficiency, the tangent space at a hyperbolic space point, denoted as $T_x{H}_c^d$, is frequently utilized. This tangent space is a Euclidean space that facilitates the use of exponential and logarithmic maps for transitioning between the hyperbolic space and the tangent space. These maps are mathematically expressed as follows:

$$\begin{array}{cc}\hfill {exp}_0^c\left(v\right)& =tanh(\sqrt{c}\parallel v\parallel )\frac{v}{\sqrt{c}\parallel v\parallel },\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill {log}_0^c\left(y\right)& =\mathrm{artanh}\left(\sqrt{c}\parallel y\parallel \right)\frac{y}{\sqrt{c}\parallel y\parallel }.\hfill \end{array}$$

(4)

Additionally, hyperbolic space accommodates the definition of scalar multiplication, which scales vectors in accordance with the hyperbolic metric:

$$r{\otimes }_{c}x=\frac{tanh\left(r\mathrm{arctanh}\right(\sqrt{c}\parallel x\parallel \left)\right)}{\sqrt{c}}x/\parallel x\parallel .$$

(5)

The introduction of these operations is paramount for the development of TKG embeddings capable of effectively modeling the intricacies of real-world data, inclusive of hierarchical and relational structures.

4. Proposed Method

4.1. Notations and Definitions

A temporal knowledge graph (TKG) is an extensive collection of temporal facts, which can be formally represented as a subset of the Cartesian product of entities, relations, entities, and time stamps, denoted as $\mathbb{G}\subset \mathbb{E}\times \mathbb{R}\times \mathbb{E}\times \mathbb{T}$. In this context:

• $\mathbb{E}$ symbolizes the set of all entities within the TKG.
• $\mathbb{R}$ represents the set of all relations that connect entities.
• $\mathbb{T}$ denotes the set of time stamps, which are used to indicate when a particular fact occurred.

Each temporal fact in $\mathbb{G}$ is expressed as a quadruple $(e_s,r,e_o,t)$, where:

• $e_s\in \mathbb{E}$ denotes the subject entity.
• $r\in \mathbb{R}$ denotes the relation that exists between the subject and the object entities.
• $e_o\in \mathbb{E}$ denotes the object entity.
• $t\in \mathbb{T}$ denotes the time stamp indicating the moment at which the relation r held between $e_s$ and $e_o$.

In the domain of TKG completion, the objective is to enhance our understanding of the underlying temporal dynamics within a knowledge graph by predicting missing entities that once participated in a relation at a specific point in time. Given a query $q=(e_q,r_q,?,t_q)$, the interpolation task is to identify the missing object entity. This is achieved by leveraging the available information from the known subject entity $e_q\in \mathbb{E}$, the relation $r_q\in \mathbb{R}$, and the timestamp $t_q\in \mathbb{T}$. And we use d to denote the embedding dimension.

4.2. Various Geometry Embeddings

The spatial properties of Euclidean, hyperbolic, and hypersphere spaces are fundamentally different, each with its own spatial metric that dictates the rules of distance and angle measurements. The Euclidean metric is the most familiar, derived from the Pythagorean theorem, and is used in the standard Euclidean space. In contrast, the hyperbolic metric is characterized by constant negative curvature, and the hypersphere space is characterized by constant positive curvature. These metrics not only shape the geometry of the spaces but also influence how geometric information propagates between them. The exponential and logarithmic mappings are crucial in transferring information across these spaces. They act as bridges, allowing the embedding of entities in one space to be translated into another. This translation is essential for integrating knowledge from different geometric perspectives and for performing complex operations that span multiple spaces.

In the realm of geometric embeddings, the concept of rotations emerges as an integral mechanism for understanding and manipulating the structure inherent within and across various geometric domains. Rotations are pivotal for learning and encoding the intrinsic geometric relationships. Within our attention-based temporal graph information embedding (ATGIE) framework, these rotations are facilitated through the employment of $2\times 2$ Givens transformation matrices. The selection of these matrices is deliberate, capitalizing on their distinctive capacity to maintain the integrity of relative distances within Riemannian non-Euclidean spaces. This attribute renders them particularly well-suited for the embeddings of hyperbolic and hypersphere spaces, thereby enhancing the model’s geometric fidelity and its ability to represent complex spatial relationships.

The block-diagonal matrix $W_{{\Theta }_r}^{rot}$ is composed of elements $G^{+}\left({\theta }_{r,i}\right)$ on its diagonal, where each $G^{+}$ is a rotation matrix parameterized by an angle ${\theta }_{r,i}$:

$$W_{{\Theta }_r}^{rot}=\mathrm{diag}\left(G^{+}\left({\theta }_{r,1}\right),\dots ,G^{+}\left({\theta }_{r,d/2}\right)\right),$$

(6)

with each $G^{+}$ defined as

$$G^{+}\left({\theta }_{r,i}\right)=\left[\begin{array}{cc}cos\left({\theta }_{r,i}\right)& -sin\left({\theta }_{r,i}\right)\\ sin\left({\theta }_{r,i}\right)& cos\left({\theta }_{r,i}\right)\end{array}\right].$$

(7)

By applying these rotations to the embeddings, we obtain rotated results for each geometric space:

$$q_{rot}^H=W_{{\Theta }_r}^{rot}{e}_s^H.$$

(8)

Drawing inspiration from translation-based [22,23] and rotation-based [25,44] methods for static KG completion, our model incorporates both transformations to embed relationship and temporal information into the temporal knowledge graph. The head entity embedding is first translated over time to create a temporal embedding, which is then rotated along the relationship vector to produce a transformed head entity embedding.

For the Euclidean space, this translation and rotation is achieved via the Hadamard product:

$$s_E=(s+t)\circ r,$$

(9)

where s is the entity embedding, t is the time embedding, and r is the relationship embedding. For non-Euclidean spaces, namely, hyperbolic and hypersphere, we apply Möbius Addition for translation and Givens transformations for rotation. These operations are then mapped to tangent space representations to facilitate interactions with the Euclidean space:

$$s_B={log}_0^{c_b}\left(B_{{\Theta }_r}^{rot}\left(h{\oplus }_{c_b}t\right)\right),$$

(10)

$$s_H={log}_0^{c_h}\left(H_{{\Theta }_r}^{rot}\left(h{\oplus }_{c_h}t\right)\right),$$

(11)

where $B_{{\Theta }_r}^{rot}$ and $H_{{\Theta }_r}^{rot}$ represent the block-diagonal matrices that are meticulously calculated via Equation (6) for the hyperbolic and hypersphere spaces, respectively. ${\oplus }_{c_b}$ and ${\oplus }_{c_h}$ represent Möbius addition with curvature $c_b$ and $c_h$, respectively, and ${log}_0^B$ and ${log}_0^{c_h}$ are the corresponding logarithmic mappings to the tangent spaces. These transformations allow for a comprehensive representation of entities and their temporal dynamics across multiple geometric spaces, thus enriching the expressive power of the knowledge graph.

4.3. Attention Mechanism for Geometry Interaction

The interaction of multiple geometric spaces in TKGs offers a rich framework for representing and reasoning about complex data. However, the utility of these embeddings can vary depending on the specific query or task at hand. Some spaces may be more adept at capturing certain types of relationships or temporal dynamics. To leverage the strengths of each geometric representation and to create a more robust and versatile model, we introduce an attention mechanism.

The attention mechanism allows our model to dynamically assign importance to each geometric space based on the query. This is achieved by computing attention weights $({\alpha }_E,{\alpha }_{E}B,{\alpha }_H)$ for the Euclidean, hyperbolic, and hypersphere spaces, respectively. The weights are derived from the interaction between an attention vector a and the embeddings $s_E$, $s_B$, and $s_H$ of the respective spaces:

$$({\alpha }_E,{\alpha }_B,{\alpha }_H)=\mathrm{Softmax}(a^T{s}_E,a^T{s}_B,a^T{s}_H),$$

(12)

where the Softmax function ensures that the attention weights sum to one, providing a normalized measure of the relevance of each space for the given query.

With the attention weights computed, we can construct a weighted sum of the embeddings from each space to form a single geometric interaction embedding $G_s$. This embedding combines the information from all spaces, weighted by their computed importance:

$$G_s={\alpha }_E{s}_E+{\alpha }_B{s}_B+{\alpha }_H{s}_H.$$

(13)

This results in a comprehensive representation that encapsulates the multi-faceted geometric interactions within the temporal knowledge graph. Finally, query embeddings are compared to target tail embeddings using the distance (Equation (2)) in a hyperbolic space with curvature $c_r$. The resulting scoring function is

$$\begin{array}{c}\hfill s(q,e_o)=-d^{c_r}{(G_s,{\mathbf{e}}_o)}^2+b,\end{array}$$

(14)

where b is the bias which act as margins in the scoring function [17,45].

4.4. Training with Cross-Entropy Loss

Our ATGIE framework is refined through the optimization of a cross-entropy loss function. This function quantifies the discrepancy between the anticipated and actual results for a specified query. Our approach incorporates uniform negative sampling, which generates negative instances for a query $q=(e_q,r_q,?,t_q)$ by uniformly selecting alternative subject entities from the space of all potential triples, thereby disrupting the correct subject entity $e_o$:

$$\begin{array}{cc}\hfill \mathcal{L}& =\sum _{(q,e)\in {\mathcal{T}}_{\mathrm{train}}}\log (1+\exp \left(Y_{e}s(q,e)\right)),\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill & \mathrm{where}{Y}_e=\left\{\begin{array}{cc}-1\hfill & \mathrm{if}e=e_o\hfill \\ 1\hfill & \mathrm{otherwise}\hfill \end{array}\right.,\hfill \end{array}$$

(16)

where ${\mathcal{T}}_{\mathrm{train}}$ is set of training queries. This formulation encourages the model to correctly identify the true object entity $e_o$ while differentiating it from the negative samples. Optimization in hyperbolic space presents unique challenges owing to its non-Euclidean nature. To address this, we adopt a strategy where all parameters are defined in the tangent space at the origin of the hyperbolic space. This allows us to leverage standard Euclidean optimization techniques during training. The embeddings are then mapped back to the hyperbolic space using the exponential map when necessary [46].

This approach strikes a balance between the theoretical benefits of hyperbolic embeddings and the practical advantages of Euclidean computation, enabling efficient training of our model while maintaining the geometric properties of the hyperbolic space. The attention mechanism for geometric interaction, as described, provides a sophisticated approach to leveraging the strengths of multiple geometric embeddings within a TKG. By dynamically weighting the contributions of each space, our model can adapt to the requirements of different queries and tasks, leading to improved performance and more nuanced insights into the structured data.

5. Experiments

In this section, we describe a series of experiments conducted to assess the performance of our proposed ATGIE. The evaluation is designed to test the effectiveness of ATGIE across a spectrum of scenarios that are representative of the challenges present in the TKG interpolation task.

5.1. Experimental Setup

5.1.1. Benchmark Datasets

To evaluate our proposed method, we performed link prediction task on four commonly used TKG benchmark datasets, namely, ICEWS14, ICEWS05-15 [40], YAGO11k [39], and GDELT [47].

Table 1. Statistics of four experimented datasets.

Dataset	ICEWS14	ICEWS05-15	YAGO11k	GDELT
Entities	7128	10,488	10,623	500
Relations	230	251	10	20
Time Stamps	365	4017	70	366
Train	72,826	386,962	16,408	2,735,685
Validation	8941	46,275	2050	341,961
Test	8963	46,092	2051	341,961

encapsulates a summary of these datasets, highlighting the ICEWS14 and ICEWS05-15 datasets’ greater quantity of quantitative relations in comparison to YAGO11k and GDELT. The ICEWS [48] serves as a comprehensive archive of political events, each indexed with a distinct timestamp. The datasets ICEWS14 and ICEWS05-15 [40] are derived subsets of the ICEWS corpus, representing events from the year 2014 and spanning the years 2005 to 2015, respectively. YAGO11k [39] is an excerpt from the YAGO3 [49] dataset, with temporal annotations articulated as time intervals. We derive the dataset from HyTE [39] to obtain the same year-level granularity by dropping the month and date information, which results in 70 different time stamps. For the GDELT dataset, we utilize the subset defined by [47], encompassing events from 1 April 2015 to 31 March 2016. Consistent with the methodology of [50], we applied identical preprocessing steps to the train, validation, and test sets, thereby framing the problem as a TKG completion (TKGC) task rather than one of extrapolation.

5.1.2. Evaluation Protocol

The evaluation of our proposed model for TKG completion is conducted through a link prediction task. This task is specifically designed to deduce incomplete temporal facts with a missing entity. It targets scenarios where the facts are represented by either a source entity s, relation r, and a missing target entity at time t, denoted as $(s,r,?,t)$, or vice versa, where the source entity is missing, represented as $(?,r,o,t)$. During the inference phase, we adhere to the candidate generation strategy as outlined in [7]. This strategy involves the creation of candidate quadruples C, which are formed by substituting either the subject s or the object o with every entity from the entity set $\mathcal{E}$, resulting in the set $C=\{(s,r,\overline{o},t):\overline{o}\in \mathcal{E}\}\cup \{(\overline{s},r,o,t):\overline{s}\in \mathcal{E}\}$. Subsequently, these quadruples are ordered by their corresponding scores in an ascending order by their scores (Equation (14)). We opt for the time-wise filtered setting as reported in [41,50] to present our experimental outcomes. This method ensures that facts not observed at time t remain eligible for assessment when evaluating a given test quadruple, as outlined in [7]. This approach is particularly well-suited for TKG reasoning, as it maintains the temporal integrity of the data while facilitating a more accurate analysis. All experiments are conducted based on a Linux machine with 16 CPU cores, 32 GB RAM and a NVIDIA 3090Ti GPU (Nvidia Corporation, Santa Clara, CA, USA). We set the dimension of the embedding to 200.

To assess the effectiveness of our approach, we rely on conventional evaluation metrics, which include the ratio of accurate triples positioned within the top ranks (Hits@1, Hits@3, and Hits@10), as well as the Mean Reciprocal Rank (MRR). In the context of these metrics, higher values are indicative of superior performance. For the purpose of our experiments, we present the mean results obtained from five independent runs, electing to exclude the variance due to its typically minimal impact.

5.1.3. Baselines

We compared with both SOTA static and temporal KGE baselines. For static baselines, we use TransE [22], DistMult [26], RotatE [25], and QuatE [51]. For TKGE methods, we consider TTransE [38], HyTE [39], TA-DistMult [40], DE-SimplE [50], ATiSE [41], TeRo [7], RotateQVS [10], and TGoemE++ [52].

5.2. Performance Comparison

The experimental results over four extrapolation datasets are listed in

Table 2. Results on link prediction task over four experimented datasets. The best score is in bold.

Dataset	ICEWS14				ICEWS05-15
Metrics	Hits@1	Hits@3	Hits@10	MRR	Hits@1	Hits@3	Hits@10	MRR
TransE [22]	0.094	-	0.637	0.280	0.090	-	0.663	0.294
DistMult [26]	0.323	-	0.672	0.439	0.337	-	0.691	0.456
RotatE [25]	0.291	0.478	0.690	0.418	0.164	0.355	0.595	0.304
QuatE [51]	0.353	0.530	0.712	0.471	0.370	0.529	0.727	0.482
TTransE [38]	0.074	-	0.601	0.255	0.084	-	0.616	0.271
HyTE [39]	0.108	0.416	0.655	0.297	0.116	0.445	0.681	0.316
TA-DistMult [40]	0.363	-	0.686	0.477	0.346	-	0.728	0.474
DE-SimplE [50]	0.418	0.592	0.725	0.526	0.392	0.578	0.748	0.513
ATiSE [41]	0.436	0.629	0.750	0.550	0.378	0.606	0.794	0.519
TeRo [7]	0.468	0.621	0.732	0.562	0.469	0.668	0.795	0.586
RotateQVS [10]	0.507	0.642	0.754	0.591	0.529	0.709	0.813	0.633
TGoemE++ [52]	0.546	0.680	0.780	0.629	0.605	0.736	0.833	0.686
ATGIE (ours)	0.563	0.691	0.797	0.642	0.616	0.749	0.840	0.691
Dataset	YAGO11k				GDELT
Metrics	Hits@1	Hits@3	Hits@10	MRR	Hits@1	Hits@3	Hits@10	MRR
TransE	0.015	0.138	0.244	0.100	0.0	0.158	0.312	0.113
DistMult	0.107	0.161	0.268	0.158	0.117	0.208	0.348	0.196
RotatE	0.103	0.167	0.305	0.167	-	-	-	-
QuatE	0.107	0.148	0.270	0.164	-	-	-	-
TTransE	0.020	0.150	0.251	0.108	0.0	0.160	0.318	0.115
HyTE	0.015	0.143	0.272	0.105	0.0	0.165	0.326	0.118
TA-DistMult	0.103	0.171	0.292	0.161	0.124	0.219	0.365	0.206
DE-SimplE	-	-	-	-	0.141	0.248	0.403	0.230
ATiSE	0.110	0.171	0.288	0.170	-	-	-	-
TeRo	0.121	0.197	0.319	0.187	-	-	-	-
RotateQVS	0.124	0.199	0.323	0.189	0.175	0.293	0.458	0.270
TGoemE++	0.130	0.196	0.326	0.195	-	-	-	-
ATGIE (ours)	0.155	0.232	0.357	0.223	0.198	0.313	0.480	0.294

. Overall, temporal knowledge graph completion methods are better than static knowledge graph completion methods, which shows the effectiveness of modeling temporal information. For the proposed ATGIE, we observe that our model outperforms all the baseline models over the four datasets across all metrics consistently. Despite the significant differences in scale, number of entities, and number of relations between the four TKGs used in our evaluation (see Table 1), the results demonstrate that our proposed model exhibits efficient reasoning performance on TKGs of different sizes and complexity levels. This showcases the versatility and efficiency of our model in addressing knowledge graph completion tasks in diverse temporal contexts.

5.3. Ablation Study of Performance

Ablation studies were conducted on two datasets, namely, ICEWS14 and ICEWS05-15, to investigate the impact of individual components on the performance of our ATGIE. The MRR performances are shown in Figure 1, where seven sub-models are compared, including (1) the original ATGIE model; (2) ATGIE without euclidean space, denoted as “-E”; (3) ATGIE without hyperbolic space, denoted as “-B”; (4) ATGIE without hypersphere space, denoted as “-H”; (5) ATGIE without both the euclidean and hyperbolic spaces, denoted as “-E-B”; (6) ATGIE without both the euclidean and hypersphere spaces, denoted as “-E-H”; and (7) ATGIE without both the hyperbolic and hypersphere spaces, denoted as “-B-H”. Based on the experimental results, it can be concluded that the embeddings of each space, including Euclidean, hyperbolic, and hypersphere, play a crucial role in determining the performance metrics of ATGIE. The ablation study helps us understand the importance of each component and highlights their contributions to the overall performance of the ATGIE.

5.4. Robustness Evaluation

To rigorously assess the robustness of the ATGIE framework, we conducted experiments under two distinct conditions: data sparsity and data noise. This was achieved by systematically manipulating the ICEWS14 dataset through the deletion or insertion of facts. Specifically, we altered the training set by randomly removing existing facts or adding new ones, targeting proportions of 25%, 50%, and 75%. Figure 2 and Figure 3 present a comparative analysis of performance across these varying levels of data sparsity and noise. In this comparison, we evaluate our ATGIE model against two baselines: TeRo [7] and RotateQVS [10]. The results showcased in Figure 2 and Figure 3 clearly illustrate that ATGIE significantly outperforms the baseline models across an extensive range of data sparsity and noise conditions. Notably, ATGIE maintains a high level of performance even under extreme conditions of 75% data sparsity or noise, achieving a Hits@1 value exceeding 0.45. This exceptional outcome underscores ATGIE’s unparalleled robustness and its proficient capability in effectively managing both sparse and noisy datasets. The superior performance of ATGIE can be attributed to its advanced design, which likely incorporates innovative mechanisms to mitigate the adverse effects of data imperfections. This makes ATGIE a highly resilient tool for applications where data quality cannot be guaranteed, offering significant advantages over conventional models in handling challenging real-world scenarios involving incomplete or inaccurate information.

6. Conclusions

In this paper, we introduce a novel attention-based geometry interaction embedding (ATGIE) method, tailored for temporal knowledge graph completion. This innovative method harnesses the collective advantages of distinct geometric domains, i.e., Euclidean, hyperbolic, and hypersphere, to more adeptly represent the complexities inherent in TKGs. ATGIE employs an attention mechanism to dynamically weigh the contributions of different geometric spaces, allowing it to adaptively form reliable spatial structures based on interactive geometric information. This multi-space modeling not only captures the diverse relationships within TKGs but also facilitates a nuanced understanding of how entities and their relationships evolve over time. Through extensive experiments, we demonstrate ATGIE’s superiority in TKGC tasks, showcasing its improvement over existing methods, robustness to noise, and sensitivity to temporal dynamics. The results highlight ATGIE’s potential to advance the state of the art in TKGC, offering a promising direction for research and application in the field.

7. Limitations and Future Work

Limitations: The ATGIE method, despite its innovative approach to temporal knowledge graph completion, comes with a set of limitations that warrant attention. A key aspect of our model is the attention mechanism, which dynamically weighs the contributions from distinct geometric domains. While this feature enhances the model’s adaptability, it also introduces sensitivity to hyperparameter settings. The process of tuning these hyperparameters is critical and can be complex, requiring a delicate balance to avoid overfitting. The model’s reliance on the attention mechanism may also make the optimization process more susceptible to local minima, thus demanding sophisticated initialization methods and advanced hyperparameter optimization techniques. Furthermore, although ATGIE demonstrates robustness against data sparsity and noise, the model’s adaptability across different domains and its scalability when applied to extensive TKGs are yet to be thoroughly investigated. The empirical validation of ATGIE’s performance on a variety of datasets and its capacity to handle large-scale TKGs with millions of entities and relationships is an area that requires further research.

Future Work: Looking ahead, several avenues for future work present themselves. First, addressing the scalability of ATGIE to very large TKGs will be a focal point. This includes developing efficient learning strategies, such as mini-batch processing or distributed computing approaches, to manage the computational demands of big data environments. Second, we recognize the need to enhance the interpretability of the attention mechanism within ATGIE. Future work will explore visualization and explanation methods to provide insights into how different geometric spaces contribute to the model’s decisions, thereby increasing trust and transparency in the model’s predictions. Lastly, we plan to investigate the long-term stability and convergence properties of the model. This includes studying the effects of different optimization algorithms and learning rate schedules on the training process to ensure reliable and consistent performance.