Research on a Knowledge Graph Embedding Method Based on Improved Convolutional Neural Networks for Hydraulic Engineering

Abstract

In response to the shortcomings of existing knowledge graph embedding strategies, such as weak feature interaction and latent knowledge representation, a unique hydraulic knowledge graph embedding method is suggested. The proposed method incorporates spatial position features into the entity-relation embedding process, thereby enhancing the representation capability of latent knowledge. Furthermore, it utilizes a multi-layer convolutional neural network to fuse features at different levels, effectively capturing more abundant semantic information. Additionally, the method employs multi-scale dilated convolution kernels to capture rich explicit interaction features across different scales of space. In this study, the effectiveness of the proposed model was validated on the link prediction task. Experimental results demonstrated that, compared to the ConvE model, the proposed model achieved a significant improvement of 14.8% in terms of mean reciprocal rank (MRR) on public datasets. Additionally, the suggested model outperformed the ConvR model on the hydraulic dataset, leading to a 10.1% increase in MRR. The results indicate that the proposed approach exhibits good applicability and performance in the task of hydraulic knowledge graph complementation. This suggests that the method has the potential to offer significant assistance for knowledge discovery and application research in the field of hydraulics.

1. Introduction

The implementation and management of hydraulic engineering and the ecological environment is complex and changeable, there are many risks during the operation of the project, and it is difficult to balance the distribution of water resources. It is important to strengthen the scientific and standardized management of hydraulic engineering. At present, most of the data related to hydraulic engineering are stored in the form of paper text and electronic documents, which have problems such as low efficiency of query and retrieval, weak content association, and insufficient intelligent decision support. Therefore, it is necessary to explore the rapid acquisition, rational organization, information sharing, and scientific utilization of the massive, multi-source, heterogeneous hydraulic engineering knowledge body.

A knowledge graph is a semantic network that represents the relationships between various entities in the real world [1]. In recent years, knowledge graphs have also been applied to the field of hydraulics. By establishing links between entities, relationships and attributes, a knowledge graph can realize the structured and unified management of large amount of multi-source heterogeneous data in the field of hydraulics. Juan D et al. [2] constructed a water quality knowledge graph using water quality data and regulatory data from the Bogota River basin. The knowledge graph was also used to uncover information on water quality violations in the Bogota River basin, providing experts with a connected and interoperable view of the data. J Yan et al. [3] proposed a method to build a knowledge graph for water affairs by integrating structured and unstructured data, aiming to meet the requirements of water information integration. They developed an information recommendation system based on the water affairs knowledge graph, which effectively addressed the needs of water information retrieval and improved water information retrieval capabilities. Duan H et al. [4] achieved a cross-domain query and retrieval of water knowledge by constructing a comprehensive knowledge graph of water resources, which integrates and associates water-related object knowledge and water discipline knowledge, thus improving the efficiency of water knowledge retrieval and the ability of knowledge mining and discovery. In addition to the construction of hydraulic knowledge graphs, many scholars have started to initially explore intelligent applications based on hydraulic knowledge graphs. The research of these applications aims to make full use of the rich information of the hydraulic knowledge graph to improve the efficiency of management and the accuracy of decision making in the field of hydraulics. Liu X et al. [5] used the text data of a water conservancy project inspection to construct the knowledge graph of the water conservancy project emergency plan, and generated the emergency plan intelligently through knowledge retrieval and reasoning, which broke the problems of low digitization, poor knowledge relevance, and insufficient intelligent decision making in the traditional water conservancy project emergency plan. Ye et al. [6] proposed a knowledge-graph-based urban flood resilience service framework to provide accurate and timely flood resilience information as well as efficient flood management solutions by enabling the sharing and intelligent application of knowledge in the flood resilience domain.

Although knowledge graphs are very effective in representing structured data, the basic symbolic nature of triples makes knowledge graphs difficult to manipulate [7]. Also, the complexity of the data’s multi-source heterogeneity and knowledge extraction [8,9,10] processes in knowledge graphs makes them suffer from incomplete information, and this incompleteness seriously affects their effectiveness in practical applications [11]. Knowledge graph embedding (KGE) [7] is a vital approach for addressing the problem of knowledge graph completion. Among the existing knowledge graph embedding models, the models based on translation distance extract the potential semantic information and structural features in the knowledge graph by simple structural or basic matrix operations, and the representative models are TransE [12], TransH [13], TransR [14], and TransD [15]. Bilinear models, such as DisMult [16] and ComplEx [17], use simple multiplicative operators to represent entities and relations and, thus, are able to capture linear associations between entities and relations. However, both types of models have weak expressiveness and an insufficient generalization ability when dealing with complex knowledge graphs. With the continuous development of deep learning technology, neural networks show a powerful nonlinear fitting ability and a good generalization performance. Therefore, the knowledge graph embedding representation model based on the deep structure of neural networks has attracted a lot of attention from researchers. Among them, as the first model to introduce convolutional neural networks to the knowledge graph embedding task, ConvE [18] used a combination of head entity embedding and relational embedding as the input for the convolutional neural network, and extracted the local interaction features between entities and relations using two-dimensional convolution for the first time. However, its standard convolution captures insufficient feature interaction information, and has low feature representation capability. InteractE [19] with improvements to ConvE, using techniques such as feature alignment and circular convolution, further enhances the local interaction of features and improves the knowledge representation. However, its multiple convolutional layer structure can lead to the loss of surface knowledge to some extent. JointE [20] jointly uses 1D convolution and 2D convolution for feature extraction. The interaction between entities and relations is facilitated by 2D convolution to fully capture the latent knowledge; 1D convolution is used to extract explicit knowledge to address the lack of surface semantics. However, the JointE model ignores the spatial location properties of entity and relationship vectors. CapsE [21], on the other hand, takes advantage of the capsule network, and is able to capture deep features of different feature mappings in the same dimension. However, due to the complex structure of the capsule network and the use of dynamic routing algorithms, the training and inference process of the model consumes a lot of computational resources and time. Unlike the aforementioned models, the ConvR [22] model reshapes the embedding parameters of relations into multiple convolutional kernels, enabling relation-specific convolution operations that capture the feature interactions between entities and relations more comprehensively. However, it focuses solely on the feature interaction of a single scale and overlooks the feature interactions across different scales, limiting the expressive power of the model.

Based on the above discussion, this study proposes a knowledge graph embedding method based on convolutional neural networks to address the limitations of feature interaction capabilities and incomplete semantic representations in knowledge graph embeddings.

The main contributions of this work are as follows:

• The integration of spatial location feature information in the embedding representation of entity relations enhances the representation of water knowledge;
• It proposed the ConvMVD model, which utilizes multi-scale dilated convolution for high and low level feature interaction, resulting in richer semantic information and more reasonable entity-relationship embedding representation;
• The ConvMVD model was applied to the hydraulic engineering dataset, and the experimental results showed that the model performed significantly better than other representative baseline methods in the linkage prediction task.

The rest of this paper is organized as follows. Section 2 introduces the techniques related to the knowledge graph embedding model ConvMVD. Section 3 elaborates on the construction process of the ConvMVD model. Section 4 presents the experimental protocol, the experimental results, and the experimental discussion. Section 5 briefly describes the formal methods for verification based on artificial intelligence techniques. Section 6 concludes the paper, and points out future research directions.

2. Related Work

2.1. Convolutional Neural Networks

A convolutional neural network (CNN) is a feedforward neural network capable of extracting features from data with convolutional structures [23]. The basic idea of CNN is to extract local features by convolving the input data with a convolutional layer, then downsampling the extracted features with a pooling layer and, finally, classifying them with a fully connected layer.

The convolutional layer is mainly divided into convolution and activation operations [24]. It is assumed that there are N neurons in a layer, and the convolution operation is performed between the feature map of the previous layer and the convolution kernel, then the new feature matrix is output by a specific activation function [25]. The mathematical expression for the convolution operation is as follows:

$$X_j^l=f\left({\displaystyle \sum }_{i\in \left(0,M\right)}{X}_i^{l-1}{K}_j^l+b_j^l\right) \left(j\in \left(0,N\right)\right)$$

(1)

In the equation, l denotes the layer number, $X_j^l$ represents the j-th feature matrix of the l-th layer, f(·) represents the activation function, M represents the total number of neurons in the previous layer, $X_i^{l-1}$ represents the i-th feature matrix passed from the previous layer, $K_j^l$ represents the weight matrix of the corresponding convolutional kernel, and $b_j^l$ represents the bias matrix.

The pooling layer [26] aims to reduce the dimensionality of the feature maps by performing downsampling on the input feature vectors using pooling kernels. Pooling can be categorized into two types: maximum pooling and average pooling, respectively:

$$P_{max}^{l\left(i,s\right)}=\underset{\left(s-1\right)W<t<sW}{\max }\left\{a^{l\left(i,t\right)}\right\} \left(\mathrm{s}=1,2,\dots \dots .,\mathrm{q}\right),$$

(2)

$$P_{avg}^{l\left(i,s\right)}=\underset{\left(s-1\right)W<t<sW}{avg}\left\{a^{l\left(i,t\right)}\right\}$$

(3)

In the equations, a^l(i,t) represents the t-th feature map of the i-th neuron in the l-th layer, W represents the width of the pooling region, s denotes the s-th pooling kernel, and $P_{avg}^{l\left(i,s\right)}$ represents the output result of pooling.

The role of the fully connected layer [27] is to integrate the relevant information for feature classification from the convolutional and pooling layers. It transforms the feature maps, carrying the relevant information into one-dimensional vectors. The expression for this process is as follows:

$$\mathrm{h}\left(\mathrm{x}\right)=\mathrm{f}\left(\mathrm{wx}+\mathrm{b}\right)$$

(4)

In the equation, x represents the input to the fully connected layer, which is the feature matrix passed down from the last convolutional layer, w represents the weight vector, and b represents the bias vector.

2.2. Dilated Convolution

Dilated convolution, also known as atrous convolution or dilated convolution, is a technique in convolutional neural networks [28]. Its main design principle is to insert a certain number of zeros (0 values) between two consecutive filter values in each dimension of the convolutional kernel, thereby increasing the receptive field of the kernel. For a two-dimensional signal X, the output obtained by applying the dilated convolution operator can be represented in the following form:

$$Y\left(m,n\right)={{\displaystyle \sum }}_{a=1}^{k_h}{{\displaystyle \sum }}_{b=1}^{k_w}X\left(m+r·a,n+r·b\right)\times {\omega }^r\left(a,b\right)$$

(5)

In the equation, Y represents the output signal, k_h and k_w represent the width and height of the convolutional kernel, and r corresponds to the dilation rate of the convolutional kernel ω^r used to sample X.

2.3. Knowledge Graph Embedding

The knowledge graph embedding approach is a technique for mapping entities and relations in the knowledge graph to a low-dimensional continuous vector space. These vector representations can capture the semantic associations between entities and relations, and be used for various knowledge-graph-related tasks such as entity relationship prediction, knowledge graph complementation [29], and recommendation. Knowledge graph embedding models can usually be classified as translational distance-based models, bilinear models, and neural network-based models.

• The model based on translation distance treats the relationship as a translation from the head entity to the tail entity in vector space, which has the features of high computational efficiency and wide application but, compared with the neural network-based and deep learning methods, it still needs to be improved in semantic feature extraction and learning;
• The bilinear model measures the similarity between entities and relationships by mapping them into a low-dimensional vector space and using bilinear functions. Its simple form is easy to implement, but its weak expressiveness limits its application in modeling complex relationships;
• The neural network-based model is to use different neural network methods to complete the feature representation of entities and relations, and is then used for inference prediction of the knowledge triad. Due to its greater advantages in semantic feature learning and extraction, it has become the mainstream research direction of current knowledge graph embedding models.

3. Materials and Methods

The process of the proposed ConvMVD model, which is based on an enhanced convolutional neural network for knowledge graph embedding, is depicted in Figure 1. Firstly, the one-dimensional feature vectors of entities and relations are jointly reshaped into two-dimensional feature maps, allowing for the aggregation of feature information from translation and rotation perspectives. Subsequently, feature extraction is performed using two-dimensional convolutional operations, where multi-scale dilated convolutions with dilation rates (1, 2, 3) are employed instead of standard convolutions to expand the receptive field. Finally, the three output feature maps are fused and reshaped into the final predicted one-dimensional feature vector.

3.1. Problem Formulation

In this paper, the knowledge graph is represented as $G=\left\{\left(h,r,t\right)|\left(h,r,t\right)\in E\times R\times E\right\}$, which is the set of a series of triples, and a single triple can be represented as (h,r,t). Where E represents the set of entities, h,t denote the head entity and tail entity, respectively, $h,t\in E$; and R represents the set of relations, $r\in R$. The purpose of knowledge graph embedding is to learn the low-dimensional vector representation $\mathrm{h},\mathrm{r}ϵ{\mathrm{M}}^{\mathrm{d}}$ of a given entity and relation, where d denotes the dimensionality of the embedding. The link prediction task was performed during our experiments, which refers to using the entities and relations that already exist in the knowledge graph to infer whether there is a relationship missing between a certain two entities. For example, given the head entity and relation, the tail entity is predicted, denoted as (h,r,?). The link prediction problem can be formally defined as learning a scoring function φ: E × R × E → M. Given an input triple x = (h,r,t), it is scored using a pre-designed scoring function.

3.2. ConvMVD

This paper proposes an improved convolutional neural network-based knowledge graph embedding model called ConvMVD, as shown in Figure 2. ConvMVD involves embedding representation, feature interaction and fusion, and link prediction.

3.2.1. Embedding Representation

The potential semantic extraction ability of the knowledge graph embedding model is significantly affected by the degree of interaction between entity embedding and relationship embedding. The embedding representation of the ConvMVD model consists of four parts: the two-dimensional feature matrix of entity relationships, the spatial translation of entity relationship vectors, the spatial rotation of entity relationship vectors, and the spatial fusion superposition of entity relationship vectors, i.e., the fusion of spatial translation and rotation. Through the combination of the above four components, the ConvMVD model can synthesize different features between entities and relations to extract potential semantic information in the knowledge graph. This comprehensive embedding representation helps the model better understand and reason about the relationships in the knowledge graph, thus enhancing its potential semantic extraction capability.

(1)

Characteristic matrix

For a knowledge graph, we initialize each head entity as a vector of head entities and each relation as a vector of relations, followed by reshaping each of them into a two-dimensional matrix and performing a join operation to obtain one of the inputs for the convolution operation. The reason for reshaping vectors into matrices is that two-dimensional convolution is superior to one-dimensional convolution in terms of interaction modeling. The process can be expressed as follows:

$$\begin{gathered} \mathrm{H}ϵ{\mathrm{M}}^{\mathrm{d}}\to hϵ{\mathrm{M}}^{\mathrm{a}\times b^{\prime }} \\ \mathrm{R}ϵ{\mathrm{M}}^{\mathrm{d}}\to rϵ{\mathrm{M}}^{\mathrm{a}\times b^{\prime }} \\ \widehat{h}=concat\left(h,r\right), \widehat{h}\in {\mathrm{M}}^{\mathrm{a}\times \mathrm{b}} \end{gathered}$$

(6)

where H denotes the head entity vector, h denotes the 2D transformed head entity vector, R denotes the relational vector, r denotes the 2D transformed relational vector, and d denotes the embedding dimension. $\widehat{h}$ denotes the 2D matrix obtained by joining h and r, concat denotes the join operation, and a,b,b′ = b/2, denotes the spatial dimension of the vector;

(2)

Spatial transformation view

Information about the spatial location features of entities and relationships plays an important role in the accuracy of link prediction tasks [30] and, by considering the location features of entities and relationships in space, link relationships in knowledge graphs can be better understood. Combining multiple views can provide a more comprehensive description of knowledge, and the different views have complementary information to each other. By combining different perspectives and features, we can obtain a richer and more diverse representation of knowledge. Therefore, this study constructs different spatial transformation views for entity vectors and relation vectors. The views are represented as V, where the first view, V_P, represents the spatial translation view of entity-relation vectors; the second view, V_R, represents the spatial rotation view of entity-relation vectors; and the third view, V_M, represents the fusion view of spatial translation and rotation. The implementation process is as follows:

First, the head entity $\mathrm{h}ϵ{\mathrm{M}}^{\mathrm{a}\times b^{\prime }}$ and the relation $\mathrm{r}ϵ{\mathrm{M}}^{\mathrm{a}\times b^{\prime }}$ are divided into two parts: $h=\left\{h_1,h_2\right\}$ and $r=\left\{r_1,r_2\right\}$, where $h_1,h_2,r_1,r_2ϵ{\mathrm{M}}^{\mathrm{a}\times b^{″}}$ and $b^{″}=b/4$. Next, the two sub-entity vectors and the two sub-relation vectors are combined separately to form four combinations of entity-relation vectors: $\left(h_1,r_1\right)$,$\left(h_1,r_2\right)$,$\left(h_2,r_1\right), \left(h_2,r_2\right)$. The spatial translation formula for entity-relation vectors is as follows:

$$V_P={\left[{\left(h_1+r_1\right)}_{\mathrm{a}\times b^{″}};{\left(h_1+r_2\right)}_{\mathrm{a}\times b^{″}};{\left(h_2+r_1\right)}_{\mathrm{a}\times b^{″}};{\left(h_2+r_2\right)}_{\mathrm{a}\times b^{″}}\right]}_{\mathrm{a}\times b}$$

(7)

where V_P denotes the spatial translation view, (h₁ + r₁) denotes the spatial translation operation of h₁ and r₁, $\left(h_1+r_1\right)ϵ{\mathrm{M}}^{\mathrm{a}\times b^{″}}$, and [;] denotes the vectors on both sides of the semicolon to be connected. The spatial rotation formula of the entity relation vector is as follows:

$$V_R={\left[{\left(h_1\ast r_1\right)}_{\mathrm{a}\times b^{″}};{\left(h_1\ast r_2\right)}_{\mathrm{a}\times b^{″}};{\left(h_2\ast r_1\right)}_{\mathrm{a}\times b^{″}};{\left(h_2\ast r_2\right)}_{\mathrm{a}\times b^{″}}\right]}_{\mathrm{a}\times b}$$

(8)

where V_R denotes the spatial rotation view, (h₁ ∗ r₁) denotes the spatial rotation operation of h₁ and r₁, $\left(h_1\ast r_1\right)ϵ{\mathrm{M}}^{\mathrm{a}\times b^{″}}$, and [;] denotes the vectors on both sides of the semicolon for concatenation.

Next, the fused view V_M is formed by combining the translation vector h + r and the rotation vector h ∗ r. This operation can be expressed as:

$$V_M={\left[{\left(h+r\right)}_{\mathrm{a}\times b^{\prime }};{\left(h\ast r\right)}_{\mathrm{a}\times b^{\prime }}\right]}_{\mathrm{a}\times b}$$

(9)

Finally, the two-dimensional feature matrix $\widehat{h}$, the spatial translation view V_P, the spatial rotation view V_R, and the fusion view V_M are concatenated as the input of the convolution layer. This process can be expressed as $\widehat{E}=concat\left(\widehat{h};V_P;V_R;V_M\right)$, where concat denotes the concatenation operation.

3.2.2. Feature Interaction and Fusion

In the feature interaction module, this model incorporates a multi-scale and multi-level feature interaction structure.

First, by adjusting the dilation rate parameter i, we generate dilated convolution kernels ${\omega }^i\in K^{k_h^i\times k_w^i}$ with different sizes, where $k_h^i=k_h+\left(k_h-1\right)\times \left(i-1\right)$ and $k_w^i=k_w+\left(k_w-1\right)\times \left(i-1\right)$. Here, k_h and k_w represent the width and height of the convolution kernel, respectively. As shown in Figure 2, these dilated convolution kernels include ${\omega }^1\in K^{3\times 3}$, ${\omega }^2\in K^{5\times 5}$, ${\omega }^3\in K^{7\times 7}$, which have different sizes. These kernels are convolved with $\widehat{E}$ to produce interaction features at multiple scales. For the m-th row and n-th column of the feature output C_i obtained by operating $\widehat{E}$ with the convolution kernel ωⁱ, it can be calculated using the following formula:

$$C_i\left(m,n\right)=\widehat{E}\ast {\omega }^i={\displaystyle \sum }_{a=1}^{k_h}{\displaystyle \sum }_{b=1}^{k_w}\widehat{E}\left(m+i·a,n+i·b\right)\times {\omega }^i\left(a,b\right)$$

(10)

In the equation, C_i represents the feature output obtained by interacting with the convolutional kernel ωⁱ with a pore size of i, and ∗ denotes the convolution operation.

Shallow features are extracted as features in earlier convolutional layers, and they are very effective for capturing local information. However, shallow features have a small field of perception, and can only focus on local areas of the input data, which may not cover the broader contextual information. As the number of convolutional layers increases, the extraction ability of deep features is enhanced and can capture richer semantic information. Deep features gradually expand the perceptual field by stacking multiple convolutional layers and nonlinear activation functions, which can focus on a larger range of contexts and capture more abstract and high-level semantic features. However, correspondingly, the detailed information of deep features may be reduced [31]. This model uses multilayer convolutional neural networks to extract features at different levels, to fuse features at different levels to take full advantage of their strengths and complementarities. First, the input feature matrix is convolved and, for shallow feature maps, it is used as input to perform more convolutions until three layers of feature maps are obtained, each corresponding to low, medium and high level features. Moreover, each layer’s feature maps represent the interaction features between entities and relations at different scales. These feature maps are aggregated together using element-wise addition. Then, a fully connected layer is applied to fuse these feature maps, resulting in a multi-scale and multi-level feature map. Finally, the feature map is flattened into a feature vector. This process can be represented as follows:

$$C_1=\widehat{E}\ast {\omega }^1$$

(11)

$$C_2=C_1\ast {\omega }^2$$

(12)

$$C_3=C_2\ast {\omega }^3$$

(13)

$$O=vec\left(add\left(C_1,C_2,C_3\right)\right)$$

(14)

where vec stands for spread operation and add stands for add-by-element operation.

3.2.3. Scoring Functions

To calculate the score of (h,r,t), we project O onto M^d using the fully connected layer, then calculate the similarity of the inner product of this vector with the tail vector to measure the plausibility of the triplet (h,r,t). The scoring function φ(h,r,t) of the model ConvMVD can be expressed as:

$$\phi \left(h,r,t\right)=f{\left(O{W}_P\right)}^{T}t$$

(15)

where f denotes the nonlinear activation function and W_P denotes the characteristic projection matrix.

4. Experiment

In this section, we begin by presenting the experimental setup of our study, including the description of the datasets used, the configuration of hyperparameters, and the evaluation metrics. The experiments are divided into three parts. The first part is the validity and stability of this paper’s model on different data sets and comparison with other KGE models. The second part consists of conducting ablation experiments to verify the superiority of our model by analyzing the impact of various components or techniques used. The third part focuses on hyperparameter optimization experiments to identify the optimal combination of hyperparameters that yield the best performance for our model.

4.1. Experimental Setting

4.1.1. Dataset

In this paper, experiments are conducted on the standard dataset FB15K-237 and the hydraulic engineering dataset Hydraulic_eng.

FB15K-237: FB15K is extracted from Freebase, an extensive knowledge base that includes information about movies, celebrities, sports teams, prizes, and so on. FB15K-237 removes reversible relationships from FB15K. The FB15K-237 dataset contains a total of 14541 entities and 237 relationships.

Hydraulic_eng: The dataset “Hydraulic_eng” is constructed with a collection of data including case studies in hydraulic engineering, fundamental knowledge of hydraulic engineering, and hydrological geographical information. It consists of 6088 entities and 509 relationships.

4.1.2. Hyperparameter Setting

In this paper, we use the grid search method for hyperparameter tuning, using Adam as the optimizer, and iterative training on the public dataset FB15K-237 and the hydrological dataset Hydraulic_eng. The hyperparameters of the grid search range as follows: the range of learning rate is {0.0001, 0.0003, 0.0005, 0.0007, 0.0009, 0.001, 0.002, 0.003, 0.004, 0.005}, the range of batch_size is {32, 48, 64, 80, 96, 112, 128, 144, 160, 176}, and the range of the embedding size is {150, 200, 250, 300, 350, 400}.

The following parameter settings are consistent across all datasets, the number of multiscale branches is set to 3, the hole rate of the i-th branch is set to i, and the standard convolutional kernel size is 3 × 3. The remaining hyperparameters are set as shown in

Table 1. Experimental hyperparameter setting.

Dataset	LR	Batch Size	Embedding Size	Input Dropout	Feature Dropout	Hidden Dropout
FB15K-237	0.001	64	350	0.2	0.3	0.2
Hydraulic_eng	0.001	64	350	0.2	0.3	0.2

, where LR is the learning rate, input dropout, feature dropout, and hidden dropout denote the dropout rates of input layer, feature map and hidden layer, respectively.

4.1.3. Evaluation Indicators

This experiment uses the link prediction task to verify the validity of the model. In the link prediction task, the metric mean reciprocal ranking (MRR), Hits@n, is usually used as the final evaluation metric of the model. For a given test triad (h,r,t), we replace the tail entity t with all other entities e ∈ E to generate candidate triads, and then score the given triad (h,r,t) and the candidate triads together, and rank the scores from low to high. MRR is the mean reciprocal ranking, which indicates the reciprocal of the average ranking of the real triad in the score ranking list. higher MRR values indicate better model better performance. It is calculated as follows:

$$MRR=\frac{1}{\left|S\right|}{\displaystyle \sum }_{i=1}^{\left|S\right|}\frac{1}{ran{k}_i}$$

(16)

where S is the set of triples, $\left|S\right|$ is the number of sets of triples, and rank_i refers to the link prediction rank of the ith triple.

The indicator Hits@n refers to the proportion of correct triples in the top n rankings, and a larger value of this indicator indicates better model performance, generally with a value of 1, 3 or 10 for n. It is calculated as follows:

$$HITS@n=\frac{1}{\left|S\right|}{\displaystyle \sum }_{i=1}^{\left|S\right|}\mathit{II}\left(ran{k}_i\le n\right)$$

(17)

The above formula involves the same symbols as those involved in the MRR calculation formula and, in addition, II(·) is the indicator function (if the condition is true then the function value is 1, otherwise it is 0). Both MRR and Hits@n fractions range from 0 to 1 [32].

4.2. Experimental Results and Analysis

4.2.1. Link Prediction Experiments

In this paper, TransE, DistMult, ConvE, ConvR, InteractE, and JointE are used as the comparison models and, to ensure a fair comparison, these results are used with exactly the same data preprocessing and measurement methods. The accuracy results of each model on the FB15K-237 dataset and the Hydraulic_eng dataset are shown in

Table 2. Experimental results of link prediction for FB15K-237 dataset.

Method	MRR	Hits@10	Hits@3	Hits@1
TransE	0.287	0.475	0.325	0.192
DistMult	0.178	0.352	0.204	0.092
ConvE	0.316	0.491	0.350	0.239
ConvR	0.350	0.528	0.385	0.261
InteractE	0.353	0.541	0.390	0.260
JointE	0.356	0.543	0.393	0.262
ConvMVD	0.363	0.565	0.421	0.236

and

Table 3. Experimental results of link prediction for Hydraulic_eng dataset.

Method	MRR	Hits@10	Hits@3	Hits@1
TransE	0.082	0.167	0.086	0.031
DistMult	0.073	0.148	0.074	0.025
ConvE	0.338	0.651	0.394	0.196
ConvR	0.317	0.618	0.373	0.179
InteractE	0.342	0.655	0.401	0.199
JointE	0.341	0.655	0.40	0.198
ConvMVD	0.349	0.665	0.411	0.204

and Figure 3, respectively. To verify the validity of the model, the experimental results use the average of the results of 500 iterations. The mean statistic reflects the average level of the model prediction, and the higher the mean values of the indicators MRR, Hits@10, Hits@3 and Hits@1, the better the average level of the model prediction. The optimal results are indicated using bold.

From Table 2 and Table 3, it can be observed that deep models have significant advantages over translation-based and bilinear models in handling knowledge graph embedding tasks. Link prediction results on the FB15K-237 dataset demonstrate that ConvMVD consistently achieves the best performance across multiple evaluation metrics. Compared to ConvE, ConvR, InteractE, and JointE models based on convolutional neural networks, ConvMVD shows improvements of 14.8%, 3.7%, 2.8%, and 2% in terms of MRR, and improvements of 35.4%, 25.9%, 22.9%, and 22.5% in terms of Hits@10, and improvements of 20.3%, 9.4%, 7.9%, and 7.1% in terms of Hits@3, respectively. These experimental results collectively indicate that the proposed model performs well in knowledge graph embedding tasks.

To validate the applicability of the ConvMVD model in hydraulic knowledge graph completion tasks, we conducted link prediction experiments on the Hydraulic_eng dataset. The experimental results, as shown in Table 3, demonstrate that ConvMVD achieves the best performance across all four evaluation metrics. Compared to the ConvE and ConvR models, ConvMVD shows improvements of 3.3% and 10.1% in terms of MRR, respectively. The performance difference is relatively small when compared to InteractE and JointE models, with improvements of 2% and 2.3% in MRR, respectively. These experimental findings indicate that the proposed model can be integrated into hydraulic engineering knowledge graph completion tasks, resulting in a more comprehensive and knowledge-rich knowledge graph. This lays the foundation for providing comprehensive and efficient hydraulic knowledge services.

In order to verify the stability of the models, 150 Monte Carlo simulations were conducted for each model to calculate the variance statistics. The variance statistic reflects the degree of deviation of the model prediction, and the lower value of variance indicates the more stable model prediction. The variance of different models is shown in

Table 4. Variance statistics of each model indicator for FB15K-237 dataset.

Method	MRR	Hits@10	Hits@3	Hits@1
TransE	1.41 × 10−5	2.38 × 10−5	1.87 × 10−5	4.51 × 10−5
DistMult	0.98 × 10−5	1.14 × 10−5	0.88 × 10−5	2.23 × 10−5
ConvE	0.0014	0.0035	0.0018	0.0007
ConvR	0.0021	0.0122	0.0041	0.0019
InteractE	0.0016	0.0034	0.002	0.0014
JointE	0.001	0.0052	0.0035	0.0023
ConvMVD	0.0004	0.0014	0.0007	0.0001

and

Table 5. Variance statistics of each model indicator for Hydraulic_eng dataset.

Method	MRR	Hits@10	Hits@3	Hits@1
TransE	0.0002	0.0009	0.0002	3.41 × 10−5
DistMult	0.0001	0.0006	0.0001	1.61 × 10−5
ConvE	0.0030	0.0079	0.0047	0.0017
ConvR	0.0046	0.0143	0.0069	0.0022
InteractE	0.0031	0.0078	0.0049	0.0018
JointE	0.0036	0.0089	0.0056	0.0020
ConvMVD	0.0022	0.0057	0.0036	0.0012

.

As can be seen from Table 4 and Table 5, where the values of variance statistics for both the shallow models TransE and DistMult are the smallest; however, combined with the results of the linked prediction experiments above, it is clear that the shallow models have the lowest prediction average. Compared with the convolutional neural network-based models ConvE, ConvR, InteractE, and JointE, the variance statistics of each metric of ConvMVD are the lowest.

The above experimental results show that the model in this paper has good effectiveness and stability compared with other benchmark models.

4.2.2. Ablation Experiments

To evaluate the impact of different convolutional approaches on model performance, we conducted an ablation experiment on the Hydraulic_eng dataset. In the ablation experiment, we replaced the multi-scale dilated convolution in the model with standard convolution and single-scale dilated convolution, respectively, for comparative analysis. The experimental results are shown in Figure 4.

In Figure 4, “Standard” represents the experimental results using standard convolution, “Single_scale” represents the experimental results using single-scale dilated convolution with a dilation rate of 2, and “Multiscale” represents the experimental results using multi-scale dilated convolution with dilation rates of [1, 2, 3]. The results demonstrate that, when using standard convolution or single-scale dilated convolution, all four evaluation metrics show varying degrees of decrease. This further validates the importance and effectiveness of mixed dilated convolutions in the model. Through the ablation experiment, we gained a deeper understanding of the impact of different convolutional approaches on model performance, providing valuable experimental evidence for model optimization and further research.

4.2.3. Hyperparameter Optimization Experiment

In the hyperparameter optimization experiments, batch_size and learning rate parameter selection experiments, and embedding size selection experiments are conducted, and the results of these experiments provide important guidance to help us choose the appropriate hyperparameter configuration to optimize the performance of the ConvMVD model. The results are shown in Figure 5 and Figure 6, respectively.

The first are the batch size and learning rate parameter optimization experiments, and the experimental results are presented in the heat map in Figure 5. In this figure, the horizontal coordinate indicates the Batch_Size and the vertical coordinate indicates the learning rate (LR), while the shades of the colors reflect the magnitude of the values of the evaluation metrics under the corresponding hyperparameter combinations. Figure 5a–d correspond to the MRR metric level, Hits@10 metric level, Hits@3 metric level and Hits@1 metric level for different hyperparameter combinations, respectively. It should be noted that a darker color in Figure 5 indicates a larger the value of each evaluation indicator under the corresponding hyperparameter combination, which indicates a better model performance. Therefore, we can conclude that the model ConvMVD exhibits optimal performance for each evaluation metric at a learning rate of 0.001 and a batch_size of 64.

Indeed, the embedding dimension can also impact the model’s performance. If it is too small, the model may struggle to fit the data adequately, while if it is too large, the model may overfit. In the embedding dimension selection experiment, we considered sizes of 150, 200, 250, 300, 350, and 400. The results are presented in Figure 6, with the optimal results highlighted in bold. From Figure 6, it can be observed that when the embedding dimension is set to 350, the ConvMVD model achieves optimal performance in most of the evaluation metrics. This suggests that an embedding dimension of 350 leads to the best results for the ConvMVD model.

5. AI-Based Formal Methods for Verification

The combination of formal methods and artificial intelligence techniques, known as AI-based formal methods, is a methodology that integrates formal methods with AI technologies. Formal methods [33] are an approach based on mathematics and logic that provides precise specifications and verifies system behavior. On the other hand, artificial intelligence technologies encompass machine learning, natural language processing, knowledge representation and reasoning, and other techniques [34].

In formal methods, the specifications and properties of a system are typically described using formal languages such as mathematical formulas, logical expressions, or models. However, traditional formal methods often require manual authoring and verification of specifications, which can become challenging and time-consuming for complex systems. Artificial intelligence technologies can assist in automating and simplifying the application and verification process of formal methods. For instance, machine learning can automatically infer and generate new specifications and properties by learning from existing patterns. Natural language processing techniques can aid in understanding and interpreting natural language descriptions of formal specifications and properties. Knowledge representation and reasoning techniques can handle and reason about complex formal specifications and properties.

By combining artificial intelligence techniques with formal methods, more efficient and intelligent solutions can be provided for system verification and specification:

• Automated specification and property generation: utilizing methods like machine learning, automatic learning and generation of formal specifications and properties can be achieved from existing system behavior data;
• Intelligent verification tools: developing intelligent verification tools using natural language processing and knowledge representation and reasoning techniques enables non-experts to easily apply formal methods for system verification;
• Efficient verification algorithms: improving the verification algorithms of formal methods through the application of machine learning and optimization techniques to enhance verification efficiency and accuracy;
• Reinforcement of formal verification: integrating techniques such as reinforcement learning to automate the formal verification process, enhancing the robustness and adaptability of verification.

However, AI-based formal methods also face certain challenges. For example, the verification process can be influenced by factors such as algorithm selection, data biases, and feature extraction. Additionally, the interpretation and explainability of verification results can be problematic, since AI technologies often operate as black-box models, making it difficult to provide detailed explanations and visualizations.

In conclusion, AI-based formal methods have tremendous potential in verifying complex systems. By combining formal specifications with AI technologies for automated verification, we can effectively analyze and validate system correctness and performance, thereby improving the quality and reliability of system designs.

6. Conclusions

In this paper, we propose a ConvMVD knowledge graph embedding model for the hydraulic knowledge graph complementation task. The experiments are evaluated using a link prediction task, and it is confirmed that most of the metrics of this paper’s model are significantly improved over the baseline models on both public and hydraulic datasets. The experimental results show that the ConvMVD model can effectively improve the feature interaction effect and enhance the knowledge representation of entities and relationships by adding entity-relationship spatial location features, fusing different levels of feature information, and taking advantage of different perceptual fields of convolutional kernels at different scales.

Although ConMVD achieves good results on the KGE tasks, the model in this paper focuses on the extraction of entity relationship feature information inside the triad, and does not consider introducing more external knowledge, such as description information and attribute information of hydraulic engineering objects, into the model. To address this limitation, we will consider incorporating the descriptive text information of the entities themselves and the path information in the knowledge graph in future work to further improve the performance of the model on the task of embedding the knowledge graph of hydraulic engineering. And, formal methods based on verification of artificial intelligence techniques will be considered to validate the robustness, reliability and interpretability of the proposed model.