DL-Aided Underground Cavity Morphology Recognition Based on 3D GPR Data

Abstract

Cavity under urban roads has increasingly become a huge threat to traffic safety. This paper aims to study cavity morphology characteristics and proposes a deep learning (DL)-based morphology classification method using the 3D ground-penetrating radar (GPR) data. Fine-tuning technology in DL can be used in some cases with relatively few samples, but in the case of only one or very few samples, there will still be overfitting problems. To address this issue, a simple and general framework, few-shot learning (FSL), is first employed for the cavity classification tasks, based on which a classifier learns to identify new classes given only very few examples. We adopt a relation network (RelationNet) as the FSL framework, which consists of an embedding module and a relation module. Furthermore, the proposed method is simpler and faster because it does not require pre-training or fine-tuning. The experimental results are validated using the 3D GPR road modeling data obtained from the gprMax3D system. The proposed method is compared with other FSL networks such as ProtoNet, R2D2, and BaseLine relative to different benchmarks. The experimental results demonstrate that this method outperforms other prior approaches, and its average accuracy reaches 97.328% in a four-way five-shot problem using few support samples.

1. Introduction

Urban areas around the world continue to experience a series of sudden sinkhole collapses that cause severe traffic disruptions and significant economic losses. Underground cavities are the main reason for the formation of sinkholes. Complex conditions such as changes in drainage patterns, excessive pavement loads, and disturbances in infrastructure construction often lead to various cavities [1]. Therefore, the cavities may vary in morphology, for example, cavities with different shapes or combinations of several basic shapes, or they may be filled with different media or have different positions and sizes. Due to the unpredictability and morphological complexity of cavities, there is a growing need for their early recognition.

Ground-penetrating radar (GPR) has gradually been applied to the detection and perception of underground cavities [2], owing to its nondestructive inspection, strong penetrating ability, and high-precision characteristics. GPR transmitters emit electromagnetic (EM) waves into the surface at multiple spatial positions, and then the reflected signal can be measured by the GPR receiver to establish a two-dimensional (2D) GPR image. Three-dimensional (3D) GPR images can be obtained immediately when multichannel GPR transmitters and receivers exist parallel to the scanning direction at the same time [3,4]. The morphological scale of a cavity can reflect the evolution speed of the cavity and the severity of future road collapse, and it can also accurately reflect the 3D space state of the cavity. Therefore, the accurate detection of morphology has scientific value for studying the mechanism of cavity formation and summarizing the corresponding prevention and repair methods. However, as the quantity of the 3D GPR data increases, the manual analysis of the GPR data becomes time-consuming and difficult to meet the requirements of the efficient and fine detection of cavity morphology.

Three challenges in automating this task cannot be ignored. The first challenge is the selection and extraction of morphological features. Environmental complexities, such as the interference of surrounding pipelines and groundwater leakage, may impose difficulties in describing cavity morphological features. It is impossible to comprehensively describe the reflection properties with one or a few features. Additionally, the acquired morphological attributes still need to be inferred and identified by experienced professionals, making it difficult to obtain a general description feature. The second challenge is the fine classification of cavity morphology in the GPR data. Due to the variety of types, varying sizes, distinct directions and extensions, and irregular shapes, cavity morphology analysis faces difficulties in identification and fine classification. The third challenge is to address the issue of insufficiently labeled GPR data. Compared with objects such as buried rebars and pipes, the scale and diameter distribution of a cavity with collapse threat is relatively large. It is very laborious to make such a large target in the lab, and the targets are generally in regular forms, not universal and representative. To obtain reliable results, the lack of sample data must be considered and addressed.

Therefore, in this study, a novel deep learning (DL)-aided framework was proposed to extract the morphological features of cavities and classify them in facing a small number of 3D GPR data. Figure 1 shows the details of the proposed framework. The contribution of this work is twofold:

(i)

First, a joint characterization algorithm was developed for cavity morphology that generates 2D morphological images and fully exploits 3D GPR spatial information;

(ii)

Second, we implemented a novel few-shot learning (FSL) network for cavity morphology classification and embedded a relation network (RelationNet) into the FSL model to adapt to different few-sample cavity scenarios.

The rest of this paper is organized as follows: Section 2 introduces the literature review of the GPR cavity detection. In Section 3, the imaging scheme of the 3D GPR data is proposed. Section 4 introduces the details of the FSL network and RelationNet structure for morphological classification. In Section 5, the experimental results are compared and analyzed, and finally, in Section 6, conclusions are drawn.

2. Literature Review

Previous studies have focused on the automated GPR cavity detection process. Qin et al. [5] proposed a pattern recognition method based on the support vector machine (SVM) classifier to identify cavities in GPR images. Park et al. [6] combined instantaneous phase analysis with the GPR technique to identify hidden cavities. Hong et al. [7] developed a new time-domain-reflectometry-based penetrometer system to accurately estimate the relative permittivity at different depths and estimate the state of a cavity. Yang et al. [8] constructed a horizontal filter to identify cavity disease and eliminate the interference of rebar echo. Based on the data collected by multisensors such as unmanned aerial vehicles (UAVs) and GPR, the authors of [9,10] detected and analyzed cavity diseases in disaster-stricken areas to rescue potential victims trapped in cavities. In 2022, Rasol et al. [11] reviewed state-of-the-art processing techniques such as machine learning and intelligent data analysis methods, as well as their applications and challenges in GPR road pavement diagnosis. To better localize pavement cracks and solve the interference of various factors in the on-site scene, Liu et al. [12] integrated a ResNet50vd-deformable convolution backbone into YOLOv3, along with a hyperparameter optimization method. To detect subsurface road voids, Yamaguchi et al. [13] constructed a 3D CNN to extract hyperbolic reflection characteristics from GPR images.

Previous results were based on the processing of only B-scans; however, once faced with specific subsurface objects, it was difficult to classify them using B-scans alone. In particular, the characteristics of various cavities in GPR B-scan images tended to be similar. Therefore, to improve the classification performance, both the GPR B-scan and C-scan images were considered in the classification process using the DL network [14,15,16,17]. Compared with the 2D GPR data, 3D data can provide rich spatial information and greatly improve the process in terms of data volume, imaging methods, and disease detection accuracy. Luo et al. [18] established a cavity pattern database including C-scans and B-scans, where the C-scan provides location information of objects, and B-scan information assists in verifying object types. Kim et al. [19] proposed a triplanar convolutional neural network (CNN) for processing the 3D GPR data, enabling automated underground object classification. Kang et al. [20] designed the UcNet framework to reduce the misclassification of cavities, and the next year, Kang et al. [21] developed a transfer-enhanced CNN to improve the classification accuracy. Khudoyarov et al. [22] proposed a 3D CNN architecture to process the 3D GPR data. The authors of another study [23] visualized and distinguished underground hidden cavities from other objects (such as buried pipes, and manholes). In 2021, Kim et al. [24] used the AlexNet network with the transfer learning technology to achieve underground object classification, further improving detection accuracy and speed. Abhinaya et al. [25] detected cavities around sewers using in-pipe GPR equipment and confirmed that YOLOv3 [26] was suitable for cavity recognition tasks. Liu et al. [27] combined the YOLO model and the information embedded in 3D GPR images to address the recognition issue of road defects. The above research demonstrated that, compared with using only B-scan images, the developed CNNs using both the B-scans and C-scans improved the classification performance. However, it was found that the cavity morphology is still indistinguishable due to the difficulty of cavity data acquisition and the lack of a GPR database.

Faced with such a problem, FSL [28,29], as a novel DL technique, was developed to generalize the network with very few or fewer training samples for each class. This changes the situation where traditional DL models must require large quantities of labeled data. FSL can be divided into three categories: model-based, optimization-based, and metric-based learning methods [30]. The model-based learning method first designs the model structure and then uses the designed model to quickly update parameters on a small number of samples, and finally directly establishes the mapping function of the input and prediction values. Santoro et al. [31] proposed the use of memory augmentation to solve this task and a memory-based neural network approach to adjust bias through weight updates. Munkhdalai et al. [32] proposed a meta-learning network, and its fast generalization ability is derived from the “fast weight” mechanism, where the gradients generated during training are used for fast weight generation. The optimization-based learning method completes the task of small sample classification by adjusting the optimization method instead of the conventional gradient descent method. Based on the fact that gradient-based optimization algorithm does not work well with a small quantity of data, Ravi et al. [33] studied an updated function or rule for model parameters. The method proposed by Finn et al. [34] can deal with situations with a small number of samples and can obtain better model generalization performance with only a small number of training times. The main advantages of this method are that it does not depend on the model form, nor does it need to add new parameters to the meta-learning network. The metric-based learning method is developed to measure the distance/similarity between the training set and the support set and completes the classification with the help of the nearest neighbor method. Vinyals et al. [35] proposed a new matching network, which aims to build different encoders for the support set and the batch set, respectively. Sung et al. [36] proposed a RelationNet network to model the measurement method, which learns the distance measurement method by training a CNN network.

3. Imaging Scheme of 3D GPR Data

3.1. The 3D GPR Data Format

The GPR data comes in three forms: A-, B-, and C-scan. The transmitter radiates EM waves to the underground, and the receiver collects signals reflected by underground objects or stratum interfaces, so as to obtain underground information. The original data format of the reflected signal is a one-dimensional (1D) waveform, which can also be called a GPR A-scan waveform. By scanning a region of interest with the single-channel GPR system, a 2D radargram can be obtained, called a GPR B-scan image. A single C-scan image is formed by imaging data points at the same depth in multiple B-scan images.

As shown in Figure 2, the x-axis is the same as the scanning direction, the y-axis denotes the width direction of the radar antenna device, and the z-axis indicates the depth direction of the measured object. The 3D GPR data can be represented as two kinds of orthogonal planes: B-scan and C-scan. B-scan and C-scan are arbitrary sections perpendicular to y- and z-axes. This is conducive to identifying the cavity morphology from multiple different perspectives, which can effectively avoid the misjudgment of a single angle in the 2D image. Therefore, 3D GPR can guarantee the accuracy of the detection type and reduce the number of core sampling verifications.

3.2. GPR Morphological Data Extraction

The hyperbolic signature in B-scan images is a kind of typical characteristic that is often used in the detection of underground objects. The C-scan image can reflect the detailed shape information of the subsurface cavity. The B-scan and C-scan images contain morphological information in length–depth and length–width directions, respectively. The color on scan images reflects the field strength (V/m) of subsurface media. In this study, we propose an automatic algorithm for cavity morphology classification using GPR B-scan and C-scan images simultaneously.

The two forms are extracted from the 3D GPR data $S$. Take the irregular cavity as an example: (1) B-scan images are sequentially extracted parallel to the XOZ plane, and their stacked display is shown in Figure 3a. It can be expressed as $S_1=\left\{B_1,B_2,B_3,\cdots ,B_n\right\}$, $n=l_y/△y$, where $B_n$ represents each slice image, $S_1$ is a collection of $n$ B-scan slices, $l_y$, and $△y$ are the model space length and space step size in the y-axis direction. (2) C-scan images are sequentially extracted parallel to the XOY plane. The horizontal slices are extracted at equal intervals, and their stacked display is shown in Figure 3b. The slices can be expressed as $S_2=\left\{C_1,C_2,C_3,\cdots ,C_m\right\}$, $m=l_{\mathrm{z}}/△z$, where $C_m$ represents each C-scan image, $S_2$ is the set of $m$ C-scan slices, $l_z$ and $△z$ are the model space length and space step size in the z-axis direction.

The typical GPR B-scan and the corresponding C-scan images of an irregular cavity are shown in Figure 4 and Figure 5. The C-scan images (right) are intersectional layers of red lines on the B-scan images (left). In the B-scan image, the horizontal x-axis and the vertical t-axis indicate the GPR scanning trajectory (m) and the two-way travel time of EM wave (ns), respectively. In the C-scan image, the x- and y-axes indicate the GPR scanning trajectory (m) and the width of GPR equipment (m), respectively. As shown in Figure 4, the irregular cavity shows a hyperbolic pattern on the B-scan image and elliptical signatures on the C-scan image. As shown in Figure 5, the rectangular cavity shows a double hyperbolic signature on the B-scan image and quadrilateral signatures (Figure 5b) on the C-scan images. Therefore, different morphologies of cavities can be well-recognized using B-scan and C-scan images.

3.3. The 2D Morphological Image Generation

Multiple B-scan and C-scan images are integrated into a 2D morphological map, where the B-scan and C-scan images are assigned to the upper and lower parts of the morphological map, respectively. Each cavity model corresponds to a 2D morphological image $I=\left\{S_1,S_2\right\}$, which consists of 8 B-scan images $S_1=\left\{B_1,B_2,B_3,B_4,B_5,B_6,B_7,B_8\right\}$, and 12 C-scan images $S_2=\left\{C_1,C_2,C_3,C_4,C_5,C_6,C_7,C_8,C_9,C_{10},C_{11},C_{12}\right\}$. The morphological image $I$ consists of $S_1$ and $S_2$, which is formed into a new $5\times 4$ matrix and can also be expressed as Equation (1). Examples of the morphological images for each model are presented in Figure 6. The 2D morphological image is then used as the input of the following deep network.

$$I=\left\{\begin{array}{cccc}{B}_1& B_2& B_3& B_4\\ B_5& B_6& B_7& B_8\\ C_1& C_2& C_3& C_4\\ C_5& C_6& C_7& C_8\\ C_9& C_{10}& C_{11}& C_{12}\end{array}\right\}$$

(1)

4. Few-Shot Learning Designed for Morphology Classification

4.1. FSL Definition

FSL is able to quickly identify new classes on very few samples. It is generally divided into three kinds of datasets: training set, support set, and testing set. The training set can be further divided into a sample set and a query set. If the support set contains $K$ labeled examples for each of $C$ unique classes, the target few-shot problem is called $C$-way $K$-shot. Figure 7 shows the FSL architecture for a four-way one-shot problem.

The parameters $\theta$ are optimized by the training set, hyperparameters are tuned using the support set, and finally, the performance of function $f\left(x,\theta \right)$ is evaluated on the test set. Each sample $\widehat{x}$ is assigned a class label $\widehat{y}$. The data structure in the training phase is constructed to be similar to that in the testing phase; that is, the sample set $S$ and query set $Q$ during the training simulate the support set and testing set at the testing time. The sample set $S=\left\{\left(x_1,y_1\right),\left(x_2,y_2\right),\dots ,\left(x_m,y_m\right)\right\}$ is built by randomly picking $C$ classes from the training set with $K$ labeled samples, and the rest of these samples are used in the query set $Q=\left\{\left(x_1,y_1\right),\left(x_2,y_2\right),\dots ,\left(x_n,y_n\right)\right\}$.

4.2. Relation Network Architecture and Relation Score Computation

The relation network (RelationNet) is a typical metric-learning-based FSL method. In essence, metric-learning-based methods [36,37,38,39,40,41] compare the similarities between query images and support classes through a feed-forward pass through an episodic training mechanism [35]. The core of RelationNet is to learn a nonlinear metric through deep CNN, rather than selecting a fixed metric function. RelationNet is a two-branch architecture that includes an embedding module and a relation module. The embedding module is used to extract image features. The relation module obtains the correlation score between query images and sample images; that is, it measures their similarity, so as to realize the recognition task of a small number of samples.

Figure 8 represents the RelationNet architecture settings for FSL. The embedding module utilizes four convolutional blocks, and each convolutional block consists of a 64-filter $3\times 3$ convolution, a batch normalization, and a ReLU nonlinearity layer. In addition to the above, the first two convolutional blocks also include a $2\times 2$ max-pooling layer, and the latter two convolutional blocks do not contain the pooling layer. The output feature maps are then obtained for the following convolutional layers in the relation module. The relation module consists of two convolutional blocks and two fully connected layers. Each convolutional block is a $3\times 3$ convolution containing 64 filters, followed by batch normalization, ReLU nonlinearity, and $2\times 2$ max-pooling. For the network architectures, in order to generate relation scores within a reasonable range, in all fully connected layers, ReLU functions are employed, except for the output layer, in which Sigmoid is used.

The prior few-shot works use fixed pre-specified distance metrics, such as the Euclidean or cosine distances, to perform classification [35,42]. Compared with the previously used fixed metrics, RelationNet can be viewed as a metric capable of learning deep embeddings and deep nonlinearities. By learning the similarity using a flexible function approximator, RelationNet can better identify matching/mismatching pairs. Sample $x_j$ in the query set $Q$ and sample $x_i$ in the sample set $S$ are fed through the embedding module $f_{\phi }$ to produce feature maps $f_{\phi }\left(x_i\right)$ and $f_{\phi }\left(x_j\right)$, respectively. Then, these two feature maps $f_{\phi }\left(x_i\right)$ and $f_{\phi }\left(x_j\right)$ are combined using the operator $ℂ\left(f_{\phi }\left(x_i\right),f_{\phi }\left(x_j\right)\right)$. After that, the combined feature map is fed into the relation module $g_{\phi }$, which finally produces a scalar in the range of 0–1 to represent the similarity between $x_i$ and $x_j$, also called the relation score. Thus, the relation score $r_{i,j}$ is generated as shown in Equation (2):

$$r_{i,j}=g_{\varphi }\left(ℂ\left(f_{\phi }\left(x_i\right),f_{\phi }\left(x_j\right)\right)\right), \mathrm{i}=1,2,\dots ,\mathrm{C}$$

(2)

Here, the mean square error loss is computed to train the model, as shown in Equation (3), regressing the relation score $r_{i,j}$ to the ground truth: the similarity of matched pairs is 1, and the similarity of unmatched pairs is 0.

$$\phi ,\varphi \leftarrow \underset{\phi ,\varphi }{\arg \min }{{\displaystyle \sum _{i=1}^m{\displaystyle \sum _{j=1}^n\left(r_{i,j}-1\left(y_i==y_j\right)\right)}}}^2$$

(3)

4.3. RelationNet-Based Cavity Morphology Classification Scheme

Based on the data and structural characteristics of the GPR cavity morphological recognition system, we divide the complex processing process into two main parts: a training phase and a testing phase, as shown in Figure 9. The cavity morphologies are discussed here, e.g., cylindrical, rectangular, spherical, and irregular hemispherical. First, a training set is inputted to learn classification rules inside the network in the training phase. Then, in the testing phase, a small number of support samples (labeled) and test samples (unlabeled) are inputted into the trained network model, and the unlabeled test samples are predicted and classified, thereby outputting the final morphological classification results.

Based on the above principle, the RelationNet-based GPR cavity morphology classification system first obtains the trained model on the training set and then recognizes the new category of cavity images. The embedding model is used to extract the feature information of each inputted GPR image and then concatenates the image features between the test sample and support sample. Then, the integrated features are inputted into the relation model for comparison. According to the comparison results, it is judged to which category the test sample belongs, so as to achieve the classification of cavity morphologies.

5. Experiments and Results

5.1. Experimental Settings

The classification category should be labeled for each morphological image since the DL-based classification algorithm is essentially a supervised learning algorithm. The choice of category is related to the morphology of the cavity. There are four types of cavities involved, namely cylindrical cavity, rectangular cavity, spherical cavity, and irregular cavity. To train CNN, the 2D morphological images with each image size of 498 × 395 × 3 pixels were fed to the input layer, and then the convolutional layers were used to extract multilevel image features by convolution kernels. A total of 68 GPR morphological images were obtained, each consisting of 8 B-scans and 12 C-scans. Among them, there were 17 spherical cavity images, 17 cylindrical cavity images, 17 rectangular cavity images, and 17 irregular cavity images.

We used the PyTorch 1.5 implementations of the LibFewShot package [43], which is a comprehensive FSL library and integrates the most advanced FSL methods. The code was implemented using NVIDIA RTX 3080Ti GPU and Intel i9-12900K CPU. In the training phase, in the FSL in all experiments, we used the Adam optimizer [44] with an initial learning rate of 10−3 and step decay. The backbone adopted Conv64F. The batch size was set to 128. In the testing phase, the test epoch was set to 10, and the test episode was 17, the four-way one-shot contained 16 query images, and the four-way five-shot had 12 query images for each of the 4 classes in each training episode.

We compared the results against those of various networks for few-shot recognition, including ProtoNet [42], R2D2 [45], and BaseLine [46]. Embedding backbones, such as Conv64F, ResNet12, and ResNet18, were also compared to identify and select the main backbone with a better performance. The main experiments were conducted on two benchmark datasets: miniImageNet [35] and tieredImageNet [47]. The miniImageNet dataset [35] consists of 60,000 color images with 100 classes, and the input images are resized to 84 × 84. The tieredImageNet dataset [47] consists of 779,165 color images with 608 classes, each of size 84 × 84.

5.2. The 3D GPR Cavity Data Acquisition

The GPR data are difficult to collect and label. To address the issue, a 3D GPR forward modeling tool, GprMax3D [48,49], is often used for the 3D modeling and simulation of underground structures. Based on the finite-difference time-domain (FDTD) method, the technique increases the volume of training data by creating synthetic GPR images. Maxwell’s equations govern the propagation of EM waves used by the GPR. Figure 10 shows the flowchart of the GprMax3D forward simulation technique. GprMax3D was used to generate synthetic GPR images [50]. To further imitate the real situation, we set up cavity objects with uneven surfaces and random media to approximate real objects. An example of the GPR system’s parameter setting is shown in

Table 1. GPR system parameters in road structure scene.

System Parameters	Value
Spatial resolution/m	0.01
Time window/ns	14
Initial coordinate of transmit antenna/m	(0.45, 1.0, 0.0)
Initial coordinate of receive antenna/m	(0.35, 1.0, 0.0)
Antenna step distance/m	(0.01, 0, 0)
Measuring point number	100
Excitation signal type	Ricker
Excitation signal frequency/MHz	800

.

Road cavity is generally distributed in the underground range of 0.3–1 m, which is also the junction of the pavement structure and the subgrade. The subgrade is generally a soil structure, while the pavement structure is generally a flexible, semi-rigid, or rigid structure, generally made of cement or asphalt concrete; in particular, its bottom layer is generally made of hard materials such as gravel. Due to the high probability of soil erosion, cavities are most likely to appear at the junction of soft and hard layers. Figure 11 shows a simulation model example of a road structure with the first layer of asphalt, the second layer of concrete, and the third layer of sandstone. Their attribute parameter settings are shown in

Table 2. Dielectric properties of road structure.

System Parameters	Relative Permittivity	Conductivity (S/m)
Air	1	0
Asphalt	6	0.005
Concrete (dry)	9	0.05
Gravel	12	0.1

. In addition, the subgrade is generally dominated by soil, and a more realistic soil model was established by simulating random media. A number of soil dispersion materials were defined as follows: soil with 50% sand, 50% clay, sand density of 2.66 g/cm³, clay bulk density of 2 g/cm³, and volumetric water content ranging from 0.001 to 0.25. These materials were distributed over a model with a volume of $2\times 1.2\times 1$ m³ (in which the soil layers were randomly distributed).

Figure 12 presents the simulated models of four representative cavity morphologies: spherical, rectangular, cylindrical, and irregular hemispherical cavities. The first three cavities have smooth surfaces, and the last one shows an uneven surface. Figure 13 shows the representative 2D GPR morphological images of cavities. Figure 13a shows the case of spherical cavities with parabolic and circular features, which can be observed on the B-scan and C-scan images, respectively. As shown in Figure 13b, rectangular cavities generally have distinguishable features, namely a double parabola shape in B-scans. Similar features are also revealed in the cylindrical case of Figure 13c. However, Figure 13b,c can be distinguished by C-scans because they, respectively, show quadrilateral signatures and double circular intersection features in C-scans. Compared with Figure 13a, it can be observed in Figure 13d that the C-scan images of the irregular cavity show unsmoothed circular features with considerable noise randomly distributed inside the circle. Due to these distinguishable and representative characteristics, the cavity morphology can be classified well with the FSL frameworks.

5.3. Classification Results and Analysis

Figure 14 shows the RelationNet-based underground cavity morphology classification results. These results were obtained based on the network settings of the backbone Conv64F, the benchmark dataset tieredImageNet, and in a four-way five-shot problem. As expected, compared with the ground truth, the irregular hemispherical cavity with significant characteristics in the GPR morphological image was correctly classified. Moreover, the classification accuracy rates of spherical, rectangular, and cylindrical cavities were 97.5%, 98.33%, and 98.33%, respectively. However, 0.83% and 1.67% of spherical were misclassified as cylindrical and hemispherical due to their similar morphological features. In addition, 1.67% of rectangular were misclassified as hemispherical due to their double parabola shapes in B-scans, while 0.83% and 0.83% of cylindrical were, respectively, misclassified as spherical and hemispherical, due to their similar features in B- or C-scans. The classification performance of RelationNet was evaluated based on the indices (precision, recall, and F-score) using the following Equations (4)–(6):

$$precision=\frac{true positive}{true positive+false positive}$$

(4)

$$recall=\frac{true positive}{true positive+false negative}$$

(5)

$$F-score=\frac{2\cdot precision\cdot recall}{precision+recall}$$

(6)

Table 3. Statistical results obtained from RelationNet (%).

System Parameters	Precision	Recall	F-Score
Spherical	99.15	97.5	98.32
Rectangular	100	98.33	99.16
Cylindrical	99.16	98.33	98.74
Hemispherical	96	100	97.96

shows the statistical results obtained from the RelationNet results, namely the precision, recall, and F-score values. For the rectangular cases, 99.15% precision and 97.5% recall indicated that false-positive (FP) and false-negative (FN) results occurred, and the number of FN results was greater than those of FP. Similarly, the 99.16% precision and 98.33% recall values of the cylindrical case meant the FP and FN alarms because RelationNet sometimes recognized a cylindrical cavity as a spherical or hemispherical. For rectangular cases, 100% precision and the relatively low recall value indicated that FN occurred due to the misclassification between rectangular and hemispherical. On the contrary, 96% precision and 100% recall values meant that the hemispherical samples were properly classified using RelationNet, but multiple samples in other categories were misclassified as hemispherical at the same time. According to the F-scores, it can be concluded that the performance of RelationNet was acceptable.

5.4. Comparison Experiments

5.4.1. RelationNet Evaluation on Different Embedding Backbones

The performance of RelationNet relies on the quality of the embedding backbone. To select a suitable main embedding backbone, we compared three different embedding backbones: Conv64F, ResNet12, and ResNet18. The above experiments were run for 10 epochs on the miniImageNet dataset. Other settings remained the same. Conv64F consisted of four convolutional blocks, and each block was composed of a convolutional layer, a batch-normalization layer, a ReLU layer, and a max-pooling layer. ResNet12 consisted of four residual blocks, each of which contained three convolutional blocks along with a skip connection layer. ResNet18 had the same architecture as used in [50].

Table 4. Comparison results on different embedding backbones (%) (the best results in bold).

Embedding Backbones	Four-Way One-Shot	Four-Way Five-Shot
Conv64F	78.097	88.934
ResNet12	69.467	72.500
ResNet18	69.926	79.865

shows the comparison results among different backbones. It can be observed that RelationNet equipped with Conv64F backbone achieved the best performance in either a four-way one-shot or four-way five-shot problem.

5.4.2. RelationNet Evaluation on Different Benchmark Datasets

Table 5. Comparison results on different benchmark datasets: miniImageNet vs. tieredImageNet (%) (the best results in bold).

Embedding Backbones	Four-Way One-Shot	Four-Way Five-Shot
miniImageNet	78.097	88.934
tieredImageNet	77.086	97.328

compares the performance of RelationNet on miniImageNet and tieredImageNet datasets by controlling the most implementation details. For this comparison, RelationNet adopted Conv64F as the main backbone. As can be seen from Table 5, in the four-way one-shot problem, the accuracy achieved on the miniImageNet dataset was slightly higher than that on the tieredImageNet dataset. In the four-way five-shot problem, competitive accuracy could be achieved using the tieredImageNet dataset compared with using the miniImageNet dataset, improving the accuracy by 8.394%. Therefore, the accuracy results indicated that RelationNet performed well when trained on the tieredImageNet dataset.

5.4.3. Performance Comparison of Different FSL Networks

To validate the effectiveness of RelationNet, the four experimental validation results of ProtoNet, R2D2, BaseLine, and RelationNet were compared. We used the exact same embedding backbone Conv64F and benchmark dataset miniImageNet. It can be observed from

Table 6. Statistical results obtained from RelationNet (%) (the best results in bold).

FSL Networks	Four-Way One-Shot	Four-Way Five-Shot
ProtoNet	70.965	85.221
R2D2	76.562	88.659
BaseLine	66.103	83.505
RelationNet	78.097	88.934

that RelationNet achieved the best results in the four-way one-shot problem, even improving the accuracy by 11.994% over the BaseLine. With the increase in the number of sample images, these four frameworks achieved substantial improvements in facing the four-way five-shot problem, and RelationNet still achieved the highest accuracy over the other three frameworks.

6. Conclusions

In this paper, we first applied the FSL technique to classify and identify cavity morphology characteristics based on the 3D GPR data. RelationNet was adopted as the FSL framework and trained end-to-end from scratch. Based on the advantages of learning a deep distance metric, RelationNet addressed the issue of insufficient cavity data and obtained the classification results using only a few samples. The experiment results demonstrated the effectiveness of using RelationNet in morphology classification performance. The RelationNet model achieved an average classification accuracy value of 97.328% in the four-way five-shot and 78.097% in the four-way one-shot problem.

There is a limitation that could be addressed in future research. In the experiments, all the models were trained on the source domain (e.g., miniImageNet and tieredImageNet) and directly tested on the target domain (e.g., cavity radar dataset). However, the performance hardly improved or significantly dropped when there was a large domain shift. In this paper, based on the fact that there was no intersection between the source set and our cavity radar dataset, there was a large domain offset between the source and target domains. Future efforts need to be made to integrate prior knowledge into FSL or explore one-shot or zero-shot classification methods.

For on-site applications, there are two limitations that could be addressed in future research. First, the real cavity data are difficult to collect for training the proposed method. Additionally, the publicly available GPR cavity datasets are limited. Efforts need to be made in the future to collect and prepare GPR datasets to facilitate the implementation of this method. Second, the proposed method was only tested for cavity morphology classification using the GprMax3D data. The scalability of the method in other challenging environments and applications needs further investigation. Future studies could test this method for collecting cavity data and classifying their morphologies in on-site city roads.