**1. Introduction**

In recent years, permeable pavement has been widely used in developed countries as part of low-impact development practices [1]. The main benefit of permeable pavement is that it can allow rainwater to pass through its pores into the ground below by filtering rainwater over the distinct layers, thereby reducing the excessive volume of waterlogging. The runoff volumes and discharge rates can be reduced from paved surfaces and the high risk of downstream flooding can be decreased. Additionally, permeable pavement contributes water quality enhancement. It can trap stormwater pollutants and prevent them from reaching downstream receiving waters [2]. Other benefits of using this special pavement are skid resistance, noise control, and surface temperature reduction [3].

However, there are some drawbacks to using permeable pavement. First, it is expensive owing to its special design. Permeable pavement consists of aggregate components with different materials such as concrete mixtures with cement, polymer, and plastic [4]. Second, it has a high maintenance cost, especially for cleaning the pavement. An industrial vacuum is needed to remove the particles that can block the spaces in the pavement. If it is not maintained properly, the water and pollutants can run off the surface, which can

**Citation:** Feri, L.E.; Ahn J.; Lutfillohonov, S.; Kwon, J. A Three-Dimensional Microstructure Reconstruction Framework for Permeable Pavement Analysis Based on 3D-IWGAN with Enhanced Gradient Penalty. *Sensors* **2021**, *21*, 3603. https://doi.org/10.3390/ s21113603

Academic Editors: Rashid Mehmood, Juan M. Corchado and Tan Yigitcanlar

Received: 28 April 2021 Accepted: 20 May 2021 Published: 21 May 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

cause flooding. Finally, it is prone to the clogging problem that is closely related to the maintenance issue. Clogging is the reduction in porosity and permeability when fine particles, sand, or clay block the spaces or pores between the pavers. The quality of permeable pavement can be measured using the clogging potential value. Clogging potential is the ratio of porosity or permeability reduction because of clogging to the initial porosity or permeability in the unclogged state [5].

Several experimental studies related to clogging in permeable pavement have been reported by researchers over the years to develop an ideal pavement design [6–9]. A good permeable pavement should have a small clogging potential value. According to the researchers, we need to understand the effects of different pore structures to yield a permeable pavement that is less susceptible to clogging [5].

To analyze the clogging potential in permeable pavements, some important properties of pavement microstructure need to be measured. Microstructure is the small-scale structure of a material that can be viewed through a microscope. The properties of microstructure are very important for understanding the material structure and studying the mechanical behavior of a material. In permeable pavement, three crucial properties for analysis are (1) porosity, (2) permeability, and (3) hydraulic conductivity [5,10]. Porosity is a fraction of the volume of pores over the total volume of a material. It is considered as a most important property since it affects the flow rates in a permeable pavement. Permeability is defined as the ability of a porous material to transmit fluids, and hydraulic conductivity is the ease with which a fluid (water) can move through porous surfaces [11]. Along with these properties, some morphological aspects also need to be carefully measured to study the clogging phenomenon in a porous medium like the permeable pavement [12].

Traditionally, to measure the clogging potential properties, we needed to conduct field experiments [6,13] using real physical materials. These are high in cost because the experiments require several samples and instruments that must be installed in the real field. The setting of a test area is another aspect that can be expensive, especially for conducting a large-scale field experiment. It is also time consuming, especially in large-scale experiments, owing to the equipment setup, material selection, and test-area preparation, which involves an intricate process. Thus, a more efficient method to measure the clogging potential properties is needed. Alternatively, they can be measured through visual experiments with image analysis and computer simulation.

A virtual experiment is another alternative to estimate the clogging potential properties [14,15]. It is more efficient in comparison to field experiments because it uses fewer samples and equipment for the experiment, and the required time is also not high as required in a field experiment. For this type of experiment, the most important aspect is the preparation of a good three-dimensional model for computer simulation. Before the upsurge of deep learning in image processing, researchers used statistical methods and some basic machine learning techniques such as support vector machines and the genetic algorithm [16,17] to reconstruct a 3D microstructure. Another method uses stochastic models such as a Gaussian random field [18] combined with hybrid optimization that generates a 3D porous material structure using a two-point correlation function and a cluster correlation function. The most common method for 3D reconstruction is the two-point correlation method [19–21]. It characterizes the microstructure based on certain statistical features and then performs an optimization process to build a 3D structure that matches those statistical features.

However, there are some limitations to the basic machine learning techniques and statistical approach. They require predefined knowledge of the materials [19], many different samples, and the generated model sometimes does not resemble the real sample. In the statistical case, there is a possibility that two materials with different properties may share first- and second-order statistical information and chord-distribution functions. Therefore, it has failed to yield good estimates of the macroscopic properties [20]. In brief, previous approaches did not yield realistic 3D microstructure reconstruction due to lack of (1) many different samples and (2) a good generative model.

To overcome these limitations, we propose a deep-learning-based 3D microstructure reconstruction framework for permeable pavement analysis as a virtual experiment method. Our system first takes a two-dimensional image to extract the latent features of the image. Then, it generates the corresponding 3D microstructure image that can be obtained through the concept of the generative adversarial network [22,23]. Since it is hard to achieve stability in the original generative adversarial network during the training process, we decided to implement a 3D improved Wasserstein generative adversarial network (3D-IWGAN) [24,25] with an enhanced gradient penalty. One of the main advantages of utilizing this generative adversarial network is that it can produce sharper, even degenerate distributions, while other 3D GAN-based methods show instability in generated results. Thus, compared to other methods [22], our framework achieves high profit from the enhanced gradient penalty and keeps the stability during the training process.

The main contributions of this paper are summarized as follows:


The remainder of this paper is organized as follows. Section 2 provides a brief overview of related research work. Section 3 presents some preliminaries of GANs. In Section 4, the 3D model generation and analysis of the generated model are explained in detail. The experiment and evaluation of the experimental results are discussed in Section 5. Finally, the conclusion of our work and future work are presented in Section 6.

### **2. Related Work**

In this section, we provide a brief review of the related work. There are three categories to be discussed: (1) microstructure reconstruction approaches, (2) general 2D/3D object reconstruction approaches, and (3) deep-learning-based microstructure reconstruction approaches.

### *2.1. Microstructure Reconstruction*

Microstructure is the small-scale structure of a material that can be viewed through a microscope. In the fields of materials science and civil engineering, microstructure properties are very important to understand the structure of a material and study its mechanical behavior.

The most common and earlier work for 3D microstructure reconstruction uses the two-point correlation method [19–21]. It characterizes the microstructure based on certain statistical features and then performs an optimization process to build a 3D structure that matches those statistical features. After this, statistical methods and basic machine learning algorithms were applied for reconstruction of 3D microstructure. Some of the well-known machine learning methods include support vector machines (SVMs) [16] and the genetic algorithm [17]. Among others, stochastic methods have shown better results for the given tasks. For example, Jiang et al. [18] proposed a method that exploits a Gaussian random field with the combination of hybrid optimization. It uses a two-point correlation function and a cluster correlation function to generate a 3D porous material structure.

### *2.2. General 2D/3D Object Reconstruction*

After the successful results shown by convolutional neural networks (CNNs) in the field of image classification and computer vision, researchers started considering using CNNs and other deep learning methods in their research on permeable pavement.

Since the core functionality of GAN enables us to generate a set of realistic (2D) images from the given sample, thus it is extended to generate 3D images (objects) [26]. Although GAN can accurately visualize the results, it suffers from a major drawback: the instability of GAN training. To guarantee the stability of GAN training, researchers proposed to leverage the Wasserstein distance (WGAN) [27] and to further improve training of the WGAN (IWGAN) [24].

Table 1 lists the different methods for object reconstruction (microstructure and nonmicrostructure). It shows that the deep learning method is more common in 2D object reconstruction than 3D object reconstruction. More specifically, there is still no application of GAN in pavement microstructure. Therefore, we attempt to fill this gap by designing a GAN for this case.

**Table 1.** 2D/3D object reconstruction methods with statistical and deep learning approaches.


### *2.3. Deep-Learning-Based Microstructure Reconstruction*

There were a few research works on deep learning applications for microstructure reconstruction before the GAN technique. One example of such applications is a research work by Cang, Ruijin, et al. [28]. It proposed a convolutional deep belief network (CDBN) to reconstruct heterogeneous materials. Recently, Tran and Tran [29] introduced a 2D microstructure reconstruction framework based on the image inpainting method to solve the microstructure reconstruction problem in three different contexts.

Owing to the proliferation of GAN, several research approaches exploit the generative power of GAN. Since GAN is used for microstructure reconstruction [30], it suffers from the intrinsic instability issue. We published the preliminary results of a deep-learning-based microstructure reconstruction using IWGAN [31].

Recently, more complex methods have been combined with GAN-based methods [32–34]. An end-to-end three-dimensional reconstruction framework of porous media from a single two-dimensional dataset has been proposed [32]. Shams et al. [33] introduces coupled generative adversarial and autoencoder neural networks to overcome instability for the reconstruction of realizations of three-dimensional data. The model gains efficient results by applying a gradient-descent-based optimization method for training and stabilizing the neural networks. A transfer learning technique [34] is exploited to integrate statistical descriptors with feature maps from a pre-trained deep neural network into an overall loss function for an optimization-based reconstruction procedure.

Our proposed system is different from the aforementioned recent approaches in the following aspects. First, we focus on reconstructing the 3D microstructure of permeable pavements and obtaining 3D images with a given 2D image. In this regard, an end-toend framework [32] has the same goal as our system. However, our system exploits the characteristics of 3D-IWGAN, whereas an end-to-end framework uses the modified version of BicycleGAN [35], which is mainly for 2D-to-2D translation. Second, we provide the more detailed steps for pre-processing and the pavement analysis methods for checking the effectiveness of generated 3D images.
