Preliminary Study of Airfoil Design Synthesis Using a Conditional Diffusion Model and Smoothing Method

Abstract

Generative models such as generative adversarial networks and variational autoencoders are widely used for design synthesis. A diffusion model is another generative model that outperforms GANs and VAEs in image processing. It has also been applied in design synthesis, but was limited to only shape generation. It is important in design synthesis to generate shapes that satisfy the required performance. For such aims, a conditional diffusion model has to be used, but has not been studied. In this study, we applied a conditional diffusion model to the design synthesis and showed that the output of this diffusion model contains noisy data caused by Gaussian noise. We show that we can conduct flow analysis on the generated data by using smoothing filters.

1. Introduction

In the process of shape design, it is common practice to develop an initial shape that satisfies the desired properties and refine their shapes incrementally. However, this process often relies on the subjective expertise and intuition of designers, leading to challenges such as difficulty in discovering novel shapes and the occurrence of rework due to the initial shapes not meeting the desired performance. The mathematical optimization methods can be used for design optimization [1], but they require the initial shapes to start the computation. To address these challenges, research has been conducted to introduce machine learning into the design process.

In the field of design using machine learning, with the advancement of deep learning, the number of studies utilizing deep generative models has increased. Generative adversarial networks (GAN) and variational autoencoders (VAE) are commonly used [2,3,4,5,6,7,8]. Generative models are utilized in a wide variety of design tasks, including automotive [9], motor [10], ship hull [11], and chair designs [12]. Among their many applications, an airfoil shape generation task is widely employed as a benchmark example, especially in generating shapes that meet performance criteria such as lift coefficients [2,13].

One difficulty in generative models to be used for design synthesis is that the generated shapes exhibit noisy lines. Ref. [2] reported such noisy lines and used filters to smoothen lines. Ref. [6] used stabilization techniques for training and obtained smooth shapes without any smoothing. The other difficulty is that although the aim of design synthesis is to obtain shapes that meet the requirements, the generated shape sometimes does not satisfy the requirements. To overcome this issue, a physics-guided model was proposed in [14].

Recently, diffusion models [15,16] have attracted attention with the advent of models enabling the conversion from text to images, such as stable diffusion [17] and DALL-E2 [18]. Such diffusion models are now widely used in many fields, including computer vision [19], object detection [20], bioinformatics [21], voice conversion [22], and density-based designs such as topology optimization [23,24]. Some studies, such as [25], compared diffusion models with other generative models, i.e., VAE and GAN, using image generation tasks, and reported that the diffusion model outperforms other models. Diffusion models have also been utilized for airfoil shape generation [26]. However, in such studies, only shapes are generated, regardless of the model’s performance. In the design synthesis, it is mandatory to obtain shapes that meet the performance requirements. For such an aim, a conditional diffusion model can be used, but it is not known whether the conditional diffusion model also exhibits the same issues as GAN and VAE, i.e., the issue of noisy lines and accuracy of requirements. In this study, we explore the application of diffusion models to airfoil shape generation, aiming to reveal the issues and provide insights. In this study, a denoising diffusion probabilistic model (DDPM) [27] is employed and used as a conditional diffusion model.

The present paper is organized as follows. In Section 2, the diffusion model is introduced. In Section 3, the architecture of the diffusion model is described. In Section 4, the numerical experiments and comparison with other generative models are described. Section 5 concludes this paper.

2. Diffusion Model

The learning process of diffusion models is divided into the forward process and reverse process, as shown in Figure 1. The process from ${\mathit{x}}_T$ to ${\mathit{x}}_0$ is called a reverse process. The reverse process is modeled by a Markov chain, which implies that the state ${\mathit{x}}_{t-1}$ is determined only by ${\mathit{x}}_t$;

$$p_{\theta }\left({\mathit{x}}_{t-1}\mid {\mathit{x}}_t\right)=\mathcal{N}\left({\mathit{x}}_{t-1};{\mathit{\mu }}_{\theta }\left({\mathit{x}}_t,t\right),{\mathsf{\Sigma }}_{\theta }\left({\mathit{x}}_{\theta },t\right)\right)$$

(1)

$$p_{\theta }\left({\mathit{x}}_{0:T}\right)=p\left({\mathit{x}}_T\right){\mathsf{\Pi }}_{t=1}^T{p}_{\theta }\left({\mathit{x}}_{t-1}\mid {\mathit{x}}_t\right).$$

(2)

The forward process, or diffusion process, is also a Markov chain, In the diffusion process, noise is added to the original data $x_t$ according to a variance schedule ${\beta }_1,\dots ,{\beta }_T$. The diffusion process is modeled as

$$q\left({\mathit{x}}_{1:T}\right)={\mathsf{\Pi }}_{t=1}^{T}q\left({\mathit{x}}_t\mid {\mathit{x}}_{t-1}\right),$$

(3)

$$q({\mathit{x}}_t\mid {\mathit{x}}_{t-1})=\mathcal{N}\left({\mathit{x}}_t;\sqrt{1-{\beta }_t}{\mathit{x}}_{t-1},{\beta }_{t}I\right).$$

(4)

The training is performed by optimizing the variational bound on negative log-likelihood $\mathbb{E}\left[-log{p}_{\theta }\left({\mathit{x}}_0\right)\right]\le L$, where the objective function L is formulated as

$$L={\mathbb{E}}_q\left[-log{\displaystyle \frac{p_{\theta }\left({\mathit{x}}_{0:T}\right)}{q\left({\mathit{x}}_{1:T}\mid {\mathit{x}}_0\right)}}\right]={\mathbb{E}}_q\left[-logp\left({\mathit{x}}_T\right)-{\mathsf{\Sigma }}_{t\ge 1}log{\displaystyle \frac{p_{\theta }\left({\mathit{x}}_{t-1}\mid {\mathit{x}}_t\right)}{q\left({\mathit{x}}_t\mid {\mathit{x}}_{t-1}\right)}}\right].$$

(5)

By using the Kullback–Leibler divergence ($D_{\mathrm{KL}}$) to directly compare $p_{\theta }\left({\mathit{x}}_{t-1}\right|{\mathit{x}}_t)$ with the forward process posterior $q\left({\mathit{x}}_{t-1}\right|{\mathit{x}}_t,{\mathit{x}}_0)$, the upperbound L is rewritten as

$$L={\mathbb{E}}_q\left[D_{\mathrm{KL}}\left(q\left({\mathit{x}}_T|{\mathit{x}}_0\right)\left|\right|p\left({\mathit{x}}_T\right)\right)+{\mathsf{\Sigma }}_{t>1}{D}_{\mathrm{KL}}(q\left({\mathit{x}}_{t-1}\right|{\mathit{x}}_t,{\mathit{x}}_0)\left|\right|p_{\theta }\left({\mathit{x}}_{t-1}\right|{\mathit{x}}_t))-log{p}_{\theta }\left({\mathit{x}}_0\right|{\mathit{x}}_1)\right]$$

(6)

On the other hand, the inverse process is a process that uses a neural network-parametrized probability model. In the inverse process, the model learns to predict the noise ${\epsilon }_t$ added to the data $x_t$ after t steps of the diffusion process. During inference, noise sampled from a multidimensional standard normal distribution $N(0,1)$ is applied to $x_T$. By denoising these samples through the inverse process for T steps, new data $x_0$ can be generated. The U-Net structure [28] is commonly used in the neural network of the inverse process. While originally devised for semantic segmentation, U-Net’s ability to extract information from inputs and return outputs of the same size makes it applicable to the inverse process.

We use the airfoil design task as a benchmark problem for comparison with other generative models. Two types of conditional diffusion models were trained: one uses liner fully connected layers, and the other uses convolutional layers. These two models use the same hyperparameters, as shown in

Table 1. Hyperparameters of the conditional diffusion models.

Layer of U-Net	4	Optimizer	Adam
Channel of U-Net	128	Learning rate	0.0001
Timestep T	400	Batch size	16
Activation function	ReLU, GeLU

. We used the classifier-free diffusion guidance [29] as the conditional diffusion model. The schematics of the conditional diffusion model applied to the airfoil geneation are illustrated in Figure 2.

3. Dataset and Evaluation

3.1. Dataset

The airfoil dataset used in this study was based on the NACA 4-digit series airfoils. These airfoils are defined by three parameters: the first digit represents the percentage of maximum camber relative to the chord length, the second digit represents the tenth percentile of the chord length for the maximum camber position, and the third and fourth digits represent the percentage of maximum thickness relative to the chord length. This dataset was the same as that used in [8].

Additionally, Joukowski airfoils were also employed. For both NACA and Joukowski airfoils, the following preprocessing steps were applied. Firstly, 248 coordinates were obtained from these airfoil profiles (Figure 3). Subsequently, the lift coefficient of the airfoil was calculated using these coordinates. XFoil [30], an airfoil analysis software, was utilized for this purpose. XFoil takes the coordinates of each point on the airfoil as the input and provides the lift coefficient as the output. However, in cases where the calculations do not converge, XFoil returns NaN. Airfoil profiles with NaN lift coefficients were excluded. For NACA airfoils, values less than 0.5 and greater than 1.2 were also excluded. These processing steps addressed data with missing values and outlier removal, aiming to stabilize the training of neural networks. Finally, the data were standardized to have a mean of 0 and a variance of 1, where the mean and variance of the lift coefficients were 0.8776 and 0.2198, respectively. Standardization was applied to both the coordinates of the airfoil and the lift coefficients. This preprocessing step was also performed with the expectation of stabilizing neural network training.

As a result of these preprocessing steps, a dataset comprising 3767 pairs of airfoil profiles and lift coefficients was obtained. This dataset served as the training dataset for the experiments conducted in this study. The relationship between the frequency of lift coefficients is illustrated in Figure 4.

3.2. Evaluation Method

Evaluation metrics for the conditional generative model were based on four indicators. The first metric measured the proportion of generated airfoil shapes for which the lift coefficient calculations converged after 40 iterations using XFoil version 6.97. This indicator not only represented the percentage of computationally feasible lift coefficients among the generated shapes but also quantitatively assessed whether the generated shapes were sufficiently airfoil-like. The second through fourth metrics utilized only those cases for which the first evaluation metric converged.

The second metric was the mean squared error between the input lift coefficients provided to the conditional generative model and the lift coefficients of the generated airfoils. This metric measured the ability to generate airfoil shapes corresponding to the specified lift coefficients. This metric is defined as

$$L_{\mathrm{MSE}}={\displaystyle \frac{1}{N}}{\mathsf{\Sigma }}_{i=1}^N{∥C_L^c-C_L^r∥}_2^2,$$

(7)

where $C_L^c$ and $C_L^r$ are the specified class label and re-calculated lift coefficient.

The third metric was the mean squared error between each generated airfoil and the average of the generated airfoils, representing the variance of the generated airfoil shapes. This metric measured the diversity of the generated airfoil shapes and is presented as

$$\mu ={\displaystyle \frac{1}{N}}{\mathsf{\Sigma }}_{i=1}^N{∥\overline{\mathit{x}}-{\mathit{x}}_i∥}_2^2,$$

(8)

where ${\mathit{x}}_i$ is the generated data, and $\overline{\mathit{x}}$ is the mean of all generated data.

The fourth metric was the maximum squared error between each generated airfoil (${\mathit{x}}_i$) and its nearest training data (${\mathit{d}}_i$). This indicator produced a larger value if the generated airfoils exhibit shapes not present in the training data. Therefore, this metric provided a measure of the uniqueness of the generated airfoil shapes concerning the training data. It is formulated as

$$\nu =\underset{{\mathit{x}}_i\in \mathrm{generated}\mathrm{data}}{max}\underset{d_j\in \mathrm{train}\mathrm{data}}{min}{\parallel {\mathit{x}}_i-{\mathit{d}}_j\parallel }_2^2$$

(9)

4. Numerical Experiments

4.1. Conditional Diffusion Model with Linear Layers

We trained the conditional diffusion model with linear layers and generated data. The results of generated data are shown in Figure 5. The data were completely random and did not make any shapes. The linear layers were used in VAE and GANs for shape design synthesis [6,13]. The linear layer was also used in the diffusion model for a MNIST task, which is a relatively easy task. However, in the diffusion model used to generate airfoil shapes, it is revealed that the linear layer is not sufficient.

4.2. Conditional Diffusion Model with Convolution Layers

We trained the conditional diffusion model with convolutional layers and generated data. The results of generated data are shown in Figure 6. The generated data consisted of airfoil shapes, but the outline was a noisy shape. Since the outline was noisy, the XFoil calculation could not be conducted for the generated shapes. In the diffusion models, the Gaussian noise was contained in the generated data. As training progresses, the Gaussian noise is reduced, but in this study, the noise remained in the final output.

Such noisy shapes are also reported in [2], which uses GAN. One approach to remove noisy lines is to use the stabilization technique of GAN [6], which cannot be used for diffusion models. The other approach is to use smoothing filters [2]. We employed the second approach in this study.

In order to smoothen the outlines, we used a filter. Since we knew that the noise is caused by Gaussian noise, a moving average filter was suitable. The moving average filter is formulated as ${\widehat{\mathit{x}}}_i={\displaystyle \frac{1}{2N+1}}{\sum }_{j=i-N}^{i+N}{\mathit{x}}_j.$ We used $N=7$ in the following experiments. The filter was applied to the generated data and the smoothed shapes were obtained (Figure 7). The noise was reduced and the outline became smooth. Since the noise was caused by the Gaussian noise, the smoothed shape was located roughly at the center of the original data.

The evaluation results are shown in

Table 2. Evaluation results of diffusion model.

	Convergence Rate	${\mathcal{L}}_{\mathbf{MSE}}$	$\mathit{\mu }$	$\mathit{\nu }$
Conditional diffusion	20.4%	0.489	6.472	0.171
Conditional $\mathcal{S}$-VAE [6,31]	89.8%	0.020	0.317	—
Conditional GAN [6]	90.4%	0.047	0.320	—

. ${\mathcal{L}}_{\mathrm{MSE}}$ was $0.489$, which is obviously larger than that of GAN (0.047) and VAE (0.017) [6]. The reason that the diffusion model shows such a large ${\mathcal{L}}_{\mathrm{MSE}}$ is because the moving average filter changes the airfoil shape (Figure 7). The error ${\mathcal{L}}_{\mathrm{MSE}}$ is attributed to two factors; one is the smoothing method, and the other is the generative model.

5. Conclusions

We have utilized DDPM to design the synthesis of airfoil shape generation. The generated outline contains noise, and the flow analysis cannot be conducted for the generated shapes. Hence, smoothing methods are necessary. Moving average methods are suitable since the noise is caused by the Gaussian noise of DDPM. One of the reasons for the instability of diffusion models comes from the small dataset size. The data used in this study consist of only 3767 data, which is smaller than average data size for diffusion models. The proposed model generates required shapes in a short time, which contributes to the design process.

In order to reduce noise, stabilization techniques are necessary. In addition, the loss function should be revised to add penalty terms for noisy lines. The accuracy of requirements, i.e., ${\mathcal{L}}_{\mathrm{MSE}}$, should be reduced by using techniques of physics-informed neural networks.