Thermal Deformation Defect Prediction for Layered Printing Using Convolutional Generative Adversarial Network

Abstract

This paper presents a Thermal Deformation defect prediction method for layered printing using Convolutional Generative Adversarial Network (CGAN). Firstly, the original manifold mesh is converted into layered image in Printing Coordinate System (PCS). The trajectory inside layered image with various infill patterns are generated for making comparisons. Inspired by monocular vision and even binocular vision, the mathematical model of thermal defect prediction via infrared thermogram is built via virtual printing of Digital Twins to preset the initial parameters of Artificial Neural Network (ANN). Particularly, the depth convolution is used to extract multi-scale features of layered image. By using transfer learning techniques to identify small sample data, the CGAN is employed to build the nonlinear implicit relations between thermal deformation and multi-scale features. The binocular stereo vision laser scanner is used to determine the actual thermal deformation of the target printed objects. The shape deformation dissimilarity can be succinctly calculated by evaluating the surface profile error via mesh registration between the original source and target mesh model. The proposed method is verified by physical experiments. The experiment proved that the proposed method can deal with the thermal deformation with more optimal parameters, which contributes to performance forward design of irregular complex parts regarding diversified customized requirements.

1. Introduction

Additive manufacturing (AM), a representative of superfield manufacturing, enables quick freeform fabrication of complex-geometry components directly from the 3D models [1]. Due to its layer-by-layer additive process, which is the common feature of AM, it can be simpler and quicker to produce complex-geometry components at one time compared with conventional manufacturing process. Most frontier research integrate virtual reality, digital twin, transfer learning, cloud computing, and Internet of Things (Iot) with AM. What is known to us all is that there are many factors which have remarkable effects on quality and efficiency, such as geometry measurement, thermal characteristics, and thermoforming.

The earlier publications studied geometry measurement and trajectory planning. Zhang et al. [2] presented a method to find printing directions that avoid placing supports in perceptually significant regions to prevent the visual impact of artifacts. Mao et al. [3] introduced a method to efficiently generate slicing plans by a new metric profile that can characterize the distribution of deviation errors along the building direction. Shen et al. [4] proposed a toolpath interpolation method for 5-degree-of-freedom (5-DoF) parallel kinematic machines which can realize the smoothness of acceleration in all toolpath components, decrease the shock to the system, and reduce the surface error after cutting. Wang et al. [5] proposed a novel 5-axis dynamic slicing algorithm to achieve non-supporting material printing.

Most research has focused on thermal engineering and thermoforming. Bugeda et al. [6] developed a finite element model for the 3D simulation of the sintering of a single track during a Selective Laser Sintering (SLS) process which takes into account both the thermal and the sintering phenomena involved in the process. Based on a series of single tracks, Song et al. [7] firstly proposed the processing windows corresponding to different melting mechanisms in terms of microstructure, roughness, densification and microhardness. In order to study these dynamic interactions, such as heat conduction, convection, radiation, melting and solidification, dynamic phase changes, and evaporation, a finite element model has been developed which uses a dynamic mesh with spatial non-linear thermal properties to track the point of laser exposure on the powder bed to study thermal evolution during Selective Laser Melting (SLM) by Zeng et al. [8]. Li et al. [9] studied the phase evolution of laser Additive Manufacture (LAM) in producing Al-5Si-1Cu-Mg alloy in detail by characterization along the deposition direction.

In the research of optimal trajectory, Yang et al. [10] proposed a fractal scanning path which is a space-filling, self-avoiding, simple, and self-similar curve for selective laser sintering (SLS). Selvakumar et al. [11] proposed an extension of a specialized solution approach, which transforms the pursuit-evasion game (in which terminal time is free) via a special state transformation into a family of games with fixed terminal time. Liu et al. [12] presented how to apply second-order cone programming, a subclass of convex optimization, to rapidly solve a highly nonlinear optimal control problem arisen from smooth entry trajectory planning with high lift/drag ratios. Bonami et al. [13] have solved the mixed-integer nonlinear programming problem using a nonlinear programming-based branch-and-bound algorithm specifically tailored to the problem.

Some researchers studied AM technology using some novel methods. Jin et al. [14] studied autonomous in-situ correction of fused deposition modeling printers using computer vision and deep learning. He et al. [15] studied machine learning for continuous liquid interface production. Liu et al. [16] studied warpage deformation mechanism and control method of fused deposition parts. Xiong et al. [17] studied online measurement of bead geometry in gas metal arc welding based AM using passive vision. Teng et al. [18] studied the effects of material property assumptions on predicted meltpool shape for laser powder bed fusion based additive manufacturing. Paul et al. developed a three-dimensional thermomechanical finite element (FE) model to [19] calculate the thermal deformation in AM parts based on slice thickness, part orientation, scanning speed, and material properties.

There has been related research in 4D printing (4DP) concerning shape deformation. Rastogi et al. [20] proposed the complexity and inflexible design that had barricaded their dimension were vanquished by 4D printing with its dynamic structures. Wang et al. [21] proposed that deformation of arbitrary deployable surface can be realized by designing the fiber trajectories. Abuzaid et al. [22] was dedicated to study the stability and shape recovery properties of 4D printed honeycomb structures using thermoplastic Shape Memory Polymer (SMP). Bodaghi et al. [23] proposed that a Finite Element (FE) formulation based on the non-linear Green-Lagrange kinematic relations coupled with a robust SMP constitutive model established to describe material tailoring in 4D printing. Sossou et al. [24] proposed a modeling framework for simulating smart materials (SMs) and conventional materials behaviors on a voxel basis for 4D printing. Ding et al. [25] designed and demonstrated several 4D rod structures with the use of a nonlinear thermomechanical computational model using 4D printing.

Aiming at improving geometric precision for 3D printing (3DP) and even 4DP, based on the previous representative work [26,27,28,29], the antecedent research background is deepened and continued, the upshot is that this paper proposed a method of deformation-induced defect prediction for layered printing using Convolutional Generative Adversarial Network (CGAN).

2. Convert Manifold Mesh into Layered Image in Printing Coordinate System (PCS)

2.1. Printing Coordinate System (PCS)

For a polyhedron manifold model $M$ to be printed in Euclidean space $R^3$, let $A_i$, $i=0,\cdots ,N-1$ be the N triangular faces, with vertices ($a_i,b_i,c_i$), which are assumed to be ordered counter clockwise (CCW) on $A_i$. The Axis-aligned Bounding Boxes (AABB) of $M$ is calculated to define the scale of $M$. Meanwhile, the stroke length $x_b,y_b,z_b$ of the AABB along $x,y,z$ direction can be reckoned. The maximum print space of a printer along $x,y,z$ direction are, respectively, $x_p,y_p,z_p$.

For any 3D component to be fabricated, the object is sliced into a number of 2D layers of defined layer thickness. In a printing coordinate system (PCS), the layer height sequence $\mathit{Z}$ about $n$ layers can be expressed as

$$\mathit{Z}=\left\{z_1,z_2,z_i,\cdots ,z_n\right\} Z_i\in R \& i\in \left[1,n\right],$$

(1)

where $z_i<z_j$, for $\forall i<j$.

The layer thickness sequence $\mathit{d}$ can be obtained:

$$\mathit{d}=diff\left(\mathit{Z}\right)=\left\{d_1,d_2,d_i,\cdots ,d_{n-1}\right\} d_i\in R \& i\in \left[1,n-1\right],$$

(2)

where $diff$ means 1st order forward difference, $\forall d_i=z_{i+1}-z_i$, $i\in \left[1,n-1\right]$.

For uniform slicing, layer thickness is equal to each other, $\forall diff\left(\mathit{d}\right) =0$. For adaptive slicing, layer thickness varies, $\exists diff\left(\mathit{d}\right)\ne 0$.

In PCS, the normalized height $h_n$ of i-th layer can be calculated hereby to represent relative height of the slicing plane:

$$h_n=\frac{z_i}{z_b} h_n\in \left(0,1\right].$$

(3)

There exists the inherent staircase effect (SE) in 3DP. The SE can be manifested by various geometric parameters, for instance, cusp height $\delta$ and volume error $V_e$. The cusp height is defined as

$$\delta =d\times \left|cos\alpha \right|,$$

(4)

$$\alpha =co{s}^{-1}\frac{{\mathit{n}}_{\mathit{z}}\times {\mathit{n}}_{\mathit{f}}}{\Vert {\mathit{n}}_{\mathit{z}}\Vert ·\Vert {\mathit{n}}_{\mathit{f}}\Vert } \alpha \in \left[0,\mathsf{\pi }\right],$$

(5)

where $d$ is layer thickness, and α is the included angle between the z normal vector ${\mathit{n}}_{\mathit{Z}}$ and facet normal vector ${\mathit{n}}_{\mathit{f}}$.

Adaptive slicing is generally realized via variable thickness using iteration. Therefore, the calculation process of layer thicknesses can be expressed in the following mathematical model.

$$\{\begin{array}{c}find \mathit{d}\\ min card\left(\mathit{d}\right)\\ Subject to: f\left(\mathit{d}\right)\le {\delta }_{max}\\ d\in \left[d_{min} ,d_{max}\right]\end{array},$$

(6)

where $card\left(\mathit{d}\right)$ is the cardinality of the $\mathit{d}$, $d\in \left[d_{min} ,d_{max}\right].$ $f\left(\mathit{d}\right)$ is the constraint function which presents the accuracy requirement, and ${\delta }_{max}$ can be the maximum cusp height of each layer.

The Printing Coordinate System (PCS) of 3D reconstructed manifold model (right human knee distal femur) is shown in Figure 1.

2.2. Trajectory Inside Layered Image with Various Infill Patterns

In order to prepare the 3D printing experiment and simulate the manufacturing process, the femur model is sliced in different layers with special infill patterns shown in Figure 2. Let $L$ be total length of open-loop trajectory, and $n_{point}$ be amount of turning points among all points $P$.

$$L={\sum }_{i=1}^{n_{point}-1}\Vert P_{}^{\left(i+1\right)}\left(x,y,z\right)-P_{}^{\left(i\right)}\left(x,y,z\right)\Vert .$$

(7)

3. Thermal Effects Prediction on Shape Deformation via Infrared Thermogram

3.1. Infrared Camera and Digital Camera

The 3D printing is realized by stacking between the multilayers with various and uneven temperatures along the trajectory. Discriminating visibility is the key technology to realize virtual printing. The visible primitives including points and facets are obtained based on convex hull principles. For the mathematical description of the three-dimensional heat conduction problem, it is necessary to combine the Fourier law and the principle of conservation of energy in order to analyze the multilayer thermodynamic equilibriums state. Inspired by monocular vision and even binocular vision, Figure 3 shows the schematic binocular stereo vision measurement of layered printing.

The melting energy consumption as follows:

$$E_{thermal}=cm\left(T_m-T_a\right)+mX=\rho V\left[c\left(T_m-T_a\right)+X\right],$$

(8)

where $E_{thermal}$ is the melting energy consumption (kJ); $c$ is the material specific heat capacity ($\mathrm{J}·{\mathrm{kg}}^{-1}·{\mathrm{K}}^{-1}$); $m$ is the weight of the filament ($\mathrm{kg}$); $T_m$ is the material melting point ($\mathrm{K}$); $T_a$ is the environment temperature ($\mathrm{K}$); $X$ is the latent heat ($\mathrm{kJ}·\mathrm{kg}$).

The thermodynamic equilibriums equation for multilayer satisfies the following classical 3D heat conduction equation.

$$\rho c\frac{\partial T}{\partial t}=\frac{\partial }{\partial x}\left(k_x\frac{\partial T}{\partial x}\right)+\frac{\partial }{\partial y}\left(k_y\frac{\partial T}{\partial y}\right)+\frac{\partial }{\partial z}\left(k_z\frac{\partial T}{\partial z}\right)+\dot{q},$$

(9)

where $\rho$ is the material density ($\mathrm{kg}·m^{-3}$); $c$ is the material specific heat capacity ($\mathrm{J}·{\mathrm{kg}}^{-1}·{\mathrm{K}}^{-1}$); $T$ is the temperature $(\mathrm{K}$); $t$ is the interaction time $(\mathrm{s}$); $k_x,k_y,k_z$ are thermal conductivities $\left(\mathrm{W}·{\mathrm{m}}^{-1}·{\mathrm{K}}^{-1})\right)$ along the directions of x, y, z axes; and $\dot{q}$ is the internal heat density ($\mathrm{W}·m^{-3}$).

When $k_x,k_y,k_z$ are the same value as a constant, the thermodynamic equilibriums equation for multilayer can be obtained from the basic principles of heat transfer and the first law of thermodynamics, as follows:

$$\frac{\partial T}{\partial t}=\alpha \left(\frac{{\partial }^{2}T}{\partial x^2}+\frac{{\partial }^{2}T}{\partial y^2}+\frac{{\partial }^{2}T}{\partial z^2}\right)+\frac{\dot{q}}{\rho c},$$

(10)

$$\alpha =k/\left(\rho c\right)$$

(11)

where $\alpha$ is the thermal diffusivity $\left({\mathrm{m}}^2·{\mathrm{s}}^{-1}\right)$, $k$ is the thermal conductivity $\left(\mathrm{W}·{\mathrm{m}}^{-1}·{\mathrm{K}}^{-1}\right)$; $c$ is the material specific heat capacity ($\mathrm{J}·{\mathrm{kg}}^{-1}·{\mathrm{K}}^{-1}$) as a constant.

The heat conduction equation is a partial differential equation of first order for time. But it is a partial differential equation of second order for space. Thus, solving the equation requires a time condition for temperature and two spatial conditions. These conditions are given as the initial conditions at $\mathrm{t}=0$ and the boundary conditions of the space.

The first type of boundary condition for the temperature is defined as follows:

$$T_{x=0}=T_s\left(t\right).$$

(12)

When the temperature $T_s$ on the boundary surface is independent of time as a constant, it is called constant wall temperature condition.

For the case where the heat flux on the boundary is specified, it can be expressed by

$$-k{\left(\frac{\partial T}{\partial x}\right)}_{x=0}=q\left(t\right).$$

(13)

When the heat flux density on the boundary is independent of time as a constant, it is called constant heat flux condition. In the case of $q=0$, which is the adiabatic condition, can be expressed as follows:

$${\left(\frac{\partial T}{\partial x}\right)}_{x=0}=0.$$

(14)

When the convection heat transfer coefficient $h$ is given on the boundary surface, which means the heat transfer between the object and the surrounding fluid, the surface heat transfer coefficient is used according to the Newton cooling formula. The boundary condition is expressed as follows:

$$-k{\left(\frac{\partial T}{\partial x}\right)}_{x=0}=h\left[T_s\left(t\right)-T_{\infty }\right],$$

(15)

where $h$ is the thermal convection coefficient $\left(\mathrm{W}·{\mathrm{m}}^{-2}·{\mathrm{K}}^{-1}\right)$; $T_{\infty }$ is the final temperature when time tends to infinity, which means $T_{\infty }$ is the temperature of ambient environment $T_a$.

Interface continuous conditions refer to two objects in full contact. The two objects of the contact surface and the temperature and heat flux values were equal. Therefore, the boundary condition is expressed as follows:

$${\left(T_1\right)}_{x=0}={\left(T_2\right)}_{x=0},$$

(16)

$$-k_1{\left(\frac{\partial T_1}{\partial x}\right)}_{x=0}=-k_2{\left(\frac{\partial T_2}{\partial x}\right)}_{x=0},$$

(17)

where $T_1$ and $T_2$ are the temperatures of two contacted objects, and $k_1$ and $k_2$ are the thermal conductivities of two contacted objects.

The end-effector moves at a constant speed of $V_F$ in the region when extruding which leads to regional sensitivity. In fact, the highest temperature in each layer is always below the end-effector. Considering only heat conduction and convection, the multilayer thermodynamic equilibriums equation can be expressed as

$$q_f=q_{f+df}+q_A+q_S+\Delta q,$$

(18)

where $q_f$ is the input heat of extruded filament (kJ); $q_{f+df}$ is the conduction heat from extruded filament to interactive element $\left(\mathrm{kJ}\right)$; $q_A$ is the convection heat to ambient air $\left(\mathrm{kJ}\right)$; $q_S$ is the convection heat to substrate $\left(\mathrm{kJ}\right)$; $\Delta q$ is internal energy change of filament microelement $\left(\mathrm{kJ}\right)$.

For most of 3D trajectory in the process of 3D printing, it can be simplified to one-dimensional problem due to the end-effector which usually moves in one direction. According to previous boundary conditions, it can be converted to the partial differential equation (PDE).

$$\rho c{\mathrm{S}}_A\frac{\partial T}{\partial x}={\mathrm{S}}_A\frac{\partial }{\partial x}\left(k\frac{\partial T}{\partial x}\right)-hP\left(T-T_a\right),$$

(19)

where ${\mathrm{S}}_A$ is cross-sectional area of filament (${\mathrm{mm}}^2$); $P$ is cross-sectional perimeter of filament ($\mathrm{mm}$). $\frac{\partial T}{\partial x}$ equals to the temperature gradient $\nabla T$.

As long as the initial temperature of filament $T_0$ when $x=0$ is obtained:

$$\nabla T=\frac{\partial T}{\partial x}=-f\left(T_0-T_a\right)e^{-fx},$$

(20)

$$f=\frac{\sqrt{1+4\alpha \beta }-1}{2\alpha }, \alpha =\frac{k}{\rho c{V}_F}, \beta =\frac{hP}{\rho c{\mathrm{S}}_A{V}_F}.$$

(21)

As long as the temperature gradient $\nabla T$ is obtained, the analysis of the process parameters is supposed to be considered in manufacturing process.

3.2. Infrared Temperature Measurement

In the actual radiation model, in addition to the energy radiated by the object itself, it can also reflect and transmit the environment radiation, so the energy reaching the infrared optical system mainly consists of three parts.

$$E=E_1+E_2+E_3,$$

(22)

where $E$ is radiant energy reaching the infrared optical system (kJ), $E_1$ is energy radiated by the object itself (kJ), $E_2$ is environmental radiation energy reflected by the target surface (kJ), $E_3$ is environmental radiation energy transmitted by target surface (kJ).

Assuming that the target emissivity is $\epsilon$ and the reflectivity is ${\rho }_r$, the transmissivity is $\tau$, and the total environmental radiation energy is $E_a$ (kJ), and the radiation energy of blackbody at the same temperature with the target surface is $E_b$ (kJ), $E$ is determined as follows.

$$E=\epsilon E_b+{\rho }_r{E}_a+\tau E_a,$$

(23)

where $\epsilon +{\rho }_r+\tau =1$.

According to Stefan-Boltzmann law, $E$ can be obtained as follows.

$${\sigma }_s{T}_r{}^4=\epsilon {\sigma }_s{T}_b{}^4+\left(1-\epsilon \right){\sigma }_s{T}_a{}^4,$$

(24)

where ${\sigma }_s=5.67\times {10}^{-8}\left(W·m^{-2}·K^{-4}\right)$ is Stefan constant, $T_r$ is radiation temperature of target surface $\left(K\right)$, $T_a$ is ambient temperature $\left(K\right)$, and $T_b$ is actual temperature of target surface $\left(K\right)$.

Then, actual temperature $T_b$ of target surface can be obtained as follows.

$$T_b=\sqrt[4]{\frac{T_r{}^4-\left(1-\epsilon \right)T_a{}^4}{\epsilon }}.$$

(25)

The thermographs using thermal field measurement is shown in Figure 4. The noise equivalent temperature difference (NEDT) of used FLUKE^® thermography is NEDT (30 °C) ≤ 0.09 °C. The instantaneous field of view (IFOV) of thermal infrared imager instrument is IFOV < 1.25 mRad. The detector resolution of the thermal infrared imager is more than 320 × 240 (76,800 pixels). The measurable temperature range is [−20 °C, 1200 °C]. The $\epsilon$ = 0.95 and $\tau$ = 100% for measured target.

3.3. Layered Deformation of the Printed Objects

The thermal effect plays an important role in thermoplastic forming. Apparently, thermal deformation is the main part of printing deformation.

The initialization probability $P_{init}$ $(P_{init}\in \left[0,1\right])$ of customized layer before defect prediction using convolutional generative adversarial network is shown in Figure 5. Specifically, in Figure 5a, the maximum probability max ($P_{init}$) = 0.400 at (145.393, 136.766) point; the minimum probability min ($P_{init}$) = 0.118 at (144.852, 123.850) point; standard deviation std ($P_{init}$) = 0.063. In Figure 5b, max ($P_{init}$) = 0.399 at (166.943, 93.617) point; the minimum probability min ($P_{init}$) = 0.055, at (168.326, 94.555) point; standard deviation std ($P_{init}$) = 0.099. In Figure 5c, max ($P_{init}$) = 0.523 at (184.364,104.073) point; the minimum probability min ($P_{init}$) = 0.070, at (193.758,119.930) point; standard deviation std ($P_{init}$) = 0.083. In Figure 5d, max ($P_{init}$) = 0.460 at (190.176,111.479) point; the minimum probability min ($P_{init}$) = 0.059 at (192.780,104.766) point; standard deviation std ($P_{init}$) = 0.085. In Figure 5e, max ($P_{init}$) = 0.396 at (186.280,95.857) point; the minimum probability min ($P_{init}$) = 0.098 at (172.151,120.081) point; standard deviation std ($P_{init}$) = 0.081. In Figure 5f, max ($P_{init}$) = 0.442 at (158.341,113.351) point; the minimum probability min ($P_{init}$) = 0.101 at (158.125,98.107) point; standard deviation std ($P_{init}$) = 0.077.

4. Surface Profile Error to Evaluate Deformation

The profile of the curve is expressed not by mathematical equation but by discrete points, which makes it difficult to evaluate the profile error. Therefore, the profile error of surface mesh can be measured by calculating the dissimilarity deviation between the source surface and the target surface based on the Surface Subdivision Method. According to the definition of the profile tolerance in International Organization for Standardization (ISO) 1101 [30], the surface profile error can be defined by the minimum diameter covering all measured points of the cluster spheres in which centers lie on the source model.

Let $M_{source}$ be source model and $M_{target}$ be target model. The key points can be selected from the measured model $M_{target}$ by using many methods, such as K-nearest neighbors (KNN) search, Least Squares (LS), etc. The deviation between the source surface S and the i-th key point $P_i^{key}$ is defined by the minimum Euclidean distance from $P_i^{key}$ to the closest point $S\left(u_i,v_i\right)$ on the model surface S.

$$P_i^{key}={\left[x_i^{key},y_i^{key},z_i^{key}\right]}^T, i\in \left[1,n\right]$$

(26)

$$S\left(u_i,v_i\right)={\left[x\left(u_i,v_i\right),y\left(u_i,v_i\right),z\left(u_i,v_i\right)\right]}^T, u\in \left[0,1\right],v\in \left[0,1\right].$$

(27)

The closest point $S\left(u_i,v_i\right)$ can be searched by solving the minimum distances.

$$\underset{\left(u_i,v_i\right)}{\mathrm{Arg} \min }\left(\sqrt{{\left(x_i^{key}-x\left(u_i,v_i\right)\right)}^2+{\left(y_i^{key}-y\left(u_i,v_i\right)\right)}^2+{\left(z_i^{key}-z\left(u_i,v_i\right)\right)}^2}\right).$$

(28)

Assuming that the i-th measured point is expressed as $P_i={\left[x_i,y_i,z_i\right]}^T$, and $P_i{}^{\prime }={\left[x_i{}^{\prime },y_i{}^{\prime },z_i{}^{\prime }\right]}^T$ is the point on the source surface S which is closest to the i-th localized measured point $P_i{}^{*}={\left[x_i{}^{*},y_i{}^{*},z_i{}^{*}\right]}^T$, the mathematical model of the surface profile error $s_{error}$ can be described as follows:

$$d_i=\Vert P_i^{*}-P_i^{\prime }\Vert {}_2,$$

(29)

$$s_{error}=max\left\{2\left|d_i\right|\right\}=max\left\{2\Vert P_i^{*}-P_i^{\prime }\Vert {}_2\right\},$$

(30)

where $d_i$ is the deviation between the i-th localized measured point and the source surface S, ${\tilde{P}}_i^{*}$ is the homogeneous coordinate of $P_i^{*}$, and ${\tilde{P}}_i$ is the homogeneous coordinate of the measured point $P_i{}^{\prime }$.

The surface profile error can be evaluated by iteratively aligning and non-rigidly deforming a measured mesh to an original source mesh. The final transformation matrix $T$ rotates and translates the measuring coordinate system using six parameters α, β, γ, Δx, Δy, Δz. Hence, $T$ can be expressed as follows:

$${\tilde{P}}_i^{*}=T·{\tilde{P}}_i, \mathrm{i}=1,2,\cdots ,n,$$

(31)

$$\mathrm{T}=\left[\begin{array}{cccc}\cos \beta \cos \gamma & \sin \alpha \sin \beta \cos \gamma - \cos \alpha \sin \gamma & \cos \alpha \sin \beta \cos \gamma + \sin \alpha \sin \gamma & \Delta x\\ \cos \beta \sin \gamma & \sin \alpha \sin \beta \sin \gamma + \cos \alpha \cos \gamma & \cos \alpha \sin \beta \sin \gamma - \sin \alpha \cos \gamma & \Delta y\\ -\sin \beta & \sin \alpha \cos \beta & \cos \alpha \cos \beta & \Delta z\\ 0& 0& 0& 1\end{array}\right],$$

(32)

where α, β, γ are the angles rotating the measuring coordinate system around the X, Y, and Z axis, respectively, and Δx, Δy, Δz are translations along the X, Y, and Z axes, respectively.

The thermal deformation comparison before and after improvement is shown in Figure 6.

5. Convolutional Generative Neural Network (CGAN) of Layer Customized Map (LCM)

The feature map $H_i$ of the $i$-th layer is determined according to the following equation:

$$H_i=f\left(W_i\otimes H_{i-1}+b_i\right)i\in \left[2,card\left(\mathit{d}\right)\right],$$

(33)

$$H_1=I_1,$$

(34)

where $W_i$ is the convolution kernel of the $i$-th layer, namely weight assigned, $b_i$ is the bias, $H_{i-1}$ is the feature map of the $i$-th layer, and $f$ is activation function. $\otimes$ is the convolutional operator.

In convolutional neural network, the definition of receptive field is the size of the region mapped by the pixels on the feature map output from each layer of the convolutional neural network on the input image. It represents the size of the perception range of neurons in different positions in the network to the original image as follows.

$$R_i=R_{i-1}+\left(\left(k_i-1\right)\ast {{\displaystyle \prod }}_{l=1}^{i-1}{s}_l\right),$$

(35)

where $R_i$ is receptive field of the $i$-th layer, $R_{i-1}$ is receptive field of the $\left(i-1\right)$-th layer, $k_i$ is convolution kernel size of the $i$-th layer, and $s_l$ is convolution kernel stride of $l$-th layer.

When the training data is sufficient, the neural network based on deep learning is introduced. The value for the $j$-th neuron of the k-th layer ($x_j^k$) is determined according to the following equation:

$$x_j^k=\sigma \left({\sum }_{i=1}^n{w}_{ji}^k·x_i^{k-1}+b_j^k\right),$$

(36)

where $w_{ji}^k$ is the weight assigned in the $j$-th neuron of the $k$-th layer to the input from $i$-th neuron of the previous layer, and $b_j^k$ is the bias, and n is the number of neurons in $\left(k-1\right)$-th layer. The bias $b_j^k$ acts as an independent polynomial term, and its value is adjusted in a similar manner to that of the weights $w_{ji}^k$. $\sigma$ is hyperbolic tangent Tanh activation function expressed by

$$\sigma \left(x\right)=\frac{1-e^{-2x}}{1+e^{-2x}}.$$

(37)

The training procedure is similar to a two-player min-max game with the following loss function.

$$\underset{G}{\min }\underset{D}{\max }V\left(D,G\right)={\mathbb{E}}_{x~p_{data}\left(x\right)}[\log D\left(x\right)]+{\mathbb{E}}_{z~p_z\left(z\right)}[\log \left(1-D\left(G\left(z\right)\right)\right)],$$

(38)

where $z$ is a noise vector sampled from distribution $p_z$ (e.g., uniform or Gaussian distribution), and $x$ is a real data from the distribution $p_{data}$.

The stochastic gradient of discriminator $D$ is updated by ascending in

$${\nabla }_{{\theta }_d}\frac{1}{m}{{\displaystyle \sum }}_{i=1}^m[\log D\left(x^{\left(i\right)}\right)+\log \left(1-D\left(G\left(z^{\left(i\right)}\right)\right)\right)],$$

(39)

where ${\nabla }_{{\theta }_d}$ is gradient solving operator of discriminator $D$, $m$ is the amount of data in a single minibatch, and $i$ is the data index.

Meanwhile, the stochastic gradient of generator $G$ is updated by descending in

$${\nabla }_{{\theta }_g}\frac{1}{m}{{\displaystyle \sum }}_{i=1}^m[\log \left(1-D\left(G\left(z^{\left(i\right)}\right)\right)\right)],$$

(40)

where ${\nabla }_{{\theta }_g}$ is gradient solving operator of generator $G$, $m$ is the amount of data in a single minibatch, and $i$ is the data index.

For output layer $Lr$, the error evaluation function $C_{Lr}$ of generator $G$ is defined as

$$C_{Lr}=\frac{1}{2m}{{\displaystyle \sum }}_x\Vert y\left(x\right)-a^{Lr}{\left(x\right)\Vert }^2,$$

(41)

where $y\left(x\right)$ is expected output in output layer $Lr$, and $a^{Lr}\left(x\right)$ is predicted output by the activation function $\sigma$.

The depth convolution with minimum residual is used to extract features of each layer shown in Figure 7.

6. Multi Objective Optimization (MOO) to Reduce Deformation

6.1. Mathematical Model of MOO

Heat on the multilayer of the structure gradually decreases due to thermal convection and external radiation, and the temperature gradually decreases. The CGAN model takes each layer end-effector temperature, temperature gradient, printing velocity as the optimization parameters and solved MOO using Non-dominated Sorting Genetic Algorithm-II to calculate thermal deformation using minimum residual depth convolution.

$$\begin{gathered} \mathrm{Find}: \mathit{x}=\left[T_n,\nabla T,V_p,\mathit{d}\right] \\ \mathrm{Minimize}: f\left(\mathit{x}\right)=\{s_{error},t_p\} \\ \mathrm{Subject} \mathrm{to}:\{\begin{array}{c}\min \left(d\right)\le d\le \max \left(d\right)\\ \min \left(V_p\right)\le V_F\le \min \left(V_p\right)\\ \min \left(T_n\right)\le T_n\le \max \left(T_n\right)\end{array}, \end{gathered}$$

(42)

where $\mathit{x}$—parameters of printing process; $\mathit{d}$—layer thickness (mm); $V_p$—printing velocity ($\mathrm{mm}·{\mathrm{s}}^{-1}$); $T_n$—end-effector temperature (K); $\nabla T$—temperature gradient ($\mathrm{K}·{\mathrm{mm}}^{-1}$); $t_T$—total print time (s); $f\left(\mathit{x}\right)$—objective function of printing process.

The total print time can also be estimated as follows:

$$t_p=\frac{L}{V_p}=\frac{r_{infill}{V}_{object}}{{\mathrm{S}}_A{V}_p},$$

(43)

where $t_p$ is the total print time $\left(s\right)$;$L$ is the length of trajectory $\left(mm\right)$;$V_p$ is the printing velocity $\left(\mathrm{mm}·{\mathrm{s}}^{-1}\right);r_{infill}$ is infill rate; $V_{object}$ is volume of object to be printed; ${\mathrm{S}}_A$ is cross area of filament.

6.2. MOO Results

The proposed method has been implemented using open source C++, OpenGL and Open Cave. The proposed method was tested in a 64-bit Windows 10 system PC environment with Intel(R) Core(TM) i5-2400 [email protected] and 32G RAM.

The Pareto Optimal Solutions (POS) using convolutional Generative Adversarial Network is shown in Figure 8 where Figure 8a shows an initial set of POS and Figure 8b shows a further optimized set of POS. By using the CGAN method, the 10 POS with best Y are, respectively, (−3.0621,25956.0000,31603.4968), (−3.0621,25956.0000,31689.7022), (−3.0621,25956.0000,31774.8731), (−3.0621,25956.0000,31857.9875), (−3.0621,25956.0000,31939.6714), (−3.0621,25956.0000,32020.4324), (−3.0621,25956.0000,32100.6860), (−3.0621,25956.0000,32180.7755), (−3.0621,25956.0000,32260.9862), (−3.0621,25956.0000,32341.5575).

The average thermal energy consumption (from Equation (22)) is decreased from 8.1582e + 04 J to 7.5606e + 04 J with ratio of 7.32% using the proposed method.

7. Physical Experiments and Results comparison

The specimens considered in the present work were fabricated on a 3DP 3D printer by using the commercial Polylactic Acid (PLA) as the fabrication material and the support material. As shown in Figure 9, the laser reconstruction equipment includes trilinear coordinates measuring instrument and binocular stereo vision laser scanner. The scanning mode has turntable scanning and fixed scanning. The scanning precision is less than 0.05 mm. The scan size can vary from 30 mm × 30 mm × 30 mm to 1200 mm × 1200 mm × 1200 mm.

The Figure 10 shows the virtual printing via digital twins considering manufacturing error. The amount of layer is thereafter same (herein 52) to enhance comparability. The lateral surface area indicates more evenly distribution after adaptive optimization.

The Figure 11 shows the comparison before and after optimization. Hereby, the maximum is marked with magenta ▲, and the minimum is marked with green ▇ (similarly hereinafter). Before optimization, $\max \left(\delta \right)$= 2.075594e-02 at $h_n$= 1.9608%, $\min \left(\delta \right)$= 4.795815e-03 at $h_n$= 90.1961%, $\mathrm{mean}\left(\delta \right)$= 0.0155, $\mathrm{std}\left(\delta \right)$= 0.0044, $\mathrm{var}\left(\delta \right)$ = 0.0000. After optimization, $\max \left(\delta \right)$= 2.695928e-02 at $h_n$= 1.9231%, $\min \left(\delta \right)$= 0 at $h_n$= 100.0000%%, $\mathrm{mean}\left(\delta \right)$= 0.0143, $\mathrm{std}\left(\delta \right)$= 0.0050, $\mathrm{var}\left(\delta \right)$= 0.0000.

The results comparison for various layer amounts are listed in

Table 1. Results comparison for various layer amounts.

Performance Parameters	Before Optimization	After Optimization	Ratio
Max $\left(\delta \right)$	2.075594e-02	2.695928e-02	29.89%
Min ($\delta$)	4.795815e-03	0	−100.00%
Max $\left(s_{error}\right)$	2.605556e-06	2.020809e-06	−22.44%
Min ($s_{error}$)	2.390000e-06	1.670376e-06	−30.11%

.

8. Conclusions

(1) A method of thermal Deformation defect deformation prediction for 3DP using Convolutional Generative Adversarial Network (CGAN) is put forward.

The multiscale Convolutional Generative Adversarial Network (CGAN) is employed to build the nonlinear implicit relations between thermal deformation and multi-scale features. The thermal deformation of each layer can be predicted with high precision for various shapes. The CGAN method is especially suitable for complex structures with diversified region-based requirements.

(2) The depth convolution is used to extract multi-scale features of LCM in each layer.

The CGAN model takes each layer end-effector temperature, temperature gradient, printing velocity as the optimization parameters and solved MOO using Non-dominated Sorting Genetic Algorithm-II to calculate thermal deformation using minimum residual depth convolution. The average thermal deformation can be decreased via CGAN. The visible primitives including points and facets are obtained based on convex hull principles, which paves the way for virtual printing via Digital Twins technology.

(3) The proposed CGAN method is verified by physical experiment.

The proposed method is verified by serialized physical experiments. The thermographs are firstly obtained using thermal field measurement to determine the temperature and temperature difference. The trilinear coordinates measuring instrument and binocular stereo vision laser scanner are then used to determine the actual thermal deformation of the printed objects. Compared with traditional methods, the CGAN method can deal with the thermal deformation with more optimal parameters which contributes to forward design of irregular complex parts.

In future, encouraged by the artificial intelligence technology, the proposed method will be planned to be more suitable for AM of more complex part structures, such as mechanical grooves, convex shoulders, matching holes, and inner cavity microfluidic channels, supported by knowledgebase big data.