Prediction of the Weld Qualities Using Surface Appearance Image in Resistance Spot Welding

Abstract

The quality of the resistance spot weld is predicted qualitatively using information from the weld’s external apparent image. The predicting tool used for weld qualities was a convolution neural network (CNN) algorithm with excellent performance in pattern recognition. A heat trace image of the weld surface was used as information on the external apparent image of welds. The materials used in the experiment were advanced high strength steel (AHSS) with 980 MPa strength, and uncoated cold-rolled (CR) steel sheets and galvannealed (GA) steel sheets were used. The quantitatively predicted weld quality information contained tensile shear strength, nugget diameter, fracture mode of welds, and expulsion occurrence. The predicted performance of the verification step of the model determined through the learning process was as follows; the predicted error rate for tensile shear strength and nugget diameter were 2.2% and 2.6%, respectively. And the predicted accuracy on fracture mode and expulsion occurrence was 100%.

1. Introduction

The ability to predict or evaluate the quality of resistance spot welding (RSW) in real-time using nondestructive methods is key to the automation of automotive assembly processes. The tensile shear strength (TSS) of the weld, nugget diameter, button size after the peel test, and failure mode (interfacial fracture or pull-out fracture) have generally been used as evaluation items of the conventional resistance spot welding. However, since the quality evaluation method requires time and manpower, many studies have been conducted to predict the quality nondestructively. Several research results predicted quality using the process variable setting conditions. Hao et al. [1] studied statistical models on nugget diameter, weld strength, and expulsion according to process variable conditions using multiple linear regression analysis. Muhammad et al. [2] proposed the models to predict the growth of nugget diameter and heat-affected zone (HAZ), which consisted of welding current, welding time and hold time as variables using the surface response analysis method in RSW of mild steel. Darwish et al. [3] studied the prediction model of TSS, which consisted of welding current, electrode force, welding time, and material thickness as variables using surface response analysis method in aluminum resistance spot welding. Choi et al. [4] studied the optimal welding conditions with electrode force and welding current as variables using the response surface analysis method and demonstrated that the minimum nugget diameter of 4$\sqrt{t}$ guaranteed the minimum button fracture in RSW of 980 MPa third-generation AHSS. Studies on the quality prediction of resistance point welds using numerical analysis have been reported as follows. Nied et al. [5] reported that the simulated results were very excellent when the finite element modeling of RSW process included heat transfer-electrical behavior-mechanical balance equations and temperature-dependent material properties. The results of the study on the quality evaluation of resistance spot welds using nondestructive detection technology were also reported. Denisov et al. [6] measured the nugget diameter using the 2D ultrasonic arrays technique (PAUT). However, this method has shown the disadvantage of requiring expensive ultrasonic equipment or added processes. Predictive models by statistical regressive methods had limitations inaccurate quality predictions because the condition of the electrode surface in continuous welding did not reflect changes in weld quality due to contamination.

Studies of predicting the weld quality using measured waveforms of welding process variables (current, voltage, dynamic resistance, displacement, etc.) have been reported. Johnson et al. showed that the weld quality was different due to variations in electrode displacement [7]. Dickinson et al. [8] studied the mechanism for controlling expulsion occurrence conditions using a dynamic resistance monitoring system that can continuously monitor voltage, current, resistance, and power during RSW. Brown et al. [9] tried to applicate data of standardized dynamic resistance waveform, welding current waveform and electrode diameter to the artificial neural network to predict the nugget diameter, which is closely related to weld strength. Dilthey et al. [10] reported a neural network, in which changes in welding parameters, test materials, and electrode wear are considered, that can perform very well in evaluating the quality of spot welds. Wan et al. [11] proposed a weld quality monitoring approach based on dynamic resistance signal to predict the nugget size and failure load for the resistance spot welding of titanium alloy. Xing et al. [12] proposed a scheme of online quality monitoring of RSW using the random forest (RF) classification featuring with dynamic resistance signals. Also Hwang et al. [13] reported that the predicted error rate was within 5% using adaptive resonance theory (ART) artificial neural network (pattern recognition). The input parameters used were welding current, the voltage between electrodes and welding power waveforms and the output parameters used were TSS and the nugget diameter.

Some studies on the quality evaluation of welds through image analysis have also been reported. Ruisz et al. [14] reported quality evaluation of resistance spot weld based on computer vision algorithm. Simončič et al. [15] predicted the resistance spot weld strength from electrode tip displacement/velocity by image processing.

On the other hand, light-weighting of the automotive body to improve fuel efficiency has been rapidly taking place around the world. As a result, the application of third-generation AHSS steel is expanding. The optimal welding conditions of these GPa-grade steels have a very narrow range compared to conventional AHSS steels, and the interfacial fracture occurs frequently in the weld fracture mode. Also, differences in weld quality may occur under the same welding conditions. Therefore, it is urgent to predict the quality of the welds in RSW of these third-generation AHSS steel.

In RSW, the shape of the weld ultimately has an important effect on the weld quality, no matter what welding equipment is used. In RSW, a heat trace is produced around the weld surface due to a high current input. The heat traces are formed in round shapes and colors. The greater the heat input and the greater the melt, the greater the range of the heat trace and the deeper the indentation depth. In other words, the increase in weld current and weld time will result in an increase in the weld and HAZ, while increasing the size of the same time.

The purpose of this study is to suggest how to predict the weld quality of GPa-grade steel, where the quality of welds (TSS, nugget diameter, and fracture mode) varies significantly despite slight changes in the process setting conditions. For this purpose, a heat trace image of the surface appearance of welds was used as a determinant factor of weld quality after welding of RSW of a two-layer 980 MPa steel. These images are slightly different between zinc-coated steel and uncoated steel in the case of GPa-grade steel with the same strength. In this study, these heat trace images were used as an input variable, and the qualities of the weld (TSS, nugget diameter, and fracture mode of the weld (interfacial or button)) were predicted through the convolution natural network (CNN) algorithm using uncoated cold-rolled (CR) and galvannealed (GA) steel sheets. The prediction performance of qualities was investigated in the learning process and verification process with different experimental data sets.

2. Experiments and Learning Methods

2.1. Materials and Welding Conditions

The materials used were GA steel sheet and CR steel sheet with the tensile strength of 980 MPa (21% elongation). The thickness of the material is 1.2 mm for both sheets. The test specimen was prepared by ISO 10447:2015 specifications. The length and width of specimen used were 100 mm and 30 mm, respectively with the overlap of 30 mm as shown in Figure 1. The chemical composition and mechanical properties of the materials used are shown in Table 1. The welding equipment used in the experiment is a medium frequency direct current (MFDC) welder.

Figure 1. Geometry of welding specimen.

Table 1. The chemical composition and mechanical properties of 980 MPa-grade GA, CR steel.

Materials	Chemical composition (wt. %)				Mechanical properties
	C	Si	Mn	Fe	Ultimate Tensile Strength (MPa)	Elongation (%)	Yield Strength (MPa)
GA steel	0.22	1.62	2.12	Bal.	988	15	400
CR steel	0.20	1.59	2.40	Bal.	990	15	500

Dome type electrode with an electrode face diameter of 6 mm is applied, whose detailed dimensions are shown in Figure 2.

Figure 2. Schematic diagram of electrode.

The process setting conditions of RSW for CNN learning are shown in Table 2. Welding parameters mentioned in Table 2 are determined through the weld lobe for same base material and joint geometry used in this study. To reduce test errors, 10-iteration welding was conducted for each welding condition. Five of ten welded specimen were used to measure TSS, while the other five were used to measure the nugget diameter. Weld current was set to three levels of 4.0, 5.5, and 7.0 kA. Weld time was also set to three levels of 250, 333, and 417 ms. The electrode force was fixed at 300 kgf.

Table 2. Welding conditions of training data set.

Welding Conditions	980 MPa-Grade GA Steel	980 MPa-Grade CR Steel
Welding current (kA)	4.0, 5.5, 7.0
Welding time (ms)	250, 333, 417
Electrode Force (kgf)	300

2.2. Weld Surface Appearance Image Processing

Figure 3 is a conceptual diagram of the surface appearance of the weld. Though not expressed in Figure 3, the photographing was carried out within an enclosed space where no outside light could enter. Illumination was used for the same exposure to the surface of the weld. The distance between the illuminating and the surface of the weld was 2 m and illuminated with an angle of 30° from vertical. The color of the light was white. The amount of light is 1600 lm. The distance between the surface of the weld and the camera was 120 mm vertically. The pixel size of the image taken was 128 × 128. The original image taken was a color image, but in this study, it was converted to a gray image and used as a quality prediction video input data. The data values of each pixel of the converted gray image were normalized as values between [0, 1] with the minimum value of 0 and the maximum value of 1. The mapped image between 0 and 1 and those image processing process is shown as Figure 4.

Figure 3. Condition of image data capture.

Figure 4. Image data conversion process.

2.3. Convolution Neural Network Algorithm for Prediction of Weld Quality

In this study, CNN was used to predict the quality of RSW. CNN is one of the deep learning algorithms that showed excellent performance in image recognition. In the general in-depth neural network, the meaning of successive lines and shapes of images is occasionally ignored when analyzing images. Each pixel datum affects all nodes. Input data of the general in-depth neural network have been flattened with simple one-dimensional vector data, and have been learned. So, limitations in the prediction accuracy have been shown. However, the convolution layer uses a kernel of a specified size in the shape of the image data. Image is reconstructed by extracting characteristic variables from areas as large as kernel sizes. So, the successive lines or characteristic data influences the results. As a result, the convolution neural network shows the prediction performance with higher accuracy in image recognition. Figure 5 shows the structure of CNN applied in this study. A total of 90 heat trace images were taken for each parameter of welding in two types of steel. The pixel size of each image was 16,384 pixels (128 × 128). After converting to Gray image, all pixel data were normalized as values between 0 and 1. Mapped image data was used as input data. A total of four CNN models for predicting the weld qualities were obtained using the input mapped data. Each model predicts the TSS, nugget diameter, fracture mode, and expulsion occurrence, which are criteria for determining the quality of RSW. Therefore, each model has different weights and biases.

Figure 5. Convolution neural network algorithm for weldability prediction.

Two layers were used as the hidden layer of CNN for predicting the TSS and the nugget diameter. Also, two layers were used as pooling layer. One layer with 200 nodes was used as the hidden layer of CNN for predicting the fracture shape and the expulsion occurrence. The method of slope used (optimizer) used Adam. The rectified linear unit (ReLU) function was used as an activation function for predicting the TSS and the nugget diameter. The sigmoid function was used as an activation function for predicting the fracture shape and the expulsion occurrence. The reason for using the sigmoid function is that it is more accurate than the ReLU when predicting the results by classifying the results of the prediction by 0 and 1, such as fracture shape and expulsion occurrence. The function of the ReLU and its differential values are shown in Equations (1) and (2), respectively.

$$\mathrm{y}=\{\begin{array}{c}x(x>0)\\ 0\left(x\le 0\right)\end{array}$$

(1)

$$\frac{\partial y}{\partial x}=\{\begin{array}{c}1(x>0)\\ 0\left(x\le 0\right)\end{array}$$

(2)

The ReLU function has zero slope value when the output value is negative and one when the output value is positive. Even if the neural network layer is thick, the differential value is 1 when the output value is positive, so the parameter renewal of each node is made during an error backpropagation procedure. In this process, the characteristics of input data greatly affect parameter updates. Thus, the ReLU function contributes to increasing the accuracy of the neural network model. The exact parameter values of the applied CNN algorithm are specified in Table 3.

Table 3. Convolution neural network algorithm for weldability prediction.

Convolution Layer 1
Kernel amount	32
Kernel size	4,4
Pooling size	2,2
Activation function	ReLU
Convolution Layer 2
Kernel amount	32
Kernel size	4,4
Pooling size	2,2
Activation function	ReLU
Fully Connected Layer
Node	200, 400, 800, 1200, 800, 400, 200
Activation function	ReLU

The learning process of neural networks is the process of finding the optimum values of weights and bias values using the training dataset. Repeated learning was conducted to find parameters (weights and bias values) with minimum error values between the actual and calculated results using the error calculation, as shown in Equation (3).

$$\mathrm{E}=\frac{1}{2}{\displaystyle {\displaystyle \sum }_k}{\left(y_k-t_k\right)}^2$$

(3)

where y_k is the desired output t_k is the target output.

3. Results and Discussion

3.1. Result of Welding Experiments for Surface Heat Trace

Figure 6 represents the heat trace images of the weld surface under several welding conditions. It has been found that the more current and time the welds have, the greater the heat marks on the surface of the specimen. The dilation of the heat trace means an increase in the TSS and the nugget diameter. The GA sheet showed heat traces smaller than CR sheet. Figure 7 is the converted gray images of welds in CR sheet used in the test. According to Figure 7, the image of the heat trace at the same welding current and time was not the same. In addition to welding conditions, many factors affect welding, so each image is very slightly different. In particular, the last image of Figure 7 was shown as a dimmer heat trace, with no apparent circumference of the heat trace circles than the other four.

Figure 6. Reference welds part images of all parameter.

Figure 7. Various gray images of welds in CR steel sheet under the welding time of 417 ms and welding current of 7 kA.

3.2. Comparison of Welding Quality Results and Predicted Values According to Welding Conditions

The actual measured values and the predicted values by CNN on the TSSs, the nugget diameters, the fracture shapes, and the expulsion occurrence at all welding conditions were shown in Table 4. In the fracture mode, the button fracture was marked as 1 and the interfacial fracture as 0. In the case of CR steel sheet, the expulsion was occurred at the welding current of 7 kA, regardless of the welding time. The TSS was shown a maximum of 19.2 kN and the nugget diameter was enlarged to 6.3 mm. In the case of GA, no expulsions were occurred at welding current 7 kA and interfacial fractures have occurred under all conditions. The TSS was shown up to 19.4 kN and the nugget diameter was enlarged up to 6.1 mm.

Table 4. Results of spot weldability for each parameter.

Order.	Welding Condition			Measured Result				Predicted Result
	Material	Current (kA)	Time (ms)	Tensile Shear Strength (kN)	Nugget Size (mm)	Fracture Mode	Expulsion	Tensile Shear Strength (kN)	Nugget Size (mm)	Fracture Mode	Expulsion
1	CR	4	250	7.3	3.6	0	0	7.3	3.6	0	0
2	CR	4	250	7.6	3.5	0	0	7.6	3.5	0	0
3	CR	4	250	7.5	3.5	0	0	7.5	3.5	0	0
4	CR	4	250	7.6	3.4	0	0	7.6	3.4	0	0
5	CR	4	250	7.5	3.5	0	0	7.6	3.4	0	0
6	CR	4	333	7.7	3.8	0	0	7.7	3.8	0	0
7	CR	4	333	8.1	3.6	0	0	8.1	3.6	0	0
8	CR	4	333	8.5	3.5	0	0	8.5	3.5	0	0
9	CR	4	333	8	3.5	0	0	8	3.5	0	0
10	CR	4	333	8	3.6	0	0	8	3.6	0	0
11	CR	4	417	8	3.9	0	0	8	3.9	0	0
12	CR	4	417	7.8	3.9	0	0	7.8	3.9	0	0
13	CR	4	417	8.3	4	0	0	8.3	4	0	0
14	CR	4	417	8.7	4	0	0	8.7	4	0	0
15	CR	4	417	8.8	4	0	0	7.6	3.7	0	0
16	CR	5.5	250	14.1	5	0	0	14.1	5	0	0
17	CR	5.5	250	14.9	4.8	0	0	14.9	4.8	0	0
18	CR	5.5	250	14.2	4.7	0	0	14.2	4.7	0	0
19	CR	5.5	250	14.5	4.7	0	0	14.5	4.7	0	0
20	CR	5.5	250	14.8	5	0	0	9.4	4.2	0	0
21	CR	5.5	333	14.6	5	0	0	14.6	5	0	0
22	CR	5.5	333	14.8	5.1	0	0	14.8	5.1	0	0
23	CR	5.5	333	15	5.2	0	0	15	5.2	0	0
24	CR	5.5	333	14.9	5.3	0	0	14.9	5.3	0	0
25	CR	5.5	333	14.7	5	0	0	14.8	5.4	0	0
26	CR	5.5	417	14.6	5.2	0	0	14.6	5.2	0	0
27	CR	5.5	417	15.3	5.4	0	0	15.3	5.4	0	0
28	CR	5.5	417	15.1	5.4	0	0	15.1	5.4	0	0
29	CR	5.5	417	14.8	5.3	0	0	14.8	5.3	0	0
30	CR	5.5	417	14.6	5.4	0	0	14.7	5.1	0	0
31	CR	7	250	17.8	5.9	1	1	17.8	5.9	1	1
32	CR	7	250	17.9	5.8	1	1	17.9	5.8	1	1
33	CR	7	250	17.8	5.5	1	1	17.8	5.5	1	1
34	CR	7	250	18	5.7	1	1	18	5.7	1	1
35	CR	7	250	17.6	5.9	1	1	18.1	5.9	1	1
36	CR	7	333	18.5	6	1	1	18.5	6	1	1
37	CR	7	333	18.5	6	1	1	18.5	6	1	1
38	CR	7	333	18.8	5.9	1	1	18.8	5.9	1	1
39	CR	7	333	18.3	5.9	1	1	18.3	5.9	1	1
40	CR	7	333	18.6	6.2	1	1	18.3	5.8	1	1
41	CR	7	417	18.4	6.2	1	1	18.4	6.2	1	1
42	CR	7	417	18.5	6.3	1	1	18.5	6.3	1	1
43	CR	7	417	17.8	5.9	1	1	17.8	5.9	1	1
44	CR	7	417	19.2	6.2	1	1	19.2	6.2	1	1
45	CR	7	417	18.9	6	1	1	14.7	5.8	1	1
46	GA	4	250	5.2	2.1	0	0	5.1	2.1	0	0
47	GA	4	250	5.2	2.2	0	0	5.2	2.2	0	0
48	GA	4	250	5.1	2.1	0	0	5.1	2.1	0	0
49	GA	4	250	4.8	2.2	0	0	4.8	2.2	0	0
50	GA	4	250	5	2.3	0	0	6.5	2.5	0	0
51	GA	4	333	6.4	2.4	0	0	6.4	2.4	0	0
52	GA	4	333	6.2	2.5	0	0	6.2	2.5	0	0
53	GA	4	333	6.3	2.5	0	0	6.3	2.5	0	0
54	GA	4	333	6.1	2.7	0	0	6.1	2.6	0	0
55	GA	4	333	5.9	2.4	0	0	6.7	2.5	0	0
56	GA	4	417	5.9	2.6	0	0	5.9	2.6	0	0
57	GA	4	417	5.9	2.3	0	0	5.9	2.3	0	0
58	GA	4	417	5.8	2.6	0	0	5.8	2.6	0	0
59	GA	4	417	6	2.7	0	0	6	2.7	0	0
60	GA	4	417	5.5	2.5	0	0	7.6	2.6	0	0
61	GA	5.5	250	12.4	4.6	0	0	12.4	4.6	0	0
62	GA	5.5	250	12.6	4.7	0	0	12.6	4.7	0	0
63	GA	5.5	250	13	4.5	0	0	13	4.5	0	0
64	GA	5.5	250	13.1	4.3	0	0	13.1	4.3	0	0
65	GA	5.5	250	12.9	4.7	0	0	12.9	4.5	0	0
66	GA	5.5	333	13.7	4.9	0	0	13.7	4.9	0	0
67	GA	5.5	333	13.8	4.9	0	0	13.8	4.9	0	0
68	GA	5.5	333	13.9	4.7	0	0	13.9	4.7	0	0
69	GA	5.5	333	13.6	4.7	0	0	13.6	4.7	0	0
70	GA	5.5	333	14.1	4.9	0	0	12.4	4.6	0	0
71	GA	5.5	417	12.9	5	0	0	12.9	5	0	0
72	GA	5.5	417	13.6	5.1	0	0	13.6	5.1	0	0
73	GA	5.5	417	13.7	5.2	0	0	13.7	5.2	0	0
74	GA	5.5	417	13.5	5.1	0	0	13.5	5.1	0	0
75	GA	5.5	417	14.2	5.2	0	0	14.2	4.9	0	0
76	GA	7	250	18.4	5.4	0	0	18.4	5.4	0	0
77	GA	7	250	18.1	5.6	0	0	18.1	5.6	0	0
78	GA	7	250	18	5.5	0	0	18	5.5	0	0
79	GA	7	250	18.5	5.3	0	0	18.5	5.3	0	0
80	GA	7	250	18.3	5.6	0	0	17.9	5.3	0	0
81	GA	7	333	18.8	5.8	0	0	18.8	5.8	0	0
82	GA	7	333	18.8	5.7	0	0	18.8	5.7	0	0
83	GA	7	333	18.8	5.7	0	0	18.8	5.7	0	0
84	GA	7	333	19.2	5.5	0	0	19.2	5.5	0	0
85	GA	7	333	19.4	5.9	0	0	15.1	5.2	0	0
86	GA	7	417	18.7	5.9	0	0	18.7	5.9	0	0
87	GA	7	417	19.1	6.1	0	0	19.1	6.1	0	0
88	GA	7	417	19.2	5.9	0	0	19.2	5.9	0	0
89	GA	7	417	19.2	5.7	0	0	19.2	5.7	0	0
90	GA	7	417	19.4	6	0	0	18.8	5.7	0	0

3.3. Neural Network Learning Using CNN

Figure 8 is a graph that compares the measured and predicted TSSs at all welding conditions of the learning process. The prediction accuracy of the TSSs used for learning process was 98.6% and the coefficient of determination was 0.9943. Figure 9 is a graph to compare between the measured and the predicted nugget diameters at all welding conditions of the learning process. The prediction accuracy of the nugget diameters used for learning process was 98.8% and the coefficient of determination was 0.9857.

Figure 8. Correlation between predicted and measured values of tensile shear strength.

Figure 9. Correlation between predicted and measured values of nugget diameter.

Figure 10 shows the predicted results of the fracture shapes and expulsions. In the fracture mode prediction, value 0 means the interfacial fracture, and 1 means the button fracture. In the expulsion occurrence prediction, 0 means no expulsion occurrence, and 1 means expulsion occurrence. As a result, both the prediction accuracies of the fracture shape and the expulsion occurrence were 100% at all welding conditions. In Figure 10, all the data on the graph is concentrated on 0 and 1, and it is seen as only one point.

Figure 10. Correlation between predicted and measured values of (a) fracture mode and (b) expulsion.

Figure 11 is a graph that is grouped by steel from the predicted results of Figure 8 and Figure 9. The predicted accuracy of TSS of CR steel was 99.5% and the coefficient of determination was 0.9978. The predicted accuracy of the nugget diameter was 98.9% and the coefficient of determination was 0.9732. The predicted accuracy of TSS of GA steel was 97.7% and the coefficient of determination was 0.9923. The predicted accuracy of the nugget diameter was 98.7% and the coefficient of determination is 0.9908. Comparing the predicted accuracy of TSS and the nugget diameter by steel type, CR steel was shown slightly higher accuracy than GA steel. This is thought to be because the heat traces vary depending on whether the surface is plated or not. In the surface heat trace images of the CR steel of Figure 6 above, the surface colors among the indented zone, shoulder zone, and HAZ were bounded and widely spread. It is thought that the difference between these colors may have affected the accuracy of the prediction. In the case of GA steel of Figure 6, the surface colors were slightly dim than CR steel. So, the prediction accuracies of GA steel were shown slightly lower than CR steel. It meant that input data were different depending on the surface heat trace image of different material types.

Figure 11. Comparison between predicted and measured values for (a) tensile shear strength of CR and (b) GA steel sheets, (c) nugget size of CR, and (d) GA steel sheets.

3.4. Verification of Predictive CNN Models

The setting conditions of RSW for CNN verification are shown in Table 5. A total of 4 welding conditions were selected and the verification experiment was repeated five times at same one condition. For these four verification test conditions, welding current and welding time were selected, except for the welding conditions used in the learning.

Table 5. Welding conditions of verification data.

Welding Conditions of verification test	980 MPa-Grade GA Steel	980 MPa-Grade CR Steel
Welding current (kA)	6	5
Welding time (ms)	300, 400	300, 400
Electrode Force (kgf)	300

Table 6 shows the measured and predicted results using the CNN model produced during the learning process in the 20 verification conditions of. The maximum prediction error was 1 kN for TSS and the error of nugget diameter was max 0.5 mm. Fracture shape and expulsion occurrence were correctly predicted in all verification tests.

Table 6. Results of verification test.

Order.	Welding Condition			Measured Result				Predicted Result
	Material	Current (kA)	Time (ms)	Tensile Shear Strength (kN)	Nugget Size (mm)	Fracture Mode	Expulsion	Tensile Shear Strength (kN)	Nugget Size (mm)	Fracture Mode	Expulsion
1	CR	5	300	12.1	3.9	0	0	12.3	3.7	0	0
2	CR	5	300	12.4	4.0	0	0	12.7	3.8	0	0
3	CR	5	300	11.8	4.0	0	0	10.2	4.1	0	0
4	CR	5	300	11.9	3.8	0	0	12.0	3.8	0	0
5	CR	5	300	12.0	3.8	0	0	11.7	3.9	0	0
6	CR	5	400	15.5	4.2	0	0	15.2	4.1	0	0
7	CR	5	400	15.9	4.2	0	0	16.4	4.2	0	0
8	CR	5	400	15.9	4.1	0	0	16.2	4.1	0	0
9	CR	5	400	15.8	4.1	0	0	15.3	4.1	0	0
10	CR	5	400	15.8	4.2	0	0	16.8	4.3	0	0
11	GA	6	300	9.7	4.5	0	0	9.5	4.6	0	0
12	GA	6	300	9.8	4.6	0	0	9.7	4.5	0	0
13	GA	6	300	10.3	4.7	0	0	10.9	4.7	0	0
14	GA	6	300	10.2	4.6	0	0	9.8	4.7	0	0
15	GA	6	300	10.0	4.7	0	0	10.2	4.6	0	0
16	GA	6	400	14.1	5.0	0	0	13.1	5.2	0	0
17	GA	6	400	14.6	4.9	0	0	14.9	5.1	0	0
18	GA	6	400	14.3	4.6	0	0	14.3	5.1	0	0
19	GA	6	400	14.3	5.0	0	0	14.2	4.9	0	0
20	GA	6	400	14.4	5.0	0	0	14.7	4.9	0	0

Figure 12 is a graph comparing the predicted and measured values of TSS and nugget diameter in the verification test results. The predicted accuracy of TSS was 97.8% and the coefficient of determination was 0.9463. For nugget diameter, the average accuracy of the forecast was 97.4% and the coefficient of determination was 0.888. The predicted results of the verification tests demonstrated that CNN quality prediction is highly accurately predicted using surface heat trace images of welds in RSW.

Figure 12. Correlation between predicted and measured values of (a) tensile shear strength and (b) nugget diameter.

Figure 13 shows a surface heat trace image of welds with lower prediction accuracy in verification tests. In Figure 13a–f, data with large errors between measured and predicted values are shown. For Figure 13a,b, the measured value was measured at 11.8 kN, while the predicted value was 10.2 kN with a 13.6% error rate. Comparing with the other surface heat trace images at the same condition, the abnormal heat trace shape stains at shoulder zone was shown, differencing from the heat trace shape of the normal circular shape. For Figure 13c,d, the measured value of TSS was 15.8 kN, the predicted value was 16.8 kN, and the error was 6.3%. Comparing with the other surface heat trace images of the same condition, the bigger black circle at the center of the indented zone was shown. In the case of Figure 13e,f, the measured value of nugget diameter was 4.6 mm while the predicted value was 5.1 mm and the error rate was 10.9%. Comparing with the other surface heat trace images of the same condition, due to the lack of focus in the photographing process, the entire weld area was blurred with a little clear boundary. The state of the image may have been viewed differently because the state of the light in the learning and the state of the light in the verification process did not match exactly. This study tried to predict qualities of welds using the surface heat trace images in RSW. By using this prediction technique, it is possible to predict the quality of the resistance spot weld in the production line. It also enables monitor the weld quality and reduces the defect rate. However, the surface heat trace images depend on heat transfer conditions, contamination of electrodes, the slope of electrodes, misalignment, etc. Therefore, it is very important to obtain an image under the same photographing condition to make accurate prediction. Prediction of weld quality when the image condition is bad or inaccurate as mentioned above will be discussed in future studies. Also, in this study, the weld quality is predicted by applying CNN to base material and welding conditions under specific conditions. If the type, thickness of base material and welding conditions are different, it is expected that accurate welding quality can be predicted by optimizing the weight between nodes through additional learning.

Figure 13. Images with inaccurate predicted values.

4. Conclusions

Using the surface heat trace image of weld in the resistance spot welding of 980 MPa-grade GA steel and CR steel as an input variable of convolution neural network, tensile shear strength, nugget diameter, fracture shape, and expulsion occurrence were predicted, and the following conclusions were obtained.

(1)

In the learning process, the coefficient of determination of tensile shear strength is 0.9943 and the coefficient of determination of nugget diameter is 0.9857. In the verification process, the predicted average error rate of tensile shear strength and nugget diameter are 3.2% and 2.6%, respectively, and the fracture shape and expulsion occurrence are accurately predicted. It has been demonstrated that accurate quality predictions can be made using the image of the welds in resistance spot welding.

(2)

Even if the surface treatment of steel is different, it has been proven that good weld quality can be predicted.

(3)

In the process of photographing a surface heat trace image, if disturbances that affect the image, such as the focus or lighting of the camera, occur, a large error in the prediction of quality can occur.