Next Article in Journal
Experimental Thrust and Specific Impulse Analysis of Pulsed Detonation Combustor
Previous Article in Journal
Improvement of Buildings’ Air Quality and Energy Consumption Using Air Purifying Paints
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Vision-Based Online Molten Pool Image Acquisition and Assessment for Quality Monitoring in Gas–Metal Arc Welding

1
Flexible Manufacturing R&D Department, Korea Institute of Industrial Technology, Incheon 21999, Republic of Korea
2
Department of Mechanical Convergence Engineering, Hanyang University, Seoul 04763, Republic of Korea
3
Advance Research Team, Hwashin, Yeongcheon 38828, Republic of Korea
*
Authors to whom correspondence should be addressed.
Appl. Sci. 2024, 14(14), 5998; https://doi.org/10.3390/app14145998
Submission received: 22 May 2024 / Revised: 18 June 2024 / Accepted: 7 July 2024 / Published: 9 July 2024
(This article belongs to the Section Materials Science and Engineering)

Abstract

:
In gas–metal arc welding (GMAW), the welding quality is influenced by various factors, including the welding conditions and external environment. Achieving a high welding quality requires careful observation and control. Welding monitoring is crucial for validating a planned welding process and assessing its quality. This study introduces a vision system for the online acquisition of clear molten pool images. It uses a small camera equipped with a complementary metal-oxide-semiconductor sensor during GMAW, as well as a method for constructing this system. The proposed vision system does not require triggering using devices such as field-programmable gate arrays, facilitating lower costs and online quality monitoring under various welding conditions during mild-steel GMAW. Accurate measurement of radiation from the weld is essential for clear observation of the molten pool during welding, emphasizing the importance of selecting an optical system with appropriate lenses and filters. In addition, appropriate digital camera parameters must be set for monitoring. High-quality images were successfully captured using the proposed method, and the image quality was evaluated using the blind/referenceless image spatial quality evaluator algorithm.

1. Introduction

During gas–metal arc welding (GMAW), welding quality is influenced by various factors, including the welding conditions and external environment. Achieving a high welding quality requires careful monitoring, which is essential for verifying the planned welding process and assessing its quality. Various technologies have been utilized for the nondestructive inspection of GMAW, such as ultrasonic inspection, vision-based quality inspection, and process signal-based quality prediction techniques. A commonly applied method for vision-camera-based quality inspection involves measuring the external weld geometry and inspecting external defects using a laser vision sensor after welding. Specifically, it determines whether the quality standards required by the consumer are satisfied by extracting feature points after scanning the weld bead surface. However, this method has limitations in real-time monitoring and requires reworking when welding defects occur, leading to an increased working time. Another technique for determining the welding quality involves collecting electrical and acoustic information in real time during welding; however, it has limitations owing to its noise in the surrounding environment. The molten pool surface provides intuitive and abundant feature information that allows the welder to assess the welding quality and adjust the welding conditions during visual inspection [1]. Intelligent systems have been developed to improve the productivity of the welding process, and the acquisition of molten pool images utilizing vision sensors is crucial for intelligent automated systems [2,3,4,5,6,7,8]. Although numerous studies have been conducted on vision monitoring systems for arc welding automation, they have primarily focused on gas tungsten arc welding, which has fewer limitations in acquiring molten pool images [9,10,11,12,13,14,15]. During GMAW, capturing images of the molten pool is challenging because of the dynamic vibrations caused by droplet transfer and volumetric changes in the arc plasma. In addition, the sufficiently strong radiation emitted from the arc plasma interrupts the observation of the region of interest (ROI), resulting in inevitable information loss.
Radiation from the arc plasma is significantly stronger than that reflected and emitted from the molten pool surface or weld. The arc plasma is not an ROI, and more information can be obtained from the molten pool surface. Previous studies have divided gas–metal arc welding monitoring methods into active- and passive-vision methods. Active vision cuts off the arc plasma using physical methods, such as neutral density (ND) filters, and utilizes separate light sources to observe the molten pool surface and droplet transfer. Generally, a laser is utilized as the light source, and images are acquired using a high-speed camera. However, the dynamics of images in active vision is unsatisfactory despite the utilization of auxiliary light sources because ND filters with excessively high optical densities are used to inhibit the interference of arc plasma [16]. Active-vision system configurations incur considerable costs.
In contrast, passive vision involves observing the molten pool by utilizing the emitted radiation with no auxiliary light source. Passive vision is suitable for intelligent welding because of its relatively simple configuration. Because vision systems focus more on applications in production lines than on research and analysis, a less costly method is reasonable. Zhang et al. [17] acquired molten pool images utilizing a 650 nm narrowband filter based only on the radiation from the molten pool during aluminum alloy gas tungsten arc welding. However, they neglected the quantum efficiency of the cameras. Jin et al. [18] prepared a passive-vision sensor for molten pool observation and an active-vision sensor for weld bead geometry measurement and utilized a screen between the torch and laser module to avoid the interference of the radiation from the arc plasma. This configuration is complex and possesses limitations in terms of its response to various welding conditions. Zhao et al. [19] selected a 660 nm band-pass filter or an 850 nm high-pass filter as the imaging band for 304 stainless steel GMAW via image quality assessment. However, clear monitoring may not be possible because the images are acquired only in the base-current section, where the arc is extinguished using a trigger device. Reisgen et al. [20] monitored gas–metal arc welding using passive vision with a monochromatic sensor. However, because the sensor was installed in front of the welding torch along the welding direction, observing the entire area of the molten pool was difficult. Cai et al. [21] conducted a study captured melt-pool images from laser–arc hybrid welding and used them for real-time monitoring. However, detailed observation of the molten pool was difficult because the interference of the arc plasma cannot be suppressed. In previous studies, only the spectroscopic characteristics of the radiation generated during welding and the quantum efficiency of cameras were considered when proposing passive vision monitoring methods.
In this study, we propose a vision system that uses a vision camera equipped with a complementary metal-oxide-semiconductor (CMOS) sensor to acquire clear molten pool images online during welding, as well as a method to construct the system. The proposed vision system is not complex and enables low-cost welding quality monitoring with a simpler configuration than conventional systems. The proposed vision system was used to perform online quality monitoring during mild-steel gas–metal arc welding. The accurate measurement of radiation from the weld is essential for observing the molten pool during welding, and selecting appropriate lenses and filters for monitoring is crucial. When the quality of the captured images was assessed using the blind/referenceless image spatial quality evaluator (BRISQUE) technique, the quality of the images acquired using the proposed method was found to be suitable for welding monitoring, thereby verifying its performance.

2. Materials and Methods

2.1. Welding Conditions

The base metal used in the experiment was ISO 630 E355 [22] with dimensions of 300 mm × 100 mm × 15 mm, while the filler wire was ISO 636-A W 42 5 2Si [23] (The Lincoln Electric Company, Cleveland, OH, USA) with diameter of 1.2 mm; their chemical compositions are listed in Table 1 and Table 2, respectively. The welding power source was a Daihen P500L (Daihen Corporation, Osaka, Japan), and the current mode was a direct current (DC) pulse. Welding was performed by installing a welding torch using YASKAWA’s six-axis robot (Yaskawa, Kitakyushu, Japan). The welding current and voltage were 250 A and 28 V, respectively. The wire feed rate was 8.5 m/min. The contact tip-to-work distance (CTWD) was 15 mm, and the welding speed was 30 cm/min. A mixture of argon and CO2 in a ratio of 9:1 was used as the shielding gas at a flow rate of 15 L/min.

2.2. Analysis of Spectroscopic Characteristics and Wavelength Band Selection

The strong radiation emitted during gas–metal arc welding makes it impossible to observe the weld using conventional vision systems. This necessitates an appropriate wavelength band for image acquisition using a vision camera to monitor the molten pool and metal during gas–metal arc welding. As illustrated in Figure 1, the radiation emitted from the arc plasma causes image saturation and hinders clear observation. Therefore, the strong radiation must be removed and filtered, and its characteristics must be analyzed to determine the appropriate conditions. Furthermore, the spectral radiance of the source that emits radiation and the quantum efficiency of the image sensor used for image acquisition must be considered. Radiation refers to the movement of heat when a certain material emits electromagnetic energy, even when the media are not in direct contact. It represents the conversion of the internal energy of a material into electromagnetic energy, which is also referred to as thermal radiation. During GMAW, thermal radiation enables observation of the molten pool using a passive vision sensor without an auxiliary light source. Planck’s law for black-body radiation is expressed by Equation (1):
B ν T = 2 h ν 3 c 2 1 e h ν / k T 1
where B ν T is the spectral radiance; h is the Planck constant; ν is the frequency of the electromagnetic radiation; c is the speed of light in vacuum; k is the Boltzmann constant; and T is the absolute temperature of the material. As the temperature of the material increases, the peak of the emitted spectral radiance shifts to shorter wavelengths. According to the theory of black-body radiation in thermodynamics, all materials emit electromagnetic radiation at absolute zero or higher temperatures. A black body is an ideal object that absorbs all incident electromagnetic radiation regardless of the frequency and incident angle. A black body emits electromagnetic radiation at a constant temperature, referred to as black-body radiation, and its spectrum is determined solely by the temperature rather than the shape or material. Figure 1 depicts Planck’s law for black-body radiation. Depending on temperature T , the molten pool in gas–metal arc welding emits more radiation than that emitted at the melting temperature of steel. According to Wang [24], the peak temperature of a molten pool of mild steel (Q235) is 3169 K, and its spectral radiance reaches a peak near the near-infrared (NIR) wavelength of 800 nm. According to Cai [25], the temperature of the arc plasma ranged from 7000 to 20,900 K when an Ar 90%–CO2 10% mixed gas was used as a protective gas. Therefore, the spectral radiance peak shifted from 600 nm to shorter wavelengths with increasing temperatures.
To acquire feature information from the molten pool during GMAW, the area where the arc plasma is formed and the metal melts must be captured. Therefore, the spectroscopic characteristics must be analyzed by measuring the radiation from both the arc plasma and molten pool, and the sum of all radiation emitted during GMAW must be considered. Therefore, contrary to a previous study [19], the irradiation from the arc plasma and the molten pool were simultaneously measured without dividing the arc plasma and the molten pool sections. The radiation emitted from mild-steel GMAW was measured using an Ocean Optics HR4000CG-UV-NIR spectrometer (Ocean Optics, Orlando, FL, USA). The fiber collected the emitted radiation and transmitted it to the detector in the spectrometer. A collimating lens, which covered a large free space and long distance, was used to measure the radiation from the arc plasma and molten pool without bias. To measure the radiation from both the arc plasma and molten pool, the angle between the collimating lens and welding torch was set to 45° (Figure 2).

2.3. System Configuration

2.3.1. Vision Camera

The configuration of the proposed vision system is shown in Figure 3. The system includes a vision camera, lens, ND filter, bandpass filter, and ultraviolet (UV) filter and is located at the rear of the welding direction. Because the molten pool head was covered by the gas nozzle when the welding torch and camera were parallel, the angle between the welding torch and camera was set to 45° to acquire images of the entire molten pools. As outlined in Section 2.2, a wavelength band was selected based on the spectroscopic analysis. Images suitable for weld observation were captured using a filter that only allowed the passage of the selected wavelength band. When images are captured during arc welding, the red–green–blue sensor captures images with higher sharpness and signal-to-noise ratio than a monochrome sensor [26]. The vision camera used in this experiment was the IDS UI-3271LE (IDS Imaging, Obersulm, Germany), which uses a Sony IMX265 global-shutter CMOS color sensor (Sony, Tokyo, Japan). Its quantum efficiency is illustrated in Figure 4.
Table 3 shows the specifications of the vision camera and the parameter values set in the experiment. The vision camera exhibited the highest quantum efficiency near the blue, green, and red wavelengths of 450 nm, 530 nm, and 610 nm, respectively. Because relatively good color quality is exhibited in each peak wavelength band, quantum efficiency is important for selecting imaging bands. In addition, when images are captured, parameters such as gain and gamma must be properly adjusted. The gain, a parameter that adjusts the brightness of an image, represents the overall brightness level. The image becomes brighter and darker when this value increases and decreases, respectively. The gamma ratio indicates the contrast between dark and bright regions of an image. The contrast increases with the gamma ratio, making the dark part darker and bright parts brighter. If it decreases, the level of color representation decreases owing to a reduction in contrast. The exposure time indicates the time at the image sensor is exposed to light. A long exposure time yields a bright image because a large amount of light is input to the sensor. Likewise, a short exposure time yields a dark image because less light is supplied. Additionally, if the exposure time is long, many movements are captured, whereas only fast-moving subjects are captured if the exposure time is short. The boundary between the molten pool and base metal must be clear to acquire feature information via image capture during GMAW. An increase in the gain increases the brightness of the overall image. If the value is extremely high, noise is generated in the image. Therefore, its value must be reduced. The gamma value must be set as high as possible to increase contrast and clearly demarcate the boundary between the bright and dark parts. If the exposure time is long, a large amount of light enters the sensor and saturates the image; therefore, the pixel clock value decreases depending on the camera specifications, limiting image capture at high frame rates. Therefore, an appropriate level was selected. The parameters were adjusted to improve the sharpness of the acquired images. Additionally, the system was connected to an industrial computer environment set up with Intel® Core™ i5-10500, 16 GB DDR4 RAM (Intel, Santa Clara, CA, USA), and NVIDIA GeForce RTX 3060 (NVIDIA, Santa Clara, CA, USA).

2.3.2. Lens

The focal length of the lens must account for both the distance between the weld and vision sensor and the size of the weld. Varifocal lenses with variable focal lengths can be used under diverse welding conditions. However, they are relatively large and expensive, and may not deliver the same level of performance as fixed-focus lenses. Consequently, the focal length of the lens must be selected based on the specific welding process and conditions to be applied. For the proposed vision system, preliminary studies led to the selection of f/4 as the f-number of the lens with a focal length of 16 mm.

2.3.3. Filter

As shown in Figure 5, a clear observation is impossible owing to the strong emitted radiation images acquired with conventional vision systems without an optical design during GMAW. Thus, an appropriate filter design is required to eliminate the emitted radiation. First, an imaging band suitable for image acquisition was selected by analyzing the relative intensity and characteristics of the wavelength band in the spectrum measured during GMAW. Because the spectral radiance was strong, owing to the high temperature of the arc plasma and molten pool, a wavelength band with a low relative intensity was selected from the measured spectrum. A wavelength band that allows clear observation of the molten pool with little interference from the arc plasma must be selected. In this case, a bandpass filter that allowed only the selected wavelength band to pass through and cut off other wavelength bands was selected. The full width at half maximum (FWHM) is important for bandpass filter selection.
Next, the overall transmittance of the radiation must be reduced, because the spectral radiance of the radiation emitted from the arc plasma is strong across the input wavelength band of the image sensor. The bandpass filter cuts off the transmittance using a predetermined optical density (OD) value in wavelength bands other than the selected wavelength band. This further reduces the transmittance when an ND filter is used. The ND filter reduces the amount of light that passes through (light energy) based on the OD value. When the bandpass and ND filters are utilized together in the proposed vision system, the overall OD value is selected based on the score of the image quality assessment described in Section 2.4. A multi-coating UV filter was used to protect the lens and filters and reduce image noise.

2.4. Image Quality Assessment

Image quality can be assessed by subjective and objective methods. Subjective assessment presents images to users, and their evaluation relies on subjective perceptions. This method is less reliable than the objective assessment method because it depends on user experience. Objective assessment methods are further classified into full-reference image quality assessment methods, which evaluate images with reference images, and no-reference image quality assessment (NR-IQA) methods, which evaluate images without reference images. In this context, the lack of images to be referred to (also known as the ground truth) arises because the radiation from the arc plasma prevents conventional vision sensors from capturing images. Consequently, the NR-IQA was selected. For NR-IQA, the BRISQUE technique introduced by Mittal et al. [27] was employed to assess image quality. BRISQUE analyzes the statistical characteristics of images to evaluate the image quality based on the observation that the image pixel statistics are distorted when the acquired image is distorted. This technique was trained to yield low scores for high quality and high scores for low quality. After selecting the conditions with the highest image quality assessment scores using the BRISQUE technique, the most suitable image acquisition conditions were determined using an ND filter.

3. Results and Discussion

3.1. Spectroscopic Characteristics

Figure 6 shows the relative intensity as a function of the wavelength of radiation measured during mild-steel GMAW using the proposed measurement system. The relative intensity of the peak is approximately 495 nm, which indicates that the temperature distribution of the arc plasma is consistent with Planck’s law for black-body radiation. Strong radiation was emitted in the wavelength band of approximately 250–600 nm, and the relative intensity was low in the wavelength band above 602 nm. To minimize the interference of the arc plasma, the wavelength band of 600 nm or less must be cut off, and the candidate wavelength band in Figure 6 must be selected. For image quality comparison, 420 nm and 508 nm were selected as wavelengths with high relative intensities, whereas 660, 752, 808, and 830 nm were selected as wavelengths with low relative intensities. Figure 7 illustrates the relative intensity of the radiation emitted in the selected wavelength bands for an FWHM of 10 nm. In Figure 7a,b, the relative intensity was high due to the intense radiation emitted from the arc plasma. Additionally, since the camera quantum efficiency in Figure 4 in the same wavelength range is significantly higher in blue and green, the arc plasma section of the image is saturated in Figure 8a,b. The relative intensity of the 660 nm wavelength in Figure 7c is low. Still, since the red in Figure 4 is very high, the image sensor captures radiation, and the image is also saturated. In Figure 7d,e, the relative intensity of radiation was above 0.1, and in Figure 7f, the relative intensity was the lowest. The discussion of images will continue in Section 3.2.

3.2. Image Acquisition

Molten pool images were captured using the proposed vision system during mild-steel GMAW. Images were captured using bandpass filters for wavelength bands with high and low relative intensities. The FWHM was 10 nm for all bandpass filters used in the experiment. Subsequently, when a filter was selected for the wavelength band suitable for monitoring, the conditions and ND filter that minimized the interference of the arc plasma were selected. Figure 8 shows the captured images. When bandpass filters of 420 nm and 508 nm with high relative intensities were used, radiation from the arc plasma made it impossible to observe the molten pool. When filters for wavelengths with low relative intensities were used, the molten pool was observed. At 660 nm, the relative intensity was low. However, the red color emitted from the molten pool reacted strongly because of the high quantum efficiency of the image sensor in Figure 4, causing the interference of arc plasma and image saturation.
At the NIR wavelengths of 752, 808, and 830 nm, the interference of the arc plasma was relatively low, and the molten pool was observed. Among them, the 830 nm wavelength exhibited the lowest interference. In addition, the edges of the molten pool and base metal were clear. Next, images were captured by combining the 830 nm band-pass and ND filters [OD0.5 (transmission: 32%), OD1.0 (transmission: 10%), and OD2.0 (transmission: 1%)] to reduce image noise, and Figure 9 shows the corresponding images. As the OD increases, the saturation area decreases; however, the sensitivity of the images decreases.

3.3. Image Quality Assessment

The quality of the images presented in Figure 8 was assessed using the BRISQUE technique. As shown in Table 4, the BRISQUE technique yields low and high scores for high- and low-quality images, respectively, as explained in Section 2.4. The 420, 508, and 660 nm images exhibited low quality owing to strong noise, resulting in high scores. In the 752 nm image, the molten pool was partly observable; however, its width could not be discerned because of the persistent interference of the arc plasma. Conversely, the 808 nm and 830 nm images demonstrated high quality with relatively low scores, with the 830 nm image exhibiting the highest quality. Adjustments to the digital camera settings were necessary for capturing the feature information of the molten pool within the same wavelength band. It is advisable to set the gain and gamma to minimum and maximum values of 0 and 2.2, respectively, to distinctly observe the boundary between the molten pool and base metal sections. As shown in Table 5, when the gamma value was 1.0, the molten pool appeared dark because of low contrast. At a gamma value of 2.2, the molten pool was clear, and the boundary with the base metal is evident owing to its high contrast. When the BRISQUE technique was employed, a gamma value of 2.2 resulted in a score of 56.09, indicating the highest quality.
Table 6 presents the images captured when the 830 nm bandpass filter was combined with the ND filter to minimize the noise generated by the saturation of the arc plasma. When combined with the OD0.5 ND filter, the saturation area of the arc plasma was reduced, and interference with the edge was prevented despite variations in the bead width depending on the welding conditions. The combination that included the OD1.0 ND filter decreased the overall image brightness. However, the saturation area of the arc plasma was further reduced, and the light reflected around the edge of the molten pool was reduced, enabling a clear area division. Therefore, this is the most suitable image for performing semantic segmentation in the welding quality monitoring model, which will be included in a future study. Owing to excessive sensitivity reduction, observations were impossible with the combination that included the OD2.0 ND filter. Therefore, the clearest observations were obtained when images were captured by combining the 830 nm bandpass and OD1.0 ND filters. The image obtained with the 830 nm band-pass alone served as the reference image, and the image quality for each OD value was assessed using BRISQUE. The OD1.0 ND filter combination yielded the highest quality, with a score of 15.30. Figure 10 shows that the interference from the arc plasma was minimized, and the boundary of the molten pool is clearly observed in the image acquired when an 830 nm band-pass and OD1.0 ND filters were combined. Figure 11 shows the image captured for 1 s under the most suitable imaging conditions for monitoring the molten pool during mild-steel gas–metal arc welding. Triggering devices such as field-programmable gate arrays can cause problems including time delays and inaccurate synchronization. The vision system proposed in this study captured high-quality molten pool images during DC pulse welding without dividing the peak current and base current sections.

3.4. Molten Pool Quality Monitoring

During arc welding, the weld quality is affected by various factors, including the external environment. Similar to manual welding, in which a welder wears a welding helmet and performs welding while observing the molten pool, it is important to observe the various characteristics of the molten pool using a vision system for robotic welding. In vision monitoring, molten pool images can be used to determine the weld quality in real time. Images of the molten pool were captured during mild-steel GMAW using the proposed vision system. Figure 12 shows the images for each quality state of the weld under various welding conditions. Figure 12a shows a humping bead that occurs when the welding speed is too high or when the flow of the molten pool is unstable. Figure 12b shows a magnetic arc blow in which the electromagnetic force accumulates, and the weld bead is tilted. Figure 12c depicts silicate island separation that causes defects in the appearance of the weld bead owing to the separation of silicate at the end of the molten pool. Figure 12d shows burn-through owing to an excessive welding current setting or excessive gap in the weld. Figure 12e,f shows the spatter and undercut caused by inappropriate welding conditions, such as current, voltage, and speed settings, respectively.

4. Conclusions

This study introduces a vision system designed to capture clear molten pool images online during GMAW, and a method to construct the system. In addition, digital camera settings, imaging wavelength bands, and optical designs have been proposed to precisely measure the molten pools during GMAW. The proposed vision system allows cost reduction with a straightforward configuration and satisfies the image quality requirements for vision-based online quality monitoring during GMAW. The results are summarized as follows.
  • Accurate measurement of radiation is crucial for clearly observing the molten pool during welding. In this study, the radiation was measured by aligning the collimating lens of the spectrometer, similar to the distance and angle between the weld and camera.
  • The selection of an appropriate lens in combination with a camera is essential for effective monitoring during welding. Factors such as the f-number, light throughput, and diffraction limit must be considered. The focal length was selected based on the ROI size and working distance between the image sensor and weld. In this study, a lens with an f-number of f/4 and focal length of 16 mm was proposed for molten pool monitoring during mild-steel GMAW.
  • During mild-steel GMAW, the imaging wavelength band with the least interference from the arc plasma and the most suitable for molten pool observation was 830 nm, with a BRISQUE score of 52.03.
  • To capture the information of the molten pool with less noise in the image during mild-steel GMAW, the combination of an 830 nm band-pass filter and an OD1.0 ND filter is the most suitable. It possessed a BRISQUE score of 15.3, which was the best score.
  • Image-quality assessment using BRISQUE confirmed that the image acquired with the proposed optical design method was suitable for monitoring with the highest quality, consistent with the assessment scores.
We propose a vision system for real-time weld-quality monitoring and a low-cost system for training unskilled welders. The radiation emitted from a weld varies depending on the welding process, base metal, filler metal, and shielding gas used. Therefore, further research will be conducted to design a vision system suitable for each welding process and welding condition, based on the proposed method. Furthermore, control systems will be developed to enhance welding quality by utilizing feature information, such as bead width, height, and weld failures, acquired from the images of this system.

Author Contributions

Conceptualization, G.-G.K. and J.Y.; methodology, G.-G.K. and D.-Y.K.; validation, G.-G.K. and J.-K.P.; formal analysis, G.-G.K. and D.-Y.K.; investigation, G.-G.K. and D.-Y.K.; resources, J.-K.P. and Y.-M.K.; writing—original draft preparation, G.-G.K. and J.Y.; writing—review and editing, G.-G.K. and Y.-M.K.; supervision, J.P. and J.Y. funding acquisition, J.-K.P. and J.Y. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the Material Parts Technology Development Project (20022489, Development and Substantiation of Joining Equipment & Smart Joining Lines for Manufacturing Electric Vehicle Chassis and Battery Cases with High-Strength Materials) funded by the “Ministry of Trade, Industry and Energy (MOTIE, Korea)” and “Korea Planning & Evaluation Institute of Industrial Technology (KEIT)”.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The original contributions presented in the study are included in the article, further inquiries can be directed to the corresponding authors.

Conflicts of Interest

Author Jong-Kyu Park was employed by the company Hwashin. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Correction Statement

This article has been republished with a minor correction to the Funding statement. This change does not affect the scientific content of the article.

References

  1. Ma, X.; Zhang, Y. Gas metal arc weld pool surface imaging: Modeling and processing. Weld. J. 2011, 90, 85s–94s. [Google Scholar]
  2. Kovacevic, R.; Zhang, Y.M.; Ruan, S. Sensing and Control of Weld Pool Geometry for Automated GTA Welding. J. Eng. Ind. 1995, 117, 210–222. [Google Scholar] [CrossRef]
  3. Liu, Y.; Shi, L.; Tian, X. Weld seam fitting and welding torch trajectory planning based on NURBS in intersecting curve welding. Int. J. Adv. Manuf. Technol. 2018, 95, 2457–2471. [Google Scholar] [CrossRef]
  4. Liu, J.; Fan, Z.; Olsen, S.I.; Christensen, K.H.; Kristensen, J.K. Boosting active contours for weld pool visual tracking in automatic arc welding. IEEE Trans. Autom. Sci. Eng. 2015, 14, 1096–1108. [Google Scholar] [CrossRef]
  5. Wang, Z. Monitoring of GMAW weld pool from the reflected laser lines for real-time control. IEEE Trans. Ind. Inform. 2014, 10, 2073–2083. [Google Scholar] [CrossRef]
  6. Dong, H.; Cong, M.; Zhang, Y.; Liu, Y.; Chen, H. Modeling and real-time prediction for complex welding process based on weld pool. Int. J. Adv. Manuf. Technol. 2018, 96, 2495–2508. [Google Scholar] [CrossRef]
  7. Chen, Z.; Chen, J.; Feng, Z. Monitoring Weld Pool Surface and Penetration Using Reversed Electrode Images. Weld. J. 2017. Available online: https://www.osti.gov/servlets/purl/1407780 (accessed on 21 May 2024).
  8. You, D.; Gao, X.; Katayama, S. WPD-PCA-based laser welding process monitoring and defects diagnosis by using FNN and SVM. IEEE Trans. Ind. Electron. 2014, 62, 628–636. [Google Scholar] [CrossRef]
  9. Wang, J.J.; Lin, T.; Chen, S.B. Obtaining weld pool vision information during aluminium alloy TIG welding. Int. J. Adv. Manuf. Technol. 2005, 26, 219–227. [Google Scholar] [CrossRef]
  10. Wu, J.; Chen, S. Software system designs of real-time image processing of weld pool dynamic characteristics. In Robotic Welding, Intelligence and Automation; Springer: Berlin/Heidelberg, Germany, 2007; pp. 303–309. [Google Scholar]
  11. Zhang, X.; Wang, F.; Chen, Y.; Zhang, H.; Liu, L.; Wang, Q. Weld joint penetration state sequential identification algorithm based on representation learning of weld images. J. Manuf. Process. 2024, 120, 192–204. [Google Scholar] [CrossRef]
  12. Hong, Y.; Yang, M.; Yuan, R.; Du, D.; Chang, B. A Novel Quality Monitoring Approach Based on Multigranularity Spatiotemporal Attentive Representation Learning During Climbing GTAW. IEEE Trans. Ind. Inform. 2024, 20, 8218–8228. [Google Scholar] [CrossRef]
  13. Baek, D.; Moon, H.S.; Park, S.-H. Optimization of weld penetration prediction based on weld pool image and deep learning approach in gas tungsten arc welding. Int. J. Adv. Manuf. Technol. 2024, 130, 2617–2633. [Google Scholar] [CrossRef]
  14. Hong, Y.; Yang, M.; Chang, B.; Du, D. Filter-PCA-based process monitoring and defect identification during climbing helium arc welding process using DE-SVM. IEEE Trans. Ind. Electron. 2022, 70, 7353–7362. [Google Scholar] [CrossRef]
  15. Chen, C.; Lv, N.; Chen, S. Welding penetration monitoring for pulsed GTAW using visual sensor based on AAM and random forests. J. Manuf. Process. 2021, 63, 152–162. [Google Scholar] [CrossRef]
  16. Lertrusdachakul, I.; Mathieu, A.; Aubreton, O. Vision-based control of wire extension in GMA welding. Int. J. Adv. Manuf. Technol. 2015, 78, 1201–1210. [Google Scholar] [CrossRef]
  17. Zhang, G.; Yan, Z.; Wu, L. Visual sensing of weld pool in variable polarity TIG welding of aluminium alloy. Trans. Nonferrous Met. Soc. China 2006, 16, 522–526. [Google Scholar] [CrossRef]
  18. Jin, B.; Li, H.; Wang, Q.; Gao, H. Online measurement of the GMAW process using composite sensor technology. Weld. J. 2017, 96, 133–142. [Google Scholar]
  19. Zhao, Z.; Deng, L.; Bai, L.; Zhang, Y.; Han, J. Optimal imaging band selection mechanism of weld pool vision based on spectrum analysis. Opt. Laser Technol. 2019, 110, 145–151. [Google Scholar] [CrossRef]
  20. Reisgen, U.; Purrio, M.; Buchholz, G.; Willms, K. Machine vision system for online weld pool observation of gas metal arc welding processes. Weld. World 2014, 58, 707–711. [Google Scholar] [CrossRef]
  21. Cai, Y.; Li, C.; Chen, H.; Xiong, J. Monitoring of distance between laser beam and arc in laser-arc hybrid welding based on deep learning. Opt. Laser Technol. 2024, 174, 110562. [Google Scholar] [CrossRef]
  22. ISO 630:1995; Structural Steels—Plates, Wide Flats, Bars, Sections and Profiles. ISO: Geneva, Switzerland, 1995.
  23. ISO 636:2017; Welding Consumables—Rods, Wires and Deposits for tungsten Inert Gas Welding of Non-Alloy and Fine-Grain Steels—Classification. ISO: Geneva, Switzerland, 2017.
  24. Wang, L.; Chen, J.; Zhang, S.; Wu, C. Numerical simulation of coupled arc-droplet-weld pool behaviors during compound magnetic field assisted gas metal arc welding. AIP Adv. 2021, 11, 065221. [Google Scholar] [CrossRef]
  25. Cai, X.; Dong, B.; Lin, S.; Murphy, A.B.; Fan, C.; Yang, C. Heat source characteristics of ternary-gas-shielded tandem narrow-gap GMAW. Materials 2019, 12, 1397. [Google Scholar] [CrossRef] [PubMed]
  26. Zhao, Z.; Sun, B.; Zhang, Y.; Bai, L.; Han, J. Weld pool image acquisition and contour extraction based on arc spectrum and camera quantum efficiency. Optik 2020, 202, 163719. [Google Scholar] [CrossRef]
  27. Mittal, A.; Moorthy, A.K.; Bovik, A.C. No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. 2012, 21, 4695–4708. [Google Scholar] [CrossRef] [PubMed]
Figure 1. Black-body radiation spectrum according to Planck’s law.
Figure 1. Black-body radiation spectrum according to Planck’s law.
Applsci 14 05998 g001
Figure 2. Schematic of the spectrometer system configuration for measuring the radiation emitted during GMAW.
Figure 2. Schematic of the spectrometer system configuration for measuring the radiation emitted during GMAW.
Applsci 14 05998 g002
Figure 3. Vision system configuration during GMAW: (a) schematic; (b) experimental system.
Figure 3. Vision system configuration during GMAW: (a) schematic; (b) experimental system.
Applsci 14 05998 g003
Figure 4. Quantum efficiency of the image sensor used in experiment as a function of the wavelength.
Figure 4. Quantum efficiency of the image sensor used in experiment as a function of the wavelength.
Applsci 14 05998 g004
Figure 5. Image of GMAW without imaging-band design.
Figure 5. Image of GMAW without imaging-band design.
Applsci 14 05998 g005
Figure 6. Radiation from arc plasma and molten pool with arc during GMAW.
Figure 6. Radiation from arc plasma and molten pool with arc during GMAW.
Applsci 14 05998 g006
Figure 7. Relative intensity of radiation emitted from mild-steel GMAW for each wavelength: (a) 420 nm; (b) 508 nm; (c) 660 nm; (d) 752 nm; (e) 808 nm; (f) 830 nm.
Figure 7. Relative intensity of radiation emitted from mild-steel GMAW for each wavelength: (a) 420 nm; (b) 508 nm; (c) 660 nm; (d) 752 nm; (e) 808 nm; (f) 830 nm.
Applsci 14 05998 g007
Figure 8. Images of the molten pool obtained during mild-steel GMAW using band-pass filters: (a) 420 nm; (b) 508 nm; (c) 660 nm; (d) 752 nm; (e) 808 nm; (f) 830 nm.
Figure 8. Images of the molten pool obtained during mild-steel GMAW using band-pass filters: (a) 420 nm; (b) 508 nm; (c) 660 nm; (d) 752 nm; (e) 808 nm; (f) 830 nm.
Applsci 14 05998 g008
Figure 9. Molten pool image obtained by combining an 830 nm band-pass filter and ND filters during mild-steel GMAW: (a) 830 nm band-pass filter + OD0.5 ND filter; (b) 830 nm band-pass filter + OD1.0 ND filter; (c) 830 nm band-pass filter + OD2.0 ND filter.
Figure 9. Molten pool image obtained by combining an 830 nm band-pass filter and ND filters during mild-steel GMAW: (a) 830 nm band-pass filter + OD0.5 ND filter; (b) 830 nm band-pass filter + OD1.0 ND filter; (c) 830 nm band-pass filter + OD2.0 ND filter.
Applsci 14 05998 g009
Figure 10. Information configuration of image obtained by combining 830 nm band-pass filter and OD1.0 ND filter during mild-steel GMAW.
Figure 10. Information configuration of image obtained by combining 830 nm band-pass filter and OD1.0 ND filter during mild-steel GMAW.
Applsci 14 05998 g010
Figure 11. Molten pool images of mild-steel GMAW obtained for 1 s; frame rate: 30 frames per second.
Figure 11. Molten pool images of mild-steel GMAW obtained for 1 s; frame rate: 30 frames per second.
Applsci 14 05998 g011
Figure 12. Weld quality status observed in the molten pool image in mild-steel GMAW: (a) humping; (b) magnetic arc blow; (c) silicate island; (d) burn-through; (e) spatter; (f) undercut.
Figure 12. Weld quality status observed in the molten pool image in mild-steel GMAW: (a) humping; (b) magnetic arc blow; (c) silicate island; (d) burn-through; (e) spatter; (f) undercut.
Applsci 14 05998 g012
Table 1. Chemical composition of the base metal ISO 630 E355 (wt.%).
Table 1. Chemical composition of the base metal ISO 630 E355 (wt.%).
CSiMnPS
0.18520.3431.3550.0110.0044
Table 2. Chemical composition of the filler wire ISO 636-A W 42 5 2Si (wt.%).
Table 2. Chemical composition of the filler wire ISO 636-A W 42 5 2Si (wt.%).
CSiMnPS
0.070.651.140.0150.01
Table 3. Vision camera specifications and parameter settings in experiments.
Table 3. Vision camera specifications and parameter settings in experiments.
SpecificationSetting Value
Resolution (h × v)2056 × 1542600 × 450
Aspect ratio (h:v)adjustable4:3
Exposure time0.024–1000 ms15 ms
Gamma0–2.22.2
Gain0–100100
Frame rateadjustable30 fps
Table 4. Image quality assessment for each wavelength band image with BRISQUE.
Table 4. Image quality assessment for each wavelength band image with BRISQUE.
Wavelength (nm)420508660752808830
Score72.1270.6361.7055.8954.1852.03
Table 5. Image quality assessment for each gamma value with BRISQUE.
Table 5. Image quality assessment for each gamma value with BRISQUE.
Gamma Value1.01.11.52.02.2
ImageApplsci 14 05998 i001Applsci 14 05998 i002Applsci 14 05998 i003Applsci 14 05998 i004Applsci 14 05998 i005
Score76.1272.3364.8960.0256.09
Table 6. BRISQUE image quality score of combining the 830 nm band-pass filter with each ND filter.
Table 6. BRISQUE image quality score of combining the 830 nm band-pass filter with each ND filter.
Optical Density0.5 (transmission: 32%)1.0 (transmission: 10%)2.0 (transmission: 1%)
ImageApplsci 14 05998 i006Applsci 14 05998 i007Applsci 14 05998 i008
Score42.8515.3034.51
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Kim, G.-G.; Kim, Y.-M.; Kim, D.-Y.; Park, J.-K.; Park, J.; Yu, J. Vision-Based Online Molten Pool Image Acquisition and Assessment for Quality Monitoring in Gas–Metal Arc Welding. Appl. Sci. 2024, 14, 5998. https://doi.org/10.3390/app14145998

AMA Style

Kim G-G, Kim Y-M, Kim D-Y, Park J-K, Park J, Yu J. Vision-Based Online Molten Pool Image Acquisition and Assessment for Quality Monitoring in Gas–Metal Arc Welding. Applied Sciences. 2024; 14(14):5998. https://doi.org/10.3390/app14145998

Chicago/Turabian Style

Kim, Gwang-Gook, Young-Min Kim, Dong-Yoon Kim, Jong-Kyu Park, Junhong Park, and Jiyoung Yu. 2024. "Vision-Based Online Molten Pool Image Acquisition and Assessment for Quality Monitoring in Gas–Metal Arc Welding" Applied Sciences 14, no. 14: 5998. https://doi.org/10.3390/app14145998

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop