Intelligent Identification of Liquid Aluminum Leakage in Deep Well Casting Production Based on Image Segmentation

Abstract

This study proposes a method based on image segmentation for accurately identifying liquid aluminum leakage during deep well casting, which is crucial for providing early warnings and preventing potential explosions in aluminum processing. Traditional DeepLabV3+ models in this domain encounter challenges such as prolonged training duration, the requirement for abundant data, and insufficient understanding of the liquid surface characteristics of casting molds. This work presents an enhanced DeepLabV3+ method to address the restrictions and increase the accuracy of calculating liquid surface areas for casting molds. This algorithm substitutes the initial feature extraction network with ResNet-50 and integrates the CBAM attention mechanism and transfer learning techniques. The results of ablation experiments and comparative trials demonstrate that the proposed algorithm can achieve favorable segmentation performance, delivering an MIoU of 91.88%, an MPA of 96.53%, and an inference speed of 55.05 FPS. Furthermore, this study presents a technique utilizing OpenCV to accurately measure variations in the surface areas of casting molds when there are leakages of liquid aluminum. In addition, this work introduces a measurement to quantify these alterations and establish an abnormal threshold by utilizing the Interquartile Range (IQR) method. Empirical tests confirm that the threshold established in this study can accurately detect instances of liquid aluminum leakage.

1. Introduction

Aluminum and its alloys are widely utilized metals, ranking second only to steel, owing to its exceptional ductility, robust resistance to corrosion, low density, impressive strength-to-weight ratio, and ability to be recycled. Their applications encompass a wide range of industries, such as aerospace, construction, automotive, and electronics [1,2,3,4]. The smelting and casting process is the first stage in preparing and processing aluminum alloy ingredients. Deep well casting, a type of Direct Chill (DC) casting, is widely employed by the majority of aluminum processing companies in China due to its cost-effectiveness and convenient maintenance [5]. The single-furnace casting process is illustrated in Figure 1. As depicted in the diagram, molten aluminum is transported from the smelting furnace through a channel called a launder to different crystallizers located on the casting mold at high temperatures. A cooling water pump transfers cooling water from the high-level water tank to the cooling water trough of the casting mold with a certain pressure. This allows for heat exchange with the high-temperature aluminum in the crystallizers, resulting in the cooling and solidification of the aluminum into ingots. Concurrently, the dummy plate, propelled by the casting machine, consistently extracts the shaped aluminum ingots from the crystallizers at a constant pace, accomplishing DC casting.

When high-temperature aluminum and water come into contact, the substantial temperature disparity causes fast heat transfer, evading a substantial amount of water. The continuous or substantial release of liquid aluminum can result in energy buildup caused by the high-temperature and high-pressure water vapor produced by the interaction of the two-phase fluids. This can lead to a forceful release of steam, which presents a significant danger to the well-being of individuals, the stability of structures, and the ecological surroundings [6]. In the past five years [7], accidents in China related to aluminum processing (deep well casting) have primarily included deep well explosions, electric shocks, scalding, and other incidents. Out of all the incidents, most (53.8%) are caused by deep well explosions due to liquid aluminum leaking. These accidents have resulted in the most severe casualties and property damage.

Scholars have primarily concentrated on investigating the processes [8,9] and conducting risk assessments [10,11] of aluminum liquid–water contact explosions to study liquid aluminum leaking. The technology for online monitoring of liquid aluminum leakage is currently at its early stage, predominantly dependent on manual experiential judgment and lacking a theoretical foundation. Due to the progress in computer vision and video monitoring technologies, many academics have conducted research in several disciplines. Zhang et al. [12] pioneered using infrared thermal imagers to detect liquid aluminum leakage. They introduced a monitoring algorithm that combines infrared image features to effectively detect high-temperature liquid aluminum leakage. Nevertheless, this approach relied on simulated experiments and did not produce a dataset derived from real-world production, hence lacking practical reference value. In addition, it is important to note that in real casting operations, a single manufacturing batch can contain hundreds of ingots. This large number of ingots can create challenges for thermal imaging cameras, as they may not be able to effectively monitor and detect any potential leaks of liquid aluminum, resulting in incomplete detection.

Presently, in the realm of aluminum processing (deep well casting), the identification of liquid aluminum leakage predominantly depends on physical examination. Production personnel inspect the casting mold and utilize their expertise to ascertain if any leakage has transpired. This approach requires a significant amount of focus and proficiency from the staff. When liquid aluminum leaks, the casting mold’s liquid level decreases significantly, leading to a major loss in the surface area of the liquid in the mold. By identifying this particular attribute, timely warning signals can be generated to encourage operators to take immediate action, thus averting ongoing leakage of liquid aluminum. However, real-time monitoring of liquid surfaces in casting molds remains challenging, requiring innovative solutions. Machine vision technology has rapidly advanced, offering technological assistance in estimating the surface area of the casting mold’s liquid. One of the breakthroughs is the segmentation of the liquid surface in casting molds, which is an important step in determining the area of the liquid surface in the casting mold.

Currently, image segmentation algorithms can be classified into two main categories: classical and deep-learning-based semantic segmentation algorithms. Conventional techniques utilize many methods to extract region information and show strong segmentation performance. However, they suffer from issues such as the requirement for domain expertise, weak generalization ability, susceptibility to background–feature mixture, low computational efficiency, and significant sensitivity to noise [13]. Conversely, deep learning approaches utilize extensive data and strong computational powers to autonomously acquire knowledge of features and patterns from the data, resulting in a more accurate representation of the semantic information contained in images [14]. Many segmentation methods are based on deep learning, with the mainstream architecture being the encoder–decoder structure. Common algorithms include FCN [15], UNet [16], and PSPNet [17], among others. Wu et al. [18] proposed a fracture segmentation method based on FCN, using synthetic seismic image data to generate the training set for seismic monitoring. Almalki et al. [19] proposed a residual UNet model based on a denoising encoder for tooth segmentation using X-ray images to identify damage to individual teeth. Wang et al. [20] suggested an improved coal gangue image segmentation network using a modified PSPNet. Their approach facilitates rapid identification of coal gangue even under conditions of adhesion and partial occlusion, achieved by replacing the backbone network and incorporating attention mechanisms. Moreover, the DeepLab series [21,22,23,24] of semantic segmentation models, introduced by Google, have gained significant recognition for their capacity to maintain intricate information and achieve exceptional segmentation results. DeepLabV3+, the latest version of this series, has been extensively utilized in several fields [25,26,27,28,29] due to its exceptional performance. Nevertheless, conventional DeepLabV3+ methods are plagued by limitations such as substantial model parameter sizes, protracted training durations, and demanding annotated data prerequisites. Consequently, they fail to fulfill the real-time and precision needs of industrial inspection [30].

Based on the previous analyses, this work introduces a two-stage casting mold liquid surface area calculation method that integrates semantic segmentation with OpenCV. This technique is utilized to identify liquid aluminum leakage.

The main contributions are as follows:

• This project specifically targets an aluminum facility in South China to address the challenges of limited datasets of casting mold liquid surface images and the expensive nature of manual annotation. A collection of casting mold photos from different time periods and casting units is gathered. Several data preparation activities are performed to create the casting mold liquid surface image dataset, such as region extraction, image annotation, and data augmentation.
• The method aims to overcome the challenges of lengthy training time, high data demands, and inadequate learning of casting mold liquid surface features in traditional DeepLabV3+ models. To enhance the model, this study replaces the original feature extraction network with ResNet-50 and integrates CBAM attention mechanisms and transfer learning methods.
• In order to address the intricacy of identifying comprehensive liquid aluminum leakage, a solution is suggested that involves calculating the surface area of the casting mold liquid using OpenCV. In addition, it establishes a measurement for variations in the liquid surface area of the casting mold and calculates an abnormal threshold using the IQR approach. Ultimately, the strategy is verified by employing production data.

The organization of this paper is structured as follows: Section 2 presents the pertinent theories and the proposed framework. Section 3 presents the dataset, examines the outcomes of image segmentation, calculates the anomaly threshold, and validates it. Section 4 provides a summary of the findings, discusses the constraints of the study, and suggests potential areas for future research.

2. Methods

2.1. Overall Algorithm Design

The flowchart depicting the proposed image-segmentation-based method for identifying liquid aluminum leakage is provided in Figure 2. The process is divided into two main parts:

Part 1: Offline modeling of casting mold liquid surface image segmentation:

a. Historical image acquisition: To augment the variety of data samples, movies documenting the casting process in diverse settings were recorded at different intervals. Subsequently, these recordings were divided into individual frames in order to acquire the initial collection of images.

b. Establishment of casting mold liquid surface dataset: In light of the lack of study in this field, creating a thorough dataset on the liquid surface of casting molds was crucial. The construction procedure encompassed the extraction of regions, denoising of images, augmentation of data, and annotation of images.

c. Enhancement of the DeepLabV3+ casting mold liquid surface segmentation model: To overcome issues such as long training times, large data requirements, and insufficient learning of casting mold liquid surface features in typical DeepLabV3+ models, the model was enhanced by replacing the original feature extraction network with the ResNet-50 network. In addition, the CBAM attention mechanism and transfer learning approaches were introduced.

Part 2: Online identification of liquid aluminum leakage:

A. Online image acquisition: The system triggers the start casting signal; then, it uses predetermined parameters to identify the camera detection region and capture real-time RGB photos.

B. The upgraded DeepLabV3+ model is used to segment the images of the casting mold liquid surface.

C. Online identification of liquid aluminum leakage: The proposed technique, utilizing OpenCV, is employed to compute the surface area of the casting mold liquid in pixels. The proposed method in this work calculates the change in the indicator of the liquid surface area of the casting mold to determine if there is any leakage of liquid aluminum. This is performed by comparing it to a threshold value.

2.2. Pre-Processing

In order to prioritize the safety of production, it is imperative that the acquisition equipment is not positioned in close proximity to the casting mold. This constraint gives rise to difficulties such as the presence of huge picture dimensions and elevated levels of noise in the photographs of the liquid surface of the original casting mold. Therefore, image processing is required, which includes extracting regions, reducing picture noise, and augmenting data, as shown in Figure 3.

(1)

Region Extraction

While capturing the video, unintended areas outside of the casting mold are also recorded, which leads to the inclusion of extra regions in the images. These additional regions can cause interference in the detection process and substantially impact the following algorithms. In order to mitigate this interference, it is essential to isolate the specific region of the casting mold liquid surface when segmenting images of the casting mold liquid surface. The process of region extraction for the casting mold liquid surface is depicted in Figure 3a. Because of the installation position constraints, the monitoring scene displays the casting mold liquid surface region as a parallelogram instead of a rectangle, which facilitates easier extraction. By utilizing the fixed camera location, these coordinates are used to extract the region of the liquid surface of the casting mold from images of the same casting ingot specification in batches. This process establishes the first dataset. The blank areas are assigned a pixel value of 0, with corresponding labels designated as “_background_” and excluded from training.

(2)

Gaussian Filtering

The presence of considerable noise in the casting mold liquid surface images, caused by limits in the clarity of surveillance video, can have a negative impact on the accuracy of casting mold liquid surface segmentation. Gaussian filtering is applied to the dataset after region extraction to mitigate this noise and enhance image quality, as depicted in Figure 3b. Gaussian filtering efficiently attenuates noise in the image following a normal distribution, enhancing image quality while maintaining the overall image characteristics. The essence of this method lies in the utilization of the Gaussian function, represented by the following formula:

$$G(x,y)=\frac{1}{2\pi {\sigma }^2}{e}^{-\frac{x^2+y^2}{2{\sigma }^2}}$$

(1)

In the equation, (x, y) denotes the coordinates of pixels within the window and σ represents the standard deviation. By selecting an appropriate σ value, the specific Gaussian function can be determined.

In Gaussian filtering, the value of the Gaussian function determines the pixel weight relative to the central pixel, with pixels farther from the center having lower weights. This strategy is utilized due to the higher probability of neighboring pixels in the image belonging to the same structure or texture, whereas pixels that are farther apart are more likely to belong to distinct structures or noise.

(3)

Data Augmentation

Because manually annotated semantic segmentation label images are scarce in this experiment, it is necessary to augment images in order to expand the diversity of data, improve the generalization of models, and reduce overfitting. Typical picture augmentation techniques include geometric changes, color alterations, and pixel transformations. This study used many augmentation techniques from the “albumentations” library [31]—including horizontal flipping, random cropping, random shifting and scaling, saturation transformation, RGB shifting, and motion blur—to enhance the dataset after Gaussian filtering, as depicted in Figure 3c.

2.3. Improved DeepLabV3+ Casting Mold Liquid Surface Segmentation Model

2.3.1. Improved DeepLabV3+ Model Structure

This research utilizes the DeepLabV3+ network model to segment the liquid surface of a casting mold. Nevertheless, the intricate characteristics of the liquid surface in casting molds make the conventional DeepLabV3+ algorithm inappropriate for segmentation tasks in actual casting procedures. Extended utilization of casting molds gradually causes the creation of aluminum dross in the liquid, leading to several small, spot-like characteristics in the captured photos. Over time and under different lighting conditions, these characteristics experience substantial alterations, and comparable characteristics are also present on the molds. Therefore, an improved DeepLabV3+ method for casting mold liquid surface segmentation is proposed to address these challenges. The overall architecture of this network is illustrated in Figure 4. The enhanced DeepLabV3+ maintains an encoder–decoder architecture. During the “encoder” phase, ResNet-50 takes the role of Xception as the network used for extracting features, resulting in the acquisition of both shallow and deep feature maps. Before additional processing, the CBAM attention mechanism enhances the extracted surface characteristics of the casting mold liquid from the primary network, hence increasing accuracy by mitigating recognition errors caused by comparable features on both the mold and liquid surface. The shallow feature map is directly sent to the decoder, whereas the deep feature map acquires information at various scales using the ASPP module to improve the recognition of features at multiple scales. The process involves combining numerous feature maps and then applying a 1 × 1 convolution. The resulting output is then fed into the decoder. The decoder takes the deep feature map generated by the ASPP module and increases its size by a factor of 4. It then combines this upsampled feature map with the shallow feature map, which has been processed through a 1 × 1 convolution and a 3 × 3 convolution. Finally, the combined feature map is upsampled by a factor of 4 again to generate a prediction mask image that matches the size of the input image.

2.3.2. ResNet-50 Backbone Network

This paper employs the ResNet network [32] as the feature extraction network for DeepLabV3+. The ResNet network addresses issues such as gradient vanishing and explosion by introducing residual modules, as illustrated in Figure 5. In conventional network architectures, data must be transmitted sequentially through several layers, such as convolutional layers and pooling layers. This process often results in the loss of gradient information in deep networks, posing a significant challenge during training. Nevertheless, the residual structure enables the network to learn residuals directly, which are the discrepancies between the desired output and the current representation. This facilitates the training of the network with extremely deep layers. The ResNet series comprises many network models with varying layer counts, namely, ResNet-18, ResNet-34, ResNet-50, ResNet-101, and ResNet-152. In order to optimize computational resources, the new model has chosen ResNet-50 as the backbone feature extraction network. ResNet-50 is a well-balanced choice that effectively balances parameter count and network depth.

2.3.3. CBAM Attention Mechanism

The DeeplabV3+ model utilizes the ASPP module to obtain information at several scales. Simply combining these features does not adequately capture the full contextual information. Thus, in the encoder architecture, instead of directly inputting the deep feature map into the ASPP module as in the original model, this study incorporates CBAM to handle the deep feature map. This stage aims to retain important feature maps while discarding less influential ones before feeding them into the ASPP module. Furthermore, in the decoder stage, prior to performing 1 × 1 convolution operations on the shallow feature maps, the CBAM attention module is incorporated to enhance the acquisition of significant characteristics on the surface of the casting mold liquid.

CBAM [33] is an attention mechanism utilized to boost the performance of convolutional neural networks (CNNs). The structure, depicted in Figure 6, comprises two distinct sub-modules: the spatial and channel attention modules. The spatial attention module emphasizes spatial information, while the channel attention module emphasizes channel information. The CBAM module offers the benefits of both reducing processing power and minimizing the number of parameters. Additionally, it can be easily integrated into existing CNN networks as a plug-and-play module.

2.4. Casting Mold Liquid Surface Area Change Threshold Calculation Method

2.4.1. Casting Mold Liquid Surface Area Calculation Based on OpenCV

The study uses the OpenCV library, with support from the numpy library, to compute the aggregate pixel value of the liquid surface, which corresponds to the extent of the casting mold’s liquid surface. The precise procedures of the dynamic algorithm for calculating the surface area of the casting mold liquid are detailed in Algorithm 1.

Algorithm 1. OpenCV implements casting mold liquid surface area calculation steps.

Input: Capture the surveillance video image at the current time T at a frame rate of 1 frame/s;

Output: Image casting mold liquid surface area ST at current time T;

Process:

(1) Pass the surveillance video liquid surface image at the current time T into the configured DeepLabV3+ network for prediction: pr = self.net(images)[0];

(2) Since the input model training and prediction image size is [512, 512], adjust the predicted image to the original size: R pr = cv2.resize(pr, (original_w, original_h), interpolation = cv2.INTER_LINEAR);

(3) Obtain the type of each pixel: pr = pr.argmax(axis = −1); (4) Open the text file named current_area and write the real-time casting mold liquid surface area in appended format:  With open (file_path, ‘a’) as file:   file.write(f’{str(np.sum(pr = = 1))}/n’);

The algorithm outlined above enables the computation of the real-time casting mold liquid surface area, denoted as S_T. This computation serves as the data foundation for the method of identifying liquid aluminum leakage discussed in subsequent sections. Visual examples of this calculation method are illustrated in Figure 7a,b.

2.4.2. Casting Mold Liquid Surface Area Change Threshold

This paper uses the absolute value of the change in liquid surface area over a certain time window, |ΔS|, as the judgment criterion for liquid aluminum leakage. The time window is determined based on data analysis and calculations.

The IQR (Interquartile Range) method is employed to determine the abnormal threshold for changes in liquid surface area; the calculation is given by

$$\{\begin{array}{l}IQR=|Q_1-Q_3|\\ LB=Q_3+1.5IQR\end{array}$$

(2)

The equation employs Q₁ and Q₃ to represent the lower and upper quartiles; IQR as the interquartile range; and LB as the lower bound for anomalies, which signifies the abnormal threshold for variations in the liquid surface area of the casting mold. Due to the necessity of calculating the absolute value of changes in liquid surface area, only the upper limit of the IQR is utilized.

2.5. Experimental Design

2.5.1. Experimental Environment and Training Parameter

The experimental platform used in this study comprises Windows 11 as the operating system, an AMD Ryzen 7 5800X 8-core Processor (AMD (Advanced Micro Devices), Santa Clara, CA, USA) as the CPU, and an NVIDIA GeForce RTX 3090 (NVIDIA, Santa Clara, CA, USA) as the GPU. PyTorch 1.10.0 serves as the deep learning framework, complemented by CUDA version 11.3 and Python version 3.8.

In the experiment of casting mold liquid surface segmentation, to maintain fairness, all model parameters are identical, as outlined in

Table 1. Casting mold liquid surface segmentation model parameter settings.

Parameters	Settings
batch_size	4
Learning rate	7 × 10−3
Epoch	80
Optimizer	SGD (Stochastic Gradient Descent)
Momentum	0.9
Loss function	CE Loss
Input shape	[512, 512]

. The Cross-Entropy Loss (CE Loss) [34] function is employed as the loss function in this study. This loss function converges quickly, and its magnitude reflects the difference between the probability distributions predicted by the model and the true label distributions. A lower loss value suggests a lower mistake rate of the model, making it well-suited for semantic segmentation tasks. Furthermore, because the feature extraction performance is not adequate when training the model from the beginning, all models utilize pre-trained main feature networks from the ImageNet dataset. The loss iteration graph in Figure 8 indicates model convergence at approximately 60 iterations, with the number of epochs set to 80 to ensure the precision and reliability of evaluation metrics.

2.5.2. Evaluation Index

The evaluation criteria in this paper are divided into two categories, which are used to assess the performance of casting mold liquid surface segmentation and aluminum leakage identification, respectively, based on the threshold of casting mold liquid surface area change.

(1) Mean Intersection over Union (MIoU), Liquid surface Intersection over Union (Liquid-IoU), and Mean Pixel Accuracy (MPA) are employed as evaluation metrics for evaluating the performance of models in casting mold liquid surface segmentation tasks. Moreover, the inference speed was measured in frames per second (FPS). FPS represents the number of 512 × 512 images processed per second, where a higher FPS indicates a faster model inference speed.

MIoU calculates the ratio of the intersection to the union of predicted and ground truth results for each class, which is then summed and averaged. This experiment presents two classes: background (0) and liquid surface (1).

$$MIoU=\frac{1}{n+1}{\displaystyle \sum _{i=0}^k\frac{p_{ii}}{{\displaystyle \sum _{j=0}^k{p}_{ij}+{\displaystyle \sum _{j=0}^k{p}_{ji}-p_{ii}}}}},i=0,1,j=0,1,n=1$$

(3)

In the equation, there are n + 1 classes, where p_ii represents the number of pixels correctly predicted as class i, p_ij represents the total number of pixels predicted as class i, and p_ji represents the total number of pixels predicted as class j.

Liquid-IoU refers to the ratio of the intersection to the union of the predicted and ground truth results for the liquid surface class.

MPA calculates the ratio of accurately predicted pixels in each class compared to the total number of pixels and subsequently calculates the average of these proportions. The specific calculation is shown as follows:

$$MPA=\frac{1}{n+1}{\displaystyle \sum _{i=0}^k\frac{p_{ii}}{{\displaystyle \sum _{j=0}^k{p}_{ij}}}},i=0,1,j=0,1,n=1$$

(4)

(2) The evaluation criteria of Aluminum leakage identification include Accuracy, false alarm rate (FAR), and missing alarm ratio (MAR), calculated using Equations (5)–(7), respectively.

$$Accuracy=\frac{(TP+TN)}{TP+TN+FP+FN}$$

(5)

$$FAR=\frac{FP}{FP+TN}$$

(6)

$$MAR=\frac{FN}{FN+TN}$$

(7)

where TP (True Positive) means that the change in liquid surface area does not exceed the threshold and liquid aluminum leakage does not occur; FN (False Negative) means that the change in liquid surface area does not exceed the threshold but liquid aluminum leakage occurs; FP (False Positive) means that the change in liquid surface area exceeds the threshold but liquid aluminum leakage does not occur; and TN (True Negative) means that the change in liquid surface area exceeds the threshold and liquid aluminum leakage occurs.

3. Case Study

3.1. Study Object and Data Collection

The study focused on the foundry workshop of an aluminum processing plant located in South China. The workshop is located on the eastern side of the plant and has a somewhat open construction. At present, there are three casting units in operation: Units No.2, No.3, and No.4, as shown in Figure 9a. The collection was collected from archival video footage recorded by AI cameras positioned overhead the casting molds of different components. Figure 9b,c depict the precise locations where the cameras and monitoring screens are installed, respectively. Data collection was conducted between March 2023 and August 2023, encompassing routine casting operations as well as occurrences of anomalies specifically connected to the loss of liquid aluminum.

Table 2. Partial dataset information.

Dataset Serial Number	Casting Date	Recording Duration	Whether Liquid Aluminum Leakage Occurs
1	11 March 2023	1 min 20 s	Yes
2	16 March 2023	50 s	No
3	21 March 2023	50 s	Yes
4	7 April 2023	56 s	Yes
5	13 April 2023	48 s	Yes
6	15 April 2023	2 min 10 s	Yes
7	18 April 2023	53 s	No
8	20 May 2023	1 min 43 s	Yes
9	28 May 2023	2 min 15 s	Yes
10	5 June 2023	58 s	No
11	20 July 2023	1 min 12 s	No

contains a selection of video data samples. The captured films were divided into individual frames at a frequency of one frame per second to create the first dataset.

3.2. Casting Mold Liquid Surface Segmentation Experiment and Result Analysis

3.2.1. Casting Mold Liquid Surface Segmentation Dataset Description

A total of 150 images, which captured the liquid surfaces of casting molds during regular casting operations, were selected from the dataset outlined in Section 3.1. These images were taken between March and April 2023. The selection criteria prioritized clear images that depicted numerous scenarios, including diverse times, lighting conditions, liquid levels, casting units, and aluminum ingot parameters. After performing region extraction, picture denoising, data augmentation, and annotation using the Labelme program, a total of 1200 images were obtained. Afterward, the photographs were divided into training and validation sets in a random manner, with a ratio of 8:2. This resulted in 960 images for training and 240 images for validation.

3.2.2. Training Strategy

The training strategy proposed in this paper for segmenting casting mold liquid surfaces using the improved DeepLabV3+ primarily involves two steps:

• Model pre-training: This paper adopts a transfer learning strategy for feature extraction, as illustrated in Figure 10. The model undergoes pretraining on the VOC dataset to create pretraining files in .pth format, after which the pre-trained weights are loaded into the model.
• Model training: The imported pre-trained model weights are used as input for retraining with the casting mold liquid surface dataset. The training process continues until the model reaches convergence.

3.2.3. Ablation Experiment

In order to evaluate the efficacy of the proposed enhanced DeepLabV3+ technique for segmenting the liquid surface in casting molds, this experiment conducted ablation investigations that specifically examined feature extraction networks, attention mechanisms, and transfer learning. The segmentation performance was tested using the original backbone networks Xception, ResNet50, and lightweight network MobilenetV2 to demonstrate the balance between model accuracy and inference time. Figure 11 displays the detection results using different feature networks as backbone networks. Based on the figure, it is clear that utilizing MobilenetV2 as the underlying network results in the highest inference speed (70.77 FPS) but the lowest MIoU (87.88%). On the other hand, when ResNet50 is used as the main network, DeepLabV3+ achieves the best level of MIoU (90.4%), with an inference speed (54.84 FPS) second only to MobileNetV2. ResNet50 is a suitable choice for the backbone feature network because of its ability to achieve high segmentation accuracy and meet real-time needs, resulting in overall improved model performance.

Additionally,

Table 3. Model ablation experimental results.

Network Models	MIoU/%	Liquid-IoU/%	MPA/%
DeeplabV3+ (ResNet50)	90.4	84.7	95.85
DeeplabV3+ (ResNet-50+CBAM)	91.0	85.55	96
DeeplabV3+ (ResNet-50+CBAM+transfer learning)	91.88	86.97	96.53

reveals that the introduction of the CBAM attention module resulted in a 0.6 percentage point increase in MIoU and a 0.85 percentage point increase in Liquid-IoU. Using the CBAM module improves the model’s capacity to acquire complex details of the liquid surface.

Furthermore, the effectiveness of transfer learning in enhancing the performance of the improved DeepLabV3+ algorithm was validated. As shown in Table 3, the results of the ablation experiments show that training with transfer learning led to a 2.27 percentage point increase in Liquid-IoU and a 1.42 percentage point increase in MIoU. This demonstrates that transfer learning effectively improves the model’s prediction accuracy.

3.2.4. Comparative Experiment

In order to evaluate and examine the effectiveness of the improved DeepLabV3+ model in segmenting the liquid surface of casting molds, this study conducted a comparative analysis involving the DeepLabV3+ model, UNet model, and PSPNet model. The comparative results are shown in

Table 4. Compare experimental results.

Network Models	Accuracy/%			Speed/FPS
	MIoU	MPA	Liquid-IoU
DeeplabV3+	88.48	95.71	81.73	37.84
UNet	90.37	94.81	84.76	24.64
PSPNet	84.96	92.11	76.03	34.66
Ours	91.88	96.53	86.97	55.05

.

The table clearly shows that the upgraded DeepLabV3+ model, as described in this research, outperforms other models in the job of segmenting liquid surfaces. It achieves an MIoU of 91.88%, an MPA of 96.53%, and a Liquid-IoU of 86.97%. The speed indicator surpasses that of other versions, reaching 55.05 FPS. The original DeepLabV3+ method exhibits the second-highest inference speed (37.84 FPS) among all models mentioned in this study. However, its accuracy is the lowest, suggesting that its processing efficiency comes at the expense of accuracy. The UNet network model demonstrates a commendable segmentation accuracy, ranking second with an MIoU of 90.37%. However, it falls behind in terms of speed (24.64 FPS), highlighting the necessity for enhanced processing performance. The PSPNet model has a relatively fast inference speed (34.66 FPS), but its segmentation accuracy is the lowest, measuring only 76.03% at a Liquid-IoU and 84.96% at an MIoU. In summary, the improved DeepLabV3+ method presented in this research successfully fulfills the need for accurate and real-time liquid surface segmentation, demonstrating strong overall performance.

In order to evaluate and compare the effectiveness of the upgraded DeepLabV3+ network against other networks, this study specifically chose casting mold liquid surface photos from the validation set. These images were picked to have different liquid levels, lighting situations, casting units, and aluminum ingot parameters. The segmentation results of different models are shown in Figure 12, where each row depicts the original, annotated, and output images of commonly used segmentation models. The PSPNet model demonstrates a subpar segmentation effect characterized by imprecise delineation of the liquid surface boundary area. It fails to accurately capture the form of the liquid surface and occasionally misclassifies the background as part of the liquid surface. Conversely, the UNet model demonstrates enhanced segmentation outcomes for night-time liquid surface images. However, it encounters difficulties with daytime images, exhibiting fragmented and insufficiently segmented liquid surface regions. The original DeepLabV3+ model also demonstrates instances of both under-segmentation and over-segmentation, along with a limited comprehension of the boundary of the liquid surface. The method presented in this study effectively reduces mis-segmentation, accurately aligns with the ground truth, and exhibits precise judgment for casting mold liquid surfaces in daytime conditions.

3.3. Casting Mold Liquid Surface Area Change Threshold Verification

Experimental analysis revealed that segmentation instability occurs due to factors such as casting lighting, steam, or other influencing factors. However, the change in casting mold liquid surface area during liquid aluminum leakage moments remains higher than during normal casting. Specifically, a 3 s time window is more effective at distinguishing liquid surface area changes between normal conditions and liquid aluminum leakage compared to 1 s or 2 s windows. Additionally, the response time is sufficiently sensitive for production personnel to take prompt action. Therefore, we choose the absolute value of the change in liquid surface area, |ΔS|, with a time window of 3 s as the judgment criterion for liquid aluminum leakage.

Computing liquid surface area and changes for consecutive normal casting datasets from May 2023, along with abnormal casting datasets (related to liquid aluminum leakage) from March to May, resulted in a cumulative dataset of liquid surface area changes over 1856 s. Applying Equation (2) produced a calculated threshold of 1095 for changes in liquid surface area.

This section aims to validate the effectiveness of the threshold for changes in liquid surface area of the casting mold in identifying liquid aluminum leakage. It utilizes datasets from June to August 2023, consisting of liquid aluminum leakage data and normal casting data (the total number of datasets is 110, and the number of aluminum leakage datasets is 25), as the verification set to assess the validity of the calculated threshold.

Here, any value exceeding the threshold in the dataset is considered abnormal. The computation results are presented in

Table 5. Validation of threshold effectiveness for casting mold liquid surface area change.

Index	Results
Accuracy	95.5%
FAR	8.3%
MAR	12.0%

.

The results show that the set threshold achieves an Accuracy of 95.5%, indicating a strong capability to detect liquid aluminum leakage effectively. Due to limited aluminum leakage data and small sample size, the false negative rate and false positive rate are relatively elevated. This finding can be applied in practical production settings to further investigate the FAR and MAR.

4. Conclusions

Real-time monitoring of abnormal liquid surface in the casting mold requires a high level of concentration from the production personnel. Leveraging advancements in computer vision, this work introduces a two-stage method based on image segmentation for calculating the liquid surface area in casting molds, providing a new visual solution for detecting aluminum leakage in deep well casting. First, given the dearth of datasets in this domain, this work constructed a dataset of liquid levels in the distribution plate utilizing production process videos for image annotation. In the stage of data preprocessing, techniques such as region extraction, Gaussian filtering, and image augmentation were employed to tackle challenges related to non-target regions, substantial noise, and complexities in data annotation within the images. Second, to improve casting mold liquid surface segmentation accuracy, an improved DeepLabV3+ model was proposed using a casting mold liquid surface image dataset; we demonstrated that the proposed model surpasses other models, reaching a Mean Intersection over Union (MIoU) score of 91.88% and an inference speed of 55.05 FPS. Third, this work introduces a method for calculating liquid surface area based on OpenCV, which involves accumulating segmented liquid surface pixels to represent the liquid surface area.

Furthermore, a metric for liquid surface area change is proposed, and an abnormal threshold is determined using the IQR method based on data from aluminum leakage casting and normal casting. This approach addresses the challenge of quantifying changes in liquid surface area during aluminum leakage. Experimental validation shows that the proposed method can effectively identify aluminum leakage with an accuracy of 95.5%.