Support Vector Machine with Robust Low-Rank Learning for Multi-Label Classification Problems in the Steelmaking Process

Abstract

In this paper, we present a novel support vector machine learning method for multi-label classification in the steelmaking process. The steelmaking process involves complicated physicochemical reactions. The end-point temperature is the key to the steelmaking process. According to the initial furnace condition information, the end-point temperature can be predicted using a data-driven method. Based on the setting value of the temperature before tapping, multi-scale predicted errors of the end-point temperature can be calculated and divided into different ranges. The quality evaluation problem can be attributed to the multi-label classification problem of molten steel quality. To solve the classification problem, considering that it is difficult to capture nonlinear relationships between the input and output in linear models, we propose a novel support vector machine with robust low-rank learning, which has the characteristics of class imbalance without label correlations; a low-rank constraint is used to deal with high-order label correlations in low-dimensional space. Furthermore, we derive an accelerated proximal gradient algorithm and then extend it to handle the nonlinear multi-label classifiers. To validate the proposed model, experiments are conducted with real data from a practical steelmaking problem. The results show that the proposed model can effectively solve the multi-label classification problem in industrial production. To evaluate the proposed approach as a general classification approach, we test it on multi-label classification benchmark datasets. The results illustrate that the proposed approach performs better than other state-of-the-art approaches across different scenarios.

1. Introduction

The steel industry is the foundation of all industrialized countries worldwide. Steel output is a common metric used by economists to assess the economic strength of different countries. The industrial scale, product quality, economic benefit, and distribution direction of the iron and steel industry all have a significant impact on the development of the country. In order to ensure that metal products have high elasticity, deep drawability, and surface quality, the smelting process must precisely manage the impurities, content, temperature, and composition of molten steel [1,2].

Iron is made in a blast furnace, and steel is made in the converter furnace during the smelting process of the steel plant. The blast furnace is an iron-making machine that performs the initial step in the smelting of iron and steel. The smelting of iron ore into pig iron in a blast furnace is a high-temperature operation. Iron ore, coke, and slag-forming flux (limestone) are loaded from the top of the furnace, then preheated air is blown along the furnace perimeter from the tuyere in the lower half. The carbon in the coke reacts with the oxygen in the air at high temperatures to reduce gases (some blast furnaces additionally inject powdered coal, heavy oil, natural gas, and other auxiliary fuels). During the rising process, these reducing gases heat the slowly descending furnace charge and convert the iron oxide in the iron ore to metallic iron. The blast furnace provides the molten iron required in converter steelmaking.

The converter steelmaking process is a batch process with excellent production effectiveness, rapid reaction response times, and quick smelting. Throughout the steelmaking process, it is crucial to increase the temperature of the molten steel to the necessary level. In addition, a certain amount of alloy needs to be added. Figure 1 shows the smelting process. First, the blast furnace’s molten iron is poured into a torpedo car, then it is transferred to the ladle at the reload station. Moreover, at the desulfurization station, activities for desulfurization and slagging-off are also carried out. After processing, oxygen is blown into the converter furnace so that it directly oxidizes with the high-temperature molten iron to remove impurities. Then the molten iron is changed into molten steel.

The composition and temperature are indicative of the quality of the molten steel. The “cold brittleness” and “thermal brittleness” of steel are caused by the high content of phosphorus and sulfur in molten steel, respectively. During the steelmaking process, phosphorus and sulfur should be removed from the molten steel. When the maximum amount of oxygen is present in molten steel, the thermal brittleness of steel will be exacerbated, and a high number of oxide inclusions will occur, necessitating deoxidation. The presence of inclusions will disrupt the continuity of the steel matrix, degrading the steel’s mechanical characteristics. Therefore, they should be removed. To fulfill the tapping criteria, the temperature must be raised in the steelmaking process, and a specific kind and amount of alloy must be added to ensure the steel composition meets the steelmaking type parameters. To conclude, the primary goals of steelmaking are to achieve decarburization, dephosphorization, desulfurization, and deoxidation by using oxygen, slag, alloy addition, and stirring. Inclusions are removed by raising the temperature and adjusting the composition, then the necessary liquid steel is obtained and transferred into suitable ingots or billets.

Pure oxygen is blown into the molten iron in a basic oxygen furnace (BOF) steelmaking process. This process must be continuously monitored to obtain high-quality molten steel. Based on the physicochemical reactions that take place in the BOF, the process can be split into three stages: the silicon and manganese reaction stage, the carbon oxidation stage, and the blowing stage. The reaction process in BOF steelmaking is summarized. First, steel scraps and hot metal are pumped into a converter furnace. Second, a water-cooled lance is used to blast high-purity oxygen into the molten steel. The temperature rises from around 1200 °C to 1700 °C as a result of certain carbon oxidation events occurring in the molten steel, which also produce carbon dioxide and carbon monoxide. Third, supplementary ingredients (such as limestone) are added to the molten steel to meet the quality requirements. Meanwhile, a mixture of inert gases (nitrogen or argon) is blown into the furnace from the bottom to ensure a sufficient reaction. Finally, a sublance is used to complete the sampling process. The continuous oxygen-blowing operation increases the consumption of metal iron and decreases the life of the furnace lining if the molten steel’s quality falls below the necessary standard. Furthermore, the blowing time is fixed in actual operation. When the appropriate blowing time is reached, the furnace is turned on to measure temperature and carbon, and the refining process is selected. If the molten steel continues to blow, productivity suffers significantly, and quality is difficult to ensure.

Because of the short smelting period of the converter (from a few minutes to more than ten minutes), it is difficult to manually manage the temperature to ensure that the target composition of molten steel is achieved. Moreover, due to the black-box smelting conditions, and the steelmaking process can only be controlled by operator experience. The iron and steel smelting, however, is a multiphase, high-temperature reaction process. Classic interface response theory can only describe and explain traditional research methods; these methods fall short of addressing the needs of industrial production. The study of complex metallurgical reactions has become a low-cost and high-efficiency process due to the development of artificial intelligence (AI). Many studies have shown that AI models detect and examine metallurgical reactions more accurately than traditional metallurgical theory [3,4]. The establishment of an AI model is extremely beneficial to researchers and operators. As a result, we construct a steelmaking process anomaly detection model that can learn effective information from the steelmaking process to judge whether there is an anomaly in the future smelting process, allowing operators to detect and correct problems in real time.

Considering that the BOF steelmaking process has high-temperature and complex physical and chemical reactions, it is difficult to establish a precise quality evaluation model for molten steel. This is due to the multi-label classification problem. To solve this problem, data-driven methods are applied to the steelmaking process. Overall, the contributions of the current study are as follows:

• We present a novel multi-label classification model called the support vector machine with robust low-rank learning (SVM-RL) and derive a kernelization SVM-RL to capture nonlinear relationships between the input and output.
• In the kernel classifier case, the surrogate least-squares hinge loss is replaced with a margin-based loss to make it smooth for efficient optimization. It is expected that this will result in avoiding some of the over-fitting problems that may be encountered, and it is used as an intermediate step to discover the structure of a dataset.
• A multi-label classification problem is derived from the practical steelmaking process, which has a black-box property. The proposed approach effectively solves this difficulty, and different benchmark problems are used to verify the performance of SVM-RL.

The rest of this paper is organized as follows. Section 2 summarizes the related literature. Our proposed method is introduced in Section 3. Section 4 shows the experimental results for the practical problem and benchmark problems. Finally, Section 5 provides conclusion and discusses future work.

2. Related Literature

In anticipating and analyzing complicated metallurgical models, AI algorithms can considerably improve the understanding of metallurgical processes. Some researchers have recently presented a variety of valuable algorithms for solving difficult metallurgical problems. They comprise regression analyses of specific quantitative descriptions as well as abnormal judgment categorization analyses. Azadi et al. [5] addressed issues such as large-scale monitoring of blast furnace operation status, multi-phase and multi-scale physicochemical changes, and long computation times, among others. They proposed a hybrid dynamic model to predict the hot metal silicon content and the slag basicity in the blast furnace process. The suggested model demonstrated good performance using actual production data from a blast furnace. Cardoso et al. [6] developed a committee machine containing 108 independent artificial neural networks to anticipate the impurity concentration of cast iron made in a blast furnace. The results of their experiments showed that this integrated method accurately estimated the impurity concentration in blast furnaces and has practical application value. Han et al. [7] proposed a bidirectional recursive multiscale convolution depth neural network algorithm based on the continuous spectral information of the furnace mouth flame in the steelmaking process. On this basis, a dynamic prediction model for the later stages of steelmaking was established, and the effectiveness of the algorithm was demonstrated through trials. Saigo et al. [8] proposed the Einstein–Roscoe expression and a transfer learning framework based on the Gaussian process to assist in measuring and estimating regression parameters for the viscosity prediction of the steelmaking process. Experiments showed the suggested technique was effective and that it outperformed other machine learning algorithms in clay prediction. Deng et al. [9] employed FactSage technology to construct a dynamic model of different oxides in the oxidation process of ultra-low carbon steel and carried out a detailed analysis of slag impurity oxides, which provided a significant contribution to the control of the steelmaking process. In light of the challenges of fewer data and large data fluctuation in blast furnace ironmaking, Gao et al. [10] developed a migration learning approach for robust fault identification and demonstrated that the algorithm can solve the abnormal diagnosis of blast furnace ironmaking through tests. Li et al. [11] developed a nonparallel hyperplane-based fuzzy classifier model to coordinate their model’s accuracy and interpretability based on the closed smelting and hysteresis features of a blast furnace system and tested the classification impact utilizing blast furnace data. On the topic of industrial failure detection, Rippon et al. [12] created a new industrial prediction classification problem and designed a machine learning methodology. They proposed a complete representation learning prediction classification framework by comparing conventional and contemporary representation learning approaches with data from an electric arc furnace. Zhang et al. [13] constructed a non-contact intelligent prediction model based on the internet of things system for predicting carbon content in molten steel and used the support vector machine method to forecast the provided carbon content criterion. The model accurately predicted and classified the steelmaking process, according to the experimental data. Zhou et al. [14] proposed a fault-detection approach based on deep learning and multi-information fusion to handle the problems of overheating, abnormal exhaust, partial melting, and other issues. Their experiments demonstrate the algorithm’s capability. In light of the coexistence of Gaussian and non-Gaussian blast furnace ironmaking data, Zhou et al. [15] suggested an integrated technique using PCA and ICA to detect and diagnose aberrant furnace conditions in the blast furnace ironmaking process.

With better knowledge of the metallurgical mechanism model, the correlation of multiple indicators of multi-output and multi-input data has been a focus of current studies in the metallurgical process. Feng et al. [16] presented a multichannel diffusion graph convolutional network using the correlation of molten steel element concentration to predict the end-point composition of molten steel. Experiments show that by mining the data correlation, the model may successfully increase end-point prediction performance. Li et al. [17] presented a novel multi-input multi-output Takagi-Sugeno fuzzy model. It has been used in the blast furnace ironmaking process. In order to evaluate the aging status of steel ladles in steelmaking plants, Vannucci et al. [18] suggested a multi-classification approach for uneven datasets. Experiments showed the model could predict when the ladle was nearing the end of its useful life. In light of the critical function of the oxygen demand changing the trend of iron and steel businesses in the iron and steel sector, Zhou et al. [19] suggested an oxygen-demand-prediction approach based on multi-output Gaussian process regression. Their algorithm’s usefulness and performance were demonstrated by comparing it to other algorithms in real-world scenarios.

3. Problem Formulation and Optimization

A significant development across several industries is machine learning (ML). It is an artificial intelligence method that can predict outcomes. One issue with predictive modeling is classification. The goal of classification is to provide a given set of input data with a class label. In scientific research, categorization models have acted as helpful AI tools in a variety of fields, such as financial credit, risk assessment [20], signal processing, and pattern recognition [21]. The main objective of these methods is to accurately forecast diverse circumstances. The support vector machine (SVM) offers a well-known and widely used classification algorithm that has sufficient capacity to generalize, fewer local minima, and minimal dependency on a small number of factors [22], and has seen success in applications as a strong, flexible classifier with excellent accuracy. However, the approach taken in conventional formulation settings cannot determine the relative relevance of distinct characteristics, and its performance may be seriously compromised when redundant variables are employed to determine the decision rule, even when they are ineffective at guessing due to the buildup of random noise, especially in a high-dimensional space [23].

When samples are not linearly separable, the kernel form of the SVM is created through the projection of input space to higher dimensional space. Although several characteristics are accessible, not all of them will be used in the classifier’s creation. If all the available input characteristics are incorporated into the model, redundant features and excessive noise may reduce the classifier’s accuracy while increasing its complexity. Obtaining a suitable solution for the kernel SVM is therefore vital since it is the most representative of the margin-based classifiers. The objective function of SVM is:

$$\begin{array}{ll}\min :& \frac{1}{2}{\Vert \mathit{w}\Vert }^2\\ s.t.& y_i({\mathit{w}}^T{\mathit{x}}_i+b_i)\ge 1\end{array}$$

(1)

where w is the weight vector, b is the paranoid item, and y_i is the ith feature value.

However, computing the original SVM directly is typically difficult as it is non-smooth. In order to convert a non-convex issue into a convex problem that can be solved by an optimization approach, the SVM algorithm employs the Lagrange multiplier method to transform the original problem into an appropriate dual problem. In the current state of machine learning algorithms, this is also the preferred technique for solutions. It builds an appropriate objective function for the optimization method without altering its original goal and simplifies or approximates problems that cannot be solved.

$$L(\mathit{w},b,\alpha )=\frac{1}{2}{\Vert \mathit{w}\Vert }^2+{\displaystyle \sum _{i=1}^m{\alpha }_i[1-y_i({\mathit{w}}^T{\mathit{x}}_i+b)]}$$

(2)

where α is the Lagrange multiplier, and the second term is called the cost function term which denotes hinge loss.

The SVM algorithm has achieved excellent results for single-label classification. However, many practical production-classification problems demonstrate the importance of multiple label groups. In image classification, for example, an image frequently contains rich semantic information, such as multiple scenes, targets, and behaviors. As a result, single-objective classification’s generalization, multi-label classification, is a more practical and universal approach. Many researchers have improved the objective function of SVM, and one of them has modified the cost function term of SVM. Specifically, the hinge loss function of the replacement SVM becomes a margin-based loss. Achieving the goal of this method involves minimizing sorting loss when the margin is large and dealing with the nonlinear case utilizing the kernel technique [24].

Based on this investigation, the surrogate least-squares hinge loss is further substituted with a margin-based loss in place of punishing the conventional cost function of SVMs. To leverage high-order label correlations under the assumption of a low-dimensional label space, we explicitly put a low-rank restriction on the kernel parameter matrix, so that the chosen features that are utilized to build the rule should be few, if not even sparse, and simultaneous categorization is possible in practice.

Mathematically, the problem becomes the following.

$$\begin{array}{l}\underset{W}{\min }\frac{1}{2}{\Vert L_{\delta }\left({\left|E-Y\circ \left(\varphi \left(X\right)W\right)\right|}_{+}\right)\Vert }_1+\frac{{\lambda }_1}{2}{\Vert W\Vert }_F^2\\ s.t \mathrm{Rank}\left(W\right)\le k\end{array}$$

(3)

where $\varphi \left(X\right)=\left[\varphi \left(x_1\right)\dots \dots \varphi \left(x_n\right)\right]$ is the approximate function of low-dimensional sample data X to high-dimensional data. $\left({\left|E-Y\circ \left(\varphi \left(X\right)W\right)\right|}_{+}\right)$ is the margin-based loss employed to approximate the thresholding 0–1 loss, $E={\left\{1\right\}}^{n\times l}$ denotes the matrix with each element equal to 1, $\circ$ denotes the Hadamard product of matrices, ${\Vert \Vert }_1$ denotes the l₁-norm, ${\Vert W\Vert }_F^2={\displaystyle \sum _{j=1}^l{\Vert W^j\Vert }^2}$ is the reciprocal of soft spacing distance, and $\mathrm{Rank}\left(W\right)$ is a low-rank restriction on the parameter matrix made under the supposition that the label space is low dimensional.

It is obvious that the restricted optimization problem in Equation (3) may be turned into the following unconstrained optimization problem by substituting the trace norm for the rank regularization.

$$\underset{A}{\min }\frac{1}{2}{\Vert L_{\delta }\left({\left|E-Y\circ KA\right|}_{+}\right)\Vert }_1+\frac{{\lambda }_1}{2}Tr\left(A^{T}KA\right)+{\lambda }_2{\Vert A\Vert }_{\ast }$$

(4)

In recent research, Zou [25] introduced a new loss function penalty formula. The experiment demonstrated that the loss penalty effect enhanced computation performance and was smoother when dealing with categorization issues. Based on this work, we propose a novel support vector machine with robust low-rank learning (SVM-RL). SVM-RL aims to minimize the margin-based loss and maximize the soft spacing distance at the same time.

For any $\delta >0,$ we define a $\delta -$ smoothed hinge loss function

$$L_{\delta }\left(u\right)=\{\begin{array}{ll}0& u\ge 1+\delta ,\\ \frac{1}{4\delta }{\left[u-\left(1+\delta \right)\right]}^2& 1-\delta <u<1+\delta ,\\ 1-u& u\le 1-\delta .\end{array}$$

(5)

$L_{\delta }(•)$ is a brand-new classifier and convex loss based on margins in $\mathrm{RKHS} H_K$ is

$${\alpha }^k=\arg {\min }_{\alpha \in R^n}\left[\frac{1}{n}{\displaystyle \sum _{i=1}^n{L}_{\delta }\left(y_i{K}_i^T\alpha \right)}+{\lambda }_1{\alpha }^{T}K\alpha +{\lambda }_{2}Loss\left(\alpha \right)\right]$$

(6)

Proposition 1.

Let$Q(\alpha )=\frac{1}{n}{\displaystyle \sum _{i=1}^n{L}_{\delta }(y_i{K}_i^T\alpha )}+{\lambda }_1{\alpha }^{T}K\alpha +{\lambda }_{2}Loss(\alpha )$, then we have$Q\left({\alpha }^k\right)-\frac{\delta }{4}\le Q\left({\alpha }^{SVM}\right)\le Q\left({\alpha }^k\right)$.

Lemma 1.

For any  $0<\delta <{\delta }^{\ast },$ define  ${\tilde{E}}_0^{\delta }=\left\{i:\left|1-y_i{K}_i^T{\alpha }^k\right|<\delta \right\}$ and  ${\tilde{\alpha }}_0^{\delta }={\alpha }^k.$ For $k=1,2,\dots \dots ,$ let

$${\tilde{\alpha }}_0^{\delta }=\arg {\min }_{\alpha \in R^n}\left[\frac{1}{n}{\displaystyle \sum _{i=1}^n{L}_{\delta }\left(y_i{K}_i^T\alpha \right)}+{\lambda }_1{\alpha }^{T}K\alpha +{\lambda }_{2}Loss\left(\alpha \right)\right]$$

(7)

subject to $1=y_i{K}_i^T{\tilde{\alpha }}_k^{\delta },\forall i\in {\tilde{E}}_{k-1}^{\delta }$ and ${\tilde{E}}_{k-1}^{\delta }=\left\{i:\left|1-y_i{K}_i^T{\tilde{\alpha }}_k^{\delta }\right|<\delta \right\}.$ Then there exists a finite $k^{\ast }$ such that ${\tilde{E}}_{k^{\ast }}^{\delta }={\tilde{E}}_{k^{\ast }-1}^{\delta }\subseteq E_0$ and ${\tilde{\alpha }}_{k^{\ast }+1}^{\delta }={\tilde{\alpha }}_{k^{\ast }}^{\delta }.$ We denote ${\tilde{E}}^{\delta }={\tilde{E}}_{k^{\ast }}^{\delta }$ and  ${\tilde{\alpha }}^{\delta }={\tilde{\alpha }}_{k^{\ast }}^{\delta }$.

Lemma 2 is that ${\tilde{\alpha }}^{\delta }$ will actually equal ${\alpha }^{SVM}$ once $\delta$ is less than ${\delta }^{\ast }.$

Lemma 2.

Suppose there exists some$i$such that$y_i{K}_i^T{\alpha }^{SVM}\ne 1.$It holds that${\tilde{\alpha }}^{\delta }={\alpha }^{SVM}$as long as$\delta <{\delta }^{\ast },$where${\delta }^{\ast }=\min \left\{{\delta }_0,4\eta \right\}.$

Remark 1.

All the proofs are similar to the reference[25], so we leave out them here owing to lack of space.

Remark 2.

Due to the superiority of the APG, Equation (3) can be guaranteed to converge to a global optimum with the$\mathrm{O}\left(\frac{1}{t^2}\right)$convergence rate.

The main flow is outlined in Algorithm 1, which summarizes the entire APG algorithm for training and tuning SVM-RL.

Algorithm 1: Accelerated Proximal Gradient Method for Equation (5)
Input: $X\in R^{n\times m},Y\in {\left\{-1,1\right\}}^{n\times l},$ the kernel matrix $K\in R^{n\times n},$ tradeoff    hyper-parameters ${\lambda }_1,{\lambda }_2$.
Output: $W^{\ast }\in R^{n\times l}$.
1	Initialize $t=1,b_1=1,\delta .$ Define the smooth function $L_{\delta }.$
2	Initialize $G_1=W_0\in R^{n\times l}$ as zero matrix.
3	while (3) not converge do
4	Compute the gradient of $\nabla G_{t}f\left(G_t\right)$.
5	$W_t={\mathrm{prox}}_{\left({\lambda }_2/L_f\right)}\left(G_t-\frac{1}{L_f}\nabla G_{t}f\left(G_t\right)\right)$.
6	$b_{t+1}=\frac{1+\sqrt{1+4b_t^2}}{2}$.
7	$G_{t+1}=W_t+\frac{b_t-1}{b_{t+1}}\left(W_t-W_{t-1}\right)$.
8	$t=t+1$.
9	Until the KKT conditions of all SVM models are satisfied
10	end
11	$W^{\ast }=W_{t-1}$.

4. Numerical Results

To demonstrate the model’s practicality, we used actual data from the steelmaking process to solve multi-label classification tasks. Furthermore, we performed a significant number of experimental evaluations on four different sets of public data. The performance of the model was consistent and high-performing, and the results outperformed existing advanced approaches, demonstrating that the model had excellent performance and universality in multi-label classification.

4.1. Experimental Details

This section provides an overview of the experimental equipment and evaluation criteria.

4.1.1. Experimental Setting

• Experimental platform: The experiment was performed on a computer with an Intel^® Core™ i5-9400f 2.90 GHz CPU, 16 GB RAM and NVIDIA GeForce GRX 1060 6 GB GPU, with a 64-bit Windows 11 operating system. The software was developed using Matlab2016.
• Parameter setup: Grid searching was used to find the model parameters in this paper. The parameters were as follows: λ₁, λ₃, $\sigma =\{{10}^{-4},{10}^{-3},\dots ,{10}^2\}$. A total of 60 percent of each dataset was chosen at random as training data, while the remaining 40 percent was used as test data. To avoid aberrant interference, each dataset was subjected to 10 independent experiments, with the results being averaged.

4.1.2. Evaluation Metrics

• Single-label classification indicators: to assess the model’s performance, we employed Precision, Recall, F1-Score, and Accuracy as indicators;

  Table 1. The confusion matrix of a single label classification.

  	Total Population	Predicted Condition Positive	Predicted Condition Negative
  True Condition	Condition Positive	True Positive (TP)	False Negative (FN)
  	Condition Negative	False Positive (FP)	True Negative (TN)

  is the confusion matrix of single-label classification.

• “Precision” indicates the fraction of predicted condition positive to total prediction condition positive:

$$\mathrm{Precision}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}$$

(8)

2.

“Recall” indicates the fraction of predicted condition positive to total condition positive:

$$\mathrm{Recall}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}$$

(9)

3.

“F1-Score” is a combination of two contradictory indicators. They are the accuracy rate and the recall rate:

$$\mathrm{F}1\text{-}\mathrm{Score}=\frac{2\times \mathrm{precision}\times \mathrm{recall}}{\mathrm{percision}+\mathrm{recall}}$$

(10)

4.

“Accuracy” is the overall categorization index, the number of valid classifications:

$$\mathrm{Accuracy}=\frac{\mathrm{TP}+\mathrm{TN}}{(\mathrm{TP}+\mathrm{FN}+\mathrm{FP}+\mathrm{TN})}$$

(11)

• Multi-label classification indicators to assess the following: Hamming Loss, Subset Accuracy, F1-Example, Ranking Loss, Coverage, Average Precision [26] as the multi-classification problem evaluation index. Assuming a given test dataset $u=\{({\mathit{x}}_i,Y_i)|1\le i<t,Y_i=\{1,-1\}\}$, f(x_i) denotes each of the multi-label classifier prediction label sets for x_i, and q is the label count.

• Hamming Loss (Hal) calculates the binary average error ratio of Tags. ${\Vert \Vert }_1$ denotes the l₁-norm.

  $$\mathrm{Hal}=\frac{1}{t}{\displaystyle \sum _{i=1}^t\frac{1}{q}}{\Vert f({\mathit{x}}_i)\oplus Y\Vert }_1$$

  (12)
• Ranking Loss (Ral) calculates the ratio of reverse labels.

  $$\begin{array}{l}\mathrm{Ral}=\frac{1}{t}{\displaystyle \sum _{i=1}^t\frac{\left|Set{R}_i\right|}{\left|Y_i^{+}\right|\left|Y_i^{-}\right|}}\\ Set{R}_i=\{(p,q)|f_p({\mathit{x}}_i)\le f_q({\mathit{x}}_i),(p,q)\in Y_i^{+}\times Y_i^{-}\}\end{array}$$

  (13)
• Coverage (Cov) calculates the number steps it takes to progress down the label sort until all the fundamental real tags are covered.

  $$\mathrm{Cov}=\frac{1}{q}(\frac{1}{t}{\displaystyle \sum _{i=1}^t\underset{l_k\in Y_i}{\max }}rank({\mathit{x}}_i,l_k)-1)$$

  (14)
• Subset Accuracy (Sa) calculates the ratio of the predicted label subset that matches the actual data label. F(x_i) is the total multi-label classifier prediction label set for x_i.

  $$\mathrm{Sa}=\frac{1}{t}{\displaystyle \sum _{i=1}^{t}F({\mathit{x}}_i)}=Y_i$$

  (15)
• F1-Example (F1e) indicates the average of F1-Score for each instance.

  $$\mathrm{F}1\mathrm{e}=\frac{1}{t}{\displaystyle \sum _{i=1}^t\frac{2|f{({\mathit{x}}_i)}^{+}\cap Y_i^{+}|}{\left|f{({\mathit{x}}_i)}^{+}\right|+\left|Y_i^{+}\right|}}$$

  (16)

  where $rank({\mathit{x}}_i,l_k)$ stands for the rank of the label l_k in the ranking list based on F(x_i) which is sorted in descending order.
• Average Precision (Ap) calculates the average score of related labels above that of a particular label.

  $$\begin{array}{l}\mathrm{Ap}=\frac{1}{t}{\displaystyle \sum _{i=1}^t\frac{1}{\left|Y_i^{+}\right|}{\displaystyle \sum _{l_j\in Y_i^{+}}\frac{\left|Set{P}_{ij}\right|}{ran{k}_F({\mathit{x}}_i,l_j)}}}\\ Set{P}_{ij}=\{l_k\in Y_i^{+}|ran{k}_F({\mathit{x}}_i,l_k)\le ran{k}_F({\mathit{x}}_i,l_j)\}\end{array}$$

  (17)

4.2. Multi-Label Classification Problems in the Steelmaking Process

4.2.1. Experimental Data and Settings

The information utilized in this experiment came from an actual BOF steelmaking process. A multi-source sensor was utilized to collect the data; a flame analyzer was used to monitor real-time temperature; sublance and tossing probes sampled the quality components in the molten steel. The above means were used to collect the data from a steel plant for one year, through the screening of missing values and abnormal values. The number of data samples was 2755 in total. The input data included the following: (1) steel scrap data: addition amount of steel scrap, etc.; (2) molten iron data: addition amount of molten iron, molten iron composition, etc.; (3) oxygen lance data: height of oxygen lance, etc.; (4) auxiliary material data: types of auxiliary material, addition amount of auxiliary material, etc.; (5) oxygen blowing data: flow of oxygen, etc.; (6) alloy data: types of alloy addition, addition amount of alloy, etc. The training data points comprised 1629 samples with 20 features, and the test data points comprised 1086 samples with 20 features. The output data comprised the variance between the end-point molten steel temperature and the actual end-point molten steel temperature before tapping. If the predicted values were larger than 15 °C and 10 °C, respectively, the label was displayed as −1. On the contrary, it is marked as 1 if values were less than or equal to 15 °C and 10 °C. The SVM was the comparison method. We used a grid search to optimize the parameters of the SVM to make the results more believable. The most impressive outcomes are highlighted.

4.2.2. Experimental Results

The suggested model’s judgment performance in the steelmaking dataset is shown in Figure 2 andf Figure 3 and

Table 2. The experimental results (mean) of comparison between RBRL, SVM, and SVM-RL on the steelmaking process dataset.

15 °C		Precision	Recall	F1-Score	Accuracy
RBRL	−1	0.92	0.80	0.86	0.77
	1	0.30	0.55	0.39
SVM	−1	0.98	0.78	0.87	0.78
	1	0.16	0.71	0.26
SVM-RL	−1	1.00	1.00	1.00	1.00
	1	1.00	1.00	1.00
10°C		Precision	Recall	F1-Score	Accuracy
RBRL	−1	0.41	0.64	0.50	0.58
	1	0.71	0.49	0.57
SVM	−1	0.47	0.66	0.55	0.57
	1	0.68	0.51	0.58
SVM-RL	−1	1.00	0.74	0.85	0.80
	1	0.55	1.00	0.71

. The index of single-label categorization was used to analyze each task in the steelmaking process. The model produced 100% correct prediction answers for the 15 °C label problem. At the same time, the model’s accuracy for the problem of 10 °C was 80%, which was effective to fulfill the site’s actual requirements. Because of the significant nonlinear interaction between converter input features, the SVM/SVR has always been an efficient approach to deal with steelmaking challenges. It has been demonstrated via comparison trials that the proposed model’s accuracy performance can be improved by at least 20%. The results of the comparison are shown in Figure 2 and Table 2. The model’s precision exceeds 20%, which was higher than the SVM in each category. The correlation of input and output features can often be leveraged to improve performance in multi-label task prediction. Simultaneously, to compare the proposed model’s convergence performance, we demonstrated the algorithm’s loss-reduction process; the comparison of the loss function curve is shown in Figure 3. As expected, the suggested model converged on the loss function more smoothly and quickly, and the final loss function was smaller. Furthermore, the curve of the model was basically in convergence when the loss function was updated, and the loss function value approached 0 after 60 iterations. However, the RBRL convergence was unstable, and the function did not converge until the maximum number of iterations was reached. This demonstrated that the proposed model was more efficient than the RBRL.

4.2.3. Sensitivity Analysis

Finally, we examined the steelmaking data’s sensitivity to the suggested model parameters $\{{\lambda }_1,{\lambda }_2,\delta \}$ in a single-variable situation. Among these, the λ₁ control model’s complexity was relatively high, and the model’s robustness was higher. λ₂ controls the impact of the output tag’s relevance on the objective function. σ is the width of the RBF kernel function; the better the nonlinearity, the higher the value; however, it is easy to overfit. The optimal hyper parameters were determined by cross-validation in the experiment, and the performance effects of the other two hyper parameters were assessed once one hyperparameter was fixed. Among them, the variation range of each parameter was [10⁻⁴, 10²]. The sensitivity analysis findings for each evaluation index are depicted in Figure 4. The hyper parametric influence of the data focused mostly on the kernel function width coefficient because of the significant nonlinear link between the input properties of steelmaking data. Because of the short kernel function width coefficient, the model could not fit the data adequately and performed poorly. The model performed best at the midway value of the kernel width coefficient, which was in good agreement with the theory. For λ₁ and λ₂, the model’s performance change trend was not evident. This suggested that λ₁, λ₂ had little interaction effect, λ₁ controlled the complexity of the model, and λ₂ adjusted the weight of the correlation between the model outputs in the objective function. The influence of these two characteristics had little bearing on the overall evaluation of each piece of work.

4.3. Benchmark Test Problems

4.3.1. Benchmark Dataset

We conducted multiple comparative experiments on four datasets covering a wide range of scales and fielded to establish the algorithm’s generalization. The details of the benchmark dataset were listed in

Table 3. Statistics of the benchmark experimental datasets. (“Cardinality” indicates the average number of labels per example. “Density” normalizes the “Cardinality” by the number of possible labels. These datasets can be obtained from the reference [27]).

Dataset	Instance	Feature	Label	Cardinality	Density	Domain
Emotions	593	72	6	1.869	0.331	music
Image	2000	294	5	1.240	0.248	image
Scene	2407	294	6	1.074	0.179	image
Yeast	2417	103	14	4.239	0.303	biology

. Six metrics were used to evaluate the multi-label classification problems.

4.3.2. Experimental Results

The multi-label classification experiments were also conducted with several state-of-the-art methods—RBRL [27], Rank-SVM [24], Rank-SVMz [28], BR [29], ML-KNN [30], CLR [31], RAKEL [32], CPNL [33] and MLFE [34]—to show the effectiveness of the SVM-RL. The comparison results with the number of best results are presented in

Table 4. Experimental results of each benchmark approach (mean) on four multi-label datasets. $\downarrow (\uparrow )$ indicates the smaller (larger) the better.

	Metric	Rank-SVM	Rank-SVMs	BR	ML-kNN	CLR	RAKEL	CPNL	MLFE	RBRL	SVM-RL
emotions	Hal $(\downarrow )$	0.189	0.201	0.183	0.200	0.182	0.177	0.183	0.186	0.181	0.167
	Ral $(\downarrow )$	0.155	0.149	0.246	0.169	0.149	0.192	0.139	0.142	0.138	0.133
	Cov $(\downarrow )$	0.294	0.291	0.386	0.306	0.283	0.338	0.277	0.282	0.277	0.284
	Sa $(\uparrow )$	0.291	0.292	0.313	0.285	0.318	0.356	0.324	0.291	0.334	0.361
	F1e $(\uparrow )$	0.645	0.675	0.620	0.605	0.624	0.679	0.684	0.621	0.666	0.682
	Ap $(\uparrow )$	0.808	0.819	0.760	0.796	0.813	0.801	0.828	0.822	0.828	0.835
image	Hal $(\downarrow )$	0.161	0.177	0.156	0.175	0.157	0.154	0.150	0.156	0.149	0.075
	Ral $(\downarrow )$	0.143	0.142	0.220	0.180	0.144	0.173	0.132	0.142	0.133	0.072
	Cov $(\downarrow )$	0.171	0.170	0.227	0.198	0.168	0.191	0.157	0.165	0.160	0.115
	Sa $(\uparrow )$	0.451	0.411	0.482	0.393	0.477	0.527	0.533	0.463	0.552	0.686
	F1e $(\uparrow )$	0.631	0.670	0.623	0.503	0.627	0.680	0.698	0.593	0.688	0.815
	Ap $(\uparrow )$	0.823	0.826	0.772	0.786	0.826	0.813	0.839	0.826	0.836	0.930
scene	Hal $(\downarrow )$	0.092	0.113	0.077	0.091	0.078	0.075	0.077	0.083	0.073	0.0884
	Ral $(\downarrow )$	0.065	0.072	0.128	0.083	0.061	0.087	0.059	0.063	0.058	0.047
	Cov $(\downarrow )$	0.068	0.075	0.119	0.084	0.064	0.089	0.064	0.067	0.062	0.055
	Sa $(\uparrow )$	0.563	0.500	0.655	0.615	0.650	0.696	0.699	0.617	0.735	0.541
	F1e $(\uparrow )$	0.664	0.756	0.717	0.678	0.718	0.756	0.802	0.685	0.803	0.595
	Ap $(\uparrow )$	0.882	0.874	0.834	0.858	0.887	0.875	0.893	0.885	0.895	0.905
yeast	Hal $(\downarrow )$	0.203	0.207	0.188	0.195	0.188	0.195	0.192	0.194	0.187	0.184
	Ral $(\downarrow )$	0.170	0.172	0.308	0.170	0.158	0.244	0.158	0.166	0.157	0.134
	Cov $(\downarrow )$	0.446	0.458	0.627	0.451	0.436	0.543	0.445	0.452	0.436	0.412
	Sa $(\uparrow )$	0.156	0.179	0.190	0.177	0.194	0.248	0.179	0.172	0.192	0.077
	F1e $(\uparrow )$	0.632	0.643	0.623	0.615	0.625	0.647	0.630	0.607	0.628	0.635
	Ap $(\uparrow )$	0.755	0.765	0.680	0.762	0.773	0.727	0.775	0.769	0.777	0.814
Score		0	0	0	0	0	2	2	0	4	17

. From the table, we can see that the proposed model has obvious advantages with a score of 17 in total. For each metric on all the datasets, SVM-RL was almost superior to the other models and had powerful generalization. Specifically, in terms of the AP, the proposed model excesses far more than any other comparison method. For the “emotions” and “scene” datasets, the results were not the best ones. Because the nuclear norm is the tightest convex approximation of the rank function, the low-rank matrix can be recovered with a high probability under certain conditions. Unfortunately, when the rank number of the low-rank matrix is larger, the low-rank solution is biased. Although some improved SVM models can achieve good performance, they are prone to over-fitting for the small and middle training samples with a single label. However, the problem can be solved by the regularization function of the proposed model. Because SVM-RL can minimize the approximate Hamming Loss with less model complexity, it is easy to control the generalization error by the mechanism; thus, the generalization ability of SVM-RL can be improved. Therefore, the experimental results verify the effectiveness of SVM-RL.

5. Conclusions

This paper was motivated by the challenging issue of solving multi-label classification problems in the steelmaking process. To address the problem, we proposed a novel support vector machine with robust low-rank learning, which had a low-rank constraint for high-order label correlations in low-dimensional space, and the accelerated proximal gradient algorithm was applied to the kernel classifier with a smooth loss function for the nonlinear multi-label classifiers. To verify the practicability of the proposed approach, we conducted experiments on a real steelmaking production process. The accuracy of the proposed model surpassed 20% of the SVM in each category. The results illustrate that SVM-RL can effectively solve the multi-label classification problem in the steelmaking process. In addition, to further validate the proposed approach, multi-label classification benchmark problems were used to test the performance of SVM-RL. The proposed model had obvious advantages with the 17 scores in total. The results indicate that SVM-RL outperforms other state-of-the-art approaches across different scenarios. Possible future work could involve exploiting sparse features in the input space and employing non-convex penalty functions and data-adaptive multi-class kernel functions when constructing the SVM in the ironmaking process.