Intelligent Hybrid Modeling of Complex Leaching System Based on LSTM Neural Network

Abstract

In order to improve the leaching efficiency of gold ore and reduce the environmental treatment cost of residual sodium cyanide, continuous stirred tank reactors are often connected in a cascade manner. A gold leaching system is a multiphase chemical reaction system, and its kinetic reaction mechanism is complex and affected by random factors. Using intelligent modeling technology to establish a hybrid prediction model of the leaching system, the dynamic performance of the process can be easily analyzed. According to the reaction principle and the theory of substance conservation, a mechanism model is established to reflect the main dynamic performance of the leaching system. In order to improve the global convergence of the optimization target, a particle swarm optimization (PSO) algorithm based on simulated annealing is used to optimize the adjustment parameters in the kinetic reaction velocity model. The multilayer long short-term memory (LSTM) neural network approach is used to compensate for the prediction errors caused by the unmodeled dynamics, and a hybrid model is established. The hybrid prediction model can accurately predict the leaching rate, which provides a reliable basis for guiding production, and also provides a model basis for process optimization, controller design, and operation monitoring. Finally, the superiority and practicability of the hybrid model are verified by a practical leaching industrial system test. The prediction model of key variables in the leaching process is established for the first time using the latest time series prediction technology and intelligent optimization technology. The research results of this paper can provide a good reference and guidance for other research on complex system hybrid modeling.

1. Introduction

A hydrometallurgical leaching system is a typical complex system involving physical and chemical reactions, variables, and uncertainties [1]. Establishing the model of this complex system is helpful to analyze the relationship between the internal structure characteristics and the external variables [2]. With the decline in ore gold grade and increasingly strict environmental protection standards, hydrometallurgy plays an increasingly important role in the production of non-ferrous metals [3]. Hydrometallurgy technology can directly deal with low-grade and complex materials, improve the comprehensive utilization rate of resources, and realize cleaner production easily [4]. Hydrometallurgy is mainly based on the disciplines of metallurgical physics chemistry and reaction engineering [5]. Leaching is an important process in hydrometallurgy. Leaching is a complex process of chemical dissolution reaction between a leaching agent and specific components in solid materials [6]. The solution containing the extracted substance can be formed by leaching and can be preliminarily separated from other insoluble components. Leaching systems are mainly based on filtration leaching and agitation leaching. The leaching rate is the key index of any leaching system, and it is usually detected by offline assay [7]. The quality of the leaching solution directly determines the quality and benefit of subsequent production processes. Based on experimental tests, computer simulations, and mathematical derivation, the dynamic model of the leaching system can be established [8]. The process model can provide the necessary scientific basis for the engineering development and application of new hydrometallurgical processes.

The main factors affecting gold cyanidation leaching are cyanide concentration, oxygen concentration, pH value of leaching solution, ore pulp temperature, ore particle size and shape, pulp concentration, leaching time, impurities, and associated minerals [9]. In order to ensure that the leaching system operates in the optimal state, it is necessary to accurately establish the prediction model of the key variables related to the leaching rate [10]. The mechanism model of the leaching system can be established by analyzing the structure and internal chemical changes of the leaching system. Combined with prior knowledge and reasonable assumptions, the mechanism model is simplified to reveal the external characteristics of the leaching system, which has important practical significance for the leaching system [11]. The influence of related variables on the leaching rate and key auxiliary variables can be determined by analyzing the mechanism model. The gold cyanidation leaching system is a complex multiphase chemical reaction process. The complicated kinetic reaction mechanism and process random factors increase the difficulty in modeling the leaching system [12]. Due to the different production technologies, equipment parameters, and operation modes, the structure and parameters of the kinetic model of the gold cyanidation leaching system are different and need to be determined by appropriate methods [13]. The achievements of researchers in determining the gold leaching reaction mechanism provide a good guide for the establishment of the leaching system model in this paper.

The mechanism model can be established by analyzing the internal laws of the system and ignoring unimportant factors. Based on strict mathematical derivation or computer simulation, the mechanism model has a strong explanatory ability and can be extended to other similar processes [14]. Conversely, the data-driven model fully mines the relationship between process data and can reflect complex information that cannot be explained by the mechanism method. The data compensation model compensates for the output errors caused by the unmodeled dynamics of the system [15]. The data model has high precision but also has the disadvantage of poor generalization ability. The hybrid model uses a mechanism model to reflect the main dynamic characteristics of the system, which reduces the requirement of sample data and improves the generalization ability of the model [16]. The hybrid model has good local approximation performance and global dynamic performance. The hybrid model has the dual advantages of high precision of the data model and the strong generalization ability of the mechanism model [17]. The hybrid models commonly used in practical applications can be divided into two types, parallel and series. The relationships between the variables and structures of cascade leaching systems are complex [18]. Abundant mechanism knowledge and prior knowledge go into creating leaching systems, so it is more suitable to establish a parallel hybrid model of such systems [19]. The LSTM neural network has unique superiorities in process data mining and time-series prediction. The LSTM is an improvement of the recurrent neural network and has a time-recurrent structure [20]. Therefore, this paper will use the LSTM network to research the hybrid modeling of the cascade leaching industrial process.

The mechanism analysis and data-driven modeling of industrial systems are hot topics for researchers. Based on Merkel’s theory and mechanistic analytical techniques, a parallel hybrid model has been constructed for a mechanical draft counter flow wet-cooling tower by using least squares support vector machine, which can guide system energy-saving optimization [21]. By analyzing experiment data and mechanism knowledge, the choke finger system is modeled by a block-oriented model with a nonlinear dynamic input [22]. Based on process knowledge and real industrial data, an intelligent operational adjustment framework based on a hybrid Bayesian network (BN) has been established for the copper cleaner flotation process [23]. A new hybrid wind-speed forecasting method based on a back propagation (BP) neural network has been proposed to forecast daily average wind speed [24]. By combing ensemble just-in-time learning and moving window techniques, a data-driven semi-supervised weighted probability partial least squares regression (SWPPLSR) is developed to obtain the unmodeled dynamics in the mechanism model, and its validity has been verified by modeling a roller hearth kiln of ternary cathode material manufacturing [25]. By combining data assignment and the parameter estimation technique, a hybrid system consisting of a Piecewise Auto-Regressive eXogeneous (PWARX) structure has been developed for a rainfall-runoff system [26]. Hybrid modeling with a data-driven scheme has been proposed in multiscale material and process design, in which mechanistic models are used to describe the process-related principle [27]. For the polyethylene production process, a hybrid model with feedforward artificial neural networks (FANNs) has been developed by adding an empirical layer to the mechanistic model. The mechanistic model utilizes fundamental material and energy balances to predict important process conditions [28]. Based on empirical mode decomposition and back-propagation neural network, a hybrid water temperature prediction model has been proposed to reduce aquaculture risks and optimize the operation of water quality management [29]. A quantitative assessment hybrid model demonstrates a superior capability to predict the time evolution of different process variables using only the initial and process conditions [30]. According to the existing knowledge of the reaction mechanism and chemical principles, a simplified mechanism model of a leaching system can be established [31]. The unknown parameters involved in the non-convex mechanism model can be optimized and solved by intelligent methods [32]. The unmodeled dynamics and prediction errors in the mechanism model can be compensated by an error data model established by neural networks [33]. These hybrid modeling strategies provide a good reference and guidance for the hybrid modeling of cascade leaching systems.

In this paper, the theory of mechanism analysis modeling and the scheme of intelligent hybrid modeling are studied for hydrometallurgical gold cyanidation leaching systems. A typical gold cyanidation leaching system is introduced and designed. The chemical principle of the cyanidation leaching system is introduced in detail. The structure and model order of the process are determined by the mechanism analysis technique. The influence of related variables on the leaching system is also discussed. Based on reasonable assumptions and simplification, the dynamic mechanism model of single and cascade gold leaching systems is established. The adjusting parameters of the gold dissolution rate and cyanide ion consumption rate are estimated by using the particle swarm optimization algorithm based on a simulated annealing strategy. The LSTM network with sliding windows is designed to compensate for the prediction error of the mechanism model, and the intelligent hybrid model is established. Based on the experimental test, the effectiveness and merit of the proposed hybrid modeling algorithm are cross-verified by comparison with the existing methods. The accurate leaching process model can not only predict the leaching rate well but also provide the basis for the model-based predictive control technology and the development of a process operation monitoring system. Based on the latest data-driven modeling strategy using an LSTM neural network or a gated recurrent unit (GRU) neural network, the proposed hybrid modeling technique can accurately predict the leaching rate in the leaching process. The research ideas proposed in this paper can provide inspiration for the modeling of other complex systems. The rest of this paper is organized as below. The hydrometallurgical gold leaching system will be introduced and designed in Section 2. Section 3 includes the chemical principle behind the gold cyanidation leaching system. The mechanism modeling of a cascade leaching system will be given in Section 4. Parameter identification of the mechanism model will be given in Section 5. Hybrid modeling techniques based on LSTM networks will be presented in Section 6. The experiment test is presented in Section 7. Finally, some conclusions are drawn in Section 8.

2. Hydrometallurgical Gold Leaching System

In this section, the basic components of the hydrometallurgical leaching process will be described and analyzed in detail to facilitate the establishment of system mechanism models. Seven tank reactors in series are adopted to carry out cyanide reactions for the hydrometallurgical gold leaching system, as shown in Figure 1. A blower is used to fill each leaching tank with air. Sodium cyanide is added to the first, second, fourth, and sixth reaction tanks. The ore pulp containing gold is concentrated and stored in buffer tanks. The insoluble gold in the solid phase reacts with sodium cyanide and oxygen to form water-soluble gold cyanide complex ions. Compressed air is passed through the bottom of the leaching tank to provide dissolved oxygen for the reaction. Pneumatic agitation is used to prevent ore pulp accumulation. The reacted ore pulp flows into the subsequent leaching tank through the overflow. The cascade leaching system involves complex physical and chemical reactions, and different leaching tanks have different kinetic reaction rates. Ore pulp flow rate, ore pulp concentration, pH value, and dissolved oxygen concentration can be measured online. However, gold grade and cyanide ion concentration can only be measured offline. Low-frequency and high-frequency disturbances exist in the measurement process, which increases the difficulty of process modeling.

The continuous stirred leaching tank reactor (CSTR) is also called a Pachuca extractor and has been adopted in the leaching system [7], as shown in Figure 2. The CSTR is the key reaction equipment in the leaching process. It is not only the reaction vessel but also the power source of the leaching process. The bottom of the leaching tank is filled with compressed air into the central pipe, forming a large number of bubbles rising along the central pipe. The bubble in the central pipe makes the volume of ore pulp expand and the density decrease, which results in the pulp pressure in the central pipe being less than the pressure outside the central pipe. Under the action of the pressure difference between the inside and outside of the pipe, the ore pulp in the pipe moves upward and flows out from the upper end of the central pipe. The ore pulp outside the central pipe slowly flows down to the bottom, which forms a closed circulation of pulp flow. The bubbles that overflow from the ore pulp are discharged through the exhaust pipe at the top of the tank.

3. Chemical Principle of Gold Cyanidation Leaching System

In this section, the reaction principle of the hydrometallurgical leaching process will be introduced in order to understand the difficulties in, and key points of, modeling. The reaction of gold with sodium cyanide is electrochemical dissolution. The oxidation and dissolution of gold occur at the anode, while the depolarization of oxygen occurs at the cathode, as shown in Figure 3.

The major chemical reactions can be expressed as

$$2\mathrm{Au}+4\mathrm{NaCN}+{\mathrm{O}}_2+{2\mathrm{H}}_2\mathrm{O}\to {2\mathrm{NaAu}\left(\mathrm{CN}\right)}_2+2\mathrm{NaOH}+{\mathrm{H}}_2{\mathrm{O}}_2$$

(1)

Equation (1) can be expressed as two battery reactions. The anode region reaction can be expressed as

$$2\mathrm{Au}+{4\mathrm{CN}}^{-}-2\mathrm{e}\to {2\mathrm{Au}\left(\mathrm{CN}\right)}_2^{-}$$

(2)

The cathode region reaction can be expressed as

$${\mathrm{O}}_2+2{\mathrm{H}}_2\mathrm{O}+2\mathrm{e}\to {\mathrm{H}}_2{\mathrm{O}}_2+{2\mathrm{OH}}^{-}$$

(3)

Concentration polarization is the most important factor affecting the polarization of the cathode and anode. The concentration polarization of oxygen and cyanide ions can be determined by Fick’s law

$$\frac{\mathrm{d}({\mathrm{O}}_2)}{\mathrm{d}t}=\frac{D_{{\mathrm{O}}_2}}{\delta }{A}_1\{[{\mathrm{O}}_2]-{[{\mathrm{O}}_2]}_i\}$$

(4)

$$\frac{\mathrm{d}({\mathrm{CN}}^{-})}{\mathrm{d}t}=\frac{D_{{\mathrm{CN}}^{-}}}{\delta }{A}_2\{[{\mathrm{CN}}^{-}]-{[{\mathrm{CN}}^{-}]}_i\}$$

(5)

where $\frac{\mathrm{d}({\mathrm{O}}_2)}{\mathrm{d}t}$ and $\frac{\mathrm{d}({\mathrm{CN}}^{-})}{\mathrm{d}t}$ are the diffusion rates of ${\mathrm{O}}_2$ and ${\mathrm{CN}}^{-}$, respectively; $D_{{\mathrm{O}}_2}$ and $D_{{\mathrm{CN}}^{-}}$ are the diffusion coefficients of dissolved oxygen and cyanide ion, respectively; $[{\mathrm{O}}_2]$ and $[{\mathrm{CN}}^{-}]$ are the concentrations of ${\mathrm{O}}_2$ and ${\mathrm{CN}}^{-}$ in the solution, respectively; ${[{\mathrm{O}}_2]}_i$ and ${[{\mathrm{CN}}^{-}]}_i$ are the concentrations of ${\mathrm{O}}_2$ and ${\mathrm{CN}}^{-}$ at the interface, respectively.

The chemical reaction rate of ${\mathrm{O}}_2$ and ${\mathrm{CN}}^{-}$ at the metal interface is much higher than the diffusion rate, i.e., ${[{\mathrm{O}}_2]}_i\approx 0$, ${[{\mathrm{CN}}^{-}]}_i\approx 0$. Thus, Equations (4) and (5) can be simplified as

$$\frac{\mathrm{d}({\mathrm{O}}_2)}{\mathrm{d}t}=\frac{D_{{\mathrm{O}}_2}}{\delta }{A}_1[{\mathrm{O}}_2]$$

(6)

$$\frac{\mathrm{d}({\mathrm{CN}}^{-})}{\mathrm{d}t}=\frac{D_{{\mathrm{CN}}^{-}}}{\delta }{A}_2[{\mathrm{CN}}^{-}]$$

(7)

The dissolution rate of gold is twice that of oxygen consumption and half that of cyanide consumption. Therefore, the dissolution rate of gold $v$ can be expressed as

$$v=\frac{1}{2}\frac{\mathrm{d}({\mathrm{CN}}^{-})}{\mathrm{d}t}=\frac{1}{2}\frac{D_{{\mathrm{CN}}^{-}}}{\delta }{A}_2[{\mathrm{CN}}^{-}]$$

(8)

or

$$v=2\frac{\mathrm{d}({\mathrm{O}}_2)}{\mathrm{d}t}=2\frac{D_{{\mathrm{O}}_2}}{\delta }{A}_1[{\mathrm{O}}_2]$$

(9)

When the above equation reaches equilibrium, it becomes

$$\frac{1}{2}\frac{D_{{\mathrm{CN}}^{-}}}{\delta }{A}_2[{\mathrm{CN}}^{-}]=2\frac{D_{{\mathrm{O}}_2}}{\delta }{A}_1[{\mathrm{O}}_2]$$

(10)

The total surface area of metal in contact with water can be expressed as $A=A_1+A_2$. The dissolution rate of gold can be expressed as

$$v=\frac{2A{D}_{{\mathrm{CN}}^{-}}{D}_{{\mathrm{O}}_2}[{\mathrm{CN}}^{-}][{\mathrm{O}}_2]}{\delta \{D_{{\mathrm{CN}}^{-}}[{\mathrm{CN}}^{-}]+4D_{{\mathrm{O}}_2}[{\mathrm{O}}_2]\}}$$

(11)

In the case of very low cyanide concentration and high dissolved oxygen concentration, Equation (11) can be simplified as

$$v=\frac{A{D}_{{\mathrm{CN}}^{-}}[{\mathrm{CN}}^{-}]}{2\delta }=k_1[{\mathrm{CN}}^{-}]$$

(12)

The dissolution rate of gold is mainly determined by the cyanide ion concentration.

In the case of high cyanide concentration and low dissolved oxygen concentration, Equation (11) can be simplified as

$$v=\frac{2A{D}_{{\mathrm{O}}_2}[{\mathrm{O}}_2]}{\delta }=k_2[{\mathrm{O}}_2]$$

(13)

The dissolution rate of gold mainly depends on the concentration of dissolved oxygen.

4. Mechanism Modeling of Cascade Leaching System

On the basis of understanding the structure, kinetic principle, and chemical reaction principle of hydrometallurgy, the mechanism model of the leaching process can be established by using the mechanism knowledge and partial differential equations. The mechanism model of the leaching system mainly consists of the mass conservation equations of gold in the ore, gold in the liquid, and cyanide ion in the liquid, as well as the corresponding kinetic reaction rate models of gold and cyanide ion. The conservation equation of gold and cyanide in single-tank leaching systems can be expressed as

$$\frac{Q_s}{M_s}(C_{s0}-C_s)-r_{Au}=\frac{d{C}_s}{dt}$$

(14)

$$\frac{Q_l}{M_l}(C_{l0}-C_l)+\frac{M_s}{M_l}{r}_{Au}=\frac{d{C}_l}{dt}$$

(15)

$$\frac{Q_l}{M_l}(C_{CN0}-C_{CN})+\frac{Q_{CN}}{M_l}-r_{CN}=\frac{d{C}_{CN}}{dt}$$

(16)

where $Q_s$ is the solid flow rate of ore pulp; $Q_l$ is the liquid flow rate of ore pulp; $C_{s0}$ and $C_{l0}$ are the initial gold concentration of the solid and liquid, respectively; $M_s$ is the mass of solid ore pulp in the leaching tank; $M_l$ is the mass of liquid ore pulp in the leaching tank; $C_{CN0}$ is the initial cyanide ion concentration of the liquid; $C_s$ and $C_l$ are the concentration of solid and liquid gold in the leaching tank, respectively; $C_{CN}$ is the liquid cyanide ion concentration; $Q_{CN}$ is the addition flow rate of sodium cyanide; $r_{Au}$ is the gold dissolution rate; $r_{CN}$ is the consumption rate of cyanide ion.

The gold dissolution rate $r_{Au}$ is a nonlinear function of the average diameter of ore particles $\overline{d}$, dissolved oxygen concentration $C_O$, liquid hydrogen ion concentration $C_{CN}$, and solid gold concentration $C_s$, and can be expressed as

$$r_{Au}=f(\overline{d},C_O,C_{CN},C_s)$$

(17)

The consumption rate of cyanide ions $r_{CN}$ is a nonlinear function of the $\overline{d}$ and $C_{CN}$, and can be expressed by

$$r_{CN}=g(\overline{d},C_{CN})$$

(18)

The solid and liquid mass of ore pulp satisfies the conservation relation. The relationship between the solid flow rate and liquid flow rate of ore pulp can be expressed as

$$Q_l=Q_s(\frac{1}{C_w}-1)$$

(19)

where $C_w$ is the ore pulp concentration.

The average residence time of the chemical reaction can be computed by

$$\tau =\frac{V}{\frac{Q_s}{{\rho }_s}+\frac{Q_l}{{\rho }_l}}\times 1000$$

(20)

where ${\rho }_s$ and ${\rho }_l$ are the solid density and liquid density of ore pulp, respectively; and $V$ is the volume of the leaching tank.

The residence mass of solid and liquid in the leaching tank can be expressed as

$$M_s=Q_s\times \tau$$

(21)

$$M_l=Q_l\times \tau$$

(22)

The leaching rate of gold can be computed by

$$y_1=\frac{C_{s0}-C_s}{C_{s0}}\times 100\%$$

(23)

The mechanism model of the leaching system is a nonlinear function and can be expressed as

$$y_1=F(C_{s0},C_{CN0},M_s,M_l,Q_s,Q_l,Q_{CN},\overline{d},C_w,V_i,{\rho }_s,{\rho }_l,C_o,r_{Au},r_{CN})$$

(24)

The leaching tank is serially connected by ore pulp overflow. The effective volume of each leaching tank is the same. It is assumed that the reactants in the leaching tank can be mixed adequately. Material isolation is neglected. The resistance of ore pulp is constant. The input and output schematic diagram of the cascade leaching system is shown in Figure 4. For the stable operation of the leaching industrial process, the mass conservation principle satisfies $Q_s=Q_{s0}$ and $Q_l=Q_{l0}$. The total leaching rate of the cascaded gold cyanidation leaching system can be expressed as

$$y=\frac{C_{s0}-C_{sN}}{C_{s0}}\times 100\%$$

(25)

where $C_{sN}$ is the solid gold concentration of the $Nth$ leaching tank.

The addition of sodium cyanide to each leaching tank is an operational variable. The model structure of each leaching tank in the cascade leaching system is the same, but the kinetic reaction velocity model parameters of different leaching tanks need to be estimated. If every leaching tank model adopts the same parameter, it will cause model mismatch and reduce the prediction accuracy of the leaching rate. Therefore, the unknown parameters in the model should be identified by the actual process data. There are many unknown parameters in the leaching process mechanism model. It is difficult for conventional optimization algorithms to obtain a satisfactory global optimal solution. It is more likely to obtain the optimal operating variable by intelligent optimization technology.

5. Parameter Estimation Based on PSO Algorithm

The gold dissolution rate $r_{Au}=f(\overline{d},C_O,C_{CN},C_s)$ and the consumption rate of cyanide ions $r_{CN}=g(\overline{d},C_{CN})$ cannot be measured using the detection equipment for online measurement. The kinetic reaction models estimated by the existing literature based on the least square method are

$$r_{Au}=(1.13\times {10}^{-3}-4.37\times {10}^{-11}{\overline{d}}^{2.93}){(C_s-C_{s\infty }(\overline{d}))}^{2.13}{C}_{CN}{}^{0.961}{C}_O{}^{0.228}$$

(26)

$$r_{CN}=(\frac{1.69\times {10}^{-8}}{{\overline{d}}^{0.547}-6.40})C_{CN}{}^{3.71}$$

(27)

where the residual gold concentration $C_{s\infty }$ in the solid is a function of $\overline{d}$ and expressed as

$$C_{s\infty }(\overline{d})=0.357(1-1.49e^{-1.76\times {10}^{-2}\overline{d}})$$

(28)

The differential equations of gold concentration and cyanide ion concentration can be expressed by

$$d{C}_s=\frac{Q_s}{M_s}\ast (C_s-C_{s\infty })-(1.13\ast {10}^{-3}-4.37\ast {10}^{-11}{\overline{d}}^{2.93}){(C_s-C_{s\infty })}^{2.13}{C}_{cn}^{0.961}{C}_0^{0.228}$$

(29)

$$d{C}_{cn}=\frac{Q_l}{M_l}\ast (C_{cn0}-C_{cn})+\frac{Q_{cn}}{M_l}-(\frac{1.69\ast {10}^{-8}}{d^{0.547}-6.40})C_{cn}{}^{3.71}$$

(30)

Due to different production environments and equipment parameters, the empirical model obtained from existing literature cannot be directly applied to the leaching system in this paper. However, it provides a good reference for constructing a leaching system model. The key adjusting parameters $u_{Au}$ and $u_{CN}$ in the gold dissolution rate and cyanide ion consumption rate models can be estimated based on the existing model structure. The parameter identification objective function is composed of the error norm between the actual leaching rate obtained by assay and the leaching rate predicted by the model, i.e.,

$$J(u_{Au},u_{CN})={\displaystyle \sum _{i=1}^N{‖y_i-{\widehat{y}}_i‖}_2^2}={\displaystyle \sum _{i=1}^N{‖y_i-F_i(r_{Au},r_{CN})‖}_2^2}$$

(31)

The reaction velocity model is a nonlinear function of adjustment parameters and the sampling data may contain measurement noise. The conventional least square optimization or Newton optimization methods cannot easily obtain satisfactory optimization results [34]. Therefore, intelligent methods should be used to optimize the solution. Among many intelligent optimization methods, the particle swarm optimization (PSO) algorithm has the advantages of small computation and strong global convergence, which is suitable for optimizing large complex process models. The intelligent PSO algorithm will be used to solve the objective function. The PSO algorithm based on a simulated annealing strategy has the ability of probability mutation and can avoid falling into the local extremum. The velocity update and position update formulas of a particle can be expressed as

$$v_{i,j}(t+1)=\phi \left\{v_{i,j}\right(t)+c_1\cdot r_1\cdot [p_{i,j}-x_{i,j}(t\left)\right]+c_2\cdot r_2\cdot [p_g{}_{,j}-x_{i,j}\left(t\left)\right]\right\}$$

(32)

$$x_{i,j}(t+1)=x_{i,j}\left(t\right)+v_{i,j}(t+1),\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}j=1,2,\dots ,d$$

(33)

where $d$ is the dimension of the particle; $r_1$ and $r_2$ are random numbers between (0, 1); $c_1$ and $c_2$ are positive learning factors (or acceleration coefficients); $\phi =\frac{2}{\left|2-C-\sqrt{C^2-4C}\right|}$ is the velocity compression factor, and $C=c_1+c_2$.

The particle fitness function constructed by the standard PSO algorithm has some limitations. Therefore, the simulated annealing algorithm is used to construct the fitness function of particles to improve the global convergence and convergence speed of the optimization algorithm. The initial temperature of the simulated annealing algorithm is taken as $t_0=f(p_g)/\ln 5$. The annealing mode is designed as $t_{k+1}=\lambda t_k$, where $\lambda \in (0,1)$ is the annealing constant. The fitness of a particle $p_i$ can be computed by

$$F_{p_i}=\frac{e^{-[f(p_i)-f(p_g)]/t}}{{\displaystyle \sum _{j=1}^N{e}^{-[f(p_{i,j})-f(p_g)]/t}}}$$

(34)

where $p_g$ is the group optimal solution and $f$ is the fitness function. The improved particle swarm optimization algorithm is more suitable for optimizing the leaching process and more likely to obtain the optimal operational variables. In the actual industrial system application, the undetermined parameters involved in the optimization algorithm can be taken according to the following criteria [35]. The number of particles is usually set to between 50 and 100. The learning factor $c_1$ and $c_2$ are usually set in the interval between 1.3 and 2.8. The value of the annealing constant is usually chosen between 0.4 and 0.6. The number of iterations can be determined according to the requirements of the industrial process and is usually set to between 500 and 1500.

6. Compensation Modeling Based on LSTM Neural Network

The mechanism model parameters of the system can be obtained by the optimization technique. However, due to the simplification in the modeling process, the prediction of the mechanism model will have a large error. Intelligent methods can be used to train the prediction errors to obtain a compensation model. The mechanism model of the leaching system is derived by rigorous mathematical operation. It can well describe the main dynamic trend in the leaching rate. However, the mechanism model cannot accurately describe all the characteristics and complex variable relationships of the actual process. When identifying the unknown parameters of the dynamic model of the leaching system, the errors caused by model structure approximation and ideal assumptions are not considered. The variation in production working conditions and random disturbances will lead to the mismatching of mechanism model parameters, which will cause a large prediction error. By mining the relationship between process data, data-driven technology can compensate for the undescribed information of the mechanism modeling method. A data-driven model is used to compensate for the output error caused by the unmodeled dynamic of the leaching system, and the hybrid model can be established. Due to the bad operating environment of the leaching process, there may be no abundant good data for network training. Therefore, the general neural network methods cannot easily obtain good error compensation results. Considering that the LSTM neural network can assign different weights to memory information by using the forget gate, this operation can weaken the adverse effects of data dispersion and dead zone cutoff on network learning. Considering the characteristics of the gold cyanidation leaching system, the parallel hybrid model is constructed by utilizing the LSTM neural network and the mechanism model, as shown in Figure 5. The data model based on an LSTM neural network compensates for the mechanism model output error. The LSTM neural network uses the prediction error $y_{error}=y-y_m$ between the mechanism model output $y_m$ and the system output $y$ to train the network and obtain the error compensation output $y_c$. The hybrid model output can be expressed as $y_h=y_c+y_m$.

The gating state of the LSTM neural network consists of a sigmoid layer and a pointwise multiplication operation. The Sigmoid layer normalizes the input signal to determine the retained information. The pointwise multiplication operation is to multiply the input data dimension-by-dimension. This operation can protect the original dimension of the data from being destroyed and ensure the composability of the output results of each gate. The LSTM neural network uses input, forgetting, and output gates to protect and control the cell state and transmission state. The input gate normalizes the information of the previous hidden state and the current input information. The forgetting gate solves the gradient attenuation problem by assigning different weights to the control information and transferring the past information to the present moment. The output gate outputs the current hidden state and passes it to the next layer. The LSTM neural network gate structure is shown in Figure 6, where $X_t$ is the batch data with $n$ samples and $x$ dimensions at time $t$; $h_{t-1}$ is the hidden state at the previous moment; $W$ is the learning weight parameter; $\odot$ represents the operation of multiplying by the corresponding element, i.e., the Hadamard product; $\sigma$ is the sigmoid function with a range of [0, 1]; and $\tanh$ is a hyperbolic tangent function with a range [−1, 1].

The hidden layers of the LSTM network not only transmit feedforward information but also transmit internal feedback information. The LSTM network can take into account not only current information but also historical useful information. The LSTM network controls the transmission state through the gating state so that the network can remember the information which needs to be remembered for a long time and forget the unimportant information. The LSTM uses constant error flow to solve the problem of gradient disappearance and gradient explosion in time-series data training. The information propagation of the LSTM neural network mainly consists of forward propagation and error backward propagation. Through forward propagation and error backward propagation, each gating unit obtains the optimal weight matrix, bias matrix, and error information proportion according to the cost function. The output layer outputs the training results, which mainly consist of matrix linear transformation and normalization function. The schematic diagram of the single-layer LSTM neural network is shown in Figure 7.

The number of hidden layer nodes has a great influence on the training and compensation of an LSTM neural network. If the number of hidden nodes is too few, the learning and information processing capability of the network will be reduced. If the number of hidden layer nodes is too many, it will more easily fall into the local extremum, increasing the complexity of the network structure and reducing the speed of network learning. The optimal number of hidden layer nodes $s$ can be determined by

$$\left\{\begin{array}{c}s<2n+1\\ s\le \sqrt{0.43mn+0.12nn+2.54m+0.77n+0.35}+0.51\\ s={\log }_{2}N\end{array}\right.$$

(35)

where $N$ is the number of modes; $n$ is the number of nodes at the input layer; and $m$ is the number of nodes at the output layer. The network node parameters used in this study are given in

Table 1. Network node parameters.

$\mathit{s}$	$\mathit{n}$	$\mathit{m}$
60	50	40

.

In general, the number of hidden layer nodes can be specified as one hundred, which can satisfy the accuracy requirements. To prevent gradient explosions, the gradient threshold can be taken as one. Network convergence can be promoted by reducing the learning rate when network training reaches half times. The cost function of the network is constructed by the mean square error (MES) approach. The optimization method ‘Adam’ is used to solve the problem [36]. Adam can ensure that each network weight maintains the same learning rate and can be adjusted separately. The adaptive learning rate of different parameters is calculated by the first and second moments of the gradient. Adam is the combination of the root mean square propagation and adaptive momentum algorithms, and its weight update formula is

$$w_t=w_{t-1}-\alpha \frac{{\widehat{m}}_t}{\sqrt{{\widehat{v}}_t}+\epsilon }$$

(36)

where $t$ is the number of iterations; $\alpha$ is the learning rate and can be taken as $\alpha =0.001$; $\epsilon$ is a small constant to increase data stability and can be taken as $\epsilon ={10}^{-8}$; $m_t$ is the exponential moving mean of the gradient, obtained from the first moment of the gradient; and $v_t$ is the square gradient, obtained by the second moment of the gradient. The update formula $m_t$ and $v_t$ are

$$m_t={\beta }_1{m}_{t-1}+(1-{\beta }_1)g_t$$

(37)

$$v_t={\beta }_2{v}_{t-1}+(1-{\beta }_2)g_t^2$$

(38)

where $g_t$ is the gradient of the objective function; ${\widehat{m}}_t$ is a modification of $m_t$; ${\widehat{v}}_t$ is a modification of $v_t$.

$${\widehat{m}}_t=\frac{m_t}{1-{\beta }_2^t}$$

(39)

$${\widehat{v}}_t=\frac{v_t}{1-{\beta }_2^t}$$

(40)

where ${\beta }_1$ and ${\beta }_2$ are the constants that control exponential decay. In this paper, the optimization method parameters of Adam are those given in

Table 2. Parameters of Adam.

Max Epoch	Gradient Threshold	Initial Learn Rate	Learn Rate Drop Period
500	0.85	0.0025	250
Learn rate drop factor	Adjustment parameter ${\beta }_1$	Adjustment parameter ${\beta }_2$	Small constant $\epsilon$
0.1	0.9	0.999	10−8

.

The prediction performance of the time series can be improved by improving data training correspondence. The corresponding step size of historical data at each point is increased and an autoregressive form is used to improve the prediction performance. A sliding window prediction algorithm is proposed based on the LSTM neural network. The prediction of a single point data $\{u_i,y_i\}$ corresponding to $y_i$ is modified to use $p$ adjacent historical data $\{u_i,\cdots ,u_{i-p},y_i,\cdots ,y_{i-p}\}$ corresponding to $y_i$ for prediction output. It should be noted that the sampled data of the actual system may contain noise and disturbance, which need to be cleaned and processed. It may be necessary to design specific test experiments to obtain the data that can fully reflect the system performance to improve the performance of the training network and the model prediction precision. The flowchart of the proposed hybrid intelligent modeling strategy is shown in Figure 8.

7. Experimental Verification

In order to verify the prediction performance of the proposed mechanism model, parameter optimization method, and hybrid model, a cascade leaching system is used for practical application tests. The variables and parameter values involved in the mechanism model are shown in

Table 3. The variables and value in the mechanistic model.

Variables	Value	Unit
$C_w$	39	%
$C_{w0}$	0	mg/kg
$C_s$	5	mg/kg
$C_l$	0	mg/kg
$C_O$	7	mg/kg
$\overline{d}$	80	μm
$C_{CN0}$	250	mg/kg
$Q_s$	2500	kg/h
$V$	68	m3
${\rho }_s$	2.8	g/cm3
${\rho }_l$	1	g/cm3

. By sampling the data between the input sodium cyanide and the output leaching rate, as shown in Figure 9, the unknown parameters in the mechanism model can be effectively optimized and determined.

In order to identify the mechanism model’s unknown parameters, the PSO algorithm is used to solve the objective function. The dimension of the particle is taken as $d=14$. The number of particles is taken as $N=50$. The number of iterations is taken as $M=500$. The positive learning factors are taken as $c_1=2.05$ and $c_2=2.05$. The annealing constant is taken as $\lambda =0.5$. The key adjustment parameters obtained by the identification algorithm are shown in

Table 4. Estimation of gold dissolution rate and cyanide ion consumption rate (N# means the Nth Reactor).

Parameter	1# Reactor	2# Reactor	3# Reactor	4# Reactor	5# Reactor	6# Reactor	7# Reactor
$u_{Au}$	3.12	3.07	3.32	3.71	3.09	3.42	4.01
$u_{CN}$	1.16	1.02	1.43	2.22	2.65	1.49	2.42

. The mechanism model can describe the main dynamic characteristics of the cascade leaching system output, as shown in Figure 10. Due to some simplifications in the process of establishing the mechanism model and the noise in the sampled data, there is a large error between the prediction output of the mechanism model and the sampling output. To improve the prediction accuracy of the model, the LSTM neural network will be used to compensate for the output error.

The 3000 data samples are divided into two parts. The first part contains 2000 data points and is applied to train the network. The second part contains the remaining 1000 data points and is used to test the network. Different adjacent point data are selected to form a sequential recursive relationship for estimating the output error. Different sliding window lengths ($p$) and numbers of hidden layers ($N_l$) are taken to construct different neural networks. For comparing the prediction performance of the proposed method, the hybrid model based on the BP neural network in the reference [37] is also tested to validate experimental data. In addition, based on the BP neural network and GRU neural network, the data-driven models without using system mechanism model information are also tested to validate experimental data respectively [38]. In order to quantitatively evaluate the prediction accuracy of different algorithms, the prediction error can be computed by $y_{error}=y-y_h$. The mean of the prediction error is computed by $y_{\mathrm{mean}}=\mathrm{mean}({\displaystyle \sum _{i=1}^{\mathrm{L}}y(i)-y_h(i)})$. The variance of the prediction error is computed by $y_{\mathrm{mean}}=\mathrm{var}({\displaystyle \sum _{i=1}^{\mathrm{L}}y(i)-y_h(i)})$. The output prediction results by different methods are shown in Figure 11 and Figure 12. The means and variances of the prediction errors by using different data-driven modeling methods are shown in

Table 5. Mean and variance of prediction errors by using different data modeling.

	Mechanistic Model	Data Model (LSTM)	Data Model (BP)	Data Model (GRU)
Mean	0.0021	0.0033	0.0051	0.0013
Variance	$2.366\times {10}^{-6}$	$5.813\times {10}^{-6}$	$1.804\times {10}^{-5}$	$5.6\times {10}^{-7}$

. The means and variances of the prediction errors by using different hybrid modeling methods are shown in

Table 6. Mean and variance of prediction errors by using different hybrid modeling.

	Hybrid Model (BP)	Hybrid Model (LSTM p = 1)	Hybrid Model (LSTM N = 3 p = 1)	Hybrid Model (LSTM p = 10)	Hybrid Model (LSTM N = 3 p = 5)
Mean	0.0011	0.0029	0.0017	0.0024	0.0005
Variance	$5.4\times {10}^{-7}$	$3.331\times {10}^{-6}$	$1.452\times {10}^{-6}$	$2.669\times {10}^{-6}$	$2.83\times {10}^{-7}$

. Note that to more clearly display the input and output data and the estimates of the different leaching process models, only a portion of the data is shown in these figures. The X-axis represents the number of samples taken by a discrete computer sampling system. If the $k$ is multiplied by the sampling interval, it will represent the system sampling time. But when computing the prediction error, all the sampled data is used. The cross-validation results show that the hybrid models have better prediction results than the mechanism model and data models. The compensation error based on the LSTM neural network is smaller than that of the BP neural network and the GRU neural network. For signals with fast amplitude changes, the multilayer LSTM neural network has better prediction results. Since the BP neural network has no time-series operation, the BP network only trains the input data at the current moment, without considering the useful information of the input data at the previous moment. The BP network uses the gradient descent method to train the network. The weights of the BP neural network easily obtain local minima, which causes the termination of the training without obtaining the global optimal solution. Since the discrete distribution of the output data is large, it is difficult to train the BP neural network effectively. The data-driven models established based on the BP network and the LSTM network have poor extrapolation adaptability and large output prediction error. Since the forget gate of the LSTM neural network can assign different weights to memory information, the adverse effects of data dispersion and dead zone cutoff on network learning are weakened, so the hybrid model with a sliding window based on the LSTM neural network has a smaller estimation variance and higher estimation stability. It can be seen that, as the number of hidden layers of the LSTM increases, the error compensation effect is better. As the sliding window length increases, the compensation effect also becomes better. However, when the sliding window length increases to a certain amount, the compensation effect will be reduced due to the transitional memory of the previous data. The hybrid model based on a three-layer LSTM neural network with sliding window length ($p=5$) has achieved a good compensation effect for the cascade leaching system. We can see that the cascade leaching process is a complex system of gold hydrometallurgy, involving many links, equipment, and operating variables. The hybrid modeling technique combines the mechanism analysis technique, intelligent optimization technique, and neural network data-driven modeling technique. Many of the latest algorithms are used in the experimental comparison test.

8. Conclusions

A hydrometallurgical leaching system with the serial connection of several leaching tanks is characterized by strong coupling, nonlinearity, and different reaction rates. According to the principles of physics and chemical reactions and the principles of mass conservation, a mechanism model has been established to reflect the dynamic characteristics of the cascade leaching system. The mechanism model mainly consists of the gold conservation equation, cyanide ion conservation equation, and the related kinetic reaction velocity equation. The reaction rate model parameters are not the same due to the different working environments of leaching tanks. The particle swarm optimization algorithm based on a simulated annealing strategy has been used to estimate the key adjusting parameters in the gold dissolution rate model and the cyanide ion consumption rate model. There are some ideal assumptions and simplifications in the process of mechanism modeling. The variation in production conditions and random disturbances can cause parameter mismatch in the mechanism model and large prediction errors. The LSTM neural network is used to mine complex relationships between process data. The output errors caused by the unmodeled dynamics of the mechanism model are trained and compensated using the LSTM approach. An intelligent hybrid model of the cascade leaching system has been established. The hybrid model of the leaching system is convenient to analyze the dynamic characteristics and understand the influence of input variables on the leaching rate. The hybrid model facilitates designing controllers and monitoring process operation, and also provides a scientific basis for new process development. Finally, the effectiveness and superiority of the proposed hybrid modeling algorithm are verified by comparison with existing methods based on the data sampled from the cascade leaching industrial process. The proposed modeling techniques and frameworks can also be optimized using other intelligent optimization algorithms with better performance and the latest data-driven modeling techniques. Note that when the proposed modeling technique is extended to other industrial systems, it will be necessary to establish a mechanism model reflecting the main dynamics of the system, and sufficient data training prediction errors are also required in data compensation modeling. Otherwise, the establishment model will contain prediction errors.