Soft Sensors for Industrial Processes Using Multi-Step-Ahead Hankel Dynamic Mode Decomposition with Control

Abstract

In this paper, a novel data-driven approach for the development of soft sensors (SSs) for multi-step-ahead prediction of industrial process variables is proposed. This method is based on the recent developments in Koopman operator theory and dynamic mode decomposition (DMD). It is derived from Hankel DMD with control (HDMDc) to deal with highly nonlinear dynamics using augmented linear models, exploiting input and output regressors. The proposed multi-step-ahead HDMDc (MSA-HDMDc) is designed to perform multi-step prediction and capture complex dynamics with a linear approximation for a highly nonlinear system. This enables the construction of SSs capable of estimating the output of a process over a long period of time and/or using the developed SSs for model predictive control purposes. Hyperparameter tuning and model order reduction are specifically designed to perform multi-step-ahead predictions. Two real-world case studies consisting of a sulfur recovery unit and a debutanizer column, which are widely used as benchmarks in the SS field, are used to validate the proposed methodology. Data covering multiple system operating points are used for identification. The proposed MSA-HDMDc outperforms currently adopted methods in the SSs domain, such as autoregressive models with exogenous inputs and finite impulse response models, and proves to be robust to the variability of systems operating points.

1. Introduction

Soft sensors (SSs) are widely exploited for monitoring and controlling industrial processes where real-time estimation of variables is essential. SSs provide mathematical solutions for estimating and predicting hard-to-measure variables (i.e., quality features) using easy-to-measure variables (i.e., quantity features) [1,2].

In automation and control strategies, SSs are applied to address the issues related to delays in measurements due to time-consuming laboratory analysis in the feedback loop. This requires that the designed SSs are capable of multi-step-ahead prediction. Applications are ubiquitous in the process industry: refineries [3], chemical plants [4], power plants [5], food processing [6], polymerization processes [7] or wastewater treatment systems [8]. Industrial processes are often highly nonlinear, suffer from intrinsic dynamic dependencies between input and output variables and may exhibit transients, intermittent phenomena and continuous spectra.

The main approach to identifying such complex nonlinear dynamical systems originates with Poincare’s studies and works on the geometry of subspaces of local linearizations around fixed points, periodic orbits and more general attractors [9]. This methodology has a deep theoretical foundation, such as the Hartman–Grobman theorem, which determines when and where it is possible to approximate a nonlinear system with linear dynamics.

On the one hand, such a geometric perspective enables the application of simple quantitative, locally linear models, such as autoregressive (ARX), principal component regression (PCR) and partial least-square regression (PLSR) models [1], and proper orthogonal decomposition [10], as well as the composition of multiple linear systems as components of more complex modeling techniques [11,12]. In this scenario, the rich linear analytical framework can be used around such operating points and is therefore suitable for linear control strategies. On the other hand, the global analysis remains qualitative and is based on computational analysis, which is not suitable for predicting, estimating and controlling nonlinear systems far from fixed points and periodic orbits. Moreover, due to the complex theoretical environment, data-driven approaches are often used for SSs to support both linear and nonlinear methods [12,13,14,15,16,17]. In this methodological and application scenario, the Koopman operator [18] can provide a theoretical tool for obtaining a global linear representation that is valid for nonlinear systems, even far from fixed points and periodic orbits. A main motivation for the adoption of the Koopman framework is the possibility to simplify the dynamics by the eigenvalue decomposition of the Koopman operator [19] and thus to represent a nonlinear dynamical system globally by an infinite-dimensional linear operator. It uses a Hilbert space of observable functions related to the state of the system to describe the space of all possible measurement state functions. It is linear and its spectral decomposition fully characterizes the behavior of a nonlinear system, without a direct relation to the operating points of the system.

The application of such a powerful tool to industrial problems, namely to obtain a finite-dimensional approximation of the Koopman operator, is a challenge of recent research [20]. Moreover, since the closed form of the Koopman operator is not always obtainable [21], data-driven algorithms are needed. Koopman mode decomposition can be performed using data-driven approaches such as dynamic mode decomposition (DMD) [22].

Applications of DMD can be found in the literature in fluid dynamics [23,24], epidemiology [25], neuroscience [26], plasma physics [27,28], robotics [29], power grid instabilities [30] and renewable energy prediction [31]. DMD represents a method for approximating the Koopman operator that provides a best-fit linear model for one-step-ahead prediction. Such an approximation might not be rich enough to describe nonlinear dynamics. To overcome this limitation and apply DMD to nonlinear industrial processes, it is possible to extend DMD with different strategies based on either nonlinear functions or delayed measurements. Extended DMD [32] and sparse identification nonlinear dynamics (SINDy) [33] belong to the first category. The second category, on which this paper focuses, is based on the use of delayed state variables obtained by the Hankel operator. Such approaches overcome the limitations of standard DMD, which cannot accurately describe systems where the number of variables is smaller than the spectral complexity. Therefore, in Hankel-based DMD, the number of variables is increased by considering time-delayed vectors in addition to the current state vector. There are a few variants of the Hankel approach for the Koopman operator: Hankel DMD (HDMD) [34], high-order DMD (HODMD) [35] and Hankel alternative view of Koopman (HAVOK) [36]. Thanks to the state variable augmentation, these methods are more robust and accurate than classical DMD and are therefore suitable for the identification of nonlinear dynamical systems and offer robust noise filtering [37,38,39].

With the aim of applying DMD-based approaches to industrial processes with exogenous inputs, the DMD with control (DMDc) approach has been proposed in the literature [40]. This is a modified version of DMD that considers both system measurements and exogenous control inputs to identify input–output relationships and the underlying dynamics. Hankel DMD with control (HDMDc) has recently been introduced to handle both time-delayed state variables and control inputs [41,42]. Applications of Koopman theory for quasiperiodically driven systems have also been presented in [43].

In this paper, we propose an extension of the HDMDc approach to multi-step-ahead (MSA) prediction (hereafter referred to as MSA-HDMDc) in the SSs design domain. This solution leverages and exploits the intrinsic HDMDc capability for forecasting [42], operating and, in addition, iterative multi-step-ahead model optimization and output prediction. This makes SSs suitable for the application of model-based online control strategies that are widely used in industrial processes, such as model predictive control (MPC) [44,45,46,47]. To evaluate the potential of the MSA-HDMDc approach in industry, and to test the robustness and reliability of the method in real-world industrial environments considering noise and uncertainty [48,49], two widely used benchmarks in the SS field are considered: the sulfur recovery unit (SRU) [1,12,17,50,51] and the debutanizer column (DC) [52,53]. Multi-step-ahead prediction of the output variables is evaluated on such datasets, and a comparison with currently adopted linear model identification techniques is performed.

The main outcomes of this work are summarized here:

• The MSA-HDMDc procedure is developed and applied in the field of soft sensors for industrial applications to perform multi-step-ahead prediction;
• A model order reduction strategy with a two-step optimization is developed to reduce the computational complexity of the identified model;
• A global linear model for a nonlinear process is identified so that model analysis and control cover multiple working points of the process.

The article is structured as follows. Section 2 describes the theoretical background, the equations and the algorithmic procedure behind the models used. The two industrial case studies, the SRU and the DC, are presented in Section 3. Section 4 and Section 5 present the simulation results for the implemented models, including the design and optimization of hyperparameters and model order reduction for both case studies. Comparisons with baseline methods are also reported in this section. Finally, conclusions are drawn in Section 6.

2. Theory Fundamentals

There are different approaches for the data-driven identification of dynamic processes. When dealing with linear models, the widely used model classes in the SS field are the AutoRegressive with eXogenous Input (ARX) and the finite impulse response (FIR) filter [1,2]. They are considered here for comparison with the new MSA-HDMDc, which has been evaluated in industrial applications. This section describes the theoretical and mathematical foundations for both the baseline and the HDMDc models. The algorithmic procedure of the MSA-HDMDc is also presented.

2.1. Baseline Methods

An ARX model set is determined by two polynomials whose degrees are $n_a$ and $n_b$, respectively:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \begin{array}{c}\hfill A(z^{-1},\theta )=1+a_1{z}^{-1}+a_2{z}^{-2}+\cdots +a_{n_a}{z}^{-n_a}\\ \hfill B(z^{-1},\theta )=b_0+b_1{z}^{-1}+b_2{z}^{-2}+\cdots +b_{n_b}{z}^{-n_b}\end{array}\end{array}\end{array}$$

(1)

where $z^{-1}$ represents the time delay operator and $\theta$ is the set of parameters:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \theta :={[a_1{a}_2\cdots a_{n_a}{b}_1{b}_2\cdots b_{n_b}]}^T\end{array}\end{array}$$

(2)

The acronym ARX can be explained in the model equation form for the calculation of $y\left(t\right)$, the predicted output at the time instant t:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill y\left(t\right)={\displaystyle \frac{B(z^{-1},\theta )}{A(z^{-1},\theta )}}u\left(t\right)+{\displaystyle \frac{1}{A(z^{-1},\theta )}}e\left(t\right)\end{array}\end{array}$$

(3)

or equivalently:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill A(z^{-1},\theta )y\left(t\right)=B(z^{-1},\theta )u\left(t\right)+e\left(t\right)\end{array}\end{array}$$

(4)

where $e\left(t\right)$ is a zero-mean white noise process and $u\left(t\right)$ is the exogenous input vector.

AR refers to the autoregressive part $A(z^{-1},\theta )y\left(t\right)$ in the model, while X refers to the exogenous term $B(z^{-1},\theta )u\left(t\right)$. The model set is completely determined once the integers $n_a$, $n_b$ and the parameter set $\theta$ have been specified.

A more general expression including an input/output delay is represented by:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill y\left(t\right)=z^{-n_k}{\displaystyle \frac{B(z^{-1},\theta )}{A(z^{-1},\theta )}}u\left(t\right)+{\displaystyle \frac{1}{A(z^{-1},\theta )}}e\left(t\right)\end{array}\end{array}$$

(5)

where $n_k$ is the number of the input–output delay samples.

When an SS is designed to replace the hardware sensors, the output regressors are not always available. In these cases, an infinite-step prediction should be performed, using as output regressors the past estimated values. As an alternative, it is preferred to not involve output regressors in the system dynamics description, and finite impulse response (FIR) models can be adopted. FIR is a special case of Equation (1), with $n_a=0$.

2.2. HDMDc Method

The algorithm produces a discrete state-space model, hence the notation for discrete instances, $x_k$, of the continuous time variable, $x\left(t\right)$, is used, where $x_k=x(k{T}_s$) and $T_s$ is the sampling time of the model. Delay coordinates (i.e., $x_{k-1}$,$x_{k-2}$, etc.) are also included in the state-space model to account for state delay in the system. This procedure allows the creation of the augmented state space relevant to model nonlinear phenomena, as discussed in Section 1. Therefore, we define a state delay vector as:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill x_{dk}={[x_{k-1}{x}_{k-2}\cdots x_{k-q+1}]}^T,\end{array}\end{array}$$

(6)

where q is the number of delay coordinates (including the current time step) of the state, with $x_{dk}\in {\mathbb{R}}^{(q-1)n_x}$ and $n_x$ the number of state variables.

The input delay vector is defined as:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill u_{dk}={[u_{k-1}{u}_{k-2}\cdots u_{k-q_u+1}]}^T,\end{array}\end{array}$$

(7)

where $q_u$ is the number of delay coordinates (including the current time step) of the inputs, with $u_{dk}\in {\mathbb{R}}^{(q_u-1)n_u}$ and $n_u$ the number of the exogenous input variables.

The discrete state-space function is defined as:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill x_{k+1}=A{x}_k+A_d{x}_{dk}+B{u}_k+B_d{u}_{dk},\end{array}\end{array}$$

(8)

where $A\in {\mathbb{R}}^{n_x\times n_x}$ is the state matrix, $A_d\in {\mathbb{R}}^{n_x\times (q-1)n_x}$ is the state delay system matrix, $B\in {\mathbb{R}}^{n_x\times n_u}$ is the input matrix and $B_d\in {\mathbb{R}}^{n_x\times (q_u-1)n_u}$ is the delay input matrix. The system output is assumed to be equal to the state, i.e., the output matrix is assumed to be the identity matrix. When dealing with system identification in which only an input/output time series is available, this assumption implies that $n_x$ should be chosen as the size of the process output vector. The training time series consists of discrete measurements of the outputs (i.e., $y_k=x_k$) and corresponding inputs (i.e., $u_k$).

The training data exploring the augmented state space, thanks to the delay shifts, are organized in the following matrices:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill X=\left[\begin{array}{ccccccccc}{x}_q& & x_{q+1}& & x_{q+2}& & \cdots & & x_{(q-1)+w}\end{array}\right]\end{array}\end{array}$$

(9)

$$X^{\prime }=\left[\begin{array}{ccccccccc}{x}_{q+1}& & x_{q+2}& & x_{q+3}& & \cdots & & x_{q+w}\end{array}\right]$$

(10)

$$X_d=\left[\begin{array}{ccccccccc}{x}_{q-1}& & x_q& & x_{q+1}& & \cdots & & x_{(q-1)+w-1}\\ ⋮& & ⋮& & ⋮& & \ddots & & ⋮\\ x_2& & x_3& & x_4& & \cdots & & x_{w+1}\\ x_1& & x_2& & x_3& & \cdots & & x_w\end{array}\right]$$

(11)

$$\begin{array}{c}\hfill \begin{array}{c}\hfill X_d^{\prime }=\left[\begin{array}{ccccccccc}{x}_q& & x_{q+1}& & x_{q+2}& & \cdots & & x_{(q-1)+w}\\ ⋮& & ⋮& & ⋮& & \ddots & & ⋮\\ x_3& & x_4& & x_5& & \cdots & & x_{w+2}\\ x_2& & x_3& & x_4& & \cdots & & x_{w+1}\end{array}\right]\end{array}\end{array}$$

(12)

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \mathrm{\Gamma }=\left[\begin{array}{ccccccccc}{u}_q& & u_{q+1}& & u_{q+2}& & \cdots & & u_{(q-1)+w}\end{array}\right]\end{array}\end{array}$$

(13)

$$\begin{array}{c}\hfill \begin{array}{c}\hfill {\mathrm{\Gamma }}_d=\left[\begin{array}{ccccc}{u}_{q-1}& u_{q+0}& u_{q+1}& \cdots & u_{(q-1)+w-1}\\ ⋮& ⋮& ⋮& \ddots & ⋮\\ u_{(q-q_u)+2}& u_{(q-q_u)+3}& u_{(q-q_u)+4}& \cdots & x_{(q-q_u)+w+1}\\ u_{(q-q_u)+1}& x_{(q-q_u)+2}& u_{(q-q_u)+3}& \cdots & u_{(q-q_u)+w}\end{array}\right]\end{array}\end{array}$$

(14)

where w represents the time snapshots and is the number of columns in the matrices, $X^{\prime }$ is the matrix X shifted forward by one time step, $X_d$ is the matrix with delay states and $\mathrm{\Gamma }$ is the matrix of inputs. Moreover, to incorporate the dynamic effect of control inputs, an extended matrix of the exogenous inputs with time shifts (i.e., ${\mathrm{\Gamma }}_d$) is created and included in the model. Equation (8) can now be combined with the matrices in Equations (9)–(14) to produce:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \left[\begin{array}{c}{X}^{\prime }\\ X_d^{\prime }\end{array}\right]=AX+A_d{X}_d+B\mathrm{\Gamma }+B_d{\mathrm{\Gamma }}_d\end{array}\end{array}$$

(15)

Note that the primary objective of HDMDc is to determine the best-fit model matrices, A, $A_d$, B and $B_d$, given the data in $X^{\prime }$, X, $X_d$, $\mathrm{\Gamma }$ and ${\mathrm{\Gamma }}_d$ [40]. Considering the definition of the Hankel matrix, H, for a generic single measurement time series, $h_k$, and applying a d time shift:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill H=\left[\begin{array}{ccccccccc}{h}_d& & h_{d+1}& & h_{d+2}& & \cdots & & h_{(d-1)+w}\\ h_{d-1}& & h_d& & h_{d+1}& & \cdots & & h_{(d-2)+w}\\ ⋮& & ⋮& & ⋮& & \ddots & & ⋮\\ h_1& & h_2& & h_3& & \cdots & & h_w\end{array}\right]\end{array}\end{array}$$

(16)

we can introduce the synoptic notation:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \begin{array}{c}\hfill X_H=\left[\begin{array}{c}X\\ X_d\end{array}\right],X_H^{\prime }=\left[\begin{array}{c}{X}^{\prime }\\ X_d^{\prime }\end{array}\right],{\mathrm{\Gamma }}_H=\left[\begin{array}{c}\mathrm{\Gamma }\\ {\mathrm{\Gamma }}_d\end{array}\right],\\ \hfill A_H=\left[\begin{array}{cc}A& A_d\end{array}\right],B_H=\left[\begin{array}{cc}B& B_d\end{array}\right]\end{array}\end{array}\end{array}$$

(17)

with $X_H\in {\mathbb{R}}^{q{n}_x\times w}$ and ${\mathrm{\Gamma }}_H\in {\mathbb{R}}^{q_u{n}_u\times w}$ the Hankel matrices for the time series $x_k$ and $u_k$, respectively. $A_H$ and $B_H$ are the transformation matrices for the augmented state and inputs, with $A_H\in {\mathbb{R}}^{q{n}_x\times q{n}_x}$ and $B_H\in {\mathbb{R}}^{q{n}_x\times q_u{n}_u}$.

Considering the matrix $\Omega \in {\mathbb{R}}^{(q{n}_x+q_u{n}_u)\times w}$ as the composition of the delayed inputs and outputs, and G as the global transformation matrix described in Equation (18).

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \Omega =\left[\begin{array}{c}{X}_H\\ {\mathrm{\Gamma }}_H\end{array}\right],G=\left[\begin{array}{c}{A}_H{B}_H\end{array}\right],\end{array}\end{array}$$

(18)

we obtain:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill X_H^{\prime }=G\Omega \end{array}\end{array}$$

(19)

A truncated Singular Value Decomposition (SVD) of the $\Omega$ matrix results in the following approximation:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \Omega \approx {\tilde{U}}_p{\tilde{\Sigma }}_p{\tilde{V}}_p^T\end{array}\end{array}$$

(20)

where the notation ^˜ represents rank-p truncation of the corresponding matrix, $\tilde{U}\in {\mathbb{R}}^{(q{n}_x+q_u{n}_u)\times p}$, $\tilde{\Sigma }\in {\mathbb{R}}^{p\times p}$, and $\tilde{V}\in {\mathbb{R}}^{w\times p}$. Then the approximation of G can be computed as:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill G\approx X_H^{\prime }{\tilde{V}}_p{\tilde{\Sigma }}_p^{-1}{\tilde{U}}_p^T\end{array}\end{array}$$

(21)

For reconstructing the approximate state matrices ${\tilde{A}}_H$ and ${\tilde{B}}_H$, the matrix ${\tilde{U}}_p$ can be split in two separate components: ${\tilde{U}}_{p1}$, related to the state, and ${\tilde{U}}_{p2}$, related to the exogenous inputs:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill {\tilde{U}}_p^T=\left[\begin{array}{cc}{\tilde{U}}_{p1}^T& {\tilde{U}}_{p2}^T\end{array}\right]\end{array}\end{array}$$

(22)

where ${\tilde{U}}_{p1}\in {\mathbb{R}}^{q{n}_x\times p}$ and ${\tilde{U}}_{p2}\in {\mathbb{R}}^{q_u{n}_u\times p}$.

The complete G matrix can be therefore split in:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill G\approx \left[\begin{array}{cc}{\overline{A}}_H& {\overline{B}}_H\end{array}\right]=\left[\begin{array}{cc}{X}_H^{\prime }{\tilde{V}}_p{\tilde{\Sigma }}_p^{-1}{\tilde{U}}_{p1}^T& X_H^{\prime }{\tilde{V}}_p{\tilde{\Sigma }}_p^{-1}{\tilde{U}}_{p2}^T\end{array}\right]\end{array}\end{array}$$

(23)

Due to the high dimension of the matrices and to obtain further optimization in the computation of the reconstructed system, a truncated SVD of the $X_H^{\prime }$ matrix results in the following approximation:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill X_H^{\prime }\approx {\widehat{U}}_r{\widehat{\Sigma }}_r{\widehat{V}}_r^T\end{array}\end{array}$$

(24)

where the notation ^{^} represents rank-r truncation, ${\widehat{U}}_r\in {\mathbb{R}}^{q{n}_x\times r}$, ${\widehat{\Sigma }}_r\in {\mathbb{R}}^{r\times r}$, and ${\widehat{V}}_r\in {\mathbb{R}}^{w\times r}$, and typically we consider $r<p$. Considering the projection of the operators ${\overline{A}}_H$ and ${\overline{B}}_H$ on the low-dimensional space we obtain:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill {\tilde{A}}_H={\widehat{U}}_r^T{\overline{A}}_H{\widehat{U}}_r={\widehat{U}}_r^T{X}_H^{\prime }{\tilde{V}}_p{\tilde{\Sigma }}_p^{-1}{\tilde{U}}_{p1}^T{\widehat{U}}_r\end{array}\end{array}$$

(25)

$$\begin{array}{c}\hfill \begin{array}{c}\hfill {\tilde{B}}_H={\widehat{U}}_r^T{\overline{B}}_H={\widehat{U}}_r^T{X}_H^{\prime }{\tilde{V}}_p{\tilde{\Sigma }}_p^{-1}{\tilde{U}}_{p2}^T\end{array}\end{array}$$

(26)

with ${\tilde{A}}_H\in {\mathbb{R}}^{r\times r}$ and ${\tilde{B}}_H\in {\mathbb{R}}^{r\times q_u{n}_u}$ The approximated discrete-time system based on the Hankel transformation of the original time series (i.e., ${\tilde{x}}_k^H$ and $u_k^H$) can be therefore represented as:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill {\tilde{x}}_{k+1}^H={\tilde{A}}_H{\tilde{x}}_k^H+{\tilde{B}}_H{u}_k^H\end{array}\end{array}$$

(27)

with $x_k^H={\widehat{U}}_r{\tilde{x}}_k^H$. The original time series $x_k$ is then extracted from $x_k^H$ considering only the rows with index $i=$$n·q+1$, where $n=0,1,\dots ,(n_x-1)$.

Figure 1 clarifies the HDMDc procedure at a higher level. The input and output measurement data from the historical dataset of an industrial process are fed into the HDMDc block that performs in sequence the Hankel transformation of the input/output variables, the merging of the state and input matrices, which integrates the control signals (DMDc), and then the model identification procedure (DMD). The DMD performs a space transformation, based on the SVD, and returns the reduced estimated state-space system representation used for the output multi-step-ahead prediction.

2.3. MSA-HDMDc Method

In this section, the proposed MSA-HDMDc method is described and the procedure is presented in Algorithm 1. It performs a model optimization and multi-step-ahead prediction of the process output on the basis of a model identified using HDMDc. The model optimization is based on a cost function depending on a combination of key performance indicators (KPIs) such as the mean average percentage error ($MAP{E}_{\%}$) and the coefficient of determination ($R^2$), adopted for the comparison of the estimated output to the measured target.

Moreover, to compare a set of models to a chosen baseline ($bl$) one, the performance improvement ($PI\%$) index is defined for each KPI as:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill KP{I}_{PI\%}={\displaystyle \frac{(T-KP{I}_{bl})-(T-KP{I}_{newmodel})}{(T-KP{I}_{bl})}}\%\end{array}\end{array}$$

(28)

with $T=0$ for $MAP{E}_{\%}$ and $T=1$ for $R^2$.

As a preliminary step, the optimal delay shifts $q^{opt}$ and $q_u^{opt}$, for the state and the input, respectively, should be determined in sequence. The selection is performed by applying the HDMDc algorithm without order reduction, adopting an optimization algorithm (e.g., grid search strategy), and comparing the prediction performances in terms of $MAP{E}_{\%}$ and $R^2$, as will be shown in Section 4.

The MSA-HDMDc algorithm is described in the following. The acquired input/output data samples are required to perform the model identification procedure. The training dataset is used to create the X and $\mathrm{\Gamma }$ matrices with the output and inputs, respectively. The state and input augmentation ($X_H$ and ${\mathrm{\Gamma }}_H$) are performed by applying the Hankel operator to the original measurements, and the extended state matrix, $\Omega$, is obtained by appending the $X_H$ and ${\mathrm{\Gamma }}_H$ matrices. The core SVD algorithm is performed on the $\Omega$ and $X_H^{\prime }$ matrices. The iteration for model optimization on multi-step-ahead prediction is then performed by determining, in sequence, the optimal reduction for the $\Omega$ and $X_H^{\prime }$ matrices. The core of the model reduction and reconstruction is performed in the function Reconstruct in Algorithm 2. It performs four operations:

• The state matrices order truncation (p,r) for model reduction;
• The determination of the HDMDc operators ${\tilde{A}}_H$ and ${\tilde{B}}_H$ as the state space representation of the identified reduced model;
• The iterative reconstruction of the multi-step-ahead estimated state in the reduced state space (${\tilde{X}}_H$), for each selected time horizon within $K_{max}$;
• The remapping of the reduced state variables to the original augmented state space (${\widehat{X}}_H$);
• The extraction of the original state variables from ${\widehat{X}}_H$, selecting the rows related to the first time shift or each state variable;
• The evaluation of the $MAP{E}_{\%}$ and $R^2$, comparing the model predictions with the target output.

The model reduction is performed in two steps. In the first step, the $\Omega$ matrix is reduced by adopting different order reductions, $p\in p_{range}$, selected on a grid with maximum value $p_{max}=n_x{q}^{opt}+n_u{q}_u^{opt}$. In this phase, the matrix $X_H^{\prime }$ is kept to the full-order $r=r_{max}=n_{x}q$. The optimal reduction order, $p^{opt}$, is determined by maximizing the cost function, $f_p$, over the $p_{range}$. The cost function consists of a linear combination of the adopted KPIs ($MAP{E}_{\%}$ and $R^2$) evaluated on the maximum prediction time step, $K_{max}$. The optimization is performed using the validation data contained in the training dataset. In the second phase, the matrix $X_H^{\prime }$ is reduced by adopting different order reductions $r\in r_{range}$, selected on a grid with maximum value $r_{max}=min(n_x{q}^{opt},p)$. The reduced matrix, $\Omega$, with order $p=p^{opt}$ is here considered. The optimal reduction order, $r^{opt}$, is determined by maximizing the cost function, $f_r$, over the $r_{range}$. The multi-step-ahead prediction can be performed by reconstructing the output dynamics of the optimal identified model.

Algorithm 1 MSA-HDMDc Algorithm

Given the training datasets $X\in {\mathbb{R}}^{n_x\times w}$, $\mathrm{\Gamma }\in {\mathbb{R}}^{n_u\times w}$; $X=[x_1{x}_2\cdots x_{m-1}]$ $X^{\prime }=[x_2{x}_3\cdots x_m]$ $\mathrm{\Gamma }=[u_1{u}_2\cdots u_{m-1}]$ Set $K_{max}$ as the maximum prediction time step value $p_{max}=n_x{q}^{opt}+n_u{q}_u^{opt}$ $r_{max}=min(n_x{q}^{opt},p)$ Set $p_{range}$ and $r_{range}$ with $r<p$ and $r<r_{max}$ $X_H$ = Hankel(X, $q^{opt}$); $X_H^{\prime }$ = Hankel($X^{\prime }$, $q^{opt}$) ${\mathrm{\Gamma }}_H$ = Hankel($\mathrm{\Gamma }$, $q_u^{opt}$) $\Omega ={\left[X_H^T{\mathrm{\Gamma }}_H^T\right]}^T$ Compute the SVD of $\Omega =\tilde{U}\tilde{\Sigma }{\tilde{V}}^T$ Compute the SVD of $X_H^{\prime }=\widehat{U}\widehat{\Sigma }{\widehat{V}}^T$ for $p\in p_{range}$ do     [$MAP{E}_{\%(p,r_{max})}$, $R_{(p,r_{max})}^2$] = Reconstruct (p,$r_{max}$, $K_{max}$) end for [$MAP{E}_{\%b{l}_p}$, $R_{b{l}_p}^2$] = Reconstruct ($p_{max}$, $r_{max}$, $K_{max})$ $f_p(MAP{E}_{\%(p,r_{max})},R_{(p,r_{max})}^2,MAP{E}_{\%b{l}_p},R_{b{l}_p}^2)=$ $MAP{E}_{PI\%}+R_{PI\%}^2$ $p^{opt}=\underset{p\in p_{range}}{argmax}{f}_p(p,r_{max})$ for $r\in r_{range}$ do     [$MAP{E}_{\%(p^{opt},r)}$,$R_{(p^{opt},r)}^2$] = Reconstruct ($p^{opt}$, $r,K_{max})$ end for [$MAP{E}_{\%b{l}_r}$,$R_{b{l}_r}^2$] = Reconstruct ($p^{opt}$, $r_{max}$, $K_{max}$) $f_r(MAP{E}_{\%(p^{opt},r_{max})},R_{(p^{opt},r_{max})}^2,MAP{E}_{\%b{l}_r},R_{b{l}_r}^2)=MAP{E}_{PI\%}+R_{PI\%}^2$ $r^{opt}=\underset{r\in r_{range}}{argmax}{f}_r(p^{opt},r)$ [$MAP{E}_{\%(p^{opt},r^{opt})}$, $R_{(p^{opt},r^{opt})}^2$]= Reconstruct ($p^{opt}$, $r^{opt},K_{max}$)

Algorithm 2 Reduction and Reconstruction Algorithm

function [$MAP{E}_{\%}$, $R^2] =$Reconstruct (p,r,K) Truncate the SVD matrices at order p: $\Omega \approx {\tilde{U}}_p{\tilde{\Sigma }}_p{\tilde{V}}_p^T$ Truncate the SVD matrices at order r: $X_H^{\prime }\approx {\widehat{U}}_r{\widehat{\Sigma }}_r{\widehat{V}}_r^T$ Compute the HDMDc operators ${\tilde{A}}_H={\widehat{U}}_r^T{X}_H^{\prime }{\tilde{V}}_p{\tilde{\Sigma }}_p^{-1}{\tilde{U}}_{p1}^T{\widehat{U}}_r$ and ${\tilde{B}}_H={\widehat{U}}_r^T{X}_H^{\prime }{\tilde{V}}_p{\tilde{\Sigma }}_p^{-1}{\tilde{U}}_{p2}^T$ for $j=1,2,\cdots ,w-K_{max}$ do     Get $x_j^H$, $u_j^H$ from the datasets $X_H$ and ${\mathrm{\Gamma }}_H$     ${\tilde{x}}_j^H={\widehat{U}}_r^T{x}_j^H$     for $k=0,1,\cdots ,K_{max}-1$ do         ${\tilde{x}}_{j+k+1}^H={\tilde{A}}_H{\tilde{x}}_{j+k}^H+{\tilde{B}}_H{u}_{j+k}^H$         ${\widehat{x}}_{j+k+1}^H=\widehat{U}{\tilde{x}}_{j+k+1}^H$     end for     ${\widehat{X}}_H=[{\widehat{x}}_1^H{\widehat{x}}_2^H\cdots {\widehat{x}}_{w-K_{max}}^H]$     Select ${\widehat{X}}_H$ rows for the first time shift for each state variable     Compute $MAP{E}_{\%}$ and $R^2$ end for Return $MAP{E}_{\%}$ and $R^2$ for the prediction step K

3. Case of Study

This section contains the description of two case studies that are widely used in the field of SSs. Both are from the petrochemical sector and are complex systems with exogenous inputs and strong nonlinearities.

3.1. The Sulfur Recovery Unit

The SRU desulphurization unit considered here is located in a refinery in Sicily (Italy), as described in [54]. SRUs in refineries are used to recover elemental sulfur from gaseous hydrogen sulfide ($H_{2}S$) contained in by-product gasses produced during the refining of crude oil and other industrial processes. Since $H_{2}S$ is a hazardous environmental pollutant, such a process is of fundamental importance.

The inlets of each SRU line receive two acid gases: MEA gas, rich in $H_{2}S$, and SWS (Sour Water Stripping) gas, rich in $H_{2}S$ and ammonia ($N{H}_3$). These input gases are combusted in two separate chamber reactors fed by a suitable airflow supply for combustion control. The output gas stream contains residues of $H_{2}S$ and sulphur dioxide ($S{O}_2$). Normally, the ratio of $H_{2}S$ to $S{O}_2$ in the tail gas must be maintained, which is specified by a setpoint. An additional secondary airflow is used as an input to improve process control. This variable is the output of a feedback control system and is used to reduce the peak values of $H_{2}S$ and $S{O}_2$.

Figure 2 represents a working scheme for an SRU line. The application of SSs is therefore necessary to estimate such concentrations.

The input and output variables available in the SRU historical dataset are listed in

Table 1. Input and output variables of the SRU models.

Variable	Description
$u^{\left(1\right)}=MEA\_GAS$	gas flow in the MEA chamber (NM3/h)
$u^{\left(2\right)}=AIR\_MEA$	airflow in the MEA chamber (NM3/h)
$u^{\left(3\right)}=SWS\_GAS$	total gas flow in the SWS chamber (NM3/h)
$u^{\left(4\right)}=AIR\_SWS$	total airflow in the SWS chamber (NM3/h)
$u^{\left(5\right)}=AIR\_ME{A}_2$	secondary air flow (NM3/h)
$y_1=\left[H_{2}S\right]$	$H_{2}S$ concentration (output 1) (mol%)
$y_2=\left[S{O}_2\right]$	$S{O}_2$ concentration (output 2) (mol%)

. MSA-HDMDc is used to estimate the $H_{2}S$ concentration (i.e., $y_1$ output).

3.2. The Debutanizer Column

The column is located at ERG Raffineria Mediterranea s.r.l. (ERGMED) in Syracuse, Italy, and is an integral part of a desulphurization and naphtha splitting plant. In the DC, propane (C3) and butane (C4) are extracted from the naphtha stream as overheads [52].

The DC is required to:

• Ensure sufficient fractionation in the debutanizer;
• Maximize the C5 content (stabilized gasoline) in the debutanizer overhead (LP gas splitter feed) while complying with the legally prescribed limit;
• Minimize the C4 (butane) content in the bottom of the debutanizer (feed to the naphtha splitter).

A detailed schematic of the debutanizer column is shown in Figure 3. It includes the following components:

• E150B heat exchanger;
• E107AB overhead condenser;
• E108AB bottom reboiler;
• P102AB head reflux pump;
• P103AB feed pump to the LPG splitter;
• D104 reflux accumulator.

A number of hardware sensors are installed in the plant to monitor product quality. The subset of sensors that are relevant for the described application are listed in

Table 2. Input and output variables of the DC models.

Variable	Description
$u^{\left(1\right)}=T040$	top temperature (°C)
$u^{\left(2\right)}=P011$	top pressure (Kg/cm2)
$u^{\left(3\right)}=F015$	top reflux (m3/h)
$u^{\left(4\right)}=F018$	top flow (m3/h)
$u^{\left(5\right)}=T004$	side temperature (°C)
$u^{\left(6\right)}=(T036+T037)/2$	$T036$ and $T037$ bottom temperatures (°C)
$y=F_{C4}$	C4 concentration in the bottom flow (%)

.

The C4 concentration in the bottom flow is estimated as the output of the designed SS. It is not detected on the bottom stream, but at the overheads of the deisopentanization column. The C4 content in the C5 depends solely on the operating conditions of the debutanizer: it can be assumed that the C4 detected in the C5 stream is that which flows from the bottom of the debutanizer. Due to the location of the analyzer, the concentration values are determined with a long delay, which is not exactly known but is constant and probably in the range of 30 min. To improve the control quality of the DC, real-time estimation of both C4 content and C5 content is required. For this purpose, a virtual sensor is needed, which is described in the following sections.

4. Sulfur Recovery Unit Results

In this section, the proposed MSA-HDMDc is applied to the SRU case study and the results are compared with the baseline models. The available dataset consists of about 14,000 samples, with a sampling period of one minute. In addition, $70\%$ of the dataset is used for model training, $15\%$ for the validation of the hyperparameters and model order reduction and, finally, the testing is performed on the remaining $15\%$ of the dataset.

4.1. Model Optimization and Hyperparameter Tuning

An MSA prediction is performed for the output, $y_1$. To validate the performance of the procedure, a time horizon of $K_{max}=30$ steps is selected. To better show the effect of the MSA prediction for different time horizons, values in the interval $K\in \{1,5,10,15,20,25,30\}$ are considered.

4.1.1. Baseline Models

Two linear models were considered: ARX and FIR. The optimal model order was selected based on the minimum description length (MDL) criterion [55]. In particular, the ARX structure was identified to have eight common poles ($n_a=8$), two zeros ($n_b=3$), and no delay for all input variables ($n_k=0$), while the FIR order parameters resulted to be $n_b=10$ and $n_k=0$.

4.1.2. MSA-HDMDc Model

In a preliminary phase of the iterative MSA-HDMDc procedure, a parametric study was performed based on model performance in terms of $MAP{E}_{\%}$ and $R^2$. Such a procedure allowed the definition of the hyperparameters related to the delay shifts q and $q_u$ applied to the state and inputs, respectively.

The first step was to determine the optimal state variable delay shift, q. A grid search strategy was applied and, for each q that lies between $q=20$ and $q=60$ with a step of 10, the full-rank HDMDc model was identified. The estimated output reconstructed at the maximum prediction step (i.e., 30 steps) was compared with the measured output, $y_1$. For statistical analysis, the validation dataset was divided into 20 subsets of 100 samples. $MAP{E}_{\%}$ and $R^2$ were evaluated for each subset, and the corresponding distribution was determined. In particular, the median value of $MAP{E}_{\%}$ was used to select the optimal hyperparameter, q.

Table 3. SRU case study: performance comparison for the selection of the $q^{opt}$. The mean value over 20 subsets of data of the $MAP{E}_{\%}$ is reported for different state time shifts, q. The KPI is evaluated for a 30-step-ahead prediction. The $PI\%$ is reported considering $q=40$ as the reference value.

State Time-Shift Optimization
$\mathbf{q}$	20	30	40	50	60
${\mathbf{MAPE}}_{\%}$	$5.53$	$5.22$	$5.19$	$5.28$	$5.46$
$\mathbf{PI}\%$	$-6.66$	$-0.58$	0	$-1.79$	$-5.13$

shows the mean value of $MAP{E}_{\%}$ over the 20 trials considering a 30-step-ahead prediction for the selected $q\in \{20,30,40,50,60\}$. The best performing model corresponds to $q^{opt}=40$, as shown by the reported $PI\%$. In a second step, assuming that the optimal value $q^{opt}=40$ is fixed, a further parametric optimization was performed by varying the Hankel shift, $q_u$, applied to the exogenous control inputs in the ${\mathrm{\Gamma }}_H$ matrix. Since $q_u\le q^{opt}$, for the system to be causal, the model was identified for each $q_u$ belonging to the set $q_u\in \{10,15,30,40\}$. The estimated output, reconstructed at each considered prediction step, was compared with the measured output, $y_1$. Considering the model with $q_u=q^{opt}=40$, corresponding to the maximum allowable value, as the baseline one, the barplot in Figure 4 shows how the system performance changes ($PI\%)$, at different prediction steps, by decreasing the value of $q_u$. It can be noticed how the reduction of the Hankel shift, $q_u$, applied to the input variables causes negative $PI\%$ and, thus, the decaying of the model performance, both in terms of $MAP{E}_{\%}$ and $R^2$ for prediction steps higher than 5. This led to the selection of $q_u^{opt}=q^{opt}=40$ as the optimal number of Hankel shifts for inputs and state variables.

4.2. Model Order Reduction

To evaluate the importance of the model reduction phase, as proposed in MSA-HDMDc, the results obtained during the model order optimization are shown for different prediction horizons. To better assess the performance of the model, both $MAP{E}_{\%}$ and $R^2$ are shown.

According to Algorithm 1, the analysis was performed considering two optimization phases, first for p and then for r, each representing the SVD truncation for the matrices $\Omega$ (Equation (20)) and $X_H^{\prime }$ (Equation (24)). In a first step, the $\Omega$ matrix was truncated from the full-order $p_{max}=n_x{q}^{opt}+n_u{q}_u^{opt}=240$ (with $q^{opt}=40$, $n_x=1$, $q_u^{opt}=40$, and $n_u=5$) to the reduced order p. Models with different p-reductions from the set $p_{range}\in \{201,202,210,220,240\}$ and with the full-order matrix $X_H^{\prime }$ with order $r_{max}=min(n_x{q}^{opt},p)=40$ are shown in Figure 5, where the indices $MAP{E}_{\%}$ and $R^2$ are given for different prediction steps. The global performance of HDMDc on the validation dataset was compared with the performance of the baseline methods (i.e., $ARX$ and $FIR$). It can be noticed that the MSA-HDMDc model outperforms the FIR model for all prediction steps and the ARX model for prediction steps greater than or equal to 5, regardless of the model order, p. The comparison of the MSA-HDMDc model for the different p also shows that the reduction of the $\Omega$ matrix to order p has a positive effect on the performance of the model for prediction steps greater than 5, and the optimal configuration is achieved for order $p=202$.

In a second step, while maintaining the optimal $\Omega$ order $p^{opt}=202$, the $X_H^{\prime }$ matrix was truncated from the full-order $r_{max}=min(n_x{q}^{opt},p^{opt})=40$ to the reduced order, r. Models with different r reductions belonging to the set $r\in \{18,20,23,25,30,35\}$ were identified. The global performance comparison on the validation dataset is shown in Figure 6, where the $PI\%$ is given for both $MAP{E}_{\%}$ and $R^2$ at different prediction steps. The reference model for the $PI\%$ is MSA-HDMDc with $p^{opt}=202$ and $X_H^{\prime }$ full-order $r_{max}=40$. It can be noticed how the MSA-HDMDc model with a reduction of $X_H^{\prime }$ to order $r=25$ outperforms the full-order model for prediction steps greater than 5. These results confirm that the optimized MSA-HDMDc order reduction allows identification of the dominant dynamics and thus introduces robustness and noise rejection features to the reduced model. These properties thus improve the long-term prediction performance compared with the full-order system. Finally, the optimal order for the MSA-HDMDc model was determined to be $p^{opt}=202$ and $r^{opt}=25$.

4.3. Model Comparisons and Discussion

In this section, the performance of the MSA-HDMDc reduced-order model is evaluated. In particular, the results for the maximum step prediction (i.e., 30 steps), are here presented. The regression plots on the test dataset, for the baseline and the optimal MSA-HDMDc models, are reported in Figure 7. The ARX model regression plot presents a slope of $0.41$ and a bias of $0.13$; the global performance over the test dataset is not considered acceptable. The FIR regression plot presents a slope of $0.76$ with a bias of $0.04$. MSA-HDMDc outperforms both the baseline models with a slope of $0.77$ and a bias of $0.068$.

The time plot in Figure 8 shows a comparison between the measured output, $y_1$, and the 30-step-ahead predicted output for the baseline and the MSA-HDMDc models for a subset of the test dataset.

As described above, the main objective of the SRU is to remove $H_{2}S$ from the gas flow. Therefore, the estimation of the output peaks is of greatest interest for the designed model. Figure 8 shows the prediction performance of the proposed model, which outperforms the results obtained with the baseline approaches, especially with respect to the peak events.

Table 4. SRU case study: $MAP{E}_{\%}$ values at different prediction steps obtained for the considered models: ARX, FIR, MSA-HDMDc ($q^{opt}=40$, $q_u^{opt}=40$, $p^{opt}=202$, $r^{opt}=25$).

${\mathbf{MAPE}}_{\%}$
Steps	$\mathbf{1}$	$\mathbf{5}$	$\mathbf{10}$	$\mathbf{15}$	$\mathbf{20}$	$\mathbf{25}$	$\mathbf{30}$
ARX	$\mathbf{2}.\mathbf{75}$	8.08	10.91	12.70	13.74	14.26	14.55
FIR	11.84	11.84	11.84	11.84	11.84	11.84	11.84
MSA-HDMDc	5.56	$\mathbf{5}.\mathbf{85}$	$\mathbf{6}.\mathbf{24}$	$\mathbf{6}.\mathbf{59}$	$\mathbf{6}.\mathbf{90}$	$\mathbf{7}.\mathbf{15}$	$\mathbf{7}.\mathbf{35}$

Bold values represent the best performance for the specific prediction steps column.

and

Table 5. SRU case study: $R^2$ values at different prediction steps obtained for the considered models: ARX, FIR, MSA-HDMDc ($q^{opt}=40$, $q_u^{opt}=40$, $p^{opt}=202$, $r^{opt}=25$).

${\mathbf{R}}^2$
Steps	$\mathbf{1}$	$\mathbf{5}$	$\mathbf{10}$	$\mathbf{15}$	$\mathbf{20}$	$\mathbf{25}$	$\mathbf{30}$
ARX	0.95	0.49	0.18	−0.02	−0.15	−0.21	−0.24
FIR	0.26	0.26	0.26	0.26	0.26	0.26	0.26
MSA-HDMDc	0.73	0.70	0.67	0.64	0.62	0.61	0.59

Bold values represent the best performance for the specific prediction steps column.

reports the $MAP{E}_{\%}$ and $R^2$ for the test dataset considering different prediction steps for the considered models. Both KPIs are in agreement and show the superiority of the MSA-HDMDc model for large prediction horizons.

As mentioned in Section 1, the strength of the Koopman operator, which forms the basis of MSA-HDMDc, lies in the identification of a global state space model that is valid, even far from specific working points and/or attractors. It differs from standard linear models that either exploit linearization around specific working points or extend the linear approximation to the entire domain. With this in mind, the exogenous inputs in the entire test interval were clustered using the k-means algorithm with squared Euclidean distance, which is commonly used in pattern recognition [56], classification and predictive modeling [57]. Such a method aims to identify different operating points contained in the input dynamics. To select the optimal number of clusters, the silhouette score distribution [58] was used, obtaining three distinct clusters. The results of such a procedure allowed us to divide the test time series into sub-intervals, each of which is associated with a cluster identifying a working point.

The results of the clustering are shown in Figure 9. The first panel shows the time evolution of the exogenous inputs as named and described in Table 1. The second field shows the different operating points, defined as clusters, identified by the k-means algorithm in the analyzed time window. The third field contains the $MAP{E}_{\%}$ time evolution, which was analyzed in time batches of 100 min for a 30-step-ahead prediction. It can be observed that, in the first and last time interval, belonging to $Cluster1$, which is representative of the majority of the dataset, all three models predict the output with good performance. For data belonging to the second and third clusters, only MSA-HDMDc guarantees a performance similar to those obtained in the previous cluster. This confirms the suitability of MSA-HDMDc to identify global models.

5. Debutanizer Column Results

In this section, the proposed MSA-HDMDc is applied to the DC case study and the results are compared with the baseline models. The available dataset consists of 4 months of data, i.e., March, May, July and September 2004, with a sampling period of 6 min. Here, $50\%$ of the dataset (March and May) is used for model training, $25\%$ (July) for the validation of the hyperparameters and model order reduction and, finally, the testing is performed on the final $25\%$ of the dataset (September).

5.1. Model Optimization and Hyperparameter Tuning

An MSA prediction on the y output is performed. To validate the performance of the procedure, a time horizon of $K_{max}=20$ steps corresponding to 120 min is selected. To better show the effect of MSA prediction for different time horizons, values in the interval $K\in \{2,5,10,20\}$ steps are considered.

5.1.1. Baseline Models

Two linear models were considered: ARX and FIR. The optimal model order was selected based on the minimum description length (MDL) criterion [55]. In particular, the ARX structure was identified to have three common poles ($n_a=3$), eight zeros ($n_b=8$) and no delay for all input variables ($n_k=0$), while the FIR order parameters resulted to be $n_b=8$ and $n_k=0$.

5.1.2. MSA-HDMDc Model

In a preliminary stage of the iterative MSA-HDMDc method, the HDMDc algorithm was used without order reduction. A grid search strategy was applied to find the optimal q in the range from $q=10$ to $q=30$. The estimated output reconstructed for the prediction step, $K_{max}$, was compared with the measured output, y. For the statistical analysis, the validation dataset was divided into 70 subsets of 100 samples. $MAP{E}_{\%}$ and $R^2$ were evaluated for each subset and the corresponding distribution was determined. In particular, the median value of $MAP{E}_{\%}$ was taken into account when selecting the optimal q hyperparameters.

Table 6. DC case study: performance comparison for the selection of the $q^{opt}$. The mean value over 70 subsets of 100 of data samples of the $MAP{E}_{\%}$ is reported for different state time shifts, q. The KPI is evaluated for a 20-step-ahead prediction. The $PI\%$ is reported considering $q=12$ as the reference value.

State Time-Shift Optimization
$\mathbf{q}$	10	12	15	17	20	30
${\mathbf{MAPE}}_{\%}$	$25.73$	$24.87$	$25.32$	$26.43$	$27.52$	$30.19$
$\mathbf{PI}\%$	$-3.43$	0	$-1.81$	$-6.28$	$-10.65$	$-17.20$

shows the mean value over the 70 trials of the $MAP{E}_{\%}$, considering a 20-steps-ahead prediction. The considered state delay shifts are $q\in \{10,12,15,17,20,30\}$. The best-performing model corresponds to $q^{opt}=12$, as shown by the $PI\%$ reported.

As a second step, considering the optimal value $q^{opt}=12$, further parametric optimization was carried out by varying the Hankel shift, $q_u$, applied to the exogenous control inputs in the ${\mathrm{\Gamma }}_H$ matrix. Being $q_u\le q^{opt}$, for the system to be causal, the model was identified for each $q_u$ belonging to the set $q_u\in \{5,10,12\}$. The estimated output, reconstructed at each considered prediction step, was compared with the measured output, y. It was found that the reduction of the Hankel shift, $q_u$, applied to the input variables causes decaying of the model performance, both in terms of $MAP{E}_{\%}$ and $R^2$. This led to the selection of $q_u^{opt}=q^{opt}=12$ as the optimal number of Hankel shifts for inputs and state variables.

5.2. Model Order Reduction

In a first step, the $\Omega$ matrix was truncated from the full-order $p_{max}=n_x{q}^{opt}+n_u{q}_u^{opt}=84$ (where $q^{opt}=12$, $n_x=1$, $q_u^{opt}=12$ and $n_u=6$) to the reduced order p. Models with different p reductions in the set $p_{range}\in \{65,66,70,84\}$ and with the full-order matrix $X_H^{\prime }$ with $r_{max}=min(n_x{q}^{opt},p)=12$ are shown in Figure 10, where the indices $MAP{E}_{\%}$ and $R^2$ are given for different prediction steps. The global performance on the validation dataset was compared with the baseline methods (i.e., $ARX$ and $FIR$). It can be seen that the MSA-HDMDc model outperforms both FIR and ARX for all prediction steps, regardless of the model order, p. When comparing MSA-HDMDc for the different p, it is also noted that reducing the $\Omega$ matrix to order p leaves the performance unchanged, as it decreases from $p=84$ to $p=66$, but starts to deteriorate at $p=65$. This leads to the conclusion that $p^{opt}=66$ holds for all prediction steps.

In a second step, while maintaining the optimal $\Omega$ order $p^{opt}=66$, the $X_H^{\prime }$ matrix was truncated from the full-order $r_{max}=min(n_x{q}^{opt},p^{opt})=12$ to the reduced order r. Models with different r reductions belonging to the set $r\in \{4,5,8,12\}$ were identified. The global performance comparison on the validation dataset is shown in Figure 11, where the $PI\%$ is given with respect to MSA-HDMDc, with $p^{opt}=66$ and $r_{max}=40$ for both $MAP{E}_{\%}$ and $R^2$ at different prediction steps. It can be seen that the MSA-HDMDc model with a reduction of $X_H^{\prime }$ to order $r=5$ outperforms the full-order model for all selected prediction steps. These results confirm that the optimized MSA-HDMDc order reduction allows to identify the dominant dynamics and thus introduces robustness and noise rejection features to the reduced model. These properties thus improve the long-term prediction performance compared with the full-order system. Finally, the optimal order for the MSA-HDMDc model was determined to be $p^{opt}=66$ and $r^{opt}=5$.

5.3. Model Comparison and Discussion

In this section, the performance of the reduced-order MSA-HDMDc model is further evaluated in terms of the predicted time series and robustness to the variability of the operating points. The time plot in Figure 12 shows a comparison between the measured output, y, and the 5- and 10-step-ahead predicted outputs for the baseline and MSA-HDMDc models for a subset of the test dataset. It shows that the prediction performance of the proposed model outperforms the results obtained with the baseline approaches, especially for the 10-step-ahead prediction, and confirms that the difference between the models becomes more pronounced at longer prediction intervals.

Table 7. DC case study: $MAP{E}_{\%}$ values at different prediction steps obtained for the considered models: ARX, FIR, MSA-HDMDc ($q^{opt}=12$, $q_u^{opt}=12$, $p^{opt}=66$, $r^{opt}=5$) in the test dataset.

${\mathbf{MAPE}}_{\%}$
Steps	$\mathbf{2}$	$\mathbf{5}$	$\mathbf{10}$	$\mathbf{20}$
ARX	4.44	13.75	29.40	53.91
FIR	57.42	57.42	57.42	57.42
MSA-HDMDc	$\mathbf{1}.\mathbf{66}$	$\mathbf{6}.\mathbf{01}$	$\mathbf{12}.\mathbf{93}$	$\mathbf{23}.\mathbf{13}$

Bold values represent the best performance for the specific prediction steps column.

and

Table 8. DC case study: $R^2$ values at different prediction steps obtained for the considered models: ARX, FIR, MSA-HDMDc ($q^{opt}=12$, $q_u^{opt}=12$, $p^{opt}=66$, $r^{opt}=5$) in the test dataset.

${\mathbf{R}}^2$
Steps	$\mathbf{2}$	$\mathbf{5}$	$\mathbf{10}$	$\mathbf{20}$
ARX	0.983	0.854	0.347	−1.20
FIR	−0.99	−0.99	−0.99	−0.99
MSA-HDMDc	$\mathbf{0}.\mathbf{998}$	$\mathbf{0}.\mathbf{974}$	$\mathbf{0}.\mathbf{890}$	$\mathbf{0}.\mathbf{661}$

Bold values represent the best performance for the specific prediction steps column.

reports the $MAP{E}_{\%}$ and $R^2$ for the test dataset considering different prediction steps for the considered models. Both KPIs are in agreement and show the superiority of the MSA-HDMDc model for all the prediction horizons.

The analysis of the model performance was performed by identifying different operating points through exogenous inputs clustering. The k-means algorithm [56,57] was applied with squared Euclidean distance. The optimal number of clusters was found to be two by using the silhouette score distribution [58]. The clustering results are reported in Figure 13. In the first panel, the time evolution of the normalized exogenous inputs, as named and described in Table 2, is shown. In the second panel, the clusters identified by the k-means algorithm are reported in the analyzed time window showing the operating points with different colors. The third panel contains the $MAP{E}_{\%}$ time evolution computed in time batches of 100 samples for a 5-step-ahead prediction. It can be seen that, for data belonging to the operating point labeled as Cluster 1 between 1400 and 2000 samples, only the MSA-HDMDc model maintains consistent performance across different system operating points. In contrast, the performance of the ARX and FIR models degrades significantly and shows high variability with changes in operating conditions. This highlights the sensitivity of the ARX and FIR models to variations in operating points and confirms the suitability of MSA-HDMDc for identifying global models that provide robust performance under different operating conditions.

6. Conclusions

In this work, we have proposed the MSA-HDMDc approach for the development of SSs in industrial applications. Two well-established benchmarks based on the SRU and DC data were considered.

The MSA-HDMDc approach focused on optimizing the identified HDMDc state-space model for multiple-step-ahead predictions. It employed the Hankel operator for state-space dimension augmentation to address nonlinear dynamics effectively through a linear state-space model.

As a preliminary step, the augmentation delay time shifts for input and state variables were optimized through statistical analyses, using the $MAP{E}_{\%}$ and $R^2$ as key performance indexes. As a second step, model order reduction was investigated to find the minimum number of Koopman space observables that better describe the nonlinear system dynamics while keeping the computational burden reasonable, aiming to obtain models that can be integrated into model-based control strategies. An iterative procedure was proposed to select the optimal order reduction, maximizing a fitness function based on performance improvements relative to full-order models. The optimal reduced MSA-HDMDc model structure was then compared for different step-ahead predictions with baseline linear identification methods (i.e., ARX and FIR), which are widely used in the SS field.

The MSA-HDMDc model outperformed, for both the SRU and the DC case studies, the baseline ARX and FIR models for high prediction steps. The model exhibited reduced sensitivity to different operational conditions compared with the baseline models. This was demonstrated through a clustering procedure applied to the input time series data, which identified and classified various operative states as clusters. The performance of the analyzed models was monitored over time and associated with these distinct states. The MSA-HDMDc model consistently maintained its performance across various conditions, whereas the ARX and FIR models showed sensitivity to these variations, resulting in decreased performance for certain clusters. This is in line with the Koopman theory that allows the identification of a global model for the system able to properly work on multiple operating points. This feature, along with the model optimization due to the SVD-based model order reduction, makes the proposed MSA-HDMDc suitable for MPC-based applications, enhancing the reliability and effectiveness of the control system in industrial applications.

The capability of the proposed methodology to identify a global model that functions across different operating points encourages further exploration of its transferability. Applying the identified model to different process lines could address the issue of data scarcity. Additionally, given that SRU and DC data exhibit strong nonlinearity and dynamic variations in operating points and spectral content, future efforts will focus on integrating MSA-HDMDc with multi-resolution DMD [59] to incorporate slow feature analysis.