Axiomatic Foundations of Anisotropy-Based and Spectral Entropy Analysis: A Comparative Study

Abstract

An axiomatic development of control systems theory can systematize important concepts. The current research article is dedicated to the investigation and comparison of two axiomatic approaches to the analysis of discrete linear time-invariant systems affected by external random disturbances. The main goal of this paper is to explore axiomatics of an anisotropy-based theory in comparison with axiomatics of a spectral entropy approach in detail. It is demonstrated that the use of the spectral entropy approach is mathematically rigorous, which allows one to prove that the minimal disturbance attenuation level in terms of an anisotropy-based control theory provides the desired performance that is not only for ergodic signals. As a result, axiomatics of the spectral entropy approach allows one to rigorously prove that anisotropy-based controllers can be used to guarantee the desired disturbance attenuation level, not only for stationary random sequences, but also for a wider set of input random signals.

1. Introduction

As a rule, the classic automatic control theory is based on the application of mathematical tools to analyze and design the control law. On the one hand, this approach allows for deriving transparent methods and numerical algorithms of analysis and control. On the other hand, it allows for providing predefined quality criteria for the closed-loop system performance in mathematical terms. The use of mathematical tools requires specifying the set of axioms and deriving rules for investigation. An axiomatic approach in automatic control theory can be found in [1,2]. Thus, this paper [1] applies an axiomatic approach to investigate the Lyapunov stability of nonlinear differential equations. The axiomatic approach is also widely used in engineering research; for example, in the paper [3], the axiomatic approach is suggested to be used for complex engineering systems designs which involve hardware, software and human interface units. Meanwhile, an axiomatic approach for the design of cyberphysical systems is discussed in paper [4].

The current research article is dedicated to the investigation and comparison of two axiomatic approaches to the analysis and control of discrete linear time-invariant systems affected by external random disturbances. The first one is called the anisotropy-based control theory; the second one is known as the spectral entropy (σ-entropy) approach. The anisotropy-based control theory appeared in the mid 1990s from the papers [5,6,7,8,9]. The anisotropy-based approach made it possible to unify within the common framework ${\mathcal{H}}_2$ and ${\mathcal{H}}_{\infty }$ control theories. As well as ${\mathcal{H}}_2$ and ${\mathcal{H}}_{\infty }$ control theories, the anisotropy-based control theory deals with external disturbance attenuation acting on the control system. It is supposed that external disturbances belong to the certain set of signals that is specified by a non-negative scalar value, called a mean anisotropy level. The mean anisotropy level is a Kullback–Leibler information divergence from the probability distribution function (p.d.f.) of the signal to a p.d.f. of white Gaussian noise. The zero mean anisotropy corresponds to the set of white Gaussian noise, while any positive mean anisotropy defines a wider set of signals. Thus, ${\mathcal{H}}_2$ and ${\mathcal{H}}_{\infty }$ control theories are limiting cases of the anisotropy-based control theory. The anisotropy-based control theory is actively developing at present. Many fruitful results can be found in [10,11,12,13].

Despite significant advantages in anisotropy-based control, there still exist some unsolved problems. It is worth mentioning that anisotropy-based control theory guarantees desired disturbance attenuation level only for ergodic processes, but real external disturbances are not stationary. Thus, it is not clear how effective anisotropy-based controllers are for real systems. To answer this question, a new approach, named the spectral entropy method, is proposed [14,15]. The first results on a linear time-invariant systems analysis appeared in [15]. It is demonstrated that spectral entropy allows for the description of a wider set of signals, including non-stationary and fading signals.

The main goal of this paper is to explore axiomatics of the anisotropy-based theory in comparison with axiomatics of the spectral entropy approach in detail. It is demonstrated that the use of a spectral entropy approach allows one to prove that a minimal disturbance attenuation level in terms of the anisotropy-based control theory is mathematically rigorous and provides a desired performance not only for ergodic signals.

The main contributions of this paper can be summarized as follows:

• Axiomatics of two approaches: anisotropy-based and spectral entropy approaches are investigated and compared for the first time.
• Different axiomatics are demonstrated to extend the set of input signals acting on the control plant substantially.
• The problem of the spectral entropy analysis is proven to have the same solution as the anisotropy-based analysis. Thus, all the results obtained for the anisotropy-based analysis and control can be directly applied to the spectral entropy analysis.

This paper is organized as follows. In Section 2, a detailed description of anisotropy-based approach axiomatics is given. Section 3 deals with axiomatics of the spectral entropy control theory. In Section 4, a discussion of the new spectral entropy approach application is presented.

2. Axiomatics of Anisotropy-Based Control Theory

As it was mentioned in the Introduction, the anisotropy-based control theory deals with an external random disturbance attenuation problem. In contrast to classical ${\mathcal{H}}_2$ and ${\mathcal{H}}_{\infty }$ control theories, the anisotropy-based control theory takes into consideration the correlation of inputting random signals. The disturbance attenuation level is calculated as a power norm ratio between an output and a random input disturbance. The anisotropy-based analysis and control is successfully developed for the linear time-invariant as well as for linear time-varying systems [16,17,18]. The analysis and control of linear time-varying systems applies the theory of matrices and matrix norms and is not considered here. To characterize a set of possible disturbances, the term “mean anisotropy” is introduced and used. The mean anisotropy level defines a set of stationary random disturbances with an unknown covariance and bounded “spectral color” [9]. It allows one to tune the controller with better disturbance attenuation properties. However, the anisotropy-based approach deals only with ergodic input signals.

Consider axiomatics of the anisotropy-based approach in detail, starting from the main six axioms:

• Linear dynamical system;
• Infinite horizon;
• Anisotropy of a random vector;
• Sequence vectorization and mean anisotropy of a random sequence;
• Kolmogorov–Szego theorem and transfer to a frequency domain;
• The quality criterion is the gain on a subset of signals whose anisotropy is less than or equal to a.

In the sequel in this paper, anisotropy-based theory statements will be reformulated in terms of:

• Equivalence relation on the set of input signals;
• The induced norm on a subset of signals, the power of which is bounded by the equivalence relation.

Consider a stable linear discrete-time system given in the state space in the following form [10]:

$$\left\{\begin{array}{cc}{x}_{k+1}\hfill & =A{x}_k+B{w}_k,x_0=0,\hfill \\ y_k\hfill & =C{x}_k+D{w}_k,k=0,1,2,\dots ,\hfill \end{array}\right.$$

(1)

where $x_k\in {\mathbb{R}}^n$ is a state vector, $W={\left\{w_k\right\}}_{k\in \mathbb{Z}}$ is a stationary Gaussian sequence of m-dimensional vectors with bounded mean anisotropy level $\overline{\mathbf{A}}\left(W\right)<a$$(a⩾0)$ and zero mean, and $y_k\in {\mathbb{R}}^p$ is an output of the system. The transfer function of system (1) is equal to $F\left(z\right)=C{(zI-A)}^{-1}B+D$.

This is the standard beginning of the problem statement in an anisotropy-based context, from which the first two axioms immediately follow:

• Linear discrete-time system;
• Infinite horizon.

At the next step, we need to describe the set of signals acting on the system. Closely related to the anisotropy-based control theory, $LQR$ or ${\mathcal{H}}_{\infty }$ deal with continuous-time deterministic signals with finite energy (i.e., ${\mathcal{L}}_2$ norm), while the $LQG$ control deals with white Gaussian noise with a known covariance. In the anisotropy-based theory, a more detailed description of the stationary sequence of random m-dimensional vectors is introduced by means of the anisotropy of the single vector and a mean anisotropy of the sequence composed of random vectors. We introduce definitions of the anisotropy and mean anisotropy, following [10].

Recall that ${\mathfrak{L}}_2^m$ is a class of ${\mathbb{R}}^m$-dimensional random vectors that are absolutely and continuously distributed with a finite second moment.

Denoted by $p_{m,\lambda }$ with $\lambda >0$, the probability density function of the Gaussian signal with a zero mean and scalar covariance matrix $\lambda I_m$ is, i.e.,

$$p_{m,\lambda }\left(x\right)={(2\pi \lambda )}^{-m/2}exp\left(-{\displaystyle \frac{{\left|x\right|}^2}{2\lambda }}\right),x\in {\mathbb{R}}^m.$$

(2)

For any $w\in {\mathfrak{L}}_2^m$ with a probability density function $f:{\mathbb{R}}^m\to {\mathbb{R}}_{+}$, its relative entropy with respect to (2) takes the form [19]:

$$\mathbf{D}\left(f\parallel p_{m,\lambda }\right)=\mathbf{E}\left[ln{\displaystyle \frac{f\left(w\right)}{p_{m,\lambda }\left(x\right)}}\right]=-h\left(w\right)+{\displaystyle \frac{m}{2}}ln(2\pi \lambda )+{\displaystyle \frac{\mathbf{E}\left[|w{|}^2\right]}{2\lambda }},$$

(3)

where

$${\displaystyle h\left(w\right)=-\mathbf{E}\left[lnf\left(w\right)\right]=-\underset{{\mathbb{R}}^m}{\int }f\left(w\right)lnf\left(w\right)dw}$$

is a differential entropy of a random vector w.

Definition 1.

Anisotropy $\mathbf{A}\left(w\right)$ of a random vector $w\in {\mathfrak{L}}_2^m$ is defined as the minimum information deviation of its distribution from Gaussian distributions on  ${\mathbb{R}}^m$ with zero mean and scalar covariance matrices

$$\mathbf{A}\left(w\right)=\underset{\lambda >0}{min}\mathbf{D}(f\parallel p_{m,\lambda }).$$

A direct calculation demonstrates that the minimum in (3) for all possible $\lambda >0$ is achieved with $\lambda =\mathbf{E}\left[{\left|w\right|}^2\right]/m$ and hence,

$$\mathbf{A}\left(w\right)=\underset{\lambda >0}{min}\mathbf{D}(f\parallel p_{m,\lambda })={\displaystyle \frac{m}{2}}ln\left({\displaystyle \frac{2\pi \mathrm{e}}{m}}\mathbf{E}\left[{\left|w\right|}^2\right]\right)-h\left(w\right).$$

Before formulating the third axiom of the anisotropy-based theory, note that the anisotropy of a random vector, being a relative entropy, has the following properties:

• $\mathbf{D}(p\parallel q)⩾0,$ and $\mathbf{D}(p\parallel q)=0$ only if $p\left(x\right)=q\left(x\right)$ for any x;
• In general case $\mathbf{D}(p\parallel q)\ne \mathbf{D}(q\parallel p).$

The second property leads directly to the fact that the relative entropy is not symmetric with respect to the rearrangement of p and q. This fact does not allow for using the relative entropy as a certain criterion, which is a certain characteristic of the two random vectors’ proximity. In the anisotropy-based theory, this drawback is overcome in the following way: the probability density functions of random vectors are compared not with each other, but with a reference vector; and the results of such a comparison are used further in theory. A random Gaussian vector with a zero mean and scalar covariance matrix $\lambda I_m$ is chosen as a reference, and since there is a whole family of such vectors, the minimum is calculated for all possible $\lambda >0$. Thus, the third axiom of the anisotropy-based theory is

• Anisotropy of a random vector.

Note that for each $w\in {\mathfrak{L}}_2^m$, its anisotropy $\mathbf{A}\left(w\right)⩾0$, with $\mathbf{A}\left(w\right)=0$, if and only if w is a Gaussian vector with zero mean and scalar covariance.

The control theory does not work with random vectors, but it does work with input and output signals, either discrete or continuous. In case of the anisotropy-based theory, the input signal is an infinite sequence of random vectors. Consequently, in this case the definition of anisotropy given above is not applicable. Instead of this term, the mean anisotropy of the random sequence is used. The mean anisotropy of the sequence is defined as a passage to the limit of a vectorized fragment of N elements of the sequence W with $N\to +\infty$.

Select a sub-sequence of the dimension $N\times m$ from the sequence W [10]

$$W_{0:N-1}=\left[\begin{array}{c}{w}_0\\ ⋮\\ w_{N-1}\end{array}\right],$$

where each vector $w_i\in {\mathbb{R}}^m$ for $i=\overline{0,(N-1)}$.

Definition 2.

Mean anisotropy of the sequence W is defined as

$$\overline{\mathbf{A}}\left(W\right)=\underset{N\to +\infty }{lim}{\displaystyle \frac{\mathbf{A}\left(W_{0:N-1}\right)}{N}}·$$

Hence, the fourth axiom is

• Mean anisotropy of a random sequence.

The term “mean anisotropy” is used to characterize a set of possible disturbances. The mean anisotropy level defines a set of stationary random disturbances with unknown covariances and bounded “spectral color”.

The definition of mean anisotropy $\overline{\mathbf{A}}\left(W\right)$ of a random sequence is given in a probability-theoretic context, while the control theory deals either in the time domain or frequency domain, which means that it uses a classical function theory apparatus. The transition from a probability-theoretic description to a functional one is provided by the Kolmogorov–Szego theorem [20]. This theorem associates one step prediction error of a random sequence with its spectral density. However, this theorem has strict limitations on the random sequence—the random signal must be a centered stationary process. From this limitation, the following axiom arises:

• Stationary random sequence.

Finally, the quality criterion in the classical anisotropy-based theory, known as the anisotropic norm of the system, is defined in the following way [10].

Denote the output sequence of the system (1) by $Y={\left\{y_k\right\}}_{k\in \mathbb{Z}}$. Define the power norm of the sequence Y as

$${\displaystyle \parallel Y{\parallel }_{\mathcal{P}}=\underset{N\to \infty }{lim}{\displaystyle \frac{1}{2N+1}}\sum _{k=-N}^N\mathbf{E}\left[|y_k{|}^2\right].}$$

Supposing that $\parallel Y{\parallel }_{\mathcal{P}}$ and $\parallel W{\parallel }_{\mathcal{P}}$ are finite, for a given system F with an input signal $W={\left\{w_k\right\}}_{k\in \mathbb{Z}}$, a root mean square gain can be defined in the form

$$Q(F,W)={\displaystyle \frac{\parallel Y{\parallel }_{\mathcal{P}}}{\parallel W{\parallel }_{\mathcal{P}}}}·$$

Definition 3.

Let W be a stationary Gaussian sequence with a bounded mean anisotropy $\overline{\mathbf{A}}\left(W\right)⩽a$, with  $a⩾0$ being a known scalar value. Then anisotropic norm of the system F is defined as

$${⫴F⫴}_a=\underset{\overline{\mathbf{A}}\left(W\right)⩽a}{sup}Q(F,W).$$

(4)

Thus, the anisotropic norm $⫴F{⫴}_a$ sets the stochastic gain by the system F of the input signal W, which leads to the last (sixth) axiom of the anisotropy-based theory:

• Anisotropic norm of the system.

Consequently, we obtain the following six axioms of the anisotropy-based control theory:

• Linear discrete-time system;
• Infinite horizon;
• Anisotropy of a random vector;
• Mean anisotropy of a random sequence;
• Stationary random sequence;
• Anisotropic norm of the system.

Presently, it is necessary to say a few words about one property of the mean anisotropy, which is not mentioned anywhere. The fact is that from a set-theoretic point of view, the mean anisotropy generates an equivalence relation on the set of input random sequences. Indeed, the mean anisotropy allows for any pair of sequences to introduce a binary equivalence relation ∼ on the set of input sequences $\mathcal{W}$ and the term “equianisotropic”, which means that the sequences have equal mean anisotropies. The equianisotropy relation introduced in this way satisfies all the equivalence conditions [21], i.e., for any sequences $\left\{w_k\right\},\left\{v_k\right\},\left\{u_k\right\}\in \mathcal{W}$, this relation is

• Reflexive as

  $$\left\{w_k\right\}\sim \left\{w_k\right\};$$
• Symmetric as

  $$\left(\left\{w_k\right\}\sim \left\{v_k\right\}\right)\Rightarrow \left(\left\{v_k\right\}\sim \left\{w_k\right\}\right);$$
• Transitive as

  $$\left(\left\{w_k\right\}\sim \left\{v_k\right\}\right)\wedge \left(\left\{v_k\right\}\sim \left\{u_k\right\}\right)\Rightarrow \left(\left\{w_k\right\}\sim \left\{u_k\right\}\right).$$

Like any equivalence relation, the equianisotropy relation on the set of input sequences $\mathcal{W}$ generates a partition of the set $\mathcal{W}$ into disjoint equivalence classes, and each class ${\mathcal{K}}_a$ is characterized by its mean anisotropy level a.

The set of all equivalence classes ${\mathcal{K}}_a$ of the set $\mathcal{W}$ with respect to the equianisotropy relation ∼ forms a factor set $\mathcal{W}/\sim$. Hence, the anisotropic norm (4) of the system is an induced norm on a subset of the factor set $\mathcal{W}/\sim$, and the “power” of this subset is determined by the condition $\overline{\mathbf{A}}\left({\mathcal{K}}_a\right)⩽a.$

Noting that axioms 3–5 serve as a detailed and consistent description of signals in the anisotropy-based theory, these three axioms can be grouped and reformulated as follows, into an equivalence relation on the set of input signals. Taking into account the properties of the mean anisotropy and keeping in mind the goal of expanding the anisotropy-based theory, the axiomatics of the anisotropy-based theory can be reformulated in the following way:

• Linear dynamics;
• Infinite horizon;
• Equivalence relation on the set of input signals;
• Induced norm on a subset of the equivalence classes factor set.

Based on these axioms, frequency and state-space domain approaches to the analysis of linear discrete-time systems appeared in [7,9]. Finally, a state-space formulae of the anisotropic norm is given by the following theorem [7].

Theorem 1.

Let the asymptotically stable system (1) satisfy inequality

$$\frac{{\parallel F\parallel }_2^2}{m}<{\parallel F\parallel }_{\infty }^2$$

with ${\parallel F\parallel }_2$ and ${\parallel F\parallel }_{\infty }$ being ${\mathcal{H}}_2$ and ${\mathcal{H}}_{\infty }$ norms of the system F.

Then, for $a>0$ there exists the unique pair $(q,R)$ of the scalar parameter $q\in \left({0,\parallel F\parallel }_{\infty }^{-2}\right)$ and the admissible solution R of the Riccati equation

$$\begin{array}{ccc}\hfill R& =& A^{\mathrm{T}}RA+q{C}^{\mathrm{T}}C+L^{\mathrm{T}}{\mathsf{\Sigma }}^{-1}L,\hfill \end{array}$$

(5)

$$\begin{array}{ccc}\hfill \mathsf{\Sigma }& =& {(I-q{D}^{\mathrm{T}}D-B^{\mathrm{T}}RB)}^{-1},\hfill \end{array}$$

(6)

$$\begin{array}{ccc}\hfill L& =& \mathsf{\Sigma }(B^{\mathrm{T}}RA+q{D}^{\mathrm{T}}C).\hfill \end{array}$$

(7)

such that

$$-\frac{1}{2}lndet\left(\frac{m{\Sigma }}{\mathrm{tr}(LP{L}^{\mathrm{T}}+\mathsf{\Sigma })}\right)=a,$$

(8)

where P is a solution of the Lyapunov equation

$$P=(A+BL)P{(A+BL)}^{\mathrm{T}}+B\Sigma B^{\mathrm{T}}.$$

The anisotropic norm (4) of system (1) is given by

$$⫴F{⫴}_a^2=\frac{1}{q}\left(1-\frac{m}{\mathrm{tr}(LP{L}^{\mathrm{T}}+\mathsf{\Sigma })}\right).$$

(9)

The state-space formulae given in the above theorem can easily be utilized for the numerical computation of the anisotropic norm using Newton iterations [9].

3. $\mathbf{\sigma }$-Entropy Analysis Method

In this section, we will provide axiomatics of the spectral entropy approach to an analysis of linear time invariant systems.

3.1. Basic Definitions of $\sigma$-Entropy Theory

Consider a plant described by a linear discrete stationary system with a zero initial condition:

$$\left\{\begin{array}{cc}{x}_{k+1}\hfill & =A{x}_k+B{w}_k,x_0=0,\hfill \\ z_k\hfill & =C{x}_k+D{w}_k,k=0,1,2,\dots ,\hfill \end{array}\right.$$

(10)

where $A\in {\mathbb{R}}^{n\times n}$, $B\in {\mathbb{R}}^{n\times m}$, $C\in {\mathbb{R}}^{p\times n}$, $D\in {\mathbb{R}}^{p\times m}$, $x_k\in {\mathbb{R}}^n$ is a state vector, $z_k\in {\mathbb{R}}^p$ is an output vector, and $w_k\in {\mathbb{R}}^m$ is a random input disturbance. The transfer function of system (10) is given by $F\left(z\right)=C{(zI-A)}^{-1}B+D$. System (10) is assumed to be asymptotically stable (i.e., the spectral radius $\varrho$ of the matrix A is less than one: $\varrho \left(A\right)<1$), the system is supposed to be controllable and observable. The system is affected by a random sequence $W={\left\{w_k\right\}}_{k\in \mathbb{Z}}$ with a zero mean

$$\mathbf{E}\left[w_k\right]=0,$$

(11)

and finite ${\mathit{l}}_2$ norm $\parallel W{\parallel }_2$

$${\displaystyle \parallel W{\parallel }_2^2=\sum _{k=-\infty }^{+\infty }\mathbf{E}\left[|w_k{|}^2\right]<\infty ,}$$

(12)

or finite power norm $\parallel W{\parallel }_{\mathcal{P}}$

$$\parallel W{\parallel }_{\mathcal{P}}^2=\underset{N\to +\infty }{lim}\left\{{\displaystyle {\displaystyle \frac{1}{2N+1}}\sum _{k=-N}^N\mathbf{E}\left[|w_k{|}^2\right]}\right\}<\infty .$$

(13)

Hereinafter, $\left|w_k\right|$ is the Euclidian norm of the vector $w_k\in {\mathbb{R}}^m$.

Obviously, analysis of system (10) with the input signals (12) or (13) are two different problems, and they have been solved separately in [15], but there is a lot in common between them. To demonstrate this, following the paper [14], unify the description of the input signals and introduce the $\mathfrak{N}$ norm of the signal in the form

$$\parallel W{\parallel }_{\mathfrak{N}}^2=\mathfrak{N}\left(w_k^{\mathrm{T}}{w}_k\right),$$

(14)

where $\mathfrak{N}$ is a linear operator that transforms the Euclidean norm of the vector $|w_k{|}^2=w_k^{\mathrm{T}}{w}_k$ into $l_2$ or into the power norm of the stochastic signal according to the following rule:

$$\mathfrak{N}(·)=\left\{\begin{array}{ccc}{\displaystyle \sum _{k=-\infty }^{k=+\infty }\mathbf{E}[·]}& & \mathrm{for}{l}_2\mathrm{norm},\hfill \\ \underset{N\to \infty }{lim}\left\{{\displaystyle {\displaystyle \frac{1}{2N+1}}\sum _{k=-N}^{k=+N}\mathbf{E}[·]}\right\}& & \mathrm{for}\mathcal{P}\mathrm{norm}.\hfill \end{array}\right.$$

Define the correlation convolution of the $K\left(l\right)$ input sequence $\{w_k$} via $\mathfrak{N}$ operator, as follows:

$$K\left(l\right)=\mathfrak{N}\left(w_{k+l}^{\mathrm{T}}{w}_k\right).$$

Fourier transform of the correlation convolution provides:

$$S_w\left(\omega \right)={\displaystyle \frac{1}{2\pi }}\sum _{l=-\infty }^{+\infty }K\left(l\right){\mathrm{e}}^{-i\omega l}.$$

Matrix $S_w\left(\omega \right)$, being a spectral density matrix of the sequence W, is Hermitian positive:

$$S_w\left(\omega \right)=S_w^{*}\left(\omega \right)>0$$

(15)

with ${}^{*}$ being a hermitian conjugation.

The inverse Fourier transform allows us to express the correlation convolution of $K\left(l\right)$ in a standard way via the spectral density matrix $S_w\left(\omega \right)$:

$${\displaystyle K\left(l\right)=\underset{-\pi }{\overset{+\pi }{\int }}{S}_w\left(\omega \right){\mathrm{e}}^{i\omega l}d\omega .}$$

Since $\mathfrak{N}$ norm of the input sequence W is equal to

$${\parallel W\parallel }_{\mathfrak{N}}^2=\mathrm{tr}K\left(0\right),$$

then $\mathfrak{N}$ norm of the sequence W can be expressed in terms of the spectral density matrix $S_w\left(\omega \right)$ as

$${\displaystyle \parallel W{\parallel }_{\mathfrak{N}}^2=\underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}{S}_w\left(\omega \right)d\omega .}$$

Similarly, $\mathfrak{N}$ norm of the output sequence Z is

$${\displaystyle \parallel Z{\parallel }_{\mathfrak{N}}^2=\underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}{S}_z\left(\omega \right)d\omega ,}$$

where $S_z\left(\omega \right)$ is a spectral density of the output sequence Z, which is expressed in terms of the transfer matrix of the system (10) $G\left(z\right)=C{(zI-A)}^{-1}B+D$ and the spectral density of the input signal $S_w\left(\omega \right)$, as follows [22]:

$$S_z\left(\omega \right)=G\left(e^{i\omega }\right)S_w\left(\omega \right)G^{*}\left(e^{i\omega }\right).$$

(16)

Consequently, $\mathfrak{N}$ norm of the output Z equals to

$${\displaystyle \parallel Z{\parallel }_{\mathfrak{N}}^2=\underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}\left[G^{*}\left(e^{i\omega }\right)G\left(e^{i\omega }\right)S_w\left(\omega \right)\right]d\omega .}$$

Presently, define the gain $\mathsf{\Theta }$ for system (10) with the input sequence W, which satisfies either condition (12), or (13), as the ratio of the $\mathfrak{N}$ norm of the output sequence Z to $\mathfrak{N}$ norm of the input sequence W:

$${\mathsf{\Theta }}^2={\displaystyle \frac{\parallel Z{\parallel }_{\mathfrak{N}}^2}{\parallel W{\parallel }_{\mathfrak{N}}^2}}={\displaystyle \frac{{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}\left[\mathsf{\Lambda }\left(\omega \right)S_w\left(\omega \right)\right]d\omega }}{{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}{S}_w\left(\omega \right)d\omega }}},$$

(17)

where $\mathsf{\Lambda }\left(\omega \right)=G^{*}\left(e^{i\omega }\right)G\left(e^{i\omega }\right)$.

Presently, following the paper [14], introduce an integral characteristic of the input signal, namely σ-entropy:

$${\displaystyle \mathfrak{S}\left(W\right)=-{\displaystyle \frac{\alpha }{2}}\underset{-\pi }{\overset{+\pi }{\int }}lndet{\displaystyle \frac{\beta S_w\left(\omega \right)}{{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}{S}_w\left({\omega }^{\prime }\right)d{\omega }^{\prime }}}}d\omega .}$$

(18)

The concept of the $\sigma$-entropy is introduced axiomatically, and only the functional relationship between $\mathfrak{S}$ and $S_w$ is strictly postulated, while the coefficients before and under the integral should be determined from some additional assumptions. Briefly discuss these coefficients: first of all, the minus sign before the integral has a physical sense, as entropy should be a non-negative value, and under $ln(·)$ there is a value less than one, so the minus is necessary. The scalar parameters $\alpha$ and $\beta$ should be positive, and specific values are not important, but they could make sense while calculating the total entropy of the system, which contains the control plant (10).

Define the $\sigma$-entropy norm $⫴F{⫴}_s$ of system (10) for a random input sequence (12) or (13) as the maximum gain (17) for all input sequences, $\sigma$-entropy (18) of which does not exceed the given non-negative value s:

$$⫴F{⫴}_s^2=\underset{\mathfrak{S}\left(W\right)⩽s}{sup}{\mathsf{\Theta }}^2=\underset{\mathfrak{S}\left(W\right)⩽s}{sup}{\displaystyle \frac{{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}\left[\mathsf{\Lambda }\left(\omega \right)S_w\left(\omega \right)\right]d\omega }}{{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}{S}_w\left(\omega \right)d\omega }}}·$$

(19)

Inequality $\mathfrak{S}\left(W\right)⩽s$ with $\mathfrak{S}\left(W\right)$ given by (18) defines the set of all possible random signals W satisfying (11)–(13), whose $\sigma$-entropy does not exceed the scalar value $s⩾0$.

3.2. $\sigma$-Entropy Norm Computation in the Frequency Domain

Before calculating $\sigma$-entropy norm (19), consider a property of $\sigma$-entropy (18), which will allow us to calculate the norm of the system in two stages. From a set-theoretic point of view, $\sigma$-entropy generates an equivalence relation on the set of input sequences. Indeed, the $\sigma$-entropy allows one to introduce the binary relation of equivalence∼for any pair of sequences; this relation is required to be “isoentropic” (i.e., to have equal $\sigma$-entropies) on the set of input sequences $\mathcal{W}$. The $\sigma$-entropy equality relation satisfies all equivalence conditions [21], i.e., for all input signals $\left\{w_k\right\},\left\{v_k\right\},\left\{u_k\right\}\in \mathcal{W}$, this relation is

• Reflexive as

  $$\left\{w_k\right\}\sim \left\{w_k\right\};$$
• Symmetric as

  $$\left(\left\{w_k\right\}\sim \left\{v_k\right\}\right)\Rightarrow \left(\left\{v_k\right\}\sim \left\{w_k\right\}\right);$$
• Transitive as

  $$\left(\left\{w_k\right\}\sim \left\{v_k\right\}\right)\wedge \left(\left\{v_k\right\}\sim \left\{u_k\right\}\right)\Rightarrow \left(\left\{w_k\right\}\sim \left\{u_k\right\}\right).$$

Like any equivalence relation, the isoentropy relation on the set of input sequences $\mathcal{W}$ generates a partition of the set $\mathcal{W}$ into disjoint equivalence classes, and each class ${\mathcal{K}}_s$ is characterized by its $\sigma$-entropy value s. Define the $\sigma$-entropy gain of system (10) as the maximum gain (17) ${\mathsf{\Theta }}_s$ on the isoentropic class ${\mathcal{K}}_s$ of input sequences, $\sigma$-entropy (18), of which is equal to the given value s, as:

$${\mathsf{\Theta }}_s^2=\underset{\mathfrak{S}\left(W\right)=s}{sup}{\mathsf{\Theta }}^2=\underset{\mathfrak{S}\left(W\right)=s}{sup}{\displaystyle \frac{{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}\left[\mathsf{\Lambda }\left(\omega \right)S_w\left(\omega \right)\right]d\omega }}{{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}{S}_w\left(\omega \right)d\omega }}}·$$

(20)

The set of all equivalence classes ${\mathcal{K}}_s$ of the set $\mathcal{W}$ with respect to the isoentropy relation∼forms a factor set $\mathcal{W}/\sim$. Hence, the $\sigma$-entropy norm (19) of the system is an induced norm on a subset of the factor set $\mathcal{W}/\sim$; the “power” of this subset is defined by

$$\mathfrak{S}\left({\mathcal{K}}_s\right)⩽s.$$

Theorem 2.

For any $s⩾0$ σ-entropy gain (20) of the system (10), the input is a stochastic discrete-time signal with a finite $\mathfrak{N}$ norm and is calculated as

$${\mathsf{\Theta }}_s^2={\displaystyle \frac{{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}\left\{\mathsf{\Lambda }\left(\omega \right){\left[I-q_s\mathsf{\Lambda }\left(\omega \right)\right]}^{-1}\right\}d\omega }}{{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}{\left[I-q_s\mathsf{\Lambda }\left(\omega \right)\right]}^{-1}d\omega }}},$$

(21)

where parameter $q_s\in \left[0,\parallel F{\parallel }_{\infty }^{-2}\right)$ is a unique solution of the equation

$${\displaystyle -{\displaystyle \frac{\alpha }{2}}\underset{-\pi }{\overset{+\pi }{\int }}lndet{\displaystyle \frac{\beta {\left[I-q_s\mathsf{\Lambda }\left(\omega \right)\right]}^{-1}}{{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}{\left[I-q_s\mathsf{\Lambda }\left({\omega }^{\prime }\right)\right]}^{-1}d{\omega }^{\prime }}}}d\omega =s.}$$

(22)

Proof. 

The definition of $\sigma$-entropy gain ${\mathsf{\Theta }}_s^2$ can be reformulated in the following form:

$$\begin{array}{c}{\mathsf{\Theta }}_s^2=\underset{S_w\left(\omega \right)}{sup}\left\{{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}\left[\mathsf{\Lambda }\left(\omega \right)S_w\left(\omega \right)\right]d\omega :}\right.\hfill \\ \hfill \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}{\displaystyle -{\displaystyle \frac{\alpha }{2}}\underset{-\pi }{\overset{+\pi }{\int }}lndet\left(\beta S_w\left(\omega \right)\right)d\omega =s,\left.{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}{S}_w\left(\omega \right)d\omega =1}\right\}.}\end{array}$$

(23)

Find the maximum of this expression by the method of indefinite Lagrange multipliers. In this case, the Lagrange function has the following form:

$$\begin{array}{c}{\displaystyle L\left[S_w\right]=\underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}\left[\mathsf{\Lambda }\left(\omega \right)S_w\left(\omega \right)\right]d\omega +}\hfill \\ \hfill \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}+{\lambda }_1\left\{{\displaystyle s+{\displaystyle \frac{\alpha }{2}}\underset{-\pi }{\overset{+\pi }{\int }}lndet\left(\beta S_w\left(\omega \right)\right)d\omega }\right\}+{\lambda }_2\left\{{\displaystyle 1-\underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}{S}_w\left(\omega \right)d\omega }\right\},\end{array}$$

where ${\lambda }_1$, ${\lambda }_2$ are Lagrange multipliers. The solution of the extremum-seeking problem can be reduced to finding a spectral density on which the variation $\delta L\left[S_w\right]$ of the Lagrange function

$${\displaystyle \delta L\left[S_w\right]=\underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}\left[\mathsf{\Lambda }\left(\omega \right)\delta S_w\left(\omega \right)\right]d\omega +{\displaystyle \frac{\alpha {\lambda }_1}{2}}\underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}\left[S_w^{-1}\left(\omega \right)\delta S_w\left(\omega \right)\right]d\omega -{\lambda }_2\underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}\left[\delta S_w\left(\omega \right)\right]d\omega }$$

is equal to zero

$${\displaystyle \delta L\left[S_w\right]=\underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}\left\{\left[\mathsf{\Lambda }\left(\omega \right)+{\displaystyle \frac{\alpha {\lambda }_1}{2}}{S}_w^{-1}\left(\omega \right)-{\lambda }_{2}I\right]\delta S_w\left(\omega \right)\right\}d\omega =0.}$$

In order to find $\delta L\left[S_w\right]=0$, it is necessary and sufficient that the expression in the square brackets equals zero:

$$\mathsf{\Lambda }\left(\omega \right)+{\displaystyle \frac{\alpha {\lambda }_1}{2}}{S}_w^{-1}\left(\omega \right)-{\lambda }_{2}I=0.$$

Thus, the spectral density matrix is equal to

$$S_w\left(\omega \right)={\displaystyle \frac{\alpha {\lambda }_1}{2{\lambda }_2}}{\left[I-{\displaystyle \frac{1}{{\lambda }_2}}\mathsf{\Lambda }\left(\omega \right)\right]}^{-1},$$

or, equivalently,

$$S_w\left(\omega \right)=\alpha p{\left[I-q\mathsf{\Lambda }\left(\omega \right)\right]}^{-1},$$

(24)

where the parameters $p={\displaystyle \frac{{\lambda }_1}{2{\lambda }_2}}$ and $q={\displaystyle \frac{1}{{\lambda }_2}}$ can be found from the following conditions:

$$\mathfrak{S}\left(W\right)=s,$$

(25)

$${\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}{S}_w\left(\omega \right)d\omega =1.}$$

(26)

According to the Equation (24), the spectral density matrix $S_w\left(\omega \right)$ commutes with the matrix $\mathsf{\Lambda }\left(\omega \right)$:

$$\mathsf{\Lambda }\left(\omega \right)S_w\left(\omega \right)-S_w\left(\omega \right)\mathsf{\Lambda }\left(\omega \right)=0.$$

In addition, matrices $\mathsf{\Lambda }\left(\omega \right)$ and $S_w\left(\omega \right)$ are Hermitian non-negatively and positively defined, as $\mathsf{\Lambda }\left(\omega \right)$ is defined in the form $\mathsf{\Lambda }\left(\omega \right)=G^{*}\left(\omega \right)G\left(\omega \right),$ and matrix $S_w\left(\omega \right)$ is given by Condition (15). It leads to

$$\begin{array}{cc}\hfill {\mathsf{\Lambda }}^{*}\left(\omega \right)& =\mathsf{\Lambda }\left(\omega \right)⩾0,\hfill \\ \hfill S_w^{*}\left(\omega \right)& =S_w\left(\omega \right)>0.\hfill \end{array}$$

Hence, there exists a unitary matrix $U^{*}\left(\omega \right)=U^{-1}\left(\omega \right)$, which brings these two matrices to a diagonal form [23] simultaneously, and the eigenvalues ${\lambda }_i\left(\omega \right)$ and $s_i\left(\omega \right)$ of matrices $\mathsf{\Lambda }\left(\omega \right)$ and $S_w\left(\omega \right)$ are connected by the ratio

$$s_i\left(\omega \right)={\displaystyle \frac{\alpha p}{1-q{\lambda }_i\left(\omega \right)}}·$$

For all the eigenvalues $s_i\left(\omega \right)$ of matrix $S_w\left(\omega \right)$ to be positive, it is necessary that the parameter q is limited:

$$0⩽q<\underset{\omega }{max}{\lambda }_{\max }^{-1}\left(\omega \right)=\parallel F{\parallel }_{\infty }^{-2},$$

where ${\lambda }_{\max }\left(\omega \right)$ is the maximum eigenvalue of matrix $\mathsf{\Lambda }\left(\omega \right)$.

To find the parameter p, apply the normalization condition for the spectral density of the input signal (26):

$${\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}{S}_w\left(\omega \right)d\omega =1.}$$

Substituting the expression for the spectral density $S_w\left(\omega \right)$ from the Equation (24) into this equality, we straightforwardly obtain

$$p={\displaystyle \frac{1}{{\displaystyle \alpha \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}{\left[I-q\mathsf{\Lambda }\left(\omega \right)\right]}^{-1}d\omega }}}·$$

Hence, the spectral density, for which gain (23) reaches its maximum value, is equal to

$$S_w(q,\omega )={\displaystyle \frac{{\left[I-q\mathsf{\Lambda }\left(\omega \right)\right]}^{-1}}{{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}{\left[I-q\mathsf{\Lambda }\left({\omega }^{\prime }\right)\right]}^{-1}d{\omega }^{\prime }}}}.$$

(27)

Notice that the spectral density matrix $S_w(q,\omega )$ is proportional to the resolvent

$R\left(\mu \right)={(\mathsf{\Lambda }-\mu I)}^{-1}$ of the linear operator $\mathsf{\Lambda }$:

$$S_w(q,\omega )\sim {[\mathsf{\Lambda }\left(\omega \right)-{\displaystyle \frac{1}{q}}I]}^{-1}.$$

Since the resolvent is an analytical function of each connected component of the resolvent set of the operator $\mathsf{\Lambda }$ [24], the spectral density of $S_w\left(q\right)$ is also an analytical function of $q\in \left[0,\parallel F{\parallel }_{\infty }^{-2}\right)$, and the variable q parametrizes the set of spectral density matrices.

Substitute the expression for spectral density (27) into the definition of $\sigma$-entropy (18):

$${\displaystyle \mathfrak{S}\left(q\right)=-{\displaystyle \frac{\alpha }{2}}\underset{-\pi }{\overset{+\pi }{\int }}lndet{\displaystyle \frac{\beta {\left[I-q\mathsf{\Lambda }\left(\omega \right)\right]}^{-1}}{{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}{\left[I-q\mathsf{\Lambda }\left({\omega }^{\prime }\right)\right]}^{-1}d{\omega }^{\prime }}}}d\omega .}$$

As the function $\mathfrak{S}\left(q\right)$ is analytic, strictly increasing, and convex [10,25] on the half-interval $\left[{0,\parallel F\parallel }_{\infty }^{-2}\right)$, Equation (25) has a unique solution

$$q_s={\mathfrak{S}}^{-1}\left(s\right).$$

Substituting expression (27) into the formula for the $\sigma$-entropy gain (20), one can obtain

$${\mathsf{\Theta }}_s^2={\displaystyle \frac{{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}\left\{\mathsf{\Lambda }\left(\omega \right){\left[I-q_s\mathsf{\Lambda }\left(\omega \right)\right]}^{-1}\right\}d\omega }}{{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}{\left[I-q_s\mathsf{\Lambda }\left(\omega \right)\right]}^{-1}d\omega }}}$$

with parameter $q_s\in \left[0,\parallel F{\parallel }_{\infty }^{-2}\right)$ being the unique solution of

$${\displaystyle -{\displaystyle \frac{\alpha }{2}}\underset{-\pi }{\overset{+\pi }{\int }}lndet{\displaystyle \frac{\beta {\left[I-q_s\mathsf{\Lambda }\left(\omega \right)\right]}^{-1}}{{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}{\left[I-q_s\mathsf{\Lambda }\left({\omega }^{\prime }\right)\right]}^{-1}d{\omega }^{\prime }}}}d\omega =s,}$$

that finishes the proof of Theorem 2. □

Thus, the $\sigma$-entropy gain ${\mathsf{\Theta }}_s$, i.e., the maximum gain on the isoentropy class of input sequences ${\mathcal{K}}_s$, the $\sigma$-entropy of which is equal to a given value s, can be calculated by Formula (21), while the log-determinant Equation (22) uniquely connects the parameter $q_s$ and the value of the $\sigma$-entropy $\mathfrak{S}\left({\mathcal{K}}_s\right)=s$ of the isoentropy class ${\mathcal{K}}_s$. Transform (21) and (22) into

$${\mathsf{\Theta }}^2\left(q\right)={\displaystyle \frac{{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}\left\{\mathsf{\Lambda }\left(\omega \right){\left[I-q\mathsf{\Lambda }\left(\omega \right)\right]}^{-1}\right\}d\omega }}{{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}{\left[I-q\mathsf{\Lambda }\left(\omega \right)\right]}^{-1}d\omega }}},$$

and

$${\displaystyle s_q=-{\displaystyle \frac{\alpha }{2}}\underset{-\pi }{\overset{+\pi }{\int }}lndet{\displaystyle \frac{\beta {\left[I-q\mathsf{\Lambda }\left(\omega \right)\right]}^{-1}}{{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}{\left[I-q\mathsf{\Lambda }\left({\omega }^{\prime }\right)\right]}^{-1}d{\omega }^{\prime }}}}d\omega ,}$$

make sure that the variable q parametrizes both $\sigma$-entropy gain $\mathsf{\Theta }\left(q\right)$ of isoentropy classes ${\mathcal{K}}_q$, and $\sigma$-entropy $s_q$ of classes ${\mathcal{K}}_q$, and the set of isoentropy classes itself, i.e., the factor set $\mathcal{W}/\sim$. Moreover, both of these dependencies are strictly increasing functions of the variable q [10,25]. This fact, together with the partitioning of the input signals set into equivalence classes, allows for calculating the $\sigma$-entropy norm (19) of the system $⫴F{⫴}_s^2={sup}_{\mathfrak{S}\left(S\right)⩽s}{\mathsf{\Theta }}^2$ in two steps:

• Find the maximum gain on the equivalence class (this problem is solved by Theorem 2);
• Determine the maximum value of $\sigma$-entropy gains $\mathsf{\Theta }\left(q\right)$ on a subset of the ${\mathcal{W}}_{q_s}$ factor set $\mathcal{W}/\sim$, where the subset ${\mathcal{W}}_{q_s}$ is given by the condition $0⩽q⩽q_s$:

  $$⫴F{⫴}_s^2=\underset{0⩽q⩽q_s}{sup}{\mathsf{\Theta }}^2\left(q\right).$$

Function $\mathsf{\Theta }\left(q\right)$ is strictly increasing and reaches its maximum at the right end of the interval, hence:

$$⫴F{⫴}_s^2={\displaystyle \frac{{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}\left\{\mathsf{\Lambda }\left(\omega \right){\left[I-q_s\mathsf{\Lambda }\left(\omega \right)\right]}^{-1}\right\}d\omega }}{{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}{\left[I-q_s\mathsf{\Lambda }\left(\omega \right)\right]}^{-1}d\omega }}}$$

(28)

with parameter $q_s\in \left[0,\parallel F{\parallel }_{\infty }^{-2}\right)$ being a unique solution of

$${\displaystyle -{\displaystyle \frac{\alpha }{2}}\underset{-\pi }{\overset{+\pi }{\int }}lndet{\displaystyle \frac{\beta {\left[I-q_s\mathsf{\Lambda }\left(\omega \right)\right]}^{-1}}{{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}{\left[I-q_s\mathsf{\Lambda }\left({\omega }^{\prime }\right)\right]}^{-1}d{\omega }^{\prime }}}}d\omega =s.}$$

(29)

3.3. $\sigma$-Entropy Norm Computation in the State Space

Before proving the theorem on the $\sigma$-entropy norm of a linear discrete-time stationary stochastic system, formulate the lemma of the inner discrete-time system [26,27] and proof the lemma on the worst case spectral density of the input signal.

Definition 4.

System $\mathsf{{\rm Y}}\left(z\right)=C{(zI-A)}^{-1}B+D$ is called inner if Y*(z) Y(z) = I.

Lemma 1.

Let $\mathsf{{\rm Y}}\left(z\right)=\left[\begin{array}{cc}A& B\\ C& D\end{array}\right]\in R{H}^{\infty }$, and X is an observability gramian of the system Υ. The system Υ is inner if and only if the following three conditions are satisfied:

$$\begin{array}{ccc}\hfill A^{\mathrm{T}}XA-X+C^{\mathrm{T}}C& =& 0,\hfill \\ \hfill B^{\mathrm{T}}XA+D^{\mathrm{T}}C& =& 0,\hfill \\ \hfill B^{\mathrm{T}}XB+D^{\mathrm{T}}D& =& I\hfill \end{array}$$

where $R{H}^{\infty }$ denotes a set of strictly proper rational matrices.

Lemma 2.

Let $F\left(z\right)=\left[\begin{array}{cc}A& B\\ C& D\end{array}\right]$ is a transfer matrix of a discrete-time system F and  $0⩽{q<\parallel F\parallel }_{\infty }^2$. Then, the matrix

$$G\left(z\right)=\left[\begin{array}{cc}A+BL& B{M}^{1/2}\\ L& M^{1/2}\end{array}\right]$$

factorizes the matrix $S_{*}\left(\omega \right)={\left[I-q{F}^{*}\left(e^{i\omega }\right)F\left(e^{i\omega }\right)\right]}^{-1}$ in the following form:

$$S_{*}\left(\omega \right)=G\left(e^{i\omega }\right)G^{*}\left(e^{i\omega }\right),$$

(30)

and matrices L and M correspond to the admissible solution of the Riccati equation R

$$\begin{array}{ccc}\hfill R& =& A^{\mathrm{T}}RA+q{C}^{\mathrm{T}}C+L^{\mathrm{T}}{M}^{-1}L,\hfill \end{array}$$

(31)

$$\begin{array}{ccc}\hfill L& =& M(B^{\mathrm{T}}RA+q{D}^{\mathrm{T}}C),\hfill \end{array}$$

(32)

$$\begin{array}{ccc}\hfill M& =& {(I-B^{\mathrm{T}}RB-q{D}^{\mathrm{T}}D)}^{-1}.\hfill \end{array}$$

(33)

Proof. 

The condition $0⩽{q<\parallel F\parallel }_{\infty }^2$ means that $I-q{F}^{*}\left(e^{i\omega }\right)F\left(e^{i\omega }\right)>0$ for any $\omega \in \left[-\pi ,\pi \right]$. Positive definiteness of matrix $I-q{F}^{*}\left(e^{i\omega }\right)F\left(e^{i\omega }\right)$ and its inverse matrix guarantee the existence of factorization $S_{*}\left(\omega \right)=G\left(e^{i\omega }\right)G^{*}\left(e^{i\omega }\right)$. Consequently, Equation (30) may be rewritten in the form $G^{-*}\left(e^{i\omega }\right)G^{-1}\left(e^{i\omega }\right)=I-q{F}^{*}\left(e^{i\omega }\right)F\left(e^{i\omega }\right)$ where $G^{-*}={\left(G^{*}\right)}^{-1}$. Denoting $\mathsf{{\rm Y}}\left(e^{i\omega }\right)=\left[\begin{array}{c}\sqrt{q}F\left(e^{i\omega }\right)\\ G^{-1}\left(e^{i\omega }\right)\end{array}\right]$, the equation ${\mathsf{{\rm Y}}}^{*}\left(e^{i\omega }\right)\mathsf{{\rm Y}}\left(e^{i\omega }\right)=I$ is satisfied, which means that matrix $\mathsf{{\rm Y}}\left(e^{i\omega }\right)$ is the inner.

Chose matrix $G\left(z\right)$ in the form

$$G\left(z\right)=\left[\begin{array}{cc}A+BL& B{M}^{1/2}\\ L& M^{1/2}\end{array}\right]$$

where L and $M=M^{\mathrm{T}}>0$ are arbitrary matrices of the corresponding dimension. By direct calculation, the inverse matrix $G^{-1}\left(z\right)$ is defined as

$$G^{-1}\left(z\right)=\left[\begin{array}{cc}A& B\\ M^{-1/2}L& M^{-1/2}\end{array}\right].$$

It is natural to consider the matrix $\mathsf{{\rm Y}}\left(z\right)$ as a transfer matrix of the system combined by two parallel subsystems $F\left(z\right)$ and $G^{-1}\left(z\right)$, then this transfer matrix can be calculated using the formula [27]

$$\mathsf{{\rm Y}}\left(z\right)=\left[\begin{array}{c}\sqrt{q}F\left(z\right)\\ G^{-1}\left(z\right)\end{array}\right]=\left[\begin{array}{ccc}A& 0& B\\ 0& A& B\\ \sqrt{q}C& 0& \sqrt{q}D\\ 0& -M^{-1/2}L& M^{-1/2}\end{array}\right].$$

Since the dynamic parts of the two subsystems are identical, the state space realization of system $\mathsf{{\rm Y}}\left(z\right)$ has the following form:

$$\mathsf{{\rm Y}}\left(z\right)=\left[\begin{array}{cc}A& B\\ \sqrt{q}C& \sqrt{q}D\\ -M^{-1/2}L& M^{-1/2}\end{array}\right].$$

Applying Lemma 1 to the system $\mathsf{{\rm Y}}\left(z\right)$, Riccati Equations (31)–(33) follow immediately. Consequently, the system $G\left(e^{i\omega }\right)$ factorizes matrix $S_{*}\left(\omega \right)={\left[I-q{F}^{*}\left(e^{i\omega }\right)F\left(e^{i\omega }\right)\right]}^{-1}$, while matrices L and M correspond to an admissible solution R of Riccati Equations (31)–(33). □

Theorem 3.

Let F be a system with a state-space realization (10). Let W be a stochastic input disturbance with bounded σ-entropy, i.e., $\mathfrak{S}\left(S\right)=s$ with $s⩾0$. Then, the σ-entropy norm of system (10) is calculated by

$$⫴F{⫴}_s^2=\frac{1}{q}\left\{1-\frac{m}{\mathrm{tr}\left(LP{L}^{\mathrm{T}}+M\right)}\right\}$$

(34)

where P satisfies the Lyapunov equation

$$P=(A+BL)P{(A+BL)}^{\mathrm{T}}+BM{B}^{\mathrm{T}},$$

(35)

matrices L and M correspond to an admissible solution R of Riccati Equations (31)–(33) and the special type equation

$$-\frac{1}{2}lndet\frac{mM}{\mathrm{tr}(LP{L}^{\mathrm{T}}+M)}=s.$$

(36)

Proof. 

Before we obtain the main result of the theorem, first define certain parameters $\alpha$ and $\beta$. For the convenience of the calculation set

$$\alpha =\frac{1}{2\pi },\beta =2\pi m.$$

$\sigma$-Entropy norm, defined in the form (28), can be transformed to the following:

$$\begin{array}{c}\hfill ⫴F{⫴}_s^2={\displaystyle \frac{{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}\left\{\frac{1}{q}\left[I-(I-q\mathsf{\Lambda }(\omega \left)\right)\right]{\left[I-q\mathsf{\Lambda }\left(\omega \right)\right]}^{-1}\right\}d\omega }}{{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}{\left[I-q\mathsf{\Lambda }\left(\omega \right)\right]}^{-1}d\omega }}}=\\ \hfill =\frac{1}{q}-\frac{2\pi m}{q}\frac{1}{{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}{\left[I-q{F}^{*}\left(e^{i\omega }\right)F\left(e^{i\omega }\right)\right]}^{-1}d\omega }}.\end{array}$$

(37)

Application of Lemma 2 to the denominator of (37) leads to

$$\begin{array}{c}\hfill {\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}{\left[I-q{F}^{*}\left(e^{i\omega }\right)F\left(e^{i\omega }\right)\right]}^{-1}d\omega =\underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}\left[G^{*}\left(e^{i\omega }\right)G\left(e^{i\omega }\right)\right]d\omega }\end{array}$$

(38)

where $F\left(z\right)=\left[\begin{array}{cc}A& B\\ C& D\end{array}\right]$ and $G\left(z\right)=\left[\begin{array}{cc}A+BL& B{M}^{1/2}\\ L& M^{1/2}\end{array}\right]$. The integral on the right-hand side of Equation (38), up to a factor of $2\pi$, is equal to the ${\mathcal{H}}_2$ norm of the matrix G:

$$\begin{array}{c}\hfill {\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}\left[G^{*}\left(e^{i\omega }\right)G\left(e^{i\omega }\right)\right]d\omega =2\pi {\parallel G\parallel }_2^2=2\pi \mathrm{tr}\left(LP{L}^{\mathrm{T}}+M\right)}\end{array}$$

(39)

where matrix P is a controllability gramian that satisfies the Lyapunov Equation (35). Substituting (39) and (38) into (37), one can obtain (34).

Now consider the equation of the Form (22) and transform its left-hand side to

$$\begin{array}{c}{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}lndet{\displaystyle \frac{2\pi m{\left[I-q\mathsf{\Lambda }\left(\omega \right)\right]}^{-1}}{{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}{\left[I-q\mathsf{\Lambda }\left({\omega }^{\prime }\right)\right]}^{-1}d{\omega }^{\prime }}}}d\omega =}\hfill \\ \hfill \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}{\displaystyle =2\pi mln{\displaystyle \frac{2\pi m}{{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}{\left[I-q\mathsf{\Lambda }\left({\omega }^{\prime }\right)\right]}^{-1}d{\omega }^{\prime }}}}+\underset{-\pi }{\overset{+\pi }{\int }}lndet{\left[I-q\mathsf{\Lambda }\left(\omega \right)\right]}^{-1}d\omega .}\end{array}$$

According to (38) and (39), the integral in the denominator of the first term is equal to ${\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}{\left[I-q\mathsf{\Lambda }\left({\omega }^{\prime }\right)\right]}^{-1}d{\omega }^{\prime }=2\pi \mathrm{tr}\left(LP{L}^{\mathrm{T}}+M\right)}$, while the second term, following [6], equals to $2\pi lndetM$; therefore,

$${\displaystyle \frac{1}{2\pi }\underset{-\pi }{\overset{+\pi }{\int }}lndet{\displaystyle \frac{2\pi m{\left[I-q\mathsf{\Lambda }\left(\omega \right)\right]}^{-1}}{{\displaystyle \underset{-\pi }{\overset{+\pi }{\int }}\mathrm{tr}{\left[I-q\mathsf{\Lambda }\left({\omega }^{\prime }\right)\right]}^{-1}d{\omega }^{\prime }}}}d\omega =lndet\frac{mM}{\mathrm{tr}(LP{L}^{\mathrm{T}}+M)},}$$

which means that the special type Equation (36) equals to (22), completing the proof. □

Note that multiplier $\beta$ is selected as $\beta =2\pi m$ to satisfy $\sigma \left(W\right)=0$ for $q=0$, i.e., to deal with white Gaussian noise. Multiplier $\alpha$ is selected equal to $\frac{1}{2\pi }$ in order to provide matching between the analytical appearance of the mean anisotropy and spectral entropy in the state-space domain.

4. Discussion

The axioms underlying the anisotropic and spectral-entropy approaches to the analysis of linear time-invariant systems have been considered above. It was demonstrated that anisotropic and $\sigma$-entropy norms have similar mathematical formulae and are defined as a supremum ratio between the power norm of the output and power norm of the input on the set of input signals, bounded by the mean anisotropy or spectral entropy level, respectively. The substantial difference lies in the use of axioms to define the anisotropy or spectral entropy level. In anisotropy-based approach information theory, the relative entropy and Kolmogorov–Szego theorem are used to interpret the random disturbance from the time domain to the frequency domain. Consequently, some significant limitations have to be taken into account. Firstly, the relative entropy requires the use of white Gaussian noise as a reference signal. Secondly, the Kolmogorov–Szego formula is valid only for stationary signals [20] Thus, a substantial drawback of the anisotropy-based control theory is that it guarantees a given disturbance attenuation level only for stationary random sequences. In the spectral entropy approach, there is no reference signal. The spectral entropy level can be calculated for all signals that have a Fourier transform. This means that the set of input signals with bounded spectral entropy (36) is wider than the set of signals with bounded mean anisotropy (8), although both formulas are similar. Frequency and time domain formulae for the calculation of anisotropic and spectral entropy norms of linear time-invariant systems are identical. Consequently, we can claim that axiomatics of the spectral entropy approach allows one to rigorously prove that anisotropy-based controllers can be used to guarantee the desired disturbance attenuation level not only for stationary random sequences but also for a wider set of input signals. Some results on the optimal and suboptimal anisotropy-based control design can be found in [10].

It also should be noted that postulating on the spectral entropy in the form of (36) allows one to develop spectral entropy control for continuous-time linear systems. The first results engaged to the analysis of linear continuous-time systems are given in [14,28,29,30].