Verification of Approximate Initial-State Opacity for Control Systems via Neural Augmented Barrier Certificates

Abstract

In this paper, we propose an augmented barrier certificate-based method for formally verifying the approximate initial-state opacity property of discrete time control systems. The opacity verification problem is formulated as the safety verification of an augmented system and is then addressed by searching for augmented barrier certificates. A set of well-defined verification conditions is a prerequisite for successfully identifying augmented barrier certificates of a specific type. We first suggest a new type of augmented barrier certificate which produces a weaker sufficient condition for approximate initial-state opacity. Furthermore, we develop an algorithmic framework where a learner and a verifier interact to synthesize augmented barrier certificates in the form of neural networks. The learner trains neural certificates via the deep learning method, and the verifier solves several mixed integer linear programs to either ensure the validity of the candidate certificates or yield counterexamples, which are passed back to further guide the learner. The experimental results demonstrate that our approach is more scalable and effective than the existing sum of squares programming method.

1. Introduction

1.1. Research Motivation

Cyber-physical systems (CPSs) consist of tightly coupled physical components and software systems and are deeply involved in safety-critical infrastructure, such as power plants, medical devices, and self-driving cars. In the last two decades, formal verification of the safety properties for cyber-physical systems has gained considerable attention [1,2]. However, security properties have not been investigated thoroughly for cyber-physical systems utill very recently [3,4]. Cyber-physical systems are security-critical, since the tight interaction between physical components and software systems may have the potential to release secret information and expose the system to malicious intruders. Therefore, ensuring the security of cyber-physical systems has become significantly important.

For cyber-physical systems, opacity verification is a vital requirement. Opacity is an information flow security property that captures whether or not the “secret” of the system can be revealed to an intruder that can infer the system’s actual behavior based on the information flow [5,6]. Opacity has been widely investigated in the domain of discrete event systems [7,8,9,10]. For continuous state-space CPSs, the opacity verification can be performed by abstraction-based techniques in which continuous-space models are approximated by discrete ones [11,12].

The approximate initial-state opacity property requires the outside intruders to infer whether the initial state of the system is a secret one by observing the output trajectories of the system [8]. A discretization-free barrier certificate-based approach for verifying the approximate initial-state opacity of discrete-time control systems was developed in [11]. The barrier certificate-based method is quite popular for the safety verification of CPSs [13,14,15], and in [11,12], an augmented barrier certificate is defined for an augmented system constructed by augmenting a control system with itself, while the initial and unsafe sets are designed to capture the secret and initial sets of the original system. Then, the existence of such barrier certificates can guarantee the approximate initial-state opacity of the original system. Moreover, a sum of squares (SOS) programming-based methodology is presented for the construction of the augmented barrier certificates. However, this methodology may suffer from scalability issues since it requires discretization of the state and input sets of the original system.

Motivated by the limitation of [11], in this work, we present a novel barrier certificate-based method for formally verifying the approximate initial-state opacity of discrete-time control systems. First, we suggest a new type of augmented barrier certificate which can produce a weaker sufficient condition for the approximate initial-state opacity. Intuitively, the augmented barrier certificates introduced in [11] guarantee that for any state run of the system starting from a secret state, all state runs starting from the non-secret state set have similar output trajectories. Therefore, the sufficient condition for the approximate initial-state opacity yielded from the augmented barrier certificates introduced in [11] is more conservative. Usually, a weaker condition for the augmented barrier certificates means that more augmented barrier certificates can be synthesized, as the expressiveness of the augmented barrier certificates is stronger.

We then suggest an algorithmic framework for synthesizing augmented barrier certificates in the form of neural networks. Our framework draws insight from the existing methods for learning the barrier certificates of hybrid systems [16,17,18] and consists of two modules. The learner trains a neural augmented barrier certificate that satisfies the barrier certificate conditions over a set of sampled data, and the verifier solves a mixed integer linear program (MILP) to either ensure the validity of the candidate augmented barrier certificate or yield counterexamples, which are passed back to further guide the learner. The loop is repeated until a trained candidate augmented barrier certificate is successfully verified. The main contributions of this paper are summarized as follows:

• We define a new type of augmented barrier certificate, which can produce a weaker sufficient condition for the approximate initial-state opacity.
• We present a framework for synthesizing neural augmented barrier certificates. In the framework, a counterexample guided procedure is adopted to speed up the construction of networks, and the verification of Rectified Linear Unit (ReLU) networks can be efficiently performed via an MILP.
• We carry out proof-of-concept case studies to empirically show the efficiency and practicability of the approach.

The structure of this paper is as follows. Section 1.2 introduces the related works. In Section 2, we give a less conservative sufficient condition with two relation functions to verify the approximate initial-state opacity, and then we train neural networks to synthesize the candidate neural barrier certificates and relation functions in Section 3.1 and convert the verification problem to a group of MILP optimization problems for validation in Section 3.2. In Section 3.3, we illustrate the barrier certificate and relation function synthesis algorithm. Section 4 provides an experimental evaluation of our algorithm with three examples, including linear, three-dimensional and non-polynomial examples. We conclude and introduce future work in Section 5.

1.2. Relate Work

Opacity is an important information flow property that was initially introduced in [5]. Specifically, the concept of language-based opacity was proposed in [19]. Aside from that, several state-based opacities were proposed, including initial-state opacity [20], K-step opacity [21], current-state opacity [22], and infinite-step opacity [23]. However, the conditions of exact opacity could be too strong for the systems. Consequently, they need to be relaxed in some application scenarios. In [8], approximate opacity was introduced and verified by approximate opacity preserving the simulation relation. In [24,25], the authors synthesized a supervisor such that the closed-loop system was opaque.

The verification of opacity has also been widely studied in [26,27,28]. Four types of opacity properties have been verified: language-based opacity, initial-state opacity, current-state opacity, and initial-and-final-state opacity [7]. In [20], the verification problem of initial-state opacity to an initial-state estimation problem was translated, and this was addressed by the construction of initial-state estimators. Moreover, current-state and initial-state opacity problems in bounded Petri nets can be efficiently solved by adopting a compact representation of the reachability graph [29]. In [30], initial-state opacity for networks of interconnected control systems was verified. In [31], the verification of opacity for recursive tile systems is presented. Note that in [32], a finite abstraction-based technique was presented for verifying approximate opacity of the class of discrete-time stochastic systems, where the original control systems were approximated by discrete ones. For continuous state–space CPSs, a barrier certificate was used to verify the approximate initial-state opacity in [11,12]. However, this methodology may suffer from scalability issues since it requires discretization of the state and input sets of the original system.

Synthesizing augmented barrier certificates via deep learning methods is subject to further verification, which is related to formal verification of the neural networks, and much research focused on formal verification of the neural networks [33,34,35]. The problem of neural network verification is NP-hard [36], where a tool was implemented called Reluplex for validating neural networks with a ReLU activation function. In [37], a framework to verify deep neural networks based on Satisfiability Modulo Theory (SMT) was introduced. The verification of the properties of deep neural networks was finished based on the simplex method in [38]. The neural network with a ReLU activation function was verified by exploiting the dependencies between the ReLU nodes [39]. Furthermore, with the candidate network with ReLU, the non-linear SMT solver iSAT3 was used for verification [35]. In [17], candidate neural barrier certificates were synthesized with a specially structured network, which transformed the verification problems of neural networks into a group of mixed-integer linear programming problems and a mixed-integer quadratically constrained problem. Zhao et al. [40] provides an innovative method for synthesizing neural networks as barrier certificates, which can give safety guarantees for neural network-controlled systems.

2. An Augmented Barrier Certificate for Approximate Initial-State Opacity

We denote $\mathbb{R}$, ${\mathbb{R}}_{\ge 0}$, $\mathbb{Q}$, and $\mathbb{N}$ as real numbers, non-negative real numbers, rational numbers, and natural numbers, respectively, and appropriate subscripts are used to restrict the sets. The empty set is represented by ∅. Given a vector x, we use ${\mathbb{R}}^n$ to represent an n-dimensional Euclidean space and ‖x‖ to represent the Euclidean norm. For a, b ∈ $\mathbb{N}$, and a ≤ b, we use [a, b] to represent a closed interval. Given two sets A and B with A ⊂ B, we define the complement of A with respect to B as $B\backslash A$ = {a∣a∈B, a∉A}. The Cartesian product of two sets A and B is defined as A × B = {(a, b) ∣a∈A, b∈B}.

2.1. Approximate Initial-State Opacity for Discrete-Time Control Systems

In this subsection, we will introduce some preliminaries:

Definition 1

([11]). A discrete-time control system (dt-cs) is defined by a tuple Π = ($\mathbb{X}$, ${\mathbb{X}}_0$, ${\mathbb{X}}_s$, ${\mathbb{X}}_{ns}$, $\mathbb{U}$, f, $\mathbb{Y}$, h) where the following are true:

• $\mathbb{X}$, $\mathbb{U}$, and$\mathbb{Y}$are the state set, control input set, and output set, respectively;
• ${\mathbb{X}}_0$, ${\mathbb{X}}_s$, and${\mathbb{X}}_{ns}$are the initial state set, secret state set, and non-secret state set, respectively, where${\mathbb{X}}_0$, ${\mathbb{X}}_s$, ${\mathbb{X}}_{ns}$ ⊆ $\mathbb{X}$, and ${\mathbb{X}}_s$⋂${\mathbb{X}}_{ns}$ = ∅;
• $f:\mathbb{X}\times \mathbb{U}\to \mathbb{X}$is the state transition function;
• $h:\mathbb{X}\to \mathbb{Y}$is the output function.

Intuitively, the discrete-time control system $\mathrm{\Pi }$ can be modeled by the following difference equations:

$$\begin{array}{c}\hfill \mathrm{\Pi }:\left\{\begin{array}{c}\mathit{x}(t+1)=f\left(\mathit{x}\right(t),\mathit{u}(t\left)\right),\hfill \\ \mathit{y}\left(t\right)=h\left(\mathit{x}\right(t\left)\right)\hfill \end{array}\right.\end{array}$$

(1)

where $\mathit{x}$: $\mathbb{N}$→$\mathbb{X}$, $\mathit{u}$: $\mathbb{N}$→$\mathbb{U}$, and $\mathit{y}$: $\mathbb{N}$→$\mathbb{Y}$ are the state signal, control input signal, and output signal, respectively. We use ${\mathit{x}}_{{\mathit{x}}_0,\mathit{u}}\left(k\right)$ to denote the state reached at time k of $\mathrm{\Pi }$ from the initial state ${\mathit{x}}_0\in {\mathbb{X}}_0$ under the input signal $\mathit{u}$ and use ${\mathit{y}}_{{\mathit{x}}_0,\mathit{u}}\left(k\right)=h\left({\mathit{x}}_{{\mathit{x}}_0,\mathit{u}}\left(k\right)\right)$ to denote the output corresponding to the state ${\mathit{x}}_{{\mathit{x}}_0,\mathit{u}}\left(k\right)$.

Given a discrete-time control system $\mathrm{\Pi }$, we want to verify the approximate initial-state opacity; that is, for each finite state run whose initial state is a secret state, there exists another finite state run starting from a non-secret state with a similar output trajectory. The following definition describes the approximate initial-state opacity of the system $\mathrm{\Pi }$:

Definition 2

([8]). Consider a discrete-time control system Π = ($\mathbb{X}$, ${\mathbb{X}}_0$, ${\mathbb{X}}_s$, ${\mathbb{X}}_{ns}$, $\mathbb{U}$, f, $\mathbb{Y}$, h) and a constant δ ∈ ${\mathbb{R}}_{\ge 0}$. The system Π is δ-approximate initial-state opacity if for any ${\mathit{x}}_0\in {\mathbb{X}}_0\bigcap {\mathbb{X}}_s$ and any finite state run ${\mathit{x}}_{{\mathit{x}}_0,\mathit{u}}$ = {${\mathit{x}}_0$, ${\mathit{x}}_1$, … ${\mathit{x}}_n$}, there exists ${\widehat{\mathit{x}}}_0$$\in {\mathbb{X}}_0\bigcap {\mathbb{X}}_{ns}$ and a finite state run ${\mathit{x}}_{{\widehat{\mathit{x}}}_0,\widehat{u}}$ = {${\widehat{\mathit{x}}}_0$, ${\widehat{\mathit{x}}}_1$, … ${\widehat{\mathit{x}}}_n$} such that

$$\begin{array}{c}\hfill \underset{k\in [0;n]}{max}∥(h\left({\mathit{x}}_k\right)-h\left({\widehat{\mathit{x}}}_k\right)∥\le \delta ,\end{array}$$

(2)

where the threshold parameter δ can be interpreted as either the measurement imprecision of the intruder or the security level the system can guarantee.

It is assumed that the secret of the system is not revealed initially; otherwise, it will violate $\delta$-approximate initial-state opacity initially. Therefore, we assume that $\forall {\mathit{x}}_0\in {\mathbb{X}}_0\bigcap {\mathbb{X}}_s$, where

$$\begin{array}{c}\hfill \{\mathit{x}\in {\mathbb{X}}_0\mid ∥h\left(\mathit{x}\right)-h\left({\mathit{x}}_0\right)∥\le \delta \}\subseteq {\mathbb{X}}_{ns}.\end{array}$$

(3)

To verify the approximate initial-state opacity of discrete-time control systems, the authors of [11,12] achieved the goal with the help of barrier certificates. In detail, for a discrete-time control system $\mathrm{\Pi }$, the approximate initial-state opacity verification problem can be cast into checking a safety property of the associated augmented system $\mathrm{\Pi }$ × $\mathrm{\Pi }$ using barrier certificates. More concretely, there is the augmented system

$$\begin{array}{cc}\hfill \mathrm{\Pi }\times \mathrm{\Pi }=& (\mathbb{X}\times \mathbb{X},{\mathbb{X}}_0\times {\mathbb{X}}_0,{\mathbb{X}}_s\times {\mathbb{X}}_s,\hfill \\ \hfill & {\mathbb{X}}_{ns}\times {\mathbb{X}}_{ns},\mathbb{U}\times \mathbb{U},f\times f,\mathbb{Y}\times \mathbb{Y},h\times h),\hfill \end{array}$$

which can be regarded as the product of a dt-cs $\mathrm{\Pi }$ and itself. We denote a pair of states in $\mathrm{\Pi }$ × $\mathrm{\Pi }$ by ($\mathit{x}$, $\widehat{\mathit{x}}$) ∈$\mathbb{X}$ × $\mathbb{X}$ and use $\mathrm{\Theta }$ = $\mathbb{X}$ × $\mathbb{X}$ to represent the state space of the augmented system. Given the initial state (${\mathit{x}}_0$, ${\widehat{\mathit{x}}}_0$) of the augmented system $\mathrm{\Pi }$ × $\mathrm{\Pi }$ and input run ($\mathit{v}$, $\widehat{\mathit{v}}$), we denote the state trajectory of $\mathrm{\Pi }$ × $\mathrm{\Pi }$ with (${\mathit{x}}_{{\mathit{x}}_0,\mathit{v}}$, ${\widehat{\mathit{x}}}_{{\mathit{x}}_0,\widehat{\mathit{v}}}$).

In order to leverage barrier certificates to verify the approximate initial-state opacity for a given discrete-time control system $\mathrm{\Pi }$, the initial state set ${\mathrm{\Theta }}_0$ and the unsafe state set ${\mathrm{\Theta }}_u$ are defined as follows:

$$\begin{array}{cc}\hfill & {\mathrm{\Theta }}_0=\{(\mathit{x},\widehat{\mathit{x}})\in ({\mathbb{X}}_0\cap {\mathbb{X}}_s)\times ({\mathbb{X}}_0\backslash {\mathbb{X}}_s)\mid ∥h\left(\mathit{x}\right)-h\left(\widehat{\mathit{x}}\right)∥\le \delta \},\hfill \\ \hfill & {\mathrm{\Theta }}_u=\{(\mathit{x},\widehat{\mathit{x}})\in \mathbb{X}\times \mathbb{X}\mid ∥h\left(\mathit{x}\right)-h\left(\widehat{\mathit{x}}\right)∥>\delta \}.\hfill \end{array}$$

(4)

The following theorem introduces a sufficient condition for verifying the approximate initial-state opacity of discrete-time control systems via a concept of augmented barrier certificates:

Theorem 1.

Given a dt-cs Π, the associated augmented system Π × Π and the sets ${\mathrm{\Theta }}_0$, ${\mathrm{\Theta }}_u$ in Equation (4), if there exists a function $B:\mathbb{X}\times \mathbb{X}\to \mathbb{R}$ such that

$$\begin{array}{cc}\hfill & \forall (\mathit{x},\widehat{\mathit{x}})\in {\mathrm{\Theta }}_{0}B(\mathit{x},\widehat{\mathit{x}})\le {ϵ}_1,\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill & \forall (\mathit{x},\widehat{\mathit{x}})\in {\mathrm{\Theta }}_{u}B(\mathit{x},\widehat{\mathit{x}})\ge {ϵ}_2,\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill & \forall (\mathit{x},\widehat{\mathit{x}})\in \mathrm{\Theta },\forall \mathit{u}\in \mathbb{U},\exists \widehat{\mathit{u}}\in \mathbb{U}\hfill \\ \hfill & B(f(\mathit{x},\mathit{u}),f(\widehat{\mathit{x}},\widehat{\mathit{u}}))-B(\mathit{x},\widehat{\mathit{x}})\le 0,\hfill \end{array}$$

(7)

where the constants ${ϵ}_1$, ${ϵ}_2$ ∈ ${\mathbb{R}}_{\ge 0}$, and ${ϵ}_2>{ϵ}_1$ are present, then the δ-approximate initial-state opacity of the dt-cs Π is guaranteed.

The function $B(\mathit{x},\widehat{\mathit{x}})$ in Theorem 1 is called an augmented barrier certificate of the augmented system $\mathrm{\Pi }$ × $\mathrm{\Pi }$. Intuitively, the augmented barrier certificate $B(\mathit{x},\widehat{\mathit{x}})$ ensures that all trajectories of the system $\mathrm{\Pi }$ × $\mathrm{\Pi }$ originating from the initial state sets ${\mathrm{\Theta }}_0$ never reach the unsafe region ${\mathrm{\Theta }}_u$, and then the safety of the dt-cs $\mathrm{\Pi }$ is guaranteed.

In this paper, we focus on the nonlinear discrete-time control systems, where the state transition function and output function are represented by multivariate elementary functions, and the sets $\mathbb{X}$, $\mathbb{U}$, $\mathbb{Y}$, ${\mathbb{X}}_0$, ${\mathbb{X}}_s$, and ${\mathbb{X}}_{ns}$ are bounded polyhedrons each defined by linear inequalities over a variable $\mathit{x}$. Concretely, multivariate elementary functions are expressed by the following grammar:

$$\begin{array}{cc}\hfill f,g::=& cx\left|f\right|f^a|ln\left(f\right)|sin\left(f\right)|cos\left(f\right)\hfill \\ \hfill & |e^f|f+g|f-g|f\times g|f/g,\hfill \end{array}$$

where $c\in \mathbb{R}$ is a real constant, $a\in \mathbb{Q}$ is a rational constant, and $\mathit{x}$ can be any system variable.

2.2. A New Augmented Barrier Certificate for Approximate Initial-State Opacity

The augmented barrier certificates introduced in Theorem 1 guarantee that all trajectories of the augmented system $\mathrm{\Pi }\times \mathrm{\Pi }$ starting from the initial state set ${\mathrm{\Theta }}_0$ never reach the unsafe states in ${\mathrm{\Theta }}_u$. This means that for any state run of the discrete-time control system $\mathrm{\Pi }$ starting from a secret state ${\mathit{x}}_0\in {\mathbb{X}}_0\cap {\mathbb{X}}_s$, all state runs starting from the non-secret state set ${\mathbb{X}}_0\backslash {\mathbb{X}}_s$ have similar output trajectories. However, according to Definition 2, the $\delta$-approximate initial-state opacity just requires that for each finite state run whose initial state is a secret state, there exists another finite state run starting from a non-secret state with a similar output trajectory. Clearly, the sufficient condition of the augmented barrier certificates introduced in Theorem 1 is more conservative, which may make the problem of $\delta$-approximate initial-state opacity verification infeasible.

In the following, we introduce a less conservative augmented barrier certificate for the verification of approximate initial-state opacity:

Theorem 2.

Consider a dt-cs Π and the associated augmented system Π × Π. If there exists a function $B:\mathbb{X}\times \mathbb{X}\to \mathbb{R}$ that satisfies the conditions in Equations (6) and (7) in Theorem 1 and moreover

$$\begin{array}{c}\hfill \forall \mathit{x}\in {\mathbb{X}}_0\cap {\mathbb{X}}_s,\exists \widehat{\mathit{x}}\in {\mathbb{X}}_0\backslash {\mathbb{X}}_s\\ \hfill ∥h\left(\mathit{x}\right)-h\left(\widehat{\mathit{x}}\right)∥\le \delta \wedge B(\mathit{x},\widehat{\mathit{x}})\le {ϵ}_1.\end{array}$$

(8)

then the δ-approximate initial-state opacity of the dt-cs Π is guaranteed.

Proof. 

Consider an arbitrary initial secret state ${\mathit{x}}_0$ and an input sequence $\mathit{u}$, as well as the corresponding state sequence ${\mathit{x}}_{{\mathit{x}}_0,\mathit{u}}$ in $\mathrm{\Pi }$. First note Equations (3) and (8), where $\{\mathit{x}\in {\mathbb{X}}_0\mid ∥h\left(\mathit{x}\right)-h\left({\mathit{x}}_0\right)∥\le \delta \}\subseteq {\mathbb{X}}_{ns}$. It follows that there exists an initial state ${\widehat{\mathit{x}}}_0\in {\mathbb{X}}_0\backslash {\mathbb{X}}_s$ for every initial state ${\mathit{x}}_0\in {\mathbb{X}}_0\cap {\mathbb{X}}_s$ such that $\parallel h\left({\widehat{\mathit{x}}}_0\right)-$$h\left({\mathit{x}}_0\right)\parallel \le \delta$, based on Equation (8). Therefore, the system is initially satisfied for $\delta$-approximate initial-state opacity. The existence of an augmented barrier certificate B($\mathit{x}$,$\widehat{\mathit{x}}$) as described in Theorem 1 guarantees that for any ${\mathit{x}}_0\in {\mathbb{X}}_0\cap {\mathbb{X}}_s$, there exists ${\widehat{\mathit{x}}}_0\in {\mathbb{X}}_0\backslash {\mathbb{X}}_s$ and a control policy $\widehat{\mathit{u}}$ such that any state sequence of $\mathrm{\Pi }$ × $\mathrm{\Pi }$ starting from (${\mathit{x}}_0$,${\widehat{\mathit{x}}}_0$) never reaches the unsafe region ${\mathrm{\Theta }}_u$ under input run $(\mathit{u},\widehat{\mathit{u}})$. In other words, the following is true:

$$\begin{array}{c}\hfill \underset{t\in [0;n]}{max}∥(h\left({\mathit{x}}_{{\mathit{x}}_0,\mathit{u}}\left(t\right)\right)-h\left({\widehat{\mathit{x}}}_{{\widehat{\mathit{x}}}_0,\widehat{\mathit{u}}}\left(t\right)\right)∥\le \delta \end{array}$$

(9)

Since ${\mathit{x}}_0$ and ${\mathit{x}}_{{\mathit{x}}_0,\mathit{u}}$ are arbitrarily chosen, we conclude that $\mathrm{\Pi }$ is the $\delta$-approximate initial-state opacity.    □

Since the condition in Equation (8) is weaker than the initial condition in Theorem 1, Theorem 2 produces a less conservative sufficient condition for the $\delta$-approximate initial-state opacity of the dt-cs $\mathrm{\Pi }$. Usually, a weaker condition on augmented barrier certificates means that more augmented barrier certificates can be synthesized, as the associated expressiveness is stronger. Therefore, the problem of verifying the $\delta$-approximate initial-state opacity for the dt-cs $\mathrm{\Pi }$ can be transformed into the problem of computing the relaxed augmented barrier certificates defined in Theorem 2.

However, the forall-exists quantifiers make the problem of computing the augmented barrier certificates given in Theorem 2 computationally difficult. From a computational point of view, the forall-exists statement in Equation (8) can be encoded as

$$\begin{array}{cc}\hfill & \forall (\mathit{x},\widehat{\mathit{x}})\in {\tilde{\mathrm{\Theta }}}_{0}B(\mathit{x},\widehat{\mathit{x}})\le {ϵ}_1\hfill \end{array}$$

(10)

where ${\tilde{\mathrm{\Theta }}}_0$ is defined as follows:

$$\begin{array}{cc}\hfill & {\tilde{\mathrm{\Theta }}}_0=\{(\mathit{x},\widehat{\mathit{x}})\in ({\mathbb{X}}_0\cap {\mathbb{X}}_s)\times ({\mathbb{X}}_0\backslash {\mathbb{X}}_s)\mid \hfill \\ \hfill & \widehat{\mathit{x}}=p\left(\mathit{x}\right)\wedge ∥h\left(\mathit{x}\right)-h\left(\widehat{\mathit{x}}\right)∥\le \delta \},\hfill \end{array}$$

(11)

by introducing a function p over $\mathit{x}$ to describe the relation between $\mathit{x}$ and $\widehat{\mathit{x}}$. Similarly, we introduce another relation function q over $\mathit{x}$, $\widehat{\mathit{x}}$, and $\mathit{u}$ to encode the forall-exists statement:

Theorem 3.

Consider a dt-cs Π and the associated augmented system Π × Π. If there exist functions $B:\mathbb{X}\times \mathbb{X}\to \mathbb{R}$, $p:\mathbb{X}\to \mathbb{R}$ and $q:\mathbb{X}\times \mathbb{X}\times \mathbb{U}\to \mathbb{R}$ which satisfy the conditions in Equations (6) and (10) and moreover

$$\begin{array}{c}\forall (\mathit{x},\widehat{\mathit{x}})\in \mathrm{\Theta },\forall (\mathit{u},\widehat{\mathit{u}})\in \mathbb{U}\times \mathbb{U},\hfill \\ \widehat{\mathit{u}}=q(\mathit{x},\widehat{\mathit{x}},\mathit{u})\in \mathbb{U}\wedge B(f(\mathit{x},\mathit{u}),f(\widehat{\mathit{x}},\widehat{\mathit{u}}))-B(\mathit{x},\widehat{\mathit{x}})\le 0.\hfill \end{array}$$

(12)

then the δ-approximate initial-state opacity of the dt-cs Π is guaranteed.

Now, the problem of verifying the $\delta$-approximate initial-state opacity for the dt-cs $\mathrm{\Pi }$ can be transformed into the problem of computing the relaxed augmented barrier certificate $B(\mathit{x},\widehat{\mathit{x}})$ and the relation functions $p\left(\mathit{x}\right)$, $q(\mathit{x},\widehat{\mathit{x}},\mathit{u})$ defined in Theorem 3.

3. Synthesis of an Augmented Barrier Certificate via Learning and Verification

In this section, we introduce an iterative framework to synthesize neural barrier certificates and relation functions to complete the verification of the approximate initial-state opacity of the dt-cs $\mathrm{\Pi }$. As shown in Figure 1, the procedure is structured as an inductive loop between a learner and a verifier. At first, we present a less conservative augmented barrier certificate for the validation of approximate initial-state opacity. Then, the learner trains the certificate functions simultaneously over sampled data points from the initial set, the unsafe set, and the state space. After that, the trained candidates are formally verified by the verifier according to the conditions in Theorem 3. Due to the structure of ReLU neural networks, the verification problem can be transformed into a set of efficiently solvable MILP problems.

3.1. Neural Certificate Training

In this subsection, we give the details for training the neural barrier certificates.

3.1.1. The Structure of Neural Networks

Consider a dt-cs $\mathrm{\Pi }$ as defined in Definition 1, where state $\mathit{x}$ and control input $\mathit{u}$ are of n and m dimensions, respectively. Now, we provide the specific structure of neural networks for certificate functions $B(\mathit{x},\widehat{\mathit{x}})$, $p\left(\mathit{x}\right)$ and $q(\mathit{x},\widehat{\mathit{x}},\mathit{u})$.

• One input layer, one output layer, and several hidden layers;
• The number of input neurons for $B,p$, and q is $2n$, n, and $2n+m$, respectively;
• Both B and q have one output neuron, while p has n output neurons;
• The ReLU function is the only legal activation function.

For the above feedforward neural network, a non-input layer neuron value is computed by the preceding layer neuron values, the layer weight matrix, and the bias vector. Let ${\mathit{x}}_0$ denote the given neuron value of the input layer, ${\mathit{z}}_k$ and ${\mathit{x}}_k$ ($1\le k\le \eta -1$) denote the neuron value of the hidden layer $L_k$ before and after ReLU activation function, respectively, and ${\mathit{x}}_{\eta }$ denote the neuron value of the output layer. Then, the forward propagation of the neural network is as follows:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{\mathit{z}}_k=W_k{\mathit{x}}_{k-1}+{\mathit{b}}_k,k=1,\dots ,\eta -1\hfill \\ {\mathit{x}}_k=ReLU\left({\mathit{z}}_k\right),k=1,\dots ,\eta -1\hfill \\ {\mathit{x}}_{\eta }=W_{\eta }{\mathit{x}}_{\eta -1}+{\mathit{b}}_{\eta }\hfill \end{array}\right.\end{array}$$

(13)

where $\eta -1$ is the number of hidden layers of the neural network. Intuitively, the output layer of neural network B and q has only one neuron, and hence their outputs are real numbers. As for the output of p, it is a vector and in the same form of the corresponding input vector.

According to Universal Approximation Theorem, we can approximate any complex rational function by the above neural network with the ReLU activation function, which makes it possible for us to synthesize the barrier certificate and relation functions. By restricting the number of hidden layers and the type of activation function, the verification problems of Equations (6), (10), and (12) can be transformed into a set of MILP problems. Then, the verification can be accomplished with the help of the optimization tool Gurobi.

3.1.2. Training Dataset Generation

The procedure for synthesizing neural networks takes a data-driven approach. Here, we discuss how to generate adequate datasets for training the candidate network. Datasets are generated while aiming at making the network fulfill the barrier certificate conditions given by Theorem 3. We sample points from the domain $\mathrm{\Theta }$, initial secret region ${\mathbb{X}}_0\cap {\mathbb{X}}_s$, and unsafe region ${\mathrm{\Theta }}_u$ of the considered system $\mathrm{\Pi }\times \mathrm{\Pi }$ for obtaining datasets $D_1$, $D_2$, and $D_3$, respectively.

The method employed for sampling is mesh generation. At first, we mesh the super rectangles in the sampling regions bounded with a fixed mesh size. Then, we filter out the points that do not satisfy the constraints of Equations (6), (10) and (12). As for the dataset $D_4$ of neural networks q, it is essentially a vector splicing of $D_3$ with the control input $\mathit{u}$. Finally, we obtain four finite data sets, $D_1$, $D_2$, $D_3$, and $D_4$.

3.1.3. Loss Function Encoding

Given datasets $D_1$, $D_2$, $D_3$, and $D_4$, according to Equations (6), (10) and (12) given by Theorem 3, the neural network B is trained to satisfy the following conditions:

$$\begin{array}{c}\hfill B(\mathit{x},p\left(\mathit{x}\right))\le 0\forall \mathit{x}\in D_1\end{array}$$

(14)

$$\begin{array}{c}\hfill B(\mathit{x},\widehat{\mathit{x}})>0\forall (\mathit{x},\widehat{\mathit{x}})\in D_2\end{array}$$

(15)

$$\begin{array}{c}\hfill B(f(\mathit{x},\mathit{u}),f(\widehat{\mathit{x}},q(\mathit{x},\widehat{\mathit{x}},\mathit{u})))-B(\mathit{x},\widehat{\mathit{x}})\le 0\forall (\mathit{x},\widehat{\mathit{x}})\in D_3,\forall \mathit{u}\in \mathbb{U}\end{array}$$

(16)

According to Equations (14)–(16) and the output bound of the control policy given by the control input $\mathit{u}$ and p, we construct sub-loss functions that lead the optimizers to build the network by minimizing the loss.

For satisfying Equation (14), with the input from dataset $D_1$, the output of B should satisfy $B(\mathit{x},p(\mathit{x}\left)\right)$ ≤ 0. Otherwise, there would be a non-zero loss. Let $c_1$ be a small non-negative tolerance. The first sub-loss function is defined as follows:

$$\begin{array}{c}\hfill L_1=\sum _{\mathit{x}\in D_1}ReLU(B(\mathit{x},p\left(\mathit{x}\right))+c_1)\end{array}$$

(17)

Similarly, for Equation (15), the output of the neural network is supposed to conform with $B(\mathit{x},\widehat{\mathit{x}})>0$. A non-zero loss is also incurred if the condition is violated. Given a small positive constant $c_2$, the second sub-loss function is defined as follows:

$$\begin{array}{c}\hfill L_2=\sum _{(\mathit{x},\widehat{\mathit{x}})\in D_2}ReLU(-B(\mathit{x},\widehat{\mathit{x}})+c_2)\end{array}$$

(18)

To meet Equation (16), with a small positive number $c_3$, the third sub-loss function is defined as follows:

$$\begin{array}{c}\hfill L_3=\sum _{(\mathit{x},\widehat{\mathit{x}})\in D_3,\mathit{u}\in \mathbb{U}}ReLU(B(f(\mathit{x},\mathit{u}),f(\widehat{\mathit{x}},q(\mathit{x},\widehat{\mathit{x}},\mathit{u})))-B(\mathit{x},\widehat{\mathit{x}})+c_3)\end{array}$$

(19)

Aside from this, the output of the control policy q has upper and lower bounds, since the value of the control input is within the given closed interval. Let $l_q$, $u_q$ be the upper and lower bounds of the control input, respectively. The output of q is between $l_q$ and $u_q$ (i.e., $q\left(\mathit{x}\right)$∈ [$l_q$, $u_q$]). After that, the fourth sub-loss function which leads the training of q is defined as follows:

$$\begin{array}{cc}\hfill L_4& =\sum _{(\mathit{x},\widehat{\mathit{x}},\mathit{u})\in D_4}(ReLU(q(\mathit{x},\widehat{\mathit{x}},\mathit{u})-u_q)\hfill \\ \hfill & +ReLU(-q(\mathit{x},\widehat{\mathit{x}},\mathit{u})+l_q))\hfill \end{array}$$

(20)

According to Equation (11), we know that $p\left(\mathit{x}\right)\in {\mathbb{X}}_0\backslash {\mathbb{X}}_s$ when input $\mathit{x}$∈${\mathbb{X}}_0$∩${\mathbb{X}}_s$. Furthermore, each unit of the output of p will be in [$l_p$, $u_p$]. Then we obtain the sub-loss function $L_5$ as:

$$\begin{array}{cc}\hfill L_5& =\sum _{\mathit{x}\in D_1}(ReLU(\parallel h(p(\mathit{x}))-h(\mathit{x})\parallel -\delta +c_4)\hfill \\ & +ReLU(p(\mathit{x})-u_p)+ReLU(-p(\mathit{x})+l_p))\hfill \end{array}$$

(21)

where $c_4$ is a small positive constant. The role of the four tolerances $c_1,c_2,c_3$, and $c_4$ is to make the non-sampled points in the neighborhood of the sampled data obtain a loss of zero. Finally, let ${\alpha }_1$, ${\alpha }_2$, ${\alpha }_3$, ${\alpha }_4$, ${\alpha }_5>0$ denote the weights of the sub-losses. The total loss function for training B, p, q can be expressed as

$$\begin{array}{c}\hfill L={\alpha }_1{L}_1+{\alpha }_2{L}_2+{\alpha }_3{L}_3+{\alpha }_4{L}_4+{\alpha }_5{L}_5\end{array}$$

(22)

Significantly, the sub-loss $L_3$ is related to the training of networks B and q. Hence, we can make ${\alpha }_3$ slightly larger to promote effective training. We minimize the total loss L for training three neural networks and adopt an Adam optimizer, which achieves quick training. When L decreases to zero, we obtain candidate networks B, p, and q at the same time.

3.2. Certificate Function Verification

Since the candidate neural certificate functions B, p, and q are trained over the sampled datasets, the empirical loss L converges to zero, which only guarantees that the candidate certificate functions satisfy the barrier certificate conditions in Theorem 3 over such sampled datasets. Thus, the candidate neural certificate functions require further verification. In this section, we present an approach to formally check whether the candidate neural certificate functions B, p, and q precisely satisfy the following barrier certificate conditions in Theorem 3:

$$\begin{array}{c}\hfill \forall (x,\widehat{x})\in {\tilde{\mathrm{\Theta }}}_{0}B(x,\widehat{x})\le 0\end{array}$$

(23)

$$\begin{array}{c}\hfill \forall (x,\widehat{x})\in {\mathrm{\Theta }}_{u}B(x,\widehat{x})>0,\end{array}$$

(24)

$$\begin{array}{c}\hfill \forall (x,\widehat{x})\in \mathrm{\Theta },\forall (u,\widehat{u})\in \mathbb{U}\times \mathbb{U},\\ \hfill \widehat{u}=q(x,\widehat{x},u)\in \mathbb{U}\wedge B(f(x,u),f(\widehat{x},\widehat{u}))-B(x,\widehat{x})\le 0.\end{array}$$

(25)

By utilizing the special structure of ReLU networks, the problem of verifying the candidate neural certificate functions B, p, and q can be encoded as several optimization problems, whose globally optimal objective values can be computed efficiently via the optimizer Gurobi. In the following, we will verify the neural certificate functions via MILP-based encoding.

Verifying Condition (Equation (23)). According to Equation (23), the neural augmented barrier certificate $B(\mathit{x},\widehat{\mathit{x}})$ should be nonpositive over the initial set ${\tilde{\mathrm{\Theta }}}_0$. To ease the presentation, we use ${\mathit{y}}_0$ and $y_{\eta }$ to denote the input and output of the neural certificate $B(\mathit{x},\widehat{\mathit{x}})$, respectively, where ${\mathit{y}}_0=({\mathit{x}}_0,{\widehat{\mathit{x}}}_0)$ and $y_{\eta }=B\left({\mathit{y}}_0\right)$. Based on the forward propagation of the neural networks in Equation (13), the problem of verifying the condition in Equation (23) can be transformed into the following optimization problem:

$$\begin{array}{cc}\hfill & {\rho }_{Init}^{\ast }=max{y}_{\eta }\hfill \\ \hfill & s.t.\left\{\begin{array}{c}{\mathit{x}}_0\in {\mathbb{X}}_0\cap {\mathbb{X}}_s,\hfill \\ p\left({\mathit{x}}_0\right)\in {\mathbb{X}}_0\backslash {\mathbb{X}}_s,\hfill \\ {\mathit{z}}_k=W_k{\mathit{y}}_{k-1}+{\mathit{b}}_k,k=1,\dots ,\eta -1,\hfill \\ {\mathit{y}}_k=ReLU\left({\mathit{z}}_k\right),k=1,\dots ,\eta -1,\hfill \\ y_{\eta }=W_{\eta }{\mathit{y}}_{\eta -1}+{\mathit{b}}_{\eta },\hfill \end{array}\right.\hfill \end{array}$$

(26)

where $\eta$ is the number of hidden layers of the neural certificate B and ${\rho }_{Init}^{\ast }$ is the globally optimal solution. Clearly, if the globally optimal solution ${\rho }_{Init}^{\ast }>0$, then the condition in Equation (23) is verified to be satisfied.

The application of ReLU activation functions improves the approximation capability of the neural networks, but additionally, the nonlinearity of the ReLU function makes the problem in Equation (26) difficult to solve directly. Fortunately, an exact MILP-based encoding of ReLU functions is introduced to treat ReLU [33]. More concretely, the ReLU function, which is defined over $\mathit{z}\in [l_z,u_z]$, can be encoded by the following linear and binary constraints:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}-\mathit{z}+\mathit{y}+l-l·t\le 0,\hfill \\ \mathit{z}-\mathit{y}\le 0,\hfill \\ \mathit{y}-u·t\le 0,\hfill \\ 0\le \mathit{y},\hfill \\ t\in \{0,1\},\hfill \end{array}\right.\end{array}$$

(27)

where t is a binary variable and l and u are constants satisfying $l\le l_z$ and $u\ge u_z$, respectively. Now, by utilizing the above MILP-based encoding, the optimization problem in Equation (26) can be rewritten as the following MILP problem:

$$\begin{array}{cc}\hfill & {\rho }_{Init}^{\ast }=max{y}_{\eta }\hfill \\ \hfill & s.t.\left\{\begin{array}{c}{\mathit{x}}_0\in {\mathbb{X}}_0\cap {\mathbb{X}}_s,\hfill \\ p\left({\mathit{x}}_0\right)\in {\mathbb{X}}_0\backslash {\mathbb{X}}_s,\hfill \\ {\mathit{y}}_0=({\mathit{x}}_0,p\left({\mathit{x}}_0\right)),\hfill \\ {\mathit{z}}_k=W_k{\mathit{y}}_{k-1}+{\mathit{b}}_k,\hfill \\ -{\mathit{z}}_k+{\mathit{y}}_k\le l_k(t_k-1),\hfill \\ {\mathit{z}}_k-{\mathit{y}}_k\le 0,\hfill \\ {\mathit{y}}_k-u_k·t_k\le 0,\hfill \\ t_k\in \{0,1\},\hfill \\ {\mathit{y}}_k\ge 0,\hfill \\ k=1,\dots ,\eta -1,\hfill \\ y_{\eta }=W_{\eta }{\mathit{y}}_{\eta -1}+{\mathit{b}}_{\eta }.\hfill \end{array}\right.\hfill \end{array}$$

(28)

For simplicity, we set $l_k$ = $-{10}^9$ and $u_k$ = ${10}^9$. This MILP problem (Equation (28)) can be solved via the optimizer Gurobi. Indeed, Gurobi returns the optimum ${\tilde{\rho }}_{Init}$ and the corresponding optimal solution ${\tilde{x}}_{Init}$, together with the maximum relative error ${\xi }_{Init}$ between ${\tilde{\rho }}_{Init}$ and the globally optimum ${\rho }_{Init}^{\ast }$, which is formally expressed as

$${\xi }_{Init}\ge \frac{|{\tilde{\rho }}_{Init}-{\rho }_{Init}^{\ast }|}{|{\tilde{\rho }}_{Init}|}.$$

As stated in [17], if ${\xi }_{Init}<1$, then ${\tilde{\rho }}_{Init}$ and ${\rho }_{Init}^{\ast }$ have the same sign. Therefore, if ${\tilde{\rho }}_{Init}\le 0$ while the maximum relative error ${\xi }_{Init}<1$, then ${\rho }_{Init}^{\ast }\le 0$, which means that the condition in Equation (23) is verified to be satisfied. Otherwise, we randomly sample a set of points in a neighborhood of the optimal solution ${\tilde{\mathit{x}}}_{Init}$ and add those data points that violate the condition in Equation (23) to the training dataset $D_1$ in the next iteration to refine the networks.

Verifying Condition (Equation (24)). According to Equation (24), the neural augmented barrier certificate $B(\mathit{x},\widehat{\mathit{x}})$ should be positive over the unsafe set ${\mathrm{\Theta }}_u$. The verification of the condition in Equation (24) takes the same form. Based on the forward propagation of neural networks in Equation (13) and the MILP-based encoding of ReLU functions, the problem of this verifying condition (Equation (24)) can be transformed into the following MILP problem:

$$\begin{array}{cc}\hfill & {\rho }_{Unsafe}^{\ast }=min{x}_{\eta }\hfill \\ \hfill & s.t.\left\{\begin{array}{c}{\mathit{x}}_0\in {\mathrm{\Theta }}_u,\hfill \\ z_k=W_k{\mathit{x}}_{k-1}+{\mathit{b}}_k,\hfill \\ -z_k+{\mathit{x}}_k\le l_k(t_k-1),\hfill \\ z_k-{\mathit{x}}_k\le 0,\hfill \\ {\mathit{x}}_k-u_k·t_k\le 0,\hfill \\ t_k\in \{0,1\},\hfill \\ {\mathit{x}}_k\ge 0,\hfill \\ k=1,\dots ,\eta -1,\hfill \\ x_{\eta }=W_{\eta }{\mathit{x}}_{\eta -1}+{\mathit{b}}_{\eta }.\hfill \end{array}\right.\hfill \end{array}$$

(29)

By solving the above MILP problem via the optimizer Gurobi, we can obtain the optimum ${\tilde{\rho }}_{Unsafe}$ and the corresponding optimal solution ${\tilde{\mathit{x}}}_{Unsafe}$ together with the maximum relative error ${\xi }_{Unsafe}$. If ${\tilde{\rho }}_{Unsafe}\ge 0$ while the maximum relative error ${\xi }_{Unsafe}<1$, then the globally optimum ${\rho }_{Unsafe}^{\ast }\ge 0$, and the condition in Equation (24) is verified to be satisfied. Otherwise, we randomly sample a set of points in a neighborhood of the optimal solution ${\tilde{x}}_{Unsafe}$ and add those data points that violate the condition in Equation (24) to the training dataset $D_2$ in the next iteration to refine the networks.

Verifying condition (Equation (25)). For the condition in Equation (25), the difference between $B(f(\mathit{x},\mathit{u}),f(\widehat{\mathit{x}},\widehat{\mathit{u}}))$, and $B(\mathit{x},\widehat{\mathit{x}})$ should be non-positive over the state set $\mathrm{\Theta }$ and input set $\mathbb{U}\times \mathbb{U}$. Here, $\widehat{\mathit{u}}=q(\mathit{x},\widehat{\mathit{x}},\mathit{u})$. Note that the condition in Equation (25) involves the valve of ${\mathit{y}}_0$ after the transition function; in other words, ${\mathit{y}}_0^{\prime }=({\mathit{x}}_0^{\prime },{\widehat{\mathit{x}}}_0^{\prime })=(f({\mathit{x}}_0,\mathit{u}),f({\widehat{\mathit{x}}}_0,\widehat{\mathit{u}}))$. Therefore, the problem of $B\left(f\right(x\left)\right)-B\left(x\right)<0$ can be transformed into finding whether the maximum value of $B\left({\mathit{y}}_0^{\prime }\right)-B\left(y_0\right)$ is negative.

Based on the forward propagation of neural networks in Equation (14) and the MILP-based encoding of ReLU functions, the problem of this verifying condition (Equation (25)) can be transformed into the following optimization problem:

$$\begin{array}{cc}\hfill & {\rho }_{Evol}^{\ast }=max{y}_{\eta }^{\prime }-y_{\eta }\hfill \\ \hfill & s.t.\left\{\begin{array}{c}{\mathit{y}}_0\in \mathrm{\Theta },\hfill \\ \mathit{u},\widehat{\mathit{u}}\in \mathbb{U},\hfill \\ {\mathit{y}}_0^{\prime }=({\mathit{x}}_0^{\prime },{\widehat{\mathit{x}}}_0^{\prime }),\hfill \\ z_k=W_k{\mathit{y}}_{k-1}+b_k,z_k^{\prime }=W_k{\mathit{y}}_{k-1}^{\prime }+b_k,\hfill \\ -z_k+{\mathit{y}}_k\le l_k(t_k-1),-z_k^{\prime }+{\mathit{y}}_k^{\prime }\le l_k(t_k^{\prime }-1),\hfill \\ z_k-{\mathit{y}}_k\le 0,z_k^{\prime }-{\mathit{y}}_k^{\prime }\le 0,\hfill \\ {\mathit{y}}_k-u_k·t_k\le 0,{\mathit{y}}_k^{\prime }-u_k·t_k^{\prime }\le 0,\hfill \\ t_k\in \{0,1\},t_k^{\prime }\in \{0,1\},\hfill \\ y_{\eta }=W_{\eta }{\mathit{y}}_{\eta -1}+b_{\eta },y_{\eta }^{\prime }=W_{\eta }{\mathit{y}}_{\eta -1}^{\prime }+b_{\eta },\hfill \\ {\mathit{y}}_k\ge 0,{\mathit{y}}_k^{\prime }\ge 0,k=1,\dots ,\eta -1.\hfill \end{array}\right.\hfill \end{array}$$

(30)

However, there may exist nonlinear terms in transition function f involved in the condition in Equation (25). To handle the nonlinearity, a piecewise linear approximation method is utilized. For a non-linear function $g\left(x\right)$ with $x\in [l_x,u_x]$, it can be approximated by two piecewise linear functions $g_l\left(x\right)$, $g_u\left(x\right)$ with the controlled approximation error $\zeta$, defined as

$$\begin{array}{c}\hfill g_l\left(x\right)\le g\left(x\right)\le g_u\left(x\right),|g_u\left(x\right)-g_l\left(x\right)|\le \zeta ,\forall x\in [l_x,u_x].\end{array}$$

Consider the condition in Equation (25) again. By utilizing the piecewise linear approximation method, the optimization problem in Equation (30) can be transformed into an MILP problem and be solved via the optimizer Gurobi. Thus, we can obtain the optimum ${\tilde{\rho }}_{Evol}$ and the corresponding optimal solution ${\tilde{\mathit{x}}}_{Evol}$ together with the maximum relative error ${\xi }_{Evol}$. If ${\tilde{\rho }}_{Evol}\ge 0$ while the maximum relative error ${\xi }_{Evol}<1$, then the globally optimum ${\rho }_{Evol}^{\ast }\ge 0$, and the condition in Equation (25) is verified to be satisfied. Otherwise, we randomly sample a set of points in a neighborhood of the optimal solution ${\tilde{x}}_{Evol}$ and add those data points that violate the condition in Equation (25) to the training dataset $D_3$ in the next iteration to refine the networks.

3.3. Algorithm

In the previous section, we introduced the complete synthesis process of candidates B, p, and q and the MILP-based verification method. Next, the synthesis process of the barrier certificate and relation functions is given in Algorithm 1.

Algorithm 1 Barrier certificate and relation function synthesis.

Require: Dt-cs $\mathrm{\Pi }$, control input $\mathit{u}$, maximum epoch number $n_{epoch}$, data batches $n_{batch}$, loss function parameters ${\alpha }_{1-5}$, $c_{1-4}$, learning rate $lr.$ Ensure:  B, p, q  1: Initialize parameters of neural network B, p and q  2: Generate training datasets $D_1$, $D_2$, $D_3$, $D_4$ and generate $n_{batch}$ batches  3: for i = 1 to $n_{epoch}$ do:  4:     $L_{epoch}$ = 0;  5:     for j = 1 to $n_{batch}$ do:  6:        calculate $L_{epoch}$ according to Equation (22);  7:        update(B, p, q);  8:     end for  9:     If $L_{epoch}$ = 0: 10:        return candidate B, p, q; 11: Validation of candidate B, p, q by solving MILP problems (28)–(30); 12: The optimizer returns the optimal solution; ${\tilde{\rho }}_{Init}$, ${\tilde{\rho }}_{Unsafe}$, ${\tilde{\rho }}_{Evol}$ and the corresponding input ${\tilde{\mathit{x}}}_{Init}$, ${\tilde{\mathit{x}}}_{Unsafe}$, ${\tilde{\mathit{x}}}_{Evol}$; 13: If ${\tilde{\rho }}_{Init}$, ${\tilde{\rho }}_{Unsafe}$, ${\tilde{\rho }}_{Evol}$ have the expected signs and ${\tilde{\xi }}_{Init}$, ${\tilde{\xi }}_{Unsafe}$, ${\tilde{\xi }}_{Evol}<1$: 14:     Return success and B, p, q; 15: Else: 16:     For the violation ${\tilde{\mathit{x}}}_{Init}$, ${\tilde{\mathit{x}}}_{Unsafe}$, ${\tilde{\mathit{x}}}_{Evol}$, sampling around them and add them to the training set; 17:     Repeat 3–10; 18: return failure;

First, the respective weights among the layers and biases of neural networks B, p, and q are stochastically initialized with a standard Gaussian distribution. Then, the training datasets are generated (lines 1–2). In each epoch, with the loss function according to Equation (22) and the Adam [41] optimizer, the weights and biases of the networks are updated in a backpropagation. By minimizing L to zero, we find the candidates B, p, and q (lines 3–10). With the help of the optimization tool Gurobi, the verification of candidates B, p, and q can be transformed into the optimization problems in Equations (28)–(30). We find the current optimal solutions ${\tilde{\rho }}_{Init}$, ${\tilde{\rho }}_{Unsafe}$, ${\tilde{\rho }}_{Evol}$ and the corresponding inputs ${\tilde{\mathit{x}}}_{Init}$, ${\tilde{\mathit{x}}}_{Unsafe}$, ${\tilde{\mathit{x}}}_{Evol}$ respectively. If the current optimal solutions ${\tilde{\rho }}_{Init}$, ${\tilde{\rho }}_{Unsafe}$, ${\tilde{\rho }}_{Evol}$ have the expected signs, and ${\tilde{\xi }}_{Init}$, ${\tilde{\xi }}_{Unsafe}$, ${\tilde{\xi }}_{Evol}<1$, then the candidates B, p, and q are globally consistent with the constraints in the conditions in Equations (23)–(25). Then, we return success and B, p, and q (Line 14). Otherwise, the sampling data points around the counterexamples are returned and added to the corresponding training datasets to refine the networks (lines 16–17). The training method based on Counterexample-Guided Inductive Synthesis (CEGIS) can train the neural network quickly and accurately. The candidates are trained and refined continuously until finding the real B, p, and q or reaching the maximum epoch number. Figure 2 is the flowchart of Algorithm 1.

4. Experimental Results and Analysis

We have implemented an augmented barrier certificate synthesis tool based on Algorithm 1 using the Pytorch 1.7 platform and Gurobi 9.5. We apply our tool to verify a number of benchmark examples from the literature comprising both polynomial and non-polynomial dynamics and compare it with the SOS programming method proposed in [11].

4.1. Examples

Vehicle Model. Consider the following vehicle model [11]:

$$\begin{array}{cc}\hfill & \mathrm{\Pi }:\left\{\begin{array}{c}{x}_1(t+1)=x_1\left(t\right)+\Delta \tau x_2\left(t\right)+0.5\Delta {\tau }^{2}u\left(t\right)\hfill \\ x_2(t+1)=x_2\left(t\right)+\Delta \tau u\left(t\right)\hfill \\ y\left(t\right)=h\left(x\right(t\left)\right)\hfill \end{array}\right.\hfill \end{array}$$

where $x_1$ and $x_2$ denote the absolute position and velocity of the vehicle, respectively, u is the control input (acceleration), and $\Delta \tau$ is the sampling time. The output is assumed to be the position of the vehicle on the road (i.e., $h\left(x\left(t\right)\right)=x_1\left(t\right)$). Suppose that the initial positions of the vehicle contain critical information which needs to be kept secret and there is a malicious intruder who can observe the behavior of the vehicle and intends to carry out an attack. Thus, we aim to verify the $\delta$-approximate initial-state opacity property for this system, where $\delta$ captures the security guarantee level in terms of the measurement precision of the intruder.

Suppose that state space $\mathbb{X}=[0,10]\times [0,0.1]$, the initial set ${\mathbb{X}}_0=[0,10]\times \left\{0\right\}$, the secret set ${\mathbb{X}}_s=[0,1]\times [0,0.1]$, the non-secret set ${\mathbb{X}}_{ns}=[1,10]\times [0,0.1]$, and the control input set $\mathbb{U}=[-0.05,0.05]$. For the augmented system $\mathrm{\Pi }\times \mathrm{\Pi }$, the initial set ${\tilde{\mathrm{\Theta }}}_0=\{\mathit{x}\in [0,1]\times \left\{0\right\}$ and $\widehat{\mathit{x}}\in [1,10]\times \left\{0\right\}|\widehat{\mathit{x}}=p\left(\mathit{x}\right),{(x_1-{\widehat{x}}_1)}^2\le {\delta }^2\}$, where $p\left(\mathit{x}\right)$ is a relation function defined in Equation (10) and the unsafe set ${\mathrm{\Theta }}_u=\{(\mathit{x};\widehat{\mathit{x}})\in \mathbb{X}\times \mathbb{X}|{(x_1-{\widehat{x}}_1)}^2>{\delta }^2\}$.

Now, we set the measurement accuracy to be $\delta =1$ and search for the augmented barrier certificate B satisfying the conditions in Theorem 2. By applying our approach, we successfully obtain the neural augmented barrier certificate B and the neural relation functions p and q, which are represented as once-hidden layer ReLU-activated neural networks with 16 neurons per layer. Therefore, the $\delta$-approximate initial-state opacity of the system $\mathrm{\Pi }$ is guaranteed.

Figure 3 (left) depicts a phase portrait of the augmented system $\mathrm{\Pi }\times \mathrm{\Pi }$ with $x_2={\widehat{x}}_2=0$. It is seen that in the projection space on the position plane, the zero sublevel set of the barrier certificate $B(\mathit{x},\widehat{\mathit{x}})$ (the yellow surface) separates ${\mathrm{\Theta }}_u$ (the red region) from all trajectories starting from ${\tilde{\mathrm{\Theta }}}_0$ (the pink line). Note that the initial set ${\tilde{\mathrm{\Theta }}}_0$ (the pink line), described by the neural relation function p, is a subset of the initial set ${\mathrm{\Theta }}_0$ (the green triangle), which is defined in [11]. This means that our relaxed augmented barrier certificate yields a weaker sufficient condition for the $\delta$-approximate initial-state opacity.

Figure 3 (right) shows the projection of a number of state trajectories on the position plane of the augmented system $\mathrm{\Pi }$×$\mathrm{\Pi }$ starting from the random initial states in ${\tilde{\mathrm{\Theta }}}_0$ under control input $\widehat{\mathit{u}}=q(\mathit{x},\widehat{\mathit{x}},\mathit{u})$, with $\mathit{u}$ taking values in $\mathbb{U}$. It can be seen that any trajectory starting from ${\tilde{\mathrm{\Theta }}}_0$ does not reach the unsafe set ${\mathrm{\Theta }}_u$ (the blue region) as the time increases.

Three-dimensional model. In this example, we consider the following three-dimensional system:

$$\begin{array}{cc}\hfill & \mathrm{\Pi }:\left\{\begin{array}{c}{x}_1(t+1)=x_1\left(t\right)+0.5x_2\left(t\right)+x_3\left(t\right)+0.5u\left(t\right)\hfill \\ x_2(t+1)=x_2\left(t\right)+0.5u\left(t\right)\hfill \\ x_3(t+1)=0.2{\left(x_3\left(t\right)\right)}^3+0.05u\left(t\right)\hfill \\ y\left(t\right)=x_1\left(t\right).\hfill \end{array}\right.\hfill \end{array}$$

Suppose that state space $\mathbb{X}=[0,4]\times [0,0.1]\times [0,0.5]$, the initial set ${\mathbb{X}}_0=\mathbb{X}$, the secret set ${\mathbb{X}}_s=[0,0.8]\times [0,0.1]\times [0,0.5]$, the non-secret set ${\mathbb{X}}_{ns}=[0.8,4]\times [0,0.1]\times [0,0.5]$, and the control input set $\mathbb{U}=[0.03,0.05]$. For the augmented system $\mathrm{\Pi }\times \mathrm{\Pi }$, the initial set ${\tilde{\mathrm{\Theta }}}_0=\{\mathit{x}\in [0,0.8]\times [0,0.1]\times [0,0.5]$, $\widehat{\mathit{x}}\in [0.8,4]\times [0,0.1]\times [0,0.5]|\widehat{\mathit{x}}=p\left(\mathit{x}\right),{(x_1-{\widehat{x}}_1)}^2\le {\delta }^2\}$, where $p\left(\mathit{x}\right)$ is a relation function defined in Equation (10) and the unsafe set ${\mathrm{\Theta }}_u=\{(\mathit{x};\widehat{\mathit{x}})\in \mathbb{X}\times \mathbb{X}|{(x_1-{\widehat{x}}_1)}^2>{\delta }^2\}$. We aim to verify the $\delta$-approximate initial-state opacity property for the system, where $\delta =2.2$ captures the security guarantee level in terms of the measurement precision of the intruder.

It suffices to synthesize certificate functions B, p, and q, satisfying the conditions in Theorem 3. By applying our approach, we successfully obtain the neural augmented barrier certificate B, represented as two hidden layer ReLU activated neural networks with 16 neurons per layer, and the neural relation functions p and q, represented as one hidden layer ReLU activated neural networks with 10 neurons per layer. Therefore, the $\delta$-approximate initial-state opacity of the given system $\mathrm{\Pi }$ is guaranteed.

Figure 4 (left) depicts phase portrait of the augmented system $\mathrm{\Pi }\times \mathrm{\Pi }$ with $x_2={\widehat{x}}_2=0.05$ and $x_3={\widehat{x}}_3=0.1$. It is seen that in the projection space on the position plane, the zero sub-level set of the barrier certificate $B(\mathit{x},\widehat{\mathit{x}})$ (the yellow surface) separates ${\mathrm{\Theta }}_u$ (the red region) from all trajectories starting from ${\tilde{\mathrm{\Theta }}}_0$ (the pink line). Compared with the initial set ${\mathrm{\Theta }}_0$ (the green triangle), which is defined in [11], the initial set ${\tilde{\mathrm{\Theta }}}_0$ (the pink line) is much smaller, which means that our relaxed augmented barrier certificate yields a weaker sufficient condition for the $\delta$-approximate initial-state opacity.

Figure 4 (right) shows the projection of a number of state trajectories on the position plane of the augmented system $\mathrm{\Pi }$×$\mathrm{\Pi }$ starting from the random initial states in ${\tilde{\mathrm{\Theta }}}_0$ under control input $\widehat{\mathit{u}}=q(\mathit{x},\widehat{\mathit{x}},\mathit{u})$, with $\mathit{u}$ taking values in $\mathbb{U}$. It can be seen that all such trajectories do not reach the unsafe set ${\mathrm{\Theta }}_u$ (the blue region).

Non-polynomial model. Consider a non-polynomial system $\mathrm{\Pi }$:

$$\begin{array}{cc}\hfill & \mathrm{\Pi }=\left\{\begin{array}{c}{x}_1(t+1)=x_1\left(t\right)+x_2\left(t\right)+u\left(t\right)\hfill \\ x_2(t+1)=e^{x_2\left(t\right)}+u\left(t\right)-1\hfill \\ y\left(k\right)=x_1\left(k\right).\hfill \end{array}\right.\hfill \end{array}$$

(31)

Suppose that the state set $\mathbb{X}=[1,5]\times [0,0.1]$, the initial set ${\mathbb{X}}_0=[1,5]\times [0,0.1]$, the secret set ${\mathbb{X}}_s=[1,1.1]\times [0,0.1]$, the non-secret set ${\mathbb{X}}_{ns}=\mathbb{X}\backslash {\mathbb{X}}_s$, and the control input set $\mathbb{U}=[0.03,0.05]$. We aim to verify the $\delta$-approximate initial-state opacity property for the system, where $\delta =0.95$ captures the security guarantee level in terms of the measurement precision of the intruder.

For the augmented system $\mathrm{\Pi }\times \mathrm{\Pi }$, the initial set ${\tilde{\mathrm{\Theta }}}_0=\{\mathit{x}\in [1,1.1]\times [0,0.1]$, $\widehat{\mathit{x}}\in [1.1,5]\times [0,0.1]|\widehat{\mathit{x}}=p\left(\mathit{x}\right),{(x_1-{\widehat{x}}_1)}^2\le {\delta }^2\}$, where $p\left(\mathit{x}\right)$ is a relation function defined in Equation (10) and the unsafe set ${\mathrm{\Theta }}_u=\{[\mathit{x};\widehat{\mathit{x}}]\in \mathbb{X}\times \mathbb{X}|{(x_1-{\widehat{x}}_1)}^2>{\delta }^2\}$. It suffices to synthesize certificate functions B, p, and q, satisfying the conditions in Theorem 3. By applying our approach, we successfully obtain the neural augmented barrier certificate B, represented as two hidden layer ReLU activated neural networks with 16 neurons per layer, and the neural relation functions p and q, represented as one hidden layer ReLU activated neural networks with 16 neurons per layer. Therefore, the $\delta$-approximate initial-state opacity of the given system $\mathrm{\Pi }$ is guaranteed.

Figure 5 (left) depicts a phase portrait of the augmented system $\mathrm{\Pi }\times \mathrm{\Pi }$ with $x_2={\widehat{x}}_2=0$. It is seen that in the projection space on the position plane, the zero sub-level set of the barrier certificate $B(\mathit{x},\widehat{\mathit{x}})$ (the yellow surface) separates ${\mathrm{\Theta }}_u$ (the red region) from all trajectories starting from ${\tilde{\mathrm{\Theta }}}_0$ (the pink line). Clearly, the initial set ${\tilde{\mathrm{\Theta }}}_0$ (the pink line) is much smaller than the initial set ${\mathrm{\Theta }}_0$ (the green triangle), which is defined in [11]. Figure 5 (right) shows the projection of a number of state trajectories on the position plane of the augmented system $\mathrm{\Pi }$×$\mathrm{\Pi }$, starting from the random initial states in ${\tilde{\mathrm{\Theta }}}_0$ under control input $\widehat{\mathit{u}}=q(\mathit{x},\widehat{\mathit{x}},\mathit{u})$, with $\mathit{u}$ taking values in $\mathbb{U}$. It is obvious that such trajectories does not reach the unsafe set ${\mathrm{\Theta }}_u$ (the blue region) as the time increases.

4.2. Performance Evaluation

The statistics of our case studies are summarized in

Table 1. Performance evaluation.

Ex	$\left|\mathit{x}\right|$	B	p	q	$\mathit{l}\mathit{r}$	${\mathit{\alpha }}_3$	${\mathit{\alpha }}_{1,2,4,5}$	${\mathit{c}}_1$	${\mathit{c}}_2$	${\mathit{c}}_3$	${\mathit{c}}_4$
Ex 1	4	4-16-1	2-16-2	5-16-1	0.01	1.2	1	0	0.01	0.01	0.001
Ex 2	6	6-16-16-1	3-10-3	7-10-1	0.1	1.5	1	0.01	0.05	0.01	0.001
Ex 3	4	4-16-1	2-16-2	5-16-1	0.01	1.2	1	0.01	0.05	0.01	0.001

. $\left|x\right|$ denotes the number of augmented system variables, $B,p$, and q denote the structure of the neural network barrier certificates and relation functions, and $lr$ denotes the learning rate. ${\alpha }_1,{\alpha }_2,{\alpha }_3,{\alpha }_4$, and ${\alpha }_5$ denote the weights of the sub-losses. The tolerances $c_1,c_2,c_3$, and $c_4$ are used for training neural barrier certificates and relation functions.

For the vehicle model, note that the SOS programming method is presented in [11] for computing the augmented barrier certificates, but it lacks soundness due to numerical errors. More concretely, for the augmented barrier certificate $B(\mathit{x},\widehat{\mathit{x}})$ and corresponding control policy $\widehat{\mathit{u}}(\mathit{x},\widehat{\mathit{x}},\mathit{u})$ given in [11], there exist some counterexamples, such as ($x_1$, ${\widehat{x}}_1$, $x_2$, ${\widehat{x}}_2$) = (0.5, 1.25, 0, 0), (0, 1.0312, 0.0625, 0.0625), (0.25, 0.0625, 0.0625, 0.0977, −0.0488), which violate the three conditions of the augmented barrier certificates defined in Theorem 1.

For the three-dimensional model, we also apply the SOS programming method presented in [11] to search for the augmented barrier certificate and the relation functions. However, the SOS programming method cannot yield such certificate functions with a degree less than six within one hour. The reason for this is that the size of the optimization problem derived from opacity verification grows exponentially with the number of state variables and the degree of the polynomial involved in the system, which imposes a serious size limitation on the system to be verified. For the augmented system $\mathrm{\Pi }\times \mathrm{\Pi }$, the number of state variables is six, and the degree of the involved polynomials is three. Therefore, due to the large size, the bilinear SOS program associated to opacity verification cannot be solved efficiently.

For the non-polynomial model, note that SOS programming method in [11] cannot deal with this non-polynomial model directly, and one should combine a recasting procedure to transform it into an equivalent polynomial model with a much higher dimension.

It is interesting to investigate what impact a less conservative sufficient condition on the synthesis of neural networks as barrier certificates and relation functions to verify the approximate initial-state opacity has. For the three examples mentioned above, there still exist counterexamples after multiple iterations of synthetic neural barrier certificates and relation functions based on the conditions in [11], and there are also counterexamples based on the SOS programming. This is because the conditions in [11] are too strict on the initial region. Based on Theorem 3, we can synthesize the valid neural barrier certificates and relation functions successfully with no counterexamples. As shown in the left part of Figure 3, Figure 4 and Figure 5, the initial region becomes much smaller under the premise of satisfying the approximate initial-state opacity.

5. Conclusions

In this paper, a method for synthesizing neural augmented barrier certificates to verify the approximate initial-state opacity of discrete-time control systems is presented. The existence of augmented barrier certificates can serve as a guarantee for approximate initial-state opacity. We propose a novel form of augmented barrier certificates that generates a weaker sufficient condition for the approximate initial-state opacity. We then employ an algorithmic framework involving a learner for training neural certificates via the deep learning method and alternate a verifier for verifying the candidate certificates by solving several mixed-integer linear programs. The verifier either ensures the validity of the candidate certificates or yields counterexamples for further training. The effectiveness of the proposed approach is illustrated via several numerical examples.