Modeling and Optimal Supervisory Control of Networked Discrete-Event Systems and Their Application in Traffic Management

Abstract

In this paper, we investigate the modeling and control of networked discrete-event systems (DESs), where a supervisor is connected to the plant via an observation channel and the control commands issued by the supervisor are delivered to the actuator of the plant via a control channel. Communication delays exist in both the observation channel and the control channel. First, a novel modeling framework for the supervisory control of DESs subject to observation delays and control delays is presented. The framework explicitly models the interaction process between the plant and the supervisor over the communication channels. Compared with the previous work, a more accurate “dynamics” of the closed-loop system is specified. Under this framework, we further discuss how to estimate the states of the closed-loop system in the presence of observation delays and control delays. Based on the state estimation, we synthesize an optimal supervisor on the fly to maximize the controlled behaviors while preventing the system from leaving the desired behaviors under communication delays. We compare the proposed supervisor with the supervisor proposed in the literature and show that the proposed supervisor is more permissive. As an application, we show how the proposed approach can be applied to manage vehicles in a signal intersection. Finally, we show how to extend the proposed framework to model a system whose actuators and sensors are distributed at different sites.

1. Introduction

The dynamics of DESs are driven by sequences of asynchronous events. The main control theory developed for DESs is the supervisory control theory, where a supervisor is desired to disable events that lead to some undesirable event sequences. Since the supervisor cannot control and observe all the events, the desired behaviors (control objective) could be unachievable. The necessary and sufficient conditions for the existence of a supervisor are characterized as controllability [1] and observability [2]. Since then, the supervisory control is extended in several directions, such as decentralized supervisory control [3], robust supervisory control [4], asynchronous supervisory control [5], and quantitative supervisory control [6].

Nowadays, in many industrial applications, the supervisor is usually connected to the plant via communication networks. Such a network structure provides efficient ways for controlling DESs. However, the communication delays existing in the observation channel and the control channel pose significant challenges to the supervisory control of DESs [7,8,9,10,11,12,13,14]. Thus, networked DESs have drawn much attention in the past few years [15,16,17,18,19,20] Most of the current works on networked supervisory control focus on verifying if a given control objective can be achieved under observation delays and control delays [20,21,22,23,24,25,26,27,28,29,30], which is known as the supervisor existence problem. When the desired language cannot be exactly achieved, one would compute a safe control policy online or offline, known as the supervisor synthesis problem [31,32,33,34,35,36]. In this paper, we focus on solving the maximally-permissive supervisor synthesis problem under observation delays and control delays. In particular, based on the infinite observed sequence of events, an online algorithm is presented in this paper to calculate a maximal supervisor under observation delays and control delays. The calculated online supervisor is optimal because (i) the system is prevented from leaving the desired language even if communication delays exist in both the observation channel and the control channel, and (ii) given (i) is satisfied, the language generated by the closed-loop system is maximized.

In the supervisor synthesis, state estimation is a crucial step in determining a valid control action after each new observation. The state estimation problem can be briefly stated as follows: estimate all of the states of the closed-loop system that may be under communication delays, given that all future control decisions are unknowable. To synthesize an optimal supervisor under communication delays, the authors in [22,24,37,38] compute the state estimate based on the open-loop system without using the information of the controls imposed on the system. As stated in [39], the state estimate calculated in [22,24,37,38] contains some states that have been prevented from reaching. Therefore, the solutions computed in [22,24,37,38] are suboptimal for the unrestricted domain of observed event sequences. In [29], the state estimates are computed based on the assumption that the control delays and the observation delays are constant. The proposed approach fails to deal with nondeterministic observation delays and control delays. Recently, the authors in [26] calculated the state estimates of the networked DESs by taking the information of the control decision’s history into consideration. Nevertheless, the work of [26] can be further improved in two directions. First, the framework of the networked supervisory control adopted in [26] is conservative in the sense that the specified language of the closed-loop system is an over-approximation of the actual language of the closed-loop system. That is, it may include some sequences that never occur in practice (see Example 1 for more details), and the state estimate computed by [26] may contain some states that the closed system never reaches. Thus, the synthesized supervisor could be restrictive in the sense that it may disable some unnecessary events. Second, the work of [26] considers only control delays. When only control delays exist, the observation of a supervisor to a string is deterministic and the control command made after a string can be uniquely determined. In practice, however, the delays often exist in both the observation channel and the control channel. If this is the case, the observation of a supervisor to a string is nondeterministic and varies with the different observations. The supervisor may make different control decisions based on different observations, which complicates the supervisor’s synthesis problem.

In this paper, a new modeling framework for the supervisory control of DESs under control delays and observation delays is first presented. Specifically, in the newly proposed framework, we model the observation channel by a sequence of pairs of an occurred event and its observation delays (called the observation channel configuration). We also model the control channel by a sequence of pairs of an issued control command and its control delays (called the control channel configuration). We then build an automaton to model the interaction process between the plant and the supervisor over the observation channel and the control channel. In the automaton, two special types of events representing the respective receptions of observable events and the executions of control commands are introduced. Each state of the automaton dynamically tracks the plant state, the current control command, the observation channel configuration, the control channel configuration, and the supervisor state. Based on the constructed automaton, the exact language of the closed-loop system can be specified. Under the framework, we then discuss how to estimate (and predict) all the states of the current (and future) closed-loop system. Without any structural assumption on the solution space, an online algorithm is finally presented to calculate a maximal network-controlled policy based on the infinite observed sequence of events. We further compare the proposed supervisor with the supervisor proposed in [26]. The previous framework may contain some physically impossible strings. This may damage the supervisor’s synthesis because a synthesized good supervisor may be mistakenly taken as a bad supervisor. There exists the possibility that all of the illegal strings that may be generated by the closed-loop system are physically impossible. In such situations, the controlled system can never reach an illegal state as all of the illegal strings never occur in reality. Since the proposed framework excludes all physically impossible strings, the state estimation is more precise than the previous approach. Thus, the proposed supervisor is more permissive than the previous one.

To show the application of the proposed modeling and control approach, we consider the vehicle management problem in a signal intersection. When a self-driving vehicle arrives at the intersection, it needs to communicate with the intersection to determine the traffic light color. If the traffic light is yellow or red, it must stop and wait until the traffic light is switched to green. Otherwise, if the traffic light is green, it can pass through the intersection. We show that the proposed approach can be used to achieve control objectives when control delays and observation delays exist.

Finally, we briefly discuss how to extend the proposed approaches to deal with non-FIFO observations and controls. Specifically, we consider a system where the actuators and the sensors are distributed at different sites. For each actuator, the supervisor sends control commands to it over an individual control channel, and for each sensor, the detected information is sent to the supervisor over an individual observation channel. Different channels may have different upper bounds of delays. Techniques are developed to model the dynamics of the closed-loop system.

The proposed supervisor synthesis approach differs from the existing works in the following sense.

• In contrast to [26], we consider both the control delays and the observation delays in this paper. That is, the observation of the supervisor to a string is nondeterministic and varies with the different observation delays. For different observations, the supervisor may make different control decisions. An event after a string may be allowed to occur after some of these control decisions but not be allowed to occur for the other control decisions. Thus, we must consider all possibilities. In addition, the closed-loop system behaviors specified in the proposed framework exclude those strings that never occur in reality and are shown to be more accurate. As a result, the supervisor can estimate the states of the closed-loop system more accurately and make control decisions more reasonable at any instant.
• Compared with [22,24,37,38], the supervisor makes control decisions based on closed-loop systems. In other words, the synthesized supervisor considers controls imposed on the system when making control decisions. Thus, the control command made by the proposed supervisor is optimal with respect to the unrestricted domain of the observed event sequences.
• Different from [29], the proposed model assumes that the observation delays and control delays are nondeterministic, which often happens. In this paper, the observation delays and control delays are measured by the number of events occurring in the plant. More specifically, the observation delays and control delays are upper-bounded by $N_o$ and $N_c$ events, respectively. That is, all of the events delayed at the observation channel can be communicated to the supervisor (in the same order that they are generated) before no more than $N_o$ event occurrences. All control commands delayed at the control channel can be executed by the actuator (in the same order that they are issued) before no more than $N_c$ events (since they are issued).

The rest of this paper is organized as follows. Section 2 presents some preliminary concepts and the required assumptions in this paper. Section 3 introduces a new modeling framework for supervisory control with observation delays and control delays. An online procedure for estimating the states of the closed-loop system is presented in Section 4. Section 5 synthesizes a maximal and safe networked supervisor on the fly. Section 6 discusses an application for the vehicle control in a signal intersection. Section 7 extends the proposed approaches to deal with non-FIFO observations and controls. Section 8 concludes this paper.

2. Preliminaries

We model a DES using a deterministic finite-state automaton $G=(Q,\Sigma ,\delta ,q_0)$, where Q is the finite set of states; $\Sigma$ is the finite set of events; $\delta :Q\times \Sigma \to Q$ is the transition function; $q_0$ is the initial state. ${\Sigma }^{*}$ is the Kleene closure of $\Sigma$, i.e., the set of all sequences over events in $\Sigma$. $\delta$ is extended to $Q\times {\Sigma }^{*}$ in the usual way [40]. The language generated by G is denoted by $\mathcal{L}(G)$. $\epsilon$ is the empty sequence. “!” means “is defined”.

Given a $s={\sigma }_1{\sigma }_2\cdots {\sigma }_k\in {\Sigma }^{*}$, we write $s^i={\sigma }_1{\sigma }_2\cdots {\sigma }_i$ for $i=1,2,\dots ,k$, and $s^0=\epsilon$. $\left|s\right|$ is the length of s. $\overline{s}=\left\{s^{\prime }\right|(\exists s^{″})s^{\prime }{s}^{″}=s\}$ denotes the set of all prefixes of s. $s_{-i}$ denotes the prefix of s, such that $|s_{-i}|=max\{0,\left|s\right|-i\}$. Let ${\Sigma }^{\le N}=\{s\in {\Sigma }^{*}:\left|s\right|\le N\}$. Let $s\backslash t$ be the suffix of s after its prefix t, i.e.,  $t(s\backslash t)=s$. If t is not a prefix of s, then $s\backslash t$ is not defined. The prefix closure of a language $L\subseteq {\Sigma }^{*}$ is denoted by $\overline{L}$. L is prefix-closed if $L=\overline{L}$. In this paper, only prefix-closed languages are considered. $\mathbb{N}$ is the set of natural numbers. Let $[0,N]=\{n\in \mathbb{N}:n\le N\}$ be the set of natural numbers no larger than N. Given $G_1$ and $G_2$, we say $G_1$ is a sub-automaton of $G_2$, denoted by $G_1⊑G_2$, if $G_1$ can be obtained from $G_2$ by deleting some states in $G_2$ and all transitions connect to these states.

In many applications, the original system G may not satisfy the desired specification. To make the system fulfill the specification, the supervisory control finds a supervisor to dynamically disable events that lead to some undesirable sequences. In general, not all of the events are controllable and observable. We partition $\Sigma ={\Sigma }_c\cup {\Sigma }_{uc}$ into the set of controllable events ${\Sigma }_c$ and the set of uncontrollable events ${\Sigma }_{uc}$. We also partition $\Sigma ={\Sigma }_o\cup {\Sigma }_{uo}$ into the set of observable events ${\Sigma }_o$ and the set of unobservable events ${\Sigma }_{uo}$. The natural projection $P:\mathcal{L}(G)\to {\Sigma }_o^{*}$ is recursively defined as: $P(\epsilon )=\epsilon$ and, for all $s,s\sigma \in \mathcal{L}(G)$, $P(s\sigma )=P(s)\sigma$, if $\sigma \in {\Sigma }_o$, and $P(s\sigma )=P(s)$, if $\sigma \in {\Sigma }_{uo}$.

We denote, in this paper, the supervisor by a pair $S=(A,\chi )$, where $A=(X,{\Sigma }_o,\xi ,x_0)$ is a deterministic automaton with $\mathcal{L}(A)={\Sigma }_o^{*}$, and $\chi :X\to 2^{\Sigma }$ is a function that specifies the set of events to be enabled. Specifically, for any $t\in {\Sigma }_o^{*}$, we denote $\chi (\xi (x_0,t))$ by the set of events to be enabled after observing t. With a slight abuse of notation, we write $S(t)=\chi (\xi (x_0,t))$. More details on the definition of S are provided in Example 1. Let $\Pi =\{\pi \in 2^{\Sigma }:{\Sigma }_{uc}\subseteq \pi \}$ be the set of all the admissible control commands. Since we cannot disable an uncontrollable event, $S(t)\in \Pi$ for all $t\in {\Sigma }_o^{*}$. The control objective in this paper is given by a specification language $K\subseteq \mathcal{L}(G)$. We assume that K can be represented by a sub-automaton $H⊑G$ of G. The automaton representation of language $K=\mathcal{L}(H)$ with $K\subseteq \mathcal{L}(G)$ can always be changed to satisfy $H⊑G$.

As shown in Figure 1, in the networked DESs, communications from the plant (supervisor) to the supervisor (plant) for the observation (control) are carried out over an observation channel (control channel) subject to random delays. We assume first-in-first-out (FIFO) is satisfied in both the observation and control, i.e., the observations of events are sent to the supervisor in the same order that they are generated and the control commands are executed in the same order that they are issued. As shown in [21,22,23,24], the delays are measured by the number of event occurrences (observable or not). We assume that (1) the observation delays are upper-bounded by $N_o$ event occurrences, i.e.,  when an observable event occurs, it can be communicated to the supervisor before no more than $N_o$ additional event occurrences; (2) the control delays are upper-bounded by $N_c$ event occurrences, i.e., after a control command is issued, it can be executed before no more than $N_c$ event occurrences. We assume that the initial control command has been deployed in the actuator of the plant beforehand. When the plant is initialized and starts to work, the initial control command can be executed without any delays.

Given system G and a supervisor S defined over ${\Sigma }_o^{*}$, we consider all possible strings, which may be generated by the closed-loop system (also called the controlled system) when the observation delays and control delays are upper-bounded by $N_o$ and $N_c$, respectively. Before that, let us first recall how the previous works specify the language of the closed-loop system under observation delays and control delays. As shown in [23], an upper bound on possible strings, denoted by ${\mathcal{L}}_a(S/G)$, which may be generated by the controlled system under observation delays and control delays is defined as follows:

• $\epsilon \in {\mathcal{L}}_a(S/G)$;
• for any $s\in {\mathcal{L}}_a(S/G)$ and $s\sigma \in \mathcal{L}(G)$ with $\sigma \in \Sigma$, $s\sigma \in {\mathcal{L}}_a(S/G)$ if $\sigma$ is enabled by one of the control commands issued in the past $N_c+N_o$ steps, i.e.,

  $$\begin{array}{cc}\hfill & [(\forall s\in {\mathcal{L}}_a(S/G))(\forall \sigma \in \Sigma )s\sigma \in \mathcal{L}(G)]s\sigma \in {\mathcal{L}}_a(S/G)\iff \hfill \\ \hfill & \sigma \in S(P(s))\cup S(P(s_{-1}))\cup \cdots \cup S(P(s_{-N_o-N_c})).\hfill \end{array}$$

In [22,24], the language ${\mathcal{L}}_a(S/G)$ is also referred to as the large language. However, as discussed in [23,26], ${\mathcal{L}}_a(S/G)$ is not the exact language that may be generated by the closed-loop system. It is essentially an over-approximation of the actual language that may be generated by the closed-loop system and may contain some sequences that never occur in reality. To make this paper self-contained, we use the following simple example to illustrate this.

Example 1. 

Consider the system G depicted in Figure 2a with $\Sigma =\{\alpha ,\beta ,\eta \}$, ${\Sigma }_o=\{\alpha ,\beta \}$, and ${\Sigma }_c=\Sigma$. Let $N_{o,1}=N_{c,1}=1$, i.e., the upper bounds of control delays and observation delays are both 1. The supervisor $S=(A,\chi )$ is depicted in Figure 2b. The function χ is specified by the set of events associated with each state in Figure 2b. Specifically, $S(\epsilon )=\chi (x_0)={\pi }_0=\{\alpha ,\eta \}$. When α is observed, automaton A moves to state $x_1$ from state $x_0$, and $S(\alpha )=\chi (x_1)={\pi }_1=\left\{\beta \right\}$. When $\alpha \beta$ is observed, automaton A moves to state $x_2$ from state $x_1$, and $S(\alpha \beta )=\chi (x_2)={\pi }_2=\left\{\eta \right\}$. For the other $t\in {\Sigma }_o^{*}\backslash \{\epsilon ,\alpha ,\alpha \beta \}$, $S(t)=\chi (x_3)={\pi }_3=\varnothing$. We first show that $\alpha \beta \alpha \in \mathcal{L}(S/G)$.

At first, we have $\epsilon \in {\mathcal{L}}_a(S/G)$. Since $\epsilon \in {\mathcal{L}}_a(S/G)$, $\alpha \in S(\epsilon )$, and $\alpha \in \mathcal{L}(G)$, by definition, $\alpha \in {\mathcal{L}}_a(S/G)$. Moreover, since $\alpha \in {\mathcal{L}}_a(S/G)$, $\beta \in S(P(\alpha ))=S(\alpha )$, and $\alpha \beta \in \mathcal{L}(G)$, by definition, $\alpha \beta \in {\mathcal{L}}_a(S/G)$. Then, since $\alpha \beta \in {\mathcal{L}}_a(S/G)$, $\alpha \in S(P({(\alpha \beta )}_{-2}))=S(\epsilon )$, and $\alpha \beta \alpha \in \mathcal{L}(G)$, by definition, $\alpha \beta \alpha \in {\mathcal{L}}_a(S/G)$. We next show that $\alpha \beta \alpha$ never occurs in practice.

Since $\alpha \in S(\epsilon )$, $\alpha \notin S(\alpha )$, and $\alpha \notin S(\alpha \beta )$, one can check that α can occur after $\alpha \beta$ only if $S(\epsilon )$ is taking effect when α occurs after $\alpha \beta$. However, since $\beta \in S(\alpha )$ and $\beta \notin S(\epsilon )$, $S(\alpha )$ must have been executed at the time β occurs after α. In other words, $S(\epsilon )$ must have been replaced by $S(\alpha )$ after the occurrence of $\alpha \beta$. Therefore, $\alpha \beta \alpha$ never occurs (under S) in reality.

To obtain the exact language of the closed-loop system, we need a new modeling framework for networked DESs, which will be discussed in the following section.

3. Modeling Framework for Networked Supervisory Control

In this section, we consider a new modeling framework for the network supervisory control of DESs. In the new framework, we model the observation channel by a sequence of observable events waiting to be communicated and their observation delays. We also model the control channel by a sequence of control commands waiting to be executed and their control delays. We then build an automaton to describe how the supervisor and the plant interact with each other over the observation channel and the control channel. It is shown that the language of the closed-loop system subject to communication delays can be simply “decoded” from sequences of the constructed automaton.

3.1. Modeling of the Communication Channels

Let us first consider the observation channel.

Definition 1. 

The observation channel configuration is defined as a sequence of pairs:

$${\theta }_o=({\sigma }_1,n_1)\cdots ({\sigma }_k,n_k),$$

where ${\sigma }_1\cdots {\sigma }_k\in {\Sigma }_o^{*}$ are the observable events (in the same order that they are generated) that have occurred but are currently delayed at the observation channel, and $n_i\in [0,N_o]$, $i=1,\dots ,k$ is the number of event occurrences since ${\sigma }_i$ occurred. If the observation channel is empty, ${\theta }_o=\epsilon$.

We denote by ${\Theta }_o\subseteq {({\Sigma }_o\times [0,N_o])}^{\le N}$ the set of all the possible observation channel configurations, where $N\in \mathbb{N}$ is the maximum length of ${\theta }_o\in {\Theta }_o$. The observation channel configuration ${\theta }_o$ is evolving when a new event occurs or a new observable event is communicated. To update ${\theta }_o$, we introduce the following operators.

• When a new event $\sigma \in \Sigma$ occurs, to update the observation channel configuration, we define the operator ${\mathbf{IN}}^{obs}:{\Theta }_o\times \Sigma \to {\Theta }_o$ as: for all ${\theta }_o\in {\Theta }_o$ and all $\sigma \in \Sigma$,

  $$\begin{array}{c}\hfill {\mathbf{IN}}^{obs}({\theta }_o,\sigma )=\left\{\begin{array}{cc}{\theta }_o^{+}\hfill & \mathrm{if}\sigma \in {\Sigma }_{uo}\hfill \\ {\theta }_o^{+}(\sigma ,0)\hfill & \mathrm{if}\sigma \in {\Sigma }_o,\hfill \end{array}\right.\end{array}$$

  (1)

  where if ${\theta }_o=({\sigma }_1,n_1)\cdots ({\sigma }_k,n_k)\ne \epsilon$, ${\theta }_o^{+}=({\sigma }_1,n_1+1)\cdots ({\sigma }_k,n_k+1)$, and if ${\theta }_o=\epsilon$, ${\theta }_o^{+}=\epsilon$.
• When a new observable event $\sigma \in {\Sigma }_o$ delayed at the observation channel is communicated, to update the observation channel configuration, we define the operator ${\mathbf{OUT}}^{obs}:{\Theta }_o\times {\Sigma }_o\to {\Theta }_o$ as: for all ${\theta }_o\in {\Theta }_o$ and all $\sigma \in {\Sigma }_o$,

  $$\begin{array}{c}\hfill {\mathbf{OUT}}^{obs}({\theta }_o,\sigma )=\left\{\begin{array}{cc}{\theta }_o\backslash ({\sigma }_1,n_1)\hfill & \mathrm{if}{\theta }_o=({\sigma }_1,n_1)\cdots ({\sigma }_k,n_k)\ne \epsilon \wedge \hfill \\ \hfill & \sigma ={\sigma }_1\hfill \\ \mathrm{undefined}\hfill & \mathrm{otherwise}.\hfill \end{array}\right.\end{array}$$

  (2)

When an event $\sigma \in \Sigma$ occurs in the plant, all the natural numbers in ${\theta }_o$ should be ’plus 1’ since they are used to count the observation delays. Furthermore, if $\sigma \in {\Sigma }_o$, by FIFO, we still need to add $(\sigma ,0)$ to the end of ${\theta }_o$ for recording the new observable event occurrence. That is what the operator ${\mathbf{IN}}^{obs}(·)$ does in Equation (1). On the other hand, when a new observable event is communicated to the supervisor, by FIFO, it must be the first event queued at the observation channel. Therefore, we define ${\mathbf{OUT}}^{obs}(·)$ to remove the first event $\sigma$ from ${\theta }_o$. Next, let us consider the control channel.

Definition 2. 

The control channel configuration is defined as a sequence of pairs:

$${\theta }_c=({\pi }_1,m_1)\cdots ({\pi }_h,m_h),$$

where ${\pi }_1\cdots {\pi }_h\in {\Pi }^{*}$ is a sequence of control commands (in the same order that they are issued) that have been issued but are currently delayed at the control channel, and $m_i\in [0,N_c]$, $i=1,\dots ,h$ is the number of event occurrences since ${\pi }_i$ has been issued. If the control channel is empty, ${\theta }_c=\epsilon$.

We denote by ${\Theta }_c\subseteq {(\Pi \times [0,N_c])}^{\le M}$ the set of all possible control channel configurations, where $M\in \mathbb{N}$ is the maximum length of ${\theta }_c\in {\Theta }_c$. To update ${\theta }_c$, we introduce the following operators.

• When a new event $\sigma \in \Sigma$ occurs in the plant, to update the control channel configuration, we define the operator $\mathbf{PLUS}:{\Theta }_c\to {\Theta }_c$ as: for all ${\theta }_c\in {\Theta }_c$,

  $$\begin{array}{c}\hfill \mathbf{PLUS}({\theta }_c)={\theta }_c^{+},\end{array}$$

  (3)

  where if ${\theta }_c=({\pi }_1,m_1)\cdots ({\pi }_h,m_h)\ne \epsilon$, ${\theta }_c^{+}=({\pi }_1,m_1+1)\cdots ({\pi }_h,m_h+1)$, and if ${\theta }_c=\epsilon$, ${\theta }_c^{+}=\epsilon$.
• When a new control command $\pi \in \Pi$ is issued by the supervisor, to update the control channel configuration, we define the operator ${\mathbf{IN}}^{ctr}:{\Theta }_c\times \Pi \to {\Theta }_c$ as: for all ${\theta }_c\in {\Theta }_c$ and all $\pi \in \Pi$,

  $$\begin{array}{c}\hfill {\mathbf{IN}}^{ctr}({\theta }_c,\pi )={\theta }_c(\pi ,0).\end{array}$$

  (4)
• When a new control command $\pi \in \Pi$ delayed at the control channel is executed, to update the control channel configuration, we define the operator ${\mathbf{OUT}}^{ctr}:{\Theta }_c\times \Pi \to {\Theta }_c$ as: for all ${\theta }_c\in {\Theta }_c$ and all $\pi \in \Pi$,

  $$\begin{array}{c}\hfill {\mathbf{OUT}}^{ctr}({\theta }_c,\pi )=\left\{\begin{array}{cc}{\theta }_c\backslash ({\pi }_1,m_1)\hfill & \mathrm{if}{\theta }_c=({\pi }_1,m_1)\cdots ({\pi }_h,m_h)\ne \epsilon \hfill \\ \hfill & \wedge \pi ={\pi }_1\hfill \\ \mathrm{undefined}\hfill & \mathrm{otherwise}.\hfill \end{array}\right.\end{array}$$

  (5)

When a new event occurs, for recording the control delays, $\mathbf{PLUS}({\theta }_c)$ adds 1 to all of the natural numbers in ${\theta }_c$. When a new control command is issued (following a new observation), ${\mathbf{IN}}^{ctr}({\theta }_c,\pi )$ adds the newly issued control command $\pi$ to the end of ${\theta }_c$. Moreover, when a new control command is executed, ${\mathbf{OUT}}^{ctr}({\theta }_c,\pi )$ removes the first control command $\pi$ from ${\theta }_c$.

3.2. Language of the Closed-Loop System

We next show how to specify the language that may be generated by the controlled system subject to observation delays and control delays. Specifying an accurate language requires the information of the control command that takes effect in the interval between two successive observable event communications. In the standard supervisory control framework without communication delays, the control command taking effect is exactly the one that was most-recently issued, which can be uniquely determined by the sequence that has been observed so far. However, in the presence of communication delays, the control commands taking effect (between two successive observable event communications) are non-deterministic.

To obtain the exact language of the closed-loop system, we construct an automaton that dynamically tracks the state of the plant, the current control command, the observation channel configuration, the control channel configuration, and the state of the supervisor. This model essentially captures the interaction process between the supervisor and the plant over the observation channel and the control channel. Before we formally construct the automaton, let us introduce two special types of events.

• To keep track of what has been successfully communicated so far, we define the bijection $f:{\Sigma }_o\to {\Sigma }_f$, such that ${\Sigma }_f=\{f(\sigma ):\sigma \in {\Sigma }_o\}$ is a set disjointed from $\Sigma$. For all $\sigma \in {\Sigma }_o$, we use $f(\sigma )$ to denote that the occurrence of $\sigma$ was communicated. Define $f^{-1}$ as, for all $f(\sigma )\in {\Sigma }_f$, $f^{-1}(f(\sigma ))=\sigma$. We extend f to a set of sequences, as $f(\epsilon )=\epsilon$ and, for all $s={\sigma }_1{\sigma }_2\cdots {\sigma }_k\in {\Sigma }_o^{*}$, $f(s)=f({\sigma }_1)f({\sigma }_2)\cdots f({\sigma }_k)\in {\Sigma }_f^{*}$. We also extend $f^{-1}$ to a set of sequences, as $f^{-1}(\epsilon )=\epsilon$ and, for all $f(s)=f({\sigma }_1)f({\sigma }_2)\cdots f({\sigma }_k)\in {\Sigma }_f^{*}$, $f^{-1}(f(s))={\sigma }_1{\sigma }_2\cdots {\sigma }_k\in {\Sigma }_o^{*}$.
• To model which control action is taken, we define bijection $g:\Pi \to {\Sigma }_g$, such that ${\Sigma }_g=\{g(\pi ):\pi \in \Pi \}$ is disjointed from $\Sigma \cup {\Sigma }_f$. For all $\pi \in \Pi$, we use $g(\pi )$ to denote that the control command $\pi$ was executed. Define $g^{-1}$ as, for all $g(\pi )\in {\Sigma }_g$, $g^{-1}(g(\pi ))=\pi$. We extend g to a set of sequences, as $g(\epsilon )=\epsilon$ and, for all $\mu ={\pi }_1{\pi }_2\cdots {\pi }_k\in {\Pi }^{*}$, $g(\mu )=g({\pi }_1)g({\pi }_2)\cdots g({\pi }_k)\in {\Sigma }_g^{*}$. We also extend $g^{-1}$ to a set of sequences, as $g^{-1}(\epsilon )=\epsilon$ and, for all $g(\mu )=g({\pi }_1)g({\pi }_2)\cdots g({\pi }_k)\in {\Sigma }_g^{*}$, $g^{-1}(g(\mu ))={\pi }_1{\pi }_2\cdots {\pi }_k\in {\Pi }^{*}$.

We show in Figure 3 how the plant interacts with the supervisor over the observation channel and the control channel. When a new observable event $\sigma \in {\Sigma }_o$ occurs in the plant, it is immediately added to the end of the observation channel. Since the observation delays are upper-bounded by $N_o$ event occurrences, the maximum observation delays after the occurrence of $\sigma$ should be no larger than $N_o$, i.e., $n_1+1\le N_o$. Similarly, since the control delays are upper-bounded by $N_c$ event occurrences, the maximum control delays after the occurrence of $\sigma$ should be no larger than $N_c$, i.e., $m_1+1\le N_c$. By FIFO, the first event delayed at the observation channel, i.e., ${\sigma }_1$ can be communicated to the supervisor. If ${\sigma }_1$ is communicated to the supervisor, we need to remove $({\sigma }_1,n_1)$ from the head of the observation channel. Meanwhile, following the observation of ${\sigma }_1$, a new control command $\chi (\xi (x,{\sigma }_1))$ is made and is added to the end of the control channel. Moreover, by FIFO, the control commands are executed in the same order that they are issued. Thus, ${\pi }_2,\dots ,{\pi }_h$ cannot be executed until ${\pi }_1$ is executed. If ${\pi }_1$ is executed, we need to remove $({\pi }_1,m_1)$ from the head of ${\theta }_c$.

Notations: Given a ${\theta }_o\in {\Theta }_o$, if ${\theta }_o=({\sigma }_1,n_1)\cdots ({\sigma }_k,n_k)\ne \epsilon$, let $\mathbf{MAX}({\theta }_o)=n_1$ be the maximum delay occurring in the observation channel, and if ${\theta }_o=\epsilon$, let $\mathbf{MAX}({\theta }_o)=0$. Similarly, given a ${\theta }_c\in {\Theta }_c$, if ${\theta }_c=({\pi }_1,m_1)\cdots ({\pi }_h,m_h)\ne \epsilon$, let $\mathbf{MAX}({\theta }_c)=m_1$ be the maximum delay occurring in the control channel, and if ${\theta }_c=\epsilon$, let $\mathbf{MAX}({\theta }_c)=0$.

Given a supervisor $S=(A,\chi )$ with $A=(X,{\Sigma }_o,\xi ,x_0)$, we formally construct $G_S=(Q_S,{\Sigma }_S,{\delta }_S,q_{0,S})$, where $Q_S\subseteq Q\times \Pi \times {\Theta }_o\times {\Theta }_c\times X$ is the state space; $q_{0,S}=(q_0,S(\epsilon ),\epsilon ,\epsilon ,x_0)$ is the initial state, where $S(\epsilon )$ is the initial control command (since the initial control command can be immediately executed when the plant starts to work, we assume that the initial control command takes effect at first); ${\Sigma }_S\subseteq \Sigma \cup {\Sigma }_f\cup {\Sigma }_g$ is the event set; the transition function ${\delta }_S:Q_S\times {\Sigma }_S\to Q_S$ is defined as:

• For all $\tilde{q}=(q,\pi ,{\theta }_o,{\theta }_c,x)\in Q_S$ and all $\sigma \in \Sigma$,

  $$\begin{array}{c}\hfill {\delta }_S(\tilde{q},\sigma )=\left\{\begin{array}{cc}{\tilde{q}}^{\prime }\hfill & \mathrm{if}\delta (q,\sigma )!\wedge \sigma \in \pi \wedge \mathbf{MAX}({\theta }_o^{+})\le N_o\wedge \mathbf{MAX}({\theta }_c^{+})\le N_c\hfill \\ \mathrm{undefined}\hfill & \mathrm{otherwise},\hfill \end{array}\right.\end{array}$$

  (6)

  where ${\tilde{q}}^{\prime }=(\delta (q,\sigma ),\pi ,{\mathbf{IN}}^{obs}({\theta }_o,\sigma ),\mathbf{PLUS}({\theta }_c),x)$;
• For all $\tilde{q}=(q,\pi ,{\theta }_o,{\theta }_c,x)\in Q_S$ and all $f(\sigma )\in {\Sigma }_f$,

  $$\begin{array}{c}\hfill {\delta }_S(\tilde{q},f(\sigma ))=\left\{\begin{array}{cc}{\tilde{q}}^{\prime }\hfill & \mathrm{if}{\mathbf{OUT}}^{obs}({\theta }_o,\sigma )!\hfill \\ \mathrm{undefined}\hfill & \mathrm{otherwise},\hfill \end{array}\right.\end{array}$$

  (7)

  where ${\tilde{q}}^{\prime }=(q,\pi ,{\mathbf{OUT}}^{obs}({\theta }_o,\sigma ),{\mathbf{IN}}^{ctr}({\theta }_c,\chi (\xi (x,\sigma ))),\xi (x,\sigma ))$;
• For all $\tilde{q}=(q,\pi ,{\theta }_o,{\theta }_c,x)\in Q_S$ and all $g(\gamma )\in {\Sigma }_g$,

  $$\begin{array}{c}\hfill {\delta }_S(\tilde{q},g(\gamma ))=\left\{\begin{array}{cc}{\tilde{q}}^{\prime }\hfill & \mathrm{if}{\mathbf{OUT}}^{ctr}({\theta }_c,\gamma )!\hfill \\ \mathrm{undefined}\hfill & \mathrm{otherwise},\hfill \end{array}\right.\end{array}$$

  (8)

  where ${\tilde{q}}^{\prime }=(q,\gamma ,{\theta }_o,{\mathbf{OUT}}^{ctr}({\theta }_c,\gamma ),x)$.

Equation (6): for any $(q,\pi ,{\theta }_o,{\theta }_c,x)\in Q_S$, an event $\sigma \in \Sigma$ can occur at q only if (i) $\sigma$ is active at q, i.e., $\delta (q,\sigma )!$; (ii) $\sigma$ is allowed to occur by the control command in use, i.e., $\sigma \in \pi$; (iii) after the occurrence of $\sigma$, the control delays and the observation delays are no larger than $N_c$ and $N_o$, respectively, i.e., $\mathbf{MAX}({\theta }_c^{+})\le N_c\wedge \mathbf{MAX}({\theta }_o^{+})\le N_o$. When $\sigma$ occurs at q, to track the plant state, we set $q\leftarrow \delta (q,\sigma )$. Meanwhile, to update the observation channel configuration and the control channel configuration, by Equations (1) and (3), we set ${\theta }_o\leftarrow {\mathbf{IN}}^{obs}({\theta }_o,\sigma )$ and ${\theta }_c\leftarrow \mathbf{PLUS}({\theta }_c)$. Since no new control command is executed, we keep $\pi$ unchanged.

Equation (7): for any $(q,\pi ,{\theta }_o,{\theta }_c,x)\in Q_S$, if a new observable event $\sigma$ is communicated (denoted by $f(\sigma )\in {\Sigma }_f$), by FIFO, $\sigma$ must be the first event queued at the observation channel, i.e., ${\mathbf{OUT}}^{obs}({\theta }_o,\sigma )!$. When $\sigma$ is communicated, by Equation (2), we set ${\theta }_o\leftarrow {\mathbf{OUT}}^{obs}({\theta }_o,\sigma )$. Meanwhile, upon the communication of $\sigma$, the supervisor moves to state $\xi (x,\sigma )$ and sends a new control command $\chi (\xi (x,\sigma ))$ to the actuator of the plant. Correspondingly, we set $x\leftarrow \xi (x,\sigma )$ and ${\theta }_c\leftarrow {\mathbf{IN}}^{ctr}({\theta }_c,\chi (\xi (x,\sigma )))$.

Equation (8): for any $(q,\pi ,{\theta }_o,{\theta }_c,x)\in Q_S$, if a new control command $\gamma \in \Pi$ is executed (denoted by $g(\gamma )\in {\Sigma }_g$), by FIFO, it must be the first control command queued at the control channel, i.e., ${\mathbf{OUT}}^{ctr}({\theta }_c,\gamma )!$. When $\gamma$ is executed, by FIFO, the control command that takes effect becomes $\gamma$, and the control commands delayed at the control channel become ${\pi }_2\cdots {\pi }_h$. Correspondingly, we have $\pi \leftarrow \gamma$ and ${\theta }_c\leftarrow {\mathbf{OUT}}^{ctr}({\theta }_c,\gamma )$.

Remark 1. 

By the construction, $G_S$ satisfies all of the assumptions made in this paper. Specifically, by Equation (6), we know the maximum delay occurring in the observation channel is no larger than $N_o$, and the maximum delay occurring in the control channel is no larger than $N_c$. By Equations (7) and (8), the delayed observable events are communicated to the supervisor in the same order that they are generated, and the delayed control commands are delivered to the plant in the same order that they are issued, i.e., both the observation channel and the control channel satisfy the FIFO property. Moreover, by Equations (7) and (8), the observation delays and control delays are non-deterministic. That is, an observable event can be communicated in any one of the following $N_o$ steps from the occurrence, and a control command can be executed in any one of the following $N_c$ steps from when it is issued.

Remark 2. 

In some control applications, there may exist communication losses between the plant and the supervisor. For example, some observable transitions may be lost when they are communicated to the supervisor. Let us denote the set of transitions of G by ${\delta }_G=\{(q,\sigma ,q^{\prime }):\delta (q,\sigma )=q^{\prime }\}$. We also denote the set of observable transitions of G by ${\delta }_{G,o}=\{(q,\sigma ,q^{\prime }):\delta (q,\sigma )=q^{\prime }\wedge \sigma \in {\Sigma }_o\}$. We partition ${\delta }_{G,o}$ into ${\delta }_{G,L}$ and ${\delta }_{G,o}\backslash {\delta }_{G,L}$, where ${\delta }_L$ is the set of transitions whose corresponding event occurrences are either observed without losses or observed with losses. To model possible observation losses, we can first refine the structure of G by adding parallel ε-transitions to the transitions that may be lost in ${\delta }_L$ and obtain $G^{\prime }=(Q,\Sigma \cup \left\{\epsilon \right\},{\delta }^{\prime },q_0)$, where ${\delta }^{\prime }=\delta \cup \{(q,\epsilon ,q^{\prime }):(q,\sigma ,q^{\prime })\in {\delta }_L\}$. Using techniques developed in this section, we can construct $G_S^{\prime }$. Note that when constructing $G_S^{\prime }$, the supervisor does not need to make any control decisions following the communication of ε ($f(\epsilon )$ occurs). Although the occurrence of ε cannot be sensed by the supervisor, all of the natural numbers in ${\theta }_c$ and ${\theta }_o$ should be added by 1 when ε occurs (as an event occurred but was lost). In this paper, we focus on dealing with the nondeterministic observation delays and control delays existing between the supervisor and the plant. The formal approaches for implementing supervisory control under communication delays and losses are beyond the scope of this paper, yet a fruitful area for future exploration.

We use the following example to further illustrate how to construct $G_S$.

Example 2. 

Consider the system G depicted in Figure 2a with $\Sigma =\{\alpha ,\beta ,\eta \}$, ${\Sigma }_o=\{\alpha ,\beta \}$, and ${\Sigma }_c=\Sigma$. Let $N_{o,1}=N_{c,1}=1$. The supervisor $S=(A,\chi )$ is depicted in Figure 2b. We now construct $G_S$ using G and S.

The initial state of $G_S$ is ${\tilde{q}}^0=(q^0,{\pi }^0,{\theta }_o^0,{\theta }_c^0,x^0)=(0,{\pi }_0,\epsilon ,\epsilon ,x_0)$. By Figure 2a, we have $\delta (q^0,\alpha )=1$. Moreover, since $\alpha \in {\pi }^0$, $\mathbf{MAX}({({\theta }_o^0)}^{+})=0\le N_o$, and $\mathbf{MAX}({({\theta }_c^0)}^{+})=0\le N_c$, by Equation (6), we have ${\delta }_S({\tilde{q}}^0,\alpha )={\tilde{q}}^1=(q^1,{\pi }^1,{\theta }_o^1,{\theta }_c^1,x^1)$, where $q^1=\delta (q^0,\alpha )$, ${\pi }^1={\pi }^0$, ${\theta }_o^1={\mathbf{IN}}^{obs}({\theta }_o^0,\alpha )$, ${\theta }_c^1=\mathbf{PLUS}({\theta }_c^0)$, and $x^1=x^0$. Since $\alpha \in {\Sigma }_o$, by Equation (1), ${\mathbf{IN}}^{obs}({\theta }_o^0,\alpha )=(\alpha ,0)$. By Equation (3), $\mathbf{PLUS}({\theta }_c^0)=\epsilon$. Therefore, ${\tilde{q}}^1=(q^1,{\pi }^1,{\theta }_o^1,{\theta }_c^1,x^1)=(1,{\pi }_0,(\alpha ,0),\epsilon ,x_0)$.

Next, consider state ${\tilde{q}}^1$. Since ${\theta }_o^1=(\alpha ,0)$, by Equation (2), ${\mathbf{OUT}}^{obs}({\theta }_o^1,\alpha )=\epsilon$. By Equation (7), ${\delta }_S({\tilde{q}}^1,f(\sigma ))=(q^2,{\pi }^2,{\theta }_o^2,{\theta }_c^2,x^2),$ where $q^2=q^1$, ${\pi }^2={\pi }^1$, ${\theta }_o^2={\mathbf{OUT}}^{obs}({\theta }_o^1,\sigma )$, ${\theta }_c^2={\mathbf{IN}}^{ctr}({\theta }_c^1,\chi (\xi (x^1,\sigma )))$, and $x^2=\xi (x^1,\alpha )$. By Figure 2b, $\xi (x^1,\alpha )=\xi (x_0,\alpha )=x_1$ and $\chi (\xi (x^1,\alpha ))=\chi (x_1)={\pi }_1$. By Equation (4), ${\mathbf{IN}}^{ctr}({\theta }_c^1,\chi (\xi (x^1,\alpha )))=({\pi }_1,0)$. Therefore, ${\delta }_S({\tilde{q}}^1,f(\sigma ))=(1,{\pi }_0,\epsilon ,({\pi }_1,0),x_1)$.

In this way, we can define all of the transitions. Finally, the complete $G_S$ is constructed as shown in Figure 4.

For all $\mu \in \mathcal{L}(G_S)$, let $\psi (\mu )$ and ${\psi }^f(\mu )$ be the sequences obtained by removing all the event occurrences in ${\Sigma }_f\cup {\Sigma }_g$ and $\Sigma \cup {\Sigma }_g$, respectively, without changing the order of the remaining event occurrences in $\mu$. For example, consider $\mu =\alpha f(\alpha )g({\pi }_1)\beta \in \mathcal{L}(G_S)$ in Figure 4. By the definitions of $\psi (·)$ and ${\psi }^f(·)$, we have $\psi (\mu )=\alpha \beta$ and ${\psi }^f(\mu )=f(\alpha )$. We extend $\psi$ and ${\psi }^f$ to a set of sequences in the usual way. Intuitively, for all $\mu \in \mathcal{L}(G_S)$, $\psi (\mu )$ tracks the sequence that occurred in the plant, and $f^{-1}({\psi }^f(\mu ))$ tracks what the supervisor observed after the occurrence of $\psi (\mu )$. The following proposition formally proves this.

Proposition 1. 

Given an arbitrary $\mu \in \mathcal{L}(G_S)$, we write ${\delta }_S(q_{0,S},\mu )=(q,\pi ,{\theta }_o,{\theta }_c,x)$. Then, (i) $q=\delta (q_0,\psi (\mu ))$ and (ii) $x=\xi (x_0,f^{-1}({\psi }^f(\mu )))$.

Proof. 

Please see Appendix A.    □

By Proposition 1, the dynamics of the closed-loop system can be simply obtained by removing all of the events in ${\Sigma }_f\cup {\Sigma }_g$ from sequences generated by $G_S$, which yields the following definition.

Definition 3. 

Given a system G and a supervisor S, we construct $G_S$ as described above. All possible strings that may be generated by the closed-loop system when the observation delays and control delays are upper-bounded by $N_o$ and $N_c$, respectively, are defined as $\mathcal{L}(S/G)=\psi (\mathcal{L}(G_S))$.

Remark 3. 

By Definition 3 and Figure 4, $\alpha \beta \alpha$ is not included in $\mathcal{L}(S/G)$. As we already discussed in Example 1, $\alpha \beta \alpha$ never occurs in practice. This example justifies the advantage of the proposed modeling framework.

Given two supervisors $S_1$ and $S_2$, we say that $S_1$ is smaller than $S_2$, denoted by $S_1\subseteq S_2$, if for all $t\in {\Sigma }_o^{*}$, $S_1(t)\subseteq S_2(t)$, and we say that $S_1$ is strictly smaller than $S_2$ if $S_1\subseteq S_2$, and there exists $t\in {\Sigma }_o^{*}$, such that $S_1(t)\subset S_2(t)$. The following proposition states that the more events a supervisor S enables, the larger the language $\mathcal{L}(S/G)$ the closed-loop system generates.

Proposition 2. 

Given two $S_i=(A_i,{\chi }_i)$ with $A_i=(X_i,{\Sigma }_o,{\xi }_i,x_{0,i})$, $i\in \{1,2\}$, we construct $G_{S_i}=(Q_{S_i},{\Sigma }_{S_i},{\delta }_{S_i},q_{0,S_i})$ as described above. Then, if $S_1\subseteq S_2$, we have $\mathcal{L}(S_1/G)\subseteq \mathcal{L}(S_2/G)$.

Proof. 

The proof is provided in Appendix B.    □

By Proposition 2, to synthesize a supervisor, such that the closed-loop language is maximal and safe, we only need to synthesize a supervisor’s maximal supervisor, such that the closed-loop system behaviors are safe. We next formally formulate the optimal supervisor synthesis problem.

3.3. Problem Formulation

Before we formally present the problem to be solved, let us first introduce the following notation. Given a supervisor S, for a prefix-closed language $L\subseteq {\Sigma }_o^{*}$, ${S|}_L$ means that S is restricted to a smaller domain L, defined as ${S|}_L(t)=S(t)$, if $t\in L$, and undefined, otherwise. Assuming that the supervisor observes $t\in {\Sigma }_o^{*}$, our goal is to compute a maximal and safe supervisor on the fly, for each $t^i\in \overline{t}$, $i=0,1,\dots ,\left|t\right|$.

Problem 1. 

Assuming that the system G executes an arbitrarily long sequence $s\in \mathcal{L}(G)$ and the current observation for s is $t\in {\Sigma }_o^{*}$, we find a supervisor S, such that:

• S is safe, i.e.,  $\mathcal{L}(S/G)\subseteq K$;
• ${S|}_{\overline{t}}$ is maximal, i.e.,  there is no other $S^{\prime }$ that satisfies (1) with $S^{\prime }{{|}_{\overline{t}}\supset S|}_{\overline{t}}$.

Remark 4. 

Since we focus on an online supervisor synthesis, we only need to ensure that the control decisions that make up the current instant are optimal. That is why S is only required to be optimal on $\overline{t}$ (instead of the whole ${\Sigma }_o^{*}$).

Remark 5. 

The solutions to Problem 1 need not be unique. Actually, there may exist several incomparable maximal solutions. In this paper, we emphasize how to online synthesize a “greedily maximal” supervisor, rather than ambitiously calculate all possible solutions.

4. State Estimation under Communication Delays

To make a control decision right after each new observation, the supervisor needs to estimate the states of the closed-loop system (subject to observation delays and control delays) on the fly. We focus on the problem of online networked state estimation in this section. Let us first introduce the definition of the networked state estimate (NSE) as follows.

Definition 4. 

Given a DES G and a supervisor S, we construct $G_S$ as described in Section 3.2. For any $t\in f^{-1}({\psi }^f(\mathcal{L}(G_S)))$, define

$$\begin{array}{cc}\hfill {\mathcal{E}}_S(t)=& \{q\in Q:(\exists \mu \in \mathcal{L}(G_S))q=\delta (q_0,\psi (\mu ))\wedge t=f^{-1}({\psi }^f(\mu ))\},\hfill \end{array}$$

(9)

as the NSE of t under S, which is the set of all the possible states that system G may be in after observing t (subject to observation delays and control delays) under S.

If S is given beforehand, we can calculate ${\mathcal{E}}_S(t)$ by constructing an observer of $G_S$ with the set of observable events ${\Sigma }_f$. However, we focus on online network supervisory control in this paper. That is, we need to calculate the state estimate right after each new communication (all future controls are unknown). To this end, besides the plant state $q\in Q$, we also need to estimate the current control command $\pi \in \Pi$, the observation channel configuration ${\theta }_o\in {\Theta }_o$, and the control channel configuration ${\theta }_c\in {\Theta }_c$, because all of them can affect the behaviors of the closed-loop system. Therefore, we denote each “state” of the closed-loop system by a four-tuple $(q,\pi ,{\theta }_o,{\theta }_c)\in Q\times \Pi \times {\Theta }_o\times {\Theta }_c$. We call such a state an augmented state. Let $\tilde{Q}=\{(q,\pi ,{\theta }_o,{\theta }_c):q\in Q\wedge \pi \in \Pi \wedge {\theta }_o\in {\Theta }_o\wedge {\theta }_c\in {\Theta }_c\}$ be the set of all the augmented states. We next show that we can estimate all possible states that the plant may be in by estimating all possible augmented states that the controlled system may reach.

To precisely estimate the augmented states, we need the following two operators.

Let $Z\in 2^{\tilde{Q}}$ be a set of augmented states calculated immediately after a new observation or the initial $Z=\varnothing$ (since the plant does not work until it is initialized, we let $Z=\varnothing$ before the initial control command is executed).The delayed unobservable reach of Z under an admissible control command $\gamma \in \Pi$, denoted by $\mathrm{DUR}(Z,\gamma )$, is defined as follows.

• If $Z=\varnothing$, then

  $$\begin{array}{c}\hfill (q_0,\gamma ,\epsilon ,\epsilon )\in \mathrm{DUR}(Z,\gamma ),\end{array}$$

  (10)

  and if $Z\ne \varnothing$, then for all $(q,\pi ,{\theta }_o,{\theta }_c)$,

  $$\begin{array}{c}\hfill (q,\pi ,{\theta }_o,{\mathbf{IN}}^{ctr}({\theta }_c,\gamma ))\in \mathrm{DUR}(Z,\gamma );\end{array}$$

  (11)
• Then, we repeatedly apply the following operations until convergence is achieved.
  • For all $(q,\pi ,{\theta }_o,{\theta }_c)\in \mathrm{DUR}(Z,\gamma )$ and all $\sigma \in \Sigma$, if $\delta (q,\sigma )!$ and $\sigma \in \pi$ and $\mathbf{MAX}({\theta }_o^{+})\le N_o$ and $\mathbf{MAX}({\theta }_c^{+})\le N_c$, then

    $$\begin{array}{c}\hfill (\delta (q,\sigma ),\pi ,{\mathbf{IN}}^{obs}({\theta }_o,\sigma ),\mathbf{PLUS}({\theta }_c))\in \mathrm{DUR}(Z,\gamma );\end{array}$$

    (12)
  • For all $(q,\pi ,{\theta }_o,{\theta }_c)\in \mathrm{DUR}(Z,\gamma )$ and all ${\gamma }^{\prime }\in \Pi$, if ${\mathbf{OUT}}^{ctr}({\theta }_c,{\gamma }^{\prime })!$, then

    $$\begin{array}{c}\hfill (q,{\gamma }^{\prime },{\theta }_o,{\mathbf{OUT}}^{ctr}({\theta }_c,{\gamma }^{\prime }))\in \mathrm{DUR}(Z,\gamma ).\end{array}$$

    (13)

If $Z=\varnothing$, no control commands have been executed. That is, $\gamma$ is the initial control command. By assumption, $\gamma$ can be executed without any delays. Hence, by Equation (10), we have $(q_0,\gamma ,\epsilon ,\epsilon )\in \mathrm{DUR}(Z,\gamma )$. Otherwise, if $Z\ne \varnothing$, $\gamma$ is not the initial control command. By FIFO, it will not be executed until all of the control commands that are now delayed at the control channel are executed. Thus, for all $(q,\pi ,{\theta }_o,{\theta }_c)\in Z$, Equation (11) adds $\gamma$ to the end of ${\theta }_c$, i.e., ${\theta }_c\leftarrow {\mathbf{IN}}^{ctr}({\theta }_c,\gamma )$. Meanwhile, Equations (12) and (13) consider the cases of “an event (observable or not) occurs” and “a control command is executed”, respectively. When there exist observation delays and control delays, only “an observable event is communicated” is observable. Therefore, $\mathrm{DUR}(Z,\gamma )$ consists of all the augmented states that may be reached from Z in an “unobservable” way.

Let $Z\in 2^{\tilde{Q}}$ be the current set of augmented states. The delayed observable reach of Z under an observable event $\sigma \in {\Sigma }_o$, denoted by $\mathrm{DOR}(Z,\sigma )$, is defined as:

$$\begin{array}{cc}\hfill \mathrm{DOR}(Z,\sigma )=& \{(q,\pi ,{\mathbf{OUT}}^{obs}({\theta }_o,\sigma ),{\theta }_c):(\exists (q,\pi ,{\theta }_o,{\theta }_c)\in Z){\mathbf{OUT}}^{obs}({\theta }_o,\sigma )!\}.\hfill \end{array}$$

(14)

Intuitively, $\mathrm{DOR}(Z,\sigma )$ includes all of the augmented states that can be reached from Z upon a new communication of $\sigma$. By FIFO, an observable event can be communicated only if it is the first event queued at the observation channel. Hence, we consider all the $\sigma \in {\Sigma }_o$, such that there exists $(q,\pi ,{\theta }_o,{\theta }_c)\in Z$ with ${\mathbf{OUT}}^{obs}({\theta }_o,\sigma )!$. When $\sigma$ is communicated, we remove $({\sigma }_1,n_1)$ from ${\theta }_o$. As we can see, we set ${\theta }_o\leftarrow {\mathbf{OUT}}^{obs}({\theta }_o,\sigma )$. We assume that $\mathrm{DOR}(Z,\sigma )$ is updated right after a new observation of $\sigma$ but before the next control command is issued. Therefore, we keep ${\theta }_c$ unchanged.

We next present how to online estimate augmented states.

Definition 5. 

Let G be a DES and S be a supervisor. We construct $G_S$ as described in Section 3.2. For a $t\in f^{-1}({\psi }^f(\mathcal{L}(G_S)))$, let ${\tilde{\mathcal{E}}}_S(t)$ be the augmented state estimate for t. ${\tilde{\mathcal{E}}}_S(t)$ is calculated by alternatively applying $DUR(·)$ and $\mathrm{DOR}(·)$ as follows.

• Initially, ${\tilde{\mathcal{E}}}_S(\epsilon )=DUR(\varnothing ,S(\epsilon ))$;
• For all $t^i,t^i\sigma \in \overline{t}$, $i=0,1,\dots ,\left|t\right|-1$,

  $${\tilde{\mathcal{E}}}_S(t^i\sigma )=\mathrm{DUR}(\mathrm{DOR}({\tilde{\mathcal{E}}}_S(t^i),\sigma ),S(t^i\sigma )).$$

Remark 6. 

${\tilde{\mathcal{E}}}_S(t^i\sigma )$ indeed online estimates the augmented states. As shown in Figure 5, the online procedure for estimating augmented states can be briefly summarized as repeatedly executing (i) an observable event occurrence $\sigma \in {\Sigma }_o$ (after $t^i$) is communicated to the supervisor, and the set of augmented states is updated to $Z^{\prime }=\mathrm{DOR}({\tilde{\mathcal{E}}}_S(t^i),\sigma )$; (ii) following the observation of σ, a new control command $\pi =S(t^i\sigma )\in \Pi$ is issued by the supervisor S. Then, the corresponding augmented state estimate is updated to ${\tilde{\mathcal{E}}}_S(t^i\sigma )=Z=DUR(Z^{\prime },\pi )$.

We next show that ${\tilde{\mathcal{E}}}_S(t)$ indeed estimates the plant state, the current control command, the observation channel configuration, and the control channel configuration. Let us first define

$$\begin{array}{cc}\hfill \mathcal{T}(t)=& \{(q,\pi ,{\theta }_o,{\theta }_c)\in \tilde{Q}:(\exists \mu \in \mathcal{L}(G_S))f^{-1}({\psi }^f(\mu ))=t\wedge \hfill \\ \hfill & {\delta }_S(q_{0,S},\mu )=(p,\gamma ,{\omega }_o,{\omega }_c,x)\wedge p=q\wedge \gamma =\pi \wedge {\omega }_o={\theta }_o\wedge {\omega }_c={\theta }_c\}.\hfill \end{array}$$

The following lemma will be used later.

Lemma 1. 

For any $t\in f^{-1}({\psi }^f(\mathcal{L}(G_S)))$ if $z=(q,\pi ,{\theta }_o,{\theta }_c)\in \mathcal{T}(t)$ and $z^{\prime }$ is the augmented state calculated by applying Equation (12) or (13) on z, then $z^{\prime }\in \mathcal{T}(t)$.

Proof. 

Without a loss of generality, we write $z=(q,\pi ,{\theta }_o,{\theta }_c)$ and $z^{\prime }=(q^{\prime },{\pi }^{\prime },{\theta }_o^{\prime },{\theta }_c^{\prime })$. Since $z\in \mathcal{T}(t)$, by the definition of $\mathcal{T}(·)$, there exists a $\mu \in \mathcal{L}(G_S)$, such that $f^{-1}({\psi }^f(\mu ))=t$ and ${\delta }_S(q_{0,S},\mu )=(p,\gamma ,{\omega }_o,{\omega }_c,x)$ with $p=q$, $\gamma =\pi$, ${\omega }_o={\theta }_o$, and ${\omega }_c={\theta }_c$. Since $z^{\prime }$ is the augmented state obtained by applying one of the operations in Equations (12)∼(13) on z, one of the following two cases must be true.

Case 1: $z^{\prime }=(\delta (q,\sigma ),\pi ,{\mathbf{IN}}^{obs}({\theta }_o,\sigma ),\mathbf{PLUS}({\theta }_c))$. By Equation (12), $\delta (q,\sigma )!$, $\sigma \in \pi$, $\mathbf{MAX}({\theta }_o^{+})\le N_o$, and $\mathbf{MAX}({\theta }_c^{+})\le N_c$. Since ${\delta }_S(q_{0,S},\mu )=(p,\gamma ,{\omega }_o,{\omega }_c,x)$, by Equation (6), ${\delta }_S(q_{0,S},\mu \sigma )=(\delta (q,\sigma ),\pi ,{\mathbf{IN}}^{obs}({\theta }_o,\sigma ),\mathbf{PLUS}({\theta }_c),x)$. Thus, $z^{\prime }\in \mathcal{T}(f^{-1}({\psi }^f(\mu \sigma )))=\mathcal{T}(t)$.

Case 2: $z^{\prime }=(\delta (q,\sigma ),{\gamma }^{\prime },{\theta }_o,{\mathbf{OUT}}^{ctr}({\theta }_c,{\gamma }^{\prime }))$. By Equation (13), ${\mathbf{OUT}}^{ctr}({\theta }_c,{\gamma }^{\prime })!$. Since ${\delta }_S(q_{0,S},\mu )=(p,\gamma ,{\omega }_o,{\omega }_c,x)$, by Equation (8), ${\delta }_S(q_{0,S},\mu g({\gamma }^{\prime }))=(q,{\gamma }^{\prime },{\theta }_o,{\mathbf{OUT}}^{ctr}({\theta }_c,{\gamma }^{\prime }),x)$. Thus, we have that $z^{\prime }\in \mathcal{T}(f^{-1}({\psi }^f(\mu g({\gamma }^{\prime })))=\mathcal{T}(t)$.    □

Theorem 1. 

Given a DES G and a supervisor S, we construct $G_S=(Q_S,{\Sigma }_S,{\delta }_S,q_{0,S})$ as described in Section 3.2. For any $t\in f^{-1}({\psi }^f(\mathcal{L}(G_S)))$, we have

$$\begin{array}{cc}\hfill {\tilde{\mathcal{E}}}_S(t)=& \{(q,\pi ,{\theta }_o,{\theta }_c)\in \tilde{Q}:(\exists \mu \in \mathcal{L}(G_S))f^{-1}({\psi }^f(\mu ))=t\wedge \hfill \\ \hfill & {\delta }_S(q_{0,S},\mu )=(p,\gamma ,{\omega }_o,{\omega }_c,x)\wedge p=q\wedge \gamma =\pi \wedge {\omega }_o={\theta }_o\wedge {\omega }_c={\theta }_c\}.\hfill \end{array}$$

Proof. 

$(\subseteq )$ We first prove ${\tilde{\mathcal{E}}}_S(t)\subseteq \mathcal{T}(t)$ by contradiction. Suppose there exists $t\in f^{-1}({\psi }^f(\mathcal{L}(G_S)))$, such that ${\tilde{\mathcal{E}}}_S(t)\neg \subseteq \mathcal{T}(t)$. Without loss of generality (w.l.o.g.), we assume that t is the shortest sequence in $f^{-1}({\psi }^f(\mathcal{L}(G_S)))$, satisfying ${\tilde{\mathcal{E}}}_S(t)\neg \subseteq \mathcal{T}(t)$. We now show $t\ne \epsilon$. By Definition 5, for any $z\in {\tilde{\mathcal{E}}}_S(\epsilon )$, there exists a sequence of augmented states $z_0{z}_1\cdots z_k$, such that $z_0=(q_0,S(\epsilon ),\epsilon ,\epsilon )$, $z_k=z$, and $z_{i+1}$ is the augmented state calculated by applying Equation (12) or (13) on $z_i$, $i=0,1,\dots ,k-1$. Since ${\delta }_S(q_{0,S},\epsilon )=(q_0,S(\epsilon ),\epsilon ,\epsilon ,x_0)$, $z_0\in \mathcal{T}(\epsilon )$. By repeatedly applying Lemma 1, $z_1,\dots ,z_k\in \mathcal{T}(\epsilon )$. Therefore, ${\tilde{\mathcal{E}}}_S(\epsilon )\subseteq \mathcal{T}(\epsilon )$.

Since $t\ne \epsilon$, we write $t=t^{\prime }\sigma$ for some $\sigma \in {\Sigma }_o$. Since ${\tilde{\mathcal{E}}}_S(t)\neg \subseteq \mathcal{T}(t)$, $\exists z\in {\tilde{\mathcal{E}}}_S(t)$ such that $z\notin \mathcal{T}(t)$. Since $z\in {\tilde{\mathcal{E}}}_S(t)$, by Definition 5, there exists a sequence of augmented states $z_0{z}_1\cdots z_k$ with $z_0=(q,\pi ,{\mathbf{OUT}}^{obs}({\theta }_o,\sigma ),{\mathbf{IN}}^{ctr}({\theta }_c,S(t^{\prime }\sigma )))$ for some $(q,\pi ,{\theta }_c,{\theta }_o)\in {\tilde{\mathcal{E}}}_S(t^{\prime })$, $z_k=z$, and $z_{i+1}$ is the augmented state calculated by applying Equation (12) or (13) on $z_i$, $i=0,1,\dots ,k-1$. Next, we prove $z_0\in \mathcal{T}(t^{\prime }\sigma )$.

Since $(q,\pi ,{\theta }_c,{\theta }_o)\in {\tilde{\mathcal{E}}}_S(t^{\prime })\subseteq \mathcal{T}(t^{\prime })$, $\exists \mu \in \mathcal{L}(G_S)$, such that $f^{-1}({\psi }^f(\mu ))=t^{\prime }$, ${\delta }_S(q_{0,S},\mu )=(p,\gamma ,{\omega }_o,{\omega }_c,x)$, and $p=q\wedge \gamma =\pi \wedge {\omega }_o={\theta }_o\wedge {\omega }_c={\theta }_c$. Since ${\mathbf{OUT}}^{obs}({\theta }_o,\sigma )!$ and ${\theta }_o={\omega }_o$, we have ${\mathbf{OUT}}^{obs}({\omega }_o,\sigma )!$. By Equation (7),

$${\delta }_S(q_{0,S},\mu f(\sigma ))=(p,\gamma ,{\mathbf{OUT}}^{obs}({\omega }_o,\sigma ),{\mathbf{IN}}^{ctr}({\omega }_c,\chi (\xi (x,\sigma ))),\xi (x,\sigma )).$$

Since $f^{-1}({\psi }^f(\mu ))=t^{\prime }$, we have $f^{-1}({\psi }^f(\mu f(\sigma )))=t^{\prime }\sigma$. By Proposition 1, we have $\xi (x,\sigma )=\xi (x_0,f^{-1}({\psi }^f(\mu f(\sigma ))))=\xi (x_0,t^{\prime }\sigma )$. By definition, we have $\chi (\xi (x,\sigma ))=S(t^{\prime }\sigma )$. Thus,

$${\delta }_S(q_{0,S},\mu f(\sigma ))=(p,\gamma ,{\mathbf{OUT}}^{obs}({\omega }_o,\sigma ),{\mathbf{IN}}^{ctr}({\omega }_c,S(t^{\prime }\sigma )),\xi (x,\sigma )).$$

By the definition of $\mathcal{T}(·)$,

$$(p,\gamma ,{\mathbf{OUT}}^{obs}({\omega }_o,\sigma ),{\mathbf{IN}}^{ctr}({\omega }_c,S(t^{\prime }\sigma )))\in \mathcal{T}(f^{-1}({\psi }^f(\mu f(\sigma )))=\mathcal{T}(t^{\prime }\sigma ).$$

Since $p=q\wedge \gamma =\pi \wedge {\omega }_o={\theta }_o\wedge {\omega }_c={\theta }_c$, $(q,\pi ,{\mathbf{OUT}}^{obs}({\theta }_o,\sigma ),{\mathbf{IN}}^{ctr}({\theta }_c,S(t^{\prime }\sigma )))\in \mathcal{T}(t^{\prime }\sigma )$. Hence, $z_0\in \mathcal{T}(t^{\prime }\sigma )$. By repeatedly applying Lemma 1, $z_1,\dots ,z_k\in \mathcal{T}(t^{\prime }\sigma )$, which contradicts $z=z_k\notin \mathcal{T}(t^{\prime }\sigma )$.

$(\supseteq )$ We next prove ${\tilde{\mathcal{E}}}_S(t)\supseteq \mathcal{T}(t)$. To prove ${\tilde{\mathcal{E}}}_S(t)\supseteq \mathcal{T}(t)$, we only need to prove that for all $\mu \in \mathcal{L}(G_S)$, if ${\delta }_S(q_{0,S},\mu )=(q,\pi ,{\theta }_o,{\theta }_c,x)$, then $(q,\pi ,{\theta }_o,{\theta }_c)\in {\tilde{\mathcal{E}}}_S(f^{-1}({\psi }^f(\mu )))$. The proof is by induction on the finite length of sequences in $\mathcal{L}(G_S)$.

Since ${\delta }_S(q_{0,S},\epsilon )=(q_0,S(\epsilon ),\epsilon ,\epsilon ,x_0)$ and $(q_0,S(\epsilon ),\epsilon ,\epsilon )\in {\tilde{\mathcal{E}}}_S(\epsilon )$, the base case is true. The induction hypothesis is that for all $\mu \in \mathcal{L}(G_S)$ with $\left|\mu \right|\le k$, we write ${\delta }_S(q_{0,S},\mu )=(q,\pi ,{\theta }_o,{\theta }_c,x)$. Then, $(q,\pi ,{\theta }_o,{\theta }_c)\in {\tilde{\mathcal{E}}}_S(f^{-1}({\psi }^f(\mu )))$.

We next prove the same is also true for $\mu e\in \mathcal{L}(G_S)$ with $\left|\mu \right|=k$. We write ${\delta }_S(q_{0,S},\mu e)=(p,\gamma ,{\omega }_o,{\omega }_c,y)$. Then, ${\delta }_S((q,\pi ,{\theta }_o,{\theta }_c,x),e)=(p,\gamma ,{\omega }_o,{\omega }_c,y)$. Since $e\in {\Sigma }_S$, $e\in \Sigma$, $e\in {\Sigma }_f$, or $e\in {\Sigma }_g$. We consider each of them separately as follows.

Case 1: $e=\sigma \in \Sigma$. Since ${\delta }_S((q,\pi ,{\theta }_o,{\theta }_c,x),\sigma )=(p,\gamma ,{\omega }_o,{\omega }_c,y)$, by Equation (6), we have

• $\delta (q,\sigma )!$, $\sigma \in \pi$, $\mathbf{MAX}({\theta }_o^{+})\le N_o$, and $\mathbf{MAX}({\theta }_c^{+})\le N_c$;
• $p=q$, $\gamma =\pi$, ${\omega }_o={\mathbf{IN}}^{obs}({\theta }_o,\sigma )$, and ${\omega }_c=\mathbf{PLUS}({\theta }_c)$.

Since $(q,\pi ,{\theta }_o,{\theta }_c)\in {\tilde{\mathcal{E}}}_S(f^{-1}({\psi }^f(\mu )))$ and Condition 1 in Case 1, by Equation (12),

$$(\delta (q,\sigma ),\pi ,{\mathbf{IN}}^{obs}({\theta }_o,\sigma ),\mathbf{PLUS}({\theta }_c))\in {\tilde{\mathcal{E}}}_S(f^{-1}({\psi }^f(\mu ))).$$

Since $\sigma \in \Sigma$, we have $f^{-1}({\psi }^f(\mu \sigma ))=f^{-1}({\psi }^f(\mu ))$. By Condition 2 in Case 1, $(p,\gamma ,{\omega }_o,{\omega }_c)\in {\tilde{\mathcal{E}}}_S(f^{-1}({\psi }^f(\mu \sigma )))$.

Case 2: $e=f(\sigma )\in {\Sigma }_f$. For brevity, we write $t=f^{-1}({\psi }^f(\mu ))$. Then, $f^{-1}({\psi }^f(\mu f(\sigma )))=t\sigma$. Since ${\delta }_S((q,\pi ,{\theta }_o,{\theta }_c,x),f(\sigma ))=(p,\gamma ,{\omega }_o,{\omega }_c,y)$, by Equation (7),

• ${\mathbf{OUT}}^{obs}({\theta }_o,\sigma )!$;
• $p=q$, $\gamma =\pi$, ${\omega }_o={\mathbf{OUT}}^{obs}({\theta }_o,\sigma )$, ${\omega }_c={\mathbf{IN}}^{ctr}({\theta }_c,\chi (y))$, and $y=\xi (x,\sigma )$.

Since ${\delta }_S(q_{0,S},\mu f(\sigma ))=(p,\gamma ,{\omega }_o,{\omega }_c,y)$, by Proposition 1, $y=\xi (x_0,t\sigma )$. Thus, $\chi (y)=S(t\sigma )$. By the induction hypothesis, $(q,\pi ,{\theta }_o,{\theta }_c)\in {\tilde{\mathcal{E}}}_S(t)$. Since ${\mathbf{OUT}}^{obs}({\theta }_o,\sigma )!$, by Equation (14), $(q,\pi ,{\mathbf{OUT}}^{obs}({\theta }_o,\sigma ),{\theta }_c)\in DOR({\tilde{\mathcal{E}}}_S(t),\sigma )$. Moreover, since $\chi (y)=S(t\sigma )$, by Equation (12),

$$(q,\pi ,{\mathbf{OUT}}^{obs}({\theta }_o,\sigma ),{\mathbf{IN}}^{ctr}({\theta }_c,\chi (y)))\in \mathrm{DUR}(\mathrm{DOR}({\tilde{\mathcal{E}}}_S(t),\sigma ),S(t\sigma )).$$

By Definition 5, we have $(q,\pi ,{\mathbf{OUT}}^{obs}({\theta }_o,\sigma ),{\mathbf{IN}}^{ctr}({\theta }_c,\chi (y)))\in {\tilde{\mathcal{E}}}_S(t\sigma )$. Thus, by Condition 2 in Case 2, $(p,\gamma ,{\omega }_o,{\omega }_c)\in {\tilde{\mathcal{E}}}_S(t\sigma )={\tilde{\mathcal{E}}}_S(f^{-1}({\psi }^f(\mu f(\sigma )))).$

Case 3: $e=g({\gamma }^{\prime })\in {\Sigma }_g$. Since $g({\gamma }^{\prime })\in {\Sigma }_g$, $f^{-1}({\psi }^f(\mu ))=f^{-1}({\psi }^f(\mu g({\gamma }^{\prime })))$. Since ${\delta }_S((q,\pi ,{\theta }_o,{\theta }_c,x),\sigma )=(p,\gamma ,{\omega }_o,{\omega }_c,y)$, by Equation (8), we have

• ${\mathbf{OUT}}^{ctr}({\theta }_c,{\gamma }^{\prime })!$;
• $p=q$, $\gamma ={\gamma }^{\prime }$, ${\omega }_o={\theta }_o$, and ${\omega }_c={\mathbf{OUT}}^{ctr}({\theta }_c,{\gamma }^{\prime })$.

Moreover, since $(q,\pi ,{\theta }_o,{\theta }_c)\in {\tilde{\mathcal{E}}}_S(f^{-1}({\psi }^f(\mu )))$ and ${\mathbf{OUT}}^{ctr}({\theta }_c,{\gamma }^{\prime })!$, by Equation (13), we have that $(q,{\gamma }^{\prime },{\theta }_o,{\mathbf{OUT}}^{ctr}({\theta }_c,{\gamma }^{\prime }))\in {\tilde{\mathcal{E}}}_S(f^{-1}({\psi }^f(\mu )))={\tilde{\mathcal{E}}}_S(f^{-1}({\psi }^f(\mu g({\gamma }^{\prime })))).$ By Condition 2 in Case 3, $(p,\gamma ,{\omega }_o,{\omega }_c)\in {\tilde{\mathcal{E}}}_S(f^{-1}({\psi }^f(\mu g({\gamma }^{\prime })))).$    □

Let $z=(q,\pi ,{\theta }_o,{\theta }_c)\in \tilde{Q}$ be a given augmented state. We denote $\mathrm{FC}(z)=q$ by the first component (the plant state) of z. (“FC” means “first component”). We extend $\mathrm{FC}(·)$ to a set of augmented states $Z\in 2^{\tilde{Q}}$ as follows: $\mathrm{FC}(Z)=\{\mathrm{FC}(z):\forall z\in Z\}$. The following corollary discusses the relationship between ${\mathcal{E}}_S(t)$ and ${\tilde{\mathcal{E}}}_S(t)$.

Corollary 1. 

Let G be a DES and S be a supervisor. We construct $G_S$ as described in Section 3.2. For any $t\in f^{-1}({\psi }^f(\mathcal{L}(G_S)))$, ${\mathcal{E}}_S(t)=\mathrm{FC}({\tilde{\mathcal{E}}}_S(t))$.

Proof. 

The proof directly follows from Theorem 1 and Definition 4.    □

By Corollary 1, we can estimate the plant states by taking the first component of the estimated augmented states. We use the following example to further illustrate our online state estimation procedure.

Example 3. 

Consider again the system G depicted in Figure 2a and the supervisor S depicted in Figure 2b. Let ${\Sigma }_o=\{\alpha ,\beta \}$, ${\Sigma }_c=\{\alpha ,\beta ,\eta \}$, and $N_o=N_c=1$. We now compute ${\tilde{\mathcal{E}}}_S(\epsilon )$, ${\mathcal{E}}_S(\epsilon )$ and ${\tilde{\mathcal{E}}}_S(\alpha )$, ${\mathcal{E}}_S(\alpha )$.

Initially, by Equation (10), $(0,{\pi }_0,\epsilon ,\epsilon )\in {\tilde{\mathcal{E}}}_S(\epsilon )$. Since $\delta (0,\alpha )=1$, $\alpha \in {\pi }_0$, and $\mathit{MAX}({\epsilon }^{+})=0\le N_c,N_o$, by Equation (12), $(1,{\pi }_0,(\alpha ,0),\epsilon )\in {\tilde{\mathcal{E}}}_S(\epsilon )$. Then, since $\delta (1,\eta )=2$, $\eta \in {\pi }_0$, $\mathit{MAX}({(\alpha ,0)}^{+})=1\le N_c$, and  $\mathit{MAX}({\epsilon }^{+})=0\le N_c$, also by Equation (12), $(2,{\pi }_0,(\alpha ,1),\epsilon )\in {\tilde{\mathcal{E}}}_S(\epsilon )$. Therefore,

$$\begin{array}{cc}\hfill {\tilde{\mathcal{E}}}_S(\epsilon )=& \{(0,{\pi }_0,\epsilon ,\epsilon ),(1,{\pi }_0,(\alpha ,0),\epsilon ),(2,{\pi }_0,(\alpha ,1),\epsilon )\}.\hfill \end{array}$$

By Corollary 1, ${\mathcal{E}}_S(\epsilon )=\{0,1,2\}$. Since ${\mathit{OUT}}^{obs}((\alpha ,0),\alpha )={\mathit{OUT}}^{obs}((\alpha ,1),\alpha )=\epsilon$, by Equation (14), $\mathrm{DOR}({\tilde{\mathcal{E}}}_S(\epsilon ),\sigma )=\{(1,{\pi }_0,\epsilon ,\epsilon ),(2,{\pi }_0,\epsilon ,\epsilon )\}.$ By Definition 5, ${\tilde{\mathcal{E}}}_S(\alpha )=\mathrm{DUR}(\mathrm{DOR}({\tilde{\mathcal{E}}}_S(\epsilon ),$

$\sigma ),S(\alpha ))$. Since $S(\alpha )$ is not the initial control command, by Equation (11),

$$(1,{\pi }_0,\epsilon ,({\pi }_1,0)),(2,{\pi }_0,\epsilon ,({\pi }_1,0))\in \mathrm{DUR}(\mathrm{DOR}({\tilde{\mathcal{E}}}_S(\epsilon ),\sigma ),S(\alpha )).$$

Then, by Equations (12) and (13), we have

$$\begin{array}{cc}\hfill {\tilde{\mathcal{E}}}_S(\alpha )=& \{(1,{\pi }_0,\epsilon ,({\pi }_1,0)),(2,{\pi }_0,\epsilon ,({\pi }_1,0)],(2,{\pi }_0,\epsilon ,({\pi }_1,1))\hfill \\ \hfill & (2,{\pi }_1,\epsilon ,\epsilon ),(1,{\pi }_1,\epsilon ,\epsilon ),(3,{\pi }_1,(\beta ,0),\epsilon ),(4,{\pi }_1,(\beta ,1)(\beta ,0),\epsilon )\}.\hfill \end{array}$$

By Corollary 1, ${\mathcal{E}}_S(\alpha )=\{1,2,3,4\}$.

5. Online Network Supervisory Control

In this section, we calculate a maximal and safe control on the fly based on the state estimation techniques developed in Section 4.

5.1. State Prediction

To determine if the control decision made at the moment is safe, we need to predict all states that we cannot prevent from reaching under observation delays and control delays. To this end, for a $z=(q,\pi ,{\theta }_c,{\theta }_o)\in \tilde{Q}$ appeared in an augmented state estimate, we construct an automaton $G_z$ to check what states the plant may reach from q, if we disable all controllable events in the future. The basic idea for the construction of $G_z$ is similar to that of $G_S$. That is, starting from z, $G_z$ dynamically tracks the plant state, the current control command, the observation channel configuration, and the control channel configuration, given that all future controls are ${\Sigma }_{uc}$.

Formally, we construct $G_z=(Q_z,{\Sigma }_z,{\delta }_z,z)$, where $Q_z\subseteq Q\times \Pi \times {\Theta }_o\times {\Theta }_c$ is the state space; $z=(q,\pi ,{\theta }_c,{\theta }_o)$ is the initial state; ${\Sigma }_z\subseteq \Sigma \cup {\Sigma }_f\cup {\Sigma }_g$ is the event set; the transition function ${\delta }_z:Q_z\times {\Sigma }_z\to Q_z$ is defined as:

• For all $z=(q,\pi ,{\theta }_o,{\theta }_c)\in Q_z$ and all $\sigma \in \Sigma$,

  $$\begin{array}{c}\hfill {\delta }_z(z,\sigma )=\left\{\begin{array}{cc}{z}^{\prime }\hfill & \mathrm{if}\delta (q,\sigma )!\wedge \sigma \in \pi \wedge \mathbf{MAX}({\theta }_o^{+})\le N_o\wedge \mathbf{MAX}({\theta }_c^{+})\le N_c\hfill \\ \mathrm{undefined}\hfill & \mathrm{otherwise},\hfill \end{array}\right.\end{array}$$

  (15)

  where $z^{\prime }=(\delta (q,\sigma ),\pi ,{\mathbf{IN}}^{obs}({\theta }_o,\sigma ),\mathbf{PLUS}({\theta }_c))$;
• For all $z=(q,\pi ,{\theta }_o,{\theta }_c)\in \tilde{Q}$ and all $f(\sigma )\in {\Sigma }_f$,

  $$\begin{array}{c}\hfill {\delta }_z(\tilde{q},f(\sigma ))=\left\{\begin{array}{cc}{z}^{\prime }\hfill & \mathrm{if}{\mathbf{OUT}}^{obs}({\theta }_o,\sigma )!\hfill \\ \mathrm{undefined}\hfill & \mathrm{otherwise},\hfill \end{array}\right.\end{array}$$

  (16)

  where $z^{\prime }=(q,\pi ,{\mathbf{OUT}}^{obs}({\theta }_o,\sigma ),{\mathbf{IN}}^{ctr}({\theta }_c,{\Sigma }_{uc}))$;
• For all $z=(q,\pi ,{\theta }_o,{\theta }_c)\in \tilde{Q}$ and all $g(\gamma )\in {\Sigma }_g$,

  $$\begin{array}{c}\hfill {\delta }_z(z,g(\gamma ))=\left\{\begin{array}{cc}{z}^{\prime }\hfill & \mathrm{if}{\mathbf{OUT}}^{ctr}({\theta }_c,\gamma )!\hfill \\ \mathrm{undefined}\hfill & \mathrm{otherwise},\hfill \end{array}\right.\end{array}$$

  (17)

  where $z^{\prime }=(q,\gamma ,{\theta }_o,{\mathbf{OUT}}^{ctr}({\theta }_c,\gamma ))$.

Since we assume all the controllable events are disabled in the future when a new observable event is communicated, we adopt the control command ${\Sigma }_{uc}$. As shown in Equation (16), we set ${\theta }_c\leftarrow \mathbf{IN}({\theta }_c,{\Sigma }_{uc})$ after the communication of $\sigma$.

The following proposition states that for any $z=(q,\pi ,{\theta }_c,{\theta }_o)\in \tilde{Q}$ and any $\nu \in \mathcal{L}(G_z)$, we cannot disable the occurrence of $\psi (\nu )$ from q even if we disable all of the controllable events in the future.

Proposition 3. 

Let G be a DES and S a supervisor. For any $\mu \in \mathcal{L}(G_S)$, we write ${\delta }_S(q_{0,S},\mu )=(q,\pi ,{\theta }_o,{\theta }_c,x)$. Let $z=(q,\pi ,{\theta }_o,{\theta }_c)$ and $G_z$ be the automaton constructed as described above. Then, if $\nu \in \mathcal{L}(G_z)$, $\psi (\mu \nu )\in \mathcal{L}(S/G)$.

Proof. 

Please see Appendix C.    □

Given a $z=(q,\pi ,{\theta }_o,{\theta }_c)\in \tilde{Q}$, all of the plant states that we cannot prevent from reaching q via some $\psi (\nu )\in \psi (\mathcal{L}(G_z))$ can be obtained by taking the first component of $Q_z$. That is,

$$\begin{array}{c}\hfill \mathrm{FC}(Q_z)=\{\delta (q,\psi (\nu )):\forall \nu \in \mathcal{L}(G_z)\}.\end{array}$$

(18)

We can prove Equation (18) by inducing the finite length of sequences in $\mathcal{L}(G_z)$, which is similar to the proof of Proposition 1, and is omitted here for brevity.

5.2. Online Algorithm

Suppose that the current observation of the system is $t\in {\Sigma }_o^{*}$. When a new event is observed, the supervisor S makes a new control command $\pi \in \Pi$, and the augmented state estimate will be updated to ${\tilde{\mathcal{E}}}_S(t\sigma )=\mathrm{DUR}(\mathrm{DOR}({\tilde{\mathcal{E}}}_S(t),\sigma ),\pi )$. As discussed in Section 5.1, for any $z=(q,\pi ,{\theta }_c,{\theta }_o)\in {\tilde{\mathcal{E}}}_S(t\sigma )$, $\mathrm{FC}(Q_z)$ collects all of the plant states that may be reached from q no matter what control commands we adopt in the future. Therefore, we define the set of “bad” augmented states as:

$$\begin{array}{c}\hfill {\mathcal{T}}_{spec}=\{z\in \tilde{Q}:\mathrm{FC}(Q_z)\cap (Q\backslash Q_H)\ne \varnothing \}.\end{array}$$

(19)

For safety, all the augmented states in ${\mathcal{T}}_{spec}$ should never be reached. To make the problem non-trivial, we assume that the controlled system is safe if we choose to disable all of the controllable events after each new observation.

With the above preparations, we are now ready to introduce our online algorithm. We first pre-compute ${\mathcal{T}}_{spec}$ offline. The networked supervisory control for G is implemented on the fly in Algorithm 1 as follows: when the supervisor receives a new observable event occurrence $\sigma$, Line 9 is executed with the new communication of $\sigma$. The set of events to be enabled following the communication of $\sigma$ is then calculated by the for-loop on Line 3, where all the controllable events are checked one by one to see if they can be enabled while the system cannot reach some “bad” augmented states. The above processes are repeated when another observable event occurrence is communicated.

Algorithm 1:Online maximal networked control

Definition 6. 

For any $t\in {\Sigma }_o^{*}$, the online network supervisor $S_t$ is defined as: for all $t^i\in \overline{t}$, $i=0,1,\dots ,\left|t\right|$, $S_t(t^i)$ is the set of events that is enabled right after the communication of $t^i$, and for all $t^{\prime }\in {\Sigma }_o^{*}$ with $t^{\prime }\notin \overline{t}$, $S_t(t^{\prime })={\Sigma }_{uc}$.

Note that $S_t$ can be represented as an automaton with a finite state space.

Remark 7. 

The maximal networked supervisors are not unique. Given a different order ${\Sigma }_c=\{{\sigma }_1,\dots ,{\sigma }_k\}$ on the controllable events, Algorithm 1 may return different results. However, the order of controllable events can be changed dynamically after each new communication, if desired. However, all of the possible supervisors returned by Algorithm 1 are safe and maximal.

Remark 8. 

Algorithm 1 tries to enable a maximum allowable set of controllable events at any instant to ensure the closed-loop system is within the desired specification language. However, as discussed in Remark 7, there may exist several incomparable maximum control decisions after each new observation. In many applications, enabling a controllable event could involve financial and human costs. In such situations, it is preferable to select a maximum allowable set of controllable events with the minimum enablement of costs at each instant. A simple approach is to consider all maximum control commands and select one with the minimum enablement cost. Another approach is to list all of the controllable events in ascending order according to their enablement costs: ${\Sigma }_c=\{{\sigma }_1,\dots ,{\sigma }_k\}$, where ${\sigma }_1$ is a controllable event that is the least costly to enable, and ${\sigma }_k$ is the event that is the most costly to enable. By the for-loop on Line 3, a controllable event with a smaller enablement cost has a priority to be considered. The first approach is optimal but needs more computational resources than the second approach. The second approach may be suboptimal but is more efficient compared with the first approach.

Remark 9. 

For a given $z\in \tilde{Q}$, we know the number of states in $G_z$ is upper-bounded by $|\tilde{Q}|$. Since the number of verifiers to be constructed is $|\tilde{Q}|$, the computational complexity for calculating ${\mathcal{T}}_{spec}$ is the order of $\mathcal{O}(|\tilde{Q}{|}^2)$. By definition, $\tilde{Q}\subseteq Q\times \Pi \times {\Theta }_o\times {\Theta }_c$, Since $\Pi \subseteq 2^{\Sigma }$, ${\Theta }_o\subseteq {({\Sigma }_o\times [0,N_o])}^{\le N}$, and ${\Theta }_c\subseteq {(\Pi \times [0,N_c])}^{\le M}$, the complexity for calculating ${\mathcal{T}}_{spec}$ is polynomial with respect to (w.r.t.) $\left|Q\right|$ and exponential w.r.t. $|\Sigma |$.

After each new communication, for each $\sigma \in {\Sigma }_c$, we need to test whether or not σ can be enabled. In each test, $\mathrm{DUR}(\Xi ,\pi )$ is updated by Line 5 and we need to search the state space $\tilde{Q}$ once. Therefore, the computational complexity of Algorithm 1 is the stepwise order of $\mathcal{O}(|\tilde{Q}|)$, which is also polynomial w.r.t. $\left|Q\right|$ and exponential w.r.t. $|\Sigma |$.

Next, we show that the control commands made at each step in Algorithm 1 guarantee that the controlled system is safe.

Theorem 2. 

Suppose the current observation of the system is $t\in {\Sigma }_o^{*}$. Let ${\tilde{\mathcal{E}}}_{S_t}(t^i)$ be the augmented state estimate for $t^i\in \overline{t}$ under $S_t$, $i=1,\dots ,\left|t\right|$. Then,

$$(\forall i=1,\dots ,|t|){\tilde{\mathcal{E}}}_{S_t}(t^i)\cap {\mathcal{T}}_{spec}=\varnothing \iff \mathcal{L}(S_t/G)\subseteq K.$$

Proof. 

Please see Appendix D. □

The following corollary states that $S_t$ is the solution to Problem 1.

Corollary 2. 

Suppose the current observation of the system is $t\in {\Sigma }_o^{*}$. The online supervisor $S_t$ derived by Algorithm 1 satisfies conditions 1 and 2 of Problem 1.

Proof. 

The proof directly follows from Problem 1 and Theorem 2. □

5.3. Comparison with the Existing Work

In this section, we compare the proposed algorithm with the algorithm proposed in [26]. Similar to [26], we assume that there are only control delays with an upper bound of $N_c$, and there are no observation delays, i.e., $N_o=0$ in this section. To make this paper self-contained, we first review the state estimation techniques proposed in [26].

A channel configuration is defined in [26] as a set of pairs in the form of $\theta =\{({\pi }_1,n_1),({\pi }_2,n_2),\dots ,({\pi }_k,n_k)\},$ where ${\pi }_i\in \Pi$ is an admissible control action that is delayed at the control channel, and $n_i\in [0,N_c]$ is a nonnegative integer indicating that the control action ${\pi }_i$ is still effective for the next $n_i$ steps. We denote by $\Gamma (\theta )$ the union of all the control actions in $\theta$, i.e., $\Gamma (\theta )={\cup }_{i=1,\dots ,k}{\pi }_i$. We also denote by $\Theta \subseteq 2^{\Pi \times [0,N_c]}$ the set of all channel configurations. To update a $\theta \in \Theta$ after a new event occurrence, we define the “next” operator $NX:\Theta \to \Theta$ as follows: for any $\theta \in \Theta$,

$$\mathrm{NX}(\theta )=\{(\pi ,n-1)\in \Pi \times \mathbb{N}:(\pi ,n)\in \theta ,n\ge 1\}.$$

$\mathrm{NX}(\theta )$ decreases the timing index of each element of $\theta$ by one unit and only keeps the elements of $\theta$ with nonnegative natural numbers. Thus, $\theta$ collects all the control actions issued in the past $N_c$ steps (including the current step).

We define an extended state as a pair of a plant state $q\in Q$ and a channel configuration $\theta \in \Theta$. Let $\widehat{Q}=Q\times \Theta$ be the set of all extended states. Let $Z\in 2^{\widehat{Q}}$ be a set of extended states and $\pi \in \Pi$ be a control action. Then, the networked unobservable reach of Z under $\pi$, denoted by $\mathrm{NU}{\mathrm{R}}_{\pi }(Z)$, is defined recursively as follows:

• For any $(q,\theta )\in Z$, we have

  $$\begin{array}{c}\hfill (q,\theta \cup \left\{(\pi ,N_c)\right\})\in \mathrm{NU}{\mathrm{R}}_{\pi }(Z);\end{array}$$

  (20)
• For any $(q,\theta )\in \mathrm{NU}{\mathrm{R}}_{\pi }(Z)$ and any unobservable event $\sigma \in {\Sigma }_{uo}$, if $\sigma \in \Gamma (\theta )$ and $\delta (q,\sigma )!$, then

  $$\begin{array}{c}\hfill (\delta (q,\sigma ),\mathrm{NX}(\theta )\cup \left\{(\pi ,N_c)\right\})\in \mathrm{NU}{\mathrm{R}}_{\pi }(Z).\end{array}$$

  (21)

Operation Equation (20) is used to add the latest control action $\pi$ into the channel configuration. Operation Equation (21) computes all the extended states that can be reached from any $(q,\theta )\in \mathrm{NU}{\mathrm{R}}_{\pi }(Z)$ via an unobservable event occurrence. In Equation (17), an event $\sigma$ can occur at an extended state $(q,\theta )$ if it is active at state q, i.e., $\delta (q,\sigma )!$, and it is allowed to occur by one of the control actions issued in the past $N_c$ steps, i.e., $\sigma \in \Gamma (\theta )$.

Let $Z\in 2^{\widehat{Q}}$ be a set of extended states and $\sigma \in {\Sigma }_o$ be an observable event. The networked observable reach ($\mathrm{NOR}$) of Z upon the occurrence of $\sigma$, denoted by $\mathrm{NO}{\mathrm{R}}_{\sigma }(Z)$, is defined as:

$$\begin{array}{c}\hfill \mathrm{N}\mathrm{O}{\mathrm{R}}_{\sigma }(Z)=\{(\delta (q,\sigma ),\mathrm{N}\mathrm{X}(\theta ))\in \tilde{Q}:(q,\theta )\in x,\sigma \in \Gamma (\theta )\}.\end{array}$$

(22)

Operation Equation (22) collects all of the extended states that can be immediately reached from elements of Z via $\sigma$.

Let S be a given networked supervisor. The set of extended states that the controlled system may be in after a communicated $t\in {\Sigma }_o^{*}$, denoted by ${\widehat{\mathcal{E}}}_S(t)$, can be calculated as follows:

• Initially, ${\widehat{\mathcal{E}}}_S(\epsilon )=\mathrm{N}\mathrm{U}{\mathrm{R}}_{S(\epsilon )}(\left\{(q_0,\varnothing )\right\})$;
• For all $t^i,t^i{\sigma }_{i+1}\in \overline{\left\{t\right\}}$, $i=0,1,\dots ,\left|t\right|-1$,

  $${\widehat{\mathcal{E}}}_S(t^i{\sigma }_{i+1})=\mathrm{N}\mathrm{U}{\mathrm{R}}_{S(t^i{\sigma }_{i+1})}(\mathrm{N}\mathrm{O}{\mathrm{R}}_{{\sigma }_{i+1}}({\widehat{\mathcal{E}}}_S(t^i))).$$

Then, it is shown by Corollary 1 of [26] that the set of plant states that the controlled system may be in after observing t can be simply obtained by taking the first components of ${\widehat{\mathcal{E}}}_S(t)$.

Let $\theta =\{({\gamma }_1,n_1),({\gamma }_2,n_2),\cdots ,({\gamma }_k,n_k)\}$ be a channel configuration and $m\in [0,N_c]$ be a non-negative integer. We denote by ${\Gamma }_{\ge m}(\theta )$ the union of all control decisions that can take effect in the next m steps, i.e.,

$$\begin{array}{c}\hfill {\Gamma }_{\ge m}(\theta )=\bigcup _{i\in [1,k]:n_i\ge m}{\gamma }_i.\end{array}$$

(23)

The uncontrollable language for $\theta$ can be defined as:

$$\begin{array}{c}\hfill L_{uc}(\theta ):=\overline{{\Gamma }_{\ge 0}(\theta ){\Gamma }_{\ge 1}(\theta )\cdots {\Gamma }_{\ge N_c}(\theta ).}\end{array}$$

(24)

Given an extended state $\tilde{q}=(q,\theta )\in \tilde{Q}$, the uncontrollable state prediction of $\tilde{q}$, denoted by $\mathrm{U}\mathrm{S}\mathrm{P}(\tilde{q})$, is defined as

$$\begin{array}{c}\hfill \mathrm{U}\mathrm{S}\mathrm{P}(\tilde{q})=\{\delta (q,s)\in Q:s\in L_{uc}(\theta )\}.\end{array}$$

(25)

The online supervisor synthesis approaches proposed in [26] mainly consist of the following two steps.

Step 1: When an observable event sequence $t\in {\Sigma }_o^{*}$ is communicated, calculate the extended state estimate ${\widehat{\mathcal{E}}}_S(t)$;

Step 2: Find a maximal control decision $\gamma \in \Gamma$, such that $\mathrm{U}\mathrm{S}\mathrm{P}(\mathrm{N}\mathrm{U}{\mathrm{R}}_{\gamma }({\widehat{\mathcal{E}}}_S(t)))\subseteq Q_H$.

Next, we use two examples to show that the proposed supervisor can be more permissive than that proposed in [26].

Example 4. 

Consider the uncontrolled system G and the desired system H depicted in Figure 6a and Figure 6b, respectively. Let ${\Sigma }_c=\Sigma$ and ${\Sigma }_o=\{\alpha ,\beta \}$.

Initially, we start from $\{(0,\varnothing )\}$ and choose a maximal control decision $S(\epsilon )={\gamma }_0$, such that $\mathrm{UPS}(\mathrm{NU}{\mathrm{R}}_{{\gamma }_0}(\left\{(0,\varnothing )\right\})\subseteq Q_H$. One can check that ${\gamma }_0=\{\alpha ,\gamma \}$ is such a maximal control action. Then, we can compute the extended state estimate ${\widehat{\mathcal{E}}}_S(\epsilon )=\mathrm{NU}{\mathrm{R}}_{{\gamma }_0}(\left\{(0,\varnothing )\right\})=\{(0,\left\{({\gamma }_0,2)\right\}),(3,\left\{({\gamma }_0,1)\right\})\}$. If α is observed, we have $\mathrm{NO}{\mathrm{R}}_{\alpha }({\widehat{\mathcal{E}}}_S(\epsilon ))=\left\{(1,\{{\gamma }_0,1\})\right\}$. Then, we again find a maximal control decision $S(\alpha )={\gamma }_1$, such that $\mathrm{UPS}(\mathrm{N}\mathrm{U}{\mathrm{R}}_{{\gamma }_1}(\left\{(1,\{{\gamma }_0,1\})\right\}))\subseteq Q_H$.

By definition, $\tilde{q}=(1,\{({\gamma }_0,1),({\gamma }_1,2)\})\in \mathrm{NU}{\mathrm{R}}_{{\gamma }_1}(\left\{(1,\{{\gamma }_0,1\})\right\})$. One can check that $\beta \notin {\gamma }_1$, because otherwise, by Equation (24), $\beta \gamma \in L_{uc}(\theta )$. Then, by Equation (25), we have $5\in \mathrm{UPS}(\tilde{q})\in Q\backslash Q_H$. Therefore, we have ${\gamma }_1=\{\alpha ,\gamma \}$. Since $\beta \notin {\gamma }_0$ and $\beta \notin {\gamma }_1$, β will never occur at State 1. Therefore, all possible behaviors that may occur under the synthesized supervisor include $\{\epsilon ,\gamma ,\alpha \}$.

In the previous framework, it was assumed that all control actions issued in the past $N_c$ steps may take effect. Thus, in Example 4, since $\gamma \in {\gamma }_0$, ${\gamma }_0$ may take effect after $\alpha \beta$. Therefore, we must disable $\beta$ after $\alpha$ to prevent the system from reaching State 5. Since $\beta \notin {\gamma }_0,{\gamma }_1$, we know that $\beta$ will never occur after $\alpha$. However, as shown in the following example, we can actually enable $\beta$ after observing $\alpha$, and the controlled system can never reach State 5.

Example 5. 

Continue with Example 4. We now show how to apply Algorithm 1 to compute an optimal supervisor.

Algorithm 1 starts from ${\Xi }^0=\varnothing$ and iterates the for-loop on Line 3 for computing a maximal $S(\epsilon )=E_a^0$, such that $\mathrm{D}\mathrm{U}\mathrm{R}(\mathrm{D}\mathrm{O}\mathrm{R}({\Xi }^0,E_a^0))\cap {\mathcal{T}}_{spec}=\varnothing$. One can check that $E_a^0=\{\alpha ,\gamma \}$ is such an optimal control decision. The augmented state estimate is updated to

$$\begin{array}{cc}\hfill {\tilde{\mathcal{E}}}_S(\epsilon )=& \{[0,E_a^0,\epsilon ,\epsilon ],[3,E_a^0,\epsilon ,\epsilon ],[1,E_a^0,(\alpha ,0),\epsilon ]\}.\hfill \end{array}$$

After that, the supervisor observes α and estimates

$${\Xi }^1=\mathrm{DOR}(\tilde{\mathcal{E}}(\epsilon ),\sigma )=\left\{[1,E_a^0,\epsilon ,\epsilon ]\right\}.$$

Then, iterating the for-loop on Line 3 leads to $S(\alpha )=E_a^1=\{\alpha ,\beta \}$. The augmented state estimate is updated to

$$\begin{array}{c}\hfill {\tilde{\mathcal{E}}}_S(\alpha )=\{[1,E_a^0,\epsilon ,(E_a^1,0)],[1,E_a^1,\epsilon ,\epsilon ],[2,E_a^1,(\beta ,0),\epsilon ]\}.\end{array}$$

Then, the supervisor observes β and estimates

$${\Xi }^2=\mathrm{DOR}(\tilde{\mathcal{E}}(\alpha ),\beta )=\left\{[2,E_a^1,\epsilon ,\epsilon ]\right\}.$$

Iterating the for-loop on Line 3, we have $S(\alpha \beta )=E_a^1=\{\alpha ,\beta \}$.

Note that we cannot enable γ after observing $\alpha \beta$. Therefore, under the synthesized supervisor, we may reach States 0, 1, 2, and 3. That is, all the possible behaviors that may be generated by the closed-loop system include $\{\epsilon ,\gamma ,\alpha ,\alpha \beta \}$.

By Examples 4 and 5, the language of the closed-loop system under the supervisor synthesized by Algorithm 1 is larger than the language of the closed-loop system under the supervisor synthesized by [26]. Since the proposed framework excludes all physically impossible strings, the state estimate calculated is more precise than that calculated by the previous approach. Thus, the proposed supervisor is more permissive than the previous one.

6. Application in Traffic Control

We consider a signalized intersection as shown in Figure 7. When a self-driving vehicle x arrives at the intersection, it needs to communicate with the intersection to observe the signal and make a control decision accordingly. The observation and control are realized through a network. Due to network characteristics, observation delays and control delays are unavoidable. We assume in this example the observation delays are upper-bounded by 1 and the control decision are upper-bounded by 1, i.e., $N_o=1$ and $N_c=1$. We define seven events as shown in

Table 1. Events in the transport safety model.

Events	Description	Controllable	Observable
a	Vehicle x arrives at the intersection	Yes	No
p	Vehicle x leaves the intersection	Yes	No
y	The traffic light is switched to yellow	No	Yes
$g_1$	The traffic light is in the first half of the green cycle	No	Yes
$g_2$	The traffic light is in the second half of the green cycle	No	Yes
$r_1$	The traffic light is in the first half of the red cycle	No	Yes
$r_2$	The traffic light is in the second half of the red cycle	No	Yes

.

Event a denotes that Vehicle x arrives at the intersection. Event p denotes that Vehicle x passes through the intersection. Event y denotes that the traffic signal is switched to yellow. The green time in one signal cycle is $t_g$ seconds, and we divide $t_g$ into $g_1$ and $g_2$ equally: $g_1$ denotes the first $t_g/2$ seconds and $g_2$ denotes the remaining $t_g/2$ seconds. Similarly, the red time in one signal cycle is $t_r$ seconds, and we divide $t_r$ into $r_1$ and $r_2$ equally: $r_1$ denotes the first $t_r/2$ seconds and $r_2$ denotes the remaining $t_r/2$ seconds.

Events a and p are controllable since Vehicle x can choose to approach or pass through the intersection. Events $r_1$, $r_2$, $g_1$, $g_2$, and y are observable but are not controllable since Vehicle x can observe but cannot change the color of the traffic light. The system model $G=(Q,\Sigma ,\delta ,q_0)$ for vehicle x is displayed in Figure 8a.

Let us interpret the construction of G in Figure 8a as follows. When Vehicle x arrives at the intersection (a occurs), the system enters State 1. If the signal in the forward direction is switched to red for no more than $t_r/2$ seconds after green, i.e., $r_1$. (respectively, red for more than $t_r/2$ seconds but no more than $t_r$ seconds after green, i.e., $r_2$, green for no more than $t_g/2$ seconds after red, i.e., $g_1$, green for more than $t_g/2$ seconds but no more than $t_g$ seconds after red, i.e., $g_2$, and yellow, i.e., y), the system makes a state transition to State 5 (respectively, States 2, 6, 3, and 4). If the system is uncontrolled, Vehicle x can pass through the intersection at any time. Hence, p can occur in States 2, 3, 4, 5, and 6. Let us suppose that the traffic light is $r_1$ when Vehicle x arrives at the intersection. Thus, the system is in State 5. If Vehicle x chooses to pass through the intersection, then the system moves to State 9. Otherwise, if Vehicle x stops at the intersection, then upon the occurrence of $r_2$, the traffic light enters the second stage of the red cycle, and the system makes a state transition from State 5 to State 2. Then, if Vehicle x chooses to pass through the intersection, the system moves from State 2 to State 9. Otherwise, by the switching rule, the traffic light is further switched to green ($g_1$ occurs), the system makes a state transition from State 2 to State 6, and so on.

By traffic laws, passing the intersection (enabling p) is not permitted when the traffic light is red or yellow when the vehicle approaches the intersection. Therefore, we should disable the occurrence of p at States 2, 4, and 5. On the other hand, we can enable the occurrence of p at States 3 and 6. In particular, when the system is in State 3, the traffic lights may be switched to yellow. Upon the occurrence of y, the system moves to State 7. By the traffic law, enabling p is legal if the traffic light is switched from green to yellow when Vehicle x is passing through the intersection. Thus, we can enable p at State 7. The desired system H is depicted in Figure 8b.

We now apply Algorithm 1 to calculate an optimal control command after each new observation. We denote ${\Xi }^i$ by the set of augmented states returned by Line 9 after the observation of the ith event. We also denote $E_a^i$ by the set of events returned by Line 7 after the observation of the ith event.

Initially, by Lines 2 and 3, we have $\Xi =\varnothing$ and $E_a\leftarrow {\Sigma }_{uc}=\{y,r_1,r_2,g_1,g_2\}$. Let ${\Sigma }_c=\{a,p\}$. By the for-loop on Line 3, we first try to add a into $E_a$ and set $\pi \leftarrow {\Sigma }_{uc}\cup \left\{a\right\}$. By the definition of $\mathrm{DUR}(·)$, one can verify that $\mathrm{DUR}(\Xi ,\pi )\cap {\mathcal{T}}_{spec}\ne \varnothing$ since only the occurrence of p can lead the controlled system to the “illegal” state 9, and p can never occur if we choose to disable p now and in the future. Thus, by Line 6, we have $E_a={\Sigma }_{uc}\cup \left\{a\right\}$. By the for-loop on Line 3, we next try to add p into $E_a$ and set $\pi \leftarrow {\Sigma }_{uc}\cup \{a,p\}$. It can be checked that $z=(5,\pi ,(r_1,0),\epsilon )\in \mathrm{DUR}(\Xi ,\pi )$ and $z\in {\mathcal{T}}_{spec}$ since $\pi$ may take effect after $a{r}_1$, and p is prevented from occurring after $a{r}_1$. Thus, we have ${\Xi }^0=\varnothing$ and $E_a^0={\Sigma }_{uc}\cup \left\{a\right\}$. Let ${\pi }_0\leftarrow E_a^0$. By definition,

$$\begin{array}{cc}\hfill \mathrm{DUR}({\Xi }^0,E_a^0)=& \{(0,{\pi }_0,\epsilon ,\epsilon ),(1,{\pi }_0,\epsilon ,\epsilon ),(2,{\pi }_0,(r_2,0),\epsilon ),(3,{\pi }_0,(g_2,0),\epsilon )\hfill \\ \hfill & (4,{\pi }_0,(y,0),\epsilon ),(5,{\pi }_0,(r_1,0),\epsilon ),(6,{\pi }_0,(g_1,0),\epsilon )\hfill \\ \hfill & (6,{\pi }_0,(r_2,1)(g_1,0),\epsilon ),(7,{\pi }_0,(g_2,1)(y,0),\epsilon ),(5,{\pi }_0,(y,1)(r_1,0),\epsilon )\hfill \\ \hfill & (2,{\pi }_0,(r_1,1)(r_2,0),\epsilon ),(3,{\pi }_0,(g_1,1)(g_2,0),\epsilon )\}.\hfill \end{array}$$

Next, if $g_2$ is communicated, by Line 9,

$$\begin{array}{cc}\hfill {\Xi }^1=\mathrm{DOR}(\mathrm{DUR}({\Xi }^0,E_a^0),g_2)=& \{(3,{\pi }_0,\epsilon ,\epsilon ),(7,{\pi }_0,(y,0),\epsilon )\}.\hfill \end{array}$$

Then, let us go to Line 2, and we have $E_a\leftarrow {\Sigma }_{uc}$. By the for-loop on Line 3, we first try to add a into $E_a$ and set $\pi \leftarrow {\Sigma }_{uc}\cup \left\{a\right\}$. One can verify that $\mathrm{DUR}(\Xi ,\pi )\cap {\mathcal{T}}_{spec}\ne \varnothing$. We next try to add p into $E_a$ and set $\pi \leftarrow {\Sigma }_{uc}\cup \{a,p\}$.

Since $(7,{\pi }_0,(y,0),\epsilon )\in {\Xi }^1$, when $\pi$ is issued, augmented states can occur in the order as follows:

$$\begin{array}{cc}\hfill & (7,{\pi }_0,(y,0),(\pi ,0))\stackrel{g(\pi )}{⟶}(7,\pi ,(y,0),\epsilon )\stackrel{r_1}{⟶}\hfill \\ \hfill & (5,\pi ,(y,1)(r_1,0),\epsilon )\stackrel{f(y)}{⟶}(5,\pi ,(r_1,0),({\pi }^{\prime },0))\stackrel{p}{⟶}(9,\pi ,(r_1,1),({\pi }^{\prime },1)),\hfill \end{array}$$

where ${\pi }^{\prime }$ is the control command made after the communication of $r_1$. As we can see, the control action $\pi$ may take effect at the time p occurs at $(5,\pi ,(r_1,0),({\pi }^{\prime },0))$, which violates the traffic law. Hence, we can only add a into $E_a$. We have $E_a^1\leftarrow {\Sigma }_{uc}\cup \left\{a\right\}$.

The above process is repeated until the vehicle passes through the intersection. The synthesized supervisor is depicted in Figure 9. For brevity, we only list all of the controllable events to be enabled at each state of the supervisor (all of the uncontrollable events are omitted). From Figure 9, to pass through the intersection safely, Vehicle x must stop if the traffic light is $r_1$, $r_2$, y, or $g_1$ when the vehicle arrives at the intersection. Vehicle x can choose to pass through the intersection only when $g_1$ is communicated, i.e., Vehicle x observes the occurrence of $g_1$.

7. Extension of the Proposed Framework

In this section, we briefly discuss how to model a system with non-FIFO observations and controls.

In many control applications, such as cyber–physical systems, the sensors are often distributed at different sites, and the detected information is communicated to the supervisor over different observation channels. Different observation channels may have different upper bounds of observation delays. The nondeterministic observation delays may change the order of events communicated to the supervisor. In other words, the supervisor may receive observable event occurrences in different orders as they occur. On the other hand, the enablement and disablement of controllable events are achieved by single actuators, and all actuators are distributed at different sites. The supervisor sends the control decisions for disabling or enabling events to the corresponding actuators upon each new event observation. Different control channels may have different upper bounds of control delays.

As shown in Figure 10, there are $|{\Sigma }_o|$ sensors, each is associated with an observable event. For brevity, we write ${\Sigma }_o=\{{\sigma }_1,\dots ,{\sigma }_n\}$, where $n=|{\Sigma }_o|$. For each ${\sigma }_i\in {\Sigma }_o$, the occurrence can be detected by sensor i and communicated to the supervisor over observation channel i. We assume that observation delays occurring in the observation channel i are upper-bounded by $N_{o,i}$ event occurrences. That is, when an event ${\sigma }_i$ occurs, it will be communicated to the supervisor before no more than $N_{o,i}$ additional event occurrences. On the other hand, there are $|{\Sigma }_c|$ actuators, and each is associated with a controllable event. We write ${\Sigma }_c=\{e_1,\dots ,e_m\}$, where $m=|{\Sigma }_c|$. For each $e_i\in {\Sigma }_c$, the enablement and disablement are achieved by actuator i, and the control decision for enabling or disabling $e_i$ is sent to the actuator i over the control channel i. We assume that control delays occurring in control channel i are upper-bounded by $N_{c,i}$ event occurrences. That is, the control decision made for an event $e_i$ can be executed before no more than $N_{c,i}$ additional event occurrences.

For each $e_i\in {\Sigma }_c$, the supervisor sends 0 or 1 to actuator i over control channel i, where “0” means “disablement” and “1” means “enablement”. Thus, we denote $\Phi =\{([0,1]\times \left\{e_1\right\})\times \cdots \times ([0,1]\times \left\{e_m\right\})\}$ by the set of all the possible control decisions that the supervisor may make. Correspondingly, we denote, in this section, the supervisor by a pair $D=(W,\omega )$, where $W=(Z,{\Sigma }_o,\zeta ,z_0)$ is a deterministic automaton with $\mathcal{L}(W)={\Sigma }_o^{*}$, and $\zeta :Z\to \Phi$ is a function that specifies control decisions for disabling or enabling $e_1,\dots ,e_m$. Specifically, for any $t\in {\Sigma }_o^{*}$, we denote $\omega (\zeta (z_0,t))$ by control decisions for disabling or enabling $e_1,\dots ,e_m$. With a slight abuse of notation, we write $D(t)=\omega (\zeta (z_0,t))$. For any $t\in {\Sigma }_o^{*}$, we have $D(t)\in \Phi$. For any $e\in {\Sigma }_c$ and any $\overline{\varphi }=[(b_1,e_1),\dots ,(b_m,e_m)]\in \Phi$, we say that e is allowed to be enabled by $\overline{\varphi }$, denoted by $e\in \overline{\varphi }$, if $(\exists i=1,\dots ,m)e=e_i\wedge b_i=1$.

Definition 7. 

The observation channel i configuration is defined as a sequence of pairs:

$${\theta }_{o,i}=({\sigma }_i,n_1)\cdots ({\sigma }_i,n_k),$$

where ${\sigma }_i\cdots {\sigma }_i\in {\Sigma }_o^{*}$ is a sequence of observable events ${\sigma }_i$ that have been detected by sensor i but were currently delayed at the observation channel i, and $n_j\in [0,N_{o,i}]$, $j=1,\dots ,k$ is the number of event occurrences since ${\sigma }_i$ occurred and was detected. If observation channel i is empty, ${\theta }_{o,i}=\epsilon$.

We denote by ${\Theta }_{o,i}\subseteq {(\left\{{\sigma }_i\right\}\times [0,N_{o,i}])}^{\le N_i}$ the set of all possible observation channel i configurations, where $N_i$ is the maximum length of ${\theta }_{o,i}$. Given a ${\theta }_{o,i}=({\sigma }_i,n_1)\cdots ({\sigma }_i,n_k)\in {\Theta }_{o,i}$, let $\mathbb{MAX}({\theta }_{o,i})=n_1$ be the maximum observation delays occurring in the observation channel i. The overall state of the observation channels is defined as a vector ${\overline{\theta }}_o=[{\theta }_{o,1},\dots ,{\theta }_{o,n}]$, where ${\theta }_{o,i}$ is the observation channel i configuration. Let ${\Theta }_o={\Theta }_{o,1}\times \cdots \times {\Theta }_{o,n}$ be the set of all the states of observation channel configurations.

• When a new event $\sigma \in \Sigma$ occurs, to update the state of the observation channel configurations, we define the operator ${\mathbb{IN}}^{obs}:{\Theta }_o\times \Sigma \to {\Theta }_o$ as: for all ${\overline{\theta }}_o=[{\theta }_{o,1},\dots ,{\theta }_{o,n}]\in {\Theta }_o$ and all $\sigma \in \Sigma$,

  $${\mathbb{IN}}^{obs}({\overline{\theta }}_o,\sigma )=[{\theta }_{o,1}^{\prime },\dots ,{\theta }_{o,n}^{\prime }],$$

  such that

  $$\begin{array}{c}\hfill {\theta }_{o,i}^{\prime }=\left\{\begin{array}{cc}{\theta }_{o,i}^{+}(\sigma ,0)\hfill & \mathrm{if}\sigma ={\sigma }_i\wedge \mathbb{MAX}({\theta }_{o,i}^{+})\le N_{o,i}\hfill \\ {\theta }_{o,i}^{+}\hfill & \mathrm{if}\sigma \ne {\sigma }_i\wedge \mathbb{MAX}({\theta }_{o,i}^{+})\le N_{o,i}\hfill \\ \mathrm{undefined}\hfill & \mathrm{otherwise},\hfill \end{array}\right.\end{array}$$

  (26)

  where if ${\theta }_{o,i}=({\sigma }_i,n_1)\cdots ({\sigma }_i,n_k)\ne \epsilon$, ${\theta }_{o,i}^{+}=({\sigma }_i,n_1+1)\cdots ({\sigma }_i,n_k+1)$, and if ${\theta }_{o,i}=\epsilon$, ${\theta }_{o,i}^{+}=\epsilon$.
• When a new observable event $\sigma \in {\Sigma }_o$ is communicated to the supervisor, to update the state of the observation channel configurations, define the operator ${\mathbb{OUT}}^{obs}:{\Theta }_o\times {\Sigma }_o\to {\Theta }_o$ as for all ${\overline{\theta }}_o=[{\theta }_{o,1},\dots ,{\theta }_{o,n}]\in {\Theta }_o$ and all $\sigma \in {\Sigma }_o$,

  $${\mathbb{OUT}}^{obs}({\overline{\theta }}_o,\sigma )=[{\theta }_{o,1}^{\prime },\dots ,{\theta }_{o,n}^{\prime }],$$

  such that

  $$\begin{array}{c}\hfill {\theta }_{o,i}^{\prime }=\left\{\begin{array}{cc}{\theta }_{o,i}\backslash {\theta }_{o,i}^1\hfill & \mathrm{if}\sigma ={\sigma }_i\wedge {\theta }_{o,i}\ne \epsilon \hfill \\ {\theta }_{o,i}\hfill & \mathrm{if}\sigma \ne {\sigma }_i\hfill \\ \mathrm{undefined}\hfill & \mathrm{otherwise},\hfill \end{array}\right.\end{array}$$

  (27)

  where ${\theta }_{o,i}^1$ is the first component of ${\theta }_{o,i}$. That is, for all ${\theta }_{o,i}=({\sigma }_i,n_1)\cdots ({\sigma }_i,n_k)\ne \epsilon$, we have ${\theta }_{o,i}^1=({\sigma }_i,n_1)$.

When an event $\sigma \in \Sigma$ occurs in the plant, all natural numbers in ${\theta }_{o,i}$ should be plus 1 since they are used to counting the observation delays. Furthermore, if $\sigma \in {\Sigma }_{o,i}$, by FIFO, we still need to add $({\sigma }_i,0)$ to the end of ${\theta }_{o,i}$ for recording the new observable event occurrence. On the other hand, when a new observable event $\sigma \in {\Sigma }_o$ is communicated to the supervisor, ${\mathbb{OUT}}^{obs}({\overline{\theta }}_o,\sigma )$ removes $\sigma$ from the head of ${\theta }_{o,i}$ if $\sigma ={\sigma }_i$.

Definition 8. 

The control channel i configuration is defined as a sequence of pairs:

$${\theta }_{c,i}=({\varphi }_1,m_1)\cdots ({\varphi }_h,m_h),$$

where ${\varphi }_1\cdots {\varphi }_h\in {(\{0,1\}\times \left\{e_i\right\})}^{*}$ is a sequence of control decisions made for enabling or disabling $e_i$ but are currently delayed at the control channel i, and $m_j\in [0,N_{c,i}]$, $j=1,\dots ,k$ is the number of event occurrences since ${\pi }_j$ has been issued. If the control channel i is empty, ${\theta }_{c,i}=\epsilon$.

We denote by ${\Theta }_{c,i}\subseteq {((\{0,1\}\times \left\{e_i\right\})\times [0,N_{c,i}])}^{\le M_i}$ the set of all the possible control channel i configurations, where $M_i\in \mathbb{N}$ is the maximum length of ${\theta }_{c,i}\in {\Theta }_{c,i}$. Given a ${\theta }_{c,i}=({\varphi }_1,m_1)\cdots ({\varphi }_h,m_h)\in {\Theta }_{c,i}$, let $\mathbb{MAX}({\theta }_{c,i})=m_1$ be the maximum control delays occurring in the control channel i. The overall state of the control channel is defined as a vector ${\overline{\theta }}_c=[{\theta }_{c,1},\dots ,{\theta }_{c,n}]$, where ${\theta }_{c,i}$ is the control channel i configuration. Let ${\Theta }_c={\Theta }_{c,1}\times \cdots \times {\Theta }_{c,m}$ be the set of all the states of control channel configurations. To update ${\overline{\theta }}_c\in {\Theta }_c$, we introduce the following operators.

• When a new event $\sigma \in \Sigma$ occurs in the plant, to update the state of the control channel configurations, define the operator $\mathbb{PLUS}:{\Theta }_c\to {\Theta }_c$ as: for all ${\overline{\theta }}_c=[{\theta }_{c,1},\dots ,{\theta }_{c,m}]\in {\Theta }_c$,

  $$\mathbb{PLUS}({\overline{\theta }}_c)=[{\theta }_{c,1}^{\prime },\dots ,{\theta }_{c,m}^{\prime }],$$

  such that

  $$\begin{array}{c}\hfill {\theta }_{c,i}^{\prime }=\left\{\begin{array}{cc}{\theta }_{c,i}^{+}\hfill & \mathrm{if}\mathbb{MAX}({\theta }_{c,i}^{+})\le N_{c,i}\hfill \\ \mathrm{undefined}\hfill & \mathrm{otherwise},\hfill \end{array}\right.\end{array}$$

  (28)

  where if ${\theta }_{c,i}=({\varphi }_1,m_1)\cdots ({\varphi }_h,m_h)\ne \epsilon$, ${\theta }_{c,i}^{+}=({\varphi }_1,m_1+1)\cdots ({\varphi }_h,m_h+1)$, and if ${\theta }_{c,i}=\epsilon$, ${\theta }_{c,i}^{+}=\epsilon$.
• When a new control command $\overline{\varphi }=[{\varphi }_1,\dots ,{\varphi }_m]\in \Phi$ is issued by the supervisor, to update the state of the control channel configurations, we define the operator ${\mathbb{IN}}^{ctr}:{\Theta }_c\times \Phi \to {\Theta }_c$ as: for all ${\overline{\theta }}_c=[{\theta }_{c,1},\dots ,{\theta }_{c,m}]\in {\Theta }_c$ and all $\overline{\varphi }=[{\varphi }_1,\dots ,{\varphi }_m]\in \Phi$,

  $${\mathbb{IN}}^{ctr}({\overline{\theta }}_c,\overline{\varphi })=[{\theta }_{c,1}^{\prime },\dots ,{\theta }_{c,m}^{\prime }],$$

  such that ${\theta }_{c,i}^{\prime }={\theta }_{c,i}({\varphi }_i,0)$ for all $i=1,\dots ,m$.
• When a new control command $\varphi \in \{0,1\}\times {\Sigma }_c$ is executed, to update the states of the control channel configurations, define the operator ${\mathbb{OUT}}^{ctr}:{\Theta }_c\times (\{0,1\}\times {\Sigma }_c)\to {\Theta }_c$ as: for all ${\overline{\theta }}_c=[{\theta }_{c,1},\dots ,{\theta }_{c,m}]\in {\Theta }_c$ and all $\varphi =(b,e)\in \{0,1\}\times {\Sigma }_c$,

  $${\mathbb{OUT}}^{ctr}({\overline{\theta }}_c,\varphi )=[{\theta }_{c,1}^{\prime },\dots ,{\theta }_{c,m}^{\prime }],$$

  such that

  $$\begin{array}{c}\hfill {\theta }_{c,i}^{\prime }=\left\{\begin{array}{cc}{\theta }_{c,i}\backslash {\theta }_{c,i}^1\hfill & \mathrm{if}e=e_i\wedge {\theta }_{c,i}\ne \epsilon \hfill \\ {\theta }_{c,i}\hfill & \mathrm{if}e\ne e_i\hfill \\ \mathrm{undefined}\hfill & \mathrm{otherwise}.\hfill \end{array}\right.\end{array}$$

  (29)

When a new event occurs, for recording the control delays, $\mathbb{PLUS}({\overline{\theta }}_c)$ adds 1 to all of the natural numbers in ${\theta }_{c,i}$. When a new control command $\overline{\varphi }=[{\varphi }_1,\dots ,{\varphi }_m]\in \Phi$ is issued (following a new observation), ${\mathbb{IN}}^{ctr}({\overline{\theta }}_c,\overline{\varphi })$ adds the newly issued control command to the end of control channel i. When a new control command is executed by actuator i, ${\mathbb{OUT}}_i^{ctr}({\overline{\theta }}_c,\varphi )$ removes the first control command $\varphi$ from ${\theta }_{c,i}$.

• To keep track of what has been successfully communicated to the supervisor so far, define bijection $h:{\Sigma }_o\to {\Sigma }_h$, such that ${\Sigma }_h=\{h(\sigma ):\sigma \in {\Sigma }_o\}$ is a set disjoint from $\Sigma$. For all $\sigma \in {\Sigma }_o$, we use $h(\sigma )$ to denote that the occurrence of $\sigma$ has been communicated to the supervisor.
• To model which control action has been executed by one of the actuators, we define bijection $d:\{0,1\}\times {\Sigma }_c\to {\Sigma }_d$, such that ${\Sigma }_d=\{d(\varphi ):\varphi \in \{0,1\}\times {\Sigma }_c\}$ is disjoint from $\Sigma \cup {\Sigma }_h$. For all $\varphi \in \{0,1\}\times {\Sigma }_c$, we use $d(\varphi )$ to denote that the control command $\varphi$ has been executed by the corresponding actuator.

Given a supervisor $D=(W,\omega )$ with $W=(Z,{\Sigma }_o,\zeta ,z_0)$, we formally construct

$$G_D=(Q_D,{\Sigma }_D,{\delta }_D,q_{0,D}),$$

where $Q_D\subseteq Q\times \Phi \times {\Theta }_o\times {\Theta }_c\times Z$ is the state space; $q_{0,D}=(q_0,D(\epsilon ),\left[{\underbrace{\epsilon ,\dots ,\epsilon }}_n\right],\left[{\underbrace{\epsilon ,\dots ,\epsilon }}_m\right],z_0)$ is the initial state; ${\Sigma }_D\subseteq \Sigma \cup {\Sigma }_h\cup {\Sigma }_d$ is the event set; the transition function ${\delta }_D:Q_D\times {\Sigma }_D\to Q_D$ is defined as:

• For all $\tilde{q}=(q,\overline{\varphi },{\overline{\theta }}_o,{\overline{\theta }}_c,z)\in Q_D$ and all $\sigma \in \Sigma$,

  $$\begin{array}{c}\hfill {\delta }_D(\tilde{q},\sigma )=\left\{\begin{array}{cc}{\tilde{q}}^{\prime }\hfill & \mathrm{if}\delta (q,\sigma )!\wedge [\sigma \in {\Sigma }_c\Rightarrow \sigma \in \overline{\varphi }]\wedge \hfill \\ \hfill & {\mathbb{IN}}^{obs}({\overline{\theta }}_o,\sigma )!\wedge \mathbb{PLUS}({\overline{\theta }}_c)!\hfill \\ \mathrm{undefined}\hfill & \mathrm{otherwise},\hfill \end{array}\right.\end{array}$$

  (30)

  where ${\tilde{q}}^{\prime }=(\delta (q,\sigma ),\overline{\pi },{\mathbb{IN}}^{obs}({\overline{\theta }}_o,\sigma ),\mathbb{PLUS}({\overline{\theta }}_c),z)$;
• For all $\tilde{q}=(q,\overline{\varphi },{\overline{\theta }}_o,{\overline{\theta }}_c,z)\in Q_D$ and all $h(\sigma )\in {\Sigma }_h$,

  $$\begin{array}{c}\hfill {\delta }_D(\tilde{q},h(\sigma ))=\left\{\begin{array}{cc}{\tilde{q}}^{\prime }\hfill & \mathrm{if}{\mathbb{OUT}}^{obs}({\overline{\theta }}_o,\sigma )!\hfill \\ \mathrm{undefined}\hfill & \mathrm{otherwise},\hfill \end{array}\right.\end{array}$$

  (31)

  where ${\tilde{q}}^{\prime }=(q,\overline{\varphi },{\mathbb{OUT}}^{obs}({\overline{\theta }}_o,\sigma ),{\mathbb{IN}}^{ctr}({\overline{\theta }}_c,\omega (\zeta (z,\sigma ))),\zeta (z,\sigma ))$;
• For all $\tilde{q}=(q,\overline{\varphi },{\overline{\theta }}_o,{\overline{\theta }}_c,z)\in Q_D$ and all $d(\varphi )\in {\Sigma }_d$,

  $$\begin{array}{c}\hfill {\delta }_D(\tilde{q},d(\varphi ))=\left\{\begin{array}{cc}{\tilde{q}}^{\prime }\hfill & \mathrm{if}{\mathbb{OUT}}^{ctr}({\overline{\theta }}_c,\varphi )!\hfill \\ \mathrm{undefined}\hfill & \mathrm{otherwise},\hfill \end{array}\right.\end{array}$$

  (32)

  where ${\tilde{q}}^{\prime }=(q,\mathbb{UD}(\overline{\varphi },\varphi ),{\overline{\theta }}_o,{\mathbb{OUT}}^{ctr}({\overline{\theta }}_c,\varphi ),z)$.

For all $\mu \in \mathcal{L}(G_D)$, let ${\psi }^{\prime }(\mu )$ be the sequences obtained by removing all the event occurrences in ${\Sigma }_h\cup {\Sigma }_d$ without changing the order of the remaining event occurrences in $\mu$. We extend ${\psi }^{\prime }(·)$ to a set of sequences in the usual way. The dynamics of the closed-loop system can be simply obtained from $G_S$ as follows.

Definition 9. 

Given system G and supervisor D, we construct $G_D$ as described above. All possible strings that may be generated by the closed-loop system with the observation delays $N_{o,1},\cdots ,N_{o,n}$ and the control delays $N_{c,1},\cdots ,N_{c,m}$, are defined as: $\mathcal{L}(D/G)={\psi }^{\prime }(\mathcal{L}(G_D))$.

By Definition 9, we can specify the dynamics of the closed-loop system when the sensors and the actuators are distributed at different sites. Furthermore, we can extend the proposed approaches to make the state estimation and synthesize the supervisor for the “distributed” system. Since this paper focuses on the case where there is one control channel and one observation channel, such an extension to the “distributed” system is beyond the scope of this paper.

8. Conclusions

In this paper, we considered the optimal supervisory control of DESs under communication delays. It is assumed that (i) delays do not change the order of the observations and controls; and (ii) both the observation delays and control delays have upper bounds. A modeling framework for supervisory control under communication delays was developed and evaluated. With this proposed framework, an online algorithm for the state estimation of the supervised system is proposed. The proposed algorithm can be used to solve the supervisor’s synthesis problem in networked DESs. Compared with the supervisor proposed in the existing work, (i) the synthesized supervisor can be more permissive as the proposed framework and state estimation approaches are more precise; (ii) the proposed framework considers the nondeterministic observation delays and control delays, which often happen. An application is provided to show how to implement the proposed algorithm. Finally, we extended the proposed framework to specify the dynamics of the closed-loop system when the sensors and actuators of the system are distributed, where delays may change the order of the observations and the controls.

One direction for future research can be to enhance the application scope of the proposed approach by accommodating communication losses in the system model. Researches can also look at how to estimate the states and synthesize supervisors when the sensors and actuators of the system are distributed.