Incremental Coding for Real-Time Remote Control over Bandwidth-Limited Channels and Its Applications in Smart Grids ^†

Abstract

Remote control over communication networks with bandwidth-constrained channels has attracted considerable recent attention because it holds the promise of enabling a large number of real-time applications, such as autonomous driving, smart grids, and the industrial internet of things (IIoT). However, due to the limited bandwidth, the sub-packets or even bits have to be transmitted successively, thereby experiencing non-negligible latency and inducing serious performance loss in remote control. To overcome this, we introduce an incremental coding method, in which the actuator acts in real time based on a partially received packet instead of waiting until the entire packet is decoded. On this basis, we applied incremental coding to a linear control system to obtain a remote-control scheme. Both its stability conditions and average linear-quadratic-Gaussian-(LQG) cost are presented. Then, we further investigated a multi-user remote-control method, with a particular focus on its applications in the demand response of smart grids over bandwidth-constrained communication networks. The utility loss due to the bandwidth constraint and communication latency are minimized by jointly optimizing the source coding and real-time demand response. The numerical results show that the incremental-coding-aided remote control performed well in both single-user and multi-user scenarios and outperformed the conventional zero-hold control scheme significantly under the LQG metric.

1. Introduction

Remote control over communication networks is expected to play a key role in the future real-time applications of autonomous driving, the industrial internet of things (IIoT), and, especially, smart grids [1]. However, compared to remote-control systems [2] that use wired communications to provide real-time and reliable data transmission, remote control over wireless communication networks suffers serious communication limits [3], in terms of bandwidth, latency, and so on. Therefore, it is essential to study suitable technologies that effectively reduce the performance loss of remote control under these communication limits [4,5].

In particular, to satisfy the extremely strict low-latency communication requirement in the scenario of remote control over communication networks, the deep integration of communication and real-time control has been further considered. More specifically, the joint optimization of communication and some specific control processes were studied in [6,7,8,9,10,11,12,13,14,15]. Firstly, Tatikonda, Sahai, and Mitter studied the design of channel coding in [6,7,8] for a general distributed linear control system. They proposed a concept of “information complexity” to characterize the channel rate required for the stabilization of different control objectives. The trade-off between the communication rate and the linear-quadratic-regulator-(LQR) cost was studied in [9] for the linear stochastic system. In [10], the communication delay in a networked Markov decision process was studied. The performance of such networked Markov decision processes was also shown to be extremely sensitive to communication latency. In the quantized linear-quadratic-Gaussian-(LQG) control problem, a lack of separation principle was found in [11]. It was shown that feedback control, estimation, and quantization cannot be separated. In [12], goal-oriented quantization was considered for practical control tasks, in which quantization policy and decision making were jointly optimized. In [13], finite-blocklength coding was jointly considered in a wireless networked control system (WNCS). The counterintuitive result found was that a scalar linear control system is stabilized with an arbitrarily large latency when the SNR of the wireless channel is high enough. In [14], a coding-free control method was proposed in WNCS, which aimed to minimize the sum control cost of multiple plants by allocation of data-transmission power. A differential coding scheme was proposed in [15], based on which the stability properties were analyzed for networked controlled linear systems. Furthermore, the effect of quantization and estimation on the stability of linear control systems has also been widely addressed, such as in quantization accuracy [16], the quantization scheme [17,18], and state estimation [19,20].

Communication-and-control co-design is particularly essential to the application of demand response in smart grids. More specifically, demand response alters the user’s demand profile effectively to make it match the electricity supply, which is important to improve the reliability of smart grid operation [21]. However, the smooth implementation of demand response requires the guarantee of extremely low communication latency [22]. The millisecond-level delay of communication may induce a certain degree of performance loss which cannot be ignored in demand response. To fulfill the extremely low latency requirement [23], the deep integration of communication and smart grids needed to be further considered. In the demand-response scenario of smart grids, the communication requirements were thoroughly studied in [24,25,26]. The wireless-communication-bandwidth requirement for demand response was studied in [24]. It was shown that the wireless bandwidth for demand response can be extremely limited as the number of users in the smart grid increases. On the other hand, with a larger number of users, demand response will become more effective [25]. Therefore, a highly scalable communication system for demand response is particularly important. Furthermore, the quality-of-service-(QoS) requirements on data transmission for real-time sensing and metering were also studied in [26]. It was shown that for the smooth implementation of various pricing schemes in demand response, the communication latency of real-time metering is required to be a few milliseconds, which is extremely strict.

The motivation for incremental coding is to achieve optimal control based on partially received packets. Similar ideas can also be found in progressive coding in image- and video-transmission problems. For example, the wavelet-transform-based method [27] is widely used in the JPEG2000. Furthermore, multiple-description coding [28] can optimize the decoding results based on each individually received data packet under unreliable channel conditions. Layered coding, on the other hand, can achieve rate scalability by providing layered embedded bit streams that can be decoded at different bit rates [29]. A closer example of incremental decoding appeared in [30], where an iterative-quantization algorithm was proposed for quantized Kalman filtering under communication-limited conditions. However, partial decoding of data packets was not considered.

In our previous work [31,32,33,34], we proposed an incremental-coding-based communication mechanism for real-time control tasks to satisfy the extremely strict requirement of communication latency, in which the source-codebook design and control process were jointly optimized. More specifically, to minimize the performance loss induced by information latency in real-time control or decision making, the incremental-coding mechanism allows the controller to take action in real time based on the currently received partial codeword instead of waiting for the entire codeword. On this basis, joint optimization of source coding and decision making was finally formulated as a general problem and solved by a dynamic-programming algorithm. However, to broaden the application scope of these works, we also needed to extend our control model to linear control systems, which are widely implemented in industrial applications, such as the demand response of smart grids.

For this paper, we focused on introducing an incremental-coding-based communication mechanism into linear control systems. More specifically, we first studied the scenario of a linear control system with a single controller and a single plant. In this scenario, an incremental-coding-based control scheme was proposed, based on which we compared LQG control performance to that of a traditional zero-hold control scheme. Then, the necessary and sufficient conditions for the plant to be stable under these two control schemes were derived, in terms of wireless bandwidth and source-code length, respectively. The analytical results showed that the stabilization conditions under these two control schemes were the same. In other words, the incremental-coding-based control scheme showed no performance gain on the stabilization of the plant, compared to the traditional zero-hold control scheme. On the other hand, the LQG performance measures of these two control schemes were further compared. In particular, the respective achievable upper bounds of the average LQG control cost under each of these two control schemes were derived. The analytical results show that the incremental-coding-based scheme significantly outperformed the zero-hold scheme under the LQG performance measure. On this basis, we extended the linear control system to the case of a multi-user control scenario. A specific multi-user control scenario for demand-response management (DRM) of smart grids was studied. In this scenario, the cumulative mean square error (MSE) between the electricity load and the supply was formulated as the LQG control cost. Then, both the centralized direct control and the distributed control through electricity price were considered in terms of DRM. The minimization of performance loss induced by the latency of the demand-scheduling signaling was finally achieved by introducing the incremental-coding-based communication mechanism with joint optimization of source coding and real-time decision making. Finally, the numerical results also showed the potential of introducing the incremental-coding mechanism in DRM.

The remainder of this paper is organized as follows. In Section 2, we introduce the model of the discrete-time scalar linear control system, along with the corresponding quantization and source-coding schemes. In Section 3, we propose the incremental-coding-based control scheme and compare its LQG control performance to that of the traditional zero-hold control scheme. In Section 4, we extend the linear control system to a specific multi-user control scenario in the DRM of the smart grid. Section 5 demonstrates our conclusions with numerical results. Finally, Section 6 concludes the paper.

2. System Model

In this section, we first introduce the composition and time discretization of a linear-time-invariant-(LTI) discrete-time control system. To satisfy the growing demand for the application of wireless networked control, the actuator and sensor are assumed to be separated and connected over a wireless channel with limited bandwidth. We then introduce the quantization and coding scheme of such a communication system with a limited bandwidth.

2.1. Discrete-Time Linear Control System

As shown in Figure 1, we consider a scalar linear control system consisting of a plant, an actuator, and a sensor. More specifically, the plant denotes the practical dynamical physical environment. The evolution of state $x\left(t\right)$ in the plant can be described by a linear differential equation as follows:

$$\dot{x}\left(t\right)=a_{c}x\left(t\right)+b_{c}u\left(t\right)+w\left(t\right),\forall t\in [0,+\infty ),$$

(1)

where $x\left(t\right)$ is the state of the plant and variable $\dot{x}\left(t\right)$ is differentiation of $x\left(t\right)$ in the time domain. Variable $u\left(t\right)$ is the control action performed by the actuator, and variable $w\left(t\right)\in [-w_0,w_0]$ is a bounded stochastic disturbance with zero mean in the state-evolution process. Parameters $a_c$ and $b_c$ are positive constants that depend on the specific dynamical physical environment. The initial state $x\left(0\right)$ belongs to a bounded set $[-x_{0,max},x_{0,max}]$. With the information of state $x\left(t\right)$ the actuator can perform appropriate $u\left(t\right)$ to make state $x\left(t\right)$ stable (approach zero).

To facilitate the design of the actuator, time is discretized into periods with duration $T_s$. In each period, the actuator performs the constant control action, i.e.,

$$u\left(t\right)=u_k,\forall k{T}_s\le t<(k+1)T_s.$$

(2)

On this basis, let $x_k$ denote the state of the plant at the beginning of k-th period, i.e., $x_k=x\left(k{T}_s\right)$. Then, the state evolution of the plant can be expressed in the following discrete-time form:

$$x_{k+1}=a{x}_k+b{u}_k+w_k,\forall k\in \mathbb{N},$$

(3)

where $x_0=x\left(0\right)\in [-x_{0,max},x_{0,max}]$. Furthermore, the parameters a, b, and the stochastic disturbance $w_k$ in Equation (3) depend on the parameters $a_c$, $b_c$ in Equation (1) and the duration $T_s$ of the control periods. By analyzing the linear differential equation [35], we can establish that

$$\begin{array}{cc}\hfill a& =e^{a_c{T}_s},\hfill \\ \hfill b& =\frac{b_c}{a_c}\left(e^{a_c{T}_s}-1\right),\hfill \\ \hfill w_k& ={\int }_{k{T}_s}^{(k+1)T_s}{e}^{a_c((k+1)T_s-\tau )}w\left(\tau \right)d\tau .\hfill \end{array}$$

(4)

As $w\left(\tau \right)$ in Equation (4) is a bounded stochastic disturbance with zero mean, such that $w\left(\tau \right)\in [-w_0,w_0]$, we can deduce that $w_k\in [-w_{max},w_{max}]$ is also a bounded stochastic disturbance with zero mean, where $w_{max}=a_c^{-1}\left(e^{a_c{T}_s}-1\right)w_0$.

We assume that the actuator cannot obtain the state of the plant by itself. To overcome this, the sensor measures the state of the plant and transmits it to the actuator periodically through a wireless channel with limited bandwidth. The measurement process of the sensor is assumed to be perfect, in which the signal distortion and the time consumption caused by the measurement can be negligible. The time interval between two measurements of the sensor is $n{T}_s$. Therefore, the actuator can obtain the information of $x_{kn},\forall k\in \mathbb{N}$ from the measurement of the sensor. Based on the information of $x_{kn}$, the actuator decides the control action $u_i$ for any $i=kn,kn+1,\cdots ,(k+1)n-1$. Based on the state evolution of the plant given in Equation (3), the state in the $k+1$-th measurement $x_{(k+1)n}$ has the following relationship with the state in the k-th measurement $x_{kn}$:

$$x_{(k+1)n}=a^n{x}_{kn}+\sum _{i=0}^{n-1}{a}^{n-1-i}b{u}_{kn+i}+\sum _{i=0}^{n-1}{a}^{n-1-i}{w}_{kn+i}.$$

(5)

2.2. Quantization and Source-Coding Schemes

The sensor measures the state of the plant with period $n{T}_s$ and transmits it to the actuator through a wireless channel with bandwidth W. In particular, the data transmission of the sensor uses BPSK modulation with the symbol period $1/W$, i.e., one bit is transmitted every $1/W$ seconds [36]. Furthermore, to align the temporal granularity of the control action and the information transmission, the duration of the control period is consistent with the symbol period, $T_s=1/W$. As a result, the sensor can transmit $n-1$ bits during the time interval between two measurements.

Before the k-th data transmission, the analog state $x_{kn}$ of the plant is quantified by a $2^{n-1}$-step uniform quantizer ${\mathcal{Q}}_{H_k}$, which is given by

$${\mathcal{Q}}_{H_k}\left(x\right)=\left\{\begin{array}{ccc}\left(⌈\frac{x}{\Delta }⌉-\frac{1}{2}\right)\Delta ,\hfill & \hfill & 0<x\le H_k,\hfill \\ \left(⌊\frac{x}{\Delta }⌋+\frac{1}{2}\right)\Delta ,\hfill & \hfill & -H_k<x\le 0,\hfill \\ 0,\hfill & \hfill & else,\hfill \end{array}\right.$$

(6)

where $H_k$ is the range of the analog state, i.e., $x_{kn}\in [-H_k,H_k]$ and the variable $\Delta =\frac{H_k}{2^{n-2}}$ is the quantized interval. After quantizing the analog state into a discrete value, the sensor implements a source codebook to encode the discrete-state value ${\mathcal{Q}}_{H_k}\left(x\right)$ into the codeword ${\mathbf{c}}_x$ with $n-1$ bits. For such a uniform quantizer ${\mathcal{Q}}_{H_k}$, the mapping between the analog state x and the codeword ${\mathbf{c}}_x$ is given as follows:

$${\mathbf{c}}_x\left(i\right)=\left\{\begin{array}{ccc}\mathbb{I}\{x>0\},\hfill & \hfill & i=1,\hfill \\ ⌊\frac{|{\mathcal{Q}}_{H_k}\left(x\right)|}{2^{n-1-i}\Delta }⌋mod2,\hfill & \hfill & 2\le i\le n-1.\hfill \end{array}\right.$$

(7)

For ease of understanding, we present the example of quantization and source coding in the case of $n=4$, as shown in Figure 2. From this example, we can see that the range of the uncertainty region of the analog state x will be reduced to exactly half of the original uncertainty region when one new bit of codeword ${\mathbf{c}}_x$ is received at the actuator. As a result, when part of codeword ${\mathbf{c}}_x$ is received, the actuator can extract the information of analog state x immediately for real-time control instead of waiting until the complete codeword is received. This new way of information utilization, named “incremental coding” [32], may improve the performance of delay-critical linear control systems substantially.

In the following, we shall compare the incremental-coding-based control scheme and the traditional zero-hold control scheme under the LQG performance measure based on the above wireless networked linear control system. In particular, such a comparison of the LQG performance measure is also extended to multi-user control scenarios.

3. Incremental-Coding-Empowered Linear Feedback Systems

In this section, we propose the incremental-coding-based control scheme and compare it to the traditional zero-hold control scheme. More specifically, under these control schemes, the stability domain of the plant is obtained in the two-dimensional space of wireless bandwidth and source-code length. On this basis, we further compare the LQG performance measure of these control schemes. The analytical results show that the incremental-coding-based scheme outperforms the zero-hold scheme significantly under the LQG performance measure.

3.1. Zero-Hold Control Scheme

In this subsection, we introduce the traditional zero-hold control scheme, in which the actuator controls the plant only when the codeword $c_x$ is fully decoded. In other words, the actuator keeps waiting until the complete information of ${\mathcal{Q}}_{H_k}\left(x\right)$ is received. Under the LQG manner, the control input $u_{kn+i},i=1,2,\cdots ,n-1$ of the zero-hold scheme is given as follows:

$$u_{kn+i}=\left\{\begin{array}{ccc}0,\hfill & \hfill & 0\le i\le n-2,\hfill \\ -\frac{a^n}{b}{\mathcal{Q}}_{H_k}\left(x_{kn}\right),\hfill & \hfill & i=n-1.\hfill \end{array}\right.$$

(8)

The result of control action $u_{(k+1)n-1}$ in the last control period can be derived, based on Equation (5). More specifically, to make $x_{(k+1)n}$ approach zero as much as possible, we should maintain that

$$\sum _{i=0}^{n-1}{a}^{n-1-i}b{u}_{kn+i}=-a^n{x}_{kn}.$$

(9)

As $u_{kn+i}=0,\forall 0\le i\le n-2$, when the actuator obtains the complete information of ${\mathcal{Q}}_{H_k}\left(x_{kn}\right)$ in the last control period, the control action should be $u_{(k+1)n-1}=-\frac{a^n}{b}{\mathcal{Q}}_{H_k}\left(x_{kn}\right)$.

Under the control action $u_{kn+i}$ given in Equation (8), the analytical results of state $x_{kn+i}$ between the k-th measurement and the $k+1$-th measurement are obtained as follows:

$$x_{kn+i}=\left\{\begin{array}{ccc}{a}^i{x}_{kn}+\sum _{l=0}^{i-1}{a}^{i-1-l}{w}_{kn+l},\hfill & \hfill & 1\le i\le n-1,\hfill \\ a^n\left(x_{kn}-{\mathcal{Q}}_{H_k}\left(x_{kn}\right)\right)+\sum _{l=0}^{n-1}{a}^{n-1-l}{w}_{kn+l},\hfill & \hfill & i=n.\hfill \end{array}\right.$$

(10)

The result of Equation (10) shows that the value of state $x_{kn+i}$ is divided into two parts. The first part is induced by the state of the k-th measurement $x_{kn}$. The second part is induced by the stochastic disturbance in the state-evolution process, which is inevitable. Fortunately, the value of the first part can be decreased significantly once the complete information of ${\mathcal{Q}}_{H_k}\left(x_{kn}\right)$ is obtained in the last control period. This is because based on Equation (6) we obtain

$$-\frac{H_k}{2^{n-1}}\le x_{kn}-{\mathcal{Q}}_{H_k}\left(x_{kn}\right)\le \frac{H_k}{2^{n-1}},$$

(11)

so that the bound is much smaller than that of original bound on $x_{kn}$, i.e., $-H_k\le x_{kn}\le H_k$.

Lemma 1. 

Under the zero-hold control scheme, the quantization-interval size $H_{k+1}$ in the $k+1$-th measurement has the following recursive relationship with $H_k$ in the k-th measurement:

$$H_{k+1}=\frac{a^n}{2^{n-1}}{H}_k+\frac{a^n-1}{a-1}{w}_{max}.$$

(12)

Proof. 

Based on the analytical results of state $x_{kn+i}$ in Equation (10), the state of the plant at the $k+1$-th measurement is given as follows:

$$x_{(k+1)n}=a^n\left(x_{kn}-{\mathcal{Q}}_{H_k}\left(x_{kn}\right)\right)+\sum _{l=0}^{n-1}{a}^{n-1-l}{w}_{kn+l}.$$

(13)

As $-w_{max}\le w_{kn+l}\le w_{max}$, then we have

$$-\frac{a^n-1}{a-1}{w}_{max}\le \sum _{l=0}^{n-1}{a}^{n-1-l}{w}_{kn+l}\le \frac{a^n-1}{a-1}{w}_{max}.$$

(14)

Substituting inequality (11) and (14) into Equation (13), we can finally ascertain that

$$\begin{array}{cc}\hfill x_{(k+1)n}& \ge -\left(\frac{a^n}{2^{n-1}}{H}_k+\frac{a^n-1}{a-1}{w}_{max}\right)\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill x_{(k+1)n}& \le \frac{a^n}{2^{n-1}}{H}_k+\frac{a^n-1}{a-1}{w}_{max}.\hfill \end{array}$$

(16)

□

3.2. Incremental-Coding-Based Control Scheme

In this subsection, we shall introduce the incremental-coding-based control scheme. By contrast with the zero-hold control scheme, the incremental-coding based-control scheme extracts the information of the state in real time based on the currently received partial codeword instead of waiting for the complete codeword.

Let ${\widehat{x}}_{kn}(kn+i)$ denote the estimation of state $x_{kn}$ in the $kn+i$-th control period based on the incremental decoding result under the given codebook in Equation (7). By taking the center point of the uncertainty region as the estimation value (this is reasonable when state $x_{kn}$ follows a uniform distribution in an uncertainty region), we have

$$\begin{array}{c}\hfill {\widehat{x}}_{kn}(kn+i)=\left\{\begin{array}{ccc}\left(⌈\frac{x_{kn}{2}^{i-1}}{H_k}⌉-\frac{1}{2}\right)\frac{H_k}{2^{i-1}},\hfill & \hfill & 0\le x_{kn}\le H_k,\hfill \\ \left(⌊\frac{x_{kn}{2}^{i-1}}{H_k}⌋+\frac{1}{2}\right)\frac{H_k}{2^{i-1}},\hfill & \hfill & -H_k\le x_{kn}<0,\hfill \end{array}\right.\end{array}$$

(17)

where $i=1,2,\cdots ,n-1$ and ${\widehat{x}}_{kn}\left(kn\right)=0$ in the case of $i=0$.

Let $V(kn+i)$ denote the estimation error of state $x_{kn}$ in the $kn+i$-th control period, i.e., $V(kn+i)=x_{kn}-{\widehat{x}}_{kn}(kn+i)$. Then, based on ${\widehat{x}}_{kn}(kn+i)$ given in Equation (17), the estimation error $V(kn+i)$ is bounded by i as

$$-\frac{H_k}{2^i}\le V(kn+i)\le \frac{H_k}{2^i}.$$

(18)

As a result, the estimation error decreases exponentially as i increases. Using the estimation of state $x_{kn}$ under the incremental-coding-based communication mechanism, the actuator adjusts control action $u_{kn+i}$ in real time to make the state approach zero as closely as possible. To this end, the control action under the incremental-coding-based scheme is given by

$$\begin{array}{c}\hfill u_{kn+i}=\frac{a^{i+1}}{b}({\widehat{x}}_{kn}(kn+i-1)-{\widehat{x}}_{kn}(kn+i)),\forall i=1,2,\cdots ,n-1,\end{array}$$

(19)

where $u_{kn}=0$, because no information of $x_{kn}$ is received in the $kn$-th control period.

Lemma 2. 

Under the incremental-coding-based control scheme, the state of the plant can be presented as the following general-term formula:

$$x_{kn+t}=a^{t}V(kn+t-1)+\sum _{l=0}^{t-1}{a}^{t-1-l}{w}_{kn+l},\forall t=1,2,\cdots ,n.$$

(20)

Proof. 

The proof of Lemma 2 is based on mathematical induction. We assume that state $x_{kn+t}$ obeys the form presented in Equation (20) when $t=i$. We then show that $x_{kn+t}$ still obeys the form presented in Equation (20) when $t=i+1$. More specifically, substituting control action $u_{kn+i}$ given by Equation (19) into the state evolution of the plant given by Equation (3), we can ascertain that

$$\begin{array}{cc}\hfill x_{kn+i+1}& =a{x}_{kn+i}+b{u}_{kn+i}+w_{kn+i}\hfill \\ \hfill & =a^{i+1}(x_{kn}-{\widehat{x}}_{kn}(kn+i-1))+\sum _{l=0}^{i-1}{a}^{i-l}{w}_{kn+l}\hfill \\ \hfill & +a^{i+1}({\widehat{x}}_{kn}(kn+i-1)-{\widehat{x}}_{kn}(kn+i))+w_{kn+i}\hfill \\ \hfill & =a^{i+1}V(kn+(i+1)-1)\hfill \\ \hfill & +\sum _{l=0}^{(i+1)-1}{a}^{(i+1)-1-l}{w}_{kn+l}.\hfill \end{array}$$

(21)

The result of Equation (21) shows that if state $x_{kn+t}$ obeys Equation (20) when $t=i$, then it still obeys Equation (20) when $t=i+1$. Fortunately, by substituting control action $u_{kn}=0$ into the state evolution of the plant, we have

$$x_{kn+1}=aV\left(kn\right)+w_{kn}.$$

(22)

As a result, the form presented in Equation (20) is established when $t=1$. On this basis, the form presented in Equation (20) is also established when $t=2,3,\cdots ,n$. □

Comparing the results of Equations (10) and (20), we note that the state of the plant under the incremental-coding-based control scheme is smaller than that under the zero-hold control scheme. In particular, based on the analytical result in Equation (20), the value of the state at the $k+1$-th measurement is obtained as

$$x_{(k+1)n}=a^n\left(x_{kn}-{\mathcal{Q}}_{H_k}\left(x_{kn}\right)\right)+\sum _{l=0}^{n-1}{a}^{n-1-l}{w}_{kn+l},$$

(23)

which is exactly the same as $x_{(k+1)n}$ under the zero-hold control scheme shown in Equation (10). As a result, when we implement the incremental-coding-based control scheme, the quantization interval size $H_{k+1}$ and $H_k$ have the same recursive relationship as shown in Lemma 1, i.e.,

$$H_{k+1}=\frac{a^n}{2^{n-1}}{H}_k+\frac{a^n-1}{a-1}{w}_{max}.$$

(24)

3.3. Stability-Analysis-and-LQG-Performance Comparison

Based on the analytical result of the state given in Equations (10) and (20), we can further study the necessary and sufficient conditions for the stabilization of the plant under the zero-hold control scheme and the incremental-coding-based control scheme, respectively. In particular, the stabilization condition of the plant in the mean-square sense can be defined as

$$\underset{t\to \infty }{lim\; sup}\mathbb{E}\left(\right|x_t{|}^2)<\infty .$$

(25)

Under the LQG control manner, we can further define the square of the state as the LQG control cost. As a result, the average control cost $C_{ave}$ is given by

$$\begin{array}{cc}\hfill C_{ave}& =\underset{T\to \infty }{lim}\frac{1}{T}\sum _{t=0}^T\mathbb{E}\left(\right|x_t{|}^2),\hfill \\ \hfill & =\underset{L\to \infty }{lim}\frac{1}{L}\sum _{k=1}^L\frac{1}{n}\sum _{t=kn+1}^{(k+1)n}\mathbb{E}\left(\right|x_t{|}^2),\hfill \\ \hfill & =\underset{L\to \infty }{lim}\frac{1}{L}\sum _{k=1}^L{C}_{ave}^{\left(k\right)},\hfill \end{array}$$

(26)

where $C_{ave}^{\left(k\right)}=\frac{1}{n}{\sum }_{t=kn}^{(k+1)n-1}\mathbb{E}\left(\right|x_t{|}^2)$ is the average cost between the k-th measurement and the $k+1$-th measurement. To ensure the stabilization condition listed in condition (25), we only need to ensure that the average LQG control cost $C_{ave}$ in Equation (26) is bounded.

Lemma 3. 

Under the zero-hold control scheme, the average cost between the k-th measurement and the $k+1$-th measurement, i.e., $C_{ave}^{\left(k\right)}$ is bounded by $H_k$ and $w_{max}$ as follows:

$$C_{ave}^{\left(k\right)}\le \frac{a^{2n}-1}{n(a^2-1)}{H}_k^2+\left(\frac{a^{2n}-1}{n(a^2-1)}-\frac{1}{a^2-1}\right){\sigma }_w^2,$$

(27)

where ${\sigma }_w^2$ is the variance of stochastic disturbance $w_k$. The same holds for (27) when $x_{kn}=\pm H_k$, and $w_{kn+i}=\pm w_{max},\forall i=0,1,\cdots ,n-1$.

Proof. 

Based on Equation (26) and the analytical results of state $x_{kn+i}$ in Equation (10), the average LQG control cost $C_{ave}^{\left(k\right)}$ under the zero-hold control scheme is given by Equation (28):

$$\begin{array}{cc}\hfill \frac{1}{n}\sum _{t=0}^{n-1}\mathbb{E}\left(\right|x_{kn+t}{|}^2)& =\frac{1}{n}\sum _{t=0}^{n-1}\mathbb{E}\left(a^{2t}{x}_{kn}^2+{\left(\sum _{l=0}^{t-1}{a}^{t-1-l}{w}_{kn+l}\right)}^2+2a^t{x}_{kn}\left(\sum _{l=0}^{t-1}{a}^{t-1-l}{w}_{kn+l}\right)\right)\hfill \\ \hfill & \stackrel{\left(\mathrm{I}\right)}{=}\frac{1}{n}\sum _{t=0}^{n-1}\mathbb{E}\left(a^{2t}{x}_{kn}^2+\sum _{l=0}^{t-1}{a}^{2(t-1-l)}{w}_{kn+l}^2\right)\hfill \\ \hfill & \stackrel{\left(\mathrm{II}\right)}{\le }\frac{1}{n}\sum _{t=0}^{n-1}\left(a^{2t}{H}_k^2+\sum _{l=0}^{t-1}{a}^{2(t-1-l)}{\sigma }_w^2\right)\hfill \\ \hfill & =\frac{a^{2n}-1}{n(a^2-1)}{H}_k^2+\left(\frac{a^{2n}-1}{n(a^2-1)}-\frac{1}{a^2-1}\right){\sigma }_w^2.\hfill \end{array}$$

(28)

More specifically, equation (I) holds in Equation (28) because the stochastic disturbance $w_{kn+i}$ is zero mean and independent of the state at the k-th measurement $x_{kn}$. Inequality (II) holds in Equation (28) because $x_{kn}\in [-H_k,H_k]$ and $w_{kn+l}\in [-w_{max},w_{max}]$. Therefore, the same holds in (27) when $x_{kn}=\pm H_k$ and $w_{kn+i}=\pm w_{max},\forall i=0,1,\cdots ,n-1$. □

Lemma 4. 

Under the incremental-coding-based control scheme, the expectation of average cost between the k-th measurement and the $k+1$-th measurement, i.e., $C_{ave}^{\left(k\right)}$ is bounded by $H_k$ and $w_{max}$ as follows:

$$C_{ave}^{\left(k\right)}\le \frac{a^{2n}-4^n}{n{4}^{(n-1)}(a^2-4)}{H}_k^2+\left(\frac{a^{2n}-1}{n(a^2-1)}-\frac{1}{a^2-1}\right){\sigma }_w^2.$$

(29)

The same holds in (29) when $x_{kn}=\pm H_k$ and $w_{kn+i}=\pm w_{max},\forall i=0,1,\cdots ,n-1$.

Proof. 

According to Lemma 2 and Equation (26), the average LQG control cost $C_{ave}^{\left(k\right)}$ under the incremental-coding-based control scheme is given by Equation (30):

$$\begin{array}{cc}\hfill \frac{1}{n}\sum _{t=0}^{n-1}\mathbb{E}\left(\right|x_{kn+t}{|}^2)& =\frac{1}{n}\mathbb{E}\left(\sum _{t=1}^{n-1}\right(a^{2t}{V}^2(kn+t-1)+{\left(\sum _{l=0}^{t-1}{a}^{(t-1-l)}{w}_{kn+l}\right)}^2\hfill \\ \hfill & +2a^{t}V(kn+t-1)\left(\sum _{l=0}^{t-1}{a}^{(t-1-l)}{w}_{kn+l}\right))+{\left|x_{kn}\right|}^2)\hfill \\ \hfill & \stackrel{\left(\mathrm{I}\right)}{=}\frac{1}{n}\mathbb{E}\left(\sum _{t=1}^{n-1}\left(a^{2t}{V}^2(kn+t-1)+\sum _{l=0}^{t-1}{a}^{2(t-1-l)}{w}_{kn+l}^2\right)+{\left|x_{kn}\right|}^2\right)\hfill \\ \hfill & \stackrel{\left(\mathrm{II}\right)}{\le }\frac{1}{n}\sum _{t=0}^{n-1}\left(a^{2t}\frac{H_k^2}{2^{2t}}+\sum _{l=0}^{t-1}{a}^{2(t-1-l)}{\sigma }_w^2\right)\hfill \\ \hfill & =\frac{a^{2n}-4^n}{n{4}^{(n-1)}(a^2-4)}{H}_k^2+\left(\frac{a^{2n}-1}{n(a^2-1)}-\frac{1}{a^2-1}\right){\sigma }_w^2.\hfill \end{array}$$

(30)

Similar to the proof of Lemma 3, equation (I) holds in Equation (30) because the stochastic disturbance $w_{kn+i}$ is zero mean and independent of the estimation error $V(kn+t-1)$. Inequality (II) holds in Equation (30) because $x_{kn}\in [-H_k,H_k]$ and $w_{kn+l}\in [-w_{max},w_{max}]$. Therefore, the same holds in (29) when $x_{kn}=\pm H_k$ and $w_{kn+i}=\pm w_{max},\forall i=0,1,\cdots ,n-1$. □

Corollary 1. 

The achievable upper bound of the average LQG control cost $C_{ave}^{\left(k\right)}$ between the k-th measurement and the $k+1$-th measurement under the incremental-coding-based control scheme outperforms that under the zero-hold control policy.

Proof. 

Based on Equations (28) and (30), we have

$$\begin{array}{cc}\hfill & \left(\frac{a^{2n}-4^n}{n{4}^{(n-1)}(a^2-4)}{H}_k^2+\left(\frac{a^{2n}-1}{n(a^2-1)}-\frac{1}{a^2-1}\right){\sigma }_w^2\right)\hfill \\ \hfill & -\left(\frac{a^{2n}-1}{n(a^2-1)}{H}_k^2+\left(\frac{a^{2n}-1}{n(a^2-1)}-\frac{1}{a^2-1}\right){\sigma }_w^2\right)\hfill \\ \hfill & =\frac{1}{n}\sum _{t=0}^{n-1}\left(a^{2t}\frac{H_k^2}{2^{2t}}+\sum _{l=0}^{t-1}{a}^{2(t-1-l)}{\sigma }_w^2\right)-\frac{1}{n}\sum _{t=0}^{n-1}\left(a^{2t}{H}_k^2+\sum _{l=0}^{t-1}{a}^{2(t-1-l)}{\sigma }_w^2\right)\hfill \\ \hfill & =\frac{H_k^2}{n}\sum _{t=0}^{n-1}\left({\left(\frac{a^2}{4}\right)}^t-{\left(a^2\right)}^t\right)\le 0.\hfill \end{array}$$

As $\left({\left(\frac{a^2}{4}\right)}^t-{\left(a^2\right)}^t\right)\le 0$ for any $t=0,1,\cdots ,n-1$, we have ${\sum }_{t=0}^{n-1}\left({\left(\frac{a^2}{4}\right)}^t-{\left(a^2\right)}^t\right)\le 0$. □

Theorem 1. 

The necessary and sufficient condition for the stability of the plant in relation to both the wireless bandwidth W and the source-code length n are as follows:

$$e^{\frac{n{a}_c}{W}}-2^{n-1}<0,$$

(31)

regardless of whether the zero-hold control scheme or the incremental-coding-based control scheme is implemented.

Proof. 

According to Equation (26), the necessary and sufficient condition in Equation (25) for the stability of the plant is equivalent to ensure that the average LQG control cost $C_{ave}^{\left(k\right)}$ is bounded for any k. Based on Lemmas 3 and 4, we note that the achievable upper bound of $C_{ave}^{\left(k\right)}$ increases linearly with $H_k^2$ and $w_{max}^2$. As $w_{max}=a_c^{-1}\left(e^{a_c{T}_s}-1\right)w_0$ is bounded, the necessary and sufficient condition in Equation (25) is thereby equivalent to ensure that $H_k$ is bounded for any positive integer k. On the other hand, the update of $H_k$ is given by

$$H_{k+1}=\frac{a^n}{2^{n-1}}{H}_k+\frac{a^n-1}{a-1}{w}_{max},$$

(32)

regardless of whether the zero-hold control scheme or the incremental-coding-based control scheme is implemented. Therefore, the condition of bounded $H_k$ for any positive integer k is equivalent to the condition $\frac{a^n}{2^{n-1}}<1$. By substituting $a=e^{a_c{T}_s}$ and $T_s=\frac{1}{W}$ into $\frac{a^n}{2^{n-1}}<1$, we obtain $e^{\frac{n{a}_c}{W}}-2^{n-1}<0$ as the necessary and sufficient condition for the stability of the plant. □

Corollary 2. 

When the plant meets the stability condition in Theorem 1, the achievable upper bound of $C_{ave}$ under the incremental-coding-based control scheme is given by

$$\begin{array}{cc}\hfill C_{ave}& =\frac{a^{2n}-4^n}{n{4}^{(n-1)}(a^2-4)}{\left(\frac{(a^n-1)2^{n-1}}{(2^{n-1}-a^n)(a-1)}{w}_{max}\right)}^2\hfill \\ \hfill & +\left(\frac{a^{2n}-1}{n(a^2-1)}-\frac{1}{a^2-1}\right){\sigma }_w^2.\hfill \end{array}$$

Proof. 

As the plant meets the stability condition in Theorem 1, we have $\frac{a^n}{2^{n-1}}<1$. Then, based on the update of $H_k$ in Equation (32), we have

$$\underset{k\to \infty }{lim}{H}_k=\frac{(a^n-1)2^{n-1}}{(2^{n-1}-a^n)(a-1)}{w}_{max}.$$

(33)

As a result, according to Lemma 4, the limit of the achievable upper bound of $C_{ave}^{\left(k\right)}$ is given by

$$\underset{k\to \infty }{lim}{C}_{ave}^{\left(k\right)}\le \frac{(a^n-1)2^{n-1}}{(2^{n-1}-a^n)(a-1)}{w}_{max}.$$

(34)

Based on Equation (26), the right-hand side of Equation (34) is also the achievable upper bound of $C_{ave}$. □

3.4. Group Incremental Coding with Finite Blocklength

We omitted the effect of channel coding in the previous analysis. However, considering that incremental coding requires splitting data packets, the impact of different blocklengths in channel coding needs to be considered. According to the conclusions of finite-blocklength coding [37], the achievable transmission rate increases with the coding blocklength. In other words, if incremental coding uses a shorter coding blocklength, the efficiency of the channel coding will be relatively reduced. Therefore, we need to strike a balance between the gains (the average LQG cost) and the disadvantages (the channel-coding efficiency) of incremental coding. This issue can be addressed by employing incremental coding at different resolutions.

From a unified perspective, if we term the current incremental coding as “incremental coding with the minimum resolution of 1 bit” and label the zero-hold strategy as “incremental coding with the maximum resolution of $n-1$ bits”, we can then explore decoding at various resolutions. For instance, if we split the complete data packet into units of k bits, it is equivalent to updating the control strategy every k time slots. With a smaller value of k, we can achieve a smaller average control cost, but at the same time the channel-coding efficiency will be reduced, and the decoding results and the actuator will need to be updated at a higher frequency. Depending on the practical scenario and constraints, an appropriate resolution k should be chosen while maintaining acceptable control costs.

4. Applications in Demand-Response Management of Smart Grids

In this section, we extend the single-user control scenario to the case of the multi-user control scenario. A specific multi-user control scenario in demand-response management of smart grids is studied. In this scenario, the cumulative MSE between the electricity load and the supply is formulated as the LQG control cost. On this basis, both the centralized and the distributed communication modes for optimization of the LQG control performance are considered. Under these two communication modes, the joint optimization of source coding and incremental-coding-based control is introduced for the minimization of the LQG control cost.

4.1. Multi-User Control Scenario in Demand-Response Management

As shown in Figure 3, the aim of DRM is to balance the total electricity load $X_L$ of the end users and the electricity supply $X_S$ of the power utility, as quickly as possible. However, due to the limited bandwidth of wireless channels and the huge number of end users, emergency-demand scheduling information is difficult to support. As a result, to balance the electricity load and supply in real-time demand response, we introduce joint optimization of the source-codebook design and incremental-coding-based control for the minimization of cumulative MSE between the electricity load and the supply.

At the beginning of each duration T, the power utility sends the demand scheduling information $\omega$ to the end users according to the total power supply $X_S$. The information $\omega$ is coded into a binary codeword $\mathcal{B}\left(\omega \right)$ by the source encoder and transmitted to the end users through the wireless channel. However, due to the limited data-transmission power, the wireless channel can only support one bit of error-free transmission per channel use. The duration of each channel use is denoted by one period and is equal to the inverse of the channel bandwidth W, i.e., $1/W$. As a result, the binary codeword $\mathcal{B}\left(\omega \right)$ is received successively by the end users. Let ${\mathcal{B}}^{\left(n\right)}\left(\omega \right)$ denote the first n segment of codeword $\mathcal{B}\left(\omega \right)$. Therefore, the received partial codeword at the n-th period of each duration is also ${\mathcal{B}}^{\left(n\right)}\left(\omega \right)$.

For end user i, there is a concave utility function $U_i\left(x\right)$ to describe the gain that end user i obtains from its electricity load x, where $U_i^{\prime }\left(0\right)=+\infty$. Based on the received partial codeword ${\mathcal{B}}^{\left(n\right)}\left(\omega \right)$ and the utility function $U_i\left(x\right)$, the end user i adjusts its electricity load in real time. Let $x_i\left(t\right)$ denote the real-time electricity load of user i, which is determined by the received partial codeword and the utility function. As a result, the total real-time electricity load $X_L\left(t\right)$ of all the users is given by

$$X_L\left(t\right)=\sum _{i=1}^K{x}_i\left(t\right).$$

(35)

To balance the total electricity load and the electricity supply of the power utility as quickly as possible, the instantaneous cost-of-demand response is defined as the square error between real-time load $X_L\left(t\right)$ and supply $X_S$. Let $C_{ave}$ denote the expected cumulative cost in the duration T, which is given by

$$C_{ave}={\mathbb{E}}_{X_S\sim \mathbf{p}}\left\{{\int }_0^T{(X_L\left(t\right)-X_S)}^{2}dt\right\}.$$

(36)

Furthermore, the expected cumulative cost $C_{ave}$ can also be presented as the LQG control cost of the linear control system, in which $T=n$, $a=0$, $u_{kn+i}=X_L\left((kn+i)T_s\right)$, and $w_{kn+t}=X_S$.

In addition to the balance of electricity load and supply, we should also consider the maximization of the users’ utility under the given electricity-supply constraint. As a result, the users’ demand response $\{x_1\left(t\right),x_2\left(t\right),\cdots ,x_K\left(t\right)\}$ is expected to converge to the optimal solution to the following problem, as t approaches T:

$$\begin{array}{cc}\hfill \underset{\{x_1,x_2,\cdots ,x_K\}}{max}& \sum _{i=0}^K{U}_i\left(x_i\right)\hfill \\ \hfill s.t.& \sum _{i=0}^K{x}_i\le X_S,\hfill \\ \hfill & x_i\ge 0,\forall i=1,2,\cdots ,K.\hfill \end{array}$$

(37)

To minimize $C_{ave}$ in Equation (36) while maximizing the total users’ utility in problem (37), both the demand-scheduling signaling and its corresponding source coding shall be carefully designed. In the remainder of this paper, we shall try different schemes to design these parameters and compare their performance in the numerical results.

4.2. Demand-Response Management under Distributed-Communication Mechanism

In this subsection, we assume that the power utility has knowledge of all the users’ utility functions. Based on this assumption, the power utility transmits the information of $x_k^{*}$ to each individual end user k under the distributed-communication mode.

More specifically, the power utility first obtains the optimal solution $\{x_1^{*},x_2^{*},\cdots ,x_K^{*}\}$ to problem (37). Then, the demand-scheduling information is set to be $\{x_1^{*},x_2^{*},\cdots ,x_K^{*}\}$ and is transmitted to the end users for adjusting the electricity load. In the remainder of this section, we shall study how to transmit the demand-scheduling information $\{x_1^{*},x_2^{*},\cdots ,x_K^{*}\}$ over the wireless channel with limited bandwidth W to reduce $C_{ave}$ as much as possible.

4.2.1. Sharing the Wireless Channel through TDMA

One straightforward way to transmit demand-scheduling information to end users is by sharing the broadcast channel via TDMA. More specifically, for each individual user i, its optimal demand response $x_i^{*}$ is determined by the power supply $X_S$ from the power utility. Define $f_i$ as a mapping from $X_S$ to $x_i^{*}$. As $X_S\in \mathcal{X}=\{X_1,X_2,\cdots ,X_n\}$, based on $f_i$ we can also obtain the alphabet ${\mathcal{X}}_i$ for $x_i^{*}$ as ${\mathcal{X}}_i=\{f_i\left(X_1\right),f_i\left(X_2\right),\cdots ,f_i\left(X_n\right)\}$. Given the alphabet ${\mathcal{X}}_i$ for user i, the scheduling information $x_i^{*}$ is coded independently as the codeword ${\mathcal{B}}_i\left(x_i^{*}\right)$ according to a designed codebook ${\mathcal{B}}_i$. Then, the codeword ${\mathcal{B}}_i\left(x_i^{*}\right)$ is transmitted to user i over the broadcast channel with bandwidth W. The channel is shared equally among all the users in the time domain. In each time slot with duration $1/W$, it supports one bit of error-free transmission, as shown in Figure 4a.

Under the TDMA framework, each user i receives one new bit of codeword ${\mathcal{B}}_i\left(x_i^{*}\right)$ every $K/W$ seconds. Then, user i adjusts its real-time electricity load $x_i\left(t\right)$ according to the received partial codeword ${\mathcal{B}}_i^{\left(n\right)}\left(x_i^{*}\right)$, where ${\mathcal{B}}_i^{\left(n\right)}\left(x_i^{*}\right)$ is the first n segment of codeword ${\mathcal{B}}_i\left(x_i^{*}\right)$. As a result, the real-time electricity load of user i is given by

$$x_i\left(t\right)=\left\{\begin{array}{ccc}w\left({\mathcal{B}}_i^{\left(n\right)}\left(x_i^{*}\right)\right),\hfill & \hfill & \frac{nK}{W}\le t\le \frac{(n+1)K}{W},n<l_{x_i^{*}},\hfill \\ x_i^{*},\hfill & \hfill & t\ge \frac{l_{x_i^{*}}K}{W},\hfill \end{array}\right.$$

(38)

where n is a non-negative integer, variable $l_{x_i^{*}}$ is the length of codeword ${\mathcal{B}}_i\left(x_i^{*}\right)$, and function $w\left(.\right)$ denotes the decision-making scheme for adjusting the real-time electricity load according to the received partial codeword.

To balance the electricity load and supply, the real-time load $x_i\left(t\right)$ of user i is expected to approach $x_i^{*}$ as much as possible. We denote the cumulative MSE between $x_i\left(t\right)$ and $x_i^{*}$ as ${\delta }_i$, which is given by

$$\begin{array}{cc}\hfill {\delta }_i& ={\int }_0^T{(x_i\left(t\right)-x_i^{*})}^{2}dt,\hfill \\ \hfill & =\frac{K}{W}\sum _{n=0}^{WT/K}{\left(w\left({\mathcal{B}}_i^{\left(n\right)}\left(x_i^{*}\right)\right)-x_i^{*}\right)}^2,\hfill \\ \hfill & =\frac{K}{W}\sum _{n=0}^{l_{x_i^{*}}}{\left(w\left({\mathcal{B}}_i^{\left(n\right)}\left(x_i^{*}\right)\right)-x_i^{*}\right)}^2.\hfill \end{array}$$

(39)

Our aim is to minimize the expectation of ${\delta }_i$ to balance the electricity load and supply as quickly as possible. As the number K of users and bandwidth W are constant, based on Equation (39) we only need to minimize the expectation of ${\sum }_{n=0}^{l_{x_i^{*}}}{\left(w\left({\mathcal{B}}_i^{\left(n\right)}\left(x_i^{*}\right)\right)-x_i^{*}\right)}^2.$ To this end, we introduce the joint optimization of the source codebook ${\mathcal{B}}_i$ and the real-time decision-making scheme $w(.)$, which can be formulated as the following problem:

$$\underset{{\mathcal{B}}_i,w(.)}{min}{\mathbb{E}}_{x_i^{*}}\left\{\sum _{n=0}^{l_{x_i^{*}}}{\left(w\left({\mathcal{B}}_i^{\left(n\right)}\left(x_i^{*}\right)\right)-x_i^{*}\right)}^2\right\}.$$

(40)

It is not a trivial work to solve problem (40) directly, as the feasible numbers of codebook ${\mathcal{B}}_i$ and decision-making scheme $w(.)$ are huge. Fortunately, this problem is a special case of problem (7) formulated in our previous work [32]. To solve this problem, a dynamic-programming algorithm is presented (Algorithm 2 in [32]), based on which we can efficiently obtain the optimal source codebook and decision-making scheme in problem (40). Let ${\xi }_i^{*}$ denote the optimal objective value of problem (40). Then, under the optimal source codebook and TDMA, the total cumulative MSE of all users, i.e., ${\delta }_{TDMA}$ is given by

$$\begin{array}{cc}\hfill {\delta }_{TDMA}& =\mathbb{E}\left\{\sum _{i=1}^K{\delta }_i\right\}=\frac{K}{W}\sum _{i=1}^K{\xi }_i^{*}.\hfill \end{array}$$

(41)

The careful reader may find that the metric ${\delta }_{TDMA}$ is different from the metric $C_{ave}$ in Equation (36). We would mention that $K\mathbb{E}\left\{{\sum }_{i=1}^K{\delta }_i\right\}$ is an upper bound of $C_{ave}$. This is because

$$\begin{array}{cc}\hfill {\int }_0^T{(X_L\left(t\right)-X_S)}^{2}dt& ={\int }_0^T{\left(\sum _{i=1}^K(x_i\left(t\right)-x_i^{*})\right)}^{2}dt,\hfill \\ \hfill & \le {\int }_0^{T}K\sum _{i=1}^K{(x_i\left(t\right)-x_i^{*})}^{2}dt,\hfill \\ \hfill & =K\sum _{i=1}^K{\delta }_i.\hfill \end{array}$$

(42)

As K and W are constant, we can minimize the upper bound of $C_{ave}$ by minimizing the result of ${\delta }_{TDMA}$.

4.2.2. Sharing the Wireless Channel through FDMA

In this part, we consider sharing the broadcast channel among the users through the FDMA mechanism. By implementing the FDMA mechanism, the information of $\{x_1^{*},x_2^{*},\cdots ,x_K^{*}\}$ can be transmitted simultaneously to the users. On the other hand, the bandwidth allocation in FDMA provides a more flexible way for resource allocation to minimize the total cumulative MSE.

More specifically, let $\{W_1,W_2,\cdots ,W_K\}$ denote the bandwidth allocation for the end users. For an individual end user i, it receives a new bit of codeword ${\mathcal{B}}_i\left(x_i^{*}\right)$ every $1/W_i$ seconds. Therefore, under the FDMA mechanism with the optimal source codebook in problem (40), the total cumulative MSE of all the users under the FDMA mechanism, i.e., ${\delta }_{FDMA}$ is given by

$${\delta }_{FDMA}=\sum _{i=1}^K\frac{{\xi }_i^{*}}{W_i}.$$

(43)

In the following, we aim to minimize ${\delta }_{FDMA}$ by bandwidth allocation. The problem can be formulated as follows:

$$\begin{array}{cc}\hfill \underset{W_1,W_2,\cdots ,W_K}{min}& \sum _{i=1}^K\frac{{\xi }_i^{*}}{W_i}\hfill \\ \hfill s.t.& \sum _{i=1}^K{W}_i=W,\hfill \\ \hfill & W_1,W_2,\cdots ,W_K\ge 0.\hfill \end{array}$$

(44)

Theorem 2. 

The optimal bandwidth allocation in FDMA for each individual end user i is $W_i^{*}$, which is given as follows:

$$W_i^{*}=\frac{W\sqrt{{\xi }_i^{*}}}{{\sum }_{j=1}^K\sqrt{{\xi }_j^{*}}}.$$

(45)

Proof. 

First of all, the objective function ${\sum }_{i=1}^K\frac{{\xi }_i^{*}}{W_i}$ of problem (44) is a convex function for $W_1,W_2,\cdots ,W_K\ge 0$. The constraints of problem (44) are all linear constraints. Therefore, the strong duality condition of problem (44) is established. Then, we can find the optimal solution $W_i^{*}$ by minimization of the following Lagrangian function:

$$F(W_i,{\lambda }_i,\mu )=\sum _{i=1}^K\frac{{\xi }_i^{*}}{W_i}+\mu \left(\sum _{i=1}^K{W}_i-W\right)-\sum _{i=1}^K{\lambda }_i{W}_i.$$

(46)

Based on the strong-duality condition, we have $\frac{\partial F}{\partial W_i^{*}}=0$. Then, we can ascertain that

$$-\frac{{\xi }_i^{*}}{W_i^{*2}}+{\mu }^{*}-{\lambda }_i^{*}=0.$$

(47)

Furthermore, based on the KKT condition, we have ${\lambda }_i^{*}{W}_i^{*}=0$. As a result, we obtain the relationship between $W_i^{*}$ and ${\mu }^{*}$:

$$W_i^{*}=\sqrt{\frac{{\xi }_i^{*}}{{\mu }^{*}}}.$$

(48)

By substituting Equation (48) into Equation (44.b), we finally obtain

$$W_i^{*}=\frac{W\sqrt{{\xi }_i^{*}}}{{\sum }_{j=1}^K\sqrt{{\xi }_j^{*}}}.$$

(49)

□

Corollary 3. 

The total cumulative MSE of all users, i.e., ${\delta }_{FDMA}$ with the optimal bandwidth allocation $W_i^{*}$ is given by

$${\delta }_{FDMA}=\frac{{\left({\sum }_{i=1}^K\sqrt{{\xi }_i^{*}}\right)}^2}{W}.$$

(50)

Proof. 

By substituting the result of $W_i^{*}$ into Equation (43), we can obtain Equation (50). □

Corollary 4. 

With the optimal bandwidth allocation $W_i^{*}$ in FDMA, the total cumulative MSE of all users, i.e., ${\delta }_{FDMA}$ shall satisfy

$${\delta }_{FDMA}\le {\delta }_{TDMA}.$$

(51)

Proof. 

The proof of inequality (51) is equivalent to show that the following inequality (52) is established:

$$\frac{{\left({\sum }_{i=1}^K\sqrt{{\xi }_i^{*}}\right)}^2}{W}\le \frac{K}{W}\sum _{i=1}^K{\xi }_i^{*}.$$

(52)

Then, inequality (52) can be transformed into the form of the Cauchy–Buniakowsky–Schwarz inequality as follows:

$${(\sqrt{{\xi }_1^{*}}+\cdots +\sqrt{{\xi }_K^{*}})}^2\le \underset{K}{\underbrace{(1+\cdots +1)}}({\xi }_1^{*}+\cdots +{\xi }_K^{*}).$$

(53)

The equal sign holds if and only if ${\xi }_1^{*}={\xi }_2^{*}=\cdots ={\xi }_K^{*}$. □

4.3. Demand-Response Management under Centralized Communication Mechanism

In this subsection, the demand-scheduling information is set to be the electricity price. According to the information of the electricity price and the utility function, the end users adjust the electricity load to maximize their own benefit, which is a special case of competitive Markov decision processes [38]. On this basis, demand response is managed under a centralized-communication mechanism.

4.3.1. Control with Knowledge of Utility Function

In this part, we still assume that the power utility has knowledge of all the users’ utility functions. Based on these utility functions, the power utility shall decide the electricity price and transmit it to the end users. For a given electricity price p, the benefit $b_i$ of user i is given by

$$b_i=U_i\left(x_i\right)-p{x}_i,$$

(54)

where $x_i$ is the electricity load of user i.

In the following, we shall study whether there exists an optimal electricity price $p^{*}\left(X_S\right)$ for a given power supply $X_S$. Based on the information of $p^{*}\left(X_S\right)$, each user i adjusts the electricity load to the optimal electricity load $x_i^{*}$ (the optimal solution to problem (37)) by maximizing its own benefit $b_i$. If such $p^{*}\left(X_S\right)$ exists, then we shall consider how to transmit it through the broadcast channel.

Lemma 5. 

There is an optimal electricity price $p^{*}\left(X_S\right)$, which is the solution λ to the following equation:

$$\sum _{i=1}^K{U_i^{\prime }}^{-1}\left(\lambda \right)=X_S,$$

(55)

where function ${U_i^{\prime }}^{-1}(.)$ is the inverse of the derivative of the utility function $U_i(.)$.

Proof. 

As the utility function $U_i(.)$ is a concave function, problem (37) is a convex problem. The Lagrangian function corresponding to problem (37) is given by

$$L(x_i,{\lambda }_i,\lambda )=-\sum _{i=0}^K{U}_i\left(x_i\right)+\lambda \left(\sum _{i=1}^K{x}_i-X\right)-\sum _{i=1}^K{\lambda }_i{x}_i.$$

(56)

Based on the strong-duality condition, we have $\frac{\partial L}{\partial x_i^{*}}=0$. Then, we can obtain

$$-U_i^{\prime }\left(x_i^{*}\right)+{\lambda }^{*}-{\lambda }_i^{*}=0.$$

(57)

On the other hand, based on the KKT condition, we have ${\lambda }_i^{*}{x}_i^{*}=0$ and ${\lambda }^{*}\ge 0$. As $U_i^{\prime }\left(0\right)=+\infty$, we have ${\lambda }_i^{*}=0$ and $U_i^{\prime }\left(x_i^{*}\right)={\lambda }^{*}$. Furthermore, as utility function $U_i(.)$ is a concave function, its derived function $U_i^{\prime }(.)$ is monotonically decreasing, and the inverse of $U_i^{\prime }(.)$ exists. Therefore, the optimal electricity load $x_i^{*}$ can be presented by

$$\left\{\begin{array}{c}{U}_i^{\prime }\left(x_i^{*}\right)={\lambda }^{*},\hfill \\ \sum _{i=1}^K{U_i^{\prime }}^{-1}\left({\lambda }^{*}\right)=X_S.\hfill \end{array}\right.$$

(58)

If the electricity price $p^{*}\left(X_S\right)$ is set to be ${\lambda }^{*}$ in Equation (58), the benefit $b_i$ of user i is $U_i\left(x_i\right)-{\lambda }^{*}{x}_i$. Therefore, for maximization of $b_i$, user i shall adjust its electricity load $x_i$ to satisfy $U_i^{\prime }\left(x_i\right)={\lambda }^{*}$, i.e., $x_i=x_i^{*}$. □

In the following, we shall design a source codebook $\mathcal{B}$ for transmitting the information of the optimal electricity price $p^{*}$ under the incremental-coding mechanism. The aim of incremental coding is minimization of $C_{ave}$. For the end users, they estimate the electricity price as ${\widehat{p}}^{\left(n\right)}=w\left({\mathcal{B}}^{\left(n\right)}\left(p^{*}\right)\right)$ in real time, based on the currently received partial codeword ${\mathcal{B}}^{\left(n\right)}\left(p^{*}\right)$. According to the estimated price $\widehat{p}$, each user i adjusts its electricity load ${\widehat{x}}_i^{\left(n\right)}={U_i^{\prime }}^{-1}\left({\widehat{p}}^{\left(n\right)}\right)$ to maximize its benefit $b_i$. Therefore, we can minimize $C_{ave}$ by optimizing codebook $\mathcal{B}$ and decision-making scheme $w(.)$ as follows:

$$\underset{\mathcal{B},w(.)}{min}{\mathbb{E}}_{p^{*}}\left\{\frac{1}{W}\sum _{n=0}^{l_{p^{*}}}{\left(\sum _{i=1}^K{U_i^{\prime }}^{-1}\left(w\left({\mathcal{B}}^{\left(n\right)}\left(p^{*}\right)\right)\right)-X_S\right)}^2\right\},$$

(59)

where $l_{p^{*}}$ is the codeword length of realization $p^{*}$. Fortunately, this problem is still a special case of problem (7) in our previous work [32]. Therefore, it can also be solved by a dynamic-programming algorithm, i.e., Algorithm 2 in [32].

4.3.2. Control without Knowledge of Utility Function

In this part, a more practical scenario is considered, in which the power utility has no knowledge of the users’ utility functions. Then, although the optimal electricity price $p^{*}\left(X_S\right)$ exists, the power utility cannot obtain it directly.

As a result, we need to design a new demand-scheduling signaling and its corresponding communication mechanism. Based on Lemma 5, we note that the optimal electricity price $p^{*}\left(X_S\right)$ is exactly the solution to Equation (55). Fortunately, the function ${\sum }_{i=1}^K{U_i^{\prime }}^{-1}(.)$ is monotone decreasing. This indicates to us that we can find $p^{*}\left(X_S\right)$ through a binary search.

More specifically, there are two initial parameters, i.e., the upper bound $p^{up}$ of the electricity price and the lower bound $p^{low}$ of the electricity price, which satisfy

$$\left\{\begin{array}{c}\sum _{i=1}^K{U_i^{\prime }}^{-1}\left(p^{low}\right)\ge max\left\{X_S\right|X_S\in \mathcal{X}\},\hfill \\ \sum _{i=1}^K{U_i^{\prime }}^{-1}\left(p^{up}\right)\le min\left\{X_S\right|X_S\in \mathcal{X}\}.\hfill \end{array}\right.$$

(60)

Based on the initial parameters $p^{up}$ and $p^{low}$, the electricity price p is defined to be $(p^{low}+p^{up})/2$. According to the electricity price p, the users adjust their electricity load to maximize their benefits. For a given price p, the total electricity load of all the users’ demand response, $X_L$, is given by

$$X_L=\sum _{i=1}^K{U_i^{\prime }}^{-1}\left(p\right).$$

(61)

The power utility has the knowledge of $X_L$. If $X_L>X_S$, then the power utility sends bit ‘0’ to the users, to indicate the update of $p^{low}=(p^{low}+p^{up})/2$. If $X_L<X_S$, then the power utility sends bit ‘1’ to the users, to indicate the update of $p^{up}=(p^{low}+p^{up})/2$. Based on this, the length of interval $[p^{low},p^{up}]$ shrinks exponentially. As $p^{*}$ always belongs to the interval $[p^{low},p^{up}]$, the total electricity load $X_L$ also converges to $X_S$ exponentially.

It is worth mentioning that introducing network coding proves beneficial in multi-receiver multicast scenarios. Take, for instance, the scenario depicted in Figure 5. When dealing with dual receivers, the successful decoding of $b_1$ and $b_2$ is achieved through a bitwise XOR operation. For example, $b_1$ and $b_2$ denote two consecutive bits after encoding, and by means of a linear network coding method, the end users can get both bits after decoding. Then we can use incremental coding with a resolution of 2 bits as mentioned at the end of the previous section. The performance analysis in this case can be further investigated in future work.

5. Numerical Result

In this section, we present the numerical results to validate our theoretical analysis. In particular, a single-user scenario and a multi-user scenario in smart grids are considered successively in the numerical results.

5.1. Single-User Scenario

In the single-user scenario, the parameters of a continuous-time scalar linear control system are set to be $a_c=0.8$, $b_c=0.5$, $w_0=1$, and $x_{0,max}=100$. The wireless bandwidth W is set to be 2 kHZ. The time interval 1 ms is normalized as 1. As a result, the parameters of the equivalent discrete-time scalar linear control system are given by $a=1.49$, $b=0.31$, and $w_{max}=0.61$.

In Figure 6, we set the measurement period $n=10$ and compare the results of the zero-hold control scheme to the incremental-coding-based control scheme in the same stochastic environment. As we can see in Figure 6, state x of the incremental-coding-based control scheme is much more stable around 0 than that of the zero-hold control scheme. In particular, state x under the zero-hold control scheme shifts away from 0 exponentially over time until a complete codeword is received in the last symbol period. On the other hand, state x under the incremental-coding-based control scheme begins to shift to 0 after the first symbol period. This is because the controller performs the control action in real time, based on a partially received codeword. As a result, the incremental-coding-based control scheme shows less LQG control cost than that of the zero-hold control scheme.

In Figure 7, we set the noise factor $w_0$ as 0.001 and 0.1, respectively, and show the LQG control cost under different quantization bit numbers. Firstly, as we can see, the LQG control cost of the incremental-coding-based control scheme is less than that of the zero-hold control scheme in all cases of quantization bit numbers. Then, for a given noise factor w, there is an optimal quantization bit number that minimizes the LQG control cost. This is because a too-large or too-small number of quantization bits will cause obvious control errors. In particular, a too-small number of quantization bits will cause more quantization error and a too-large number of quantization bits will cause more transmission delay, thereby inducing more noise in the control process. On the other hand, when the noise factor $w_0$ is 0.001, the optimal quantization bit number is 3. But when the noise factor $w_0$ is 0.1, the optimal quantization bit number is 2. This is because, in cases of smaller noise factor, the quantization error is more important than the noise error. Therefore, a greater quantization bit number should be considered, to reduce the quantization error.

5.2. Multi-User Scenario

In this subsection, we consider a multi-user scenario in the demand response of smart grids. We set the total power supply at the end of the power utility, $X_S$, to be an independent and identically distributed random variable with values drawn from the set $\mathcal{X}=\{4,5,6,7,8,9,10\}$. The associated discrete probability distribution is represented by $\mathbf{p}=\{0.1,0.1,0.2,0.3,0.15,0.05,0.1\}$. The power utility broadcasts scheduling information to the users, where the number of users is set to $K=3$. The utility function of each user is set to $U_i\left(x_i\right)=c_{i}ln(1+x_i)$, where $c_1=c_2=2$, $c_3=4$. The bandwidth of the wireless channel is W, in which the corresponding symbol period $\frac{1}{W}$ is normalized as 1. The duration of state $X_S$ is set to be $T=20$.

Figure 7. Comparison of LQG control cost under different quantization bit numbers.

Figure 7. Comparison of LQG control cost under different quantization bit numbers.

In Figure 8, we compare demand-response performance under different control mechanisms. More specifically, we show the expectation of ${\sum }_{i=1}^K{(x_i-x_i^{*})}^2$ in the entire process of scheduling signaling transmission. As we can see in Figure 8, compared to centralized control mechanisms, distributed control through electricity price not only exhibits the least cumulative cost 1.4755, but also reduces the cost to 0 fastest, i.e., using 25% of the time of centralized control with FDMA. On other hand, we can also see that the cumulative cost corresponding to centralized control with FDMA is about 3.9199 and smaller than 4.9319 of TDMA. This is consistent with Corollary 4.

In Figure 9, under the distributed-control mechanism, we compare the demand-response performance of cases of power utility with or without knowledge of the users’ utility functions. As we have seen, under the distributed-control mechanism, when the power utility has knowledge of all the users’ utility functions, the total electrical load converges to the supply faster than in the case where users’ utility functions are unknown. On the other hand, as a binary search is implemented, the total electricity load converges fast enough in the case that the users’ utility functions are unknown.

6. Conclusions

In this paper, we introduced an incremental-coding-based communication mechanism into a remote linear control process over communication networks. Firstly, we considered a linear control system with a single controller and a single plant, and we proposed a control scheme based on incremental coding. Both the stabilization condition of the plant and the average LQG control cost were analyzed under the incremental-coding-based control scheme. The analytical results showed that the proposed incremental-coding-based control scheme significantly outperformed the traditional zero-hold control scheme under the LQG performance measure. Furthermore, we extended the linear control system from a single-user to a multi-user control scenario. A specific multi-user control scenario in demand-response management of smart grids was studied. By introducing the incremental-coding-based communication mechanism with joint optimization of source coding and decision making, the minimization of the performance loss that is induced by the latency of demand scheduling signaling was finally achieved.