A Class of Distributed Online Aggregative Optimization in Unknown Dynamic Environment

Abstract

This paper considers a class of distributed online aggregative optimization problems over an undirected and connected network. It takes into account an unknown dynamic environment and some aggregation functions, which is different from the problem formulation of the existing approach, making the aggregative optimization problem more challenging. A distributed online optimization algorithm is designed for the considered problem via the mirror descent algorithm and the distributed average tracking method. In particular, the dynamic environment and the gradient are estimated by the averaged tracking methods, and then an online optimization algorithm is designed via a dynamic mirror descent method. It is shown that the dynamic regret is bounded in the order of $\mathcal{O}\left(\sqrt{T}\right)$. Finally, the effectiveness of the designed algorithm is verified by some simulations of cooperative control of a multi-robot system.

1. Introduction

The distributed online optimization problem for multi-agent systems has received considerable attention in the past few decades [1,2]. The objective is to minimize a global time-varying cost function, written as the sum of local convex functions, under the constrict that each individual agent has only knowledge of the local convex functions.

Recently, some researchers have focused on distributed online aggregative optimization, which is a special class of online optimization problems. In distributed online aggregative optimization, the local cost functions include some aggregative terms motivated by many real applications. These aggregative terms make the design and analyses of the online optimization algorithm more challenging. Some tracking methods were presented to solve distributed online aggregative optimization problems. For example, a distributed aggregative gradient tracking algorithm is proposed and analyzed in [3] to solve a distributed online aggregative optimization problem. It is shown that convergence to the optimal variable is linear. Different from [3], a set constraint is considered in [4], where an online distributed gradient tracking algorithm is proposed to solve a distributed online aggregative optimization problem with exact gradient information and stochastic/noisy gradients, respectively. The upper boundary of dynamic regret is analyzed and given, and some simulations on the target surrounding problem are provided to verify the effectiveness of the designed algorithm. The authors of [5] considered the distributed online aggregative optimization without the assumption of boundedness of the gradients and the feasible sets. In particular, a projected aggregative tracking algorithm was presented in [5], and the authors showed that the dynamic regret is bounded by a constant term and a term related to time variations.

Dynamic environments are common in the real applications of online optimization [6,7,8,9,10,11]. For example, the authors of [6] showed that many online learning problems, including the dynamic texture analysis, solar flare detection, sequential compressed sensing of a dynamic scene, traffic surveillance, tracking self-exciting point processes, and network behavior in the Enron email corpus, can benefit from the incorporation of a dynamic environment. In [7], the tracking problem of a time-varying parameter with unknown dynamics was studied as an online optimization in a dynamic environment. The distributed tracking and tracking of dynamic point processes on network problems were solved in [8,9] using mirror descent methods with dynamical environments, respectively. The localization of sensor networks problem was solved in [10] by using a distributed online bandit learning algorithm over a multi-agent network with a dynamical environment. Note that the dynamic environments were assumed to be known in [6,7,8,9,10]. In contrast with [6,7,8,9,10], an unknown dynamic environment was considered in [11], where the dynamic regret was shown to be bounded.

Based on the above discussions, the distributed online aggregative optimization with a dynamic environment is motivated by many real applications. The problem has many applications, especially under the condition that the dynamic environment is unknown. Therefore, this paper considers a class of distributed online aggregative optimization problems with an unknown dynamic environment. The main contributions are as follows. Some aggregative terms and an unknown dynamic environment are simultaneously considered in this paper for an online optimization problem. The problem formulation comes from some real applications, e.g., the cooperative control problem for a multi-robot system in the simulation section. Compared with [3,4,5,6,7,8,9,10,11], the dynamic environment was not considered in [3,4,5], and the aggregative terms were not considered in [6,7,8,9,10,11], respectively. Therefore, the design and analyses of the optimization algorithm in this paper are more challenging than in [3,4,5,6,7,8,9,10,11]. In particular, some averaged tracking methods estimate the dynamic environment and the gradient. Based on the estimations, an online optimization algorithm is designed via a dynamic mirror descent method. It is shown that the dynamic regret is bounded in the order of $\mathcal{O}\left(\sqrt{T}\right)$, and some simulations are proposed to verify the effectiveness of the designed algorithm.

The remaining parts of this paper are organized as follows. In Section 2 and Section 3, some preliminaries and problem formulation are presented. The main results are proposed in Section 4. In Section 5, the performance of the designed algorithm is verified by some simulations. Section 6 concludes the paper.

2. Preliminaries

2.1. Notations

${\mathbb{R}}^n$ denotes the n-dimensional Euclidean space. $\parallel x\parallel$ denotes the 2-norm of a vector x. $\left[T\right]$ with positive integer T denotes the set $\{1,2,\dots ,T\}$. $\langle x,y\rangle$ denotes the inner product of vectors $x\in {\mathbb{R}}^n$ and $y\in {\mathbb{R}}^n$, i.e., $\langle x,y\rangle =x^{T}y$ where $x^T$ denotes the transpose of x. For a function $f(x,t):{\mathbb{R}}^n\times \mathbb{R}\to \mathbb{R}$, $\nabla f(x,t)$ denotes the gradient of $f(x,t)$ with respect to a vector x. ${\sigma }_2\left(W\right)$ denotes the second-largest singular value of matrix W.

2.2. Graph Theory

For a multi-agent system, we use a directed graph $\mathcal{G}=(\mathcal{V},\mathcal{E})$ to describe the information exchange within it, where $\mathcal{V}=\{1,2,\dots ,N\}$ is the node set and $\mathcal{E}\subset \mathcal{V}\times \mathcal{V}$ is the edge set. $(j,i)\in \mathcal{E}$ represents that agent i can obtain information from agent j. The self-loop is unconsidered, i.e., $(i,i)\notin \mathcal{E}$. ${\mathcal{N}}_i=\{j:(j,i)\in \mathcal{E}\}$ denotes the in-neighbor set of node i. If for every $(j,i)\in \mathcal{E}$, there exists $(i,j)\in \mathcal{E}$, then the graph is an undirected graph. A path between nodes i and j is a sequence of edges $(i,{\mathcal{V}}_1),({\mathcal{V}}_1,{\mathcal{V}}_2),\dots ,({\mathcal{V}}_k,j)$ in the graph $\mathcal{G}$ where ${\mathcal{V}}_l,l=1,2,\dots ,k$ are some distinct nodes. An undirected graph is connected if there is a path between each pair of nodes.

2.3. Bregman Divergence

The mirror descent algorithm based on Bregman divergence is frequently used and effective in online optimization [8]. The Bregman divergence is defined as follows. Let $R:{\mathbb{R}}^n\to \mathbb{R}$ be a strongly convex function which satisfies

$$R\left(x\right)\ge R\left(y\right)+\langle x-y,\nabla R\left(y\right)\rangle +{\displaystyle \frac{1}{2}}{\parallel x-y\parallel }^2,\forall x,y\in {\mathbb{R}}^n,$$

(1)

and define the Bregman divergence $D_R(x,y)$ as

$$D_R(x,y)=R\left(x\right)-R\left(y\right)-\langle x-y,\nabla R\left(y\right)\rangle .$$

(2)

3. Problem Formulation

Considering a network system composed of N agents, and there exists a sequence of time-varying convex function $f_t\left(x\right):{\mathbb{R}}^n\to \mathbb{R}$ over the network composed of $f_{i,t}(x,v\left(x\right)):{\mathbb{R}}^n\times {\mathbb{R}}^d\to \mathbb{R}$. Considering the following optimization problem,

$$\begin{array}{ccc}\hfill \min {.}_{x\in \mathcal{X}}{f}_t\left(x\right)& =& \sum _{i=1}^N{f}_{i,t}(x,v\left(x\right)),\hfill \end{array}$$

(3)

where $i\in \left[N\right]$, $t\in \left[T\right]$, $x\in \mathcal{X}\subseteq {\mathbb{R}}^n$, ${\psi }_i:{\mathbb{R}}^n\to {\mathbb{R}}^d$, and $v:{\mathbb{R}}^n\to {\mathbb{R}}^d$ is the aggregative variable defined by $v\left(x\right)={\displaystyle \frac{1}{N}}{\sum }_{i=1}^N{\psi }_i\left(x\right)$. The function $f_{i,t}(x,v\left(x\right))$ is convex and assigned to agent i. Supposing there exists a sequence of unknown stable non-expansive mapping $A_t\in {\mathbb{R}}^{n\times n}$, i.e., $\parallel A_t\parallel \le 1$, such that

$$\begin{array}{c}\hfill x_{t+1}^{\ast }=A_t{x}_t^{\ast }+{\vartheta }_t,\end{array}$$

(4)

where $x_t^{\ast }={\mathrm{argmin}}_{x\in \mathcal{X}}{f}_t\left(x\right)$ and ${\vartheta }_t$ is an unstructured and unknown noise. Assuming that agent i independently observes the mapping $A_t$, and $A_{i,t}$ denotes the observed value in time t. Moreover, assuming that $A_t^{\ast }$ is the optimal observed value for all agents and $A_t^{\ast }$ is the optimal solution to the following optimization problem:

$$\begin{array}{c}\hfill A_t^{\ast }=\underset{{\tilde{A}}_t}{argmin}\sum _{i=1}^N{\parallel {\tilde{A}}_t-A_{i,t}\parallel }^2.\end{array}$$

(5)

It follows from (4) and (5) that

$$\begin{array}{c}\hfill x_{t+1}^{\ast }=A_t^{\ast }{x}_t^{\ast }+{\theta }_t,\end{array}$$

(6)

where ${\theta }_t=(A_t-A_t^{\ast })x_t^{\ast }+{\vartheta }_t$.

Distributed Online Aggregative Optimization Problem:

The objective is to generate a sequence $x_{i,t},i\in \left[N\right],t\in \left[T\right]$ to minimize the following dynamic regret,

$$\begin{array}{c}\hfill Reg\left(T\right):=\sum _{t=1}^T\sum _{i=1}^N{f}_{i,t}\left(x_{i,t}\right)-\sum _{t=1}^T\sum _{i=1}^N{f}_{i,t}\left(x_t^{\ast }\right),\end{array}$$

(7)

where $x_t^{\ast }$ is the optimal solution to (3) which satisfies constraint (6), i.e., $x_t^{\ast }={\mathrm{argmin}}_{x\in \mathcal{X}}{f}_t\left(x\right)$ while $x_t^{\ast }$ satisfies constraint (6).

Remark 1.

The online aggregative optimization problems have been studied in [4,5]. However, this paper considers an unknown dynamic environment, which is more challenging than [4,5], i.e., the mapping $A_t$.

Let $\nabla {\psi }_i\left(x_i\right)$, ${\nabla }_1{f}_{i,t}(x_i,v)$ and ${\nabla }_2{f}_{i,t}(x_i,v)$ denote ${\nabla }_{x_i}{\psi }_i\left(x_i\right)$, ${\nabla }_{x_i}{f}_{i,t}(x_i,v)$ and ${\nabla }_v{f}_{i,t}(x_i,v)$, respectively. Let $W\in {\mathbb{R}}^{N\times N}$ be the adjacency matrix of network $\mathcal{G}$, and ${\eta }_t\in \mathbb{R},t\in \left[T\right]$ be the global step-size which will be used to design an online optimization algorithm in the next section. Let $\overline{\mathcal{X}}\subseteq {\mathbb{R}}^n$ and $\overline{\mathcal{Y}}\subseteq {\mathbb{R}}^d$ be some convex sets which will be defined in the next section. Some necessary assumptions are as follows.

Assumption 1.

Graph $\mathcal{G}$ is undirected and connected, and W is doubly stochastic, i.e.,

$$\begin{array}{c}\hfill \sum _{i=1}^N{W}_{ij}=\sum _{j=1}^N{W}_{ij}=1,\end{array}$$

and there exists a constant $\alpha \in (0,1)$ such that $W_{ij}\ge \alpha$ when $W_{ij}\ge 0$, and $W_{ii}\ge \alpha$ for all $i\in \left[N\right]$.

Assumption 2.

For $x,m\in \overline{\mathcal{X}}$, $y,n\in \overline{\mathcal{Y}}$, $i\in \left[N\right]$ and $t\in \left[T\right]$, the functions $f_{i,t}\left(x\right)$, ${\nabla }_1{f}_{i,t}(x,y)$, ${\nabla }_2{f}_{i,t}(x,y)$, ${\psi }_i\left(x\right)$, and $\nabla {\psi }_i\left(x\right)$ are Lipschitz continuous, and the functions ${\psi }_i\left(x\right)$, ${\nabla }_1{f}_{i,t}(x,y)$ and ${\nabla }_2{f}_{i,t}(x,y)$ are bounded, i.e.,

$$\begin{array}{ccc}& & \iota \parallel x-m\parallel \le \parallel f_{i,t}\left(x\right)-f_{i,t}\left(m\right)\parallel \le L\parallel x-m\parallel \hfill \\ & & \parallel {\nabla }_1{f}_{i,t}(x,y)-{\nabla }_1{f}_{i,t}(m,n)\parallel \le L(\parallel x-m\parallel +\parallel y-n\parallel )\hfill \\ & & \parallel {\nabla }_2{f}_{i,t}(x,y)-{\nabla }_2{f}_{i,t}(m,n)\parallel \le L(\parallel x-m\parallel +\parallel y-n\parallel )\hfill \\ & & \parallel {\psi }_i\left(x\right)-{\psi }_i\left(m\right)\parallel \le L\parallel x-m\parallel \hfill \\ & & \parallel \nabla {\psi }_i\left(x\right)-\nabla {\psi }_i\left(m\right)\parallel \le L\parallel x-m\parallel \hfill \\ & & \parallel {\psi }_i\left(x\right)\parallel \le H,\parallel {\nabla }_1{f}_{i,t}(x,y)\parallel \le H,\parallel {\nabla }_2{f}_{i,t}(x,y)\parallel \le H,\hfill \end{array}$$

where L is the Lipschitz constant, $\iota >0$ and $H>0$.

Assumption 3.

There exists

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill |D_R(x,z)-D_R(y,z)|& \le K\parallel x-y\parallel \hfill \\ \hfill D_R(x,\sum _{i=1}^{N}a\left(i\right)y_i)& \le \sum _{i=1}^{N}a\left(i\right)D_R(x,y_i)\hfill \\ \hfill |\nabla R(x)-\nabla R(y\left)\right|& \le L_R\parallel x-y\parallel ,\hfill \end{array}\end{array}$$

where $x,y,y_i,z\in \overline{\mathcal{X}}$, $a\left(i\right)$ is on the N-dimensional simplex, and K and $L_R$ are some Lipschitz constants.

Assumption 4.

There exists positive constants $M_A$ and M, such that $\parallel A_t^{\ast }-I_n\parallel \le M_A{\eta }_t$, and ${\sum }_{i=1}^N{\parallel A_{i,t}-A_{i,t-1}\parallel }_1\le M{\eta }_t$, where ${\eta }_t$ is the global step sequence. Moreover, $D_R(A_t^{\ast }x$, $A_t^{\ast }y)\le D_R(x,y)$ holds for $x,y\in \overline{\mathcal{X}}$ and $t\in \left[T\right]$.

Remark 2.

Assumptions 1, 3, and 4 come from [6,7,8,11]. They are common in mirror descent algorithms for distributed online optimization. Assumption 2 is also common in online aggregation optimization [4].

4. Main Result

Inspired by [11], consider the following online algorithm:

$$\begin{array}{cc}\hfill {\widehat{A}}_{i,t}& =\sum _{j=1}^N{W}_{ij}{\widehat{A}}_{j,t-1}+A_{i,t}-A_{i,t-1}\hfill \end{array}$$

(8a)

$$\begin{array}{cc}\hfill {\widehat{x}}_{i,t+1}& =\underset{x\in \mathcal{X}}{\mathrm{argmin}}\{{\eta }_t\langle x,{\nabla }_{i,t}\rangle +D_R(x,y_{i,t})\}\hfill \end{array}$$

(8b)

$$\begin{array}{cc}\hfill x_{i,t+1}& =\sum _{j=1}^N{W}_{ij}{\widehat{A}}_{j,t}{\widehat{x}}_{i,t+1}\hfill \end{array}$$

(8c)

$$\begin{array}{cc}\hfill y_{i,t+1}& =\sum _{j=1}^N{W}_{ij}{x}_{j,t+1}\hfill \end{array}$$

(8d)

$$\begin{array}{cc}\hfill v_{i,t+1}& =\sum _{j=1}^N{W}_{ij}{v}_{j,t}+{\psi }_i\left(x_{i,t+1}\right)-{\psi }_i\left(x_{i,t}\right)\hfill \end{array}$$

(8e)

$$\begin{array}{cc}\hfill z_{i,t+1}& =\sum _{j=1}^N{W}_{ij}{z}_{j,t}+\nabla {\psi }_i\left(x_{i,t+1}\right)-\nabla {\psi }_i\left(x_{i,t}\right),\hfill \end{array}$$

(8f)

where $i\in \left[N\right]$, $t\in \left[T\right]$, ${\nabla }_{i,t}={\nabla }_1{f}_{i,t}(x_{i,t},v_{i,t})+{\nabla }_2{f}_{i,t}(x_{i,t},v_{i,t})z_{i,t}$, ${\eta }_t$ is the global step sequence, ${\widehat{x}}_{i,t+1}$, $y_{i,t+1}$, ${\widehat{A}}_{i,t+1}$, $v_{i,t+1}$ and $z_{i,t+1}$ are some auxiliary variables, and $x_{i,t+1}$ is the designed sequence. In (8), the initial states are chosen such that ${\widehat{A}}_{i,0}=A_{i,0}={\mathbf{0}}_{n\times n}$, $y_{i,0}=x_{i,0}={\widehat{x}}_{i,0}={\mathbf{0}}_n$, $v_{i,0}={\psi }_i\left(x_{i,0}\right)$ and $z_{i,0}=\nabla {\psi }_i\left(x_{i,0}\right)$.

It follows from (8) that the states in (8) are bounded. Let sets $\overline{\mathcal{X}}$ and $\overline{\mathcal{Y}}$ satisfy $\mathcal{X}\subseteq \overline{\mathcal{X}}$, $x_{i,t}\in \overline{\mathcal{X}}$, $y_{i,t}\in \overline{\mathcal{X}}$ and $v_{i,t}\in \overline{\mathcal{Y}}$. Let $P={max}_i\{v_{i,1},z_{i,1}\}$, $Q={sup}_{x\in \mathcal{X}}\parallel x\parallel$, $R^2={sup}_{x,y\in \mathcal{X}}{D}_R(x,y)$, and

$$\begin{array}{ccc}\hfill ▵A_{i,t}& =& \sum _{j=1}^N{W}_{ij}{\widehat{A}}_{j,t}-A_t^{\ast },{\overline{z}}_t={\displaystyle \frac{1}{N}}\sum _{i=1}^N{z}_{i,t}\hfill \\ \hfill {\overline{x}}_{t+1}& =& {\displaystyle \frac{1}{N}}\sum _{i=1}^N{x}_{i,t+1},\nabla v\left(x\right)={\displaystyle \frac{1}{N}}\sum _{i=1}^N\nabla {\psi }_i\left(x\right)\hfill \\ \hfill {\tilde{\nabla }}_{i,t}& =& {\nabla }_1{f}_{i,t}(x_{i,t},v\left(x_{i,t}\right))+{\nabla }_2{f}_{i,t}(x_{i,t},v\left(x_{i,t}\right))\nabla v\left(x_{i,t}\right).\hfill \end{array}$$

Remark 3.

We have provided a comparative analysis of problem formulation of online optimization in

Table 1. Comparison of problem formulation and algorithm of online optimization.

Reference	Aggregative Terms	Dynamic Environment	Algorithm/Technique
[3]	Yes	No	Distributed aggregative gradient tracking
[4]	Yes	No	Online distributed gradient tracking
[5]	Yes	No	Projected aggregative tracking
[6]	No	Known	Dynamic mirror descent
[7]	No	Known	Online gradient descent
[8]	No	Known	Decentralized mirror descent
[10]	No	Known	Adaptive gradient method
[9]	No	Known	Distributed bandit online leaning
[11]	No	Unknown	Gradient tracking via DAT
This paper	Yes	Unknown	Aggregative gradient tracking via DAT

, where $\mathrm{DAT}$ denotes distributed average tracking. It follows that the problem considered in this paper is more challenging than [3,4,5,6,7,8,9,10,11]. Algorithms (8a)–(8d) come from [11]. In fact, algorithms (8a)–(8d) have been used in [11] to solve a class-distributed online optimization problem with an unknown mapping. However, the aggregative variable $v\left(x\right)$ is not considered in [11]. This makes the design and analyses of the optimization algorithm more challenging. In particular, it follows from (3) that, for agent i, the gradient of the function $f_{i,t}(x,v\left(x\right))$ with respect to x is ${\nabla }_1{f}_{i,t}(x_{i,t},v\left(x_{i,t}\right))+{\nabla }_2{f}_{i,t}(x_{i,t},v\left(x_{i,t}\right))\nabla v\left(x_{i,t}\right)$. But $v\left(x_{i,t}\right)$ and $\nabla v\left(x_{i,t}\right)$ are unknown for agent i due to the topology constraint. Therefore, the algorithms (8e) and (8f) are added by using the distributed average tracking method to estimate $v\left(x_{i,t}\right)$ and $\nabla v\left(x_{i,t}\right)$. Based on (8e) and (8f), the new gradient ${\nabla }_{i,t}={\nabla }_1{f}_{i,t}(x_{i,t},v_{i,t})+{\nabla }_2{f}_{i,t}(x_{i,t},v_{i,t})z_{i,t}$ is introduced in algorithm (8b). Hence, algorithms (8e) and (8f) are used to achieve aggregative gradient tracking via $DAT$. A strict analysis of dynamic regret of algorithm (8) will be presented as follows.

Some lemmas and the main result are presented as follows, and the proofs are given in the Appendix A, Appendix B, Appendix C, Appendix D, Appendix E, Appendix F and Appendix G.

Lemma 1.

Consider the mapping (5) and the algorithm (8a), and supposing that Assumptions 1 and 4 hold. Then for all $i\in \left[N\right]$ and $t\in \left[T\right]$, there exists

$$\begin{array}{c}\hfill \parallel ▵A_{i,t}\parallel \le {\displaystyle \frac{8M{\eta }_t}{1-\lambda }},\end{array}$$

where $\lambda \in (0,1)$.

Lemma 2.

Considering algorithm (8f), and supposing that Assumptions 1 and 2 hold, then for all $i\in \left[N\right]$ and $t\in \left[T\right]$, there exists,

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \parallel z_{i,t}-{\overline{z}}_t\parallel \le {\Delta }_1,\parallel {\nabla }_{i,t}\parallel \le {\Delta }_2,\end{array}\end{array}$$

where ${\Delta }_1=NP\gamma +{\displaystyle \frac{2NL\gamma \beta }{1-\beta }}+4L$, ${\Delta }_2=H{\Delta }_1+HL+H$, $\gamma ={(1-{\displaystyle \frac{\alpha }{2N^2}})}^{-2}$, and $\beta =1-{\displaystyle \frac{\alpha }{2N^2}}$.

Lemma 3.

Considering algorithms (8a)–(8d), and supposing that Assumptions 1–4 hold, then for all $i\in \left[N\right]$ and $t\in \left[T\right]$, there exists,

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \parallel x_{i,t+1}-{\overline{x}}_{t+1}\parallel & \le {\Delta }_3\sum _{\tau =0}^t{\sigma }_2^{t-\tau }\left(W\right){\eta }_{\tau },\hfill \end{array}\end{array}$$

(9)

where ${\Delta }_3=\sqrt{N}\left[{\Delta }_2+M_{A}Q+{\displaystyle \frac{8MQ}{1-\lambda }}\right]$.

Lemma 4.

Considering algorithm (8), and supposing that Assumptions 1–3 hold, then for all $i\in \left[N\right]$ and $t\in \left[T\right]$, there exists,

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \parallel x_{i,t+1}-x_{i,t}\parallel & \le {\Delta }_4{\eta }_t+2{\Delta }_3\sum _{\tau =0}^t{\sigma }_2^{t-\tau }\left(W\right){\eta }_{\tau }\hfill \\ \hfill \parallel v_{i,t}-v\left(x_{i,t}\right)\parallel & \le N\gamma P{\beta }^t+{\Delta }_5{\eta }_{t-1}+{\Delta }_6\sum _{\tau =0}^t{\sigma }_2^{t-\tau }\left(W\right){\eta }_{\tau }\hfill \\ \hfill \parallel z_{i,t}-\nabla v\left(x_{i,t}\right)\parallel & \le N\gamma P{\beta }^t+{\Delta }_5{\eta }_{t-1}+{\Delta }_6\sum _{\tau =0}^t{\sigma }_2^{t-\tau }\left(W\right){\eta }_{\tau },\hfill \end{array}\end{array}$$

(10)

where ${\Delta }_4={\Delta }_2+M_{A}Q+{\displaystyle \frac{8MQ}{1-\lambda }}$, ${\Delta }_5=L{\Delta }_4({\displaystyle \frac{N\gamma \beta }{1-\beta }}+2)$, and ${\Delta }_6=2L{\Delta }_3({\displaystyle \frac{N\gamma \beta }{1-\beta }}+3)$.

Lemma 5.

Considering algorithm (8), and supposing that Assumptions 1, 3 and 4 hold, then for all $i\in \left[N\right]$ and $t\in \left[T\right]$, there exists,

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill & \sum _{t=1}^T\sum _{i=1}^N({\displaystyle \frac{1}{{\eta }_t}}{D}_R(x_t^{\ast },y_{i,t})-{\displaystyle \frac{1}{{\eta }_t}}{D}_R(x_t^{\ast },{\widehat{x}}_{i,t+1}))\hfill \\ \hfill \le & {\displaystyle \frac{2N{R}^2}{{\eta }_{T+1}}}+\sum _{t=1}^T{\displaystyle \frac{NK}{{\eta }_{t+1}}}\parallel x_{t+1}^{\ast }-A_t^{\ast }{x}_t^{\ast }\parallel +\sum _{t=1}^T\sum _{i=1}^N{\displaystyle \frac{{\eta }_t}{{\eta }_{t+1}}}{\Delta }_7\parallel A_t^{\ast }{x}_t^{\ast }-x_{i,t+1}\parallel ,\hfill \end{array}\end{array}$$

(11)

where ${\Delta }_7={\displaystyle \frac{8L_{R}MQ}{(1-\lambda )}}$.

Lemma 6.

Considering algorithm (8), and supposing that Assumptions 1–3 hold, then for all $i\in \left[N\right]$ and $t\in \left[T\right]$, there exists,

$$\begin{array}{c}\hfill \begin{array}{cc}& \parallel {\tilde{\nabla }}_{i,t}-{\nabla }_{i,t}\parallel \le (L+L^2+H)\{N\gamma P{\beta }^t+{\Delta }_5{\eta }_{t-1}+{\Delta }_6\sum _{\tau =0}^t{\sigma }_2^{t-\tau }\left(W\right){\eta }_{\tau }\}.\hfill \end{array}\end{array}$$

The main result of this paper is as follows.

Theorem 1.

Considering the optimization problem (3), constraint (6), and algorithm (8), and supposing that Assumptions 1–4 hold. Let ${\eta }_t=\eta >0$ and $\iota >{\Delta }_7$. Then there exists

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \mathrm{Reg}\left(T\right)\le {\displaystyle \frac{L}{\iota -{\Delta }_7}}\left\{{\Delta }_8\eta T+{\displaystyle \frac{2N{R}^2+NK{C}_T}{\eta }}+{\Delta }_9\right\},\end{array}\end{array}$$

where $C_T={\sum }_{t=1}^T\parallel {\theta }_t\parallel$, ${\Delta }_8=2Q(L+L^2+H)\left\{{\Delta }_5+{\displaystyle \frac{{\Delta }_6}{1-{\sigma }_2\left(W\right)}}\right\}+{\displaystyle \frac{N{(H+HL)}^2}{2}}+{\displaystyle \frac{2N(H+HL){\Delta }_3}{1-{\sigma }_2\left(W\right)}}$, and ${\Delta }_9=2NQ{\Delta }_7+{\Delta }_7{C}_T+2Q{N}^2\gamma P(L+L^2+H){\displaystyle \frac{\beta }{1-\beta }}$.

Remark 4.

The condition $\iota >{\Delta }_7$ is reasonable since a sufficiently small ${\Delta }_7$ is chosen by decreasing $L_R$. It follows from the boundary $\mathrm{Reg}\left(T\right)$ in Theorem 1 that a proper step is ${\eta }_t={\eta }_0/\sqrt{T}$ with ${\eta }_0>0$. In such a case, the regret $\mathrm{Reg}\left(T\right)$ is bounded in the order of $\mathcal{O}\left(\sqrt{T}\right)$ for any arbitrary $T>1$.

5. Simulations

Inspired by [5,8], considering a cooperative control problem for a multi-robot system in a plant $\mathcal{X}\subseteq {\mathbb{R}}^2$, in particular, a moving target with state $\mathcal{T}\in \mathcal{X}$ is modeled as

$${\mathcal{T}}_{t+1}=A_t{\mathcal{T}}_t+{\vartheta }_t^{\circ },$$

(12)

where $A_t\in {\mathbb{R}}^{2\times 2}$ and ${\vartheta }_t^{\circ }\in {\mathbb{R}}^2$ is the noise. Moreover, there is a multi-robot network composed of N robots aiming to protect the target, while there are N intruders aiming to capture the target. Let ${\mathcal{I}}_{i,t}\in \mathcal{X}$ denote the state of the ith intruder at time t, which is assigned to robot i. Supposing that the formation of a multi-robot network is defined by a virtual leader–follower formation method, let $x_t^{\circ }$ denote the leader’s state which is unknown for all robots. In particular, let $x_{i,t}\in \mathcal{X}$ denote the estimated value of $x_t^{\circ }$ by robot i at time t, and the state of robot i is given by ${\psi }_i\left(x_{i,t}\right)={\beta }_i{x}_{i,t}+{\gamma }_i$, where ${\beta }_i$ and ${\gamma }_i$ represent an offset of robot i to the virtual leader $x_{i,t}$.

The multi-robot network is used to protect the moving target by choosing an optimal virtual leader such that the average state of all the robots is close to the target. Meanwhile, each robot is simultaneously close to the associated intruder, referring to Figure 1. Let $v\left(x_t\right)={\displaystyle \frac{1}{N}}{\sum }_{i=1}^N{\psi }_i\left(x_{i,t}\right)$ denote the averaged state of robots where $x_t=\mathrm{col}(x_{1,t},x_{2,t},\dots ,x_{N,t})$. Then, the cooperative control problem can be modeled as the following optimization problem:

$$\begin{array}{c}\hfill \min {.}_{x_{i,t}\in \mathcal{X}}{f}_t\left(x_t\right)=\sum _{i=1}^N{f}_{i,t}(x_{i,t},v\left(x_t\right)),\end{array}$$

(13)

where

$$f_{i,t}(x_{i,t},v\left(x_t\right))={\alpha }_1\parallel v\left(x_t\right)-{\mathcal{T}}_t{\parallel }^2+{\alpha }_2{\parallel {\psi }_i\left(x_{i,t}\right)-{\mathcal{I}}_{i,t}\parallel }^2,$$

${\alpha }_1>0$, and ${\alpha }_2>0$. In fact, the objective is to cooperate in seeking an optimal virtual leader, which is the optimal solution to the problem (13).

It is worth pointing out that the estimated virtual leader should be consensus. Therefore, let $x_t^{\ast }\in \mathcal{X}$ be the consensus solution to problem (13). Then $x_t^{\ast }$ should be the solution to the following online optimization problem:

$$\begin{array}{ccc}\hfill x_t^{\ast }=\min {.}_{x\in \mathcal{X}}{f}_t\left(x\right)& =& \sum _{i=1}^N{f}_{i,t}(x,v\left(x\right)),\hfill \end{array}$$

(14)

where ${\psi }_i\left(x\right)={\beta }_{i}x+{\gamma }_i$ and $v\left(x\right)={\displaystyle \frac{1}{N}}{\sum }_{i=1}^N{\psi }_i\left(x\right)$. In addition, it follows from (12) that $x_t^{\ast }$ should also have the mapping $A_t$, i.e., $x_t^{\ast }$ satisfies the following constraint:

$$x_{t+1}^{\ast }=A_t{x}_t^{\ast }+{\vartheta }_t,$$

(15)

where ${\vartheta }_t\in {\mathbb{R}}^2$ is an unknown and unstructured noise. Note that $A_t$ is unknown for all robots. Hence, $A_{i,t}$ denotes the observed mapping of $A_t$ at time t for robot i. Problem (14) and constraint (15) take the form of problem (3) and constraint (6), so Algorithm 1 can be used to solve problem (14) with constraint (15).

Let $N=4$, $\mathcal{X}=\{x\in {\mathbb{R}}^2|\parallel x\parallel \le 3\}$, $A_t=\left(\begin{array}{cc}1& (1+cost)\ast {10}^{-3}\\ 0& 1\end{array}\right)$, ${\vartheta }_t^{\circ }$ is a random vector chosen from a normal distribution with zero mean and standard deviation $[1;2]\ast {10}^{-3}$, ${\mathcal{I}}_{i,t}={\mathcal{T}}_t+\mathrm{col}(cos(2i\pi /N),sin(2i\pi /N))$, ${\beta }_i=1$, ${\gamma }_i=\mathrm{col}(cos(2i\pi /N),$$sin(2i\pi /N))$, ${\alpha }_1=0.1$, ${\alpha }_2=0.2$, the unknown noise ${\vartheta }_t$ satisfies $\parallel {\vartheta }_t\parallel \le 0.2$, and $A_{i,t}=A_t+{\mu }_t{I}_2$ where ${\mu }_t$ is a random number chosen from a normal distribution with zero mean and standard deviation ${10}^{-3}$.

Choosing $T=1000$, ${\eta }_t=0.001$ and $R\left(x\right)={0.01\parallel x\parallel }^2$, and supposing that an arbitrary doubly stochastic W is given, it can be verified that Assumptions 1–4 and the condition $\iota >{\Delta }_7$ hold. The simulations are performed on a computer equipped with AMD Ryzen 9 5950X 16-core CPU, 64G RAM, and Nvidia RTX 3080Ti GPU. Figure 2 shows the trajectory $x_{1,t}$ of robot 1 by using Algorithm 1. The dynamic regret of Algorithm 1 is indicated in Figure 3, and the regret $\mathrm{Reg}\left(T\right)$ in Figure 3 is bounded. Therefore, Theorem 1 is verified by Figure 2 and Figure 3.

6. Conclusions

This paper introduces an online optimization algorithm for addressing distributed online aggregative problems featuring dynamic environments. With this algorithm, the dynamic regret converges to a boundary without relying on the condition that the dynamic environment is known. Future research includes the distributed online optimization problems in the context of time-varying directed networks.