Communication-Efficient Zeroth-Order Adaptive Optimization for Federated Learning

Abstract

Federated learning has become a prevalent distributed training paradigm, in which local devices collaboratively train learning models without exchanging local data. One of the most dominant frameworks of federated learning (FL) is FedAvg, since it is efficient and simple to implement; here, the first-order information is generally utilized to train the parameters of learning models. In practice, however, the gradient information may be unavailable or infeasible in some applications, such as federated black-box optimization problems. To solve the issue, we propose an innovative zeroth-order adaptive federated learning algorithm without using the gradient information, referred to as ZO-AdaFL, which integrates the zeroth-order optimization algorithm into the adaptive gradient method. Moreover, we also rigorously analyze the convergence behavior of ZO-AdaFL in a non-convex setting, i.e., where ZO-AdaFL achieves convergence to a region close to a stationary point at a speed of $O(1/T)$ (T represents the total iteration number). Finally, to verify the performance of ZO-AdaFL, simulation experiments are performed using the MNIST and FMNIST datasets. Our experimental findings demonstrate that ZO-AdaFL outperforms other state-of-the-art zeroth-order FL approaches in terms of both effectiveness and efficiency.

1. Introduction

In the last few years, machine learning has received a lot of significant attention and enthusiasm. Deep learning especially has been successful in a range of applications, including ChatGPT, which is a notable example [1]. In deep learning, deep neural network models can be trained by using numerous data. Nevertheless, most of the data are generated and owned by personal devices like smartphones and PCs. The conventional method is to collect and transmit data to a central point, such as a data center. However, the data may leak the privacy of users during the process of transmission. Hence, it is difficult to train machine learning models by transmitting data from all users to the central server (CS), since the privacy and security of data need to be protected [2,3,4]. To mitigate this issue, FL is proposed in [5,6], where the raw data of the participating devices are kept locally; they participate in global machine learning model training in a collaborative manner by uploading local models instead of uploading data directly to the CS.

Because of its superiority, recently, a variety of optimization algorithms have been devoted to FL. One of the most popular FL approaches is the FedAvg algorithm [5]. Moreover, its variants have also been effectively discussed and understood recently [7,8,9]. Nonetheless, the aforementioned algorithms update the model parameters using stochastic gradient descent (SGD), which results in slow training and suboptimal performance. For this reason, adaptive gradient methods [10,11] were introduced in [12], which presents FedYogi and FedAdaGrad, respectively. Meanwhile, their convergence behaviors were also analyzed rigorously. Despite this progress, however, these algorithms need to have access to the gradient information during the training process. Nevertheless, this information might be unavailable in some real-world scenarios such as federated black-box optimization [13] and federated hyperparameter tuning [14]. Therefore, existing federated adaptive gradient methods are unsuitable for these scenarios.

One of the most popular gradient-free methods is the zeroth-order optimization algorithm, which has been successfully used in handling a variety of problems in the real world. The gradient information is expensive or infeasible, but the computation of objective functions is only available in these scenarios. Such examples include the black-box adversarial attack on structured prediction [15], reinforcement learning [16], and deep neural networks (DNNs) [17,18,19]. Due to its success, zeroth-order optimization has made great progress in recent years. Nesterov and Spokoiny [20] introduced a zeroth-order degree decrease (ZO-GD) algorithm by using a two-point Gaussian stochastic gradient estimator. Moreover, each gradient approximation requires only $O\left(1\right)$ different function values. To improve the performance of zeroth-order optimization, a zeroth-order stochastic variance reduction algorithm known as ZO-SVRG was presented by Liu et al. [21]. Soon after, the ZO-AdaMM method was introduced in [22] as a zeroth-order adaptive momentum method for black-box optimization. The aim of this method is to reduce the large variance caused by the gradient estimator. Furthermore, an analysis of the convergence performance of ZO-AdaMM was also provided. The aforementioned efforts were made for the centralized machine learning framework. Nevertheless, the design and analysis of the zeroth-order optimization algorithm for the federated learning framework are still limited. Very recently, Fang et al. [23] presented the FedZO algorithm; several years ago, a federated zeroth-order optimization algorithm was presented, in which the zeroth-order gradient is integrated into federated learning. The SGD-based update rule is used in FedZO, which suffers from slow training speed. However, algorithmic and theoretical developments for federated adaptive zeroth-order optimization are still barely understood, to our knowledge.

Aspiring to remedy this gap in the research, this paper develops a communication efficient zeroth-order adaptive federated learning, referred to as ZO-AdaFL, under a non-convex setting. In each round of communication, ZO-AdaFL adopts the stochastic gradient estimator for multiple local model updates. Then, the adaptive gradient method is used to achieve a global update without first-order information. Meanwhile, multiple local iterations are implemented at each round of communication in ZO-AdaFL, which can reduce the communication overheads and is suitable in dealing with federated learning with numerous participants. Moreover, we also provide a theoretical guarantee for ZO-AdaFL for non-convex objective functions. This paper’s main contributions are summarized in the following points:

• We propose an algorithm for federated learning that aims to enhance communication efficiency, called zeroth-order adaptive federated learning (ZO-AdaFL), which inherits the framework of the FedAdam algorithm. Moreover, ZO-AdaFL only queries the values of objective functions without using gradient or Hessian information.
• We establish the convergence analysis of the proposed ZO-AdaFL algorithm under the non-convex setting. In particular, we prove that ZO-AdaFL achieves a convergence rate of $O(1/T)$ with K local iterations and full device participation, where T represents the total number of iterations.
• We conduct various experiments to validate the performance of ZO-AdaFL. The experimental results show that ZO-AdaFL is effective and efficient compared with state-of-the-art federated zeroth-order learning methods.

The remaining work is organized as shown below. Section 2 introduces related work. We propose zeroth-order adaptive federated optimization in Section 3. In Section 4, we present the convergence analysis. We carefully verify our theoretical analysis through experiments in Section 5. We present the conclusion of this paper and discuss future work in Section 6. The notation interpretation is shown in

Table 1. Notation summary table.

Notation	Definition
m	The number of clients.
${\mathbb{M}}_t$	The subset of randomly selected participating clients.
$\left|{\mathbb{M}}_t\right|$	The number of randomly selected participating clients.
$\left|Y\right|$	The size of the data point subset owned by the i-th client.
${\mathcal{D}}_i,\left|{\mathcal{D}}_i\right|$	The dataset and dataset size of client i.
$f_{\mu }\left(\mathbf{x}\right)$	Client’s loss function.
d	Dimension of the model parameters.
$\parallel ·\parallel$	The ${𝓁}_2$ norm of vectors.
${(·)}^T$ and ${(·)}^H$	The transpose and the conjugate transpose.
$\mathbf{m}$ and $\mathbf{v}$	The first-order and second-order momentum parameters.
$\widehat{\nabla }F$	Unbiased gradient estimator.
${\eta }_l,\alpha$	Stands for the local step and the global step, respectively.
$K,T$	Represents the number of local iterations and the total number of global iterations, respectively.

.

Throughout this paper, we use $\mathbb{R}$ and $\mathbb{E}[·]$ to denote the real number set and the expectation operator, respectively. Scalars, vectors, and matrices are denoted by regular letters, bold lower-case letters, and bold upper-case letters, respectively. The notation $diag\left(\mathbf{x}\right)$ is used to represent a diagonal matrix with the diagonal entries determined by $\mathbf{x}$.

2. Related Work

In this section, we give a brief overview of related work on SGD, adaptive gradient methods, federated learning, zeroth-order optimization, respectively.

2.1. SGD and Adaptive Gradient Methods

SGD [24] is a popular method for training machine learning models, but it can be sensitive to parameter settings and slow to converge when dealing with heavy-tailed stochastic gradient noise. To address these issues, adaptive gradient methods like AdaGrad [25], RMSProp [26], and AdaDelta [27] can be used. Adam [10] and its variant AMSGrad [11] have been proposed. These methods are widely used in training deep neural networks and other variants [28,29,30] also have a crucial role in enhancing various aspects of the adaptive gradient method.

2.2. Federated Learning

FL is an innovative machine learning approach first mentioned by Google [5] in 2016 to address issues such as data privacy protection, security, and centralized processing bottlenecks. The technique allows data to be stored and models to be trained on local devices, but it also raises challenges in terms of communication efficiency, privacy protection, and device selection. To address these challenges effectively, FL [6] has now attracted considerable attention from the research community. Federated averaging (FedAvg) [5], as a classical framework, has strongly contributed to the rapid development of the field of FL through the periodic SGD updating of averages. Stich [31] provided a concise theoretical convergence guarantee for local SGD. Following the FedAvg algorithm, numerous other first-order optimization algorithms have been presented, e.g., FedProx [8], FedNova [7], and SCAFFOLD [32]. Furthermore, Reddi et al. [12] have also introduced a few adaptive federated optimization approaches in recent years, such as FedAdagrad, FedYogi, and FedAdam. These optimization approaches aim to yield even faster convergence to address the convergence problems of FedAvg. However, the mainstream first-order optimization algorithms have the following problems: the learning rate is too low, resulting in the too-slow convergence of the loss function; the learning rate is too high, which may affect the convergence and lead to fluctuations in the loss function on the minimum value, or even divergence, and is more sensitive to the parameters; the loss function is prone to converging to the local optimum, and it is difficult to jump out of the saddle point. To further reduce the communication overhead, several second-order optimization algorithms were proposed, such as GIANT [33], FedDANE [34], FedNL [35], and FedNew [36]. The second-order optimization algorithm accelerates the first-order gradient descent by using the curvature correction of the second-order derivative of the objective function. Compared to the first-order optimizer, it converges faster, highly approximates the optimal value, and the descent path is geometrically more consistent with the true optimal descent path. Although first-order and second-order optimization algorithms have made significant contributions to federated learning, they may not be suitable for scenarios where derivative or Hessian information is unavailable or large-scale datasets need to be trained. In such cases, zeroth-order optimization algorithms can be more effective. Zeroth-order optimization algorithms are advantageous due to their computational efficiency, independence from the gradient problem, and ability to adapt to non-smooth and discrete objective functions. Therefore, using zeroth-order optimization algorithms in these situations can be more efficient and effective.

2.3. Zeroth-Order Optimization

The ZO algorithm commonly uses a gradient estimator, such as the one-point or two-point gradient estimator, to approximate the full-gradient. The one-point gradient estimator estimates the gradient $\widehat{\nabla }f\left(\mathbf{x}\right)$ by evaluating the function $f(·)$ at a random nearby location to the point $\mathbf{x}$ [37,38]. On the other hand, the two-point gradient estimator uses two stochastic function queries to compute the finite difference [20,39]. ZO stochastic gradient descent (ZO-SGD) [40] and ZO stochastic coordinate descent (ZO-SCD) [41] are able to converge quickly in unconstrained stochastic optimization problems with a convergence rate of $O(\sqrt{d}/\sqrt{T})$; the number of optimization variables is d with T iterations. To enhance the iteration complexity of ZO algorithms, the variance reduction technique has been applied to ZO-SGD and ZO-SCD, resulting in stochastic-variation-reduced ZO algorithms, with an enhanced convergence rate in T, i.e., $O(d/T)$ [21,42,43]. Several recent works [13,44] have concentrated on the study of distributed zeroth-order optimization. In particular, the authors of [44] have developed a ZONE algorithm using the primal–dual technique. However, ZONE requires $O\left(T\right)$ sampling complexity per iteration. More comprehensive discussion on distributed zeroth-order optimization methods can be found in [45,46].

3. Zeroth-Order Adaptive Federated Optimization

Most federated optimization algorithms that utilize SGD to update the model have loss function values that are slow to change in some dimensions. They tend to fall into local minima and at saddle points in high-dimensional spaces, which leads to larger variance values of the loss function and hinders the algorithm’s convergence speed. In this paper, we devote much attention to two-point gradient estimators that aim to make the variance of the loss function smaller, thus speeding up the convergence of the algorithm and improving the complexity bounds of the ZO algorithms. Firstly, we analyze the non-convex optimization problems of FL; secondly, we propose an FL optimization algorithm called ZO-AdaFL. An overview flowchart of the ZO-AdaFL algorithm is shown in Figure 1.

3.1. Problem Formulation

In FL, general FL algorithms have difficulty solving complex optimization models without gradients or where obtaining gradients is costly. This scenario occurs in many real-world applications, including—but not limited to—federal black-box attacks on DNNs [13]. The objective function in FL is typically non-convex due to the heterogeneity of client data. Furthermore, the problem of federated black-box attacks on DNN models is non-convex. Meanwhile, considering the practical application scenario of large-scale device participation and black-box attacks in FL, this paper aims to study the non-convex optimization problem in FL; the specific problem is described here:

$$\underset{\mathbf{x}\in {\mathbb{R}}^d}{min}f\left(\mathbf{x}\right)=\frac{1}{m}\sum _{i=1}^m{f}_i\left(\mathbf{x}\right),$$

(1)

where m is the total number of clients, $\mathbf{x}\in {\mathbb{R}}^d$ represents the model parameter with dimension d, and $f\left(\mathbf{x}\right)$ represents the global loss function on the CS. In Equation (1), $f_i\left(\mathbf{x}\right)$ is a local non-convex loss function and is a measure of the expectation of risk above locally distributed data, denoted by $f_i\left(\mathbf{x}\right)={\mathbb{E}}_{\xi \sim {\mathcal{D}}_i}\left[F_i(\mathbf{x};\xi )\right],$ where $\xi \sim {\mathcal{D}}_i$ represents a random variable, $\xi$, uniformly sampled from ${\mathcal{D}}_i$; $F_i(\mathbf{x};\xi )$ denotes the loss relative to $\xi$ which is evaluated at the model parameter $\mathbf{x}$.

In the FedAvg, an optimization solution is proposed for the problem involved in problem Equation (1). During the t-th iteration, the CS provides client i in the chosen subset $M_t$ with the model ${\mathbf{x}}_t$. Using the learning rate ${\eta }_l$, the client i performs the K steps of the SGD update locally to obtain the local model ${\mathbf{x}}_{t,K}^i$. Each client can perform model updates based on local data and send the model parameter differences ${\Delta }_t^i={\mathbf{x}}_{t,K}^i-{\mathbf{x}}_t$ back to the CS. Then, the differences in the global model, denoted as ${\Delta }_t$, are updated by the CS through the simple averaging of the local model differences, ${\Delta }_t^i$. The global model ${\mathbf{x}}_{t+1}$ is updated using the function ${\mathbf{x}}_{t+1}={\mathbf{x}}_t+{\Delta }_t$, which is the same approach as that taken for the direct average local model ${\mathbf{x}}_{t,K}^i$, i.e., ${\mathbf{x}}_{t+1}={\mathbf{x}}_t+\frac{1}{n}{\sum }_{i\in S_t}({\mathbf{x}}_{t,K}^i-{\mathbf{x}}_t)=\frac{1}{n}{\sum }_{i\in S_t}{\mathbf{x}}_{t,K}^i$.

FedAdam was then proposed among several adaptive optimization methods in federated learning [12]. FedAdam replaces FedAvg’s SGD approach by introducing the Adam optimizer as a global update rule that can better accommodate different local variances ${\Delta }_t^i$ and average them out to ${\Delta }_t$. The CS then uses the Adam optimizer to perform an accurate and fast update of the global model based on this global variance ${\Delta }_t$:

$${\mathbf{m}}_t={\beta }_1{\mathbf{m}}_{t-1}+(1-{\beta }_1){\Delta }_t,$$

(2)

$${\mathbf{v}}_t={\beta }_2{\mathbf{v}}_{t-1}+(1-{\beta }_2){\Delta }_t^2,$$

(3)

$${\mathbf{x}}_{t+1}={\mathbf{x}}_t+{\alpha }_t\frac{{\mathbf{m}}_t}{\sqrt{{\mathbf{v}}_t}+ϵ},$$

(4)

where ${\Delta }_t$ serves as a pseudo gradient, and the global update can be considered as a single-step Adam update that uses ${\Delta }_t$. There have also been a number of algorithm variants proposed that make subtle modifications to the variance term ${\mathbf{v}}_t$, including FedAdagrad [12], FedYogi [12] and FedAMSGrad [47]. In order to prevent the ${\mathbf{v}}_t$ term from becoming too small and causing instability in the optimization process, stability is ensured by adding $ϵ$ to Equation (4).

3.2. ZO-AdaFL Algorithm

As shown in Algorithm 1, this section proposes a general zeroth-order algorithm for an adaptive federated optimization framework. The ZO-AdaFL algorithm aims to eliminate gradient dependence and focuses on reducing the model exchange frequency. To achieve these goals, in each round of communication, the ZO-AdaFL algorithm cleverly employs a gradient estimator and performs a K-step stochastic zeroth-order updating strategy. Each round of our ZO-AdaFL algorithm has the following four parts.

Broadcast Global Model: The CS chooses any collection of clients from the M clients for training at the beginning of round t, denoted as ${\mathbb{M}}_t$. Subsequently, each client in collection ${\mathbb{M}}_t$ obtains a current global model parameter ${\mathbf{x}}_t$ in a broadcast from the CS.

Local Model Update: Each selected client, $i\in {\mathbb{M}}_t$, then trains its own local model parameters, ${\mathbf{x}}_{t,K}^i$, using K-steps of stochastic zeroth-order updating at a learning rate of ${\eta }_l$. Specifically, at the k-th iteration of the t-th round, client i computes a two-point stochastic gradient estimator [45], as shown below.

$$\widehat{\nabla }{F}_i({\mathbf{x}}_{t,k}^i;{\xi }_{t,k}^i)=(d/{\mu }_t)[F_i({\mathbf{x}}_{t,k}^i+{\mu }_t{\mathbf{u}}_{t,k};{\xi }_{t,k}^i)-F_i({\mathbf{x}}_{t,k}^i;{\xi }_{t,k}^i)]{\mathbf{u}}_{t,k},\forall i\in \left[m\right],$$

(5)

where ${\mathbf{x}}_{t,k}^i$ denotes the local model of client i, and ${\xi }_{t,k}^i$ is a personalized update of client i in each iteration. Specifically, ${\xi }_{t,k}^i$ is a random variable obtained by sampling a local data distribution ${\mathcal{D}}_i$ of client i at the k-th iteration of the t-th round. ${\mathbf{u}}_{t,k}$ is a random direction vector obtained by evenly sampling on the unit sphere ${\mathbb{S}}^d$ of dimension d. ${\mu }_t$ denotes the magnitude of the movement in that random direction, i.e., the positive step size.

Afterwards, client i performs a stochastic zeroth-order update to update its local model, as follows:

$${\mathbf{x}}_{t,k+1}^i={\mathbf{x}}_{t,k}^i-{\eta }_l\widehat{\nabla }{F}_i({\mathbf{x}}_{t,k}^i;{\xi }_{t,k}^i),k\in \left[K\right],$$

(6)

where ${\eta }_l$ represents the learning rate. Client i obtains a new local model parameter ${\mathbf{x}}_{t,K}^i$ through a total of k iterations.

Local Model Uploading: After the local training is completed, all of the clients from ${\mathbb{M}}_t$ obtain the updated part of the local model parameters, i.e., ${\Delta }_t^i={\mathbf{x}}_{t,K}^i-{\mathbf{x}}_t,i\in {\mathbb{M}}_t$. Then, they upload them to the CS.

Global Model Update: The CS aggregates ${\Delta }_t^i$ to obtain ${\Delta }_t$ after it receives the local model update from the selected client i, i.e., ${\Delta }_t=\frac{1}{|{\mathbb{M}}_t|}{\sum }_{i\in {\mathbb{M}}_t}{\Delta }_t^i$. ${\Delta }_t$ acts as a pseudo gradient to calculate momentum ${\mathbf{m}}_t$ and variance ${\mathbf{v}}_t$, following Equations (2) and (3). Subsequently, the CS obtains the new global model, that is, ${\mathbf{x}}_{t+1}={\mathbf{x}}_t+{\alpha }_t\frac{{\mathbf{m}}_t}{\sqrt{{\widehat{\mathbf{v}}}_t}+ϵ}$, where ${\widehat{\mathbf{v}}}_t=max({\widehat{\mathbf{v}}}_{t-1},{\mathbf{v}}_t)$. Note that this is the same as the AMSGrad update rule [11], which addresses Adam’s non-convergence problem [10] by using non-decreasing ${\mathbf{v}}_t$.

Algorithm 1 Zeroth-Order Adaptive Optimization for Federated Learning (ZO-AdaFL)

Require: initial model ${\mathbf{x}}_1$, local step size ${\eta }_l$, global step size ${\alpha }_t$, smoothing parameter $\mu >0$, ${\beta }_1,{\beta }_2,ϵ$.   1: ${\mathbf{m}}_0\leftarrow 0,{\mathbf{v}}_t\leftarrow 0$   2: for $t=1,2,\dots ,T$ do   3:    Randomly sample a subset ${\mathbb{M}}_t$ from m.   4:    The CS broadcasts the global model ${\mathbf{x}}_t$ to the subset ${\mathbb{M}}_t$ of clients   5:    ${\mathbf{x}}_{t,0}^i={\mathbf{x}}_t$   6:    for every client $i\in {\mathbb{M}}_t$ in parallel do   7:      for $k=0,1,\dots ,K-1$ do   8:         Compute local stochastic gradient estimator: ${\widehat{\mathbf{g}}}_{t,k}^i=\widehat{\nabla }{F}_i({\mathbf{x}}_{t,k}^i;{\xi }_{t,k}^i)$ by Equation (5)   9:         Perform local update: ${\mathbf{x}}_{t,k+1}^i={\mathbf{x}}_{t,k}^i-{\eta }_l{\widehat{\mathbf{g}}}_{t,k}^i$ 10:      end for 11:      Compute the local model updates: ${\Delta }_t^i={\mathbf{x}}_{t,K}^i-{\mathbf{x}}_t$ 12:    end for 13:    The CS aggregates the local model updates: ${\Delta }_t=\frac{1}{|{\mathbb{M}}_t|}{\sum }_{i\in {\mathbb{M}}_t}{\Delta }_t^i$ 14:    Update ${\mathbf{m}}_t={\beta }_1{\mathbf{m}}_{t-1}+(1-{\beta }_1){\Delta }_t$ 15:    Update ${\mathbf{v}}_t={\beta }_2{\mathbf{v}}_{t-1}+(1-{\beta }_2){\Delta }_t^2$ 16:    Option: ${\widehat{\mathbf{v}}}_t=max({\widehat{\mathbf{v}}}_{t-1},{\mathbf{v}}_t)$, update ${\mathbf{x}}_{t+1}={\mathbf{x}}_t+{\alpha }_t\frac{{\mathbf{m}}_t}{\sqrt{{\widehat{\mathbf{v}}}_t}+ϵ}$ 17: end for

4. Convergence Analysis

The presentation in this section is about the theoretical convergence results of ZO-AdaFL. We focus on the setting of non-convex loss functions. In order to perform the convergence analysis, we need some assumptions. These assumptions are presented below.

Assumption 1

(Smoothness). Each loss function on the i-th worker $F_i\left(\mathbf{x}\right)$ is L-smooth, i.e., $\forall \mathbf{x},\mathbf{y}\in {\mathbb{R}}^d$,

$$\begin{array}{c}\hfill |F_i\left(\mathbf{x}\right)-F_i\left(\mathbf{y}\right)-\langle \nabla F_i\left(\mathbf{y}\right),\mathbf{x}-\mathbf{y}\rangle |\le \frac{L}{2}{\parallel \mathbf{x}-\mathbf{y}\parallel }^2.\end{array}$$

Assumption 1 also implies the L-gradient Lipschitz condition, i.e., $\parallel \nabla F_i\left(\mathbf{x}\right)-\nabla F_i\left(\mathbf{y}\right)\parallel \le L\parallel \mathbf{x}-\mathbf{y}\parallel$. Assumption 1 is a standard assumption in non-convex optimization problems, which has been also adopted in [10,11,48,49].

Assumption 2

(Bounded Gradient). Each loss function on the i-th worker $F_i\left(\mathbf{x}\right)$ has $G_{zo}$ bounded stochastic gradient on ${𝓁}_2$, i.e., for all ξ, we have $\parallel \nabla F_i(\mathbf{x},\xi )\parallel \le G_{zo}$.

The assumption of bounded gradient is usually adopted in adaptive gradient methods [10,11,28,50].

Assumption 3

(Unbiasedness). The stochastic gradient $\nabla F_i(\mathbf{x},\xi )$ is an unbiased estimate of $\nabla f_i\left(\mathbf{x}\right)$, i.e.,

$$\begin{array}{c}\hfill {\mathbb{E}}_{\xi }[\nabla F_i(\mathbf{x},\xi )]=\nabla f_i\left(\mathbf{x}\right),\forall \mathbf{x}\in {\mathbb{R}}^d,\forall i\in \left[m\right].\end{array}$$

Assumption 4

(Bounded Variance). Each stochastic gradient on the i-th worker has a bounded local variance, i.e., for all $\mathbf{x},i\in \left[m\right]$, we have $\mathbb{E}\left[\parallel \nabla f_i(\mathbf{x},\xi )-\nabla F_i{\left(\mathbf{x}\right)\parallel }^2\right]\le {\sigma }_l^2$, and the loss function on each worker has a global variance bound, $\frac{1}{m}{\sum }_{i=1}^m{\parallel \nabla F_i\left(\mathbf{x}\right)-\nabla f\left(\mathbf{x}\right)\parallel }^2\le {\sigma }_h^2$, where ${\sigma }_l$ and ${\sigma }_h$ represent the local stochastic gradient variance and global variance, respectively.

Assumption 4 is commonly applied in FL optimization issues [12,48,49]. The bounded local variance reflects the stochastic gradient stochasticity for each client, and the bounded global variance reflects the heterogeneity of the dataset across devices. When the value of the local variance is 0 (i.e., ${\sigma }_h=0$), it represents a setting where the dataset for each client has the same distribution, i.e., independent and identically distributed (i.i.d.).

Here, we show the convergence results of ZO-AdaFL using an all-hands-on-deck scheme, where each client participates in the communication round and the model update, denoted as $|{\mathbb{M}}_t|=m,\forall t\in \left[t\right]$. Before proving the convergence results of ZO-AdaFL, several auxiliary lemmas are introduced.

Lemma 1.

Under Assumption 1, using the ZO-AdaMM update rule [22], we have

$$\begin{array}{cc}\hfill \mathbb{E}[f_{\mu }\left({\mathbf{z}}_{t+1}\right)-f_{\mu }\left({\mathbf{z}}_1\right)]& \le \sum _{t=1}^T\mathbb{E}\left[\langle \nabla f_{\mu }\left({\mathbf{x}}_t\right),{\mathbf{z}}_{t+1}-{\mathbf{z}}_t\rangle \right]\hfill \\ \hfill & \phantom{=}+\frac{4L_g+5L_g{\beta }_1^2}{2{(1-{\beta }_1)}^2}\sum _{t=1}^T\mathbb{E}\left[{\parallel {\mathbf{x}}_{t+1}-{\mathbf{x}}_t\parallel }^2\right].\hfill \end{array}$$

(7)

Proof. 

See Appendix A. □

Lemma 2.

Suppose Assumptions 1–4 hold. Then, according to the ZO-AdaMM update rule [22], for the first term of Lemma 1, we have

$$\begin{array}{cc}\hfill & \sum _{t=1}^T\mathbb{E}\left[\langle \nabla f_{\mu }\left({\mathbf{x}}_t\right),{\mathbf{z}}_{t+1}-{\mathbf{z}}_t\rangle \right]\hfill \\ \hfill & \le \frac{{\alpha }_t{\beta }_1{\eta }_{l}K{G}_{zo}^2}{1-{\beta }_1}\sum _{t=1}^T\mathbb{E}\left[\parallel {\widehat{\mathbf{V}}}_t^{-1/2}-{\widehat{\mathbf{V}}}_{t-1}^{-1/2}{\parallel }_1\right]\hfill \\ & \phantom{=}-\frac{3\sqrt{2}{\alpha }_t{\eta }_l^3{L}^{2}d{K}^3-\sqrt{2}{\alpha }_t{\eta }_{l}K}{2}\sum _{t=1}^T\mathbb{E}\left[\parallel \frac{\nabla f_{\mu }\left({\mathbf{x}}_t\right)}{\sqrt[4]{{\beta }_2{\mathbf{v}}_{t-1}+ϵ}}{\parallel }^2\right]\hfill \\ \hfill & \phantom{=}-\frac{{\alpha }_t{\eta }_l^3{L}^{2}d{K}^{3}T}{\sqrt{2ϵ}}({\sigma }_l^2+3{\sigma }_h^2)-\frac{{\alpha }_t{\eta }_l^3{K}^3{\mu }^2{d}^2{L}^{4}T}{4\sqrt{2ϵ}}\hfill \\ \hfill & \phantom{=}-\frac{{\alpha }_t\sqrt{2(1-{\beta }_2)}{G}_{zo}}{ϵ}\sum _{t=1}^T\mathbb{E}[\parallel {\Delta }_t{\parallel }^2]\hfill \\ \hfill & \phantom{=}+\frac{\sqrt{2}{\alpha }_t{\eta }_l}{2K{m}^2}\sum _{t=1}^T\mathbb{E}\left[\parallel \frac{1}{\sqrt[4]{{\beta }_2{\mathbf{v}}_{t-1}+ϵ}}\sum _{i=1}^m\sum _{k=0}^{K-1}\nabla f_{\mu }^i\left({\mathbf{x}}_{t,k}^i\right){\parallel }^2\right].\hfill \end{array}$$

(8)

Proof. 

See Appendix B. □

Lemma 3.

Then, with Assumptions 1–4 and ${\eta }_l\le \frac{1}{3KL\sqrt{d}}$, for the second term of Lemma 1, we have

$$\begin{array}{cc}\hfill & \frac{4L+5L{\beta }_1^2}{2(1-{\beta }_1^2)}\sum _{t=1}^T\mathbb{E}[\parallel {\mathbf{x}}_{t+1}-{\mathbf{x}}_t{\parallel }^2]\hfill \\ \hfill & \le \sum _{t=1}^T{\alpha }_t^{2}d\frac{4L+5L{\beta }_1^2}{2(1-{\beta }_1^2)}\frac{1-{\beta }_1}{1-{\beta }_2}\frac{1}{1-\gamma }.\hfill \end{array}$$

(9)

Proof. 

The proof follows from Lemma 2.4 in [22]. □

Theorem 1.

Suppose Assumptions 1–4 hold. If the local learning rate satisfies ${\eta }_l\le min\left\{\frac{1}{3KL\sqrt{d}},\right.\left.\frac{ϵ}{2K\sqrt{2}{\alpha }_t{\eta }_l\sqrt{1-{\beta }_2}{G}_{zo}}\right\}$, then, under the condition of the full participation of the devices, the iterates of ZO-AdaFL in Algorithm 1 satisfy

$$\begin{array}{cc}\hfill & \mathbb{E}[\parallel \nabla f_{\mu }\left({\mathbf{x}}_t\right){\parallel }^2]\le 2\sqrt{{\beta }_2{\eta }_l^2{K}^2{G}^2+ϵ}\left[\frac{f_{\mu }^0-f_{\mu }^{*}}{AT}+\frac{\Phi }{T}-\Psi \right],\hfill \end{array}$$

(10)

where $A=\sqrt{2}(3{\eta }_l^2{L}^{2}d{K}^2-1)$, $\Phi =\frac{{\beta }_1{G}_{zo}^{2}d}{(1-{\beta }_1)\sqrt{ϵ}}$, and $\Psi =\frac{1}{A}\left[\frac{{\eta }_l^2{L}^{2}d{K}^2}{\sqrt{2ϵ}}({\sigma }_l^2+3{\sigma }_h^2)+\frac{{\eta }_l^2{K}^2{\mu }^2{d}^2{L}^4}{4\sqrt{2ϵ}}+\frac{\sqrt{2(1-{\beta }_2)}{G}_{zo}{\eta }_l{\sigma }_l^2}{mϵ}-{\alpha }_{t}d\frac{4L+5L{\beta }_1^2}{2{\eta }_{l}K(1-{\beta }_1^2)}\frac{1-{\beta }_1}{1-{\beta }_2}\frac{1}{1-\gamma }\right]$. Moreover, as the communication rounds increase, ZO-AdaFL converges to the neighborhood of a solution.

Proof of Theorem 1.

See Appendix C. □

Remark 1.

Equation (10) shows that the upper bound of the minimum gradient squared ${min}_{t\in \left[T\right]}\mathbb{E}[\parallel \nabla f\left({\mathbf{x}}_t\right){\parallel }^2]$ in the global model sequence is closely related to the total number of steps T, and it decreases as $T\to \infty$. Additionally, the convergence rate of the algorithm depends on ${\sigma }_l$ and ${\sigma }_h$. In the scenario where each worker has the same data distribution (i.e., i.i.d.), the global variance is zero (${\sigma }_h=0$), and the variance term Ψ will be reduced, relying less on the number of local steps K.

Remark 2.

Note that, compared with FedAdam [12], our theoretical analysis of ZO-AdaFL improves upon previous work by providing a comprehensive analysis of ZO-AdaFL with a non-zero momentum term. In contrast, the analysis in reference [12] only focuses on the scenario where ${\beta }_1=0$.

5. Simulation Results

In this section, we introduce the results of our simulated experiments, evaluating the performance of the ZO-AadFL algorithm, which is proposed for federated black-box attacks. We verify the effectiveness of our ZO-AadFL under federated black-box attacks and the faster convergence speed of ZO-AadFL with the involvement of large-scale devices through different experimental setups.

5.1. Federated Black-Box Attack

Deep-learning-based image classification algorithms are mostly trained under carefully made datasets. For images outside the dataset or slightly modified images, the recognition ability of the network is often affected to some extent. Under this phenomenon, adversarial attacks begin to be included in the examination of network model robustness. By adding different noises or transforming certain areas of the image to generate antagonistic samples, the samples attack the network model to achieve the purpose of confusing the network. Since the black-box possesses the characteristic of not being able to directly observe the gradient information of the function, we need to solve the black-box attack optimization problem with zeroth-order optimization. We consider a well-trained DNN classifier model with a testing accuracy of $99.4\%$ on the standard MNIST dataset [13], and let this model simulate a federated black-box model. The federated black-box attack aims to jointly produce a universal adversarial perturbation that makes the perturbation image visually imperceptible, but this can mislead the classifier.

5.2. Experiment Setup

In this experiment setting, we randomly selected 200 samples from the training set of digit class “4”, and then distributed $|{\mathcal{D}}_i|=60$ samples to each client i. The number of clients was set to $N=50$. Moreover, ${\mathbf{v}}_0={\widehat{\mathbf{v}}}_0={10}^{-5}$ and ${\mathbf{m}}_0=0$, ${\beta }_1=0.9$, ${\beta }_2=0.99$, $ϵ={10}^{-8}$. The learning rate and step size were set to ${\alpha }_t=0.02$ and ${\eta }_l=0.001$, respectively.

5.3. Experimental Results and Analysis

In the experiment, we observed the influence of a different number of local iterations and the number of participating clients on the convergence of ZO-AdaFL algorithm. The results are shown in Figure 2.

In Figure 2a, we demonstrate the effect that the number of local iterations has on the convergence performance of ZO-AadFL when the total number of clients involved $m=50$. Specifically, the relationship between the black-box attack loss and the number of communication rounds is presented by changing the number of local iterations $K\in \{10,50,100\}$. As shown in Figure 2a, as K increases, the attack loss of the ZO-AadFL algorithm decreases and the convergence rate tends to accelerate. It is worth noting that the ZO-AadFL algorithm converges to different losses for different K; this is because, with the non-convex nature of the federated black-box attack problem on DNN models, multiple saddle points exist. As a result, the ZO-AadFL algorithm may become trapped in any of these saddle points. In addition, Figure 2b shows the black-box attack losses of ZO-AadFL for different numbers of participating clients when the number of local iterations is $K=50$. Specifically, the relationship between attack losses and communication rounds is presented by changing the number of participating clients $m\in \{10,30,50\}$. It can be observed that the ZO-AadFL is able to effectively reduce attack losses and produce a better convergence effect with the increase in m, because the more clients participate in training, the higher the precision of model training.

We further verified the effectiveness of the proposed ZO-AdaFL algorithm, and compared the performance of our algorithm with those of three baseline algorithms—i.e., the DZOPA [13], FedZO [23], and ZONES [44] algorithms—on training MNIST and FMNIST datasets, respectively, with the CNN model. The results are depicted in Figure 3. We compared communication rounds and the black-box attack loss for several algorithms on each dataset.

For the MNIST experiments, Figure 3a shows the convergence result of ZO-AdaFL and other ZO optimization baselines. In the initial stage, the attack loss of each algorithm is high. As the number of communication rounds increases, the attack loss of all algorithms gradually decreases. Furthermore, our ZO-AdaFL algorithm’s loss decreases at a faster rate and ultimately reaches the lowest loss value. Note that—although DZOPA and FedZO have similar final performances—the DZOPA algorithm does not perform as well as FedZO when the number of communication rounds is low. Throughout the experiment, although the attack loss of the ZONES algorithm is always kept at a relatively lower level than that of FedZO, it is more volatile and less stable. Compared with the FedZO algorithm, the ZO-AadFL algorithm reduces the attack loss faster and has better convergence in the first 100 rounds. Based on the comparative analysis of the experimental results of the FedZO algorithm and the ZO-AadFL algorithm, we find that the loss of the ZO-AadFL algorithm is smaller than that of the FedZO algorithm, both in the initial and final stages. In contrast, ZO-AadFL significantly outperforms the other three baselines in terms of final black-box attack loss. This suggests that ZO-AadFL can be an ideal solution for zeroth-order optimization algorithms in the absence of first-order information.

For FMNIST experiments, Figure 3b shows the convergence result of ZO-AdaFL and other ZO optimization baselines. Obviously, our ZO-AadFL algorithm converges with the number of algorithmic communication rounds grown. Furthermore, the final black-box attack loss performance of ZO-AadFL algorithm is better than FedZO and DZOPA algorithms. However, compared to the ZONES algorithm, our algorithm has a higher attack loss rate. To further analyze the reasons for this phenomenon, we provide additional experimental results about the attack success rate and the distortion loss against 600 communication rounds in

Table 2. Summary of attack success rate and distortion loss for universal attack against under $T=600$ iterations on the FMNIST dataset.

Methods	DZOPA [13]	FedZO [23]	ZONES [44]	ZO-AdaFL
Attack success rate	48.36%	83.72%	76.28%	89.66%
Distortion loss	8.60	8.95	44.82	23.23

. It is important to note that all algorithms start with no perturbation. Consequently, the attack loss gradually decreases with the number of algorithmic communication rounds, which grows till it converges to zero, demonstrating a successful attack. Meanwhile, there is a possibility of an increase in distortion loss. In these circumstances, if the algorithm can quickly converge to where the optimal trade-off between zero attack loss and low distortion loss is reached, then the algorithm achieves optimal attack performance. In other words, there exists a trade-off between attack loss and perturbation distortion. Combining Figure 3b and Table 2, the ZONES algorithm achieves better attack loss performance due to its higher disturbance loss compared to our proposed algorithm. However, the success rate of our proposed algorithm in attacking is higher. Moreover, our proposed algorithm sacrifices disturbance loss to achieve lower attack loss and higher attack success rate, compared to the DZOPA and FedZO algorithms.

6. Conclusions and Future Work

In this paper, we propose a zeroth-order adaptive optimization algorithm for FL, named ZO-AdaFL, which is the first application of zeroth-order optimization and adaptive optimization to FL. ZO-AdaFL adopts the zero-order optimization algorithm of estimating the gradient by function values to approximate the gradient to solve the FL black-box optimization problem, and the adaptive algorithm avoids FL falling into a local optimum under the non-convex setup condition of full device participation. The convergence speed of the ZO-AdaFL algorithm is also guaranteed. We also analyze the convergence behavior of the ZO-AdaFL algorithm theoretically and prove its convergence rate, $O(1/T)$, in a non-convex setting. The experiments conducted on a variety of benchmarks confirmed our theoretical analysis. In this work, we assume that evaluation on $f_i\left(\mathbf{x}\right)$ can be made in the absence of noise or error, which may limit the applicability of this result, since—in many practical scenarios—the functional values are obtained through some noisy measurement procedures. Because the variance of the stochastic gradient estimator can be unbounded, in the future, we hope to further incorporate the idea of variance reduction to minimize the constraints. Meanwhile, a distributed zero-order algorithm can be investigated on this basis.