Latency Analysis of UAV-Assisted Vehicular Communications Using Personalized Federated Learning with Attention Mechanism

Abstract

In this paper, unmanned aerial vehicle (UAV)-assisted vehicular communications are investigated to minimize latency and maximize the utilization of available UAV battery power. As communication and cooperation among UAV and vehicles is frequently required, a viable approach is to reduce the transmission of redundant messages. However, when the sensor data captured by the varying number of vehicles is not independent and identically distributed (non-i.i.d.), this becomes challenging. Hence, in order to group the vehicles with similar data distributions in a cluster, we utilize federated learning (FL) based on an attention mechanism. We jointly maximize the UAV’s available battery power in each transmission window and minimize communication latency. The simulation experiments reveal that the proposed personalized FL approach achieves performance improvement compared with baseline FL approaches. Our model, trained on the V2X-Sim dataset, outperforms existing methods on key performance indicators. The proposed FL approach with an attention mechanism offers a reduction in communication latency by up to 35% and a significant reduction in computational complexity without degradation in performance. Specifically, we achieve an improvement of approximately 40% in UAV energy efficiency, 20% reduction in the communication overhead, and 15% minimization in sojourn time.

1. Introduction

Unmanned aerial vehicles (UAVs) find diverse applications in vehicular communications in sixth-generation (6G) networks. UAVs can serve as aerial computing equipment or aerial base stations (BSs) to provide a wide range of services to vehicular networks [1]. Moreover, UAVs have a great number of degrees of freedom in their trajectory and can achieve line of sight (LoS) with vehicles [2]. However, optimizing the available UAV battery power during massive data transfers and multi-link communications originating from multiple vehicles is a critical issue [3]. The operational height and elevation angle of a UAV also impact its link reliability and coverage range. The optimal trajectory of UAVs is subject to interference constraints while maximizing the coverage area. The mobility of vehicles and the UAV and the associated Doppler spread further impact the precision in channel estimation methods [4].

In 6G communications, the volume of data traffic has surged as sensors generate, offload, and process huge amounts of data [5]. These applications require substantial computing resources and need energy-efficient mechanisms to store and process the incoming packets at the UAV’s server. A combination of federated learning (FL) and mobile edge computing (MEC) facilitates collaborative intelligence in vehicular edge computing [6]. Mobile cloud computing (MCC) and MEC have been used as viable solutions to process computationally intensive tasks at vehicular edge servers [7]. In some approaches, Lyapunov optimization has been used to reduce latency, enhance energy efficiency, and achieve convergence of underlying machine learning models. When MEC is combined with FL, partial offloading improves the quality of service (QoS) in low-latency and delay-sensitive vehicular communications [8]. Edge intelligence for FL-driven 6G communications was discussed in [9], while emerging wireless networks equipped with machine learning were discussed in [10]. An example scenario with integration of a UAV in vehicular communications is depicted in Figure 1. Here, a conventional road side unit (RSU) communicates with vehicles. To facilitate this communication, the UAV can provide communication range for all vehicles [11]. With the UAV serving as a mobile base station (BS), the handoff delay can be reduced to minimal. Figure 1 illustrates that with a traditional terrestrial BS, the signal strength is low when the vehicles are distant from the BS. Figure 1 also illustrates that the handoff delay accumulation can be reduced even in the presence of some no-coverage zones by deploying a UAV as a mobile BS. However, a UAV has limited battery power and needs to remain in a vehicle’s LoS for a prolonged time.

The paradigm of FL addresses challenges such as limited scalability of centralized computing server, centralized storage and management of vehicular data, computational complexity at the vehicular edge, and inconsistencies due to the straggler effect [12]. Excluding stragglers reduces the amount of training data, leading to sparse gradients and model aggregation errors [13]. Furthermore, federated stochastic gradient descent (FedSGD) and federated averaging (FedAvg) limit the scalability of FL approaches [14]. Some recent applications of distributed FL approaches for intelligent transportation systems (ITSs) have been discussed in [15]. Furthermore, the transmission of messages from multiple vehicles leads to wastage of communication resources when the messages at successive transmission time intervals (TTIs) have similar content [16]. The relevance of a global model depends on data distributions among vehicles. Specifically, a global model works well when vehicles’ data is independent and identically distributed (i.i.d.). Hence, this paper explores the application of personalized federated learning (PFL) in UAV-assisted C-V2X communications to optimize the available UAV transmit power and addresses resource demands of vehicles [17]. The multi-agent PFL is further enhanced using an attention mechanism, which considers each vehicle and the UAV to determine the transmission, scheduling and queuing strategy. Based on an adaptive asynchronous optimal policy, the vehicles maximize a global objective function [18]. The asynchronous FL-based task offloading schemes minimize delay and power consumption. Transformer personalization methods have also been explored to model the dependency, priority, and associations between tasks [19].

For trajectory prediction, hidden Markov models (HMMs) and graph theory-based approaches have been applied and explored in existing works such as [20,21,22]. By considering state transitions between an agent’s trajectories as Markovian dynamics, a likelihood matrix is generated to predict the trajectory. Memoryless Markov models, however, assume that an agent’s present state separates its future trajectory from its past locations [23]. In UAV-assisted communications, the UAV’s state transitions may not only depend on the current state, but also on the past trajectories, covering different vehicles [24]. Consequently, HMM-based models do not explain the heterogeneity of UAVs’ trajectories, resulting from their different past locations and random trajectories. The attention mechanism determines the degree of dependence of future UAV trajectories and vehicle states on a UAV’s available battery power [25]. Using model aggregation based on PFL, we predict UAV trajectory coordinates [26]. To maximize learning efficiency, our method allows local model training with subsequent parameter transmission to the UAV [27]. The simulations indicate that our proposed approach significantly improves the computation rate and maintains queue stability compared to several baseline methods.

1.1. Related Work

Machine learning and game-theoretic approaches continue to find novel applications in performance optimization of vehicular communication networks. The recent advances in UAV-assisted 6G communications that are driving the developments in ITSs have been discussed in [28] and UAV-assisted satellite–aerial–ground integrated networks have been analyzed in [29]. Based on Stackelberg game theory, a novel approach for coordinated UAV-assisted vehicular communication networks was proposed in [30]. The performance of vehicular communications was enhanced using multiple candidates for single-subframe resources in [31].

1.1.1. Personalized Federated Learning Approaches and Their Applications in UAV-Assisted Communications

A variant of FL where the vehicles train personalized local models and make predictions jointly with the UAV’s global model is known as PFL. The complexity of the central shared model can be minimized while maintaining the benefits of joint training [32]. In recent applications of PFL, a cloud-based indoor positioning model aggregates parameters from multiple environments to build a cloud-based encoder to create personalized models based on channel state information (CSI). The authors of [33] proposed weight anonymized factorization-based FL for performance improvement and fairness [33]. In addition, a clustered FL algorithm maintains multiple global models to explore latent relationships between clients. The model losses and learning errors are minimized to achieve the optimal number of clusters [34]. Compared to FedAvg and FedSGD, the federated Gaussian process regression (FGPR) framework improves personalization [35]. Dynamic reward shaping improves the selection of actions as state parameters dynamically change, while maintaining reward balance [36].

The authors of [37] proposed an approach that captures common priors from distributed data and generates customized models for heterogeneous clients. Here, a layer-wise PFL algorithm generates and learns different parameters. In a recent work, PFL and DRL maximize the network throughput that adapts to time-varying network states [38]. The authors of [39] proposed a communication-efficient FL through gradient compression and adaptive aggregation. The proposed approach reduced communication overhead between clients and the server [39]. However, a large number of hyper-parameters leads to congestion. To improve communication efficiency, a dynamic sampling and selective masking was used to select fewer clients dynamically in [40]. For efficient wireless FL using layers for personalization, the proposed approach reduced congestion and derived an upper bound for non-convex loss functions. The method also revealed that dynamic network topologies in FL help new participants to reach higher accuracy [41].

The authors of [42] proposed a communication-efficient PFL algorithm and introduced a personalization parameter. The proposed approach improved the model’s accuracy and accelerated model convergence by varying the personalization parameter [42]. The authors of [43] proposed a hierarchical attention-enhanced meta-learning approach that analyzes similarities in clients’ data. The method achieves a tradeoff between the clients’ personalized models and a common model [43]. In a recent approach, collaborative edge caching was investigated, where fog access points make caching decisions when communicating with a macro base station (MBS) [44]. The authors of [45] applied variational inequality, an incentive-based FL approach, and the block coordinate descent algorithm to edge-based FL [45]. The authors of [46] explored mechanisms to avoid duplicate computations and adapt to data distribution shifts. The authors of [17] considered non-convex Gaussian mixture model (GMM) optimization in edge computing environments and proposed adaptive solutions to the GMM problem. By using an attention mechanism, the priority of services was determined, and resource allocation mechanisms were developed to maintain QoS. The results demonstrated a decrease in task processing delays, and effectively delivered QoS [14].

1.1.2. Personalized Federated Learning Approaches for Performance Enhancement in Aerial–Ground Integrated Networks

Existing FL schemes do not adequately account for performance complexity when clients conduct a varying number of training iterations [47]. When clients have low computational or communication capabilities, global model aggregation is slow, causing higher latency. The authors of [48] proposed time-sensitive FL and investigated whether FL leads to improvement in the convergence rate. The authors proposed a DRL mechanism to develop an optimal deterministic algorithm for assigning local iterations based on the computational capabilities of clients [48]. In addition, to minimize training time, the authors in [49] implemented a framework that proposes a spatial–temporal learning optimization based on the resources available at the UAV. The solution approach in [49] utilizes dual fitting to create an online algorithm with a competitive ratio and reduces average flow time by 19% [49]. The authors of [50] proposed collaborative neural filtering for communication efficiency. To reduce model complexity, the authors constructed a standard neural collaborative filtering to select smaller payloads. FL models from selected nodes resulted in a payload-efficient global model. The authors of [51] integrate FL with multi-agent DRL in the vehicular environment enhance energy efficiency and task completion rates. The task-offloading mechanism establishes an FL framework to determine resource allocation proportions [51]. Conventionally, communication and information fusion models have been designed based on belief maps, and multi-objective optimization problems have been formulated as Markov decision processes (MDPs) [52].

A novel method examines the criterion for selecting participating devices under energy constraints and their impact on device selection. To improve training efficiency, the method uses the age-of-information (AoI) metric to determine how quickly participants update their gradients [53]. The authors of [54] propose an innovative Stackelberg game-based framework that explores global loss minimization and latency minimization for faster convergence. To reduce communication rounds, global loss minimization is formulated at the leader, while latency minimization is formulated at the followers. Utilizing monotonic optimization and matching theory, they achieve an optimal follower strategy through subproblems pertaining to resource allocation and sub-channel assignment [54]. The authors of [55] obtained an upper bound for the convergence rate and proposed a new algorithm for selecting devices based on the age of update. It was shown that AoI-based device selection accelerates convergence rates and makes efficient use of sub-channels [55].

The authors of [56] investigate an actor–critic framework for heterogeneous asynchronous FRL. A three-layer MEC system involves entities acting both as actors, interacting with the environment, and critics, learning optimal policies, with independent training for each agent. The method solves the problem of non-stationary environments and enables cooperation among entities [57]. To minimize delay, the authors in [58] formulate an MDP. A solution approach based on DDPG and k-nearest neighbors is used [58]. In a novel approach in [59], an actor network generates user scheduling, UAV trajectory, and power allocation policies, updating dual variables. The authors of [60] proposed an air slicing in which multiple UAVs serve as aerial BSs. With network slicing, the network efficiency is maximized while achieving a simultaneous improvement in latency. The authors formulate a stochastic game in MDP environments where multiple adaptive agents compete simultaneously [60].

The authors of [24] formulated uncertainty minimization as a reward maximization problem and proposed a deep Q-network to optimize task offloading and flying orientation, where different parameters affect the search performance [24]. Another approach transferred wireless power from towers to UAVs and minimized computation overheads; less data led to overfitting. Providing joint power charging between towers and UAVs using workload-aware scheduling algorithms a solution approach was proposed based on MDP to handle workload-aware scheduling decisions [23]. For a dynamic wireless sensor network that employs orthogonal frequency division multiplexing (OFDM), a UAV trajectory design problem is investigated in [61]. A similar approach reduced delay on edge computing networks by reducing dynamic variability in task arrival [62]. In [63], a reinforcement learning (RL)-based cooperative navigation algorithm is proposed to plan efficient paths to avoid collisions.

For multi-UAV cooperative navigation, a prioritized experience replay strategy was designed by considering both temporal difference errors and node connectivity [63]. By optimizing location and velocity at each time slot, a UAV was trained to navigate faster [64]. Multi-agent communication explores relationships between message entropy and bandwidth and introduces mutual information to approximate message entropy [64]. Using hierarchical federated learning (HFL), clusters of UAVs were divided into cluster heads [65]. With content caching, vehicles’ context information can be used to allow UAVs to cache popular content, where clusters of UAVs are formed based on vehicles’ mobility patterns [66]. In another approach, using adaptive FRL, the UAV energy consumption is calibrated to determine the local update frequency in [67]. Other work aimed to optimize network environments and fluctuations in data traffic, and developed an algorithm to select a UAV as a central server [68].

The authors of [69] developed an approach with multiple objectives for UAVs with power constraints [69]. An optimization problem for dynamic data caching and computation offloading to reduce the average processing delay and maximize cache hits for UAVs was formulated in [70]. An optimization problem was constructed to minimize FL convergence time considering constraints such as uplink latency, selection strategy, local model training latency, and energy consumption in [71]. The authors of [72] optimized the position of UAVs based on the instantaneous signal-to-interference noise ratio (SINR). The resource allocation strategy improved the FL execution by 66.67% and attained lower latency and energy consumed compared to other resource allocation techniques [72]. The authors of [73] enhance the generalization capability of routing decisions using FL where domain controllers and servers communicate. The proposed method facilitates knowledge transfer, accelerates training, and reduces communication costs [73]. The authors of [74] integrate FL with imitation learning to coordinate UAVs’ maneuvers. The authors remove biased estimations of imitation parameters using generative adversarial imitation learning. The work aims to regularize the federated gradient updates to achieve efficient parameter interactions and coordinated swarm policies using self-imitated learning [74].

1.2. Contributions

This paper proposes PFL with an attention mechanism. As each vehicle gathers diverse data with varying statistical characteristics, challenges arise in effectively generalizing the global model at the UAV. The global model may yield suboptimal performance due to differences in data distributions and transmission patterns. This degradation in the global model becomes significant as the diversity among local data from different vehicles continues to increase [14]. The attention weights capture the relationship between the hidden states and the past trajectories of the UAV. By leveraging the collective data from all participating vehicles, the proposed approach integrates an attention mechanism to enhance collaboration among vehicles with similar data distributions and mitigates data scarcity and straggler issues [26]. Instead of using a single global model to aggregate local updates, the UAV stores several personalized models, which encapsulate common data from multiple vehicles [75]. The UAV aggregates several global models based on the data distribution parameters calculated by each vehicle [76]. The principal contributions are listed as follows:

• For PFL model convergence, optimal policy evaluation, and improvement, we consider the multi-objective MDP as a sequence of parameterized optimization problems. The optimal solutions correspond to the optimal actions in the proposed bin-packing problem (BPP). This leads to an improvement of state evaluation and action evaluation in a reduced action space.
• We investigate the vehicle selection losses and communication errors in the PFL model convergence rate. We identify the factors that impact the global optimum and minimize training loss [77].
• We analyze the importance of past UAV trajectories in the proposed system using the V2X-Sim dataset from [78] and the LTE I/Q dataset from [79]. Consequently, we develop a sequential prediction model for future UAV trajectories.
• We extend our previous works in [80,81] by incorporating FL with an attention mechanism and predict the UAV trajectories that lead to maximum utilization of available battery power [82].

1.3. Organization

The remainder of this paper is structured as follows: Section 2 outlines our system model. In Section 3, we formulate the proposed optimization problem as a Markov decision process (MDP). Section 4 is our proposed solution based on PFL with an attention mechanism to minimize the UAVs’ power consumption. Section 5 discusses the results and investigates the variations in UAV power consumption and optimal UAV trajectory prediction. Lastly, Section 6 concludes the paper.

2. System Model

Figure 2 illustrates the system model comprising the UAV–vehicle communication architecture that comprises $\{{\mathbf{V}}_1$, …, ${\mathbf{V}}_n\}$ vehicles. The communication channel uses orthogonal time–frequency space (OTFS) modulation, which introduces frequency and spatial selectivity in the fading channel. It considers both LoS and NLoS components to estimate the channel characteristics. The UAV trajectory is a set of sequential coordinates denoted by ${\mathsf{\Delta }}_{(x,y)}$. The trajectory coordinates comprise hovering and turning points, represented as ${\mathsf{\Delta }}_{(x,y)}\in [{\mathsf{\Delta }}_{(0,0)},{\mathsf{\Delta }}_{(0,1)}\dots {\mathsf{\Delta }}_{(1,0)},{\mathsf{\Delta }}_{(1,1)}\dots {\mathsf{\Delta }}_{(m,m)}]$.

The past UAV coordinates are stored and are used to predict the future UAV path. We model the changes in the UAV trajectory as incremental steps based on their velocity and directions. We assume a system with slot duration $\tau$ that consists of a block-fading model. The downlink channel gain from the UAV (u) to a vehicle (v) is given by Equation (1).

$${\displaystyle {\overline{g}}_{u\to v}^{\left(t\right)}}={\left|{\displaystyle h_{u\to v}^{\left(t\right)}}\right|}^2{\displaystyle {\sigma }_{u\to v}^{\left(t\right)}},t=1,2,\dots ,\tau$$

(1)

where ${\displaystyle {\sigma }_{u\to v}^{\left(t\right)}}$ represents the large-scale fading component that varies as vehicles change their position. The location of vehicle v at slot t is represented as ${\displaystyle {\mathrm{x}}_v^{\left(t\right)}}$. The large-scale fading in $dB$ is given by Equation (2).

$${\displaystyle {\sigma }_{\mathrm{dB},u\to v}^{\left(t\right)}}=\mathrm{PL}\left({\mathrm{x}}_u,{\displaystyle {\mathrm{x}}_v^{\left(t\right)}}\right)+{\displaystyle {\vartheta }_{u\to v}^{\left(t\right)}}$$

(2)

where $PL$ is the path loss and follows log-normal shadowing from ${\mathrm{x}}_u$ to ${\displaystyle {\mathrm{x}}_v^{\left(t\right)}}$. The standard deviation in log-normal shadowing is denoted by ${\displaystyle {\vartheta }_{u\to v}^{\left(t\right)}}$ that varies with the displacement of the vehicle during the last slot. The SINR ${\displaystyle {\gamma }_v^{\left(t\right)}}$ at a vehicle at time slot t is defined as a function of the communication bandwidth ${\displaystyle {\mathbf{b}}_w^{\left(t\right)}}$ and power ${\displaystyle {\mathbf{p}}_w^{\left(t\right)}}$ given by Equation (3) as

$${\displaystyle {\gamma }_v^{\left(t\right)}}\left({\displaystyle {\mathbf{b}}_w^{\left(t\right)}},{\displaystyle {\mathbf{p}}_w^{\left(t\right)}}\right)={\displaystyle \frac{{\displaystyle {\overline{g}}_{{\displaystyle b_u^{\left(t\right)}}\to v}^{\left(t\right)}}{\displaystyle p_v^{\left(t\right)}}}{{\sum }_V{\overline{g}}_{{\displaystyle b_u^{\left(t\right)}}\to v}{\displaystyle p_v^{\left(t\right)}}+{\mathcal{N}}^2}}$$

(3)

where ${\mathcal{N}}^2$ is the power spectral density of additive white Gaussian noise. The downlink spectral efficiency ${\displaystyle C_v^{\left(t\right)}}$ of vehicle v at time t is given by Equation (4).

$${\displaystyle C_v^{\left(t\right)}}=log\left(1+{\displaystyle {\gamma }_v^{\left(t\right)}}\left({\displaystyle {\mathbf{b}}_w^{\left(t\right)}},{\displaystyle {\mathbf{p}}_w^{\left(t\right)}}\right)\right)$$

(4)

The channel coefficient is modeled as follows:

$$h_{u,v}=\sqrt{{\displaystyle \frac{{\kappa }_u}{{\kappa }_u+1}}}{\overline{h}}_{u,v}+\sqrt{{\displaystyle \frac{1}{{\kappa }_u+1}}}{\tilde{h}}_{u,v}$$

(5)

The symbols considered in this paper are described in

Table 1. Definition of symbols used in this paper.

Symbol	Definition
$\mathsf{\Delta }$	UAV trajectory coordinates, denoted by $[{\mathsf{\Delta }}_1,{\mathsf{\Delta }}_2,\dots ,{\mathsf{\Delta }}_n]$
$\mathcal{H}$	UAV’s altitude in meters (m)
$\mathbf{V}$	Number of vehicles $\{{\mathbf{V}}_1$, …, ${\mathbf{V}}_n\}$
${\mathsf{\Delta }}_{(x,y)}$	UAV trajectory; is a set of sequential coordinates
$\mathcal{T}$	UAV’s flying time
$\tau$	Slot duration for communication between vehicles and UAV
${\displaystyle {\overline{g}}_{u\to v}^{\left(t\right)}}$	Downlink channel gain from UAV to a vehicle
${\left|{\displaystyle h_{u\to v}^{\left(t\right)}}\right|}^2$	Channel coefficient
${\displaystyle {\sigma }_{u\to v}^{\left(t\right)}}$	Large-scale fading component
${\displaystyle {\mathrm{x}}_v^{\left(t\right)}}$	Location of vehicle v at slot t
${\displaystyle {\vartheta }_{u\to v}^{\left(t\right)}}$	Standard deviation in log-normal shadowing
${\displaystyle {\gamma }_v^{\left(t\right)}}$	SINR at receiver at time slot t
${\displaystyle {\mathbf{b}}_w^{\left(t\right)}}$	Communication bandwidth available in a transmission window
${\displaystyle {\mathbf{p}}_w^{\left(t\right)}}$	UAV’s available power in a TTI
${\mathcal{N}}^2$	Power spectral density of noise
${\displaystyle C_v^{\left(t\right)}}$	Downlink spectral efficiency of vehicle v at time t
${\kappa }_u$	Rician K-factor
$c_q$	Bin set capacity
$a_{\mathsf{\Psi }}$	Packet set transmitted from a vehicle
${\mathsf{\Psi }}_i$	Size of each packet comprising PFL model hyper-parameters
${\zeta }_{ik}$	Decision variable to denote if packet i is assigned to queue k
${\varphi }_k$	Decision variable to denote whether bin k is utilized
${\mathcal{D}}^{\left(t\right)}$	Total delay encountered by a packet
${\displaystyle d_v^t}$	Delay experienced by a packet communicated from vehicle v
${\displaystyle {\varsigma }_{i,j}^{\left(k\right)}}\left(t\right)$	Binary variable to identify a vehicle scheduled for transmission or waiting state
${\mathsf{\psi }}^{\left(t\right)}$	Vector consisting of different packets comprising PFL model hyper-parameters
${\tau }_d$	Delay experienced by a vehicle while downloading global model hyper-parameters
${\tau }_u$	Delay experienced by a vehicle while uploading local model hyper-parameters
$t_i$	Time duration of the TTI under consideration
$p_{i,j}$	Power consumption of a UAV in a TTI
$a,b$	Major and minor axes of UAV’s elliptical path
$\alpha$	Tuning factor to expand or contract the elliptical trajectory
${\mathcal{D}}_n$	Dataset with n samples
$T_r\left(x\right)$	Trajectory storage vector
$\mathcal{A}$	Set of available actions for UAV
$V^{\ast }\left(s_t\right)$	State value
$Q^{\ast }(s_t,a_t)$	Action value that implies the expected cumulative reward
$\mathcal{R}(s_t,a_t)$	Accumulated reward for state–action pair
$\mathcal{H}\left({\pi }_{\varphi }\right)$	Policy learned by the UAV at height $\mathcal{H}$
${\gamma }_d$	Discount coefficient
$\phi$	Personalization parameter
${\mathbb{E}}_{a_t\sim \pi }$	Expectation over state s
${\overrightarrow{\gamma }}_T$	Global model, collection of sequential data gathered during repeated transmissions
$p_{\theta }$	Probability of state transitions of UAV
$\theta$	Set of parameters of local models
${\mathit{A}}_t$	Attention mechanism to capture dominant past states’ influence on future state transitions
$m_{\varphi }\left({\overrightarrow{\mathsf{\Psi }}}_T\right|{\overrightarrow{\gamma }}_T)$	Variational distribution that approximates the posterior $p_{\theta }\left({\overrightarrow{\mathsf{\Psi }}}_T\right|{\overrightarrow{\gamma }}_T)$

.

3. Problem Formulation

In this paper, an optimization problem is formulated to jointly optimize the UAV’s trajectory, UAV’s available battery power, vehicles’ packet transmission scheduling, and the queue length at the UAV. The vehicles compute the local stochastic gradient and communicate with the UAV to seek a global optimum. Subsequently, we minimize the queue length at the UAV by reducing the transmission rate from multiple vehicles [83]. The data distribution experiences unpredictable changes and needs to periodically update the training set. The queue at the UAV follows the standard $M/M/k$ model and is a bin set of capacity $c_q$, comprising a packet set $a_{\mathsf{\Psi }}$, where the size of each packet is ${\mathsf{\Psi }}_i$. To optimize the queue length at the UAV, we formulate a bin-packing problem (BPP) as an MDP to pack all the packets into a queue such that the total number of queues is minimized for queue capacity $c_q$. The variable ${\zeta }_{ik}$ denotes whether packet i is assigned to queue k, and the variable ${\varphi }_k$ denotes whether bin k is utilized. The problem ($\mathit{P}1$) is formulated as follows in Equation (6):

$$\begin{array}{cc}\hfill \mathit{P}1:& {\displaystyle \underset{{\mathit{\psi }}^{\left(t\right)},{\mathcal{D}}^{\left(t\right)}}{min}}\{w_1{\displaystyle \underset{S{P}_1}{\underbrace{\left[{\displaystyle \sum _{t=1}^T}min\left(\underset{v\in \mathcal{V}}{max}\left({\displaystyle d_v^t}\right),{\tau }_d\right)+T{\tau }_u\right]}}}+\hfill \\ & w_2{\displaystyle \underset{S{P}_2}{\underbrace{\left[{\displaystyle \sum _{t=1}^T}\sum _{v\in \mathcal{V}}{\displaystyle {\mathcal{D}}_v^{\left(t\right)}}min({\displaystyle d_v^{\left(t\right)}},{\tau }_d)\right]}}}+w_3{\displaystyle \underset{S{P}_3}{\underbrace{\left[{\displaystyle \sum _{t=1}^T}\sum _{v\in \mathcal{V}}\left(\sum _{j\in \mathcal{V}}\sum _{k\in \mathcal{K}}{t}_i{\displaystyle p_{i,j}^{\left(k\right)}}{\displaystyle {\varsigma }_{i,j}^{\left(k\right)}}\left(t\right)\right)\right]\}}}}\hfill \\ & \mathrm{subject}\mathrm{to}\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill C1:& min\sum _{v\in V}{\varphi }_k,{\varphi }_k\in \{0,1\},\forall v\in \mathcal{V}\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill C2:& \sum _{i\in a_{\mathsf{\Psi }}}{\zeta }_{ik}{\mathsf{\Psi }}_i\le c_{q{\varphi }_k},\forall k\in K,\forall v\in \mathcal{V}\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill C3:& \sum _{k\in K}{\zeta }_{ik}=1\forall i\in a_{\mathsf{\Psi }},{\zeta }_{ik}\in \{0,1\},\forall i\in a_{\mathsf{\Psi }}\forall k\in K\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill C4:& \left|\right|{\mathsf{\Delta }}_{i(t+1)}-{\mathsf{\Delta }}_{i\left(t\right)}\left|\right|\le v_{max}\left(t\right)t_{i,i+1}\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill C5:& \left|\right|{\mathsf{\Delta }}_{i(t+1)}-{\mathsf{\Delta }}_{i\left(t\right)}\left|\right|\ge d_{min}\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill C6:& {\mathsf{\Delta }}_{(x,y)}={\mathsf{\Delta }}_{(x_0,y_0)}·e^{-\alpha \left({\displaystyle \frac{x^2}{a^2}}+{\displaystyle \frac{y^2}{b^2}}\right)}+\mathcal{H}\hfill \end{array}$$

(12)

$$\begin{array}{cc}\hfill C7:& w_1+w_2+w_3=1\hfill \end{array}$$

(13)

where problem ($\mathit{P}1$) is a multi-objective and multi-variable non-convex optimization problem. In problem ($\mathit{P}1$) in Equation (6), the parameter ${\mathit{\psi }}^{\left(t\right)}$ is a vector consisting of different packets comprising PFL model hyper-parameters. Rather than transmitting the local model hyper-parameters from all the vehicles to the UAV, PFL aims to select a subset of vehicles that communicate their local model hyper-parameters to the UAV in a TTI. The variable ${\tau }_d$ is the delay experienced by a vehicle while downloading the global model hyper-parameters from the UAV. The term ${\tau }_u$ implies the delay experienced by a vehicle while uploading its local model hyper-parameters to the UAV. The variable $t_i$ is the time duration of the TTI under consideration and $p_{i,j}$ is the power consumption of a UAV in a TTI.

The constraint $C1$ emphasizes the decision variable ${\varphi }_k$ that denotes whether bin k is utilized in a transmission window or is idle. The constraint $C2$ limits packet size ($\psi$). This ensures that the queue does not exceed the capacity and limits the utilization of the UAV’s resources. The constraint $C3$ introduces a control parameter for the transmitted data. The constraints $C4$ and $C5$ restrict the UAV’s trajectory within the vehicle’s coverage range such that the UAV traverses a minimum distance ($d_{min}$) and receives packets from at least one vehicle in a TTI. The maximum velocity of the UAV is restricted to ($v_{max}$). In constraint $C6$, the UAV’s trajectory is constrained to an elliptical path. In constraint $C6$, the variables a and b indicate the major axis and minor axis of the UAV’s elliptical path. We introduce a variable $\alpha$ as a tuning factor to expand or contract the elliptical trajectory. The selection of an elliptical trajectory over a circular trajectory was motivated by the results demonstrated in [84,85,86]. The authors in these works postulate that, for UAVs, an elliptical path is a common trajectory for surveillance, target tracking, and other applications. Through extensive experiments and analysis, the authors have illustrated that an elliptical path provides optimal obstacle avoidance and motion stability compared to a circular path [84]. Moreover, a rectangular or a square path has abrupt discontinuities and may lead to a UAV becoming stuck in a specific segment of the path. The variable $\alpha$ is a tuning factor between 0 and 1. When $\alpha$ = 0, the UAV is stationary at height ($\mathcal{H}$) and does not move along the x and y axes. When $\alpha$ = 1, the largest possible elliptical trajectory is realized. For other values of $\alpha$ between 0 and 1, the area of the elliptical path gradually increases as $\alpha$ is increased from 0 to 1. The constraint $C7$ implies that the weighting coefficients in Equation (6) should add to unity. In this paper, we use block coordinate descent (BCD) to decompose the optimization problem $\mathit{P}1$ into three subproblems. BCD optimizes a subset of variables in a subproblem while keeping others constant. This process is repeated until convergence by optimizing smaller blocks of variables in each TTI. To account for the underlying hardware’s computing limitations, it is important to achieve a positive semi-definite matrix and to have the rank of the variables’ matrices as less than or equal to one. This allows the non-convex optimization problem to be transformed into a convex optimization problem. If the weighting coefficients $w_1$, $w_2$, and $w_3$ in Equation (6) do not add to unity or are large integers, it takes a comparatively longer time to solve all the subproblems.

4. Solution Approach

Using BCD, problem $\mathit{P}1$ in Equation (6) is divided into three subproblems. Subproblem $S{P}_1$ pertains to optimizing the UAV trajectory; subproblem $S{P}_2$ pertains to optimizing the available battery power; and $S{P}_3$ aims to optimize the queue occupancy at the UAV. We also utilize personalized federated learning (PFL) with an attention mechanism to reduce the number of communication rounds between the vehicles and UAV. The reduction in communication overhead is realized by selecting a subset of vehicles for communicating local model hyper-parameters in each TTI. When, instead of all the vehicles, a subset of vehicles communicate their local model hyper-parameters with the UAV, a lesser number of communication rounds can be used to transmit the relevant information to the UAV for further processing. Figure 3 illustrates the PFL approach.

4.1. Training Vehicles with Data

We consider a dataset ${\mathcal{D}}_n$ with n samples that comprises a d-dimensional feature space bounded and closed in ${\mathbb{R}}^d$. The vehicle selection is assigned to the UAV according to a policy that varies with the UAV’s battery power and trajectory coordinates. The non-optimal actions are iteratively removed from the action set. We introduce a trajectory storage vector $T_r\left(x\right)$ that stores the difference between the two adjacent trajectory coordinates. We identify the trajectory predictors of $T_r\left(x\right)$ using samples from ${\mathcal{D}}_n$. To build estimators of $T_r\left(x\right)$, we calculate an optimal minimax rate to estimate $T_r\left(x\right)$ using ${\mathcal{D}}_n$. The optimal actions are updated as $V\left(s\right)={max}_a\{Q(s,a):a\in \mathcal{A}\left(s\right),\mathcal{A}\left(s\right)\subseteq \mathcal{A}\}$, where $\mathcal{A}$ denotes the available actions for the vehicles and the UAV. The state value $V^{\ast }\left(s_t\right)$ is the total reward a vehicle can accumulate and an action value $Q^{\ast }(s_t,a_t)$ is the cumulative reward. The Bellman equations are given by Equations (14) and (15) as

$$V^{\ast }\left(s_t\right)=\underset{a_t}{max}\{\mathcal{R}(s_t,a_t)+V^{\ast }\left(s_{t+1}\right)\}\forall t\in T$$

(14)

where $\mathcal{R}(s_t,a_t)$ is the accumulated reward for a state–action pair. The value $Q_{\overline{\theta }}$ is updated as in Equation (15):

$$Q_{\overline{\theta }}\left(s_t,a_{t+1}\right)=min\left(Q_{{\overline{\theta }}_1}\left(s_t,a_t\right),Q_{{\overline{\theta }}_2}\left(s_t,a_t\right)\right)+{\gamma }_d{\mathbb{E}}_{a_t\sim \pi }\left[Q_{\overline{\theta }}\left(s_t,a_t\right)+\phi \mathcal{H}\left({\pi }_{\varphi }\right)\right]$$

(15)

where $\mathcal{H}\left({\pi }_{\varphi }\right)$ is the policy learned by the UAV at height $\mathcal{H}$. At different heights, the available battery power is different, leading to different policies, ${\gamma }_d$ is the discount factor; $a_{(t+1)}$ is the action at the next time; and $\phi$ is the personalization parameter. The expectation ${\mathbb{E}}_{a_t\sim \pi }$ is over state s. The policy aims to maximize the cumulative reward and hence we parameterize the policy using past interactions. For UAV trajectory prediction, we propose a multidimensional time series. The UAV’s action space is continuous and an actor–critic algorithm is used to train the trajectory predictor. This extracts the temporal dependencies among vehicles’ data to forecast the data traffic of each vehicle when spatial correlations exist. This enables PFL to learn the long-term dependency among the local models of different clusters. As illustrated in Figure 4, the local model hyper-parameters are stored and communicated to the UAV. Using PFL, the redundant local model hyper-parameters are discarded to reduce the overall computational complexity of the proposed solution approach.

4.2. Action Set for Vehicles and UAV

The action set comprises five vectors $[a_1,a_2,a_3,a_4,a_5]$. Here, $a_1$ represents the characteristics of a packet, its current state, type of packet, and processing time. The vector $a_2$ indicates the priority of a packet. The vector $a_3$ implies a UAV’s processor utilization, queue occupancy status, queuing delay, and processing delay for a packet. The vector $a_4$ indicates the status of the UAV queue, queue length, waiting time, and the number of processed packets. The vector $a_5$ represents the local model hyper-parameters of a vehicle’s clusters and rejected redundant local models in each TTI. The rewards accumulated by the vehicles are stored in memory used to train a global model. At slot t, each vehicle observes its local state and decides its next action. We quantify the estimator performance by its squared-$L^2\left(p\right)$ error, which implies the precision in estimating the UAV’s trajectory as

$$\mathsf{\psi }\left(\widehat{T}\right)\triangleq \mathbb{E}{\displaystyle {\parallel {\widehat{T}}_r\left(x\right)-T_r\left(x\right)\parallel }_{L^2\left(p\right)}^2}$$

(16)

where $L^2\left(p\right)$ is the $L^2$ norm. The magnitude of Equation (16) varies with all possible local model transmissions from individual vehicles and clusters to the UAV and identifies a configuration ${\displaystyle p_{{\theta }^{\ast }}^{\ast }}$ as

$$\begin{array}{c}\hfill {\displaystyle p_{{\theta }^{\ast }}^{\ast }}\in arg\underset{p_{\theta }\in p_{\mathsf{\Theta }}}{max}{\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^V}\mathcal{L}(p_{\theta };{\displaystyle {\mathcal{D}}_{train}^{\left(i\right)}},{\displaystyle {\mathcal{D}}_{valid}^{\left(i\right)}}),\end{array}$$

(17)

where $\mathcal{L}$ is the loss function, and ${\displaystyle {\mathcal{D}}_{train}^{\left(i\right)}}$ and ${\displaystyle {\mathcal{D}}_{valid}^{\left(i\right)}}$ are the training dataset and validation dataset. The posterior distribution for an action selected by a vehicle pertaining to a UAV’s state is $\forall a\in {\mathcal{A}}_v,\forall v\in \{a_1,a_2,a_3,a_4,a_5\}\}$, and ${\mathcal{H}}_t$ is the history of the past heights and trajectories of the UAV up to iteration t. The batch selection scheme encourages diverse exploration and vehicle selection via a Poisson point process. The method has an exponential complexity for n samples.

4.3. Personalized Federated Learning with Attention Mechanism

At the UAV, we fit the global model, denoted as ${\overrightarrow{\gamma }}_T$, which is a set of sequential data collected and stored during repeated transmissions. A UAV’s global model hyper-parameter vector is

$${\overrightarrow{\gamma }}_T={\displaystyle {\left\{{\mathcal{D}}_t\right\}}_{t=1}^T},$$

(18)

where ${\mathcal{D}}_t$ is the data gathered during the $t^{th}$ TTI, and T implies the total number of coordinate changes traversed by the UAV. At each time step t, the UAV’s location is characterized over the hovering time T. The progression of states correspond to different trajectory coordinates ${\mathbf{\Delta }}_{(x,y)}$. We model the joint distribution of the UAV’s states and past observations as

$$\begin{array}{c}\hfill p_{\theta }({\overrightarrow{\gamma }}_T,{\overrightarrow{\mathsf{\Psi }}}_T)={\displaystyle \prod _{t=1}^T}\underset{\mathbf{Past}-\mathbf{trajectory}}{\underbrace{p_{\theta }\left(x_t\right|{\psi }_t)}}\underset{{\mathbf{UAV}}^{\prime }\mathbf{s}\mathbf{future}\mathbf{trajectories}}{\underbrace{p_{\theta }\left({\psi }_t\right|{\overrightarrow{\gamma }}_{t-1},{\overrightarrow{\mathsf{\Psi }}}_{t-1})}}\end{array}$$

(19)

where ${\overrightarrow{\mathsf{\Psi }}}_t$ is the set of local model hyper-parameters. The transition probability in Equation (19) assumes that the UAV’s state at time t depends on the past observations $({\overrightarrow{\gamma }}_{t-1},{\overrightarrow{\mathsf{\Psi }}}_{t-1})$. This is different from the Markovian assumption, where $p_{\theta }\left({\psi }_t\right|{\overrightarrow{\gamma }}_{t-1},{\overrightarrow{\mathsf{\Psi }}}_{t-1})=p_{\theta }\left({\psi }_t\right|{\psi }_{t-1})$, i.e., future states depend only on the current state. To capture non-Markovian dynamics using PFL and an attention mechanism, the state transitions are modeled as

$$p_{\theta }\left({\psi }_t\right|{\overrightarrow{\gamma }}_{t-1},{\overrightarrow{\mathsf{\Psi }}}_{t-1})=p_{\theta }\left({\psi }_t\right|{\overrightarrow{\mathsf{\psi }}}_t,{\overrightarrow{\mathsf{\Psi }}}_{t-1})$$

(20)

where ${\overrightarrow{\mathsf{\psi }}}_t=\{{\displaystyle {\psi }_1^t},\dots ,{\displaystyle {\psi }_{t-1}^t}\}$ is a set of attention weights to infer the statistical characteristics of the future states. The attention weights are used to determine the influence of a UAV’s past states on the future state transitions through a non-linear mapping based on the attention mechanism [87,88,89,90,91]. The attention mechanism is commonly used in Transformer architectures and utilizes query, key, and value vectors to calculate the attention weights [87]. In PFL, a vehicle is designated to communicate with the UAV in a TTI. In that TTI, the other vehicle’s model parameter transmissions will be redundant until the next TTI. An adaptive weighting approach compares the statistical similarity in model weight parameters of different vehicles as well as vehicles and the UAV. If two vehicles have a high similarity, they are considered to belong to a single subset, so the UAV can communicate with any one vehicle out of the two. If there are three such vehicles in a subset, the UAV can communicate with any one out of these three vehicles because all three vehicles are communicating similar data. Moreover, in a TTI, more than one such subset can exist. Hence, in order to ensure that the UAV does not miss different types of information transmitted by different vehicles, the UAV should communicate with at least one vehicle from each subset. The attention mechanism trains the vehicles and the UAV to learn the state transitions from a sequence of past data [88]. Note, the vehicles generate local models from a limited number of data samples from each TTI. The previously learned local models are augmented with the arrival of new data, allowing the agents to adapt the current PFL model to new model updates. The attention function ${\mathbf{A}}_t$ is introduced to utilize the features from the previous time steps. As the UAV’s trajectory coordinates are determined, the corresponding attention weights cluster the vehicles with similar local models within a selected TTI. This approach utilizes the past knowledge of optimal trajectory coordinates, where the UAV can serve a maximum number of vehicles while adapting to fresh local model hyper-parameters received from the vehicles.

The attention weights ${\overrightarrow{\mathsf{\psi }}}_t$ from past states at time t are gathered from the UAV’s context ${\overrightarrow{\gamma }}_t$ using an attention mechanism ${\mathit{A}}_t$ as ${\overrightarrow{\mathsf{\psi }}}_t=A_t\left({\overrightarrow{\gamma }}_t\right)$, where $\mathbf{A}$ is a deterministic attention mechanism that leads to a sequence of functions ${\left\{A_t\right\}}_t$, $A_t:\to {[0,1]}^t$. Here, $\mathit{A}$ is a set of actions that maps a UAV’s context ${\overrightarrow{\gamma }}_t$ to a set of attention weights ${\overrightarrow{\mathsf{\psi }}}_t$. The model output is a sequence of attention weights at each time step that enables to learn the model hyper-parameter $\theta$ and to infer a UAV’s current state from the posterior distribution $p_{\theta }\left({\overrightarrow{\mathsf{\Psi }}}_t\right|{\overrightarrow{\gamma }}_t)$. Here, we utilize variational learning to jointly learn the model hyper-parameter $\theta$ and an inference network assists to approximate the posterior distribution $p_{\theta }\left({\overrightarrow{\mathsf{\Psi }}}_t\right|{\overrightarrow{\gamma }}_t)$. Using variational learning, we maximize the expected lower bound as

$$\begin{array}{c}\hfill log{p}_{\theta }\left({\overrightarrow{\gamma }}_T\right)\ge {\mathbb{E}}_{m_{\varphi }}\left[log{p}_{\theta }({\overrightarrow{\gamma }}_T,{\overrightarrow{\mathsf{\Psi }}}_T)-log{m}_{\varphi }\left({\overrightarrow{\mathsf{\Psi }}}_T\right|{\overrightarrow{\gamma }}_T)\right]\end{array}$$

(21)

where $m_{\varphi }\left({\overrightarrow{\mathsf{\Psi }}}_T\right|{\overrightarrow{\gamma }}_T)$ is an intermediate variational distribution that approximates the posterior distribution $p_{\theta }\left({\overrightarrow{\mathsf{\Psi }}}_T\right|{\overrightarrow{\gamma }}_T)$. This intermediate variational distribution is modeled as $m_{\varphi }\left({\overrightarrow{\mathsf{\Psi }}}_T\right|{\overrightarrow{\gamma }}_T)$ using a jointly trained inference network as

$$\begin{array}{c}\hfill {\varphi }^{\ast },{\theta }^{\ast }=arg\underset{\varphi ,\theta }{max}{\mathbb{E}}_{m_{\varphi }}\left[log{p}_{\theta }({\overrightarrow{\gamma }}_T,{\overrightarrow{\mathsf{\Psi }}}_T)-log{m}_{\varphi }\left({\overrightarrow{\mathsf{\Psi }}}_T\right|{\overrightarrow{\gamma }}_T)\right]\end{array}$$

(22)

Hence, by calculating the estimations of $\theta$ and $\varphi$ from the training data, we calculate $p_{\theta }({\overrightarrow{\gamma }}_T,{\overrightarrow{\mathsf{\Psi }}}_T)$ to extract the environmental knowledge, and the inference network $m_{\varphi }\left({\overrightarrow{\mathsf{\Psi }}}_T\right|{\overrightarrow{\gamma }}_T)$ to infer the trajectory of the UAV. The inference network $m_{\varphi }\left({\overrightarrow{\mathsf{\Psi }}}_T\right|{\overrightarrow{\gamma }}_T)$ infers the structure of the posterior distribution.

$$p_{\theta }\left({\overrightarrow{\mathsf{\Psi }}}_T\right|{\overrightarrow{\gamma }}_T)=p_{\theta }\left({\psi }_1\right|{\overrightarrow{\gamma }}_T){\displaystyle \prod _{t=1}^T}{p}_{\theta }\left({\psi }_t\right|{\overrightarrow{\mathsf{\psi }}}_{t-1},{\overrightarrow{\mathsf{\Psi }}}_{t-1},{\overrightarrow{\gamma }}_{t:T})$$

(23)

For the inference network, for all TTIs,

$$m_{\varphi }\left({\overrightarrow{\mathsf{\Psi }}}_T\right|{\overrightarrow{\gamma }}_T)=m_{\varphi }\left({\psi }_1\right|{\overrightarrow{\gamma }}_T){\displaystyle \prod _{t=1}^T}{m}_{\varphi }\left({\psi }_t\right|{\overrightarrow{\mathsf{\psi }}}_{t-1},{\overrightarrow{\mathsf{\Psi }}}_{t-1},{\overrightarrow{\gamma }}_{t:T}).$$

(24)

The attention weights generated in $\mathit{A}$ are used to infer the statistical characteristics of ${\psi }_t$. We calculate an estimate of the distribution $\approx p_{\theta }\left({\psi }_t\right|{\overrightarrow{\gamma }}_t)$ to consider the impact of past states with their relevant contributions, which are indicated by the attention weights. We estimate the gradients via stochastic backpropagation and update the parameters of the attention mechanism from their maximum likelihood estimates. The attention function enables sharing of the parameters between the vehicle’s clusters to accelerate learning. The attention function $m_{\varphi }\left({\psi }_t\right|{\overrightarrow{\gamma }}_T)$ stores the attention weights with ${\overrightarrow{\mathsf{\Psi }}}_{t-1}$ and ${\overrightarrow{\gamma }}_{t:T}$ for state transitions. The inference network leads the posterior network to learn the relevant trajectory coordinates and the variance of gradient estimates. This enables us to precisely identify the UAV’s location and the vehicles to which the resources should be allocated in a TTI.

5. Simulation Results and Discussion

We processed the datasets on an Amazon elastic compute-2 (EC-2) instance and the PFL collaborators were constructed using TensorFlow and the TensorFlow Federated frameworks. For PFL model training, we split the V2X-Sim dataset into 60% training and 40% testing data, and for UAV trajectory prediction, we split the LTE-I/Q dataset into 60% training and 40% testing data. We randomly train the agents for approximately 25–30 sets of training instances. Each training set consists of 50 instances, comprising model hyper-parameter transmissions from the vehicles to the UAV. The packet size was varied between 1 byte and 10 megabytes and the number of packets was varied from 100 to 4000. The inter-arrival time interval for packets at the UAV was varied between 100 ms and 1000 ms. In each instance, the capacity of the queue was set as up to 10,000 packets in each bin.

Table 2. Simulation parameters.

Parameter	Value
Vehicle Mobility	Manhattan Mobility
Number of vehicles (V)	1–100
Number of $\mathcal{PS}$ in UAV	1
UAV deployment altitude	100 m–2 km
Elliptical path’s major axis	300 m–1000 m
Elliptical path’s minor axis	150 m–550 m
Edge server location	In vehicle
Communication frequency	5.9 GHz
Modulation technique	OTFS
Distance between vehicles	10–100 m
Road length ($R_L$)	1–4 km
Vehicle speed	0–100 kmph
Mean speed of vehicles	50 km/h
Standard deviation in vehicle speed	10 km/h
Packet size for gross data offloading	1 byte–3 megabytes
Packet size of FL models	1 byte–10 megabytes
Dataset used	V2X-Sim, LTE I/Q
Inter-arrival time for packets at UAV	100 ms–1000 ms
OTFS base station transmit power	40 dBm (10 W)
UAV transmission power	20 dBm (100 mW)
UAV receiving threshold	−80 dBm
Vehicle transmission power	25 dBm (316.2 mW)
Noise power	−50 dBm (${10}^{-8}$ W)

lists the main parameters used in the simulations.

5.1. Performance Comparison of PFL with Traditional FL Methods

In this section, we compare the performance of our proposed PFL approach with FedAvg and FedSGD for a varying number of vehicles (V) and varying road lengths ($R_L$). We assume a scenario with 100 vehicles. Our algorithm converged after 30 iterations and processed 100% of the local model hyper-parameter updates from all the vehicles in the FedAVg and FedSGD scenarios. However, as all the vehicles communicated their local model hyper-parameter updates to the UAV, there was a notable delay in receiving the global model hyper-parameter from the UAV. Using PFL, only a subset of vehicles in each TTI communicated their local model hyper-parameter updates to the UAV, resulting in lower latency and sojourn time.

The parameter percentage of latency violations represents the percentage of instances where the round-trip latency exceeds a predefined threshold. The benchmark predefined threshold values are specified and periodically updated in the 3rd Generation Partnership Project (3GPP) specifications [1,92,93]. However, in our simulated environment, because of limited computing power, we did not set the threshold to a lower value between 2 ms and 5 ms, as appears in some 3GPP specifications. In our simulations, the maximum delay in ms was around 30 ms for the FL, FedAvg, and PFL approaches. So, we set the latency threshold ($L_{max}$) as 30 ms, and for any instance when the delay, comprising queuing delay plus processing delay, exceeded 30 ms, we assumed the packet was either dropped or retransmitted. In an iteration, if the delay exceeds the $L_{max}$ of 30 ms, we consider that as an instance of latency violation. As the number of vehicles was increased, a larger number of packets exceeded the latency threshold $L_{max}$ of 30 ms for the queuing delay and processing delay metrics; this is plotted as the percentage of latency violations in Figure 5 and Figure 6. It was noted that the percentage of latency violations was around 30% for PFL and 15% for PFL with attention mechanism. The 50% reduction in the percentage of latency violations for PFL with attention mechanism validates the advantage of using attention weights and utilizing dominant states for UAV trajectory prediction. In PFL, a minor increase in delay is observed compared to PFL with attention mechanism to ensure that the UAV’s processing capacity is not exceeded and the queue does not overflow. The proposed PFL approach ensures that whenever the latency threshold constraint is breached, the next set of arriving packets are dropped. If the UAV’s queue or buffer becomes available in this TTI, the dropped packets are retransmitted. Otherwise, the next set of packets is transmitted in the next TTI. Some other recent works that have utilized a threshold latency as a metric for performance evaluation are [94,95].

The percentage of latency violations in PFL and conventional FL is illustrated in Figure 5 and Figure 6. Here, we vary the number of vehicles (V) from 1 to 100; the vehicle velocity is set as 60 km/h and 80 km/h; and $R_L$ is set as 2 km and 4 km. The local learning rate for each vehicle is initially set to 0.005 and increased gradually with periodic increments. The number of global model and local model update iterations is set to 2000 and 1000, respectively. In each training round in PFL, 40–70% of vehicles are selected to participate in the training of the global model at the UAV. The subset of vehicles selected by the UAV is based on the similarity of data distributions in the model hyper-parameters. Out of two vehicles with high similarity in their data distributions, only one vehicle is selected by the UAV. This saves redundant computations and lowers latency.

As noted from Figure 5 and Figure 6, the percentage of latency violations lies between 25 and 30% for FedAvg and between 23 and 31% for FedSGD. For the PFL approach, for a similar number of vehicles, the percentage of latency violations lies between 18 and 28%, indicating a gain in performance. PFL uses an adaptive weighting approach that compares the statistical similarity in model weight parameters of different vehicles as well as vehicles and the UAV. If two vehicles have a high similarity, they are treated as a single cluster, so the UAV can communicate with any of the vehicle out of the two. If there are three such vehicles in a cluster, the UAV can communicate with any one out of these three vehicles as all three vehicles are communicating similar data. Moreover, in a TTI, more than one such cluster can exist. So, to not miss different types of information transmitted by different vehicles, the UAV should communicate with at least one vehicle from each cluster. The lower latency in packet transmissions is further noted from Figure 7. The average delay experienced by a packet in PFL and conventional FL is illustrated in Figure 7.

5.2. Performance Comparison of Attention-Mechanism-Based PFL with Traditional FL Methods

In this section, we compare the performance of the proposed attention mechanism-based PFL approach with the traditional FL methods, particularly FedAvg and FedSGD, when varying the number of vehicles (V) and varying road length ($R_L$). The percentage of latency violations in PFL with attention mechanism and conventional FL and the average delay experienced by a packet are illustrated in Figure 8. During the local model hyper-parameter updates, each local model weight is normalized to a value between 0.20 and 0.85 to measure the variance in the data distribution during the dataset partitioning. The FedAvg and FedSGD algorithms emphasize vehicle-specific model hyper-parameter transmissions to the UAV. As illustrated, PFL with attention mechanism also significantly outperforms FedAvg and FedSGD as well as PFL, which implies that the attention mechanism leverages the similarity of data distributions in clusters of vehicles to improve the performance over heterogeneous data.

It is also observed that a vehicle accumulates a higher reward to maintain a higher probability to remain in a cluster during all TTIs. The higher reward of a vehicle emphasizes consistent presence in a PFL cluster and model transmission to the UAV, which improves the global model’s convergence rate. The consistency of local model updates and transmission to the UAV increases the accumulated reward and leads to quicker local model convergence. However, the trend is adversely impacted as the number of vehicles increases, as more vehicles complete their local model hyper-parameter updates and compete to gain access to the UAV’s resources, i.e., bandwidth and available battery power over state–action pairs. As compared to PFL, where the percentage of latency violations lies between 18 and 28%, the percentage of latency violations for PFL with attention mechanism decreases considerably. As noted from Figure 8, the percentage of latency violations for PFL with attention mechanism lies between 12 and 20%, marking a steep decline of 15% as compared to PFL.

The variation in average latency (ms) in the FedSGD and PFL with attention mechanism scenarios is illustrated in Figure 9. Figure 10 illustrates the variation in average packet delay (ms) in PFL for a varying number of vehicles (V) for different road lengths ($R_L$). As illustrated in Figure 10, when the road length ($R_L$) is increased to 4 km, the delay exceeds the latency threshold ($L_{max}$) of 30 ms. From Figure 9 and Figure 10, we observe that there is an approximate 20% reduction in latency for the same number of vehicles that are served by the UAV in a given TTI when utilizing the trajectory predictions of the PFL model with attention mechanism. The attention mechanism leads to improved trajectory prediction compared to conventional FL as well as PFL. Also, PFL with attention mechanism handles the data heterogeneity effectively. Figure 11 illustrates the variations in average latency (ms) in the FedAvg, FedSGD, PFL, and PFL with attention mechanism scenarios with a varying number of vehicles (V) for road length ($R_L$) = 2 km. Figure 11 illustrates the outputs of Figure 8 and Figure 9 on a single plot for comparative analysis. It is observed from Figure 11 that the average latency achieved in the case of FedAvg and FedSGD over a road length ($R_L$) of 2 km for varying vehicle velocities is between 16 ms and 24 ms. For the same vehicle velocities and road length, the average latency for PFL and PFL with attention mechanism is between 10 ms and 18 ms. Hence, PFL and PFL with attention mechanism lead to an approximately 30% reduction in latency, that comprises queuing delay and processing delay, compared to conventional FL approaches such as FedAvg and FedSGD. However, a significant discernible difference in the latency of PFL and PFL with attention mechanism is not observed in Figure 11. This is attributed to the delay accumulated in calculating the attention weights and to identifying the dominant local model hyper-parameters to be transmitted from vehicles to the UAV. Although this difference in the two approaches is negligible over a smaller road length, this difference may be larger and significant in real-world traffic scenarios.

5.3. Variation in UAV’s Transmit Power (dBm) with Number of Vehicles (V)

The variation in the UAV’s transmit power (dBm) vs. number of vehicles (V) for FedAvg and PFL is illustrated in Figure 12. The variation in the UAV’s transmit power (dBm) vs. number of vehicles (V) for FedSGD and PFL with attention mechanism is illustrated in Figure 13 and Figure 14 for V = 1–100. We use the Adam optimizer to update the inference network. We set the UAV’s target region to an elliptical-shaped area where 100 vehicles are randomly and uniformly distributed on the road, with a direct LoS with the UAV. We set the initial coordinates of the UAV to [15, 15], [80, 80], [15, 80], and [80, 15] m. The UAV’s coordinates along the x-axis and y-axis are bound by an elliptical path and the altitude along the z-axis is varied between 100 m and 2 km. Within the elliptical path, the UAV can traverse the trajectory coordinates from a theoretically infinite number of possible paths, distributed as a Poisson point process.

It is noted from Figure 12 that for FedAvg, for 500 ms of training iterations, the UAV power consumption is approximately 14 dBm. For a similar duration of training iterations, the UAV power consumption is approximately 8 dBm for the PFL scenario, implying a 40% reduction in battery power consumption. This reduction in the UAV’s battery power consumption is due to the fact that a reduced number of vehicles compete for the UAV’s communication and computation resources in PFL. When the vehicle velocity is increased to 80 km/hr and the road length ($R_L$) is increased to 2 km, it can be noted that PFL with attention mechanism leads to the UAV’s battery power consumption being similar to that of PFL at lower vehicle velocity. It is evident from Figure 12 that the UAV power consumption is higher in the conventional FL scenario since all the vehicles transmit their local model hyper-parameters to the UAV. Conversely, in the PFL scenario, the UAV power consumption is marginally lower as only a select number of vehicles from a subset are required to transmit their local model hyper-parameters to the UAV.

Additionally, each vehicle generates multiple packets in each TTI. The UAV moves around certain areas, since its coverage range is limited to an elliptical path and it should serve a maximum number of vehicles to increase the fairness index and maximize its reward. With target trajectory prediction, the UAV can arrive at the next set of coordinates from where it can be in the LoS of the maximum number of vehicles. From the trend in Figure 13 and Figure 14, when using PFL with attention mechanism, we observe a reduction of approximately 20% in the UAV’s transmit power and an approximate 15% time reduction in model convergence. For a similar environment, PFL offers a reduction of 15% in the UAV’s transmit power and energy or a 10.5% time reduction in model convergence. Hence, using PFL, improved prediction accuracy in the UAV’s trajectory leads to an approximate reduction of 15% in the UAV’s transmit power, with a 20% reduction in communication overhead from vehicles. Combining PFL with attention mechanism leads to a further reduction in the UAV’s transmit power as well as a reduction in the communication overhead from vehicles.

Figure 15 illustrates the variation in the UAV’s transmit power (dBm) with V for varying vehicle speed and road length (${\mathcal{R}}_L$) for FedAvg: The FedSGD, PFL, and PFL with attention mechanism scenarios over model training time slots. Figure 15 illustrates the outputs of Figure 12 and Figure 13 on a single plot for comparative analysis. It is observed from Figure 15 that the UAV’s average transmit power (dBm) is approximately 14 dBm for FedSGD and 8 dBm for PFL mechanism. This implies an approximate 40% reduction in the UAV’s transmit power when using PFL. For PFL with attention mechanism, the transmit power is approximately 10 dBm. This slight increase in the UAV’s transmit power as compared to PFL is due to the fact that a portion of the UAV’s power is dissipated in selecting a vehicle for local model hyper-parameter transmission from a subset of vehicles. This selection is based on the attention weights that allow the UAV to identify a vehicle with dominant local model hyper-parameters to be transmitted from vehicles to the UAV. Although this difference in the two approaches, i.e., PFL and PFL with attention mechanism, is negligible over a smaller road length, this difference may be larger and significant in real-world traffic scenarios.

5.4. Variation in Average Mean Square Error (MSE) Losses with Personalization Parameter ($\phi$) Using PFL and PFL with Attention Mechanism

The variation in normalized MSE losses with number of training episodes for the PFL and PFL with attention mechanism scenarios is illustrated in Figure 16 and Figure 17 for V = 1–100. We evaluate the effectiveness of the proposed PFL model aggregation approach.

The UAV’s transmit power and bandwidth limits are considered along with the vehicles’ wireless resource limitations. As the selected vehicles upload their local model hyper-parameters for aggregation, PFL requires more computation and bandwidth resources than the PFL approach with attention mechanism. It is observed from the model convergence characteristics that for the available battery power at the UAV, the MSE of PFL is reduced by approximately 6.75% for $V=50$, by 8.5% for $V=100$, and by 12.5% for PFL with attention mechanism, in comparison with the conventional FL approaches. For the UAV’s optimal trajectory prediction with PFL and PFL with attention mechanism on the LTE I/Q dataset, the MSE is reduced by 9.5% for $V=50$, 5.5% for $V=100$, and 3.75% for PFL with attention mechanism, in comparison with the conventional FL approaches. The above results demonstrate the superior performance of our proposed PFL method. Here, we set the learning rate to 0.001 and utilize the Adam optimizer to update the model in subsequent iterations.

To verify the advantages of the PFL with attention mechanism, we compare its performance with PFL to regularize each local loss function and to address the data heterogeneity. Moreover, in PFL, the selected vehicles upload the entire model to the UAV for aggregation in each round. In each round, the selected vehicles sequentially train the feature extractor and predictor. Hence, the PFL algorithm converged after approximately 50 iterations, whereas PFL with attention mechanism converged after approximately 25 iterations. We selected 60% of the vehicles such that at least one vehicle was selected from each cluster. The personalization parameter was varied for other vehicles in a cluster. We tuned each vehicle’s personalization parameter to select between the cluster-wise embedding vectors with lower dimensionality and to train the model in fewer iterations. We also verified that the tunable parameter to balance the training performance and power consumption of the UAV with a varying number of vehicles impacts the average delay.

As illustrated in Figure 18, as the number of vehicles and vehicle velocity increases, the vehicles consume more energy, resulting in scheduling more data transmissions to the UAV. On the V2X-Sim dataset, the PFL results in 4.50% and 3.50% improvement in delay for $\phi$ = 0.001 and $\phi$ = 0.01, and obtains a slight improvement in delay when $\phi$ = 0.1 compared with the traditional FL algorithms. The simulation results reveal that the proposed approach can schedule fewer data samples and still achieve higher learning accuracy. The performance gain is derived from the joint optimization of vehicle selection, bandwidth allocation, communication and computation time, and TTI allocation policies. It is noted that if $\phi$ is too large, there is less emphasis on the UAV’s power consumption and more vehicles are clustered together. This increases the UAV’s power consumption and also increases delay. Thus, the value of the personalization parameter ($\phi$) should be adjusted to optimize the training performance while satisfying the transmit power constraints of the UAV and overall delay. As we increased $\phi$, the model convergence time improved gradually. Figure 18 also implies that the attention mechanism maintains personalization through an adaptive approach. PFL also outperforms the traditional FL methods in terms of average delay for a varying number of vehicles and road length. Figure 19 illustrates the variation in normalized MSE losses with number of training episodes for varying the personalization parameter $\phi$. As illustrated in Figure 19, as the personalization parameter $\phi$ is gradually increased from 0.0000001 to 0.01 in multiples of 10, the MSE error losses tend to gradually decrease. For all the values of the personalization parameter $\phi$, the PFL model tends to converge after approximately 200 training episodes. However, for larger values of $\phi$ = 0.001 and $\phi$ = 0.01, the initial MSE losses are slightly lower compared to smaller values of $\phi$ = 0.0001 and $\phi$ = 0.00001. This implies a reduction in model convergence time and hence a reduction in the UAV’s power consumption when computing global model hyper-parameters from local model hyper-parameters. Furthermore, it is noted that the local model hyper-parameters generated using the attention mechanism add approximately an additional 10% of attention weight parameters compared to the hyper-parameters of the PFL model. Although PFL uses a slightly smaller ratio of personalized parameters, it significantly outperforms FedAvg and FedSGD on latency and the UAV’s power consumption metrics. Hence, PFL leads to a significant improvement in the models’ ability to overcome data heterogeneity. The decrease in the MSE losses for PFL and PFL with attention mechanism indicates that the performance of PFL is improved with the attention mechanism.

5.5. Discussion and Comparison of the Proposed PFL Approach with Existing Works

In

Table 3. Performance comparison of the proposed PFL approach with recent applications of different deep learning and federated learning methods in vehicular communications.

Reference	Proposed Method	Objectives	Cost Function	Reported Results
[96]	Alternative scheduling mechanisms	Low packet drop ratio Packet retransmission	Percentage of dropped messages Network latency	Achieved lower network latency
[97,98]	Proposed deep Q-networks Used actor–critic mechanism	Enhanced agent mobility Improved decision making on centralized server	System’s energy consumption Model complexity Task completion time	Improved system performance
[99]	FL for network performance optimization	Lowering model convergence time for FL Lowering model training loss for FL	Optimal user selection for task association Joint learning for task association	Lowered FL model convergence time by 56% Improvement in FL model accuracy by 3%
[65]	Power allocation Resource allocation Lower latency	Less power consumption Less queuing delay	Round-trip delay	Lesser data exchange by 79% Smaller queue length by 60% Lower power consumption
[100]	Satisfaction-based multi-armed bandit approach	Optimal UAV trajectories Efficient resource allocation	Transmit power Received signal strength	Achieved improvement in throughput Lower system outage probability
[80]	Q-learning and FRL	Minimize local model hyper-parameters Minimize communication rounds	Number of communication rounds FRL model convergence time	Reduced communication rounds Quicker model convergence for FRL
Our Work	PFL PFL with attention mechanism	Minimize average latency and communication overhead Minimize UAV’s power consumption	Average latency (ms) UAV’s power consumption (dBm)	Reduced average latency and communication overhead Reduced UAV’s power consumption UAV’s optimal trajectory prediction

, we have compared our proposed solution approach using PFL with the previous FL and DRL approaches. Whereas earlier state-of-the-art FL and DRL methods achieved an approximate 50% reduction in model convergence time, our method based on PFL achieved an approximate reduction of 40% in the number of transmitted local model hyper-parameters from the vehicles to the UAV for a similar convergence time and similar number of training episodes. The proposed PFL approach also reduced the communication overhead between the vehicles and the UAV. However, unlike existing FL approaches, where joint learning from multiple datasets leads to higher model convergence time, our proposed PFL approach reduces the UAV’s power consumption in a TTI. This is attributed to the fact that in PFL, rather than communicating with each vehicle, the UAV only communicates with select vehicles in a subset in each TTI. Another phenomenon observed for PFL with attention mechanism was that the average latency and model convergence time was lower by approximately 20% as compared to FedAvg and FedSGD. This also implies that PFL is suited to scenarios that consist of non-i.i.d. training data. To fine-tune the performance of our model, we introduced a personalization parameter and noted that utilizing PFL consistently improved the latency and the UAV’s power consumption by 10–15% as compared to FedAvg and FedSGD.

6. Conclusions

This paper proposed a PFL approach to maximize the available UAV battery power and minimize the sojourn time in UAV-assisted C-V2X communications. The proposed PFL approach utilizes an attention mechanism to model aggregation errors and ensure an adaptive and personalized vehicle selection in an uncertain and time-varying environment, subject to bandwidth, UAV power, and vehicle QoS constraints. Furthermore, using an attention mechanism, our PFL model achieved a reduction in model training time with an increase in the number of vehicles compared to existing FL methods. The PFL mechanism addresses the straggler effect by quantifying the contribution of vehicles participating in the model computation and training. This enables us to achieve a balance between model accuracy and straggler impact by dynamically managing the straggler rate. To address packets of varying size, arrival rate, and service rate, we proposed that PFL automates the selection of vehicles by the UAV through attentive state-space modeling. This is achieved by clustering the vehicles to model the similarities between the datasets in a cluster. The experimental findings reveal a greater than 25% reduction in training time and 20% increase in model accuracy across varying non-i.i.d. data scenarios. We propose using PFL with attention mechanism to collaborate with vehicles with similar data distribution through attention-based grouping. PFL achieves approximately 20% lower latency than conventional FL methods such as FedAvg and FedSGD. In the future, we aim to include the complexity in assessing changes in the channel state when using multi-modal datasets. The proposed approach can be improved to predict outcomes such as recurrence of similar data transmissions and incorporate channel state-based predictive information. In addition, future research includes extending the proposed PFL framework to adaptive optimization methods or group-based training methods.

Moreover, in future work, the authors aim to conduct an in-depth computational complexity analysis of the proposed approach. The model training complexity can act as a severe bottleneck for real-world deployment of PFL approaches. Furthermore, introducing a channel state estimator into the system model is expected to contribute to reducing the model convergence-time. A few other challenges in applying PFL and attention mechanism to UAV-assisted vehicular communications are to track the changes in the UAV and vehicle states with variations in the instantaneous channel characteristics. The feasibility of utilizing reward functions based on channel estimation is an avenue for further work.