Multi-Dimensional Resource Allocation for Covert Communications in Multi-Beam Low-Earth-Orbit Satellite Systems

Abstract

Satellite communication systems, especially multi-beam low-Earth-orbit (LEO) satellites, could cater to the needs of different industrial applications through flexible resource allocation. Unfortunately, due to the wide coverage of LEO satellites, the data exchange within the LEO satellite networks suffers from the risk of eavesdropping and malicious jamming, which could severely degrade the performance of the industrial production process. To address such challenges, this paper introduces a multi-dimensional resource allocation strategy to facilitate covert communication within the multi-beam LEO satellite network. Our approach ensures the rate requirements of different user equipments while preventing the detection of communication signals by an eavesdropping geostationary orbit (GEO) satellite. Specifically, we formulate an optimization problem that jointly optimizes satellite beam-hopping scheduling, frequency band allocation, and the transmit power of different user equipments, under the covertness constraint. By introducing auxiliary binary variables, we transform this optimization problem into a Mixed-Integer Linear Programming (MILP) problem, which allows us to utilize machine learning-based techniques for efficient solution finding. The simulation results demonstrate the effectiveness of our proposed scheme.

1. Introduction

With the wide spread of mobile communication technologies, the demand for ubiquitous network access is increasing significantly. The development of industrial application in remote areas urgently requires the support of communication networks, for example, the transmission of extraction information during combustible ice extraction in offshore regions and the transmission of mechanical drawings for aircraft parts manufacturing in desert areas. However, geographic limitations make the deployment of traditional base station equipment difficult, failing to meet the communication requirements of offshore and remote industrial production [1]. Satellite communication systems, as a crucial branch of modern communication technology, establish seamless global connectivity networks with unique advantages [2,3]. The wide coverage of satellite systems makes them ideal solutions to fill the coverage gaps of ground networks and promote global interconnectivity. Compared to geostationary orbit (GEO) and medium-Earth-orbit (MEO) satellites, low-Earth-orbit (LEO) satellites enjoy shorter communication distances and thus have the potential to meet the latency and reliability requirements of industrial applications [4].

Due to the uneven geographical distribution of user equipments and the diversity in their communication demands, effectively scheduling limited LEO satellite resources to meet the communication needs of user equipments is a significant challenge. Traditional static single-beam satellite resource allocation strategies lack flexibility, making it difficult to address the diverse requirements of industrial applications, resulting in low resource utilization and poor communication quality. Multi-beam satellites, leveraging beam-hopping technology, can flexibly allocate beams and frequency resources within each time slot to serve users in different regions, which enables efficient utilization of on-board resources [5,6,7]. Furthermore, multi-beam satellites can achieve robust interference management through minimizing inter-beam interference with strategic beam planning and precise spectrum management [8]. Due to the limited on-board resources and the diverse needs of users, it is urgent to develop a resource allocation strategy that enhances resource utilization and communication efficiency while coordinating interference.

In recent years, some studies have focused on the efficient allocation of limited resources on LEO satellites to enhance the performance of satellite-to-ground link transmissions. In [9], Kodheli et al. design an uplink resource allocation strategy for a narrowband Internet of Things (IoT) system on LEO satellites, which takes into account the varying channel conditions of the satellite and the data transmission requirements of user equipments. In [10], Guo et al. propose a beam-hopping resource allocation strategy for scenarios with overlapping coverage by multiple LEO satellites. This scheme considers both inter-satellite and intra-satellite interference during beam-hopping scheduling and communication tasks and effectively minimizes the total transmission time. With the booming development of artificial intelligence (AI) and deep neural networks (DNNs), some multi-beam satellite resource allocation schemes based on deep reinforcement learning (DRL) have also been proposed. In [11], Huang et al. consider a joint optimization problem of resource allocation and power control in a LEO satellite–ground network based on rate-splitting multiple access (RSMA). To address the challenge of continuous power control and discrete resource allocation under limited system information, the authors propose a framework based on DRL, which employs the deep Q-network (DQN) algorithm for discrete resource allocation and the proximal policy optimization (PPO) algorithm for continuous power control, aiming to maximize the joint objective. To better address the uneven and time-varying nature of ground communication service requests, Lin et al. propose a joint beam-hopping and bandwidth allocation strategy based on a multi-agent DRL framework [12]. This scheme achieves improved data throughput performance and latency fairness, while also demonstrating strong generalization capabilities.

Despite the extensive research on multi-beam satellite network, the security of wireless transmission between the user equipments and the LEO satellite has not been adequately considered. Due to the broadcast nature of wireless channels, satellite communication systems are susceptible to eavesdropping and malicious jamming. Some electronic reconnaissance satellites with specific functions may eavesdrop on important information transmitted by ground nodes without permission, causing security issues such as sensitive data leakage [13]. In addition, the presence of signals may attract the attention of some active eavesdroppers, who can impose malicious interference to disrupt the transmission. Traditionally, the security of satellite communications has relied primarily on encryption technology and physical layer security (PLS) [14,15,16]. Encryption can safeguard data confidentiality and integrity but requires intricate key management, algorithm updates, and remains susceptible to risks like password cracking. Physical layer security (PLS) utilizes modulation techniques and channel characteristics to reduce eavesdropping risks without traditional encryption. However, eavesdroppers with advanced decoding capabilities can overcome its defenses to obtain confidential information successfully [17]. Additionally, both of these measures are also susceptible to interference attacks by active eavesdroppers, causing communication interruptions due to the exposure of signal transmission behaviors. Different from the above traditional methods, covert communication technology aims to ensure privacy and security by hiding wireless signals, thereby avoiding detection by unauthorized eavesdroppers [18,19].

Motivated by the above discussion, we investigate the design of a multi-dimensional resource allocation strategy to facilitate efficient and covert data transmission within the multi-beam LEO satellite network. Specifically, we jointly optimize satellite beam-hopping scheduling, frequency band allocation, and the transmit power of different user equipments to maximize the rate and covert requirements of different user equipments. By introducing auxiliary binary variables, we convert this optimization problem into a Mixed-Integer Linear Programming (MILP) problem so that machine learning-based techniques can be utilized for efficient solution finding. The main contributions of this paper are as follows:

• To meet the transmission requirements of various user equipments, we investigate how to utilize a multi-dimensional resource allocation strategy in the multi-beam LEO satellite network. Unlike existing works, we additionally consider the covert transmission requirements and examine their impact on satellite resource allocation.
• We derive the uplink covert transmission constraint and formulate the design of the multi-beam LEO satellite resource allocation strategy as an optimization problem. In our formulation, we jointly optimize satellite beam-hopping scheduling, frequency resource allocation, and transmit power of user equipments at different time slots.
• To efficiently find a solution, we first employ linearization techniques to transform the optimization problem into a MILP problem. Subsequently, we apply a machine learning-based method, specifically a Tree Markov Decision Process algorithm, to solve this MILP problem. Simulation results demonstrate that our approach is effective.

2. System Model and Problem Formulation

The considered multi-beam LEO satellite covert communication scenario is shown in Figure 1, which includes I user equipments (UEs), a legitimate LEO satellite with altitude H, and an eavesdropping GEO satellite with altitude $H_e$. The set of UEs is denoted as $\mathcal{I}=\left\{i|i=1,2,\dots ,I\right\}$. There are two types of UEs: the normal user equipments (NUEs), and the covert user equipments (CUEs) requiring covert data transmission. The set of CUEs is denoted as $\mathcal{J}\subset \mathcal{I}$, $\mathcal{J}=\left\{j|j=1,2,\dots ,J\right\}$. The LEO satellite makes resource scheduling decisions in a time-slotted fashion, where the length of each time slot is $\delta$ and the set of time slots is denoted as $\mathcal{T}=\left\{t|t=1,2,\dots ,T\right\}$. In order to meet the communication requirements of UEs and realize flexible multi-dimensional communication resource allocation, the LEO satellite employs the beam-hopping strategy to receive UE data. At each time slot, the LEO satellite illuminates certain beam areas. The set of beam areas is denoted as $\mathcal{N}=\left\{n|n=1,2,\dots ,N\right\}$. The carrier frequency of transmitted signals is denoted as f and the system bandwidth is B. Both CUEs and NUEs transmit uplink signals to the legitimate LEO satellite on different bands, and the set of bands is denoted as $\mathcal{M}=\left\{m|m=1,2,\dots ,M\right\}$. At time slot t, UE i selects a power level from the set $\left\{P_k|k=1,2,\dots ,K\right\}$ on each band for data transmission.

In our proposed scenario, the eavesdropping GEO satellite aims to detect whether there exists covert data from its received signals. The CUEs aim to transmit covert uplink data to the legitimate LEO satellite while ensuring that the wireless signals are not detected by the eavesdropping GEO satellite.

2.1. Satellite Channel Model

Multi-beam LEO satellites usually utilize beam-forming technology to receive signals in specified directions, while the UEs transmit signals through an omnidirectional antenna. The channel power gain between UE i and beam area n can be calculated by [20]

$$h_{i,n}={\displaystyle \frac{g\left({\theta }_{i,n}\right)GtGr}{L_f}}$$

(1)

where $Gt$ is the maximum transmitting antenna gain of UEs and $Gr$ is the maximum receiving antenna gain of the LEO satellite. According to Friis formula, the free space path loss $L_f$ can be expressed as

$$L_f={\displaystyle \frac{{\left(4\pi d_{i}f\right)}^2}{c^2}}$$

(2)

where c denotes the light speed, $d_i=\sqrt{H^2+s_i^2}$ represents the distance between the LEO satellite and UE i, and $s_i$ is the distance between the UE i and the LEO satellite projection point. The parameter $g\left({\theta }_{i,n}\right)$ in (1) is the beam gain factor, where ${\theta }_{i,n}$ denotes the angle between UE i and the center of LEO satellite beam area n. The beam gain factor can be calculated by [21]

$$g\left({\theta }_{i,n}\right)={\left({\displaystyle \frac{J_1\left(u\right)}{2u}}+36{\displaystyle \frac{J_3\left(u\right)}{u^3}}\right)}^2$$

(3)

$$u=2.70123{\displaystyle \frac{sin{\theta }_{i,n}}{sin{\theta }_{3\mathrm{dB}}}}$$

(4)

where ${\theta }_{3\mathrm{dB}}$ is the deviation angle of half energy attenuation, which determines the radius of beam areas. $J_1(·)$ and $J_3(·)$ are first-kind Bessel functions of order 1 and 3, respectively.

Similar to (1) and (2), the channel power gain between UE i and the eavesdropping GEO satellite can be calculated by

$$h_{e,i}={\displaystyle \frac{GtG{r}_e{c}^2}{{\left(4\pi H_{e}f\right)}^2}}$$

(5)

where $G{r}_e$ is the maximum receiving antenna gain of the GEO satellite. Since the eavesdropping GEO satellite is very far from the ground, the deviation of the receiving angle can be ignored and the communication distance can be approximated by the orbit altitude.

2.2. Covert Communication Model

In the secure satellite communication scenario, an eavesdropping GEO satellite aims to detect whether there is covert data from the received signal, while CUEs try to transmit more covert data without being eavesdropped. The eavesdropping GEO satellite aims to detect the transmission activities from the following binary hypothesis test:

$$\left\{\begin{array}{cc}{\mathcal{H}}_0:& {\left[y_{e,l}\left(t\right)\right]}_{1\le l\le L}={\left[x_{nue,l}\left(t\right)+n_{e,l}\right]}_{1\le l\le L}\\ {\mathcal{H}}_1:& {\left[y_{e,l}\left(t\right)\right]}_{1\le l\le L}={\left[x_{nue,l}\left(t\right)+x_{cue,l}\left(t\right)+n_{e,l}\right]}_{1\le l\le L}\end{array}\right.$$

(6)

where ${\left[y_{e,l}\left(t\right)\right]}_{1\le l\le L}$ denotes the received signal samples and $L=\delta B$ is the number of samples collected by the eavesdropping GEO satellite. ${\mathcal{H}}_0$ represents the case that UEs only send normal data, and ${\mathcal{H}}_1$ represents the case that UEs send both normal data and covert data. $n_{e,l}$ is the noise at the eavesdropping GEO satellite and it follows the complex circularly symmetric Gaussian distribution with mean 0 and variance ${\sigma }^2$. $x_{nue,l}\left(t\right)$ is the received signal from NUEs at time slot t, and $x_{cue,l}\left(t\right)$ is the received signal from CUEs at time slot t, which can be expressed as

$$x_{nue,l}\left(t\right)={\displaystyle \sum _{i\in \mathcal{I}\setminus \mathcal{J}}}{\displaystyle \sum _{m\in \mathcal{M}}}\sqrt{h_{e,i}{P}_{i,t}^m}{x}_{i,m,l}$$

(7)

$$x_{cue,l}\left(t\right)={\displaystyle \sum _{j\in \mathcal{J}}}{\displaystyle \sum _{m\in \mathcal{M}}}\sqrt{h_{e,j}{P}_{j,t}^m}{x}_{j,m,l}$$

(8)

where $P_{i,t}^m$ and $P_{j,t}^m$ denote the transmit power of NUE i and CUE j on the band m at time slot t. $x_{i,m,l}$ denotes the l-th symbol transmitted by NUE i on the band m and it follows $x_{i,m,l}\sim \mathcal{CN}\left(0,1\right)$, while $x_{j,m,l}$ denotes the l-th symbol transmitted by CUE j on the band m and it also follows $x_{j,m,l}\sim \mathcal{CN}\left(0,1\right)$. The eavesdropper detection can produce two error decisions: false alarm probability $\alpha$ and missed detection probability $\beta$. The covert performance of the system is usually measured by the total detection error probability: $p_e=\alpha +\beta$. Similar to the existing studies [22], the covert transmission requirements of CUEs can be guaranteed if $p_e\ge 1-\epsilon$, where $\epsilon$ is a small positive constant.

According to [18], when the eavesdropper adopts the optimal decision threshold, the lower bound of the total detection error probability is given as $1-\sqrt{\mathcal{D}\left({\mathbb{P}}_{1,m}\left|\right|{\mathbb{P}}_{0,m}\right)}/2$, where ${\mathbb{P}}_{1,m}$ is the probability distribution of ${\left[y_{e,l}\left(t\right)\right]}_{1\le l\le L}$ when ${\mathcal{H}}_1$ is true, and ${\mathbb{P}}_{0,m}$ is the probability distribution of ${\left[y_{e,l}\left(t\right)\right]}_{1\le l\le L}$ when ${\mathcal{H}}_0$ is true. $\mathcal{D}\left({\mathbb{P}}_{1,m}\left|\right|{\mathbb{P}}_{0,m}\right)$ is the Kullback–Leibler (KL) divergence of these two distributions. Given the parameter $\epsilon$, we can meet the covert transmission requirements of CUEs as long as

$${\displaystyle \sum _{m\in \mathcal{M}}}\mathcal{D}\left({\mathbb{P}}_{1,m}\left|\right|{\mathbb{P}}_{0,m}\right)\le {\displaystyle \frac{2{\epsilon }^2}{\delta B}}$$

(9)

Based on the above discussion, we propose a covertness constraint and a optimization problem of the multi-dimensional resource scheduling strategy in the following subsection.

2.3. Problem Formulation

We define a binary variable $b_{i,t}^{m,k}\in \{0,1\}$ that represents the transmitting decision of UE i, i.e., $b_{i,t}^{m,k}=1$ represents UE i transmitting signals with power $P_k$ on the band m at time slot t, while $b_{i,t}^{m,k}=0$ represents UE i not transmitting signals with power $P_k$ on the band m at time slot t. UEs can only select one transmit power on the band m at each time slot, which can be expressed as

$${\mathrm{C}}_1:{\displaystyle \sum _{k\in \mathcal{K}}}{b}_{i,t}^{m,k}=1$$

(10)

At time slot t, the transmit power of UE i on the band m can be expressed as

$$P_{i,t}^m={\displaystyle \sum _{k\in \mathcal{K}}}{b}_{i,t}^{m,k}{P}_k$$

(11)

We also define a binary variable $c_{n,t}\in \{0,1\}$ that represents the beam-hopping scheduling decision of the LEO satellite, i.e., $c_{n,t}=1$ represents the beam area n illuminated at time slot t, while $c_{n,t}=0$ represents the beam area n not illuminated at time slot t. The satellite illuminates a certain number of beams at time slot t to receive UE data. Additionally, only $N_0$ beam areas can be illuminated at each time slot due to the limited satellite payload resources, which can be expressed as

$${\mathrm{C}}_2:{\displaystyle \sum _{n\in \mathcal{N}}}{c}_{n,t}=N_0$$

(12)

The uplink communication rate of UE i can be expressed as

$$R_i={\displaystyle \sum _{t\in \mathcal{T}}}{\displaystyle \sum _{m\in \mathcal{M}}}{\displaystyle \frac{B}{M}}{log}_2\left(1+{\displaystyle \frac{{\displaystyle \sum _{n\in \mathcal{N}}}{h}_{i,n}{c}_{n,t}{\displaystyle \sum _{k\in \mathcal{K}}}{b}_{i,t}^{m,k}{P}_k}{{\displaystyle \sum _{i^{\prime }\in \mathcal{I},i^{\prime }\ne i}}{\displaystyle \sum _{n\in \mathcal{N}}}{h}_{i^{\prime },n}{c}_{n,t}{\displaystyle \sum _{k\in \mathcal{K}}}{b}_{i^{\prime },t}^{m,k}{P}_k+{\sigma }^2}}\right)$$

(13)

where ${\sigma }^2$ represents the noise power of the LEO satellite channel. In order to meet the communication requirements of NUEs, there is a rate constraint for all NUEs, which can be expressed as

$${\mathrm{C}}_3:R_i\ge D,i\in \mathcal{I}\setminus \mathcal{J}$$

(14)

where D denotes the rate threshold for NUEs.

At the same time, we consider the worst case scenario, where the eavesdropping GEO satellite eavesdrops on all transmission bands. Based on the above discussion about covert communication, the KL-divergence in (9) can be calculated by

$$\mathcal{D}\left({\mathbb{P}}_{1,m}\left|\right|{\mathbb{P}}_{0,m}\right)=\left[{\displaystyle \frac{{\displaystyle \sum _{j\in \mathcal{J}}}{\displaystyle \sum _{k\in \mathcal{K}}}{h}_{e,j}{b}_{j,t}^{m,k}{P}_k}{{\displaystyle \sum _{i\in \mathcal{I},i\notin \mathcal{J}}}{\displaystyle \sum _{k\in \mathcal{K}}}{h}_{e,i}{b}_{i,t}^{m,k}{P}_k+{\sigma }_e^2}}-ln\left(1+{\displaystyle \frac{{\displaystyle \sum _{j\in \mathcal{J}}}{\displaystyle \sum _{k\in \mathcal{K}}}{h}_{e,j}{b}_{j,t}^{m,k}{P}_k}{{\displaystyle \sum _{i\in \mathcal{I},i\notin \mathcal{J}}}{\displaystyle \sum _{k\in \mathcal{K}}}{h}_{e,i}{b}_{i,t}^{m,k}{P}_k+{\sigma }_e^2}}\right)\right]$$

(15)

where ${\sigma }_e^2$ represents the noise power of the eavesdropping GEO satellite channel. Furthermore, according to the Taylor expansion, when $x\ge 0$, there is always $x-lnx\le x^2/2$. Thus, the upper bound of the KL divergence can be expressed as

$$\mathcal{D}\left({\mathbb{P}}_{1,m}\left|\right|{\mathbb{P}}_{0,m}\right)\le {\displaystyle \frac{1}{2}}{\left({\displaystyle \frac{{\displaystyle \sum _{j\in \mathcal{J}}}{\displaystyle \sum _{k\in \mathcal{K}}}{h}_{e,j}{b}_{j,t}^{m,k}{P}_k}{{\displaystyle \sum _{i\in \mathcal{I},i\notin \mathcal{J}}}{\displaystyle \sum _{k\in \mathcal{K}}}{h}_{e,i}{b}_{i,t}^{m,k}{P}_k+{\sigma }_e^2}}\right)}^2$$

(16)

Based on (9), the covert transmission can be guaranteed as long as

$${\displaystyle \frac{1}{2}}{\displaystyle \sum _{m\in \mathcal{M}}}{\left({\displaystyle \frac{{\displaystyle \sum _{j\in \mathcal{J}}}{\displaystyle \sum _{k\in \mathcal{K}}}{h}_{e,j}{b}_{j,t}^{m,k}{P}_k}{{\displaystyle \sum _{i\in \mathcal{I},i\notin \mathcal{J}}}{\displaystyle \sum _{k\in \mathcal{K}}}{h}_{e,i}{b}_{i,t}^{m,k}{P}_k+{\sigma }_e^2}}\right)}^2\le {\displaystyle \frac{2{\epsilon }^2}{\delta B}}$$

(17)

Thus, the covertness constraint of our multi-dimensional satellite resource scheduling strategy can be expressed as

$${\mathrm{C}}_4:{\displaystyle \sum _{m\in \mathcal{M}}}\left({\displaystyle \frac{{\displaystyle \sum _{j\in \mathcal{J}}}{\displaystyle \sum _{k\in \mathcal{K}}}{h}_{e,j}{b}_{j,t}^{m,k}{P}_k}{{\displaystyle \sum _{i\in \mathcal{I},i\notin \mathcal{J}}}{\displaystyle \sum _{k\in \mathcal{K}}}{h}_{e,i}{b}_{i,t}^{m,k}{P}_k+{\sigma }_e^2}}\right)\le \sqrt{{\displaystyle \frac{4{\epsilon }^2}{\delta B}}}$$

(18)

The optimization goal of our scheme is to maximize the total communication rate of CUEs under the aforementioned constraints. Similar to the successive interference cancellation (SIC) in non-orthogonal multiple access (NOMA) technology [23], the legitimate LEO satellite removes the NUEs signal from the received signals. As a result, the communication of CUEs is exclusively affected by the interference from other CUEs. To simplify the notation, we define $\mathcal{B}=\left\{b_{i,t}^{m,k}\forall i,t,m,k\right\}$ and $\mathcal{C}=\left\{c_{n,t}\forall n,t\right\}$ to denote the sets of variables. The optimization problem is formulated as

$$\begin{array}{cc}\hfill {\mathbf{P}}_1:& {\displaystyle \underset{\mathcal{B},\mathcal{C}}{max}}{\displaystyle \sum _{j\in \mathcal{J}}}{\displaystyle \sum _{t\in \mathcal{T}}}{\displaystyle \sum _{m\in \mathcal{M}}}{\displaystyle \frac{B}{M}}{log}_2\left(1+{\displaystyle \frac{{\displaystyle \sum _{n\in \mathcal{N}}}{h}_{j,n}{c}_{n,t}{\displaystyle \sum _{k\in \mathcal{K}}}{b}_{j,t}^{m,k}{P}_k}{{\displaystyle \sum _{j^{\prime }\in \mathcal{J}{j}^{\prime }\ne j}}{\displaystyle \sum _{n\in \mathcal{N}}}{h}_{j,n}{c}_{n,t}{\displaystyle \sum _{k\in \mathcal{K}}}{b}_{j,t}^{m,k}{P}_k+{\sigma }^2}}\right)\hfill \end{array}$$

(19a)

$$\begin{array}{cc}\hfill & \mathrm{s}.\mathrm{t}.{\mathrm{C}}_1,{\mathrm{C}}_2,{\mathrm{C}}_3,{\mathrm{C}}_4\hfill \end{array}$$

(19b)

3. TMDP-Based Resource Allocation Algorithm and Problem Linearization

In this section, we first analyze the characteristics of optimization problem ${\mathbf{P}}_1$. ${\mathbf{P}}_1$ is a Mixed-Integer Nonlinear Programming (MINLP) problem that includes logarithmic terms, making it difficult to solve due to its numerous constraints. To address this challenge, we transform the problem ${\mathbf{P}}_1$ into a MILP problem by introducing auxiliary binary variables, which allows us to apply a MILP solution based on machine learning techniques.

3.1. TMDP-Based Resource Allocation Algorithm

The optimization problem ${\mathbf{P}}_1$ is classified as a MINLP problem, which includes logarithmic terms in its objective function, complicating the solution process. Although Karush–Kuhn–Tucker (KKT) conditions and other methods can be applied to determine optimality in nonlinear programming, the presence of integer constraints limits the feasible solution space, making conventional nonlinear solving methods potentially unsuitable. Therefore, we consider linearizing the optimization problem ${\mathbf{P}}_1$, transforming it into a MILP problem. This transformation not only allows us to leverage existing efficient MILP solvers, but also significantly reduces the computational complexity, thereby improving solution efficiency.

The most commonly used exact algorithm for solving MILP problems is the Branch-and-Bound (B&B) algorithm. This algorithm iteratively narrows down the variable ranges through branching and bounding, progressively approaching the optimal solution. It primarily relies on the concept of a search tree, where the feasible solution space is viewed as the root node. Branches are created according to certain rules, and the lower bounds of the objective values (for minimization problems) are computed in each branch to prune branches that cannot yield better solutions. The key to the B&B algorithm lies in determining the branching strategy.

In recent years, the design of branching strategies based on machine learning has gained considerable attention. These strategies, formulated by learning the target problem, can potentially outperform traditional, problem-agnostic branching strategies. Based on the work of Scavuzzo et al. [24], we aim to apply a machine learning-based MILP algorithm known as the Tree Markov Decision Process (TMDP) method. The TMDP framework generalizes the traditional MDP by incorporating a tree structure that better captures the branching process in MILP solvers. This method has demonstrated superior performance over traditional solvers like SCIP in several MILP problems, such as the Multiple Knapsack problem.

To utilize the TMDP method for solving MILP problems, it is essential to first linearize the original MINLP problem. Thus, we consider transforming the optimization problem into a MILP problem in the next subsection.

3.2. Optimization Problem Linearization

We define a vector variable ${\mathit{u}}_t$ that combines the variables $b_{i,t}^{m,k}$ and $c_{n,t}$, which can be expressed as

$${\mathit{u}}_t=\left[c_{1,t},\dots ,c_{N,t},b_{1,t}^{1,1},\dots ,b_{1,t}^{1,K},b_{2,t}^{1,1},\dots ,b_{2,t}^{1,K},\dots ,b_{I,t}^{1,K},b_{1,t}^{2,1},\dots ,b_{I,t}^{M,K}\right]$$

(20)

The length of the variable ${\mathit{u}}_t$ is $N+IKM$, and the elements in ${\mathit{u}}_t$ are all binary variables; thus, there are $2^{N+IKM}$ combinations corresponding to the resource scheduling at the current time slot t. This large number of combinations greatly increases the program complexity and computational time. Therefore, it is necessary to eliminate some of the irrational combinations to mitigate this issue. Due to constraints (10) and (12), most values of ${\mathit{u}}_t$ are unavailable. It can be seen that the number of element “1” in ${\mathit{u}}_t$ should be $N_0+IM$; thus, there are actually $\left({\displaystyle \genfrac{}{}{0pt}{}{N+IKM}{N_0+IM}}\right)$ combinations of variable ${\mathit{u}}_t$. By traversing these $\left({\displaystyle \genfrac{}{}{0pt}{}{N+IKM}{N_0+IM}}\right)$ combinations, we find that most of them are still unavailable. Therefore, we only consider the combinations that satisfy constraints (10) and (12). As a result, the effective number of combinations in our proposed scheme is significantly reduced, which can be calculated by

$$Q=\left({\displaystyle \genfrac{}{}{0pt}{}{N}{N_0}}\right)K^{IM}$$

(21)

Then, we define an auxiliary binary variable ${\omega }_t^q\in \{0,1\}$, where $q\in \mathcal{Q}=\left\{1,2,3,\dots Q\right\}$. The variable ${\omega }_t^q$ represents the satellite beam-hopping scheduling decision and UE transmitting decision at time slot t. For instance, if ${\omega }_t^q=1$, it represents the value of ${\mathit{u}}_t$ corresponds to the q-th combination, which is selected from all the Q combinations. Obviously, the satellite scheduling decision and UE transmitting decision at time slot t are unique, which can be expressed as

$${\displaystyle \sum _{q\in \mathcal{Q}}}{\omega }_t^q=1$$

(22)

At time slot t, the value of ${\mathit{u}}_t$ is uniquely determined for any $q\in \mathcal{Q}$, and its corresponding variables $b_{i,t}^{m,k}$ and $c_{n,t}$ are also uniquely determined as $b_{i,t,q}^{m,k}$ and $c_{n,t,q}$. Therefore, the communication rate of UE i at time slot t and scheduling decision q can be expressed as

$${\rho }_{i,t}^q={\displaystyle \sum _{m\in \mathcal{M}}}{\displaystyle \frac{B}{M}}{log}_2\left(1+{\displaystyle \frac{{\displaystyle \sum _{n\in \mathcal{N}}}{h}_{i,n}{c}_{n,t,q}{\displaystyle \sum _{k\in \mathcal{K}}}{b}_{i,t,q}^{m,k}{P}_k}{{\displaystyle \sum _{i^{\prime }\in \mathcal{I},i^{\prime }\ne i}}{\displaystyle \sum _{n\in \mathcal{N}}}{h}_{i^{\prime },n}{c}_{n,t,q}{\displaystyle \sum _{k\in \mathcal{K}}}{b}_{i^{\prime },t,q}^{m,k}{P}_k+{\sigma }^2}}\right)$$

(23)

Similarly, the communication rate of CUE j at time slot t and scheduling decision q can be expressed as

$${\psi }_{j,t}^q={\displaystyle \sum _{m\in \mathcal{M}}}{\displaystyle \frac{B}{M}}{log}_2\left(1+{\displaystyle \frac{{\displaystyle \sum _{n\in \mathcal{N}}}{h}_{j,n}{c}_{n,t,q}{\displaystyle \sum _{k\in \mathcal{K}}}{b}_{j,t,q}^{m,k}{P}_k}{{\displaystyle \sum _{j^{\prime }\in \mathcal{J}{j}^{\prime }\ne j}}{\displaystyle \sum _{n\in \mathcal{N}}}{h}_{j,n}{c}_{n,t,q}{\displaystyle \sum _{k\in \mathcal{K}}}{b}_{j,t,q}^{m,k}{P}_k+{\sigma }^2}}\right)$$

(24)

We denote ${\chi }_t^q$ as

$${\chi }_t^q={\displaystyle \sum _{m\in \mathcal{M}}}{\displaystyle \frac{{\displaystyle \sum _{j\in \mathcal{J}}}{\displaystyle \sum _{k\in \mathcal{K}}}{h}_{e,j}{b}_{j,t,q}^{m,k}{P}_k}{{\displaystyle \sum _{i\in \mathcal{I},i\notin \mathcal{J}}}{\displaystyle \sum _{k\in \mathcal{K}}}{h}_{e,i}{b}_{i,t,q}^{m,k}{P}_k+{\sigma }_e^2}}$$

(25)

These above parameters ${\rho }_{i,t}^q$, ${\psi }_{j,t}^q$ and ${\chi }_t^q$ can be calculated by traversing all the Q combinations one by one; thus, they are constants in the optimization problem. Based on the above discussion, the original optimization problem ${\mathbf{P}}_1$ can be transformed into a MILP problem ${\mathbf{P}}_2$ as follows:

$$\begin{array}{cc}\hfill {\mathbf{P}}_2:{\displaystyle \underset{{\omega }_t^q}{max}}& {\displaystyle \sum _{j\in \mathcal{J}}}{\displaystyle \sum _{t\in \mathcal{T}}}{\displaystyle \sum _{q\in \mathcal{Q}}}{\omega }_t^q{\psi }_{j,t}^q\hfill \end{array}$$

(26a)

$$\begin{array}{cc}\hfill \mathrm{s}.\mathrm{t}.& {\displaystyle \sum _{q\in \mathcal{Q}}}{\omega }_t^q=1\hfill \end{array}$$

(26b)

$$\begin{array}{cc}\hfill & {\displaystyle \sum _{t\in \mathcal{T}}}{\displaystyle \sum _{q\in \mathcal{Q}}}{\omega }_t^q{\rho }_{i,t}^q\ge D,i\in \mathcal{I}\setminus \mathcal{J}\hfill \end{array}$$

(26c)

$$\begin{array}{cc}\hfill & {\omega }_t^q{\chi }_t^q\le \sqrt{{\displaystyle \frac{4{\epsilon }^2}{\delta B}}}\hfill \end{array}$$

(26d)

Compared to the optimization problem ${\mathbf{P}}_1$, there are no more logarithmic terms in the problem ${\mathbf{P}}_2$. As both the objective function and constraints of ${\mathbf{P}}_2$ are linear, the original problem has been transformed into a MILP problem. Subsequently, this transformation allows us to utilize the TMDP method to solve the MILP problem, which demonstrates great performance in the next section.

4. Simulation Results

In this section, we show the simulation results of the multi-dimensional satellite resource scheduling strategy. We assume that the satellite beam areas are adjacent to each other, and the beam radius can be calculated by $r=Htan{\theta }_{3\mathrm{dB}}$. The location of UEs in each beam area is randomly generated. We set five beam areas corresponding to five UEs, including two CUEs. Specifically, we set UE 4 and UE 5 as CUE 1 and CUE 2. As for the location of UEs and the LEO satellite, we establish a plane coordinate system; the origin (0,0) corresponds to the ground projection point of the LEO satellite. The center point coordinates of the five beam areas are listed as follows: (−150 km, 0), (0, 0), (150 km, 0), (−75 km, −150 km), (75 km, −150 km). In the communication scenario, UEs transmit uplink signals through two frequency bands, with two optional power levels and five transmission time slots. According to the characteristics of the LEO satellite communication system, the signal frequency f is set to 20 GHz and the bandwidth B is 400 MHz. Since the location of each UE is randomly generated, the simulation results are, unless otherwise stated, averages of five experiments. The detailed parameter settings are shown in

Table 1. Simulation Parameters.

Parameter	Symbol	Value
Number of UEs	I	5
Number of CUEs	J	2
Number of bands	M	2
Bandwidth	B	400 MHz
Number of beam areas	N	5
Number of beams	$N_0$	3
Transmit power	$P_k$	[10 W 100 W]
Number of time slots	T	5
Length of each time slot	$\delta$	10 ms
Rate threshold for NUEs	D	4 Mbps
Signal carrier frequency	f	20 GHz
LEO satellite altitude	H	1000 km
GEO satellite altitude	$H_e$	35,786 km
3 dB beam angel	${\theta }_{3dB}$	$5^{\circ }$
Beam radius	r	87 km
Receiving antenna gain	$Gr$	30 dB
Eavesdropper antenna gain	$G{r}_e$	40 dB
Transmitting antenna gain	$Gt$	20 dB
Noise power of LEO channel	${\sigma }^2$	−90 dBW
Noise power of GEO channel	${\sigma }_e^2$	−90 dBW
Covertness parameter	$\epsilon$	0.1

below.

The impact of bandwidth B and the number of beams $N_0$ on the total communication rate of CUEs is shown in Figure 2. It can be seen that increasing bandwidth can significantly improve the total rate of CUEs. With the increase in $N_0$, UEs can transmit signals through more beams; thus, the total rate of CUEs is also improved. It is worth noting that according to our proposed covertness constraint (18), increasing bandwidth makes this constraint more strict. Therefore, it is necessary to limit the bandwidth within a certain range when the number of transmit power is limited.

The impact of the covert parameter $\epsilon$ on the rate of each UE is illustrated in Figure 3. It is shown that the increase in $\epsilon$ corresponds to the increase in communication rates for both CUE 1 and CUE 2. This phenomenon can be attributed to the fact that a higher $\epsilon$ value relaxes the covertness constraint (18), thereby enhancing the communication rates. However, it should be noted that this relaxation comes at the expense of reducing the covertness of the communication system. Therefore, it is also necessary to limit $\epsilon$ with a certain range.

The impact of the rate threshold D for NUEs on the rate of each UE is illustrated in Figure 4. It is shown that as D increases, the average rates for CUEs decrease. This is because increasing the rate threshold D will make the constraint (14) more strict, and we can only sacrifice a part of the CUE performance to meet the communication requirements of NUEs in the limited on-board resources. This simulation result shows that the LEO satellite makes tradeoffs according to the requirements of different UEs in the process of resource allocation.

The communication rate of each UE and the beam scheduling decision at each time slot t are shown in

Table 2. The beam scheduling decision at each time slot. For instance, when $t=1$, “01011” represents the case that beam area 2, beam area 4, and beam area 5 are illuminated.

$\mathit{t}=1$	$\mathit{t}=2$	$\mathit{t}=3$	$\mathit{t}=4$	$\mathit{t}=5$
01011	00111	10011	01011	10011

and Figure 5. This simulation result is generated based on a specific set of conditions with $N_0=3$ and $D=10$ Mbps. It can be seen that in order to improve the total communication rate of CUEs, the LEO satellite illuminates beam area 4 and beam area 5 at all time slots. Meanwhile, in order to guarantee the communication rate of NUEs, the LEO satellite should illuminate other beam areas at certain time slots. Taking NUE 1 as an example, its rate at the first two time slots is very low; thus, at the third time slot, the LEO satellite illuminates beam area 1, which meets its communication rate requirement.

In Figure 6, we compare the impact of our proposed resource allocation scheme with two other schemes on the total rate of CUEs. We first utilize the beam round robin scheme, where the beam areas illuminated by the LEO satellite at each time slot are fixed. Specifically, the illuminated areas at the five time slots are 123, 234, 345, 451, and 512, respectively, with $N_0=3$. For instance, when $t=1$, “123” represents the case that area 1, area 2, and area 3 are illuminated. As shown in the figure, this round robin scheme performs worse than our proposed scheme because it does not optimize time resource allocation. Next, we utilize a heuristic algorithm as the comparison, which is similar to the greedy algorithm for solving MILP problems. In the comparison algorithm, it optimizes the LEO satellite scheduling decision at each time slot by selecting the decision with the highest CUE rate. If the rate constraint (14) and the covertness constraint (18) are not satisfied, the current decision is discarded, and the next highest CUE rate decision is selected until an optimal solution is found. It can be seen that the performance of comparison algorithm is lower than the TMDP method of our scheme. This is because the comparison algorithm optimizes decisions at each time slot independently, without considering the overall time scheduling problem. This simulation result demonstrates that our proposed multi-beam LEO satellite multi-dimensional resource allocation strategy is more effective.

5. Conclusions

In this paper, we propose a multi-dimensional resource allocation strategy to facilitate covert communication within the multi-beam LEO satellite network. Our approach ensures the rate requirements of different user equipments while preventing the detection of communication signals by an eavesdropping GEO satellite. Specifically, we formulate an optimization problem that jointly optimizes satellite beam-hopping scheduling, frequency band allocation, and the transmit power of different user equipments, under the covertness constraint. Then, we transform the optimization problem into a MILP problem by introducing additional auxiliary binary variables, which allows us to utilize machine learning-based techniques for efficient solution finding. Specifically, we apply a TMDP method to address this MILP problem. Simulation results demonstrate that our proposed scheme significantly improves the communication efficiency in the multi-beam LEO satellite network.

The overall framework of our proposed resource scheduling strategy is novel, and there are several directions of extension for future research. Firstly, additional performance metrics such as energy efficiency and queuing delay can be incorporated to enhance the resource allocation strategy. Additionally, refining the communication requirements of user equipments to account for sudden business demands, regional variations, and other factors can improve the adaptability of our strategy. In order to establish a more accurate channel model, factors such as the satellite movement speed, multi-path fading, and rain/fog fading can also be considered. Furthermore, the corresponding solving algorithms should be optimized, and the strategies for covert communication between multi-beam LEO satellites are also worth studying. Based on these considerations, we aim to further explore and refine the multi-beam LEO satellite resource allocation strategy for covert communication in our future work.