Utility Maximization for the IRS-Assisted Wireless Powered Communication Network with Multiple Service Providers

Abstract

We consider a multi-device wireless-powered communication network, where an intelligent reflecting surface (IRS) is deployed to assist the wireless energy transfer (WET) from the power station (PS) to the information transmitter (IT) and the wireless information transfer (WIT) from the IT to the devices. The IRS, IT, and PS belong to different service providers, where the PS receives revenue from the IT for WET and pays fees to its energy source and the IRS, and the IT receives revenue from the devices for WIT and pays fees to the PS and the IRS. We model the interactions between the IT and the PS through a Stackelberg game and aim to achieve a win-win situation between them in terms of utility. Specifically, we solve a follower problem that maximizes the utility of the PS by jointly optimizing the IRS reflection coefficients and the transmit power of the PS, and a leader problem that maximizes the utility of the IT by jointly optimizing the energy price, the transmit power allocation of the IT, and the time allocation as well as the IRS reflection coefficients in the WET and WIT phases. The results show that, by deploying IRS, it is possible to effectively enhance the signal strength and thus increase the revenue of the system, and by using Time Division Multiple Access (TDMA), the performance of the system is greatly improved by coordinating the timing of the reception of the signal by each device.

1. Introduction

With the advent of large-scale deployment of battery-powered devices, it may be critical to extend device lifetime in energy-constrained Internet of Things (IoT) networks. Wireless energy transfer (WET) technology has been widely explored for its ability to provide power to energy-constrained devices in a wireless and flexible manner [1]. Therefore, a wireless-powered communication network (WPCN) can be established using WET technology. In a WPCN, the information transmitter (IT) usually adopts a harvest-then-send transmission mode; i.e., it first harvests energy during the WET phase and then uses the harvested energy for information transmission. However, the communication performance of a device in a WPCN can be degraded by the double path loss effect caused by channel path losses in both the WET and wireless information transmission (WIT) phases. Therefore, it is essential for a WPCN to eliminate the double path loss effect and ensure the communication performance of the devices. An intelligent reflecting surface (IRS) consists of a large number of passive reflecting units, each of which can independently adjust its reflection amplitude and phase for the incident signal. Based on this advantage, the wireless communication channel can be intelligently reconfigured by an IRS. In [2], by optimizing the IRS reflection phase shifts as well as the precoding vectors, the secure communication rate of the system with an IRS was significantly increased compared to that of the system without one. Reference [3] focuses on the phase estimation of individual channels in IRS-assisted communication systems with single-antenna transceivers under the correlated Rayleigh fading. In [4], an IRS assisted the simultaneous wireless information and power transfer system by optimizing the transmit precoders at the access point and the phase shifts at the IRS, and the transmit power required at the AP was significantly reduced. Thus, use of the IRS is expected to provide an effective solution for eliminating the double-path loss effect in a WPCN.

There have been existing works considering the use of an IRS to assist a WPCN [5,6]. In [5], the results indicated that dynamic IRS reflection coefficients have practical significance in improving the performance of WPCN systems. In [6], the authors considered an intelligent reflecting surface (IRS)-empowered wireless-powered communication network (WPCN) in an Internet of Things (IoT) network. In both cases, the average sum rate of a system with an IRS is greater than that of a system without one. Therefore, the use of an IRS can create a favorable propagation environment, effectively reducing the double-path loss and improving the performance of a WPCN. In addition, most of the current research on IRS-assisted WPCNs considers only a single service provider, and the resulting optimization problems can be solved using centralized optimization algorithms.

In a WPCN, there may be multiple service providers, such as an energy service provider and an information service provider. Unlike the single service provider systems, there are interactions between each service provider, and each service provider pursues its own utility maximization, so it is desired to optimize the parameters of different service providers to achieve a win-win situation for all service providers. The use of game theory can effectively achieve this goal through sequential strategic optimization interactions among different service providers. In [7], a multi leader single-follower (MLSF) Stackelberg game was formulated to study the resource allocation problem in a WPCN with cooperative jamming. The cooperative and non-cooperative relationships between the wireless nodes and the hybrid access points were studied in [8]. Reference [9] focused on the energy game model between the Base Station and D2D transmitter, and the results showed that the use of game theory can effectively improve the system utility. The results of these works demonstrated that game theory can provide a framework for sequential strategic optimization interaction among service providers with different optimization goals in a WPCN. However, these existing works did not consider the use of an IRS to assist different service providers.

A few studies have attempted to apply game theory to IRS-aided communication systems. Reference [10] studied an evolutionary game model where users interact with dynamic network service providers to adjust their utility. In [11], monetary incentives were considered to study the energy interaction between an IT and a power station (PS) belonging to different operators in an IRS-assisted WPCN. Deploying an IRS can effectively improve system performance, but in practice, the use of an IRS is not free of charge. If there is no revenue, the service provider of the IRS should stop providing the IRS service [12]. Therefore, the IRS should be considered as an independent service provider. To the best of our knowledge, there are few studies on IRS-assisted WPCNs considering the IRS as an independent service provider, which motivated the present study.

In this paper, we consider an IRS-assisted multi-device WPCN. The network first performs WET and then performs WIT. In the WET phase, the IT harvests energy from the PS, and in the WIT phase, the IT transmits information to the devices in the manner of time division multiple access (TDMA). The IRS is used for signal quality improvement during the WET and WIT phases. The PS, the IT, and the IRS are different service providers, and the use of the IRS is not free. The IT purchases energy from PS, so it can decide how much energy to purchase and can act as the leader of the system. The PS, as a follower, provides energy to the IT to earn profits. The IT pays the cost of obtaining energy and obtains profits through transmitting information. The network optimizes the transmit power of the PS, the energy price, the transmit power allocation of the IT, and the time allocation as well as the IRS reflection coefficients in the WET and WIT phases to maximize the utilities of the PS and the IT, respectively. We use the Stackelberg game to model the interactions of different service providers and propose efficient algorithms to achieve a win-win situation for multiple service providers. The main contributions of this paper are outlined as follows.

•

To the best of our knowledge, this is the first paper to consider the IRS as an independent service provider. In contrast to [13], we deploy an IRS in order to improve the signal strength of the wireless communication system as well as to avoid possible obstruction of the signal. The IRS in [11] provides the service to the device for free, which is not possible in reality. If there is no revenue, the IRS will refuse to provide the service, so this question considers the IRS as an independent service provider and charges for the service provided. We have not set a limit on the minimum value of the signal that the IT transmits to the user, which is unreasonable; rather, we have set a limit on the signal that the IT transmits to the user through TDMA. Furthermore, we thought it was reasonable to limit the maximum transmit power of the PS. Unlike the existing works, we assume here that the signal enhancement service provided by the IRS is not free, and the cost of using the IRS is modeled as proportional to the number of IRS reflection units activated. We also assume that the IT and the PS are different service providers. We use the Stackelberg game to provide a hierarchical framework for the complex optimization interactions among multiple service providers, and to formulate the utility maximization problems for the IT and the PS as leader and follower problems, respectively.

•

To solve the non-convex follower problem, we propose an efficient alternating optimization-based algorithm that decomposes the considered problem into a transmit power optimization sub-problem and an IRS reflection coefficient optimization sub-problem and solve them alternatively and iteratively. To solve the IRS reflection coefficient optimization sub-problem, we use a transformation that converts a non-trackable equality constraint into an inequality constraint, and use a penalty-based technique to ensure the equivalence of such a transformation. We also apply the successive convex approximation (SCA) technique to find a locally optimal solution to this sub-problem.

•

We also propose a similar alternating optimization-based algorithm for the leader problem, which decomposes it into an IRS reflection coefficient optimization sub-problem, a WET time portion optimization sub-problem, a WIT time portion optimization sub-problem, a transmit power allocation sub-problem, and an energy price optimization sub-problem. The difficulty in solving this problem lies in solving the IRS reflection coefficient optimization sub-problem. We apply the closed-form fractional programming (FP) approach, the penalty-based equivalent transformation, and the SCA technique to obtain a locally optimal solution to this sub-problem. In addition, we obtain the optimal (and closed-form) solution to the other subproblems.

•

We show that the Stackelberg equilibrium between the PS and IT can be achieved through theoretical analysis and simulations. Simulation results also show that our proposed algorithm is able to maximize the utilities of the PS and IT, and that the use of the IRS significantly improves the utilities of both the PS and IT.

2. System Model

As shown in Figure 1, we consider an IRS-assisted WPCN that includes a PS, an IT, K devices, and an IRS with N reflection units. The PS, IT, and devices are all equipped with a single antenna. In order to fully investigate the role of IRS in wireless communication systems, we consider the IRS as an independent service provider, and the results provided by the computations can be directly used by other service providers. The entire transmission block of the considered network can be divided into a WET phase and a WIT phase. In the WET phase, the PS transmits energy to power the IT; in the WIT phase, the IT transmits information to the devices. The IRS enhances the signal quality of the WET and WIT phases through intelligent signal reflection. These two phases operate in a TDMA manner. Specifically, the entire transmission block is divided into $K+1$ time slots, with the WET and WIT phases occupying the first and remaining K time slots, respectively [14]. T denotes the total duration of the transmission block, and the durations of the WET and WIT phases can be written as $t_{0}T$ and ${\sum }_{k=1}^K{t}_{k}T$, respectively, where $t_0$ and $t_k$ denote the time portions of the energy transfer from the PS to the IT and the information transfer from the IT to device k, respectively, and they should be non-negative, i.e.,

$$t_k\ge 0,\forall k.$$

(1)

For simplicity, we assume a normalized transmission block duration, i.e., $T=1$ s. Thus, $t_k$ and $t_0$ should also satisfy the total transmission block duration constraint:

$$\sum _{k=1}^K{t}_k+t_0\le 1.$$

(2)

The PS, IT, and IRS are different service providers. In particular, the IT is an information service provider, the PS is an energy service provider, and the IRS is a signal enhancement service provider. The PS transfers energy to the IT and receives payment for the energy from the IT. The IT transmits information to the devices and receives payment for the information from the devices [15]. The IRS receives payment from the PS and IT in the WET and WIT phases, respectively, for its signal reflection, which improves the channel quality of both phases. The energy and information service providers try to allocate their resources to achieve their own utility maximization, and using conventional independent optimization approaches that ignore the interactions of their resource allocation strategies may cause one service provider’s utility maximizing resource allocation strategy to reduce the utility of the others. Therefore, we use the Stackelberg game to provide a hierarchical framework for the sequential strategy interactions between the information service provider (the IT) and the energy service provider (the PS). We consider a buyer-dominated energy market, where the IT is the leader who can determine the price of energy transfer and the PS is the follower [16].

The channel coefficients from the PS to the IRS, from the IRS to the IT, from the PS to the IT, from the IT to the IRS, from the IRS to the k-th device, and from the IT to the k-th device are denoted by ${\mathit{h}}_r\in {\mathbb{C}}^{N\times 1}$, ${\mathit{h}}_{it}^H\in {\mathbb{C}}^{1\times N}$, $h_{pt}^H\in {\mathbb{C}}^{1\times 1}$, ${\mathit{g}}_r\in {\mathbb{C}}^{N\times 1}$, ${\mathit{g}}_{i{d}_k}^H\in {\mathbb{C}}^{1\times N}$, and $h_k^H\in {\mathbb{C}}^{1\times 1}$, respectively. We use the quasi-static block flat-fading channel model to model all channels, and thus assume that all channel coefficients are constant in each transmission block and may vary over different blocks. To obtain the upper bound on the performance of the considered network, we assumed that the channel state information (CSI) of all channels is perfectly known [17,18,19].

In the WET phase, the PS transfers energy with transmit power ${\mathit{P}}_{\mathrm{p}}$ subject to the minimum and maximum limits:

$$0\le P_{\mathrm{p}}\le P_{\max },$$

(3)

where $P_{\max }$ is the maximum transmit power of the PS. The PS needs to pay the energy cost to its energy source according to the value of ${\mathit{P}}_{\mathrm{p}}$, and the cost is $t_0(a{P}_{\mathrm{p}}^2+b{P}_{\mathrm{p}})$, where $a>0$ and $b>0$ are predetermined cost parameters [13].

The IRS reflects the energy signal with a reflect beamforming matrix $\mathbf{\Phi }\triangleq \mathrm{diag}\left(\mathit{\phi }\right)$, where $\mathit{\phi }\triangleq {\left[w_{\mathrm{E},1},\dots ,w_{\mathrm{E},N}\right]}^T\in {\mathbb{C}}^{N\times 1}$, and the operator diag$\left(\mathit{a}\right)$ forms a diagonal matrix with its diagonal elements equal to the vector $\mathit{a}$. For the n-th reflection unit of the IRS, if it is not activated, then the n-th diagonal element of $\mathbf{\Phi }$ is equal to zero, i.e., $w_{\mathrm{E},n}=0$, and if it is activated, then $w_{\mathrm{E},n}=e^{j{\theta }_{\mathrm{E},n}}$, where ${\theta }_{\mathrm{E},n}\in \left[0,2\pi \right)$ is the reflection phase shift of the n-th reflection unit of the IRS during the WET phase. The IRS charges the PS according to the number of activated IRS reflection units during the WET phase. The fee charged by the IRS can be written as $r{∥\mathbf{\Phi }∥}_0$, where r is the price per activated reflection units of the IRS, and ${∥\mathbf{\Phi }∥}_0$ denotes the ${\ell }_0$-norm of $\mathbf{\Phi }$, representing the number of non-zero diagonal elements in $\mathbf{\Phi }$. Let $\mathbf{\Phi }(n,n)$ denote the n-th diagonal element of $\mathbf{\Phi }$. If the n-th reflection unit of the IRS is activated, then $\left|\mathbf{\Phi }\right(n,n\left)\right|=1$, and if it is not activated, then $\left|\mathbf{\Phi }\right(n,n\left)\right|=0$. This condition is equivalent to the following equality constraint

$$\left|\mathbf{\Phi }\left(n,n\right)\right|\left(1-\left|\mathbf{\Phi }\left(n,n\right)\right|\right)=0,n=1,2,\dots ,N.$$

(4)

With (4), ${∥\mathbf{\Phi }∥}_0$ is equal to the number of activated IRS reflection units.

Thus, the energy harvested by the IT is

$$\mathit{E}=t_0\eta min\left\{{\mathit{P}}_{\mathrm{p}}{\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^2,P_{\mathrm{sat}}\right\},$$

(5)

where $\eta \in \left(0,1\right]$ is the energy harvesting efficiency, $P_{\mathrm{sat}}$ is the saturation power threshold of the IT’s energy harvester [20], and the noise power at the energy harvester has been ignored [21]. The IT pays the PS according to the energy it receives. Let $\lambda$ be the price of the power received by the IT per Watt. The revenue received by the PS from the IT is $\lambda t_0{P}_{\mathrm{p}}{|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H|}^2$.

Let $U_{\mathrm{p}}\left(P_{\mathrm{p}},\mathbf{\Phi }\right)$ denote the utility of the PS, which is defined as the difference between the revenue of the PS and the payments of the PS for the IRS’s signal enhancement and the energy cost. Therefore, the utility of the PS can be expressed as

$$\begin{array}{cc}\hfill U_{\mathrm{p}}\left(P_{\mathrm{p}},\mathbf{\Phi }\right)=& \lambda t_0{P}_{\mathrm{p}}{\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^2-r{∥\mathbf{\Phi }∥}_0-t_0\left(a{P}_{\mathrm{P}}^2+b{P}_{\mathrm{p}}\right).\hfill \end{array}$$

(6)

In the WIT phase, the IT transmits information with the harvested energy to the devices to obtain revenue. Let $p_k$ denote the transmit power allocated for the information transmission of the k-th device, and the maximum transmit energy of the IT should not be greater than its harvested energy, i.e.,

$$\sum _{k=1}^K{t}_k{p}_k\le t_0\eta min\left\{{\mathit{P}}_{\mathrm{p}}{\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^2,P_{\mathrm{sat}}\right\},$$

(7)

and

$$p_k\ge 0,\forall k.$$

(8)

The IRS reflects the signal for the k-th device with a reflect beamforming matrix ${\mathbf{\Psi }}_k\triangleq \mathrm{diag}\left({\mathit{\psi }}_k\right)$, where ${\mathit{\psi }}_k\triangleq {\left[w_{k,1},\dots ,w_{k,N}\right]}^T\in {\mathbb{C}}^{N\times 1}$. For the n-th reflection unit of the IRS, if it is not activated for the k-th device, then the n-th element of ${\mathbf{\Psi }}_k$ is equal to zero, i.e., $w_{k,n}=0$, and if it is activated, then $w_{k,n}=e^{j{\theta }_{k,n}}$, where ${\theta }_{k,n}\in \left[0,2\pi \right)$ is the reflection phase shift of the n-th reflection unit of the IRS for the k-th device in the WIT phase. The IRS charges the IT for the number of activated IRS reflection units in the WIT phase. The fee charged by the IRS at time slot k is $r{∥{\mathbf{\Psi }}_k∥}_0$. Similar to $\mathbf{\Phi }$, we introduce the following constraints to ensure that ${∥{\mathbf{\Psi }}_k∥}_0$ is equal to the number of activated IRS reflection units in time slot k:

$$\begin{array}{cc}\hfill & \left|{\mathbf{\Psi }}_k\left(n,n\right)\right|\left(1-\left|{\mathbf{\Psi }}_k\left(n,n\right)\right|\right)=0,n=1,2,\dots ,N,k=1,2,\dots ,K.\hfill \end{array}$$

(9)

Thus, the achievable rate of the k-th device in bits/Hertz (bits/Hz) in time slot k can be written as

$$R_k=t_k{log}_2\left(1+{\displaystyle \frac{p_k{\left|{\mathit{g}}_{i{d}_k}^H{\mathbf{\Psi }}_k{\mathit{g}}_r+h_k^H\right|}^2}{{\sigma }^2}}\right),$$

(10)

where ${\sigma }^2$ is the noise power at each device’s receiver. Let $R_{\min }$ be the minimum rate requirement of each device, and the rates of the devices should satisfy the following minimum rate constraint:

$$t_k{log}_2\left(1+{\displaystyle \frac{p_k{\left|{\mathit{g}}_{i{d}_k}^H{\mathbf{\Psi }}_k{\mathit{g}}_r+h_k^H\right|}^2}{{\sigma }^2}}\right)\ge R_{\min },\forall k.$$

(11)

The sum rate of all devices is

$$R_{\mathrm{sum}}=\sum _{k=1}^K{R}_k.$$

(12)

Let $\beta$ be the revenue from transmitting the achievable rate per bits/Hz. The revenue of the IT during the WIT phase is $\beta R_{\mathrm{sum}}$ [22].

Let $U_{\mathrm{t}}\left(\lambda ,t_k,p_k,{\mathbf{\Psi }}_k,t_0\right)$ denote the utility of the IT, which is defined as the difference between the revenue earned from providing the information transmission service to the devices and the payments paid to the IRS for signal enhancement and the PS for power transfer. Therefore, the utility of the IT can be expressed as

$$\begin{array}{cc}\hfill U_{\mathrm{t}}\left(\lambda ,t_k,p_k,{\mathbf{\Psi }}_k,t_0\right)=& \beta \sum _{k=1}^K{t}_k{log}_2\left(1+{\displaystyle \frac{p_k{\left|{\mathit{g}}_{i{d}_k}^H{\mathbf{\Psi }}_k{\mathit{g}}_r+h_k^H\right|}^2}{{\sigma }^2}}\right)\hfill \\ \hfill & \hfill -\lambda t_0{P}_{\mathrm{p}}{\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^2-r\sum _{k=1}^K{∥{\mathbf{\Psi }}_k∥}_0.\end{array}$$

(13)

The rest of this paper is organized as follows. Section 2 describes the system model and the relevant parameters. Section 3 introduces the leader problem and the follower problem. Section 4 introduces the Stackelberg Game-Based Optimization Algorithm. Section 5 presents the experimental results of the proposed system model. Section 6 provides the conclusions of our work.

3. Problem Formulation

To achieve a win-win situation for multiple service providers, the transmit power of the PS and the IRS reflection coefficients for the WET phase are jointly optimized to maximize the utility of the PS. The energy price, the time portions of all time slots, the transmit powers of the IT for all devices, and the IRS reflection coefficients for the WIT phase are jointly optimized to maximize the utility of the IT. The optimization is performed in the Stackelberg game framework, where a leader problem and a follower problem are to be solved, respectively.

The leader problem aims to maximize the utility of the IT by jointly optimizing the IRS reflection coefficients for all devices in the WIT phase ${\left\{{\mathbf{\Psi }}_k\right\}}_1^K$, the energy price $\lambda$, the time portions of the WET and WIT phases ${\left\{t_k\right\}}_0^K$, and the IT’s transmit power allocated to the devices ${\left\{p_k\right\}}_1^K$, subject to each device’s minimum rate constraint (11), the IRS’s reflection coefficient constraint (9), the non-negative energy price constraint, the total transmission duration constraint (2), the IT’s maximum transmit energy constraint (7), the non-negative energy allocation constraint (8), and the non-negative time portion constraint (1). Thus, the leader problem can be formulated as

$$\begin{array}{cc}\hfill \underset{\lambda ,{\left\{{\mathbf{\Psi }}_k\right\}}_1^K,{\left\{t_k\right\}}_0^K,{\left\{p_k\right\}}_1^K}{max}& \beta \sum _{k=1}^K{t}_k{log}_2\left(1+{\displaystyle \frac{p_k{\left|{\mathit{g}}_{i{d}_k}^H{\mathbf{\Psi }}_k{\mathit{g}}_r+h_k^H\right|}^2}{{\sigma }^2}}\right)-\lambda t_0{P}_{\mathrm{p}}{\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^2\hfill \end{array}$$

$$\begin{array}{cc}\hfill & -r\sum _{k=1}^K{∥{\mathbf{\Psi }}_k∥}_0\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& \left(1\right),\left(2\right),\left(7\right),\left(8\right),\left(9\right),\left(11\right),\hfill \end{array}$$

(14a)

$$\begin{array}{cc}\hfill & \lambda \ge 0,\hfill \end{array}$$

(14b)

The follower problem aims to maximize the utility of the PS by jointly optimizing the PS’s transmit power $P_{\mathrm{p}}$ and the IRS reflection coefficients for the WET phase $\mathbf{\Phi }$, subject to the constraints of the maximum transmit power of the PS in (3) and the IRS’s reflection coefficients in (4). Thus, the follower problem can be formulated as

$$\begin{array}{cc}\hfill \underset{P_{\mathrm{p}},\mathbf{\Phi }}{max}& \lambda t_0{P}_{\mathrm{p}}{\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^2-t_0\left(a{P}_{\mathrm{p}}^2+b{P}_{\mathrm{p}}\right)-r{∥\mathbf{\Phi }∥}_0\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& \left(3\right),\left(4\right).\hfill \end{array}$$

(15a)

The leader problem and the follower problem together form a Stackelberg game problem. We consider using the backward induction method to solve this problem [21]. Specifically, we first solve the follower problem (15). Then we substitute the obtained solution $\mathbf{\Phi }$ and $P_{\mathrm{p}}$ into the leader problem (14) and solve it.

4. Stackelberg Game-Based Optimization Algorithm

In this section, we first solve the follower problem (15) and the leader problem (14), respectively. We then prove that the Stackelberg equilibrium can be achieved, resulting in a win-win situation for the multiple service providers in the considered network.

4.1. Proposed Algorithm for the Follower Problem (15)

In problem (15), the optimization variables $P_{\mathrm{p}}$ and $\mathbf{\Phi }$ are coupled in the objective function but are not in the constraints. This motivates us to solve problem (15) by decomposing it into two sub-problems and solving them alternatively. Specifically, Sub-Problem 1 optimizies $\mathbf{\Phi }$ with fixed $P_{\mathrm{p}}$, and Sub-Problem 2 optimizies $P_{\mathrm{p}}$ with fixed $\mathbf{\Phi }$. The procedures for solving the sub-problems and the overall algorithm are presented below.

4.1.1. Solution to Sub-Problem 1

Sub-Problem 1 optimizes $\mathbf{\Phi }$ with fixed $P_{\mathrm{p}}$. Since ${\parallel \mathbf{\Phi }\parallel }_0$ in (15a) is non-convex, $|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H{|}^2$ in (15a) is difficult to handle, and the left-hand side (LHS) of (4) is non-linear. Sub-Problem 1 is a non-convex optimization problem and is difficult to solve. We tackle these difficulties as follows. First, the modulus of the diagonal elements of $\mathbf{\Phi }$ is either 0 or 1, so ${∥\mathbf{\Phi }∥}_0$ = ${∥\mathbf{\Phi }∥}_F^2$ = ${∥\mathit{\phi }∥}_2^2$, where ${∥\mathbf{\Phi }∥}_F$ denotes the Frobenius norm of $\mathbf{\Phi }$, and ${∥\mathit{\phi }∥}_2$ denotes the ${\ell }_2$-norm of $\mathit{\phi }$. Second, we define $\mathit{m}\triangleq \mathrm{diag}\left({\mathit{h}}_{it}^H\right){\mathit{h}}_r\in {\mathbb{C}}^{N\times 1}$, and $|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H{|}^2$ in (15a) can be transformed to $|{\mathit{\phi }}^H\mathit{m}+h_{pt}^H{|}^2$. Third, $|\mathbf{\Phi }\left(n,n\right)|\left(1-|\mathbf{\Phi }\left(n,n\right)|\right)$ in (4) can be written as ${\mathit{\phi }}^H{\mathit{e}}_i{\mathit{e}}_i^H\mathit{\phi }\left(1-{\mathit{\phi }}^H{\mathit{e}}_i{\mathit{e}}_i^H\mathit{\phi }\right)$, where ${\mathit{e}}_i$ represents a vector where the i-th element is 1 and the other elements are 0. With the above transformations, we further introduce auxiliary variables $\mathit{\omega }\triangleq {\left[{\omega }_1,\dots ,{\omega }_N\right]}^T$ and a penalty factor $\mu$ to resolve the non-linear issue of (4), and then we can write Problem (15) into a more tractable form:

$$\begin{array}{cc}\hfill \underset{\mathit{\phi },{\left\{{\omega }_i\right\}}_1^N}{max}& \lambda t_0{P}_{\mathrm{p}}{\left|{\mathit{\phi }}^H\mathit{m}+h_{pt}^H\right|}^2-r{∥\mathit{\phi }∥}_2^2-\mu \sum _{i=1}^N{\omega }_i\hfill \end{array}$$

(16a)

$$\begin{array}{cc}\hfill \mathrm{s}.\mathrm{t}.& 0\le {\mathit{\phi }}^H{\mathit{e}}_i{\mathit{e}}_i^H\mathit{\phi }\left(1-{\mathit{\phi }}^H{\mathit{e}}_i{\mathit{e}}_i^H\mathit{\phi }\right)\le {\omega }_i,i=1,2,\dots ,N.\hfill \end{array}$$

(16b)

Note that the penalty factor $\mu$ can guarantee that the solution to Problem (16) can satisfy constraint (4). This is because, when $\mu \to \infty$, the optimal solution of ${\omega }_i\to 0$, and in this case, (16b) is equivalent to (4). Therefore, we can solve Problem (16) to obtain the solution to Sub-Problem 1.

Since the term ${\left|{\mathit{\phi }}^H\mathit{m}+h_{pt}^H\right|}^2$ in (16a) is non-convex with respect to $\mathit{\phi }$, and (16b) is a non-convex constraint, Problem (16) is still difficult to solve. We use the SCA technique to solve Problem (16) iteratively. We suppose that ${\mathit{\phi }}^{\left(u\right)}$ is the solution of (16) obtained at the u-th iteration. The first-order Taylor expansions of ${\left|{\mathit{\phi }}^H\mathit{m}+h_{pt}^H\right|}^2$ and ${\mathit{\phi }}^H{\mathit{e}}_i{\mathit{e}}_i^H\mathit{\phi }\left(1-{\mathit{\phi }}^H{\mathit{e}}_i{\mathit{e}}_i^H\mathit{\phi }\right)$ at ${\mathit{\phi }}^{\left(u\right)}$ can be respectively written as

$$\begin{array}{c}\hfill -{\left|{{\mathit{\phi }}^H}^{\left(u\right)}\mathit{m}+h_{pt}^H\right|}^2+2\mathrm{Re}\left\{{\left({\mathit{\phi }}^H\mathit{m}+h_{pt}^H\right)}^H\left({{\mathit{\phi }}^H}^{\left(u\right)}\mathit{m}+h_{pt}^H\right)\right\},\end{array}$$

(17)

and

$$\begin{array}{cc}\hfill & {{\mathit{\phi }}^H}^{\left(u\right)}{\mathit{e}}_i{\mathit{e}}_i^H{\mathit{\phi }}^{\left(u\right)}\left(1-{{\mathit{\phi }}^H}^{\left(u\right)}{\mathit{e}}_i{\mathit{e}}_i^H{\mathit{\phi }}^{\left(u\right)}\right)\hfill \\ \hfill & +{\left(2{\mathit{e}}_i{\mathit{e}}_i^H{\mathit{\phi }}^{\left(u\right)}-4{\mathit{e}}_i{\mathit{e}}_i^H{\mathit{\phi }}^{\left(u\right)}{{\mathit{\phi }}^H}^{\left(u\right)}{\mathit{e}}_i{\mathit{e}}_i^H{\mathit{\phi }}^{\left(u\right)}\right)}^T\left(\mathit{\phi }-{\mathit{\phi }}^{\left(u\right)}\right),\hfill \end{array}$$

(18)

where $\mathrm{Re}\left\{·\right\}$ denotes the real part operator. By replacing ${\left|{\mathit{\phi }}^H\mathit{m}+h_{pt}^H\right|}^2$ and ${\mathit{\phi }}^H{\mathit{e}}_i{\mathit{e}}_i^H\mathit{\phi }\left(1-{\mathit{\phi }}^H{\mathit{e}}_i{\mathit{e}}_i^H\mathit{\phi }\right)$ with (17) and (18) in Problem (16), respectively, the problem to be solved in the $(u+1)$-th iteration can be constructed as

$$\begin{array}{cc}\hfill \underset{\mathit{\phi },{\left\{{\omega }_i\right\}}_1^N}{max}& \lambda t_0{P}_{\mathrm{p}}\left(-{\left|{{\mathit{\phi }}^H}^{\left(u\right)}\mathit{m}+h_{pt}^H\right|}^2\right)+2\lambda t_0{P}_{\mathrm{p}}\mathrm{Re}\left\{{\left({\mathit{\phi }}^H\mathit{m}+h_{pt}^H\right)}^H\left({{\mathit{\phi }}^H}^{\left(u\right)}\mathit{m}+h_{pt}^H\right)\right\}\hfill \\ \hfill & -r{∥\mathit{\phi }∥}_2^2-\mu \sum _{i=1}^N{\omega }_i\hfill \end{array}$$

(19a)

$$\begin{array}{cc}\hfill \mathrm{s}.\mathrm{t}.& 0\le {{\mathit{\phi }}^H}^{\left(u\right)}{\mathit{e}}_i{\mathit{e}}_i^H{\mathit{\phi }}^{\left(u\right)}\left(1-{{\mathit{\phi }}^H}^{\left(u\right)}{\mathit{e}}_i{\mathit{e}}_i^H{\mathit{\phi }}^{\left(u\right)}\right)\hfill \\ \hfill & +{\left(2{\mathit{e}}_i{\mathit{e}}_i^H{\mathit{\phi }}^{\left(u\right)}-4{\mathit{e}}_i{\mathit{e}}_i^H{\mathit{\phi }}^{\left(u\right)}{{\mathit{\phi }}^H}^{\left(u\right)}{\mathit{e}}_i{\mathit{e}}_i^H{\mathit{\phi }}^{\left(u\right)}\right)}^T\left(\mathit{\phi }-{\mathit{\phi }}^{\left(u\right)}\right)\le {\omega }_i,\hfill \\ \hfill & i=1,2,\dots ,N.\hfill \end{array}$$

(19b)

Problem (19) is a convex optimization problem and can be solved by a standard solver like CVX.

4.1.2. Solution to Sub-Problem 2

Sub-Problem 2 optimizes $P_{\mathrm{p}}$ with fixed $\mathbf{\Phi }$ in Problem (15) and can be written as

$$\begin{array}{cc}\hfill \underset{P_{\mathrm{p}}}{max}& \lambda t_0{P}_{\mathrm{p}}{\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^2-t_0\left(a{P}_{\mathrm{p}}^2+b{P}_{\mathrm{p}}\right)\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& \left(3\right).\hfill \end{array}$$

(20a)

The objective function (20a) is a quadratic concave function of $P_{\mathrm{p}}$, and the constraint (3) is linear, so Problem (20) is a convex optimization problem. By setting the first-order derivative of (20a) to zero, we have the following equation:

$$\lambda t_0{\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^2-2a{t}_0{P}_{\mathrm{p}}-b{t}_0=0.$$

(21)

By solving (21), we obtain its solution $P_{\mathrm{p}}={\displaystyle \frac{\lambda {\gamma }_e-b}{2a}}$, where ${\gamma }_e={|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H|}^2$. Let $P_{\mathrm{p}}^{*}$ denote the optimal solution to Problem (20), and according to the constraint (3), we can obtain $P_{\mathrm{p}}^{*}$ as

$$\begin{array}{c}\hfill P_{\mathrm{p}}^{*}=\left\{\begin{array}{c}\hfill {\displaystyle \frac{\lambda {\gamma }_e-b}{2a}},0\le {\displaystyle \frac{\lambda {\gamma }_e-b}{2a}}\le P_{\max },\\ \hfill 0,{\displaystyle \frac{\lambda {\gamma }_e-b}{2a}}\le 0,\\ \hfill P_{\max },{\displaystyle \frac{\lambda {\gamma }_e-b}{2a}}\ge P_{\max }.\end{array}\right.\end{array}$$

(22)

4.1.3. Overall Algorithm for Problem (15)

The overall algorithm for Problem (15) is summarized in Algorithm 1, where $U_{\mathrm{p}}^{\left(t\right)}=g(P_{\mathrm{p}}^{\left(t\right)},{\mathit{\phi }}^{\left(t\right)})$ denotes the objective value of Problem (15) with the variables $P_{\mathrm{p}}^{\left(t\right)}$ and ${\mathit{\phi }}^{\left(t\right)}$. The algorithm solves Sub-Problems 1 and 2 in Steps 3–6 and Step 7, respectively. Since the locally optimal solution to Sub-Problem 1 and the optimal solution to Sub-Problem 2 can be obtained at each iteration, the objective function value of Problem (15) is monotonically non-decreasing. Furthermore, since $P_{\mathrm{p}}$ is upper bounded by $P_{\max }$, there must be an upper bound on the objective function value of Problem (15). Therefore, Algorithm 1 is guaranteed to converge (Lemma 2, [23]). The flowchart of Algorithm 1 is shown in Figure 2.

Algorithm 1 Proposed Algorithm for Problem (15).

1: Initialization: Set initial value for $P_{\mathrm{p}}^{\left(0\right)}$. Set $U_{\mathrm{p}}^{\left(0\right)}$ = 0, $t=0$, ${\mathit{\epsilon }}_0=1\times {10}^{-3}$, and ${\chi }_0=1\times {10}^{-8}$. 2: repeat 3:       repeat 4:          Set initial value for $\mu$. 5:          Given $P_{\mathrm{p}}^{\left(t\right)}$, solve problem (16) and denote the obtained solution as ${\mathit{\phi }}^{\left(t+1\right)}$. 6:          If ${\mathit{\phi }}^H{\mathit{e}}_i{\mathit{e}}_i^H\mathit{\phi }\left(1-{\mathit{\phi }}^H{\mathit{e}}_i{\mathit{e}}_i^H\mathit{\phi }\right)>{\chi }_0$, set $\mu =2\mu$. 7:       until ${\mathit{\phi }}^H{\mathit{e}}_i{\mathit{e}}_i^H\mathit{\phi }\left(1-{\mathit{\phi }}^H{\mathit{e}}_i{\mathit{e}}_i^H\mathit{\phi }\right)\le {\chi }_0$. 8:       Given ${\mathit{\phi }}^{\left(t+1\right)}$, obtain $P_{\mathrm{p}}^{\left(t+1\right)}$ according to (22). 9:       Set $t=t+1$. 10:     Set $U_{\mathrm{p}}^{\left(t\right)}=g(P_{\mathrm{p}}^{\left(t\right)},{\mathit{\phi }}^{\left(t\right)})$. 11: until ${\displaystyle \frac{\left|U_{\mathrm{p}}^{\left(t\right)}-U_{\mathrm{p}}^{\left(t-1\right)}\right|}{U_{\mathrm{p}}^{\left(t\right)}}}\le {\epsilon }_0$.

4.2. Proposed Algorithm for the Leader Problem (14)

We substitute the obtained solution to the follower problem (15), i.e., $P_{\mathrm{p}}^{*}$ and $\mathit{\phi }$, into the leader problem (14), and solve it as follows. In Problem (14), the optimization variables $\lambda ,{\left\{t_k\right\}}_0^K,{\left\{p_k\right\}}_1^K$, and ${\left\{{\mathbf{\Psi }}_k\right\}}_1^K$ are intricately coupled in the objective function, and the objective function is not jointly concave with respect to them. Furthermore, the LHS of the equality constraint (9) is non-linear, so Problem (14) is non-convex and difficult to solve optimally. To solve this problem efficiently, we use the alternating optimization algorithm to decouple the optimization variables and decompose Problem (14) into five sub-problems that are solved alternately. Specifically, Sub-Problem 1 optimizes ${\left\{{\mathbf{\Psi }}_k\right\}}_1^K$ with a fixed $\lambda$, ${\left\{t_k\right\}}_0^K$, and ${\left\{p_k\right\}}_1^K$; Sub-Problem 2 optimizes $t_0$ with a fixed $\lambda$, ${\left\{{\mathbf{\Psi }}_k\right\}}_1^K$, ${\left\{t_k\right\}}_1^K$, and ${\left\{p_k\right\}}_1^K$; Sub-Problem 3 optimizes ${\left\{t_k\right\}}_1^K$ with a fixed $\lambda$, $t_0$, ${\left\{{\mathbf{\Psi }}_k\right\}}_1^K$, and ${\left\{p_k\right\}}_1^K$; Sub-Problem 4 optimizes ${\left\{p_k\right\}}_1^K$ with a fixed $\lambda$, ${\left\{t_k\right\}}_0^K$, and ${\left\{{\mathbf{\Psi }}_k\right\}}_1^K$; Sub-Problem 5 optimizes $\lambda$ with a fixed ${\left\{{\mathbf{\Psi }}_k\right\}}_1^K$, ${\left\{t_k\right\}}_0^K$, and ${\left\{p_k\right\}}_1^K$. The procedures for solving all sub-problems and the overall algorithm are presented below.

4.2.1. Solution to Sub-Problem 1

Sub-Problem 1 optimizes ${\left\{{\mathbf{\Psi }}_k\right\}}_1^K$ in Problem (14) with a fixed $\lambda$, ${\left\{t_k\right\}}_0^K$, and ${\left\{p_k\right\}}_1^K$. Similar to (15), by setting ${\mathit{g}}_k\triangleq \mathrm{diag}\left({\mathit{g}}_{i{d}_k}^H\right){\mathit{g}}_r$ and replacing $\left|{\mathbf{\Psi }}_k\left(n,n\right)\right|\left(1-\left|{\mathbf{\Psi }}_k\left(n,n\right)\right|\right)$ with ${\mathit{\psi }}_k^H{\mathit{e}}_i{\mathit{e}}_i^H{\mathit{\psi }}_k(1-{\mathit{\psi }}_k^H{\mathit{e}}_i{\mathit{e}}_i^H{\mathit{\psi }}_k)$ in (9), we can write Sub-Problem 1 as

$$\begin{array}{cc}\hfill \underset{{\left\{{\mathit{\psi }}_k\right\}}_1^K}{max}& \beta \sum _{k=1}^K{t}_k{log}_2\left(1+{\displaystyle \frac{p_k{\left|{{\mathit{\psi }}_k}^H{\mathit{g}}_k+h_k^H\right|}^2}{{\sigma }^2}}\right)-r\sum _{k=1}^K{∥{\mathit{\psi }}_k∥}_2^2\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& \left(11\right),\hfill \end{array}$$

(23a)

$$\begin{array}{cc}\hfill & {\mathit{\psi }}_k^H{\mathit{e}}_i{\mathit{e}}_i^H{\mathit{\psi }}_k\left(1-{\mathit{\psi }}_k^H{\mathit{e}}_i{\mathit{e}}_i^H{\mathit{\psi }}_k\right)=0,i=1,2,\dots ,N,k=1,2,\dots ,K.\hfill \end{array}$$

(23b)

The logarithmic functions in (23a) are non-concave with respect to ${\mathit{\psi }}_k$, so Problem (23) is still difficult to solve. We apply the closed-form FP approach [24] to solve it efficiently, which involves the following two steps.

Step 1: Lagrangian Dual Transform. By introducing auxiliary variables $\mathit{s}\triangleq {\left[s_1,\dots ,s_K\right]}^T$ and after some manipulations, Problem (23) can be equivalently transformed into the following form [24]

$$\begin{array}{cc}\hfill \underset{{\left\{{\mathit{\psi }}_k\right\}}_1^K,{\left\{s_k\right\}}_1^K}{max}& \beta \sum _{k=1}^K{t}_k{log}_2\left(1+s_k\right)-\beta \sum _{k=1}^K{t}_k{s}_k+\beta \sum _{k=1}^K{t}_k{\displaystyle \frac{\left(1+s_k\right)\left(\frac{p_k\left|{{\mathit{\psi }}_k}^H{\mathit{g}}_k+h_k^H\right|}{{\sigma }^2}\right)}{\left(1+\frac{p_k{\left|{{\mathit{\psi }}_k}^H{\mathit{g}}_k+h_k^H\right|}^2}{{\sigma }^2}\right)}}\hfill \\ \hfill & -r\sum _{k=1}^K{∥{\mathit{\psi }}_k∥}_2^2\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& \left(11\right),\left(\text{23b}\right).\hfill \end{array}$$

(24a)

Since ${\left\{{\mathit{\psi }}_k\right\}}_1^K$ and ${\left\{s_k\right\}}_1^K$ are coupled in Problem (24), we solve it by iteratively updating ${\left\{{\mathit{\psi }}_k\right\}}_1^K$ and ${\left\{s_k\right\}}_1^K$. When ${\left\{{\mathit{\psi }}_k\right\}}_1^K$ is fixed, Problem (24) is a convex optimization problem. Therefore, by solving the equation that the first-order derivative of (24a) is zero, the optimal solution of ${\left\{s_k\right\}}_1^K$ can be obtained:

$$s_k^{*}={\displaystyle \frac{p_k{\left|{{\mathit{\psi }}_k}^H{\mathit{g}}_k+h_k^H\right|}^2}{{\sigma }^2}},\forall k.$$

(25)

When ${\left\{s_k\right\}}_1^K$ are fixed, by defining $\widehat{s_k}\triangleq t_k\left(1+s_k^{*}\right){\displaystyle \frac{p_k}{{\sigma }^2}}$, Problem (24) can be reduced to

$$\begin{array}{cc}\hfill \underset{{\left\{{\mathit{\psi }}_k\right\}}_1^K}{max}& \sum _{k=1}^K{\displaystyle \frac{\widehat{s_k}\left({\left|{{\mathit{\psi }}_k}^H{\mathit{g}}_k+h_k^H\right|}^2\right)}{1+\frac{p_k{\left|{{\mathit{\psi }}_k}^H{\mathit{g}}_k+h_k^H\right|}^2}{{\sigma }^2}}}-r\sum _{k=1}^K{∥{\mathit{\psi }}_k∥}_2^2\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& \left(11\right),\left(\text{23b}\right).\hfill \end{array}$$

(26a)

Step 2: Quadratic Transform. Problem (26) is a multiple-ratio FP problem. To solve this problem, we introduce auxiliary variables $\mathit{\rho }\triangleq {\left[{\rho }_1,\dots ,{\rho }_K\right]}^T$ and transform it equivalently into the following form [24]:

$$\begin{array}{cc}\hfill \underset{{\left\{{\mathit{\psi }}_k\right\}}_1^K,\rho }{max}& 2\sum _{k=1}^K\sqrt{\widehat{s_k}}\mathrm{Re}\left\{{\rho }_k^H\left({{\mathit{\psi }}_k}^H{\mathit{g}}_k+h_k^H\right)\right\}-\sum _{k=1}^K{\left|{\rho }_k\right|}^2\left(1+{\displaystyle \frac{p_k{\left|{{\mathit{\psi }}_k}^H{\mathit{g}}_k+h_k^H\right|}^2}{{\sigma }^2}}\right)\hfill \\ \hfill & -r\sum _{k=1}^K{∥{\mathit{\psi }}_k∥}_2^2\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& \left(11\right),\left(\text{23b}\right).\hfill \end{array}$$

(27a)

Since ${\left\{{\mathit{\psi }}_k\right\}}_1^K$ and $\mathit{\rho }$ are coupled in Problem (27), we solve it by iteratively updating ${\left\{{\mathit{\psi }}_k\right\}}_1^K$ and $\mathit{\rho }$. When ${\left\{{\mathit{\psi }}_k\right\}}_1^K$ are fixed, the optimal solution for $\mathit{\rho }$ is

$${\rho }_k^{*}={\displaystyle \frac{\sqrt{\widehat{s_k}}\left({{\mathit{\psi }}_k}^H{\mathit{g}}_k+h_k^H\right)}{1+\frac{p_k{\left|{{\mathit{\psi }}_k}^H{\mathit{g}}_k+h_k^H\right|}^2}{{\sigma }^2}}},\forall k.$$

(28)

By substituting (28) into Problem (27), we can find that Problems (27) and (26) are equivalent, so we will focus on solving Problem (27) in the following.

With given ${\rho }_k^{*}$, similar to Problem (15), we introduce auxiliary variables $\mathit{\kappa }\triangleq {\left[{\kappa }_1,\dots ,{\kappa }_N\right]}^T$ and a penalty factor $\nu$ into Problem (27) to resolve the non-linear issue of (23b), and we transform Problem (27) into

$$\begin{array}{cc}\hfill \underset{{\left\{{\mathit{\psi }}_k\right\}}_1^K,{\left\{{\kappa }_i\right\}}_1^N}{max}& 2\sum _{k=1}^K\sqrt{\widehat{s_k}}\mathrm{Re}\left\{{\rho }_k^H\left({{\mathit{\psi }}_k}^H{\mathit{g}}_k+h_k^H\right)\right\}-\sum _{k=1}^K{\left|{\rho }_k\right|}^2\left(1+{\displaystyle \frac{p_k{\left|{{\mathit{\psi }}_k}^H{\mathit{g}}_k+h_k^H\right|}^2}{{\sigma }^2}}\right)\hfill \\ \hfill & -r\sum _{k=1}^K{∥{\mathit{\psi }}_k∥}_2^2-\nu \sum _{i=1}^N{\kappa }_i\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& \left(11\right),\hfill \end{array}$$

(29a)

$$\begin{array}{cc}\hfill & 0\le {\mathit{\psi }}_k^H{\mathit{e}}_i{\mathit{e}}_i^H{\mathit{\psi }}_k\left(1-{\mathit{\psi }}_k^H{\mathit{e}}_i{\mathit{e}}_i^H{\mathit{\psi }}_k\right)\le {\kappa }_i,i=1,2,\dots ,N,k=1,2,\dots ,K.\hfill \end{array}$$

(29b)

Problem (29) is difficult to solve since the constraints (11) and (29b) are non-convex. We use the SCA technique to overcome such a difficulty. Similar to (15), we assume that ${\mathit{\psi }}_k^{\left(q\right)}$ is the solution obtained at the q-th iteration, by replacing the terms ${\left|{{\mathit{\psi }}_k}^H{\mathit{g}}_k+h_k^H\right|}^2$ in (11) and ${\mathit{\psi }}_k^H{\mathit{e}}_i{\mathit{e}}_i^H{\mathit{\psi }}_k\left(1-{\mathit{\psi }}_k^H{\mathit{e}}_i{\mathit{e}}_i^H{\mathit{\psi }}_k\right)$ in (29b) with their first-order Taylor expansions at ${\mathit{\psi }}_k^{\left(q\right)}$, respectively, the problem to be solved in the (q + 1)-th iteration can be constructed as

$$\begin{array}{cc}\hfill \underset{{\left\{{\mathit{\psi }}_k\right\}}_1^K,{\left\{{\kappa }_i\right\}}_1^N}{max}& 2\sum _{k=1}^K\sqrt{\widehat{s_k}}\mathrm{Re}\left\{{\rho }_k^H\left({{\mathit{\psi }}_k}^H{\mathit{g}}_k+h_k^H\right)\right\}-\sum _{k=1}^K{\left|{\rho }_k\right|}^2\left(1+{\displaystyle \frac{p_k{\left|{{\mathit{\psi }}_k}^H{\mathit{g}}_k+h_k^H\right|}^2}{{\sigma }^2}}\right)\hfill \end{array}$$

$$\begin{array}{cc}\hfill & -r\sum _{k=1}^K{∥{\mathit{\psi }}_k∥}_2^2-\nu \sum _{i=1}^N{\kappa }_i\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& -{\left|{{{\mathit{\psi }}_k}^H}^{\left(q\right)}{\mathit{g}}_k+h_k^H\right|}^2+2\mathrm{Re}\left\{{\left({\mathit{\psi }}_k^H\mathit{m}+h_k^H\right)}^H\left({{{\mathit{\psi }}_k}^H}^{\left(q\right)}\mathit{m}+h_k^H\right)\right\}\hfill \end{array}$$

(30a)

$$\begin{array}{cc}\hfill & \ge {\displaystyle \frac{\left(2^{\frac{{\mathrm{R}}_{\min }}{t_k}}-1\right){\sigma }^2}{p_k}},k=1,2,\dots ,K,\hfill \\ \hfill & 0\le {{{\mathit{\psi }}_k}^H}^{\left(q\right)}{\mathit{e}}_i{\mathit{e}}_i^H{\mathit{\psi }}_k^{\left(q\right)}\left(1-{{{\mathit{\psi }}_k}^H}^{\left(q\right)}{\mathit{e}}_i{\mathit{e}}_i^H{\mathit{\psi }}_k^{\left(q\right)}\right)\hfill \\ \hfill & +{\left(2{\mathit{e}}_i{\mathit{e}}_i^H{\mathit{\psi }}_k^{\left(q\right)}-4{\mathit{e}}_i{\mathit{e}}_i^H{\mathit{\psi }}_k^{\left(q\right)}{{{\mathit{\psi }}_k}^H}^{\left(q\right)}{\mathit{e}}_i{\mathit{e}}_i^H{\mathit{\psi }}_k^{\left(q\right)}\right)}^T\left({\mathit{\psi }}_k-{\mathit{\psi }}_k^{\left(q\right)}\right)\le {\kappa }_i,\hfill \end{array}$$

(30b)

$$\begin{array}{cc}\hfill & i=1,2,\dots ,N,k=1,2,\dots ,K.\hfill \end{array}$$

(30c)

Problem (30) is a convex optimization problem, and we can use a standard solver like CVX to solve it efficiently.

4.2.2. Solution to Sub-Problem 2

Sub-Problem 2 optimizes $t_0$ with a fixed ${\left\{{\mathbf{\Psi }}_k\right\}}_1^K$, $\lambda$, ${\left\{p_k\right\}}_1^K$, and ${\left\{t_k\right\}}_1^K$, and it can be written as

$$\underset{t_0}{max}-\lambda t_0{P}_{\mathrm{p}}^{*}{\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^2$$

(31a)

$$\mathrm{s}.\mathrm{t}.\sum _{k=1}^K{t}_k+t_0\le 1,$$

(31b)

$$\begin{array}{cc}\hfill & t_0\ge 0,\hfill \end{array}$$

(31c)

$$\begin{array}{cc}\hfill & \sum _{k=1}^K{t}_k{p}_k\le t_0\eta min\left\{{\mathit{P}}_{\mathrm{p}}^{*}{\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^2,P_{\mathrm{sat}}\right\}.\hfill \end{array}$$

(31d)

According to (31b–d), we obtain $t_0\le 1-{\sum }_{k=1}^K{t}_k$, $t_0\ge 0$, and $t_0\ge ϵ$, respectively, where $ϵ={\displaystyle \frac{{\sum }_{k=1}^K{t}_k{p}_k}{\eta min\left\{P_{\mathrm{p}}^{*}{\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^2,P_{\mathrm{sat}}\right\}}}$. Based on these inequalities and the fact that (31a) is decreasing with $t_0$, we can obtain the optimal solution to Problem (31), denoted by $t_0^{*}$, in the following expression

$$\begin{array}{c}\hfill t_0^{*}=\left\{\begin{array}{cc}\hfill ϵ,& ϵ>0,\hfill \\ \hfill 0,& \mathrm{otherwise}.\hfill \end{array}\right.\end{array}$$

(32)

4.2.3. Solution to Sub-Problem 3

Sub-Problem 3 optimizes ${\left\{t_k\right\}}_1^K$ with a fixed ${\left\{{\mathbf{\Psi }}_k\right\}}_1^K$, $\lambda$, $t_0$, and ${\left\{p_k\right\}}_1^K$, and can be written as

$$\underset{{\left\{t_k\right\}}_1^K}{max}\beta \sum _{k=1}^K{t}_k{log}_2\left(1+{\displaystyle \frac{p_k{\left|{\mathit{g}}_{i{d}_k}^H{\mathbf{\Psi }}_k{\mathit{g}}_r+h_k^H\right|}^2}{{\sigma }^2}}\right)$$

(33a)

$$\mathrm{s}.\mathrm{t}.t_k{log}_2\left(1+{\displaystyle \frac{p_k{\left|{\mathit{g}}_{i{d}_k}^H{\mathbf{\Psi }}_k{\mathit{g}}_r+h_k^H\right|}^2}{{\sigma }^2}}\right)\ge R_{\min },k=1,2,\dots ,K,$$

(33b)

$$\begin{array}{cc}\hfill & \sum _{k=1}^K{t}_k+t_0\le 1,t_k\ge 0,k=1,2,\dots ,K,\hfill \end{array}$$

(33c)

$$\begin{array}{cc}\hfill & \sum _{k=1}^K{t}_k{p}_k\le t_0\eta min\left\{{\mathit{P}}_{\mathrm{p}}^{*}{\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^2,P_{\mathrm{sat}}\right\}.\hfill \end{array}$$

(33d)

Problem (33) is a linear programming problem and can be solved with a standard solver such as CVX.

4.2.4. Solution to Sub-Problem 4

Sub-Problem 4 optimizes ${\left\{p_k\right\}}_1^K$ with a fixed ${\left\{{\mathbf{\Psi }}_k\right\}}_1^K$, $\lambda$, and ${\left\{t_k\right\}}_0^K$, and can be written as

$$\underset{{\left\{p_k\right\}}_1^K}{max}\beta \sum _{k=1}^K{t}_k{log}_2\left(1+{\displaystyle \frac{p_k{\left|{\mathit{g}}_{i{d}_k}^H{\mathbf{\Psi }}_k{\mathit{g}}_r+h_k^H\right|}^2}{{\sigma }^2}}\right)$$

(34a)

$$\mathrm{s}.\mathrm{t}.t_k{log}_2\left(1+{\displaystyle \frac{p_k{\left|{\mathit{g}}_{i{d}_k}^H{\mathbf{\Psi }}_k{\mathit{g}}_r+h_k^H\right|}^2}{{\sigma }^2}}\right)\ge R_{\min },k=1,2,\dots ,K,$$

(34b)

$$\begin{array}{cc}\hfill & p_k\ge 0,k=1,2,\dots ,K,\hfill \end{array}$$

(34c)

$$\begin{array}{cc}\hfill & \sum _{k=1}^K{t}_k{p}_k\le t_0\eta min\left\{{\mathit{P}}_{\mathrm{p}}^{*}{\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^2,P_{\mathrm{sat}}\right\}.\hfill \end{array}$$

(34d)

Problem (34) is a convex optimization problem and can be solved with a standard solver such as CVX.

4.2.5. Solution to Sub-Problem 5

Sub-Problem 5 optimizes $\lambda$ with a fixed ${\left\{{\mathbf{\Psi }}_k\right\}}_1^K$, ${\left\{p_k\right\}}_1^K$, and ${\left\{t_k\right\}}_0^K$, and can be written as

$$\underset{\lambda }{max}-\lambda t_0{P}_{\mathrm{p}}^{*}{\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^2$$

(35a)

$$\mathrm{s}.\mathrm{t}.\sum _{k=1}^K{t}_k{p}_k\le t_0\eta min\left\{{\mathit{P}}_{\mathrm{p}}^{*}{\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^2,P_{\mathrm{sat}}\right\},$$

(35b)

$$\begin{array}{cc}\hfill & \lambda \ge 0.\hfill \end{array}$$

(35c)

Note that, according to (22), $P_{\mathrm{p}}^{*}$ in (35a,b) is a function of $\lambda$. According to the constraint (35b), Problem (35) can be solved in the cases of ${\mathit{P}}_{\mathrm{p}}^{*}{|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H|}^2\le P_{\mathrm{sat}}$ and ${\mathit{P}}_{\mathrm{p}}^{*}{|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H|}^2>P_{\mathrm{sat}}$, respectively.

When ${\mathit{P}}_{\mathrm{p}}^{*}{\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^2\le P_{\mathrm{sat}}$, according to the expression of $P_{\mathrm{p}}^{*}$ given in (22), there are three cases to consider. Case 1: if $P_{\mathrm{p}}^{*}={\displaystyle \frac{\lambda {\gamma }_e-b}{2a}}$, Problem (35) can be written as

$$\underset{\lambda }{max}-{\displaystyle \frac{t_0{\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^2}{2a}}\left({\gamma }_e{\lambda }^2-b\lambda \right)$$

(36a)

$$\begin{array}{cc}\hfill \mathrm{s}.\mathrm{t}.& \sum _{k=1}^K{t}_k{p}_k\le {\displaystyle \frac{t_0\eta {\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^2}{2a}}\left(\lambda {\gamma }_e-b\right),\hfill \\ \hfill & \left(\text{35c}\right).\hfill \end{array}$$

(36b)

$$\begin{array}{c}\hfill {\lambda }^{*}=\left\{\begin{array}{c}{\displaystyle \frac{2a{\sum }_{k=1}^K{t}_k{p}_k+b\eta t_0{\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^2}{\eta t_0{\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^4}}, P_{\mathrm{p}}^{*}={\displaystyle \frac{\lambda {\gamma }_e-b}{2a}},\hfill \\ 0, P_{\mathrm{p}}^{*}=0,\hfill \\ {\displaystyle \frac{2a{P}_{\max }+b}{{\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^2}},P_{\mathrm{p}}^{*}=P_{\max }.\hfill \end{array}\right.\end{array}$$

(37)

By analyzing the first-order derivative of (36a) and according to the constraints (36b) and (35c), the optimal solution of Problem (36), denoted by ${\lambda }^{*}$, should be ${\lambda }^{*}={\displaystyle \frac{2a{\sum }_{k=1}^K{t}_k{p}_k+b\eta t_0{\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^2}{\eta t_0{\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^4}}$. Case 2: if $P_{\mathrm{p}}^{*}=0$, then ${\lambda }^{*}=0$. Case 3: if $P_{\mathrm{p}}^{*}=P_{\max }$, then according to the condition ${\displaystyle \frac{\lambda {\gamma }_e-b}{2a}}\ge P_{\max }$ in (22), ${\lambda }^{*}={\displaystyle \frac{2a{P}_{\max }+b}{{\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^2}}$. Therefore, when ${\mathit{P}}_{\mathrm{p}}^{*}{\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^2\le P_{\mathrm{sat}}$, the expression for ${\lambda }^{*}$ is

Similarly, when ${\mathit{P}}_{\mathrm{p}}^{*}{\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^2>P_{\mathrm{sat}}$, ${\lambda }^{*}$ can be obtained as

$$\begin{array}{c}\hfill {\lambda }^{*}=\left\{\begin{array}{c}\hfill {\displaystyle \frac{2a{P}_{\mathrm{sat}}+b{\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^2}{{\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^4}},P_{\mathrm{p}}^{*}={\displaystyle \frac{\lambda {\gamma }_e-b}{2a}},\\ \hfill {\displaystyle \frac{2a{P}_{\max }+b}{{\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^2}},P_{\mathrm{p}}^{*}=P_{\max }.\end{array}\right.\end{array}$$

(38)

4.2.6. Overall Algorithm for Problem (14)

The overall algorithm for Problem (14) is presented in Algorithm 2, where $R^{\left(p\right)}=f\left({\left\{{\mathit{\psi }}_k^{\left(p\right)}\right\}}_1^K\right)$ denotes the objective value of Problem (29) with variables ${\left\{{\mathit{\psi }}_k^{\left(p\right)}\right\}}_1^K$ in iteration p, and $U_{\mathrm{t}}^{\left(q\right)}=z\left({\left\{p_k^{\left(q\right)}\right\}}_1^K,{\lambda }^{\left(q\right)},{\left\{{\mathit{\psi }}_k^{\left(q\right)}\right\}}_1^K,{\left\{t_k^{\left(q\right)}\right\}}_0^K\right)$ denote the objective values of Problem (14) with variables ${\left\{p_k^{\left(q\right)}\right\}}_1^K$, ${\lambda }^{\left(q\right)}$, ${\left\{{\mathit{\psi }}_k^{\left(q\right)}\right\}}_1^K$, and ${\left\{t_k^{\left(q\right)}\right\}}_0^K$ in iteration q. In each iteration of Algorithm 2, the following inequalities hold

$$\begin{array}{cc}\hfill & z\left({\left\{p_k^{\left(q\right)}\right\}}_1^K,{\lambda }^{\left(q\right)},{\left\{{\mathit{\psi }}_k^{(q-1)}\right\}}_1^K,t_0^{\left(q\right)},{\left\{t_k^{\left(q\right)}\right\}}_1^K\right)\hfill \end{array}$$

$$\begin{array}{cc}\hfill \le & z\left({\left\{p_k^{\left(q\right)}\right\}}_1^K,{\lambda }^{\left(q\right)},{\left\{{\mathit{\psi }}_k^{\left(q\right)}\right\}}_1^K,t_0^{\left(q\right)},{\left\{t_k^{\left(q\right)}\right\}}_1^K\right)\hfill \end{array}$$

(39a)

$$\begin{array}{cc}\hfill \le & z\left({\left\{p_k^{\left(q\right)}\right\}}_1^K,{\lambda }^{\left(q\right)},{\left\{{\mathit{\psi }}_k^{\left(q\right)}\right\}}_1^K,t_0^{\left(q+1\right)},{\left\{t_k^{\left(q\right)}\right\}}_1^K\right)\hfill \end{array}$$

(39b)

$$\begin{array}{cc}\hfill \le & z\left({\left\{p_k^{\left(q\right)}\right\}}_1^K,{\lambda }^{\left(q\right)},{\left\{{\mathit{\psi }}_k^{\left(q\right)}\right\}}_1^K,t_0^{\left(q+1\right)},{\left\{t_k^{\left(q+1\right)}\right\}}_1^K\right)\hfill \end{array}$$

(39c)

$$\begin{array}{cc}\hfill \le & z\left({\left\{p_k^{\left(q+1\right)}\right\}}_1^K,{\lambda }^{\left(q\right)},{\left\{{\mathit{\psi }}_k^{\left(q\right)}\right\}}_1^K,t_0^{\left(q+1\right)},{\left\{t_k^{\left(q+1\right)}\right\}}_1^K\right)\hfill \end{array}$$

(39d)

$$\begin{array}{cc}\hfill \le & z\left({\left\{p_k^{\left(q+1\right)}\right\}}_1^K,{\lambda }^{\left(q+1\right)},{\left\{{\mathit{\psi }}_k^{\left(q\right)}\right\}}_1^K,t_0^{\left(q+1\right)},{\left\{t_k^{\left(q+1\right)}\right\}}_1^K\right).\hfill \end{array}$$

(39e)

where (39a) holds because ${\left\{{\mathit{\psi }}_k^{\left(q\right)}\right\}}_1^K$ is the locally optimal solution to Problem (29); (39b) holds because $t_0^{\left(q\right)}$ is the optimal solution to Problem (31); (39c) holds because ${\left\{t_k^{\left(q\right)}\right\}}_1^K$ is the optimal solution to Problem (33); (39d) holds because ${\left\{p_k^{\left(q\right)}\right\}}_1^K$ is the optimal solution to Problem (34); (39e) holds because ${\lambda }^{\left(q\right)}$ is the optimal solution to Problem (35). Due to the non-decreasing trend of the objective value over iterations shown in (39) and the existence of an upper bound on the objective value, Algorithm 2 is guaranteed to converge (Lemma 2, [23]). During each iteration, the gains of the leader and the follower are increased, and the optimal solution to the leader and follower problems is obtained when the Stackelberg equilibrium is reached, and if one party disrupts the equilibrium for self-interest, the gains of the other party are reduced, and at this point the equilibrium is broken, and the win-win situation is also destroyed.

In the inner layer, the main complexity of Algorithm 1 comes from Step 5, In Step 5, the complexity is $\mathcal{O}\left(L_{outer}{L}_{inner}{N}^{3.5}\right)$, where $L_{outer}$ and $L_{inner}$ denote the number of iterations required for reaching convergence in the inner and outer layers, respectively. The complexity of Algorithm 2 is $\mathcal{O}\left(N_{int}(\left(N^{2}K+2K+2\right){\left(3K+5\right)}^3+({\left(N^{2}K+2K+2\right)}^2{\left(3K+5\right)}^2\right)+$${\left(N^{2}K+2K+2\right)}^2)$, where $N_{int}$, $\left(N^{2}K+2K+2\right)$ and $\left(3K+5\right)$ denote the number of iterations required for reaching convergence, the number of variables, and the iteration number, respectively. The flowchart of Algorithm 2 is shown in Figure 3.

Algorithm 2 Algorithm for the Leader Problem (14).

1: Initialization: Set initial values for ${\left\{{\mathit{\psi }}_k^{\left(0\right)}\right\}}_1^K$, ${\left\{s_k^{\left(0\right)}\right\}}_1^K$, ${\left\{{\rho }_k^{\left(0\right)}\right\}}_1^K$, ${\lambda }^{\left(0\right)}$, ${\left\{p_k^{\left(0\right)}\right\}}_1^K$, and ${\left\{t_k^{\left(0\right)}\right\}}_0^K$. Set $q=0$, $U_{\mathrm{t}}^{\left(0\right)}$ = 0, ${\epsilon }_0=1\times {10}^{-3}$, and ${\chi }_0=1\times {10}^{-8}$. 2: repeat 3:        Set $p=0$ and $R^{\left(0\right)}=f\left({\left\{{\mathit{\psi }}_k^{\left(0\right)}\right\}}_1^K\right)$. 4:        repeat 5:           Set initial value for $\nu$. 6:           repeat 7:            Given ${\left\{s_k^{\left(p\right)}\right\}}_1^K$ and ${\left\{{\rho }_k^{\left(p\right)}\right\}}_1^K$, solve Problem (23) and denote the obtained solution as ${\left\{{\mathit{\psi }}_k^{(p+1)}\right\}}_1^K$. 8:            If there exists k such that ${\mathit{\psi }}_k^H{\mathit{e}}_i{\mathit{e}}_i^H{\mathit{\psi }}_k\left(1-{\mathit{\psi }}_k^H{\mathit{e}}_i{\mathit{e}}_i^H{\mathit{\psi }}_k\right)>{\chi }_0$, then set $\nu =2\nu$. 9:           until ${\mathit{\psi }}_k^H{\mathit{e}}_i{\mathit{e}}_i^H{\mathit{\psi }}_k\left(1-{\mathit{\psi }}_k^H{\mathit{e}}_i{\mathit{e}}_i^H{\mathit{\psi }}_k\right)\le {\chi }_0$, $\forall k$. 10:           Given ${\left\{{\rho }_k^{\left(p\right)}\right\}}_1^K$ and ${\left\{{\mathit{\psi }}_k^{(p+1)}\right\}}_1^K$, obtain ${\left\{s_k^{\left(p+1\right)}\right\}}_1^K$ according to (25). 11:           Given ${\left\{s_k^{\left(p+1\right)}\right\}}_1^K$ and ${\left\{{\mathit{\psi }}_k^{(p+1)}\right\}}_1^K$, obtain ${\left\{{\rho }_k^{\left(p+1\right)}\right\}}_1^K$ according to (28). 12:           Set $p=p+1$. 13:           Set $R^{\left(p\right)}=f\left({\left\{{\mathit{\psi }}_k^{\left(p\right)}\right\}}_1^K\right)$. 14:        until ${\displaystyle \frac{\left|R^{\left(p\right)}-R^{\left(p-1\right)}\right|}{R^{\left(p\right)}}}\le {\epsilon }_0$. 15:        Set ${\mathit{\psi }}_k^{\left(q+1\right)}$=${\mathit{\psi }}_k^{\left(p\right)},\forall k$. 16:        Given ${\left\{p_k^{\left(q\right)}\right\}}_1^K$, ${\lambda }^{\left(q\right)}$, ${\left\{{\mathit{\psi }}_k^{\left(q+1\right)}\right\}}_1^K$, and ${\left\{t_k^{\left(q\right)}\right\}}_1^K$, obtain $t_0^{\left(q+1\right)}$ according to (32). 17:        Given ${\left\{p_k^{\left(q\right)}\right\}}_1^K$, ${\lambda }^{\left(q\right)}$, ${\left\{{\mathit{\psi }}_k^{\left(q+1\right)}\right\}}_1^K$, and $t_0^{\left(q+1\right)}$, solve Problem (33) and denote the obtained solution as ${\left\{t_k^{\left(q+1\right)}\right\}}_1^K$. 18:        Given ${\left\{t_k^{\left(q+1\right)}\right\}}_0^K$, ${\lambda }^{\left(q\right)}$, and ${\left\{{\mathit{\psi }}_k^{\left(q+1\right)}\right\}}_1^K$, solve Problem (34) and denote the obtained solution as ${\left\{p_k^{\left(q+1\right)}\right\}}_1^K$. 19:        Given ${\left\{p_k^{\left(q+1\right)}\right\}}_1^K$, ${\left\{t_k^{\left(q+1\right)}\right\}}_0^K$, and ${\left\{{\mathit{\psi }}_k^{\left(q+1\right)}\right\}}_1^K$, if ${\mathit{P}}_{\mathrm{p}}{\left|{\mathit{h}}_{it}^H\mathbf{\Phi }{\mathit{h}}_r+h_{pt}^H\right|}^2\le P_{\mathrm{sat}}$, then obtain ${\lambda }^{\left(q+1\right)}$ according to (37); otherwise, obtain ${\lambda }^{\left(q+1\right)}$ according to (38). 20:        Set $q=q+1$. 21:        Set $U_{\mathrm{t}}^{\left(q\right)}=z\left({\left\{p_k^{\left(q\right)}\right\}}_1^K,{\lambda }^{\left(q\right)},{\left\{{\mathit{\psi }}_k^{\left(q\right)}\right\}}_1^K,{\left\{t_k^{\left(q\right)}\right\}}_0^K\right)$. 22: until ${\displaystyle \frac{\left|U_{\mathrm{t}}^{\left(q\right)}-U_{\mathrm{t}}^{(q-1)}\right|}{U_{\mathrm{t}}^{\left(t\right)}}}\le {\epsilon }_0$.

4.2.7. Stackelberg Equilibrium Analysis

After solving Problem (15) and Problem (14), the Stackelberg equilibrium between the PS and IT can be achieved [11]. The reason is given below. Let $\left(P_{\mathrm{p}}^{*},{\mathbf{\Phi }}^{*}\right)$ and $\left(P_{\mathrm{p}},\mathbf{\Phi }\right)$ denote the obtained locally optimal solution and a feasible solution to the follower problem (15), respectively. In addition, let $\left({\lambda }^{*},{\left\{t_k^{*}\right\}}_0^K,{\left\{{\mathbf{\Psi }}_k^{*}\right\}}_1^K,{\left\{p_k^{*}\right\}}_1^K\right)$ and $\left(\lambda ,{\left\{t_k\right\}}_0^K,{\left\{p_k\right\}}_1^K,{\left\{{\mathbf{\Psi }}_k\right\}}_1^K\right)$ denote the obtained locally optimal solution and a feasible solution to the leader problem (14), respectively. For the follower problem (15), the following inequality holds

$$\begin{array}{cc}\hfill & U_{\mathrm{p}}\left({\lambda }^{*},{\left\{t_k^{*}\right\}}_0^K,{\left\{p_k^{*}\right\}}_1^K,{\left\{{\mathbf{\Psi }}_k^{*}\right\}}_1^K,P_{\mathrm{p}}^{*},{\mathbf{\Phi }}^{*}\right)\ge U_{\mathrm{p}}\left({\lambda }^{*},{\left\{t_k^{*}\right\}}_0^K,{\left\{p_k^{*}\right\}}_1^K,{\left\{{\mathbf{\Psi }}_k^{*}\right\}}_1^K,P_{\mathrm{p}},\mathbf{\Phi }\right).\hfill \end{array}$$

(40)

For the leader problem (14), the following inequality holds

$$\begin{array}{cc}\hfill & U_{\mathrm{t}}\left({\lambda }^{*},{\left\{t_k^{*}\right\}}_0^K,{\left\{p_k^{*}\right\}}_1^K,{\left\{{\mathbf{\Psi }}_k^{*}\right\}}_1^K,P_{\mathrm{p}}^{*},{\mathbf{\Phi }}^{*}\right)\ge U_{\mathrm{t}}\left(\lambda ,{\left\{t_k\right\}}_0^K,{\left\{p_k\right\}}_1^K,{\left\{{\mathbf{\Psi }}_k\right\}}_1^K,P_{\mathrm{p}}^{*},{\mathbf{\Phi }}^{*}\right).\hfill \end{array}$$

(41)

According to the definition of the Stackelberg equilibrium in [25], the Stackelberg equilibrium is reached as long as the inequalities (40) and (41) are satisfied. Therefore, by solving Problems (15) and (14), the obtained solution $\left({\lambda }^{*},{\left\{t_k^{*}\right\}}_0^K,{\left\{p_k^{*}\right\}}_1^K,{\left\{{\mathbf{\Psi }}_k^{*}\right\}}_1^K,P_{\mathrm{p}}^{*},{\mathbf{\Phi }}^{*}\right)$ is the Stackelberg equilibrium point for the considered Stackelberg game.

5. Simulation Results

We evaluate the performance of the proposed Stackelberg game-based optimization algorithm through computer simulations. In the simulations, the proposed algorithm is compared to the following benchmark schemes.

(1) 

W/o IRS: This scheme does not use an IRS. It maximizes the utility of the PS by optimizing $P_{\mathrm{p}}$ using an algorithm similar to Algorithm 1, and it maximizes the utility of the IT by optimizing ${\left\{p_k\right\}}_1^K$, ${\left\{t_k\right\}}_0^K$, and $\lambda$ using an algorithm similar to Algorithm 2.

(2) 

Fixed time: This scheme fixes the time allocation ${\left\{t_k\right\}}_0^K$. It maximizes the utility of the PS by optimizing $P_{\mathrm{p}}$ and $\mathbf{\Phi }$ using an algorithm similar to Algorithm 1, and maximizes the utility of the IT by optimizing ${\left\{p_k\right\}}_1^K$, $t_0$, $\lambda$, and ${\left\{{\mathbf{\Psi }}_k\right\}}_1^K$ using an algorithm similar to Algorithm 2.

(3) 

Fixed time & w/o IRS: This scheme does not use an IRS and fixes the values of ${\left\{t_k\right\}}_0^K$. It maximizes the utility of the PS by optimizing $P_{\mathrm{p}}$ using an algorithm similar to Algorithm 1, and maximizes the utility of the IT by optimizing ${\left\{p_k\right\}}_1^K$, $t_0$, and $\lambda$ using an algorithm similar to Algorithm 2.

(4) 

Rate maximization: This scheme ignores the cost of using the IRS and uses all IRS reflection units. It maximizes the sum rate of all devices by jointly optimizing the IRS reflection coefficients in the WET phase $\mathbf{\Phi }$, the transmit power of the PS $P_{\mathrm{p}}$, the IRS reflection coefficients for all devices in the WIT phase ${\left\{{\mathbf{\Psi }}_k\right\}}_1^K$, the IT’s transmit power allocated to the devices ${\left\{p_k\right\}}_1^K$, and the time portions of the WET and WIT phases ${\left\{t_k\right\}}_0^K$ under the constraints (1), (2), (3), (7), (8), and (11). The considered optimization problem is

$$\begin{array}{cc}\hfill \underset{{\left\{t_k\right\}}_0^K,{\left\{p_k\right\}}_1^K,{\left\{{\mathbf{\Psi }}_k\right\}}_1^K,P_{\mathrm{p}},\mathbf{\Phi }}{max}& \sum _{k=1}^K{t}_k{log}_2\left(1+{\displaystyle \frac{p_k{\left|{\mathit{g}}_{i{d}_k}^H{\mathbf{\Psi }}_k{\mathit{g}}_r+h_k^H\right|}^2}{{\sigma }^2}}\right)\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& \left(1\right),\left(2\right),\left(3\right),\left(7\right),\left(8\right),\left(11\right),\hfill \end{array}$$

(42a)

$$\begin{array}{cc}\hfill & \left|w_{E,n}\right|=1,\forall n,\hfill \end{array}$$

(42b)

$$\begin{array}{cc}\hfill & \left|w_{k,n}\right|=1,\forall n,\forall k.\hfill \end{array}$$

(42c)

This scheme uses the alternating optimization algorithm to solve this problem, and the procedure is similar to Algorithm 2.

In a two-dimensional coordinate system, the IRS, the IT, and the PS are located at $\left(5,1\right)$ m, $\left(5,0\right)$ m, and $\left(0,0\right)$ m, respectively. The devices are randomly distributed in a circle centered at $\left(10,0\right)$ m with a radius of 0.5 m. The large-scale fading path loss model for all channels is given by $L\left(d\right)=C_0{\left(d/d_0\right)}^{-\alpha }$, where $C_0$, d, and $\alpha$ denote the path loss at the reference distance $d_0$ = 1 m, the propagation distance, and the path loss exponent, respectively. The path loss exponents of the channels from the PS to the IT, from the IT to the IRS, from the IRS to the devices, and from the IRS to the IT are set to 2.5, and the path loss exponents of the channels from the PS to the IT and from the IT to the devices are set to 3.5. The small-scale fading of all channels is modeled by the Rayleigh fading model. Unless otherwise specified, the other simulation parameters are set as follows: $N=30$, $\eta =0.8$, $\beta =10$, $r=0.01$, $a=1$, $b=1$, ${\sigma }^2=-80$ dBm, $C_0=-30$ dB, and $R_{\min }=0.01$ bits/Hz.

5.1. Impact of the Number of Devices

Figure 4 shows the utilities of the IT achieved by different schemes versus the number of devices K. It can be observed that the utilities of the IT achieved by all schemes increase as the number of devices increases except for the “rate maximization” scheme. This is because the more devices there are, the more revenue the IT earns. In the “rate maximization” scheme, the utilities are basically unchanged, because the IT will maximize the transmission power of the PS, thus achieving the maximum achievable rate, which this scheme aims to maximize when there is only one device. The proposed algorithm has a lower utility than the “rate maximization” scheme, but achieves a higher utility than the other benchmark schemes. This is due to the fact that IRS enhances the strength of the signal in the WET and WIT phases and uses TDMA in the WIT phase to co-ordinate the timing of the reception of the signal by each device. The system has a higher degree of flexibility and is therefore able to realize higher utilities than other schemes. It outperforms the “w/o IRS” scheme due to the signal enhancement of the IRS in the WET and WIT phases. The “fixed time” scheme has an even lower utility than the “w/o IRS” scheme, demonstrating the effectiveness of time allocation in improving the utility of IT.

Figure 5 shows the utilities of the PS achieved by different schemes versus the number of devices K. It is observed that the utilities of the PS achieved by all schemes increase as K increases. The proposed algorithm has the highest utility. The proposed algorithm achieves higher utility than the “w/o IRS” scheme, demonstrating that the use of an IRS can achieve a higher utility. In addition, the “rate maximization” scheme achieves a negative utility, because it only considers maximizing the sum rate of the devices and ignores the utility of the PS. Additionally, the “rate maximization” scheme activates all IRS reflection units in the WET phase, resulting in a high cost. The results of Figure 4 and Figure 5 demonstrate that the proposed algorithm can achieve a win-win situation between the PS and IT.

Figure 6 shows the sum rates of all devices achieved by different schemes versus the number of devices K. It is observed that the sum rates achieved by all schemes increase as K increases. The “rate maximization” scheme has the highest sum rate because it only considers maximizing the sum rate, ignores the utility of the IT, and treats the IRS as a free service by activating all reflection units of the IRS. In addition, although the proposed algorithm does not aim to maximize the sum rate, it achieves a significantly higher sum rate than the “w/o IRS”, “fixed time”, and “fixed time & w/o IRS” schemes, demonstrating that the use of an IRS can not only improve the utilities of the IT and PS, but also increase the sum rate of the devices. Furthermore, the difference in sum rate between the proposed algorithm and the “fixed time” scheme shows that the time allocation in the WET and WIT phases has a decisive influence on the performance of the system. Therefore, in practice, we need to consider the time for each device to receive the signal and deploy the IRS to improve the signal strength and thus increase the utility of the system.

5.2. Impact of the Maximum Transmit Power of the PS

Figure 7 shows the utilities of the IT achieved by different schemes versus the maximum transmit power of the PS $P_{\max }$ for $K=5$. It can be observed that the proposed algorithm achieves the highest utility. This is due to the fact that, for the same parameters, the proposed scheme can increase the benefit of the system due to its superiority in terms of performance. Therefore, in order to increase the utility of the system, parameters can be set slightly larger in practice. Furthermore, it can be observed that the utility of the IT of the proposed algorithm first increases and then remains unchanged as $P_{\max }$ increases. This is because, when $P_{\max }$ is not large, as $P_{\max }$ increases, the energy harvested by the IT increases, resulting in an increase in the revenue earned by the IT. On the other hand, when $P_{\max }$ is sufficiently large, the Stackelberg equilibrium is reached at a certain value of $P_{\max }$ (about 33 dBm), and the utility of the IT is maximized at that point. The utilities of the “w/o IRS” and “fixed time & w/o IRS” schemes do not increase much with $P_{\mathrm{p}}$ due to the absence of the IRS.

Figure 8 shows the utilities of the PS achieved by different schemes versus the maximum transmit power of the PS $P_{max}$ for $K=5$. It can be observed that, as $P_{\max }$ increases, the utilities of the PS achieved by all schemes increase first and then tend to remain unchanged. This is because, when $P_{\max }$ is not large, the increase in $P_{\max }$ increases the energy sold by the PS, thus increasing its utility. On the other hand, when $P_{\max }$ is sufficiently large, the Stackelberg equilibrium is reached at about $P_{\max }=33$ dBm, and all systems have no incentive to change their resource allocations after reaching the equilibrium, which could reduce utility.

5.3. Impact of the Price of the Activated IRS Reflection Unit

The numbers of activated IRS reflection units in different schemes versus the price of the activated IRS reflection units r for $K=5$ is shown in Figure 9, where “proposed-WIT” and “proposed-WET” stand for the proposed algorithm in the WIT and WET phases, respectively, and “fixed time-WIT” and “fixed time-WET” stand for the “fixed time” scheme in the WIT and WET phases, respectively. In Figure 9, it can be observed that the number of activated IRS reflection units decreases in all schemes as the price r increases. This is because, as the price of activated IRS reflection units increases, the IT and PS use fewer IRS reflection units to reduce costs and increase their utilities. For the PS, as r increases, the number of activated IRS reflection units used in the WET phase decreases more slowly than that in the WIT phase to allow the IT to harvest more energy. For the “fixed time” scheme, there is no time allocation for the WIT phase, resulting in lower revenues for the IT and PS, as compared to the proposed algorithm. Therefore, to increase the utility, the system with the “fixed time” scheme needs to reduce costs and thus uses a lower number of activated IRS reflection units in the WET and WIT phases, as compared to that with the proposed algorithm.

6. Conclusions

We have studied a multi-device IRS-assisted WPCN with multiple devices, where the PS, the IT, and the IRS belong to different service providers. We set up a Stackelberg game framework to model the resource allocation strategy interactions among different service providers (the PS and the IT). For the PS, we formulate a follower problem, where its utility is maximized by jointly optimizing the IRS’s reflect beamforming and its transmit power. For the IT, we formulate a leader problem, where its utility is maximized by jointly optimizing the energy price, the time allocation, the IRS’s reflect beamforming, and its transmit power allocation. To solve these problems, we have proposed efficient algorithms based on the alternating optimization technique. The SCA technique has been used to solve the follower problem, and the closed-form FP approach and the SCA technique have been used to solve the leader problem. Simulation results show that the proposed algorithm can effectively improve the PS and IT’s utilities, as compared to some baseline algorithms, and can achieve the Stackelberg equilibrium between different service providers, thus achieving a win-win situation for the PS and IT. Simulation results also show that the IRS can significantly improve all service providers’ utilities by enhancing the signal quality, even when the cost of the IRS is taken into account.