Budgeted Thompson Sampling for IRS Enabled WiGig Relaying

Abstract

Intelligent reconfigurable surface (IRS) is a competitive relaying technology to widen the WiGig coverage range, as it offers an effective means of addressing blocking issues. However, selecting the optimal IRS relay for maximum attainable data rate is a time-consuming process, as it requires WiGig beamforming training (BT) to tune the phase shifts (PSs) for WiGig base station (WGBS) and IRS relays. This paper proposes a self-learning-based budgeted Thomson sampling approach for IRS relay probing (BTS-IRS) to address this challenge. The BT time cost of probing the IRS relay is incorporated into the main BTS formula, where both payoff and cost posterior distributions are sampled separately, their ratio is estimated, and the arm/IRS relay with the highest ratio is decided. This enables the IRS relay to be chosen with the lowest BT time cost. Numerical results demonstrate the improved performance of the BTS-IRS relaying technique regarding BT time consumption/cost, spectral efficiency, and attainable data rate when compared to other benchmarks.

1. Introduction

The high-frequency range of 30–300 GHz of WiGig communications is being touted as an essential element of the upcoming beyond fifth-generation (B5G) and six-generation (6G) wireless networks as it provides an extensive spectrum for transmission [1]. Nevertheless, its high operating frequency nature has an effect on severe channel characteristics with sensitive blocking, as individual shadowing and raindrops can certainly affect WiGig communication linkage [1]. Furthermore, this limits the coverage of WiGig owing to its short transmission scale [2]. Thus, diverse beamforming training (BT) approaches were commonly utilized in conjunction with WiGig transmissions to expand its communication range [2]. Nevertheless, this BT methodology yields a considerable time overhead, which hinders the attainable data rate of the WiGig linkage.

Currently, intelligent reconfigurable surface (IRS) is recommended as a critical enabling technology for future B5G/6G wireless networks [3]. The IRS panels comprise passive antenna elements that are manipulated to facilitate the orientation of incoming electromagnetic waves (EMs) towards a designated receiver by calibrating their phase shifts (PSs) [4]. Hence, it significantly boosts and extends the wireless transmission/reception range [3,4].

In contrast to traditional relaying methods, such as amplify and forward (AF) or decode and forward (DF), IRS is particularly well-suited for installation at low-cost, given the absence of the costly radio frequency (RF) chains required by traditional relaying [5]. Therefore, the literature studied and recommended IRS applications in various wireless scenarios [3,4].

Additionally, there is a mutually beneficial relationship in IRS-enabled WiGig communications. The IRS can act as a relay for the WiGig signal, extending its range and circumventing obstacles. On the other hand, WiGig base stations (WGBS) can use IRS to direct beams to specific WiGig receivers or destinations. This concept can also be applied to terahertz (THz) communications, which have fragile channels and high levels of path blockage in forthcoming 6G networks [6]. Furthermore, IRS-enabled WiGig relaying effectively diminishes the increased complexity of the WiGig traditional relaying by getting rid of the enormous RF chains. Hence, some related work studied IRS-enabled WiGig communications, particularly in elevated blockage scenarios [7,8,9]. The literature of [10,11,12] has extensively explored various challenges such as the estimation of the cascaded IRS-enabled WiGig channel and optimizing antenna phase shifts of both the WGBS and IRS relay [13,14] (Herein, the IRs panels act as relays and we are trying to find the best nearby panel. A similar term is used in [13,14]). The use of IRS relay probing can provide a range of real-world applications, including extending the coverage range of wireless communication networks in rural and remote areas, enabling emergency response and disaster relief, improving wireless capacity, helping to develop and deploy B5G/6G systems, and providing a reliable and consistent connection for the Internet of Things (IoT).

Our paper is motivated by the investigation of WiGig-aided IRS relaying. In recent years, WiGig and IRSs have been widely heralded as groundbreaking technologies that could enhance the performance of wireless communication systems, particularly in terms of increased spectral efficiency and reduced energy consumption. Nonetheless, the performance of IRSs is heavily contingent upon the selection of precoding techniques and the training time required to determine the optimal precoder weights. This is due to the ability of IRS to manipulate the signal via tuning the reflectivity coefficients of its elements to be adaptable against channel conditions. Therefore, our objective is to identify the optimal surrounding IRS panel that can be utilized to relay information from WGBS to the mobile user when LoS path is blocked. However, this research focuses on finding the supreme IRS relay amongst the scattered relays (IRS boards), where this targeted relay/board maximizes the attainable throughput at the WiGig client. This is anticipated as a complex practical issue in spreading IRS-empowered WiGig relaying. The hardship of this issue comes from the significant BT computational overhead needed to simultaneously tune the phase shifts of the WGBS and IRS relay, as it is computationally intensive (First, the phase shifts from WGBS to the probed IRS panel have to be tuned. Then, another tuning process at IRS is in order to direct the signal to the client). Hence, brute forcing all of the equipped IRS boards employing relay searching techniques to locate the supreme one looks like an inoperable solution due to the severe time consumption. Hence, in this paper, the IRS relay probing issue is handled and formulated as a budgeted multi-armed bandit (MAB) methodology [15].

The key findings and achievements outlined in this paper are listed as follows:

• The WiGig IRS relay probing problem is mathematically formulated as an optimization problem in order to choose the supreme IRS relay that maximizes the attainable data rates of WiGig communication linkages. Still, the selection process is constrained by the time consumption of the BT procedure of the chosen IRS panel;
• WiGig-IRS relay probing issue is reformulated as online budgeted MAB, where budgeted Thompson sampling (BTS-IRS) with guaranteed time-cost performance is envisioned to handle it. Herein, the WGBS, IRS boards/relays, throughput, and BT are the bandit player, arms, reward, and cost, respectively;
• The proposed BTS-IRS algorithm demonstrates superior performance compared to other benchmark schemes through numerous numerical simulations in various scenarios as the BT processing time is amended to the classical TS formula as a budget/cost;
• Numerous numerical evaluations were carried out to illustrate the superiority of the envisioned BTS scheme compared to other benchmarks such as the brute force solution, the classical TS algorithm [16,17], and random selection of IRS relays. Moreover, BTS-IRS is compared to previous cost-effective UCB-based MAB schemes. The BTS-IRS scheme exhibited an average spectral efficiency that was superior to other benchmark schemes, with comparable BT time computation.

The rest of this paper is organized as follows. Section 2 details existing research related to the topic. Section 3 presents the WiGig-IRS system model, which includes the IRS-assisted WiGig channel model and the optimization problem for IRS relay probing. Section 4 presents the proposed BTS-IRS scheme. Numerical simulations are discussed in Section 5. Finally, Section 6 offers the conclusions and findings.

2. Related Works

Due to the advanced technology of EM, IRS-aided telecommunication is envisioned to burn line-of-sight difficulties of future wireless communications [3,4]. IRS panels can assist satellite communications [18,19,20], IoT networks [21], UAV communications [7], etc. Besides its energy saving, it overcomes the half-wavelength issue of antenna elements. Moreover, IRSs reuse EM waves resulting in satisfactory power expenditure. Furthermore, its innovative radio propagation capabilities enable ubiquitous secured wireless transmissions [22]. Two main energetic applications of IRS in wireless systems are IRS modulators and IRS relays. The authors of [23] investigated IRS-enhanced modulations. This can be achieved via adjusting the IRS reflection coefficients to efficiently forward the transmitted signal from the antenna feeder via modulation. In IRS relay implementations, the transmitted power from the WGBS to the IRS Panel is redirected to the client via continuous readjustment of the PSs [24]. Few related studies investigated the impact of IRS on WiGig wireless networks. In [25], coauthors of this work proposed double-stage MAB methodologies for WiGig-IRS wireless systems to select the best route that attains the maximum spectral efficiency (SE) via the IRS panel. Still, no relay probing consideration was performed in this work. Moreover, they also leveraged MAB schemes in [7] to determine the best route via IRS in envisioned UAV-mounted IRS system that enhances the WiGig communications in hotspot zones. Furthermore, the authors of [8] derived a tractable model for coverage of IRS-aided WiGig networks based on stochastic geometry. A federated learning (FL) aided model of WiGig-enabled IRS systems was envisioned in [9]. The authors of [10] explored an effective two-stage channel estimation formulation for hybrid IRS-assisted WiGig MIMO systems, while [11] derived it via atomic norm minimization. In [12], a proposal was advanced to optimize the average spectral efficiency in an IRS-aided WiGig MIMO architecture, which incorporated the use of hybrid precoders for the WGBS and passive precoders for the IRS, taking advantage of time-varying channel state information (CSI). Furthermore, the work of [24] studied a multi-user hybrid precoding (HP) scheme for IRS-enhanced WiGig systems. The work of [26] envisioned a deep learning (DL) scheme that attains the optimal communication rate for IRS systems. In [27], the authors derived a machine learning (ML) enabled solution for efficient beam management in IRS-aided WiGig systems. Hybrid IRS relay structures were established in [28] with better performance than traditional IRS models. Precise SINR formulations for IRS-enabled WiGig AF relay were formulated in [29]. They attained the optimal PSs with maximal end-to-end SINR via allocating the optimal power of AF relay. The work of [30] investigated the utilization of sequential fractional programming (SFP) for multi-objective optimization of passive beamforming at WiGig-assisted IRS, user association, and power allocation. Furthermore, different IRS-aided WiGig BT structures were explored in [31] to determine the optimal transmission coefficients. IRS panels were utilized to assist cell-free MIMO communications in [32,33]. The research in [34] examined the issue of selecting the IRS in vehicular systems, where the vehicle chooses the IRS panel that results in the highest received signal. In spite of their work, [32,33,34] did not deal with WiGig IRS relaying in addition to neglecting the study of BT time minimization problem. In [35], coauthors of this work contributed to the IRS millimeter wave relaying using cost-effective MABs using modified UCB schemes, i.e., MAB-CE1 and MAB-CE2. Since TS is better than UCB, this paper leverages BTS for WiGig-IRS relay probing with guaranteed performance.

3. System Model and Problem Statement

The system model depicted in Figure 1 involves utilizing a group of IRS relays, numbered from 1 to R, within the coverage area of a WiGig system that is enabled by IRS. The IRS relays are strategically placed to boost the WiGig linkage. The figure demonstrates the total blockage of the direct link, i.e., line of sight (LoS) between the WGBS and the mobile user (MU)/client. Therefore, the WGBS should choose one of the surrounding IRS panels to relay its data to the ME via the NLoS path that avoids the blocker. The WGBS is comprised of M antenna elements which are arranged linearly in a uniform fashion. In addition, each IRS panel has an array of N antenna elements with a uniform planar layout. The IRS controller tunes the phase shifts of the IRS antenna arrays based on the phase shift vector chosen via WGBS.

To manage information between the WGBS and IRS relay, an assigned communication channel between them is operated. Following IRS-assisted WiGig channel models provided in [25,36], the delivered signal at the MU can be mathematically described as:

$$y_r={\mathit{h}}_{rU}^H{\mathbf{\Theta }}_{r_i}{\mathit{H}}_{Br}{\mathit{f}}_{j}x+n,$$

(1)

where $1\le r\le R,1\le i\le \left|{\mathsf{\Omega }}_r\right|$ and $1\le j\le \left|\mathcal{F}\right|$.

In addition, $y_r$ and x are the received and sent symbols tracked via IRS relay r, correspondingly. In addition, the transmission power equals $P=\mathit{E}\left[x{x}^H\right]$, where ${\left(\dot{}\right)}^H$ is the Hermitian transpose, and $n\sim \mathcal{CN}\left(0,{\sigma }_0^2\right)$ identifies noise. ${\mathit{f}}_j\in {\mathit{C}}^{M\times 1}$ is the analog precoder vector (Further investigations of digital or hybrid precoder types will be considered as future work) sized by $M\times 1$ of WGBS, while ${\mathsf{\Theta }}_{r_i}\in {\mathit{C}}^{N_r\times N_r}$ is the $N_r\times N_r$ diagonal matrix whose main diagonal is the IRS board antenna elements’PS vector r. Analog precoders have reduced complexity compared to digital or hybrid precoders. It can be implemented with low-complexity circuits, which makes them suitable for low-cost and low-power IRS systems. In addition, its ability to implement continuous phase shifts, which can improve the performance of the IRS system. In contrast, digital precoders are limited to discrete phase shifts, which can result in reduced performance. i and j define the indexes of the utilized ${\mathsf{\Theta }}_r$ and $\mathit{f}$, where ${\mathsf{\Omega }}_r$ and $\mathcal{F}$ identify the sets of accessible PSs, vectors of IRS relay r, and WGBS, accordingly. ${\mathit{H}}_{Br}\in {\mathit{C}}^{N_r\times M}$ is the $N_r\times M$ channel matrix between WGBS and IRS relay r. ${\mathit{h}}_{rU}\in {\mathit{C}}^{N_r\times 1}$ is the $N_r\times 1$ channel vector between the IRS relay r and MU. ${\mathit{H}}_{Br}$ and ${\mathit{h}}_{rU}$ can be expressed as follows [25,36]:

$${\mathit{H}}_{Br}=\sqrt{\frac{N_{r}M}{K_{Br}}}\sum _{k=1}^{K_{Br}}{\eta }_k{\mathit{a}}_r({\pi }_k^{\left(AoA\right)},O_k^{\left(AoA\right)}){\mathit{g}}_B{\left({\pi }_k^{\left(AoD\right)}\right)}^H$$

(2)

$${\mathit{h}}_{rU}=\sqrt{\frac{N_r}{K_{rU}}}\sum _{k=1}^{K_{rU}}{\nu }_l{\mathit{a}}_r\left({\theta }_k^{\left(AoD\right)},{\varphi }_k^{\left(AoD\right)}\right)$$

(3)

where $K_{Br}$ is the linkage paths between WGBS and IRS relay r, and $K_{rU}$ is the paths between IRS relay r and MU. ${\eta }_k$ and ${\nu }_k$ are the coefficients of large-scale fading [36] that are distributed as complex Gaussian i.e., $\mathcal{CN}\left(0,{10}^{\frac{{PL}_k}{10}}\right)$, where ${PL}_k\left(dB\right)=-PL\left(d_0\right)-10\alpha {log}_{10}\left(d\right)-{\rho }_l$. $PL\left(d_0\right)$ is the path loss at a reference distance $d_0=1$ m [36], $\alpha$ identifies path loss exponent, d is the distance, and ${\rho }_k\sim \mathcal{CN}\left(0,{\sigma }_{{\rho }_l}^2\right)$ is the shadowing term of path l in $K_{Br}$ and $K_{rU}$. ${\mathit{a}}_r\left({\theta }_k^{\left(AoA\right)},{\varphi }_k^{\left(AoA\right)}\right)$, and ${\mathit{g}}_B\left({\pi }_k^{\left(AoD\right)}\right)$ are the response vectors of the l-th path array at the IRS relay r and BS, where the angle of arrival (AoA) is ${\pi }_k^{\left(AoA\right)}{o}_k^{AoA}$ and ${\pi }_k^{\left(AoD\right)}$ is the angle of departure (AoD) [25,36]. ${\mathit{a}}_r\left({\theta }_k^{\left(AoD\right)},{\varphi }_k^{\left(AoD\right)}\right)$ is the response vector of the $k^{th}$ path at the IRS panel r, where ${\theta }_k^{\left(AoD\right)}$ is the AOD azimuth angle and ${\varphi }_k^{\left(AoD\right)}$ is the AOD elevation angle. Mainly, ${\mathit{a}}_r\left(\theta ,\varphi \right)$ is mathematically stated as follows [25]:

$${\mathit{a}}_r\left(\theta ,\varphi \right)=\frac{1}{\sqrt{N_r}}{\left[1,\dots ,e^{j\frac{2\pi }{\lambda }s\left(ysin\left(\theta \right)+zcos\left(\varphi \right)\right)},\dots \right]}^T,$$

(4)

where s, $\lambda$ refer to antenna spacing and carrier wavelength, accordingly, and $0\le y,z\le \left(\sqrt{N_r}-1\right)$. Similarly, ${\mathit{g}}_B\left({\pi }_k^{\left(AoD\right)}\right)$ is given as [25]:

$${\mathit{g}}_B\left(\theta \right)=\frac{1}{\sqrt{M}}{\left[1,\dots ,e^{j\frac{2\pi }{\lambda }snsin\left(\theta \right)},\dots \right]}^T,$$

(5)

where $0\le n\le \left(M-1\right)$.

The goal of the IRS-assisted WiGig relay selection problem is to select the optimal IRS relay, $r^{*}$, to maximize the attainable throughput, $\mathsf{\Psi }$, measured in bits per second. This can be mathematically represented as

$$r^{*}=\underset{r}{max}\left({\mathsf{\Psi }}_r\right)=\underset{r}{max}\left(BW{\psi }_r\left(\frac{T_D}{T_D+{\sum }_{r=1}^{M_Q}{T}_{{BT}_r}}\right)\right)$$

(6a)

$${\psi }_r={log}_2\left(1+\frac{P\left({\mathit{h}}_{rU}^H{\mathsf{\Phi }}_{r_i}{\mathit{H}}_{Br}{\mathit{f}}_j\right){\left({\mathit{h}}_{rU}^H{\mathsf{\Phi }}_{r_i}{\mathit{H}}_{Br}{\mathit{f}}_j\right)}^H}{{\sigma }_0^2}\right)$$

(6b)

s.t

$$i\in {\mathsf{\Omega }}_r,$$

(6c)

$$M_Q\in R,$$

(6d)

$$j\in \mathcal{F}.$$

(6e)

where $BW,T_D$ are the utilized bandwidth and data transmission time, respectively. ${\sum }_{r=1}^{M_Q}{T}_{{BT}_r}$ is the overall beamforming time needed to probe $M_Q\in R$ IRS relays prior to deciding the supreme one with the highest data rate. $T_{{BT}_r}$ is the beamform training time of relay r, mathematically expressed as:

$$T_{{BT}_r}=\left|{\mathsf{\Omega }}_r\right|\left|\mathcal{F}\right|{\Delta }_{BT},$$

(7)

where ${\Delta }_{BT}$ is the BT time of single pair of ${\mathsf{\Phi }}_{r_i}$ and ${\mathit{f}}_j$.

Brute-force searching all of the accessible IRS relays, i.e., $M_Q=R$, then deciding the supreme one, i.e., $r^{*}$, with maximum $\psi$, causes great degradation of the attainable data rate $\mathsf{\Psi }$ due to the huge overall BT time cost with all spread R IRS relays. The supreme data rate performance issues from only probing one relay with the largest spectral efficiency. Hence, in this paper, a self-learning-assisted solution via formulating the issue under investigation as a BMAB bounded by the chosen IRS relay’s BT cost will be envisioned, and every time slot t, a single IRS relay is chosen.

4. Envisioned BTS-IRS Algorithm

Herein, we will briefly discuss the vision behind MABs, including BMABs, and then re-formulate (6) as a BMAB game. Then, the BTS-IRS scheme with efficient time cost will be highlighted to deal with the discussed optimization problem.

4.1. MAB Concept

MAB is a well-known issue in reinforcement learning in which a decision-maker must choose which arm of a k-armed bandit to pull at each round in order to maximize their cumulative reward over time. The decision-maker does not know the true expected reward for each arm and must learn this through trial and error. MAB is a promising online/self-learning methodology with a broad range of feasible applications [15]. In the typical setting of MAB, a player intends to maximize his longstanding gain via attempting k arms $\mathit{k}\in \{1,2,\dots ,\mathcal{K}\}$ of the slot machine. Hence, the player subsequently draws one arm per time t, i.e., ${\mathit{k}}_t$, and collects its attainable payoff, ${\mathcal{R}}_{{\mathbf{k}}_t}$. Initially, the player explores all of the arms once at least to notice their payoffs. Hence, the player decides his next chosen arm upon those noticed payoffs. Typically, the player handles the exploitation-exploration compromise, i.e., exploiting the largest reward arm discovered till now or exploring new ones [15,17,37,38]. BMABs are similar to MABs, except that playing any arm is associated with paying a cost to reveal the reward. The way of handling the cost mathematically reveals the BMAB type. As some types embed the cost term in both exploration–exploitation terms of the BMAB algorithms, and others only subtract the cost term in the MAB exploration term. Hence, in BMABs, the player attempts to maximize his payoff and minimize his paid cost. Thus, BTS is envisioned to handle IRS relay probing scenario [35,39]. As will be shown later, the BTS solution is a sub-optimal one, where it intends to minimize its regret defined as:

$$\mathbb{R}\left(T_{Hr}\right)=\mathbb{E}\left[\sum _{t=1}^{T_{Hr}}{\mu }_{{\mathbb{R}}_{(i,t)}^{*}}-{\mu }_{{\mathbb{R}}_{(i,t)}}\right],$$

(8)

At time horizon $T_{Hr}$, the optimal arm’s mean rewards are denoted by ${\mu }_{{\mathbb{R}}_{(i,t)}^{*}}$, and the mean reward from the selected arm by the BTS scheme is denoted by ${\mu }_{{\mathbb{R}}_{(i,t)}}$. The expectation operator $\mathbb{E}$ is derived from the random selection of ${\mathbb{R}}_{(i,t)}$ from the BTS scheme. BTS endeavors to mitigate instant regret over t, yet the cumulative regret is still increased over $T_{Hr}$. This leads to suboptimal performance of the BTS algorithm over t, while it is close to optimal over the extended training period of $T_{Hr}$.

The IRS-assisted WiGig relaying described in (6) can be re-formulated as BMAB by taking into account the computational time cost of the chosen relay. The player in this scenario is the WGBS, whose objective is to obtain the maximal long-term reward, i.e., the attainable $\psi$, within a certain time horizon $T_{Hr}$, via choosing among the accessible IRS relays (arms) R with simultaneously reducing the computational beamforming time cost of the chosen IRS relay at each t. Hence, (6) is re-expressed as a sequential time optimization formula, expressed as:

$${\max }_{{\mathbb{L}}_{\left(1\right)},\dots ,{\mathbb{L}}_{\left(T_{Hr}\right)}}\frac{BW}{T_{Hr}}\sum _t\sum _r{{\mathbb{L}}_{r\left(t\right)}\psi }_{r\left(t\right)}\left(\frac{T_d}{T_d+T_{{BT}_{r\left(t\right)}}}\right)$$

(9a)

$${\psi }_{r\left(t\right)}={log}_2\left(1+\frac{P\left({\mathit{h}}_{r\left(t\right)U}^H{\mathsf{\Phi }}_{r{\left(t\right)}_i}{\mathit{H}}_{Br\left(t\right)}{\mathit{f}}_j\right){\left({\mathit{h}}_{r\left(t\right)U}^H{\mathsf{\Phi }}_{r_i\left(t\right)}{\mathit{H}}_{Br\left(t\right)}{\mathit{f}}_j\right)}^H}{{\sigma }_0^2}\right)$$

(9b)

S.t

$$T_{H_r}\in (0,Z^{+}),$$

(9c)

$$i\in {\mathsf{\Omega }}_r,$$

(9d)

$$j\in \mathcal{F},$$

(9e)

$$T_{{BT}_{r\left(t\right)}}\le \underset{r}{max}\left(T_{{BT}_r}\right),$$

(9f)

$$\sum _r{\mathbb{L}}_{r\left(t\right)}=1,1\le r\le R.$$

(9g)

where $Z^{+}$ is the set of positive integers. ${\mathbb{L}}_{r\left(t\right)}$ is a linkage indicator, which indicates whether IRS relay r is chosen to build a WiGig relay connection at time t, with a value of 1 indicating a selection and 0 indicating otherwise. The restriction (9f) states that the resulting block transmission time cost from choosing IRS relay r is limited by maximum $T_{BT}$ among all of the available IRS panels R. The restriction (9g) assures that a single IRS relay is pulled at time t. Equation (9) involves sequentially selecting the IRS relay over the bandit time $T_{Hr}$, where the chosen one must have the highest spectral efficiency while also satisfying the block transmission time cost bound. The transformation of (6a) into (9a) helps to decrease the beamform training time cost for the problem of selecting IRS relays by only choosing one IRS board at a time, which simulates the optimal process leading to observable increment in the achievable data rate.

4.2. Envisioned BTS Algorithm

BTS is a variant of the classic TS algorithm that is used for solving BMAB problem types [38]. It is a mathematically rigorous method for balancing the exploration and exploitation of different arms (IRS relays in our case) in order to maximize the expected reward (spectral efficiency) over a given budget (BT time). It can be mathematically modeled as a Bayesian optimization problem with the following components: Prior distributions: Each arm is associated with a prior distribution over its reward distribution. These distributions are updated as more observations are made. Posterior distributions: The posterior distribution over the reward of each arm is obtained by combining the prior with the observed rewards. Sampling: A sample is drawn from the posterior distribution of each arm, and the arm with the highest sample is chosen for the next round. Budget constraint: A budget constraint is imposed, which limits the total number of times that an arm can be pulled over a fixed period. BTS is mathematically sound and can be proven to have theoretical guarantees for performance. Hence, in our case i.e., IRS-relay probing, the prior distribution is the Gaussian distribution impacted by the additive white Gaussian noise, and the budget is the maximum allowable BT time. BTS-IRS optimal strategy is by the continuous selection of the optimal arm, which has the maximum ratio of mean payoff to mean cost. Hence, BTS selects that optimal arm as much as possible during the time horizon.

Algorithm 1 details the envisioned BTS-IRS. Its inputs are $T_{{BT}_r}$, ${\mathsf{\Omega }}_r\forall r\in R$, and $\mathcal{F}$. The output is the chosen IRS relay at trial t, $r^{*}\left(t\right)$. As an initialization step, $x_r\left(t\right)=0$, which is the number of draws of IRS panel r at time t. In addition, ${\overline{\psi }}_r\left(0\right)=0$ and ${\overline{T}}_{B{T}_{r\left(0\right)}}=0$, where the former is the spectral efficiency/reward of IRS panel r and the latter is the time cost of IRS panel r. The algorithm starts for the whole time horizon to draw the supreme IRS selected using the BTS policy. This is carried out by sampling spectral efficiencies ${\theta }_i^r\left(t\right)$ and BT time cost ${\theta }_i^c\left(t\right)$ from a normal distribution, and then drawing the IRS relay with the largest spectral efficiency to BT cost ratio i.e., $r^{*}\left(t\right)={arg\; max}_{i\in R}\frac{{\theta }_i^r\left(t\right)}{{\theta }_c^r\left(t\right)}$. Afterwards, we receive the spectral efficiency ${\psi }_{r^{*}\left(t\right)}$ and BT cost $T_{BT{r}^{*}\left(i\right)}$ from this specific IRS relay and update the distribution of both spectral efficiency and BT cost as follows:

$${\overline{\psi }}_{r^{*}\left(t\right)}=\frac{1}{x_{r^{*}\left(t\right)}}\sum _{i=1}^t{\mathbb{L}}_{r^{*}\left(i\right)}{\psi }_{r^{*}\left(i\right)}$$

(10a)

$${\overline{T}}_{BT{r}^{*}\left(t\right)}=\frac{1}{x_{r^{*}\left(t\right)}}\sum _{i=1}^t{\mathbb{L}}_{r^{*}\left(i\right)}{T}_{BT{r}^{*}\left(i\right)}$$

(10b)

Algorithm 1: Proposed BTS-IRS Algorithm

5. Numerical Analysis

Herein, numerical simulations are implemented to validate the efficiency of the envisioned BTS-IRS scheme by comparing it with benchmark techniques, including brute force search, random selection, and naïve TS algorithms. As prescribed in Figure 1, our experimental setup considers a certain coverage region, where the WGBS is located on one edge of the region and the UE is positioned on the other end. The direct link between both is blocked. A random number of IRS boards are spread inside the communication area. The target is to find out the best probed IRS panel/relay. The entire set of IRS relays is systematically evaluated through a brute-force search to identify the maximal rewarded IRS relay, i.e, spectral efficiency. The random IRS relay approach selects a relay randomly. The classical TS method does not include any cost function and only considers the TS values. The relay with the highest TS is then chosen to establish the WiGig relay link, which only improves the distribution of spectral efficiencies. The classical TS method is used as a benchmark for the proposed BTS-IRS, as it is the original variation of BTS. Additionally, the performance of the brute-force search scheme is also included as it is a conventional method that leads to optimal data rate, yet with paying the highest BT time cost. The comparison also includes the results of the brute force search and random selection methods, which are important to include as the brute force search is a traditional heuristic approach and the random selection method is a simpler algorithm with lower computational requirements. Furthermore, previous work of [35] is included in the comparison to compare MAB-CE1, which is a budgeted UCB algorithm [40] that reflects the cost in both exploitation exploration terms, and MAB-CE2, which is derived from an explore then commit algorithm [39]. MAB-CE2 first explores the probed IRS relays and then decides the lowest cost (BT time) IRS relay from a set of highly rewarded IRS relays [35].

The setup scenario considers IRS relays to be randomly distributed across a simulated area of 2.5 Km${}^2$, unless otherwise specified. The WGBS and UE are located at opposite extremities of the simulated region. For consistency, the radiation patterns of WGBS and IRS panels must be equal in size to their respective antenna sizes, i.e., $\left|F\right|=M$ and $|{\mathsf{\Omega }}_r|=N_r$. The number of beams issued from WGBS is 16. The IRS panels have changed UPA sizes and beam count consequently, where $M_q$ uniformly lies between 60 and 600 antenna elements.

Table 1. Simulation parameters.

Parameter	Value
$P_t$	30 dBm
$BW$	2.16 GHz
${\sigma }_0^2$	−114 dBm
$K_{Br}$	5
$K_{ru}$	5
M	16
$\left|F\right|$	16
$T_{Hr}$	1000
$T_D$	20 ms
${\tau }_{BT}$	46 $\mathsf{\mu }$s
$PL\left(d_0\right)$	61 dB

states other important simulation parameters.

5.1. Performance versus Different Numbers of IRS Relays

Figure 2 plots the mean spectral efficiency of the compared schemes versus different numbers of IRS relays at 30 dBm transmitter power. As the IRS relays increase, the spectral efficiencies of all IRS relay probing schemes also increase due to the increased probability of obtaining a good IRS relay. The brute-force scheme has the optimum performance due to its strategy of finding out the supreme relay after searching all scattered relays. Moreover, the random approach delivers the worst performance as it stochastically decides IRS relay without caring about spectral efficiency values. Classical TS shows better performance than BTS-IRS, MAB-CE1, and MAB-CE2 due to its continuous selection of the highest rewarded IRS relay. BTS-IRS delivers better performance than other cost-effective MAB schemes due to its efficient decision policy. At eight IRS relays, around 95%, 77%, 75%, 61%, and 40% of the brute-force performance is attained using classical TS, BTS-IRS, MAB-CE1, MAB-CE2, and random IRS relay selection schemes.

5.2. Performance versus Different Values of Transmitter Power

Figure 3 previews the performance comparison of the IRS relay probing schemes in terms of mean spectral efficiency versus the transmitter power using 20 IRS panels/relays. For all schemes, as the transmitter power increases, the spectral efficiencies also increase consequently. The brute-force scheme has the optimum performance due to its policy of probing all accessible IRS relays. The random scheme has the worst performance because of the stochastic IRS relay probing policy. The classical TS-IRS scheme achieves better spectral performance than the proposed BTS-IRS because of its policy of choosing the highest spectral efficiency IRS relay without considering any cost. Our proposed BTS-IRS scheme has better performance than previously cost-effective, i.e., MAB-CE1 and MAB-CE2 [35], solutions due to its ease of probing policy upon the posterior distributions of the best IRS relay. At the transmitter power of 10 dBm, around 93%, 75%, 40%, 35%, and 10.46% of the brute force performance are achieved using classical TS, BTS-IRS, MAB-CE1, MAB-CE2, and random IRS relay probing, accordingly.

Figure 4 presents the mean throughput performance of all compared probing methods versus the transmitter power using 20 IRS relay. The brute-force scheme has the worst performance due to the effect of large BT time accumulated from trying all possible relays. Our envisioned BTS-IRS has comparable throughput performance compared to other schemes due to its efficient handling of BT. When the transmitter power equals 30 dBm, around 16, 15, 12, 9, and 5 times of the brute-force value are achieved utilizing BTS-IRS, MAB-CE2, TS, MAB-CE1, and random IRS relay probing, accordingly.

Figure 5 illustrates the mean BT time cost in ${10}^{-5}$ s of the IRS relay probing schemes under comparison versus the transmitter power at a fixed number of IRS relays which is 20. It has been observed that the BT cost remains constant when the transmission power is varied. The brute-force search delivers the highest BT performance, which is 0.92 msec and is the highest performance followed by TS-IRS due to its careless strategy of BT cost, as it cares only about picking up the relay with the highest spectral efficiency. The random IRS probing delivers a constant BT cost, i.e., 46 s performance, with lower performance than classical TS because of its randomized IRS relay probing strategy. The proposed BTS-IRs deliver comparable BT time performance. Note that MAB-CE2 has the lowest BT consumption due to its selection strategy of choosing the minimum cost arm from a set of highly rewarded arms after sufficient exploration. This ensures the effectiveness of solving the problem using the powered MAB methodologies that take into its account the BT cost.

5.3. Complexity Analysis

The IRS-enabled WiGig relay probing problem has two major sources of complexity: the complexity issued from the Beamforming Training (BT) process and the utilized algorithm computational complexity.

The main complexity source is the BT one, which consumes a significant amount of time (46 $\mathsf{\mu }$s) to test one TX/RX beam pair. The brute-force search algorithm has the largest BT complexity since it exhaustively evaluates all accessible IRS relays prior to deciding the supreme one $\mathcal{O}\left(R\left|{\mathsf{\Omega }}_r\right|\left|\mathcal{F}\right|\right)$. On the other hand, algorithms like the classical TS, and the random selection have much lower BT complexity due to the one BT process required per time slot, i.e., $\mathcal{O}\left(\left|{\mathsf{\Omega }}_r\right|\left|\mathcal{F}\right|\right)$. Our envisioned BTS-IRS is $\mathcal{O}\left(ln\left|{\mathsf{\Omega }}_r\right|\left|\mathcal{F}\right|\right)$ [41].

The algorithmic computational complexity, i.e., the execution speed of the algorithm, is minimal when compared to the complexity of the BT process. BTS-IRs and TS computational complexities are similar with $\mathcal{O}\left(R+1\right)$. This is due to sampling the 1D Gaussian Random Variable and updating its associated parameters [41].

6. Conclusions

This paper handled the problem of IRS-enabled WiGig relay probing by formulating it using a budgeted MAB setup. The main target is to find out the supreme IRS relay with the highest data rate and lowest BT time cost. Hence, we proposed a BTS-IRS algorithm, a variant of theoretically guaranteed TS, where the optimal IRS relay is selected via the maximal ratio of spectral efficiency and BT time cost. BTS-IRS delivered higher performance than the classical TS, brute force, and random IRS probing schemes in terms of spectral efficiency mean throughput. It also delivers comparable BT time consumption performance compared to MAB-CE1 and MAB-CE2. This paper paves the way for using advanced MAB schemes to handle various problems in IRS-enabled wireless communication topics effectively.