Research on Network Handover Based on User Movement Prediction in Visible Light Communication and Wi-Fi Heterogeneous Networks

Abstract

This paper addresses the handover challenge in indoor visible light communication and Wi-Fi heterogeneous networks, proposing an adaptive handover strategy based on user trajectory prediction. Extracting meaningful and important location points from massive trajectory data for clustering, an improved hidden Markov model is used to predict the user’s next location by analyzing the patterns of the user’s historical mobile trajectory data. The Q-learning algorithm is then used to determine the optimal network handover based on the current network state, while a seamless handover protocol is introduced to ensure successful network transition and uninterrupted data transmission. Compared with the traditional STD-LTE handover scheme, the proposed algorithm can reduce vertical handover rates by up to 32% during fast walking. When indoor user connections increase, the algorithm can maintain high fairness and high throughput when indoor user connections increase, verifying that it is robust in different scenarios.

1. Introduction

The widespread adoption of mobile networks has led to a dramatic rise in the number of Internet users in a short period of time. Additionally, the rapid development of the Industrial Internet has driven a sharp increase in the number of mobile devices accessing the Internet and the demand for related applications [1]. Due to the gradual depletion of radio spectrum resources, Wireless Fidelity (Wi-Fi), which operates at 2.4 GHz, has a limited spectrum bandwidth, which restricts its ability to accommodate the rapidly increasing data traffic and bandwidth demands. Visible light communication (VLC) offers abundant spectrum resources and combines lighting and communication, and it has the potential to become the next generation of wireless communication technology [2]. The heterogeneous network formed by VLC complementing Wi-Fi can provide indoor users with faster, greener, and more reliable communication services. However, since the coverage areas of the two networks overlap with each other in real scenarios, the random movement of users between different access points will lead to frequent network handovers. Research on efficient user movement prediction algorithms and solving the problems of the reasonable selection of network access points have become important challenges for heterogeneous communication networks.

Depending on whether the network belongs to the same technology mode before and after the handover, the handover mode can be categorized into two types [3]: vertical handover (VHO) and horizontal handover (HHO). HHO refers to the handover between the same technology, while VHO refers to the handover between different technologies. In indoor VLC/Wi-Fi heterogeneous networks, there is more literature on optimizing the VHO problem, and all of these studies have achieved specific results. Bao et al. proposed a handover decision mechanism based on channel-adaptive dwell time, which dynamically adjusts the dwell time to adapt to changes in the network environment [4]. Okine et al. proposed a hybrid application-aware VHO scheme that combines network state and application requirements (e.g., delay, bandwidth) to optimize the handover decision [5]. Babalola et al. predicted VHO using a hidden Markov model [6]. Liang et al. proposed combining a hierarchical analysis method with a cooperative game model to decide whether to perform VHO [7].

The VHO decision scheme implies a direct transfer to Wi-Fi for communication when the quality of VLC communication is degraded or interrupted. However, too many connections can easily lead to Wi-Fi overload and reduce system performance. To solve the network selection problem, Wu et al. proposed a centralized handover scheme using the statistical information of channel blocking [8], which employs optimization techniques to maximize the overall system throughput. Some scholars proposed using fuzzy logic [9] and artificial neural networks [10] in indoor heterogeneous networks. These methods optimize handover decisions using system parameters such as channel state information, user speed, and data rate. However, fuzzy logic rules must be predefined and adjusted, increasing practical applications’ complexity and maintenance cost. Artificial neural networks need many iterations to reach a stable state. An analysis of the above literature reveals that none of these studies consider the actual scenarios of mobile users. Additionally, they all suffer from high computational complexity and frequent handovers.

To better adapt to the dynamic needs of mobile users, trajectory prediction techniques can be introduced into indoor heterogeneous networks, avoiding the problem of frequent handovers caused by relying on a single piece of information (e.g., received signal strength) in the traditional methods. In indoor space, the density-based spatial clustering of applications with noise (DBSCAN) algorithm [11,12] is mainly used to mine mobile users’ trajectory features. Cheng et al. converted indoor location data into a sequence of stationary points with semantic information and clustered the trajectories using the DBSCAN algorithm [13]. Lan et al. utilized a density clustering algorithm to cluster the trajectories and computed the similarity between trajectories to identify the anomalous trajectories [14]. Laursen et al. proposed a tracking algorithm based on hidden Markov models (HMM) to predict indoor user trajectories [15]. Qiao et al. proposed methods that can adaptively change the HMM parameters to realize the prediction of different types of user trajectory data [16]. Wang et al. proposed a Markov chain-based campus trajectory prediction model by building a state transfer matrix to describe the transfer probabilities between different times and locations [17]. Existing studies show that traditional methods mainly cluster the spatial density of trajectory points. However, these methods ignore key features in the trajectory, such as time, direction, and speed. Directly analyzing the raw data will have the problem of a more extensive sample set, and the temporal performance of the convergence of the clustering is not quite in line with the expectation. In the aspect of the trajectory prediction, the prediction by using the traditional HMM will still have the inaccurate results. This is because the state transfer probability matrix describes the probability of a user moving from one location to another. Existing research does not take into account the problem that the user stays in a location for too long, which leads to a state transfer probability of 0 and partial prediction failure.

To address these problems, this paper proposes a network handover algorithm based on predicting the user’s mobile trajectory. The algorithm applies to VHO and HHO problems in indoor VLC/Wi-Fi heterogeneous networks. It reduces the number of handovers while improving system throughput, providing users with more stable and reliable communication services. The main contributions of this paper can be summarized as the two following points:

1. Important location points are extracted from the massive trajectory data to be clustered using the DBSCAN algorithm, and the clustering results are used as the hidden state of HMM to predict the user’s next possible location. This avoids the problem of prediction failure caused by the state transfer probability matrix being zero during the prediction process by introducing a linear smoothing mechanism.

2. An adaptive network handover strategy based on Q-learning is proposed. The strategy selects the best network access point for the user by considering the current network state, user trajectory, and service quality. A seamless handover protocol is introduced throughout the handover process for seamless data transmission.

The rest of the paper is organized as follows: Section 2 presents the system model for VLC/Wi-Fi heterogeneous networks, including the channel model and optical path blocking model for VLC and Wi-Fi. Section 3 details the proposed network handover algorithm based on location prediction, mainly consisting of trajectory clustering, movement prediction, and seamless handover protocol. Section 4 verifies the effectiveness of the algorithm in this paper. Finally, Section 5 summarizes this paper and outlines future research directions.

2. System Model

Figure 1 shows the indoor VLC/Wi-Fi heterogeneous network transmission model. Sixteen VLC access points (APs) are deployed in an array on the ceiling for downlink data transmission. At the same time, Wi-Fi AP are centrally located on the ceiling for uplink and partial downlink data transmission. The receiver is equipped with a photoelectric detector. All VLC APs and Wi-Fi APs are connected to a central controller via power lines [18]. The central controller monitors real-time communication transmission, collects channel status information at intervals, and manages user data.

2.1. VLC Channel Model

The indoor VLC channel model consists of both line-of-sight (LOS) and non-line-of-sight (NLOS) links. Although NLOS can provide complementary communication through reflected signals, its received power is typically more than 7 dB lower than the worst line-of-sight link [19]. The path loss of NLOS signals in typical indoor environments (e.g., an office or a factory) with wall reflectivity of about 0.7 can be expressed as follows:

$$P_{\mathrm{NLOS}}=P_t\cdot H_{\mathrm{LOS}}\cdot {\displaystyle \frac{\rho A_{\mathrm{eff}}}{{(d_1+d_2)}^2}}{\cos }^m(\varphi )\cos (\theta )$$

(1)

where $\rho$ is the reflectivity, $P_t$ is the transmit power, $H_{\mathrm{LOS}}$ is the channel gain of the LOS link, $A_{\mathrm{eff}}$ is the effective receiving area of the receiver, $\varphi$ is the LED transmitting angle, $\psi$ is the PD receiving angle, and $m=-\ln 2/\ln (\cos {\varphi }_{1/2})$ is the Lambertian radiation coefficient; $d_1$ and $d_2$ are the distances from the transmitting end to the reflecting surface and from the reflecting surface to the receiving end, respectively. Komine T. et al. experimentally determined that the reflected light’s contribution to the total received power of the receiver contribution is very low [20]. As shown in Figure 2, direct light occupies the vast majority of the received light, as high as 95.16%, and the proportion of primary and secondary reflected light is relatively low due to the secondary path loss and reflection efficiency limitations and the NLOS is significantly affected by multipath interference and dynamic masking. Therefore, this paper uses LOS as the primary channel model for indoor visible light communication to study the network handover problem under this link.

The optical channel gain from VLC AP to the user is expressed as

$$H_{i,j}^{\mathrm{VLC}}(0)=\left\{\begin{array}{ll}{\displaystyle \frac{(m+1)A_{\mathrm{eff}}}{2\pi D_{i,j}^2}}{\cos }^m\varphi T_s(\psi )g(\psi )\cos (\psi ),\hfill & 0⩽\psi ⩽{\psi }_c\hfill \\ 0,\hfill & \psi >{\psi }_c\hfill \end{array}\right.$$

(2)

where $m=-\ln 2/\ln (\cos {\varphi }_{1/2})$ is the Lambertian radiation coefficient; $A_{\mathrm{eff}}$ is the effective receiving area of the receiver; $D_{i,j}$ is the linear distance between the transmitter and receiver; $\varphi$ is the light-emitting diode emission angle; $\psi$ is the photoelectric detector receiving angle; $T_s(\psi )$ is the optical filter gain; $g(\psi )$ is the optical concentrator gain; and ${\psi }_c$ is the receiver half field angle, which determines whether the user can receive the optical signal from the line-of-sight link. The optical concentrator gain is given by [21]

$$\mathrm{g}(\psi )=\left\{\begin{array}{c}{\displaystyle \frac{n^2}{{\sin }^2{\psi }_c}},0\le \psi \le {\psi }_c\\ 0,\psi \ge {\psi }_c\end{array}\right.$$

(3)

where $n$ is the refractive index of the optical receiver. The signal-to-interference-plus-noise ratio (SINR) of user $j$ served by VLC AP $i$ is expressed as

$${\xi }_{i,j}^{\mathrm{VLC}}={\displaystyle \frac{{[\gamma P_{\mathrm{t}}^{\mathrm{VLC}}{H}_{i,j}^{\mathrm{VLC}}(0)]}^2}{{\displaystyle \sum _{k\in K}}{[\gamma P_{\mathrm{t}}^{\mathrm{VLC}}{H}_{k,j}^{\mathrm{VLC}}(0)]}^2+N_{\mathrm{VLC}}{B}_{\mathrm{VLC}}}}$$

(4)

where $\gamma$ is the photoelectric conversion coefficient of the receiver; $P_{\mathrm{t}}^{\mathrm{VLC}}$ is the transmitting power; $K$ is the interference set of VLC APs; $H_{k,j}^{\mathrm{VLC}}(0)$ is the interference channel gain of AP to user $j$; $N_{\mathrm{VLC}}$ is the noise power spectral density (approximately 1 × 10⁻²¹ in an indoor environment), and $B_{\mathrm{VLC}}$ is the bandwidth of the visible light communication system.

2.2. Wi-Fi Channel Model

The Wi-Fi channel gain for user $j$ is expressed as

$$G_{\mathrm{WiFi}}^j={\left|H_{\mathrm{WiFi}}\right|}^2{10}^{{\displaystyle \frac{-L(d_j)+X_{\sigma }}{10}}}$$

(5)

where $H_{\mathrm{WiFi}}$ is the channel transfer function, which follows the standard Rayleigh distribution; $X_{\sigma }$ represents shadow fading in the Wi-Fi channel, typically modeled using a zero-mean lognormal distribution to simulate the random variation in signal strength due to obstacles, with a standard deviation of $10 \mathrm{dB}$ [22]; $L(d_j)$ denotes the free-space path loss, and $d_j$ is the length of the communication link between Wi-Fi AP and user $j$. The signal-to-noise ratio (SNR) between the Wi-Fi router and user $j$ is formulated by

$${\xi }_{\mathrm{WiFi}}^j={\displaystyle \frac{G_{\mathrm{WiFi}}^j{P}_{\mathrm{WiFi}}}{N_{\mathrm{WiFi}}{B}_{\mathrm{WiFi}}}}$$

(6)

where $N_{\mathrm{WiFi}}$ is the power spectral density of the noise at the receiver, while $B_{\mathrm{WiFi}}$ and $P_{\mathrm{WiFi}}$ represent the system bandwidth and transmit power of the Wi-Fi AP, respectively.

2.3. Light-Path Blockage

In downlink visible light communication, the optical path blockage will affect the signal transmission quality. This paper uses two parameters, namely the occurrence rate and occupation rate, to model optical path obstruction. The occurrence rate denoted by ${\lambda }_u$ is defined as the average number of blockages occurring in the optical path per unit of time. To quantify the effect of blockages, it is assumed that they follow the Poisson point process, which is generally used to model unpredictable stochastic events. The occupation rate ${\eta }_u$ is defined as the proportion of time occupied by the optical path blockage. The range is $0<{\eta }_u<1$ [23].

In practice, when obstacles are near the transmitter or receiver, the LOS path is blocked, leading to a significant reduction in signal-to-noise ratio. User equipment (UE) receives minimal signals from LOS but may detect some NLOS signals. However, the total received signal power from the NLOS path is typically less than 20% [24]. In this paper, it is assumed that the received signal strength of UE is zero during the blockage period.

3. Network Handover Algorithm Based on Location Prediction

The network algorithm based on location prediction is divided into six parts, as shown in Figure 3. The indoor trajectory grid sequence is formed based on the mobile trajectory data matched by the central controller during signal acquisition. The historical mobility data are analyzed using data mining techniques to form trajectory clusters, and trajectory location prediction is conducted with the HMM. The mobility prediction module predicts the subsequent trajectory based on the user’s current position. The network at the predicted locations prepares in advance to minimize the handover failures caused by handover peaks and adjusts the handover management parameters accordingly.

3.1. Parameter Definition

A single moving trajectory point as follows

$$p=\{p\mid p=(x,y,time,angle)\}$$

(7)

The sequence of moving trajectories, consisting of multiple moving trajectory points in chronological order, as follows

$$originalTrail=\{p_1,\dots ,p_n\}$$

(8)

Based on the historical trajectory data, the shortest distance between two neighboring and non-repeating trajectory points in the indoor space is calculated, which is defined as the unit length, as follows

$$d(p_i,p_j)=\sqrt{{(x_i-x_j)}^2+{(y_i-y_j)}^2}$$

(9)

According to the defined unit length, the indoor space model is meshed to generate a meshed indoor space model. The collected trajectory data are projected according to the grid model, and a grid sequence is generated as follows

$$gridSequence=\{g_1,g_2,\dots ,g_m\},g_i=\{id,stoptime,angle\}$$

(10)

where $id$ is the grid number, $stoptime$ is the user dwell time, $angle$ is user deflection direction.

User dwell time: In a grid cell, if multiple trajectory points belong to the same grid but at different times, the time difference between these points is called dwell time. If the dwell time exceeds the predefined threshold, the grid cell is considered a dwell state, as shown in Equation (11).

$$stoptime=\left\{\begin{array}{c}0,g_j[id]\ne g_{i1}[id]\parallel g_{ik}[time]-g_{i1}[time]<{\delta }_t\\ g_{ik}[time]-g_{i1}[time],g_{ik}[id]=g_{i1}[id]\end{array}\right.$$

(11)

Due to the large scale of indoor mobile object trajectories and their redundant characteristics, the original trajectory data include many repetitive and consecutive trajectory points. The large trajectory dataset consumes significant storage space. Therefore, the original trajectory needs to be preprocessed.

The specific steps are as follows:

• The deflection angle is obtained by calculating the angle between two trajectory points, which serves as the basis for analyzing changes in trajectory direction.
• The user’s residence time is calculated based on its definition. Redundancies in the trajectory sequence are supplemented to ensure data integrity, facilitating the analysis of user behavior within the grid area.
• The deflection angle is standardized. The angle interval is divided into eight non-overlapping sub-intervals, and the number of sub-interval angles is counted to analyze the user’s movement distribution and enrich the trajectory prediction.

3.2. Trajectory Clustering

DBSCAN algorithm is a density-based spatial clustering algorithm. It uses a region with sufficient density as the distance center and connects samples of the same category. These connected samples continuously expand the region, this process known as clustering. In dealing with spatiotemporal trajectory data, DBSCAN can effectively deal with data with uneven density. Mobile trajectories are usually not uniformly distributed; for example, an important machine next to the work console and other locations may be very dense. DBSCAN can automatically adapt to the density distribution of the data through the density threshold and fast clustering. In the data collection, there will be noise points caused by measurement errors. DBSCAN can categorize low-density points as “noise” without forcing them to belong to a specific clustering, which makes DBSCAN highly robust when dealing with mobile trajectory data.

The residence time of the user in a particular state and the density of trajectories passing through a grid cell can highlight the importance of that location, such as near a key machine or working console. These important locations are extracted from the processed trajectory data. Then, the DBSCAN algorithm is used to identify high-density regions and group them into clusters. The definition of important location points is provided in Equation (12). A location is regarded as important if any one of the three conditions is satisfied.

$$\left\{\begin{array}{c}stoptime>{\delta }_t\\ num(angle)>{\delta }_a,({\displaystyle \frac{\pi }{4}}\le angle\le {\displaystyle \frac{7\pi }{4}})\\ N>{\delta }_n\end{array}\right.$$

(12)

where $stoptime$ is the dwell time, and ${\delta }_a$ is the threshold for the number of deflection angles. In this paper, we assume that user behavior changes when ${\displaystyle \frac{\pi }{4}}\le angle\le {\displaystyle \frac{7\pi }{4}}$, and ${\delta }_n$ is the threshold for the number of grid trajectory points.

3.3. Movement Prediction

In this section, clusters generated from clustering important location points in Section 3.2 are utilized as hidden states, and the HMM predicts user movement trajectories. The parameters of the HMM are defined as follows:

Hidden state sequence $C$: Hidden state sequences are generated by clustering historical grid trajectory sequences $\{C_1,C_2,\dots C_m\}$.

Observed state sequence $O$: Define the sequence of grid trajectories $\{g_1,g_2,\dots g_n\}$ as the observation state.

The state transfer probability matrix $A$:

$$A=\left[\begin{array}{cccc}{a}_{11}& a_{12}& \dots & a_{1m}\\ a_{12}& \dots & \dots & a_{12}\\ \dots & a_{ij}& a_{ii}& \dots \\ a_{m1}& a_{m2}& \dots & a_{mm}\end{array}\right],\sum _{j=1}^m{a}_{ij}=1$$

(13)

where $a_{ij}$ represents the state transfer probability between hidden states $C_i$ and $C_j$. To satisfy the constraints of the state transfer probability matrix $A$, $a_{ij}^{\prime }$ is first normalized, and then $a_{ij}$ is calculated.

$$a_{ij}^{\prime }=[{\beta }_a^{(ij)},{\beta }_{st}^{(ij)},{\beta }_{dis}^{(ij)}]\left[\begin{array}{c}{\omega }_a\\ {\omega }_{st}\\ {\omega }_{dis}\end{array}\right]$$

(14)

where ${\beta }_a^{(ij)}$ denotes the normalized angular eigenvalue between $C_i$ and $C_j$, ${\beta }_{st}^{(ij)}$ represents the residence time eigenvalue of $C_i$, ${\beta }_{dis}^{(ij)}$ is the reciprocal of the Euclidean distance between $C_i$ and $C_j$, and $\left[\begin{array}{c}{\omega }_a\\ {\omega }_{st}\\ {\omega }_{dis}\end{array}\right]$ denotes the weight set for the three eigenvalues.

Observation probability matrix $B$:

$$B=\left[\begin{array}{cccc}{b}_{11}& b_{12}& \cdots & b_{1n}\\ b_{21}& \cdots & \cdots & b_{2n}\\ \cdots & b_{ij}& b_{ii}& \cdots \\ b_{m1}& b_{m2}& \cdots & b_{mn}\end{array}\right],\sum _{j=1}^n{b}_{ij}=1$$

(15)

where $b_{ij}$ is the transfer probability between grid cells and the clusters of the hidden state sequences, to satisfy the constraints of the state transfer probability matrix $B$, $b_{ij}^{\prime }$ is first normalized, and then $b_{ij}$ is obtained.

$$b_{ij}^{\prime }=\left\{\begin{array}{c}1,g_j\in C_i\\ \left[{\gamma }_a^{(ij)},{\gamma }_{dis}^{(ij)}\right]\left[\begin{array}{c}{w}_a\\ w_{dis}\end{array}\right],g_j\notin C_i\end{array}\right.$$

(16)

where $g_j$ is the grid cell, ${\gamma }_a^{(ij)}$ is the eigenvalue of the normalized angle between $C_i$ and $g_j$, and ${\gamma }_{dis}^{(ij)}$ is the inverse of the Euclidean distance between $C_i$ and $g_j$.

Initial state probability matrix $\lambda ={[0,\dots ,{\displaystyle \frac{1}{k}},0,{\displaystyle \frac{1}{k}},\dots ,0]}^T$, the $k$ clusters neighboring indoor entrances are selected and their initial states are set to be uniformly distributed with an initial state probability of ${\displaystyle \frac{1}{k}}$ and the rest of the state probabilities are zero.

After determining the parameters of each part of the HMM, we train the model using the Baum–Welch algorithm [25]. In user movement, the user may stay at a specific location for a long time, resulting in the continuous trajectory points projected to the same cluster. The state transfer probability $a_{ij}$ of the classical HMM may become zero, which will cause the prediction failure, and this problem can be solved by introducing a smoothing factor ${\epsilon }_a$. The user dwell time obeys a long-tailed distribution. Find the maximum dwell time for all grids in cluster $C_t$, as shown in the following equation.

$$stoptime(C_t)=\underset{1\le v\le k}{\max }(stoptime(g_{tv}))$$

(17)

where $g_{tv}$ denotes the grid point $v$ in Cluster $C_t$, $k$ is the number of grid points in Cluster $C_t$. $stoptime(C_t)$ denotes the maximum value of dwell time for all grid points $g_{tv}$ in Cluster $C_t$.

The residence time is mapped using a mapping function $L\mathrm{ogistic}$ in the range of [0,1] as follows:

$${\epsilon }_a=L\mathrm{ogistic}(stoptime(C_t)),0<{\epsilon }_a\le \max (a_{tz})$$

(18)

The smoothing factor ${\epsilon }_a$ dynamically adjusts the state transfer probability. This ensures that the state transfer probability does not drop to zero, even if the state residence time is extended, as follows:

$$a_{ij}^{\prime }=\left\{\begin{array}{c}{\epsilon }_a,i=j\\ (1-{\epsilon }_a)a_{ij},j\ne i\end{array}\right.$$

(19)

By introducing ${\epsilon }_a$, $a_{ij}$ can obtain suitable non-zero values when continuously in the same cluster. In this way, the model can perform effective state transfer even in the case of consecutive identical cluster sequences, avoiding the problem of prediction failure.

After obtaining the trained HMM, we use the Viterbi algorithm for location prediction. We return the obtained maximum hidden state sequence to the grid cells within the cluster, which is the location where the user is most likely to move.

3.4. Seamless Network Handover

Q-learning is a reinforcement learning algorithm based on value iteration and independent of the environment state. It relies on the interaction between the agent and the environment to iteratively learn an optimal strategy and make the best decision. We apply this algorithm to network handover. Due to its low complexity and computational simplicity, it significantly simplifies the network handover implementation process. The algorithm identifies the optimal action for each corresponding state by continuing to explore the action-state space until convergence.

In this paper, the central controller is considered an agent that performs the assignment of network access points. Based on the network state $S=(s_1,s_2,s_3)$, the greedy strategy selects the optimal action from set $A=(a_1,a_2,a_3)$ with probability $1-\epsilon$, derived from feedback on state transition and reward. Another action is randomly performed with probability $\epsilon$. The detailed parameters are defined as follows:

• State: $S=(s_1,s_2,s_3)$,where $s_1$ represents the current service network, either VLC or Wi-Fi; $s_2$ indicates the VLC coverage area to which the user will move in the future; and $s_3$ indicates the current quality of service, represented by 0 or 1, depending on whether the SINR is below or above the threshold (10 dB).

Action: $A=(a_1,a_2,a_3)$,where $a_1$ represents handover to the best VLC, $a_2$ represents handover to Wi-Fi, and $a_3$ represents no handover.

Reward function: The reward function combines the key elements of handover success, user satisfaction with the data rate, and handover cost.

$$R=\mathrm{I}\ast \left[\zeta \ast \beta +(1-\delta )(1-\beta )\right]$$

(20)

The parameters are defined as follows:

$\mathrm{I}\in \{0,1\}$ is the handover success identifier. If the handover is successful, then $\mathrm{I}=1$; if it fails, $\mathrm{I}=0$.

$\zeta ={\displaystyle \frac{r_{current}}{r_{previous}}}$ represents user satisfaction, which indicates the data rate ratio after handover compared to before handover. The higher the value of $\zeta$, the more significant the improvement in the data rate the user experiences after handover, leading to greater satisfaction with the network service.

$\delta$ represents the handover cost. HHO involves only handover between the APs of the same technology and has a low latency of approximately 200 ms. VHO requires cross-technology negotiation and resource reconfiguration. It incurs a higher latency, approximately 500 ms [26]. Based on the measured data and the literature [26], the handover cost is set to ${\delta }_{NHO}=0$, ${\delta }_{HHO}=0.3$, and ${\delta }_{VHO}=0.7$, respectively, to quantify the difference in resource overhead for different handover types.

$\beta \in \{0,1\}$ is a tradeoff between the data rate and the number of handovers. When $\beta =1$, the reward function simplifies to $R=\mathrm{I}\ast \zeta$. At this point, the reward favors actions that achieve the maximum data rate, with less consideration given to the number of handovers required. In this case, the algorithm will prioritize pursuing a high data rate, even if it may result in more handover actions. When $\beta =0$, the reward function becomes $R=\mathrm{I}\ast (1-\delta )$. At this point, the algorithm prioritizes minimizing the number of handovers. It may sacrifice some data rate to reduce handovers.

Since handover introduces data overhead in addition to information transmission, it can degrade the system performance. Therefore, this paper introduces a sequence number (SN) state synchronization mechanism and an acknowledgment number (Ack) mechanism within the transmission control protocol [27] during network handover to ensure seamless handover and uninterrupted data transmission, as depicted in Figure 4. The specific implementation process is as follows:

• The central controller periodically collects the user’s location and network status parameters.
• When the central controller decides that network handover is required, it sends a handover request to the target AP, and the target AP returns an Ack as a handover confirmation.
• During handover decisions, the central controller sends an SN status request to the UE to synchronize downlink data transmission. It then forwards the SN status reported by UE to the target AP to ensure data continuity.
• Throughout the negotiation process, the UE maintains communication with the original network until the handover is successful.

In Figure 4, during each iteration, the agent initially observes the environmental state at the current time t and selects an action based on the greedy strategy to obtain immediate rewards. Next, the agent updates the Q-value corresponding to the current state and action based on the maximum expected discounted reward $\max Q(s_{t+1},a^{\prime })$ at time $t+1$. The Q-value update equation is as follows:

$$Q(s,a)\leftarrow Q(s,a)+\alpha [r+\gamma \max Q(s_{\mathrm{t}+1},a^{\prime })-Q(s,a)]$$

(21)

where $\alpha$ represents the learning rate, which affects the update speed of the Q table; $r$ denotes the immediate reward; $\gamma$ is the discount rate, representing the impact of future rewards on the current step. When $\gamma \to 0$, the agent focuses more on the reward obtained by the current action; when $\gamma \to 1$, the agent considers future rewards more heavily. $\max Q(s_{\mathrm{t}+1},a^{\prime })$ represents the maximum Q value among the available actions in the next state. This process is repeated until the final state is reached, marking the completion of a learning episode. Through repeated learning, the algorithm gradually learns the optimal action for each state, corresponding to the behavior that maximizes the long-term reward in each state, forming the optimal policy set. Over time, the exploration rate gradually decreases, the algorithm gradually shifts from the initial exploration to exploiting the best-known strategy.

4. Discussion

The simulation parameters for the indoor VLC/Wi-Fi heterogeneous network are shown in

Table 1. Simulation parameters.

Parameter	Value
Room size (length by width by height)	15 m × 15 m × 3 m
The cost of VHO, $C_V$	500 ms
The cost of HHO, $C_H$	200 ms
The physical area of the PD, $A_{\mathrm{eff}}$	1 cm2
Field of view semi-angle of the PD, ${\psi }_c$	90°
Detector responsivity, $\gamma$	0.53 A/W
Half-intensity radiation angle, ${\varphi }_{1/2}$	60°
System bandwidth per VLC, $B_{\mathrm{VLC}}$	20 MHz
Transmitted optical power per VLC, $P_{\mathrm{t}}^{\mathrm{VLC}}$	3.5 W
Noise power spectral density in VLC, $N_{\mathrm{VLC}}$	10−21 A2/Hz
System bandwidth per Wi-Fi, $B_{\mathrm{WiFi}}$	20 MHz
Transmitted power per Wi-Fi, $P_{\mathrm{WiFi}}$	15 dBm
Noise power spectral density in Wi-Fi, $N_{\mathrm{WiFi}}$	−174 dBm/Hz

. The maximum number of iterations for the algorithm is set to 3000, with the $\epsilon$, $\alpha$, and $\gamma$ set to 0.1, 0.5, and 0.2, respectively. In this paper, two schemes are introduced for comparison with the proposed algorithms: the traditional standard STD-LTE handover scheme and the improved fuzzy logic handover scheme (FL-HO) [28], with the handover threshold and trigger time set to 1 dB and 320 ms, respectively.

4.1. Performance Indicators

(1)

Throughput

Throughput is a key measure of a user’s data transmission efficiency in a wireless network. It represents the amount of actual data successfully transmitted per unit of time. The throughput calculation in this paper takes into account the user’s SNR and the type of network connected (VLC or Wi-Fi), and the throughput calculation is expressed as:

$$r_j=\left\{\begin{array}{ll}{\displaystyle \frac{B_{\mathrm{VLC}}}{2}}\cdot {\log }_2\left(1+{\displaystyle \frac{e}{2\pi }}{\xi }_{\mathrm{VLC}}^{i,j}\right),\hfill & \mathrm{for} \mathrm{VLC} \mathrm{AP}\hfill \\ B_{\mathrm{WiFi}}\cdot {\log }_2\left(1+{\xi }_{\mathrm{WiFi}}^j\right),\hfill & \mathrm{for} \mathrm{WiFi} \mathrm{AP}\hfill \end{array}\right.$$

(22)

where ${\xi }_{\mathrm{VLC}}^{i,j}$ and ${\xi }_{\mathrm{WiFi}}^j$ are the SNR between the user and the AP, and $B_{\mathrm{VLC}}$ and $B_{\mathrm{WiFi}}$ are the bandwidths of VLC and Wi-Fi, respectively.

(2)

Handover rate

The handover rate is an important measure of the frequency of the handover in a network. It is often used to evaluate the efficiency and stability of network handover algorithms. The calculation of the handover rate is expressed as follows:

$$\mathrm{Handover}\hspace{0.33em}\mathrm{Rate}={\displaystyle \frac{{\mathrm{N}}_{\mathrm{HO}}}{\mathrm{T}}}$$

(23)

where ${\mathrm{N}}_{\mathrm{HO}}$ is the number of handovers occurring during the observation time $\mathrm{T}$. Frequent handover not only consumes additional network resources but also interrupts data transmission, significantly affecting the user experience and the overall stability of the network.

(3)

Fairness

Jain’s fairness index is an important index to measure the fairness of resource allocation in network handover, and the calculation of fairness is expressed as:

$$J={\displaystyle \frac{{\left({\sum }_{i=1}^N{x}_i\right)}^2}{N{\sum }_{i=1}^N{(x_i)}^2}}$$

(24)

where $N$ denotes the number of users and $x_i$ denotes the amount of resources allocated to the user (e.g., bandwidth, throughput), which in this paper refers to the user’s throughput. The range of values is [0, 1]. This index can help evaluate whether users have fair access to network resources when the handover is between different access technologies. A high fairness index implies that the difference in throughput between users is slight, and the allocation of network resources is more balanced.

4.2. Performance Analysis

4.2.1. Trajectory Prediction Analysis

In order to verify the influence of the parameters on the prediction accuracy, this paper tests the influence of different values of the smoothing factor ${\epsilon }_a$ on the prediction accuracy, as shown in

Table 2. Effect of smoothing factor on prediction accuracy.

Smoothing Factor	Prediction Accuracy
0.1	0.68
0.3	0.72
0.5	0.82
0.7	0.71
0.8	0.64
0.9	0.56

. The experiments show that, when ${\epsilon }_a$ varies from 0 to 1, the model’s prediction accuracy tends to increase and decrease. The accuracy is highest when ${\epsilon }_a=0.5$. When ${\epsilon }_a<0.3$, the model cannot effectively alleviate the user stay problem. When ${\epsilon }_a>0.7$, excessive smoothing leads to the loss of trajectory dynamic features and a significant increase in the error rate. This indicates that moderate smoothing can effectively alleviate the problem of zero-state transfer probability, but excessive smoothing will blur the dynamic characteristics of the trajectory.

This paper selects five sets of user trajectories from the dataset to simulate both the classical and the improved HMM. Figure 5 illustrates the comparison of prediction errors before and after the improvement, showing that the prediction error rate of the classical HMM is generally higher than that of the improved HMM. This is because the indoor space is limited, causing mobile objects to be constrained in their movement, resulting in many repetitive trajectories. Consequently, the classical HMM will have a transition probability of 0, leading to prediction failure. On the other hand, the improved HMM enhances the prediction accuracy by addressing the state dwell problem.

Figure 6 compares the prediction accuracy under different data repetition rates. It shows that the difference in prediction accuracy between the classical and improved HMM is slight when the trajectory repetition rate is low. However, as the user continues to move and the trajectory repetition rate increases, the rate of decrease in the prediction accuracy for the classical HMM becomes more extensive. In contrast, the decrease for the improved HMM is more minor.

4.2.2. Network Handover Analysis

(1)

Average throughput

Figure 7 demonstrates the variation of system throughput with user speed. It can be seen that when the user speed is low (v < 1 m/s), the difference in throughput among the three handover methods is not apparent. As the user speed increases, the channel conditions change more frequently, leading to an increase in the number of handovers and a decrease in throughput. At normal walking speed (v = 2 m/s), the throughput is slightly higher than the two compared algorithms. This is because the method in this paper still maintains a specific exploratory ability while accumulating certain knowledge and can better adapt to the dynamically changing network environment. The handover mechanism of STD-LTE is based on a fixed threshold that triggers frequent handovers in the face of fast user movement, which reduces the adequate transmission time. The decision rule of the FL-HO scheme is fixed, and the decision-making is not precise enough in high-speed movement. In contrast, the algorithm in this paper is based on user trajectory prediction and reinforcement learning, which can dynamically adjust the handover strategy according to the network state. When the user is moving at low speed or in a stationary state, the decision-making process is slower and the algorithm accumulates less experience. As a result, the throughput is lower than that of the FL-HO algorithm at v < 1.5 m/s. As the speed increases, the algorithm is triggered more frequently, and it can learn faster to select a more suitable network (e.g., Wi-Fi) for fast-moving users. When the user speed is 4 m/s, the throughput of the algorithm in this paper is about 42% higher than that of the traditional STD-LTE scheme and about 27% higher than that of the FL-HO scheme.

Figure 8 shows the variation of the system throughput with the number of room users in the case of humans’ normal indoor walking speed of 2 m/s. With the increase in the number of users, the throughput of STD-LTE and fuzzy logic methods gradually decreases, while the throughput of this paper’s algorithm remains relatively stable. When the number of users is 22–26 people, the throughput slightly decreases because of the increased competition for the system resources. However, it is still higher than the other two comparison algorithms. When the number of users reaches 26, the throughput is 462 Mbps, about 33% higher than the STD-LTE scheme, and about 15% higher than the FL-HO scheme.

Figure 9 demonstrates the effect of the different optical path-blocking incidence $\lambda$ on the system throughput, where the number of users in the room is set to 15 and the speed is 2 m/s. The optical path blocking rate indicates the average number of times that the optical path is blocked per unit of time. Modeling the optical path blocking as a Poisson process, uniformly distributed between 0 and 1, ensures that the average number of blocking events occurring per unit of time is constant and each blocking event occurs independently. When $\lambda =2$, the frequency of blockings are low, and handovers are primarily driven by user mobility. The throughput of our proposed scheme is 28% higher than that of STD-LTE scheme. The FL-HO scheme which also considers user mobility in its decision-making process has a performance close to that of the proposed scheme when $\lambda$ is low. As $\lambda$ increases, the throughput of all schemes decreases. However, the decreasing trend of our proposed scheme is slower. When $\lambda$ increases from 2 times/minute to 10 times/minute, the throughput achieved by the proposed scheme drops from 461 Mbps to 408 Mbps (only 11%). The throughput of STD-LTE decreases by 35%, and the throughput of FL-HO decreases by 22%. This demonstrates that our algorithm maintains a good performance even under frequent optical path blockings.

(2)

Handover rate

Figure 10 discusses the changes in the horizontal and VHO rates due to different speeds. As users traverse the network coverage areas at higher speeds, the handover rate increases proportionally. When speed increases from 0.5 m/s to 4.0 m/s, the horizontal and VHO rates show a general upward trend across all schemes, but the handover rate of the proposed algorithm rises more gradually. When v = 4 m/s, the proposed algorithm reduces the HHO rate by 42% and 26%, and the VHO rate by 32% and 15% compared to STD-LTE scheme and FL-HO scheme. This occurs because the proposed algorithm dynamically adjusts its handover strategy based on current network conditions, learning through interactions with the environment. Therefore, as user speed increases, the learning process accelerates with more frequent handovers, allowing the decision-making process to find the optimal action for future states more quickly.

Figure 11 shows the variation of the handover rate for different optical path-blocking scenarios. When the optical path blockage occurs, the UE needs to handover to other access points, increasing the handover rate, verified for both comparison schemes in the figure. In contrast, our proposed scheme reduces the HHO rate from 0.48 to 0.31 as optical path blockings increase. This reduction occurs because our scheme predicts the user’s next position, identifies whether the user will enter a blocking region, and avoids unnecessary handovers in these areas. When $\lambda$ increases from 5 to 10 times per minute, the VHO rate increases from 0.53 to 0.62, a slight increase of 17%. At $\lambda =10$ times per minute, the scheme proposed in this paper reduces the handover rate most significantly, reducing the horizontal and vertical handover rates by 56% and 53%, respectively, compared to STD-LTE. Compared with the FL-HO scheme, it reduces the handover rate by 35% and 17%, respectively. While our algorithm demonstrates effectiveness in reducing the handover rate under typical conditions, the method struggles to precisely determine user positioning and optimal network access points under severe indoor interference scenarios. This limitation manifests particularly in industrial environments with electromagnetic interference affecting Wi-Fi transmission, or when randomly positioned obstacles disrupt VLC line-of-sight links, potentially triggering excessive network handovers.

(3)

User fairness

Figure 12 shows the variation of user fairness between different algorithms under different numbers of users. As can be seen from the figure, the fairness of STD-LTE decreases significantly with the increase in the number of users. When the VLC signal strength decreases, STD-LTE decides to handover based only on the signal strength, and users will handover to Wi-Fi for communication to pursue a better signal quality. However, the bandwidth resources of Wi-Fi APs are limited, and an increase in the number of user connections reduces the average available bandwidth per user, decreasing the average throughput. At the same time, VLC AP resources may remain underutilized, and the average throughput of users served by VLC APs may increase. This imbalance in resource allocation widens the throughput difference between users, leading to decreased fairness. The higher fairness of the scheme proposed in this paper and the FL-HO scheme is achieved because the FL-HO scheme constructs a centralized optimization problem, which takes the handover cost due to optical path blocking and user movement as the primary considerations. The scheme proposed in this paper can determine the user’s movement trend and demand in advance, and the designed reward function introduces the handover cost, balancing parameter, encouraging the algorithm to select the AP that can satisfy the user’s data rate demand and has low handover cost while guaranteeing the success of handover. For high-speed mobile users, the algorithm will tend to select Wi-Fi directly because the frequent handover to VLC may lead to the optical link blocking and low data rate, obtaining lower rewards; while for low-speed mobile users, direct HHO can obtain higher rewards, ensuring the fairness of the overall network.

(4)

Convergence case

Figure 13 shows the reward convergence of the algorithm for different numbers of user connections, and the number of iterations required for the algorithm to converge increases gradually with the number of people. This is due to the rise in network complexity, increase in state space and action space, increase in diversity of different users’ movement trajectories, data requirements, and optical link occlusion scenarios, resulting in the algorithm requiring more iterations to explore and learn from these complexities. Despite increasing users leading to slower convergence and lower reward values, the algorithm can gradually converge to a steady state, showing good robustness.

5. Conclusions

To improve the efficiency and stability of indoor VLC/Wi-Fi heterogeneous network handover, this paper proposes a network handover algorithm based on mobility trajectory prediction, suitable for VHO and HHO in heterogeneous networks. The algorithm simplifies the indoor spatial model, and by analyzing the user’s historical movement data and combining it with the improved HMM algorithm, the user’s position at the next moment can be accurately predicted, enabling more accurate handover decisions. Specifically, when the user speed is 4 m/s, the proposed algorithm improves throughput performance by approximately 42% compared to the traditional handover algorithm. As the number of users increases, the proposed algorithm better maintains high fairness. When the number of connected users reaches 26, the throughput is improved by about 33% over the traditional algorithm. In terms of handover rate, as the user speed increases, the algorithm proposed in this paper can reduce the HHO rate by 42% and the VHO rate by 32%.

Although the proposed algorithm performs well in most scenarios, the trajectory prediction module relies on the completeness and consistency of historical data. The prediction accuracy may decline when sudden changes occur in the user’s movement pattern or when the data are insufficient. An online learning framework can be introduced to enable the model to dynamically update parameters and adapt to changes in user behavior in real-time. The network handover algorithm proposed in this paper accounts for two key factors: user mobility and network signal quality. However, when additional environmental factors are introduced, such as network load and user preferences, the state space becomes more complex. As a result, Q-learning may experience slower convergence and higher computational complexity. Future research can further explore advanced multi-objective optimization methods to better balance these parameters. Moreover, advanced reinforcement learning algorithms can be implemented to handle high-dimensional state spaces, ensuring more stable communication services for users.