A Tilt Visible Light Positioning System Based on Double LEDs and Angle Sensors

Abstract

Visible light positioning (VLP) has been studied widely due to its high accuracy and low cost in the field of location-based services (LBS). However, many existing VLP systems have the requirements that the receiver should be placed horizontally and more than three LED lamps should be used, which are difficult to meet in practical scenarios. Therefore, it is necessary to develop a novel VLP algorithm for tilted conditions. An effective and simple VLP system while the receiver is tilted based on double-LED lamps is proposed in this paper. The vertical position can be determined by combining the information from angle sensors with geometric information. Through analyzing the imaging characteristics of the tilted state, we can utilize the relationship of similarity to calculate the location of the mobile receiver. Experimental results show that the positioning accuracy of our proposed algorithm can reach 5.48 cm.

1. Introduction

With the development of indoor location-based services (LBS), indoor positioning is in urgent need. The widely used Global Positioning System (GPS) cannot be directly applied to an indoor environment because the signal from satellites may be blocked by the wall of buildings [1]. Besides, traditional indoor positioning systems (IPS) such as wireless local area network (WLAN), radio-frequency identification (RFID), Bluetooth and ultra-wideband (UWB) have various limitations in terms of high cost, low accuracy and electromagnetic interference [1]. In contrast, visible light positioning (VLP) based on LED and visible light communication technology is a better choice. Firstly, it does not have the limitations mentioned above. Secondly, energy-efficient LED lamps can be seen everywhere in daily life. Most importantly, the high-frequency flickers are invisible to human eyes, which means that the LED lamps can realize the dual functions of lighting and positioning.

According to the type of receiver, VLP systems can be divided into two categories: image sensor (IS)-based VLP [2,3,4,5] and photodiode (PD)-based VLP [6,7,8,9,10]. For the PD-based VLP, the methods of time of arrival (TOA) [11] and time difference of arrival (TDOA) [12] require high-precision time measurement devices; the received signal strength (RSS) [13] is sensitive to light intensity variation. Angle of arrival (AOA) and RSS have been combined to achieve three-dimensional positioning [14], but the computational complexity and positioning accuracy may not meet our needs. IS-based VLP is more suitable to be applied in indoor environments because it is not sensitive to the diffuse reflection of the light signal and the light intensity variation compared with the PD-based VLP [15,16]. Moreover, smartphones equipped with complementary metal oxide semiconductor (CMOS) image sensors have been widely used, which means that we do not have to pay for the extra receiver.

For IS-based VLP, the LED lamps were treated as point sources without geometric information in the early positioning research. Though centimeter-level accuracy has been achieved in some VLP systems, at least three LEDs should be captured in a picture [17,18]. However, this requirement is difficult to meet in reality, since the camera’s field of view (FOV) is limited, and LED lamps are usually deployed sparsely in a building. Thus, the robustness and flexibility of the VLP system are greatly restricted. In [5,19], angle sensors were employed to “compensate” the receiver’s orientation information in the case of the shortage of LED lamps, and the accuracies of 3.85 and 6.5 cm were achieved. The inaccuracy of azimuthal angle is the main source of the navigation and positioning errors [20,21]. In [22], a positioning accuracy of 17.52 cm was achieved based on a single LED lamp with a marginal point. However, the marked point would be difficult to recognize for the camera as the distance increases. Moreover, most of the algorithms that have been proposed are only applicable when the receiver is in a horizontal state [3,5,23,24,25,26], which restricts the applications of VLP systems.

A tilt positioning system based on double LED and angle sensors is proposed to solve the above problems in this paper. We further explored the geometric features of images and then constructed the equations for the target position. For angle sensors, unlike previous studies such as [23,24,27], only the pitch and roll angles are needed from angle sensors to participate in calculation and selection in our system, which avoids interference from geomagnetism. Generally, VLP is operated in a closed indoor environment, which causes the azimuthal angle measured by the geomagnetic sensor to be affected by walls or other buildings, and azimuthal angle is involved in the calculation in many traditional VLP positioning algorithms. We solved two practical problems to a certain extent: more than three LED lamps are required and algorithms may be invalid when the mobile receiver is tilted. Moreover, the algorithm in this paper is simple to implement and computationally efficient, and it can be practical in subway stations, underground parking lots and so on.

2. Positioning System

2.1. System Overview

The architecture of our proposed scheme is shown in Figure 1a. LEDs with the radius of R are modulated to broadcast their location signals, and they flash at a frequency that cannot be detected by our eyes. Spatially varying strips can be captured by the camera according to the rolling shutter effect (RSE) [28] as shown in Figure 1b. The mobile receiver can be tilted arbitrarily while capturing, but it should be certain that double-LED lamps are in the FOV of the camera. Then, geometric information and the ID information of LEDs can be obtained through image processing technology. We can obtain the 3D position of the mobile receiver from our proposed algorithm after obtaining the required data.

Figure 2a shows the system model of the proposed algorithm. The centers of LED lamps in a 3D world coordinate system are recorded as $\left({\mathrm{X}}_1,{ \mathrm{Y}}_1,{ \mathrm{Z}}_1\right)$ and $\left({\mathrm{X}}_2,{ \mathrm{Y}}_2,{ \mathrm{Z}}_2\right)$. Since the lamps are placed at the same height, ${\mathrm{Z}}_1={\mathrm{Z}}_2$. The midpoint of the lens is estimated to be the terminal position in the 3D coordinate system, which is recorded as point P $\left(\mathrm{X},\mathrm{Y},\mathrm{Z}\right)$. After capturing, the coordinates in the image coordinate system can be calculated as follows:

$$\mathrm{u}=\frac{i}{di}+{\mathrm{u}}_0$$

(1)

$$\mathrm{v}=\frac{j}{dj}+{\mathrm{v}}_0$$

(2)

where $(\mathrm{u}$, $\mathrm{v})$ is the coordinate in the pixel coordinate system, and $\left({\mathrm{u}}_0,{\mathrm{v}}_0\right)$ is the midpoint.$(i$,$j)$ is the coordinate in the image coordinate system. The unit conversion relationship is 1 pixel = $dj$ mm. The mapped points corresponding to the center of LED1 and LED2 are recorded as $\left(i_1,j_1\right) \mathrm{and} \left(i_2,j_2\right)$, and they are both located in the center of the imaging ellipses. The semimajor axes of ellipses are $\mathrm{a}1$ and $\mathrm{a}2$, as shown in Figure 2b. $\left(i_0, j_0\right)$ is the midpoint of the image coordinate system. $\mathrm{f}$ is the parameter of the camera’s focal length. We can read the pitch angle $\mathsf{\alpha }$along the X-axis and the roll angle $\mathsf{\beta }$ along Y-axis directly through angle sensors. The 3D target location of $\mathrm{P}$ is determined by the following description.

2.2. Principle of Positioning

2.2.1. Calculating Z Coordinate

In the proposed system, an obvious similarity can be found when the receiver is tilted, which refers to that the proportions of the length in a three-sided pyramid is equal to that of another three-sided pyramid. As shown in Figure 2a, the intersection line between the horizontal plane and the imaging plane is exactly parallel to the line where the major axis of the ellipse is located. Thus, the major axis is parallel to a certain radius of the LED lamp, and the two three-sided pyramids in red color in Figure 2a are similar because all corresponding angles between them are the same. Then, we can obtain two equations about $Z$.

$$\frac{Z_1-Z}{R} =\frac{h_1}{a_1}$$

(3)

$$\frac{ Z_2-Z}{R} =\frac{h_2}{a_2}$$

(4)

where $h$_k$\left(k=1, 2\right)$ is the vertical distance from the lens to the horizontal plane where the major axis is located.

Figure 3 shows different situations based on the position of the ellipse in the image. Then, we can obtain $h_k$ in two cases for each LED.

$$\left[\begin{array}{c}{h_1}^{\prime }\\ {h_1}^{″}\end{array}\right]=\left[\begin{array}{cc}f& -d1\\ f& d1\end{array}\right]\left[\begin{array}{c}\cos t\\ \tan t\ast \cos t\end{array}\right]$$

(5)

$$\left[\begin{array}{c}{h_2}^{\prime }\\ {h_2}^{″}\end{array}\right]=\left[\begin{array}{cc}f& -d2\\ f& d2\end{array}\right]\left[\begin{array}{c}\cos t\\ \tan t\ast \cos t\end{array}\right]$$

(6)

where $d_k$=$\sqrt{{\left(i_k-i_0\right)}^2+{\left(j_k-j_0\right)}^2}$ (k = 1, 2); $t$ is the angle between the imaging plane and the horizontal plane, which can be calculated through $\cos \left(t\right)=\cos \left(\mathsf{\alpha }\right)\ast \cos \left(\mathsf{\beta }\right)$. ${h_k}^{\prime }\left(k=1, 2\right)$ and $h_k^{″}\left(k=1, 2\right)$ represent the $h_k$ when the ellipse is on the upper or lower side of the horizontal line passing through the center. However, only two of ${h_k}^{\prime }\left(k=1, 2\right)$ and $h_k^{″}\left(k=1, 2\right)$ are correct, since the state of each ellipse is fixed in one image.

We do not pick the correct $h_k\left(k=1, 2\right)$ for each ellipse in the next step, but substitute ${h_k}^{\prime }$ and $h_k^{″}$ into $h_k$(k = 1, 2) in Equations (3) and (4).

$$\left[\begin{array}{c}{Z}_{11}\\ Z_{12}\end{array}\right]=\left[\begin{array}{c}{Z}_1\\ Z_1\end{array}\right]-\frac{R}{a_1}·\left[\begin{array}{c}{h_1}^{\prime }\\ {h_1}^{″}\end{array}\right]$$

(7)

$$\left[\begin{array}{c}{Z}_{21}\\ Z_{22}\end{array}\right]=\left[\begin{array}{c}{Z}_2\\ Z_2\end{array}\right]-\frac{R}{a_2}·\left[\begin{array}{c}{h_2}^{\prime }\\ {h_2}^{″}\end{array}\right]$$

(8)

where only two of $Z_{mn}$(m = 1, 2; n = 1, 2) are the target values for Z in a picture, and they should be the same in theory. However, the two correct values are different due to the existence of deviations in reality. So, there is a difference between $Z_{1p}$ and $Z_{2q}$ (p = 1, 2; q = 1, 2), and Z can be calculated by

$$Z=\frac{{Z_{1p}}^{\prime }+{Z_{2q}}^{\prime }}{2}$$

(9)

where ${Z_{1p}}^{\prime } \mathrm{and} {Z_{2q }}^{\prime }$are the ${\mathrm{Z}}_{1p}$ and ${\mathrm{Z}}_{2q}$ (p = 1, 2; q = 1, 2) with the smallest difference.$Z_{\mathrm{mn}}$ is the $Z$ coordinate value obtained under different assumptions. However, the previous calculation contains right and wrong situations, and the actual situation is only the correct situation. Thus, it is impossible to determine the true value of $Z$. $Z_{1\mathrm{p}}$ and $Z_{2\mathrm{q}}$ are the cases when m = 1 and m = 2 in $Z_{\mathrm{mn}}$. $Z_{1\mathrm{p}}$ represents the two $\mathrm{Z}$ coordinates calculated according to LED1, one of which is correct and the other of which is wrong; $Z_{2\mathrm{q}}$ represents the two $Z$ coordinates calculated based on LED2, one of which is correct and the other of which is wrong similarly. The two correct results of $Z_{\mathrm{mn}}$ should be the same theoretically, but the two correct values will not be completely equal due to the existence of the deviation. Thus, we calculated the average of the two values with the smallest difference as the final Z coordinate, because the theoretical difference between the wrong value and the right value of Z is $\frac{2\ast d_1\ast \sin (t)\ast R}{{\mathrm{a}}_1}$ or $\frac{2\ast {\mathrm{d}}_2\ast \sin (t)\ast R}{{\mathrm{a}}_2}$. The difference varies with the change of t or d_k. The four Z_mn values will actually be very close when t or d_k is small or even equal to 0. Therefore, the average value can also be regarded as the value of $Z$.

2.2.2. Calculating X and Y Coordinates

The similarity relationship described above can also be expressed by the following equations:

$$\frac{a_1}{R} =\frac{\sqrt{f^2+d_1^2}}{\sqrt{{\left(X-X_1\right)}^2+{\left(Y-Y_1\right)}^2+\left(Z-Z_1\right)^2}}$$

(10)

$$\frac{a_1}{R} =\frac{\sqrt{f^2+d_2^2}}{\sqrt{{\left(X-X_2\right)}^2+{\left(Y-Y_2\right)}^2+\left(Z-Z_2\right)^2}}$$

(11)

where only X and Y are unknown. Substituting the Z coordinate obtained into (10) and (11), we can obtain two different solutions for (X, Y), which can be regarded as the intersection points of two circles, as shown in Figure 4. However, only one of the solutions is the real position of the receiver, since the same values of $t$, $d_k$ and $a_k$ may be taken in the symmetrical position of the LED lamps.

The following description is the process of selecting the position of the receiver based on the vector and angle information. The double-LED vector ${\mathrm{L}}_1$ refers to the direction vector from the center of LED2 to the center of LED1 in the world coordinate system; ${\mathrm{the} \mathrm{vector} \mathrm{of} \mathrm{receiver} \mathrm{L}}_2$refers to the direction vector from the bottom of the phone to the top in the world coordinate system, as shown in Figure 5. The angle between L1 and L2 is denoted as $\mathsf{\theta }$, which can be calculated through a captured image of the LED lightings and can be expressed as (12).

$$\left\{\begin{array}{c}arctan\left(\frac{j_2-j_1}{i_2-i_1}\right); i_2-i_1>0\\ arctan\left(\frac{j_2-j_1}{i_2-i_1}\right)+\pi ; j_2-j_1\ge 0,i_2-i_1<0\\ \theta =arctan\left(\frac{j_2-j_1}{i_2-i_1}\right)-\pi ; j_2-j_1<0, i_2-i_1<0\\ +\frac{\pi }{2}; j_2-j_1>0, i_2-i_1=0\\ -\frac{\pi }{2}; j_2-j_1<0, i_2-i_1=0\\ undefined; j_2-j_1=0,i_2-i_1=0\end{array}\right\}$$

(12)

It is required that the receiver is always placed towards the double LED, as shown in Figure 5. When the angle $\mathsf{\theta }$ is acute and the receiver is tilted to the left, or the angle $\mathsf{\theta }$ is obtuse and the receiver is tilted to the right, the solution on the right side of L1 should be chosen. Conversely, when the angle $\mathsf{\theta }$ is acute and the receiver is tilted to the right, or when the angle $\mathsf{\theta }$ is obtuse and the receiver is tilted to the left, the solution on the left side of L1 should be chosen. The target position can be determined from two solutions according to this method.

3. Experiment and Analysis

A series of experiments were conducted to test the performance of the proposed system. As shown in Figure 6, we tested the accuracy in an area of 400 cm × 200 cm × 250 cm (L × W × H). The radius of the LED was 8.75 cm. The larger the radius of the LED, the smaller the relative error caused by the blurred boundary. On–off keying intensity modulation (OOK IM) was used while driving the LED lamps. The purpose of modulation is to determine the position of the LEDs in the 3D world coordinate system based on the stripe information. More information about modulation and demodulation can be found in our previous work [29,30]. We connected the 60 V DC power supply to the circuit board so that the DC power with a flashing signal would light up the LEDs. The modulator and the receiver are shown in Figure 6b,c. We used a Huawei P20 phone as the receiver to capture double-LED lamps and then calculate the position. The specific system parameters are shown in

Table 1. Parameters in this study.

Parameter	Value
Resolution	1920 × 1080
The focal length	18 mm
FOV of camera	ϕ ≈ ${50.4}^{°}$
Power of each LED	18 W
Camera exposure time	0.05 ms

.

In this experiment to test the accuracy, the coordinates of the double LEDs were set to $\left(0,30,250\right)$ and $\left(0,-30,250\right)$; the unit is cm. We evenly took 16 points to record the times at the height of 0.1 m. In order to reduce accidental errors, we tested six times for every point, keeping the compound angle t constant every time. The 3D positioning results are shown in Figure 7. The average distance from the receiver to the double-LED lamps is recorded as D.

As shown in Figure 7a, an average error of 4.28 cm was achieved when the vertical distance was 240 cm and the tilt angle $\mathrm{t}$ was less than 3$°$. To further analyze the performance of our proposed algorithm, the cumulative distribution function (CDF) of estimated errors was generated under different conditions. More than $90\%$ of the errors are less than $5.908 \mathrm{cm}$ when$\mathrm{t}<3° \mathrm{and} \mathrm{D}\approx 250 \mathrm{cm}$, as shown in Figure 8a. Figure 8b shows the results of positioning when $\mathrm{t}\approx 30° \mathrm{and} \mathrm{D}\approx 250 \mathrm{cm}$. The large deviation is mainly caused by the assumption of the weak perspective projection, which means that the object size should be small enough with respect to the distance from the viewer to the object [31]. Thus, we conducted the third experiment under this condition. Figure 7b and Figure 8c show the results when $\mathrm{t}\approx 30° \mathrm{and} \mathrm{D}>300 \mathrm{cm}$. An average error of 6.8 cm was achieved and over $90\%$ of positioning error was within 10.22 cm. In addition, we conducted additional tests at different heights to confirm the feasibility of the algorithm. Figure 7e,h and Figure 8d,e show the high accuracy of our algorithm at the heights 0.5 and 1 m.

Two error sources were unavoidable in the process of the experiments. One is that there may be errors in the placement of the camera and the installation of LEDs. The other is that the image becomes blurred gradually as the distance increases. In the case of considering experimental errors, the average error of the algorithm is within 5.48 cm, which indicates that the error of our proposed algorithm is theoretically lower.

4. Conclusions

In this paper, we propose a tilt visible light positioning system based on double LEDs and angle sensors, which explored the vertical distance from the lens to the horizontal plane where the major axis of the ellipse is located. This system solves several problems: the requirement for horizontal placement of the receiver, the dependence on a geomagnetic sensor and the requirement of more than three LEDs in some VLP systems. In the process of experimental verification, our proposed algorithm had an excellent performance, achieving an average 3D positioning error of 5.54 cm in an area of 400 cm × 200 cm × 250 cm. It is possible to capture a clearer image by increasing the pixel size of the receiver and controlling the distance D, which will further improve the positioning accuracy. We also hope that future research will further reduce the dependence on angle sensors.