ZEROES: Robust Derivative-Based Demodulation Method for Optical Camera Communication

Abstract

Most of Optical Camera Communication (OCC) systems benefit from the rolling shutter mechanism of Complementary Metal-Oxide Semiconductor (CMOS) cameras to record the brightness evolution of the Light-Emitting Diode (LED) through dark and bright strips within images. While this technique enhances the maximum achievable data rate, the main difficulty lies in the demodulation of the signal extracted from images, subject to blooming effect. Thus, two main approaches were proposed to deal with this issue, using adaptive thresholds whose value evolves according to amplitude changes or detecting signal variations with the first-order derivative. As the second method is more robust, a new demodulation method based on the detection of the zeros of the first-order derivative of the extracted signal was proposed in this paper. Obtained results clearly show an improvement in the extracted signal demodulation compared to other methods, achieving a raw Bit Error Rate (BER) of 10⁻³ around 50 cm in a Line-Of-Sight scenario, and increasing the maximum communication distance by 43.5%, reaching 330 cm in the case of a Non-Line-Of-Sight transmission.

1. Introduction

The development of Light Emitting Diode (LED) technology has brought a revolution in the lighting field, with greater energy efficiency than incandescent and halogen light bulbs [1]. Moreover, it also represents a tremendous opportunity for alternative and complementary wireless communication to Radio Frequency (RF) systems, which are now subject to spectral crowding [2] due to the massive use of new technologies such as Internet of Things (IoT) or cloud computing. Visible Light Communication (VLC) [3], a subset of Optical Wireless Communication (OWC) [4], takes advantages of the fast switching state of LED to transmit data at a frequency higher than the critical flicker fusion frequency of human eye in order to provide both communication and illumination at the same time. While a VLC transmission is generally based on a LED at the emission and a photodiode (PD) at the reception, Optical Camera Communication (OCC) [5] is exclusively restricted to the use of a camera as a receiver, directly benefiting from the omnipresence of smartphones for a rapid, large-scale and low-cost deployment of OWC systems in daily life as it does not require any hardware modifications, only software implementation.

Even though OCC promises great accessibility, the achievable data rate remains the major limitation of this technology, considering the low frame rate of most cameras (about 30 or 60 fps) in accordance with the Nyquist frequency. However, a clever approach was proposed to overcome this limitation by taking advantage of the architecture of Complementary Metal-Oxide Semiconductor (CMOS) cameras [6], making smartphones cheap and equipping them with ease of integration compared to Charge-Coupled Device (CCD) cameras. While the picture-taking of CCD cameras is based on the global shutter, meaning that all the pixels composing the image sensor are exposed simultaneously to the light, CMOS cameras are based on the rolling shutter, where pixels are illuminated column-wise [7]. Although rolling shutter is considered as a major drawback in photography since causing the deformation of fast-moving objects due to the column shift mechanism, the latter becomes an opportunity for OCC systems to enhance the data rate, considering the column shift frequency as sampling rate instead of frame rate. In doing so, the achievable data rate increases from a few bits per second to several thousand bits per second [8], which becomes enough to perform a lot of communication applications. Indeed, by setting the transmission frequency of the light source up to twice as low the column-shift frequency, bright and dark strips will appear on the resulting image, respectively, corresponding to ON and OFF states of the LED during image capture. This sequence of bright and dark strips is extracted from the image using specifically designed Region-Of-Interest (ROI) techniques [9,10], and the original information is thereafter recovered by demodulating the signal. However, the main difficulty of this operation comes from the blooming effect, characterized by the amplitude variation of the extracted signal due to a difference in light exposure intensity of all pixels, the most aligned to the source being the brightest. Since a demodulation based on constant-value thresholding is therefore no longer adapted, adaptive threshold methods [11] have been developed in order to correctly demodulate the extracted signals, but their performance still remains limited on large and rapid amplitude variations signals. Moreover, other approaches based on the first-order derivative of the extracted signal have been proposed in order to directly identify the state changes by determining both local maximum and minimum values, named peaks and valleys, respectively [12,13], or even the second-order derivative [14]. Even though developed methods based on derivatives have improved the demodulation efficiency, noisy perturbations and low amplitude variations between states remain the main limitations of these techniques since derivative greatly depends on the amplitude variations of the signal. Therefore, the main contribution of this article is to propose a new demodulation method, the Zeros Evaluation for Robust and Optimal Edges Selection (ZEROES) method, based on the detection of zeros of the derivative for determining the precise signal edge position using linear functions, achieving better communication performance due to its greater level of robustness in comparison with the demodulation methods previously presented. With experiments conducted in both Line-Of-Sight (LOS) and Non-Line-Of-Sight (NLOS) scenarios, the comparison of demodulation efficiency through the measurement of the Bit Error Rate (BER) and the percentage of bits received clearly show a significant improvement of the communication performance when using the ZEROES method.

The rest of this paper is organized as follows: Section 2 presents the different processing steps of OCC images. Section 3 describes the ZEROES developed method. Section 4 details the experimental setup employed to perform experiments. Section 5 is dedicated to the obtained results with their discussions. Finally, a conclusion summarizes this work, and a discussion about our future work is presented.

2. Optical Camera Communication

While VLC receivers based on photodiodes convert the incident light into a photocurrent, a camera returns a color image of the light source and its surrounding environment. This fundamental difference in the output signal of an image sensor therefore requires more complex operations at the receiver to process the signal. Despite the increased difficulty, a camera offers several advantages over a photodiode, such as a larger field of view, a spatial separation of light providing a clear distinction between different light sources, or a wavelength separation of the incident light into RGB component through the Bayer filter. With all the benefits brought by OCC, a wide range of applications have been identified in a wide variety of areas, such as indoor localization [15], Vehicle-to-Vehicle (V2V) communication [16], or IoT applications [17]. All the different steps required in the processing of the OCC images based on rolling shutter, from the image capture to the original information recovering, are illustrated in Figure 1.

2.1. Rolling Shutter

In order to achieve a data rate suitable for real-life applications, OCC is taking advantage of the rolling shutter mechanism of CMOS cameras to increase the number of received bits per image through bright and dark strips. However, to ensure a good contrast between the strips, and thus a good signal reception, not only must the light source frequency be set down to half the shutter speed in accordance with Nyquist criterion, but the camera must also be correctly configured. Indeed, both exposure, representing the light exposition time of each pixel, and the International Standard Organization (ISO) value, corresponding to the image sensor sensitivity, need a proper parametrization depending on the environment. With these conditions met, pixel columns will generate a bright strip when illuminated by the light source or a dark strip when the LED is off. Hence, the higher the light source frequency, the narrower the bright and dark strips will be, resulting in a higher data rate as long as the strips remain discernible from one another [18]. The values of these parameters are experimentally determined in Section 4.

2.2. Image Conversion

With the information transmitted through the brightness variation of a white LED, images contain a redundancy of the information through the three color channels. Usually, RGB images are firstly converted in another representation to reduce the image dimensionality, such as grayscale [6] or value [19] (from hue saturation value representation), therefore reducing the whole processing time. Hence, in the grayscale representation, each pixel is a linear combination of the three RGB pixel values, whereas only the maximum pixel value is retained in the value representation. Thus, the value representation provides a better contrast between bright and dark strips on colorful object while adding a reasonable amount of noise, whereas the grayscale representation is more resilient against chromatic aberration [20]. In fact, this phenomenon occurs when the lens fails to focus the color light in a same pixel due to a refractive index changing with wavelength, which is even more pronounced when the contrast between the strips is high. Hence, as illustrated in Figure 2a, chromatic aberration makes the dark strips boundaries colorful, which are therefore considered as bright strips in value representation (Figure 2c), while the impact of the color boundaries on the strips detection is less important in grayscale (Figure 2b). Moreover, as images are captured with rolling shutter, a pixel column therefore contains a redundancy of the transmitted information. Thus, processing all the pixels of a column is not required to correctly recover the transmitted information. The same remains true to a lesser extend for the pixel rows when the transmitted signal is oversampled, resulting in wider bright and dark strips in the image.

Therefore, the image can be downsampled in height (keeping 1 pixel row over ${\mathrm{\Delta }}_{row}$, with ${\mathrm{\Delta }}_{row}$ being the decimation factor along the pixel rows) and width (keeping 1 pixel column over ${\mathrm{\Delta }}_{col}$, with ${\mathrm{\Delta }}_{col}$ being the decimation factor along the pixel columns) in order to reduce the processing time aiming for a real-time execution [21].

2.3. Region of Interest

Once the image converted and downsampled, the transmitted signal encoded through the dark and bright strips’ pattern must be extracted from images. The operation represents a crucial step of the processing since a poor signal detection will inevitably lead to a poor signal demodulation, resulting in transmission errors. While this operation remains quite simple on images only composed of dark and bright strips with a high contrast, this becomes much more complex when the distance between the light source and the camera increases or when the surrounding environment interferes with the communication. Thus, different solutions have been proposed to address this issue, being derived from computer vision techniques to detect the light source position in the image using a Gaussian blur filter and a binary threshold [9], or using statistical methods, such as Principal Component Analysis (PCA). PCA is a dimension reduction method that defines a sequence of unit vectors, named principal components, where each vector represents the line that best fits the data by minimizing the average squared perpendicular distance from the points to the line while being orthogonal to the previous vector. By applying PCA directly to images, considered as data arrays, the transmitted signal can be extracted thanks to the high correlation between pixel rows, since the signal is recorded in the bright and dark strips’ pattern, vertically aligned along the image [19]. The latter is the ROI method selected to perform the conducted experiments due to its performance and robustness.

2.4. Demodulation

The main difficulty of the demodulation lies in the blooming effect affecting the extracted signal amplitude. As the demodulation intends to differentiate between the signal logic states to recover the binary signal, a sudden and significant change in the extinction ratio (ER) between the bright and dark strips therefore compromises the transmitted data integrity, considering a constant-value threshold demodulation. Hence, two different approaches were considered to overcome the blooming effect issue. The first consists of using adaptive thresholds whose values follow the signal amplitude variations, third-order polynomial curve fitting (3rd-OPF) [11]. The second, more robust against the blooming effect, is based on the first-order derivative calculation to identify the rising and falling edges of the extracted signal. Indeed, different methods have been proposed, such as the Boundary Pixel-Based Piecewise Linear Function (BPBPLFT) scheme [12], which uses combined linear functions as thresholds based on the position of signal edges, the HyperSight (HS) method that directly performs the demodulation from the normalized first-order derivative without properly using a threshold [13] or by directly identifying the strips transitions based on the derivative slopes [14]. Hence, this paper introduces a new demodulation method based on a first-order derivative approach, described in Section 3, whose performance is compared to other proposed methods in the literature.

2.5. Binarization

Finally, the demodulated signal is converted into a binary signal using the $P_b$ factor, representing the number of pixels contained per bit, given by [19]

$${\displaystyle P_b=\frac{{\mathrm{\Sigma }}_{width}\times fps}{{\mathrm{\Delta }}_{col}\times bps}}$$

(1)

where $fps$ is the camera frame rate expressed in frames per second, $bps$ is the signal bit rate in bits per second, ${\mathrm{\Delta }}_{col}$ is the column decimation factor, and ${\mathrm{\Sigma }}_{width}$ is the total pixel width of a frame, taking into account the Inter-Frame Gap (IFG) [22], representing the gap time between consecutive frames when the camera cannot receive the incident light, as illustrated in Figure 3:

With the decimation factor ${\mathrm{\Delta }}_{col}$ reducing the amount of pixels in the extracted signal to process, the latter must be carefully selected according to the oversampling factor of the camera in order to ensure the correct recovery of the information. There is therefore a trade-off to be made between the required processing time of the signal and the communication performance, especially with a poor contrast between the bright and dark strips due to the presence of noise.

3. Method Description

While several demodulation methods have been implemented to determine the width of bright and dark strips, dealing with the blooming effect through adaptive thresholds or first-order derivative, none of the proposed methods, to the best of authors’ knowledge, remain robust enough when the environment is becoming more complex. Hence, a new demodulation method is proposed with the aim of achieving greater robustness within complex environments in order to enhance OCC systems’ performance, therefore extending the range of possible applications. As illustrated in Figure 4, the proposed ZEROES method is based on a first-order derivative approach for the demodulation of the extracted signal (x). Indeed, the latter is firstly filtered and normalized ($x_n$) in order to prepare and enhance the derivative computation. Then, from the peaks and valleys ($PV$) of the derivative ($x_n^{\prime }$), representing its local maximums and minimums, associated zeros are deduced ($zeros$), where the slope of the extracted signal is constant, leading to the identification of the extracted signal boundaries ($bnd$). Finally, the switching point ($switch$) of the logical state is determined from the boundary positions by computing the intersection point between the signal and the constant straight line whose value is the average of the boundary points. Thus, the accurate detection of the signal boundaries through the zeros’ values of the derivative, combined with the optimal identification of the logical state switching points, therefore provides great robustness in demodulation with ZEROES.

3.1. Moving Average and Normalization

Since the ZEROES method intends to identify the zeros of the first-order derivative based on the positions of peaks and valleys, noisy variations of the extracted signal x result in short-term fluctuations that therefore interfere with a proper zeros’ detection (Figure 5). The extracted signal x shown in Figure 5 was obtained after the decimation of columns (${\mathrm{\Delta }}_{col}=4$) and rows (${\mathrm{\Delta }}_{row}=100$) of a 3040 × 4056 (N × M) resolution image in order to reduce the whole amount of data to process while reducing the noise. Then, a moving average was applied to smooth out the signal noise by computing an average value for each point based on the neighboring points with a sliding window $W=\lfloor P_b\rceil$, with $\lfloor .\rceil$ being the round operator.

$${\displaystyle \overline{x}\left(i\right)=\frac{1}{2W+1}\sum _{j=-W}^{W}x(i+j)}$$

(2)

Thus, the moving average computed $\forall i\in [\lfloor W/2\rfloor ,$$\lfloor N/{\mathrm{\Delta }}_{col}\rfloor -\lfloor W/2\rfloor ]$, with $\lfloor .\rfloor$ being the floor operator, acts as a low-pass filter by attenuating the noisy high frequencies of the signal while preserving the low frequencies containing the transmitted information. Then, the moving averaged signal $\overline{x}$ is normalized in $x_n$ between 0 and 100 to ensure a proper upcoming zeros’ detection of the first-order derivative regardless of the transmission conditions.

$${\displaystyle x_n\left(i\right)=\frac{\overline{x}\left(i\right)-{\overline{x}}_{min}}{{\overline{x}}_{max}-{\overline{x}}_{min}}\times 100}$$

(3)

with ${\overline{x}}_{max}$ and ${\overline{x}}_{min}$ being the maximum and minimum values of $\overline{x}$, respectively.

As shown in Figure 6, the extracted signal after moving average filtering presents less noisy fluctuations while retaining sharp edges for the good of the demodulation. However, signal improvement is achieved only at the cost of a reduced extinction ratio, which affects the correct demodulation.

3.2. First-Order Differential Calculation

Once the extracted signal is correctly filtered by the moving average and shaped by normalization, the first-order derivative ${x_n}^{\prime }$ is computed while setting all the points with an absolute derivative value lower than the threshold $T_h$ to zero (Figure 7). In fact, removing the low-amplitude variations of the derivative provides an improvement in the identification of the logical state changes, lowering the probability of a false zeros detection. Since the derivative is computed on a normalized signal $x_n$, the value of $T_h$, defining the minimal amplitude of a lobe, remains the same from one image to another and is fixed to 1, assuming that the low noisy fluctuations derivative of the normalized signal would not exceed 1. However, the derivative thresholding results in the emergence of isolated points in the signal, characterized by a derivative sign different from its neighbor points. When a sudden change in the derivative sign occurs on a single point, the corrupted values of ${x_n}^{\prime }\left(i\right)$ become equal to the average of its neighboring values ${x_n}^{\prime }(i-1)$ and ${x_n}^{\prime }(i+1)$, as follows:

$${\displaystyle {x_n}^{\prime }\left(i\right)=\left\{\begin{array}{cc}\hfill & \mathrm{if}sgn\left(x_n^{\prime }\left(i\right)\right)\ne sgn\left(x_n^{\prime }(i-1)\right)\hfill \\ \frac{{x_n}^{\prime }(i-1)+{x_n}^{\prime }(i+1)}{2},\hfill & \mathrm{and}sgn\left(x_n^{\prime }\left(i\right)\right)\ne sgn\left(x_n^{\prime }(i+1)\right)\hfill \\ \hfill & \forall i\in [1,i_{max}-1]\hfill \\ {x_n}^{\prime }\left(i\right),\hfill & \mathrm{otherwise}\hfill \end{array}\right.}$$

(4)

where $sgn\left(\right)$ is the sign function. Then, the thresholding operation described above is once again repeated in order to eliminate the resulting values with an absolute value below $T_h$.

As illustrated in Figure 7, the derivative filtering mechanism described above combined with the moving average ensures a nearly free-noise derivative by setting all the values between the red dotted lines to zero and smoothing the outlier values.

3.3. Detection of Peaks and Valleys

Due to the square-shaped waveform of the emitted signal, the first-order derivative $x_n^{\prime }$ is supposed to present alternating positive and negative lobes, respectively, indicating the rising and falling edges of the signal. Thus, a consecutive series of positive derivative values represents a positive lobe whose maximum value is a peak, while a consecutive series of negative derivative values represents a negative lobe whose minimum value is a valley (Figure 8). However, when the extracted signal x does not present sufficiently sharp edges due to a low contrast between strips, noise, or distortion, the detected peaks and valleys $PV$ are not all necessarily valid, leading to demodulation errors if not taken into account. This is indeed the case for the peak surrounded by the blue circle in Figure 8.

Fortunately, these wrong peaks and valleys can be detected through two different criteria:

–

The amplitude difference between a successive peak/valley.

–

The alternating peak/valley pattern.

Thus, the validity of each peak and valley is checked by considering the $PV$ value with the greatest amplitude $max\_PV$ as being valid. Then, from $max\_PV$, $prev\_PV$ and $next\_PV$ are determined, which are defined as the first peak/valley before and after $max\_PV$, respectively, with an absolute amplitude greater than or equal half $|PV\_max|$ (Figure 9). Hence, the validity of each peak/valley is firstly checked with $PV$ before $prev\_PV$, followed by those after $next\_PV$ (Figure 9). A peak/valley is only considered as valid if

$${\displaystyle PV\left(i\right)\ge \frac{\left|PV\right(i+1)+PV(i+2\left)\right|}{2+T_h}}$$

(5)

$\forall i\in [prev\_PV-1,2]$ for $prev\_PV$ validation; otherwise, $\forall i\in [next\_PV+1,N_{PV}-2]$ for $next\_PV$, while checking whether $N_{PV}$ is the number of $PV$ that was previously detected. If a peak or valley does not satisfy this condition, it is not considered as a peak or valley anymore in order to prevent any false detection; otherwise, it is recorded in $PV\_left$ or $PV\_right$ (Figure 9). Now that all the peaks and valleys are valid in terms of amplitude, the alternating peak/valley pattern must henceforth be checked. Indeed, in order to prevent these errors from happening, if two consecutive peaks or valleys are detected, only the portion of the derivative containing the most valid peak and valley is retained for further processing, as illustrated with the red area in Figure 9.

As clearly seen in Figure 9, only peaks and valleys corresponding to the actual rising/falling edges of the extracted signal are taken into account (red crosses), with the peaks and valleys with too low of an amplitude being left out. Moreover, in order to prevent transmission errors, the largest series including the alternating peak and valley pattern is retained, deliberately ignoring parts of the derivative that would cause errors (red zone).

3.4. Zeros’ Detection

Since peaks and valleys only provide imprecise information about the signal edges, by returning the position of the maximum or minimum variation between consecutive points, using them as an edge indicator therefore introduces an uncertainty about the total pixel length of states, which leads to demodulation errors. In order to achieve greater precision in the detection of edge position, zeros of a derivative, indicating the start and end of each state, are used instead. Indeed, two zeros’ values (before and after) are assigned to each peak/valley, corresponding to the extreme points of the associated lobe. Thus, the previous zero is defined as the first value before the peak/valley with a sign different from the peak/valley, while the next zero is defined as the first value after the peak/valley with a sign different from the peak/valley (blue squares in Figure 10).

However, when a state amplitude is not constant due to the blooming effect, especially in phases of increasing or decreasing light intensity, the associated slope is therefore not null. If the absolute slope value is ever above the threshold $T_h$, it will not be filtered when computing the derivative (Section 3.2) and be considered as part of the lobe, distorting the zeros’ detection. In order to take this situation into account, a derivative point within a lobe is only considered as a zero if

$${\displaystyle \left(|x_n^{\prime }\left(i\right)|\le \frac{\left|PV\right|}{2}and\left|x_n^{\prime }\left(i\right)\right|\le T_h\right)or\left(sgn\left(PV\right)\ne sgn\left(x_n^{\prime }\right)\right)}$$

(6)

with $x_n^{\prime }\left(i\right)$ being the derivative point of the associated $PV$. Thus, Figure 10 showsthe pair of zeros detected (blue squares) for each valid peak and valley. While all detected zeros correspond here to the sign change point with a null amplitude, it should be noted that this is not always the case, especially in complex environments, due to the detection condition of Equation (6).

3.5. Boundaries’ Detection

The position of the signal boundaries of $x_n$, defined as the limit points of a logic state, is directly obtained from the indices of the zeros detected. With the aim of accurately detecting the edges of $x_n$, the position of each boundary is adjusted according to the definition of signal rise time, characterizing the required time for the signal to switch from one state to another. Indeed, considering the absolute amplitude difference between boundaries from the same edge as $bnd\_diff$, the boundary of the low state is defined as the first value greater than or equal to the detected low boundary + 10% of $bnd\_diff$, while the boundary of the high state is defined as the first value less than or equal to the high boundary − 10% of $bnd\_diff$.

As illustrated with green diamonds in Figure 11, zeros of the derivative in association with the rise/fall time detected in the signal provide an accurate detection of the signal state boundaries, especially when a state’s amplitude is fluctuating.

3.6. Edges’ Detection

Achieving a great accuracy in the detection of the edges of the signal $x_n$, corresponding to the switching instants of logical levels, provides a better robustness in the demodulation. Thus, as an edge represents the tipping point between an high and low level, or vice versa, its position is defined as the equidistant point along the signal $x_n$ between the associated boundaries (Figure 12). To do so, the average value of boundaries from a same edge is considered as the ordinate of the corresponding switch point, while its abscissa is determined by computing the intersection point between the rising/falling edge and the constant straight line of the $switch$ value (Figure 12).

Thus, the slope a of the straight line passing through $P_h$ and $P_l$, the first points of each edge higher than and lower than the associated $switch$ value, respectively, and the corresponding y-intercept b are given by

$${\displaystyle a\left(j\right)=\frac{y_{P_h}-y_{P_l}}{x_{P_h}-x_{P_l}}}$$

(7)

$${\displaystyle b\left(j\right)=y_{P_h}\left(j\right)-a\left(j\right)\times x_{P_h}\left(j\right)}$$

(8)

Then, from the equation of a first-order function $y=ax+b$, the abscissa of the switch $switch\_idx$ is given by

$${\displaystyle switch\_idx\left(j\right)=\frac{switch\left(j\right)-b\left(j\right)}{a\left(j\right)}}$$

(9)

As $switch\_idx$ is determined by the intersection of two straight lines, the position of the switch is therefore more accurate than just selecting the point of the edge closer to the $switch$ value. As clearly shown in Figure 13, the switch points represented by red triangles oriented according to the rising or falling edges are indeed located halfway between the boundary points of the edges illustrated with green diamonds.

3.7. Demodulation

Finally, the precise detection of state switching points provides a demodulated signal with great accuracy, as shown in Figure 14, despite noise and the blooming effect distorting the extracted signal x.

Binary information is easily recovered from a demodulated signal using the $P_b$ value (introduced in Section 2), which is used to determine the number of bits contained in each state based on the number of pixels.

4. Experimental Setup

This section provides a comprehensive description of the experimental setup implemented to perform OCC transmission. In this study, both Line-Of-Sight (Figure 15a) and Non-Line-Of-Sight (Figure 15b) scenarios are considered, with a direct and indirect light transmission from the transmitter to the receiver, respectively. While the camera only receives light directly emitted by the LED source in LOS transmission, NLOS transmission takes advantage of the light reflections in the surrounding environment to increase the light source area within an image [23].

4.1. Transmission

In an attempt to reach the conditions of a real-world application, the implemented transmitter (TX) is build around the CILAOS LED panel from Holight [24]. The latter is specifically designed for indoor lightning and consists of 4 modules of 20 Surface-Mounted Device (SMD) LEDs connected in parallel to emit white light while a 30 × 30 cm polycarbonate sheet ensures uniform light diffusion. In the case of LOS transmission, the center of the LED panel is positioned at a height of 120 cm, facing the camera; in the NLOS scenario, the light source is situated 130 cm above the ground, facing a white wall with an orientation of 30° relative to the wall’s normal and at a distance of 110 cm from it. The OOK modulation of the emitted light is managed by the bulk controller for LED LM3409 from Texas Instruments [25], offering both analog and Pulse-Width Modulation (PWM) dimming to control the current powering the LED.

In order to facilitate the distinction between high and low logical levels emitted by the source, and therefore enhancing the bright and dark strips in the image, PWM dimming (consisting of successively turning on and off the LED panel) was implemented here to drive the LED. The STM32H7 microcontroller controls the LM3409 switching state by generating the message that needs to be transmitted at a data rate of 4 kbps, consisting of a 10-bit header and a 40-bit payload. The different parameters of the transmitter are summarized in

Table 1. Summary of TX parameters.

TX Parameters	LOS	NLOS
LED power	19.91 W	19.91 W
Panel dimension	30 × 30 cm	30 × 30 cm
Panel height	120 cm	130 cm
Panel/wall distance	/	110 cm
Panel/wall orientation	/	30°
Header	10 bits	10 bits
Payload	40 bits	40 bits
Data rate	4 kbps	4 kbps
Modulation	OOK	OOK

.

4.2. Reception

The reception of LED brightness variations is operated by the Sony IMX477R 12MP image sensor of the Raspberry Pi HQ camera [26], equipped with a 6 mm wide-angle lens to cover a large view of the transmitter and its nearby environment. The camera is positioned at a height of 120 cm in each scenario, and it is positioned at a lateral distance of 105 cm from the LED panel in the case of NLOS transmission. Image capture is performed by a Raspberry Pi 4 through a Python program configuring the camera’s parameters based on the Picamera2 library. Image resolution is set to the maximum of 4056 × 3040, the frame rate is fixed to 40 fps, and the minimum exposure time of 1/16667 s is selected in order to achieve better contrast between the bright and dark strips. Since the ISO value mainly depends on ambient light, its parametrization must therefore be carried out in each environment; here, it is set at 5769. Considering that the image sensor is oriented in landscape mode, 90° clockwise rotation is applied on images to recover the usual vertical strips. Through the $P_b$ parameter, which is equal to 8.25 here, the bit period of the transmitted signal is known by the receiver. In order to minimize the whole processing time, images are decimated in both columns and rows (${\mathrm{\Delta }}_{col}$ = 4, ${\mathrm{\Delta }}_{row}$ = 100) to reduce their resolution (from 3040 × 4056 to 41 × 760). The different parameters of the receiver (RX) are summarized in

Table 2. Summary of RX parameters.

RX Parameters	LOS	NLOS
Camera	Raspberry Pi HQ	Raspberry Pi HQ
Camera height	120 cm	120 cm
Camera/LED lateral dist.	/	105 cm
Resolution	4056 × 3040	4056 × 3040
Focal length	6 mm	6mm
Frame rate	40 fps	40 fps
Exposure time	1/16667 s	1/16667s
$T_{shutter}$	23.3 ms	23.3 ms
ISO	5769	5769
Average ambient light	1173 lx	3724 lx
$P_b$	8.25	8.25
${\mathrm{\Delta }}_{col}$, ${\mathrm{\Delta }}_{row}$	4, 100	4, 100

.

5. Results

In order to evaluate the robustness and efficiency of the ZEROES method, experiments are conducted in both LOS and NLOS scenarios to measure its performance in terms of communication and execution times. These results are also compared to other demodulation methods—also based on the first-order derivative like BPBPLFT [12] and HS [13] or on an adaptive threshold with 3rd-OPF [11]—using the exact same ROI method (here, PCA) [19] to only see the effect of demodulation on performance. To achieve this, 300 images are captured for each camera position with a distance to the LED source ranging from 40 to 100 cm with a 10 cm step in LOS, while the distance from the camera to the wall (without considering LED/wall distance) increases from 50 to 450 cm with a 25 cm step in NLOS. All taken images are transferred to the computer in order to be post-processed by a Python program (available at [27]). As images taken in LOS are subject to chromatic aberration, images are converted in grayscale, while value representation is used in NLOS to enhance the strips’ contrast. While the efficiency is evaluated through the percentage of received bits, the robustness is measured with the Bit Error Rate, defined as the number of errors divided by the number of received bits. To ensure the statistical reliability of the BER measurement, only the BER values computed with a minimum of 10 errors are considered in the results. Furthermore, in order to provide a clearer visualization of the BER and percentage of received bits as a function of the distance, an extrapolation curve is plotted for each demodulation method (Figure 16, Figure 17, Figure 18 and Figure 19).

5.1. Line of Sight

Due to the number of bits missed during the IFG (indicated by the red area in Figure 16) and the ones lost during the transmission itself, the camera can never receive 100% of the transmitted data. In fact, while the percentage of received bits of the demodulation methods remains relatively stable, evolving between 87 and 90%, ZEROES tends to lose more data as the distance increases from 60 cm, receiving only 82.2% of the transmitted data at 100 cm. This result difference is explained by the fact that ZEROES deliberately does not consider the bits identified as incorrect in order to minimize transmission errors.

Figure 16. Received bits of demodulation methods in LOS.

Figure 16. Received bits of demodulation methods in LOS.

Indeed, while the BER of the demodulation methods tends to increase as a function of distance, except for BPBPLFT, the ZEROES method outperforms the others in the received errors under a distance of 75 cm, at a point where BPBPLFT reaches same performance. In order to better compare the methods’ performance, a maximum BER of 10⁻³ (indicated by the red dashed line in Figure 17) serves as reference since it ensures error-free transmission when employing traditional error-correcting codes. Thus, of all the implemented methods, ZEROES is the only one reaching a BER of 10⁻³ at a distance of about 50 cm from the LED source (Figure 17).

Figure 17. Bit Error Rate of demodulation methods in LOS.

Figure 17. Bit Error Rate of demodulation methods in LOS.

However, in this case, only 87.3% of the transmitted data are received. In comparison with the BPBPLFT method at the same distance, 89.7% of the data are received but with a BER of 9.9 × 10⁻³. Therefore, the additional loss of 7.5% in the received bits introduced by ZEROES is largely compensated by receiving approximately 10 times less errors in the data.

5.2. Non Line of Sight

Similarly to the results obtained in LOS, the same variations in the percentage of received bits as a function of the distance are also observed in NLOS (Figure 18). Indeed, while other methods receive a stable amount of bits when increasing the communication distance between 87 and 90%, ZEROES is rather stable until 300 cm receiving approximately 86% of the data before losing more bits with the distance, reaching 80.5% at 450 cm.

Figure 18. Received bits of demodulation methods in NLOS.

Figure 18. Received bits of demodulation methods in NLOS.

However, in comparison with LOS results, 3rd-OPF and HS methods still present similar performance but without being able to reach a BER less than 10⁻³ over the entire measurement range (Figure 19). However, BPBPLFT reaches this value at a distance around 230 cm, while the BER of ZEROES is less than 10⁻³ until 330 cm, increasing the maximum achievable distance by 43.5%.

Hence, taking BPBPLFT as reference at this distance (330 cm), the loss of 3% in the received data with the use of ZEROES is once again largely compensated by a number of errors in the received data being reduced by a factor of about 7. Thus, ZEROES improves the communication distance of the transmission but at the cost of a reduced number of received bits. There is therefore a trade-off to deal with between the effective data rate and the number of transmission errors for the ZEROES method.

Figure 19. Bit Error Rate of demodulation methods in NLOS.

Figure 19. Bit Error Rate of demodulation methods in NLOS.

5.3. Execution Time

Regarding the execution time of each method, the runtime of a hundred executions is measured, but only the median time is considered in order to avoid the effect of a non-real-time execution. The experiment was conducted on a HP-ZBook-17-G2 equipped with a 2.8 GHz Intel Core i7 4810MQ processor, 32 GB RAM, running the Ubuntu 20.04 64-bit version. The results displayed in Figure 20 show that PCA combined with ZEROES is the most time-consuming method, taking 4.7 ms to process one image, which is approximately 5 times lower than the time between each image capture when considering a frame rate of 40 fps (1/40 = 25 ms). Therefore, the ZEROES method remains fully compliant with a real-time OCC application, considering both the gain in communication performance and the short execution time of the proposed method.

6. Discussion

The main limitation of a correct demodulation of OCC images based on rolling shutter lies in the blooming effect. The aim of this work was to implement a new demodulation method, named ZEROES, which relies on the first-order derivative of the extracted signal to accurately detect changes in signal states while removing the blooming effect. The conducted experiments demonstrated the robustness of the ZEROES method by comparing its performance to other demodulation methods while using the same PCA method for ROI detection. The obtained results show that the ZEROES method outperforms other methods in both LOS and NLOS scenarios by achieving a target BER of 10⁻³ in LOS at a distance of approximately 50 cm (when other methods cannot), and by increasing the communication distance in NLOS by 43.5% compared to BPBPLFT. However, the robustness of the ZEROES method against transmission errors results in a reduced effective data rate, receiving 85.5% of the transmitted data with a BER of 10⁻³ at 330 cm, while BPBPLFT achieves 89.9% at a shorter distance of 230 cm. Therefore, the combination of PCA and ROI with the ZEROES demodulation method paves the way for the use of OCC in daily life, especially in NLOS scenarios, with execution times being compatible with real-time applications.