*2.4. RGB Sensor Processing for HR Estimation Using Tapered Window, Signal Reconstruction based on Softsig and MUSIC Algorithm*

The fundamental method of HR estimation using an RGB camera has been described previously [15]. The RGB camera senses tiny color fluctuations in the facial skin with other noise. To remove the noise components, methods such as independent component analysis (ICA) and soft signature-based extraction (Softsig) [24] are used. In this study, we introduce the tapered window and signal reconstruction method into HR estimation for a stable measurement, which achieved an infection screening system. The observed RGB time-series data have components of heartbeat, motion artifact and noise from other light sources. The tapered window and signal reconstruction method is based on the Softsig demix heartbeat signal. Figure 4 shows an overview of HR estimation in this system.

**Figure 4.** Block diagram of signal processing for HR estimation. (**a**) RGB video with ROI detected by OpenCV. (**b**) RGB ROI image applied to tapered window. (**c**) Raw RGB time-series data and reconstruction vector *<sup>V</sup>* = *vr*, *vg*, *vb* determined by kurtosis of spectra. (**d**) Reconstructed signal using *V*. (**e**) Power spectra obtained by MUSIC.

Tapered window, which is a general window function, was applied to the detected facial ROI (Figure 4b). In facial ROI, the edge area suffers from the lag affected by the face tracker. On the other hand, the ROI center can achieve a stable tracking of the facial skin. Therefore, we adopted tapered window to weighted ROI to reduce the noise raised by facial tracking. A 1d-tapered window is represented as

$$
tau\_{1d}(i) = \begin{cases}
0.5\mathbf{x}(i) \left(1 - \cos\left(\frac{2\pi i}{2m}\right)\right) & (i = 0, 1, 2, \dots, m - 1) \\
0.5\mathbf{x}(i) \left(1 - \cos\left(\frac{2\pi (n - i - 1)}{2m}\right)\right) & (i = n - m, \dots, n) \\
\mathbf{x}(i) & (otherwise),
\end{cases}
\tag{2}
$$

where *m* indicates the tapered portion and has a value of 0.05 · *n*. To apply the tapered window to a 2d-image, the 2d-tapered window is expressed as

$$
tau\_{2d}(\mathbf{x}, \ y) = \mathbf{t} \mathbf{p} \mathbf{e} r\_{1d}(\mathbf{x}) \cdot \mathbf{t} \mathbf{p} \mathbf{e} r\_{1d}(y),\tag{3}$$

where *x* and *y* are the x-coordinates and y-coordinates of ROI, respectively.

The aim of signal reconstruction is to find a reconstruction vector *<sup>V</sup>* = *vr*, *vg*, *vb* for extracting the heartbeat signal by utilizing the difference among RGB absorption. Reconstructing a BVP signal using three RGB channels to optimize a linear function for improving the signal-to-noise ratio. According to a previous study, the reflection strength of the heartbeat is referred to as the relation in G>B>R order among the RGB channels. Using this relation, signal reconstruction can be expressed as

$$\mathbf{y}(t) = \boldsymbol{\upsilon}\_{\mathcal{T}} \mathbf{x}\_{\mathcal{T}}(t) + \boldsymbol{\upsilon}\_{\mathcal{S}} \mathbf{x}\_{\mathcal{S}}(t) + \boldsymbol{\upsilon}\_{\mathcal{b}} \mathbf{x}\_{\mathcal{b}}(t), \tag{4}$$

where *vr*, *vg*, and *vb* are the reconstruction vector. While this method is based on the Softsig method, we improved the determined method for vector V. To recover the pulse signal, we selected V to maximize the kurtosis of the spectra in the HR range of [0.75–4.0 Hz] (Figure 4c).

Finally, the MUSIC method was introduced to realize HR and RR measurements within a short time period. This method permits the realization of high-resolution HR and RR frequency estimation based on short-period measurement data Equation (5) expresses the spectrum estimation formula of the MUSIC method [14]:

$$\mathcal{S}\_{\text{MLISIC}}(f) = \frac{1}{\sum\_{k=M+1}^{p} \left| \varepsilon^{T}(f)\mathcal{W}\_{k} \right|^{2}} \times \frac{1}{\delta f'},\tag{5}$$

where *e*(*fi*) represents a complex sinusoidal wave vector and *Wk* represents the eigenvector of the correlation matrix. This system applies the MUSIC method separately to the HR and RR time-series data obtained from the video. In the case of heartbeat, the peak of 0.75–3.0 Hz (45–180 beats per minute (bpm)) of the obtained spectrum was assumed to be the HR.
