*2.1. Problem Definition of Human Detection in a Vehicle*

In this paper, Figure 2 shows the information detected via radar sensor according to the increased level of motion of passengers in a vehicle. Here, the x-axis indicates the amount of movement by a human and the y-axis represents the Doppler frequency of the echo signal.

**Figure 2.** Information detected from a radar sensor according to amount of movement by a human on a seat in the vehicle.

object using the echo generated by the human's In a vehicle, when a passenger is sleeping or a still human is sitting on a seat, Doppler radar sensors can detect breathing signals from vital sign detection area in Figure 2. Accordingly, we can easily distinguish between a human and an inanimate object using the echo generated by the human's respiration.

However, if a passenger moves with much motion on the seat, as shown in the motion detection area of Figure 2, the Doppler components issued by the body motion can mask most of the weak vital sign signals. In such cases, if an object is moving with considerable motion, we can detect the object using the Doppler echo level. However, with only the Doppler component, it is impossible to confirm whether or not a human has been detected.

Moreover, in the motion and vital sign hazard area, the detection of vital signs depends entirely on the amount of human movement. In other words, when a human is moving with relatively slight motion, motion and vital signals can appear together, whereas the vital signals can be immediately masked when a passenger is moving with a Doppler volume above a certain level.

Therefore, in both the motion detection area and the motion and vital sign hazard area, we cannot distinguish between a human and another object by the presence of vital signals or Doppler signals alone. To overcome this problem, we propose here a human recognition scheme using the characteristics of the echoed Doppler spectra of the object.

Generally, in signals reflected from a walking or running human, sidebands appear around the Doppler frequency due to the non-rigid motion. That is, various Doppler echoes can be extracted as reflected scatters of human components, such as the body, arms, and feet. Moreover, the distribution of Doppler scattering points received from a human can vary greatly during the measurement time [14,15]. In one earlier study [14], based on the FMCW radar, we proposed a concept to distinguish between pedestrians and vehicles on a road by detecting the range and velocity and analyzing the pattern of the Doppler spectrum. In addition, in another study [15], we proposed a classification algorithm for humans and vehicles that extracted feature vectors from the received FMCW radar signal and applied them to machine learning.

From these hints, in vehicle applications, although the received power is weak and there are fewer scattering points compared to the case of a walking human, the received signal has multiple reflection points echoed from the head, torso, waist, arms, pelvis, and other parts of the passenger's body. Moreover, because the human in the vehicle cannot move constantly, the Doppler spectrum will vary more over time.

's

Thus, we can find certain patterns in the micro-Doppler image of a moving passenger. In this paper, we distinguish between humans and other objects by using feature vectors extracted from this pattern, together with the vital signal extracted through additional signal processing, and applying the features to machine learning.

#### *2.2. Concept of Proposed Human Recognition* '

Figure 3a presents the proposed concept of recognizing a passenger in a vehicle based on the Doppler spectra and vital signs of the passenger. In Figure 3b, we illustrate an example of the scattering points reflected from the passenger and from a box. In this case, we cannot know which aspects of the human's various components are reflected. Moreover, we cannot know the extent to which individual parts of body contribute to the reflection.

**Figure 3.** Proposed human recognition scheme together with micro-Doppler and vital signals using machine learning for a Doppler radar sensor: (**a**) top block diagram of the proposed algorithm, (**b**) example of Doppler scattering points of a human, (**c**) example of Doppler scattering points of a non-human and a non-human object.

However, as explained above, if a human is moving, the Doppler spectrum reflected by the human is expected to spread and change over time due non-rigid motion, as shown in Figure 3b. On the other hand, echoes from a non-human object such as a moving box are expected to have a sharp type of Doppler spectrum, as shown in Figure 3c. These characteristics are the motivation behind the algorithm proposed in this paper.

The proposed human recognition scheme is divided into two parallel signal processing parts: the micro-Doppler-based motion feature extraction part and the Doppler frequency-based vital sign feature extraction part.

The I and Q signals reflected from the object are sampled through an ADC (Analog Digital Converter). In this paper, we set the ADC sampling rate to 1 kHz, which is a high enough value to digitalize the received signal form the CW radar system.

,

,

,

, 

The corresponding digitized data are inputted into two signal processing parts. In the micro-Doppler-based motion feature extraction part, we can extract two feature vectors (*x1* and *x2*) that can determine whether or not human motion appears. Details are presented below in Figures 4 and 5.

,

, passenger's ,

ℎ,

**Figure 4.** Detail algorithm steps for micro-Doppler-based motion feature extraction.

,

**Figure 5.** Concept of sliding window-based micro-Doppler generation for motion feature extraction.

In the Doppler frequency-based vital sign feature extraction part, the third vector *x3* is extracted to determine whether the vital signs exist. Figure 6 shows the detailed flow of the signal processing together with a corresponding description.

**Figure 6.** Detailed algorithm steps for Doppler frequency-based vital sign feature extraction.

ℎ, Figure 4 shows the proposed signal processing flow of the micro-Doppler-based motion feature extraction part. The process has four steps, as shown below.

First, in the pre-filtering step, from the I and Q signals, the DC and high-frequency components are removed using a DC filter and an LPF (Low Pass Filter), respectively. In the paper, the DC filter is implemented by subtracting the average value from the raw signal. We also design the LPF on the 64th order with a cut-off frequency *fh*,*<sup>m</sup>* of 30 Hz.

*Sensors* **2020**, *20*, 6202

Second, we conduct the Fourier transform by multiplying the coefficients of a Hamming window with *Nwin*,*<sup>m</sup>* to suppress the side-lobe of the frequency spectrum and the FFT (Fast Fourier Transform) with *K<sup>m</sup>* points for conversion to the frequency domain.

For a detailed explanation of the Fourier transform step, we illustrate the data processing timing flow of the original raw signal in Figure 5.

We employ the STFT (Short Time Fourier Transform) technique based on a sliding window. Because the required Doppler frequency resolution used to separate motion and vital signs is set to approximately 0.5 Hz, we select a window size *Nwin*,*<sup>m</sup>* of 2000, which means that the measurement time has a *twin*,*<sup>m</sup>* value of 2 s. Moreover, because the maximum Doppler frequency of a passenger's motion is about 10 Hz, we set the sliding step time *tstep*,*<sup>m</sup>* to 0.1 s for sliding window, indicating *Nstep*,*<sup>m</sup>* sample of 100.

Thus, in this paper, a Doppler spectrum with 2048 FFT points is generated from 2000 samples of the original signal. This procedure is repeated continuously with a sliding window in 0.1 s steps, as shown in Figure 5.

Next, in the micro-Doppler generation step, the magnitude of the Doppler frequency spectrum is calculated using the root-square function, and then the background noise is removed using the previously measured noise spectral distribution information. The generated Doppler frequency spectrum is saved into FIFO (First Input First Output) memory, and the micro-Doppler image is composed by each Doppler spectrum such as the right side of Figure 4. Here, FIFO memory can store as many Doppler spectra as the number of sub-frames *L*.

Finally, in the motion feature extraction step, as shown in Figure 4, we can obtain two feature vectors by analyzing the distribution of the Doppler scattering points over all sub-frames. This procedure is carried out in a Doppler scattering point analyzer, which will be described later and is shown in Section 2.3.

Figure 6 shows the procedure of the Doppler frequency-based vital sign feature extraction part, which is a simple technique. The process has also four steps, as shown below.

The roles of the pre-filtering step and the Fourier transform step are identical to those in the case of Figure 4. However, the LPF is on the 32th order with a cut-off frequency *fh*,*<sup>v</sup>* of 1 Hz. Moreover, the length of the Hamming window and the number of FFT points are expressed by *Nwin*,*<sup>v</sup>* and *Kv*, respectively.

We also employ STFT technique with a sliding window, as shown in Figure 7. Here, because the maximum Doppler frequency of human vital signs is less 1 Hz, the sliding step time *tstep*,*<sup>v</sup>* is set to 1 sec, with reference to a *Nstep*,*<sup>v</sup>* sample of 1000.

**Figure 7.** Concept of sliding window-based Doppler frequency extraction for vital sign feature extraction.

Moreover, for vital sign signals, because the minimum frequency is bounded in the range of 0.1 Hz to 0.5 Hz, the window size *Nwin*,*<sup>v</sup>* is set to 8000. Thus, we measure the received signal during a *twin*,*<sup>v</sup>* time of 8 s in order to confirm the presence of a breathing signal. As a result, the FFT point is set to 8192.

Next, in the Doppler frequency selection step, the absolute values are calculated as the Doppler spectrum, and rectangular windowing is applied in the frequency domain in order to consider only values below the *fh*,*<sup>v</sup>* frequency, such as the right side of Figure 6.

() = Count{(, ) > 0} for = 1~

()

{(, ), = 1~, = 1~}

first frame's

'extended degree of scattering points' <sup>1</sup>

'different degree of scattering points' <sup>2</sup>

over time, the passenger's movements may not be continuous. Thus, the distribution of

<sup>1</sup> = {∑ () =1

<sup>2</sup> = {∑ |( + 1) − ()| −1 =1

} ⁄

{

}⁄( − 1)

(), = 1~

}

2048

2

Finally, in vital sign feature extraction step, as shown in Figure 6, we can set a threshold of scattering points with a magnitude greater than the noise, and we determine vital signs based on survival values. The details pertaining to this are described later and are shown in Figure 8. <sup>3</sup> = { if > 0, then logic ′1′ else logic ′0′

3

() > reference value} for = 1~

'presence of s'

3

= Count{

**Figure 8.** Process of extracting both motion and vital sign features from the micro-Doppler and Doppler spectra with the same time interval.

the three features proposed in this paper, using only the "if–else" syntax in real time [15]. As explained in Figures 5 and 7, while the Doppler spectra for motion and vital signs are generated every 0.1 s and 1 s, respectively, according to the step time of the sliding window. That is, while one Doppler spectrum for vital sign detection is generated, ten Doppler spectra for motion detection are completed. Thus, in order to synchronize the two parts and effectively recognize human motion, we generate a micro-Doppler image using ten sub-frame spectra.
