**1. Introduction**

The fact that children are dying in hot vehicles has recently become a major social issue. Thus, the European NCAP (New Car Evaluation Program) has recommended the installation of CPD (Child Presence Detection) technology on all new cars starting in 2020 [1]. Moreover, many countries' safety regulators have also considered rules that could mandate CPD systems aimed to detect a child left in a vehicle. To support such a system, various sensors capable of detecting objects in vehicles or monitoring vehicle body status are required [2,3].

Another application of passenger detection is in electric vehicles. In electric vehicles, the heating and air conditioning functions depend on the efficiency of the battery [4]. If heating and cooling systems in vehicles can be automatically controlled for each seat, battery consumption can be decreased. Thus, to support these functions, technology to detect passengers in each seat is required.

One additional application for occupant recognition is in self-driving vehicles. The driving operation for an autonomous vehicle strongly depends on the presence or absence of passengers. That is, when occupants are riding, the comfort and reliability of passengers become very important issues [5]. Moreover, depending on whether the occupant is sleeping or moving, the self-driving style can differ. Thus, it is very important to assess the occupancy and status of passengers in every seat.

For the various applications described above, the performances capabilities of sensors to detect passenger are very important. The characteristics of these sensors are described below.

Among various methods, one very simple approach is to use a pressure sensor. However, when any object is placed on a seat, it is impossible to determine whether or not the object is a human. Another solution is to measure and use the distance from the object by means of an ultrasonic sensor. However, this method also cannot distinguish between different types of objects.

Recently, thermal infrared sensors have been attracting attention given their ability to check for the presence of passengers using human temperature. However, this method is highly sensitive to a person's clothing or to the external temperature condition.

The use of a camera is also a very effective solution for these applications. Specifically, because stereo cameras and depth cameras can measure the distance to an object, they can recognize various motions of a human when applying deep learning with image features. However, camera sensors are limited due to the external lighting conditions. Another disadvantage is that the amount of computation for image processing is excessive. Moreover, the installation of a camera inside their vehicles may cause consumers to reject these vehicles due to privacy issues.

Recently, a radar sensor-based occupant detection system has attracted attention, as radar is robust to external conditions [6]. Moreover, radar can distinguish between a moving object and a stationary object, and these systems can also monitor vital signal of sleeping or non-moving humans in a vehicle [7].

To detect the motion of an object and to detect human vital signs using a radar sensor, popular types currently in use are impulse UWB (Ultra Wide Band) radar, FMCW (Frequency Modulated Continuous Wave) radar, and CW (Continuous Wave) radar.

In impulse UWB radar, because it is possible to measure a high-resolution range, we can distinguish chest movements due to breathing by measuring changes in distance values. Because high-resolution range detection is most advantageous, UWB radar is widely used for vital sign recognition [8,9]. Earlier work proposed the concept of detecting the vital signals of a passenger by mounting a UWB radar sensor in a vehicle [8]. In another approach [9], a UWB radar sensor was used to detect human vital signs for each seat, applying the features extracted from the detected range into the machine leaning approach, such as a SVM (Support Vector Machine). However, these two related works only focused on non-moving humans, and did not consider moving humans or other objects.

Although UWB radar is very popular, the Doppler component cannot be detected in order to distinguish between a stationary object and a moving object. Thus, an additional algorithm is required using measured distances. In addition, because this type transmits an impulse-shaped waveform in the time domain, high peak transmission is limited. This can result in a low SNR (Signal-to-Noise Ratio) over a certain distance.

Recently, because FMCW radar can measure both the distance and the Doppler information, FMCW radar has come to be commonly used in commercial applications. Moreover, when detecting changes in phases over several periods, the respiration period was extracted in earlier works [10,11]. In addition, one related study [10] presented a method that separated the vital signs reflected from two humans using a high-resolution algorithm, in that case the MUSIC (Multiple Signal Classifier) algorithm. However, neither method focused on only vital sign signal detection, nor were they intended for in-vehicle applications.

Despite the fact that the FMCW radar is widely used in commercial applications, a PLL (Phase Loop Lock) circuit is also required to synchronize the transmit waveform phases and to ensure linearity during the modulation step. Moreover, because FMCW radar can detect moving and stationary objects, extra algorithms to distinguish them are necessary.

Finally, the CW radar sensor is a popular radar sensor due to its very simple hardware structure. However, because CW radar can only receive Doppler signals, these sensors can only detect moving objects, and cannot detect the range. Thus, in CW radar, breathing signals can be measured by analyzing the Doppler signal generated from the chest movements [7,12,13]. One earlier study [12] presented the concept of recognizing a human remaining in a vehicle using CW radar by cancelling the background noise. In another work [13], an algorithm for detecting not only respiration, but

also the heartbeat, was proposed. Because CW radar is somewhat sensitive to external noise, one study [7] proposed attaching an accelerometer to the radar sensor to record vibrations of the vehicle itself. However, these three related works also considered stationary humans when extracting human vital signs.

Although though CW radar sensors have various disadvantages, they can be easily applied in various applications as a low-cost senor compared to UWB radar and FMCW radar. Thus, in this paper, we employ the CW radar type to realize a passenger detection system with a very simple architecture.

To effectively confirm the existence of a human in a vehicle, we can recognize a human who is still, sleeping, or moving using the Doppler signal measured using the CW radar sensor. That is, we can determine whether or not the detected moving object is a human and can extract vital signals from a non-moving human on the seat.

If a human is moving on a seat, the echo signal of the human's vital signs can be masked by the Doppler signal of the human's motion. In such cases, it is difficult to determine whether a human is present or not in a vehicle using only the detected vital signal. If a human is moving on a seat, the echo signal of the human's Doppler signal of the human's motion. In such

Moreover, if an inanimate object is moving on a seat or if the vehicle itself has vibration, the radar system should be able to recognize a Doppler signal. Thus, when using only the presence of the Doppler echo, it is impossible to determine whether or not a human is occupying a seat.

Thus, in this paper, we propose a human recognition concept as part of our effort to implement a proper passenger detection system, as shown in Figure 1. The wait mode transmits the received radar signal into the motion-detection mode and the vital sign detection mode, with both modes operating in parallel.

**Figure 1.** Human recognition concept for in-vehicle application. Here, the car photo on the left, also used in an earlier work [2], was modified somewhat.

Based the results of both modes, in the decision mode, human recognition is determined, and the system reverts back to the wait mode.

For the vital sign detection mode, we design a simple vital sign detection algorithm to determine whether breathing is present or not. Thus, we extract one feature vector to indicate the presence of vital signs.

of the human's torso, head, shoulders, arms, waist, and thighs, micr For the motion detection mode, we propose algorithms to determine whether or not a moving object in a vehicle is a human. In the proposed method, we use the human characteristics of non-rigid motion. That is, in the case of a human, because the radar signal is reflected from various components of the human's torso, head, shoulders, arms, waist, and thighs, micro-Doppler effect appears. Thus, in paper, we initially generate a micro-Doppler image in the time-frequency domain. Next, we design two new feature vectors that suitably represent the characteristics of a moving human in a vehicle from the micro-Doppler image.

Finally, in the decision mode, we conduct machine learning using a BDT (Binary Decision Tree), which has a very simple structure, and the proposed three features to determine the presence of passengers in vehicles.

Thus, we extract three features using actual measurement data from a CW radar transceiver and verify the proposed machine learning-based human recognition scheme.

In Section 2, we present the proposed human recognition scheme with machine learning. In Section 3, we present the verification results using actual data from a 2.45 GHz CW radar front-end module and a real-time data acquisition module. Finally, we present the conclusion of our study and the suggestions for future work in the final Section.

#### **2. Proposed Human Recognition Scheme in a Vehicle**
