Non-Imaging Fall Detection Based on Spectral Signatures Obtained Using a Micro-Doppler Millimeter-Wave Radar

Abstract

Falls are the leading cause of accidents among the elderly population. In recent years, radar has been employed in fall detection due to its superior sensing capabilities, small dimensions, low cost and primarily non-intrusive sensing capabilities in addition to its robustness under a range of heat and lighting conditions. In this paper, we present a technique for identifying when a person is falling using a low-power millimeter-wave radar operating in the W-band. This detection, conducted in real time, is based on the transmission of a continuous wave and heterodyning of the received signal reflected from the person to obtain micro-Doppler shifts associated with the person’s motion. These results make it possible to obtain a high-quality time-frequency distribution and spectrogram, from which the person’s unique fall movement characteristics can be determined. In this paper, we present experimental results based on 94 GHz real radar data obtained from a falling person. This carrier frequency is higher than that of current systems, allowing higher frequency resolution and more accurate results. Compared to other tracking systems, this sensor does not simulate or violate privacy. However, the high-frequency system enables high-resolution realizations with high reliability.

1. Introduction

Global aging is a phenomenon known to have resulted from increasing longevity. Every year, the proportion population aged over 65 is growing [1,2]. The aging process is characterized by unfavorable changes in the body’s systems and organs, which may cause impaired coordination and loss of balance during changes in body position. These changes increase the chance of falls. Falls are the most common cause of accidents among the elderly [3] and are responsible for various injuries, including serious injuries that frequently lead to functional dependencies. The most common such injury is hip fracture. One of the circumstances influencing the severity of fall consequences in the elderly is the length of time for which the victim is waiting, immobile on the ground, for help. Less time spent waiting for help is related to less-severe health consequences and a higher chance of successful recovery and return to normal life.

Consequently, experts are increasingly working toward the development of effective fall detection systems and techniques. Several fall detection systems have recently been proposed in the literature [4,5]. Most studies divide fall detectors into two main categories: wearable and non-wearable. Wearable devices [4,5,6] are inexpensive but have some significant disadvantages, including the need to remember to wear the device. An elderly person may forget to put on the device or refuse to wear it for comfort reasons. Moreover, push-button devices are battery-operated and require the individual to be able to press the button after a fall. Fall detection systems that are non-wearable seem to be more promising than those that are wearable, because they do not require additional effort from the individual after the system has been installed at the user’s home.

Non-wearable fall detection methods are non-obtrusive and can be optical (e.g., video-based) [4,7] or non-optical, based on ultrasonic [8], pressure [9] or vibration sensors [10], among others. In this work, we focus on fall detection based on millimeter-wave (MMW) radar.

Recent technological developments have resulted in MMW radar becoming an increasingly commercialized and applicable technology. This radar is employed as a detection measure in many applications, such as in collision avoidance radars in automobiles [11] and in stand-off remote sensors in homeland security applications [12,13,14]. Detection of a person using Doppler MMW radar has several advantages compared to other remote sensing technologies. Due to its high transmission frequency, the transmission power is low, its dimensions are small and the transmission angle is wide.

The micro-Doppler signature of a moving person is a time-varying frequency shift caused by the relative movement of separate parts of the body; it can be studied to extract additional information on an individual’s movement characteristics [13]. Several works exploiting micro-Doppler signatures for human identification have been published [13,15,16]. To enable the detection of a fall incident, we started with the development of motion classification algorithms. These provide the system with knowledge regarding various types and forms of motion and enable it to identify specific patterns and detect anomalies. This paper is concerned with practical issues involving efficient algorithm system implementations. The goal of this study was to develop a motion classifier and build a fall detection application that can be further developed and deployed in users’ homes.

This paper is organized as follows: In Section 2, we describe the principles of continuous-wave micro-Doppler radar operation. In Section 3, we explain how to create an optimal spectrogram using short-time Fourier transform. The experimental setup is presented in Section 4. In Section 5, we describe the method of fall detection using spectral signatures and present the experimental results. In Section 6, we summarize and conclude the paper.

2. Continuous-Wave Micro-Doppler Radar

In this study, fall detection is based on a continuous-wave (CW) micro-Doppler millimeter-wave (MMW) radar. A CW is a sine wave of constant amplitude and frequency, and it is considered to be of infinite duration. The Doppler effect refers to the shift in frequency caused by a moving object. An additional shift in frequency can be caused by the object vibrating or spinning. When this happens, the Doppler-shifted signal becomes modulated; this is known as the micro-Doppler effect. This modulation can have a specific pattern or signature for which algorithms can be developed for required applications; this will be expanded on later.

Figure 1 presents the basic scheme of the CW micro-Doppler radar used in this study. A CW signal, ${\stackrel{\sim }{E}}_T$, with a constant carrier frequency $f_0$ is transmitted toward an object:

Figure 1. A general scheme of a continuous-wave (CW) micro-Doppler radar.

Figure 1. A general scheme of a continuous-wave (CW) micro-Doppler radar.

$${\stackrel{\sim }{E}}_T\left(t\right)=\Re \left(A_T{e}^{j2\pi f_{0}t}\right)$$

(1)

where t indicates the dependence on time, $\Re (·)$ denotes the real part of the complex number and $A_T$ is the amplitude of the transmitted signal. The received signal returning from the moving object at distance $r\left(t\right)$ is:

$${\stackrel{\sim }{E}}_R\left(t\right)=\Re \{A_R{e}^{-j[2k·r(t)-\theta ]}·e^{j2\pi f_{0}t}\}$$

(2)

where $A_R$ is the amplitude of the received signal, $k=2\pi f_0/c$ is the wavenumber, c is the speed of light and $\theta$ is a constant for the phase shift. A factor of 2 is applied to account for the path from the radar to the object and the return. At the radar, the transmitted signal (1) is multiplied with the received signal (2). After low pass filtering (LPF), the intermediate frequency (IF) signal obtained is:

$$\stackrel{\sim }{V}\left(t\right)={\stackrel{\sim }{E}}_T\left(t\right)·{\stackrel{\sim }{E}}_R^{\ast }\left(t\right)=\Re \left\{A_T{A}_R{e}^{j[2k·r(t)-\theta ]}\right\}$$

(3)

From this equation, it can be seen that the resulting phase changes over time:

$$\phi \left(t\right)=2k·r\left(t\right)-\theta$$

(4)

Through the use of (4) and the familiar connections from mechanics, ${\displaystyle \frac{d}{dt}}r\left(t\right)=v\left(t\right)$ and ${\displaystyle \frac{1}{2\pi }}{\displaystyle \frac{\partial \phi \left(t\right)}{\partial t}}=f_d\left(t\right)$, it is possible to express the instantaneous Doppler frequency shift as:

$$f_d\left(t\right)={\displaystyle \frac{1}{2\pi }}{\displaystyle \frac{\partial \phi \left(t\right)}{\partial t}}={\displaystyle \frac{1}{2\pi }}2k·v\left(t\right)·cos\alpha ={\displaystyle \frac{2f_0}{c}}·v\left(t\right)·cos\alpha$$

(5)

where $\alpha$ is the angle between the direction of the transmitted/reflected signal and the direction of moving of the person. The resulting instantaneous Doppler frequency $f_d\left(t\right)$ of the IF signal obtained at the multiplier output is proportional to the object velocity $v\left(t\right)$.

3. Creating an Optimal Spectrogram

The product obtained from the radar output is a time-dependent voltage. Short-time Fourier transform (STFT) is performed to extract the instantaneous Doppler frequency $f_d\left(t\right)$, which is directly proportional to the object velocity $v\left(t\right)$. STFT is used to analyze how the Doppler frequency of a non-stationary signal (i.e., the radar’s output voltage) changes over time. The STFT of a signal is calculated by sliding a window of length W over the signal and calculating the discrete Fourier transform (DFT) of the windowed data. The window has an overlap of length L over the previous window. For example, Figure 2a shows a non-stationary signal, chirp, in which a cosine signal frequency linearly increases. Figure 2b illustrates the signal distribution to the windows and the overlap between them. Figure 2c shows the DFT of each windowed segment; here, it can already be seen how the Doppler frequency content changes over time. We used Matlab to perform the STFT calculations and generate different graphs.

The final spectrogram is the same as Figure 2c but taking a different view. The intensity of the detected signal is represented by color. Blue represents low-level intensities, while red is for higher IF signal strength. The choice of the window length on which the DFT will be performed is highly important in creating an optimal spectrogram. The window should be long enough to achieve the resolution required to identify the different micro-Doppler frequency shifts, but not so long as to enable following the temporal behavior of the instantaneous frequency. For example, if the selected window is too short when obtaining the spectrogram for the chirp signal, we sample a tiny part of the signal at a time, corresponding to an approximately square signal. Therefore, we obtain a sequence of sinc signals in the spectrogram, as shown in Figure 3a. If the selected window is too long, we sample a wide number of frequencies at a time, thus obtaining several frequencies in each window and a non-continuous graph as the spectrogram result, as shown in Figure 3c. Finally, Figure 3b shows when the window selection is suitable, and the result of the spectrogram is continuous and smooth.

4. CW Micro-Doppler MMW Radar

4.1. CW Micro-Doppler MMW Radar Experimental Setup

A schematic illustration of the MMW radar is shown in Figure 4. First, an RF signal generator produces a CW signal at 15.67 GHz. Next, the signal frequency is up-converted by an X6 multiplier to the W-band regime, resulting in a 94 GHz CW carrier. The output signal is then coupled to a circulator and a mixer serving as a product detector. The signal is transmitted from the circulator to the target. The received signal, scattered from the moving target, is amplified by a low noise amplifier (LNA) and inserted into the mixer. Finally, the Doppler frequency is transmitted to the scope, where the information is processed into a spectrogram.

4.2. Advantages of Using MMW in Radar Fall Detection

MMW or extremely high frequency (EHF) refers to the band of frequencies in the electromagnetic spectrum from 30 to 300 gigahertz (GHz). Our radar system operates at 94 GHz. There are several advantages to using MMW in radar fall detection in comparison to the approaches used in other published studies. First, MMW allows higher frequency resolution and more accurate results. Second, it enables the utilization of small-aperture directive antennas, resulting in smaller systems. Finally, at 94 GHz, the atmospheric attenuation is low (i.e., within the atmospheric transmission window). Therefore, this radar can perform in places where the atmosphere is not optimal for camera sensors, such as saunas.

5. Fall Detection Application

Table 1. Comparison of fall detection systems.

Type of Sensing	Cost	Algorithm Type	Motion Classification	Disadvantages	Imaging
Wearable sensors-3D accelerometer	cheap	Threshold-based	Not classified	High false rate, no remote sensing	Not imaging
Camera-based sensing	medium	Detection based on human skeleton, falling angle vertical projection	Classified	High computing resources to process data continuously, not portable, limited sensing area	Imaging
Radar-based sensing	medium	Micro-Doppler time-frequency analyze	Classified	limited sensing area	Not imaging

compares the various modes of fall detection systems, showing the classifications and disadvantages. These are useful for system design.

5.1. Method of Fall Detection Using Spectral Signature

Since the system uses a CW radar, the proposed model is the most simplistic for application when the expected reflected signal frequency is up to about 10 kHz, and there are no stringent requirements for signal processing units. Therefore, the signal processing performed on a simple microcontroller can be implemented. The disadvantages of the system are that fall detection occurs only for movement perpendicular to the radar system. Therefore, the Doppler received signal will be only the distraction levy on the radial axis for movement not perpendicular to the distraction [13]. However, several mechanisms can overcome this problem. For example, installing two systems on two perpendicular walls or identifying the object’s location by calculating the angle relative to the radar system [17]. The spectral signature describing a person’s movement can be detected using the CW micro-Doppler MMW radar described in Section 4.1 and to generate the spectrogram described in Section 3. By examining these spectral signatures detected by the radar, it is possible to identify the kind of user movement. We measured four different kinds of motion: falling, sitting, walking and falling while walking. The differences between the spectral signatures detected using the radar could be seen for these four movements. As a result of these differences, we determined that is possible to identify and distinguish a falling person using the radar.

In order to demonstrate the accuracy of the spectral signature results obtained using the radar, we simultaneously performed measurements with video processing. We filmed the person’s movement with a simple camera, and using the “Tracker” program, we followed the movements of various body parts. “Tracker” is a free video analysis and modeling tool built on the Open Source Physics. Next, we compared the results obtained from the two different measurement modalities.

5.2. Experimental Results

Typical time domains of the IF signals resulting from the four different motions are shown in Figure 5. We generated a spectrogram for each type of motion, as can be seen in Figure 6. These spectrograms contain valuable information related to human motion characteristics. The results are obtained consistently for a specific target but depend on the target’s characteristics, such as height, and the system’s characteristics, such as the transmission frequency and the viewing angle.

5.2.1. Falling Experiment

For the scenario of a person falling, we measured an individual’s spectral signature using the radar, as shown in Figure 6a. In addition, we tracked the movement of the head using video processing. Figure 7 shows three frames from the video of the falling person; the red markings indicate the head movement tracking. Figure 8 shows the results of the two experiments described above.

It can be seen that the shape of the analyzed video signal fits the spectrogram from the radar. The inaccuracy of the slopes and the maximum point are attributed to the video camera’s low frame rate. From the spectrogram, the duration of a person’s fall was determined to be $T_1=1.96$ s and the maximum frequency $f_{max}=1.6$ kHz. Using Equation (5), and in the case that $\alpha =0$, i.e., the person’s motion is in the direction of the radar, it is possible to calculate the maximum speed during this motion $v_{max}=2.55$ m/s. The time taken for the frequency to decrease back to $f=0$ was $T_2=0.425$ s; thus, the decreasing slope was $m_n=3.764$ kHz/s. Note that, in reality, the expected slope should be steeper, because the subject in this experiment stopped himself before hitting the ground in order to avoid injury.

5.2.2. Sitting Experiment

For the scenario of a sitting person, we measured the test subject’s spectral signature using the radar, as shown in Figure 6b. We also tracked the movement of his head using video processing. Figure 9 shows three frames from the video of the sitting test subject; the red markings show the tracking of his head movement. We chose to track the head because it is the uppermost part of the body and therefore has the highest velocity during this motion. Figure 10 shows the results of the two experiments described above.

The fusion of the sensor’s radar and video helps to verify the results. However, in practice, the system will be radar-based only. Therefore, the following analysis deals with the results obtained from the spectrogram. Figure 10 clearly shows the accuracy of the spectrogram result. We split the head motion into four parts in order to better understand the spectrogram result. First, in the time interval $2.6\mathrm{s}<t<3.4\mathrm{s}$, the head is accelerating forward with the test subject’s body and the frequency changes in the range $0<f_d<0.695$ kHz; therefore, there is a positive slope of $m_p=0.868$ kHz/s. Second, in the time interval $3.4\mathrm{s}<t<3.7\mathrm{s}$, the subject accelerated backward while their velocity was still in a forward direction up to the point where their velocity reached 0. The Doppler frequency, from the spectrogram, decreased to $f_d=0.38$ kHz and the slope of this part was $m_n=0.393$ kHz/s. Third, in the time interval $3.7\mathrm{s}<t<4.2\mathrm{s}$, the subject’s head movement and velocity were in a backwards direction. The frequency increased to $f_d=0.8$ kHz with a slope of $m_p=0.814$ kHz/s. Finally, in the time interval $4.2\mathrm{s}<t<5\mathrm{s}$, the frequency decreased to $f_d=0$. In this interval, the frequency has a decreasing slope of $m_n=1$ kHz/s.

5.2.3. Walking Experiment

For the scenario of a walking person, we measured the subject’s spectral signature using the radar, as shown in Figure 6c. We also tracked the movement of his head, shoulder and right hand through video processing. Figure 11 shows three frames from the video of the walking person; the tracking of the head movement is indicated by red markers, shoulder tracking by green markers and tracking of the right hand by blue markers.

Figure 12 shows the results obtained in the experiments described above. In this figure, correspondence between the spectrogram and the video analysis results can be seen. In the movement of the right hand, indicated as a blue line, two “hills” can be seen. The second hill is in the time interval $4.07\mathrm{s}<t<5.14\mathrm{s}$. In the time interval $4.07\mathrm{s}<t<4.5\mathrm{s}$, the frequency increases from 0 to its highest value of $f_d=1.4$ kHz; thus, there is a positive slope of $m_p=3.25$ kHz/s. In the time interval $4.5\mathrm{s}<t<5.14\mathrm{s}$, the frequency decreased to 0 with a slope of $m_n=2.18$ kHz/s. The two other “hills” observed in the spectrogram, which look similar to those attributed to the right-hand tracker signal, are due to movement of the left hand.

The power due to shoulder and head movement is higher than the power due to right-hand movement, but their characteristics are different from those of the falling spectrogram. Therefore, head and shoulder movement during walking is not likely to cause an erroneous result for radar detection.

5.2.4. Falling while Walking Experiment

For the scenario of a person falling while walking, we measured the test subject’s spectral signature using the radar, as shown in Figure 6d. We also tracked the movement of the subject’s head, shoulder and right hand using video processing. Figure 13 shows three frames from the video of the person falling while walking; the red markers show the tracking of the head movement, the shoulder tracking is shown with green markers and right-hand tracking is shown with blue markers.

Figure 14 shows that the video analysis signals have the same shape as the signals in the spectrogram, but with a higher frequency value because of the video camera’s relatively low frame rate. The walking part of this motion is similar to the results of person when only walking, as discussed in Section 5.2.4. Therefore, only the fall time of this motion will be discussed in the following. In the falling time interval $2.608\mathrm{s}<t<3.28\mathrm{s}$, the frequency changed in the range $0.515\mathrm{kHz}<f_d<2.11\mathrm{kHz}$, and the slope was $m_p=2.37$ kHz/s. Afterwards, in the time interval $3.28\mathrm{s}<t<3.647\mathrm{s}$, the Doppler frequency decreased to 0 with a slope of $m_n=5.86$ kHz/s.

5.2.5. Experimental Summary Results

Table 2. Movement characteristics.

Movement Type	${\mathit{m}}_{\mathit{p}}$ (kHz/s)	${\mathit{T}}_{\mathit{p}}$ (s)	${\mathit{m}}_{\mathit{n}}$ (kHz/s)	${\mathit{T}}_{\mathit{n}}$ (s)
Falling	1.042	1.535	3.764	0.425
Sitting	0.868	0.8	1	0.8
Walking	3.25	0.43	2.18	0.64
Falling while walking	2.37	0.672	5.86	0.367

summarizes the results of the experiments. For each type of movement, the table shows the relevant times and their corresponding slopes. It is possible to see the characteristics of the falling movement and the features of the other movements that can disturb the effectiveness of radar application.The results shown are a typical adult target. The results depend on the characteristics of the target, such as height and age; see for example, [13]. Similar results were obtained from measurements of different heights of 150 cm to 190 cm, with up to 10% difference.

From Table 2, it can be seen that the slopes for the falling scenarios are significantly higher than those for the other movements. This suggests that the scenario of a falling person can be discerned with certainty using the MMW radar. An application condition could be defined as $m_n>3.7$ kHz/s—that is, when a movement occurs in which the negative slope meets this condition, a fall would be detected by the radar.

6. Conclusions

In this paper, we describe the development of an MMW- adar for application in fall detection. We examined four radar signals corresponding to four user movements: falling, sitting, walking and falling while walking. Using Matlab, STFT calculations were conducted on the received radar signals, yielding spectrograms with different spectral signatures for each type of movement. The experimental spectrogram results show that the negative slope for the falling movement was unique because of its high slope value. Therefore, a condition for detecting a falling person using the MMW radar could be implemented to send alerts when a user has fallen.

Moreover, we performed video processing simultaneously with the radar detection. We filmed the test subject’s movement by camera and then used the “Tracker” program to follow the movements of various body parts. The comparison results obtained from the two different modalities showed a high correlation.

Our MMW radar fall detection system operating at 94 GHz has several advantages over other tools in the current literature, allowing higher frequency resolution and more accurate results. It enables the utilization of small-aperture directive antennas, resulting in the possibility of developing smaller systems. It can perform in places where the atmosphere is not optimal for camera sensors. Finally, it does not simulate or infringe on privacy.