Thermal-Signature-Based Sleep Analysis Sensor

Abstract

This paper addresses the development of a new technique in the sleep analysis domain. Sleep is defined as a periodic physiological state during which vigilance is suspended and reactivity to external stimulations diminished. We sleep on average between six and nine hours per night and our sleep is composed of four to six cycles of about 90 min each. Each of these cycles is composed of a succession of several stages of sleep that vary in depth. Analysis of sleep is usually done via polysomnography. This examination consists of recording, among other things, electrical cerebral activity by electroencephalography (EEG), ocular movements by electrooculography (EOG), and chin muscle tone by electromyography (EMG). Recordings are made mostly in a hospital, more specifically in a service for monitoring the pathologies related to sleep. The readings are then interpreted manually by an expert to generate a hypnogram, a curve showing the succession of sleep stages during the night in 30s epochs. The proposed method is based on the follow-up of the thermal signature that makes it possible to classify the activity into three classes: “awakening,” “calm sleep,” and “restless sleep”. The contribution of this non-invasive method is part of the screening of sleep disorders, to be validated by a more complete analysis of the sleep. The measure provided by this new system, based on temperature monitoring (patient and ambient), aims to be integrated into the tele-medicine platform developed within the framework of the Smart-EEG project by the SYEL–SYstèmes ELectroniques team. Analysis of the data collected during the first surveys carried out with this method showed a correlation between thermal signature and activity during sleep. The advantage of this method lies in its simplicity and the possibility of carrying out measurements of activity during sleep and without direct contact with the patient at home or hospitals.

1. Introduction

Sleep is a state of relative consciousness in which humans, nay, all living beings, engage voluntarily or involuntarily in a state of physical and moral recovery. The study carried out by the authors of [1] highlights the link between the degradation of our socio-economic status (SES) and the deterioration of our sleep that affects our mental and physical well-being. In [2], the authors studied the sleep relationship with the immune response regulation process and concluded that sleep deprivation altered immune function and, conversely, that immune dysfunction will ultimately alter the quality of sleep. Sleep is thus frequently disturbed in the case of critically ill patients who often suffer from mediocre sleep and loss of circadian rhythm [3]. The authors of [4], based on the work of [5], presented a review of the role of slow sleep and paradoxical sleep in the consolidation of the memorization process, a particular challenge for future research.

Understanding the structure of sleep began early enough and was based on observing and recording the experience of a night's sleep. Sleep occupies a good part of our life and intervenes in the physical and psychological recovery. The interest aroused allowed the appearance and the evolution of instruments for the analysis of sleep, such as the electroencephalograph in 1937, which makes it possible to record the cerebral activity from the electrical signals recorded at different positions of the scalp. The signal obtained is an electroencephalogram (EEG). Several aspects of the experience of a night of sleep have motivated several research themes related to the study of sleep. The management of respiratory or behavioral pathologies during sleep caused the authorities to create specialty poles in sleep medicine, and they provided hospitals and clinics with sleep analysis centers. Medical centers specializing in the management of sleep-related pathologies have acquired a set of instruments in addition to the electroencephalograph that record several physiological signals at the same time. In addition to the electroencephalogram (EEG), an electrocardiogram (ECG) monitors heart rate, an electromyogram (EMG) monitors the electrical activity of the muscles (chin and leg), and an electrooculogram (EOG) monitors the eyes. Conceptions of sleep, long considered as a cessation of activity and a slowing of metabolism, suddenly changed in 1953 when paradoxical sleep, a period of intense cerebral activity, was discovered [6]. This sleeping period is organized in five or six cycles of 90 min each, and these same cycles consist of stages ranging from awakening to deep sleep and from paradoxical sleep to light sleep. The authors of [7] provided an overview of the organization of normal human sleep and highlight the importance of a clear appreciation of the characteristics of sleep to provide a solid context and a model to understand the clinical conditions in which “normal” sleep is altered. The authors of [8] presented investigations into the physiological processes involved in the synchronization and regulation of our daily cycles of sleep and wakefulness.

In [9], the problems of recording and analyzing biomedical signals as well as methods exploiting their various characteristics during sleep are discussed. The use of signals recorded during sleeping period results in a night's sleep experience in the form of a hypnogram, which consists of a succession of stages relating to different stages of sleep. The doctors and researchers have established criteria allowing them to somehow translate the signals obtained with a polysomnography examination into a hypnogram. These criteria forged the scoring techniques as early as 1937 with the EEG signals and subsequently evolved with the addition of EOG and EMG signals [10]. These criteria were recorded according to two methods: Rechtschaffen and Kales [11], considers the sleep staging in six stages and the second method have been proposed by the American Association of Sleep Medicine (AASM), [12,13,14], since 2007 is the most popular method where the sleep staging is considered in five stages. Thus, the authors of [14] proposed a study of scoring methods based on a large number of markers, the objective of which was to examine areas of disagreement to inform future revisions of the AASM Manual for Sleep Logging and Associated Events.

To date, sleep scoring is always carried out based on a visual analysis of the recordings made. This method of obtaining a hypnogram is the reference in the analysis of sleep [6] even if new computer techniques try to produce hypnograms in automatic or semi-automatic ways [15,16]. The authors of [17] investigated the different methods of sleep classifications based on the visual analysis of chronograms recorded with polysomnography. The authors of [17] thus compared these classical methods with new automatic methods. Concerning these methods based on visual analysis, the authors [17] highlighted the limitations due to the physiology of the human eye and the visual cortex. The advantage of automating scoring is the idea of saving time on the one hand and on the other hand of proposing a procedure of scoring stably and reproducibly from one patient to another by adopting an algorithm that can master the scoring, which can also constitute a reference for manual scoring.

The increase in the aging population, coupled with the complexity of examinations such as sleep analysis, has led to the emergence of research axes to defer some of this analysis and follow-up to patients' homes. The principle is to sort upstream before undertaking more in-depth analyses, which would be carried out exclusively at the level of hospitals and retirement homes. In this approach, mobile polysomnography devices have been developed and are being used. However, the large number of wires (30) needed for this exploration influenced the acceptability of this technique. In addition, a doubt remains regarding the results and has committed doctors to propose that patients are kept for several nights so that appropriate conditions are met.

Among the simplifying ideas, most methods seek the link between sleep and the physical activity or movement [18,19], body temperature or electro-dermic activity [20]. Among these methods, those based on the monitoring of physical activity have benefited from the rapid evolution of mobile applications. The latter are in most cases equipped with sensors such as accelerometers, which can be exploited to quantify the sporting activity or to follow the movement and the efforts in the games.

These applications have served as a springboard for new health-related applications. Thus, applications have been developed that offer hypnogram-like graphs to provide information about the night's experience and even provide an assessment of sleep quality based on the amount of deep sleep (less agitation) during the night. Other devices in the form of wristbands have been proposed, and, according to the same principle, graphs and sleep assessments are offered. These devices require permanent contact with the patient. The authors of [21] reviewed a set of research articles related to the use of actigraphy for sleep analysis. The authors concluded that, in a clinical context, actigraphy is reliable for assessing sleep in patients with insomnia and in people less likely to tolerate a polysomnograph (PSG). In [22], the authors exploited a video capture in the infrared domain to raise the level of activities of the wrist and the body and compared the results obtained with those of an activity-measuring device based on a motion sensor.

The authors of [9], examining the various signals used in sleep analysis, recounted the role of actigraphy and video recording in the medical follow-up of sleep apnea. To evaluate the sleep–wake rhythm in ambulatory conditions, in [23], the authors introduced a variable (TAP) based on a combination of three signals: wrist-measured temperature, motor activity, and body position in the bed. With the introduction of a combination of three measures, the authors of [24] proposed a global solution to assess the circadian system state and sleep–wake rhythms with a higher reliability level than the actigraphy.

In this paper, we describe a method that follows the same approach as that used in [25]: an analysis of sleep in the sense of physical activity, analyzing the evolution of human thermal radiation during overnight sleep. This thermal radiation is due to the internal heat of the human body whose core temperature is fixed around 37.2 °C. Its variation outside the body is produced by the influence of the ambient heat. To capture this thermal radiation, we used a thermopile sensor targeting the upper area of the human body (trunk and head). To label the experience of a night of sleep, we included, in the experimental device, a thermal camera that produces images in a medical format and provides information as to the temperature of the target according to the spatial distribution. We also included a three-axis accelerometer to detect night movements. The synchronization of the data was achieved by measuring the acquisition time to the nearest second. Staging results with classification algorithms such as K-means applied to the data of a labeled night's sleep confirm our approach to the use of heat radiation to perform a sleep analysis preparatory to a more in-depth analysis with methods using polysomnography. The measurement provided by this new system can be validated with the classical polysomnography-based method by integrating the sensor and its processing unit into the measurement chain realized by the telemedicine platform developed within the framework of the Smart-EEG project [26].

The rest of the document is organized as follows. In Section 2, we provide a brief description of the proposed device used to analyze sleep and the methodology used to exploit the data collected. The experimental results of this work are described in Section 3. In Section 4, orientations for future work are identified, and the performance compared to conventional methods are discussed. Final remarks and future directions for current research are also given in this section.

2. Materials and Methods

2.1. Materials

The acquisition device used for the sleep analysis process is shown in Figure 1. This system consists of a set of sensors positioned remotely, separate from the patient, and another set of sensors that are carried by the patient. These remote sensors consist in a thermal camera and a thermopile sensor. The sensors carried by the patient (see

Table 1. Description of the sensors.

	Thermopile	Thermal Camera	Accelerometer	iButton
Frequency sampling (Hz)	1	0.1	100	≤0.016
Reference	TMP007	FLIRC2	XSENS	DS1921H/Z
The measured data	Room temperature and patient and bed set	Thermal images and temperatures of the targeted area	Acceleration according to 3 axes	Skin temperature
Sensors

) consist in an inertial unit and local temperature sensors placed on the wrist, trunk, and foot. Another local temperature sensor is used to measure the ambient temperature. In adding the inertial unit, we improve the results of certain wearable devices, the results of which are based on monitoring activity recorded by the tri-axial accelerometer sensors.

2.1.1. Sensors and Measured Quantities

The architecture of the final sleep analysis device is built around the thermopile sensor. We chose a sensor with a wide field of view to gain sensitivity. The TMP007 [27] is a thermal infrared (IR) sensor sensitive to radiation in the IR spectrum of wavelengths of approximately 4–16 μm. The TMP007 measures the temperature of an object by sensing the infrared radiation emitted by the object and converting the voltage generated to a digital reading of the temperature.

2.1.2. Data Collection Protocol

Thermal Image Acquisition

Thermal images are acquired every 10s. These images in medical format allowed us to obtain two pieces of information each time. The main information regards the experience of a night of sleep obtained by visual analysis by the expert in order to label the different events related to the patient's posture changes in his/her bed. The second piece of information extracted from the thermal images pertains to the evolution of the temperature corresponding to two zones: the first targeting the upper part of the “bed + patient”. The second zone is chosen to target a region whose temperature is close to ambient temperature.

Upper “Bed + Patient” Temperature

The final objective of our project is to develop a sleep analysis device based on the use of a thermal sensor composed of one or more thermopile sensors. We chose the TMP007 [27], a thermopile sensor built around a network of micro-thermocouples that embed an electronic circuit. The sensor outputs at a rate of one measurement per second, a potential difference proportional to the radiation sensed. A calculation from this quantity resulting from the radiation emitted by the targeted zone and the ambient temperature allows us to obtain the temperature corresponding to the targeted zone. The sensor measures the ambient temperature on its own from a conventional temperature sensor placed judicially in the circuit so as to be thermally insulated therefrom. The accuracy and errors of remote temperature measurements by thermal sensors are largely related to the quality of the ambient temperature measurement.

Wrist Acceleration Measurement

In order to compare the responses of the thermopile sensor with the responses of the most popular systems in the analysis of sleep by wristwatch instruments with accelerometers and carried on the wrists, we have inserted in our device an inertial unit of XSENS [28]. The acquisition is carried out at a frequency of 100 Hz. The measurement makes it possible to obtain the acceleration of the wrist according to the three axes. This choice of using an inertial unit is linked to the possibility of supplementing our estimation of the motion by the angular velocity vector of the wrist along the three axes.

Wrist, Distal, and Proximal Skin Temperature Measurement

Several works [9,24,29,30] have referred to the links between body temperature and sleep. We distinguish between two types of temperature-related data, which tell us about the body temperature, which must be stable around 37.2 °C, and those related to the cutaneous or skin temperature, which varies according to the ambient temperature and the measurement location. The link with sleep is exploited in the study of sleep by analyzing the circadian rhythm. This rhythm is accompanied by a rather fine variation in body temperature of less than 1 °C around 37.2 °C. The day/night alternation corresponds, on the one hand, to the awakening/sleep alternation and to the alternation between high and low temperatures.

During sleep, the body temperature decreases while it increases during the day. The cutaneous temperature, unlike the body temperature, increases during sleep and decreases during waking. Work in [31] relates the uniformization of the cutaneous temperature of the entire body during deep sleep. To complement our knowledge, we have a set of iButton temperature sensors [30], which have the appearance of jacket buttons and operate autonomously. The measurements are taken at a rate corresponding to one measurement per minute. A sensor of the same type will give an estimate of the ambient temperature.

2.2. Data Processing and Interpretation

2.2.1. Features Extraction and Filtering

The reference in terms of the classification of sleep is the classification obtained in the case of sleep analysis by a polysomnography. In this process, the total duration of the recording of the data is subdivided into epochs of 30 s each and sometimes less in the case of certain pathologies (sleep apnea). The manual approach based on observation of the recorded signals consists in assigning epochs to the five stages of sleep.

In the proposed process, we collect data at different frequencies, and the time axis is defined by universal time, which facilitates the comparison of the different data in order to find the correspondences between these heterogeneous quantities.

To exploit the data in order to test the classification, we decided to resample the data series according to two acquisition rates:

• 1 Hz, to operate at the same frequency as the thermopile sensor;
• 0.1 Hz, to operate at the same frequency as the thermal camera.

As proposed by several authors [21,20], we have filtered the data from the accelerometer through a bandpass filter in the range of 0.3–3 Hz. This choice is related to the purpose of keeping only information related to voluntary movements that would correspond to states of awakening or light sleep.

2.2.2. Data Clustering and Classification

Our first approach to data analysis is to see if there is a natural tendency for the data to reflect the indications obtained through the labeling process. For this, we used an algorithm to cluster the data; our choice was based on a simple algorithm so that K-means could be implemented [32]. The partitioning process begins with the choice of cluster number. The next step is to group the data around these clusters according to a Euclidean distance criterion. The choice of the number of clusters as well as the method of choosing the first clusters impacts the result of the classification.

We have fixed the number of clusters at 5 and 3 to see if the trend of the dispersion of the data reveals a grouping close to the stages of sleep noted by our method.

3. Results

The experiment carried out consisted in the recording overnight of the following data: using a thermopile and a thermal camera, ambient temperature, bed temperature, and patient temperature and, using an accelerometer, the acceleration of movement of the right wrist (see

Table 2. Description of the collected data.

Data	Thermopile	Thermal Camera	Accelerometer
Frequency sampling (Hz)	1	0.1	100
Quality of FS (%)	100	97.65	94.96
Number of Observations	28,591	2865	2,703,130
Collection time	23:12:59 to 07:09:29	23:10:26 to 07:08:42	23:12:59 to 07:09:29

).

3.1. The Collected Data

3.1.1 Thermal Temperature

The thermopile sensor, having a larger field of view of around 90°, was placed above the bed at a distance close to 2 m. We initially chose this configuration to ensure that we captured all radiation emitted by the patient and the bed set. The signal obtained in this configuration was rather noisy, and the filtering applied did not bring about a clear improvement, as shown in Figure 2.

This original configuration has two main problems: The thermopile viewing angle (90°) did not match the one of the camera (45° × 34°) [33], and the circuit board affected the ambient temperature. The last problem was solved by moving the thermopile away from the circuit board (to eliminate the circuit board influence of the ambient temperature), while the first problem was solved by bringing the sensor closer to the subject and by limiting the diaphragm. These solutions improved the sensitivity of the thermopile sensor, as shown in Figure 3.

3.1.2. Skin Temperature

In Figure 4, it is observed that the axillary temperature under the armpit is more stable and elevated compared to those of the hand or the foot. During sleep, the body temperature decreases while it increases during the day. The cutaneous temperature, unlike the body temperature, increases during sleep and decreases during waking. Figure 5 presents the distal-proximal gradient (DPG) corresponding to two days and one night of sleep. The goal is to explore the possibilities to predict sleep-onset latency of night-time sleep for infants as mentioned in [34].

The authors of [29] studied the link between sleep and body temperature and used the DPG to link circadian rhythm to the sleep cycle.

3.1.3. Acceleration Module

The movement of the patient during the night informs us of the character of sleep in terms of peaceful or restless. One of the most commonly used sensors for following movement is the inertial sensor. Figure 6 below shows the raw measurements of the acceleration of the right wrist movement along the three axes. On the top is another graph that represents the acceleration module, which indicates the occurrence of movements during the night.

3.2. Comparison of Thermal Radiation and Actigraphy

The results obtained validate, as shown in Figure 7, the use of thermal radiation in place of the actigraphy data proposed by the portable apparatuses.

3.3. K-Means Classification of Data Collected Overnight

The result of the classification using the K-means algorithm is presented graphically in Figure 8. We have defined k (number of clusters) as 5 to see if we can observe all sleep stages. It should be noted that the first phase (Stage 1) occupies the beginning of sleep, and then, when the patient leaves the bed, this observation indicates that Step 1 corresponds to an awakening state. Stage 5, which is present in the early stages of sleep, returns at night to a fairly low degree. It is closer to periods of passage between two stages: from light sleep to deep sleep or vice versa. Stages 3 and 4 correspond more to a phase of light sleep, whereas Phase 2 shows a greater tendency toward deep sleep.

4. Discussion and Conclusions

Sleep analysis is usually performed in a hospital setting equipped with a device comprising a polysomnograph (PSG), which is a device for collecting several physiological signals. Problems are often encountered in the use of a PSG: first, the complexity of the implementation of the medical protocol, which includes the installation of different measurement cables, and second, the influence on the analyzed sleep due to the acceptability of the patient of the analytical device. The sleep analyzed may therefore not correspond to sleep habitually experienced by the patient at home or under conditions without discomfort. To solve these problems, medical practitioners extend the duration of the analysis over several nights in order to realize at least one exploitable analysis.

From this statement, in the present study, we propose a non-invasive method for sleep analysis with a device without any contact with the patient. This device is based mainly on the measurement of the thermal radiation emanating from the patient. Our study was carried out within the framework of the improvement of the device developed in the study proposed by the author of [25], and our final objective is to integrate the device in the measurement chain developed within the Smart-EEG [26].

The authors of [25] developed a device based on the use of a thermopile sensor for the detection of the presence of a patient in a bed. This study [25] conducted in a hospital setting allowed the author to extend the application of the thermopile sensor to detect the stages of sleep by exploiting the variation in the thermal signature during the night. Based on this information, the author claims to distinguish deep sleep from agitated sleep. To validate these conclusions, experiments were carried out in comparison with a professional polysomnograph.

The present study focuses on the validation of the use of a thermal radiation sensor as a sleep analysis sensor. In our validation device, we introduced a thermal camera that was meant to replace one or more thermopile sensors, which allowed us to extract two pieces of information: the variation in the temperature of the “bed + patient” set and images (of a medical format) taken throughout the night. To measure the amount of movement, we connected our device to an actimeter consisting of an inertial unit fixed to the wrist of the patient. This allowed us to detect the acceleration of the wrist along the three axes. The results obtained from the analysis of a given night of sleep allowed us to validate the use of a thermopile sensor for the simultaneous extraction of several pieces of information, such as skin temperature, actimetry, and the presence, absence, and position of a patient in a bed. Although this information was not corroborated by a device incorporating a PSG, it was sufficient to record the sleep stages corresponding to rest periods and periods of activity and awakening.

Compared to other studies, the study carried out in [23,24] offers a rather interesting prospect where our sleep analysis device can be exploited by an analysis of thermal radiation. The author proposes a new variable, TAP, which combines three pieces of information to accurately predict the rest–activity period.

In terms of this TAP variable, one could employ our device as a single sensor to deliver information in the form of states that toggle between rest, activity, or awakening. One study proposed in [22] is similar to the process developed in our study. The authors of [22] compared an analysis of motion based on measurements of wrist accelerations with an analysis of extracted motion based on infrared images captured during a night of sleep.

In conclusion, our device should ultimately result in a network of thermopile sensors associated with an electronic signal processing system. Via the Smart-EEG sensor chain [26], practitioners will be able to acquire data related to actimetry, patients’ cutaneous temperatures, and PSG information.