1. Introduction
Vital signs, such as body temperature, pulse rate, blood pressure, and the Respiration Rate (RR), are used to monitor the clinical status of patients and are crucial in detecting medical problems as they represent the fundamental functions of the body [
1]. The RR is an early indicator for clinical deterioration and is therefore an integral component of numerous early clinical warning scores [
2,
3,
4,
5]. Changes in the RR can predict conditions including respiratory failure [
2], cardiac arrest [
3,
5], chronic heart failure [
3], pulmonary embolism [
3], weaning failure [
3] or Intensive Care Unit (ICU) readmission [
3]. Liu et al. [
3] reported that deviations in the RR from its normal range can identify risks for medical emergencies 24 h before their occurrence with a probability of 95%. They also noted that the RR provides a better discriminator than blood pressure or heart rate in identifying high-risk patients. To measure the RR, the common practice involves manually counting the number of breaths within a specific time frame [
3,
6,
7]. According to Drummond et al. [
4], the measured RR can vary by 2 to 6 breaths per minute (bpm) between observers. Moreover, in busy clinical environments, there is often insufficient time to measure longer time frames for each patient and repeat measurements regularly. Consequently, shorter time intervals of 15 to 30 s are typically used, further contributing to the variance [
4,
7]. This variance can result in up to 40% of failures in the early warning scores due to inaccurate RR measurements [
4].
Measuring a single RR value is crucial, but continuous monitoring of respiration over an extended period can be necessary in certain cases. For instance, respiration has been examined for diagnosing and monitoring patients with conditions like chronic obstructive pulmonary disease [
6], pneumonia [
3,
5] or sleep breathing disorders [
6]. Continuous respiration monitoring also finds application in diagnostic imaging and radiation therapy, where the identification and elimination of motion artifacts caused by patients breathing can enhance measurement quality [
5]. While more accurate methods for measuring respiration signals, such as spirometry [
6], plethysmography, capnography, piezoelectric and bioimpedance-based sensors [
7] exist, these methods are often intrusive, interfere with the normal breathing or cause discomfort for patients. The current diagnosis of sleep breathing disorders relies on polysomnography, which is highly uncomfortable and often results in non-representative sleep [
6]. In some cases, infants in the neonatal intensive care unit are too small to use the face masks of the traditional methods. Recently, there has been a growing awareness of the importance of respiratory monitoring for patients with viral infections, such as COVID-19 [
2,
5]. Minimal contact between clinical staff and the patient is needed. Consequently, contactless continuous monitoring systems have gained greater interest to address these challenges [
5]. The aim is to reduce the restrictions and improve patient comfort [
2] while simultaneously providing an automated, reliable and accurate solution [
3]. Contactless systems differ in their underlying mechanisms and have different advantages and limitations [
3]. Massaroni et al. [
5] provided a comprehensive review of these contactless techniques. The contactless methods measure respiratory sounds, air humidity [
8], air temperature, cardiac activity modulation or chest wall movements. To measure respiratory sounds in a contactless manner, environmental microphones are used. The advantage lies in its commercial availability, but challenges persist in distinguishing between inhaling and exhaling sounds and in its sensitivity to environmental noise [
9]. Zhang et al. used electrochemical humidity sensors with a breathing mask to measure the respiration rate and detect respiration patterns [
8]. Thermal cameras are used for the contactless measurement of air temperature, while red, green, and blue (RGB) cameras can capture cardiac activity. However, 2D imaging-based methods like RGB cameras or thermal cameras usually require the patients face to be visible, limiting their utility in regard to patient positions and ambient lighting conditions [
10].
To measure the chest wall movements and extract respiration parameters, several technologies have been used, including marker-based systems [
9], laser vibrometry [
11], radiofrequency sensors [
12,
13], terahertz sensors [
14], ultrasound [
15], RGB cameras [
16,
17,
18] and depth sensors [
2]. While marker-based systems are classified as contactless, they still require markers to be attached to the body. Consequently, some of the disadvantages of contact-based methods persist, and these systems are in general expensive [
5]. Hence, marker-based systems are often unsuitable for contactless continuous monitoring [
5]. Although the physical principles differ among these sensors, they all rely on distance information between the body surface and the sensor. Depth sensors, also known as 3D sensors, can be based on the principle of Time-of-Flight (ToF) [
10,
19,
20,
21,
22,
23] and structured light [
24,
25,
26,
27]. Many studies focus solely on extracting the average RR, while some extract the Breath-to-Breath Respiration Rate (BRR), measure volume variations, or extract a continuous respiration signal. Among these 3D sensing technologies, ToF sensors have been predominantly used for monitoring breathing activity [
5] due to the sensor’s commercial availability and low cost. The Microsoft Kinect v2 ToF sensor has been employed in multiple studies [
19,
21,
28]. Skeletal information was automatically provided by the Kinect SDK 2.0. The Region of Interest (ROI) can be defined based on the joint positions of the shoulders, hips, and mid-spine. However, these systems are limited to capture subjects facing the camera, as the joints cannot be detected and the ROI computation fails otherwise. Kempfle et al. [
19] and Aoki et al. [
21] indicate that such a method is only suitable for non-moving subjects. To address this limitation, Kempfle et al. [
28] detected and used the throat region, minimally affected by respiratory motion, as a reference for overall body motion estimation. This method enabled more accurate measurement of respiratory motion by removing the overall body motion from the signal. Their approach achieved a correlation to the signal from a respiratory belt of over 0.9 for more than half the users, even with body motion. However, when the throat region is covered due to head movements, changes in position, or occlusion, the correlation dropped below 0.4, restricting the feasibility the feasibility of the method for various patient positions. Gleichauf et al. [
10] proposed the use of a ToF camera in combination with a microwave interferometric radar sensor, specifically for Neonatal Intensive Care Units (NICUs). The method was tested using a self developed neonatal simulation system for different breathing patterns. The ToF camera outperformed the radar sensor in the normal range of the RR but led to less accurate results in the low RR region, differing 31 bpm from the reference rate. The ToF camera’s strong sensitivity to noise and reflection caused the algorithm to detect additional peaks at low RRs. The radar sensor was found to be sensitive to body motion artifacts, such as swaying arms. By combining the two methods, they were able to extract an RR in the normal range with a maximum deviation of 3 bpm compared to the reference, by complementing each other’s drawbacks. Another 3D sensing technology, the structured light approach, was used in [
24]. The system was specifically designed for individuals with a normal breathing rate. Different scenarios, including slight body movement, different clothing, ambient lighting conditions, and user orientations were compared. The system demonstrated correlations between 0.92 and 0.98 with the spirometer across these scenarios. However, the main limitation identified was the patient’s orientation, allowing only a maximum deviation of 25
from the frontal view. Consequently, the system is restricted to a limited range of patient positions. The approach proposed in [
25] was also based on structured light, using a pattern composed of RGB primary colors to illuminate the scene. The subjects were unclothed and laying still. While an interference effect from ambient light on measurement accuracy was observed, it was not quantified. Under the predefined conditions, they were able to generate motion trajectories of a specific point near the diaphragm. However, the results were not verified using a reference system.
In many studies, the selection of the ROI was manually conducted [
20,
23,
29,
30,
31]. However, a recent study [
32] focused on the automated selection of the ROI. This process involved dividing each depth frame into sub-regions, averaging the depth in each region, and then applying blind-source separation to each time signal. A scoring system was applied to assess the resulting signals based on their likelihood of originating from respiratory activity. As a reference, a respiration belt was used, with participants instructed to minimize body movement and breathe normally. On average, errors between 2.89 and 11.2 bpm were observed in the detected RR. It is important to note that the scoring system relies on characteristics of normal breathing and is not suitable for detecting abnormalities. Despite recent advancements in contactless respiration monitoring techniques, challenges and questions remain unsolved [
7]. The primary challenges include the system’s robustness in varying ambient lighting conditions, body movement, patient positions, range of RRs, clothing and also the overall accuracy in general. Moreover, many studies focus solely on extracting the average RR rather than a continuous respiration signal.
In recent years, remarkable progress has been achieved in contactless 3D shape measurement technologies in general, which is driven by their increasing demand across various industries [
33]. Those advances had, among others, been made in the field of Fringe Projection Profilometry (FPP). According to several authors [
33,
34,
35], FPP is one of the most popular and promising technologies for 3D measurements. Systems based on FPP show advantages in its achievable speed, depth resolution, accuracy, affordability, versatility and robustness [
33,
36] in comparison to other 3D sensing technologies such as ToF cameras or stereo vision.
This paper introduces a FPP-based contactless respiration monitoring approach and its evaluation to overcome some of the challenges described above. The focus lies on the analyzing algorithm for extracting the RR and a respiration signal from the FPP 3D measurements. The system is evaluated in terms of its overall accuracy, robustness and limitations.