1. Introduction
The statistic of World Health Organization (WHO) in 2016 has shown that the cardiovascular diseases (CVDs) are the number 1 cause of death globally [
1]. A review of the European Society of Cardiology has mentioned that heart rate is an independent cardiovascular risk factor [
2], while the American Heart Association (AHA) has noticed that to control the heart rate within a relatively normal range is one of the treatment goals [
3]. Therefore, a comfortable and easy-to-use design of heart rate monitoring device will be the trend in the following years.
Photoplethysmography (PPG) is one of the optical techniques that has been developed for experimental use in vascular disease [
4] and has applications on measuring the oxygen saturation, blood pressure, cardiac output [
5], and determining heat stress level by using frequency analysis of PPG and its derivatives [
6]. To compare PPG with other heart rate detection methods like electrocardiography (ECG), PPG is more convenient to use and easier to wear [
5]. Users can easily wear the PPG devices on the earlobe, on the finger, on the wrist, or on the toe [
7,
8,
9,
10,
11]. Different positions are chosen to detect the signal in order to improve the signal quality. Though PPG devices have advantages like being light-weighted, portable, and easy-to-use, motion artifacts are still the problems that have to overcome [
5,
12,
13,
14,
15].
In the previous studies, problems such as weight of device, or whether the device is comfortable to wear or not are highly related to the acceptability and popularity of the users. Several later studies have proposed PPG detection on the side surface of the computer mouse [
16,
17,
18]. However, users have to force their thumbs to contact the surface of the PPG sensor, which may not be comfortable. In this study, we suggest an alternate sensing position, developed by placing the multi-PPG sensor on the top surface of the mouse, which makes contact with the user’s palm. In this solution, multiple sensors can increase the contact area and the probability of signal detection, and users can use the mouse comfortably instead of forcing their finger skin to touch the sensor.
The PPG signal suffers from motion artifacts easily, and the signal quality is constrained by the position chosen for signal detection [
10,
19]. Lee et al. have indicated that the greatest and worst cause of artifact noise that contaminates the PPG signals is motion artifacts produced from physical activity and body movement [
20]. In order to achieve clear PPG and improve the accuracy of pulse rate detection, filter and amplification are required [
21].
Tran and Chung have provided a robust algorithm for peak detection that can eliminate the fake peaks and correct error peaks in the original PPG signal under challenging movement conditions in real time on a personal computer mouse [
18]. Nevertheless, the error rate of the algorithm still increases due to mouse movements that cause more motion artifacts [
14,
15,
20]. Alzahrani et al. and Warren et al. have constructed a design of multi-PPG sensors combining with an accelerometer on wearable devices to reduce motion artifacts [
22,
23]. These structures, however, have the disadvantages of increasing costs and algorithm complexities.
The purpose of our study is to implement a comfortable and easy-to-use module with a stable and accurate pulse rate detection on a computer mouse that most people are using everyday either in the office or at home, as multi-channel sensors can handle displacement and misalignment of the palm [
22]. We proposed a multi-sensor module design (which is smaller than the previous mentioned studies [
16,
17,
18]) on the top surface of the mouse, and other circuits including the microcontroller are inside the mouse. Therefore, users don’t have to force their hands holding the mouse while they are using the mouse. The multi-channel sensor structure can not only raise the area contacted by the users, but also increase the sensitivity and predictivity. Furthermore, a weighted average method for the mixed signal can emphasize the better signal with a higher weight and the poorer signal with a lower weight to improve the quality of the signal and to increase the accuracy of detection.
The paper is organized as follows:
Section 2 describes the hardware structure and the proposed method to perform pulse rate detection based on a multi-sensor; the experimental device and the experimental procedures of each stage are stated in
Section 3;
Section 4 summarizes the experimental results of the proposed method and the comparisons to each channel and both the average and the weighted average method; and finally, some conclusions are discussed in
Section 5.
2. Materials and Methods
2.1. System Architecture
The proposed structure consists of a microcontroller (NUC120, Nuvoton Technology Corporation, Hsinchu City, Taiwan, which is an ARM®Cortex™-M0-based microcontroller), a Bluetooth module, the four-PPG sensor with four sets of band-pass filter with an amplifier, and a programmable-gain amplifier (PGA117, Texas Instruments, Dallas, TX, United States). The signal is extracted from each PPG sensor (uPI Semiconductor, Hsinchu County, Taiwan, with a wavelength of 850 nm) as one signal channel, which includes an infrared LED and a photodiode (PD).
The designed system structure is shown in
Figure 1, which is based on the design of our previous study [
24]. The LED emits infrared light to the skin, and then the photodiode receives the reflected signal that is dependent on the changes in the blood pulse and generates the PPG signal at the output of the sensor. The PPG signal will then input through a band-pass filter and an amplifier. The filtered and amplified signal will next input into a microcontroller to evaluate the pulse rate by using the algorithm and finally output the PPG waveform of each channel and a mixed PPG to a PC, which is a user interface, for waveforms and pulse rate display, and for health data storage.
2.2. Hardware Design
The Cortex™-M0 based microcontroller NUC120 (Nuvoton Technology Corporation, Hsinchu City, Taiwan) is used as the main controller in our design, and it is embedded in the computer mouse combined with other related circuits. The Inter Integrated Circuit (I2C) is used to control the PPG sensors, and the Serial Peripheral Interface (SPI) is used to control the PGA switching on modifying the magnitude factor of different PPG channels. The filtered and amplified PPG signal will input through a built-in analog-to-digital converter (ADC) with a resolution of 12 bits and the sampling rate is 200 Hz, and then the output signal will be communicated to the personal computer by the Bluetooth module. The waveform and the evaluated pulse rate will be displayed on the computer and the data is stored and can be analyzed by the application program.
In this work, we propose a pulse rate detection device using a computer mouse (TCN183, T.C. STAR, New Taipei City, Taiwan, with dpi of 800). Our proposed system is embedded in the internal of the mouse and only the multi-PPG sensor of the PPG sensor module is placed on the top surface of the mouse as shown in
Figure 2 with 2 cm wide and 2 cm long. The PPG sensors 1, 2, 3, and 4 show the position of each PPG channel on the sensor, respectively.
2.3. Algorithm Development
In the previous hardware structure section, we have acquired the PPG signals from the hardware. All algorithms including the peak detection, pulse rate calculation, and digital signal processing are done in the microcontroller. In order to improve the quality of the PPG acquisition, we first input the PPG signal into a digital filtering process to eliminate the noise. Then, a waveforms mixing process is to emphasize the needed PPG signal to increase the accuracy of peaks detection and pulse rate calculation. The system flow of the designed process in this paper is depicted in
Figure 3.
2.3.1. Digital Filtering Process
After analog signal processing in the previous hardware structure section, the extracted PPG signal has become more obvious and much cleaner than before, for which the signal is small and noise contaminated. However, there is still some high frequency noise disturbing the PPG signal due to hardware structure. Therefore, a 40th-order low-pass FIR (Finite Impulse Response) filter with a cutoff frequency of 5 Hz is designed to reduce the high frequency noise.
2.3.2. Waveforms Mixing Process
Since blood vessels spreading in the palm are diverse, the difficulties of PPG detection may be increased according to the position of palm that contacts the PPG sensors. Therefore, this study proposes to expand the area of detection using the structure of multi-sensors, and then mix the signals of each channel to form a new PPG signal that improves the quality of the signal and makes the detection easier and more convenient.
Two methods are used as comparisons in our proposed design, the average, and the weighted average method, with the purpose of observing whether the weighted average method is a better way to improve signal quality than the average method.
Average Method: In order to get the average of the waveforms, four channels of the signal processed PPG signals will be summed and then divided by the number of channels. The signal
of each channel
i by each sample point
n are summed and then divided by the number of channels
i, which is four in this case, and carried out the result of the calculated mixed signal
as Equation (
1):
Weighted Average Method: In this method, the peak detection (from
Section 2.3.3) is applied in the beginning of each time instant and then the better quality signal channels will be emphasized by increasing their weights according to Equation (
2):
The decision of adjustment is described as three cases:
Increasing Weight: , the case of peak detected in the process of peak detection and the peak-to-peak interval (PPI) also satisfied the requested range. The decision weight of and represent the weights of the current and the previous sample point, respectively. Constant affects the weight of each signal point and is set to 2. The adjusted weight is constrained in the range of 2 to 20 in this study by empirical tests.
Decreasing Weight: , if peak is detected, but the requested range of PPI is not satisfied, the peak will be recognized as a noise and the weight will be decreased; if no peak is detected in the requested range of PPI, a decreasing weight is also applied.
Unchanged Weight: , otherwise stated in the previous two cases.
Therefore, a mixed waveform of four channels can be decided by two different methods above. According to the result
of each sample either in the average method Equation (
1), or the weighted average method Equation (
2), we can then combine all sample points from each method into a whole waveform
.
2.3.3. Peak Detection and Pulse Rate Calculation
We can observe from the PPG signal that each pulse raises rapidly. Thus, the slope value of PPG signals and dynamic thresholding are used to detect the peaks.
Figure 4 depicts the peak detection flow used in this study [
25]. In the beginning of the test, the Flag is set to false. When the slope value is larger than the threshold (Th), the Flag is set to true and this slope is recorded in the buffer. While the Flag is true and the slope value is smaller than or equals to 0, a peak index (peak position) is recorded. The maximum slope value from the buffer will be used to update the new threshold as adaptive thresholding and then clean the buffer to restart the peak detection.
A PPI check is then applied after peak detection to prevent noise contaminations. When the PPI is in the requested range, a peak is detected and finally outputs the pulse rate (
). The requested PPI range is set to
based on our previous work [
25]. The pulse rate is calculated as Equation (
3), where the
is determined by the peak-to-peak interval between the
nth and the
th peak and
is the sampling rate. The peak detection and pulse rate calculation procedure are referenced and adapted based on our previous research [
24,
25]:
4. Results and Discussion
4.1. Accuracy Evaluation and Comparison
In order to provide a comparison of the proposed algorithm, we compare the acquired PPG signal from our proposed device and the reference ECG signal from the reference device to evaluate the configuration parameters , , and .
Table 2 summarizes the results of the
,
, and
of each channel and the mixed signal under the rest environment for all subjects. The performance of the mixed signal from the average method has an
of
and an
of
, and the performance of the weighted average method has an
of
and an
of
. While the worst performance under the rest environment is the signal from
, it still has the
of
and a low
of
. Therefore, a high
and a low
of the proposed system is proved under the rest environment.
In the stages of slow movement and rapid movement, we have performed two directions of each stage: the horizontal and the vertical movement.
Table 3 and
Table 4 show a comparison among the results of the
(%) of each channel and the mixed waveform from the average method and the proposed method of every subject for the slow horizontal movement and the rapid horizontal movement, respectively. The average
of each channel and both mixed waveform methods of all subjects are also listed.
The waveforms of the rapid movement stage of each channel and the mixed signal by the average and the proposed weighted average method for subject 6 is shown in
Figure 7. As stated in the previous research [
26], a PPG peak comes after the ECG R-wave in the same cardiac cycle. We can obviously observe from each channel that there may be more than one peak appearing between the ECG R-peak and the following R-peak, which may affect the accuracy of detection. Moreover, the signal channels are motion contaminated due to a rapid movement of the mouse, and the peak detection result of the mixed signal by the average method is still affected by the fluctuations of motions, while the proposed weighted average method is almost close to the reference ECG signal. In the circle where some non-detected peaks of the average method occurred is due to the fact that the slope between the time period of 3 to 3.5 s does not exceed the maximum slope and leads to undetected peaks in the following few samples, while the weighted average method does amplify those points and detected the peaks.
Table 5 is the
(%) result of each channel and the mixed waveform from the average and the weighted average method of every subject for the browsing experiment. The combination results of each channel and both mixed waveform methods of all subjects are also listed. As can be observed, though the low
channels differ from subjects, the evaluated performance of the proposed weighted average method still results in lower failure of detection than the mixed signal of the average method. The waveform of the browsing stage for subject 9 is shown as
Figure 8. It is shown that the signal is disturbed by moving and clicking. Especially in
, the consecutive fluctuations in 0.5 to 2 s appear when clicking occurs. In the circle is the period where current PPI exceeds the requested range and therefore comes out as a non-detected peak in the weighted average method.
Table 6 and
Table 7 summarized the results of all experiment stages by the average method and the weighted average method of all subjects, respectively. Comparing the
of each method, we can observe that the proposed weighted average method has lower
than the average method. The proposed method also has higher
and
than the average method of all experiment stages, as it was known that a higher
is desirable to provide better prediction accuracy [
28]. The
is used to assess the repeatability, the precision, and the reproducibility of the methods, and a higher
is desirable to provide better performance [
29]. Therefore, it can be concluded from the results that a mixed waveform by the weighted average method has better performance including high accuracy and detection rate on peak detection than the average method.
4.2. Effect of Sensor Positioning
In the four stages of six movements, the better performance signal channel differs from each subject. However, after the process of waveform mixing by the proposed algorithm, the and the of the mixed signal maintain the best or the second best. These have proved that even different habits of mouse use will affect the position detected, and the structure of multi-sensor can increase the probability of signal extraction.
4.3. Effect of Moving and Clicking of the Computer Mouse
There are two important movements of mouse usage: moving and clicking. From the stage of slow movement and rapid movement in the previous section, more motion artifacts are found in rapid movement stage than in slow movement stage. Though the sensitivity and the detection rate are reduced for rapid movement stage either for the results of the proposed algorithm or the average method, the evaluated result still remains a high and a low .
In the browsing stage, the tester has to use the mouse to browse the electronic document that combines mouse clicking and moving, therefore causing the decreasing of the sensitivity as compared to the rest environment. The reason for the loss of sensitivity is that the palm may leave a distance from the sensor when clicking occurs; thus, unexpected peaks appear and the difficulties of detection also increase. From the result of browsing by the proposed method in
Table 5, failure of detection appears in most channels for almost all the subjects, but this issue is overcome with the proposed method, which will amplify the cleaner signal and reduce the effects of noise. The overall results of the proposed method in
Table 7 also show the
and the
of the detected PPG signal.
4.4. Eye Safety Concern and Power Saving
As optical measurement like PPG has been widely used nowadays, a growing concern about the possible use of LED arrays that might pose a potential threat has been derived in some studies [
30]. Concerning the eye safety related to the power of infrared LEDs when the palm is not covering the mouse, a power saving mode is designed in this study to detect whether the user’s palm is contacting the mouse or not. When no palm (pulse rate) is detected, the turn-on time of the LED lights will be reduced by lowering its modulation frequency. Therefore, the controls of the PPG sensors are added not only for power saving, but also for protecting our eyes.
4.5. Limitations of the Study and Future Works
There are still some limitations in this study. The limitation of the contact sensor is that the sensors must be contacted with the palm while doing the measurement. In the normal computer use, the signals recorded with the system can be relatively piecewise periods of PPG signals since the user often loosens his/her grip on the mouse when he/she is typing the text with a keyboard. Therefore, the PPG signals were recorded only when the user is gripping the mouse.
The results in the study are carried out by limits of young and healthy subjects; thus, in the future, more subjects from different ages (young to old) and more conditions, like small and large palms, people with poorer circulation, or people with arrhythmias will be tested.
Moreover, motion artifacts still negatively impact measurement accuracy. Therefore, how to determine the signal quality is important [
12,
31,
32]. In the results of the experiments, we found some sensitivity loss in the movement stages, which is due to the peaks being non-detected. Therefore, a more robust peak detection by event-related moving averages with dynamic threshold is considered [
13] to improve the performance of peak detection in the future.
5. Conclusions
A real-time pulse rate monitoring mouse with a multi-sensor structure has been proposed in this study, and a weighted average method to adjust the weight of the signal according to the quality of the channel, which increase the accuracy and the stability of PPG peak detection. The mixed waveform process by the proposed method emphasizes the cleaner signal, therefore reducing the disturbance of noise effectively.
Despite the performance results by the weighted average method of each subject possibly not being the best, the overall sensitivity and the failed detection rate of the processed signal outperform most single channels. The four experiment stages have proved that the proposed multi-sensor structure and the weighted average method for the mixed waveform process can raise the usability of the detected PPG signal efficiently and effectively, therefore increasing the probability of PPG signal detection on palms.
Since the proposed design is simple, easy-to-use, and low-cost, the device could be served as a measurement tool to collect physiological signals and play a major role in early detection of diseases. Therefore, the proposed design is suitable as a solution to overcome barriers to improve health care in low- and middle-income countries [
33].