Next Article in Journal
Optimizing Edge Detection Efficiency with a Grünwald–Letnikov Fractional Network
Previous Article in Journal
Stage-Aware Interaction Network for Point Cloud Completion
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Heart Pulse Transmission Parameters of Multi-Channel PPG Signals for Cuffless Estimation of Arterial Blood Pressure: Preliminary Study †

Institute of Measurement Science, Slovak Academy of Sciences, 84104 Bratislava, Slovakia
*
Author to whom correspondence should be addressed.
This study is an extended version of the paper entitled “Cuffless Estimation of Arterial Blood Pressure Based on Heart Pulse Transmission Parameters Determined from Multi-Channel PPG Signals” presented at the 6th International Conference on Advances in Signal Processing and Artificial Intelligence (ASPAI’ 2024), 17–19 April 2024, Funchal (Madeira Island), Portugal.
Electronics 2024, 13(16), 3297; https://doi.org/10.3390/electronics13163297
Submission received: 9 July 2024 / Revised: 12 August 2024 / Accepted: 19 August 2024 / Published: 20 August 2024
(This article belongs to the Section Circuit and Signal Processing)

Abstract

:
The paper describes a method developed for the indirect cuffless estimation of arterial blood pressure (ABP) from two/three-channel photoplethysmography (PPG) signals. It is important when the actual ABPs cannot be measured, e.g., during scanning inside a magnetic resonance imager. The proposed procedure uses heart pulse transmission parameters (HPTPs) extracted from the second derivative PPG signals. The linear regression method was used to calculate the relation between the determined HPTPs and the ABPs measured in parallel by a blood pressure monitor. The ABP values were estimated by the inverse conversion characteristic calculated from these linear relations. Three auxiliary investigations were performed first to find appropriate settings for PPG signal processing. We tested the accuracy of ABP estimation using two small corpora of multi-channel PPG records sensed during our previous experiments. We also analyzed the distribution of the determined HPTP values depending on the hand and gender for the mapping of a mutual relationship of HPTPs and measured ABPs. The final estimation errors were evaluated graphically (by correlation scatter plots and Bland–Altman plots) and numerically (by a correlation coefficient between the measured and estimated ABPs and by enumeration of the relative estimation error). The obtained results achieve acceptable mean values of −2.6/−3.5 mm Hg for systolic/diastolic ABPs.

1. Introduction

Persons examined in a magnetic resonance imager (MRI) are exposed to noise and vibration, causing them stress that manifests mainly by heart rate (HR) and arterial blood pressure (ABP) changes [1]. The HR changes can be detected from a photoplethysmography (PPG) signal, while the systolic or diastolic blood pressure (SBP or DBP) values are measured by a blood pressure monitor (BPM) [2]. However, this type of a measurement arrangement is less comfortable for tested persons, and it causes problems with practical realization of experiments. It means that repeated squeezing of the arm by a pressure cuff during the whole experiment increases the negative stress and can induce a distortion of measured signals and subsequently determined parameters in the final effect. The practical difficulty of this type of measurement follows from the fact that the BPM device cannot be directly used for measurement inside the scanning area of a running MRI device due to interaction with a working magnetic field and a strong radiofrequency (RF) disturbance. The pressure cuff originally also includes a metal brace, which must be exchanged by another one from copper or brass material. Therefore, many researchers try to dispense with this type of direct blood pressure measurement approach and to use a cuffless method based on an estimation of ABP values from a PPG signal. It is well known that the second-derivative PPG wave (SD-PPG) [3] consists of five areas corresponding to the time-domain features [4]. These parameters can be calculated by a linear regression using the mean square method [5,6], nonlinear regression algorithms [7], or least squares support vector machine algorithm [8]. The most recent methods based on one-channel PPG signals use the pre-trained artificial neural networks for the continual estimation of the systolic blood pressure [9]. This approach enables creating more compact wearable devices for the monitoring of sporting and other human activities or the control of different patients’ vital parameters in a hospital environment.
The precision of ABP estimation may be improved by the method based on heart pulse transmission parameters (HPTPs). Originally, the pulse transmission time (PTT) represented the time difference between the R peak of the electrocardiogram (ECG) and the systolic peak of the PPG signal. Due to a linear relationship between PTT and ABP [10,11,12], the estimation of the blood pressure using the PTT is more accurate than using the PPG alone [13,14]. The PTT can also be determined from two or more PPG waves picked up by sensors placed at the defined distance Dx [7,15]. Another parameter describing the current state of the cardiovascular system is the pulse wave velocity (PWV) [10,15,16,17]. In the current work, we also show the usefulness of the derived parameters: relative PTT (rPPT) and relative PWV (rPWV).
In general, an optical PPG sensor can work on the transmission or reflection principle. The transmission type of a sensor probe has usually the form of a finger ring or clip with a light source (one or more LED elements) and a photodetector placed on opposite sides of the sensed human tissue—this type is often used in oximeters. The reflection type of an optical sensor is placed mainly on fingers or a wrist, and it is typically fixed by an elastic/textile ribbon or integrated as a part of wearable devices—fitness bracelets, smart watches, etc. In this case, the photodetector measures the intensity of the light reflected from the skin, and it is placed on the same side of the skin surface as the light source transmitter. Multi-channel PPG signals picked up simultaneously by sensors mounted on different parts of hands or legs [18,19] are successfully used for this task. In continuous long-term screening and monitoring, sampling at lower frequencies may be used, e.g., 128 Hz in a wrist-based wearable device for atrial fibrillation detection [20]. Special wearable smart devices used for the monitoring of multiple vital parameters contain sensors for the continual acquisition of ECG and PPG signals [21,22]. Commercial wearable PPG sensors use typical sampling frequencies between 50 and 100 Hz [23]. The higher sampling rate (typically ≥ 500 Hz) is justified for analysis of the systolic pulse with higher accuracy, and it is also important for the precise determination of PTT and PWV parameters.
For evaluation and testing of the stability of the ABP estimation method, the PPG corpora are often used. The well-known PPG signal database MIMIC-II [24] consists of collected multi-parameter records (PPG, ABP, RESP, and so on) taken from patients located in an Intensive Care Unit (ICU). The main problem of this database is that it includes only one-channel PPG signals, so the PTT as well as the PWV parameters cannot be determined. In addition, the PPG data are sampled at 125 Hz, and raw signals are partially corrupted. Due to manipulation and other operations performed in real conditions of the ICU, we cannot use this corpus to evaluate the performance of the currently developed ABP estimation method. Other open-access datasets, such as WESAD (Wearable Stress and Affect Detection) [25] or PPG-BP [26], have the same limitation of one-channel PPG signals only. Therefore, for purpose of this work, two small corpora of PPG records collected in the frame of our previous research [27,28] were finally used.
The main motivation of this work was to test whether these HPTPs are suitable for ABP estimation and whether they give sufficient estimation accuracy. This paper describes the procedure for HPTPs determination from the preprocessed two/three-channel SD-PPG signals. The linear regression method is used to perform ABP estimation from the HPTPs. At first, auxiliary investigations were performed with the aim of finding appropriate settings for PPG signal processing. They include an analysis of (1) the effect of different fS on the precision of the determined HPTPs; (2) the influence of the signal filtering effect on PPG wave properties; and (3) the simulation of consequences of the absolute measurement error of Dx distance on the precision of the PWV parameter. Next, the distribution analysis of the determined HPTPs depending on the type of the hand (left/right) and gender (males/females) was realized to map the mutual relationship between HPTPs and the measured SBP, DBP, and HR values. The main estimation experiment consists of testing multi-channel PPG records. The relative estimation error (REE) is evaluated to verify the estimation accuracy of SBP or DBP values. The partial results determined separately from the two used databases, types of hand, and genders of the testing subjects were compared numerically. Final estimation errors for both databases together were graphically evaluated by the scatter plots mapping correlations between the measured and estimated ABP values and using Bland–Altman plots.

2. Methods

2.1. PPG Wave Processing and HPTPs Determination

The picked-up PPG signal has a typical amplitude modulation with a partially linear trend (LT), and it usually contains a superimposed noise component, or it is partially disturbed or degraded. Therefore, the sensed raw PPG signal must be pre-processed for further use. In the frame of de-trending operation, the LT is calculated by the mean square method, and the LT removal procedure is applied. Then, the de-trended PPG signal is smoothed by a moving average (MA) filter with constant coefficients given by the relation
y i = 1 2 N X + 1 j = N X N X x i + j ,
where x(i)/y(i) is the i-th input/output sample of the signal and NX is half the length of the MA window.
Analysis of the pre-processed PPG signal starts with localization of the systolic peaks PSYS and determination of the pulse amplitude (Ap) and the pulse width (at one half of the amplitude) Wp—see an example in Figure 1.
Next, the heart pulse periods THP (in samples) are determined to calculate the heart rate HR in beats per minute (bpm) for the sampling frequency fS as
HR = 60 × fS/THP [bpm].
The PTT and other derived parameters are then calculated from the difference ∆ PSYS in samples between adjacent PSYS positions of two/or more PPG waves (see an example in Figure 2). Using the sampling frequency fS in kHz, the PTT in ms is determined as
PTT = ∆ PSYS/fS [ms].
The pulse wave velocity represents the relationship between the PTT, and Dx denotes the distance between the sites of PPG sensors
PWV = Dx/PTT [m/s].
The relative parameter rPTT is defined as a percentual ratio invariant on the current HR value
rPTT = (PTT/THP) × 100 [%].
Similar to (5), the relative PWV is defined as
rPWV = Dx/rPTT [%].
HPTPs determined in this way are then statistically processed to obtain representative values (one per the whole analyzed PPG signal record) for further use in the estimation process. The simplest method for calculation of the representative final value VFINAL is to use the mean value (VMEAN). However, the probability distribution of the analyzed parameter is often non-Gaussian, so the mean value will not provide a correct result. In this case, it is better to determine the value VHMAX corresponding to the maximum occurrence oMAX [%] in the histogram; however, oMAX must be relatively high (typically more than 25% of the amplitude). Otherwise, better precision is achieved by calculation using the mean method
V F I N A L = V H M A X   for   o M A X   25   % V M E A N   for   o M A X < 25   % .
The whole process of creation of a database of the HPTPs from the PPG waves is described by the block diagram in Figure 3.

2.2. ABP Values Estimation Using HPTPs from Multi-Channel PPG Signals

The proposed system for the estimation of SBP and DBP values is based on using the database of HPTPs from the pre-processed multi-channel SD-PPG signal records together with the corresponding measured heart rate (HRBPM) and BP values (BPBPM)—see the block diagram in Figure 4. In the first step, the linear regression method is applied to find a linear relation between the determined heart pulse transmission parameters and the HRBPM and BPBPM values measured in parallel. Then, the inverse conversion characteristic is calculated to estimate ABP values. The partial results of estimated SBP or DBP values per a used type of HPTP are next processed in the comparison block. Outlying values are eliminated here using limits of minimum (SBPMIN and DBPMIN) and maximum (SBPMAX and DBPMAX) ABPs in mmHg. These limits are derived from the ranges of real measured SBP or DBP values stored in the processed database. The final output SBP or DBP values are calculated for the entire tested PPG signal as medians of the remaining estimated values, satisfying the minimum/maximum limit conditions for the given parameter. For the evaluation of performance of the proposed estimation system, the SBP or DBP values estimated from the testing PPG signal records are numerically matched with the known ABP values. The graphical comparison based on scatter plots between the measured and estimated ABPs as well as the Bland–Altman plots are performed.
To enumerate the overall estimation accuracy, the percentage relative estimation error parameter is calculated as
REEABP = (ABPEST − BPBPM)/BPBPM × 100 [%]
where ABPEST represents the estimated SBP or DBP and BPBPM is the real value measured. The obtained accuracy is compared with other related works using the mean absolute error (MAE) parameter calculated as the mean of a simple absolute difference ∆ ABP = ABPEST − BPBPM in mmHg. For the mapping of correlations between the measured and estimated ABP values, we used scatter plots. For this purpose, the Pearson correlation coefficient R can be calculated as
R X , Y = 1 N 1 n = 1 N X n μ X / σ X Y n μ Y / σ Y ,
where X represents the vector of measured ABPs and Y is the vector of estimated ABP values, while µ and σ denote the mean and standard deviation of the input vectors X and Y consisting of N elements.

3. Materials, Experiments, and Results

3.1. Description of the Used PPG Signal Corpora

Two small PPG signal databases were used in this work: the PPG corpus DB1 consists of two-channel PPG signals originated from ten persons (7 male and 3 females, aged from 22 to 60 years—further called ”M11–M17” and “F11–F13”) [27]. The PPG corpus DB2 contains three-channel PPG signals from 12 subjects (8 males and 4 females, with a mean age of 50 years—next called ”M21–M28” and “F21–F24”) [28]. All tested persons are right-hand dominant, non-smoker volunteers—the authors themselves and colleagues from the IMS SAS in Bratislava. As our institute holds no approval for patient examination in standard medical practice, we are oriented only on non-clinical research.
In the case of two-channel PPG records included in the DB1 corpus, the first optical PPG sensor was always placed on the wrist artery, and the second one was worn successively on each of the fingers of the left/right hand (P1–P5)—see a principal arrangement photo in Figure 5. In this way, the number of possible sequences of HPTPs determined per person was 10 (5 × 2), and for all persons, we obtained the total amount of 100 HPTP sequences. For the three-channel PPG records stored in the database DB2, the first reflectance optical sensor was again placed on a wrist, the second one was worn on a forefinger (P4), and the transmittance sensor was put on a pinkie (P1). In this case, we obtained 2 × 2 = 4 sequences of HPTPs per person and 12 × 4 = 48 HPTP sequences in total.
All PPG signal records were picked up using the sampling frequency equal to 1 kHz. The sensing of PPG signals was accompanied by the parallel measurement of BP/HR values using the automatic BPM device Microlife BP A150-30 AFIB by Microlife AG, Swiss Corporation, Widnau, Switzerland. To prevent any possible negative influence of an inflated pressure cuff of the BPM on a tested person’s blood system, the PPG signal was picked up from the fingers of the opposite hand. A typical duration of each PPG signal record was 64 s, so about 60 ÷ 80 PPG cycles can be localized, and a similar amount of PPG features can be determined (the first and the last cycles are ignored). The actual number of PPG cycles that depends on the current HR was always sufficient to obtain stable and credible statistical results necessary for the final success of the whole estimation process.
While Dx distances are schematically shown in Figure 5 by lines, in fact, we tried to measure the length of the path between the beginning point (at the wrist W) and the end points (fingers P1–5) along the blood arteries by a flexible meter. The distances WP1–5 of a hand were measured for each of the tested persons before the PPG signal sensing started. The methodology applied follows from the fact that the blood really flows in this way in a measured hand. These measurement results were very individual-dependent, so Dx was finally measured with the maximum accuracy of 5 mm. As follows from the detailed analysis performed earlier [27], the absolute measurement error of Dx distances affected the relative percentage error of the calculated pulse wave velocity. The obtained results show that the caused inaccuracy of the PWV values is detectable—it can bring the REE error up to 3% in the worst case: using the shortest finger P5 corresponding to the smallest PTT value of about 15 ms. Therefore, it can be stated that this inaccuracy of manual Dx measurement has no essential effect on the final estimation precision.
PPG signals were picked up with the help of two special prototypes of wearable PPG sensors using Arduino compatible micro-controller boards [29] based on the processor ATmega328 by Atmel Company, San Jose, CA, United States, with integrated 10-bit A/D converters. The two-channel realization consists of two reflectance sensors with fully integrated analogue interfaces: the Crowtail-Pulse Sensor ER-CT010712P by Elecrow Company, Shenzhen, China and the Gravity Heart Rate Sensor (SEN0203) by Zhiwei Robotics Corp., Shanghai, China. The second prototype, enabling a three-channel sensing of PPG signals, consists of one transmittance PPG optical pulse sensor Easy Pulse Mikro (designed by Embedded Lab, Williamsburg, VA, USA, including ER-CDE17527M by Kyoto Electronics Manufacturing Co., Ltd., Kyoto, Japan) and two PPG optical sensors of Pulse Sensor Amped (Adafruit 1093) type, by Adafruit Industries, NY, USA, working in reflectance mode. In both prototypes, the data transfer to the control device (laptop, tablet, etc.) was realized via a Bluetooth (BT) serial connection working in the 4.1 standard at 4.2 GHz. For long-term measurement experiments, both sensors were powered from the 5 V power bank via the USB port. For the short-term measurement of PPG signals, it was also possible to use a 3.7 V rechargeable polymer–lithium–ion (Li-Po) cell to power the sensor. For further experimental measurements inside the scanning area of the open-air MRI device, all parts of the PPG sensors are fully shielded in aluminum boxes and assembled from elements of non-ferromagnetic materials [28].
The PPG signal processing and implementation of the whole estimation algorithm were realized in the Matlab environment (ver. 2019a).

3.2. Performed Analysis, Experiments, and Results

To find proper parameter settings for the processing of multi-channel PPG signals, the following auxiliary investigations were performed:
Mapping of possible dependence of precisions of PPG wave parameters on the sampling frequency—see the numerical comparison in Table 1:
  • Analysis of influence of the applied sampling frequency on the detected heart periods THP,PSYS in samples, and PTT in ms for fS = {125, 250, 500, 1000, and 2000} Hz;
  • Comparison of the relative percentage error of ∆ PSYS (RE3PSYS) and the absolute error of PTT (AE3PTT) for the maximum considered absolute error of ∆ PSYS determination equal to three samples (AE3PSYS).
Analysis of the MA filtering effect on the PPG signal properties (the systolic pulse amplitude Ap and width Wp, heart pulse periods THP) for half the window length NX = {0, 4, 8, 12, 16, and 32} samples—see visualization for the PPG wave in Figure 6 (NX = 0 represents no filtering).
Before practical use of the PPG signal corpora, the distribution analysis of the determined HPTPs in dependence on the hand (left/right) and the gender (male/female) must be realized—see the graphical comparison in Figure 7. The first analysis of mutual positions of the SBP/DBP values measured by the BPM device and HPTPs was already performed in our previous work [27]. In the frame of the current work, a more complex mapping of mutual positions of HPTPs and measured SBP, DBP, and HR values was performed as documented by the graphs in Figure 8. Finally, the results of distribution of HR values depending on the hand used (left/right/both) together with the correlation coefficient R between the measured HRBPM and determined HRPPG values are depicted in four graphs in Figure 9.
The main estimation experiment consists of testing the PPG records from DB1 and DB2 and evaluating the partial as well the summary estimation accuracy. The evaluation was performed numerically (by the mean REE values per HPTP type) and also graphically (using the correlation scatter plots and Bland–Altman plots). Table 2 compares the numerical results of the obtained REE separately for the databases DB1 and DB2, for particular HPTPs, and for all parameters together. Table 3 shows correlation between the measured and estimated SBP or DBP values—separately for the databases DB1 and DB2, the type of the hand, and the gender of the testing subject. Figure 10 contains the scatter plots showing the correlation obtained between the measured (SBPBPM or DBPBPM) and estimated (SBPEST or DBPEST) values together with the calculated coefficient R. Figure 11 presents the Bland–Altman plots of the final estimation accuracy for SBP or DBP using data of both databases together and all tested persons. Finally, the accuracy of the proposed estimation algorithm is checked from the viewpoint of the Association for the Advancement of Medical Instrumentation (AAMI) and the British Hypertension Society (BHS) standard (without using the grading criteria) [30]. A numerical comparison of the achieved MAE and SD values is presented in Table 4. The matching of obtained results using three categories of absolute difference ∆ ABP values (5 mmHg, 10 mmHg, and 15 mmHg) by the BHS methodology is shown in Table 5.

4. Discussion

In practice, the sampling frequency fS of about 150 Hz is sufficient for sensing of the PPG signals used for the determination of HR values [31] and basic features of PPG waves (heart pulse amplitude and range, signal ripple, etc.). Our first auxiliary experiment confirmed that higher fS values (at least 1 kHz) should be applied to obtain correct and precise HPTP values. Otherwise, the distances ∆ PSYS determined in samples can be too short (see values in the first line in Table 1), which causes high errors in the time domain (for three-sample inaccuracy, it is up to 24 ms in the case of fS = 125 Hz, while it is only 1.5 ms for fS = 2 kHz). This effect has also a great influence on the accuracy stability of the determined PPT and other derived parameters.
The second auxiliary analysis demonstrates the “integration” effect of the MA filter applied on the smoothed PPG signal. The graphical form of the obtained results shows that increasing the length 2 NX +1 of the MA filter window has a direct influence on the recorded raw PPG signal: a too high value of the NX parameter causes an undesirable effect on the amplitude Ap and the width Wp of systolic pulses—see Figure 6. On the other hand, smoothing of the PPG wave by the MA filter has also some effect on the position and sharpness of the systolic pulse PSYS, which may result in small fluctuations of the determined heart pulse period THP and the subsequently determined distances ∆ PSYS. Therefore, to obtain a properly smoothed PPG signal without any secondary negative influence, the window duration and the sample rate are set by a compromise. The settings finally chosen for PPG signal processing in the main ABP estimation experiments were NX = 32 and fS = 1 kHz. In accordance with the results of the previously performed analysis of a limited precision of manually measured distances Dx [27], this effect on the inaccuracy of PWV and rPWV values can be omitted for the purpose of this work.
Prior to practical experiments with the PPG signal corpora, the distribution analysis of determined HPTPs in dependence on the finger (P1–P5), the type of the hand (left/right), and the gender (males/females) was realized. The graphical results in Figure 8 are divided into two set of graphs, separately for male and female tested persons. In each of the graph pairs (Figure 8b–d), there are visible differences between HPTPs determined from the PPG signals sensed on the left and the right hand. There are also detectable smaller differences between male and female subjects, which are primarily caused by the mentioned higher length of fingers (see the first graph pair in Figure 8a). The next comparison analysis—the mapping of mutual positions of HPTPs with ABP and HR values—also shows a data grouping effect depending on the position of PPG signal sensing and type of tested subjects (male/female)—see two set graphs for SBP and DBP parameters in Figure 9. For this reason, the results of the main estimation experiments should be selected and analyzed depending on the gender and the type of the hand. The last preliminary analysis was aimed at a comparison of the correlation between the HR values measured by the BPM device and those determined from the PPG signal—see Figure 10. The distribution of HR values separately for left/right hands and summary results for both hands as well as the corresponding correlation coefficient R show minimal differences. With respect to the determined minimum areas of 95% confidence bounds, the best result (maximum R coefficient) was obtained for HR values from the left hands—compare plots in Figure 9b–d. Based on this previous analysis of the range of SBP or DBP values determined from the PPG signal database, the minimum/maximum limits were set as follows: SBP MIN = 80 mmHg, SBP MAX = 150 mmHg; DBP MIN = 50 mmHg, DBP MAX = 105 mmHg. On the other hand, the conversion characteristic used for ABP values estimation is created by a linear regression approach (where the conversion line is defined by an origin in Cartesian coordinates and a direction); it can be also extrapolated for the testing of PPG signals with ABP values lying outside these chosen intervals. However, in this case, we may expect a significantly increased estimation error of SABP/DABP values.
From the point of view of the obtained ABP estimation results, the numerical comparison in Table 2 shows negative REE (ABPEST < BPBPM) and low standard deviation (less than 3%) for DB1. For DB2, REE is positive, with a higher standard deviation (more than 10%). In both databases, the estimation errors are higher for DBP values. In both cases, the HPTPs were taken from the joined left and right hands from all persons involved in each of the analyzed databases. Detailed results of the correlation between the measured and estimated SBP and DBP values depending on the used database (DB1 or/and DB2), the type of the hand, and the gender of the testing subject are enumerated in Table 3. It shows a better correlation for HPTPs from the joined left and right hands and all tested subjects: the estimated SBP values have always lower R values than the DBP ones, which is probably due to the larger range of pressure values. The higher correlation between the measured and estimated BPs for DB1 than for DB2 in Table 3 is in correspondence with the lower SD of REE for DB1 (compare Table 2).
Figure 10 and Figure 11 represent the final results for data of all tested persons from both databases together. The scatter plot graphs confirm the partial analysis results—showing higher variance of the estimation error in the case of SBP values (the summary correlation coefficient R is lower than for DBP values). Also, the Bland–Altman plots show lower standard deviation values for the estimated DBP values, but the situation with mean values is opposite: SBP values were finally estimated with the lower MAE. For both parameters, the mean ∆ SBP/DBP values have a negative character. It is in correspondence with the observed trend of REE values for the database DB1. The DB1 contains more PPG records, so its influence outweighed the DB2 having positive REE values (compare the numerical results in Table 2). In summary, the MAE values obtained from our current work are principally comparable with the results of other cuffless BP estimation systems using PPT and/or PWV parameters as documented the Table 6.

5. Conclusions

The performed experiments confirm the practical functionality of the proposed method for the estimation of SBP and DBP based on heart pulse transmission parameters determined from multi-channel SD-PPG waves. The final mean estimation errors achieved using the merged DB1 and DB2 databases were MAESBP = −2.6 ± 10.7, MAEDBP = −3.5 ± 9.7 mmHg. Therefore, the MAE results practically fulfill the AAMI recommendation, but in the case of the required standard deviation, the limit of the maximum equal to 8 mmHg was exceeded. According to the BHS standard, the developed estimation algorithm works with an absolute error Δ ABP lower than 5 mmHg in 65.1/47.8 percentage cases of SBP/DBP values, 93.3/100% were predicted with Δ ABPs lower than 10 mmHg, and finally 100% were predicted with Δ ABPs lower than 15 mmHg, as documented in Table 5. Generally, it holds that the evaluated algorithm is considered as acceptable if the absolute difference ∆ ABP (difference between the estimated and the real ABP value) is less than 10 mmHg [30]. Hence, these results can be regarded as promising for this first-step experiment as well as in comparison with other cuffless BP estimation systems in Table 6. The second merit of this work is related with our long-term research aim. The currently performed experiments confirmed that multi-channel PPG signals sensed by special prototypes of optical sensors in the low magnetic field environment with a high RF disturbance present inside the scanning area of a running MRI device are fully usable for this purpose and can bring an applicable substitution of direct ABP measurement by a blood pressure device.
However, further improvements are necessary before practical usage of the proposed estimation method. We will try to eliminate the limited precision of manually measured distances Dx with the aim of minimizing the possible effect on the inaccuracy of PWV and rPWV values. It could be achieved by using another (semi-automatic) approach based on the determination of distances from images. A similar technique is used for the tracing of human movement when the kinetic sensors placed on different parts of a body (arm or leg) are covered by small reflexing targets to be easily sensed and traced by cameras [32]. Next, it is well known that the quality of the sensed PPG signals depends essentially on the actual state of the skin at the place of an optical sensor. It means the color of the skin surface together with the temperature, humidity, and the pressure effect of the worn sensor influence the features of the sensed PPG signals. Therefore, in the near future, we plan to perform more measurement experiments with the aim of collecting another multi-channel PPG signal database including also the attached temperature, humidity and pressure values measured by a contact method.
Finally, the basic limitation of the current work consists of a relatively small number of processed PPG signal records (taken from the databases DB1 and DB2). Next, the processed ranges of SBP and DBP values were relatively close: SBP ∊ <94~154> mmHg, and DBP ∊ <57~92> mmHg. These close ranges have a positive effect on the robustness (stability) and accuracy of the estimation procedure when the ABP values of the tested PPG signal lie in these intervals. Otherwise, if the SBP or DBP values of the tested PPG signals are outside these ranges, the estimation procedure also works, but it produces a higher estimation error. To eliminate this negative effect on the accuracy of the whole estimation process, we must collect one larger database accompanying the measured SBP and DBP values in a wider range. It is also necessary for the better practical applicability of this developed method to a broader population with more variable ABP values. Another way to solve this problem is to establish some cooperation with the nearest medical centers in Bratislava (Slovakia), Vienna (Austria), or Brno (Czech Republic).

Author Contributions

Conceptualization and methodology, J.P., A.P. and I.F.; data collection, processing, and visualization, J.P.; writing—original draft preparation, J.P. and A.P.; writing—review and editing, A.P.; project administration, J.P. and I.F.; funding acquisition, J.P. All authors have read and agreed to the published version of the manuscript.

Funding

This work was funded by the Slovak Scientific Grant Agency project VEGA2/0004/23.

Informed Consent Statement

Not applicable.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors on request.

Acknowledgments

We would like to thank all our colleagues who participated in the PPG signal recording as well as the BP and HR measurement experiments.

Conflicts of Interest

The authors declare no conflicts of interest.

Abbreviations

The following abbreviations are used in this manuscript:
MRIMagnetic resonance imager
HRHeart rate
ABPArterial blood pressure
PPGPhotoplethysmography
SBPSystolic blood pressure
DBPDiastolic blood pressure
BPMBlood pressure monitor
RFRadiofrequency
SD-PPGSecond-derivative PPG
HPTPHeart pulse transmission parameter
PTTPulse transmission time
ECGElectrocardiogram
PWVPulse wave velocity
rPTTRelative pulse transmission time
rPWVRelative pulse wave velocity
ICUIntensive Care Unit
REERelative estimation error
LTLinear trend
MAMoving average
MAEMean absolute error
BTBluetooth
AAMIAssociation for the Advancement of Medical Instrumentation
BHSBritish Hypertension Society

References

  1. Steckner, M.C. A review of MRI acoustic noise and its potential impact on patient and worker health. eMagRes 2020, 9, 21–38. [Google Scholar] [CrossRef]
  2. Celka, P.; Charlton, P.H.; Farukh, B.; Chowienczyk, P.; Alastruey, J. Influence of mental stress on the pulse wave features of photoplethysmograms. Healthc. Technol. Lett. 2020, 7, 7–12. [Google Scholar] [CrossRef] [PubMed]
  3. Nitzan, M.; Ovadia-Blechman, Z. Physical and physiological interpretations of the PPG signal. In Photoplethysmography: Technology, Signal Analysis, and Applications; Kyriacou, P.A., Allen, J., Eds.; Elsevier: London, UK, 2022; pp. 319–339. [Google Scholar] [CrossRef]
  4. Liu, M.; Po, L.M.; Fu, H. Cuffless blood pressure estimation based on photoplethysmography signal and its second derivative. Int. J. Comput. Theory Eng. 2017, 9, 202–206. [Google Scholar] [CrossRef]
  5. Kachuee, M.; Kiani, M.M.; Mohammadzade, H.; Shabany, M. Cuff-less high-accuracy calibration-free blood pressure estimation using pulse transit time. In Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS), Lisbon, Portugal, 24–27 May 2015; pp. 1006–1009. [Google Scholar] [CrossRef]
  6. Slapnicar, G.; Luštrek, M.; Marinko, M. Continuous blood pressure estimation from PPG signal. Informatica 2018, 42, 33–42. [Google Scholar]
  7. Mousavi, S.; Firouzmand, M.; Charmi, M.; Hemmati, M.; Moghadam, M.; Ghorbani, Y. Blood pressure estimation from appropriate and inappropriate PPG signals using a whole-based method. Biomed. Signal Process. Control 2019, 47, 196–206. [Google Scholar] [CrossRef]
  8. Zhou, Y.P.; Jiang, J.H.; Lin, W.Q.; Xu, L.; Wu, H.L.; Shen, G.L.; Yu, R.Q. Artificial neural network-based transformation for nonlinear partial least-square regression with application to QSAR studies. Talanta 2007, 71, 848–853. [Google Scholar] [CrossRef] [PubMed]
  9. Casadei, B.C.; Gumiero, A.; Tantillo, G.; Della Torre, L.; Olmo, G. Systolic Blood Pressure Estimation from PPG Signal Using ANN. Electronics 2022, 11, 2909. [Google Scholar] [CrossRef]
  10. McCarthy, B.M.; O’Flynn, B.; Mathewson, A. An Investigation of Pulse Transit Time as a Non-Invasive Blood Pressure Measurement Method. J. Phys. Conf. Ser. 2011, 307, 012060. [Google Scholar] [CrossRef]
  11. Arza, A.; Lázaro, J.; Gil, E.; Laguna, P.; Aguiló, J.; Bailon, R. Pulse transit time and pulse width as potential measure for estimating beat-to-beat systolic and diastolic blood pressure. In Proceedings of the IEEE Computing in Cardiology, Zaragoza, Spain, 22–25 September 2013; pp. 887–890. [Google Scholar]
  12. Johnson, M.; Jegan, R.; Mary, X. Performance measures on blood pressure and heart rate measurement from PPG signal for biomedical applications. In Proceedings of the IEEE International Conference on Innovations in Electrical, Electronics, Instrumentation and Media Technology, Coimbatore, India, 3–4 February 2017; pp. 311–315. [Google Scholar] [CrossRef]
  13. Cattivelli, F.S.; Garudadri, H. Noinvasive cuffless estimation of blood pressure from pulse arrival time and heart rate with adaptive calibration. In Proceedings of the 6th International Workshop on Wearable and Implantable Body Sensor Networks, Berkeley, CA, USA, 3–5 June 2009; pp. 114–119. [Google Scholar] [CrossRef]
  14. Xu, Y.; Zhang, W.; Wang, D.; Ping, P. A calibration method for cuffless continue blood pressure measurement using Gaussian normalized pulse transit time. In Proceedings of the 2018 IEEE International Instrumentation and Measurement Technology Conference (I2MTC), Houston, TX, USA, 14–17 May 2018; pp. 1–5. [Google Scholar] [CrossRef]
  15. Wang, Y.J.; Chen, C.H.; Sue, C.Y.; Lu, W.H.; Chiou, Y.H. Estimation of blood pressure in the radial artery using strain-based pulse wave and photoplethysmography sensors. Micromachines 2018, 9, 556. [Google Scholar] [CrossRef] [PubMed]
  16. Gesche, H.; Grosskurth, D.; Küchler, G.; Patzak, A. Continuous blood pressure measurement by using the pulse transit time: Comparison to a cuff-based method. Eur. J. Appl. Physiol. 2012, 112, 309–315. [Google Scholar] [CrossRef] [PubMed]
  17. Mishra, B.; Thakkar, N. Cuffless blood pressure monitoring using PTT and PWV methods. In Proceedings of the International Conference on Recent Innovations in Signal Processing and Embedded Systems (RISE), Bhopal, India, 27–29 October 2017; pp. 395–401. [Google Scholar] [CrossRef]
  18. Padilla, J.; Berjano, E.J.; Sáiz, J.; Fácila, L.; Díaz, P.; Mercé, S. Assessment of relationships between blood pressure, pulse wave velocity and digital volume pulse. In Proceedings of the IEEE 2006 Computers in Cardiology, Valencia, Spain, 17–20 September 2006; pp. 893–896. [Google Scholar]
  19. Wang, L.; Lo, B.P.; Yang, G.Z. Multichannel reflective ppg earpiece sensor with passive motion cancellation. IEEE Trans. Biomed. Circuits Syst. 2007, 1, 235–241. [Google Scholar] [CrossRef] [PubMed]
  20. Bonomi, A.G.; Schipper, F.; Eerikäinen, L.M.; Margarito, J.; Van Dinther, R.; Muesch, G.; De Morree, H.M.; Aarts, R.M.; Babaeizadeh, S.; McManus, D.D.; et al. Atrial fibrillation detection using a novel cardiac ambulatory monitor based on photo-plethysmography at the wrist. J. Am. Heart Assoc. 2018, 7, e009351. [Google Scholar] [CrossRef] [PubMed]
  21. Randazzo, V.; Ferretti, J.; Pasero, E. A Wearable Smart Device to Monitor Multiple Vital Parameters—VITAL ECG. Electronics 2020, 9, 300. [Google Scholar] [CrossRef]
  22. Kim, J. Simultaneous Voltage and Current Measurement Instrumentation Amplifier for ECG and PPG Monitoring. Electronics 2021, 10, 679. [Google Scholar] [CrossRef]
  23. Charlton, P.H.; Marozas, V. Wearable photoplethysmography devices. In Photoplethysmography: Technology, Signal Analysis, and Applications; Kyriacou, P.A., Allen, J., Eds.; Elsevier: London, UK, 2022; pp. 401–438. [Google Scholar] [CrossRef]
  24. Saeed, M.; Villarroel, M.; Reisner, A.T.; Clifford, G.; Lehman, L.W.; Moody, G.; Heldt, T.; Kyaw, T.H.; Moody, B.; Mark, R.G. Multiparameter intelligent monitoring in intensive care II (21): A public-access ICU database. Crit. Care Med. 2011, 39, 952–960. [Google Scholar] [CrossRef]
  25. WESAD: Multimodal Dataset for Wearable Stress and Affect Detection|Ubiquitous Computing. Available online: https://www.eti.unisiegen.de/ubicomp/home/datasets/icmi18/index.html.en?lang=en (accessed on 3 March 2024).
  26. Liang, Y.; Chen, Z.; Liu, G.; Elgendi, M. A New, Short-Recorded Photoplethysmogram Dataset for Blood Pressure Monitoring in China. Sci. Data 2018, 5, 180020. Available online: https://figshare.com/articles/PPG-BP_Database_zip/5459299 (accessed on 5 July 2023). [CrossRef] [PubMed]
  27. Přibil, J.; Přibilová, A.; Frollo, I. Analysis of heart pulse transmission parameters determined from multi-channel PPG signals acquired by a wearable optical sensor. Meas. Sci. Rev. 2023, 23, 217–226. [Google Scholar] [CrossRef]
  28. Přibil, J.; Přibilová, A.; Frollo, I. Triple PPG sensor for measurement of heart pulse transmission parameters in weak magnetic field environment. In Proceedings of the 14th International Conference on Measurement, Smolenice, Slovakia, 29–31 May 2023; pp. 67–70. [Google Scholar] [CrossRef]
  29. Smith, A.G. Introduction to Arduino: A Piece of Cake; CreateSpace Independent Publishing Platform: North Charleston, SC, USA, 2011; ISBN 1463698348. [Google Scholar]
  30. Stergiou, G.S.; Alpert, B.; Mieke, S.; Asmar, R.; Atkins, N.; Eckert, S.; Frick, G.; Friedman, B. A universal standard for the validation of blood pressure measuring devices: Association for the Advancement of Medical Instrumentation/European Society of Hypertension/International Organization for Standardization (AAMI/ESH/ISO) Collaboration Statement. Hypertension 2018, 71, 368–374. [Google Scholar] [CrossRef]
  31. Macfarlane, P.W.; van Oosterom, A.; Pahlm, O.; Kligfield, P.; Janse, M.; Camm, J. Comprehensive Electrocardiology, 2nd ed.; Springer-Verlag London Limited: London, UK, 2011. [Google Scholar] [CrossRef]
  32. Oh, J.; Eltoukhy, M.; Kuenze, C.; Andersen, M.S.; Signorile, J.F. Comparison of predicted kinetic variables between Parkinson’s disease patients and healthy age-matched control using a depth sensor-driven full-body musculoskeletal model. Gait Posture 2020, 76, 151–156. [Google Scholar] [CrossRef]
Figure 1. An example of a raw absolute PPG signal with localized three systolic peaks and detailed situation around the second heart pulse P2SYS with an amplitude (AP2), a width at half the amplitude (WP2—determined at level L1/2), and heart pulse periods (THP1,2). fS = 1 kHz.
Figure 1. An example of a raw absolute PPG signal with localized three systolic peaks and detailed situation around the second heart pulse P2SYS with an amplitude (AP2), a width at half the amplitude (WP2—determined at level L1/2), and heart pulse periods (THP1,2). fS = 1 kHz.
Electronics 13 03297 g001
Figure 2. An example of determining time differences between systolic pulses ∆ PSYSa,b from two PPG waves sensed in parallel (PPGA and PPGB); fs = 1 kHz.
Figure 2. An example of determining time differences between systolic pulses ∆ PSYSa,b from two PPG waves sensed in parallel (PPGA and PPGB); fs = 1 kHz.
Electronics 13 03297 g002
Figure 3. Block diagram of the approach used for creation of a database of HPTPs from the PPG waves.
Figure 3. Block diagram of the approach used for creation of a database of HPTPs from the PPG waves.
Electronics 13 03297 g003
Figure 4. Block diagram of the used method for testing the correctness and evaluating the precision of estimated SBP/DBPs.
Figure 4. Block diagram of the used method for testing the correctness and evaluating the precision of estimated SBP/DBPs.
Electronics 13 03297 g004
Figure 5. A photo of principal arrangement for pick-up of two-channel PPG signals by optical PPG sensors located on a wrist (W) and fingers (P1–5) at distances Dx1–5.
Figure 5. A photo of principal arrangement for pick-up of two-channel PPG signals by optical PPG sensors located on a wrist (W) and fingers (P1–5) at distances Dx1–5.
Electronics 13 03297 g005
Figure 6. Influence of MA filtering on the PPG signal using different NX values: (a) selected part of a raw PPG signal with detailed situation around the three heart systolic pulses P1–3SYS, (b) bar graph of systolic pulse amplitudes Ap and heart pulse periods THP, (c) bar graph of a pulse width Wp. NX = 0 represents no filtering; fS = 1 kHz, WP at level L1/2 = 650.
Figure 6. Influence of MA filtering on the PPG signal using different NX values: (a) selected part of a raw PPG signal with detailed situation around the three heart systolic pulses P1–3SYS, (b) bar graph of systolic pulse amplitudes Ap and heart pulse periods THP, (c) bar graph of a pulse width Wp. NX = 0 represents no filtering; fS = 1 kHz, WP at level L1/2 = 650.
Electronics 13 03297 g006
Figure 7. Comparison of HPTP parameters separately for a hand and a gender type per finger P1–P5: (a) mean Dx distances, (b) PTT values, (c) PWV values, (d) rPTT values; for male (upper plots) and female (lower plots) testing subjects.
Figure 7. Comparison of HPTP parameters separately for a hand and a gender type per finger P1–P5: (a) mean Dx distances, (b) PTT values, (c) PWV values, (d) rPTT values; for male (upper plots) and female (lower plots) testing subjects.
Electronics 13 03297 g007
Figure 8. Mutual positions of ABP values measured by the BPM device and HPTPs using all tested persons grouped by L/R hands of males/females for: (a) SBP-PTT, (b) SBP-PWV [27], (c) SBP-rPTT, (d) SBP-rPWV, (e) DBP-PTT, (f) DBP-PWV, (g) DBP-rPTT [27], (h) DBP-rPWV parameters.
Figure 8. Mutual positions of ABP values measured by the BPM device and HPTPs using all tested persons grouped by L/R hands of males/females for: (a) SBP-PTT, (b) SBP-PWV [27], (c) SBP-rPTT, (d) SBP-rPWV, (e) DBP-PTT, (f) DBP-PWV, (g) DBP-rPTT [27], (h) DBP-rPWV parameters.
Electronics 13 03297 g008
Figure 9. Analysis of correlation between measured HRBPM and determined HRPPG values: (a) mutual positions of HRs, (b) for left hands, (c) for right hands, (d) for left and right hands together; R—calculated correlation coefficient; all tested subjects using the DB1 database.
Figure 9. Analysis of correlation between measured HRBPM and determined HRPPG values: (a) mutual positions of HRs, (b) for left hands, (c) for right hands, (d) for left and right hands together; R—calculated correlation coefficient; all tested subjects using the DB1 database.
Electronics 13 03297 g009
Figure 10. Scatter plots of correlations between measured (ABPBPM) and estimated (ABPEST) values together with calculated R coefficient for (a) SBP values, (b) DBP values; used merged DB1 and DB2 databases, joined left and right hands, all testing subjects.
Figure 10. Scatter plots of correlations between measured (ABPBPM) and estimated (ABPEST) values together with calculated R coefficient for (a) SBP values, (b) DBP values; used merged DB1 and DB2 databases, joined left and right hands, all testing subjects.
Electronics 13 03297 g010
Figure 11. Visualization of Bland–Altman plots of the final estimation accuracy for (a) SBP values, (b) DBP values; used merged DB1 and DB2 databases, joined left and right hands, all testing subjects.
Figure 11. Visualization of Bland–Altman plots of the final estimation accuracy for (a) SBP values, (b) DBP values; used merged DB1 and DB2 databases, joined left and right hands, all testing subjects.
Electronics 13 03297 g011
Table 1. Influence of the used fs on mean values of THP, ∆ PSYS, PTT parameters, relative percentage of ∆ PSYS error, and absolute PTT error for the considered absolute error equal to 3 samples.
Table 1. Influence of the used fs on mean values of THP, ∆ PSYS, PTT parameters, relative percentage of ∆ PSYS error, and absolute PTT error for the considered absolute error equal to 3 samples.
fS [Hz]THP-PPGA, B (1) [Samples]PSYS [Samples]PTT [ms]RE3PSYS [%]AE3PTT [ms]
12591 ± 15 ± 0.840.1 ± 6.864.124
250180 ± 210 ± 1.539.5 ± 6.529.912
500359 ± 319 ± 3.238.9 ± 6.315.46
1000718 ± 639 ± 6.438.6 ± 6.27.83
20001435 ± 1377 ± 12.338.4 ± 6.13.91.5
(1) Mean HRs = 83.7 bpm.
Table 2. Mean REE percentage values per HPTP type—separately for databases DB1 and DB2 for left and right hands together.
Table 2. Mean REE percentage values per HPTP type—separately for databases DB1 and DB2 for left and right hands together.
HPTP TypeSBP (DB1)DBP (DB1)SBP (DB2)DBP (DB2)
PTT−2.5 ± 2.3−3.6 ± 2.40.1 ± 9.42.9 ± 10.1
PWV−2.7 ± 2.4−4.2 ± 2.80.1 ± 9.63.3 ± 9.8
rPTT−3.5 ± 2.3−3.9 ± 2.51.5 ± 8.93.2 ± 10.0
rPWV−3.1 ± 2.2−4.0 ± 2.62.1 ± 8.83.5 ± 10.1
all−3.1 ± 2.2−3.9 ± 2.60.9 ± 9.03.2 ± 9.9
Table 3. Correlation between the measured and estimated SBP/DBP values—separately for the databases DB1 and DB2s, the type of the hand, and the gender of testing subjects.
Table 3. Correlation between the measured and estimated SBP/DBP values—separately for the databases DB1 and DB2s, the type of the hand, and the gender of testing subjects.
R [-]L-Hand (1)R-Hand (1)Male (2)Female (2)All
SBP (DB1)0.9360.9350.8950.8450.936
DBP (DB1)0.9940.9920.9520.9450.994
SBP (DB2)0.8180.8400.8710.8340.845
DBP (DB2)0.8960.9100.8730.9350.912
(1) Male and female together. (2) Joined left and right hands.
Table 4. Comparison of the current work results with the AAMI standard.
Table 4. Comparison of the current work results with the AAMI standard.
MAE (mmHg)SD (mmHg)
AAMIBP≤5≤8
This workSBP−2.6410.7
DBP−3.469.7
Table 5. Percentage comparison of Δ ABP values using the BHS methodology.
Table 5. Percentage comparison of Δ ABP values using the BHS methodology.
Δ ABP ≤ 5 mmHgΔ ABP ≤ 10 mmHgΔ ABP ≤ 15 mmHg
SBP65.1%93.3%100%
DBP47.8%100%100%
Table 6. Comparison of the obtained MAEs with other cuffless BP estimation systems.
Table 6. Comparison of the obtained MAEs with other cuffless BP estimation systems.
StudyNo. of Subj.Used ParametersfS [Hz]MAESBP [mmHg]MAEDBP [mmHg]
Arza et al. [11]16PTT + PW from ECG + PPG250/10002.72 ± 9.22.16 ± 6.0
Cattivelli et al. [13]25PTT from ECG + PPG1000−0.41 ± 7.8−0.1 ± 4.9
Johnson et al. [12]5PTT from two PPG1000−0.4 ± 50.8 ± 7
McCarthy et al. [10]6PTT + PWV from ECG + PPG400−0.82 ± 5.3
Mishra et al. [14]32PTT + PWV from ECG + PPG3165.15 + 11/−125.36 + 10/−14
Our12 + 104 HPTPs from two PPG1000−2.6 ± 10.7−3.5 ± 9.7
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Přibil, J.; Přibilová, A.; Frollo, I. Heart Pulse Transmission Parameters of Multi-Channel PPG Signals for Cuffless Estimation of Arterial Blood Pressure: Preliminary Study. Electronics 2024, 13, 3297. https://doi.org/10.3390/electronics13163297

AMA Style

Přibil J, Přibilová A, Frollo I. Heart Pulse Transmission Parameters of Multi-Channel PPG Signals for Cuffless Estimation of Arterial Blood Pressure: Preliminary Study. Electronics. 2024; 13(16):3297. https://doi.org/10.3390/electronics13163297

Chicago/Turabian Style

Přibil, Jiří, Anna Přibilová, and Ivan Frollo. 2024. "Heart Pulse Transmission Parameters of Multi-Channel PPG Signals for Cuffless Estimation of Arterial Blood Pressure: Preliminary Study" Electronics 13, no. 16: 3297. https://doi.org/10.3390/electronics13163297

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop