Assessing Cardiac Sympatho-Vagal Balance Through Wavelet Transform Analysis of Heart Rate Variability
Abstract
:1. Introduction
1.1. Wavelet Transformation
1.1.1. Wavelets
1.1.2. Continuous Wavelet Transform (CWT)
1.1.3. Discrete Wavelet Transform (DWT)
1.1.4. Multi-Resolution Analysis
1.1.5. Multilevel Decomposition
2. Materials and Methods
3. Results and Discussion
4. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Berntson, G.G.; Bigger, J.T.; Eckberg, D.L.; Grossman, P.; Kaufmann, P.G.; Malik, M.; Nagaraja, H.N.; Porges, S.W.; Saul, J.P.; Stone, P.H.; et al. Heart rate variability: Origins, methods, and interpretive caveats. Psychophysiology 1997, 34, 623–648. [Google Scholar] [CrossRef] [PubMed]
- Kim, H.-G.; Cheon, E.-J.; Bai, D.-S.; Lee, Y.H.; Koo, B.-H. Stress and heart rate variability: A meta-analysis and review of the literature. Psychiatry Investig. 2008, 15, 235–245. [Google Scholar] [CrossRef] [PubMed]
- Hon, E. Electronic evaluations of the fetal heart rate patterns preceding fetal death: Further observations. Am. J. Obstet. Gynecol. 1965, 87, 814–826. [Google Scholar]
- Kleiger, R.E.; Miller, J.P.; Bigger, J.T., Jr.; Moss, A.J. Decreased heart rate variability and its association with increased mortality after acute myocardial infarction. Am. J. Cardiol. 1987, 59, 256–262. [Google Scholar] [CrossRef]
- Tsuji, H.; Venditti, F.J.; Manders, E.S.; Evans, J.C.; Larson, M.G.; Feldman, C.L.; Levy, D. Reduced heart rate variability and mortality risk in an elderly cohort. The Framingham Heart Study. Circulation 1994, 90, 878–883. [Google Scholar] [CrossRef] [PubMed]
- Thayer, J.F.; Yamamoto, S.S.; Brosschot, J.F. The relationship of autonomic imbalance, heart rate variability and cardiovascular disease risk factors. Int. J. Cardiol. 2010, 141, 122–131. [Google Scholar] [CrossRef] [PubMed]
- Carney, R.M.; Blumenthal, J.A.; Stein, P.K.; Watkins, L.; Catellier, D.; Berkman, L.F.; Czajkowski, S.M.; O’connor, C.; Stone, P.H.; Freedland, K.E. Depression, heart rate variability, and acute myocardial infarction. Circulation 2001, 104, 2024–2028. [Google Scholar] [CrossRef] [PubMed]
- Gorman, J.M.; Sloan, R.P. Heart rate variability in depressive and anxiety disorders. Am. Heart J. 2000, 140, S77–S83. [Google Scholar] [CrossRef]
- Chalmers, J.A.; Quintana, D.S.; Abbott, M.J.-A.; Kemp, A.H. Anxiety disorders are associated with reduced heart rate variability: A meta-analysis. Front. Psychiatry 2014, 5, 80. [Google Scholar] [CrossRef]
- Abhishekh, H.A.; Nisarga, P.; Kisan, R.; Meghana, A.; Chandran, S.; Raju, T.; Sathyaprabha, T.N. Influence of age and gender on autonomic regulation of heart. J. Clin. Monit. Comput. 2013, 27, 259–264. [Google Scholar] [CrossRef]
- Voss, A.; Schulz, S.; Schroeder, R.; Baumert, M.; Caminal, P. Methods derived from nonlinear dynamics for analysing heart rate variability. Philos. Trans. R. Soc. A Math. Phys. Eng. Sci. 2009, 367, 277–296. [Google Scholar] [CrossRef]
- Agelink, M.W.; Malessa, R.; Baumann, B.; Majewski, T.; Akila, F.; Zeit, T.; Ziegler, D. Standardized tests of heart rate variability: Normal ranges obtained from 309 healthy humans, and effects of age, gender, and heart rate. Clin. Auton. Res. 2001, 11, 99–108. [Google Scholar] [CrossRef]
- Umetani, K.; Singer, D.H.; McCraty, R.; Atkinson, M. Twenty-Four Hour Time Domain Heart Rate Variability and Heart Rate: Relations to Age and Gender Over Nine Decades. J. Am. Coll. Cardiol. 1998, 31, 593–601. [Google Scholar] [CrossRef]
- Barantke, M.; Krauss, T.; Ortak, J.; Lieb, W.; Reppel, M.; Burgdorf, C.; Pramstaller, P.P.; Schunkert, H.; Bonnemeier, H. Effects of Gender and Aging on Differential Autonomic Responses to Orthostatic Maneuvers. J. Cardiovasc. Electrophysiol. 2008, 19, 1296–1303. [Google Scholar] [CrossRef] [PubMed]
- Dinas, P.C.; Koutedakis, Y.; Flouris, A.D. Effects of active and passive tobacco cigarette smoking on heart rate variability. Int. J. Cardiol. 2013, 163, 109–115. [Google Scholar] [CrossRef]
- Ralevski, E.; Petrakis, I.; Altemus, M. Heart rate variability in alcohol use: A review. Pharmacol. Biochem. Behav. 2019, 176, 83–92. [Google Scholar] [CrossRef]
- Schnell, I.; Potchter, O.; Epstein, Y.; Yaakov, Y.; Hermesh, H.; Brenner, S.; Tirosh, E. The effects of exposure to environmental factors on Heart Rate Variability: An ecological perspective. Environ. Pollut. 2013, 183, 7–13. [Google Scholar] [CrossRef]
- Elsenbruch, S.; Wang, Z.; Orr, W.C.; Chen, J.D.Z. Time-frequency analysis of heart rate variability using short-time Fourier analysis. Physiol. Meas. 2000, 21, 229–240. [Google Scholar] [CrossRef] [PubMed]
- Acharya, U.R.; Joseph, K.P.; Kannathal, N.; Lim, C.M.; Suri, J.S. Heart rate variability: A review. Med. Biol. Eng. Comput. 2006, 44, 1031–1051. [Google Scholar] [CrossRef]
- Boashash, B. Time-Frequency Signal Analysis and Processing: A Comprehensive Reference; Academic Press: Cambridge, MA, USA, 2015. [Google Scholar]
- Daubechies, I. Ten Lectures on Wavelets; SIAM: Philadelphia, PA, USA, 1992. [Google Scholar]
- Saxena, S.; Kumar, V.; Hamde, S. QRS detection using new wavelets. J. Med. Eng. Technol. 2002, 26, 7–15. [Google Scholar] [CrossRef]
- Nouira, I.; Abdallah, A.B.; Kouaja, I.; Bedoui, M.H. Comparative study of QRS complex detection in ECG. Int. J. Biomed. Biol. Eng. 2012, 6, 593–597. [Google Scholar]
- Nouira, I.; Abdallah, A.; Bedoui, M.H.; Dogui, M. A Robust R Peak Detection Algorithm Using Wavelet Transform for Heart Rate Variability Studies. Int. J. Electr. Eng. Inform. 2013, 5, 270–284. [Google Scholar] [CrossRef]
- Li, C.; Zheng, C.; Tai, C. Detection of ECG characteristic points using wavelet transforms. IEEE Trans. Biomed. Eng. 1995, 42, 21–28. [Google Scholar] [CrossRef] [PubMed]
- Zheng, H.; Wu, J. A Real-Time QRS Detector Based on Discrete Wavelet Transform and Cubic Spline Interpolation. Telemed. e-Health 2008, 14, 809–815. [Google Scholar] [CrossRef]
- Mallat, S.G. A theory for multiresolution signal decomposition: The wavelet representation. IEEE Trans. Pattern Anal. Mach. Intell. 1989, 11, 674–693. [Google Scholar] [CrossRef]
- Rioul, O. A discrete-time multiresolution theory. IEEE Trans. Signal Process. 1993, 41, 2591–2606. [Google Scholar] [CrossRef]
- Pale, U.; Thürk, F.; Kaniusas, E. Heart rate variability analysis using different wavelet transformations. In Proceedings of the 2016 39th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), Opatija, Croatia, 30 May–3 June 2016. [Google Scholar] [CrossRef]
- Acharya, U.R.; Vidya, K.S.; Ghista, D.N.; Lim, W.J.E.; Molinari, F.; Sankaranarayanan, M. Computer-aided diagnosis of diabetic subjects by heart rate variability signals using discrete wavelet transform method. Knowl.-Based Syst. 2015, 81, 56–64. [Google Scholar] [CrossRef]
- Ergen, B. Comparison of Wavelet Types and Thresholding Methods on Wavelet Based Denoising of Heart Sounds. J. Signal Inf. Process. 2013, 4, 164–167. [Google Scholar] [CrossRef]
- Goldberger, A.L.; Amaral, L.A.N.; Glass, L.; Hausdorff, J.M.; Ivanov, P.C.; Mark, R.G.; Mietus, J.E.; Moody, G.B.; Peng, C.-K.; Stanley, H.E. PhysioBank, PhysioToolkit, and PhysioNet: Components of a New Research Resource for Complex Physiologic Signals. Circulation 2000, 101, E215–E220. [Google Scholar] [CrossRef] [PubMed]
- Moody, G.B.; Mark, R.G. The impact of the MIT-BIH Arrhythmia Database. IEEE Eng. Med. Biol. Mag. 2001, 20, 45–50. [Google Scholar] [CrossRef] [PubMed]
- Elgendi, M.; Jonkman, M.; De Boer, F. Frequency Bands Effects on QRS Detection. Biosignals 2010, 2003, 2002. [Google Scholar]
- Cartas-Rosado, R.; Becerra-Luna, B.; Martínez-Memije, R.; Infante-Vázquez, Ó.; Lerma, C.; Pérez-Grovas, H.; Rodríguez-Chagolla, J.M. Continuous wavelet transform based processing for estimating the power spectrum content of heart rate variability during hemodiafiltration. Biomed. Signal Process. Control 2020, 62, 102031. [Google Scholar] [CrossRef]
- Malik, M.; Bigger, J.T.; Camm, A.J.; Kleiger, R.E.; Malliani, A.; Moss, A.J.; Schwartz, P.J. Heart rate variability: Standards of measurement, physiological interpretation, and clinical use. Eur. Heart J. 1996, 17, 354–381. [Google Scholar] [CrossRef]
- Dar, M.N.; Akram, M.U.; Usman, A.; Khan, S.A. ECG biometric identification for general population using multiresolution analysis of DWT based features. In Proceedings of the 2015 Second International Conference on Information Security and Cyber Forensics (InfoSec), Cape Town, South Africa, 15–17 November 2015. [Google Scholar] [CrossRef]
- Hernando, A.; Posada-Quintero, H.; Peláez-Coca, M.D.; Gil, E.; Chon, K.H. Autonomic Nervous System characterization in hyperbaric environments considering respiratory component and non-linear analysis of Heart Rate Variability. Comput. Methods Programs Biomed. 2022, 214, 106527. [Google Scholar] [CrossRef]
- Pomeranz, B.; Macaulay, R.J.; Caudill, M.A.; Kutz, I.; Adam, D.; Gordon, D.; Kilborn, K.M.; Barger, A.C.; Shannon, D.C.; Cohen, R.J.; et al. Assessment of autonomic function in humans by heart rate spectral analysis. Am. J. Physiol. Circ. Physiol. 1985, 248, H151–H153. [Google Scholar] [CrossRef] [PubMed]
- Appel, M.L.; Berger, R.D.; Saul, J.; Smith, J.M.; Cohen, R.J. Beat to beat variability in cardiovascular variables: Noise or music? J. Am. Coll. Cardiol. 1989, 14, 1139–1148. [Google Scholar] [CrossRef] [PubMed]
- Malliani, A.; Pagani, M.; Lombardi, F.; Cerutti, S. Cardiovascular neural regulation explored in the frequency domain. Circulation 1991, 84, 482–492. [Google Scholar] [CrossRef] [PubMed]
- Montano, N.; Ruscone, T.G.; Porta, A.; Lombardi, F.; Pagani, M.; Malliani, A. Power spectrum analysis of heart rate variability to assess the changes in sympathovagal balance during graded orthostatic tilt. Circulation 1994, 90, 1826–1831. [Google Scholar] [CrossRef] [PubMed]
- Houston, T.P.; Elster, A.B.; Davis, R.M.; Deitchman, S.D. The US Preventive Services Task Force Guide to Clinical Preventive Services, AMA Council on Scientific Affairs. Am. J. Prev. Med. 1998, 14, 374–376. [Google Scholar] [CrossRef]
- Sztajzel, J. Heart rate variability: A noninvasive electrocardiographic method to measure the autonomic nervous system. Swiss Med. Wkly. 2004, 134, 514–522. [Google Scholar] [CrossRef]
- Pagani, M.; Lombardi, F.; Guzzetti, S.; Rimoldi, O.; Furlan, R.; Pizzinelli, P.; Sandrone, G.; Malfatto, G.; Dell’Orto, S.; Piccaluga, E. Power spectral analysis of heart rate and arterial pressure variabilities as a marker of sympatho-vagal interaction in man and conscious dog. Circ. Res. 1986, 59, 178–193. [Google Scholar] [CrossRef] [PubMed]
- German-Sallo, Z. Wavelet Transform based HRV Analysis. Procedia Technol. 2014, 12, 105–111. [Google Scholar] [CrossRef]
- Shaffer, F.; Ginsberg, J.P. An Overview of Heart Rate Variability Metrics and Norms. Front. Public Health 2017, 5, 258. [Google Scholar] [CrossRef] [PubMed]
Level | Scale | Frequency Band (Hz) |
---|---|---|
1 | 2 | 64–128 |
2 | 4 | 32–64 |
3 | 8 | 16–32 |
4 | 16 | 8–16 |
5 | 32 | 4–8 |
Level | Scale | Frequency Band (Hz) |
---|---|---|
1 | 2 | 90–180 |
2 | 4 | 45–90 |
3 | 8 | 22.5–45 |
4 | 16 | 11.25–22.5 |
5 | 32 | 5.625–11.25 |
6 | 64 | 2.8125–5.625 |
LF/HF Ratio | ||||||
---|---|---|---|---|---|---|
Index | 16,265 | 16,786 | 19,140 | 106 | 112 | 117 |
Method | ||||||
MODWPT | 3.188 | 1.869 | 1.682 | 0.103 | 1.032 | 0.424 |
MODWT | 3.164 | 1.875 | 1.700 | 0.098 | 1.028 | 0.852 |
Record No. | Power (ms2) | Total Power | Normalized Power (n.u) | LF/HF Ratio | ||||
---|---|---|---|---|---|---|---|---|
VLF | LF | HF | VLF | LF | HF | |||
16,265 | 982.206 | 797.42 | 251.961 | 2031.588 | 0.483 | 0.392 | 0.124 | 3.164 |
16,272 | 3695.852 | 871.607 | 334.761 | 4902.22 | 0.753 | 0.177 | 0.068 | 2.603 |
16,273 | 1923.648 | 613.323 | 213.616 | 2750.588 | 0.699 | 0.222 | 0.077 | 2.871 |
16,420 | 497.22 | 634.57 | 56.026 | 1187.818 | 0.418 | 0.534 | 0.047 | 11.326 |
16,483 | 505.501 | 648.268 | 81.5 | 1235.27 | 0.409 | 0.524 | 0.065 | 7.954 |
16,539 | 2869.741 | 1244.762 | 2916.094 | 7010.597 | 0.409 | 0.174 | 0.415 | 0.42 |
16,773 | 3323.282 | 2259.39 | 775.58 | 6358.26 | 0.522 | 0.355 | 0.121 | 2.913 |
16,786 | 1284.794 | 1068.521 | 569.724 | 2923.039 | 0.439 | 0.365 | 0.194 | 1.875 |
16,795 | 4178.642 | 1943.284 | 1074.122 | 7196.048 | 0.58 | 0.27 | 0.149 | 1.809 |
17,052 | 3553.84 | 1132.412 | 6411.643 | 5327.416 | 0.667 | 0.212 | 0.12 | 1.766 |
17,453 | 1568.02 | 1000.653 | 3384.886 | 2907.171 | 0.539 | 0.344 | 0.116 | 2.956 |
18,177 | 1150.257 | 749.455 | 441.022 | 2340.735 | 0.491 | 0.320 | 0.188 | 1.699 |
18,184 | 2325.916 | 1015.988 | 2211.593 | 3563.063 | 0.652 | 0.285 | 0.062 | 4.593 |
19,088 | 727.481 | 366.368 | 231.762 | 1325.611 | 0.548 | 0.276 | 0.174 | 1.58 |
19,090 | 1194.602 | 617.317 | 144.911 | 1956.831 | 0.61 | 0.315 | 0.074 | 4.259 |
19,093 | 3770.705 | 1456.669 | 325.436 | 5552.81 | 0.679 | 0.262 | 0.058 | 4.476 |
19,140 | 806.541 | 479.895 | 280.741 | 1567.178 | 0.514 | 0.306 | 0.179 | 1.709 |
19,830 | 247.947 | 85.615 | 22.06 | 365.63 | 0.678 | 0.261 | 0.06 | 4.332 |
Index | Power (ms2) | Total Power | Normalized Power (n.u) | LF/HF Ratio | ||||
---|---|---|---|---|---|---|---|---|
VLF | LF | HF | VLF | LF | HF | |||
100 | 295.895 | 136.511 | 863.592 | 1295.999 | 0.228 | 0.105 | 0.666 | 0.158 |
101 | 740.593 | 508.507 | 967.957 | 2217.058 | 0.334 | 0.229 | 0.436 | 0.525 |
103 | 874.086 | 283.325 | 619.681 | 1777.093 | 0.491 | 0.159 | 0.348 | 0.457 |
105 | 278.359 | 291.088 | 1080.02 | 1649.463 | 0.168 | 0.176 | 0.654 | 0.269 |
106 | 4155.21 | 1863.38 | 18,948.4 | 24,967 | 0.166 | 0.074 | 0.758 | 0.098 |
107 | 83.027 | 275.126 | 904.511 | 1262.666 | 0.065 | 0.217 | 0.716 | 0.304 |
108 | 2258 | 3386 | 7143 | 12,788 | 0.176 | 0.264 | 0.558 | 0.474 |
109 | 225.357 | 55.861 | 506.856 | 788.076 | 0.285 | 0.07 | 0.643 | 0.11 |
111 | 248.827 | 179.993 | 728.289 | 1157.111 | 0.215 | 0.155 | 0.629 | 0.247 |
112 | 108.338 | 37.878 | 36.819 | 183.036 | 0.591 | 0.206 | 0.201 | 1.02 |
113 | 2017.82 | 2532.15 | 4264.53 | 8814.49 | 0.228 | 0.287 | 0.483 | 0.593 |
114 | 836.476 | 788.307 | 5093.99 | 6718.77 | 0.125 | 0.117 | 0.758 | 0.154 |
115 | 3063.9 | 1841.02 | 1896.19 | 6801.099 | 0.45 | 0.27 | 0.278 | 0.97 |
116 | 267.311 | 574.905 | 1630.48 | 2472.701 | 0.108 | 0.232 | 0.659 | 0.352 |
117 | 1026.8 | 222.463 | 261.004 | 1510.326 | 0.679 | 0.147 | 0.172 | 0.852 |
118 | 898.244 | 699.618 | 1809.2 | 3407.059 | 0.263 | 0.205 | 0.531 | 0.386 |
119 | 966.203 | 1168.92 | 21,469.9 | 23,605 | 0.04 | 0.049 | 0.909 | 0.054 |
121 | 847.944 | 171.545 | 205.958 | 1225.449 | 0.691 | 0.139 | 0.168 | 0.832 |
122 | 928.243 | 128.752 | 71.893 | 1128.889 | 0.822 | 0.114 | 0.063 | 1.79 |
123 | 4304.29 | 6332.78 | 2945.29 | 13,582.36 | 0.316 | 0.466 | 0.216 | 2.15 |
124 | 1353.45 | 435.352 | 1467.37 | 3256.174 | 0.415 | 0.133 | 0.45 | 0.296 |
200 | 691.461 | 881.709 | 3113.06 | 4686.227 | 0.147 | 0.188 | 0.664 | 0.283 |
201 | 22,255 | 13,128.3 | 31,395.5 | 66,778.85 | 0.333 | 0.196 | 0.47 | 0.418 |
202 | 4171.18 | 3406.07 | 8072.44 | 15,649.7 | 0.266 | 0.217 | 0.515 | 0.421 |
205 | 128.279 | 81.507 | 378.745 | 588.532 | 0.217 | 0.138 | 0.643 | 0.215 |
209 | 2886.21 | 580.797 | 769.583 | 4236.593 | 0.681 | 0.137 | 0.181 | 0.754 |
210 | 1139.49 | 2012.61 | 4775.11 | 7927.211 | 0.143 | 0.253 | 0.602 | 0.421 |
212 | 345.922 | 380.276 | 539.868 | 1266.067 | 0.273 | 0.3 | 0.426 | 0.704 |
213 | 15.81 | 53.506 | 173.516 | 243.834 | 0.068 | 0.219 | 0.711 | 0.308 |
214 | 2396.8 | 3278.84 | 9936.07 | 15,611.7 | 0.153 | 0.21 | 0.636 | 0.329 |
215 | 80.294 | 229.978 | 924.845 | 1235.119 | 0.065 | 0.186 | 0.748 | 0.248 |
217 | 661.746 | 749.83 | 2301.01 | 3712.586 | 0.178 | 0.201 | 0.619 | 0.325 |
220 | 1865.91 | 1179.83 | 3409.84 | 6455.575 | 0.289 | 0.182 | 0.528 | 0.346 |
221 | 1636.79 | 4791.15 | 15,135.5 | 21,563.49 | 0.075 | 0.222 | 0.701 | 0.316 |
222 | 6280.23 | 12,581.8 | 8126.31 | 26,988.38 | 0.232 | 0.466 | 0.301 | 1.54 |
223 | 649.899 | 333.746 | 971.99 | 1955.637 | 0.332 | 0.17 | 0.497 | 0.343 |
228 | 1758.14 | 2596.81 | 8272.69 | 12,627.64 | 0.139 | 0.205 | 0.655 | 0.313 |
230 | 1894.66 | 991.77 | 308.061 | 3194.425 | 0.593 | 0.31 | 0.096 | 3.21 |
231 | 33,216.2 | 4396.81 | 3197.72 | 40,810.67 | 0.813 | 0.107 | 0.078 | 1.38 |
232 | 14,254.9 | 111,483 | 221,810 | 347,547.8 | 0.041 | 0.32 | 0.638 | 0.502 |
233 | 122.723 | 1905.31 | 1998.67 | 23,119.26 | 0.053 | 0.082 | 0.864 | 0.095 |
234 | 393.933 | 113.916 | 167.856 | 675.706 | 0.582 | 0.168 | 0.248 | 0.678 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Nelushi, A.M.; Manathunga, C.H.; Shantha Gamage, N.G.S.; Nakayama, T. Assessing Cardiac Sympatho-Vagal Balance Through Wavelet Transform Analysis of Heart Rate Variability. Appl. Sci. 2025, 15, 1687. https://doi.org/10.3390/app15041687
Nelushi AM, Manathunga CH, Shantha Gamage NGS, Nakayama T. Assessing Cardiac Sympatho-Vagal Balance Through Wavelet Transform Analysis of Heart Rate Variability. Applied Sciences. 2025; 15(4):1687. https://doi.org/10.3390/app15041687
Chicago/Turabian StyleNelushi, A.M., C.H. Manathunga, N.G.S. Shantha Gamage, and Tadachika Nakayama. 2025. "Assessing Cardiac Sympatho-Vagal Balance Through Wavelet Transform Analysis of Heart Rate Variability" Applied Sciences 15, no. 4: 1687. https://doi.org/10.3390/app15041687
APA StyleNelushi, A. M., Manathunga, C. H., Shantha Gamage, N. G. S., & Nakayama, T. (2025). Assessing Cardiac Sympatho-Vagal Balance Through Wavelet Transform Analysis of Heart Rate Variability. Applied Sciences, 15(4), 1687. https://doi.org/10.3390/app15041687