The Effect of the COVID-19 Pandemic on Physicians’ Use and Perception of Telehealth: The Case of Lebanon
Abstract
:1. Introduction
- RQ1: How did Lebanese physicians adjust their use of telehealth during the COVID-19 pandemic?
- RQ2: How did the experience of using telehealth during the COVID-19 pandemic affect the perceptions of Lebanese physicians regarding telehealth?
2. Materials and Methods
2.1. Qualitative Study
2.2. Quantitative Study
- The age, gender, and specialty of the physicians;
- The types of telehealth activities they engaged in before the COVID-19 pandemic, and the activities they engaged in more during the COVID-19 pandemic;
- The tools they used for telehealth before the COVID-19 pandemic, and the tools they used more during the COVID-19 pandemic;
- Their perceptions regarding various aspects of telehealth before, and during, the COVID-19 pandemic.
3. Results
3.1. Themes Identified in the Interviews
3.2. Questionnaire Respondents
3.3. Telehealth Activities before the COVID-19 Pandemic
3.4. Telehealth Activities That Were Done More Frequently during the COVID-19 Pandemic
3.5. Tools Used for Telehealth before the COVID-19 Pandemic
3.6. Tools That Were More Frequently Used for Telehealth during the COVID-19 Pandemic
3.7. Perceptions Regarding Telehealth
4. Discussion
4.1. Principal Results
4.2. Implications
4.3. Limitations and Future Research
5. Conclusions
Author Contributions
Funding
Conflicts of Interest
Appendix A. Semi-Structured Interview Guide
- -
- Can you please tell me about your understanding of what telehealth is?
- -
- What kind of telehealth have you done before the COVID-19 pandemic?
- -
- What kind of telehealth have you done during the COVID-19 pandemic?
- -
- What tools did you use for telehealth before the COVID-19 pandemic?
- -
- What tools did you use for telehealth during the COVID-19 pandemic?
- -
- What were the problems that you encountered while doing telehealth during the COVID-19 pandemic?
- -
- When and where were you providing telemedicine from before the COVID-19 pandemic?
- -
- When and where were you providing telemedicine from during the COVID-19 pandemic?
- -
- Have you ever been paid for a telemedicine service before the COVID-19 pandemic and how?
- -
- Have you ever been paid for a telemedicine service during the COVID-19 pandemic and how?
- -
- What was your opinion regarding telehealth before the COVID-19 pandemic?
- -
- Did your opinion change after the COVID-19 pandemic?
- -
- Are you going to invest more of your time in telehealth once the crisis is over? Or will you return to baseline?
- -
- What kind of telehealth are you thinking of investing in in the future?
- -
- Did you think telemedicine is equally effective as a face-to-face consultation before the COVID-19 pandemic?
- -
- Did your opinion change after the COVID-19 pandemic?
- -
- Did you think the patients’ satisfaction is the same with telemedicine as with a face-to-face consultation?
- -
- Did your opinion change after the COVID-19 pandemic?
- -
- What are the situations in which you wouldn't accept to use telemedicine?
- -
- Were these situations different during the COVID-19 pandemic?
- -
- How much would you price a telemedicine consultation in comparison to a face to face consultation and why?
- -
- Are you aware of any law in relation to telemedicine?
- -
- Does the law still apply during the COVID-19 pandemic?
- -
- Do you think we need stricter or less strict laws around telemedicine?
- -
- Did you feel you are taking a higher risk when providing telemedicine before the COVID-19 pandemic?
- -
- Did you feel you are taking a higher risk when providing telemedicine during the COVID-19 pandemic?
References
- Republic of Lebanon—Ministry of Public Health Lebanon. COVID-19 Status 21 Feb-18 Mar 2020; Republic of Lebanon—Ministry of Public Health Lebanon: Beirut, Lebanon, 2020.
- The Council of Ministers of Lebanon. Declaration of a State of Emergency to Fight Against the Spread of Coronavirus; The Council of Ministers of Lebanon: Beirut, Lebanon, 2020.
- Al-Markazia. News A Request for Physicians to Use Phones and Applications to Provide Healthcare Services and Prescriptions; Al-Markazia: Beirut, Lebanon, 2020. [Google Scholar]
- Saleh, S.; Khodor, R.; Alameddine, M.; Baroud, M. Readiness of healthcare providers for eHealth: The case from primary healthcare centers in Lebanon. BMC Health Serv. Res. 2016, 16, 644. [Google Scholar] [CrossRef] [Green Version]
- Aoun, E.; Monin, E. Lebanon: 2016 TV Audience Measurement; IPSOS: Beirut, Lebanon, 2017. [Google Scholar]
- Saleh, S.; Farah, A.; El Arnaout, N.; Dimassi, H.; El Morr, C.; Muntaner, C.; Ammar, W.; Hamadeh, R.; Alameddine, M. mHealth use for non-communicable diseases care in primary health: Patients’ perspective from rural settings and refugee camps. J. Public Health 2018, 40, ii52–ii63. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Chelala, E.; Saleh, N.; Dirani, A.; Fadlallah, A.; Baz, P.; Slim, E.; Halaby, G.; Bejjani, R. Screening of diabetic retinopathy and maculopathy in Lebanese population using retinography and SD-OCT: The role of telemedicine. J. Med. Liban. Leban. Med. J. 2015, 63, 59–65. [Google Scholar] [CrossRef]
- Haddad, R.N.; Sakr, C.; Khabbaz, L.; Azouri, H.; Eid, B. Telephone consultation and prescription in pediatrics: Contributing factors and impact on clinical outcomes. Front. Pediatr. 2019, 7, 515. [Google Scholar] [CrossRef] [Green Version]
- Ammar, W. Health Beyond Politics; World Health Organization, Regional Office for the Eastern Mediterranean: Beirut, Lebanon, 2009. [Google Scholar]
- Chatty, D.; Mansour, N.; Yassin, N. Bedouin in Lebanon: Social discrimination, political exclusion, and compromised health care. Soc. Sci. Med. 2013, 82, 43–50. [Google Scholar] [CrossRef] [PubMed]
- Noubani, A.; Diaconu, K.; Ghandour, L.; El Koussa, M.; Loffreda, G.; Saleh, S. A community–based system dynamics approach for understanding factors affecting mental Health and Health seeking behaviors in Beirut and Beqaa regions of Lebanon. Glob. Health 2020, 16, 1–13. [Google Scholar] [CrossRef] [PubMed]
- Chen, B.; Cammett, M. Informal politics and inequity of access to health care in Lebanon. Int. J. Equity Health 2012, 11, 23. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Vidal-Alaball, J.; Franch-Parella, J.; Lopez Seguí, F.; Garcia Cuyàs, F.; Mendioroz Peña, J. Impact of a Telemedicine Program on the Reduction in the Emission of Atmospheric Pollutants and Journeys by Road. Int. J. Environ. Res. Public Health 2019, 16, 4366. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Baayoun, A.; Itani, W.; El Helou, J.; Halabi, L.; Medlej, S.; El Malki, M.; Moukhadder, A.; Aboujaoude, L.K.; Kabakian, V.; Mounajed, H. Emission inventory of key sources of air pollution in Lebanon. Atmos. Environ. 2019, 215, 116871. [Google Scholar] [CrossRef]
- Buckley, K.M.; Tran, B.Q.; Prandoni, C.M. Receptiveness, use and acceptance of telehealth by caregivers of stroke patients in the home. Online J. Issues Nurs. 2004, 9, 9. [Google Scholar]
- Broens, T.H.F.; Huis in’t Veld, R.M.H.A.; Vollenbroek-Hutten, M.M.R.; Hermens, H.J.; van Halteren, A.T.; Nieuwenhuis, L.J.M. Determinants of successful telemedicine implementations: A literature study. J. Telemed. Telecare 2007, 13, 303–309. [Google Scholar] [CrossRef] [PubMed]
- Chen, J.; Lan, Y.-C.; Chang, Y.-W.; Chang, P.-Y. Exploring doctors’ willingness to provide online counseling services: The roles of motivations and costs. Int. J. Environ. Res. Public Health 2020, 17, 110. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Smith, A.C.; Thomas, E.; Snoswell, C.L.; Haydon, H.; Mehrotra, A.; Clemensen, J.; Caffery, L.J. Telehealth for global emergencies: Implications for coronavirus disease 2019 (COVID-19). J. Telemed. Telecare 2020, 26, 309–313. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Braun, V.; Clarke, V.; Hayfield, N.; Terry, G. Thematic analysis. In Handbook of Research Methods in Health Social Sciences; Springer: Singapore, 2019; pp. 843–860. [Google Scholar]
- MacNeill, V.; Sanders, C.; Fitzpatrick, R.; Hendy, J.; Barlow, J.; Knapp, M.; Rogers, A.; Bardsley, M.; Newman, S.P. Experiences of front-line health professionals in the delivery of telehealth: A qualitative study. Br. J. Gen. Pract. 2014, 64, e401–e407. [Google Scholar] [CrossRef] [Green Version]
- Liu, J.; Bian, Y.; Ye, Q.; Jing, D. Free for caring? The effect of offering free online medical-consulting services on physician performance in e-health care. Telemed. E-Health 2019, 25, 979–986. [Google Scholar] [CrossRef]
- Guo, S.; Guo, X.; Fang, Y.; Vogel, D. How doctors gain social and economic returns in online health-care communities: A professional capital perspective. J. Manag. Inf. Syst. 2017, 34, 487–519. [Google Scholar] [CrossRef]
- Catalán-Matamoros, D. The Role of Mass Media Communication in Public Health. Available online: https://www.intechopen.com/books/health-management-different-approaches-and-solutions/the-role-of-mass-media-communication-in-public-health (accessed on 14 December 2011).
- Feldman, B. Is It Safe to Use Zoom? Available online: https://nymag.com/intelligencer/2020/04/the-zoom-app-has-a-lot-of-security-problems.html (accessed on 1 July 2020).
- Ly, B.A.; Labonte, R.; Bourgeault, I.L.; Niang, M.N. The individual and contextual determinants of the use of telemedicine: A descriptive study of the perceptions of Senegal’s physicians and telemedicine projects managers. PLoS ONE 2017, 12, e0181070. [Google Scholar] [CrossRef]
- Daniel, F.; Jabak, S.; Sasso, R.; Chamoun, Y.; Tamim, H. Patient-physician communication in the era of mobile phones and social media apps: Cross-sectional observational study on Lebanese physicians’ perceptions and attitudes. JMIR Med. Inf. 2018, 6, e18. [Google Scholar] [CrossRef]
- Kassak, K.M.; Ghomrawi, H.M.; Osseiran, A.M.A.; Kobeissi, H. The providers of health services in Lebanon: A survey of physicians. Human Resour. Health 2006, 4, 4. [Google Scholar] [CrossRef] [Green Version]
- Youssef, J. Economic Overview Lebanon. Available online: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3519485 (accessed on 6 July 2020).
- Arawi, T. The Lebanese physician: A public’s viewpoint. Dev. World Bioeth. 2010, 10, 22–29. [Google Scholar] [CrossRef]
- Shaya, B.; Al Homsi, N.; Eid, K.; Haidar, Z.; Khalil, A.; Merheb, K.; Haidar, G.H.-A.; Akl, E.A. Factors associated with the public’s trust in physicians in the context of the Lebanese healthcare system: A qualitative study. BMC Health Serv. Res. 2019, 19, 525. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Khoja, S.; Scott, R.E.; Casebeer, A.L.; Mohsin, M.; Ishaq, A.F.M.; Gilani, S. e-Health readiness assessment tools for healthcare institutions in developing countries. Telemed. E-Health 2007, 13, 425–432. [Google Scholar] [CrossRef] [PubMed]
- Van Dyk, L. A review of telehealth service implementation frameworks. Int. J. Environ. Res. Public Health 2014, 11, 1279–1298. [Google Scholar] [CrossRef] [Green Version]
- Yusif, S.; Hafeez-Baig, A.; Soar, J. e-Health readiness assessment factors and measuring tools: A systematic review. Int. J. Med. Inf. 2017, 107, 56–64. [Google Scholar] [CrossRef] [PubMed]
- Carroll, J. Completing design in use: Closing the appropriation cycle. In Proceedings of the 13th European Conference on Information Systems, Turku, Finland, 14–16 June 2004. [Google Scholar]
- Helou, S.; Abou-Khalil, V.; Yamamoto, G.; Kondoh, E.; Tamura, H.; Hiragi, S.; Sugiyama, O.; Okamoto, K.; Nambu, M.; Kuroda, T. Understanding the situated roles of electronic medical record systems to enable redesign: Mixed methods study. JMIR Human Factors 2019, 6, e13812. [Google Scholar] [CrossRef]
- Eedy, D.J.; Wootton, R. Teledermatology: A review. Brit. J. Dermatol. 2001, 144, 696–707. [Google Scholar] [CrossRef] [Green Version]
- What Is Telehealth? How Is Telehealth different from Telemedicine?|HealthIT.gov. Available online: https://www.healthit.gov/faq/what-telehealth-how-telehealth-different-telemedicine (accessed on 30 May 2020).
Themes | Subthemes | Quotes from the Interviews |
---|---|---|
Telehealth Activities | ||
Telemedicine | Physicians receive test results from patient over the phone or internet | “Patients sometimes send me their test results when they receive them as a photo over WhatsApp” |
Physicians receive test results from centers over the internet | “I work with a center, and they gave me an ID and password to log in and see the test results” | |
Patients consult with physicians over the phone or internet | “I give consultations to Lebanese patients living abroad. They come to the clinic when they are in Lebanon and we continue the follow-up through the phone when they leave Lebanon” “Sometimes the patient’s family sends me videos of the patient having a seizure, which is helpful” | |
Physicians prescribe medications to patients over the phone or internet | “I send patient lots of prescriptions through WhatsApp and through email” | |
Public Awareness | Physicians provide public awareness through internet or media channels | “I participate in awareness campaigns on the radio, on TV, and on social media” |
Continuing Medical Education | Physicians attend health-related webinars/conferences online | “Webinars are time and money sparing tools, we used to travel to attend, now we can easily access them, especially with the high cost associated with conferences” |
Physicians are part of an online group that discusses health-related issues | “I am in many WhatsApp groups with colleagues” | |
Research | Physicians collaborate on health-related research, using the phone or the internet. | “I have been doing a PhD remotely. We have our meetings online and share the data between Lebanon and France” |
Administration | Physicians attend health-related meetings online | “I participate in a board meeting, with international researchers and practitioners, on a video platform, once a month” |
Teaching | Physicians give health-related training/lessons to students online | “Before, I didn’t do online teaching. But now we’re obliged to do it” |
Telehealth perceptions | ||
Need for Telehealth | Telehealth is needed in Lebanon | “I do a little more diagnosis online especially for people from remote areas because they cannot drive to Beirut” |
Wanting to invest in telehealth | “I need to invest more in telehealth, not because of COVID-19. I already had plans to do that anyway before the epidemic” “Interested in using telehealth for international telemedicine and more international research” | |
Health-Related Discussions on Social Media | Discussion of health topics on social media polarizes people’s opinions | “Things get polarized on WhatsApp groups because some people are louder, they chat a lot, send a lot of texts, even when they are wrong, because other people are busy, you hear a lot of echo” |
Effectiveness of Telemedicine | Inability to get the complete picture of the case with remote consultations | “Triage is more effective face-to-face: once a patient came to my clinic, the moment he walked into the clinic, the color of his face was reddish, he told me he took many pills of aspirin before coming, I sent him directly to the emergency department. Maybe I would not have noticed this with telemedicine. The appearance of the person, his general presentation, is very important” |
The first visit cannot be done remotely | “I don’t usually use telemedicine for the first visit, or to ask for a medical inquiry” | |
Efficiency of Telemedicine | Remote consultations are more time-consuming than face-to-face consultations | “It’s time-consuming”, and “it’s an interruption of daily activities” “We have connection cuts. We stop the call, call again. This is not practical; it takes more time than it should” “The platform that I use is very inflexible. I cannot start a consultation earlier even if the previous one ended prematurely; this creates a lot of dead time” |
Patient Satisfaction with Telemedicine | Patient satisfaction is different with remote consultation and face to face consultations | “Some patients might think it’s more beneficial, because they have a free medical act over the phone. Some patients might abuse it” “Patient satisfaction is not the same: They do not feel the same, they do not feel like it’s the same kind of service, [and] they are reluctant to pay the same amount for a visit. I think the visit would be shorter. It will be less personal and friendly” “We should ask the patients if they prefer face-to-face consultations, but I think they prefer it, definitely” |
Safety of Telemedicine | Remote consultations are not always safe for the patients | “Whenever there is any risk in providing telemedicine, I avoid it and tell them I have to see them to examine them or send them to the emergency room” “I have a major concern with children—you have to think about physical abuse with a child, how can I really address it, how can I have a good examination...” “I cannot be sure that the patient fully understands the medical recommendation on the phone. Usually there is a family member with them at the clinic, so I also explain it to them” |
Remote consultations are not always safe for the doctors | “I feel some risk with telemedicine. Before COVID-19 the risk was higher because you were more accountable, but now you will not be held accountable. I think I took a little bit higher risk during COVID-19” | |
Medicolegal Aspects | Existing telehealth regulations are not adequate | “We had to act in an emergency setting despite knowing that there are regulations that are missing” “We should have more regulation, now more than ever, concerning telehealth in general and telemedicine specifically” |
Remuneration for Telemedicine Services | Remote consultations are offered as an extra free service | “When someone calls for a few minutes, I don’t ask any payment for it” “Someone proposed to pay for the phone consultation, and I refused” “I don’t expect to get paid from patients that I have been seeing for a long time, with the understanding that they would come back and resume their treatment” |
Management of Telemedicine Practice | Remote consultations are provided at anytime | “If this continues, we would need to do something different, allocate time in the clinic with appointments, and give them a slot on an online platform, and be in the clinic to have the chart in front of us and update the chart” |
Remote consultations are provided anywhere | “I answer the patients whenever I can, from the car, from my home…” “Being at home is not very practical for all kinds of communication” |
Characteristics | Frequency | Percentage | |
---|---|---|---|
Age | 25–30 | 2 | 1.43% |
31–35 | 12 | 8.57% | |
36–40 | 44 | 31.43% | |
41–45 | 40 | 28.57% | |
46–50 | 10 | 7.14% | |
51–56 | 10 | 7.14% | |
56–60 | 11 | 7.86% | |
60–65 | 9 | 6.43% | |
66+ | 2 | 1.43% | |
Gender | Male | 93 | 66.4% |
Female | 47 | 33.6% | |
Other | 0 | 0% | |
Specialty | Anesthesia | 10 | 7.14% |
Cardiology | 3 | 2.14% | |
Dermatology | 5 | 3.57% | |
Endocrinology | 5 | 3.57% | |
ENT | 4 | 2.86% | |
Family medicine | 9 | 6.43% | |
Gastroenterology | 10 | 7.14% | |
General medicine | 2 | 1.43% | |
General surgery | 8 | 5.71% | |
Infectious diseases | 1 | 0.71% | |
Internal medicine | 2 | 1.43% | |
Nephrology | 1 | 0.71% | |
Neurology | 3 | 2.14% | |
Obstetrics and Gynecology | 11 | 7.86% | |
Oncology | 8 | 5.71% | |
Ophthalmology | 1 | 0.71% | |
Orthopedics | 5 | 3.57% | |
Osteopathic medicine | 1 | 0.71% | |
Pediatric surgery | 1 | 0.71% | |
Pediatrics | 21 | 15.00% | |
Plastic surgery | 4 | 2.86% | |
Pneumology | 2 | 1.43% | |
Psychiatry | 11 | 7.86% | |
Radiology | 5 | 3.57% | |
Rheumatology | 1 | 0.71% | |
Urology | 5 | 3.57% | |
Vascular and thoracic surgery | 1 | 0.71% |
Perceptions | Strongly Disagree | Disagree | Undecided | Agree | Strongly Agree | Mann–Whitney U test | |
---|---|---|---|---|---|---|---|
Telehealth is needed in Lebanon | Before | 7% | 21% | 30% | 31% | 11% | z-score = −5.1, p < 0.00001, U = 6671 |
After | 6% | 5% | 16% | 49% | 25% | ||
I want to invest more in telehealth | Before | 7% | 31% | 29% | 27% | 6% | z-score = −5.5, p < 0.00001, U = 6352 |
After | 6% | 8% | 21% | 45% | 20% | ||
Discussing health topics on social media helps align people's opinions | Before | 5% | 20% | 16% | 51% | 9% | z-score = −5.1, p < 0.00001, U = 8320 |
After | 4% | 11% | 14% | 48% | 22% | ||
Remote consultations are less time-consuming than face-to-face consultations | Before | 12% | 50% | 21% | 14% | 4% | z-score = −2.7, p = 0.00782, U = 8360 |
After | 8% | 41% | 16% | 27% | 7% | ||
I can have a complete picture of the case with remote consultations | Before | 21% | 46% | 20% | 14% | 0% | z-score = −2.4, p = 0.01778, U = 8567 |
After | 13% | 41% | 23% | 18% | 5% | ||
The first visit can be done by remote consultation | Before | 36% | 45% | 13% | 4% | 2% | z-score = −4.5, p < 0.00001, U = 7107 |
After | 19% | 40% | 11% | 27% | 2% | ||
The patient is as satisfied with a remote consultation as face-to-face | Before | 23% | 53% | 19% | 4% | 1% | z-score = −5.5, p < 0.00001, U = 6345 |
After | 11% | 33% | 27% | 28% | 1% | ||
Remote consultations are safe for the patient | Before | 21% | 49% | 23% | 5% | 1% | z-score = −5.6, p < 0.00001, U = 6323 |
After | 12% | 28% | 24% | 28% | 9% | ||
Remote consultations are safe for the doctor | Before | 21% | 41% | 24% | 10% | 4% | z-score = −4.9, p < 0.00001, U = 6805 |
After | 10% | 28% | 21% | 30% | 11% | ||
Existing telehealth regulations are adequate | Before | 29% | 36% | 24% | 11% | 1% | z-score = −1.9, p = 0.05486, U = 8882 |
After | 29% | 20% | 27% | 19% | 5% | ||
I should be compensated for remote consultations | Before | 7% | 17% | 21% | 34% | 20% | z-score = −4.4, p < 0.00001, U = 7143 |
After | 4% | 8% | 10% | 36% | 41% | ||
I should make a specific time for remote consultations | Before | 9% | 24% | 22% | 34% | 11% | z-score = −4.7, p < 0.00001, U = 6928 |
After | 5% | 6% | 18% | 42% | 29% | ||
I should have a specific location to give remote consultations | Before | 11% | 27% | 31% | 28% | 4% | z-score = −4, p = 0.00008, U = 7452 |
After | 8% | 13% | 27% | 34% | 18% |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Helou, S.; El Helou, E.; Abou-Khalil, V.; Wakim, J.; El Helou, J.; Daher, A.; El Hachem, C. The Effect of the COVID-19 Pandemic on Physicians’ Use and Perception of Telehealth: The Case of Lebanon. Int. J. Environ. Res. Public Health 2020, 17, 4866. https://doi.org/10.3390/ijerph17134866
Helou S, El Helou E, Abou-Khalil V, Wakim J, El Helou J, Daher A, El Hachem C. The Effect of the COVID-19 Pandemic on Physicians’ Use and Perception of Telehealth: The Case of Lebanon. International Journal of Environmental Research and Public Health. 2020; 17(13):4866. https://doi.org/10.3390/ijerph17134866
Chicago/Turabian StyleHelou, Samar, Elie El Helou, Victoria Abou-Khalil, Jad Wakim, Jeanine El Helou, Alain Daher, and Charline El Hachem. 2020. "The Effect of the COVID-19 Pandemic on Physicians’ Use and Perception of Telehealth: The Case of Lebanon" International Journal of Environmental Research and Public Health 17, no. 13: 4866. https://doi.org/10.3390/ijerph17134866
APA StyleHelou, S., El Helou, E., Abou-Khalil, V., Wakim, J., El Helou, J., Daher, A., & El Hachem, C. (2020). The Effect of the COVID-19 Pandemic on Physicians’ Use and Perception of Telehealth: The Case of Lebanon. International Journal of Environmental Research and Public Health, 17(13), 4866. https://doi.org/10.3390/ijerph17134866