Psoriasis—A Cancer Risk Factor?
Abstract
:1. Introduction
- -
- Provide descriptive epidemiological data on the association of psoriasis with malignancies in Northeastern Romania;
- -
- Quantify the risk of developing different forms of cancer in patients with psoriasis compared to the general population;
- -
- Analyze a possible benefit of implementing a cancer screening/prevention program in patients with psoriasis.
2. Materials and Methods
3. Results
4. Discussion
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Parisi, R.; Symmons, D.; Griffiths, C.; Ashcroft, D.M. Global Epidemiology of Psoriasis: A Systematic Review of Incidence and Prevalence. J. Investig. Dermatol. 2013, 133, 377–385. [Google Scholar] [CrossRef] [Green Version]
- Kimball, A.B.; Gladman, D.; Gelfand, J.; Gordon, K.; Horn, E.J.; Korman, N.J.; Korver, G.; Krueger, G.G.; Strober, B.E.; Lebwohl, M.G. National Psoriasis Foundation clinical consensus on psoriasis comorbidities and recommendations for screening. J. Am. Acad. Dermatol. 2008, 58, 1031–1042. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Nestle, F.O.; Kaplan, D.H.; Barker, J. Psoriasis. N. Engl. J. Med. 2009, 361, 496–509. [Google Scholar] [CrossRef] [PubMed]
- Pouplard, C.; Brenaut, E.; Horreau, C.; Barnetche, T.; Misery, L.; Richard, M.-A.; Aractingi, S.; Aubin, F.; Cribier, B.; Joly, P.; et al. Risk of cancer in psoriasis: A systematic review and meta-analysis of epidemiological studies. J. Eur. Acad. Dermatol. Venereol. 2013, 27, 36–46. [Google Scholar] [CrossRef] [PubMed]
- Trafford, A.; Parisi, R.; Kontopantelis, E.; Griffiths, C.E.M.; Ashcroft, D. Association of Psoriasis With the Risk of Developing or Dying of Cancer: A Systematic Review and Meta-analysis. JAMA Dermatol. 2019, 155, 1390–1403. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Margolis, D.; Bilker, W.; Hennessy, S.; Vittorio, C.; Santanna, J.; Strom, B.L. The risk of malignancy associated with psoriasis. Arch. Dermatol. 2001, 137, 778–783. [Google Scholar] [PubMed]
- Brauchli, Y.B.; Jick, S.; Miret, M.; Meier, C.R. Psoriasis and Risk of Incident Cancer: An Inception Cohort Study with a Nested Case–Control Analysis. J. Investig. Dermatol. 2009, 129, 2604–2612. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Paul, C.F.; Gourraud, P.-A. Cancer Risk Evaluation in Psoriasis: In Search of the Holy Grail? J. Investig. Dermatol. 2009, 129, 2547–2549. [Google Scholar] [CrossRef] [Green Version]
- Prizment, A.E.; Alonso, A.; Folsom, A.R.; Ahmed, R.L.; Virnig, B.A.; Warshaw, E.M.; Anderson, K.E. Association between psoriasis and incident cancer: The Iowa’s Women’s Health Study. Cancer Causes Control. 2011, 22, 1003–1010. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Svahn, H.; Sigurgeirsson, B.; Pukkala, E.; Lindelöf, B.; Berne, B.; Hannuksela, M.; Poikolainen, K.; Karvonen, J. Trioxsalen bath PUVA did not increase the risk of squamous cell skin carcinoma and cutaneous malignant melanoma in a joint analysis of 944 Swedish and Finnish patients with psoriasis. Br. J. Dermatol. 1999, 141, 497–501. [Google Scholar] [CrossRef] [Green Version]
- Hannuksela-Svahn, A.; Pukkala, E.; Läärä, E.; Poikolainen, K.; Karvonen, J. Psoriasis, its Treatment, and Cancer in a Cohort of Finnish Patients. J. Investig. Dermatol. 2000, 114, 587–590. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Ji, J.; Shu, X.; Sundquist, J.; Hemminki, K. Cancer risk in hospitalised psoriasis patients: A follow-up study in Sweden. Br. J. Cancer 2009, 100, 1499–1502. [Google Scholar] [CrossRef] [PubMed]
- Lindelöf, B.; Eklund, G.; Lidén, S.; Stern, R. The prevalence of malignant tumors in patients with psoriasis. J. Am. Acad. Dermatol. 1990, 22, 1056–1060. [Google Scholar] [CrossRef]
- Vaengebjerg, S.; Skov, L.; Egeberg, A.; Loft, N.D. Prevalence, Incidence, and Risk of Cancer in Patients With Psoriasis and Psoriatic Arthritis: A Systematic Review and Meta-analysis. JAMA Dermatol. 2020, 156, 421–429. [Google Scholar] [CrossRef] [PubMed]
- Paul, C.F.; Ho, V.C.; McGeown, C.; Christophers, E.; Schmidtmann, B.; Guillaume, J.-C.; Lamarque, V.; Dubertret, L. Risk of Malignancies in Psoriasis Patients Treated with Cyclosporine: A 5 y Cohort Study. J. Investig. Dermatol. 2003, 120, 211–216. [Google Scholar] [CrossRef] [Green Version]
- Geller, S.; Xu, H.; Lebwohl, M.; Nardone, B.; Lacouture, M.E.; Kheterpal, M. Malignancy Risk and Recurrence with Psoriasis and Its Treatments: A Concise Update. Am. J. Clin. Dermatol. 2017, 19, 363–375. [Google Scholar] [CrossRef]
- Chen, Y.-J.; Wu, C.-Y.; Chen, T.-J.; Shen, J.-L.; Chu, S.-Y.; Wang, C.-B.; Chang, Y.-T. The risk of cancer in patients with psoriasis: A population-based cohort study in Taiwan. J. Am. Acad. Dermatol. 2011, 65, 84–91. [Google Scholar] [CrossRef]
- Tsai, T.-F.; Wang, T.-S.; Hung, S.-T.; Tsai, P.I.-C.; Schenkel, B.; Zhang, M.; Tang, C.-H. Epidemiology and comorbidities of psoriasis patients in a national database in Taiwan. J. Dermatol. Sci. 2011, 63, 40–46. [Google Scholar] [CrossRef] [PubMed]
- Boffetta, P.; Gridley, G.; Lindelöf, B. Cancer Risk in a Population-Based Cohort of Patients Hospitalized for Psoriasis in Sweden. J. Investig. Dermatol. 2001, 117, 1531–1537. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Lee, J.H.; Kim, H.; Han, K.D.; Kim, H.-N.; Park, Y.M.; Lee, J.Y.; Park, Y.-G.; Lee, Y.B. Cancer risk in 892 089 patients with psoriasis in Korea: A nationwide population-based cohort study. J. Dermatol. 2018, 46, 95–102. [Google Scholar] [CrossRef] [PubMed]
- Stern, R.S.; Väkevä, L.H.; PUVA Follow-up Study. Noncutaneous malignant tumors in the PUVA follow-up study: 1975–1996. J. Investig. Dermatol. 1997, 108, 897–900. [Google Scholar] [CrossRef] [Green Version]
- Kano, Y.; Chiba, M.; Yagita, A.; Shiohara, T. Complete resolution of psoriasis vulgaris after excision of thyroid cancer. Int. J. Dermatol. 2008, 36, 280–282. [Google Scholar] [CrossRef] [PubMed]
- Fuxench, Z.C.C.; Shin, D.B.; Beatty, A.O.; Gelfand, J. The Risk of Cancer in Patients With Psoriasis: A Population-Based Cohort Study in the Health Improvement Network. JAMA Dermatol. 2016, 152, 282–290. [Google Scholar] [CrossRef] [Green Version]
- Yang, H.; Brand, J.S.; Li, J.; Ludvigsson, J.F.; Ugalde-Morales, E.; Chiesa, F.; Hall, P.; Czene, K. Risk and predictors of psoriasis in patients with breast cancer: A Swedish population-based cohort study. BMC Med. 2017, 15, 1–8. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Olsen, J.H.; Moller, H.; Frentz, G. Malignant tumors in patients with psoriasis. J. Am. Acad. Dermatol. 1992, 27, 716–722. [Google Scholar] [CrossRef]
- Pasche, B. Differential Effects of Aspirin before and after Diagnosis of Colorectal Cancer. JAMA 2013, 309, 2598–2599. [Google Scholar] [CrossRef] [PubMed]
- Erlinger, T.P.; Platz, E.A.; Rifai, N.; Helzlsouer, K.J. C-Reactive Protein and the Risk of Incident Colorectal Cancer. JAMA 2004, 291, 585–590. [Google Scholar] [CrossRef] [Green Version]
- Kane, S.; Khatibi, B.; Reddy, D. Higher Incidence of Abnormal Pap Smears in Women with Inflammatory Bowel Disease. Am. J. Gastroenterol. 2008, 103, 631–636. [Google Scholar] [CrossRef]
- Contreras, W.R.; Fuentes, H.M.; Nava, J.I.G.; Rincón, A.E.S.; Salcedo, J.V.; Rosas, M.P.; Rodríguez, L.M.B.; Trujillo, X.; Flores, M.R.; Hernández, B.T.; et al. Prevalence and cervical human papilloma virus associated factors in patients with rheumatoid arthritis. Ginecol. Obstet. México 2008, 76, 9–17. [Google Scholar]
- Santana, I.U.; Gomes, A.D.N.; Lyrio, L.D.; Grassi, M.F.; Santiago, M.B. Systemic lupus erythematosus, human papillomavirus infection, cervical pre-malignant and malignant lesions: A systematic review. Clin. Rheumatol. 2010, 30, 665–672. [Google Scholar] [CrossRef]
- Rojo-Contreras, W.; Olivas-Flores, E.M.; Gamez-Nava, J.; Montoya-Fuentes, H.; Trujillo-Hernandez, B.; Trujillo, X.; Suarez-Rincon, A.; Baltazar-Rodriguez, L.; Sanchez-Hernandez, J.; Ramirez-Flores, M.; et al. Cervical human papillomavirus infection in Mexican women with systemic lupus erythematosus or rheumatoid arthritis. Lupus 2011, 21, 365–372. [Google Scholar] [CrossRef]
- Kim, S.C.; Glynn, R.J.; Giovannucci, E.; Hernández-Díaz, S.; Liu, J.; Feldman, S.; Karlson, E.W.; Schneeweiss, S.; Solomon, D.H. Risk of high-grade cervical dysplasia and cervical cancer in women with systemic inflammatory diseases: A population-based cohort study. Ann. Rheum. Dis. 2014, 74, 1360–1367. [Google Scholar] [CrossRef]
- Takeshita, J.; Grewal, S.; Langan, S.M.; Mehta, N.N.; Ogdie, A.; Van Voorhees, A.S.; Gelfand, J.M. Psoriasis and comorbid diseases: Implications for management. J. Am. Acad. Dermatol. 2017, 76, 393–403. [Google Scholar] [CrossRef] [Green Version]
- Nijsten, T.E.; Stern, R.S. The Increased Risk of Skin Cancer Is Persistent after Discontinuation of Psoralen+Ultraviolet A: A Cohort Study. J. Investig. Dermatol. 2003, 121, 252–258. [Google Scholar] [CrossRef]
- Stern, R.S.; Study, P.F. The risk of squamous cell and basal cell cancer associated with psoralen and ultraviolet A therapy: A 30-year prospective study. J. Am. Acad. Dermatol. 2012, 66, 553–562. [Google Scholar] [CrossRef]
- Bruynzeel, I.; Bergman, W.; Hartevelt, H.; Kenter, C.; Van De Velde, E.A.; Schothorst, A.; Suurmond, D. High single-dose European PUVA regimen also causes an excess of non-melanoma skin cancer. Br. J. Dermatol. 1991, 124, 49–55. [Google Scholar] [CrossRef] [PubMed]
- Gelfand, J.M.; Shin, D.B.; Neimann, A.L.; Wang, X.; Margolis, D.J.; Troxel, A.B. The Risk of Lymphoma in Patients with Psoriasis. J. Investig. Dermatol. 2006, 126, 2194–2201. [Google Scholar] [CrossRef] [Green Version]
- Gelfand, J.M.; Berlin, J.; Van Voorhees, A.; Margolis, D.J. Lymphoma Rates Are Low but Increased in Patients With Psoriasis: Results from a population-based cohort study in the United Kingdom. Arch. Dermatol. 2003, 139, 1425–1429. [Google Scholar] [CrossRef] [PubMed]
- Stern, R.S. Lymphoma Risk in Psoriasis: Results of the PUVA follow-up study. Arch. Dermatol. 2006, 142, 1132–1135. [Google Scholar] [CrossRef]
- Ekström, K.; Hjalgrim, H.; Brandt, L.; Baecklund, E.; Klareskog, L.; Ekbom, A.; Askling, J. Risk of malignant lymphomas in patients with rheumatoid arthritis and in their first-degree relatives. Arthritis Rheum. 2003, 48, 963–970. [Google Scholar] [CrossRef] [PubMed]
- Gu, Y.; Nordstrom, B.L. The risk of malignancy among biologic-naïve pediatric psoriasis patients: A retrospective cohort study in a US claims database. J. Am. Acad. Dermatol. 2017, 77, 293–301. [Google Scholar] [CrossRef] [PubMed]
- Archier, E.; Devaux, S.; Castela, E.; Gallini, A.; Aubin, F.; Le Maître, M.; Aractingi, S.; Bachelez, H.; Cribier, B.; Joly, P.; et al. Carcinogenic risks of Psoralen UV-A therapy and Narrowband UV-B therapy in chronic plaque psoriasis: A systematic literature review. J. Eur. Acad. Dermatol. Venereol. 2012, 26, 22–31. [Google Scholar] [CrossRef]
- Rademaker, M.; Rubel, D.M.; Agnew, K.; Andrews, M.; Armour, K.S.; Baker, C.; Foley, P.; Gebauer, K.; Goh, M.S.; Gupta, M.; et al. An Australian/New Zealand Narrative. Australas. J. Dermatol. 2018, 60, 12–18. [Google Scholar] [CrossRef]
- Maiorino, A.; De Simone, C.; Perino, F.; Caldarola, G.; Peris, K. Melanoma and non-melanoma skin cancer in psoriatic patients treated with high-dose phototherapy. J. Dermatol. Treat. 2016, 27, 443–447. [Google Scholar] [CrossRef] [PubMed]
- McKenna, K.; Patterson, C.; Handley, J.; McGinn, S.; Allen, G. Cutaneous neoplasia following PUVA therapy for psoriasis. Br. J. Dermatol. 1996, 134, 639–642. [Google Scholar] [CrossRef] [PubMed]
- Paradisi, A.; Tabolli, S.; Didona, B.; Sobrino, L.; Russo, N.; Abeni, D. Reduced frequency of melanoma in 72,739 patients with psoriasis: A retrospective study. Eur. J. Dermatol. 2015, 25, 133–137. [Google Scholar] [CrossRef] [PubMed]
- Dinarello, C.A. Anti-inflammatory Agents: Present and Future. Cell 2010, 140, 935–950. [Google Scholar] [CrossRef] [Green Version]
- Kandiel, A.; Fraser, A.G.; Korelitz, B.I.; Brensinger, C.; Lewis, J.D. Increased risk of lymphoma among inflammatory bowel disease patients treated with azathioprine and 6-mercaptopurine. Gut 2005, 54, 1121–1125. [Google Scholar] [CrossRef] [Green Version]
- Masunaga, Y.; Ohno, K.; Ogawa, R.; Hashiguchi, M.; Echizen, H.; Ogata, H. Meta-Analysis of Risk of Malignancy with Immunosuppressive Drugs in Inflammatory Bowel Disease. Ann. Pharmacother. 2007, 41, 21–28. [Google Scholar] [CrossRef]
- Muellenhoff, M.W.; Koo, J.Y. Cyclosporine and skin cancer: An international dermatologic perspective over 25 years of experience. A comprehensive review and pursuit to define safe use of cyclosporine in dermatology. J. Dermatol. Treat. 2011, 23, 290–304. [Google Scholar] [CrossRef]
- Buchbinder, R.; Barber, M.; Heuzenroeder, L.; Wluka, A.; Giles, G.; Hall, S.; Harkness, A.; Lewis, D.; Littlejohn, G.; Miller, M.H.; et al. Incidence of melanoma and other malignancies among rheumatoid arthritis patients treated with methotrexate. Arthritis Rheum. 2008, 59, 794–799. [Google Scholar] [CrossRef]
- Salliot, C.; Van Der Heijde, D. Long-term safety of methotrexate monotherapy in patients with rheumatoid arthritis: A systematic literature research. Ann. Rheum. Dis. 2008, 68, 1100–1104. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Krathen, M.S.; Gottlieb, A.B.; Mease, P.J. Pharmacologic Immunomodulation and Cutaneous Malignancy in Rheumatoid Arthritis, Psoriasis, and Psoriatic Arthritis. J. Rheumatol. 2010, 37, 2205–2215. [Google Scholar] [CrossRef] [PubMed]
- Garcia-Doval, I.; Descalzo, M.; Mason, K.; Cohen, A.; Ormerod, A.; Gómez-García, F.; Cazzaniga, S.; Feldhamer, I.; Ali, H.; Herrera-Acosta, E.; et al. Cumulative exposure to biological therapy and risk of cancer in patients with psoriasis: A meta-analysis of Psonet studies from Israel, Italy, Spain, the U.K. and Republic of Ireland. Br. J. Dermatol. 2018, 179, 863–871. [Google Scholar] [CrossRef]
- Dommasch, E.; Abuabara, K.; Shin, D.B.; Nguyen, J.; Troxel, A.; Gelfand, J.M. The risk of infection and malignancy with tumor necrosis factor antagonists in adults with psoriatic disease: A systematic review and meta-analysis of randomized controlled trials. J. Am. Acad. Dermatol. 2011, 64, 1035–1050. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Peleva, E.; Exton, L.; Kelley, K.; Kleyn, C.; Mason, K.; Smith, C.H. Risk of cancer in patients with psoriasis on biological therapies: A systematic review. Br. J. Dermatol. 2017, 178, 103–113. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Abuabara, K.; Azfar, R.; Shin, D.; Neimann, A.; Troxel, A.; Gelfand, J. Cause-specific mortality in patients with severe psoriasis: A population-based cohort study in the U.K. Br. J. Dermatol. 2010, 163, 586–592. [Google Scholar] [CrossRef] [Green Version]
- Shu, X.; Ji, J.; Sundquist, J.; Sundquist, K.; Hemminki, K. Survival in cancer patients hospitalized for psoriasis: A population-based cohort study in Sweden. Br. J. Dermatol. 2011, 165, 129–136. [Google Scholar] [CrossRef]
- Reddy, S.; Martires, K.; Wu, J.J. The risk of melanoma and hematologic cancers in patients with psoriasis. J. Am. Acad. Dermatol. 2017, 76, 639–647. [Google Scholar] [CrossRef]
- Elder, J.T.; Bruce, A.T.; Gudjonsson, J.E.; Johnston, A.; Stuart, P.E.; Tejasvi, T.; Voorhees, J.J.; Abecasis, G.; Nair, R.P. Molecular Dissection of Psoriasis: Integrating Genetics and Biology. J. Investig. Dermatol. 2010, 130, 1213–1226. [Google Scholar] [CrossRef] [Green Version]
- Beyaert, R.; Beaugerie, L.; Van Assche, G.; Brochez, L.; Renauld, J.-C.; Viguier, M.; Cocquyt, V.; Jerusalem, G.; Machiels, J.-P.; Prenen, H.; et al. Cancer risk in immune-mediated inflammatory diseases (IMID). Mol. Cancer 2013, 12, 98. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Psoriasis + Cancer (%) | Other Dermatoses + Cancer (%) | |
---|---|---|
No. Patients | 40 | 399 |
Sex | ||
Male | 26 (65%) | 221 (55.39%) |
Female | 14 (35%) | 178 (44.61%) |
Mean age | 60.87 | 65.93 |
Area of residence | ||
Urban | 25 (62.50%) | 220 (55.14%) |
Rural | 15 (37.50%) | 179 (44.86%) |
Cancer Types * | Psoriasis | Other Dermatoses | Chi2 Test | Odd Ratio | Relative Risk | ||
---|---|---|---|---|---|---|---|
n (%) | n (%) | p | OR | 95%CI | RR | 95%CI | |
CNS | 3 (7.5%) | 2 (0.5%) | 0.001 | 16.09 | 2.61–99.4 | 15.0 | 2.58–86.9 |
Upper aerodigestive | 4 (10.0%) | 9 (2.3%) | 0.006 | 4.81 | 1.41–16.41 | 4.43 | 1.43–13.75 |
Endocrine | 3 (7.5%) | 10 (2.5%) | 0.076 | 3.15 | 0.83–11.97 | 2.99 | 0.86–10.43 |
Bladder | 3 (7.5%) | 10 (2.5%) | 0.076 | 3.15 | 0.83–11.97 | 2.99 | 0.86–10.43 |
Lung | 3 (7.5%) | 12 (3.0%) | 0.136 | 2.61 | 0.71–9.68 | 2.49 | 0.73–8.47 |
Prostate | 3 (7.5%) | 21 (5.3%) | 0.351 | 1.46 | 0.42–5.12 | 1.43 | 0.44–4.57 |
Breast | 4 (10.0%) | 29 (7.3%) | 0.533 | 1.42 | 0.47–4.26 | 1.38 | 0.51–3.72 |
Colorectal | 5 (12.5%) | 40 (10.0%) | 0.242 | 1.28 | 0.48–3.46 | 1.25 | 0.52–2.98 |
Cervical | 2 (5.0%) | 16 (4.0%) | 0.764 | 1.26 | 0.28–5.69 | 1.25 | 0.30–5.23 |
Liver | 1 (2.5%) | 8 (2.0%) | 0.833 | 1.25 | 0.15–10.28 | 1.25 | 0.16–9.72 |
Blood | 5 (12.5%) | 58 (14.5%) | 0.726 | 0.90 | 0.34–2.38 | 0.86 | 0.37–2.02 |
Cutaneous lymphomas | 2 (5.0%) | 41 (10.3%) | 0.285 | 0.46 | 0.11–1.98 | 0.49 | 0.12–1.94 |
Melanomas | 1 (2.5%) | 53 (9.8%) | 0.048 | 0.17 | 0.002–1.29 | 0.19 | 0.03–1.32 |
Kaposi sarcoma | 1 (2.5%) | 57 (14.3%) | 0.036 | 0.15 | 0.002–1.14 | 0.18 | 0.02–1.23 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Gheucă-Solovăstru, L.; Vâţă, D.; Halip, A.I.; Patraşcu, A.; Cozma, A.; Porumb-Andrese, E. Psoriasis—A Cancer Risk Factor? Appl. Sci. 2021, 11, 8366. https://doi.org/10.3390/app11188366
Gheucă-Solovăstru L, Vâţă D, Halip AI, Patraşcu A, Cozma A, Porumb-Andrese E. Psoriasis—A Cancer Risk Factor? Applied Sciences. 2021; 11(18):8366. https://doi.org/10.3390/app11188366
Chicago/Turabian StyleGheucă-Solovăstru, Laura, Dan Vâţă, Alina Ioana Halip, Adriana Patraşcu, Andreea Cozma, and Elena Porumb-Andrese. 2021. "Psoriasis—A Cancer Risk Factor?" Applied Sciences 11, no. 18: 8366. https://doi.org/10.3390/app11188366
APA StyleGheucă-Solovăstru, L., Vâţă, D., Halip, A. I., Patraşcu, A., Cozma, A., & Porumb-Andrese, E. (2021). Psoriasis—A Cancer Risk Factor? Applied Sciences, 11(18), 8366. https://doi.org/10.3390/app11188366