A Specific Inflammatory Profile Underlying Suicide Risk? Systematic Review of the Main Literature Findings
Abstract
:1. Introduction
2. Methods
2.1. Search Strategy and Study Selection
2.2. Study Design and Eligibility Criteria
2.3. Selection Criteria
2.4. Recorded Variables
3. Results
3.1. Number of Selected Studies
3.2. Studies Directly Analyzing Inflammatory Abnormalities in Suicidal (Completed Suicides or Those with a History of Suicide Attempts) Patients
3.3. Studies Analyzing Inflammatory Abnormalities in Suicidal Patients (with a Positive History of Suicide Attempts and Suicidal Ideation)
3.4. Studies Exploring Inflammatory Abnormalities in Depressed Patients with Suicidal Ideation vs. Those Without
3.5. Studies Investigating Inflammatory Abnormalities in Patients with a History of Suicide Attempts vs. Those Without
3.6. Studies Exploring Cytokines Abnormalities in Suicide Completers vs. Subjects Who Died for Other Causes
3.7. Studies Investigating Inflammatory Abnormalities in Depressed Patients Who May be Indirectly at Risk for Suicide
4. Discussion
Summary of Main Findings and Review of Study Designs
5. Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
References
- Bachmann, S. Epidemiology of Suicide and the Psychiatric Perspective. Int. J. Environ. Res. Public Health 2018, 15, 1425. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Pompili, M.; Gibiino, S.; Innamorati, M.; Serafini, G.; Del Casale, A.; De Risio, L.; Palermo, M.; Montebovi, F.; Campi, S.; De Luca, V.; et al. Prolactin and thyroid hormone levels are associated with suicide attempts in psychiatric patients. Psychiatry Res. 2012, 200, 389–394. [Google Scholar] [CrossRef] [PubMed]
- Pompili, M.; Shrivastava, A.; Serafini, G.; Innamorati, M.; Milelli, M.; Erbuto, D.; Ricci, F.; Lamis, D.A.; Scocco, P.; Amore, M. Bereavement after the suicide of a significant other. Indian J. Psychiatry 2013, 55, 256–263. [Google Scholar] [CrossRef] [PubMed]
- Pandey, G.N.; Rizavi, H.S.; Zhang, H.; Bhaumik, R.; Ren, X. Abnormal protein and mRNA expression of inflammatory cytokines in the prefrontal cortex of depressed individuals who died by suicide. J. Psychiatry Neurosci. 2018, 43, 376–385. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Courtet, P.; Giner, L.; Seneque, M.; Guillaume, S.; Olie, E.; Ducasse, D. Neuroinflammation in suicide: Toward a comprehensive model. World J. Biol. Psychiatry 2016, 17, 564–586. [Google Scholar] [CrossRef] [PubMed]
- Black, C.; Miller, B.J. Meta-analysis of cytokines and chemokines in suicidality: Distinguishing suicidal versus nonsuicidal patients. Biol. Psychiatry 2015, 78, 28–37. [Google Scholar] [CrossRef]
- Serafini, G.; Pompili, M.; Seretti, E.M.; Stefani, H.; Palermo, M.; Coryell, W.; Girardi, P. The role of inflammatory cytokines in suicidal behavior: A systematic review. Eur. Neuropsychopharmacol. 2013, 23, 1672–1686. [Google Scholar] [CrossRef]
- Lindqvist, D.; Janelidze, S.; Hagell, P.; Erhardt, S.; Samuelsson, M.; Minthon, L.; Hansson, O.; Björkqvist, M.; Träskman-Bendz, L.; Brundin, L. Interleukin-6 is elevated in the cerebrospinal fluid of suicide attempters and related to symptom severity. Biol. Psychiatry 2009, 66, 287–292. [Google Scholar] [CrossRef]
- Janelidze, S.; Mattei, D.; Westrin, A.; Träskman-Bendz, L.; Brundin, L. Cytokine levels in the blood may distinguish suicide attempters from depressed patients. Brain Behav. Immun. 2011, 25, 335–339. [Google Scholar] [CrossRef] [Green Version]
- Ducasse, D.; Olié, E.; Guillaume, S.; Artéro, S.; Courtet, P. A meta-analysis of cytokines in suicidal behavior. Brain Behav. Immun. 2015, 46, 203–211. [Google Scholar] [CrossRef]
- Shelton, R.C.; Claiborne, J.; Sidoryk-Wegrzynowicz, M.; Reddy, R.; Aschner, M.; Lewis, D.A.; Mirnics, K. Altered expression of genes involved in inflammation and apoptosis in frontal cortex in major depression. Mol. Psychiatry 2011, 16, 751–762. [Google Scholar] [CrossRef] [PubMed]
- Pandey, G.N.; Rizavi, H.S.; Ren, X.; Fareed, J.; Hoppensteadt, D.A.; Roberts, R.C.; Conley, R.R.; Dwivedi, Y. Proinflammatory cytokines in the prefrontal cortex of teenage suicide victims. J. Psychiatr. Res. 2012, 46, 57–63. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Grunebaum, M.F.; Ramsay, S.R.; Galfalvy, H.C.; Ellis, S.P.; Burke, A.K.; Sher, L.; Printz, D.J.; Kahn, D.A.; Mann, J.J.; Oquendo, M.A. Correlates of suicide attempt history in bipolar disorder: A stress-diathesis perspective. Bipolar Disord. 2006, 8, 551–557. [Google Scholar] [CrossRef] [PubMed]
- Eisenberger, N.I.; Moieni, M.; Inagaki, T.K.; Muscatell, K.A.; Irwin, M.R. In sickness and in health: The Co-regulation of inflammation and social behavior. Neuropsychopharmacology 2017, 42, 242–253. [Google Scholar] [CrossRef]
- Moieni, M.; Eisenberger, N.I. Effects of inflammation on social processes and implications for health: Effect of inflammation on social processes. Ann. N. Y. Acad. Sci. 2018, 1428, 5–13. [Google Scholar] [CrossRef]
- Haapakoski, R.; Mathieu, J.; Ebmeier, K.P.; Alenius, H.; Kivimäki, M. Cumulative meta-analysis of interleukins 6 and 1β, tumour necrosis factor α and C-reactive protein in patients with major depressive disorder. Brain Behav. Immun. 2015, 49, 206–215. [Google Scholar] [CrossRef] [Green Version]
- Stieglitz, J.; Trumble, B.C.; Thompson, M.E.; Blackwell, A.D.; Kaplan, H.; Gurven, M. Depression as sickness behavior? A test of the host defense hypothesis in a high pathogen population. Brain Behav. Immun. 2015, 49, 130–139. [Google Scholar] [CrossRef] [Green Version]
- Gimeno, D.; Kivimäki, M.; Brunner, E.J.; Elovainio, M.; De Vogli, R.; Steptoe, A.; Kumari, M.; Lowe, G.D.O.; Rumley, A.; Marmot, M.G.; et al. Associations of Creactive protein and interleukin-6 with cognitive symptoms of depression: 12-year follow-up of the Whitehall II study. Psychol. Med. 2009, 39, 413–423. [Google Scholar] [CrossRef]
- Wium-Andersen, M.K.; Ørsted, D.D.; Nielsen, S.F.; Nordestgaard, B.G. Elevated Creactive protein levels, psychological distress, and depression in 73 131 individuals. JAMA Psychiatry 2013, 70, 176. [Google Scholar] [CrossRef] [Green Version]
- Miller, A.H.; Raison, C.L. The role of inflammation in depression: From evolutionary imperative to modern treatment target. Nat. Rev. Immunol. 2016, 16, 22–34. [Google Scholar] [CrossRef] [Green Version]
- Mondelli, V.; Vernon, A.C.; Turkheimer, F.; Dazzan, P.; Pariante, C.M. Brain microglia in psychiatric disorders. Lancet Psychiatry 2017, 4, 563–572. [Google Scholar] [CrossRef] [Green Version]
- Rajkowska, G.; Legutko, B.; Moulana, M.; Syed, M.; Romero, D.G.; Stockmeier, C.A.; Miguel-Hidalgo, J.J. Astrocyte pathology in the ventral prefrontal white matter in depression. J. Psychiatr. Res. 2018, 102, 150–158. [Google Scholar] [CrossRef] [PubMed]
- Mechawar, N.; Savitz, J. Neuropathology of mood disorders: Do we see the stigmata of inflammation? Transl. Psychiatry 2016, 6, e946. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Liberati, A.; Altman, D.G.; Tetzlaff, J.; Mulrow, C.; Gøtzsche, P.C.; Ioannidis, J.P.; Clarke, M.; Devereaux, P.J.; Kleijnen, J.; Moher, D. The Prisma statement for reporting systematic reviews and metaanalyses of studies that evaluate healthcare interventions: Explanation and elaboration. BMJ 2009, 339, b2700. [Google Scholar] [CrossRef] [Green Version]
- Conejero, I.; Jaussent, I.; Cazals, A.; Thouvenot, E.; Mura, T.; Le Bars, E.; Guillaume, S.; Squalli, S.; Courtet, P.; Olié, E. Association between baseline pro-inflammatory cytokines and brain activation during social exclusion in patients with vulnerability to suicide and depressive disorder. Psychoneuroendocrinology 2019, 99, 236–242. [Google Scholar] [CrossRef]
- Pandey, G.N.; Rizavi, H.S.; Ren, X.; Dwivedi, Y.; Palkovits, M. Region-specific alterations in glucocorticoid receptor expression in the postmortem brain of teenage suicide victims. Psychoneuroendocrinology 2013, 38, 2628–2639. [Google Scholar] [CrossRef] [Green Version]
- Wang, Q.; Roy, B.; Turecki, G.; Shelton, R.C.; Dwivedi, Y. Role of complex epi-genetic switching in tumor necrosis factor-alpha upregulation in the prefrontal cortexof suicide subjects. Am. J. Psychiatry 2018, 175, 262–274. [Google Scholar] [CrossRef] [Green Version]
- Bay-Richter, C.; Linderholm, K.R.; Lim, C.K.; Samuelsson, M.; Träskman-Bendz, L.; Guillemin, G.J.; Erhardt, S.; Brundin, L. A role for inflammatory metabolites as modulators of the glutamate N-methyl-D-aspartate receptor in depression and suicidality. Brain Behav. Immun. 2015, 43, 110–117. [Google Scholar] [CrossRef]
- Sublette, M.E.; Galfalvy, H.C.; Fuchs, D.; Lapidus, M.; Grunebaum, M.F.; Oquendo, M.A.; Mann, J.J.; Postolache, T.T. Plasma kynurenine levels are elevated in suicide attempters with major depressive disorder. Brain Behav. Immun. 2011, 25, 1272–1278. [Google Scholar] [CrossRef] [Green Version]
- Gabbay, V.; Klein, R.G.; Guttman, L.E.; Babb, J.S.; Alonso, C.M.; Nishawala, M.; Katz, Y.; Gaite, M.R.; Gonzalez, C.J. A preliminary study of cytokines in suicidal and nonsuicidal adolescents with major depression. J. Child Adolesc. Psychopharmacol. 2009, 19, 423–430. [Google Scholar] [CrossRef]
- Kim, Y.K.; Lee, S.W.; Kim, S.H.; Shim, S.H.; Han, S.W.; Choi, S.H.; Lee, B.H. Differences in cytokines between non-suicidal patients and suicidal patients in major depression. Prog. Neuropsychopharmacol. Biol. Psychiatry 2008, 32, 356–361. [Google Scholar] [CrossRef] [PubMed]
- Lee, K.M.; Kim, Y.K. The role of IL-12 and TGF-beta1 in the pathophysiology of major depressive disorder. Int. Immunopharmacol. 2006, 6, 1298–1304. [Google Scholar] [CrossRef] [PubMed]
- Knowles, E.E.M.; Curran, J.E.; Göring, H.H.H.; Mathias, S.R.; Mollon, J.; Rodrigue, A.; Olvera, R.L.; Leandro, A.; Duggirala, R.; Almasy, L.; et al. Family-based analyses reveal novel genetic overlap between cytokine interleukin-8 and risk for suicide attempt. Brain Behav. Immun. 2019, 80, 292–299. [Google Scholar] [CrossRef]
- Keaton, S.A.; Madaj, Z.B.; Heilman, P.; Smart, L.; Grit, J.; Gibbons, R.; Postolache, T.T.; Roaten, K.; Achtyes, E.D.; Brundin, L. An inflammatory profile linked to increased suicide risk. J. Affect. Disord. 2019, 247, 57–65. [Google Scholar] [CrossRef] [PubMed]
- Melhem, N.M.; Munroe, S.; Marsland, A.; Gray, K.; Brent, D.; Porta, G.; Douaihy, A.; Laudenslager, M.L.; DePietro, F.; Diler, R.; et al. Blunted HPA axis activity prior to suicide attempt and increased inflammation in attempters. Psychoneuroendocrinology 2017, 77, 284–294. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Mendlovic, S.; Mozes, E.; Eilat, E.; Doron, A.; Lereya, J.; Zakuth, V.; Spirer, Z. Immune activation in non-treated suicidal major depression. Immunol. Lett. 1999, 67, 105–108. [Google Scholar] [CrossRef]
- O’Donovan, A.; Rush, G.; Hoatam, G.; Hughes, B.M.; McCrohan, A.; Kelleher, C.; O’Farrelly, C.; Malone, K.M. Suicidal ideation is associated with elevated inflammation in patients with major depressive disorder. Depress. Anxiety 2013, 30, 307–314. [Google Scholar] [CrossRef]
- Grassi-Oliveira, R.; Brieztke, E.; Teixeira, A.; Pezzi, J.C.; Zanini, M.; Lopes, R.P.; Bauer, M.E. Peripheral chemokine levels in women with recurrent major depression with suicidal ideation. Braz. J. Psychiatry 2012, 34, 71–75. [Google Scholar] [CrossRef] [Green Version]
- Coryell, W.; Wilcox, H.; Evans, S.J.; Pandey, G.N.; Jones-Brando, L.; Dickerson, F.; Yolken, R. Aggression, impulsivity and inflammatory markers as risk factors for suicidal behavior. J. Psychiatr. Res. 2018, 106, 38–42. [Google Scholar] [CrossRef]
- Janelidze, S.; Ventorp, F.; Erhardt, S.; Hansson, O.; Minthon, L.; Flax, J.; Samuelsson, M.; Traskman-Bendz, L.; Brundin, L. Altered chemokine levels in the cerebrospinal fluid and plasma of suicide attempters. Psychoneuroendocrinology 2013, 38, 853–862. [Google Scholar] [CrossRef]
- Vargas, H.O.; Nunes, S.O.; Pizzo de Castro, M.; Bortolasci, C.C.; Sabbatini Barbosa, D.; Kaminami Morimoto, H.; Venugopal, K.; Dodd, S.; Maes, M.; Berk, M. Oxidative stress and lowered total antioxidant status are associated with a history of suicide attempts. J. Affect. Disord. 2013, 150, 923–930. [Google Scholar] [CrossRef]
- Isung, J.; Mobarrez, F.; Nordström, P.; Asberg, M.; Jokinen, J. Low plasma vascular endothelial growth factor (VEGF) associated with completed suicide. World J. Biol. Psychiatry 2012, 13, 468–473. [Google Scholar] [CrossRef] [PubMed]
- Isung, J.; Aeinehband, S.; Mobarrez, F.; Mårtensson, B.; Nordström, P.; Asberg, M.; Piehl, F.; Jokinen, J. Low vascular endothelial growth factor and interleukin-8 in cerebrospinal fluid of suicide attempters. Transl. Psychiatry 2012, 2, e196. [Google Scholar] [CrossRef] [PubMed]
- Nässberger, L.; Träskman-Bendz, L. Increased soluble interleukin-2 receptor concentrations in suicide attempters. Acta Psychiatr. Scand. 1993, 88, 48–52. [Google Scholar] [CrossRef] [PubMed]
- Boehm, J.; Fischer, K.; Bohnert, M. Putative role of TNF-alpha, interleukin-8 and ICAM-1 as indicators of an early inflammatory reaction after burn: A morphological and immunohistochemical study of lung tissue of fire victims. J. Clin. Pathol. 2010, 63, 967–971. [Google Scholar] [CrossRef] [PubMed]
- Tonelli, L.H.; Stiller, J.; Rujescu, D.; Giegling, I.; Schneider, B.; Maurer, K.; Schnabel, A.; Möller, H.J.; Chen, H.H.; Postolache, T.T. Elevated cytokine expression in the orbitofrontal cortex of victims of suicide. Acta Psychiatr. Scand. 2008, 117, 198–206. [Google Scholar] [CrossRef] [Green Version]
- Torres-Platas, S.G.; Cruceanu, C.; Chen, G.G.; Turecki, G.; Mechawar, N. Evidence for increased microglial priming and macrophage recruitment in the dorsal anterior cingulate white matter of depressed suicides. Brain Behav. Immun. 2014, 42, 50–59. [Google Scholar] [CrossRef]
- Dunjic-Kostic, B.; Ivkovic, M.; Radonjic, N.V.; Petronijevic, N.D.; Pantovic, M.; Damjanovic, A.; Poznanovic, S.T.; Jovanovic, A.; Nikolic, T.; Jasovic-Gasic, M. Melancholic and atypical major depression--connection between cytokines, psychopathology and treatment. Prog. Neuropsychopharmacol. Biol. Psychiatry 2013, 43, 1–6. [Google Scholar] [CrossRef]
- Li, Z.; Qi, D.; Chen, J.; Zhang, C.; Yi, Z.; Yuan, C.; Wang, Z.; Hong, W.; Yu, S.; Cui, D.; et al. Venlafaxine inhibits the upregulation of plasma tumor necrosis factor-alpha (TNFα) in the Chinese patients with major depressive disorder: A prospective longitudinal study. Psychoneuroendocrinology 2013, 38, 107–114. [Google Scholar] [CrossRef]
- Kim, Y.K.; Hong, J.P.; Hwang, J.A.; Lee, H.J.; Yoon, H.K.; Lee, B.H.; Jung, H.Y.; Hahn, S.W.; Na, K.S. TNF-alpha -308G>A polymorphism is associated with suicide attempts in major depressive disorder. J. Affect. Disord. 2013, 150, 668–672. [Google Scholar] [CrossRef]
- Wiener, C.D.; Moreira, F.P.; Portela, L.V.; Strogulski, N.R.; Lara, D.R.; da Silva, R.A.; Souza, L.D.M.; Jansen, K.; Oses, J.P. Interleukin-6 and Interleukin-10 in mood disorders: A population-based study. Psychiatry Res. 2019, 273, 685–689. [Google Scholar] [CrossRef] [PubMed]
- Enache, D.; Pariante, C.M.; Mondelli, V. Markers of central inflammation in major depressive disorder: A systematic review and meta-analysis of studies examining cerebrospinal fluid, positron emission tomography and post-mortem brain tissue. Brain Behav. Immun. 2019, 81, 24–40. [Google Scholar] [CrossRef] [PubMed]
- Brundin, L.; Erhardt, S.; Bryleva, E.Y.; Achtyes, E.D.; Postolache, T.T. The role of inflammation in suicidal behaviour. Acta Psychiatr. Scand. 2015, 132, 192–203. [Google Scholar] [CrossRef] [PubMed]
- Ackerman, K.D.; Martino, M.; Heyman, R.; Moyna, N.M.; Rabin, B.S. Stressor induced alteration of cytokine production in multiple sclerosis patients and controls. Psychosom. Med. 1998, 60, 484–491. [Google Scholar] [CrossRef] [PubMed]
- Maes, M. Depression is an inflammatory disease, but cellmediated immune activation is the key component of depression. Prog. Neuropsychopharmacol. Biol. Psychiatry 2011, 35, 664–675. [Google Scholar] [CrossRef] [PubMed]
- Maes, M. Major depression and activation of the inflammatory response system. Adv. Exp. Med. Biol. 1999, 461, 25–46. [Google Scholar] [CrossRef] [PubMed]
- Walker, A.K.; Kavelaars, A.; Heijnen, C.J.; Dantzer, R. Neuroinflammation and comorbidity of pain and depression. Pharmacol. Rev. 2013, 66, 80–101. [Google Scholar] [CrossRef] [Green Version]
- Lopresti, A.L.; Maker, G.L.; Hood, S.D.; Drummond, P.D. A review of peripheral biomarkers in major depression: The potential of inflammatory and oxidative stress biomarkers. Prog. Neuropsychopharmacol. Biol. Psychiatry 2014, 48, 102–111. [Google Scholar] [CrossRef] [Green Version]
- Serafini, G.; Adavastro, G.; Canepa, G.; Capobianco, L.; Conigliaro, C.; Pittaluga, F.; Murri, M.B.; Valchera, A.; De Berardis, D.; Pompili, M.; et al. Abnormalities in Kynurenine Pathway Metabolism in Treatment-Resistant Depression and Suicidality: A Systematic Review. CNS Neurol. Disord. Drug. Targets 2017, 16, 440–453. [Google Scholar] [CrossRef]
- Dantzer, R.; O’Connor, J.C.; Lawson, M.A.; Kelley, K.W. Inflammation-associated depression: From serotonin to kynurenine. Psychoneuroendocrinology 2011, 36, 426–436. [Google Scholar] [CrossRef] [Green Version]
- Müller, N.; Myint, A.M.; Schwarz, M.J. The impact of neuroimmune dysregulation on neuroprotection and neurotoxicity in psychiatric disorders--relation to drug treatment. Dialogues Clin. Neurosci. 2009, 11, 319–332. [Google Scholar] [PubMed]
- Raison, C.L.; Borisov, A.S.; Broadwell, S.D.; Capuron, L.; Woolwine, B.J.; Jacobson, I.M.; Nemeroff, C.B.; Miller, A.H. Depression during pegylated interferon-alpha plus ribavirin therapy: Prevalence and prediction. J. Clin. Psychiatry 2005, 66, 41–48. [Google Scholar] [CrossRef] [PubMed]
- Fragoso, Y.D.; Frota, E.R.; Lopes, J.S.; Noal, J.S.; Giacomo, M.C.; Gomes, S.; Gonçalves, M.V.; da Gama, P.D.; Finkelsztejn, A. Severe depression, suicide attempts, and ideation during the use of interferon beta by patients with multiple sclerosis. Clin. Neuropharmacol. 2010, 33, 312–316. [Google Scholar] [CrossRef] [PubMed]
- Cacioppo, S.; Frum, C.; Asp, E.; Weiss, R.M.; Lewis, J.W.; Cacioppo, J.T. A quantitative meta-analysis of functional imaging studies of social rejection. Sci. Rep. 2013, 3, 1–3. [Google Scholar] [CrossRef] [PubMed]
- Holmes, S.E.; Hinz, R.; Conen, S.; Gregory, C.J.; Matthews, J.C.; Anton-Rodriguez, J.M.; Gerhard, A.; Talbot, P.S. Elevated translocator protein in anterior cingulate in major depression and a role for inflammation in suicidal thinking: A positron emission tomography study. Biol. Psychiatry 2018, 83, 61–69. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Olié, E.; Jollant, F.; Deverdun, J.; de Champfleur, N.M.; Cyprien, F.; Le Bars, E.; Mura, T.; Bonafé, A.; Courtet, P. The experience of social exclusion in women with a history of suicidal acts: A neuroimaging study. Sci. Rep. 2017, 7, 1–8. [Google Scholar] [CrossRef]
- Chase, H.W.; Segreti, A.M.; Keller, T.A.; Cherkassky, V.L.; Just, M.A.; Pan, L.A.; Brent, D.A. Alterations of functional connectivity and intrinsic activity within the cingulate cortex of suicidal ideators. J. Affect. Disord. 2017, 212, 78–85. [Google Scholar] [CrossRef] [Green Version]
- Kim, J.W.; Szigethy, E.M.; Melhem, N.M.; Saghafi, E.M.; Brent, D.A. Inflammatory markers and the pathogenesis of pediatric depression and suicide: A systematic review of the literature. J. Clin. Psychiatry 2014, 75, 1242–1253. [Google Scholar] [CrossRef]
- Khandaker, G.M.; Cousins, L.; Deakin, J.; Lennox, B.R.; Yolken, R.; Jones, P.B. Inflammation and immunity in schizophrenia: Implications for pathophysiology and treatment. Lancet Psychiatry 2015, 2, 258–270. [Google Scholar] [CrossRef] [Green Version]
- Cipriani, A.; Zhou, X.; Del Giovane, C.; Hetrick, S.E.; Qin, B.; Whittington, C.; Coghill, D.; Zhang, Y.; Hazell, P.; Leucht, S.; et al. Comparative efficacy and tolerability of antidepressants for major depressive disorder in children and adolescents: A network meta-analysis. Lancet 2016, 388, 881–890. [Google Scholar] [CrossRef]
- Luft, M.J.; Lamy, M.; DelBello, M.P.; McNamara, R.K.; Strawn, J.R. Antidepressant-Induced Activation in Children and Adolescents: Risk, Recognition and Management. Curr. Probl. Pediatr. Adolesc. Health Care 2018, 48, 50–62. [Google Scholar] [CrossRef] [PubMed]
- Maes, M. Evidence for an immune response in major depression: A review and hypothesis. Prog. Neuropsychopharmacol. Biol. Psychiatry 1995, 19, 11–38. [Google Scholar] [CrossRef]
Author (s), Year | Sample Characteristics | Study Design | Inflammation Measurement | Main Findings | Limitations | Conclusions |
---|---|---|---|---|---|---|
Conejerob et al. (2019) [25] | 42 SA, 40 affective controls, 19 healthy controls | Cross-sectional case-control study | IL-1β IL-6 TNF-α IL-2 | IL-1β was negatively associated with right orbitofrontal cortex activation in ESE vs. INC, whereas IL-2 was positively associated with activation of the right anterior cingulate cortex, insula, and orbitofrontal cortex in ESE vs. INC. | 1. Medicated patients reflecting real-life conditions were recruited; 2. only females were included; 3. results for smoking status and BMI were not controlled; 4. the healthy controls group was smaller than the affective controls and SA groups. | Baseline IL-1β and IL-2 blood levels are differentially associated with cerebral activation involved in the perception of social exclusion, independently of suicidal behavior. |
Wang et al. (2019) [27] | 16 non-psychiatric controls and 43 suicide subjects (21 MDD-suicides and 22 suicides with other psychiatric disorders) | Cross-sectional case-control study | TNF-α | TNF-α expression was significantly higher in dlPFC of suicide subjects regardless of psychiatric diagnosis. Its expression level was also increased in MDD subjects who died by causes other than suicide. Conversely, the expression of miR-19a-3p was upregulated in suicide subjects. | 1. The power of the study is quite low; 2. the study lacks depressed patients without suicidal ideation. | This study provides mechanistic insights into the dysregulation of TNF-α gene in suicide brain, which could potentially be involved in suicidal behavior. |
Keaton et al. (2019) [34] | 66 females with mood and anxiety disorders | Cross-sectional study | IL-6 IL-8 | Increased IL-6, lymphocytes, monocytes, white blood cell count, and polymorphonuclear leukocyte count levels significantly impacted suicide risk (the latter two inferring the strongest influence). IL-8 was independently and negatively associated with enhanced suicide risk, even after adjusting for confounders. | 1. The cross-sectional study design; 2. the findings may only identify associations and are not able to prove causality. | The biological profile of patients assessed to be at increased suicide risk differed from that associated with depression. |
Knowles et al. (2019) [33] | 1882 subjects of which 159 with SA and 135 with SI | Cross-sectional study | IL-6 IL-8 | IL-8 and IL-6 shared significant genetic overlap with risk for SA (for IL-6 this was attenuated when BMI was included as a covariate). The genetic overlap between IL-8 and SA risk was significant only in females but not males. | 1. Personality traits could confound or mediate the main results; 2. the results may be population (Mexican American) specific; 3. increased circulating cytokine levels were observed in response to stress; 4. environmental influences might have increased inflammation. | Cytokine abnormalities are not a secondary manifestation of suicidal behavior, but play a fundamental role in the pathophysiology of suicide attempts. |
Coryell et al. (2018) [39] | Patients in major depressive episodes who had a history of two or more SA (n = 79), or no history of SA (n = 123) | Cross-sectional study | CRP IL-6 IL-1β IL-1ra TNF-α. | One of five of the inflammatory markers (IL-1β), distinguished the two groups with lower values in the SA group. IL-1β levels correlated inversely with measures of aggression but neither impulsivity or aggressive behavior appear to explain the association between IL-1β levels and SA status. | 1. Small sample sizes; 2. patients described here were not treatment naïve; 3. antidepressant treatment may or may not have influenced cytokine levels. | Results identify recent aggressive behavior, higher levels of impulsivity, and lower levels of IL-1β as risk factors for a history of multiple SA in a group with major depressive episodes. These measures appear to be additive in their effects. |
Melhem et al. (2018) [35] | 38 with SA, 40 with SI, 37 healthy controls | Cross-sectional case-control study | HCC | Lower HCC [β = −0.55, 95% CI (−0.96, −0.13), p = 0.01, ES = −0.54] were found in those with SA for the first time compared to those with SI and controls. | 1. It is not clear whether HPA axis dysregulation exists prior to suicidal behavior or as a consequence of an attempt because existing studies examine cortisol levels in subjects with history of suicidal behaviour; 2. small sample size; 3. cross-sectional study design. | This is the first study to differentiate youths who attempt suicide from those with SI on HCC. The present study also showed that low HCC precedes SA. |
Pandey et al. (2018) [4] | 24 depressed individuals who died by suicide and 24 non-psychiatric controls | Cross-sectional case-control study | IL-1β, IL-6, TNF-α, Lymphotoxin A, Lymphotoxin B, IL-8, IL-10, IL-13 | Protein levels of IL-1β, IL-6, TNF-α, and lymphotoxin a were significantly increased, and levels of anti-inflammatory cytokine IL-10 and of IL-1ra were significantly reduced in the prefrontal cortex of depressed individuals who died by suicide compared with controls. | 1. Some of the suicide group had been taking antidepressant medications at the time of death. | Alterations of cytokines may be associated with the pathophysiology of depressed suicide and there may be an imbalance between pro- and anti-inflammatory cytokines in subjects who died by suicide. |
Torres-Platas et al. (2015) [47] | 24 depressed suicides and 17 controls (without psychiatric, neurological or inflammatory illnesses) | Cross-sectional study | Gene expression of IBA1 and MCP-1 | Gene expression of IBA1 and MCP-1 was upregulated in depressed suicides. In addition, mRNA for CD45 was also increased in depressed suicides. An increase compared to controls was found in the proportion of blood vessels surrounded by a high density of CD45-IR cells (non-significant difference). | 1. Brains from depressed suicides may be more highly exposed to peripheral cytokines crossing the blood-brain barrier; 2. other types of macrophages (including microglia), together with perivascular macrophages, may account for the observed increase in IBA1-IR cells associated with blood vessels in case samples. | Depression- and suicide-associated increases in circulating pro-inflammatory cytokines may be linked to low-grade cerebral neuroinflammation involving the recruitment of circulating monocytes. |
Pandey et al. (2013) [26] | 24 teenage suicide victims and 24 matched normal controls | Cross-sectional case-control study | IL-1β, IL-6, TNF-α | The mRNA and protein expression levels of IL-1β, IL-6, and TNF-α were significantly increased in Brodmann area 10 of suicide victims compared with normal controls. | 1. Stress effects (with associated changes in GR-α) on the hippocampus cannot be observed in children or adolescents, but only in adults; 2. changes in the GR-α may be not related to suicidal behavior but may change as a result of early life trauma. | Increases of pro-inflammatory cytokines in the post-mortem brain of teenage suicide victims suggest that TNF-α, IL-1β, or IL-6 are associated with the neurobiology of suicide and that targeting these cytokines may help in developing new therapies for the treatment of suicidal behavior. |
Kim et al., (2013) [50] | 204 patients with SA and 97 control patients without SA | Cross-sectional case-control study | TNF-α, IL-10, IFN-γ | The GG genotype of the TNF-α −308G>A polymorphism significantly increased SA risk. IFN-γ +874A>T and IL-10 −1082A>G were not associated with risk for suicide. Lethality of the SA was not associated with any of the three cytokine genotypes or allele types. | 1. The relatively small sample size; 2. cross-sectional study design | TNF-α −308G>A polymorphism may be considered an independent risk factor for SA in MDD. |
Vargas et al. (2013) [41] | 342 subjects divided into those with (N = 141) and without (N = 201) a history of SA | Cross-sectional case-control study | CRP Fibrinogen ESR IL-6 TNF-α | Subjects with SA had higher nitricoxide metabolites and lipid hydroperoxides levels and reduced plasma total antioxidant potential than those without. Regression analyses showed that unipolar/bipolar disorder, female gender, smoking behavior, and lipid hydroperoxides were linked to a history SA independently of classical risk factors. | 1. Participants were recruited from the center for smoking cessation treatment; 2. inflammatory, oxidative/nitrosative stress and metabolic biomarkers were assessed at baseline and correlated with a history of SA (as trait and not state-markers); 3. results may only delineate associations and not causality | Oxidative stress, nitrosative stress, lowered antioxidant levels may play a role in the pathophysiology of suicidal behavior independently from the effects of depression and smoking and classical suicide predictors (e.g., years of education and marital status). |
Dunjic-Kostic et al. (2013) [48] | 29 melancholic, 18 atypical MDD patients, and 39 healthy controls | Cross-sectional case-control study | TNF-α, IL-6 | IL-6 was significantly elevated in MDD-M. Lower TNF-α serum level was found both in melancholic patients and those with atypical depression. We detected a positive correlation between cytokine levels in atypical, but not in melancholic subjects. Clinical parameters (e.g., duration of illness, current episode, age of onset) were related to cytokine levels in atypical depression, while the duration of lifetime exposure to antidepressant treatment correlated to IL-6 serum levels in both patients with melancholic and atypical depression. | 1. The cross-sectional study design; 2. the lack of information regarding the association between inflammatory markers and antidepressant treatment over time | The study showed certain differences in pro-inflammatory cytokine serum levels in melancholic and atypical depressed patients than healthy subjects. Importantly, IL-6 elevation might represent a state indicator for acute exacerbation, especially in melancholic patients. |
Grassi-Oliveira et al. (2012) [38] | 30 female outpatients with recurrent MDD divided in two groups according with the presence/ absence of SI, and 16 healthy controls | Cross-sectional case-control study | MCP-1, CCL2, CCL5, CCL11 | MDD patients with suicidal ideation presented lower levels of MCP-1, CCL2 and CCL5 and higher levels of CCL11 compared to healthy controls. These differences remained significant after adjusting for depression severity. | 1. Small sample size; 2. participants included in MDD groups were taking antidepressants; 3. nicotine users from the sample were not excluded; 4. only female participants were included. | Findings indicated that the presence of recurrent MDD with suicidal ideation is associated with differences in inflammatory chemokines when compared to those without suicidal ideation. |
Isung et al. (2012b) [43] | 43 medication-free suicide attempters and 20 healthy male controls | Cross-sectional study | CSF VEGF, CSF IL-8, and IL-6 levels measured with an ultra-sensitive immunoassay system | Suicide attempters showed lower CSF VEGF and IL-8 levels than healthy controls. Also, a significant negative correlation was observed between CSF VEGF and severity of depression. A more severe depressive state was correlated with low CSF levels of VEGF reflecting a lack of trophic support to neurons and down-regulation of hippocampal neurogenesis. | The study was cross-sectional in nature | IL-8 may be crucial in neuroprotection. A role for an impaired innate immunity and dysregulation of neuroprotection has been suggested in both depression and suicidal behavior. |
Sublette et al. (2011) [29] | Fourteen subjects with MDD and a history of SA compared with 16 MDD patients without and 31 healthy controls | Cross-sectional case-control study | KYN, TRP, and the cytokine activation marker neopterin were investigated using high performance liquid chromatography | A priori planned contrasts showed that KYN was higher in the MDD SA subgroup compared with MDD non-attempters. KYN but not TRP was associated with attempt status, and only suicide attempters showed a positive correlation of the cytokine activation marker neopterin with the KYN:TRP ratio, suggesting that KYN production may be influenced by inflammatory processes among suicide attempters. | 1. This study did not measure inflammatory indices apart from KYN, TRP, and neopterin levels; 2. the small sample size | These preliminary results suggest that KYN and related molecular pathways may be implicated in the pathophysiology of suicidal behavior. Our findings raise the possibility that pharmacologic manipulation of KYN levels might reduce suicide risk. |
Janelidze et al. (2011) [9] | 47 SA and 17 non-suicidal depressed patients 16 healthy controls. | Cross-sectional study | IL-2 IL-6 TNF-α | Increased levels of IL-6 and TNF-α as well as decreased IL-2 concentrations in SA were found compared to non-suicidal depressed patients and healthy controls. | 1. Retrospective data, such as the duration of treatment and disease, were not registered; 2. data on smoking habits were not available; 3. the SA method, which was intoxication in 91% of patients in our sample; 4. storage time between sample collection and cytokine analysis was different for suicide attempters and depressed/healthy control subjects. | These results demonstrate for the first time that suicidal patients display a distinct peripheral blood cytokine profile compared to non-suicidal depressed patients. |
Boehm et al. (2010) [45] | 40 forensic autopsy cases of burn victims were examined 1 h after fire exposure and compared with 48 autopsy cases divided in post-mortem burns, deaths (e.g., hemorrhagic shock, railway suicide deaths) | Post-mortem study | TNF-α, IL-8, and ICAM-1 measured using immunohistochemical studies of lung tissue probes | Significantly higher extent of intra-alveolar edema was observed in the lungs of burn victims compared to other groups. A distinct expression of TNF-α, but not IL-8 or ICAM-1 was found in macrophages of all groups. A significantly stronger positivity of TNF-α in the group of burn victims was reported in intravascular erythrocytes when compared with other control groups. | 1. The survival times (>1 h) of the cohorts may be too short to reach the phase of leukocyte immigration for further morphological changes (no reactive inflammatory cell infiltrates were found); 2. the small sample size limits the generalization of the present findings. | A non-specific immune response to fire-induced inhalation trauma was demonstrated by the positive reaction of TNF-α in erythrocytes of burn victims. |
Lindqvist et al. (2009) [8] | 63 SA divided in violent or non violent SA and 47 healthy controls | Cross-sectional case-control design | CSF and plasma IL-1β, IL-6, IL-8, TNF-α were measured. The relation between cytokines and monoamine metabolites, 5-HIAA, HVA, and MHPG in CSF was also evaluated | Suicide attempters showed significantly higher CSF IL-6 levels compared to healthy controls. Specifically, violent SA showed the highest IL-6. A significant positive correlation between MADRS scores and CSF IL-6 levels was found in all patients. CSF 5-HIAA and HVA were found to correlate with IL-6 and TNF-α but not with MHPG. | 1. In order to test whether IL-6 is directly related to depressive and suicidal symptoms, cytokines should be administered peripherally or into the CNS; 2. the interaction between cytokines, monoamines, and HPA axis was not assessed. | CSF IL-6 plays a crucial role in suicidal behavior presumably through alterations of dopamine and serotonin metabolism. |
Gabbay et al. (2009) [30] | 12 suicidal adolescents, 18 non-suicidal adolescents with MDD, 15 controls | Cross-sectional case-control study | IFN-γ, TNF-α, IL-6, IL-1β, IL-4 | Suicidal adolescents with MDD had significantly decreased plasma TNF-α concentrations compared to non-suicidal adolescents with MDD. IFN-γ was increased in both suicidal and non-suicidal adolescents with MDD compared to controls. | 1. The cohort size was relatively modest; 2. a substantial proportion of patients was receiving psychotropic medications, which have been reported to induce negative immunoregulatory effects in adults with MDD; 3. due to the small sample, a multiple comparison correction to preserve statistical power was not carried out. | These preliminary findings suggest that immune system dysregulation may be associated with suicidal symptomatology in adolescent MDD. |
Tonelli et al. (2008) [46] | Post-mortem samples from the Brodman area 11 of 34 completed suicides and 17 controls | Post-mortem study | The expression of mRNA species for TNF-α, IL-1β, IL-4, IL-5, IL-6, and IL-13 measured using real-time- CRP | Female suicide victims showed increased expression of IL-4 whereas males suicide victims exhibited increased IL-13. Female suicide victims also showed higher but not significant TNF-α expression. | Controls were not matched for age, toxicology was incomplete in 82% of cases, and limited information were available on psychological diagnosis. In addition, accurate interactions between alcohol on inflammatory processes with the expression of cytokines were not reported. | Increased expression of mRNA transcripts of Th2 cytokines were found in the human orbitofrontal cortex of completed suicides. |
Kim et al. (2008) [31] | 36 MDD patients with recent SA, 33 non-suicidal MDD patients, and 40 normal controls | Cross-sectional case-control study | IL-6, IL-2, IFN-γ, IL-4, TGF-β1, Th1/Th2 ratio | Non-suicidal MDD patients had higher IL-6 levels than suicidal MDD patients and normal controls, while suicidal MDD patients had lower IL-2 than non-suicidal patients and normal controls. Both MDD groups, with or without attempted suicide, had lower IFN-γ and IL-4 and higher TGF-β1 levels. HDRS scores had positive correlations with IL-6, IFN-γ, Th1/Th2 ratio and negative correlations with IL-4 in non-suicidal depression patients. Suicidal MDD patients had no significant correlations between the LSARS or RRR scores and cytokine release. | 1. The effects of various confounding factors from these data cannot be excluded; 2. the existence of potential differences between those who agreed to participate and those that did not; 3. only in vitro mitogen-stimulated cytokine production (and not in vivo serum or plasma levels of cytokines) before the MDD treatment was measured. | The immune response has distinct differences between non-suicidal patients and suicidal patients. |
Lee and Kim (2006) [32] | 48 suicidal MDD patients, 47 non-suicidal MDD patients, 91 controls | Cross-sectional case-control study | In vitroTGF-β1 levels were investigated | In vitro TGF-β1 levels were significantly higher in suicidal MDD patients and non-suicidal MDD patients than controls. | 1. In vitro but not CSF TGF-β1 levels were measured; 2. methods of suicide attempt were not controlled; 3. the fact that blood for the quantification of the TGF was drawn after fasting for controls and 2 h later from the admission for suicidal depressive patients may represent a bias. | In vitro TGF-β1 levels may play a relevant role in MDD but not in suicidal behavior. |
Mendlovic et al. (1999) [36] | 9 patients with MDD that lasted 2–12 weeks and 9 age- and sex- matched controls | Cross-sectional case-control study | IFN-γ, IL-2, IL-4, IL-5, IL-10 secretion measured from PHA-stimulated lymphocytes | The stimulated lymphocytes of suicidal depressed patients secreted significantly more IFN-γ than those of healthy controls. Non-suicidal depressed patients secreted significantly less IFN-γ as compared with controls. Also, suicidal depressed patients secreted less IL- 4 and IL-5 as compared with non-suicidal depressed patients (although the difference was not statistically significant). | The small sample size limits the generalization of the present findings | Th1 activation in suicidal depression might reflect a unique form of autoimmune suicide. |
Author (s), Year | Sample Characteristics | Study Design | Inflammation Measurement | Main Findings | Limitations | Conclusions |
---|---|---|---|---|---|---|
Bay-Richter et al. (2014) [28] | 30 subjects (43% MDD) with SA; 36 healthy controls | 2-year longitudinal study | IL-6; CSF kynurenic acid; CSF quinolinic acid | Quinolinic acid was increased and kynurenic acid decreased over time in suicidal patients vs. healthy controls. A significant association between lower kynurenic acid and severe depressive symptoms as well as a link between higher IL-6 levels and more severe suicidal symptoms were reported. | 1. The study is correlational in nature and does not test direct causality in the patients; 2. Most patients were on psychoactive medications; 3. gender and age of both controls and patients were not matched; 4. repeated lumbar punctures are carried out only on suicide attempters and not healthy controls | A long-term dysregulation of the kyn pathway in the CNS of SA was demonstrated. An increased load of inflammatory cytokines was coupled to more severe symptoms. |
Li et al. (2013) [49] | 64 first-episode drug-naïve MDD patients and 64 matched healthy controls | 8-week case-control study | TNF-α | Plasma TNF-α levels were significantly decreased after venlafaxine treatment. Compared to non-responders, responders had a greater reduction in TNF-α levels which was linked to the greater reduction rate of HDRS-17. The plasma TNF-α levels were equally higher in both suicidal and non-suicidal MDD patients relative to healthy controls on admission. | 1. The naturalistic observation design may not completely control confounding factors such as selective bias; 2. the levels of serotonin and norepinephrine were not detected; 3. the dexamethasone/CRH test was not performed; 4. the long-term effects of venlafaxine on plasma TNF-α levels have been not evaluated. | MDD but not per se suicide was associated with the increased plasma TNF-α levels that may be inhibited using venlafaxine |
O’Donovan et al. (2013) [37] | 76 MDD and 48 healthy controls | Follow-up study | TNF-α; IL-6; IL-10; CRP | Patients with MDD and higher SI had higher inflammatory index scores than both controls and MDD patients with lower SI. Patients with lower SI were not different from controls on the inflammatory index as well. Follow-up analyses indicated that differences between MDD patients with high vs. lower SI were independent of depression severity and recent SA. | 1. The cross-sectional study design; 2. The small sample size; 3. The assumption of antidepressants and other medications; 4. the use of a structured interview measure of depression severity (HDRS) and not self-report measure; 5. The inclusion of a small number of patients with SA in the previous month. | SI may be uniquely linked to inflammation in depressed patients. |
Janelidze et al. (2013) [40] | 41 patients with SA, 22 non-suicidal psychiatric patients, 43 healthy controls | 12 year follow-up study | CCL11; IP-10; CXCL10; MIP-1β, CCL4; MCP-1, CCL2; MCP-4,CCL13; TARC,CCL17 | CSF eotaxin-1, MIP-1β (CCL4), MCP-1 (CCL2), MCP-4 (CCL13), and TARC (CCL17) were significantly lower in SA. Lower chemokine levels were specifically linked to psychotic symptoms and pain. | 1. The mismatch in gender distribution in the CSF Index study; 2. differences in the recruitment time between suicide attempter and healthy control groups in the CSF Index study; 3. multiple analyses were performed on the present data. | When compared to non-attempters, abnormally reduced chemokine levels in suicide attempters, both in the acute suicidal setting and after the long-term period, were reported. |
Isung et al. (2012a) [42] | 58 suicide attempters | 13 year longitudinal study | IL-1α; IL-1β; IL-2; IL-4; IL-6; IL-8; IL-10; IFN-γ; TNF-α; MCP-1; EGF; VEGF | Significantly lower VEGF levels were found in the seven patients after a mean follow-up of 13 years before completing suicide. VEGF also showed a trend for negative correlation with the planning subscale of SIS. A trend might be shown for lower IL-2 and higher IFN-γ levels in suicide victims. | 1. Degradation of several cytokines has been observed after 4 years of storage at –80 °C; 2. the data on smoking habits and exercise were not available. | Further support for a role of inflammation in the pathophysiology of suicidal behavior has been provided. VEGF may be significantly related with suicide risk. |
Nassberger and Trask- man- Bendz (1993) [44] | Medication-free suicide attempters with depressive disorders | Follow-up study | Urine and CSF soluble IL-2 receptor concentration presumably reflecting an activation of T-lymphocytes | A median soluble IL-2 receptor concentration far above the range of healthy controls was found. High levels of the soluble IL-2 receptor concentration were also reported at follow-up. There was a tendency of an association between soluble IL-2 receptor concentration and the ratio of norepinephrine-epinephrine in 24-h urine, as well as plasma and CSF 4-hydroxy-3- methoxymethyl-glycol. | 1. The small sample size limits the generalization of the present findings; 2. the study lacks of a placebo group. | Psychiatric patients investigated after a suicide attempt showed an imbalance of the immune system. |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Serafini, G.; Parisi, V.M.; Aguglia, A.; Amerio, A.; Sampogna, G.; Fiorillo, A.; Pompili, M.; Amore, M. A Specific Inflammatory Profile Underlying Suicide Risk? Systematic Review of the Main Literature Findings. Int. J. Environ. Res. Public Health 2020, 17, 2393. https://doi.org/10.3390/ijerph17072393
Serafini G, Parisi VM, Aguglia A, Amerio A, Sampogna G, Fiorillo A, Pompili M, Amore M. A Specific Inflammatory Profile Underlying Suicide Risk? Systematic Review of the Main Literature Findings. International Journal of Environmental Research and Public Health. 2020; 17(7):2393. https://doi.org/10.3390/ijerph17072393
Chicago/Turabian StyleSerafini, Gianluca, Valentina Maria Parisi, Andrea Aguglia, Andrea Amerio, Gaia Sampogna, Andrea Fiorillo, Maurizio Pompili, and Mario Amore. 2020. "A Specific Inflammatory Profile Underlying Suicide Risk? Systematic Review of the Main Literature Findings" International Journal of Environmental Research and Public Health 17, no. 7: 2393. https://doi.org/10.3390/ijerph17072393