Next Article in Journal
Variation in the Distribution of Four Cacti Species Due to Climate Change in Chihuahua, Mexico
Previous Article in Journal
Preventive Care Use among the Belgian Elderly Population: Does Socio-Economic Status Matter?
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
IJERPH
Volume 11
Issue 1
10.3390/ijerph110100373
ijerph-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Effectiveness of a Multi-Component Smoking Cessation Support Programme (McSCSP) for Patients with Severe Mental Disorders: Study Design

by
Maria Paz Garcia-Portilla
1,2,
Leticia Garcia-Alvarez
2,
Pilar Alejandra Saiz
1,2,*,
Eva Diaz-Mesa
2,
Gonzalo Galvan
3,4,
Fernando Sarramea
5,
Josefa Garcia-Blanco
5,
Edorta Elizagarate
6,7 and
Julio Bobes
1,2
1
Departmento de Psiquiatría, Universidad de Oviedo, 33006 Oviedo, Spain
2
Centro de Investigación Biomédica en Red de Salud Mental, CIBERSAM, Universidad de Oviedo, 33006 Oviedo, Spain
3
Universidad Nacional de la Patagonia Austral, Puerto San Julián, 9310 Santa Cruz, Argentina
4
Instituto Interuniversitario de Postgrado en Salud, Santa Rosa, 6300 La Pampa, Argentina
5
Complejo Hospitalario de Jaén, Equipo de Salud Mental de Andújar, 23740 Jaén, Spain
6
Unidad de Psicosis Refractarias, Hospital Psiquiátrico de Álava, 01006 Vitoria, Spain
7
Centro de Investigación Biomédica en Red de Salud Mental, CIBERSAM, Universidad del País Vasco, 48940 Leioa-Bizkaia, Spain
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2014, 11(1), 373-389; https://doi.org/10.3390/ijerph110100373
Submission received: 1 November 2013 / Revised: 13 December 2013 / Accepted: 14 December 2013 / Published: 24 December 2013
Browse Figure
Versions Notes

Abstract

:
Only a few studies have examined the efficacy and safety of smoking cessation programmes in patients with mental disorders. The aim of this paper is to describe in detail the methodology used in the study as well as the Multi-component Smoking Cessation Support Programme in terms of pharmacological treatments and psychological interventions. An open-label 9-month follow-up study was conducted in Spain. A total of 82 clinically stable outpatients with schizophrenia, schizoaffective or bipolar disorder were enrolled. Treatment consisted of a programme specifically developed by the research team for individuals with severe mental disorders. The programme consisted of two phases: (1) weekly individual motivational therapy for 4–12 weeks, and (2) a 12-week active treatment phase. During this phase, at each study visit patients received a one- or two-week supply of medication (transdermal nicotine patches, varenicline or bupropion) with instructions on how to take it, in addition to group psychotherapy for smoking cessation. Evaluations were performed: (1) at the time of enrolment in the study, (2) during the 12-week active treatment phase of the study (weekly for the first 4 weeks and then biweekly), and (3) after the end of this phase (two follow-up assessments at weeks 12 and 24). Evaluations included: (1) smoking history, (2) substance use, (3) psychopathology, (4) adverse events, and (5) laboratory tests. The importance of this study lies in addressing a topical issue often ignored by psychiatrists: the unacceptably high rates of tobacco use in patients with severe mental disorders.

1. Introduction

Although smoking rates are declining in the general population in developed countries [1], patients with long-standing mental health disorders are almost twice as likely to smoke as people without such problems [2]. In fact smoking prevalence rates among people with mental disorders (schizophrenia, mood and anxiety disorders) are two to four times higher than in the general population [3,4]. Prevalence is greatest in patients with schizophrenia with rates around 64% followed by patients with bipolar disorder, with rates around 44% [5]. In Spain the reported tobacco use rate in patients with schizophrenia was 54.4% [6] and 51.5% in patients with bipolar disorder [7,8]. These rates are practically twice that of the general Spanish population (26.4%) [9].
This exceptionally high prevalence, explained by several biological and psychosocial factors, contributes to the high rates of medical morbidity and mortality in patients with severe mental disorders [6,7,10,11]. Life expectancy was found to be 25 years lower in patients with mental illnesses, mainly due to tobacco-related diseases [12]. In this respect the Spanish Consensus on Physical Health of Patients with Schizophrenia [13] and with Bipolar Disorder [14] rank respiratory disorders (standardized mortality ratio of 3–7 in patients with bipolar disorder) followed by cardiovascular disease as the leading causes of the increased mortality rates in these patients. With respect to morbidity, tobacco use seems to be one of the most important risk factors for the greater risk of coronary disease both in patients with schizophrenia [6,15,16,17] and bipolar disorder [7].
In addition to its negative impact on physical health, tobacco use has been associated with more excitement and agitation symptoms [18,19], greater severity of global psychopathology as measured by the Clinical General Impression (CGI) scale [6], and positive psychotic symptoms [6,20] in patients with schizophrenia, although the effect size was small [6]. In patients with bipolar disorders the results are controversial; while some studies showed tobacco use to be associated with greater disorder severity [21,22] others did not [7]. Recently a relationship was also reported between smoking and use of a greater number of psychopharmaceuticals for the treatment of bipolar disorder [7]. Furthermore, an association was found between tobacco use and greater attempted suicide rates in patients with bipolar disorders [23,24] and with schizophrenia [25]. Some studies have also reported beneficial effects of tobacco on patients’ mental health. Thus improvements in negative symptoms [20,26], spatial working memory and attention deficit [27], and sensory gating [28] have been reported. However we believe that these beneficial effects do not justify maintaining the smoking habit in this population, as tobacco is associated with more than 4,000 toxins and 60 carcinogens, and nicotine can be delivered more safely through approved drugs.
Studies on the smoking patterns of individuals with severe mental disorders have shown that they started smoking an average of 5 years prior to illness onset [29,30,31], smoked a large number of cigarettes per day [29], had higher plasma nicotine levels than smokers without mental disorders [32,33] and were more nicotine-dependent than general population [16,34,35,36].
While smokers with severe mental disorders have quite low smoking cessation rates, they have demonstrated high levels of motivation to quit [37,38], persistent attempts [39] and good tolerance of short-term nicotine abstinence without significant clinical exacerbation [40,41,42,43]. Despite all the above, only a few studies have examined the efficacy and safety of smoking cessation programmes in patients with mental disorders. For this reason we decided to develop a Multi-component Smoking Cessation Support Programme (McSCSP) tailored to the specific characteristics of individuals with severe mental disorders, as recommended by the European Association of Psychiatry (EPA) in its Position Statement on Smoking and Strategies for Smoking Cessation in People with Mental Illness [44], and to determine its clinical effectiveness. The McSCSP consisted of two phases: (1) weekly individual motivational therapy for 4–12 weeks, and (2) a 12-week active treatment phase. During this phase, at each study visit patients received a one- or two-week supply of medication with instructions on how to take it, in addition to group psychotherapy for smoking cessation.
The aim of this paper is to describe in detail the methodology used in the study, i.e., inclusion and exclusion criteria, clinical evaluations including smoking history, substance use, psychopathology, adverse events, anthropometrics, vital signs, and laboratory tests, as well as the McSCSP in terms of pharmacological treatments and psychological interventions.

2. Methods

2.1. Study Design

This is a non-randomized, open-label (both the researchers and participants know which treatment is being administered), 9-month follow-up study conducted at three sites in Spain (Oviedo, Vitoria, and Jaén) between March 2011 and June 2013. The study was conducted at two Mental Health Centres (Oviedo and Jaén) and at the Refractory Psychosis Unit where outpatients from the city of Vitoria were referred for the study.
The Clinical Research Ethics Committee of Hospital Universitario Central de Asturias in Oviedo, Spain approved the study protocol. Written informed consent was obtained from all subjects prior to enrolment.

2.2. Subjects

The sample size needed for this study was determined using the EpiInfo v.6.0 program. Considering a population of 150 patients with severe mental disorder for each of the three psychiatrists participating in the study, a 6-month successful smoking cessation rate of 12% of the patients at six months, and a confidence interval level of 95%, a sample size of n = 23 per centre is needed to achieve the stated goals.
A total of 82 outpatients with schizophrenia, schizoaffective or bipolar disorder on maintenance treatment were enrolled in the study (74.4% schizophrenia or schizoaffective disorder and 25.6 bipolar disorder). Four of them dropped-out before initiating the active treatment phase. Of the 78 patients who were included in the active phase of treatment, 50% received varenicline, 46.2% 24-hour nicotine transdermal patches, and 3.8% bupropion SR. A total of 62.8% of the patients completed the 12-week active treatment phase. There was no statistically significant differences in the retention rate among the three drugs (see Figure 1).
Ijerph 11 00373 g001 1024
Figure 1. Patients’ disposition.
Figure 1. Patients’ disposition.
Ijerph 11 00373 g001
Inclusion criteria were: (1) patients with a confirmed DSM-IV diagnosis of schizophrenia, schizoaffective or bipolar disorder, clinically stable (i.e., without hospitalization or acute exacerbation) in the six months prior to enrolment in the study and on maintenance treatment; (2) currently smoking 15 or more cigarettes/day without a period of smoking abstinence longer than one month during the previous year; (3) Fagerström Test for Nicotine Dependence score ≥4 (moderate); (4) breath carbon monoxide (CO) level >9 particles per million (ppm); (5) between 18 and 65 years of age; (6) without suicidal ideation; and (7) written informed consent to participate in the study.
Patients were excluded if they had (1) a total score >70 on the Positive and Negative Symptoms Scale (PANSS) for patients with schizophrenia, or >14 on the Hamilton Depression Rating Scale (HDRS) or >6 on the Young Mania Rating Scale (YMRS) for patients with bipolar disorder; (2) serious suicidal behaviour or thoughts in the last six months; (3) severe and unstable somatic illness including but not limited to acute myocardial infarction, unstable arrhythmia, stroke; (4) history of organic brain damage including epilepsy, tumours, head injuries with significant cognitive impairment; (5) significant renal impairment (creatinine ≥ 1.5 mg/dL); and (6) liver function tests more than twice the upper limit of normal values.

2.3. Diagnostic Interview

Diagnosis was confirmed by the psychiatrists participating in the study using the Schizophrenia, Schizoaffective and Bipolar sections of the Spanish version of the Structured Clinical Interview for DSM-IV Axis I Disorders (SCID-I) [45].

2.4. Demographic and Clinical Data

Demographic and clinical data were collected at baseline. Demographic data included: birth date, age, gender, civil status, educational level, main occupation, and current working status. Clinical data included: primary diagnosis, secondary diagnosis, length of illness, first episode, previous suicide attempts, and current psychopharmacological treatment. Changes in psychopharmacological treatment were recorded at every visit.

2.5. Clinical Evaluations

Psychometric and biological evaluations included smoking history, substance use, psychopathology, adverse events, anthropometrics, vital signs and laboratory tests (see Table 1). For all instruments used in the study we employed the validated Spanish versions with the exception of the Columbia Suicide Severity Rating Scale (CSSRS), which is currently being validated (Project PI: Pilar A. Saiz), and the Richmond test which, although it has not been validated in Spanish, is widely used in our country [46].
Table
Table 1. Psychometric and biological evaluations.
Table 1. Psychometric and biological evaluations.
Area of AssessmentPsychometric Instruments/ Biological ParametersInclusion CriteriaExclusion Criteria
Smoking historyPattern of tobacco use
  • Age at first use
  • Number of years smoked
  • Smoking pack-year (SPY)
  • Cigarette brand
  • Cigarettes per day (CPD)
  • Breath carbon monoxide (CO) level




  • ≥15 cigarettes/day
  • >9 ppm
Nicotine dependence
  • Fagerström Test for Nicotine Dependence (FTND)
  • Glover-Nilsson Smoking Behavioral Questionnaire
  • Total score ≥ 4
Motivation to quit
  • Richmond test
  • University of Rhode Island Change Assessment (URICA) scale
Substance useCaffeine
  • Daily consumption
Other
  • Drug Use Table—Individual Substances of the Addiction Severity Index 6th version (ASI6)
PsychopathologySchizophrenia or schizoaffective patients
  • Positive and Negative Syndrome Scale (PANSS)
  • Total score > 70
Patients with bipolar disorder
  • Hamilton Depression Rating Scale (HDRS)
  • Young Mania Rating Scale (YMRS)
  • Total score > 14
  • Total score > 6
Both types of patients
  • SCID-I: Schizophrenia, Schizoaffective and Bipolar sections
  • Columbia Suicide Severity Rating Scale (CSSRS)
  • Clinical Global Impression (CGI)
  • Meeting criteria for one disorder
  • Serious suicidal behaviours or thoughts
Adverse events
  • UKU Side Effects Rating Scale (UKU)
Biological evaluationAnthropometrics
  • Weight, height, body mass index (BMI), waist circumference
Vital signs
  • Blood pressure, heart rate
Laboratory tests
  • Renal function tests: creatinine, BUN, glomerular filtration rate
  • Liver function tests: aspartate aminotransferase (AST), alanine aminotransferase (ALT), gamma glutamyl transpeptidase (GGT), total bilirubin (TBIL)
  • Lipid profile: triglycerides, total cholesterol, HDL cholesterol, LDL cholesterol
  • Creatinine ≥ 1.5

  • Liver function tests > 2 × the upper limit of normal

2.5.1. Smoking History

Pattern of tobacco use was described with the following parameters: age at first use, number of years smoked, pack-years, cigarette brand, cigarettes per day (CPD), and breath carbon monoxide (CO) level.
Smoking packyears (SPY): Tobacco exposure has a cumulative effect so it is necessary to consider not only current consumption but also global lifetime consumption. SPY is a numerical value that represents the amount an individual has smoked over his/her life. The following formula is used to calculate it: (number of cigarettes per dayA × years of consumption of amountA/20) + (number of cigarettes per dayB × years of consumption of amountB/20) + ... + (number of cigarettes per dayn × years of consumption of amountn/20). A calculator can be downloaded from http://smokingpackyears.com/calculate [47].
Cigarettes per day (CPD): The number of CPD can be considered a face valid measure of nicotine dependence with low predictive abstinence value [48]. As there is no consensus on how to best define “light” and “heavy” smokers, we classified subjects into three categories according to the following classification: self-reported CPD <10 = light smoker; 11–20 = moderate smoker; and >20 = heavy smoker.
Breath CO levels: Breath CO was measured with a portable piCOsimpleTM Smokerlyzer® monitor (Bedfont Scientific Ltd., Kent, UK). In keeping with the majority of studies we used a cutoff point for current smokers of 9 ppm. Since smokers have diurnal variation in CO [49,50] it was measured between 9 a.m. and 11 a.m.
Nicotine dependence: This was evaluated using the Fagerström Test for Nicotine Dependence (FTND) [50] and the Glover-Nilsson Smoking Behavioral Questionnaire [51]. The FTDN comprises six items that evaluate the degree of nicotine dependence. The FTDN total score ranges from 0 to 10 and categorizes patients as having mild (0–3), moderate (4–7), and severe (8–10) nicotine dependence. It was recently recommended by the European Psychiatric Association for use in daily clinical practice [44].
The Glover-Nilsson test consists of 11 items that capture behavioural nicotine dependence. It classifies behavioural dependence into four levels according to total score: 0–11 = mild; 12–22 = moderate; 23–33 = strong; 34–44 = very strong.
Motivation to quit: The Richmond test [52] and the University of Rhode Island Change Assessment (URICA) scale [53] were used to measure the individual’s motivation to quit smoking and his/her readiness to change, respectively. The Richmond test is a 4-item instrument that evaluates motivation to quit smoking. Total score ranges between 0 and 10. Scores between 0 and 4 indicate low motivation; 5–6 moderate motivation; and 7–10 high motivation to quit.
The URICA is a 32-item scale that contains four 8-item subscales measuring the stages of change: Precontemplation (PC), Contemplation (C), Action (A), and Maintenance (M). A continuous “Readiness to Change” score can be obtained using the following formula [(Avg C + Avg A + Avg M) − Avg PC], where Avg represents the average score of each of the four subscales in order to evaluate an individual’s level of motivation for change. Since there are no normative scores for the Spanish population we decided to use subscale scores instead of the readiness to change score to determine each patient’s stage of the change.

2.5.2. Substance Use

The use of the following substances was assessed with the Drug Use Table—Individual Substances of the Spanish Addiction Severity Index 6th version (ASI6) [54]: marijuana, sedatives, cocaine, stimulants, hallucinogenics, heroin, methadone, other opiates, and inhalants. For each substance we determined the age at first use, years of regular use, if used 50 or more days (lifetime), use in past 30 days, and if used as treatment (last 30 days). In addition we recorded daily caffeine consumption by asking about daily consumption of coffee, tea, cola and energy drinks (Burn, Red Bull, Fire, etc.). Caffeine consumption was recorded as number of cups of coffee per day based on the following equivalence: 1 cup of coffee = 1 energy drink = 2 cups of tea = 3 colas [55].

2.5.3. Psychopathology

In patients with schizophrenia we used the PANSS [56], which measures the severity of positive, negative, and general psychopathology symptoms. In patients with bipolar disorder we used the HDRS [57] and the YMRS [58], which rate the severity of depressive and manic symptoms, respectively. In both types of patients we utilized the CGI [59] for assessing severity and change in global psychopathology, and the CSSRS to evaluate suicidal thoughts and behaviours.

2.5.4. Adverse Events

Unwanted side effects of pharmacological treatment were assessed by means of the UKU Side Effects Rating Scale (UKU) [60]. In addition, we will also get information about them through the psychometric ratings, anthropometric measures, laboratory results and spontaneous patient’ self-reports.

2.5.5. Anthropometrics, Vital Signs, and Laboratory Tests

Weight and height were measured without jackets and shoes. Body mass index (BMI) was calculated as body weight (kg) divided by the square of height (m2). Waist circumference was measured at the high point of the iliac crest and at the level of the umbilicus. Heart rate was measured as number of beats per minute using the wrist palpation method. Blood pressure consisted of a single seated determination. Blood samples were collected if patients confirmed having fasted for at least 8 h. Creatinine, BUN, glomerular filtration rate, aspartate aminotransferase (AST), alanine aminotransferase (ALT), gamma glutamyl transpeptidase (GGT), total bilirubin (TBIL), triglycerides, total cholesterol, and HDL and LDL cholesterol were tested.

2.6. Assessments

All subjects were evaluated at the time of enrolment in the study before starting the motivational therapy. During the 12-week active treatment phase patients were assessed weekly for the first 4 weeks and then biweekly. After the end of this phase, two follow-up assessments were done at weeks 12 and 24 (study weeks 24 and 36 respectively). Table 2 displays the timing and the parameters evaluated.
Table
Table 2. Task schedule.
Table 2. Task schedule.
Study PhaseMotivationActive TreatmentFollow-up
Visit numberMV1V0V1V2V3V4V5V6V7V8FU- V1FU- V2
Weekw-12 to w-4w0w1w2w3w4w6w8w10w12w24w36
Inclusion/Exclusion criteriaX
Informed consentX
Demographic dataX
Clinical dataX
SCID-I: Schizophrenia, Schizoaffective and Bipolar sectionsX
Smoking history
Pattern of tobacco useX
Cigarettes per day (CPD)XXXXXXXXXXX
Breath CO levelXXXXXXXXXXXX
Fagerström Test for Nicotine Dependence (FTND)XXXXXXXXXXXX
XXXXXXXXXXXX
Glover-Nilsson Smoking Behavioral QuestionnaireXX
XX
Richmond test
University of Rhode Island Change Assessment (URICA) scale
Substance use
Daily caffeine consumptionXXXXXXXXXXXX
Drug Use, Table from the Addiction Severity Index 6th version (ASI6)XXXXXXXXXXXX
Psychopathology
Positive and Negative Syndrome Scale (PANSS)XXXXXXXXXXXX
XXXXXXXXXXXX
Hamilton Depression Rating Scale (HDRS)XXXXXXXXXXXX
XXXXXXXXXXXX
Young Mania Rating Scale (YMRS)XXXXXXXXXXXX
Columbia Suicide Severity Rating Scale (CSSRS)XXXXXXXXXXXX
XXXXXXXXXX
Clinical Global Impression (CGI)
Severity
Change
Adverse events
UKU Side Effects Rating Scale (UKU)XXXXXXXXXXXX
Biological evaluation
Anthropometrics and vital signsXXXXXXXXXXXX
Laboratory testsX X XXX
Notes: FU: follow-up; M: motivation; V: visit; w: week; w-12 to w-4: between 12 and 4 weeks before starting Active Treatment (V0, w0).

2.7. Statistical Plan

We will determine short-term (at the end of the 12-week active treatment phase), and 3- and 6-months after active treatment outcomes. The primary outcome measure will be smoking cessation, a composite measure formed by the patient's self-report of previous 7-day abstinence confirmed by breath CO levels ≤ 9 ppm. It will be also considered as main outcome measure the proportion of subjects with at least 50% reduction in the number of cigarettes per day. Secondary outcome measures will be safety, including changes in the symptoms of the primary illness, and tolerability.

2.8. Treatment

2.8.1. Pharmacological Treatment

The pharmacological treatments used in the study were those approved for smoking cessation (bupropion sustained release, nicotine replacement therapy (transdermal patches) and varenicline) and considered first-line options by the US Public Health Service in 2008 [61]. Recently the European Psychiatric Association (EPA) [44] recommended drug treatment with nicotine, varenicline or bupropion for even a mild degree of tobacco dependence in every psychiatric patient. However, it should be pointed out that, in its Therapeutic Prescribing Guidelines, the Agencia Española de Medicamentos y Productos Sanitarios does not include varenicline among the drugs approved for smoking cessation and specifically contraindicates the use of bupropion in patients with bipolar disorder [62].
For each patient the drug was chosen based on the clinical characteristics of his/her mental disorder, smoking pattern and previous smoking cessation experiences, somatic comorbidities and their pharmacologic treatments, as well as patient preference. Varenicline was given according to the usual schedule, i.e., 0.5 mg/day for the first 3 days, 0.5 mg BID on days 4–7, and 1 mg BID for the remaining 11 weeks. Bupropion SR was given as recommended, i.e., 150 mg/day for the first 6 days and 150 mg BID for the remaining treatment period. Twenty-four-hour nicotine transdermal patches were given to patients at doses of 14, 21, 28 or 35 mg based on their tobacco use during the last 12 weeks.
In the case of psychopathological exacerbation or serious adverse events the pharmacological treatment was discontinued and the subject was withdrawn from the study.

2.8.2. Psychological Interventions

The study included two types of sequential psychological interventions. Firstly those patients willing to quit smoking and participate in the study received individual motivational therapy to prepare them for the active treatment phase. Secondly group psychotherapy was provided as part of the active treatment phase along with pharmacotherapy.
Motivational therapy: Those smokers who wanted to quit received individual motivational therapy before entering the active treatment phase. The programme was flexible, lasting between 4 and 12 sessions depending on each subject’s characteristics, and was based on the motivational interviewing technique [63]. Its main objective was to help patients move forward through the Stages of Change Model [64] to the stages of Contemplation or Action. The main issues addressed during the sessions were (1) pros and cons of smoking, (2) health and financial burden of smoking, and (3) concerns about quitting.
We included the financial burden of smoking because we believe that in patients with severe mental disorder this is a very important point as shown by Lawn [65]. Many of them are financially dependent on a mental disability benefit and spend a large part of their monthly income on cigarettes.
Group therapy: A specific intensive 12-week manualized group therapy programme for individuals with severe mental was developed by the research team based on available smoking cessation guidelines and our own experience. The programme is suitable for delivery by healthcare providers experienced in working with patients with severe mental disorders and in conducting psychoeducational groups, including psychiatric nurses, psychologists and psychiatrists. Session content covered decisional balance, nicotine withdrawal and psychiatric symptoms, weight gain, control techniques, advantages and disadvantages of the pharmacological treatment options, improving adherence, and preventing relapses. In addition to group therapy, patients had to do homework that was reviewed at the beginning of the next session. Each treatment session lasted 60–75 min and included six to eight patients who achieved the Contemplation or Action stages of the Prochaska and DiClemente Model [64].

3. Discussion

This study tries to determine the effectiveness of a McSCSP specifically developed for use in individuals with severe mental disorders. The importance of this study lies in addressing a topical issue often ignored by psychiatrists: the unacceptably high rates of tobacco use in patients with severe mental disorders. The EPA—Position Statement on Smoking and Strategies for Smoking Cessation in People with Mental Illness concludes that “since tobacco dependence is a dependence disorder, psychiatrists are the experts in performing interventions in this area. It is their duty to do so in view of the major impact of tobacco dependence on, for example, the metabolism of psychotropic treatments, morbidity (such as lung cancer) and mortality” [44]. Furthermore, the relative absence of published results in this field adds more value to our study. In this study we have shown that such programmes are feasible and can be implemented in centers treating patients with severe mental illness since 66% of them completed the active treatment phase.
An advantageous feature of the study is its external validity and therefore the generalizability of our results. The inclusion and exclusion criteria used allowed us to enrol “real world” patients seen in daily clinical practice. Only those patients with an acute exacerbation of their mental illness or in whom the three study drugs were contraindicated were excluded from the study. Although we utilized the Spanish version of the URICA test [53] to identify each patient's stage of change according to the Prochaska and DiClemente Model [64], this information was not considered an inclusion or exclusion criterion due to the lack of Spanish normative scores or cutoff scores, data in people with severe mental disorders, and published doubts about its usefulness [66,67]. We hope to identify whether any of the stages of change is associated with higher rates of smoking cessation.
Another strength of this study is the sample size. Although 82 patients is not a very large sample, to our knowledge the majority of the published studies in patients with severe mental disorders included far fewer patients or were case reports. However, the lack of a control group may be considered an important shortcoming. The open-label design of the study could be also considered a limitation of the study. However, this study was intended to be a pragmatic trial of the effectiveness of a multi-component programme specifically designed for individuals with severe mental disorders rather than a clinical trial designed to determine the comparative efficacy of the approved drugs for smoking cessation.
One strong point of our study in comparison with other smoking cessation studies is the comprehensive clinical evaluation that was done. In addition to taking an exhaustive smoking history we assessed the use of other substances, the psychopathology severity including suicidal thoughts and behaviours, and adverse events using validated psychometric instruments. Standard laboratory tests were also performed. Conversely, a weak point is the lack of collection on somatic comorbidities. However, since we have anthropometric data and laboratory results we can obtain information on obesity, metabolic syndrome and cardiovascular risk, the most frequent somatic comorbidities in these patients and highly associated with their premature mortality.
Finally, the McSCSP was developed with the aim of providing mental healthcare professionals with an intervention that could be applied in specialized routine clinical practice. It may be said that 60 to 75 min is too long for implementation in standard mental health centres. However since it is designed for use in severely ill patients in specialized medical facilities (psychiatric units), and given the potential benefits to both the healthcare system and patients, as well as the group modality of the intervention (60–75 min for 6 to 8 patients), this should not constitute a barrier, or worse, an excuse not to address the problem. This will be of interest if we are able to demonstrate that our smoking cessation programme is safe and effective in patients with severe mental disorders and should thus be disseminated for use in everyday clinical practice in Spain.

4. Conclusions

The prevalence rate of tobacco dependence among patients with severe mental disorders is about twice that of people without these disorders. In addition, tobacco is one of the most important avoidable factors contributing to the high rates of morbidity and mortality seen in these patients. Therefore, psychiatrists, clinical psychologists and psychiatric nurses, as experts in addiction treatment, should motivate and help their patients to quit smoking

Acknowledgments

This study was conducted without financial support from any pharmaceutical company. It was partly supported by the Spanish Ministry of Science and Innovation, Instituto de Salud Carlos III (FIS PI10/01758), and CIBERSAM. We thank Sharon Grevet for her English assistance. She was sponsored by research grants.

Conflicts of Interest

Maria Paz Garcia-Portilla, Pilar Alejandra Saiz, and Eva Diaz-Mesa were investigators of the Clinical Study sponsorized by Pfizer: “Estudio de fase 4, doble ciego, controlado con placebo, multicéntrico y de 12 semanas de duración para evaluar la seguridad y la eficacia del tartrato de Vareniclina (CP-526,555) 1 mg dos veces al día para dejar de fumar en pacientes con depression” (A phase 4 12-week, double-blind, placebo-controlled, multicenter study evaluating the safety and efficacy of varenicline tartrate (CP-526,555) 1 mg BID for smoking cessation in subjects with depression) (Protocol A3051122). Julio Bobes was the principal investigator of the study center. The rest of the authors have no declaration of conflicts of interest to make.

References

  1. Shafey, O.; Eriksen, M.; Ross, H.; Mackay, J. The Tobacco Atlas, 3rd ed.; American Cancer Society: Atlanta, GA, USA, 2010. [Google Scholar]
  2. Lancet, E. Smoke alarm: Mental illness and tobacco. Lancet 2013, 381, 1071. [Google Scholar] [CrossRef]
  3. Lising-Enriquez, K.; George, T.P. Treatment of comorbid tobacco use in people with serious mental illness. J. Psychiatr. Neurosci. 2009, 34, 1–2. [Google Scholar]
  4. Tidey, J.W.; Rohsenow, D.J.; Kaplan, G.B.; Swift, R.M.; Ahnallen, C.G. Separate and combined effects of very low nicotine cigarettes and nicotine replacement in smokers with schizophrenia and controls. Nicotine Tob. Res. 2013, 15, 121–129. [Google Scholar] [CrossRef]
  5. Dickerson, F.; Stallings, C.R.; Origoni, A.E.; Vaughan, C.; Khushalani, S.; Schroeder, J.; Yolken, R.H. Cigarette smoking among persons with schizophrenia or bipolar disorder in routine clinical settings, 1999–2011. Psychiatr. Services 2013, 64, 44–50. [Google Scholar] [CrossRef]
  6. Bobes, J.; Arango, C.; Garcia-Garcia, M.; Rejas, J. Healthy lifestyle habits and 10-year cardiovascular risk in schizophrenia spectrum disorders: An analysis of the impact of smoking tobacco in the clamors schizophrenia cohort. Schizophr. Res. 2010, 119, 101–109. [Google Scholar] [CrossRef]
  7. Garcia-Portilla, M.P.; Saiz, P.A.; Benabarre, A.; Florez, G.; Bascaran, M.T.; Diaz, E.M.; Bousono, M.; Bobes, J. Impact of substance use on the physical health of patients with bipolar disorder. Acta Psychiatr. Scand. 2010, 121, 437–445. [Google Scholar]
  8. Gonzalez-Pinto, A.; Gutierrez, M.; Ezcurra, J.; Aizpuru, F.; Mosquera, F.; Lopez, P.; de Leon, J. Tobacco smoking and bipolar disorder. J. Clin. Psychiat. 1998, 59, 225–228. [Google Scholar] [CrossRef]
  9. Encuesta Nacional de Salud (National Health Survey); Ministerio de Sanidad: Madrid, Spain, 2006.
  10. Ziedonis, D.; Hitsman, B.; Beckham, J.C.; Zvolensky, M.; Adler, L.E.; Audrain-McGovern, J.; Breslau, N.; Brown, R.A.; George, T.P.; Williams, J.; et al. Tobacco use and cessation in psychiatric disorders: National institute of mental health report. Nicotine Tob. Res. 2008, 10, 1691–1715. [Google Scholar] [CrossRef]
  11. Ripoll, R.M. Lifestyle Medicine: The importance of considering all the causes of disease. Rev. Psiquiat. Salud Ment. 2012, 5, 48–52. [Google Scholar] [CrossRef]
  12. Colton, C.W.; Manderscheid, R.W. Congruencies in increased mortality rates, years of potential life lost, and causes of death among public mental health clients in eight states. Prev. Chron. Dis. 2006, 3. Available online: http://www.cdc.gov/pcd/issues/2006/apr/05_0180.htm (accessed on 23 December 2013).
  13. Ruiz, J.S.; Garcia, J.B.; Ruiloba, J.V.; Ubago, J.G.; Gonzalez, M.P.J.-P. Consensus on physical health of patients with schizophrenia from the spanish societies of psychiatry and biological psychiatry. Acta. Esp. Psiquiat. 2008, 36, 251–264. [Google Scholar]
  14. Bobes, J.; Ruiz, J.S.; Montes, J.M.; Mostaza, J.; Rico-Villademoros, F.; Vieta, E. Spanish consensus on physical health of patients with bipolar disorder. Rev. Psiquiatr. Salud Ment. 2008, 1, 26–37. [Google Scholar] [CrossRef]
  15. Chapman, S.; Ragg, M.; McGeechan, K. Citation bias in reported smoking prevalence in people with schizophrenia. Aust. N. Z. J. Psychiat. 2009, 43, 277–282. [Google Scholar] [CrossRef]
  16. De Leon, J.; Diaz, F.J. A meta-analysis of worldwide studies demonstrates an association between schizophrenia and tobacco smoking behaviors. Schizophr. Res. 2005, 76, 135–157. [Google Scholar] [CrossRef]
  17. Goff, D.C.; Cather, C.; Evins, A.E.; Henderson, D.C.; Freudenreich, O.; Copeland, P.M.; Bierer, M.; Duckworth, K.; Sacks, F.M. Medical morbidity and mortality in schizophrenia: Guidelines for psychiatrists. J. Clin. Psychiat. 2005, 66, 183–194. [Google Scholar] [CrossRef]
  18. Aguilar, M.C.; Gurpegui, M.; Diaz, F.J.; de Leon, J. Nicotine dependence and symptoms in schizophrenia: Naturalistic study of complex interactions. Br. J. Psychiat. 2005, 186, 215–221. [Google Scholar] [CrossRef]
  19. de Leon, J.; Diaz, F.J.; Aguilar, M.C.; Jurado, D.; Gurpegui, M. Does smoking reduce akathisia? Testing a narrow version of the self-medication hypothesis. Schizophr. Res. 2006, 86, 256–268. [Google Scholar] [CrossRef]
  20. Ziedonis, D.M.; Kosten, T.R.; Glazer, W.M.; Frances, R.J. Nicotine dependence and schizophrenia. Hosp. Community Psychiat. 1994, 45, 204–206. [Google Scholar]
  21. Ostacher, M.J.; Nierenberg, A.A.; Perlis, R.H.; Eidelman, P.; Borrelli, D.J.; Tran, T.B.; Ericson, G.M.; Weiss, R.D.; Sachs, G.S. The relationship between smoking and suicidal behavior, comorbidity, and course of illness in bipolar disorder. J. Clin. Psychiat. 2006, 67, 1907–1911. [Google Scholar] [CrossRef]
  22. Vanable, P.A.; Carey, M.P.; Carey, K.B.; Maisto, S.A. Smoking among psychiatric outpatients: Relationship to substance use, diagnosis, and illness severity. Psychol. Addict. Behav. 2003, 17, 259–265. [Google Scholar] [CrossRef]
  23. Goldstein, B.I.; Birmaher, B.; Axelson, D.A.; Goldstein, T.R.; Esposito-Smythers, C.; Strober, M.A.; Hunt, J.; Leonard, H.; Gill, M.K.; Iyengar, S.; et al. Significance of cigarette smoking among youths with bipolar disorder. Amer. J. Addiction 2008, 17, 364–371. [Google Scholar]
  24. Ostacher, M.J.; Lebeau, R.T.; Perlis, R.H.; Nierenberg, A.A.; Lund, H.G.; Moshier, S.J.; Sachs, G.S.; Simon, N.M. Cigarette smoking is associated with suicidality in bipolar disorder. Bipolar Disord. 2009, 11, 766–771. [Google Scholar] [CrossRef]
  25. Altamura, A.C.; Bassetti, R.; Bignotti, S.; Pioli, R.; Mundo, E. Clinical variables related to suicide attempts in schizophrenic patients: A retrospective study. Schizophr. Res. 2003, 60, 47–55. [Google Scholar]
  26. de Leon, J. Smoking and vulnerability for schizophrenia. Schizophr. Bull. 1996, 22, 405–409. [Google Scholar] [CrossRef]
  27. Sacco, K.A.; Termine, A.; Seyal, A.; Dudas, M.M.; Vessicchio, J.C.; Krishnan-Sarin, S.; Jatlow, P.I.; Wexler, B.E.; George, T.P. Effects of cigarette smoking on spatial working memory and attentional deficits in schizophrenia: Involvement of nicotinic receptor mechanisms. Arch. Gen. Psychiat. 2005, 62, 649–659. [Google Scholar] [CrossRef]
  28. Leonard, S.; Adams, C.E. Smoking cessation and schizophrenia. Amer. J. Psychiat. 2006, 163, 1877. [Google Scholar] [CrossRef]
  29. Baker, A.; Richmond, R.; Haile, M.; Lewin, T.J.; Carr, V.J.; Taylor, R.L.; Constable, P.M.; Jansons, S.; Wilhelm, K.; Moeller-Saxone, K. Characteristics of smokers with a psychotic disorder and implications for smoking interventions. Psychiat. Res. 2007, 150, 141–152. [Google Scholar] [CrossRef]
  30. Levander, S.; Eberhard, J.; Lindstrom, E. Nicotine use and its correlates in patients with psychosis. Acta Psychiat. Scand. Suppl. 2007, 27–32. [Google Scholar] [CrossRef]
  31. McEvoy, J.P.; Allen, T.B. The importance of nicotinic acetylcholine receptors in schizophrenia, bipolar disorder and tourette’s syndrome. Curr. Drug Targets CNS Neurol. Disord. 2002, 1, 433–442. [Google Scholar] [CrossRef]
  32. McKee, S.A.; Weinberger, A.H.; Harrison, E.L.; Coppola, S.; George, T.P. Effects of the nicotinic receptor antagonist mecamylamine on ad-lib smoking behavior, topography, and nicotine levels in smokers with and without schizophrenia: A preliminary study. Schizophr. Res. 2009, 115, 317–324. [Google Scholar] [CrossRef]
  33. Strand, J.E.; Nyback, H. Tobacco use in schizophrenia: A study of cotinine concentrations in the saliva of patients and controls. Eur. Psychiat. 2005, 20, 50–54. [Google Scholar] [CrossRef]
  34. Etter, M.; Mohr, S.; Garin, C.; Etter, J.F. Stages of change in smokers with schizophrenia or schizoaffective disorder and in the general population. Schizophr. Bull. 2004, 30, 459–468. [Google Scholar] [CrossRef]
  35. Weinberger, A.H.; Sacco, K.A.; Creeden, C.L.; Vessicchio, J.C.; Jatlow, P.I.; George, T.P. Effects of acute abstinence, reinstatement, and mecamylamine on biochemical and behavioral measures of cigarette smoking in schizophrenia. Schizophr. Res. 2007, 91, 217–225. [Google Scholar] [CrossRef]
  36. Williams, J.M.; Ziedonis, D. Addressing tobacco among individuals with a mental illness or an addiction. Addict. Behav. 2004, 29, 1067–1083. [Google Scholar] [CrossRef]
  37. Siru, R.; Hulse, G.K.; Tait, R.J. Assessing motivation to quit smoking in people with mental illness: A review. Addiction 2009, 104, 719–733. [Google Scholar] [CrossRef]
  38. Ashton, M.; Miller, C.L.; Bowden, J.A.; Bertossa, S. People with mental illness can tackle tobacco. Aust. N. Z. J. Psychiat. 2010, 44, 1021–1028. [Google Scholar]
  39. Evins, A.E.; Cather, C.; Rigotti, N.A.; Freudenreich, O.; Henderson, D.C.; Olm-Shipman, C.M.; Goff, D.C. Two-year follow-up of a smoking cessation trial in patients with schizophrenia: Increased rates of smoking cessation and reduction. J. Clin. Psychiat. 2004, 65, 307–311. [Google Scholar] [CrossRef]
  40. Baker, A.; Richmond, R.; Haile, M.; Lewin, T.J.; Carr, V.J.; Taylor, R.L.; Jansons, S.; Wilhelm, K. A randomized controlled trial of a smoking cessation intervention among people with a psychotic disorder. Amer. J. Psychiat. 2006, 163, 1934–1942. [Google Scholar] [CrossRef]
  41. Dalack, G.W.; Becks, L.; Hill, E.; Pomerleau, O.F.; Meador-Woodruff, J.H. Nicotine withdrawal and psychiatric symptoms in cigarette smokers with schizophrenia. Neuropsychopharmacology 1999, 21, 195–202. [Google Scholar] [CrossRef]
  42. Evins, A.E.; Deckersbach, T.; Cather, C.; Freudenreich, O.; Culhane, M.A.; Henderson, D.C.; Green, M.F.; Schoenfeld, D.A.; Rigotti, N.A.; Goff, D.C. Independent effects of tobacco abstinence and bupropion on cognitive function in schizophrenia. J. Clin. Psychiat. 2005, 66, 1184–1190. [Google Scholar] [CrossRef]
  43. George, T.P.; Vessicchio, J.C.; Sacco, K.A.; Weinberger, A.H.; Dudas, M.M.; Allen, T.M.; Creeden, C.L.; Potenza, M.N.; Feingold, A.; Jatlow, P.I. A placebo-controlled trial of bupropion combined with nicotine patch for smoking cessation in schizophrenia. Biol. Psychiat. 2008, 63, 1092–1096. [Google Scholar] [CrossRef]
  44. Rüther, T.; Bobes, J.; De Hert, M.; Svensson, T.; Mann, K.; Batra, A.; Gorwood, P.; Möller, H.-J. EPA—Position statement on smoking and strategies for smoking cessation in people with mental illness. Eur. Psychiat. 2013, 3. [Google Scholar] [CrossRef]
  45. First, M.; Spitzer, R.; Gibbon, M.; Williams, J. Entrevista Clínica Estructurada para los Trastornos del Eje I del DSM-IV—Versión Clinica (SCID-I); Masson: Barcelona, Spain, 2001. [Google Scholar]
  46. Puente, D.; Cabezas, C.; Rodriguez-Blanco, T.; Fernandez-Alonso, C.; Cebrian, T.; Torrecilla, M.; Clemente, L.; Martin, C. The role of gender in a smoking cessation intervention: A cluster randomized clinical trial. BMC Public Health 2011, 11, 369. [Google Scholar] [CrossRef]
  47. Masters, N.; Tutt, C. Smoking Pack Years Calculator. Available online: http://smokingpackyears.com/calculate (accessed on 23 December 2013).
  48. Heatherton, T.F.; Kozlowski, L.T.; Frecker, R.C.; Fagerstrom, K.O. The fagerstrom test for nicotine dependence: A revision of the fagerstrom tolerance questionnaire. Br. J. Addict. 1991, 86, 1119–1127. [Google Scholar] [CrossRef]
  49. Jarvis, M.J.; Tunstall-Pedoe, H.; Feyerabend, C.; Vesey, C.; Saloojee, Y. Comparison of tests used to distinguish smokers from nonsmokers. Amer. J. Public Health 1987, 77, 1435–1438. [Google Scholar] [CrossRef]
  50. Becona, E.; Vazquez, F.L. The fagerstrom test for nicotine dependence in a Spanish sample. Psychol. Rep. 1998, 83, 1455–1458. [Google Scholar]
  51. Nerin, I.; Crucelaegui, A.; Novella, P.; Beamonte, A.; Sobradiel, N.; Bernal, V.; Gargallo, P. Assessment of behavioral dependence with the glover-nilsson test in smoking cessation treatment. Arch. Bronconeumol. 2005, 41, 493–498. [Google Scholar] [CrossRef]
  52. Richmond, R.L.; Kehoe, L.A.; Webster, I.W. Multivariate models for predicting abstention following intervention to stop smoking by general practitioners. Addiction 1993, 88, 1127–1135. [Google Scholar] [CrossRef]
  53. Gomez-Pena, M.; Penelo, E.; Granero, R.; Fernandez-Aranda, F.; Alvarez-Moya, E.; Santamaria, J.J.; Moragas, L.; Aymami, M.N.; Bueno, B.; Gunnard, K.; et al. Motivation to change and pathological gambling: Analysis of the relationship with clinical and psychopathological variables. Br. J. Clin. Psychol. 2011, 50, 196–210. [Google Scholar] [CrossRef]
  54. Mesa, E.M.D.; Garcia-Portilla, P.; Saiz, P.A.; Bascaran, T.B.; Casares, M.J.; Fonseca, E.; Carreno, E.; Florez, G.; Guardia, J.; Ochoa, E.; et al. Psychometric performance of the 6th version of the addiction severity index in Spanish (ASI-6). Psicothema 2010, 22, 513–519. [Google Scholar]
  55. Lozano, R.P.; Garcia, Y.A.; Tafalla, D.B.; Albaladejo, M.F. Caffeine: A nutrient, a drug or a drug of abuse. Adicciones 2007, 19, 225–238. [Google Scholar]
  56. Peralta, V.; Cuesta, M.J. Psychometric properties of the positive and negative syndrome scale (PANSS) in schizophrenia. Psychiat. Res. 1994, 53, 31–40. [Google Scholar] [CrossRef]
  57. Bobes, J.; Bulbena, A.; Luque, A.; Dal-Re, R.; Ballesteros, J.; Ibarra, N. A comparative psychometric study of the Spanish versions with 6, 17, and 21 items of the Hamilton depression rating scale. Med. Clin. 2003, 120, 693–700. [Google Scholar]
  58. Colom, F.; Vieta, E.; Martinez-Aran, A.; Garcia-Garcia, M.; Reinares, M.; Torrent, C.; Goikolea, J.M.; Banus, S.; Salamero, M. Spanish version of a scale for the assessment of mania: Validity and reliability of the young mania rating scale. Med. Clin. 2002, 119, 366–371. [Google Scholar] [CrossRef]
  59. Guy, W. Early Clinical Drug Evaluation (Ecdeu) Assessment Manual; National Institute Mental Health: Rockville, MD, USA, 1976.
  60. Lingjaerde, O.; Ahlfors, U.G.; Bech, P.; Dencker, S.J.; Elgen, K. The UKU side effect rating scale. A new comprehensive rating scale for psychotropic drugs and a cross-sectional study of side effects in neuroleptic-treated patients. Acta Psychiat. Scand. Suppl. 1987, 334, 1–100. [Google Scholar]
  61. Fiore, M.; Jaen, C.R.; Baker, T.B.; Guideline, P. Treating tobacco use and dependence: 2008 update USA. Public health service clinical practice guideline executive summary. Respir. Care 2008, 53, 1217–1222. [Google Scholar]
  62. AEMPS. Guía de Prescripción Terapéutica. Available online: http://www.aemps.gob.es/medicamentosUsoHumano/portada/home.htm (accessed on 23 December 2013).
  63. Miller, W.R.; Rollnick, S. Motivational Interviewing: Preparing People for Change, 2nd ed.; Guilford Press: New York, NY, USA, 2002. [Google Scholar]
  64. Prochaska, J.O.; DiClemente, C.C. Stages and processes of self-change of smoking: Toward an integrative model of change. J. Consult. Clin. Psychol. 1983, 51, 390–395. [Google Scholar]
  65. Lawn, S. Australians with mental illness who smoke. Br J Psychiat. 2001, 178. [Google Scholar] [CrossRef]
  66. Riemsma, R.P.; Pattenden, J.; Bridle, C.; Sowden, A.J.; Mather, L.; Watt, I.S.; Walker, A. Systematic review of the effectiveness of stage based interventions to promote smoking cessation. BMJ 2003, 326, 1175–1177. [Google Scholar]
  67. West, R. Time for a change: Putting the transtheoretical (stages of change) model to rest. Addiction 2005, 100, 1036–1039. [Google Scholar]

Share and Cite

MDPI and ACS Style

Garcia-Portilla, M.P.; Garcia-Alvarez, L.; Saiz, P.A.; Diaz-Mesa, E.; Galvan, G.; Sarramea, F.; Garcia-Blanco, J.; Elizagarate, E.; Bobes, J. Effectiveness of a Multi-Component Smoking Cessation Support Programme (McSCSP) for Patients with Severe Mental Disorders: Study Design. Int. J. Environ. Res. Public Health 2014, 11, 373-389. https://doi.org/10.3390/ijerph110100373

AMA Style

Garcia-Portilla MP, Garcia-Alvarez L, Saiz PA, Diaz-Mesa E, Galvan G, Sarramea F, Garcia-Blanco J, Elizagarate E, Bobes J. Effectiveness of a Multi-Component Smoking Cessation Support Programme (McSCSP) for Patients with Severe Mental Disorders: Study Design. International Journal of Environmental Research and Public Health. 2014; 11(1):373-389. https://doi.org/10.3390/ijerph110100373

Chicago/Turabian Style

Garcia-Portilla, Maria Paz, Leticia Garcia-Alvarez, Pilar Alejandra Saiz, Eva Diaz-Mesa, Gonzalo Galvan, Fernando Sarramea, Josefa Garcia-Blanco, Edorta Elizagarate, and Julio Bobes. 2014. "Effectiveness of a Multi-Component Smoking Cessation Support Programme (McSCSP) for Patients with Severe Mental Disorders: Study Design" International Journal of Environmental Research and Public Health 11, no. 1: 373-389. https://doi.org/10.3390/ijerph110100373

Article Metrics

Back to TopTop