Next Article in Journal
Unified Protocol for the Transdiagnostic Treatment of Emotional Disorders in Medical Conditions: A Systematic Review
Previous Article in Journal
Analysis of Bio-Risk Management System Implementation in Indonesian Higher Education Laboratory
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Optimizing Tobacco Advertising Bans in Seven Latin American Countries: Microsimulation Modeling of Health and Financial Impact to Inform Evidence-Based Policy

by
Ariel Bardach
1,2,
Andrea Alcaraz
1,
Javier Roberti
1,2,*,
Agustín Ciapponi
1,2,
Federico Augustovski
1,2 and
Andrés Pichon-Riviere
1,2
1
Department of Health Technology Assessment and Economic Evaluation, Institute for Clinical Effectiveness and Health Policy, Ravignani 2024, Buenos Aires 1414, Argentina
2
CIESP–CONICET, Centre for Research in Epidemiology and Public Health/National Scientific and Technical Research Council, Ravignani 2024, Buenos Aires 1414, Argentina
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2021, 18(10), 5078; https://doi.org/10.3390/ijerph18105078
Submission received: 15 March 2021 / Revised: 10 April 2021 / Accepted: 12 April 2021 / Published: 11 May 2021
(This article belongs to the Section Health Behavior, Chronic Disease and Health Promotion)

Abstract

:
Introduction: In Latin America, tobacco smoking prevalence is between 6.4% and 35.2%. Governments have been making efforts to support the regulation of advertising and, in many cases, banning advertising and promotion of tobacco altogether. The objective of this study was to evaluate the potential impact on health and economic outcomes of optimizing a ban on tobacco advertising and sponsorship in Argentina, Bolivia, Brazil, Chile, Colombia, Mexico, and Peru. Methods: We built a probabilistic microsimulation model, considering natural history, direct health system costs, and quality of life impairment associated with main tobacco-related diseases. We followed individuals in hypothetical cohorts and calculated health outcomes on an annual basis to obtain aggregated 10-year population health outcomes (deaths, events, healthy years of life) and costs. To populate the model, we performed a rapid review of literature to calculate intervention effectiveness. Results: With current policies, over 10 years, in Argentina, Bolivia, Brazil, Chile, and Colombia a total of 50,000 deaths and 364,000 disease events will be averted, saving $7.2 billion. If the seven countries strengthened their policies and implemented a comprehensive ban with 100% compliance, 98,000 deaths and 648,000 events would be averted over 10 years, saving almost $15 billion in healthcare costs. Conclusions: Optimizing a ban on tobacco advertising and sponsorship would substantially reduce deaths, diseases, and health care costs attributed to smoking. Latin American countries should not delay the full implementation of this strategy.

1. Introduction

Every year, over seven million people die around the world because of the tobacco pandemic, the single most preventable cause of premature mortality; moreover, between 1990 and 2017, the total number of smoking-attributable deaths increased by 24.9% [1]. The expenditure in healthcare for smoking-attributable diseases currently exceeds USD 400 billion per year and the economic cost of smoking represents USD 1436 billion [2]. In Latin America, the annual consumption of tobacco per person ranges from 160 to 2000 cigarettes and the tobacco smoking prevalence between 6.4% and 35.2% [3].
Tobacco advertising in its many forms increases tobacco consumption [4]. Marketing of tobacco products includes direct advertising on television, radio, magazines, newspapers, billboards, and retail points-of-sale (POS) whereas indirect marketing refers to the free distribution of products, promotional discounts, the appearance of tobacco products in television or films, sponsorship of sport and music events, and the distribution of nontobacco products identified with tobacco brand names [5]. Since the early days of the entertainment industry, audiences have viewed smoking in films; and the contribution of this strategy in the promotion of smoking has been long studied, with a focus on smoking initiation in adolescents [6,7,8,9,10]. Indeed, POS marketing has been associated with an increased smoking susceptibility, experimentation, and uptake [11,12,13]. A more recent and growing marketing strategy is the tobacco advertising and branding on social media and other internet-based advertising [14,15].
Governments have been making efforts to strengthen the regulation of advertising and, in many cases, banning it altogether [4]. In fact, although comprehensive bans of advertising can result in a decline in the per capita consumption, limited bans have no significant impact on consumption because it leads to a change of advertising media, from those banned media towards those media that are still allowed [4,16]. On the opposite side, tobacco and advertising industries contend that these marketing regulations have negative economic consequences, infringe legal rights of tobacco companies, create regulatory redundancy, and that evidence supporting their effects is insufficient [17].
In 2008, the member states of the World Health Organization (WHO) adopted a treaty to provide a context for the development of tobacco control policy; the WHO Framework Convention on Tobacco Control (FCTC). The FCTC introduced a practical and cost-effective package of six evidence-based measures, referred to as MPOWER, and these measures correspond to the provisions of the framework. These measures include: Monitoring tobacco use and tobacco control measures (Article 20); Protecting people from tobacco smoke (Article 8); Offering help to quit tobacco (Article 14); Warning people about the dangers of tobacco (Articles 11 and 12); Enforcing bans on tobacco advertising, promotion and sponsorship (Article 13); and Raising tobacco taxes (Article 6) [18,19]. Despite an accelerating impact of MPOWER policies, the implementation of legislation related to tobacco advertising at a global level is still limited [20,21,22]; moreover, scarcity of quality information at a country-level, lobbying, and constituency building from the tobacco industry have delayed the implementation and enforcement of these measures in Latin America [23,24]. In the group of studied countries, Peru and Bolivia have the minimum level of implementation; Argentina and Mexico are in an intermediate level of implementation with a ban of advertising on national television, radio, print media, and some direct or indirect forms of advertising, and although Colombia, Brazil, and Chile have the maximum level of implementation with a comprehensive ban of all forms of advertising, there is still room for improvement in the compliance of measures [25].
The objective of this study was to assess the 10-year health and economic impact of the current country-specific legislation and implementation related to bans on tobacco advertising, promotion, and sponsorship; and to compare this impact to the expected effects of moving to total ban of advertising in Argentina, Bolivia, Brazil, Chile, Colombia, Mexico, and Peru, using a probabilistic state-transition microsimulation model developed to estimate the burden of smoking-attributable disease and the cost-effectiveness of tobacco control policies and interventions in Latin America.

2. Methods

The model used in this study is an individual-based Markov model, or first-order Monte Carlo technique [26]. The smoking burden model allows for the estimation of health and economic impact of tobacco, at present and after implementation of interventions to reduce smoking prevalence. This model has been validated and applied in studies carried out in 12 Latin American countries that estimated the burden of disease and the expected impact of tobacco tax increases and other strategies [25,27,28,29,30,31,32,33].

2.1. Disease Burden Modeling

The model considers the natural history, the costs, and the quality-of-life losses associated with main tobacco-related diseases (coronary and non-coronary heart disease, cerebrovascular disease, chronic obstructive pulmonary disease, pneumonia, influenza, lung cancer, and nine other neoplasms). We followed up individuals in hypothetical cohorts and calculated health outcomes on an annual basis to obtain aggregated long-term population health outcomes and costs. For acute events, we calculated age and gender-specific absolute risks based on national mortality rates and the lethality of the event. Then, the baseline risk in non-smokers is calculated based on the smoking prevalence in each age and sex group, and the relative risk of smoking associated with each event. For cancers, we obtained incidence statistics for each age and sex with Global Cancer Observatory (GCO -GLOBOCAN) for each country [34].
The model updates the input parameters for each subject in yearly cycles and calculates individual lifetime risks of occurrence of each event, disease progression, and death, based on demographic attributes, smoking status, and clinical conditions according to the underlying risk equations. The main outcomes are life years, quality-adjusted life years, disease events, hospitalizations, disease incidence, and disease costs. We calculated the years of life lost (YLL) due to smoking-related diseases at a population level as the sum of years of life lost due to premature death (PYLL); and years of life lost due to living with a poor quality of life (YLL-QL). As the model does not directly calculate the consequences of passive smoking and perinatal effects, based on the results of previous studies, we assumed that these causes impose an additional burden of 13.6% for men and 12% for women [35].

2.2. Policy Effectiveness Modeling

Tobacco control policies have an effect mediated by a reduction in consumption. This lower consumption at the country level is a consequence of both a reduction in the number of cigarettes smoked per smoker, and lower tobacco prevalence due to an increase in quitting rates (short term) and lower tobacco initiation rates in the medium and long term. To estimate the impact of implementing strategies on tobacco advertising and sponsorship ban, the smoking prevalence post-intervention was calculated as:
Prevalencepost = Prevalencepre − (EmIpPrevalencepre)
where Prevalencepre is the prevalence of smokers before the intervention, Em is the effectiveness of the intervention expressed as relative reduction in tobacco consumption, and Ip it is the proportion of variation in consumption that impacts smoker prevalence. Different studies have estimated that, in the short and medium term, approximately half of the reduction in consumption is a consequence of reduced prevalence and the other half is explained by reduced consumption of continuing smokers [36,37,38,39,40].

2.3. Model Scenarios and Base Case Analysis

To estimate the potential impact of banning advertising, promotion, and sponsorship, we considered and analyzed progressive estimates, over ten years, in each country in three scenarios (short-term: 2 years, mid-term: 5 years, and long-term: 10 years) to calculate the reduction of the health burden associated with the reduction in cigarette consumption.
  • Short-term scenario: In this scenario, we assumed that a 50% reduction in consumption would have an impact on prevalence (Ip = 0.5) which, in turn, led to an increase in former smokers. This conservative scenario is more likely to occur in the short term, as it does not include the intervention effects in preventing people from starting to smoke or the health benefits of smoking fewer cigarettes for those who continued smoking.
  • Mid-term scenario: Similar to the previous scenario, but also including the potential effects associated with a decrease in number of cigarettes smoked in people who continued smoking. Considering that low-intensity smokers have, on average, 75% less excess disease risk than a high-intensity smoker when compared to non-smokers (82% less for lung cancer, 57% less for ischemic heart disease and 80% less for chronic obstructive pulmonary disease (COPD)), we assumed that a reduction in the number of cigarettes would result in a proportional reduction in the 75% of the excess risk difference between a smoker and a former-smoker [41].
  • Long-term scenario: This is the maximum effect over ten years. Similar to the previous scenario but with a 75% reduction in consumption affecting prevalence (Ip = 0.75); the population of former smokers remains constant in relation to the baseline, with decrease in prevalence and an increase in the number of non-smokers.
The base case consisted of comparing health benefits and costs of current tobacco advertising and sponsorship policy in each country to those predicted by implementing a complete ban. To estimate disease burden and costs of the strategy, we assumed a lineal evolution from scenario 1 to scenario 2 within five years, and then to scenario 3 between years six to ten.
The burden of disease attributable to smoking was estimated for these scenarios based on these estimates of changes in smoking prevalence and new proportions of smokers, former smokers, and non-smokers. Health impact was calculated as the observed difference between baseline burden (status quo) and the complete ban strategy estimates, in terms of deaths, disease events, years lived, disability, and health costs. More information about the model can be found in the publications in which it was described, evaluated, or used [25,26,27,28,29,30,31,32,33,42,43,44,45], and in the technical reports with findings on the burden of disease (available from www.iecs.org.ar/tabaco, access date 30 March 2021).

2.4. Information Sources for the Model

Epidemiological Information

To populate the simulation model, we obtained data through a review of the literature on MEDLINE, EMBASE, CENTRAL, SOCINDEX, EconLit, LILACS, NBER, CRD and Cost Effectiveness Analysis Registry, the International Tobacco Health Conference Paper Index, and Cochrane Tobacco Addiction Review Group register. In addition, we reviewed grey literature from ministries of health or of finance, Pan American Health Organization, and regional congresses proceedings. We obtained updated information on tobacco use from Global Adult Tobacco Surveys (GATS) and national risk factor surveys. Researchers from participating countries provided information from civil registrations, vital statistics, and hospital discharge databases to estimate specific case fatality rates.

2.5. Cost Information

We performed a literature search to identify reported costs of events and developed a common costing methodology to estimate costs through a micro-costing or macro-costing approach, depending on the information availability. Then, we used a spreadsheet for each event, with the frequency, the use rate, and the unit cost of health resources. We constructed ad hoc micro-costing exercises, based on experts’ opinions, clinical guidelines, and a review of healthcare facility records. The costs of malignancies other than lung cancer were based on cost of each cancer relative to lung cancer costs and consensus using a Delphi method exercise with oncology experts from studied countries. Where local information was unavailable, we extrapolated the model to approximate costs of events. In those cases, we used the average proportion that represents event cost divided by per capita GDP in Argentina, Chile, and Mexico; then, on this average proportion, the per capita GDP of the country of interest was applied to obtain estimates.
All costs were first estimated in the local currency; consumer price indices, published by the statistics institutes of each country, were used for cost adjustments. Finally, for comparability, all costs in local currency were converted to International dollar (I$) based on the World Bank purchasing power parity exchange rates for 2018.

2.6. Estimation for Intervention Impact

For the assessment of the impact of banning tobacco advertising, promotion, and sponsorship, three possible levels of implementation were considered. At the minimum level of implementation, the absence of a ban and a ban restricted only to national television, radio, or print media were included. The literature agrees that this type of ban has limited or no effect; so, we treated this as absence of a ban. This minimum or no implementation is used as the reference category, with no effectiveness (0%). The intermediate level of implementation covers national television, radio, or print media, and some direct and/or indirect forms of advertising. Our review showed that the effectiveness on relative reduction in per capita consumption is wide-ranging, from 0% to 13.6%; we selected 1% as central estimator as it was the most consistent value suggested by the major sources and used in a study including 102 countries. The maximum level of implementation was the ban of all forms of advertising, direct and indirect, including product display at POS. In this category associated with legislation, the level of compliance to the ban will clearly determine effectiveness and the potential impact of taking the measure to the maximum level. The range of reduction in per capita tobacco consumption associated with this level is 5% to 23.5% (see Table 1). As a central estimator, the two best options were: a reduction of cigarette consumption of 7.4% described for 22 resource-rich countries (cited by the literature) and 9% reported in a study of 102 countries. Supplemental Table S1 shows the studies used as sources for effectiveness range.

2.7. Calibration and Validation of the Model

We applied the International Society for Pharmacoeconomics and Outcomes Research criteria for model development and reporting to calibrate the model in each country; we compared the disease-specific mortality rates predicted by the model with the national statistics for 16 conditions (excluding COPD mortality, which is universally agreed to be underestimated in national statistics) [46]. Sex- and age-specific model outputs were compared to the source rates and deviations from the expected values were analyzed. Predicted rates within 10% of the references were considered acceptable. In case of greater deviation, risk equations were modified until the parameter was in the acceptable range. Goodness of fit was additionally assessed by plotting predicted versus observed values outcomes, fitting a linear curve through the points with the intercept set at zero, and obtaining a squared linear correlation coefficient. We externally validated the model, checking the results against results of other epidemiological and clinical studies not used for equation estimation or model development.

3. Results

3.1. Data to Populate the Model

We identified all the epidemiological and cost parameters needed to populate the model and show the main results of input parameters in Table 1. The rapid review on the effectiveness of ban of advertising and sponsorship showed that smoking prevalence could be reduced by 1% for a partial ban and 9% for a comprehensive ban.

3.2. Current Policies in the Seven Countries

Peru and Mexico are currently at the minimum level of implementation of a ban on advertising and promotion of tobacco products. At the intermediate level of implementation of the ban, which covers national television, radio, or print media, as well as some, but not all, direct and/or indirect forms of advertising are Argentina, Bolivia, and Chile. Finally, at the maximum level of implementation with a comprehensive ban of all forms of advertising, but with varying levels of compliance, are Brazil and Colombia.
Table 2 shows current policies in the seven countries studied. According to the Report on Tobacco Control in the Americas 2016, Mexico prohibits direct advertising on national radio and television, on billboards, and advertisements from abroad. As for indirect advertising, the ban prohibits product promotion, but allows advertising and display at POS. In Mexico, compliance with the guidelines of Articles 13 and 15 of the WHO FCTC is required to fully comply with the recommendations of the FCTC and to improve the effectiveness of this intervention. Finally, Peru has a ban that does not cover national television, radio, or print media; the MPOWER report score for direct advertising is 9 and does not apply to indirect advertising (available from www.iecs.org.ar/tabaco, access date 30 March 2021) [26,27,28,29,30,31]. According to the categorization in the 2015 MPOWER report, in Argentina, Bolivia, and Chile, the ban covers national television, radio, or print media, as well as some, but not all, direct and/or indirect forms of advertising. The report assigns the following scores on a scale of 0 to 10. Argentina: 9 for direct advertising and 6 for indirect advertising; Bolivia: 9 for direct advertising and 3 for indirect advertising; Chile: 6 for direct advertising and 10 for indirect advertising. In the case of Brazil and Colombia, the ban covers all forms of direct and indirect advertising, with the following scores. Brazil with a score of 9 for both direct and indirect advertising; Colombia: 10 for direct advertising and 5 for indirect advertising.

3.3. Health and Economic Effects of Current Strategies Implementation in the Seven Countries

The policies in advertising that are currently in place in Argentina, Bolivia, Brazil, Chile, and Colombia are already producing health benefits and saving health costs to their systems as a result of their potential to avert a total of 50,000 deaths over the next 10 years. Specifically, over the next 10 years, 199,000 cardiac diseases, 47,000 cerebrovascular diseases, 92,912 COPD cases, and 26,000 cases of cancer could be averted; also, over 1.6 million years of life could be added. Averted events could represent savings totaling I$7.2 billion over the same period. With the largest population in the group of studied countries, Brazil could avert 40,000 deaths, and generate I$5.8 billion in savings (see Table 3). Of note, Mexico and Peru have limited or no bans now.

3.4. Additional Potential Benefits of Strengthening Bans of Advertising and Sponsorship

With no ban or a limited ban of tobacco advertising and sponsorship, if Mexico and Peru advanced from their current category to a partial ban, in the next 10 years, Mexico would avert 1700 deaths, almost 10,000 events, would add 47,000 years lived, and save I$313.2 million, whereas Peru would avert 588 deaths, 2300 events, add 14,400 years lived, and save I$60.2 million in health costs. However, if these countries moved to a full ban and implementation, over the next 10 years, Mexico would avert an additional 15,300 deaths, 87,441 events, would add 419,000 years lived, and save US$2.8 billion, whereas Peru would avert 5300 deaths, 20,400 events, add 130,000 years lived, and save I$541.4 million in health costs. Of note, these results are incremental and should be added to the benefits obtained in the first step, with a partial implementation.
In a second group are Argentina, Bolivia, and Chile, which have bans in intermediate level, if these three countries increased compliance to 100% of existing bans, in the next 10 years, the three countries would avert an additional 759 deaths, 3200 events, would add 19,000 years lived, and save I$111.4 million. Brazil and Colombia already have comprehensive bans with a 90% and 75% compliance, respectively. If they increased compliance to 100%, they would avert an additional 8300 deaths, 55,000 events, would add 239,000 healthy years lived, and save I$1.07 billion in health costs. Of note, these benefits are added to the benefits obtained as a result of the status-quo restrictions.
In a scenario in which all seven countries implemented a comprehensive ban with a 100% compliance, the total numbers over 10 years would be: 98,000 premature deaths, 648,000 events, a total of 3 million healthy years of life would be added, and a total of almost I$15.7 billion in direct healthcare expenses of diseases attributable to smoking would be saved (Table 3).

4. Discussion

Our findings show that the benefits of policies already implemented in Argentina, Bolivia, Brazil, Chile, and Colombia could avert 50,108 deaths, add 1.6 million healthy life years, and save I$7.2 billion, and that optimized ban tobacco advertising and sponsorship in the seven countries, including Mexico and Peru—which have no bans at the moment—could avert more 98,000 premature deaths, and 649,000 events, adding 3 million healthy life years over ten years. This would translate into significant savings for the health system across the Latin American region, with total healthcare savings over I$15 billion.
The smoking epidemic is well recognized in Latin America; changes are urgently needed to continue reducing mortality due to smoking-related diseases [47]. Estimates suggest that during the last decade, the impact of tobacco was equivalent to 2 to 6 years of losses in life expectancy for males who smoke tobacco at 50 years [47]. In the region, the tobacco industry has been offering a strong opposition and, as a result, passing control legislation has proved difficult [48]. Uruguay has been presented as an example in its efforts to reduce tobacco consumption, followed by Brazil and Panama; in the rest of the region, however, results are varied [48].
In Latin American countries there exists an inverse relationship between income level and smoking prevalence, showing that tobacco inflicts greater harm on the most disadvantaged groups [24]. Moreover, it has been suggested that a ban on tobacco advertising could have greater impact on deprived groups, with lower education levels, who are more enticed by advertising [16]. Effective restrictions on advertising are needed in the region; however, bans need to be comprehensive to achieve a reduction in tobacco consumption [4]. Compared to other tobacco control policies, the definition, implementation, and enforcement of tobacco advertising bans is more difficult because of the ever-changing nature of tobacco promotion [49]. These initiatives also require continuous monitoring of, and response to, industry efforts to circumvent bans through new ways to promote tobacco [49,50]. Although the industry contests the bans of advertising, often these result in a higher market concentration benefiting transnational tobacco companies [49,50].
Levy et al. showed the significant potential of adopting the highest level of MPOWER tobacco control measures between 2007 and 2014, with an estimated 22 million smoking-associated deaths averted [51]. The impact of any tobacco control intervention depends on the extent of the already implemented control policies or whether several policies are implemented simultaneously [5,52]. In fact, a ban on advertising should be used in the context of other measures. In other studies, our group estimated the health and cost effects of applying tax increase, plain packaging, and smoke-free strategies in the region and found that the first strategy had the most significant effect [22,25,44,45]. However, as it occurs with the other measures, the effects of a comprehensive ban on tobacco advertising and sponsorship would result in important health benefits and savings for the health systems in the region.
Policies should also include POS marketing, one of the few remaining channels to promote tobacco in many countries [12,53]. As governments implement regulations in traditional media outlets, tobacco companies increasingly market their products to customers at the POS, with product placement, brand exhibition, and price discounts. It has been shown that smoking prevalence decreased as the exposure to a POS display ban increased, with a reduction of 5% and 9% in the male and female populations, respectively [53]. A study in Canada showed that smokers—those most likely to oppose regulations—supported banning POS display of tobacco products, and this should help persuade policymakers to take action [54].
Our study has limitations that should be acknowledged. The main limitation is the scarcity of quality evidence on the effectiveness of policies. We estimated direct medical costs related to smoking, a part of total financial burden of tobacco, but not indirect costs, such as productivity losses. The model did not include some conditions related to exposure to tobacco such as breast cancer, diabetes, liver cancer, or kidney failure. In addition, we did not incorporate the use of alternative nicotine delivery products. Another limitation to be considered was that it was not always possible to include high-quality epidemiological information to populate the model. In addition, due to scarcity of information, changes in demographic, economic, and healthcare system characteristics over time were not included in the model. However, our findings offer a robust estimate of financial burden of smoking in seven countries of Latin America, with the best available sources of information in each country, applying a uniform and replicable method.
In conclusion, advertising ban policies currently in place are producing non-negligible health and economic benefits in five of those countries studied. However, the adoption of a comprehensive ban on tobacco advertising is necessary to substantially enhance the health and financial benefits of reducing the smoking epidemic in the region.

Supplementary Materials

The following are available online at https://www.mdpi.com/article/10.3390/ijerph18105078/s1, Table S1: Studies used as sources for effectiveness range.

Author Contributions

Conceptualization, A.B., F.A. and A.P.-R.; methodology, A.B., F.A. and A.P.-R.; software, A.C., F.A. and A.P.-R.; validation, A.A. and A.B.; formal analysis, A.B., A.A., A.C. and F.A.; investigation, A.B., A.A., A.C., F.A. and J.R.; resources, A.A. and A.B.; data curation, A.B. and A.C.; writing—original draft preparation, J.R. and A.B.; writing—review and editing, J.R., A.B. and F.A.; visualization, A.B. and J.R.; supervision, A.P.-R.; project administration, A.B.; funding acquisition, A.P.-R. All authors have read and agreed to the published version of the manuscript.

Funding

International Development Research Center-Canada. GRANT Nº 107978-001 and the Institute for Clinical Effectiveness and Health Policy, Argentina.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Data supporting results can be found at IECS website (www.iecs.org.ar/tabaco, access date 30 March 2021).

Acknowledgments

The authors would like to thank Verónica Schoj from the International Heart Foundation-Argentina, for her constructive views in the early phases of this work, Natalie Soto, Sebastián García-Martí, and María Belén Rodríguez for their help in the initial stages, and librarian Daniel Comandé, and Darío Balán, from the Institute for Clinical Effectiveness and Health Policy, for his help with the electronic searches.

Conflicts of Interest

All authors declare no conflicts of interest.

References

  1. GBD 2017 Risk Factors Collaborators. Global, regional, and national comparative risk assessment of 84 behavioural, environmental and occupational, and metabolic risks or clusters of risks for 195 countries and territories, 1990–2017: A systematic analysis for the Global Burden of Disease Study 2017. Lancet 2018, 392, 1923–1994. [Google Scholar] [CrossRef] [Green Version]
  2. Goodchild, M.; Nargis, N.; D’Espaignet, E.T. Global economic cost of smoking-attributable diseases. Tob. Control. 2018, 27, 58–64. [Google Scholar] [CrossRef] [PubMed]
  3. Ng, M.; Freeman, M.K.; Fleming, T.D.; Robinson, M.; Dwyer-Lindgren, L.; Thomson, B.; Wollum, A.; Sanman, E.; Wulf, S.; Lopez, A.D.; et al. Smoking Prevalence and Cigarette Consumption in 187 Countries, 1980-2012. JAMA 2014, 311, 183–192. [Google Scholar] [CrossRef] [Green Version]
  4. Saffer, H.; Chaloupka, F. The effect of tobacco advertising bans on tobacco consumption. J. Health Econ. 2000, 19, 1117–1137. [Google Scholar] [CrossRef]
  5. Levy, D.T.; Tam, J.; Kuo, C.; Fong, G.T.; Chaloupka, F. The Impact of Implementing Tobacco Control Policies: The 2017 Tobacco Control Policy Scorecard. J. Public Health Manag. Pr. 2018, 24, 448–457. [Google Scholar] [CrossRef]
  6. Dalton, A.M.; Sargent, J.D.; Beach, M.L.; Titus-Ernstoff, L.; Gibson, J.J.; Ahrens, M.B.; Tickle, J.J.; Heatherton, T.F. Effect of viewing smoking in movies on adolescent smoking initiation: A cohort study. Lancet 2003, 362, 281–285. [Google Scholar] [CrossRef]
  7. Mejia, R.; Pérez, A.; Peña, L.; Kollath-Cattano, C.; Morello, P.; Braun, S.; Hardin, J.W.; Thrasher, J.F.; Sargent, J.D. Smoking in Movies and Adolescent Smoking Initiation: A Longitudinal Study among Argentinian Adolescents. J. Pediatr. 2017, 180, 222–228. [Google Scholar] [CrossRef] [Green Version]
  8. Morgenstern, M.; Sargent, J.D.; Engels, R.C.; Scholte, R.H.; Florek, E.; Hunt, K.; Sweeting, H.; Mathis, F.; Faggiano, F.; Hanewinkel, R. Smoking in Movies and Adolescent Smoking Initiation. Am. J. Prev. Med. 2013, 44, 339–344. [Google Scholar] [CrossRef] [Green Version]
  9. Sargent, J.D.; Beach, M.L.; Adachi-Mejia, A.M.; Gibson, J.J.; Titus-Ernstoff, L.T.; Carusi, C.P.; Swain, S.D.; Heatherton, T.F.; Dalton, M.A. Exposure to Movie Smoking: Its Relation to Smoking Initiation Among US Adolescents. Pediatrics 2005, 116, 1183–1191. [Google Scholar] [CrossRef] [Green Version]
  10. Mejia, R.M.; Morello, P.; Pérez, A.; Peña, L.; Braun, S.N.; Santillán, E.A.; Barrientos, I.; Rosaura, P.H.; Erika, A.-V.; Christy, K.-C. El cine induce a los adolescentes a consumir tabaco. Es necesario implementar políticas públicas para contrarrestar este fenómeno. Rev. Asoc. Médica Argent. 2018, 131, 24–31. [Google Scholar]
  11. Paynter, J.; Edwards, R. The impact of tobacco promotion at the point of sale: A systematic review. Nicotine Tob. Res. 2009, 11, 25–35. [Google Scholar] [CrossRef] [PubMed]
  12. Shang, C.; Huang, J.; Cheng, K.-W.; Li, Q.; Chaloupka, F.J. Global Evidence on the Association between POS Advertising Bans and Youth Smoking Participation. Int. J. Environ. Res. Public Health 2016, 13, 306. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Haw, S.; Currie, D.; Eadie, D.; Pearce, J.; MacGregor, A.; Stead, M.; Amos, A.; Best, C.; Wilson, M.; Cherrie, M.; et al. The impact of the point-of-sale tobacco display ban on young people in Scotland: Before-and-after study. Public Heal. Res. 2020, 8, 1–118. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  14. Dunlop, S.; Freeman, B.; Perez, D.; Ribisl, K.; Bromberg, J. Exposure to internet-based tobacco advertising and branding: Results from population surveys of Australian youth 2010–2013. J. Med. Internet Res. 2016, 18, e104. [Google Scholar] [CrossRef] [PubMed]
  15. Kids CfT-f, Institute for Global Tobacco Control JHBSoPH, Disease IUaTaL. In Assessing Compliance with Tobacco Advertising, Promotion, and Sponsorship Bans: A “How-To” Guide for Observing the Internet, Sponsored Events, and Corporate Social Responsibility; Institute for Global Tobacco Control: Baltimore, MD, USA, 2014.
  16. Blecher, E. The impact of tobacco advertising bans on consumption in developing countries. J. Health Econ. 2008, 27, 930–942. [Google Scholar] [CrossRef]
  17. Savell, E.; Gilmore, A.B.; Fooks, G. How Does the Tobacco Industry Attempt to Influence Marketing Regulations? A Systematic Review. PLoS ONE 2014, 9, e87389. [Google Scholar] [CrossRef] [PubMed]
  18. World Health Organization (WHO). Tobacco Free Initiative: MPOWER. Available online: http://www.who.int/tobacco/mpower/en/ (accessed on 24 November 2016).
  19. World Health Organization (WHO). Convenio Marco de la OMS Para el Control del Tabaco; World Health Organization: Geneva, Switzerland, 2003. [Google Scholar]
  20. Gulland, A. WHO urges countries to fight tobacco industry’s interference. BMJ 2017, 358, j3416. [Google Scholar] [CrossRef]
  21. Levy, D.T.; Li, Y.; Yuan, Z. Impact of nations meeting the MPOWER targets between 2014 and 2016: An update. Tob. Control. 2019, 29, 231–233. [Google Scholar] [CrossRef]
  22. Flor, L.S.; Reitsma, M.B.; Gupta, V.; Ng, M.; Gakidou, E. The effects of tobacco control policies on global smoking prevalence. Nat. Med. 2021, 27, 239–243. [Google Scholar] [CrossRef]
  23. Guindon, G.E.; Paraje, G.R.; Chaloupka, F.J. The Impact of Prices and Taxes on the Use of Tobacco Products in Latin America and the Caribbean. Am. J. Public Health 2015, 101, 1868–1869. [Google Scholar] [CrossRef]
  24. Bardach, A.; Perdomo, H.; Gándara, R.A.R.; Ciapponi, A. Income levels and prevalence of smoking in Latin America: A systematic review and meta-analysis. Rev. Panam. Salud. Publica 2016, 40, 263–271. [Google Scholar]
  25. Pichon-Riviere, A.; Alcaraz, A.; Palacios, A.; Rodríguez, B.; Reynales-Shigematsu, L.M.; Pinto, M.; Castillo-Riquelme, M.; Torres, E.P.; Osorio, D.I.; Huayanay, L.; et al. The health and economic burden of smoking in 12 Latin American countries and the potential effect of increasing tobacco taxes: An economic modelling study. Lancet Glob. Health 2020, 8, e1282–e1294. [Google Scholar] [CrossRef]
  26. Pichon-Riviere, A.; Augustovski, F.; Bardach, A.; Colantonio, L. Development and Validation of a Microsimulation Economic Model to Evaluate the Disease Burden Associated with Smoking and the Cost-Effectiveness of Tobacco Control Interventions in Latin America. Value Health 2011, 14, S51–S59. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  27. Alcaraz, A.; Caporale, J.; Bardach, A.; Augustovski, F.; Pichon-Riviere, A. Carga de enfermedad atribuible al uso de tabaco en Argentina y potencial impacto del aumento de precio a través de impuestos. Rev. Panam. Salud. Publica 2016, 40, 204–212. [Google Scholar]
  28. Pinto, M.; Bardach, A.; Palacios, A.; Biz, A.; Alcaraz, A.; Rodriguez, B.; Augustovski, F.; Pichon-Riviere, A. Carga do tabagismo no Brasil e benefício potencial do aumento de impostos sobre os cigarros para a economia e para a redução de mortes e adoecimento. Cadernos Saúde Pública 2019, 35, e00129118. [Google Scholar] [CrossRef] [PubMed]
  29. Bardach, A.E.; Caporale, J.E.; Alcaraz, A.; Augustovski, F.; Huayanay-Falconí, L.; Loza-Munarriz, C.; Hernández-Vásquez, A.; Pichon-Riviere, A. Carga de enfermedad por tabaquismo e impacto potencial del incremento de precios de cigarrillos en el Perú. Rev. Peru. Med. Exp. Salud Pública 2016, 33, 651–661. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  30. Pichon-Riviere, A.; Bardach, A.; Augustovski, F.; Alcaraz, A.; Reynales-Shigematsu, L.M.; Pinto, M.T.; Castillo-Riquelme, M.; Torres, E.P.; Osorio, D.I.; Huayanay, L.; et al. Financial impact of smoking on health systems in Latin America: A study of seven countries and extrapolation to the regional level. Rev. Panam. Salud. Publica 2016, 40, 213–221. [Google Scholar] [PubMed]
  31. Bardach, A.; Cañete, F.; Sequera, V.; Palacios, A.; Alcaraz, A.; Rodríguez, B.; Caporale, J.; Augustovski, F.; Pichon-Riviere, A. Burden of disease attributable to tobacco use in Paraguay, and potential health and financial impact of increasing prices through taxing. Rev. Peru. Med. Exp. Salud Pública 2018, 35, 599–609. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  32. Castillo-Riquelme, M.; Bardach, A.; Palacios, A.; Pichón-Riviere, A. Health burden and economic costs of smoking in Chile: The potential impact of increasing cigarettes prices. PLoS ONE 2020, 15, e0237967. [Google Scholar] [CrossRef]
  33. Peña, E.; Osorio, D.; Gamboa, Ó.; Caporale, J.; Augustovski, F.; Alcaraz, A.; Bardach, A.; Mejía, A.; Pichon-Riviere, A. Carga de enfermedad atribuible al uso de tabaco en Colombia y potenciales beneficios sanitarios y económicos del aumento del precio del cigarrillo mediante impuestos. Rev. Colomb. Cancerol. 2019, 23, 135–143. [Google Scholar] [CrossRef] [Green Version]
  34. Bray, F.; Ferlay, J.; Soerjomataram, I.; Siegel, R.L.; Torre, L.A.; Jemal, A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J. Clin. 2018, 68, 394–424. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  35. Centers for Disease Control and Prevention (CDC). Smoking-attributable mortality, years of potential life lost, and productivity losses. United States 2000–2004. MMWR Morb. Mortal. Wkly. Rep. 2008, 57, 1226–1228. [Google Scholar]
  36. Jha, P.; Chaloupka, F.J. The Taxation of Tobacco Products. In Chaloupka PJaF, Editor. Tobacco Control in Developing Countries; Oxford University Press: Oxford, UK, 2000. [Google Scholar]
  37. Centers for Disease Control and Prevention. Response to Increases in Cigarette Prices by Race/Ethnicity, Income, and Age Groups—United States, 1976–1993. MMWR. Morb. Mortal. Wkly. Rep. 1998, 47, 605–609. [Google Scholar]
  38. Forster, M.; Jones, A.M. The role of tobacco taxes in starting and quitting smoking: Duration analysis of British data. J. R. Stat. Soc. Ser. A 2001, 164, 517–547. [Google Scholar] [CrossRef]
  39. Nicolás, A.L. How important are tobacco prices in the propensity to start and quit smoking? An analysis of smoking histories from the Spanish National Health Survey. Health Econ. 2002, 11, 521–535. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  40. Wasserman, J.; Manning, W.G.; Newhouse, J.P.; Winkler, J.D. The effects of excise taxes and regulations on cigarette smoking. J. Health Econ. 1991, 10, 43–64. [Google Scholar] [CrossRef] [Green Version]
  41. Doll, R.; Peto, R.; Boreham, J.; Sutherland, I. Mortality in relation to smoking: 50 years’ observations on male British doctors. BMJ 2004, 328, 1519. [Google Scholar] [CrossRef] [Green Version]
  42. Alcaraz, A.; Caporale, J.; Bardach, A.; Augustovski, F.; Pichon-Riviere, A. [Burden of disease attributable to tobacco use in Argentina and potential impact of price increases through taxes]. Rev. Panam. Salud Pública 2016, 40, 204–212. [Google Scholar]
  43. Pichon-Riviere, A.; Bardach, A.; Augustovski, F.; Alcaraz, A.; Reynales-Shigematsu, L.M.; Pinto, M.T.; Castillo-Riquelme, M.; Torres, E.P.; Osorio, D.I.; Huayanay, L.; et al. Impacto económico del tabaquismo en los sistemas de salud de América Latina: Un estudio en siete países y su extrapolación a nivel regional. Rev. Panam. Salud. Publica 2016, 40, 213–221. [Google Scholar]
  44. Alcaraz, A.; Hernández-Vásquez, A.; Palacios, A.; Rodríguez, B.; Roberti, J.; García-Martí, S.; Ciapponi, A.; Augustovski, F.; Bardach, A.; Pichon-Riviere, A. Health and Economic Impact of Health Warnings and Plain Tobacco Packaging in Seven Latin American Countries: Results of a Simulation Model. Nicotine Tob. Res. 2020, 22, 2032–2040. [Google Scholar] [CrossRef]
  45. Bardach, A.; Rodríguez, M.B.; Ciapponi, A.; Augustovski, F.; Andrea, A.; Soto, N.; Virgilio, S.; Reynales-Shigematsu, L.M.; Roberti, J.; Pichón-Riviere, A. Smoke-Free Air Interventions in Seven Latin American Countries: Health and Financial Impact to Inform Evidence-Based Policy Implementation. Nicotine Tob. Res. 2020, 22, 2149–2157. [Google Scholar] [CrossRef]
  46. Weinstein, M.C.; O’Brien, B.; Hornberger, J.; Jackson, J.; Johannesson, M.; McCabe, C.; Luce, B.R. Principles of good practice for decision analytic modeling in health-care evaluation: Report of the ISPOR task force on good research practices—Modeling studies. Value Health 2003, 6, 9–17. [Google Scholar] [CrossRef] [Green Version]
  47. Palloni, A.; Novak, B.; Pinto-Aguirre, G. The Enduring Effects of Smoking in Latin America. Am. J. Public Health 2015, 105, 1246–1253. [Google Scholar] [CrossRef] [PubMed]
  48. Burki, T.K. Latin America makes progress on tobacco control. Lancet Respir. Med. 2017, 5, 470. [Google Scholar] [CrossRef] [Green Version]
  49. Hiilamo, H.; Glantz, S. FCTC followed by accelerated implementation of tobacco advertising bans. Tob. Control. 2017, 26, 428–433. [Google Scholar] [CrossRef]
  50. Mirza, M. Advertising Restrictions and Market Concentration in the Cigarette Industry: A Cross-Country Analysis. Int. J. Environ. Res. Public Health 2019, 16, 3364. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  51. Levy, D.T.; Yuan, Z.; Luo, Y.; Mays, D. Seven years of progress in tobacco control: An evaluation of the effect of nations meeting the highest level MPOWER measures between 2007 and 2014. Tob. Control. 2018, 27, 50–57. [Google Scholar] [CrossRef]
  52. Wu, Y.; Wang, Z.; Zheng, Y.; Wang, M.; Wang, S.; Wang, J.; Wu, J.; Wu, T.; Chang, C.; Hu, Y. The impact of comprehensive tobacco control policies on cardiovascular diseases in Beijing, China. Addiction 2021. [Google Scholar] [CrossRef]
  53. He, Y.; Shang, C.; Huang, J.; Cheng, K.-W.; Chaloupka, F.J. Global evidence on the effect of point-of-sale display bans on smoking prevalence. Tob. Control. 2018, 27, e98–e104. [Google Scholar] [CrossRef] [PubMed]
  54. Brown, A.; Boudreau, C.; Moodie, C.; Fong, G.T.; Li, G.Y.; McNeill, A.E.; Thompson, M.; Hassan, L.M.; Hyland, A.; Thrasher, J.F.; et al. Support for removal of point-of-purchase tobacco advertising and displays: Findings from the International Tobacco Control (ITC) Canada survey. Tob. Control. 2012, 21, 555–559. [Google Scholar] [CrossRef] [PubMed]
Table 1. Main inputs for the simulation model.
Table 1. Main inputs for the simulation model.
Characteristics ArgentinaBoliviaBrazilChileColombiaMexicoPeru
Population (2015)43,416,75510,724,705207,847,52817,948,14148,228,704127,017,22431,376,670
Smoking prevalence 1
  Male23.420.118.035.220.119.823.5
  Female18.617.711.331.39.96.415.3
Crude mortality rate (Male/Female per 10,000) 2
 Acute myocardial infarction 46.1/33.18.4/5.516.0/11.08.3/4.919.0/13.719.9/13.974.6/57.3
 Other cardiovascular causes 118.7/104.50.9/0.53.8/2.97.4/8.42.3/1.72.2/3.151.8/57.2
 Cerebrovascular disease52.5/43.98.4/8.08.8/7.99.8/9.68.5/9.38.1/8.152.6/50.7
 Pneumonia, influenza104.4/72.417.4/15.99.1/8.54.2/4.03.6/3.14.0/3.1221.0/199.0
 COPD4.3/1.91.1/1.36.6/4.53.7/2.87.9/5.87.5/5.633.2/25.3
 Lung cancer 15.6/4.63.7/3.14.3/2.53.9/2.23.3/1.92.5/1.213.5/10.4
Estimated direct health costs of smoking-related conditions in USD millions
 Acute myocardial infarction324251145006394438354848.62663
 Other cardiovascular causes 243238351881270215343190.41850
 Annual cardiovascular follow-up 12832024409144434,7951240.61171
 Cerebrovascular disease 3429452324304443121744119.15058
 Pneumonia/influenza2172763612353251309.9174
 COPD 4439439694824613334639236.24363
 Lung cancer 517,392886212,27921,72710,49913,792.614,081
 Mouth cancer 512,5236381960215,64475609930.69251
 Esophageal cancer 14,610744412,16118,251882011,585.711,828
 Stomach cancer 514,262726715,07417,816861011,309.911,546
 Pancreatic cancer 511,827602611,61614,77471409378.99575
 Kidney cancer 512,5236381463215,64475609930.610,138
 Tax revenue on smoking 61926.221.595111346.51742237.473.5
 GDP (2015) 6583,168.633,1971,774,725240,215.7292,080.11,144,331.3192,083.7
 GDP per capita (2015) 613,4323095853913,384605690096122
 Price elasticity of demand −0.299−0.85−0.48−0.45−0.780−0.45−0.7
 Total health expenditure (%GDP)4.86.38.37.87.26.35.5
Abbreviations: GDP, gross domestic product; COPD, chronic obstructive pulmonary disease. Key: 1 Population ≥ 35 years expressed in millions; 2 Mortality rate per 10,000 people; 3 Values include first and following years; 4 COPD mild, moderate, and serious included; 5 Treatment costs of following years are included; 6 In millions of US dollars; exchange rate as mean in December 2015 according to central banks in each country.
Table 2. Current advertising ban level and estimated effectiveness for included countries.
Table 2. Current advertising ban level and estimated effectiveness for included countries.
CharacteristicCountry
ArgentinaBoliviaBrazilChile ColombiaMexicoPeru
Current ban levelPartialPartialComprehensivePartialComprehensiveAbsent/LimitedAbsent/Limited
Compliance, %7555908075Non-applicableNon-applicable
Estimated effectiveness—Status Quo, %(absolute prevalence reduction)0.750.558.100.806.75--
Table 3. Estimated cumulative 10-year benefits to ban tobacco advertising.
Table 3. Estimated cumulative 10-year benefits to ban tobacco advertising.
CountryAdvertising BanNumber of Cases Averted Life-Years GainedCosts Averted International Dollars
DeathMIStrokeCOPDCancer
ArgentinaStatus quo (partial ban, 75% compliance)13782183791197963733,127182,534,000
Step 1: Increase compliance to 100% *45972826466021211,04260,903,000
Step 2: Comprehensive ban with 100% compliance *14,88323,571853921,3716876257,7671,966,285,000
BoliviaStatus quo (partial ban, 55% compliance)1418215522441361123,210,000
Step 1: Increase compliance to 100%1166712718433297118,883,000
Step 2: Comprehensive ban with 100% compliance207112062279330059953,185339,496,000
BrazilStatus quo (comprehensive ban, 90% compliance)40,063171,26533,87376,32221,7291375,7695,830,459,000
Step 1: Increase compliance to 100%5569 20,707409592282627166,336689,054,000
ChileStatus quo (partial ban, 80% compliance)729917853165829419,083125,663,000
Step 1: Increase compliance to 100%18423121541874480931,613,000
Step 2: Comprehensive ban with 100% compliance73819286863916,7852978193,2121,272,435,000
ColombiaStatus quo (comprehensive ban, 75% compliance)779724,60510,89812,7293012203,1411,045,827,000
Step 1: Increase compliance to 100%2787 879538964550107772,615377,631,000
MexicoStep 1: Partial ban with 100% compliance170544031072360064146,564313,186,000
Step 2: Comprehensive ban with 100% compliance15,34339,626964532,4035767419,0752,781,093,000
PeruStep 1: Partial ban with 100% compliance588357553114421214,43360,151,000
Step 2: Comprehensive ban with 100% compliance52943210497410,2931911129,896541,355,000
Total Status Quo50,108199,05246,57092,91225,7131,634,7317,207,693,000
Total Comprehensive ban with 100% compliance98,132311,23990,868196,84848,7203,006,63615,659,778,000
Key: International dollar rates 2018: Argentine peso: 14.09, Brazilian real: 2.77, Chilean peso: 2.2, Colombian peso: 412.36, Mexican peso: 1328.53, Peruvian sol: 1.74. Note: * results are incremental, i.e., numbers in step 1 should be added to status quo, and then numbers of step 2 should be added to the previous two.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Bardach, A.; Alcaraz, A.; Roberti, J.; Ciapponi, A.; Augustovski, F.; Pichon-Riviere, A. Optimizing Tobacco Advertising Bans in Seven Latin American Countries: Microsimulation Modeling of Health and Financial Impact to Inform Evidence-Based Policy. Int. J. Environ. Res. Public Health 2021, 18, 5078. https://doi.org/10.3390/ijerph18105078

AMA Style

Bardach A, Alcaraz A, Roberti J, Ciapponi A, Augustovski F, Pichon-Riviere A. Optimizing Tobacco Advertising Bans in Seven Latin American Countries: Microsimulation Modeling of Health and Financial Impact to Inform Evidence-Based Policy. International Journal of Environmental Research and Public Health. 2021; 18(10):5078. https://doi.org/10.3390/ijerph18105078

Chicago/Turabian Style

Bardach, Ariel, Andrea Alcaraz, Javier Roberti, Agustín Ciapponi, Federico Augustovski, and Andrés Pichon-Riviere. 2021. "Optimizing Tobacco Advertising Bans in Seven Latin American Countries: Microsimulation Modeling of Health and Financial Impact to Inform Evidence-Based Policy" International Journal of Environmental Research and Public Health 18, no. 10: 5078. https://doi.org/10.3390/ijerph18105078

APA Style

Bardach, A., Alcaraz, A., Roberti, J., Ciapponi, A., Augustovski, F., & Pichon-Riviere, A. (2021). Optimizing Tobacco Advertising Bans in Seven Latin American Countries: Microsimulation Modeling of Health and Financial Impact to Inform Evidence-Based Policy. International Journal of Environmental Research and Public Health, 18(10), 5078. https://doi.org/10.3390/ijerph18105078

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop