Next Article in Journal
Ending the HIV Epidemic: Identifying Barriers and Facilitators to Implement Molecular HIV Surveillance to Develop Real-Time Cluster Detection and Response Interventions for Local Communities
Previous Article in Journal
Leisure Programmes in Hospitalised People: A Systematic Review
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Patterns of Emergency Room Visits for Respiratory Diseases in New York State in Relation to Air Pollution, Poverty and Smoking

by
Najm Alsadat Madani
1,2,* and
David O. Carpenter
1,2,*
1
Department of Environmental Health Science, School of Public Health, 1 University Place, University at Albany, Rensselaer, NY 12144, USA
2
Institute for Health and the Environment, 5 University Place, University at Albany, Rensselaer, NY 12144, USA
*
Authors to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2023, 20(4), 3267; https://doi.org/10.3390/ijerph20043267
Submission received: 30 December 2022 / Revised: 30 January 2023 / Accepted: 11 February 2023 / Published: 13 February 2023
(This article belongs to the Section Environmental Health)

Abstract

:
We have explored differences in rates of emergency room (ER) visits for respiratory diseases in the counties of New York State (NYS) in relation to levels of air pollution, poverty, and smoking. Air pollution information was derived from the National Emissions Inventory, which provides information on road, non-road, point, and non-point sources of 12 different air pollutants. This information is only available at the county level. Four types of respiratory diseases were considered: asthma, chronic obstructive pulmonary disease (COPD), acute lower respiratory diseases, and acute upper respiratory diseases. Asthma ER visits were elevated in counties with greater total air pollution. All forms of respiratory diseases were elevated in counties with a greater rate of poverty, although this may reflect the fact that poor people often use ERs for routine care. There was a very strong association between rates of smoking for COPD and acute lower respiratory diseases. There was an apparent negative association between smoking and asthma ER visits, but this must reflect the fact that smoking was much more common in upstate counties while asthma was more common in the New York City area, where air pollution is high. Air pollution was much greater in urban than in rural areas. Our evidence indicates that air pollution is the greatest risk factor for asthma attacks, whereas smoking is the greatest risk factor for chronic obstructive pulmonary disease (COPD) and lower respiratory disease. Poor people are more vulnerable to all forms of respiratory diseases.

1. Introduction

Respiratory diseases are a major cause of human morbidity and mortality all over the world. In 2015, lower respiratory diseases were the 3rd highest cause of years of life lost, while COPD was the 10th and asthma the 37th in terms of global burden of disease [1]. There are major public health consequences associated with asthma as a well-known respiratory disease, including high morbidity and mortality in severe cases for both children and adults [2]. Around 334 million people worldwide suffer from asthma [3]. Asthma affects 8.4% of Americans, compared with 4.3% of the worldwide population, and its prevalence is on the rise both domestically and internationally. In minorities and lower socioeconomic groups, the impact is most noticeable [4].
Globally, chronic respiratory diseases remain a major cause of death and disability. Chronic respiratory diseases affected over 544 million people worldwide in 2017, an increase of 39.8% since 1990 [5]. The world over, chronic obstructive pulmonary disease (COPD) is a common respiratory disease associated with high morbidity and mortality rates [6]. A number of factors will likely influence the future development of COPD, including longer life expectancy, occupational and environmental exposures, and the increasing prevalence of asthma. Other factors are expected to continue to influence future trends globally, including air pollution and chronic respiratory infections such as tuberculosis [7].
Studies show that air pollution can cause serious health consequences. In urban areas, especially in developing nations, air pollution is a major concern due to its harmful effects on respiratory health [8]. A study examined the association between respiratory emergency department (ED) visits (2011–2015) and biorefinery exposure in residences. A significant increase in respiratory ED visits was reported among residents living within 10 km of biorefineries [9].The causes of respiratory diseases are multiple. Many such diseases are infectious, while others are due to exposure to air pollution or exposure to toxic chemicals. Rates of smoking, poverty and other personal behavioral patterns within different populations are major factors that influence geographic rate differences. While these disparities are particularly pronounced globally, there are also distinct differences in patterns of respiratory diseases within smaller geographic areas.
We have examined the patterns for emergency room visits for respiratory diseases other than cancer within the counties in New York State (NYS). There are 62 counties in New York, and they vary widely in size and population density. They also differ greatly in rates of air pollution, poverty, and smoking. While it would be far preferable to be able to analyze the patterns of respiratory diseases at a smaller geographic unit, data on total levels of air pollution on such a scale as zip codes or census tracts are not available. We have utilized data from the National Emissions Inventory (NEI), which provide results of road, non-road, point source, and non-point source air pollution. Only point source air levels are available at the zip code level, and point source emissions are less than 5% of total air pollution. Therefore, even though counties are relatively large and there will be diverse sources of air pollution within each county, this is the only geographic level for which complete data is available. We have chosen to study asthma, COPD, acute lower respiratory diseases, and acute upper respiratory infections for which patients commonly go to emergency rooms.

1.1. Asthma

Asthma is a chronic disease caused, aggravated, or exacerbated by various chemical, physical, and biological exposures both indoors and outdoors [10]. The prevalence of asthma is a consequence of several factors, including genetics, allergies, and environmental exposures. Among environmental risk factors are allergies to dust mites and cockroaches, tobacco, air pollution, microbes, and stress [11]. According to the hygiene hypothesis, early life excessive cleanliness can lead to an immune system that overreacts to triggers [12]. There is a large body of evidence showing that various air pollutants are associated with an increased risk of asthma attacks. Even at levels below the current limit values, long-term exposure to air pollution, specifically from fossil fuel combustion, is associated with adult-onset asthma. A study with a mean follow-up period of 16.6 years for 98,326 participants in three cohorts in Denmark and Sweden revealed that 1965 participants had asthma and reported the associations in fully adjusted models with hazard ratios for particulate matter (PM2.5), nitrogen dioxide (NO2), and black carbon (BC) [13]. Another study of 4,154,887 respiratory admissions in Italy between 2006 and 2015 showed that short-term exposure to particulate matter (PM) harms the respiratory system, especially for asthma and COPD [14].

1.2. Chronic Obstructive Pulmonary Disease (COPD)

COPD has been a major public health problem for decades, and there are numerous studies that show air pollution increases the risk of COPD. Over 3 million people die worldwide every year from COPD [15]. COPD is a group of severe lung diseases such as chronic bronchitis and emphysema that cause difficulty breathing by preventing air from flowing through the airways. The most common cause of COPD in developed countries is smoking [16]. It has been suggested that outdoor air pollution exposure is associated with COPD prevalence and incidence. COPD patients experience increased respiratory symptoms when exposed to outdoor air pollutants [17]. Studies have shown that short-term exposure to air pollution increases the risk of emergency room (ER) visits for lower respiratory diseases in environments with relatively low air pollutants. In one study, the association between ER visits and ambient air pollutants for COPD and acute lower respiratory diseases was tested using a case-crossover design. NO2, PM2.5, and sulfur dioxide (SO2) all showed positive results in males, while ozone (O3) and SO2 showed positive results in females [18]. A study using data from the UK Biobank on 303,887 individuals aged 40–69 years found that ambient air pollution results in reduced lung function and an increased prevalence of COPD. Moreover, PM2.5, particulate matter (PM10), and NO2 concentrations are all associated with a higher prevalence of COPD [19].

1.3. Acute Lower Respiratory Diseases

In this paper, we studied acute bronchitis and bronchiolitis as acute lower respiratory diseases. Several studies have found that exposure to gaseous air pollution increases the risk of acute lower respiratory diseases. A study on children who visited ER services for bronchiolitis during the period 1997–2001 in Paris suggests that air pollution may contribute to acute severe bronchiolitis. Air pollutants such as PM10, black smoke, SO2, and NO2 were positively associated with hospitalizations and consultations after adjusting for public holidays, holidays, and meteorological variables [20]. Another study on 29,688 children’s hospital admissions in Hong Kong from 2008 to 2017 revealed that elevated NO2 concentration was associated with higher hospitalization rates for acute bronchiolitis; the risk of hospitalization for acute bronchiolitis increased statistically significantly with cumulative NO2 exposure over the range 66.2–119.6 µg/m3 [21].

1.4. Acute Upper Respiratory Diseases

This paper examines acute upper respiratory diseases including acute laryngopharyngitis, acute upper respiratory infections of other multiple sites, and acute upper respiratory infections of unspecified sites. Epidemiological studies have shown an association between exposure to air pollution and an increased risk of upper respiratory diseases. A study on ER visits data from three hospitals in Lanzhou, China between January 2014 and December 2018 revealed that an increase in ER visits for upper respiratory tract infections was linked to short-term exposure to air pollution, particularly SO2 and carbon monoxide (CO). At lag 07, the percentage increase for each 1mg/m3 increase in CO was 20.8% (95% CI: 11.8, 30.5) [22]. According to another study on pediatric ER visits in 66 hospitals in Shanghai, China from 2016 to 2018, short-term exposure to air pollutants such as ultrafine particles could increase the risk of ER visits for upper respiratory tract infections among children (1.14, 95% CI: 1.02–1.28) [23]. Several studies indicate that ambient air pollution increases the risk of upper respiratory tract infections. According to a study conducted at Wuhan University Hospital in Wuhan, China, gaseous pollutants negatively impact health more than solid pollutants. A significant number of clinic visits for upper respiratory tract infections were caused by SO2 and NO2, to which women seemed more susceptible than men [24]. Based on data on hospital visits from 1 October 2010 to 30 September 2012 from the Beijing Medical Claim data, researchers found that a 10 µg/m3 increase in PM2.5 concentration was associated with 0.84% (95% confidence interval, 0.05–1.64%) increases in hospital admissions for upper respiratory tract infections at lag 0–3 days, but not with emergency rooms or outpatients [25].
While respiratory infections are clearly due to infectious agents, the vulnerability to infection varies with factors such as poverty, immune system status, and access to and quality of health care. In this paper, we seek to clarify the relationships between air pollution, smoking, and poverty with respiratory diseases, to gain a better understanding of the importance of the role these factors have in human health.

2. Methods

2.1. Health Data

Hospitals located in and regulated by NYS are required to report all diagnoses, up to 15 per inpatient or ER visit, to the NYS Department of Health (NYS DOH) upon discharge. This diagnostic report system is known as the Statewide Planning and Research Cooperative System (SPARCS), and was established in 1979. The SPARCS data that we have contains all ER visits to state-regulated ERs (all but Veterans Administration or Indian Health Service clinics) from 2010–2018. With the help of R studio software version 2022.07.1 Build 554, the total number of ER visits and primary diagnoses of respiratory diseases for each year was calculated.
The International Classification of Diseases, Ninth Revision, and Clinical Modification (ICD-9-CM) and The International Classification of Diseases, Tenth Revision, and Clinical Modification (ICD-10-CM,) and the SPARCS Data Dictionary were used to find the appropriate code and category for each respiratory disease. Codes are as follows: asthma (ICD9 = 493; ICD10 = J45), COPD (ICD9 = 490, 491, 492, 494, 496; ICD10 = J40, J41, J42, J43, J44, J47), acute lower respiratory infections (acute bronchitis and bronchiolitis) (ICD9 = 466; ICD10 = J20, J21), and acute upper respiratory infections of multiple or unspecified sites (ICD9 = 465; ICD10 = J06).
The number of ER visits for each disease for each year was calculated according to the ICD code. In the next step, this data was categorized based on each county to have the crude number of ER visits for each disease per year at the county level. The Federal Information Processing Standard (FIPS) code was used to categorize data at the county level. In the last step, we calculated the rate of ER visits for each county base on the crude number of ER visits and the population of that county.

2.2. Population Data

The total population in each county of NYS was downloaded from the United States Census Bureau database for 2010 and the estimated population for 2018 was downloaded from NYS DOH. We interpolate the population numbers between 2010 and 2018 using the Equation (1) below.
Population   Year         = Population   2010 + Year   2010 ( Population   2018           Population   2010 ) / 8         where   year   = 2011 ,   2012 , , 2017
The rates of ER visits for respiratory diseases were calculated by dividing the crude number of ER visits for each respiratory disease in each county by the population of that county. The total rate of ER visits from 2010–2018 was calculated by adding the rate of ER visits for each respiratory disease in the years 2010–2018. The rate of ER visits for each respiratory disease by county is given by the following Equation (2).
The   rate   of   ER   visits   for   each   respiratory   disease   by   county             = the   number   of   ER   visits   for   each   respiratory   disease / county   population

2.3. Exposure Data

The exposure data came from the United States Environmental Protection Agency (US EPA) National Emissions Inventory (NEI), a data set of air emissions and pollutants from point, non-point, road, and non-road sources by county. The FIPS code was used to merge data for each pollutant at the county level from all four sources of pollution. The total pollution was calculated by adding together the data for each county of NYS for 2011, 2014, and 2017 divided by 3 for each chemical—total volatile organic compounds (VOCs), PM2.5, acetaldehyde, formaldehyde, benzene, ethylbenzene, toluene, xylene, SO2, NO2, CO, and lead—from all four different sources of pollution (point, non-point, road, and non-road). Total pollution for lead was only available for point and non-point sources. Land area in square miles for each county was found in the NYS DOH database. Pollution per square mile was calculated by dividing the total pollution for each chemical by the land area in square miles.
Point sources include estimates of emissions from large, fixed, stationary sources such as industrial facilities, electric power plants, airports, as well as smaller industrial, commercial, and non-industrial sources. In addition to point sources, non-point sources include sources that are too small in magnitude to be reported as point sources, such as household heating, commercial combustion, asphalt paving, and commercial and consumer solvent use. Emissions from on-road vehicles running on gasoline, diesel, or other fuels are included in the NEI on-road sources. Light-duty and heavy-duty vehicles operating on roads, highway ramps, and while idling are also considered on-road sources. Fuel sources that use gasoline, diesel, and other fuels are included among NEI non-road sources. There are several types of non-road sources, including construction equipment, lawn, and garden equipment, aircraft ground support equipment, locomotives, and commercial marine vessels.
With the help of ArcGIS software version Desktop 10.8.1, maps were created to show the total air pollution (other than lead) per square mile. The shape files were downloaded from the Topologically Integrated Geographic Encoding and Referencing system (TIGER) that represents the U.S. Census Bureau’s geographic spatial data.

2.4. Poverty Data

Each county’s percentage of poverty was downloaded from the United States Census Bureau database for 2010 to 2018 separately. The data is called the Small Area Income and Poverty Estimates (SAIPE) Program which gives the percentage of people in poverty. The SAIPE program estimates are based on several data sources, such as aggregate tax, Supplemental Nutrition Assistance Program (SNAP) benefits, and poverty incidence data. Some of these data sources are publicly available, while others are not. In SAIPE estimates, poverty incidence is defined similarly to that in the American Community Survey (ACS). Depending on family size and composition, the Census Bureau determines who is poor based on money income thresholds. Poverty is defined when a family’s total income is less than the threshold for a family. In this paper, we calculated the total percentage of poverty by adding the data together for each county in NYS from 2010 to 2018 and dividing by 9, to obtain annual rates.

2.5. Smoking Data

The crude rate of smoking in each county was downloaded from The Behavioral Risk Factor Surveillance System (BRFSS) database for 2016 and 2018. The total rate of smoking was determined by adding the crude rate of smoking in 2016 and 2018 together for each county divided by 2. We defined different statistical models to determine the impact of smoking on the rate of ER visits for respiratory diseases.

2.6. Statistical Analysis

R studio software version 2022.07.1 Build 554 was used to create figures. Correlation analysis was used to assess the relationships between 12 different air pollutants and quantify how closely related any two variables are. A positive correlation occurs when air pollutants change in the same direction. The FIPS code was used to merge health data and pollution data to have the rate of ER visits for respiratory diseases and pollution per square mile at the county level in one dataset.
Regression analysis aims to determine how the rate of ER visits for respiratory diseases is impacted by exposure to 12 air pollutants (total VOCs, PM2.5, acetaldehyde, formaldehyde, benzene, ethylbenzene, toluene, xylene, SO2, NO2, CO, and lead). All these chemicals were aggregated as a separate variable, and regression models were developed to assess the association between air pollution exposure and the rate of ER visits for respiratory diseases.
In order to determine the best fitting model for air pollution adjusted for smoking and poverty, we compared different family regression models. For the purpose of assessing the quality of regression models and choosing the best fitting model, we used a performance package. To measure the performance of a regression model, different statistical regression metrics were calculated, including root mean squared error (RMSE) and residual standard error (RSE), or sigma. The regression models were conducted with p < 0.05 as the threshold for statistical significance. After choosing the best family model to be quasi-Poisson regression, we ran different models in our study to assess the association between air pollution and ER visits for respiratory diseases, and to determine the impact of smoking and poverty on the rate of ER visits for respiratory diseases.
Among unadjusted models, our response variable is the rate of ER visits; the predictor variable in the first group of models is air pollution, in the second group of models it is poverty, and in the third group of models it is smoking.
Among adjusted models, the first group of multiple regression models assesses the association between exposure to air chemicals and the rate of ER visits adjusted for poverty. The second group of multiple regression models examines the association between air pollution and the rate of ER visits adjusted for smoking, and the third group of multiple regression analyses was run to find the relationship between air pollutants and respiratory diseases adjusted for both poverty and smoking.
In the last step, models adjusted for smoking and poverty were compared to unadjusted models using the performance package. Different statistical regression metrics including Nagelkerke R2, RMSE, and RSE (known as sigma), were calculated to compare these different regression models.

3. Results

3.1. Rate of ER Visits for Diseases

Figure 1 shows the ER visits rate for respiratory diseases, and Table S3 presents details. Figure 1a presents the rate of ER visits for asthma. The highest rates of ER visits for asthma were in Bronx, Kings (Brooklyn), and New York (Manhattan) Counties. Asthma ER visits were also elevated in the New York City suburbs, Monroe County which contains the city of Rochester, the Capital Region of Albany and Schenectady Counties, and Montgomery County which contains the city of Amsterdam. The counties with the least ER visits for asthma were Saratoga, Schoharie, and Tioga, which are rural. Figure 1b,c shows the ER visits rate for COPD and acute lower respiratory diseases. In contrast to the situation with asthma, the New York City counties and the surrounding suburban areas all showed low rates of acute lower respiratory diseases and COPD. These diseases were also relatively low in the upstate urban areas but were very elevated in rural counties, especially those in the northern part of the state. Figure 1d presents the ER visits rate for acute upper respiratory diseases. Montgomery, Cortland, and Bronx are the counties with the highest rate of ER visits for upper respiratory diseases, and Greene, Saratoga, and Tioga are counties with the lowest rate.
The National Center for Health Statistics provides information about urban—rural classification of counties for 2013. According to this dataset Allegany, Chenango, Delaware, Essex, Greene, Hamilton, Lewis, Schuyler, Sullivan, and Wyoming are among rural counties. Urban counties include Bronx, Erie, Kings, Monroe, New York, Queens, and Richmond County. Bronx, Kings, New York, Queens and Richmond Counties are the five NYC counties; Erie County contains the city of Buffalo; and Monroe contains the city of Rochester.

3.2. County-Level Air Pollution

The total air pollution per square mile at the county level was obtained from the National Emissions Inventory and is shown in Figure 2a and in Table S4. The counties with the highest rate of air pollution per square mile are New York, Kings, and Queens—all of which are New York City counties. Air pollution is also high in Nassau County on Long Island, as well as the other New York City suburbs. There are clear elevations in air pollution in the major upstate urban areas: Erie and Niagara Counties in the west, where Buffalo and Niagara Falls are located; Monroe County to the east, home to Rochester; further east in Onondaga County with the city of Syracuse; and in the Albany capital district still further east. This is clear evidence that traffic is a major contributor to local air pollution. Counties with the lowest air pollution per square mile are Hamilton, Delaware, and Lewis—all rural counties. The pollution data for four of the counties studied are not available in the NEI-EPA database, distinguishable by the gray color on the map. Also shown in Figure 2b is a table giving the correlations among the 12 air pollutants that are included in the total air pollutants values. All components are very highly correlated, which prevents us from examining the health effects of individual chemicals. The least highly correlated air pollutant is SO2, which comes from coal combustion, but even SO2 has 95% correlation with most of the others.

3.3. Poverty and Smoking

Figure 3a shows the poverty rate in NYS; the top three counties with the highest poverty rate are Bronx, Kings (Brooklyn), and Chautauqua in the low western part of the state. The counties with the lowest rate of poverty are Saratoga, Nassau, Putnam, which are relatively affluent suburban counties. Figure 3b shows the county-wide pattern of smoking. Smoking rates are much higher in rural than urban counties, and this is true even in urban counties with a relatively high rate of poverty, such as the Bronx. Orleans, Oswego, Franklin, and all rural counties in northern New York are counties with the highest rate of smoking. Nassau, Westchester, Rockland, and all New York City suburbs are counties with the lowest rate of smoking in NYS. Table S3 summarizes the data about the rate of poverty and smoking at the county level.

3.4. Regression Models

We moved from one regression family to another family to find the best fitting model for our study. Different statistical regression metrics such as RMSE and RSE (or sigma) were calculated to measure the performance of a regression model. Table 1 summarizes the results of the models’ comparison. By comparing RMSE with the sigma of the different regression family models, we could see that the quasi-Poisson model is the best model to define the association between exposure to air pollution and the rate of ER visits adjusted for smoking and poverty.

3.4.1. Air Pollution Models

Table 2 summarizes the association between exposure to air pollution and the rate of ER visits for respiratory diseases, and Figure 4 represents this association. According to the results of quasi-Poisson regression models, air pollution is positively associated with a statistically significant increased rate of ER visits for asthma (p-value = 7.14 × 10−5) but not for the other respiratory diseases. The rates of ER visits for acute lower respiratory diseases and COPD are negatively associated with air pollution. Acute upper respiratory diseases have a p-value of 0.176, which is not an indicator of significant association. It is apparent that the data from only a few counties strongly influence these results.

3.4.2. Poverty Models

Air pollution is not the only possible risk factor for an increased rate of ER visits for different diseases. Other expected risk factors include poverty and smoking. Figure 5 represents the association between poverty and respiratory diseases. As shown in Table 3, poverty is associated with elevated risk for all these diseases, and the associations are statistically significant. For COPD, the p-value is 0.064. Table S1 presents the model results of the association between exposure to air chemicals and the rate of ER visits adjusted for poverty. The findings of these multiple regression models confirm previous findings that poverty is highly associated with all respiratory diseases.

3.4.3. Smoking Models

The association between smoking and diseases is shown in Figure 6 and Table 4. Smoking is highly associated with COPD (p-value = 0.000487) and acute lower respiratory diseases (p-value = 0.012402), and those associations are statistically highly significant. For asthma, the p-value is 0.012723 and the association is negative. As discussed below, this finding cannot be correct. For acute upper respiratory diseases, the p-value is 0.966042 which is not an indicator of a statistically significant association. Exposure to air chemicals and the rate of emergency room visits adjusted for smoking are presented in Table S2. Across all diseases, smoking is associated with high rates of COPD and acute lower respiratory disease.

3.4.4. Air Pollution Models Adjusted for Poverty and Smoking

Table 5 provides us with the results of the multiple regression models to determine the association between air pollution and respiratory diseases adjusted for both smoking and poverty. Poverty is associated with all respiratory diseases. Poverty associations are statistically highly significant for all respiratory diseases. Smoking is positively associated with COPD and negatively with asthma. Among all diseases adjusted for both smoking and poverty, air pollution is statistically highly linked to asthma and negatively related to COPD.

3.5. Comparison of the Adjusted Models with the Unadjusted Model

Models adjusted for smoking and poverty were compared to unadjusted models to find the best fitting model to define this association. As in previous sections, different statistical regression metrics such as Nagelkerke R2 root mean squared error (RMSE), and residual standard error (RSE—known as sigma)—were applied to measure the performance of the regression models. Table 6 summarizes the results of the models’ comparison. By comparing Nagelkerke R2, RMSE, and sigma of the adjusted model to the unadjusted model, we could see that for all diseases, the adjusted model is a better model to define the association between exposure to air pollution and the rate of ER visits.

4. Discussion

4.1. Respiratory Diseases Are Very Serious Contributors to Global Morbidity and Mortality

In this research, air pollution has been shown to be the greatest risk factor for asthma attacks. A higher level of total air pollution was associated with more asthma ER visits. The results of our research are in line with many studies that confirm the link between exposure to air pollution and the increased rate of asthma. While the results of this current study provide interesting information about the geographic differences in ER visits for respiratory diseases in one US state, it is clear that none of the models is adequate because we have been unable to clearly detect the expected adverse effect of air pollution on any of the diseases studied other than asthma. There are several possible reasons for this. There are likely to be other variables beyond smoking and poverty that influence the results. Age may be one, as residents in many rural communities are older than urban residents and are also much less diverse than in urban areas. We have not controlled for race/ethnicity or sex. County-level data may be just too large a geographic area and the sources of air pollution and the residents too diverse. However, we clearly find the expected associated associations with smoking, especially with COPD, and poverty with all diseases.

4.2. Patterns of Respiratory Diseases in NYS: Role of Urban vs. Rural Areas, and of Population Density

The amount of air pollutants released in NYS is by far the greatest in the New York City counties, which is not surprising because of the greater population density here and the very high level of automobile and truck traffic, especially diesel buses and trucks. There are also clear elevations in the other larger urban areas, from the Buffalo/Niagara Falls area in the west, the upstate urban areas in Rochester, Syracuse, and the city area of Albany, Schenectady, and Troy in the East. In urban and rural settings, pollutants vary in level and type. Several factors can affect air pollution in an urban area, including the layout of the city, the road density, the population density, and the localized population concentration [26]. The quality of the air improves with a shift from urban to rural areas. Compared with large central metropolitan counties, noncore counties had fewer mean number of PM2.5 days, decreasing from a total of 11.21 to 0.95. Compared with large central metropolitan counties, nonmetropolitan counties had lower mean PM concentrations, decreasing from 11.15 µg/m3 to 8.87 µg/m3 [27].
The most striking observation of the levels of air pollutants is the great degree to which their concentrations are correlated. As it was shown in Figure 2b, of the 12 air pollutants studied, most had correlations of 0.99 to 1.00. Only two showed less than a 0.95 correlation, one being lead, which isn’t surprising since there are few sources of lead to the environment now. The other is SO2, a product of burning high sulfur coal or gas. The tight correlation among the other air pollutants is strong evidence that oil and gas combustion is the common source of formation and release of all of the remaining chemicals, including the other five criteria air pollutants and the various VOCs.
There are two major implications of these results. Because these pollutants are all released together, it is very difficult if not impossible to identify which one or ones are responsible for any particular disease. The second is that the major source is vehicle traffic and fossil fuel combustion. This further indicates how important moving away from fossil fuel combustion is, if we are to both reduce climate change and protect the public from the health hazards resulting from air pollution.

4.3. Role of Poverty

Poverty is well known to be one of the strongest risk factors for ill health. This is a consequence of several different factors. Poor people often live in areas with more contamination, have less access to quality health care, and practice less healthy lifestyles. Our data show a large and consistent increase in all respiratory diseases we have monitored in relation to poverty, including both upper and lower respiratory infections. Our results are consistent with other studies and COPD is more common in rural areas. A study in New Hampshire reported that rural counties had higher rates of COPD exacerbation encounters than non-rural counties when poverty and other sociodemographic factors were taken into account [28].
While to a certain degree, respiratory infections may be elevated in people living in poverty because of greater exposure to infectious agents, this elevation may primarily reflect access to care. Poor people often do not have health insurance or access to primary preventive health care. When they are ill, they then go to the local emergency room. We have no way to determine the degree to which this is a factor with the present data, but we consider it likely to be a very important factor.
Another important consideration in this study is that poor people are more likely to use emergency rooms for primary care than is the case for more affluent people. There are many people in the US that do not have health insurance, and for these people the ER is their major source of health care. There are two major federal healthcare programs. Medicaid provides access to health care for poor people and people with some disabilities and Medicare provides access to health care for old people. But others must depend on private insurance, which is expensive. The result is that many working poor and otherwise healthy younger people lack any form of health insurance and use emergency rooms when ill. Thus, the population of individuals found in our SPARCS ER dataset does not reflect a general cross-section of the state population.

4.4. Role of Smoking

We had not expected to see such a clear difference in rates of smoking in relation to urban as compared to rural counties. Smoking rates in all of the primary urban counties are not high. This is especially true for NYC and is particularly striking in that smoking rates in the upstate counties are highly correlated with poverty levels, but that is not true in NYC where there is a lot of poverty. Despite high rates of poverty, the smoking rates in NYC and even in the surrounding counties are very low, and are much lower again than in the upstate urban areas.
Our study shows that COPD is highly associated with smoking, and these findings are consistent with other studies. According to Strzelak et al., tobacco smoke causes inflammation, allergies, and other lung diseases by altering immune responses because exposure to tobacco smoke results in an imbalance of antioxidants and oxidants that leads to oxidative stress, inflammation of the mucosa, as well as increased expression of inflammatory cytokines [29].

4.5. Strengths

This study provides information on outdoor air pollution from all sources, including road, non-road, point source, and non-point source. Another important strength herein is the availability of a very large number of state-wide ER visits, with data from 2010 to 2018. This gives it strong statistical significance. Taking advantage of its geographical targeting, policy and public health leaders in New York can better understand the prevalence and distribution of respiratory diseases.

4.6. Limitations

Other than asthma, we have been unable to document the known effect of outdoor air pollution on respiratory diseases even with the use of multiple statistical models. This must be due to confounders that we have not controlled for or inadequacies of the models we have used. A wide range of respiratory diseases is linked to air pollution, as described in the Introduction. We have no control over indoor air pollution, which is probably more important than outdoor air pollution. We do not have information about people’s occupations and the air pollutants that they might be exposed to in their jobs. The data on ozone is not available in the NEI-EPA data set, and in our analysis of the adverse effect of air pollution, we did not consider ozone. Another limitation is that we find each of the 12 different air pollutants to be so highly correlated at the county level that this prevents us from identifying which specific contaminants are most associated with the different respiratory diseases.

4.7. Suggestions for Future Research

Further study of the effects of air pollutants on respiratory health in other states or countries would be helpful to confirm some of the results of this study. It would be interesting to compare findings in order to have a deeper understanding of the adverse effects of air pollution on human health. The model could be improved by adding other confounders such as age, sex, or race. We recommend this study be conducted in a smaller geographical area, such as zip codes, although there the problem of individual air pollutants not being commonly available in smaller geographic areas would remain. There needs to be further research in order to elucidate the contribution of specific VOCs (especially benzene and formaldehyde) and ozone as causes of respiratory diseases.

5. Conclusions

We have monitored levels of 12 air pollutants and found that total air pollution is much greater in urban than in rural areas, while rates of smoking are much higher in rural areas. We find a very strong association between geographic differences in levels of 12 air pollutants and emergency room visits for asthma. However, because concentrations of 12 different air pollutants are so tightly coupled, it is not possible to determine which substances contribute the most to the risk of asthma attack. There is also a very strong association between rates of smoking and COPD and acute lower respiratory diseases. Differences in levels of poverty show that poverty increases the rates of all respiratory diseases, although this may at least in part reflect the fact that many poor people use emergency rooms for primary care. However, our data and models are unable to detect the known elevated risk of outdoor air pollution on respiratory diseases other than asthma. Unfortunately, our study has not been able to answer the critical question of the relative roles of particulate matter, nitrogen and sulfur gases, and specific VOCs, on respiratory diseases, because levels of all the different air pollutants are very tightly coupled within relatively large geographic areas such as counties.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/ijerph20043267/s1, Table S1: The association between air pollution and respiratory diseases adjusted for poverty, Table S2: The association between air pollution and respiratory diseases adjusted for smoking, Table S3: Rate of ER visits for respiratory disease, rate of smoking, and rate of poverty at the county level, Table S4: Chemical pollution per square mile at the county level.

Author Contributions

Conceptualization, N.A.M. and D.O.C.; methodology, N.A.M. and D.O.C.; software, N.A.M.; validation, N.A.M. and D.O.C.; formal analysis, N.A.M. and D.O.C.; investigation, N.A.M. and D.O.C.; data curation, N.A.M.; writing—original draft preparation, N.A.M.; writing—review and editing, N.A.M. and D.O.C.; visualization, N.A.M. and D.O.C.; supervision, D.O.C.; project administration, N.A.M. and D.O.C. All authors have read and agreed to the published version of the manuscript.

Funding

Supported by the Institute for Health and the Environment and in part by RO1 ES019620.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Exposure data:
Poverty data:
Smoking data:
Health data:
SPARCS dataset is not an open access and needs application to be submitted to New York State, Department of Health.
Population data:

Conflicts of Interest

This publication was produced from raw data purchased from or provided by the New York State Department of Health (NYSDOH). However, the conclusions derived, and views expressed herein, are those of the authors and do not reflect the conclusions or views of NYSDOH. NYSDOH, its employees, officers, and agents make no representation, warranty or guarantee as to the accuracy, completeness, currency or suitability of the information provided here.

References

  1. Wang, H.; Naghavi, M.; Allen, C.; Barber, R.M.; Carter, A.; Casey, D.C.; Charlson, F.J.; Chen, A.Z.; Coates, M.M.; Coggeshall, M.; et al. Global, Regional, and National Life Expectancy, All-Cause Mortality, and Cause-Specific Mortality for 249 Causes of Death, 1980–2015: A Systematic Analysis for the Global Burden of Disease Study 2015. Lancet 2016, 388, 1459–1544. [Google Scholar] [CrossRef] [PubMed]
  2. Dharmage, S.C.; Perret, J.L.; Custovic, A. Epidemiology of Asthma in Children and Adults. Front. Pediatr. 2019, 7, 246. [Google Scholar] [CrossRef] [PubMed]
  3. Enilari, O.; Sinha, S. The Global Impact of Asthma in Adult Populations. Ann. Glob. Health 2019, 85, 2. [Google Scholar] [CrossRef] [PubMed]
  4. Loftus, P.A.; Wise, S.K. Epidemiology of Asthma. Curr. Opin. Otolaryngol. Head. Neck. Surg. 2016, 24, 245–249. [Google Scholar] [CrossRef]
  5. Soriano, J.B.; Kendrick, P.J.; Paulson, K.R.; Gupta, V.; Abrams, E.M.; Adedoyin, R.A.; Adhikari, T.B.; Advani, S.M.; Agrawal, A.; Ahmadian, E.; et al. Prevalence and Attributable Health Burden of Chronic Respiratory Diseases, 1990–2017: A Systematic Analysis for the Global Burden of Disease Study 2017. Lancet Respir. Med. 2020, 8, 585–596. [Google Scholar] [CrossRef]
  6. Barnes, P.J.; Burney, P.G.J.; Silverman, E.K.; Celli, B.R.; Vestbo, J.; Wedzicha, J.A.; Wouters, E.F.M. Chronic Obstructive Pulmonary Disease. Nat. Rev. Dis. Primers 2015, 1, 15076. [Google Scholar] [CrossRef]
  7. Rosenberg, S.R.; Kalhan, R.; Mannino, D.M. Epidemiology of Chronic Obstructive Pulmonary Disease: Prevalence, Morbidity, Mortality, and Risk Factors. Semin. Respir. Crit. Care Med. 2015, 36, 457–469. [Google Scholar] [CrossRef] [PubMed]
  8. Kumar, A.; Dhakhwa, S.; Dikshit, A.K. Comparative Evaluation of Fitness of Interpolation Techniques of ArcGIS Using Leave-One-Out Scheme for Air Quality Mapping. J. Geovis. Spat. Anal. 2022, 6, 9. [Google Scholar] [CrossRef]
  9. Lee, E.K.; Romeiko, X.X.; Zhang, W.; Feingold, B.J.; Khwaja, H.A.; Zhang, X.; Lin, S. Residential Proximity to Biorefinery Sources of Air Pollution and Respiratory Diseases in New York State. Environ. Sci. Technol. 2021, 55, 10035–10045. [Google Scholar] [CrossRef]
  10. Paterson, C.A.; Sharpe, R.A.; Taylor, T.; Morrissey, K. Indoor PM2.5, VOCs and Asthma Outcomes: A Systematic Review in Adults and Their Home Environments. Environ. Res. 2021, 202, 111631. [Google Scholar] [CrossRef]
  11. Toskala, E.; Kennedy, D.W. Asthma Risk Factors. Int. Forum. Allergy Rhinol. 2015, 5, S11. [Google Scholar] [CrossRef] [PubMed]
  12. Ege, M.; Rompa, S. The Hygiene Hypothesis of Allergy and Asthma. Encycl. Immunobiol. 2016, 5, 328–335. [Google Scholar] [CrossRef]
  13. Liu, S.; Jørgensen, J.T.; Ljungman, P.; Pershagen, G.; Bellander, T.; Leander, K.; Magnusson, P.K.E.; Rizzuto, D.; Hvidtfeldt, U.A.; Raaschou-Nielsen, O.; et al. Long-Term Exposure to Low-Level Air Pollution and Incidence of Asthma: The ELAPSE Project. Eur. Respir. J. 2021, 57, 2003099. [Google Scholar] [CrossRef] [PubMed]
  14. Renzi, M.; Scortichini, M.; Forastiere, F.; de’ Donato, F.; Michelozzi, P.; Davoli, M.; Gariazzo, C.; Viegi, G.; Stafoggia, M.; Ancona, C.; et al. A Nationwide Study of Air Pollution from Particulate Matter and Daily Hospitalizations for Respiratory Diseases in Italy. Sci. Total Environ. 2022, 807, 151034. [Google Scholar] [CrossRef] [PubMed]
  15. Rabe, K.F.; Watz, H. Chronic Obstructive Pulmonary Disease. Lancet 2017, 389, 1931–1940. [Google Scholar] [CrossRef]
  16. Lin, J.-L.; Thomas, P.S. Current Perspectives of Oxidative Stress and Its Measurement in Chronic Obstructive Pulmonary Disease. COPD J. Chronic Obstr. Pulm. Dis. 2010, 7, 291–306. [Google Scholar] [CrossRef]
  17. Hansel, N.N.; McCormack, M.C.; Kim, V. The Effects of Air Pollution and Temperature on COPD. J. Chronic Obstr. Pulm. Dis. 2016, 13, 372–379. [Google Scholar] [CrossRef]
  18. Szyszkowicz, M.; Kousha, T.; Castner, J.; Dales, R. Air Pollution and Emergency Department Visits for Respiratory Diseases: A Multi-City Case Crossover Study. Environ. Res. 2018, 163, 263–269. [Google Scholar] [CrossRef]
  19. Doiron, D.; de Hoogh, K.; Probst-Hensch, N.; Fortier, I.; Cai, Y.; de Matteis, S.; Hansell, A.L. Air Pollution, Lung Function and COPD: Results from the Population-Based UK Biobank Study. Eur. Respir. J. 2019, 54, 1802140. [Google Scholar] [CrossRef]
  20. Ségala, C.; Poizeau, D.; Mesbah, M.; Willems, S.; Maidenberg, M. Winter Air Pollution and Infant Bronchiolitis in Paris. Environ. Res. 2008, 106, 96–100. [Google Scholar] [CrossRef]
  21. Leung, S.Y.; Lau, S.Y.F.; Kwok, K.L.; Mohammad, K.N.; Chan, P.K.S.; Chong, K.C. Short-Term Association among Meteorological Variation, Outdoor Air Pollution and Acute Bronchiolitis in Children in a Subtropical Setting. Thorax 2021, 76, 360–369. [Google Scholar] [CrossRef]
  22. Liu, Y.; Wang, Y.; Dong, J.; Wang, J.; Bao, H.; Zhai, G. Association between Air Pollution and Emergency Department Visits for Upper Respiratory Tract Infection InLanzhou, China. Environ. Sci. Pollut. Res. 2022, 29, 28816–28828. [Google Scholar] [CrossRef]
  23. Li, H.; Li, X.; Zheng, H.; Liu, L.; Wu, Y.; Zhou, Y.; Meng, X.; Hong, J.; Cao, L.; Lu, Y.; et al. Ultrafine Particulate Air Pollution and Pediatric Emergency-Department Visits for Main Respiratory Diseases in Shanghai, China. Sci. Total Environ. 2021, 775, 145777. [Google Scholar] [CrossRef]
  24. Zhang, F.; Zhang, H.; Wu, C.; Zhang, M.; Feng, H.; Li, D.; Zhu, W. Acute Effects of Ambient Air Pollution on Clinic Visits of College Students for Upper Respiratory Tract Infection in Wuhan, China. Environ. Sci. Pollut. Res. Int. 2021, 28, 29820–29830. [Google Scholar] [CrossRef] [PubMed]
  25. Zhang, D.; Tian, Y.; Zhang, Y.; Cao, Y.; Wang, Q.; Hu, Y. Fine Particulate Air Pollution and Hospital Utilization for Upper Respiratory Tract Infections in Beijing, China. Int. J. Environ. Res. Public Health 2019, 16, 533. [Google Scholar] [CrossRef] [PubMed]
  26. Clark, L.P.; Millet, D.B.; Marshall, J.D. Air Quality and Urban Form in U.S. Urban Areas: Evidence from Regulatory Monitors. Environ. Sci. Technol. 2011, 45, 7028–7035. [Google Scholar] [CrossRef] [PubMed]
  27. Strosnider, H.; Kennedy, C.; Monti, M.; Yip, F. Rural and Urban Differences in Air Quality, 2008-2012, and Community Drinking Water Quality, 2010-2015—United States. MMWR Surveill. Summ. 2017, 66, 1–10. [Google Scholar] [CrossRef] [PubMed]
  28. Warner, J.S.; Bryan, J.M.; Paulin, L.M. The Effect of Rurality and Poverty on COPD Outcomes in New Hampshire: An Analysis of Statewide Hospital Discharge Data. Chronic. Obstr. Pulm. Dis. 2022, 9, 500–509. [Google Scholar] [CrossRef]
  29. Strzelak, A.; Ratajczak, A.; Adamiec, A.; Feleszko, W. Tobacco Smoke Induces and Alters Immune Responses in the Lung Triggering Inflammation, Allergy, Asthma and Other Lung Diseases: A Mechanistic Review. Int. J. Environ. Res. Public Health 2018, 15, 1033. [Google Scholar] [CrossRef] [Green Version]
Figure 1. Rate of ER visits for respiratory diseases 2010–2018 by county: (a) asthma; (b) COPD; (c) acute lower respiratory diseases; (d) acute upper respiratory diseases. The insert shows the New York City region at higher magnification.
Figure 1. Rate of ER visits for respiratory diseases 2010–2018 by county: (a) asthma; (b) COPD; (c) acute lower respiratory diseases; (d) acute upper respiratory diseases. The insert shows the New York City region at higher magnification.
Ijerph 20 03267 g001
Figure 2. (a) Total air pollution per square mile by county; (b) correlation between 12 air pollutants.
Figure 2. (a) Total air pollution per square mile by county; (b) correlation between 12 air pollutants.
Ijerph 20 03267 g002
Figure 3. Rate of poverty and smoking in New York State by county: (a) poverty; (b) smoking.
Figure 3. Rate of poverty and smoking in New York State by county: (a) poverty; (b) smoking.
Ijerph 20 03267 g003
Figure 4. The association between air pollution and respiratory diseases.
Figure 4. The association between air pollution and respiratory diseases.
Ijerph 20 03267 g004
Figure 5. The association between poverty and respiratory diseases.
Figure 5. The association between poverty and respiratory diseases.
Ijerph 20 03267 g005
Figure 6. The associations between smoking and respiratory diseases.
Figure 6. The associations between smoking and respiratory diseases.
Ijerph 20 03267 g006
Table 1. Comparison of models from the different regression model families.
Table 1. Comparison of models from the different regression model families.
Respiratory DiseaseNameModelRMSESigmaPerformance Score
AsthmaQuasi-Poissonglm0.0130.0680.988
Quasibinomialglm0.0130.0700.964
Gaussianglm0.0170.0170.500
Linear lm0.0170.0170.500
Inverse Gaussianglm0.0152.2210.250
COPDQuasi-Poissonglm0.0150.0770.985
Quasibinomialglm0.0150.0790.985
Gaussianglm0.0150.0160.763
Linear lm0.0150.0160.763
Inverse Gaussianglm0.0162.0940.000
Table 2. The association between air pollution and respiratory diseases.
Table 2. The association between air pollution and respiratory diseases.
Respiratory DiseaseTermEstimateStandard ErrorStatisticp-Value
AsthmaIntercept−3.2870.081−40.6013.15 × 10−43
Air Pollution0.1920.0454.2907.14 × 10−5
COPDIntercept−3.3420.064−51.9174.70 × 10−49
Air Pollution−0.2630.129−2.0470.045411
Acute Lower Respiratory Intercept−3.3910.087−38.9363.03 × 10−42
Air Pollution−0.2850.182−1.5700.12211
Acute Upper RespiratoryIntercept−2.8440.077−36.9375.21 × 10−41
Air Pollution0.0830.0601.3720.175528
Table 3. The association between poverty and respiratory diseases.
Table 3. The association between poverty and respiratory diseases.
Respiratory DiseaseTermEstimateStandard ErrorStatisticp-Value
AsthmaIntercept−3.3310.059−56.2235.89 × 10−51
Poverty0.3590.0458.0536.33 × 10−11
COPDIntercept−3.3270.064−52.2203.42 × 10−49
Poverty0.1140.0601.8880.064274
Acute Lower Respiratory Intercept−3.3810.083−40.6033.14 × 10−43
Poverty0.1650.0762.1720.034117
Acute Upper RespiratoryIntercept−2.8860.064−45.4296.99 × 10−46
Poverty0.2920.0515.6715.18 × 1007
Table 4. The associations between smoking and respiratory diseases.
Table 4. The associations between smoking and respiratory diseases.
Respiratory DiseaseTermEstimateStandard ErrorStatisticp-Value
AsthmaIntercept−3.2820.085−38.8133.60 × 10−42
Smoking−0.2140.083−2.5740.012723
COPDIntercept−3.3460.062−54.3653.74 × 10−50
Smoking0.2290.0623.7040.000487
Acute Lower Respiratory Intercept−3.3890.082−41.1781.46 × 10−43
Smoking0.2140.0832.5840.012402
Acute Upper RespiratoryIntercept−2.8400.078−36.5479.21 × 10−41
Smoking−0.0030.078−0.0430.966042
Table 5. The association between air pollution and respiratory diseases adjusted for smoking and poverty.
Table 5. The association between air pollution and respiratory diseases adjusted for smoking and poverty.
Respiratory DiseaseTermEstimateStandard ErrorStatisticp-Value
AsthmaIntercept−3.3470.048−69.8981.20 × 10−54
Air Pollution0.0770.0342.2520.028437
Smoking−0.1740.052−3.3610.00143
Poverty0.3160.0359.0072.42 × 10−12
COPDIntercept−3.3640.060−55.9541.69 × 10−49
Air Pollution−0.2210.131−1.6900.096765
Smoking0.1480.0692.1490.036128
Poverty0.1380.0672.0720.043017
Acute Lower RespiratoryIntercept−3.4290.083−41.3961.37 × 10−42
Air Pollution−0.3470.214−1.6240.110298
Smoking0.0930.0931.0020.321012
Poverty0.2290.0882.5930.012203
Acute Upper RespiratoryIntercept−2.8850.065−44.4743.17 × 10−44
Air Pollution0.0050.0650.0750.940876
Smoking−0.0220.071−0.3050.761745
Poverty0.2890.0555.2582.56 × 10−6
Table 6. Comparison of the adjusted models with the unadjusted model.
Table 6. Comparison of the adjusted models with the unadjusted model.
Respiratory DiseaseModelR2 NagelkerkeRMSESigma
AsthmaAdjusted0.6710.0130.068
Unadjusted0.2390.0250.102
COPDAdjusted0.3130.0150.077
Unadjusted0.1190.0170.086
Acute Lower Respiratory Adjusted0.2440.0200.102
Unadjusted0.0790.0220.111
Acute Upper RespiratoryAdjusted0.3510.0280.111
Unadjusted0.0320.0340.133
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Madani, N.A.; Carpenter, D.O. Patterns of Emergency Room Visits for Respiratory Diseases in New York State in Relation to Air Pollution, Poverty and Smoking. Int. J. Environ. Res. Public Health 2023, 20, 3267. https://doi.org/10.3390/ijerph20043267

AMA Style

Madani NA, Carpenter DO. Patterns of Emergency Room Visits for Respiratory Diseases in New York State in Relation to Air Pollution, Poverty and Smoking. International Journal of Environmental Research and Public Health. 2023; 20(4):3267. https://doi.org/10.3390/ijerph20043267

Chicago/Turabian Style

Madani, Najm Alsadat, and David O. Carpenter. 2023. "Patterns of Emergency Room Visits for Respiratory Diseases in New York State in Relation to Air Pollution, Poverty and Smoking" International Journal of Environmental Research and Public Health 20, no. 4: 3267. https://doi.org/10.3390/ijerph20043267

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop