Next Article in Journal
Health Literacy and Preventive Behaviors Modify the Association between Pre-Existing Health Conditions and Suspected COVID-19 Symptoms: A Multi-Institutional Survey
Previous Article in Journal
Prevalence of Type 2 Diabetes and Its Association with Added Sugar Intake in Citizens and Refugees Aged 40 or Older in the Gaza Strip, Palestine
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
IJERPH
Volume 17
Issue 22
10.3390/ijerph17228597
ijerph-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Selecting Indicators to Monitor and Assess Environmental Health in a Portuguese Urban Setting: A Participatory Approach

by
Marta Salgado
1,*,
Ana C. L. Vieira
2,
Anália Torres
3 and
Mónica D. Oliveira
2
1
Institute of Environmental Health (ISAMB), Faculty of Medicine, University of Lisbon, 1649-028 Lisboa, Portugal
2
Center for Management Studies of Instituto Superior Técnico (CEG-IST), Instituto Superior Técnico, Universidade de Lisboa, 1049-001 Lisboa, Portugal
3
Valorsul, Estação Mercadorias Bobadela, Plataforma Ribeirinha CP Lisboa, 2696-801 São João da Talha, Portugal
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2020, 17(22), 8597; https://doi.org/10.3390/ijerph17228597
Submission received: 13 October 2020 / Revised: 12 November 2020 / Accepted: 17 November 2020 / Published: 19 November 2020
(This article belongs to the Section Environmental Health)
Browse Figure
Versions Notes

Abstract

:
Environmental health (EH) is influenced by complex interactions between health and the built and natural environments, there being little research on its specificities in urban settings. The use of suitable indicators to monitor and assess EH is fundamental in informing evidence-based interventions at the local level. A participatory approach to selecting indicators to inform the monitoring and assessment of EH in Lisbon is herein considered. Evidence derived from a systematic review of literature and data from Lisbon and Portuguese databases were analyzed by 12 Portuguese experts in individual semi-structured interviews. The interviews aimed at identifying relevant indicators and important emerging issues in the Lisbon urban setting. The outputs from the interviews were validated by a two-round Web-Delphi process in which panelists (22 experts) from different areas of expertise expressed their views regarding the relevance of the indicators for the analysis of EH in urban settings. Seventeen indicators were validated in the Web-Delphi process. High participation achieved along this process supports the view that this participatory approach was useful for validation. Results from the adopted participatory approach point out gaps in the collection of noise and mobility indicators data and raise emerging issues on housing indicators that require further research. The results also suggest the need for local action to improve indicators and tools in order to help the monitorization of EH in urban contexts. The adopted participatory approach can be replicated for other Portuguese and European urban settings.

1. Introduction

The United Nations 2030 Agenda for sustainable development goals (SDGs) recall the importance of placing at the top of the political agenda the pursuit of good health and wellbeing (SDG-3) and the promotion of sustainable cities and communities (SDG-11) [1]. Pursuing such goals requires the use of health information systems that enable the monitoring and assessing of health in multiple dimensions, with such systems entailing specificities across geographical settings [2,3]. Within such settings, it is recognized that health outcomes have a strong relationship with the built and natural environment, with this link becoming more evident in urban areas [4,5]. Although the disparities between urban–rural settings have been narrowing, residents in urban areas are more affected by physical factors [6,7]. As cities are home to more than 60% of the world’s population [5,8], the growing understanding of these relationships in the way cities are planned and built and how this impacts health has evolved over the years [9]. Nowadays, sustainable cities have come to the forefront of environmental health (EH) priorities [10].
The environmental health field comprises the monitorization and surveillance of built and natural determinants like air quality, noise, green spaces area, and walkability, and is concerned with the prevention of disease and with the creating of health-supportive environments [11,12]. One of the most demanding challenges for researchers and policymakers within the field is to isolate built and natural environment dimensions and their interactions and to be able to define an approach to managing these interactions, in order to create value. Associated with this challenge, comes the need to do so while collecting population’ intelligence through collaborative practices and to support innovation in EH interventions [13,14]. EH indicators are particularly important in monitoring and assessing the environment status and its trends in urban settings [15], in order to improve interventions, allocate resources, and investigate the effectiveness of health policies [16]. It is known that the collection of monitoring and surveillance data is a key task for many governmental organizations and industries charged with ensuring the wellbeing of the population [17]. Environmental interactions are difficult to quantify and evaluate, and to gather evidence-based information for decision-making is far from straightforward [10,18]. As it is essential to build evidence-based tools for the development of EH interventions, there must be a rationale on how to select the dimensions and indicators to assess EH [19]. Nevertheless, using this type of evidence to inform the design and evaluation of policies in urban settings is an under-researched area [20]. Mason and Lindberg [21] and Pine and Glonti [22] reported that researchers and policymakers value data and expertise in the local context. The literature also recalls that built and natural environment indicators must have a proven effect on health and that their selection should be based on a sound approach, so that they can become a resource in identifying potential risks to health [23].
Methods for selecting indicators vary from statistical and causal models [15] to systematic reviews of the literature [24,25] and participatory processes [26,27,28,29]. The use of participatory processes in EH research is especially important because of the complexity of the EH concept and of the interactions between EH determinants and health outcomes, as well as because of the importance of engaging experts with different knowledge in expressing their views and discussing local EH problems [30,31]. The perception of the complexity and specificities of urban settings also makes it crucial to consider a variety of experts’ views to increase the probability that the selected indicators will be considered credible, scientific, and relevant to EH interventions [32]. Multiple qualitative methods like interviews, surveys, or Delphi have been used for understanding complex issues related to EH analysis. Interviewing is a way of collecting data and can be used in various situations to cover a wide range of health topics [33]. Yet Brewerton [34] argues that interviews can have low reliability due to their openness to different types of bias. To overcome issues of the validity and reliability of the findings of an interview, other methods must be applied to serve as a guarantee of the participants’ performance. Another qualitative method widely used in health research is the Delphi process which captures experts’ opinion and can be used to promote consensus [35]. The Web-Delphi is being increasingly explored due to the use of technological platforms having the advantage of not requiring face to face contact, removing geographical barriers, and allowing the involvement of a large number of experts [36]. Delphi processes have been used to identify criteria to characterize and differentiate models of palliative care in healthcare services [27], to define core areas of EH [30], and to compare scenarios in order to improve urban sustainability development [37,38]. In this article, we propose the use of a Web-Delphi process as a tool to validate the information collected for a set of built and natural indicators.
Lisbon has been recently selected as European Green Capital 2020 with an ambitious agenda for the next decade, under the motto “Choose to evolve: measures for 2030” [39]. This agenda is fully aligned with the SDGs-2030 Agenda and implementing its goals requires setting out the data to be collected in the urban context, to monitor EH and to assess the extent to which goals are achieved. Accordingly, there is an opportunity and need to develop tools (for instance, based on dashboards and/or on composite indicators) that enable the monitorization and improvement of EH by local authorities and industries.
Thus, with a view to collecting key information for improving EH, this study explores an integrated participatory process to involve EH stakeholders and experts in the selection of the relevant dimensions and indicators from built and natural environment determinants, in order to monitor and assess EH in Lisbon, Portugal.

2. Materials and Methods

For the purpose of selecting relevant dimensions and indicators relevant for the evaluation of EH in Lisbon—informed by evidence, by data, and by the views of multiple EH stakeholders and experts—we designed a multi-methodology with three stages (see Figure 1 for an overview). The first stage comprises a systematic review of the literature that focuses on environmental determinants in urban settings and which is described in detail in Salgado et al. [20], along with a search and collection of data from Portuguese and Lisbon databases relevant to assess EH in urban settings. In the second stage, the outputs of the literature review were used as an input to semi-structured interviews. In the interviews, a restricted panel of Portuguese experts from relevant EH backgrounds made a detailed analysis and evaluation of the collected evidence and data and discussed which dimensions and indicators should be used to monitor and assess EH in Lisbon. Interviews were selected as they are powerful for eliciting narrative and complex data and allow researchers to investigate experts’ views in greater depth [33]. The third stage departed from the knowledge collected during the semi-structured interviews, and consisted of a two-round Web-Delphi process to promote a commitment among a larger panel of national experts towards consensus regarding the inclusion of the final set of dimensions and indicators in a tool to monitor and assess EH in Lisbon, as well as for collecting insights and issues related to these indicators. The Web-Delphi was used as a validation stage with the aim of providing complementary completeness, plausibility, credibility, and confirmation of the knowledge acquired in the semi-structured interviews [40], while enabling experts to interact in an anonymous format. Stages 1, 2 and 3 are detailed below.

2.1. Evidence and Data Collection: Systematic Review of Literature and Search on Portuguese Databases

The systematic review of literature reported in Salgado et al. [20] aimed at identifying the dimensions and indicators within the socioeconomic, built environment, natural environment, healthcare, and behavior determinants that impact on health in urban settings. For that purpose, the review used a search strategy based on the keywords “population health”, “city OR cities OR town OR “metropolitan area” OR “urban environment”, “indicators OR determinants”. The period covered in the search was from 2008 to 2018 and searches were performed in PubMed, Web of Science, Scopus and SciELO Portugal databases. Through screening of the literature, 94 studies were selected and included for analysis. The review detected case-control, cohorts, cross-sectional and ecological studies quantifying causal inferences that provided direct evidence to link an environmental risk factor, measured by the indicators, to a health outcome [41]. The results from the systematic review that focused on the physical determinants, i.e., built and natural environment, were summarized and used to build an initial list of potential dimensions and indicators relevant to evaluate EH in urban settings within the scope of this study.
A search in Portuguese national and local open-access databases, the Instituto Nacional de Estatística (National Institute of Statistics, INE, Portugal) [42], Lisbon city council [15], and PORDATA [43], was performed to collect data and analyze available information for each indicator proposed in the initial list, and to inquire about the spatio-temporal availability of these indicators for the city of Lisbon. Accordingly, a list of thirty-four indicators grouped into nine dimensions was gathered. The indicators and dimensions were organized according to the categories of built or natural environment. Along with the list of indicators, a description of each indicator was defined, together with information concerning geographical availability and measurement periodicity.

2.2. Evidence and Data Analysis and Assessment: Semi-Structured Interviews

Given the complexity of the evidence and the nature of data collected, face-to-face semi-structured interviews were conducted for the indicators and for data analysis and assessment. This method was selected for two reasons. First, the associated bidirectional communication between interviewer and interviewee helps in obtaining more detailed and in-depth information. Tacit in this reasoning is that interviewees cannot share a common vocabulary and some aspects might have to be clarified considering the different areas of expertise, improving the validity of the results [44,45]. Second, it allows the asking of other questions outside the semi-structured questionnaire to clarify data assessment and to ask follow-up questions. This flexibility in sharing knowledge was deemed appropriate for the context and helped generating unexpected insights.
Twelve semi-structured interviews were conducted with Portuguese experts working in Lisbon, chosen to represent equitably the variety of perspectives concerning EH: experts in built environment, natural environment, EH, and from national regulation institutions (see Table 1). Involved interviewees had to fulfill at least one of the following criteria: (1) more than five years of experience in EH-related research projects for the Lisbon area; (2) having publications related to air quality, urban sustainability, renewable energy consumption and production, influence of green areas on health, water quality; or (3) being superior technicians from national regulation institutions. The experts received a formal invitation via email along with a description of the study and the process of informed consent. The experts who consented to be interviewed received the questionnaire template in advance (see Supplementary Material) with the purpose of exploring the questions more systematically and comprehensively, as well as to keep the interview focused on the desired line of action. The questionnaire consisted of three parts. The first part collected interviewee’s analysis of the results on EH dimensions from the literature. The second part included questions related to the built and natural environment determinant indicators collected in national and local databases. The third part asked for opinions regarding missing information on indicators that should be included in order to evaluate EH.
In the first part, the interviewees rated their level of agreement with the inclusion of each dimension used to evaluate EH in Lisbon making use of a five-level Likert scale (SA: Strongly disagree, D: Disagree, N: Neither agree nor disagree, A: Agree, SA: Strongly agree), with an extra option of DNK/DNWA: do not know/do not want to answer. In the second part, the same rationale was followed for rating the level of agreement, with the inclusion of the indicators previously collected from the national databases. The interviewees were also asked to analyze the categorization of indicators within the dimensions, and to indicate which indicators could be seen as redundant, i.e., which could be considered as capturing the same phenomena. In the third part, the participants were invited to state if any indicators considered relevant to monitor and assess EH were missing, regardless of the availability of data for Lisbon, and if there was a possible strategy to measure those indicators. In the end, for each indicator for which data was only available at the national level, interviewees were asked whether the national data could be used as a proxy for local data. The answers were written by the interviewer on interview sheets. When processing the information gathered in all the interviews, the dimensions and indicators were considered relevant if more than 50% of the interviewees stated “strongly agree” or “agree” and, at the same time, did not have more than 25% of “strongly disagree” or “disagree” responses. Conversely, indicators with more than 75% of either “strongly disagree” or “disagree” were rejected. If any of the interviewees choose “DNK/DNWA” for any question, the answer was not considered for that dimension or indicator.

2.3. Validation: Web-Delphi Process

The use of the Web-Delphi method had a two-fold objective. Firstly, it aimed at gathering opinions of a wide range of experts with diverse backgrounds, and coming from various geographical locations, regarding the relevance of the indicators for assessing EH in urban settings, in order to validate the results from the interviews. Secondly, it exposed experts to alternative viewpoints from other experts, while allowing them to interact and reconsider their responses. The two-round Web-Delphi was conducted between February and March 2020.

2.3.1. Delphi Panel

Evidence suggests that knowledge generation processes can be improved by ensuring diversity among the recruited participants [46]. For the purpose of this study, the Delphi panel members were defined as a group of members knowledgeable of and with work experience in the areas being studied and able to participate in the process [47]. The panel was chosen based on a “snowball” effect [48]. The interviewees were not included in the panel but recommended at least two other experts to participate in the Delphi process. The panel included a variety of expertise in air quality, epidemiology, environmental health, national institutions of health regulation, noise, public health, and urban sustainability. Based on these criteria, 55 Portuguese experts were invited to participate in the Web-Delphi. There is no general or specific rule for an optimal number of panelists, with this number varying in many Delphi studies [49,50,51]. In this study, the focus was on involving participants with heterogenous expertise and covering all relevant EH perspectives, as well as including experts from national and local institutions [52,53]. The main areas of expertise of Delphi participants are presented in Table 1.

2.3.2. Web-Delphi Design

The information resulting from the interviews was taken as the input into the Web-Delphi process. If an indicator had been agreed for inclusion by a majority (more than 50%) of interviewees, that indicator was considered in the Web-Delphi rounds. The questionnaires were implemented via a web platform (available at www.welphi.com) [54]. As evidence suggests that a high number of rounds can result in a lower response rate [47,55], and Dalkey and Brown [56] notice that the best accuracy in response rates is obtained with two-round Delphi iterations with the drop-out increasing after two iterations, a two-round process design was adopted.
The process started with the online invitation of each panelist, containing instructions to access the Welphi platform, the informed consent, information regarding the study, and the first round questionnaire. The participants were asked to answer within ten workdays of receiving the email. Two reminders were sent to those participants who did not complete the survey. The Welphi platform was used to communicate with the panelists. Participants who did not complete the round during the deadline were not invited to the second round.
In the first round, panel members were asked to indicate their level of agreement or disagreement with each indicator on a five-level Likert scale (SD, D, N, A, SA), with the option of answering DNK/DNWA. Specific rules were applied to deal with differences in opinion and to measure the level of agreement. The rule adopted for the inclusion of an indicator, i.e., to have more than 50% of the panel members stating “strongly agree” and “agree” (agreement), and at the same time having less than 33% of answers rating as “strongly disagree” and “disagree” (SA+A > 50% and SD+D < 33.3%), were approved by an absolute majority. Additionally, an indicator receiving more than 75% of “SA” and “A” responses was then approved by qualified majority. Conversely, indicators with more than 50% of either “strongly disagree” or “disagree” responses were immediately rejected by absolute majority. In the second round, the rules for approval and rejection, as applied in the first round, were kept. The Web-Delphi questionnaire included the list of 17 indicators which resulted from the semi-structured interviews, along with information regarding the respective dimensions and spatio-temporal availability for Lisbon. The participants also had the opportunity to make comments and suggestions about each indicator in a comment section provided in the Welphi platform. In the second round, panel members had the opportunity to see the results from the first round (both participants’ choices and comments) and were invited to keep or revise their answers if they considered this appropriate.

2.3.3. Statistical Analysis

The Web-Delphi process aimed at validating the results from the semi-structured interviews by promoting a higher level of agreement among the experts towards a consensus, there being nevertheless an expectation of divergence of opinion regarding either the inclusion or exclusion of some of the indicators. To understand such differences in opinion, Cronbach’s α was calculated to understand the extent to which there was alignment among all panelists. Cronbach’s α has been used previously as an association index to measure the level of agreement among panelists [57] and was used during each round of the Web-Delphi to estimate the association of the ratings in the unidimensional Likert scale [58]. The meaning of the value of Cronbach’s α depends on the context, but when Cronbach’s α is close to 1.0, it means that the participants’ choices are strongly associated, while values close to 0 suggest that the participants’ choices are highly unrelated to one another. A cutoff of the Cronbach’s α of 0.70 was adopted, taking the view that values above 0.7 are aligned with the inclusion of the indicator within an environmental health monitorization tool [16].

3. Results

3.1. Evidence and Data Collection: Systematic Review of Literature and Search of Portuguese Databases

Based upon the review of the literature, a summary of the 34 indicators related to built and natural environment determinants and with the potential to impact health in urban settings was selected (see Table 2). The selection was based on the causal inferences reported mainly in the cross-sectional and cohort studies included in the review of the literature, to ensure the validity of the evidence withdrawn. To complement this evidence, an additional search in national and local databases was performed to acquire spatio-temporal availability information (see Supplementary Material) for indicators measured in the Lisbon region and in Portugal. A description with details regarding the periodicity of measurement, geographical level and units of measure was made for each indicator (see Supplementary Material). The indicators for which information is currently available for the national level were identified, as well as the indicators lacking data. The evidence and data collection process identified five dimensions (green spaces, housing, mobility, safety, and sanitation) 18 indicators from the built environment; and four dimensions (air quality, noise, soil, and water quality) and 16 indicators for the natural environment. The indicators were grouped into these dimensions and an initial list of nine potential dimensions and 34 indicators was proposed for analysis in the next stage (see Supplementary Material and Table 2).

3.2. Evidence and Data Analysis and Assessment: Semi-Structured Interviews

The 12 semi-structured interviews were performed with experts whose main expertise related to the built environment, natural environment, EH, and national health regulation institutions, as shown in Table 1.
Each interview took a median of 55 min (range 45–70 min). The first part of the interview gathered views from participants regarding EH dimensions to be used to monitor and assess EH in Lisbon. From the set of nine dimensions, six dimensions reached a high level of agreement—more than a 50% agreement rate—about their relevance in evaluating EH in Lisbon. Over half of the interviewees disagreed with the inclusion of the safety and sanitation dimensions. Soil dimension was indicated to be important in evaluating EH in rural settings, but for cities, it was not seen as relevant for inclusion.
In the second part, the interviewees stated their agreement regarding the indicators collected in the national and local databases search. An early finding at this stage was that experts were very concerned about the availability of the data when stating their agreement regarding the inclusion and/or exclusion of the indicators, even if they had been instructed to consider indicators for which data might not exist. For some indicators, the interviewees chose not to state their opinion (DNK/DNWA). From the initial list of 34 indicators, a total of 16 (47%) were considered as relevant for monitoring and assessing EH in Lisbon by the majority of the interviewees. The indicator “Public water supply” in the “Water quality” dimension was considered as important only for cities affected by drought episodes, which is not the case for Lisbon. The “Build-up area” indicator was considered redundant in the light of other indicators included in the “Green spaces” and “Mobility” dimensions. The “Protected area surface”, “Protected area proportion” and “Number of gardens and parks” indicators were also considered redundant. Indicators measuring the area of leisure parks and community gardens achieved a higher agreement than the indicators considered redundant (see Table 2).
The last part of the interview gathered opinions regarding missing information that should be included to monitor and assess EH, with six interviewees proposing an additional indicator— “energy poverty vulnerability index”—for the housing dimension.
Based on the evidence and information collected during the semi-structured interviews, a final set of indicators and dimensions was organized. Hence the built environment determinant was divided into three dimensions—green spaces, housing, and mobility- and the natural environment determinant into another three dimensions—air quality, noise, and water quality. These six dimensions encompassed a total of 17 indicators, six in the built environment determinants, and 11 in the natural environment determinants. This set of dimensions and indicators was the input for the Web-Delphi process in the validation stage.

3.3. Validation: Web-Delphi Process

3.3.1. Panel Participation

From the 55 experts invited for the Web-Delphi process, 29 accepted the invitation and completed the first round of the Web-Delphi questionnaire (53% response rate). In the second round, 22 experts completed the questionnaire (76% response rate). A high response rate was achieved from the built environment experts where all participants completed both rounds. The group of respondents in the final round had a balanced number of experts from all areas of expertise (see Table 1).

3.3.2. Indicators

A list of 17 indicators, grouped in six dimensions, resulted from the interviews and was validated as relevant in monitoring and assessing EH in Lisbon city, as captured by the high agreement rate among experts in the Web-Delphi. In summary, ten indicators were validated by absolute majority (SA > 50% and SD+D < 33.3%): nine from the natural environment determinant and one from the green spaces dimension (“Area of leisure parks and gardens”). Seven indicators were validated by qualified majority (SA+A > 50%): five from the built environment and two from the natural environment. The five indicators from the built environment included “Area of community gardens” from the green spaces dimension, and four indicators in the housing and mobility dimensions. The two indicators included from the natural environment were the SO2 and Lden indicators. Results show that the larger group validated the work taken from the evidence and data collection and from the interviews. The “Energy poverty vulnerability index” and the “Água segura” indicators reached the highest level of agreement (Table 3).

3.3.3. Group Agreement Analysis

Analyzing the Cronbach’s α index results and the distribution of the group’s responses along the two rounds of the Web-Delphi process, it is possible to understand if the heterogeneity of Delphi panel members (in terms of expertise) translated into variability in Delphi answers in each indicator. Cronbach’s α for the first round of the Web-Delphi process was 0.80, indicating substantial agreement among the panelists’ answers from the beginning of the process. 22 experts completed the second round, having the opportunity to see the results from the first round and to revise the answers, with Cronbach’s α increasing from 0.80 to 0.84. An alpha higher than the established reliability cutoff of 0.70 indicates that, despite the different areas of knowledge from experts, they had similar views regarding the relevance of the indicators to monitor and assess EH in Lisbon.

4. Discussion

Our results contribute to deepening the knowledge of which indicators are relevant to monitor and assess EH in urban settings, and answered the need, identified in other studies, for the development of new approaches to selecting indicators to monitor and assess EH and to improving collection of EH-related data [59]. This is, to the best of our knowledge, one of the first studies attempting to combine empirical evidence, collected data, and experts’ views in the selection of dimensions and indicators to monitor and assess EH in a city in a holistic way, taking the view that the selection of indicators is at the cornerstone of the monitorization and assessment of EH [60].

4.1. Methods

The proposed multi-methodology was designed to help in structuring a transparent and tendentially more informed set of indicators useful in analyzing EH in the Lisbon urban area. The combination of interviews and Delphi processes was successful in promoting agreement on a set of indicators.
In this study, the level of agreement was used instead of consensus because it is easily interpretable, consensus being a specific case of agreement (perfect agreement) [61]. To our knowledge there are few studies in which the Delphi method using a web-platform was employed as a validation tool. The high level of agreement achieved in the two rounds proved sufficient to validate the inclusion of the indicators obtained from a smaller group of experts in a set of semi-structured interviews. The opportunity given in the Web-Delphi process to the experts to change their opinion, as a result of considering the views of their peers, was revealed to be a strength of the process. Cronbach’s α association index higher than 0.80 indicates that, besides having different areas of expertise, the experts appeared to have a shared knowledge of which indicators should be included to assess EH in Lisbon, and they were aware of the need to monitor EH with suitable indicators in the urban setting context.

4.2. EH in Urban Settings

Previous research has shown that EH has distinct patterns across urban and rural settings [62], with the results of our study providing evidence aligned with the literature. In rural settings, water and sanitation dimensions have been shown to have more impact on health [63], and Badland et al. [25] showed that air pollution, housing, public open spaces, and transport are the most important aspects in assessing health in urban settings. The set of green spaces, housing, mobility, air and water quality, and noise indicators validated in our study corroborate these findings, as well as revealing information gaps in data collected for the indicators used to analyze EH within the built and natural environment dimensions.
Although there is a lack of data for some built environment indicators in Lisbon, a high level of agreement was achieved on which of these indicators were relevant in monitoring EH. Out of the six indicators validated in the Web-Delphi for built environment dimensions, the “Area of community gardens” indicator was the one with the lowest agreement. Home and Vieli [64] state that urban agriculture performed in community gardens can meet some of the basic nutrition needs of the urban population and potentially provide economic, social, and mental and wellbeing benefits. During the interviews, the advantage of including this indicator was clear (83% agreement rate) while in the Web-Delphi this indicator achieved a rate of agreement of 55%. The difference in the results could be explained by the controversy around the importance of this type of green space when looked at alongside the availability of parks and gardens. The agreement attained towards the inclusion of the “Road network” and “Number of road vehicles” indicators can be further associated with the recognition of their impact on air and noise pollution [37,65]. In fact, to improve EH in the Lisbon context, some interventions have tried to control the number of cars in the city and to increase the number of cycling roads [37,39,65]. The surprising result from our participatory approach was the inclusion of the energy poverty vulnerability indicator in the housing dimension. Despite the need for data collection, this result corroborates that energy poverty vulnerability is a growing societal challenge that puts the welfare of the population at risk. The access to suitable housing conditions is one of the SDGs targets, and our results are in agreement with the idea that in Portugal there is growing awareness and research on the housing conditions dimension [66].
As opposed to the lack of information regarding built environment indicators, a larger number of natural environment indicators is regularly measured in Portugal. Air and water quality indicators are publicly available, updated, and achieved a high rate of agreement in the Web-Delphi. From the air quality indicators, the SO2 indicator was the only one included by qualified majority (an absolute majority was observed for the other air quality indicators). In fact, SO2 is one of the most important pollutants in the atmosphere that contributes to the degradation of buildings, and the main source is industry; exposure to high SO2 concentrations can lead to cardio-respiratory mortality and morbidity [67]. However, one should note that Lisbon is not an industrialized city, which could possibly have influenced the results achieved in the Web-Delphi. Another dimension validated in our process was noise. The causality of the effect of noise on health has not been sufficiently elucidated, and this dimension is often missing from frameworks to assess health [68]. However, the results of our study show that its potential impact on health should not be neglected, and measures of sound level should be up to date and tools designed to monitor EH in urban contexts. Overall, our results regarding the natural environment are aligned with the evidence showing air quality and access to good quality water as major concerns to urban residents [69] and point out the importance of including the noise dimension.
Many policies have been adopted in Lisbon assuming the interconnectivity between the built and the natural environments. For instance, interventions in mobility, such as reducing cars in the city and improving shared commuting, have been adopted for reducing the negative impact of air and noise pollution on health [3,39]. Another example is the awareness of improving access to green spaces, which also recognizes the need to pay proper attention to soil-related indicators like the presence of unsealed soils [70]. In our study, soil did not reach the level of agreement to be included as relevant to assess EH in Lisbon. This may potentially be explained by the fact that soil is generally taken as relevant for health in rural settings and perceived as a mere supporting platform in urban areas. Nevertheless, the literature has pointed out that urban soils can improve the wellbeing of residents and mitigate the effects of the current climate crises, and it may be necessary to start collecting data for these indicators at local level [71].
Following the findings of our study, it is clear that there is a need for gathering diverse and robust evidence to be further used to inform not only the monitoring of EH in urban settings but also the design and analysis of the impact of EH policies. The list of indicators that resulted from this study within SDGs and within the scope of an European Green City can be a starting point in identifying entry points for planning interventions targeting increased EH in Lisbon; and the set of indicators and dimensions could be the basis to discuss indicators and data collection in other European urban settings and the development of EH related information systems. The next stage of our work could involve structuring the set of selected and validated indicators into tools via which the EH improvements could be assessed and monitored (for instance, within a dashboard or within a EH index for urban settings).

4.3. Strengths and Limitations

The complex and multisectoral nature of EH required a multi-disciplinary approach to develop a robust and comprehensive set of key dimensions and indicators to improve health in the Lisbon context. The adopted approach guaranteed that indicators from different areas of concern were canvassed in the final recommendations. By requiring that the initial set of indicators was filtered, analyzed, discussed, and validated through the experience of Portuguese experts, this study also serves as a test for the adequacy of the selected indicators in monitoring EH in the Lisbon city, and a template for comparison with other cities. The Web-Delphi process is structured and transparent and has shown to be useful as a validation tool. In addition, the use of a web-platform to build and deliver the questionnaire and to follow-up the process increased its efficiency. To address the potential bias of the Delphi studies, feedback was provided in an impartial way without influencing the responses. The anonymity of the experts was also assured.
Despite these aspects, this study design has several limitations. Semi-structured interviews allow the interviewer to discuss the questionnaire, but may be subject to bias, although the researcher was aware of the need to stay open to the suggestions of the interviewees. A further limitation of the study was the method used to identify the experts for the semi-structured interviews and for the Web-Delphi. While the recruitment protocol aimed to engage a multidisciplinary sample of experts, it should be acknowledged that a degree of convenience sampling was unavoidable and could have been tackled with a broader search to identify experts. Other limitations include the lack of guidance on Cronbach’s α thresholds, with 0.70 being adopted as the level after which there was a concordance between experts. Given the complexity of the EH concept, the results of this study could be improved if a higher number of experts were involved.

5. Conclusions

Aligned with the SDGs and with the current labelling of Lisbon as the European Green Capital 2020, this study led to the identification of six dimensions and 17 indicators essential to monitor EH in Lisbon city. A key finding is that a lack of data precludes the use of relevant indicators to monitor and assess EH in the Lisbon urban context. It is common that indicators are insufficiently tracked, and that many researchers and policymakers favor using fewer indicators with already collected data, rather than compiling a list of relevant indicators and then pushing for data collection. The adopted participatory approach highlights the usefulness of involving relevant experts from key fields of expertise in the selection of dimensions/indicators regarding EH; and the combined use of evidence, interviews and of a Web-Delphi process providing a consensual list of key indicators has shown to be transparent and flexible for replication in other EH contexts. The work developed in this study can inform the construction of tools to monitor EH, as well as helping policymakers in the definition of improvement goals and also in monitoring the extent to which those goals are fulfilled.

Supplementary Materials

The following are available online at https://www.mdpi.com/1660-4601/17/22/8597/s1, Interview template.

Author Contributions

Conceptualization; M.S. and M.D.O.; Investigation, M.S., Writing—Original draft preparation, M.S., Visualization, M.S.; Validation, A.C.L.V. and M.D.O.; Writing—Review & Editing, A.C.L.V. and M.D.O.; Supervision, A.T. and M.D.O. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Acknowledgments

Marta Salgado research was supported by the Portuguese Foundation for Science and Technology (FCT) and Valorsul S.A. under the scholarship number PDE/BDE/120465/2016. Ana Vieira and Mónica Oliveira work were supported by the Portuguese Foundation for Science and Technology under the MEDI-VALUE project (Grant no. PTDC/EGE-OGE/29699/2017).

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. United Nations General Assembly. Transforming Our World: The 2030 Agenda for Sustainable Development (Resolution A/RES/70/1 Adopted by the UN General Assembly on 25 September 2015); United Nations: New York, NY, USA, 2015; p. 35. [Google Scholar]
  2. Oliveira, G.M.; Vidal, D.G.; Ferraz, M.P.; Cabeda, J.; Pontes, M.; Maia, R.L.; Calheiros, J.M.; Barreira, E. Measuring Health Vulnerability: An Interdisciplinary Indicator Applied to Mainland Portugal. Int. J. Environ. Res. Public Health 2019, 16, 4121. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Oliveira, G.M.; Vidal, D.G.; Maia, R.L. Monitoring Portuguese living conditions at local scale: A case study based on sustainable development indicators. Int. J. Sustain. Dev. World Ecol. 2019, 27, 140–152. [Google Scholar] [CrossRef]
  4. Costa, C.; Santana, P.; Dimitroulopoulou, S.; Burström, B.; Borrell, C.; Schweikart, J.; Dzúrová, D.; Zangarini, N.; Katsouyanni, K.; Deboosere, P.; et al. Population Health Inequalities Across and Within European Metropolitan Areas through the Lens of the EURO-HEALTHY Population Health Index. Int. J. Environ. Res. Public Health 2019, 16, 836. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  5. Ramirez-Rubio, O.; Daher, C.; Fanjul, G.; Gascon, M.; Mueller, N.; Pajín, L.; Plasencia, A.; Rojas-Rueda, D.; Thondoo, M.; Nieuwenhuijsen, M.J. Urban health: An example of a “health in all policies” approach in the context of SDGs implementation. Glob. Health 2019, 15, 87. [Google Scholar] [CrossRef]
  6. Qiu, Y.; Lu, W.; Guo, J.; Sun, C.; Liu, X. Examining the Urban and Rural Healthcare Progress in Big Cities of China: Analysis of Monitoring Data in Dalian from 2008 to 2017. Int. J. Environ. Res. Public Health 2020, 17, 1148. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  7. Wu, C.Y.H.; Evans, M.B.; Wolff, P.E.; Gohlke, J.M. Environmental Health Priorities of Residents and Environmental Health Professionals: Implications for Improving Environmental Health Services in Rural Versus Urban Communities. J. Environ. Health 2017, 80, 28–36. [Google Scholar]
  8. United Nations, Department of Economic and Social Affiars, Population Division. The World’s Cities in 2018—Data Booklet; (ST/ESA/ SER.A/417); United Nations: New York, NY, USA, 2018. [Google Scholar]
  9. Sardain, A.; Tang, C.; Potvin, C. Towards a dashboard of sustainability indicators for Panama: A participatory approach. Ecol. Indic. 2016, 70, 545–556. [Google Scholar] [CrossRef] [Green Version]
  10. Ceretti, M.N.; Medeiros-Sousa, A.R.; Slovic, A.D. An Environmental Health Typology as a Contributor to Sustainable Regional Urban Planning: The Case of the Metropolitan Region of São Paulo (MRSP). Sustainability 2019, 11, 5800. [Google Scholar] [CrossRef] [Green Version]
  11. Guidotti, T.L. Environmental health needs a new paradigm, I. getting back in focus. Arch. Environ. Occup. Health 2018, 73, 267–269. [Google Scholar] [CrossRef] [Green Version]
  12. Brousmiche, D.; Occelli, F.; Genin, M.; Cuny, D.; Deram, A.; Lanier, C. Spatialized composite indices to evaluate environmental health inequalities: Meeting the challenge of selecting relevant variables. Ecol. Indic. 2020, 111, 106023. [Google Scholar] [CrossRef]
  13. Djalali, A.; Gianfrate, V.; Longo, D.; Vai, E. An advanced design approach to support urban transformations through multi-stakeholder collaborations. Strat. Des. Res. J. 2020, 12, 235–246. [Google Scholar] [CrossRef]
  14. Rydin, Y.; Bleahu, A.; Davies, M.; Dávila, J.D.; Friel, S.; De Grandis, G.; Groce, N.; Hallal, P.C.; Hamilton, I.; Howden-Chapman, P.; et al. Shaping cities for health: Complexity and the planning of urban environments in the 21st century. Lancet 2012, 379, 2079–2108. [Google Scholar] [CrossRef] [Green Version]
  15. Maroosi, M.; Mesdaghinia, A.; Alimohammadi, M.; Naddafi, K.; Mahvi, A.H.; Nodehi, R.N. Developing environmental health indicators [EHIs] for Iran based on the causal effect model. J. Environ. Health Sci. Eng. 2019, 17, 273–279. [Google Scholar] [CrossRef] [PubMed]
  16. Lichtveld, M.; Covert, H.; Sherman, M.; Shankar, A.; Wickliffe, J.K.; Alcala, C.S. Advancing Environmental Health Literacy: Validated Scales of General Environmental Health and Environmental Media-Specific Knowledge, Attitudes and Behaviors. Int. J. Environ. Res. Public Health 2019, 16, 4157. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  17. Eyles, J.; Furgal, C. Indicators in Environmental Health: Identifying and Selecting Common Sets. Can. J. Public Health 2002, 93, S62–S67. [Google Scholar] [CrossRef]
  18. Hunter, D.J. Relationship between evidence and policy: A case of evidence-based policy or policy-based evidence? Public Health 2009, 123, 583–586. [Google Scholar] [CrossRef]
  19. Freitas, Â.; Santana, P.; Oliveira, M.D.; Almendra, R.; Costa, J.C.B.E.; Costa, C.A.B.E. Indicators for evaluating European population health: A Delphi selection process. BMC Public Health 2018, 18, 557. [Google Scholar] [CrossRef]
  20. Salgado, M.; Madureira, J.; Mendes, A.S.; Torres, A.; Teixeira, J.P.; Oliveira, M.D. Environmental determinants of population health in urban settings. A systematic review. BMC Public Health 2020, 20, 853. [Google Scholar] [CrossRef]
  21. Mason, K.; Lindberg, K.; Read, D.; Borman, B. The Importance of Using Public Health Impact Criteria to Develop Environmental Health Indicators: The Example of the Indoor Environment in New Zealand. Int. J. Environ. Res. Public Health 2018, 15, 1786. [Google Scholar] [CrossRef] [Green Version]
  22. Pineo, H.; Glonti, K.; Rutter, H.; Zimmermann, N.; Wilkinson, P.; Davies, M. Characteristics and use of urban health indicator tools by municipal built environment policy and decision-makers: A systematic review protocol. Syst. Rev. 2017, 6, 2. [Google Scholar] [CrossRef] [Green Version]
  23. McMillan, S.S.; King, M.; Tully, M.P. How to use the nominal group and Delphi techniques. Int. J. Clin. Pharm. 2016, 38, 655–662. [Google Scholar] [CrossRef] [PubMed]
  24. Harbers, M.M.; Verschuuren, M.; De Bruin, A. Implementing the European Core Health Indicators (ECHI) in the Netherlands: An overview of data availability. Arch. Public Health 2015, 73, 9. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  25. Badland, H.; Whitzman, C.; Lowe, M.; Davern, M.; Aye, L.; Butterworth, I.; Hes, D.; Giles-Corti, B. Urban liveability: Emerging lessons from Australia for exploring the potential for indicators to measure the social determinants of health. Soc. Sci. Med. 2014, 111, 64–73. [Google Scholar] [CrossRef] [PubMed]
  26. Jiang, Y.; Mao, F.; Li, Y.; Liu, J.; Zhang, Y.; Jiang, Y.; Zhao, D.; Chen, W.; Nicholas, S.; Zhang, Y.; et al. Construction of China cardiovascular health index. BMC Public Health 2018, 18, 937. [Google Scholar] [CrossRef] [Green Version]
  27. Firth, A.; O’Brien, S.M.; Guo, P.; Seymour, J.; Richardson, H.; Bridges, C.; Hocaoglu, M.B.; Grande, G.; Dzingina, M.; Higginson, I.J.; et al. Establishing key criteria to define and compare models of specialist palliative care: A mixed-methods study using qualitative interviews and Delphi survey. Palliat. Med. 2019, 33, 1114–1124. [Google Scholar] [CrossRef] [Green Version]
  28. Chun, S.; Nam, K. A public healthscape framework for assessing geriatric long-term care public hospitals in South Korea by a Delphi expert consultation. Build. Environ. 2019, 165, 106291. [Google Scholar] [CrossRef]
  29. Pikora, T.; Giles-Corti, B.; Bull, F.; Jamrozik, K.; Donovan, R. Developing a framework for assessment of the environmental determinants of walking and cycling. Soc. Sci. Med. 2003, 56, 1693–1703. [Google Scholar] [CrossRef]
  30. Ratnapradipa, D.; Brown, S.L.; Wodika, A.B. Examining the Breadth and Depth Of Environmental Health Through a Modified Delphi Technique. Am. J. Health Educ. 2011, 42, 50–57. [Google Scholar] [CrossRef]
  31. Allen, B.L.; Lees, J.; Cohen, A.K.; Jeanjean, M. Collaborative Workshops for Community Meaning-Making and Data Analyses: How Focus Groups Strengthen Data by Enhancing Understanding and Promoting Use. Int. J. Environ. Res. Public Health 2019, 16, 3352. [Google Scholar] [CrossRef] [Green Version]
  32. Ratnapradipa, D.; Middleton, W.K.; Wodika, A.B.; Brown, S.L.; Preihs, K. What does the public know about environmental health? A qualitative approach to refining an environmental health awareness instrument. J. Environ. Health 2015, 77, 22–28. [Google Scholar]
  33. Alshenqeeti, H. Interviewing as a Data Collection Method: A Critical Review. Engl. Linguist. Res. 2014, 3, 39–45. [Google Scholar] [CrossRef]
  34. Brewerton, P.; Millward, L. Organizational Research Methods; SAGE Publications, Ltd.: London, UK, 2001; pp. 67–113. [Google Scholar]
  35. Dalkey, N.; Helmer, O. An Experimental Application of the DELPHI Method to the Use of Experts. Manag. Sci. 1963, 9, 458–467. [Google Scholar] [CrossRef]
  36. Vieira, A.C.L.; Oliveira, M.D.; Costa, C.B.E. Enhancing knowledge construction processes within multicriteria decision analysis: The Collaborative Value Modelling framework. Omega 2020, 94, 102047. [Google Scholar] [CrossRef]
  37. Perveen, S.; Yigitcanlar, T.; Kamruzzaman, M.; Agdas, D. How can transport impacts of urban growth be modelled? An approach to consider spatial and temporal scales. Sustain. Cities Soc. 2020, 55, 102031. [Google Scholar] [CrossRef]
  38. Wang, W.-M.; Peng, H.-H. A Fuzzy Multi-Criteria Evaluation Framework for Urban Sustainable Development. Mathematics 2020, 8, 330. [Google Scholar] [CrossRef] [Green Version]
  39. Lisboa Green Capital 2020. Available online: https://lisboagreencapital2020.com/ (accessed on 29 June 2020).
  40. Subiyakto, A.; Ahlan, A.R.; Putra, S.J.; Kartiwi, M. Validation of Information System Project Success Model. SAGE Open 2015, 5, 215824401558165. [Google Scholar] [CrossRef] [Green Version]
  41. Petrisor, B.; Keating, J.; Schemitsch, E. Grading the evidence: Levels of evidence and grades of recommendation. Injury 2006, 37, 321–327. [Google Scholar] [CrossRef]
  42. Lawrence, R.J. Urban Environmental Health Indicators: Appraisal and Policy Directives. Rev. Environ. Health 2008, 23, 299–325. [Google Scholar] [CrossRef]
  43. PORDATA. Base de Dados de Portugal Contemporâneo. Available online: https://www.pordata.pt/Portugal (accessed on 30 September 2020).
  44. Van Dijk, J. Popularising Delphi method. Qual. Quant. 1989, 23, 189–203. [Google Scholar] [CrossRef]
  45. Barriball, K.L.; While, A. Collecting data using a semi-structured interview: A discussion paper. J. Adv. Nurs. 1994, 19, 328–335. [Google Scholar] [CrossRef]
  46. Dupras, C.; Birko, S.; Affdal, A.O.; Haidar, H.; Lemoine, M.-E.; Ravitsky, V. Governing the futures of non-invasive prenatal testing: An exploration of social acceptability using the Delphi method. Soc. Sci. Med. 2020, 112930. [Google Scholar] [CrossRef] [PubMed]
  47. Cerè, G.; Rezgui, Y.; Zhao, W. Urban-scale framework for assessing the resilience of buildings informed by a delphi expert consultation. Int. J. Disaster Risk Reduct. 2019, 36, 101079. [Google Scholar] [CrossRef]
  48. Taylor, S.J.; Bogdan, R. Introduction to Qualitative Research Methods: The Search for Meanings; John Wiley &Sons, Inc.: New York, NY, USA, 1984. [Google Scholar]
  49. Adam, L.; Jones, T.; Brömmelstroet, M.T. Planning for cycling in the dispersed city: Establishing a hierarchy of effectiveness of municipal cycling policies. Transportation 2018, 47, 503–527. [Google Scholar] [CrossRef] [Green Version]
  50. Degeling, C.; Johnson, J.; Ward, M.; Wilson, A.; Gilbert, G. A Delphi Survey and Analysis of Expert Perspectives on One Health in Australia. EcoHealth 2017, 14, 783–792. [Google Scholar] [CrossRef] [Green Version]
  51. Nor, M.Z. Developing a preliminary questionnaire for the faculty development programme needs of medical teachers using Delphi technique. J. Taibah Univ. Med Sci. 2019, 14, 495–501. [Google Scholar] [CrossRef]
  52. Von Der Gracht, H.A. Consensus measurement in Delphi studies. Technol. Forecast. Soc. Chang. 2012, 79, 1525–1536. [Google Scholar] [CrossRef]
  53. Cullerton, K.; Adams, J.; Francis, O.; Forouhi, N.; White, M. Building consensus on interactions between population health researchers and the food industry: Two-stage, online, international Delphi study and stakeholder survey. PLoS ONE 2019, 14, e0221250. [Google Scholar] [CrossRef] [Green Version]
  54. Welphi. 2019. Available online: http://www.welphi.com/ (accessed on 2 April 2020).
  55. Hasson, F.; Keeney, S.; McKenna, H. Research guidelines for the Delphi survey technique. J. Adv. Nurs. 2000, 32, 1008–10015. [Google Scholar]
  56. Dalkey, N.; Brown, B.; Cochran, S. Use of self-ratings to improve group estimates. Technol. Forecast. 1970, 1, 283–291. [Google Scholar] [CrossRef]
  57. Meijering, J.V.; Kampen, J.K.; Tobi, H. Quantifying the development of agreement among experts in Delphi studies. Technol. Forecast. Soc. Chang. 2013, 80, 1607–1614. [Google Scholar] [CrossRef]
  58. Tavakol, M.; Dennick, R. Making sense of Cronbach’s alpha. Int. J. Med. Educ. 2011, 2, 53–55. [Google Scholar] [CrossRef] [PubMed]
  59. Tisch, C.; Pearson, A.L.; Kingham, S.; Borman, B.; Briggs, D. Environmental health indicators: A review of initiatives worldwide. Manag. Environ. Qual. Int. J. 2014, 25, 446–466. [Google Scholar] [CrossRef]
  60. Goldman, L.; Coussens, C.M. Environmental Health Indicators: Bridging the Chasm of Public Health and the Environment—Workshop Summary; National Academic Press: Washington, DC, USA, 2004. [Google Scholar]
  61. Frumkin, H. Environmental Health: From Global to Local, 2nd ed.; Jossey-Bass: San Francisco, CA, USA, 2010. [Google Scholar]
  62. Luo, R.; Zhang, C.; Liu, Y. Health Risk Assessment Indicators for the Left-Behind Elderly in Rural China: A Delphi Study. Int. J. Environ. Res. Public Health 2020, 17, 340. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  63. Nelson-Nuñez, J.; Walters, J.P.; Charpentier, D. Exploring the challenges to sustainable rural drinking water services in Chile. Water Policy 2019, 21, 1251–1265. [Google Scholar] [CrossRef]
  64. Home, R.; Vieli, L. Psychosocial outcomes as motivations for urban gardening: A cross-cultural comparison of Swiss and Chilean gardeners. Urban For. Urban Green. 2020, 52, 126703. [Google Scholar] [CrossRef]
  65. Nieuwenhuijsen, M.J. Urban and transport planning, environmental exposures and health-new concepts, methods and tools to improve health in cities. Environ. Health 2016, 15 (Suppl. 1), 161–171. [Google Scholar] [CrossRef] [Green Version]
  66. Gouveia, J.P.; Palma, P.; Simoes, S. Energy poverty vulnerability index: A multidimensional tool to identify hotspots for local action. Energy Rep. 2019, 5, 187–201. [Google Scholar] [CrossRef]
  67. Wu, Y.; Li, R.; Cui, L.; Meng, Y.; Cheng, H.; Fu, H. The high-resolution estimation of sulfur dioxide (SO2) concentration, health effect and monetary costs in Beijing. Chemosphere 2020, 241, 125031. [Google Scholar] [CrossRef]
  68. Dzhambov, A.; Markevych, I.; Tilov, B.; Dimitrova, D.D. Residential greenspace modifies the effect of road traffic noise exposure on mental health in youth. Eur. J. Public Health 2017, 27, 1–9. [Google Scholar] [CrossRef] [Green Version]
  69. Strosnider, H.; Kennedy, C.; Monti, M.; Yip, F. Rural and Urban Differences in Air Quality, 2008–2012, and Community Drinking Water Quality, 2010–2015—United States. MMWR. Surveill. Summ. 2017, 66, 1–10. [Google Scholar] [CrossRef]
  70. Da Silva, R.T.; Fleskens, L.; Van Delden, H.; Van Der Ploeg, M. Incorporating soil ecosystem services into urban planning: Status, challenges and opportunities. Landsc. Ecol. 2018, 33, 1087–1102. [Google Scholar] [CrossRef] [Green Version]
  71. Calzolari, C.; Tarocco, P.; Lombardo, N.; Marchi, N.; Ungaro, F. Assessing soil ecosystem services in urban and peri-urban areas: From urban soils survey to providing support tool for urban planning. Land Use Policy 2020, 99, 105037. [Google Scholar] [CrossRef]
Ijerph 17 08597 g001 550
Figure 1. Overview of the approach used to select the key dimensions and indicators, from built and natural environment determinants, to evaluate environmental health (EH) in the Lisbon urban setting.
Figure 1. Overview of the approach used to select the key dimensions and indicators, from built and natural environment determinants, to evaluate environmental health (EH) in the Lisbon urban setting.
Ijerph 17 08597 g001
Table
Table 1. Profile of the participants in stages 2 and 3 of the approach used to select the key dimensions and indicators to evaluate EH in the Lisbon urban setting.
Table 1. Profile of the participants in stages 2 and 3 of the approach used to select the key dimensions and indicators to evaluate EH in the Lisbon urban setting.
Stage 2Stage 3
ItemInterviews1st Round Web-Delphi2nd Round Web-Delphi
Gender, n (%)
Female6 (50%)14 (48%)10 (45%)
Male6 (50%)15 (51%)12 (55%)
Main area of expertise, n (%)
Built environment3 (25%)5 (17%)5 (23%)
Natural environment3 (25%)9 (31%)5 (23%)
Environmental health3 (25%)10 (35%)8 (36%)
National health regulation institutions3 (25%)5 (17%)4 (18%)
Table
Table 2. List of the 34 indicators retrieved from the literature and from databases search, and synthesis of experts’ opinions collected in the semi-structured interviews regarding the relevance of indicators in monitoring and assessing EH in the Lisbon urban setting.
Table 2. List of the 34 indicators retrieved from the literature and from databases search, and synthesis of experts’ opinions collected in the semi-structured interviews regarding the relevance of indicators in monitoring and assessing EH in the Lisbon urban setting.
Built Environment Determinant
DimensionIndicatorDisagreement (%)Agreement (%)Included
Green SpacesNumber of zoos, botanic parks, and aquariums *75.00No
Protected area surface (km2)83.40No
Protected area proportion (km2)75.00No
Number of gardens and parks75.00No
Area of leisure parks and gardens (m2)083.3Yes
Area of community gardens (m2)083.3Yes
HousingNumber of classic family housing buildings *83.30No
Number of buildings by geographic location and type *75.00No
MobilityBuilt-up areas (km2) *66.70No
Road vehicles by type and fuel *16.758.4Yes
Traffic accidents with victims/1000 inhabitants66.70No
Cycle roads (km)075.0Yes
Road network (km)075.0Yes
Proportion of resident population using individual mode of transport while commuting *66.70No
Proportion of resident population using public transport mode during commuting *58.30No
SafetyPolice-reported crimes/1000 inhabitants *58.30No
SanitationUrban waste collection *58.30No
Population served by wastewater drainage systems *58.30No
Natural Environment Determinant
Air qualityPM2.5 (particulate matter)083.3Yes
PM10 (particulate matter)083.3Yes
O3 (ozone)083.3Yes
NO2 (nitrogen dioxide)083.3Yes
NOx (nitrogen oxides)8.3375.0Yes
SO2 (sulfur dioxide)083.3Yes
CO (carbon monoxide)083.3Yes
C6H6 (benzene)8.3375.0Yes
NoiseExposure to high noise75.016.7No
Suffering from noise *66.725.0No
Lden (day-evening-night equivalent sound level)083.3Yes
Ln (day-night equivalent sound level)083.3Yes
Water quality“Água segura” *,a091.7Yes
Public water supply *58.30No
SoilSoil organic carbon content *,b1000No
Soil sealing index *,c75.016.7No
Legend: * indicators for which information is currently available only for the national level; a: “Água segura” is the Portuguese indicator that measures the percentage of compliance with the parametric values established in the legislation; b: amount of organic carbon capable of affecting groundwater; c: percentage of unsealed soil.
Table
Table 3. Indicators approved by majority decision rules after the second round, grouped by built and natural environment determinants.
Table 3. Indicators approved by majority decision rules after the second round, grouped by built and natural environment determinants.
DeterminantDimensionIndicatorsAgreement Rate (%)Inclusion By
Built environment Green spacesArea of leisure parks and gardens63.6Absolute
Area of community gardens54.5Qualified
HousingEnergy poverty vulnerability index95.5Qualified
MobilityCycle roads86.4Qualified
Number of road vehicles81.8Qualified
Road network77.3Qualified
Natural environment Air qualityPM2.5 86.4Absolute
PM10 63.6Absolute
O3 63.6Absolute
NO2 63.6Absolute
NOx 59.1Absolute
SO2 86.4Qualified
CO59.1Absolute
C6H654.5Absolute
NoiseLden90.9Qualified
Ln63.6Absolute
Water quality“Água segura”95.5Absolute
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Salgado, M.; Vieira, A.C.L.; Torres, A.; Oliveira, M.D. Selecting Indicators to Monitor and Assess Environmental Health in a Portuguese Urban Setting: A Participatory Approach. Int. J. Environ. Res. Public Health 2020, 17, 8597. https://doi.org/10.3390/ijerph17228597

AMA Style

Salgado M, Vieira ACL, Torres A, Oliveira MD. Selecting Indicators to Monitor and Assess Environmental Health in a Portuguese Urban Setting: A Participatory Approach. International Journal of Environmental Research and Public Health. 2020; 17(22):8597. https://doi.org/10.3390/ijerph17228597

Chicago/Turabian Style

Salgado, Marta, Ana C. L. Vieira, Anália Torres, and Mónica D. Oliveira. 2020. "Selecting Indicators to Monitor and Assess Environmental Health in a Portuguese Urban Setting: A Participatory Approach" International Journal of Environmental Research and Public Health 17, no. 22: 8597. https://doi.org/10.3390/ijerph17228597

APA Style

Salgado, M., Vieira, A. C. L., Torres, A., & Oliveira, M. D. (2020). Selecting Indicators to Monitor and Assess Environmental Health in a Portuguese Urban Setting: A Participatory Approach. International Journal of Environmental Research and Public Health, 17(22), 8597. https://doi.org/10.3390/ijerph17228597

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop