Next Article in Journal
Assessing the Hydrodynamic Response of the Mar Menor Lagoon to Dredging Inlets Interventions through Numerical Modelling
Previous Article in Journal
Performance Evaluation of Small Sized Powdered Ferric Hydroxide as Arsenic Adsorbent
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Improving Public Participation Processes for the Floods Directive and Flood Awareness: Evidence from Cyprus

by
Katerina Charalambous
*,
Adriana Bruggeman
,
Elias Giannakis
and
Christos Zoumides
Energy, Environment and Water Research Center, The Cyprus Institute, 20 Kostantinou Kavafi Street, Aglantzia, Nicosia 2121, Cyprus
*
Author to whom correspondence should be addressed.
Water 2018, 10(7), 958; https://doi.org/10.3390/w10070958
Submission received: 5 July 2018 / Revised: 14 July 2018 / Accepted: 18 July 2018 / Published: 20 July 2018
(This article belongs to the Section Water Resources Management, Policy and Governance)

Abstract

:
Public participation is integrated in the European Floods Directive to ensure engagement of societal actors in selecting and accepting measures. This study assesses the Directive’s public participation process and provides recommendations for its improvement by using Cyprus as a case study. Interviews with the organizers and attendees of the public consultations were carried out to evaluate the process while a citizen survey examined people’s flood awareness and opinions of three household-level flood protection measures (permeable pavements, rainwater harvesting systems, and green roofs). Public consultation organizers were generally satisfied with the process while participants suggested better structured information and a more participatory approach. The majority (77%) of the survey respondents did not know if they lived in a designated flood risk area while 93% were unaware of the public consultations carried out for the Floods Directive. Their perception about the effectiveness of the three flood protection measures was positively associated with their willingness to implement them. The results indicated the need for more participatory methods in the public participation process and better strategies to increase awareness and the engagement of people in flood management. Establishing procedures for evaluating the effectiveness of public participation could contribute to the recognition and improvement of the process.

1. Introduction

Flood management has seen a shift from conventional, structural flood defense to a more integrated flood risk management approach in which structural defense measures are complemented with non-structural or ‘soft’ measures to reduce flooding effects [1]. Public participation contributes to flood risk management through the exchange of information and the co-operation and improved coordination of the public and various authorities, which results in informed decision-making and the implementation of measures [1,2,3,4]. Additionally, taxpayers and voters should be kept informed about investments made with public funds. Locals can also play an active role in hazard prevention [5]. Public awareness and public participation are interrelated and form the basis of a flood resilient society [1,6].
Governments and local authorities have been developing strategies to promote flood awareness. Mees et al. [7] examined the different forms of citizen involvement in the decision making process for flood risk management and the implementation of the respective measures in five European countries (England, Belgium-Flanders, France, the Netherlands, and Poland). They found that in England when compared to the other four countries, the less dominant role of the state in flood management has led to higher involvement of citizens in the process and the shared responsibility between citizens and the state in flood preparedness. The United Kingdom (UK) has been proactive in raising flood awareness by providing online flood risk maps and information as well as setting up flood volunteer groups and telephone and email services [8,9]. Nye et al. [10] explored the increasing trends towards public engagement in the UK’s flood and coastal risk management and noted the need for flexible approaches. In Switzerland, municipalities are obliged to take hazard maps into account for city planning and to inform people in flood hazard areas of the risks and any compulsory or voluntary protective measures. Maidl and Buchecker [11] found that a single one-way information campaign where letters were sent to all property owners in one in 100-year flood zones in Zurich had a limited effect. Similar to other studies (e.g., [6]), they related the low level of flood protection awareness and preparedness to the absence of recent flood experience. Birchard and Kazmierczak [8] suggest that policy makers and emergency planners should revisit the methods with which risk is communicated. Sörensen et al. [12] highlight how floods open up new approaches to urban planning, which requires better communication and clarification of responsibilities between various authorities and stakeholders.
Direct financial incentives (e.g., subsidies) and indirect financial incentives (e.g., reduction of storm water utility fees) have been used to encourage homeowners and developers to implement non-structural flood prevention measures such as green roofs in countries like Germany and the U.S. [13]. However, such policies are often affected by budget constraints [9,13]. Domènech and Saurí [14] found that regulations and partial subsidies are effective strategies for advocating the adoption of rainwater harvesting systems at single or multi-family homes, but awareness campaigns about the benefits and requirements of the technology are also needed. Birchard and Kazmierczak [8] found in their survey that, in flood risk areas in England and Wales, the median amount of money people were willing to spend on flood protection measures was less than one fourth of the cost of the measures. They suggest that, besides subsidies and other policy tools such as systems of no-interest loans for flood resistance and resilience products, building regulations can also be revised to ensure that all new and refurbished buildings in high flood-risk areas are equipped to be flood resistant or resilient. However, they did not investigate people’s opinions of such incentives.
The European Floods Directive (FD) adopted in October 2007 aims to reduce and manage flood risks. It requires member states to identify areas of potential significant flood risk and produce flood hazard and risk maps based on which flood risk management plans (FRMPs) are developed. These plans focus on flood prevention as well as protection and preparedness by taking into account the characteristics of each particular river basin [15]. Public participation is integrated in the process of the FD as well as in the Water Framework Directive (WFD) to ensure a wide societal engagement in the identification, selection, and acceptance of the best and most cost-effective measures [16]. Three forms of public participation with an increasing level of involvement apply to the FD. These three forms are information supply, which is to be ensured, consultation, and active involvement, which are to be encouraged [17].
Information supply is considered to be the foundation for any form of public participation [16]. Although the requirement is access to background information and not necessarily active dissemination of information, the latter is essential for consultation and active involvement [16]. Consultation means that the public is invited to respond to plans and proposals developed by the authorities through public hearings, surveys, and interviews. Active involvement is a higher level of participation than consultation. It implies that stakeholders participate in the planning process by discussing issues and contributing to their solution. It is noted that the different levels of participation build on each other. For example, consultation implies an information supply and active involvement implies a consultation [16].
Hartman and Spit [18] believe that the FD has achieved a discourse and inspired experiments with new approaches. However, it remains vague on the reasons and goals of collaboration and participation. Since FRMPs only have to define measures and not address how flood protection will be achieved, public participation seems to be carried out in the hope to enhance flood protection [19]. However, water authorities have been noted to have difficulties implementing public participation processes [18]. In response to criticism after the damaging June 2013 flood in Germany that participatory processes had delayed the implementation of flood protection, Otto et al. [20] analyzed 67 disputes and found that, in half of them, there was discontent with the process of public involvement. While many recognize the scope and challenges for achieving active involvement of stakeholders [19], there is limited documentation on public participation in flood risk management planning under the FD. The evaluation of the public participation processes for the first cycle of the FD implementation will contribute to the improvement of the FRMPs in the second planning cycle. Lessons through the implementation process are expected [17], which may reveal the need for procedural requirements to improve coordination and integration of public participation processes [3,21]. At the same time, many European research projects are engaging stakeholders in the research process and developing innovative ways of connecting science and society to address societal challenges [22].
In Cyprus, due to the island’s semi-arid climate, the prevailing water-related challenge is water scarcity. As a result, authorities have given less attention to storm water management and flood mitigation in urban planning and awareness of the matter is low [23]. Nevertheless, rain events of intense nature occur regularly and often cause flooding. The implementation of the FD has necessitated commitment by the Water Development Department (WDD), which is the responsible government authority, to focus on flood management. For the first implementation cycle of the FD, the WDD identified areas of potentially significant flood risk based on the occurrence of historical flooding events. These events have been categorized as either flash, urban, pluvial, river floods, or a combination of the previous. Hydrologic models were subsequently used to map flood hazard areas with river flooding being the main focus. In many urban areas, river flooding is enhanced by urban and pluvial floods because runoff from the impermeable areas drains to rivers.
In the Program of Measures (PoM) developed for the FRMP, the WDD identifies measures to reduce the risk of flooding. The PoM does not specify the type of floods each measure addresses but classes them in one of four axes of flood management, i.e., prevention, protection, preparedness, and restoration [24]. Prevention measures include the avoidance of constructing houses and industries in flood-prone areas, the adaptation of future developments to the risk of flooding, and the promotion of appropriated land use. Protection measures aim to reduce the likelihood of flooding either through structural or non-structural measures, while preparedness measures focus on emergency planning and awareness raising for how to respond to a flood. Restoration measures aim to mitigate the social and economic impact of flooding on the affected population and to return to normal conditions as soon as possible.
The public participation process for the implementation of the FD in Cyprus has a dual aim [25]. The first aim is to actively engage the administrations of municipalities and communities in the formulation of the FRMP and PoM. The second aim is to diffuse and communicate the technical parameters of the FRMP to institutional and local organizations and citizens and to achieve a common understanding and acceptance of the plan. The effective implementation of the FD entails that affected and responsible parties are involved in the formation of the FRMP while, during and after the process, citizens should be informed and prepared for floods [25]. Therefore, the term “public” is used to refer to citizens as well as non-governmental organizations and “stakeholders” is used to refer to the governmental and administrative authorities and organizations that have an immediate role and involvement in flood management such as the Town Planning Department, Public Works Department, and administrations of municipalities in flood-affected areas. Thus, in the “public participation process,” both the public and stakeholders are involved.
This study aims to provide recommendations for the improvement of the Floods Directive’s public participation process using Cyprus as a case study. The specific objectives of this research are (i) to review the public participation process for the implementation of the first cycle of the Floods Directive in Cyprus; (ii) to assess the implementation of the public consultation process for the Floods Directive in Cyprus; and (iii) to examine public awareness regarding flood hazards and citizens’ opinions and willingness to implement three household-level flood protection measures. Interviews were made with the organizers and attendees of the public consultations for the first FRMP cycle and a survey with 391 citizens was carried out. Results from the interviews are used to assess the process of public consultation while the survey examined public awareness and involvement in flood management.

2. Materials and Methods

2.1. Review of the Public Participation Process

Information on the implementation of the first cycle of the FD as well as the public participation process followed was retrieved from the WDD website [26]. Additional information on the process and the involvement of stakeholders was collected from interviews carried out with WDD officers. The public participation process was outlined in a step-by-step chronological order.

2.2. Interviews on the Public Consultation Process

Semi-structured interviews were carried out with 10 individuals involved in the public consultation process of the FD implementation in Cyprus. The individuals interviewed represent both sides of the process. Five of the interviewees are people in authoritative positions responsible for implementing the Directive (technical officers of the WDD) and five are attendees at the public consultation events (representatives from government authorities, municipalities, non-government organizations, and citizens). Two of the five public consultation attendees were from stakeholders actively involved during the preparation of the FRMP. Interviewees were asked to express their overall opinion regarding the public consultation process as well as their considerations on how this procedure can be improved.

2.3. Survey on Flood Awareness and Flood Protection Measures

2.3.1. Questionnaire Survey on Flood Awareness and Flood Protection Measures

A questionnaire survey was designed to examine public awareness of floods and citizens’ opinions on three household-level flood protection measures. The survey focused on the urban area of Nicosia, which is the capital of Cyprus, where three streams (the Pedieos, Klimos, and Kalogeros Rivers) were identified to create flood risk areas. The questionnaire was kept short with a total of 10 questions (see Appendix A) to ensure completion by both people who are concerned about floods and people who are not concerned about floods. The survey was first pilot-tested and adjusted accordingly.
The survey was carried out in the linear park along the Pedieos River and in the Mall of Cyprus, which is the largest indoor shopping center in Nicosia. The Pedieos Park is a flood hazard area and is visited by many people living in its vicinity [27]. Survey questions inquired whether people lived in a flood-affected area and whether they had attended public consultation events. A large flood hazard map for urban Nicosia was displayed on a 0.8 m × 2 m roll-up banner using the official flood hazard areas developed for the Floods Directive [28]. The map allowed respondents to identify whether their house is located in a flood hazard area (Figure 1). All passers-by were approached, survey questions were posed, and replies were filled out by the researchers.
The PoM for the first implementation cycle of the FD promotes “horizontal” household-level protection (PRO) measures that homeowners can adopt. “Horizontal” measures are measures that are applicable to the whole island such as practices, protocols, and legislative changes including the establishment of monetary incentives (subsidies). Conventional anti-flood measures that are applicable to individual identified flood hazard areas are referred to as “specific” measures. Measure PRO-CY_04 recommends incorporation of planning requirements for the reduction of runoff from residential and industrial properties and suggests examples such as retention ponds and pervious pavements. Measure PRO-CY_06 promotes the institutionalization of financial incentives for the adoption of systems or techniques that allow for the reduction of runoff from properties (for existing buildings) such as replacement of impermeable areas with planted surfaces. Similar to PRO-CY_06, measure PRO-CY_07 also promotes financial support for the adoption of rainwater harvesting systems (for existing properties) and mentions that the collected water could be used for garden irrigation. Based on these flood protection measures, the three measures selected for the survey were permeable pavements, rainwater harvesting systems, and green roofs. Green roofs were considered representative of Measure PRO-CY 06 and were chosen as an upcoming green infrastructure plan with multiple environmental benefits including storm water runoff delay and retention [29,30,31].
The survey included short descriptions, pictures, and estimated costs of the three selected household-level measures. The cost of each individual measure was estimated through literature research as well as consultation and personal communication with local suppliers and developers. An approximate cost of €250 per m2 was estimated for the permeable pavements. The system would include gravel pavers and a 25-cm aggregate base [32,33]. The costs for a rainwater harvesting system includes gutters, mesh screens, pipes, pumps, pressure tanks, filtering, disinfection, and a 10-m3 storage tank, which were estimated to add up to €4000 [34,35]. Costs for installing green roofs differ depending on the type of materials used for the drainage, the filter layer, the substrate type and depth, and the plant species used. An estimated cost of €200 per m2 for a green roof system that would include an insulation mat, a drainage layer, filter sheet, 25-cm substrate layer, plants, and an irrigation system was established with the help of a local green roof expert [36].

2.3.2. Statistical Methods

Pearson chi-square tests of independence were used to examine whether unpaired data on two categorical variables of interest expressed in a contingency table, i.e., a table that displays the frequency distribution of the categorical variables are independent of each other. The following hypotheses were tested: H0-the variables of interest are independent and H1-the variables of interest are associated. More precisely, the relationships explored are shown below.
  • Is there a relation between living in flood-affected areas or being concerned about floods and interest in attending flood information meetings?
  • Does citizens’ perception of the effectiveness of the three household-level flood protection measures influence whether they believe these measures should be subsidized and to what extent?
  • Are citizens who live in flood-affected areas or who are concerned about floods more willing to implement the three household-level flood protection measures?
  • Does citizens’ perception of the measures’ effectiveness affect their willingness to implement them?
The chi-square test statistic computes how the pattern of observed frequencies differs from the pattern of expected frequencies and is given by the formula [37] below.
X 2 = i = 1 n ( O i E i ) 2 E i
where n is the number of cells in the contingency table, O i is the observed frequency, and E i is the expected frequency in cell i of the table. For the test of interdependence, a chi-squared probability of less than 0.1 indicates that the null hypothesis (H0) is rejected. All calculations were performed in SPSS 13.0 [38].

3. Results and Discussion

3.1. Public Participation Process for the First FRMP Cycle

For the implementation of the FD, the WDD had outsourced most of the work to environmental consultancy companies through public procurement procedures. Nineteen areas of potential significant flood risk were identified based on newspaper and archival research as well as communication with municipalities and communities. Flood hazard and risk maps were created by flood modeling and mapping. The FRMP was developed along with the PoM, which proposed 28 horizontal measures and 10 specific measures. The measures were suggested by the WDD and its consultants. Prioritization of the measures based on their effectiveness and level of difficulty to implement was carried out followed by a cost-benefit analysis.
Table 1 presents the public participation activities that took place during the first cycle of the FD implementation in Cyprus. As part of the process, three information meetings were held to introduce the areas of potential significant flood risk (January 2012) and to present the flood hazard and risk maps (March 2013, July 2014). During the presentation of the preliminary flood risk and hazard maps (March 2013), there were municipalities and communities that wanted to influence the setting of the flood zones as these would affect land value. However, WDD officials stated that the process carried out was on a purely scientific basis. Based on the Directive’s three public participation forms, these three information meetings can be considered to be public consultation events since they were organized for people to respond to plans and proposals. However, the WDD designated the timeframe following the drafting of the FRMP and the PoM (December 2015–February 2016) to be the consultation period. A Steering Committee was created to assist in the drafting and the implementation of the FRMP and the PoM. This committee consisted of 25 people from 14 stakeholders. The involvement of these stakeholders can be classified as active involvement, according to the three types of public participation.
The length of the public consultation period for the FD was only two months (December 2015–February 2016) due to delays in initiating the process. During this time, the FRMP and PoM was made available on the WDD website and people were able to fill in an online questionnaire providing their feedback and opinions. Furthermore, during this period, the WDD organized four public consultation events at a district level. The WDD had also decided to merge the public consultations for the second River Basin Management Plan (RBMP) with the events for the first FRMP instead of carrying them out separately.
The first set of meetings (January 2016) targeted stakeholders and were organized to accommodate an exchange of information and a more in-depth discussion about site-specific issues. The second set of meetings (February 2016) targeted the public and other governmental and non-governmental organizations that are not directly affected or involved in the plan. Suggestions from the district-level meetings were incorporated in the FRMP and PoM before the Pancyprian public consultation, which was attended by stakeholders, various government organizations, and the public (February 2016). The invitations for the consultation meetings were sent by fax and email to the targeted governmental and private organizations while citizens were invited to all events through press releases and announcements on the WDD website. Attendees at the public consultation events were also requested to complete assessment forms on the consultation process. Approximately 60% of the public consultation participants stated that they were very satisfied with the meeting.

3.2. Interview Results on the Public Consultation Process

Public participation in environmental matters is something new for the Cypriot society. According to one of the interviewed WDD officials, the public consultation process for the first cycle of the RBMP “was perhaps the first largest outreach event carried out in Cyprus.” For this first RBMP consultation process, WDD officials found that people appreciated being approached and included in the process and began to comprehend the concept of public consultations. However, they were somewhat disappointed with the overall results considering the funds and energy spent.
Focusing on the first FRMP consultation process, according to all interviewed WDD officials, public consultations are “an important and useful tool, especially because local people are more aware of flooding problems in their area than the WDD officers.” They noted that perhaps the most important aspect of the consultation is that it offers a medium for measures to become known and accepted by people especially for measures that the public will have to adopt and implement. On the downside, they found that the process was time-consuming and costly. They followed all the necessary procedures outlined by the EU and were overall satisfied with the process. The meetings were considered to be overall successful, according to the interviewed WDD officers. Two of the interviewed officials stated that the public consultations were “relatively well attended considering the lack of flooding problems and society’s involvement in the state’s environmental matters.”
They noted that those who attended the public consultation events were people whose properties are affected and people with complaints. One of the officials noted that certain groups rather than individuals were more “verbal” during the sessions such as the Cyprus Real Estate Association, which did not want the maps to be publically available since it would lead to a loss in land value. There was little response to the online questionnaire by the public. All officials mentioned that the process could have been better attended and more efficient if there had been meetings carried out on a local level with smaller groups and focus on specific river basin issues. However, they all noted that, due to lack of time, staff, and money, this approach was not feasible.
The interviewed WDD officials recognized that merging the public consultations for the WFD and FD made the process quite lengthy and it was difficult to keep people’s attention for the full day. The approach for these meetings was more top-down compared to the public consultation for the first cycle of the RBMP because of a loss of enthusiasm on the WDD’s part, which is also due to the limited staff and budget following the country’s economic crisis in 2013. The WDD officers were also stricter in adding measures that were suggested by the public during the second cycle of the RBMP to the PoM.
Opinions on the consultation process were different from the perspective of the attendees. They believed that public participation is important for knowledge exchange, to raise awareness about water issues, to create a more active society that is engaged in environmental issues, and to keep citizens continuously updated and involved in such matters. However, they were disappointed with the public consultations in Cyprus. They found that the events were poorly attended. The events could have been better advertised and held outside working hours.
One interviewee mentioned his impressions from the first public consultation for the WFD: “simple folks that are interested in their own well-being took the microphone and wasted our time by talking only about their own problems.” According to the interviewee, “the process could have been more productive if the consultation was among a closed scientifically-focused group rather than open to the public. Giving the floor to everyone to express an opinion is rather time-consuming and not always relevant to the topic,” which is something that other interviewees also expressed. Two of the interviewed attendees feel that public opinion is not truly taken into consideration in the process. There is a lack of clarity created by officials and politicians with hidden information and decisions already made. One of the two also stated that “to improve the island’s water problems and for people to be informed and considered in the process, a general change in the island’s governance and political system is needed.”
Some of the interviewees pointed out that the material provided for the consultation was overwhelming and not structured in such a way that both laymen and technical specialists can easily find the information they need. Two interviewees emphasized the need for an intermediary between the public and the national authorities who develop the FRMP and PoM since there is “a large technical knowledge gap.” They commented that “the people who attend these meetings, including representatives of local authorities, do not have the technical knowledge or support to understand what the problems are and what is being proposed.” They suggested that the merging of municipalities and communities into larger administrative entities would allow the appointment of technical staff who could collect and diffuse flood information. These local experts could act as an intermediary between national-level officials that are in charge of implementing the FD and the public.

3.3. Survey Results

3.3.1. Public Awareness of Floods and the Floods Directive

The questionnaire survey was conducted with 391 people of which 47.3% of the responses were collected from the Pedieos Linear Park and 52.7% from the Mall of Cyprus. Numerically, this is 0.17% of the population in urban Nicosia. Table 2 presents the characteristics of the 391 survey respondents as well as their past flood experience and awareness of the Floods Directive. The survey captured a relatively well-balanced sample of people who were not concerned about floods (46.5%) and people who were moderately or strongly concerned about floods (53.5%).
Strikingly, out of the 343 people who responded that the area they live in is not affected by floods, 60 people (17.5%) actually live in an officially designated flood hazard area, according to the displayed map, and they were not aware of it. Out of the 48 people who responded that they do live in an area that is affected by floods, 19 did not live in a designated flood hazard area while the residence of 18 respondents was outside the map. One explanation for this finding is that many urban areas are susceptible to pluvial flooding, which may occur everywhere [39]. However, the flood hazard maps for Cyprus that were developed in the first FD cycle focused on fluvial floods since these were considered to cause greater damage based on historical facts and the existence of built-up areas along rivers [28].
The majority of the respondents (300) did not know if their residence is in a designated flood hazard area or not. From the total sample, only 8 people responded that their area of residence has been designated as a flood hazard area. One of those was wrong while, for four respondents, it could not be verified because their house was located outside the map. From the 83 people that responded negatively on this matter, seven people were wrong while 18 lived outside the area shown on the map. A comparison between the two survey locations revealed that, out of the 71 people whose residence is in a designated flood hazard area, 53 (75%) were visitors of the Pedieos Park. Regarding respondents’ awareness of the public consultations carried out for the implementation of the FD, the majority of people (366) answered negatively. Therefore, there appears to be low awareness of the Floods Directive, the public consultation events, and potentially limited dissemination of information regarding flooding.
One-hundred fifty-five respondents (40%) stated that they would be interested in attending a flood information meeting. It is important to mention that the survey was carried out after an extended dry period, which did influence people’s initial approach to the survey. Many people commented that since there is no rain flooding is not an issue. One particular person recalled how she had been carried away in her car by severe floods in July 1995. However, she was only moderately concerned about floods because she said, “it doesn’t rain as it used to.” It should be noted that a few days after the survey, a three-day rain event did cause flooding problems in various urban areas of the island. Furthermore, we did find a statistically significant relation between being affected by floods and interest in attending a flood information meeting (p = 0.002). Therefore, if people are affected by floods and are aware of flood information meetings, they would be likely to attend. Similarly, we found statistically significant relationships between concern about floods and interest in attending an information meeting both for people living in a flood hazard area (p = 0.062) and for people outside the flood hazard areas (p = 0.000). Therefore, outreach on flood matters could increase people’s attendance to flood information meetings.

3.3.2. Opinions of House-Hold Level Flood Protection Measures

Figure 2 presents the perception of respondents regarding the effectiveness of the three selected household-level flood protection measures in reducing floods in semi-arid cities. The majority of respondents believe that permeable pavements and rainwater harvesting systems are effective in reducing flooding in urban areas. Yet, green roofs were considered the least effective option with only 33% of the respondents agreeing that this measure could be effective in reducing flooding.
The respondents’ opinions on the extent to which the measures should be subsidized are presented in Figure 3. A statistically significant relationship (p = 0.000) was found between people’s perception of the effectiveness of the measures and the percentage of the cost they believed that the government should cover for the measures to be implemented. It is not surprising that people would expect household-level flood protection measures to be subsidized based on their effectiveness to reduce flood. Although the respondents’ opinion on the share of the cost that should be subsidized for the three measures is rather even, there is a slightly larger preference for subsidies for rainwater collection systems. This may affirm the fact that people do not consider flooding to be as important an issue as droughts since their reason for installing rainwater harvesting systems could be that they want to store as much water as possible.
During the survey, there were repeated comments by respondents that “subsidies should depend on people’s income” and “they should only be given to those who live in a flood hazard area.” Birchard and Kazmierczak [8] found a very low adoption rate of flood protection measures through a telephone survey of 961 homeowners in England and Wales. These authors note that the cost of government support for all properties in flood risk areas would be excessive, but they considered subsiding flood protection for the poorest and oldest householders a moral imperative. In their survey of 86 people living in flood affected areas in South Bohemia (Czech Republic), Vávra et al. [40] found that 74% of the respondents believe that subsidies should be provided to farmers who increase retention of rainwater on their land.
In Cyprus, there are currently no financial incentives or legislation for land owners to manage storm water. The only exception is the Sewerage Board of Limassol-Amathus in Limassol district, which encourages homeowners to direct storm water from their property to their disused cesspits. It has also adopted regulatory measures requiring new houses to construct 10-m3 rainwater collection cisterns per property of 800 m2. Additionally, it promotes the construction of permeable pavements and other sustainable urban drainage systems. Prior to the 2013 economic crisis, the WDD offered subsidies for four household-level water-saving measures, which were the installation of a hot water re-circulator (€220), the installation of a grey water recycling system (€3000), the drilling of a borehole for watering gardens (€ 700), and the connection of a borehole with toilet cisterns (€700) [41]. In 2009, 1408 applications were approved in which the majority (789) were for garden boreholes [23].
From the 391 respondents, 61% had a house or planned on building one in the near future. Figure 4 presents the likelihood of respondents implementing the three-household measures. There was no statistically significant relation between respondents who lived in an area affected by floods and their willingness to implement the measures. However, our analysis revealed a statistically significant relationship between people concerned about flooding and their willingness to implement rainwater harvesting systems (p = 0.075). As pointed out by a number of respondents, “the rainwater harvesting system is much simpler to install than the other two measures” since it does not require removing or enhancing the infrastructure of the house. In this case, simplicity to install, low cost, and additional benefits to homeowners may be important elements for examining to make such measures appealing to individuals.
From a telephone survey in the UK, Harries [42] found that there are situations in which flood experience may reduce the adoption of household flood protection measures due to an increase in anxiety and avoidant behavior. On the contrary, a survey of 1375 questionnaires across 13 locations in six European countries (Belgium, Finland, Germany, Ireland, Italy, and Scotland) found that flood experience was associated with flood awareness and flood preparedness [6]. Rözer et al. [39] conducted telephone interviews to explore the behavior of private households towards flood risk in three case studies in Germany 17 months after three different pluvial flood events. Findings revealed that the majority of households in all three study areas were not aware of the pluvial flood risk while the preparedness level of affected households improved considerably after the flood events. Similarly, Buchecker et al. [5] found that negative emotions from past flood events will motivate individuals to seek information and consider installing household-level measures.
Our survey also found a statistically significant relationship (p = 0.00) between people’s perception of the effectiveness of the three measures and their willingness to implement them. This finding supports the need to conduct interactive awareness campaigns and inform the public about flood protection measures. The majority of respondents (63%) were interested to receive more information on the three household measures presented. Although, we made an effort during the survey to give a concise presentation of each measure with an illustration and brief description, many respondents had an initial difficulty comprehending them. Thus, apart from providing basic information through leaflets, people could be triggered to implement some of the measures if they observe these technologies in action. Owusu et al. [9] found that awareness of personal-level flood protection measures was strongly associated with recent public education campaigns from their 256-questionnaire survey in Scotland. Birchard and Kazmierczak [8] also highlight the need for a more targeted education and awareness of homeowners about available flood protection measures.
Overall, the survey findings indicate that there is a lack of public awareness of flooding. This implies that the Floods Directive in Cyprus needs to be better disseminated by the authorities who should provide motives for the public to become more involved in the process. From their evaluation of 43 local flood risk management strategies in England, Benson et al. [43] pointed out that communication with the public on flood risks appeared optimal where strategies were produced in a non-technical way. This included the use of maps, photographs, and case studies presented in meetings and through other media. Similarly, Maidl and Buchecker [11] observed that flood awareness in Zurich, Switzerland was not anchored in the public mind. They also found that the proportion of property owners who could correctly identify the risk level of their own area was extremely low (17%). As the authors mention, effective communication of hazard maps and other risk prevention tools is still a challenge.
As in the case of Maidl and Buchecker [11], the survey carried out in urban Nicosia did increase people’s awareness of flooding and protection measures. A number of people were interested and happy to talk about flooding issues and their personal experiences as well as to give their opinions on the three measures. When given a choice, people do have an opinion about different options. This outcome highlights the importance of science and society interaction and indicates the benefits of more proactive awareness campaigns. Although information online is available and easy to access and understand, active surveys and campaigns tend to communicate information and engage people more effectively.

4. Conclusions

Public participation is included in the FD process to ensure the efficient implementation of the Directive but also to encourage public involvement and, therefore, improve flood awareness. Considering the completion of the FD’s first cycle, the aim of this paper is to shed light on lessons learned from the public participation process and provide recommendations for its improvement.
First, findings from both the interviews and the survey indicate the need for a more participatory approach in the public participation process. Instead of presenting measures that have been selected by the authorities, attendees at public consultation meetings could be given time to discuss the advantages and disadvantages of potential flood management options in smaller groups and to prioritize all suggested measures. Additionally, engagement in the process beyond the meetings could be enhanced by reporting the results of the public consultations with technical feedback on the reasons of including or excluding suggested measures.
Second, a better strategy is needed for announcing public consultation events and for increasing the attendance at these events. It could be advantageous to organize meetings and disseminate information after flood events since long dry periods can lead to people forgetting about floods and their impact. Furthermore, meetings could be carried out in flood hazard areas to address site-specific issues and measures. This would also reduce the length of public consultation meetings, which may then be more appealing for people to attend. Additionally, events could be held during non-working hours, which would enable a larger share of the public to attend. The use of non-technical language is also important for ensuring people’s understanding of what is presented and keeping their attention.
Third, surveys, proactive dissemination of information, and subsidies can increase awareness and the engagement of people. Interactive websites can be created to enable people to easily identify what areas are prone to flooding. Moreover, message boards that display the flood hazard area and introduce measures that can be implemented at a household level can be put up near flood hazard areas. Furthermore, promoting household-level flood protection measures through subsidies will increase the public’s awareness and their willingness to implement them.
The above suggestions require that authorities devote more time and effort in incorporating participatory processes and promoting public engagement. However, increased involvement of the public could also lead to the implementation of more household-level and more green measures, which may possibly reduce the cost of large infrastructure projects. This also relates to the need for establishing procedures for evaluating the effectiveness of the FD public participation process. Such procedures could improve the recognition of the importance of public participation and consultation and its subsequent optimization.

Author Contributions

K.C. and A.B. conceptualized the research. K.C. carried out the interviews with stakeholders and led the questionnaire survey in which she was assisted by all co-authors. E.G. carried out the statistical analysis. K.C. prepared the draft manuscript while the review and editing was completed by A.B., E.G. and C.Z.

Funding

The research has received financial support from the European Union Seventh Framework Program BEWATER Project (GA 612385).

Acknowledgments

The authors would like to thank the interviewed stakeholders especially Kostas Aristeidou from the Water Development Department for sharing knowledge and information on the public consultations for the Floods Directive. They would also like to acknowledge Aitor Herrero for his help in carrying out the questionnaire survey and thank all those who participated.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

Survey on Flood Management
Thank you for agreeing to fill out this questionnaire. Please circle which answers are applicable for you:
(1)
Do you live in an area affected by flooding?
YES/NO
(2)
Has your area been designated as a flood risk area for the EU Floods Directive?
YES/NO/I don’t know
(3)
Does the Flood Hazard Map (see banner), show that you live in a flood hazard area?
YES/NO/My house is not on the map
(4)
Are you concerned about flooding?
Not concerned/Moderately concerned/Greatly concerned
(5)
Were you aware of public consultation meetings carried out for the implementation of the EU Floods Directive?
Yes and I have attended
Yes, but I did not attend
No, I was not aware
(6)
Would you attend an information meeting on flooding?
YES/NO
(7)
Do you think the following household-level measures are effective in reducing floods in cities for countries like Cyprus?
Please rate from 1–5 (1 = strongly disagree, 5 = strongly agree)
Permeable pavement       1/2/3/4/5
Rainwater collection system   1/2/3/4/5
Vegetated roof         1/2/3/4/5
(8)
Do you think the government should subsidize these measures and by how much?
Permeable pavement       0%/25%/50%/75%/100%
Rainwater collection system   0%/25%/50%/75%/100%
Vegetated roof         0%/25%/50%/75%/100%
(9)
A. Do you own or plan on building a house: YES/NO
B. Would you consider implementing any of these measures?
Please rate from 1–5 (1 = highly unlikely 5 = highly likely)
Permeable pavement       1/2/3/4/5
Rainwater collection system   1/2/3/4/5
Vegetated roof         1/2/3/4/5
(10)
Would you be interested in getting more information about the benefits of these measures?
YES/NO
Gender:Male/FemaleEducation:Primary-Secondary/College-University
Age Group:18–30/31–60/61+Post Code:

References

  1. Challies, E.; Newig, J.; Thaler, T.; Kochskämper, E.; Levin-Keitel, M. Participatory and collaborative governance for sustainable flood risk management: An emerging research agenda. Environ. Sci. Policy 2016, 55, 275–280. [Google Scholar] [CrossRef] [Green Version]
  2. World Meteorological Organization. Social Aspects and Stakeholder Involvement in Integrated Flood Management. 2006. Available online: http://www.adpc.net/v2007/Resource/downloads/socialaspect13oct_2.pdf (accessed on 1 May 2017).
  3. Albrecht, J. Legal framework and criteria for effectively coordinating public participation under the Floods Directive and Water Framework Directive: European requirements and German transposition. Environ. Sci. Policy 2016, 55, 368–375. [Google Scholar] [CrossRef]
  4. Dillon, P.; Bellchambers, R.; Meyer, W.; Ellis, R. Community perspective on consultation on urban stormwater management: Lessons from Brownhill Creek, South Australia. Water 2016, 8, 170. [Google Scholar] [CrossRef]
  5. Buchecker, M.; Ogasa, D.; Maidl, E. How well do the wider public accept integrated flood risk management? An empirical study in two Swiss Alpine valleys. Environ. Sci. Policy 2016, 55, 309–317. [Google Scholar] [CrossRef]
  6. Bradford, R.; O’Sullivan, J.; van de Craats, I.; Krywkow, J.; Rotko, P.; Aaltonen, J.; Bonaiuto, M.; Dominicis, S.D.; Waylen, K.; Schelfaut, K. Risk perception—Issues for flood management in Europe. Nat. Hazards Earth Syst. Sci. 2012, 12, 2299–2309. [Google Scholar] [CrossRef]
  7. Mees, H.; Crabbé, A.; Alexander, M.; Kaufmann, M.; Bruzzone, S.; Lévy, L.; Lewandowski, J. Coproducing flood risk management through citizen involvement: Insights from cross-country comparison in Europe. Ecol. Soc. 2016, 21, 3. [Google Scholar] [CrossRef]
  8. Birchard, E.; Kazmierczak, A. Are homeowners willing to adapt to and mitigate the effects of climate change? Clim. Chang. 2012, 112, 633–654. [Google Scholar] [CrossRef]
  9. Owusu, S.; Wright, G.; Arthur, S. Public attitudes towards flooding and property-level flood protection measures. Nat. Hazards 2012, 77, 1963–1978. [Google Scholar] [CrossRef]
  10. Nye, M.; Tapsell, S.; Twigger-Ross, C. New social directions in UK flood risk management: Moving towards flood risk citizenship? J. Flood Risk Manag. 2011, 4, 288–297. [Google Scholar] [CrossRef]
  11. Maidl, E.; Buchecker, M. Raising risk preparedness by flood risk communication. Nat. Hazards Earth Syst. Sci. 2015, 15, 1577–1595. [Google Scholar] [CrossRef] [Green Version]
  12. Sörensen, J.; Persson, A.; Sternudd, C.; Aspegren, H.; Nilsson, J.; Nordström, J.; Jönsson, K.; Mottaghi, M.; Becker, P.; Pilesjö, P.; et al. Re-thinking urban flood management—Time for a regime shift. Water 2016, 6, 332. [Google Scholar] [CrossRef]
  13. Carter, T.; Fowler, L. Establishing green roof infrastructure through environmental policy instruments. Environ. Manag. 2008, 42, 151–164. [Google Scholar] [CrossRef] [PubMed]
  14. Domènech, L.; Saurí, D. A comparative appraisal of the use of rainwater harvesting in single and multi-family buildings of the Metropolitan Area of Barcelona (Spain): Social experience, drinking water savings and economic costs. J. Clean. Prod. 2011, 19, 598–608. [Google Scholar] [CrossRef]
  15. European Commission. Directive 2007/60/EC of the European Parliament and of the Council of 23 October 2007 on the Assessment and Management of Flood Risks. 2007. Available online: https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32007L0060&from=EN (accessed on 1 September 2016).
  16. European Commission. Common Implementation Strategy for the Water Framework Directive (2000/60/EC). 2003. Available online: http://ec.europa.eu/environment/water/water-framework/objectives/pdf/strategy2.pdf (accessed on 1 September 2016).
  17. European Commission. Links between the Floods Directive (FD, 2007/60/EC) and Water Framework Directive (WFD, 2000/60/EC). 2014. Available online: https://circabc.europa.eu/sd/a/124bcea7-2b7f-47a5-95c7-56e122652899/inks%20between%20the%20Floods%20Directive%20and%20Water%20Framework%20Directive%20-%20Resource%20Document (accessed on 1 September 2016).
  18. Hartmann, T.; Spit, T. Legitimizing differentiated flood protection levels—Consequences of the European flood risk management plan. Environ. Sci. Policy 2016, 55, 361–367. [Google Scholar] [CrossRef]
  19. Newig, J.; Challies, E.; Jager, N.; Kochskämper, E. What role for public participation in implementing the EU Floods Directive? A comparison with the Water Framework Directive, early evidence from Germany and a research agenda. Env. Pol. Gov. 2014, 24, 275–288. [Google Scholar] [CrossRef]
  20. Otto, A.; Hornberg, A.; Thieken, A. Local controversies of flood risk reduction measures in Germany. An explorative overview and recent insights. J. Flood Risk Manag. 2018, 11, S382–S394. [Google Scholar] [CrossRef]
  21. Hedelin, B. The EU floods directive in Sweden: Opportunities for integrated and participatory flood risk planning. J. Flood Risk Manag. 2017, 10, 226–237. [Google Scholar] [CrossRef]
  22. Levidow, L.; Neubauer, C. EU research agendas: Embedding what future? Sci. Cult. 2014, 23, 397–412. [Google Scholar] [CrossRef]
  23. Charalambous, K.; Bruggeman, A.; Bakirtzis, N.; Lange, M. Historical flooding of the Pedieos River in Nicosia, Cyprus. Water Hist. 2016, 8, 191–207. [Google Scholar] [CrossRef]
  24. Water Development Department. Programme of Measures for the Flood Risk Management Plan, River Basin of Cyprus (Period 2016–2021), Carried Out by LDK Consultants and ECOS. 2016. Available online: http://www.moa.gov.cy/moa/wdd/wdd.nsf/all/56EB6A9E19D6E1FBC2257F2B002DA93D/$file/Programma_Metron_Plimiron_Nov_16.pdf?openelement (accessed on 1 November 2016). (In Greek)
  25. Water Development Department. Report on the Results of the Public Consultation for the 1st Flood Risk Management Plan in Cyprus, Carried out by LDK Consultants and ECOS; Water Development Department: Nicosia, Cyprus, 2016.
  26. Water Development Department. European Directive 2007/60/EC and the Cyprus Legislation for Floods. Available online: http://www.moa.gov.cy/moa/wdd/wdd.nsf/all/23FC10198BC67AC5C2257FB70019986A?opendocument (accessed on 1 May 2017).
  27. Giannakis, E.; Bruggeman, A.; Poulou, D.; Zoumides, C.; Eliades, E. Linear parks along urban rivers: Perception of thermal comfort and climate change adaptation in Cyprus. Sustainability 2016, 8, 1023. [Google Scholar] [CrossRef]
  28. Water Development Department. Flood Risk Management Plan for Cyprus, Carried Out by LDK Consultants and ECOS. 2016. Available online: http://www.moa.gov.cy/moa/wdd/wdd.nsf/all/56EB6A9E19D6E1FBC2257F2B002DA93D/$file/Sxedio_Diaxirisis_Plimiron_Nov_2016.pdf?openelement (accessed on 1 November 2016). (In Greek)
  29. Blank, L.; Vasl, A.; Levy, S.; Grant, G.; Kadas, G.; Dafni, A.; Blaustein, L. Directions in green roof research: A bibliometric study. Build. Environ. 2013, 66, 23–28. [Google Scholar] [CrossRef]
  30. Getter, K.; Rowe, D. The role of extensive green roof in sustainable development. HortScience 2016, 41, 1276–1285. [Google Scholar]
  31. Trogrlić, R.Š.; Rijke, J.; Dolman, N.; Zevenbergen, C. Rebuild by design in Hoboken: A design competition as a means for achieving flood resilience of urban areas through the implementation of green infrastructure. Water 2018, 10, 553. [Google Scholar] [CrossRef]
  32. Permeable Pavement. Low Impact Development Center. Available online: http://www.lid-stormwater.net/permpaver_costs.htm (accessed on 25 October 2016).
  33. Daliko Ltd.; Nicosia, Cyprus. Personal Communication, 2016.
  34. Kim, H.; Li, M.; Kim, H.; Lee, H. Cost-benefit analysis and equitable cost allocation for a residential rainwater harvesting system in the city of Austin, Texas. Int. J. Water Resour. Dev. 2015, 32, 749–764. [Google Scholar] [CrossRef]
  35. Kostantinou, K.; VitaPlast Ltd., Nicosia, Cyprus. Personal Communication, 2016.
  36. Georgiou, E.; Landscape Architect, Nicosia, Cyprus; Orphanos, G.; Horticulturist, Nicosia, Cyprus. Personal Communication, 2016.
  37. Gaur, A.; Gaur, S. Statistical Methods for Practice and Research: A Guide to Data Analysis Using SPSS; SAGE Publications India Pvt Ltd.: New Delhi, India, 2009. [Google Scholar]
  38. SPSS Inc. SPSS 13.0 Brief Guide; SPSS Inc.: Chicago, IL, USA, 2004. [Google Scholar]
  39. Rözer, V.; Müller, M.; Bubeck, P.; Kienzler, S.; Thieken, A.; Pech, I.; Schröter, K.; Buchholz, O.; Kreibich, H. Coping with pluvial floods by private households. Water 2016, 8, 304. [Google Scholar] [CrossRef]
  40. Vávra, J.; Lapka, M.; Cudlínová, E.; Dvořáková-Líšková, Z. Local perception of floods in the Czech Republic and recent changes in state flood management strategies. J. Flood Risk Manag. 2017, 10, 238–252. [Google Scholar] [CrossRef]
  41. Charalambous, K.; Bruggeman, A.; Lange, M. Policies for Improving Water Security, the Case of Cyprus; CLICO Project Report (Contract No: SSH-CT-2010-244443); The Cyprus Institute: Nicosia, Cyprus, 2011; Available online: https://www.cyi.ac.cy/images/projects/eewrc/clico/CLICO_WP4_REPORT_Cyprus_Mar2011.pdf (accessed on 1 September 2016).
  42. Harries, T. The anticipated emotional consequences of adaptive behaviour—Impacts on the take-up of household flood-protection measures. Environ. Plan. A 2012, 44, 649–668. [Google Scholar] [CrossRef]
  43. Benson, D.; Fritsch, O.; Langstaff, L. Local flood risk management strategies in England: Patterns of application. J. Flood Risk Manag. 2016, 11, S827–S837. [Google Scholar] [CrossRef]
Figure 1. A Pedieos Park visitor identifies his house on the flood hazard map during the survey.
Figure 1. A Pedieos Park visitor identifies his house on the flood hazard map during the survey.
Water 10 00958 g001
Figure 2. Survey respondents’ perception of the effectiveness of the three household-level flood protection measures.
Figure 2. Survey respondents’ perception of the effectiveness of the three household-level flood protection measures.
Water 10 00958 g002
Figure 3. Survey respondents’ opinion on the percentage of the flood protection measures’ cost that should be subsidized.
Figure 3. Survey respondents’ opinion on the percentage of the flood protection measures’ cost that should be subsidized.
Water 10 00958 g003
Figure 4. Likelihood that survey respondents will implement the household-level flood protection measures.
Figure 4. Likelihood that survey respondents will implement the household-level flood protection measures.
Water 10 00958 g004
Table 1. Public participation activities carried out during the first cycle of the Floods Directive (FD) implementation in Cyprus (FRMP = Flood Risk Management Plan, PoM = Programme of Measures, RBMP = River Basin Management Plan).
Table 1. Public participation activities carried out during the first cycle of the Floods Directive (FD) implementation in Cyprus (FRMP = Flood Risk Management Plan, PoM = Programme of Measures, RBMP = River Basin Management Plan).
Date of EventTitle of EventContent
January 2012Presentation of the identified areas of potential significant flood riskIntroduction to FD, presentation of 19 flood risk areas and methodology
March 2013Presentation of preliminary flood hazard and risk mapsUpdate on FD implementation progress and preliminary maps
July 2014Presentation of flood hazard and risk mapsPresentation of maps and methodology
May 2015–April 2016Steering committee meetings on the FRMP and PoM (Total of 5 meetings)Drafting of FRMP and PoM, comments on reports and drafts, incorporation of suggestions in PoM
December 2015–February 2016Consultation period
Presentation of 1st FRMP and PoM
Preliminary FRMP and 7-page questionnaire available online for comments and suggestions by the public
January 2016District-level stakeholder consultation meetings on the 2nd RBMP and 1st FRMP and their PoMs (2 meetings, total attendance: 72 people)Presentation of the RBMP and FRMP and their PoMs, questions, and discussion
February 2016District-level public consultation meetings on the 2nd RBMP and 1st FRMP and their PoMs (2 meetings, total attendance: 76 people)Presentation of the RBMP and FRMP and their PoMs, questions, and discussion
February 2016Pancyprian public consultation on the 2nd RBMP and 1st FRMP and their PoMs (Total attendance: 43 people)Presentation of RBMP and FRMP and their PoMs (including amendments from prior district meetings)
Questions and discussion
Table 2. Survey respondents’ characteristics, past flood experience, and awareness of the Floods Directive.
Table 2. Survey respondents’ characteristics, past flood experience, and awareness of the Floods Directive.
Characteristics of the Survey Respondents
Distribution of Respondents%
Gender  
  Male54.5
  Female45.5
Age  
  18–4029.9
  41–6453.2
  65+16.9
Education  
  Primary–Secondary26.9
  College–University73.1
Respondents’ Flood Experience and Awareness of the Floods Directive
Do you live in an area affected by flood?
  Yes12.3
  No87.7
Is your area designated as a flood risk area?
  Yes2.0
  No21.2
  Don’t know76.7
Does the map show that you live in a designated flood hazard area?
  Yes18.2
  No58.6
  My house is not on the map23.3
Are you concerned about flooding?
  Not concerned46.5
  Moderately concerned46.3
  Greatly concerned7.2
Are you aware of the FD public consultation meetings?
  Yes, and attended2.6
  Yes, but did not attend2.6
  No, not aware93.6
Are you willing to attend an information meeting on floods?
  Yes39.6
  No60.4

Share and Cite

MDPI and ACS Style

Charalambous, K.; Bruggeman, A.; Giannakis, E.; Zoumides, C. Improving Public Participation Processes for the Floods Directive and Flood Awareness: Evidence from Cyprus. Water 2018, 10, 958. https://doi.org/10.3390/w10070958

AMA Style

Charalambous K, Bruggeman A, Giannakis E, Zoumides C. Improving Public Participation Processes for the Floods Directive and Flood Awareness: Evidence from Cyprus. Water. 2018; 10(7):958. https://doi.org/10.3390/w10070958

Chicago/Turabian Style

Charalambous, Katerina, Adriana Bruggeman, Elias Giannakis, and Christos Zoumides. 2018. "Improving Public Participation Processes for the Floods Directive and Flood Awareness: Evidence from Cyprus" Water 10, no. 7: 958. https://doi.org/10.3390/w10070958

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop