Next Article in Journal
Research on the Diffusion Model of Cable Corrosion Factors Based on Optimized BP Neural Network Algorithm
Previous Article in Journal
Examination of User Emotions and Task Performance in Indoor Space Design Using Mixed-Reality
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Buildings
Volume 13
Issue 6
10.3390/buildings13061484
buildings-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Evaluating the Impact of Public Participation Processes on Participants in Smart City Development: A Scoping Review

by
Coline Senior
*,
Alenka Temeljotov Salaj
,
Agnar Johansen
and
Jardar Lohne
Department of Civil and Environmental Engineering, Faculty of Engineering, Norwegian University of Science and Technology (NTNU), 7491 Trondheim, Norway
*
Author to whom correspondence should be addressed.
Buildings 2023, 13(6), 1484; https://doi.org/10.3390/buildings13061484
Submission received: 9 May 2023 / Revised: 1 June 2023 / Accepted: 2 June 2023 / Published: 8 June 2023
(This article belongs to the Topic Smart Cities: Infrastructure, Innovation, Technology, Governance and Citizenship)
Browse Figures
Versions Notes

Abstract

:
This paper provides an overview of the current state of research on the evaluation methods of participatory processes in smart cities. Specifically, it aims to identify and analyze existing evaluation methods and frameworks for public participation (PP) in smart city development. The study focuses on the evaluation of participatory processes to find key indicators and enable an assessment of PP from multiple perspectives. A scoping literature review was conducted to analyze the past ten years of scientific literature on the topic. Relevant literature was retrieved from Scopus, Web of Science, and Google Scholar, and articles were selected based on a set of criteria to ensure quality and relevance. We found that digital participation, also known as “e-participation”, is the most used method either exclusively or mixed with in-person methods. The level of participation achieved was mostly limited to the first two degrees of PP (i.e., information and consultation), and only a few papers addressed the highest degrees (i.e., agenda-setting and co-management). The impact on participants was mostly related to knowledge and skills, awareness raising, and satisfaction with the process and method. This paper highlights the potential to upskill citizens and enhance their understanding of sustainable urban development, fostering their commitment to achieving the United Nations’ sustainability goals for climate change mitigation in the urban context.

1. Introduction

There has been a substantial amount of literature produced on the concept of smart city since the term was first introduced in 1994, with a significant increase in attention following its adoption as a cornerstone of the European Union’s development strategy in 2010 [1,2]. The European Union continues to intensify its efforts and funding capacity for smart city projects. In total, the funding program Horizon Europe will invest 360 million euros in research and innovation actions linked to the “Climate-neutral and Smart cities” mission [3]. Smart cities rely on the deployment and use of information and communication technologies, not as an end in itself but as a means to foster sustainable urban development and enhance the quality of life for their citizens [4]. Thus, smart cities play a critical role in the pursuit of sustainable development; by leveraging technology, smart cities can enhance energy efficiency, reduce greenhouse gas emissions, improve public transportation, and promote sustainable economic growth and social development [5].
The development of smart cities is a complex process that requires the involvement and engagement of citizens to ensure sustainability and success. The United Nations Sustainable Development Goal (SDG) 11 emphasizes the importance of sustainable urbanization, which includes making cities inclusive, safe, resilient, and sustainable. To achieve this goal, cities must adopt participatory approaches that engage citizens in decision-making processes, such as those related to urban planning and transportation infrastructure. Similarly, SDG 13 focuses on climate action and the need to reduce greenhouse gas emissions. Citizen engagement and upskilling are essential in achieving this goal as well, as individuals need to be aware of their impact on the environment and to understand how they can reduce their carbon footprint [6,7]. Citizens’ lack of motivation to engage in participatory processes makes it challenging to recruit representative samples [8,9,10]. In addition, the choice of participation method determines which demographics are capable or not of engaging. At the end of the process, the results of public participation can be difficult to implement in the plans, especially if essential knowledge about the issue at stake is not sufficiently transferred to participants [11]. A change toward more sustainable behaviors is also necessary to achieve the goals of sustainable urban development [12]. Therefore, evaluating the effectiveness of participatory processes is crucial for understanding how to improve citizen participation in smart city projects. As the goals of public participation in smart cities go beyond the mere recruitment of a high number of participants and the collection of individual opinions, it is important to identify what citizens can get out of these processes rather than only what they contribute to them in order to create a common project that caters for the needs of all citizens today and in the future, ensuring the uptake of technologies and inspiring change toward sustainable behaviors.
This paper proposes a scoping literature review of the current state of research covering evaluation methods of public participation in smart cities. This review aims to identify existing frameworks and indicators used in the scientific literature to analyze not only the outcomes but also the extent to which these consider what citizens gain in the process. We will address the following research questions:
RQ1. What research has been carried out on the evaluation of public participation methods in smart cities?
RQ2. In which ways do the public participation processes impact the participants?
The findings from this review will be used to develop an approach for upskilling citizens in smart city projects, specifically in topics related to sustainability and smart cities. This is directly connected to achieving the United Nations’ sustainability goals for climate change mitigation in the built environment.
The study uses a scoping literature review analysis of specific elements in the evaluation methods of public participation, focusing on the impact on participants. The past ten years of scientific literature on the topic are reviewed, and methods and sets of indicators are analyzed to identify possible research gaps and potential for further research. Google Scholar, Web of Science, and Scopus were used to search the literature, and the selection was based on a set of criteria to ensure the scientific quality of each source. Only papers written in English and published in peer-reviewed journals were further analyzed.
We first provide an overview of the importance of participatory processes in the development of smart cities. We then proceed to describe the methodology used in the scoping literature review, including the search strategy and selection criteria. The findings are presented, including an analysis of the concentration of the literature on European smart cities, the level of public participation, and the impact on participants covered. The paper concludes by discussing the implications of the findings, including the potential for upskilling citizens in sustainability topics, and proposing avenues for further research.
The study provides a valuable contribution to the literature by identifying research gaps and the potential for further research in the evaluation of public participation in the specific context of smart city development. The originality of the study is its focus on the participants themselves rather than the outcomes of participatory processes. Additionally, the study offers practical implications for policymakers and stakeholders involved in the development of smart cities by demonstrating the importance of citizen engagement and the potential for upskilling citizens in sustainability topics.

2. Public Participation in Smart Cities

Public participation is a broad and multi-dimensional concept that has been discussed extensively in various fields of research, including urban planning, public policy, and environmental studies [13,14,15,16]. According to Arnstein (1969) [17], public participation is a process that allows citizens to actively engage in decision-making processes and to have a genuine impact on the outcomes of those processes. This definition emphasizes the importance of power dynamics between the government and citizens, with public participation serving as a means of empowering citizens and ensuring that their interests are taken into account. Several scholars have proposed typologies or models to conceptualize the different levels and types of public participation. For instance, Pretty (1995) [18] proposed a ladder of citizen participation which ranges from “manipulative participation” to “interactive participation and self-mobilization”, with each rung representing a different level of citizen involvement and influence in the decision-making processes. Rowe and Frewer (2000) identified three levels of public participation: passive, where citizens receive information but do not provide feedback; consultative, where citizens provide feedback but do not have decision-making power; and collaborative, where citizens and decision-makers work together to create a mutually beneficial solution. More recently, Ringholm, Nyseth, and Sandkjær Hanssen (2018) [19] presented a model in which the legal requirements are the starting point for assessing the level of participation, thus catering for the specific context in which the participatory process is implemented. In the context of urban planning and development, public participation is increasingly being recognized as an essential component of sustainable urban governance [20,21,22]. In recent years, there has also been growing interest in public participation in smart cities, in which information and communication technologies are used to enhance the quality of life for citizens [23,24,25]. This emerging literature on smart cities emphasizes a citizen-centric approach rather than solely focusing on technology [22,24,26]. In this context, a smart city can be defined as a space where local governments, citizens, and various stakeholders collaborate on initiatives aimed at improving overall living conditions [27,28]. By recognizing the importance of involving citizens in decision-making processes, public participation processes can be initiated as a means to minimize conflicts and objections to planning proposals. Given that citizens are the primary beneficiaries of smart city services and environments, it is argued that their involvement should be integral to the development process [23,25,29]. In smart cities, public participation requires a new approach that takes into account their unique features, such as the use of technology and data-driven decision-making [23,30]. This new governance approach should also be more flexible and adaptable than traditional urban governance models to make room for integrating knowledge-sharing perspectives and developing organizational learning capabilities [10,31].
In smart cities, the potential for technology to increase citizen engagement and participation in decision-making processes is often referred to as “e-participation”. E-participation refers to the use of digital technologies to facilitate citizen participation in government decision-making processes. Smart cities are often seen as ideal contexts for e-participation due to their advanced technological infrastructure and the availability of data [32].
By engaging citizens in decision-making processes, governments and other stakeholders can ensure that their policies and projects are aligned with citizens’ needs and values, which can improve the effectiveness and legitimacy of governance [33]. However, the effectiveness of public participation can be limited by various factors, including power imbalances, lack of resources and skills among citizens, and inadequate communication between stakeholders [13,34].
Therefore, it is essential to understand the various dimensions and levels of how public participation is evaluated as well as the factors that enable or hinder its effectiveness in order to design and implement participatory processes that are inclusive, transparent, and effective.

3. Methods

3.1. Research Design and Search Process

This study uses the scoping review methodology to give an overview of a relatively recent field, namely, the evaluation of public participation in smart cities. The flexibility of this method compared with a systematic literature review makes it especially relevant to conducting an exploratory study [35]. The scoping review method was preferred to a systematic one as it helps clarify the context, inquiries, and the extent of the available evidence. By pinpointing areas where evidence is insufficient, scoping reviews can assist sponsors and researchers in prioritizing their limited resources to address the most pressing questions to be further researched and represent a stepping stone toward a systematic review or an empirical study [36]. This scoping review followed the five stages of the methodological framework for scoping studies [37]. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) flow chart (Figure 1) illustrates the preliminary steps to arrive at the final sample to be examined in the study [35]. PRISMA-ScR is an adapted version of the original twelve points of the PRISMA method [38,39]. After defining the research question (as described in Section 1), we identified relevant studies using the following databases and search engines: Scopus, Web of Science, and Google scholar. As the field is rather narrow, the latter was used as an additional tool to expand the sample to include studies that might not have appeared in Scopus and Web of Science due to the search on “Title-Abstract-Keywords” in these databases. Then we selected the studies, extracted the data, and compiled and reported the results. The following section describes each step of the scoping review process. The initial search string for “Evaluation of public participation” AND “Smart cities” was extended to a parallel search on “Evaluation of citizen engagement” AND “Smart cities” to retrieve publications that focused on evaluating the methods used to engage citizens and not only the public participation process itself.
The PRISMA-ScR guidelines were followed in the current study for conducting the scoping literature review, as shown in Figure 1. The following 12 items (Table 1) for conducting the review process were addressed, with the exception of items 9, 11, and 12 as they are not applicable for scoping reviews [35].

3.2. Selecting Studies

The inclusion criteria were as follows:
  • The topic is Smart cities and Public participation and/or Citizen engagement, and it mentions its evaluation or a dimension thereof.
  • The study should include criteria and/or indicators to evaluate public participation or citizen engagement in smart cities.
  • Articles are written in English and accessible to authors.
Exclusion criteria:
  • Editorials and perspectives, conference proceedings, theses, project reports.
  • Articles written in languages other than English.
Some studies were kept in the next step (i.e., charting the data) despite not primarily aiming to evaluate public participation or citizen engagement in smart cities, as some of their data addressed it as one of the dimensions to consider in the overall evaluation of smart cities.

3.3. Charting the Data

Three types of categories were defined:
  • General information
  • Core data
  • Definitions
The “general information” included the sub-categories Authors, First author’s country, Title of article or book chapter, Publication year, Journal or Book, Location of the study, Authors’ keywords, Purpose of the study, Methods, Results, and Implications. Information regarding the first author’s country and location of the study was collected to map the concentration of scientific production by world region and to identify the “hot spots”. Purpose, Method, Results, and Implications were directly extracted from the abstracts and were used to assess the relevance of the study. This category was used to obtain a general overview and identify characteristics of the scientific literature connected to the research questions. At this stage, the full texts were read, and twelve articles and seven books were excluded from the review due to poor quality of writing, not meeting the inclusion criteria, or falling outside of the scope of this review.
The “core data” included “Participation method”, “Purpose of the participation method/process”, “Evaluation method”, “What is measured”, “How it is measured”, and “What is missing”. These sub-categories were defined to address the scope of the research questions (i.e., public participation in smart cities and how it is evaluated).
The category “Definitions” collected each paper’s definition of smart city, public participation, and/or citizen engagement. The collected data were used to build a common understanding of the research field among authors and to define the theoretical framework of this scoping literature review.

3.4. Data Analysis

The extracted data were then analyzed by the authors, following the aforementioned categories, and visualized with the help of selected graphic charts. A refined coding step was introduced to ease the further analysis of the data, and the level of participation achieved was introduced during this phase.
Ringholm, Nyseth and Sandkjær Hanssen (2018) [19] defined five levels of participation to assess participatory processes (Table 2). These levels are adapted from Sherry Arnstein’s ladder of participation [17].
In this scoping literature review, the levels were used to categorize the studies based on the degree of participation achieved in the participatory processes. The main advantage of using the adapted model is that it is a simplified version with practical implications for assessing the level of participation according to national regulations.
When more “mixed” or “combined approaches” were identified in the literature which could not be classified within those levels, the studies were marked with a “*”. These included multipurpose studies that did not focus on a specific process or method, but proposed evaluation models based on theoretical evidence. These will be discussed as another cluster of papers in the next section.

Methodological Limitations

One of the limitations of this scoping review is that the categories for data collection and further analysis were defined by the authors themselves and do not follow a specific framework. This approach, called grounded theory, is based on the idea that theory should emerge from data, rather than being imposed on it. Glaser and Strauss (1967) [40] argue that the researcher should immerse themselves in the data, constantly comparing and contrasting different pieces of information to identify patterns, relationships, and concepts. Through this process, the researcher can generate theories that are grounded in the data and are thus more likely to be relevant and applicable to the real world.
Another methodological limitation is the exclusion of gray literature and conference proceedings in the examined sample, limiting the scope of this literature review to peer-reviewed articles and books.

4. Results

4.1. Overview of the Sampled Literature

In total, 35 papers were identified, and data were collected into the categories previously described in Section 3.3. Even though the initial search included the period 2010–2022, relevant literature was only found in the period 2015–2022 (Figure 2). The first authors are primarily affiliated with the Netherlands (n = 6), Italy (n = 4), and Germany (n = 4) (Figure 3). European cities are over-represented (78%) in the examined papers compared with other world regions. The sampled literature is rather evenly spread across journals with only three journals counting more than one article (Figure 4).

4.2. Choice of Research Methods

Qualitative research methods dominate in the sampled literature (68%), followed by mixed-methods studies (25%). Only a few studies used quantitative methods (9%), while the book included in the sample [41] is unclassified as the majority of the chapters do not present a clear method. The sampled literature presents mostly case studies (54%), followed by literature reviews (31%).
The qualitative studies ranged from 2 [42] to 50 [43] participants, with workshops and interviews being the most-used methods. Studies using survey questionnaires in combination with other methods presented samples ranging between 16 [44] and 328 [45] respondents. The only study using a solely quantitative method analyzed a sample of 309 city policies [46]. Two pilot studies were also identified in the sampled literature. In the pilot study by Ambrose (2020) [47], 15 self-selected citizens took part in a one-time event. Paskaleva and Cooper (2018) [48] presented a framework first developed together with 25 participants and then pilot-tested in 6 cities without documenting the total number of participants in each of them.
Literature reviews range from 16 [49] to 114 [50] analyzed papers. In addition, four studies presented a literature review as part of a conceptual paper [51,52,53,54].

4.3. Participation Method and Evaluation

This section aims to classify the papers according to the participation method used in the studies, the level of participation, the type of impact on participants, and the specific attributes associated with it.
This section describes the sampled literature to address the research question of how public participation is evaluated in smart cities. First, we give an overview of the type of participation methods used in the studies (Figure 5), then the level of participation achieved, the impacts on participants that were in focus, and the evaluation indicators.
Digital participation, also known as “e-participation”, was the most commonly represented method in the sampled literature, either exclusively (n = 12) or mixed with in-person methods (n = 11). Twelve of these studies that focused on e-participation reported on the use of visualization tools and methods (e.g., planning support systems, 3D models, data visualization apps).

4.3.1. Level of Participation

The majority of the papers (n = 18) included in this scoping review addressed the first two degrees of citizen participation, namely, “information” and “consultation”. Only a few (n = 5) addressed the highest degree of citizen participation (i.e., “co-management”); these will be presented in more detail at the end of this section (Figure 6). Some of the studies did not refer to a specific case or participatory process but proposed new methodologies for the evaluation of public participation (n = 8). The latter are not included in the figure below but are described at the end of the section.

Information Only

Three studies referred solely to the “information” level of public participation [44,46,47]. In these papers, the goal of participation was to inform people and ultimately induce a change in behavior.

Consultation Only

The primary focus of four studies addressing only “consultation” was on collecting input from citizens to inform the planning process [42,54,57,58]. The others aimed to induce a change of a behavior [56], to initiate a social-ecological transformation [55], and to co-create public value for and with citizens [48,54].

Information and Consultation

Eight papers addressed both “information” and “consultation” [1,43,50,53,59,61,62]. Public participation in these studies had a primary goal of informing the urban-planning process. In addition, [60] also included a goal of improving public service delivery.

Dialogue

One of the studies referring to “dialogue” [45] describes the design and evaluation of a prototype computational model that fuels audience engagement and collaboration of cultural organizations. The authors of this paper consulted cultural stakeholder groups (e.g., journalists, artists, art lovers, and representatives of art institutions). Another study by Mcevoy et al. (2020) [63] presents a research approach to evaluate the role of a collaborative planning support tool in an urban-planning process. The authors used a case-study approach to examine the role of the tool through a conceptual framework of analysis that focused on the context, use, and impacts of the tool. They found that the tool supported dialogue and improved communication.

Agenda Setting

Two papers addressed “agenda setting” in addition to “information” and “consultation” [64,65]. In these papers, the authors focused on processes that involved citizens in their co-design in order to improve smart city services [64] and to increase their sense of ownership and to improve their quality of life and social cohesion [65].

Co-Management

Only five studies focused on the highest degree of citizen participation— “Co-management”—and included dimensions of training in different domains. The common goal of these processes was to upskill citizens to empower them to co-manage the planning process.
Russo et al. (2020) [68] aimed to construct collective knowledge, taking a further step into co-management by training citizens to develop new skills and raising aspiration and inspiration, thereby creating a collaborative circle among all partners. Afzalan and Muller (2018) [66] proposed building community consensus by cultivating trust and mobilizing citizen action and generating support. Zhong (2022) [70] describes a collaborative process in which different groups (district, neighborhood, community) and citizens were not only informed, consulted, and trained but also empowered to undertake parts of the reconstruction work themselves. Flacke et al. (2020) [67] introduced a process of public dialogue, social learning, and citizen training using an interactive collaborative tool to facilitate active participation and co-management to achieve policy goals.
Yoo (2021) [69] presents a case study of the Seoul smart city portal, whose main goals are participation, communication, cooperation, and urban problem-solving. Training citizens is discussed in terms of promoting creativity to foster technological vocations in children, actively training people in digital skills for future employment, and providing social, intergenerational, territorial, and gender-inclusion training. Citizens make suggestions regarding projects, attend events, and receive training on new technologies in smart cities. The portal also includes a ‘Smart Citizen Lab’ section to recruit citizens as experts to participate in finding solutions to urban problems. In addition, citizens can be integrated into evaluation teams for new technology product alternatives and participate in tutoring programs.

Multipurpose Studies

In addition to the above, eight multipurpose studies were identified. These were not focused on a specific participatory process or method but proposed evaluation models based on theoretical evidence [41,49,51,52,71,72,73,74].
In their studies, Foth (2016) [52] and Foth (2017) [51] discuss the concept of participation, co-creation, and public space in urban interaction design. The author introduces five possible ways to think about the “urban user” and the implications that follow, including the user as a city resident; the user as a consumer of city services; the user as a participant in the city’s community consultations; the user as co-creator in a collaborative approach to city-making; and the user re-thought as part of a much larger and more complex ecosystem of more-than-human worlds and cohabitation. The article by Clemente, Civierob and Celluralea (2019) [71] focuses on the transition of urban districts to Positive Energy Districts (PEDs). The authors developed a framework to facilitate the evaluation of the positioning and improvement of each smart city solution considered in the study. The framework identified strategies and stakeholders’ commitment to promote Smart Urban District or PED transitions, and identified implementation domains for PED transition and products and solutions. Ben (2020) [72] focuses on understanding how Internet of Things (IoT) technologies used to deliver services affect the working and living logistics of citizens. She conceptualizes a participatory approach to the design of smart city services, in which citizens are engaged from the start to implementation. Flacke, Shrestha, and Aguilar (2020) [49] reviewed the literature to discuss the use of interactive Participatory Sensing Systems (PSS) applications, tools, and participatory processes in order to analyze the impacts of these systems on participation. The results of the study showed that these systems can be used to increase community engagement, create a shared understanding of the issues under investigation, and allow participants to work through their interests. The article by Mohseni (2021) [73] focuses on the concept of smart cities and how they can potentially empower citizens. Based on a categorization of the available smart city definitions, the author presents a model which classifies smart cities based on their potential to empower citizens. The key factors influencing the improvement of the quality of smart urban life are explored by Reza, Bigdeli and Moinifar (2022) [74]. The results of the study indicate that the integration of dimensions such as health and hygiene, public education, security, social coherence and participation, consumption of resources and the environment, culture, and housing and buildings with the development of smart information and communications technology is the most significant contributory factor in improving the quality of smart urban life. The book written by Chilvers (2015) [41] explores how participation can be reconfigured and remade, particularly from a co-productionist standpoint. It examines the situated pragmatics of participation, considering how a co-productionist standpoint offers a different way of imagining participation and how it alters notions of reflexivity, learning, and critique.

4.3.2. Evaluation Method

The majority of the studies used a qualitative approach to evaluate public participation (n = 16). These are followed by mixed methods (n = 9) and quantitative methods (n = 7) (Figure 7). Three of the papers did not include a clear evaluation method (N/A) as these were providing reflections on the purpose of public participation.

4.3.3. Impact on Participants

This section focuses on the purpose of evaluation, looking for indicators to assess the impact of public participation processes on the participants in the studies. “Impact on participants” encompasses what participants get out of the process (e.g., gained awareness, behavioral change, likeliness of future engagement). The main dimensions identified in these studies were “awareness”, “change of behavior”, “knowledge and skills”, “participants’ satisfaction”, and “participants’ attitudes” (Table 3).
The other papers discussed different dimensions of public participation, for example, the level of participation, the usability of digital platforms, expectations toward the process, outcomes of the process, and the level of interactivity between municipalities and citizens.
“Satisfaction” is mentioned in six of the studies. Here there is a difference between those examining the participants’ satisfaction with the participatory process [49,53,70], the choice of method for participation [62], and the general public service delivery [56,79].
“Awareness” is also mentioned in six articles; five of them consider it as a dimension to monitor dynamically throughout the participation process [47,49,58,65,67], while the last one takes it as a starting point to assess the pre-existing awareness of the population on the risks of flood in the city [59]. This is justified by the focus of the study, which aims to evaluate a city’s preparedness to face environmental risks related to climate change.
Out of 35 papers, only few studies were classified at the highest level of participation, namely, “Dialogue”, “Agenda setting”, and “Co-management” (Figure 6). The majority of the papers (65%) cover the lowest levels of participation (i.e., “Information” and “Consultation”). We find that awareness and behavioral change are focused upon at the information level; awareness, satisfaction, and attitudes at the consultation level; future engagement, knowledge, skills, and learning and satisfaction for those combining information and consultation levels. We also observe that within studies mentioning the highest level of participation, “Co-management”, four of five papers discuss the impact on participants, but only one, the study by Flacke et al. (2020), mentions more than one dimension: awareness, future engagement, knowledge, skills, and learning and satisfaction.

5. Discussion

In this paper we set out to address the following two research questions: RQ1—What research has been carried out on the evaluation of public participation methods in smart cities? RQ2—In which ways do the public participation processes impact the participants? With these questions, we aim to identify evaluation frameworks that could provide indicators to assess whether the public participation processes succeed in empowering citizens and inspire a shift toward more sustainable behaviors, upskilling them and enhancing their understanding of sustainable urban development and fostering their commitment to achieving the United Nations’ sustainability goals for climate-change mitigation in the urban context.
Most of the research that has been carried out regarding the evaluation of public participation in smart cities is concentrated on European cases, which can be understood by the intensified funding efforts of the European Union into research on smart cities. The predominant research methodology to evaluate public participation is qualitative approaches. Our review of the scientific literature on the evaluation of public participation in smart cities identified several limitations. Most of the studies did not actually measure the learning progress or outcomes achieved through public participation, and even though citizens were engaged, they were more informed and consulted than empowered. There is a lack of clear evaluation criteria and indicators, with evaluations mostly conducted through discussions and interviews with researchers without providing evidence of the implementation of the proposed evaluation frameworks in practice. Only one article presented how they measured learning outcomes for the citizens [67]. They showed that improved learning led to the results of the participation process being easily implemented as they were directly related to the needs and challenges of the urban area. Some studies only considered specific public participation methods, such as public displays, and planning support systems (PSS), limiting the applicability of their findings to other methods. Lastly, some studies proposed frameworks for the co-evaluation of smart city services or governance capacity in general, but with public participation being only one of the dimensions and not the main focus. Some of the studies broadened the traditional sampled population to involve artists, activists, older citizens, and NGOs, but we were not able to identify studies involving the next generation of smart citizens, i.e., children and youth.
This section is structured in a way that compares the level of participation achieved (from information to co-management) with the impact on the participants considered in the studies (awareness, behavioral change, future engagement, knowledge, skills and learning, satisfaction and attitudes). We seek to identify whether the level of participation that can be achieved relates to different impacts on participants considered.
Knowledge, skills, and learning are found at various levels of participation, both at the basic level of information and consultation [1,50,53], at the intermediate level of dialogue [63], and at the highest level of co-management [67,68,69]. Citizen empowerment is the common thread among the studies at the co-management level, as they utilize digital tools to not only empower citizens during the participatory process but also to enable them to acquire new knowledge and skills for future use. In the case of PSS tools and game simulations, these are not presented as “stand-alone” solutions as they are used during supervised sessions. Those tools allowed the improvement of the citizens’ understanding of the issue at stake but were not developed to be intuitively used by citizens alone as they required the supervision of planners or other professionals. The digital platform presented by Yoo (2021) [69] supports the education and training of citizens in topics directly relevant to the implementation of smart city projects. In this case, citizens are not mere recipients of new technologies but take an active role in developing and implementing new solutions using the resources made available by the municipality.
Focusing solely on giving sufficient knowledge about the project and the process to participants will only allow them to reach the level of information and consultation. To reach the “Dialogue” level, the use of digital tools can help to facilitate the communication and collaboration between different stakeholders [63]. They found that the PSS tool played an important role in facilitating learning through the improved input and process. Flacke et al. (2020) [67] takes a step forward toward co-management by using a PSS tool for public participation that enables participants to understand the complexity of the planning process and to take an active role in it. They presented the benefits of communicating complex matters through visualization in order for citizens to understand the context and challenges related to a specific urban area. The tool enabled citizens to share their local knowledge and helped the planners to identify the most relevant locations for the projects. In the case of Russo et al. (2020) [68], the knowledge acquired by participants through gaming simulations allowed those with minimal knowledge to “go up and carry out quality projects” in city logistics.
The “knowledge and skills” dimension is the most represented one in the sampled literature. This indicates its importance in public participation in smart cities. However, most studies provide only recommendations and only a few give actual measurements or concrete methods for assessing increased or gained knowledge. The study by Yoo (2021) presents a case in which citizens can self-monitor their progress by keeping track of the training they have received on the digital platform. In their study, Hofmann et al. (2020) propose comparing perceived with actual knowledge gains in the process, which also relies on the participants’ self-assessment and reflection. Overall, the knowledge and skills that participants gain during the participatory processes seem to be difficult to assess and insufficiently documented, despite being one of the dimensions to be accounted for in the effectiveness of public participation [34,80].
The scope of this review was limited to peer-reviewed articles, and gray literature and conference proceedings were excluded from the final sample. More recent studies, especially those conducted during the COVID-19 pandemic, are not represented due to the time-period limitation and the longer publishing time in certain journals. In addition, within the sampled literature, emerging countries are not sufficiently represented, which limits the practical implications of our scoping review mostly to developed countries. This is especially important to point out as the uptake of technologies is slower and the culture for public participation in planning is generally lower in these countries, and these can be, to some extent, tokenized by the political institutions [81,82]. These aspects should be factored in when looking to evaluate public participation processes in smart cities, especially those involving the use of digital technologies to encourage a shift toward more sustainable behaviors. In the context of emerging countries, the sole implementation of ICTs will not be sufficient if it does not go hand-in-hand with improved quality of life and access to satisfactory infrastructures and services [82].

6. Conclusions

This scoping literature review revealed the scarcity of scientific production on the evaluation of public participation in smart cities in general and a high representation of European smart cities. There is an underrepresentation of other world regions in the identified studies.
Within this body of literature, only a few studies presented a citizen-centric evaluation. Despite having identified some common criteria focused on the impact on participants, only a few studies provided clear indicators to be measured. More tools for actively involving citizens in decision-making processes in smart city projects are needed.
The main methodological limitation of the present scoping literature review is the exclusion of gray literature and conference proceedings. One of the resulting shortcomings is the absence of studies conducted during the COVID-19 pandemic period, during which e-services and e-participation methods developed rapidly in response to the global restrictions on physical interactions. More work is needed to understand how public participation is evaluated in practice beyond the conceptual frameworks developed in the scientific literature. Gamification and the use of serious games are gaining momentum in the literature on public participation, with studies demonstrating their potential to improve processes by breaking down the complexity of smart city development projects and engaging the younger public that tend to be underrepresented in traditional processes [83,84,85,86].
Furthermore, the influence of the COVID-19 pandemic on public participation processes should be investigated to reveal how the communication and knowledge transfer operated during this period.
This scoping review provides a valuable contribution to the literature by identifying research gaps and potential for further research in the evaluation of public participation in the specific context of smart city development. The predominant research methodology to evaluate public participation is based on qualitative approaches, and most studies do not provide a method for measuring the learning progress or outcomes achieved during the participatory process.
To address these limitations, further research is needed to investigate other world regions and to measure the learning progress and outcomes of public participation. Additionally, more empirical studies need to be conducted to reveal the most effective methods and tools to foster a positive culture for participation in communities and to improve their knowledge and understanding of sustainability issues in the urban context.

Author Contributions

Conceptualization, C.S. and A.T.S.; methodology, J.L.; validation, C.S., A.T.S., A.J. and J.L.; formal analysis, C.S. and A.T.S.; investigation C.S.; resources, C.S.; writing—original draft preparation, C.S. and A.T.S.; writing—review and editing, C.S., A.T.S., A.J. and J.L.; visualization, C.S.; supervision, A.T.S. and A.J.; funding acquisition, A.J. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by Nordforsk, research project “CaPs-Citizens as Pilots of smart cities”, grant number 95576.

Data Availability Statement

Data sharing is not applicable.

Acknowledgments

This study is supported by the Department of Civil and Environmental Engineering, Faculty of Engineering, Norwegian University of Science and Technology (NTNU).

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Mangnus, A.C.; Vervoort, J.M.; Renger, W.; Nakic, V.; Rebel, K.T.; Driessen, P.P.J.; Hajer, M. Envisioning Alternatives in Pre-Structured Urban Sustainability Transformations: Too Late to Change the Future? Cities 2022, 120, 103466. [Google Scholar] [CrossRef]
  2. Collins, D.; Johansen, A.; Kalsaas, B.T.; Temeljotov-Salaj, A.; Hamdy, M. Brought by Degrees: A Focus on the Current Indicators of Lean ‘Smartness’ in Smart Cities. In Proceedings of the 29th Annual Conference of the International Group for Lean Construction (IGLC), Lima, Peru, 14–17 July 2021; pp. 167–176. [Google Scholar] [CrossRef]
  3. European Commission Climate-Neutral and Smart Cities. Available online: https://research-and-innovation.ec.europa.eu/funding/funding-opportunities/funding-programmes-and-open-calls/horizon-europe/eu-missions-horizon-europe/climate-neutral-and-smart-cities_en (accessed on 14 April 2023).
  4. Caragliu, A.; del Bo, C.; Nijkamp, P. Smart Cities in Europe. J. Urban Technol. 2011, 18, 65–82. [Google Scholar] [CrossRef]
  5. Schaffers, H.; Komninos, N.; Pallot, M.; Aguas, M.; Almirall, E.; Bakici, T.; Barroca, J.; Carter, D.; Corriou, M.; Fernadez, J. Smart Cities as Innovation Ecosystems Sustained by the Future Internet; HAL: Lyon, France, 2012. [Google Scholar]
  6. Preston, S.; Mazhar, M.U.; Bull, R. Citizen Engagement for Co-Creating Low Carbon Smart Cities: Practical Lessons from Nottingham City Council in the UK. Energies 2020, 13, 6615. [Google Scholar] [CrossRef]
  7. Schleicher, K.; Schmidt, C. Citizen Science in Germany as Research and Sustainability Education: Analysis of the Main Forms and Foci and Its Relation to the Sustainable Development Goals. Sustainability 2020, 12, 6044. [Google Scholar] [CrossRef]
  8. Temeljotov Salaj, A.; Gohari, S.; Senior, C.; Xue, Y.; Lindkvist, C. An Interactive Tool for Citizens’ Involvement in the Sustainable Regeneration. Facilities 2020, 38, 859–870. [Google Scholar] [CrossRef]
  9. Jowkar, M.; Temeljotov-Salaj, A.; Lindkvist, C.M.; Støre-Valen, M. Sustainable Building Renovation in Residential Buildings: Barriers and Potential Motivations in Norwegian Culture. Constr. Manag. Econ. 2022, 40, 161–172. [Google Scholar] [CrossRef]
  10. Lim, S.B.; Yigitcanlar, T. Participatory Governance of Smart Cities: Insights from e-Participation of Putrajaya and Petaling Jaya, Malaysia. Smart Cities 2022, 5, 71–89. [Google Scholar] [CrossRef]
  11. Chen, F.-H. Assessing Sustainable Development Initiatives in Central Taiwan Science Park: A Study of Residents’ Opinions and the Impact on the Urban Ecosystem. Buildings 2023, 13, 1202. [Google Scholar] [CrossRef]
  12. Khansari, N.; Mostashari, A.; Mansouri, M. Impacting Sustainable Behavior and Planning in Smart City. Int. J. Sustain. Land Use Urban Plan. 2014, 1, 46–61. [Google Scholar] [CrossRef]
  13. Innes, J.E.; Booher, D.E. Reframing Public Participation: Strategies for the 21st Century. Plan. Theory Pract. 2007, 5, 419–436. [Google Scholar] [CrossRef] [Green Version]
  14. Rowe, G.; Frewer, L.J. Public Participation Methods: A Framework for Evaluation. Sci. Technol. Hum. Values 2000, 25, 3–29. [Google Scholar] [CrossRef]
  15. Reed, M.S. Stakeholder Participation for Environmental Management: A Literature Review. Biol. Conserv. 2008, 141, 2417–2431. [Google Scholar] [CrossRef]
  16. Bovaird, T. Beyond Engagement and Participation: User and Community Coproduction of Public Services. Public Adm. Rev. 2007, 67, 846–860. [Google Scholar] [CrossRef]
  17. Arnstein, S.R. A Ladder Of Citizen Participation. J. Am. Inst. Plan. 1969, 35, 216–224. [Google Scholar] [CrossRef] [Green Version]
  18. Pretty, J.N. Participatory Learning for Sustainable Agriculture. World Dev. 1995, 23, 1247–1263. [Google Scholar] [CrossRef]
  19. Ringholm, T.; Nyseth, T.; Sandkjær Hanssen, G. Participation According to the Law? Eur. J. Spat. Dev. 2018, 67, 1–20. [Google Scholar] [CrossRef]
  20. Gordon, E.; Schirra, S.; Hollander, J. Immersive Planning: A Conceptual Model for Designing Public Participation with New Technologies. Environ. Plan. B Urban Anal. City Sci. 2011, 38, 505–520. [Google Scholar] [CrossRef]
  21. Degbelo, A.; Granell, C.; Trilles, S.; Bhattacharya, D.; Casteleyn, S.; Kray, C. Opening up Smart Cities: Citizen-Centric Challenges and Opportunities from GIScience. ISPRS Int. J. Geo-Inf. 2016, 5, 16. [Google Scholar] [CrossRef] [Green Version]
  22. Gohari, S.; Baer, D.; Nielsen, B.F.; Gilcher, E.; Situmorang, W.Z. Prevailing Approaches and Practices of Citizen Participation in Smart City Projects: Lessons from Trondheim, Norway. Infrastructures 2020, 5, 36. [Google Scholar] [CrossRef] [Green Version]
  23. Berntzen, L.; Johannessen, M.R. The Role of Citizen Participation in Municipal Smart City Projects: Lessons Learned from Norway. In Public Administration and Information Technology; Gil-Garcia, J.R., Pardo, T.A., Nam, T., Eds.; Springer: Berlin/Heidelberg, Germany, 2016; Volume 11, pp. 299–314. [Google Scholar]
  24. Simonofski, A.; Vallé, T.; Serral, E.; Wautelet, Y. Investigating Context Factors in Citizen Participation Strategies: A Comparative Analysis of Swedish and Belgian Smart Cities. Int. J. Inf. Manag. 2019, 56, 102011. [Google Scholar] [CrossRef]
  25. Goodman, N.; Zwick, A.; Spicer, Z.; Carlsen, N. Public Engagement in Smart City Development: Lessons from Communities in Canada’s Smart City Challenge. Can. Geogr. 2020, 64, 416–432. [Google Scholar] [CrossRef]
  26. Hollands, R.G. Will the Real Smart City Please Stand up? Intelligent, Progressive or Entrepreneurial? City 2008, 12, 303–320. [Google Scholar] [CrossRef]
  27. Gil-Garcia, J.R.; Pardo, T.A.; Nam, T. A Comprehensive View of the 21st Century City: Smartness as Technologies and Innovation in Urban Contexts. In Smarter as the New Urban Agenda: A Comprehensive View of the 21st Century City; Gil-Garcia, J.R., Pardo, T.A., Nam, T., Eds.; Public Administration and Informaiton Technology; Springer: Berlin/Heidelberg, Germany, 2016; Volume 11, pp. 1–19. ISBN 978-3-319-17620-8. [Google Scholar]
  28. Biloria, N. From Smart to Empathic Cities. Front. Archit. Res. 2020, 10, 3–16. [Google Scholar] [CrossRef]
  29. Nam, T.; Pardo, T.A. Smart City as Urban Innovation: Focusing on Management, Policy, and Context. In Proceedings of the 5th International Conference on Theory and Practice of Electronic Governance, Tallinn, Estonia, 26–29 September 2011; pp. 185–194. [Google Scholar] [CrossRef]
  30. Yue, A.; Mao, C.; Chen, L.; Liu, Z.; Zhang, C.; Li, Z. Detecting Changes in Perceptions towards Smart City on Chinese Social Media: A Text Mining and Sentiment Analysis. Buildings 2022, 12, 1182. [Google Scholar] [CrossRef]
  31. Israilidis, J.; Odusanya, K.; Mazhar, M.U. Exploring Knowledge Management Perspectives in Smart City Research: A Review and Future Research Agenda. Int. J. Inf. Manag. 2021, 56, 101989. [Google Scholar] [CrossRef]
  32. Kopackova, H.; Komarkova, J.; Horak, O. Enhancing the Diffusion of E-Participation Tools in Smart Cities. Cities 2022, 125, 103640. [Google Scholar] [CrossRef]
  33. Johansen, A.; Collins, D.; Temeljotov-Salaj, A.; Hagehaugen, G. Down by the Fjord: Successful Public and Private Collaboration in a Neighbourhood Redevelopment Project. Manag. Procure. Law 2022, 40, 1–10. [Google Scholar] [CrossRef]
  34. Laurian, L.; Shaw, M.M. Evaluation of Public Participation: The Practices of Certified Planners. J. Plan. Educ. Res. 2009, 28, 293–309. [Google Scholar] [CrossRef]
  35. Tricco, A.C.; Lillie, E.; Zarin, W.; O’Brien, K.K.; Colquhoun, H.; Levac, D.; Moher, D.; Peters, M.D.J.; Horsley, T.; Weeks, L.; et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann. Intern. Med. 2018, 169, 467–473. [Google Scholar] [CrossRef] [Green Version]
  36. Munn, Z.; Peters, M.D.J.; Stern, C.; Tufanaru, C.; McArthur, A.; Aromataris, E. Systematic Review or Scoping Review? Guidance for Authors When Choosing between a Systematic or Scoping Review Approach. BMC Med. Res. Methodol. 2018, 18, 1–7. [Google Scholar] [CrossRef]
  37. Arksey, H.; O’Malley, L. Scoping Studies: Towards a Methodological Framework. Int. J. Soc. Res. Methodol. Theory Pract. 2005, 8, 19–32. [Google Scholar] [CrossRef] [Green Version]
  38. Ullah, F.; Al-Turjman, F. A Conceptual Framework for Blockchain Smart Contract Adoption to Manage Real Estate Deals in Smart Cities. Mach. Learn. Appl. Secur. 2023, 35, 5033–5054. [Google Scholar] [CrossRef]
  39. Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G.; Altman, D.; Antes, G.; Atkins, D.; Barbour, V.; Barrowman, N.; Berlin, J.A.; et al. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med. 2009, 6, e1000097. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  40. Glaser, B.; Strauss, A. Applying Grounded Theory. In The Discovery of Grounded Theory: Strategies of Qualitative Research; Aldine de Gruyter: Berlin, Germany, 1967; p. 13. [Google Scholar]
  41. Chilvers, J. Remaking Participation; Kearnes, M., Ed.; Routledge: Abingdon-on-Thames, UK, 2015; ISBN 9780415857390. [Google Scholar]
  42. Lin, Y.; Kant, S. Using Social Media for Citizen Participation: Contexts, Empowerment, and Inclusion. Sustainability 2021, 13, 6635. [Google Scholar] [CrossRef]
  43. Gudowsky, N.; Sotoudeh, M.; Capari, L.; Wil, H. Transdisciplinary Forward-Looking Agenda Setting for Age-Friendly, Human Centered Cities. Futures 2020, 90, 16–30. [Google Scholar] [CrossRef]
  44. Francisco, A.; Taylor, J.E. Understanding Citizen Perspectives on Open Urban Energy Data through the Development and Testing of a Community Energy Feedback System. Appl. Energy 2019, 256, 113804. [Google Scholar] [CrossRef]
  45. Psomadaki, O.I.; Dimoulas, C.A.; Kalliris, G.M.; Paschalidis, G. Digital Storytelling and Audience Engagement in Cultural Heritage Management: A Collaborative Model Based on the Digital City of Thessaloniki. J. Cult. Herit. 2019, 36, 12–22. [Google Scholar] [CrossRef]
  46. Caragliu, A.; Del Bo, C.F. Smart Innovative Cities: The Impact of Smart City Policies on Urban Innovation. Technol. Forecast. Soc. Change 2019, 142, 373–383. [Google Scholar] [CrossRef]
  47. Ambrose, A. Walking with Energy: Challenging Energy Invisibility and Connecting Citizens with Energy Futures through Participatory Research. Futures 2020, 117, 102528. [Google Scholar] [CrossRef]
  48. Paskaleva, K.; Cooper, I. Technovation Open Innovation and the Evaluation of Internet-Enabled Public Services in Smart Cities. Technovation 2018, 78, 4–14. [Google Scholar] [CrossRef] [Green Version]
  49. Flacke, J.; Shrestha, R.; Aguilar, R. Strengthening Participation Using Interactive Planning Support Systems: A Systematic Review. ISPRS Int. J. Geo-Inf. 2020, 9, 49. [Google Scholar] [CrossRef] [Green Version]
  50. Billger, M.; Thuvander, L.; Wästberg, B.S. In Search of Visualization Challenges: The Development and Implementation of Visualization Tools for Supporting Dialogue in Urban Planning Processes. Environ. Plan. B Urban Anal. City Sci. 2017, 44, 1012–1035. [Google Scholar] [CrossRef]
  51. Foth, M. Participation, Co-Creation, and Public Space. J. Public Space 2017, 2, 21–36. [Google Scholar] [CrossRef] [Green Version]
  52. Foth, M. Participatory Urban Informatics: Towards Citizen-Ability. Smart Sustain. Built Environ. 2016, 7, 4–19. [Google Scholar] [CrossRef]
  53. Hofmann, M.; Münster, S.; Noennig, J.R. A Theoretical Framework for the Evaluation of Massive Digital Participation Systems in Urban Planning. J. Geovisualization Spat. Anal. 2020, 4, 3. [Google Scholar] [CrossRef] [Green Version]
  54. Castelnovo, W.; Misuraca, G.; Savoldelli, A. Smart Cities Governance: The Need for a Holistic Approach to Assessing Urban Participatory Policy Making. Soc. Sci. Comput. Rev. 2016, 34, 724–739. [Google Scholar] [CrossRef]
  55. Shah, S.H.; Rodina, L.; Burt, J.M.; Gregr, E.J.; Chapman, M.; Williams, S.; Wilson, N.J.; Mcdowell, G. Unpacking Social-Ecological Transformations: Conceptual, Ethical and Methodological Insights. Anthr. Rev. 2018, 5, 250–265. [Google Scholar] [CrossRef]
  56. Ghorbanzadeh, O.; Moslem, S.; Blaschke, T. Sustainable Urban Transport Planning Considering Different Stakeholder Groups by an Interval-AHP Decision Support Model. Sustainability 2019, 11, 9. [Google Scholar] [CrossRef] [Green Version]
  57. Biermann, H.; Philipsen, R.; Brell, T.; Ziefle, M. User Perspectives, Expectations, and Challenges of Data and Information Distribution in Autonomous Driving. Hum. Intell. Syst. Integr. 2020, 1, 53–70. [Google Scholar] [CrossRef]
  58. Bozdağ, A.; İnam, Ş. Collaborative Land Use Planning in Urban Renewal. J. Urban Reg. Anal. 2021, 13, 323–342. [Google Scholar] [CrossRef]
  59. Brockhoff, R.C.; Koop, H.A.S.; Snel, A.W.K. Pluvial Flooding in Utrecht: On Its Way to a Floof-Proof City. Water 2019, 11, 1501. [Google Scholar] [CrossRef] [Green Version]
  60. Mahou-lago, X.M.; Varela-álvarez, E.J. Innovation and Opportunities for Citizen Participation in Spanish Smart Cities. In Smarter as the New Urban Agenda: A Comprehensive View of the 21st Century City; Springer: Berlin/Heidelberg, Germany, 2016; pp. 367–392. ISBN 9783319176208. [Google Scholar]
  61. Clarinval, A.; Simonofski, A.; Vanderose, B.; Dumas, B. Public Displays and Citizen Participation: A Systematic Literature Review and Research Agenda. Transform. Gov. People Process Policy 2021, 15, 1–35. [Google Scholar] [CrossRef]
  62. Bouzguenda, I.; Fava, N.; Alalouch, C. Would 3D Digital Participatory Planning Improve Social Sustainability in Smart Cities? An Empirical Evaluation Study in Less-Advantaged Areas. J. Urban Technol. 2022, 29, 41–71. [Google Scholar] [CrossRef]
  63. Mcevoy, S.; van de Ven, F.H.M.; Brolsma, R.; Slinger, J.H. Evaluating a Planning Support System’s Use and E Ff Ects in Urban Adaptation: An Exploratory Case Study from Berlin, Germany. Sustainability 2020, 12, 173. [Google Scholar] [CrossRef] [Green Version]
  64. Mainka, A.; Hartmann, S.; Meschede, C.; Stock, W.G. Open Government: Transforming Data into Value-Added City Services. In Citizen’s Right to the Digital City; Springer: Berlin/Heidelberg, Germany, 2015. [Google Scholar] [CrossRef]
  65. Fredericks, J. From Smart City to Smart Engagement: Exploring Digital and Physical Interactions for Playful City-Making. In Making Smart Cities More Playable; Springer: Berlin/Heidelberg, Germany, 2020. [Google Scholar] [CrossRef] [Green Version]
  66. Afzalan, N.; Muller, B. Online Participatory Technologies: Opportunities and Challenges for Enriching Participatory Planning. J. Am. Plan. Assoc. 2018, 84, 162–177. [Google Scholar] [CrossRef]
  67. Flacke, J.; de Boer, C.; Van den Bosch, H.; Pfeffer, K. Handbook of Support Science; Geertman, S., Stillwell, J., Eds.; Edward Elgar Publishing Inc.: Cheltenham, UK, 2020; ISBN 9781788971089. [Google Scholar]
  68. Russo, F.; Calabrò, T.; Iiritano, G.; Pellicanò, D.S.; Petrungaro, G.; Sostenibile, E.; Studi, U. International Journal of Sustainable Development and Planning City Logistics Between International Vision and Local Knowledge to Sustainable Development: The Regional Role on Planning and on Public Engagement. Int. J. Sustain. Dev. Plan. 2020, 15, 619–629. [Google Scholar] [CrossRef]
  69. Yoo, Y. Toward Sustainable Governance: Strategic Analysis of the Smart City Seoul Portal in Korea. Sustainability 2021, 13, 5886. [Google Scholar] [CrossRef]
  70. Zhong, L.; Bai, L.; Xiang, M. Evaluation Index System for Public Participation in Post-Disaster Housing Reconstruction in Dujiangyan. Open House Int. 2022, 47, 107–121. [Google Scholar] [CrossRef]
  71. Clemente, C.; Civierob, P.; Celluralea, M. Solutions and Services for Smart Sustainable Districts: Innovative Key Performance Indicators to Support Transition. Int. J. Sustain. Energy Plan. Manag. 2019, 24, 95–106. [Google Scholar]
  72. Ben, E.R. Methodologies for a Participatory Design of IoT to Deliver Sustainable Public Services in “Smart Cities”. In Beyond Smart and Connected Governments: Sensors and the Internet of Things in the Public Sector; Springer: Berlin/Heidelberg, Germany, 2020; pp. 49–68. [Google Scholar] [CrossRef]
  73. Mohseni, H. Public Engagement and Smart City Definitions: A Classifying Model for the Evaluation of Citizen Power in 2025 Tehran. GeoJournal 2021, 86, 1261–1274. [Google Scholar] [CrossRef]
  74. Reza, M.; Bigdeli, V.; Moinifar, M. Technological Forecasting & Social Change The Structural Model of Indicators for Evaluating the Quality of Urban Smart Living. Technol. Forecast. Soc. Chang. 2022, 176, 121427. [Google Scholar] [CrossRef]
  75. Mannarini, T.; Talò, C. Evaluating Public Participation: Instruments and Implications for Citizen Involvement. Community Dev. 2013, 44, 239–256. [Google Scholar] [CrossRef]
  76. Argyris, C. Single-Loop and Double-Loop Models in Research on Decision Making. Adm. Sci. Q. 1976, 21, 363–375. [Google Scholar] [CrossRef]
  77. Boyer, E.J.; Van Slyke, D.M.; Rogers, J.D. An Empirical Examination of Public Involvement in Public-Private Partnerships: Qualifying the Benefits of Public Involvement in PPPs. J. Public Adm. Res. Theory 2016, 26, 45–61. [Google Scholar] [CrossRef]
  78. Frewer, L. Risk Perception, Social Trust, and Public Participation in Strategic Decision Making: Implications for Emerging Technologies. Ambio 1999, 28, 569–574. [Google Scholar]
  79. Castelnovo, W. Co-Production Makes Cities Smarter: Citizens’ Participation in Smart City Initiatives. In Co-Production in the Public Sector; Fugini, M., Bracci, E., Sicilia, M., Eds.; Springer International Publishing: Berlin/Heidelberg, Germany, 2016; pp. 97–117. ISBN 9783319305585. [Google Scholar]
  80. Innes, J.E.; Booher, D.E. Collaborative Rationality as a Strategy for Working with Wicked Problems. Landsc. Urban Plan. 2016, 154, 8–10. [Google Scholar] [CrossRef]
  81. Malek, J.A.; Lim, S.B.; Yigitcanlar, T. Social Inclusion Indicators for Building Citizen-Centric Smart Cities: A Systematic Literature Review. Sustainability 2021, 13, 376. [Google Scholar] [CrossRef]
  82. Alonso, F.; Faus, M.; Tormo, M.T.; Useche, S.A. Could Technology and Intelligent Transport Systems Help Improve Mobility in an Emerging Country? Challenges, Opportunities, Gaps and Other Evidence from the Caribbean. Appl. Sci. 2022, 12, 4759. [Google Scholar] [CrossRef]
  83. Cavada, M.; Rogers, C.D.F. Serious Gaming as a Means of Facilitating Truly Smart Cities: A Narrative Review. Behav. Inf. Technol. 2019, 39, 695–710. [Google Scholar] [CrossRef]
  84. Gabriel, S.; Schmölzer, B. Climate4Kids: A Gamified App Teaching about Climate Change. In Proceedings of the 16th European Conference on Game-Based Learning, Lisbon Portugal, 6–7 October 2022; pp. 236–243. [Google Scholar]
  85. West, M.; Yildirim, O.; Harte, A.E.; Ramram, A.; Fleury, N.W.; Carabias, V. Enhancing Citizen Participation through Serious Games in Virtual Reality. In Proceedings of the 24th International Conference on Urban Planning, Regional Development and Information Society, Karlsruche, Germany, 2–4 April 2019; Volume 4, pp. 881–888. [Google Scholar]
  86. Latifi, G.R.; Monfared, M.P.; Khojasteh, H.A. Gamification and Citizen Motivation and Vitality in Smart Cities: A Qualitative Meta-Analysis Study. GeoJournal 2022, 87, 1217–1230. [Google Scholar] [CrossRef]
Buildings 13 01484 g001 550
Figure 1. Preferred Reporting Items for Systematic Reviews and Meta-Analyses and Meta Analyses extension for Scoping Reviews (PRISMA-ScR) flow chart. Source: authors.
Figure 1. Preferred Reporting Items for Systematic Reviews and Meta-Analyses and Meta Analyses extension for Scoping Reviews (PRISMA-ScR) flow chart. Source: authors.
Buildings 13 01484 g001
Buildings 13 01484 g002 550
Figure 2. Sampled literature clustered by number of articles published per year. Source: authors.
Figure 2. Sampled literature clustered by number of articles published per year. Source: authors.
Buildings 13 01484 g002
Buildings 13 01484 g003 550
Figure 3. Geographic distribution of the location of the studies by world region. Source: authors.
Figure 3. Geographic distribution of the location of the studies by world region. Source: authors.
Buildings 13 01484 g003
Buildings 13 01484 g004 550
Figure 4. Journal distribution in the sampled literature.
Figure 4. Journal distribution in the sampled literature.
Buildings 13 01484 g004
Buildings 13 01484 g005 550
Figure 5. Type of public participation method identified in the sampled literature.
Figure 5. Type of public participation method identified in the sampled literature.
Buildings 13 01484 g005
Buildings 13 01484 g006 550
Figure 6. Level of participation achieved in the studies. The colors are set according to the degree of participation, from the lowest (lightest shade) to the highest (darkest shade) [1,42,43,44,45,46,47,48,50,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70]. Source: authors.
Figure 6. Level of participation achieved in the studies. The colors are set according to the degree of participation, from the lowest (lightest shade) to the highest (darkest shade) [1,42,43,44,45,46,47,48,50,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70]. Source: authors.
Buildings 13 01484 g006
Buildings 13 01484 g007 550
Figure 7. Evaluation method. Source: authors.
Figure 7. Evaluation method. Source: authors.
Buildings 13 01484 g007
Table
Table 1. PRISMA checklist.
Table 1. PRISMA checklist.
ItemPRISMA Checklist
1The review is based on keyword-based articles retrieved from Scopus, WoS, and Google Scholar repositories published in the last 10 years.
2The keywords must be present in one of the main sections of the articles to be eligible for inclusion in the study.
3Scopus, WoS, and Google Scholar repositories can be respectively accessed at scopus.com/search/form.uri?display=basic, https://www.webofscience.com/wos/woscc/basic-search and https://scholar.google.com/.
4Search strings are used for conducting the literature search process, as listed above in Section 3.1.
5The selection process consists of keyword searching, screening, and removing duplicates. See a detailed description in Section 3.2.
6The selected articles are analyzed by charting the data in Excel as described in Section 3.3.
7The categories for charting the data consist of general information, core data, and definitions (see Section 3.3).
8The included sources of evidence are typically not critically appraised for scoping reviews [35]. Therefore, the present study does not report on the appraisal.
9Summary measures are not applicable for scoping reviews [35].
10The synthesis of the results is presented as a “map” of the data in the form of diagrams and tables and in a descriptive format aligning with the review’s objectives (see Section 4).
11Risk of bias across studies is not applicable for scoping reviews [35].
12Additional analyses are not applicable for scoping reviews [35].
Table
Table 2. Levels of citizen participation defined by Ringholm, Nyseth and Sandkjær Hanssen (2018).
Table 2. Levels of citizen participation defined by Ringholm, Nyseth and Sandkjær Hanssen (2018).
LevelsDefinitions by Ringholm et al. (2018) [19]
Co-management “citizens’ initiatives, which can be used to initiate new planning processes or revise old ones”
Agenda setting “have the opportunity to influence the agenda by putting forward themes that should be included in the planning strategy”
Dialogue “two-way communication—a discussion, debate or deliberation regarding an issue between the municipalities and interested parties”
Consultation “the plan proposal is made publicly available so that everyone has access to it and can provide feedback”
Information “advertising the establishment of the planning process”
Table
Table 3. Impact on participants identified in the papers.
Table 3. Impact on participants identified in the papers.
Type of ImpactDefinitionEvaluation Indicators Identified in the Studies
Awareness
(n = 6)		Participants and the general public are informed about the issue and realize what is at stake in this particular matter [34].Level of awareness—pre- and post-experience [47];
community knowledge, local sense of urgency [59]; participants are informed and increase their knowledge about an issue [49,67]; raising awareness through community networks [65]; raising awareness on the concept of urban renewal and its implementation [58].
Behavioral change
(n = 4)		Process that involves modifying or altering one’s behavior or habits in response to internal or external cues, stimuli, or incentives. Behavioral change can be intentional or unintentional and can occur at individual, group, or societal levels.Likeliness to change behavior based on the tour of the facility [47];
likeliness to change behavior based on the feedback provided by the tool [44]; participants change behavior to address a challenging situation [49,67].
Future
engagement
(n = 4)		The likelihood or willingness of individuals to participate in urban planning processes again in the future after their initial experience [75].Actions to be incorporated in future engagement practices [1]; subjective assessments of deliberation quality and likelihood of participating in future participation processes [53]; community building: development of new collaborations, improved social cohesion [67]; integrating new approaches for further collaboration—casting a wider net across the top and bottom stakeholders and collectively bringing people together [65].
Knowledge, skills, learning
(n = 7)		In single-loop learning, participants seek to achieve their intended goals by making incremental improvements to their strategies and actions. In double-loop learning, participants question and potentially change their underlying assumptions, which leads to a deeper understanding of the causes of the problems and enables them to challenge and change established ways of thinking and operating [76].Allows those who had minimal knowledge of city logistics to go up and carry out quality projects [68]; improved understanding of the problem [49,67]; preconditions for a successful process include a good learning situation [50]; actual and perceived knowledge gains regarding the process’ subject matter [53]; appear to have contributed to learning through the provision of content and improved communication and interaction [63]; digital competency enhancement education: citizens can register and access online education computer resources, workshops, etc. and can review the history of the training they have received [69].
Satisfaction
(n = 6)		Individuals’ positive or negative impression of their experience with a particular initiative or participatory process [77].Satisfaction of different groups concerning the public transportation system [56]; levels of satisfaction with the participation method [62]; participants’ satisfaction with process and outcomes [49]; overall satisfaction of participating in post-disaster reconstruction [70]; satisfaction with the process [53]; user-satisfaction measures [54].
Attitudes
(n = 4)		Attitudes toward public participation are shaped by factors such as trust in government, perceived risks and benefits, and individual values and beliefs [78].Participants’ attitudes toward technology [57]; no dominating person or group [49,67]; negative attitudes between NGOs and local administration [58].
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Senior, C.; Temeljotov Salaj, A.; Johansen, A.; Lohne, J. Evaluating the Impact of Public Participation Processes on Participants in Smart City Development: A Scoping Review. Buildings 2023, 13, 1484. https://doi.org/10.3390/buildings13061484

AMA Style

Senior C, Temeljotov Salaj A, Johansen A, Lohne J. Evaluating the Impact of Public Participation Processes on Participants in Smart City Development: A Scoping Review. Buildings. 2023; 13(6):1484. https://doi.org/10.3390/buildings13061484

Chicago/Turabian Style

Senior, Coline, Alenka Temeljotov Salaj, Agnar Johansen, and Jardar Lohne. 2023. "Evaluating the Impact of Public Participation Processes on Participants in Smart City Development: A Scoping Review" Buildings 13, no. 6: 1484. https://doi.org/10.3390/buildings13061484

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop