3.1.5. Data Extraction in the Inclusion Stage

In stage 4 (inclusion), the authors performed an additional backward–forward search [74,81] on each identified article besides the identified studies from the Google Scholar database search.

The backward–forward search (involving articles dated before or after the identified article) observed the detailed themes or indicators relevant to this study, as the potential article titles were indirectly related to citizen centricity and smart cities. backward–forward search (involving articles dated before or after the identified article) observed the detailed themes or indicators relevant to this study, as the potential article titles were indirectly related to citizen centricity and smart cities.

The authors considered the backward–forward search to be an important step in finding the themes or indicators that were directly related to answering the research question. Hence, five records were found in the backward search, whereas three studies were found in the forward search. Finally, a total of 79 articles were finalized for performing a thematic analysis. The search protocol process is presented in Figure 3 below. The authors considered the backward–forward search to be an important step in finding the themes or indicators that were directly related to answering the research question. Hence, five records were found in the backward search, whereas three studies were found in the forward search. Finally, a total of 79 articles were finalized for performing a thematic analysis. The search protocol process is presented in Figure 3 below.

*Sustainability* **2020**, *12*, x FOR PEER REVIEW 8 of 32

**Figure 3.** The execution stages of the systematic literature review (source: adapted from [79,82]). **Figure 3.** The execution stages of the systematic literature review (source: adapted from [79,82]).

3.1.6. Data Analysis and Risk of Bias

authored by [52], and the highest number of articles in a year was 14 articles in 2016.

3.1.6. Data Analysis and Risk of Bias

Based on the selected articles, the authors conceptualized the formation of items. A common name was used to describe similar concepts from various authors during the line-by-line coding of the thematic analysis process [74]. For example, "participation" was used as a common name to describe similar concepts of "engagement, collaboration and involvement" [83]. Figure 4 depicts the distribution of the reviewed papers per year, with the earliest article appearing in 1969, which was Based on the selected articles, the authors conceptualized the formation of items. A common name was used to describe similar concepts from various authors during the line-by-line coding of the thematic analysis process [74]. For example, "participation" was used as a common name to describe similar concepts of "engagement, collaboration and involvement" [83]. Figure 4 depicts the distribution of the reviewed papers per year,

**4. Findings**

*4.1. Systematic Literature Review*

*Sustainability* **2020**, *12*, x FOR PEER REVIEW 9 of 32

with the earliest article appearing in 1969, which was authored by [52], and the highest number of articles in a year was 14 articles in 2016.

**Figure 4.** The distribution of the reviewed papers per year (source: authors). **Figure 4.** The distribution of the reviewed papers per year (source: authors).

Furthermore, it was discovered that the four most selected publishers were Wiley-Blackwell (nine articles), Springer (eight articles), Taylor & Francis (six articles), and Elsevier (six articles). Organizations such as the OECD, the World Bank, and universities also contributed to this group of articles (Appendix B). Furthermore, it was discovered that the four most selected publishers were Wiley-Blackwell (nine articles), Springer (eight articles), Taylor & Francis (six articles), and Elsevier (six articles). Organizations such as the OECD, the World Bank, and universities also contributed to this group of articles (Appendix **??**).

According to [79], no guarantees were provided on the assessment or interpretation of the appropriate content by systemic reviewers. Thus, the authors aimed to report the possible risk of bias across selected studies, depending on the qualitative judgment from the authors when interpreting the contents and extracting suitable themes or items to answer the research question. According to [79], no guarantees were provided on the assessment or interpretation of the appropriate content by systemic reviewers. Thus, the authors aimed to report the possible risk of bias across selected studies, depending on the qualitative judgment from the authors when interpreting the contents and extracting suitable themes or items to answer the research question.

### *3.2. Verification by Practitioners 3.2. Verification by Practitioners*

4.1.1. The Items for the Construct of a Citizen-Centric Smart City (DV 1)

With the structuring of indicators through the rigorous process of the systematic literature review, all the indicators were verified again by practitioners in the fields of smart city and participation. The verification process also assisted in reducing the risk of bias from qualitative judgment by the authors. The selected practitioners consisted of two groups, which were the "power holders" with 19 people (11 local authority officers, 5 federal or state officers, and 3 politicians), and the "have-not citizens" as the remaining half (8 representatives of community organizations or residents, 2 non-governmental organizations, 3 academicians, and 6 private sector representatives). Overall, the selected informants provided ideas and suggestions to validate the studied items. With the structuring of indicators through the rigorous process of the systematic literature review, all the indicators were verified again by practitioners in the fields of smart city and participation. The verification process also assisted in reducing the risk of bias from qualitative judgment by the authors. The selected practitioners consisted of two groups, which were the "power holders" with 19 people (11 local authority officers, 5 federal or state officers, and 3 politicians), and the "have-not citizens" as the remaining half (8 representatives of community organizations or residents, 2 non-governmental organizations, 3 academicians, and 6 private sector representatives). Overall, the selected informants provided ideas and suggestions to validate the studied items.

Nonetheless, some practitioners were uneasy about how ordinary citizens could understand the technical terms in the verification process. With such limitations, more examples relevant to common terms were applied in the items, such as grants, as opposed to contracts in the form of finance in running city programs (refer to Appendix C for the terms adjusted by practitioners). Nonetheless, some practitioners were uneasy about how ordinary citizens could understand the technical terms in the verification process. With such limitations, more examples relevant to common terms were applied in the items, such as grants, as opposed to contracts in the form of finance in running city programs (refer to Appendix **??** for the terms adjusted by practitioners).

be more focused on citizens' needs and not merely on technology to reach a consensus with citizens regarding realizing the state of citizen centricity in a smart city. LAs also learned to delegate power

In searching for and suggesting the items for building the study constructs, the authors found
