An Assessment Model for Sustainable Cities Using Crowdsourced Data Based on General System Theory: A Design Science Methodology Approach

Abstract

In the quest to understand urban ecosystems, traditional evaluation techniques often fall short due to incompatible data sources and the absence of comprehensive, real-time data. However, with the recent surge in the availability of crowdsourced data, a dynamic view of urban systems has emerged. Recognizing the value of these data, this study illustrates how these data can bridge gaps in understanding urban interactions. Furthermore, the role of urban planners is crucial in harnessing these data effectively, ensuring that derived insights align with the practical needs of urban development. Employing the Design Science Methodology, the research study presents an assessment model grounded in the principles of the city ecosystem, drawing from the General System Theory for Smart Cities. The model is structured across three dimensions and incorporates twelve indicators. By leveraging crowdsourced data, the study offers invaluable insights for urban planners, researchers, and other professionals. This comprehensive approach holds the potential to revolutionize city sustainability assessments, deepening the grasp of intricate urban ecosystems and paving the way for more resilient future cities.

1. Introduction

In recent years, the concept of smart cities has gained significant attention in urban development, aiming to enhance the quality of life for city residents. The increasing urban population, which now accounts for over half of the global populace and consumes 75% of the world’s energy [1], has led to interest in creating efficient, productive, and sustainable urban environments using innovative technologies and data-driven strategies. Smart cities strive to address diverse objectives, acting as catalysts for urban progress by embracing physical, social, and digital dimensions, anticipating challenges, and fostering integrated services and innovation within urban institutions [2].

Urban planners, as the architects of city development, play an indispensable role in translating these smart city concepts into actionable strategies. Their expertise and on-ground experience ensure that technological advancements align with genuine urban needs and the broader objectives of sustainable development [3]. The smart city ecosystem is rooted in general systems theory, which envisions cities as systems composed of various systems, each reflecting different facets of smart city attributes [4], such as sustainability, urbanization, quality of life, and intelligence. These traits form the basis for assessing a city’s level of intelligence and sustainability using smart city evaluation methodologies [5]. Integral to the advancement of smart cities is the evaluation their potential based on smart city evaluation models that play a role in achieving the aforementioned goals [6]. Successful smart city assessments provide guidance for decision making while gauging the alignment of implementation with desired trajectories [7]. Thus, the need for assessment models and data sources plays a pivotal role.

Alongside the evolving concept of smart cities, there is an imperative to grasp and implement the principles of sustainable urban development. A sustainable city goes beyond just emphasizing technology and innovation; it prioritizes a harmonious balance of social, economic, and environmental facets. This vision aligns seamlessly with the Sustainable Development Goals (SDGs) [8]. One of the cardinal objectives of a sustainable city is to foster the achievement of these SDGs. Within the framework of smart cities, the deployment of technology and data-driven strategies must be carried out with a keen eye on sustainable development principles. Therefore, the evaluation model emphasizes assessing a city’s ‘sustainability’ as a pivotal step towards fulfilling the objectives of the SDGs.

Numerous smart city assessment models have emerged, encompassing paradigms like Sustainable Development Indicators [9,10], Lisbon ranking for smart sustainable cities [11], Smart City Performance [12], IESE Cities in Motion Index [13], ITU-T Y.4903/L.1603 Indicators [14], Sustainability Perspectives Indicators [15], Smart City Index Master Indicators Survey [10,16], Dimensions of the Smart City of Vienna UT [17], Characteristics of Smart City Indicators [18], Criteria set for evaluating smart cities [19], China smart city performance [20], Juniper analysis of smart city frameworks [21], Assessing the Effectiveness of Smart Transport [22], Smart City Dimension [23], and City Sustainability Assessment [24]. These models draw from a range of resources, including primary and secondary data [25].

Primary data are generally collected through questionnaires, interviews, surveys, on-site analyses, field observations, and photographic documentation, while secondary data sources encompass census data, city audits, annual reports, and historical records. However, both primary and secondary data have limitations. Primary data collection can be resource-intensive and time-consuming, leading to prolonged assessments. Conversely, secondary data may suffer from obsolescence and data quality issues [26]. Governmental statistical data form a foundation for smart city assessments in current models. While these data provide a broad overview of cities, these data possess limitations, such as their static nature and inability to capture real-time shifts in urban environments. Establishing unified evaluation standards across diverse cities also presents a significant challenge [27]. Traditional evaluation models based on statistical data struggle to monitor real-time developments within the dynamic landscape of smart cities [27].

A notable limitation in existing smart city assessment models is their inability to effectively understand and evaluate interactions among distinct systems within urban systems [25]. As the smart city landscape evolves, this should be addressed to improve the accuracy and efficacy of smart city assessments. In summary, the trajectory of smart cities in the past decade reflects efforts to leverage technology and data for urban improvement. With a focus on multifaceted attributes and holistic development, evaluating smart city potential through diverse models and data sources holds the key to steering urban centers towards a more intelligent and sustainable future.

Leveraging big data through strategic crowdsourcing presents a solution for overcoming challenges in smart city assessments [25,28]. Crowdsourced data offer advantages in creating precise evaluation models, enhancing accuracy, and uncovering urban dynamics. This approach can be integrated into real-time assessments, reducing data collection costs while maintaining integrity [24,29]. Utilizing crowdsourced data, driven by the widespread use of information and communication technologies (ICT), allows for the assessment of smart cities. With a substantial increase in home internet adoption (64.6 percent of the global population, meaning that as of April 2023, there are approximately 5.18 billion internet users (statista.com (accessed on 1 August 2023)), a wealth of social media data are accessible from platforms like Facebook, Instagram, Twitter, and more.

Building on our understanding of the challenges, limitations, and untapped potential within crowdsourced data, this research article focuses on developing an advanced smart city assessment paradigm. This model places emphasis on the strategic integration of crowdsourced data, addressing existing limitations and gaps in conventional smart city evaluations. The envisioned model introduces more comprehensive and sustainable indicators to underpin smart city assessments. Concurrently, it establishes crucial connections between inter-indicator dynamics, a crucial step towards comprehensively evaluating the sustainability of urban landscapes.

To this end, this article is presented in a structured manner, beginning with a Literature Review (Section 2), in which we delve into previous research, general system theory, and the significance of crowdsourced data. The Methods section (Section 3) elucidates the methodological framework utilized in the development of the assessment model for sustainable cities. The Results section (Section 4) showcases our findings, while the Discussion section (Section 5) contains an examination of the implications of these results, especially in the context of crowdsourced data from citizens. The manuscript culminates in the Conclusions section (Section 6), offering a summary of this study and paving the way for other researchers by providing recommendations for future research.

2. Literature Review

2.1. Smart City Assessment

In the broader scheme of urban development, a pressing concern arises from the omission of core global issues fundamental to sustainable cities. The political underpinnings of city planning, which decisively shape the mission and objectives of city authorities, cannot be overlooked [30]. The global emphasis on sustainability, underscored by the Sustainable Development Goals (SDGs), accentuates sustainable development with a keen focus on socio-economic and environmental facets [31]. These facets are intrinsically tied to the imperatives of climate change mitigation and adaptation. Such policy goals, which are paramount to shaping sustainable cities, often remain absent in many smart city assessments. As city authorities or urban planners strive to harness technological advancements in their quest for ‘smartness’, it is equally vital to ensure these endeavors are aligned with the broader objectives of global sustainability. Only then can cities truly evolve as smart, resilient, and sustainable urban habitats that are responsive to both current challenges and future uncertainties.

Notably, smart city assessment is an emergent domain rich with potential for future exploration and development [25]. As highlighted by the authors of [7], the primary objective of evaluating smart cities is to obtain feedback and guidance pivotal for decision making, ensuring whether the implementation of a particular project or initiative is helping to move towards the desired direction. Furthermore, these assessments can serve as a performance monitoring tool, providing a platform to evaluate the benefits for various stakeholders, ranging from city authorities (urban planners), investors, funding institutions, and researchers to the general populace [6].

Central to the assessment of smart cities is the focus on two aspects: ‘smartness’ and/or ‘sustainability’. For instance, the authors of [32] embarked on a dual-aspect evaluation that encompassed six dimensions: living, economy, mobility, governance, environment, and people. The primary data employed originated from expert interviews, while the analytic approach utilized both the Analytic Network Process (ANP) and TOPSIS. Similarly, ref. [33] employed the same six dimensions but sourced their primary data from surveys and questionnaires, leveraging the Fuzzy Synthetic Evaluation (FSE) for analysis. The study by the authors of [11] centered on both smartness and sustainability but prioritized social, economic, and environmental assessment dimensions. Their secondary data drew from the Data Urban Audit and Eurostat Database, with hierarchical clustering and Principal Component Analysis (PCA) being the chosen analytic methods.

There are also studies singularly focused either on the smartness or sustainability of smart cities. For the former, ref. [19]’s research prioritized six dimensions, including environment, living, mobility, economy, people, and governance, gathering primary data through interviews and surveys and applying the Multi-Attribute Decision Making (MADM) analysis. A study by [20] evaluated smartness using dimensions such as infrastructure, governance, economy, people, and environment. This study relied on secondary data from the China City Statistical Yearbook and air quality reports, and TOPSIS was chosen as the analytic approach. On the issue of sustainability, the authors of [34] employed dimensions like innovation, economy, infrastructure, services, mobility, and environment. Their secondary data were sourced from the China Statistical Yearbook and the Yearbook of China Information Industry, deploying both PCA and back propagation (BP) for analysis. Another sustainability-focused study by the authors of [35] considered social, economic, and environmental dimensions through using secondary data derived from previous city rankings.

From the aforementioned studies on smart city evaluations, it can be seen that traditional data sources, including surveys, questionnaires, and interviews for primary data and statistical reports for secondary data, remain predominant. However, with technological advancements and data evolution, there is burgeoning potential for more dynamic evaluations using easily accessible data sources like crowdsourcing [29]. Yet, this vast potential remains largely untapped due to the lack of innovative evaluation models capable of harnessing and researching this information [25,26].

In addition to the research that has been elaborated upon, there are also various types of models and frameworks used to evaluate smart cities. The source data for this evaluation come from primary or secondary sources of the same type. Various models and frameworks from the literature are presented in

Table 1. Models and frameworks for smart city assessments.

No.	Ref.	Number of Indicators
		Social	Economy	Environment
1	Smart Sustainable City Indicators [9,10]	6	3	5
2	Sustainable Development Indicators [10]	11	3	6
3	Smart City Index Master [10,16]	0	3	3
4	Lisbon ranking for smart sustainable cities [11]	6	6	6
5	Smart city performance index [12]	3	4	4
6	IESE Cities in Motion Index 2018 [13]	13	8	11
7	ITU-T Y.4903/L.1603 [14,36]	6	7	6
8	Sustainability Perspectives Indicators [15]	11	5	13
9	Dimensions of the smart city Vienna UT [17]	0	6	4
10	Characteristics Smart City [18]	0	3	3
11	Criteria set for evaluating smart cities [19]	0	5	7
12	China smart city performance [20]	0	3	3
13	Sustainable development of communities [21]	0	5	7
14	Assess effectiveness of the smart transport [22]	0	0	2
15	Smart City Dimension [23]	0	4	7
16	City Sustainability Assessment [24]	12	7	5
17	Smart Sustainable Cities [34]	0	4	5
18	Global Power City Index 2018 [37]	0	5	3
19	ITU-T Y.4901/L.1601 [6,36,38]	6	7	6
20	ITU-T Y.4902/L.1602 [6,36,39]	6	7	6

.

2.2. Smart City Ecosystems

Smart city ecosystems delineate the dynamic interplay between urban planners, citizens, and various stakeholders. Central to this concept are specially designed systems that deliver a range of services. From transportation and public safety to healthcare, education, and social services, these systems are intertwined. Their collective purpose is not just to function in isolation but to collaborate, share resources, and exchange information to achieve overarching urban objectives [40].

Moreover, the synergy between urban planners and citizens within these ecosystems encapsulates a strategy for fostering sustainable cities [30]. This strategy is crystallized as output in the smart city input–output (I/O) model. Structured around (1) inputs like human talent, knowledge, ICT infrastructure, and financial assets, (2) dynamic processes or throughputs, (3) tangible outputs or applications, and (4) eventual outcomes or externalities, this model paints a comprehensive picture. While governance and leadership amplify the value of resources, converting them into actionable outputs, the externalities highlight the consequences derived from these processes, directly influencing the city’s sustainability metrics [41].

Drawing parallels with General System Theory (GST), city ecosystems are perceived as a mosaic of interconnected components working in harmony to create a sophisticated entity. Such systems are defined by their spatial extent, temporal dynamics, and their interaction with the surrounding environment. Their constitution and purpose are evident in their structure and functionality, respectively [4,42]. Interestingly, when viewed through the GST lens, there emerges a distinct difference in how we perceive traditional cities versus their smart counterparts.

Historically, as per GST, a city was perceived as “a vast, enduring human conglomeration with intricate subsystems like sanitation, utilities, land usage, housing, and transportation” [43]. Yet, a more contemporary definition, as posited by the authors of [4] and grounded in GST, envisions a city as “a habitat interwoven with myriad systems. Within this habitat, elements like energy networks or power plants represent the energy system, while entities like vehicles or infrastructures epitomize the transportation system.” This distinction is graphically illustrated in Figure 1a.

In traditional cities, systems typically interact only with their immediate environment. This condition means that most systems largely stand alone and cannot operate in conjunction with other systems. On the other hand, the essence of a smart city is to connect various systems amongst themselves. In Figure 1a, the (sub)system of transportation does not interact with other systems but only with its environment. In Figure 1b, individual systems from transportation interrelate with systems from the energy system. In smart cities, such interconnections among systems can represent the exchange of information (resources).

Evaluating smart cities is an integral aspect of the smart city concept. The interactions between urban planners and citizens within urban systems necessitate feedback and assessment [40]. This aligns with the GST framework, which emphasizes “problem solving in a real-world situation” [44]. Such evaluations are crucial to determine if the solutions are tailored to the city’s needs.

2.3. Crowdsourced Data

Crowdsourced data represent a grassroots effort to harness information from the public and funnel it into specific media channels, transcending mere geographic parameters [45]. These data hold immense potential in deciphering urban dynamics and the underlying patterns that drive them. These data have proven instrumental in tackling challenges and bridging critical gaps in data analysis that conventional urban methodologies struggle to address [46]. Crowdsourced data offer real-time insights, portraying the present state of affairs accurately. Moreover, the collection methods for these data present a cost-effective alternative to traditional data acquisition methods like governmental surveys or censuses. Central to the significance of crowdsourced data is the voluntary generation of these data by community members who contribute information tied to urban activities [47].

As highlighted by the authors of [47], crowdsourced data originate from three primary sources: (1) social media, including platforms like Facebook, Twitter, Instagram, and Sina Weibo; (2) Point of Interest (POI) data derived from sources such as OpenStreetMap, business mapping services like Google Places and Gaode Maps, and check-in records from social networks like Foursquare or Yelp; and (3) websites offering web services and open street maps. An alternate classification offered by the authors of [48] expands this scope to four categories: (1) social media platforms like Flickr, Foursquare, Instagram, Tencent QQ, Twitter, and Weibo; (2) outdoor activity-sharing platforms like Condoon, Geocaching, GPSies, MapMyFitness, Strava, and Wikiloc; (3) community knowledge portals such as eBird and iNaturalist; and (4) cellular signal data furnished by telecommunications companies.

To harness the full potential of crowdsourced data in urban planning and support the Sustainable Development Goals, it is essential to actively engage with the requirements of urban planners as end-users [49]. Urban planners need accurate, timely, and relevant information to support their strategic decisions. Crowdsourced data, with their dynamic and real-time nature, have the potential to meet these requirements. However, to be truly effective, the data must be presented in formats that are accessible, analyzable, and integrable into existing urban planning processes. One of the significant challenges in utilizing crowdsourced data in urban planning is their effective embedding within the policy decision-making cycle. For crowdsourced data to be impactful, these data must be meticulously embedded at every stage of the policy cycle, from problem identification, policy formulation, and implementation to evaluation [50]. Decisions based on crowdsourced data will be more relevant and effective in addressing the intricate challenges of urban environments if this integration is achieved. In the context of city sustainability, crowdsourced data can be a valuable resource for detecting environmental changes, monitoring climate impacts, and assessing the effectiveness of mitigation and adaptation interventions. To ensure these data add value, there is a need to focus on developing “win-win” solutions that benefit both parties and policy co-benefits related to the sustainable development components. This requires close collaboration between urban planners, authorities, communities, and crowdsourced platform providers.

It is evident that social media provides a wealth of crowdsourced data. Illustrated in Figure 2 is the process through which such data are procured from social media platforms. The agility of crowdsourced data’s updates, stemming from the frequent nature of social media interactions, not only reflects the latest developments but also captures the prevailing conditions accurately. The economic advantage of crowdsourced data over conventional sources like surveys or official censuses is a compelling factor. Enriched by the voluntary contributions of community members, crowdsourced data provide a trove of insights pertaining to urban life [47].

In short, crowdsourced data are a tool for decoding urban dynamics and patterns and capable of untangling problems and bridging gaps left by traditional urban data analysis methods. Using social media as a data source provides benefits for stakeholders in supporting decision making through smart city assessment. These benefits include enhanced city transparency, data-driven decision making for funding allocations, and a channel for expressing civic aspirations to local authorities. The ample amount of crowdsourced data derived from social media serve as a valuable resource that can inform new strategies and streamline the complexities of smart city development.

3. Methods

The procedures for the development of a sustainable city assessment model using crowdsourced data adopt the Design Science Methodology. This methodology integrates conceptual approaches from behavioral science with the design science paradigm. The goal is not only to understand and apply the model but also to evaluate its relevance [51]. There are three important aspects in this methodology: environment, IS research, and knowledge base. The methodology adopted is shown in Figure 3 [52].

Based on Figure 3, the environment is defined as a space that contains interesting phenomena, which is divided into three main elements: people, organizations, and technology [51]. These elements form a framework of tasks, problems, and opportunities that match business needs. Business needs, in turn, are an important milestone in connecting theoretical needs with practical needs, especially in formulating dimensions and indicators for sustainable city assessment models. To clarify, “Tasks”, in this research study, are viewed from an application domain perspective, where ‘People’ focuses on the role of citizens in smart city initiatives and ‘Organization’ highlights the importance of improving city competitiveness and the quality of public services, while ‘Technology’ refers to the processes of smart city development based on technological advancements and urban needs. Meanwhile, conditions need to be addressed to meet business needs, such as the efficiency of the assessment process, the use of cheap data sources via ICT, and understanding interactions between city systems. On the other hand, there are opportunities that can be improved, such as reducing time and costs in the assessment process, providing a real-time picture of city sustainability, and utilizing crowdsourced data.

A knowledge base is defined as a collection of data that form the basis for research. There are two main components in a knowledge base: foundation and methodology. Foundations include pre-existing data such as basic theories, frameworks, instruments, constructs, models, and other methods. Meanwhile, methodology refers to a series of guidelines or procedures applied in research [53]. In the context of this research study, the knowledge base includes the following: (1) approaches to designing assessments, such as taxonomy, (2) existing artifacts such as taxonomies, frameworks, and models for smart city assessment, (3) theories related to concepts and dimensions pertinent to smart (sustainable) cities, and (4) existing methods for validating taxonomies or models. In the design science methodology in information systems, there are two crucial activities that are considered to be the heart of research: the creation of artifacts and the validation of these results. The main goal of these two activities is to solve problems or achieve predetermined research objectives. In the context of this research study, the research steps begun with establishing dimensions and indicators. The next step was to develop relevant keywords for the crowdsourced data. After that, validation was carried out by experts. The final step is to propose an assessment model for sustainable cities. The research process is illustrated in Figure 4.

In this research study, a taxonomy approach was adopted to determine dimensions and indicators using Multi-Criteria Analysis (MCA) as the method. MCA is considered an efficient decision-making instrument in dealing with complex issues with many criteria, both qualitative and quantitative [54]. The results of this taxonomy will later become the basis for selecting the most appropriate criteria and indicators for a sustainable city assessment model. Furthermore, the taxonomy development structure is divided into three main aspects: Principle, Criteria, and Indicator. The “Principle” is the basis that determines the context for the development of the model, with a focus on the assessment of smart cities within the scope of sustainable cities. “Criteria”, or, in the context of this research study, “dimensions”, function as standards in the assessment process, providing a model for measuring performance and integrating information from indicators. “Indicators” are specific elements in the ecosystem that inform about certain criteria, providing a detailed view that represents all the elements in the assessment. The process of preparing a taxonomy based on MCA is as illustrated in Figure 5 [54].

In the effort to develop a taxonomy for dimensions and indicators, the data used can be seen in Table 1. The data include frameworks, instruments, models, and methods that have previously been used in smart city assessments, called knowledge bases. However, in selecting dimensions and indicators, the decision is based on the outcomes of the smart city I/O model [41]. This selection process can be seen in Figure 6: The first stage involves collecting framework data, instruments, models, and methods related to smart city assessments. After that, the next step is to evaluate whether each entry of the data includes dimensions that are relevant to the outcome of the smart city I/O model, which includes social, economic, and environmental aspects. If appropriate and relevant, these dimensions or indicators will be integrated into the taxonomy that is being developed [55].

In the taxonomy development process, choosing the right keywords is crucial for ensuring efficiency in data collection. After the dimensions and indicators have been determined, the next step is to select relevant keywords, which are usually taken from the existing knowledge base and used as the foundation for formation. The selection of keywords must consider the relevance and suitability of each indicator, including the initial indicators that existed before the taxonomy was formed. Some guidelines for determining keywords include the following: (1) keywords must reflect the context of the indicator, (2) they must match the meaning of the indicator, (3) they can be sourced from initial indicators before a specific theme is determined, and (4) they must be able to represent urban ecosystem dynamics well. Once the keywords are defined, they will be used to collect data from crowdsourced sources. The term crowdsourced data refers to data generated voluntarily by citizens regarding urban dynamics in certain media [39]. In the context of this research study, the crowdsourced data in question focus on opinions in text form, as these data have the potential to be processed directly for urban sustainability assessments.

After the formation of the taxonomy—which includes dimensions, indicators, and keywords—is complete, the next step is the validation process, which includes expert refinement. The aim of this refinement is to evaluate the extent to which the elements in the taxonomy are suitable for use in urban sustainability assessments. The questionnaire was designed as a validation tool, with questions organized based on the taxonomy that has been created. This questionnaire is semi-structured [56], incorporating closed questions, allowing respondents to rank the relevance of criteria or indicators via a 5-point Likert scale wherein (1)—not at all relevant, (2)—slightly relevant, (3)—moderately relevant, (4)—very relevant, and (5)—extremely relevant, and the questions are open for additional suggestions or comments [57]. For the selection of experts, we used a purposive sampling method, focusing on individuals with special expertise in the field of smart cities, sustainable cities, or smart city assessments [58]. The data collection process begins by contacting potential experts via email and short message. If they agree, a questionnaire is sent. The number of invited experts is regulated in such a way so as to ensure adequate representation in order to guarantee the validity of the results. Next, the responses from experts are summarized and analyzed using the measures of central tendency and variability method [56]. If necessary, further discussions are held with the experts. The results of this analysis are used to determine the ranking and final selection of dimensions, indicators, and keywords. As a rule of thumb, this study used a mean score of 3.7 or more as a cut-off for selecting criteria and indicators to minimize the variability in the responses from the experts [59]. Profiles of the experts involved in our validation process are shown in

Table 2. Expert profiles.

No.	Position	Country	Expertise	Experience
1	Associate Professor	Indonesia	Green IT, e-government, smart cities, e-learning, and IT public services	10–15 years
2	Professor	Indonesia	Computer vision, information systems, human factors, and smart cities.	>20 years
3	Associate Professor	Indonesia	Open government data, smart cities, network security, and digital forensics investigations.	15–20 years
4	Associate Professor	Indonesia	Open government data, smart cities, data mining, information systems, and technology adoption.	15–20 years
5	Associate Professor	Malaysia	User experiences, human–computer interactions, sustainability, and gerontechnology.	5–10 years
6	Associate Professor	Malaysia	Information systems, project management, and sustainable governance.	10–15 years
7	Professor	Indonesia	Smart system platforms and ecosystems, IT architecture and governance, and smart cities.	>20 years
8	Professor	Malaysia	IT governance, urban development, social media, data analytics, and fintech.	>20 years

.

The final stage is the preparation and analysis of an assessment model with a special focus on urban sustainability. The model was formed based on the Smart City Reference Architecture Framework, which organizes the assessment structure into three main components: background (issues), objectives (goals), and assessment metrics (measures) [60]. The ‘issue’ component discusses the urgency and importance of conducting sustainability assessments in cities that have implemented the smart city concept. Because smart city implementation requires significant resource investment, evaluation from multiple points of view, including citizen perspectives through crowdsourced data, becomes very important. The ‘goals’ component defines the objective of the assessment, which, in this case, is the sustainability of the city. This goal is expressed through various dimensions and indicators (in this context, referred to as Key Goal Indicators (KGI) and Key Performance Indicators (KPI)). In the ‘measures’ component lie the metrics or data that will be used for assessment. In the context of this research study, the main focus is on crowdsourced data, which are collected based on the keywords that have been developed for each indicator. The details of all these stages will be further described in the Results and Discussion sections of the present study (Section 4 and Section 5). Thus, the model not only offers a framework for assessing urban sustainability but also explains how the model is expected to function in practice. The model uses a smart city reference architecture, as shown in Figure 7 [60].

4. Results

Based on the research procedures outlined in the Methods section, the following section explores the results achieved in each research phase, beginning with the dimensions and indicators and ending with the formation of the model.

4.1. Taxonomy of Dimensions and Indicators

The taxonomy is formulated based on thematic indicators emphasizing three sustainability dimensions: social, economic, and environmental. These three dimensions represent the output in the smart city input–output model. Within the smart city ecosystems, this output takes the form of services (systems) where interactions occur between urban planners and citizens. These systems represent General Systems Theory (GST), which serves as a comprehensive framework for understanding the dynamics of systems within smart city ecosystems. This is rooted in the contrast between traditional cities and their smart counterparts [44,61]. In essence, a traditional city is characterized as an expansive and enduring human habitat that comprises multifaceted systems encompassing vital domains such as sanitation, utilities, land allocation, housing, and transportation. However, this accelerated development also results in challenges in effectively orchestrating urban expansion [43]. Conversely, a smart city is an amalgamation of diverse systems that are interlinked and divisible into interdependent systems. Examples include the interlinked system of energy grids and power generation within the energy realm and vehicles and infrastructure components within the transportation realm [4].

The urban ecosystem, in the context of the smart city paradigm and considering GST, is guided by the need to address urban demands (urban needs). This is intensified by the interplay of technology-driven advancements (technology push) woven into the input–output framework of the smart city model [41]. This gives rise to a process through which a smart city systematically cultivates sustainability, encompassing three fundamental dimensions: the societal, economic, and environmental realms. Thus, the dimensions that take center stage in crafting a comprehensive taxonomy, forming the bedrock for the development of this assessment model for urban sustainability, closely align with the outcomes set forth by the smart city model’s input–output structure, spanning social, economic, and environmental considerations. These three dimensions collectively serve as the benchmark (triple bottom lines) for evaluating the sustainability of a city [11,24,35].

The development of the indicator taxonomy should be guided by the established dimensions. The selection of indicators is outlined in Figure 6. This ensures that the indicators align with the social, economic, and environmental dimensions. These indicators are derived from a diverse array of artifacts, encompassing taxonomies, frameworks, and models linked to the assessment of smart cities, as evidenced in reference [53]. These sources amount to a comprehensive knowledge base with a total of thirty-one (31) entries. It is important to note that these sources undergo a curation process wherein inclusion is based on the presence of at least one dimension (social, economic, or environmental). Taxonomies, frameworks, or models that fail to meet this criterion are excluded from the indicators.

The selected results consist of twenty (20) taxonomies that serve as the artifacts used to develop indicators for this study, as shown in Table 1. The indicator taxonomies for each individual dimension are created through a process of categorization wherein indicators that share semantic affinities or contextual correlations are unified. This is carried out to prevent redundancy, given the magnitude of indicators—totaling eighty (80) for the social dimension, ninety-five (95) for the economic dimension, and one hundred and twelve (112) for the environmental dimension. The integration of indicators, guided by contextual information, serves as the foundation for the delineation of indicator themes. This strategic selection is geared towards procuring tangible insights when intertwined with crowdsourced data. Notably, the social dimension emerges as a focal point of diversity, encompassing a comprehensive spectrum of five distinct indicator themes: Equity, Health, Education, Security, and Culture and Equality (

Table 3. Indicators and themes for the social dimension.

Indicators	Themes
Asset equity [9,10] Housing [6,9,10,11,12,14,15,36,38,39] Social inclusion [6,11,14,36,38,39] Price of property [13]	Equity
Health [6,9,10,11,13,14,15,36,38,39] Health Status [12] Hospitals [13] Mortality [13,15] Nutritional regime [15] Sanitation conditions [15] Drinking water [15]	Health
Education [6,9,10,11,14,36,38,39] Educational level [15] Literacy [15]	Education
Security [9,10,12] Population [9,10,15] Safety [6,11,12,14,36,38,39] Crime rate [13] Unemployment [13] Global Peace Index [13] Global Slavery Index [13] Government response to situations of slavery [13] Terrorism [13] Violence [15]	Security
Culture [6,11,14,36,38,39] Female workers [13] Happiness Index [13] Gender equality [15]	Culture and Equality

).

The theme of Equity comprises the concept of asset equity, supported by housing, social inclusion, and property prices, offering a multidimensional perspective on the subject. Housing is related to the quality and accessibility of housing options. The notion of social inclusion underscores the imperative of fostering a diverse and integrated society. Furthermore, property prices introduce a critical lens, considering how property values influence equitable resource distribution. Together, these elements constitute Equity and its connections to various facets of society.

In the domain of well-being, the exploration of Health takes on a holistic approach, considering an array of factors that shape the wellness of individuals and communities. The central pillar, Health, focuses on vitality and well-being. Health status delves into specific health conditions experienced by individuals. Hospitals are important institutions that play a critical role in providing healthcare services. Mortality statistics provide insights into life expectancy and the broader effects of health on longevity. Nutritional regimes shed light on the impact of dietary patterns on health outcomes. Sanitation conditions contribute to disease prevention by ensuring a hygienic environment. Additionally, the availability of drinking water stands as a requirement for maintaining robust health. These interconnected elements together constitute the thematic framework of Health.

On knowledge and growth, the theme of Education draws from education, educational level, and literacy. At its core, Education encapsulates the broader concept of learning and knowledge acquisition. Educational level explores the stages and degrees of education attained by individuals, reflecting their educational journey. Literacy emphasizes the ability to read, write, and comprehend information, playing a role in personal development and societal progress. Collectively, these elements are connected to personal growth, intellectual enrichment, and the overall advancement of communities and nations.

Regarding safety and stability, the thematic framework of Security comprises the concepts of security, population dynamics, safety measures, crime rates, unemployment rates, the Global Peace Index, the Global Slavery Index, government responses to slavery, terrorism, and violence, collectively shaping a multidimensional understanding of security. At its core, Security embodies protection and well-being. Population considerations offer insights into the demographic context that influences security dynamics. Safety emphasizes safeguarding individuals and communities from harm. Crime rates and Unemployment reflect the interplay of socio-economic stability with overall security. Metrics such as the Global Peace Index and the Global Slavery Index offer nuanced perspectives on global stability and human rights. Government responses to slavery underscore the role of governance in addressing violations. Terrorism and violence shed light on threats to security and societal cohesion. These elements reveal the connections between societal well-being, governance, human rights, and global stability.

Lastly, Culture and Equality consist of the concepts of culture, female workers’ participation, the Happiness Index, and gender equality. This offers a nuanced lens through which to explore the interplay between cultural dynamics and the pursuit of equality. Culture, as the core element, encompasses the mosaic of traditions, values, and beliefs that shape societies. Female workers serve as a prism to examine gender roles, labor participation, and women’s empowerment within cultural contexts. The Happiness Index illuminates well-being and contentment within cultural and societal settings. Gender equality, a pivotal pursuit, underscores fairness and opportunities for all genders. Together, these elements reveal the connections between cultural heritage, societal inclusivity, individual fulfillment, and the ongoing journey toward greater equality.

Within the realm of the economic dimension, four main indicator themes emerge: Innovation, Income, Infrastructure, and Business Opportunity. The Innovation theme consists of eleven indicators, while the Income theme has eight indicators. Infrastructure has six pivotal indicators, while Business Opportunity has as many as twenty-two indicators, as presented in

Table 4. Indicators and themes for the economy dimension.

Indicators	Themes
Entrepreneurship and innovation [6,9,10,11,14,16,17,18,19,20,36,38,39] Ability to transform [17] Innovation Industries [12] Innovative spirit [17,21,23] Innovative output [34] Entrepreneurial enterprises [34]	Innovation
Availability of employment finding services [19] Employment [6,11,14,19,20,36,38,39] GDP estimate [13] GDP [13] GDP per capita [13,20] Labor Force Participation [12] Talent Pool [12] Human Capital [37]	Income
Local and global connection [9,10,16,21] ICT Infrastructure [6,11,14,36,38,39] Physical infrastructure [6,11,14,36,38,39] Headquarters [13] Use of information and communication technologies [23] Global interconnectedness [34]	Infrastructure
Productivity [6,9,10,11,12,13,14,16,17,18,21,34,36,38,39] Trade [6,11,14,36,38,39] Economic image and trademarks [17] Flexibility of labor market international embeddedness [17,21] Economic performance [15] Trading [11,15] Financial status [15] material consumption [15] Energy consumption [15] Economic Vitality and Planning [18] Online services made it easy to start a new business [19] E-commerce companies [19] Time required to start a business [13] Ease of starting a business [13] Motivation for early-stage entrepreneurial activity [13] Competitiveness [23] Socially responsible use of resources [23] Market size [37] Market Attractiveness [37] Business Environment [37] Ease of Doing Business [37] Public sector [6,14,36,38,39]	Business Opportunity

.

The theme of Innovation has Entrepreneurship and innovation as its core, encompassing the ability to transform, innovation industries, innovative spirit, innovative output, and entrepreneurial enterprises. These indicators create a comprehensive framework for understanding the multifaceted dynamics of innovation and its far-reaching impacts. Entrepreneurship and innovation serve as the basis, emphasizing the proactive and creative endeavors that drive progress. Ability to transform underscores the capacity to adapt and evolve in response to changing environments. Innovation industries delve into the sectors that fuel technological and creative advancement. Innovative spirit embodies the mindset of curiosity, experimentation, and pushing boundaries. Innovative output showcases the tangible results of novel ideas and approaches. Entrepreneurial enterprises showcase the role of business ventures in driving innovation-driven economies. Collectively, these elements reveal the connections between human ingenuity, economic growth, and the continuous pursuit of advancement.

The theme of Income consists of availability of employment finding services, employment, GDP estimate, GDP, GDP per capita, labor force participation, talent pool, and human capital. These indicators offer a comprehensive lens through which to understand the complex dynamics that shape individual and societal economic prosperity. The availability of employment finding services lays the groundwork for job opportunities and workforce support. Employment serves are the core, highlighting the level of participation in productive endeavors. GDP estimate and GDP provide macroeconomic insights into the overall economic activity and production. GDP per capita offers a measure of economic output per person, reflecting the standard of living. Labor force participation captures the engagement of the population in the workforce. Talent pool showcases the skills and expertise available within a society. Human capital emphasizes the knowledge and capabilities that contribute to economic growth. Collectively, these elements shed light on the connections between employment, economic output, individual potential, and the broader socio-economic landscape.

The thematic construct of Infrastructure includes local and global connections. Additionally, the construct also includes ICT Infrastructure, encompassing the digital frameworks that underpin modern communication and technology. Physical infrastructure includes the tangible systems that support societies, such as transportation and utilities systems. Headquarters serve as nodal points of administrative and have operational significance. The use of information and communication technologies showcases the role of digital tools in enhancing efficiency and connectivity. Global interconnectedness highlights the interdependence and collaboration across borders. Collectively, these elements offer a perspective on the foundational systems that enable societies to function, connect, and thrive in an increasingly interconnected world.

The thematic construct of Business Opportunity consists of productivity, trade, economic image and trademarks, flexibility of labor market international embeddedness, economic performance, trading, financial status, material consumption, energy consumption, economic vitality and planning, online services that make it easy to start a new business, E-commerce companies, the time required to start a business, the ease of starting a business, motivation for early-stage entrepreneurial activity, competitiveness, the socially responsible use of resources, market size, market attractiveness, business environment, the ease of doing business, and the public sector. These indicators allow for a greater understanding of the dynamics that define business opportunities and their impact on economic growth. The amalgamation of these elements underscores the interrelationships between productivity, trade, technological innovation, market conditions, regulatory environments, and economic values. Collectively, they compose the theme of Business Opportunity, revealing the web of factors that shape entrepreneurial endeavors, economic prosperity, and the broader business landscape.

The subsequent thematic indicators are closely related to the environmental dimension, which is organized into three distinct themes: Air, Energy, and Public Facilities. The theme focusing on Air Quality consists of fifteen indicators, while the Energy theme is derived from six key indicators. The theme of Public Facilities has thirteen indicators. These themes are presented in

Table 5. Indicators and themes for the environment dimension.

Indicators	Themes
PM2.5 and PM10 [13] Pollution [13,17] Air quality [6,14,15,19,36,37,38,39] Availability and quality of apps for air pollution monitoring [18,19,21] Air pollution index [20] Volume of CO2 emissions [13,22] Pollution control [23] Quality of air and water [6,11,13,14,15,23,36,38,39] Monitoring emissions [23] Industrial wastewater [24] Industrial waste gas emissions [24] Industrial solid waste discharge [13,24] Discharge of hazardous waste [24] Natural Environment [37] Noise [6,11,14,36,38,39]	Air
Recycling [19] Renewable energy production [19] Energy consumption [19] Energy management [18,23] Energy [6,11,12,14,36,38,39] Energy Efficiency [21]	Energy
Attractivity of natural conditions [17] Sustainable resource management [17] Environmental protection [17] Basic sanitation quality [19] Smart building and renovation [12,21] Urban and Resource planning [21] Expenses for urban amenities [22] Green area per capita [15,20] Level of waste reuse and recycle [20] Improvements of waste discarding [23] House and facility management [23] Vehicle for city environmental [12,24] Environmental Quality/Sustainability [6,14,18,36,38,39]	Public Facilities

.

The theme of “Air” consists of PM2.5 and PM10 particulate matter, pollution, air quality, availability and quality of applications for air pollution monitoring, air pollution index, the volume of CO₂ emissions, pollution control, air and water quality, emission monitoring, industrial wastewater, emissions of industrial waste gases, discharge of industrial solid waste, the release of hazardous waste, the natural Environment, and noise. These are related to the dynamics of air quality and environmental well-being. These interrelated elements encompass the physical constitution of the atmosphere, the extent and impact of pollution, technological strides in monitoring practices, and the regulatory endeavors to manage emissions and waste discharge. These elements form the theme of “Air” combining air quality, environmental sustainability, and the overall vitality of ecosystems and communities.

The theme of “Energy” consists of renewable energy production, energy consumption, energy management, energy, energy efficiency, and recycling. These allow for a greater understanding of the dynamics of the production, consumption, and sustainability of energy resources. Renewable energy production encapsulates the utilization of environmentally friendly sources like solar, wind, and hydroelectric power. Energy consumption reflects the societal demand and utilization of energy for various purposes. Energy management delves into strategic planning to optimize energy use. Energy is the foundational unit that powers countless aspects of modern life. Energy efficiency underscores the use of energy resources to minimize waste. Recycling highlights the circular economy concept that repurposes and reuses materials to conserve energy. Together, these elements constitute Energy, showing the relationships between resource availability, environmental impact, technological innovation, and the pursuit of sustainable energy practices.

The theme of “Public Facilities” consists of the allure of natural surroundings, the stewardship of sustainable resources, environmental safeguarding, the caliber of fundamental sanitation, intelligent construction and refurbishment, urban and resource planning, investments in urban amenities, per capita green space availability, the degree of waste reuse and recycling, enhancements in waste disposal practices, management of residences and facilities, vehicles for urban environmental initiatives, and the continuum of environmental quality and sustainability. These result in a comprehensive framework that elucidates the infrastructure and strategies needed to enhance the quality of communal spaces and the overall environment. From the preservation of natural aesthetics to the pursuit of sustainable resource stewardship and from sanitation standards to pioneering technological advancements, these elements shape the theme of “Public Facilities”, showing the relationships between resource allocation, waste management, urban development, and the holistic prosperity of societies.

The progression from the thematic indicators leads to the subsequent stage: the creation of a taxonomy that encompasses dimensions and indicators (guided by MCA methodology). Primarily, this entails the establishment of the fundamental taxonomy. The focal point of the principal taxonomy is the notion of a sustainable city, which serves to evaluate the feasibility of city sustainability. A sustainable city, in essence, embodies the fulfillment of urban requisites while safeguarding prospective interests [62]. Subsequently, the identification of criteria within this study is linked to the dimensions of sustainable cities. These dimensions encompass a triad of facets: social, economic, and environmental. The social dimension centers around the exploration of both the affirmative and adverse repercussions stemming from the urban ecosystem, the processes, establishments, and undertakings of citizens in their communal existence [63]. On the other hand, the economic dimension delves into the impacts tied to the efficient and conscientious utilization of urban resources [64], evaluated through the lens of economic performance [63]. Meanwhile, the environmental dimension explores the effects emanating from the city’s endeavors towards achieving environmental sustainability within the complex tapestry of the urban ecosystem [64,65].

Moving ahead, the third facet involves the identification of indicators as dynamic variables or integral constituents for each dimension. Within the framework of this taxonomy, these indicators encompass the full spectrum of thematic indicators, spanning the realms of the social, economic, and environmental dimensions. In consequence, the outcomes yielded by the taxonomy of dimensions and indicators metamorphose into the foundation for the evolution of a comprehensive model for evaluating the sustainability of a city, as depicted in Figure 8. The social dimension encapsulates Equity, Health, Education, Security, and Culture and Equality. The economic dimension encompasses innovation, income, infrastructure, and prospects for business pursuits. Finally, the environmental dimension encompasses the elements of air, energy, and public facilities.

Figure 8 presents a comprehensive taxonomy that serves as the foundation for constructing a sustainable city assessment model. This taxonomy is anchored in three primary dimensions—social, economic, and environmental sustainability—and encapsulates a total of twelve key indicators. For social sustainability, the pivotal elements encompass Equity, Health, Education, Security, and Culture and Equality. These indicators offer a holistic perspective of a city’s level of social welfare. Within the economic dimension, the study underscores indicators like innovation, income, infrastructure, and business opportunities, which are instrumental in deciphering the economic vitality of an urban setting and its potential for sustainable progression. In addressing environmental sustainability, the research scrutinizes indicators such as air quality, energy sustainability, and the availability of public amenities deemed crucial for evaluating a city’s ecological equilibrium.

In the broader scope of urban sustainability, it is imperative to align the objectives of sustainable cities with overarching policy goals, notably the Sustainable Development Goals (SDGs). These global goals accentuate the importance of amalgamating socio-economic and environmental facets that are intrinsically linked to the imperatives of climate change mitigation and adaptation. Such alignment ensures that cities not only strive for ‘smartness’ but also contribute meaningfully to global sustainability targets.

To foster comprehension, measurements for these indicators are deduced from evaluating the perceptions of the general populace. Such perceptions, hailing from views shared on social media and other platforms, are categorized as crowdsourced data. The sentiments of the public are discerned via sentiment analysis. To ascertain the precision and pertinence of the data, a keyword-centric selection process, correlating each viewpoint with the specific sustainability issue or indicator it is associated with, is recommended.

4.2. Keywords for Crowdsourced Data

Keyword formulation for crowdsourced data requires synchronization with the themes of indicators for each dimension. These keywords are pivotal for harvesting relevant crowdsourced data to assess the thematic core of the indicators. This keyword selection follows a detailed guideline, as mentioned in the Methods section. It considers factors such as synonymous meanings, terms closely related to the indicators, and the foundational indicators that underpin these dimensions. The resulting data from the keyword design focus solely on text relating to citizens’ opinions or perspectives on each indicator. These text data undergo sentiment analysis to weigh each established indicator, as presented in the taxonomy results regarding the dimensions and indicators.

Furthermore, this systematic approach guarantees the efficient and accurate aggregation of data scattered across social media. Due to this approach, the number of keywords varies across indicators. The proposed keywords for each indicator meet the set criteria but can be expanded if needed in the assessment process. A list of suggested keywords for each indicator is shown in

Table 6. Keywords for crowdsourced data.

Indicators	Keywords for Crowdsourced Data
Equity	equity, house, housing, apartment, property
Health	health, hospital, health center, nutrition, sanitation, drinking water
Education	education, literacy, schooling, campus, college
Security	security, unemployment, slavery, crime, criminality, peace, violence, terrorism, terrorist, terror
Culture and equality	culture, equality, population, female workers
Innovation	entrepreneur, company, innovation, technology, industry, transformation
Income	income, salary, employment, poverty rate, finances, talent, human capital
Infrastructure	infrastructure, cooperation, connections
Business opportunity	economic performance, consumption, market, trade, competitiveness, productivity, business
Air	air, pollution, emissions, defilement, waste
Energy	renewable energy, electricity, green industry, solar energy
Public facilities	green space, parks, city parks, vehicles, public transport, environmental facilities (equipment)

.

4.3. Expert Validation

The process of expert validation involves conducting a thorough evaluation of the appropriateness and significance of the taxonomy and keywords linked with the crowdsourced data. These components serve as the foundation on which the assessment model is created. The validation panel is composed of eight individuals. In the expert assessment, the verification procedure is carried out using a mix of verbal discourse and succinct elucidations. This process is streamlined through the utilization of a validation form instrument. The outcomes of expert validation are illustrated in Figure 9.

The outcomes of expert validation, as illustrated in Figure 9, show different scores for all indicators and keywords with insignificant differences. Notably, the “public facilities, energy, and air” indicators attain the highest mean score with 4.75, whereas the “income” indicator registers the lowest score of 4.13 on the one-to-five scale. This pattern agrees with the standard deviation data, underscoring that the “public facilities, energy, and air” indicator garners the lowest score, while the “income” indicator secures the highest rating. However, the results of our expert validation show that all indicators and keywords surpass the established threshold of 3.7. In essence, this shows the validity of all indicators and keywords across each dimension.

4.4. Sustainable City Assessment Model

The sustainable city assessment model was developed within the context of the globally recognized Sustainable Development Goals (SDGs), emphasizing sustainable development in terms of socio-economic and environmental components, especially those related to climate change mitigation and adaptation. This model was prepared using a smart city reference architecture framework, consisting of three components: issues, goals, and measures.

The Issues component addresses the outcome of the assessment process. For the sustainable city assessment model, the issues highlighted stem from the challenges depicted in Figure 3. These consist of validation, knowledge base, and understanding. Validation pertains to ensuring the smart city implementation aligns with the intrinsic requirements of the city’s ecosystem and global sustainability targets. The knowledge base underscores the invaluable contributions of urban planners, whose expertise forms the bedrock of decisions pertaining to sustainable urban development. Understanding emphasizes the imperative for urban planners to comprehend and evaluate interactions within city ecosystems, ensuring technological solutions tackle genuine urban challenges.

The goals component encompasses facets that measure the sustainability of cities. Urban planners define these objectives based on their comprehensive understanding of urban requirements and challenges, always ensuring alignment with broader global sustainability objectives, especially those set by the SDGs. Major goals embody the essence of sustainable cities, mid-goals delineate dimensions like social, economic, and environmental sustainability, while sub-goals denote specific indicators for each dimension.

Conversely, the measures component facilitates the assessment calculations for predetermined indicators in the sustainable city model. The interpretation of these measurements is crucial, and urban planners are at the forefront, ensuring these resonate with real urban challenges and policy goals. The methodology integrates crowdsourced data, keywords, and weighting. Crowdsourced data collated from diverse platforms provide a real-time pulse of urban sentiments, enabling planners to derive actionable intelligence. Keywords ensure that the data are relevant to the evaluated indicators. Advanced data analysis techniques, encompassing deep learning, semantic analysis, sentiment weighting, and exploratory data analysis, are deployed. This synergy of sophisticated tools and urban planners’ expertise guarantees assessment results that are both theoretically robust and pragmatically pertinent.

Deep learning ensures the precise categorization of crowdsourced data based on defined dimensions and indicators. Semantic analysis and sentiment weighting assign values to each indicator, mirroring the general sentiment towards specific urban challenges. Exploratory data analysis provides insights into data distribution, empowering urban planners in their policy cycle decision-making processes, particularly in designing and implementing “win-win” solutions and policy co-benefits targeting climate change mitigation and adaptation. Consequently, Figure 10 presents a model sculpted from this exhaustive process, underlining the significance of urban planners in sustainable city assessments focusing on a given city’s alignment with global sustainability objectives.

5. Discussion

Concluding on a broader note, it is essential to recognize the overarching global frameworks such as the SDGs, which provide guidance on urban sustainability. These goals emphasize the need for a comprehensive approach that addresses both socio-economic and environmental dimensions, especially those related to climate change mitigation and adaptation. As cities evolve towards ‘smart’ solutions, alignment with these international benchmarks ensures that progress is consistent with worldwide aspirations.

The smart city assessment model has been specifically designed to align with these global benchmarks and support sustainable city development. A pivotal aspect of this model is the integration of crowdsourced data. In this landscape, urban planners emerge as critical players, leveraging these models to bridge the gap between technological advances and the genuine challenges faced by cities. Their expertise ensures that insights derived from such models are tailored to the real needs of urban development and, more importantly, are embedded in the policy cycle decision-making process. This embedding guarantees that the use of crowdsourced data is geared towards the creation and implementation of “win-win” solutions and policy co-benefits, aligning with the broader objectives of sustainable development.

The development of this assessment model has been systematic, encompassing stages like selecting an assessment domain, determining indicators based on themes, structuring a taxonomy of dimensions and indicators, choosing keywords for each indicator, validation by experts, and culminating with the formulation of an assessment model. This process is rooted in design science methodology, ensuring the resulting model is both valid and credible. Furthermore, the emphasis on urban planner requirements ensures the crowdsourced data are utilized effectively, focusing on the primary objective of developing solutions that offer co-benefits in terms of socio-economic and environmental components of sustainable development, especially targeting climate change mitigation and adaptation.

Additionally, this investigation emphasizes the importance of the citizens’ perspective, acquired through crowdsourced data, in evaluating the smart city ecosystem. Their views on various facets of smart cities, including social, economic, and environmental dimensions, form a crucial component in instilling trust in policy decisions. This aligns with the foundational principles of smart cities where residents, city administration, and technology converge as the core pillars of urban development [66,67]. By anchoring insights from global frameworks like the SDGs into this understanding, one can ensure that urban strategies are comprehensive, addressing both local challenges and global sustainability goals.

In the context of smart cities, citizens play an instrumental role in driving urban sustainability [64]. Their active participation not only aids in sculpting superior city systems but also amplifies their sense of ownership and responsibility towards the urban environment. Urban planners value this citizen feedback as it offers them a firsthand insight into the sentiments and needs of the residents. By providing feedback on smart city program implementations through various platforms, they ensure these programs address societal needs [68,69], fostering a more collaborative and inclusive approach to crafting sustainable urban landscapes.

To capture society’s perspective via crowdsourced data, the assessment model presents a plethora of dimensions and indicators spanning social, economic, and environmental facets. In the social sphere, there are five proposed indicators: Equity, Health, Education, Security, and Culture and Equality. Equity is related to people’s views on the ownership of assets such as houses and other property [70]; Health includes health services ranging from hospitals to sanitation and access to clean water [71]; Education includes literacy and educational institutions such as schools and universities [69]; Security relates to people’s perceptions of security in the city, including in terms of crime and violence [72]; and Culture and Equality relate to society’s views on the role of culture and equality in everyday life [73].

From an economic perspective, there are four indicators, namely Innovation, Income, Infrastructure, and Business Opportunities, which are used to describe the economic conditions of a city from the perspective of citizens, based on data obtained through crowdsourcing. Firstly, ‘Innovation’ measures how cities innovate in the areas of entrepreneurship, technological progress, and industry [74]. Secondly, ‘Income’, focuses on a city’s financial resources, such as its GDP, wages, employment opportunities, and financial stability [75]. Thirdly, ‘Infrastructure’ measures the physical and non-physical connectivity of the city, involving aspects such as cooperation and connections [76]. Finally, ‘Business Opportunities’ reflects a city’s ability to support economic growth, assessing aspects such as productivity and the business environment [77]. It is worth noting that urban planners rely heavily on such economic indicators to shape their strategies, ensuring that urban development is not just sustainable but also economically viable.

Finally, in the environmental scope, there are three proposed indicators: Air, Energy and Public Facilities. Air indicators relate to air purity and its implications in urban environments [78]. This includes pollution, emissions, environmental contamination, and waste management [79]. Energy indicators are related to energy consumption and active efforts to use sustainable resources [80]. This includes various aspects, such as renewable energy, electricity use, environmentally friendly industrial practices, and solar power initiatives [81]. Finally, the Public Facilities indicator is related to the city’s proactive efforts to provide facilities that are aligned with the city’s environmental priorities [82]. This includes aspects such as the provision of green spaces, parks, urban gardens, efficient vehicle systems, public transportation, and the availability of other facilities and equipment in the urban environment [83,84]. Urban planners, being at the forefront of city development, utilize these indicators to ensure that cities not only grow but do so sustainably, ensuring a balance between development and environmental preservation.

The findings of this study provide an overview of the important role of dimensions and indicators in assessment models for sustainable cities. Urban planners equipped with such models can make more informed decisions, ensuring that urban development is holistic and caters to all facets of sustainability. These dimensions and indicators are specifically designed to support the assessment objectives, which focus on utilizing crowdsourced data from the community. Active community participation plays an important role in creating a sustainable smart city. Apart from that, the main advantage of this assessment model is its ability to obtain information about people’s views through keywords related to urban ecosystems in social, economic, and environmental aspects. Using this model approach in smart city assessments could provide one way to solve some of the challenges of how ecosystems in a city interact with each other.

Furthermore, the use of this model in the city sustainability assessment process offers an alternative solution in accessing real-time and cost-effective data, the accessibility of which has been limited in conventional evaluation techniques. Traditional methods, which often rely on scattered data sources, have historically been costly and unable to reflect real-time conditions in cities. Utilizing crowdsourced data opens avenues for all urban stakeholders to submit improvement recommendations. This not only empowers citizens by progressively recognizing them as active participants in policy making [85] but also signals a paradigm shift. Traditionally, cities have predominantly adopted a top-down approach, engaging various stakeholders such as businesses, governmental agencies, and city administrations with the aim of gathering public opinions and feedback on urban planning and development [86]. However, with the introduction of the proposed sustainable city assessment model, there has been a pivot towards a more inclusive bottom-up approach, emphasizing grassroots involvement.

On the other hand, the proposed model, if used in the assessment process, offers data that are cheap and accessible at any time, meaning the data are able to reflect the real conditions of the city. Urban planners, often constrained by budget and resource limitations, can greatly benefit from such cost-effective and real-time data, ensuring that their planning and development strategies are grounded in the current realities of the city. In addition, the resulting model is based on taxonomy and keywords, and it has been validated by experts in the field, thus providing reliable assessment results. Although expert validation significantly reduces the potential for subjective bias, it is important to note that the resulting models are not completely free from the influence of expert perspectives. The collective expertise and differing experiences of experts can still influence the results of these models.

6. Conclusions

This study aimed to develop a sustainable city assessment model using crowdsourced data. This model has been developed to overcome the difficulties in assessing smart cities, especially the challenges related to data collection in interactions between urban systems. Model development was carried out systematically using a design science methodology approach. The resulting model provides a comprehensive picture of the urban ecosystem, which is described through indicators for each dimension (social, economic, and environmental indicators). The model has also gone through expert validation to ensure the accuracy and relevance of the integrated dimensions and indicators. In addition, the architecture of this assessment model has been designed based on three main pillars: issues, goals, and measures. The architecture used was adapted from the smart city reference architecture to ensure that the model is not only easy to understand but also practical for implementation in the sustainable city assessments.

The contribution of this research paper to the understanding of smart cities is undeniably significant, especially in its focus on assessing urban sustainability. As cities worldwide grapple with the challenges of urbanization and climate change, such insights become increasingly invaluable. Building on this contribution, it is crucial to highlight the central role of urban planners in shaping and implementing sustainable urban models. As the main stakeholders responsible for the development, definition, and delivery of sustainable urban solutions, their insights and expertise should be at the forefront of any urban sustainability discussions. This study acknowledges this imperative and strives to ensure that their voices and perspectives are integrated at every stage of model implementation. However, further studies need to be conducted to empirically prove the effectiveness of the resulting model in the assessment process while building an analytical model. Moreover, for future studies, the model can be expanded to larger areas, linking it with the smartness aspect of a city, such as the Internet of Things (IoT) in urban planning and the involvement of artificial intelligence in smart city sustainability. In the future, the synergy between sustainability and smartness will play a pivotal role in shaping future urban landscapes, ensuring they are both resilient and efficient.