4.2.1. Review of Research Model

In this part, the survey results of all respondents (*n* = 408) in both cities were inputted. As mentioned above, the convergence validity of the SEM was determined based on the outer loadings (*Li*), the reliability of the measurement variables (*L*<sup>2</sup> *<sup>i</sup>* ), and the Average Variance Extracted, or AVE.

According to the analysis of the initial model shown in Table 6, the outer loadings (*Li*) came below 0.5 in case of "the expectation level for reliability", which corresponds to variable No. 3 within "user expectation." Accordingly, this research excluded the variable from the research model, and carried out the analysis again, which led to the results that the outer loading value stood at 0.07 or higher, and therefore proved its validity and reliability. The analysis of reliability *L*<sup>2</sup> *<sup>i</sup>* showed the level of 0.5 or higher to guarantee its credibility, and the AVE value also comes to 0.5 or higher to secure the reliability of the individual variables and the validity of latent variables.

**Table 6.** Convergent Validity Result of Barcelona and Songdo (*n* = 408).


The internal reliability evaluation of each variable is shown in Table 7, and the Cronbach's Alpha, rho\_A and CR values came to 0.6, 0.7, and 0.6 or higher, respectively, indicating that all potential variables have internal reliability. Moreover, the results of the discriminant validity by Fornell-Larker and the Chi-square test also turned out to be suitable, though they are not presented in this research. As the cross-validity of latent variables by HTMT criteria did not include one within the 90% confidence interval, each variable can be said to secure the discriminant validity. As such, the SEM proposed in this study secured consistency, reliability and validity for each item. Then, the PLS-SEM algorithm and bootstrapping were executed to analyze the SEM and derive the outer loading (regression coefficient).

As shown in Table 8, "satisfaction" revealed the highest explanatory power (0.694) based on the modified R2 criteria, which means that this research model is best optimized for measuring the actual satisfaction level. In addition, the VIF values of all variables in this SEM came to less than five, and no inter-multicollinearity occurred. Each measuring variable included in the structural model also came to less than five, indicating that there was no inter-multicollinearity.


**Table 7.** Result of Reliability-Validity of SEM's Latent Variables (Barcelona and Songdo, *n* = 408).

<sup>1</sup> Not includes 1 in Confidence Interval.

**Table 8.** Result of Coefficient of Determination on Latent Variables (based on R2 and Adj. R2).


As shown in Table 9, all the single paths presented in this study model were found to have explanatory power within the scope of being statistically accepted. When setting the respondents' satisfaction as a final path, the highest value (based on t-stat.) turned out to be "cognition" of technologies, facilities, and services of a smart city, which was emphasized by this study; this was followed by "Perceived Value (PV)", "User Characteristics (UC)" and "User Expectation (UE)".

**Table 9.** Result of Paths on a Pair of Latent Variables (Barcelona and Songdo, *n* = 408).


\*\* *p* < 0.000, \* *p* < 0.05.

That is, citizens' satisfaction can be maximized by the fair level of cognition of and accessibility to the smart city elements (t = 7.505, *p* = 0.000). This also indicates that the value of each TFS that citizens think of for their expenditure (t = 5.373, *p* = 0.000), socioeconomic features such as age, gender, and income level of citizens (t = 4. 929, *p* = 0.000) and their overall expectation level with a smart city (t = 4.472, *p* = 0.000), can lead to their higher satisfaction. These results suggest that there is a need to better consider such factors as citizens' age, gender, and income (based on questions about how much each smart city infrastructure is exposed to citizens), when drawing a plan regarding smart city elements

from citizens' perspectives. It also means that, if charging for those technologies, facilities and services are needed, and economic reviews on whether the price was affordable for the current residents are needed. It is safe to say that the level of satisfaction formed through this consideration would lead to their continuous use and utilization of smart city materials (t = 4.929, *p* = 0.000) based on its reliability (t = 13.419, *p* = 0.000).

#### 4.2.2. Analysis of All Paths of Research Model (Barcelona and Songdo)

The individual paths are determined by the mediating effect of each variable inserted into the structural model. The path model presented by this research as well as the value and the flow of the path set by 5000 times of bootstrapping are shown below. As previously mentioned, this research model is based on the SEM that evaluates images of traditional goods, services, or their suppliers, and it involves users' expectations for smart city elements, perceived quality and value of goods after use, and chances of their future utilization based on the level of satisfaction. In this regard, as shown in Figure 3, this research model proves that technologies, facilities and services of a smart city can be considered as traditional goods and services and they can be evaluated from the users' viewpoint.

**Figure 3.** Overall result of SEM based on Expected Research Model (*n* = 408, based on T-stats. and bootstrapping).

There exists a lot of research models, paths and analysis methods presented by bootstrapping. As the latent variable in each index group shows the generalized representation in accordance with the increase in the value, however, this study is to take the upper value of each index group and their index variables which affect the satisfaction for interpretation. First, each citizen's characteristics affect their satisfaction with the TFS, with the impacts becoming greater according to their gender, followed by age and income level. Second, (1) user expectation, (2) perceived quality after use, and (3) perceived value based upon the perceived quality all affect their satisfaction, but the perceived value, which combines (1) and (2) turned out to have greater impacts on users' satisfaction. That is, the perceived value, which measures how appropriate their payment for those smart city elements was, impacts on their satisfaction the most. Third, "Accessibility" to each facility and service directly affects users' overall satisfaction. Access directly to goods when necessary contributes more to the improvement of their satisfaction than any other indirect methods, such as promotion campaigns. Fourth, users' intention of continuing the use of the TFS can be boosted when the goods were better than what users expected. Given that the installation of smart city infrastructure is still in the early stage, and many see chances of

its development further, it is crucial to improve the quality of the TFS beyond people's expectations, rather than focusing on quantitative satisfaction.

#### *4.3. Analysis Results of Research Model for Each City*

When the analysis results of the two cities were combined and then inserted into the research model, it was identical to the initial model presented by this study. However, as mentioned in the beginning, o this structural model may reveal differences when the results of each city are separately put into the research model to consider the fact that there are differences in their business methods.

In particular, it is necessary to note that the above-mentioned various Customer Satisfaction Indexes are similar in their basic structure, but have been modified and sometimes improved to fit for each country or an institution. So, the chances are that, considering business characteristics of urban planning projects or economic and cultural differences caused by their geopolitical locations, the basic model proposed above may be different among cities. Accordingly, in this section, the two subjects were analyzed in the same way through the execution of bootstrapping based on the SEM used before.

In the analysis process, based on the evaluation results of the significance and the suitability of each path presented by the bootstrapping results, the low value among the variables within the statistically insignificant latent variable group was first removed. In case the result was not significant, the same process was repeatedly executed—after removing the latent variable group and re-executing bootstrapping—until significant results were derived. The results and differences of the two cities using this process are as follows.

#### 4.3.1. Evaluation and Comparison of Measurement Model

Through the above-mentioned process, analysis continued by roving variables until the statistical suitability was obtained. One variable group (LV) was removed from each research model of the two cities, and the final verification results for the validity showed that both cities show higher values in the outer loading (*Li*), reliability (*L*<sup>2</sup> *<sup>i</sup>* ) and AVE (Average Variance Extracted) than the average. In regard to the details, all measurement items in the "Perceived Quality" of Barcelona were removed as they failed to explain the path of the SEM, and all items in the "user characteristics" in Songdo were taken out due to their failure to explain the path of the structural model.

After excluding the variable group that does not match the research model as shown in Table 10, the review of the reliability and discriminant validity (HTMT) of the measurement indicators for the results of the two cities were conducted, which showed that the receptive reliability and the validity were secured in most items. In the case of Songdo, Rho\_A, an item for evaluating the reliability in the Accessibility group, came below the standard value. However, it still satisfied the CR value (0.6 or more) which is the baseline of the representative reliability set by this study. It also exceeded the standard value in the calculation of Cronbach's Alpha value (0.6 or more) and HTMT (90% confidence interval 1), which are deemed to have the strictest explanatory power, so that it was determined to be acceptable (see Table 11).

After removing groups with a low explanatory power and verifying the consistency and reliability of each city, this study conducted bootstrapping 5000 times with the results, and analyzed the structural model and then derived the outer loading (standardized regression coefficient) as follows.

Here, as shown in Table 12, the approach is a control variable of the "Satisfaction" so that it was not included in calculating the regression coefficient. The analysis results showed that the satisfaction has the highest explanatory power for both models in terms of R2 standards, proving that both research models are suitable for measuring the satisfaction of technologies, facilities and services of a smart city. In addition, the value of multicollinearity (variance inflation factor, or VIF) of all variables included in the SEM of the two cities came between 1.000 and 2.555 to hover below 5, which was an acceptable result because each variable was not highly correlated with others.


**Table 10.** Convergent Validity Result of Barcelona and Songdo (*n* = 408).

**Table 11.** Result of Reliability and Validity of SEM's Latent Variables.


**Table 12.** Results of Each Model's Coefficient of Determination on Latent Variables (based on R2 and Adj. R2).


The suitability of each single path is as shown in the Tables 13 and 14, and all the paths were found to be statistically significant, and no multicollinearity problem was generated, which guarantee the explanatory power for each path. Similarities and differences drawn from the results between the two cities in terms of each independent path are as follows. First, in the single route that ends with the satisfaction, accessibility has the greatest impacts on the satisfaction in both cities, which is the same result as the analysis of their converged data. This reaffirms that the TFS of a smart city will be able to boost the satisfaction of users or consumers when they are properly placed at a location they want or appropriately guided.


**Table 13.** Results of Paths on a Pair of Latent Variables (Barcelona).

\*\* *p* < 0.000, \* *p* < 0.05.

**Table 14.** Results of Paths on a Pair of Latent Variables (Songdo).


\*\* *p* < 0.000, \* *p* < 0.05.

Second, in the single path of Barcelona, the relations between socioeconomic characteristics of citizens (t = 11.914, *p* = 0.000) and their satisfaction, as well as the satisfaction and expectation for resolving complaints (t = 11.302, *p* = 0.000), showed quite high values. That is, given that citizens' age, gender, and income levels affect the satisfaction level, it is crucial to introduce technologies, facilities and services that best fit for citizens' characteristics in case of established cities. It also can be inferred that listening to users' complaints and providing a swift resolution could enhance the sustainability of a smart city from citizens' perspective. Therefore, smart city-related technologies, facilities and services in established cities should consider the current status of the urban space and features of citizens, such as their age, gender, and income level, and it is advised to supply fresh technologies, facilities and services, which proactively reflect the citizens' opinions. This implies the need for the comprehensive consideration of both physical and social factors of a city, including its form, related policy measures, and unique characteristics.

Third, in the single path of Songdo, the path between user expectation and perceived quality (t = 15.035, *p* = 0.000) showed a particularly high value compared to others. This indicated that perceived quality, which means the level of satisfaction of individuals' expectations, is crucial in Songdo, in contrast to Barcelona where the factor was excluded. Citizens' personal features are important in both areas, but their current social and economic circumstances turned out to specifically affect the satisfaction in Barcelona, while users' judgments on individual values is the decisive factor in Songdo.

In comparison to Barcelona, Songdo excludes the demographic and economic features of each citizen from the structural model. What this suggests is that a project to build a new city needs to identify a wider range of demands expected to be made by an unspecified number of people, and to prioritize advanced functions, design, and the usability of advanced smart city infrastructure.

#### 4.3.2. Analysis Results of Research Model Routes

The value of an individual path is determined by the mediating effect of each variable inserted into the structural model. The results of the two cities, as shown in Figures 4 and 5, which are summarized in accordance with the path model presented here, and the value and the path flow determined by 5000 times of bootstrapping, are as follows. As explained in the survey results involving the two cities, the level of satisfaction with smart city elements in both cases is set by users' expectations before their usage and their perceived value after the experience, based upon accessibility, and the satisfaction has a series of paths affecting the possibility of their continued use. This highlights the need for the TFS of a smart city to be considered as just the same as traditional public goods, and from the perspective of users, i.e., citizens.

**Figure 4.** Overall result of SEM (Barcelona, *n* = 212, based on values of T-stats. and bootstrapping).

**Figure 5.** Overall result of SEM (Songdo, *n* = 197, based on values of T-stats. and bootstrapping).

However, the two cities showed structural differences. First, the satisfaction in Barcelona, where the TFS were introduced in existing urban spaces, was decided by users' expectations before their firsthand use and their perceived value after the experience, which was affected by individual's characteristics. In the case of Songdo, where a new city development project is under way, individual user's characteristics are not factors to be considered, but the quality of the goods evaluated after use leads to their perceived value and satisfaction. However, given that perceived quality is not suitable for the structural model in the case of Barcelona, smart city infrastructures in an established city are needed to be chosen after taking into consideration the features of each citizen, because their expectations for the goods and their value in comparison with users' payment could boost their satisfaction in such an established city.

Among major variables, first, in terms of users' expectation, the importance of the reliability of smart city infrastructures was emphasized in both Barcelona and Songdo. Reliability is expected to improve their level of satisfaction with their expenditure for the goods. Second, the perceived value, which is decided after their experiences, was significantly higher in both cities in terms of "the appropriateness of the price set for facilities" than "the appropriateness of facilities compared to costs." This could indicate that individuals may drop their intention to use the smart city facilities and services in case the price set is higher than the amount they are willing to pay. So, any decision to charge goods or to add additional charges would be required to be made carefully, in order that they do not incur the improvement of the goods.

Third, "overall satisfaction" is the most important factor, among its three items, for technologies, facilities, and services for a smart city to be introduced to existing urban spaces, such as Barcelona. However, in the case of newly built cities, such as Songdo, their satisfaction "compared to the best level of the goods" was higher than their overall level of satisfaction or previous expectations. This result implies that if this research model is verified through further studies as a tool to compare a new city with an established one, the quality improvement in the form, design, and service of the smart city elements could enhance users' satisfaction, as it will better reflect citizens' expectations.

#### **5. Implication and Discussion**

This study aims to monitor users' level of satisfaction with the technologies, facilities, and services of a smart city at a time when the paradigm and the focus of a modern smart city has been changing from technology to citizens. It also aims to present a standardized evaluation system and verify it, so as to explore the possibility of its application down the road. For this purpose, the Structural Equation Model was established based on the basic structural model of the Customer Satisfaction Index, which has been widely used across the world, as well as on recent theories. A survey was also conducted involving citizens in two cities to compare their results. In order to draw implications in terms of urban planning, analysis results of the two cities were compared: one is where a smart city infrastructure was established in an existing urban space, and the other is where a smart city project was implemented in a newly built town.

As a result, the combined results of the examination of Songdo and Barcelona showed the identical structure to the prediction model based on the Customer Satisfaction Index. In other words, from citizens' point of view, the level of satisfaction with a smart city is affected by users' expectations for its technologies, facilities, and services before having first-hand experiences, their perceived quality after use, cost of the goods, and the accessibility to each facility. Those factors, as well as individuals' socio-economic characteristics, were found to have guaranteed the possibility of their continuous use. The results indicate that it is important to prioritize the introduction of facilities that residents want in order to achieve a citizens-centric smart city, just as highlighted in many studies. Such a phased introduction is advised in accordance with each city's demographic, social and economic features, rather than applying advanced technology-based platforms competitively in a drastic manner, as was called for by some previous studies and companies.

Second, the results of the analysis of each city through the research model revealed differences from the basic model, just as forecasted, but their structural features bore similarities. In Barcelona, each citizen's socio-economic characteristics well explained their satisfaction with smart city infrastructures and their sustainability, while "Perceived Quality" was not suitable for the model. This means that characteristics of each dweller and their expectations can affect satisfaction. In the case of Songdo, where a new city development and construction project takes place, city members' socio-economic features do not fit for the model. Instead, the reliability of the goods, and moves to meet their personal expectations, were found to boost their satisfaction. From urban planning perspectives, these results indicate that that it is important to adopt credible and high-quality technologies, facilities, and services for a smart city, given the influx of unspecified people, while affordability for existing residents is key to effectively enhancing their overall satisfaction, just as many studies, such as [62,63], pointed out.

This study, however, has limitations in that the comparison of two cities, rather than multiple ones, could not ensure cited academic implications, so follow-up studies are deemed required by involving more entities of a similar scale, form, and conditions to expand this discourse. Many previous studies point to differences among smart cities in accordance with different urban spaces and business methods. At a time when discussions on the establishment of a citizens-centric smart city have expanded, it is crucial to spot differences among the technologies, facilities, and services of a smart city over the course of urban regeneration, redevelopment, and restoration projects, and to make suggestions to come up with proper policy measures suitable for each project [64]. Further requirements are the establishment and/or the revision of related legislation and systems in public sector, such as an opinion gathering system for planning, a monitoring and ex-post evaluation system focusing on citizens, and a certification and evaluation mechanism to maximize the application of those results to wider projects such as a national land planning. In order to create and manage a 'Sustainable Smart City', it is also necessary to extend financial support for the private companies or organizations which produce and distribute the smart city technologies, facilities, and services, as well as the expansion of partnerships between public and private entities, among other things [65]. This is the way to Smart City 2.0 and above, where citizens can actively participate and experience, going beyond Smart City 1.0, where citizens were limited to passive participation by technocracy.

In this regard, the research carries significance as it examined the standardized framework to evaluate smart cities from the citizens' point of view and carried out their basic verification to highlight that smart city infrastructures can be seen as a new commodity of a city and, at the same time, a means of improving the quality of citizens' lives. Moreover, this study has shed light on the point that a smart city would not be a matter of international competition but a useful way of improving the sustainability of a city or a region, which could be further examined by additional studies down the road.

**Author Contributions:** Conceptualization, J.O.; methodology, J.O.; software, J.O.; validation, J.O. and M.S.; formal analysis, J.O.; investigation, M.S.; resources, J.O.; data curation, M.S.; writing original draft preparation, J.O.; writing—review and editing, M.S.; visualization, J.O.; supervision, M.S.; project administration, J.O.; funding acquisition, M.S. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by a research entitled "A Study on Urban Green New Deal Implementation Using the Third Sector" to be published by the Korea Research Institute for Human Settlements in October 2021.

**Informed Consent Statement:** Informed consent was obtained from all subjects of survey involved in the study.

**Data Availability Statement:** Not applicable.

**Acknowledgments:** The research model and the result in this study are based on the research results of the first author's thesis 'Analysis of Smart City Service Evaluation Model in Citizen-friendly Aspect' (Department of Architecture, Korea University, 2019).

**Conflicts of Interest:** The authors declare no conflict of interest.
