*3.1. Research Framework and Model*

It is necessary to listen to citizens, who are end-users of services and the subject of reproducing public information, in order to build a citizens-centric smart city. In the field of academic research, the establishment of a method to systematically perform their opinion gathering is also needed. As [50] stated, there exists a wide range of differences in interpreting a smart city in terms of system, governance, information, space, and design, and even some theories clash. However, in the area of an actual smart city project, the standardization of its technologies, facilities, and services has been called for [51,52].

As such, this study aims to present a standardized research model, which is expected to present the analysis of citizens' use of related goods, as well as the implications for promoting their continuous utilization when related entities create, operate, and manage a smart city. It also aims to establish an evaluation model based on a widely used ex-post evaluation research model for consumers, and to lay the groundwork for standardizing tools to collect and assess citizens' opinions on smart city infrastructures. This research will also verify the evaluation model by carrying out surveys and their quantitative analysis based on the population and samples to discuss their future applications.

In order to establish the evaluation model, this study adopted the basic structure of the Customer Satisfaction Index. The structure of this research model differs according to usages by each entity, but this has significance as it would be a tool to explore consumers' cognition on public or private goods both in the past and at present; their experiences, and the possibility of their continuous use down the road.

Just as shown in Figure 1, major Customer Satisfaction Indexes include ACSI (American Customer Service Index), SCSB (The Swedish Customer Loyalty Barometer), ECSI (European Customer Loyalty Barometer), and NCSI (National Customer Service Index). They are based on the Structural Equation Modeling (SEM), which began to be discussed in the 1970s and was expanded to analyze service quality, or SERVQUAL, in 1980s [53].

These SEMs are used not only to measure corporate products, but also to evaluate the level of satisfaction [54] of both tangible and intangible goods provided by private and public entities as well as their brands. Despite some differences depending on countries, cities and evaluation agencies, the basic SEM structure for each index bears similarities [55].

Looking into details, the variables—"Customer Expectations" with goods, "Perceived Quality" that they evaluate after use of the goods and services, and "perceived value" of the goods for consumers' expenses—affect the overall and comparative "satisfaction," and those variables are defined as being interconnected. "Satisfaction" is also linked to "complaint", which means that users' satisfaction can be affected by suppliers' responses when they have problems with the goods. It also indicates their expectations of the responses

they receive when they have complaints. "Complaint" can then lead to sustainability, as the handling of complaints would affect consumers' decisions on whether to continue to use the supplies and/or recommend them to others. In this regard, the SEM structure is expected to help discover the perceived advantages and disadvantages of goods, which consumers felt in the past, feel at present, and will feel in the future, and to draw implications for the improvement of the goods.

**Figure 1.** Expected research model and directions of the paths; (**a**) United States (ACSI); (**b**) Singapore (CSISG); (**c**) EU(ECSI); (**d**) Republic of Korea (NCSI); (**e**) Sweden SCSB; (**f**) Norway (NCSB).

In other words, it enables a diachronic point of view in learning the formation of consumers' cognition regarding goods in the past; their current satisfaction; and a will to use in the future. Each item of the variable is also placed on a Likert scale, which could lead to various implications through the selective analysis of items when necessary, without using the SEM. In terms of standardization, the SEM in particular could show some differences between results shown in advanced nations and in emerging nations, just as [56] pointed out. However, it is noteworthy that the very basic structure is still effectively applicable to diverse industry fields within a country or a city [57]. In this context, if the SEM structure is applied to the evaluation of a citizens-centric smart city and it could secure statistical explanatory power, the smart city can be regarded as a kind of new goods, industry, or service of a country or a city. However, the traditional SEM model was modified here, as follows, to evaluate a smart city from citizens' point of view, just as shown in Figure 2.

First, the relationship among variable groups was re-established. The SEM for the above-mentioned Customer Satisfaction Index has already been verified through various studies. However, this study specifically considered citizens, who consume technologies, facilities and services of a smart city and produce related information. As such, "User Characteristics (UC)" was added to the modified version, just as shown in A, to see if each respondent's demographic, economic, and social features can be well reflected to the service evaluation model. In addition, the study directly linked the 'User Characteristics' to "user Satisfaction" to determine if citizens' characteristics affect their levels of satisfaction with a smart city, and if so, how much impact it would have, through which the research can draw implications for related studies on a citizens-centric smart city.

**Figure 2.** Expected research model and directions of the paths.

Second, evaluation items to measure the accessibility of a smart city's TFS from the citizens' point of view were added and verified. The insertion of these variables is based on revelations that smart city-related services provided in each country, city or region have not been frequently used, in case they are not well recognized by the people due to their low accessibility or usability [58,59]. This concept is similar to "affordance", a behavioral psychology term, which means a way to define individuals' behaviors through a series of experiences and knowledge. This study, however, puts a greater value on the accessibility to discuss users' overall access to a number of goods, rather than on a single item with a clear intention. In fact, smart city-related studies have concentrated on accessibility in multiple aspects, such as its improvement in order to promote citizens' mobility both indoors and outdoors [60]; the consideration of a greater number of populations to achieve a smart but still inclusive city [59,60]; and the public access to data collected, processed and disclosed through smart city tools.

In terms of the creation of a citizens-centric smart city, no matter how many technologies, facilities and services are adopted, they would ultimately be unsustainable if they fail to actually be utilized. So, smart city-related services are bound to be affected by their accessibility as well as being linked to citizens' intentions to experience them, and the citizens' satisfaction. Therefore, this study constructed the SEM by directly connecting "accessibility" with "satisfaction" and "complaint", just as shown in Figure 2. The variables and items of each group are stated in Table 1.

Indicators here are similar or identical to those in the above-mentioned consumer satisfaction model, and they are presented in the form of questions to enhance respondents' understanding. All the items are on the Likert 11 scale, ranging from 0 to 10, and all items are required to be answered. In case of "complaint", however, the items are designed to assess respondents' expectation of suppliers' problem solving because they may not have any complaints with the TFS of a smart city.

Along with the establishment of the evaluation model, this study presented for the questionnaire comprehensive information on the technologies, facilities and services of the smart city established in the target sites, to help respondents better understand what the TFS means, as some might have not recognized them even though they have used the items. So, the list of their TFS was drawn up based upon their administrative information, related

reports, and academic documents. Second, interviews were conducted with government officials, scholars, and researchers in the related fields of each city to redefine the TFS currently in operation, as is shown in Table 2 below. Third, the subjects of the survey were confined to those who have had hands-on experience with TFS, so as to increase the objectivity of this research and the verification ability of the evaluation model. In order to make sure that they have used the TFS, photos and other visual materials of the infrastructure were provided to the respondents.


## **Table 1.** List of Evaluation Criteria.

**Table 2.** List of Smart City Technology, Facility, and Service (TFS) of Barcelona and Songdo.

