Next Article in Journal
Assessing Green and Blue Water Footprints in the Supply Chain of Cocoa Production: A Case Study in the Northeast of Colombia
Previous Article in Journal
Valuing Urban Landscape Using Subjective Well-Being Data: Empirical Evidence from Dalian, China
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

How to Evaluate Smart Cities’ Construction? A Comparison of Chinese Smart City Evaluation Methods Based on PSF

1
School of Economics and Management, Harbin Institute of Technology, Weihai 264209, China
2
Zhongshan Institute, University of Electronic Science and Technology of China, Zhongshan 528402, China
*
Author to whom correspondence should be addressed.
Sustainability 2018, 10(1), 37; https://doi.org/10.3390/su10010037
Submission received: 16 November 2017 / Revised: 18 December 2017 / Accepted: 19 December 2017 / Published: 25 December 2017

Abstract

:
With the rapid development of smart cities in the world, research relating to smart city evaluation has become a new research hotspot in academia. However, there are general problems of cognitive deprivation, lack of planning experience, and low level of coordination in smart cities construction. It is necessary for us to develop a set of scientific, reasonable, and effective evaluation index systems and evaluation models to analyze the development degree of urban wisdom. Based on the theory of the urban system, we established a comprehensive evaluation index system for urban intelligent development based on the people-oriented, city-system, and resources-flow (PSF) evaluation model. According to the characteristics of the comprehensive evaluation index system of urban intelligent development, the analytic hierarchy process (AHP) combined with the experts’ opinions determine the index weight of this system. We adopted the neural network model to construct the corresponding comprehensive evaluation model to characterize the non-linear characteristics of the comprehensive evaluation indexes system, thus to quantitatively quantify the comprehensive evaluation indexes of urban intelligent development. Finally, we used the AHP, AHP-BP (Back Propagation), and AHP-ELM (Extreme Learning Machine) models to evaluate the intelligent development level of 151 cities in China, and compared them from the perspective of model accuracy and time cost. The final simulation results show that the AHP-ELM model is the best evaluation model.

1. Introduction

The globalization trend has greatly expanded the dimensions and populations of cities in recent years, and city planning and coordination has also been improved in a pluralistic way [1]. During this period, problems and contradictions of urban development have been increasing as the days have passed. Several issues have largely curbed the sustainable and harmonious development of cities, such as the aging of the population, the insufficient of per capita resources allocation, the urban traffic congestion, the decline of urban operation efficiency, the backward environmental governance, and the destruction of urban ecology and so on. Along with the development trend of cities and the need to urgently solve the contradictions, the management and development of a good city not only need the help of the traditional city management measures, but also require new technology which can be more scientific and harmonious [2,3].
Innovation and the development of new communication technologies such as Internet of things (IoTs), big data, and cloud computing have provided new solutions to cities’ governance and maintenance. They have built a new model for new cities’ construction [4,5,6]. Integration of information and industrialization have made traditional cities gradually evolve into a new form of ecological social organization. The smart city is based on this trend, and traditional industrial society is transiting to modern innovation society. It can be said that a smart city provides a reliable solution to this process. A smart city gives an optimal approach for a resource-conserving and environmentally-friendly society. From a technical point of view, smart cities can realize the computing and integration of a ubiquitous network, overall perception, and connectivity through the comprehensive application of IoTs, big data, augmented reality, and computing cloud technology. From the perspective of social evolution, the intelligent construction of a city is also needed through the Living Lab, spatial information grid, and data grid integrated application tools and methods, so as to realize background knowledge and open vision, thus coordinating the sustainable development of social form innovation.
In recent years, the global smart city has entered the stage of high-speed development, and it will gradually realize the vision of “total connectivity, comprehensive perception, and intelligence”. The sixth world smart city conference kicked off in Barcelona in 2016 [7], with delegations from more than 600 cities all over the world, and numerous enterprises and experts in communications participated in it. It predicted that by the end of 2016, global smart cities will be worth up to 40 billion US dollars. Additionally, there will be more than 20 countries enacting smart cities’ development planning policies by 2017. They will identify the investment project priority system matching with city information, and prioritize related information technology and the business process. The Prospective Research Report [8] indicated that there were 311 prefecture-level cities and cities in China working on smart city construction by the end of 2015. A total of 158 smart cities have been built and widely used and promoted in more than 70 areas. Therefore, at the planning level, China attaches great importance to the construction and development of smart cities.
In order to solve the problems of supply shortage and the urban infrastructure demand, lack of information resources standardization, insufficient network information security, lag of governance mode of urban government, and lack of technology in the industry driving effect, etc., our paper will summarize the origin and connotation of the smart cities, the urban systems theory, and the people-oriented, city-system, and resources-flow (PSF) evaluation theory. Combining the development characteristics of China’s smart cities, the comprehensive evaluation index system for urban intelligent development is established. Based on the analytic hierarchy process (AHP) and Back Propagation (BP) neural network theory, an evaluation model of the intelligent development of 151 cities in China is evaluated. Finally, we compared the model precision and time cost. This will help to find the bottleneck in smart cities’ construction and provide an effective basis for scientific measures of the development of urban wisdom.

Literature Review

With the development of the smart city concept, research on the connotation and development of the smart city is increasing significantly. The two groundbreaking theories of Graham and Pomeroy, S. M. [9,10] in the 1990s laid the foundation for smart city theory. Previous studies have generally considered urban infrastructure construction as the most important factor. In addition to building, transportation, and other physical facilities, information technology has also dominated the fundamental functions of the city. The research explained how information communication technology influences the development of cities. The introduction of information technology to the evaluation of smart cities is an inevitable requirement of technological development. It is also an important turning point in urban evaluation.
Since then, Allwinkle [11] made a comparative study of Graham and Mitchell. The IBM [3], Forrester Research, Natural Resources Defense Council (NRDC), and the European Intelligence Council, from a different perspective, have given the definition of a smart city [5]. These definitions are highly consistent, that is, smart cities rely on social, public, information, and commercial infrastructure of the city to promote the construction of smart cities through the circulation of resources [12].
Evaluation of the city can support investors as an important guide for the cities to judge their strengths and to define their strategies for future development. Scholars take a different approach to intelligent city assessment. Giffinger R. [13] summarizes the evaluation system as the economy, people, governance, mobility, environment, and living. However, for different cities, there is a need to combine their own characteristics and the analysis of regional diversity [6]. Since then, Etzkowitz H. [14] has comprehensively considered the triple helix of University-Industry-Government. In this way, the intent of urban evaluation is not just to apply the index system, but to find the relationship among social groups. Deakin M. proposed the theory and method to evaluate the correlation of wisdom urban. He chose the triple helix theory to construct the dynamic space of a regional innovation system [15,16,17,18]. On that basis, he reflected on the governance of smart cities, and went on to explain smart cities in terms of the social networks, cultural attributes, and environmental capacities [19,20,21]. Lombardi P. also used the Analytic Network Process (ANP) method to evaluate European cities based on the theory of triple helix. Moreover, he integrated social relations into the evaluation system, and presented an advanced triple helix network model [22,23].
Countries around the world have put forward the related methods of urban construction evaluation. In the 1950s, the United States and Japan began to study a city informatization level evaluation system [24]. The model of Machlup, Borat, the information index, and Information Utilization Potential is a typical method to evaluate urban informatization [25]. Influenced by the development of information technology and Internet of things, IBM first proposed the concept of an Intelligent earth, and cisco put forward the concept of Global Intelligent. There are many international universities that are also involved in the intelligence development study of the index system, such as the University of California, Vienna University of Technology, and the University of Ljubljana, etc. [26]. The global information technology report, jointly released by the World Economic Forum (WEF) [27] and INSEAD [28], introduces the network readiness index (NRI) [29], which provides a basic methodology that has become the most authoritative benchmark preparation tool in information technology reports. In response to the impact of ICT technology, Japan has formulated the u-japan strategy [30] and compared the information of the world’s major countries based on the international assessment of ICT infrastructure construction. In September 2001, the leading company in the field of electronic communications, Kang Sajige, established an evaluation index system that basically covered all levels of smart cities.
In addition, there are many related researches on intelligent city evaluation models and system constructions. Richard Florida [31] and Komninos N. [32] evaluated smart cities from three perspectives. The first is the wisdom of the working population city level, innovation ability, and creativity; The second is associated with the collective wisdom of urban residents, the collective wisdom by differentiation, forming a more creative collaboration competition mechanism; The third is the review of the artificial intelligence embedded city: communication technology infrastructure, digital information space, and provide citizens with online services, etc.

2. Evaluation Index System of Smart City

2.1. Evaluation Index System Based on PSF Evaluation Model

2.1.1. PSF Evaluation Model

The national center for the interconnectivity intelligent city has developed a PSF evaluation model [33] based on the nature of the operating system of a smart city, and it accurately describes the entire idea of the smart city system and its architecture. The three letters of PSF represent People-Oriented, City-System, and Resource-Flow. This model of Resource-Flow (F) from bottom to top corresponds to the input support layer, City-System (S) corresponds to the application layer, and People-Oriented (P) corresponds to the core target layer. Further details are:
(1)
Input support layer, corresponding to resource flow. It covers the infrastructure, the service platform, the flow of resources, the flow of funds, and so on, to meet the flow of resources and exchange, thus providing important material for the urban wisdom and development process.
(2)
System application layer, corresponding to the urban system. Mainly includes environmental protection, urban planning management operations, seamless link wisdom industry development, social system, and intelligent application links such as input and output analysis. It is based on the emergence of large data, such as deep learning technology, providing personalized, customized services for the city, and urban economy to communication, management, service, security, and so on.
(3)
Core target layer, corresponding to people-oriented. “People-oriented” is the core goal of urban smart development. It contains citizens as the core service object, service, and value for the implementation of the people’s livelihood, with the people’s actual demand in the urban development as the basic goal of urban development and guides residents in seeking a superior work environment, life scenes, and community experience, providing a sustained and effective power.

2.1.2. Comprehensive Evaluation Index System

According to the PSF evaluation model, the three levels respectively correspond to the six level indexes of the comprehensive evaluation index system for urban intelligent development. Specific indexes are distributed as Figure 1.
The input support layer corresponds to the intelligent infrastructure elements. Infrastructure is the skeleton of the resource circulation exchange, with the aid of the grid system, Internet system, transport system, e-government system, medical system, and infrastructure that can greatly promote the urban traffic, meaning that resource and material flow has an efficient circulation and exchange. It is a city’s responsibility to implement the basic material safeguard of wisdom. The system application layer corresponds to sub-elements of wisdom, intelligence services management component factors, economic factors, and wisdom guarantee child elements. The application layer of the system not only needs to possess rich urban management experience and government administrative services, but also needs to have the corresponding economic development level to support and match the supporting social security, so as to guarantee the coordinated and fair development of urban wisdom. The core target layer corresponds to the elements of the intelligent crowd. The city of wisdom is ultimately required to serve the general population, and the original executor can be a part of it, so the people who have contributed to the city of wisdom and the benefits of the city should form the core of the target.
Wisdom city construction, management, and maintenance represents very complicated system engineering, so the evaluation of a city’s wisdom is extremely complex and involves the dimensions of the wisdom urban system characteristics. We need to achieve the goal of the city’s intelligence and development of the overall level of intelligence, and the country’s policy, inner law of cities, and the reality of urban acumen to lead the overall evaluation system. Based on the PSF evaluation model, the integrated evaluation system of urban intelligent development is divided into three levels by referring to the inclusion elements of each hierarchy. The first layer is the target layer, which represents the comprehensive evaluation index for the intelligent development of a city. The second layer is the criterion layer, also known as the primary index (B), respectively, for smart individual (B1), smart management (B2), smart services (B3), smart economy (B4), smart guarantee (B5), and smart infrastructure (B6). The comprehensive evaluation system of urban intelligent development is shown in Table 1. B1 explains the elements of wisdom in the core target layer and takes people as the basic unit of the wisdom city. B2, B3, B4, and B5 explain the four subsystems in the system application layer, and they can measure the level of intelligent application, economy, management, service, and guarantee of wisdom through the technical level and service level. B6 explains the basic elements of the urban basic material in the supporting layer.

3. Construction of Comprehensive Evaluation Model

3.1. Modeling Tools and Methods

3.1.1. Hybrid Neural Network Model

In the decision-making process, AHP fully embodies the high consistency with the human decision-making process, which is easy to use and widely applicable, but it is difficult to accurately describe the nonlinear relationship among variables as a linear model. The Artificial Neural Networks (ANNs) is a nonlinear model that has a strong tolerance for faults and the ability to generalize, a very good explanation for nonlinear models and parallel computational functions. In addition, it can realize self-learning, and with the enlargement of the training sample, the accuracy of the model will be higher and higher. But the single ANN cannot accurately describe the performance of each index and the weakness of the system structure. So, in order to give the model a good stability and accuracy, we use the AHP-ANNs hybrid method to build the Urban Intelligent Development Model.
In the hierarchical structure of the AHP, the first layer is the expected target of the research question, the middle layer is called the criterion layer (usually the primary index), and the lowest level is the scheme layer (usually the underlying index). There are n elements in this structure, and their weights are W i , i = 1 , 2 , 3 , , n and i = 1 n W i = 1 . We obtained a judgment matrix based on expert advice. Then, the consistency of hierarchical order ranking and judgment matrix is verified by Cholesky decomposition of the judgment matrix to obtain the maximum characteristic root λ m a x and feature vector W i .
To define the consistency index C I = λ max n n 1 , if C I = 0 , the judgment matrix has a satisfactory consistency. The greater the value, the lower the consistency of the judgment matrix. The average random consistency index C I is defined to represent the consistency test value of the judgment matrix of different order Numbers. With C R = C I R I as the consistency ratio, if C R < 0.10 , the judgment matrix is consistent. On the contrary, we think that the matrix is not consistent, and we need to adjust the judgment values a i j . Finally, we need to check the total hierarchy sorting by calculating the level of all the elements for the target layer weight coefficient of hierarchy total sorting. If the consistency check of the total sequencing ratio C R < 0.10 , it passes this test. Conversely, the judgment matrix needs to be recalibrated.
The typical topology of the BP network is shown in Figure 2. The diagram contains an input layer consisting of p neuron nodes, an implicit layer consisting of m neuron nodes, and an output layer consisting of q neuron nodes. V = [ v 1 , v 2 , , v m ] represents the weight of the connection input layer and the hidden layer neuron node, and W = [ w 1 , w 2 , , w q ] represents the weight of the neural node of the hidden layer and the output layer.
For the neuron nodes on the hidden layer:
n e t j = i = 1 p v i j x i θ j , j = 1 , 2 , , m
o j = f ( n e t j )
For the neuron nodes on the output layer:
n e t k = j = 1 m w j k o j θ k , k = 1 , 2 , , q
y k = f ( n e t k )
In this case, f ( ) is an activation function, usually a Sigmoid function.
f ( x ) = 1 1 + e x
The essence of the BP network learning process is to adjust the weights of the network through its target value t and the predicted value y , and its learning process includes two stages: forward propagation and error retransmission. More specifically, in the forward propagation phase, the training samples are first passed into the network through the input layer and then transmitted to the output layer via the hidden layer. In the phase of error reverse propagation, the network adjusts the network error to the network to achieve the goal or reach the epoch, according to the error between the target value t and the predicted value y .
Let’s say that D t r a i n is the training set { ( x k , y k ) | x p , y q } k = 1 N . The BP network uses the error function as the performance function to adjust the power value Δ W .
E = 1 2 t y 2 = 1 2 i = 1 n ( t i y i ) 2
Δ W = η E W
For the input weight value W and the output weight V , the updated formula is as follows:
W ( n ) = W ( n 1 ) + Δ W ( n )
V ( n ) = V ( n 1 ) + Δ V ( n )
Among them,
{ Δ w j k = η E w j k = η δ k y o j Δ v i j = η E v i j = η δ j o x i
{ δ k y = E y k f ( n e t k ) = ( t k y k ) y k ( 1 y k ) δ j o = E o j f ( n e t j ) = ( k w j k δ k y ) y j ( 1 y j )
The above process is called the standard BP network. In order to further improve the performance of standard BP, we introduced a momentum factor, α , so the weighting formula of the BP network was modified to:
W ( n ) = W ( n 1 ) + Δ W ( n ) + α Δ W ( n 1 )
V ( n ) = V ( n 1 ) + Δ V ( n ) + α Δ V ( n 1 )
The number of nodes of the hidden layer neurons can be determined by the empirical formula:
m = p + q + a
p represents the number of nodes of the input layer neuron, q represents the number of nodes of the output layer neuron, and a is the constant between [ 0 , 10 ] . In practice, the number of neuron nodes in the hidden layer needs to be determined by the Trial and Error method.

3.1.2. ELM Model

The extreme learning machine (ELM) model was first proposed by Huang et al. [34], from Nanyang Technological University in Singapore, and presented a strict theoretical proof in the literature [35]. Compared with the single-hidden layer feedforward neural network (SLFNs) based on gradient descent, ELM can randomly set the weight of the input and determine the size of the hidden layer weight with the least square method. ELM is an advanced machine learning technology, which is simple and easy to implement, and has great advantages in dealing with complex problems such as high-dimensional data sets and noise [36].
Set the training sample set ( x i , y i ) to represent a collection of N different samples. x i = [ x i 1 , x i 2 , , x i p ] T p , y i = [ y i 1 , y i 2 , , y i q ] T q . For the regression problem, q = 1 . For classification problems, q represents categories or tags.
The ELM network has N h hidden nodes, and its mathematical model can be expressed as:
y i = j = 1 N h β i f ( w i x j + b i )
f ( x ) is the activation function, and it represents the following mapping relationship: .
The above equation can be expressed as the matrix form below:
Y = H β = [ h 1 , h 2 , , h N h ] β = [ f ( w 1 x 1 + b 1 ) f ( w N h x 1 + b N h ) f ( w i x j + b i ) f ( w 1 x N + b 1 ) f ( w N h x N + b N h ) ] × [ β 1 β N ]
w i = [ w i 1 , w i 2 , , w i q ] T , i = 1 , 2 , , N h is the weight vector that connects the i hidden neurons and inputs to the neuron; β = [ β i 1 , β i 2 , , β i q ] T is the weight vector that connects to the i hidden neurons and output neurons; and H is the output matrix of the hidden layer.
Here, we take the Sigmoid as the activation function. The Sigmoid function is defined as:
f ( x ) = 1 1 + e x
We assume that the input weight W and the partial b of the hidden neuron are known, so the only parameter of the linear model Y = H β can be expressed in this formula. The weight β can be obtained by solving the least squares solution β ^ of the line
ar model.
β ^ = min β | w , b H β - Y
It can also be expressed as:
β ^ = H Y
Matrix H is the Moore-Penrose generalized inverse of matrix H . It can be calculated by orthogonal projection, such as single value decomposition (SVD).

3.2. Synthesis and Processing of Comprehensive Evaluation Model

3.2.1. Data Normalization Processing

In the comprehensive evaluation index system of urban intelligent development, the provenance, type, and unit of raw data are different and the data cannot be compared directly. In order to follow-up data processing convenience and accelerate the convergence of data to ensure the subsequent simulation, we must therefore ensure the continuity and regularity of the data before the evaluation data normalization processing. Our paper adopts the Min-Max normalization method to deal with the cost and effect data.

3.2.2. The Determination of Index Weight

The target layer A in the comprehensive evaluation index system is the integrated evaluation index system of urban intelligent development, and the criterion layer Bi includes: smart infrastructure (B1), intelligent management (B2), intelligent service (B3), smart economy (B4), intelligent crowd (B5), and wisdom guarantee (B6). The scheme layer corresponds to the secondary index, which is recorded as Xi (i = 1, 2, …, 16). To find the weight of the criteria and the levels of the protocol, our paper builds a seven judgments matrix in the questionnaire, which are: Project target layer matrix, Smart infrastructure matrix, Intelligent management matrix, Intelligent service matrix, Intelligent economic matrix, Intelligent crowd matrix, and Wisdom assurance matrix. We used MATLAB R2017a to rank the hierarchy order and the total order of the above constructed judgment matrix, and the consistency test was conducted. Then, we adjusted the judgment value of the judgment matrix that failed to pass the consistency test, and the weight coefficient W i and the consistency test of the final judgment matrix were as follows, as shown in Table 2.

3.2.3. Determination of the Sample Target Value

Through the multi-factor comprehensive evaluation method, the evaluation formula of the urban intelligent development comprehensive evaluation index is expressed as:
Y = i = 1 16 X i W i
where Y is the comprehensive evaluation index, X i is the standardized value of the single index data, and W i is the weight corresponding to the index. W i can be determined through AHP.

3.2.4. Comprehensive Evaluation Model

The comprehensive evaluation model of urban intelligent development was obtained through the BP and ELM model, as follows. The calculation process of the comprehensive evaluation model is shown in Figure 3.
Y = f ( ( f ( V x θ ) ) W θ )
Y = H β = [ h 1 , h 2 , , h N h ] β

4. Results

We looked up the China Statistical Yearbook in 2015 and the public information of China’s provincial statistics bureau [37]. The open data provided by the Tongheng smart city institute [38], Tsinghua university, is also one of the data sources of our paper. Since the time of the statistics is not synchronized, it is critical to ensure the reliability and integrity of the data, and given the sluggish nature of the intelligent cities to respond to national policies, we use the data from 2015 to analyze it. Here, we take 151 cities with different economic scales in China as the research objects, and collect the comprehensive evaluation index data of urban intelligent development. The sample set involves 151 records, with 16 inputs (i.e., 16 secondary indicators), which we classify as training sets and test sets. Among them, the training set contains 100 training samples, and the test set contains 51 test samples. In the process of model training, the training samples are required to be typical, uniform, and diverse, and we selected data from 20 cities as a training sample; the remaining 80 training samples are selected from the sample to be selected randomly from the sample. The 20 cities include those such as Beijing, Shanghai, and Hangzhou, which also include Ningbo and Wuxi, which exhibit relatively stable economic development. They also include Liupanshui, Yulin, and Anhua, and other less developed cities.
After normalization of the data, we calculated the weight coefficients of each index.
For AHP, the judgment matrix A can be obtained by the expert survey method.
For BP, the activation function f ( ) is set to the Sigmoid function; The allowable error value (goal) is set to 1.00 × 10−6; The maximum iteration number (epoch) is set to 200; the momentum factor α is set to 0.95; and the learning rate η is set to 0.7.
For ELM, the activation function f ( ) is also set to the Sigmoid function. In BP and ELM, the mean square error (MSE) of the test samples is used to measure the accuracy of the model.
For BP and ELM, the number of neurons in hidden layer nodes is obtained by the trial and error method. The BP model test error changes with the number of neuron nodes, as shown in Table 3.
In the BP model, the number of neuron nodes in the hidden layer is set to 14. In this parameter setting, the training error and the test error are shown in Figure 4. The ELM model test error changes with the number of neuron nodes, as shown in Table 4.
It can be seen that in the ELM model, the number of neuron nodes in the hidden layer is set to 20.
Some results of the comprehensive evaluation index of urban intelligent development using AHP, AHP-BP, and AHP-ELM are shown in Table 5, and the results of these three models correspond to Y 1 , Y 2 , and Y 3 , respectively.
In order to make a better assessment of the level of urban intelligence, we have five evaluation criteria for the results of the sample values, which are “Excellent”, “Good”, “Fair”, “Pass”, and “Fail”. According to the above classification criteria, the classification results of the comprehensive evaluation model of intelligent development in three different cities are shown in Table 5. Figure 5 shows the distribution of the evaluation results obtained by using three methods.
AHP, AHP-BP, and AHP-ELM city wisdom of the development comprehensive evaluation index of the classification result is generally consistent, conforming to the law of development of the smart city. The comparison of the comprehensive evaluation index of urban intelligent development based on these three models is shown in Figure 6.
In the cities of Hangzhou, Ningbo, Qingdao, and Nanjing, there is a high energy efficiency, geographical environment, economic development, intelligent industry, and urban wisdom in the upper part of the country. In the AHP model, Hangzhou, Ningbo, Shenzhen, Zhuhai, Foshan, Xiamen, Guangzhou, Qingdao, and Nanjing are rated as “good”, and the AHP-BP model and the AHP-ELM model are consistent with the AHP model.
Cities such as Luzhou, Qinzhou, and Haidong are restricted by geographical environment, low economic development level, imbalance of urban industrial structure, high brain drain rate, low urban innovation vitality, and corresponding smart city infrastructure construction level backward. These cities’ intellectual development degree is in the lower reaches of China. In the AHP, AHP-BP, and AHP-ELM model, all seven cities were rated as “Fail”, and the results of these three were consistent.
In fact, Beijing and Shanghai are the first of the nation’s smart cities. On the one hand, they have a huge economic scale and a high concentration of resources, but on the other hand, with good infrastructure to build, and better management and services, these cities’ intellectual development is at the forefront of China. In the results of the AHP and AHP-ELM model, Beijing, Shanghai, and Wuxi are classified as “excellent” grades, which are consistent with the reality. However, in the results of the AHP-BP model, Wuxi and Shanghai were divided into “good” grades, which did not tally with their actual development conditions.
From the above analysis, compared with the AHP-BP model, the AHP-ELM model is more accurate for the comprehensive evaluation of urban acumen. As can be seen from Table 3 and Table 4, the test error of AHP-BP is 2.00 × 10−4 and of AHP-ELM is 8.05 × 10−5. Obviously, the accuracy of the AHP-ELM model is better than the AHP-BP model. Besides, Table 6 shows that the AHP-ELM model takes a very short time.
Considering the factors of model precision and calculation costs, we believe that the performance of the AHP-ELM model is far superior to that of the AHP-BP model. So, this hybrid evaluation model has improved and focused on the information about urban informationization, without using the information technology infrastructure as a critical point, and thinking about the hardware and other dimensions that smart cities need to develop. In addition, the application of the PSF model also improves the subjectivity of the evaluation index. Moreover, to a certain extent, it eases the wisdom city facing the broad indicators on the evaluation of the problem. In the sample of China’s cities, we discovered that the AHP-ELM model could be effective in breaking the bottleneck in intelligent city evaluation.

5. Further Research

Although our research can provide guidance for the design of comprehensive evaluation indexes of urban intelligent development to some extent, we can construct a reference model for the evaluation of urban intelligent development, but for the shortcomings in our research, we can also consider the following perspectives in future work.
In the future, we can further enrich our evaluation index system, which may include intelligent humanity (education is also included in this category), technological innovation capability, and green development capacity. This will give a more complete explanation of the level of humanistic innovation and green development in the construction of smart cities. In addition, more types of evaluation models can be used for comprehensive comparison. The practicability of the model can also be verified by time series analysis. In the current smart city evaluation index system, some indexes are difficult to quantify, and the fuzzy theory can be considered to deal with such data. Our overall assessment of the development of urban intelligence is a static assessment, and given that the development of the city is made by the dynamic evolution of the economic, environmental, social, and technological subsystems, we can use the method of dynamic assessment to capture the potential dynamics of the development of the city.

6. Conclusions

From the perspective of the evolution logic of the urban system, a smart city is the only solution to the problems and contradictions that have become increasingly intensified in the process of urban development. Based on the PSF evaluation model, we constructed a comprehensive evaluation index system for urban intelligent development, and used the analytic hierarchy process and neural network as modeling tools to construct AHP-BP and AHP-ELM. We then tested the performance of the model in 151 cities in China. We have established a comprehensive evaluation index system for urban intelligent development. Based on the analysis of the overall assessment of the development of intelligent cities in China, we have built a model of the integrated assessment of urban intelligence based on the theory of the PSF model. In addition, we have constructed a comprehensive evaluation model of urban intelligent development. Using the AHP model synthetic evaluation index as the benchmark value, the integrated evaluation model of urban intelligent development is constructed by using the neural network model.
Experimental results show that the test error of the AHP-ELM model is less than that of AHP-BP in model precision. In addition, in terms of computational overhead, the AHP-ELM model is much less time-consuming than the AHP-BP model.

Acknowledgments

This study was supported by the National Social Science Foundation of China (Grant No. 17CGJ002).

Author Contributions

In this study, Hongbo Shi improved the research method and logic of the paper; Tianyi Zhang prepared research data for this study and analyzed the data; Xiaowei Lin conceived and designed the study framework and provided the method; Sang-Bing Tsai provided revised advice; The manuscript was produced through contributions of all authors; all authors have given approval to the final version of the manuscript.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

PSFPeople-Oriented, City-System and Resources-Flow
AHPAnalytic Hierarchy Process
BPBack Propagation
ELMExtreme Learning Machine
IoTsInternet of Things
ANNsArtificial Neural Networks
SVDSingle Value Decomposition

References

  1. Chen, Y.Y. Study on the reasons and policies towards the rapid expansion of large cities. J. City Plan. Rev. 2003, 24, 33–41. (In Chinese) [Google Scholar]
  2. Zhang, H.B. Analysis of logical path of smart city. J. Intell. Build. 2014, 12, 59–63. (In Chinese) [Google Scholar]
  3. Harrison, C.; Eckmane, B.; Hamilton, R. Foundations for smarter cities. IBM J. Res. Dev. 2010, 54, 1–16. [Google Scholar] [CrossRef]
  4. Wang, Z.Q.; He, Y. Research on key technologies of smart city construction. J. Intell. Process. Appl. 2016, 6, 69–73. (In Chinese) [Google Scholar]
  5. Komninos, N. Intelligent cities: Innovation, knowledge systems, and digital spaces. Int. J. Technol. Manag. 2004, 28, 646–649. [Google Scholar]
  6. Lazaroiu, G.C.; Roscia, M. Definitional Methodology for the Smart Cities Model. Energy 2012, 47, 326–332. [Google Scholar] [CrossRef]
  7. Feng, J.W. Global smart city is entering a phase of rapid development. J. Internet Things Technol. 2016, 12, 12–18. (In Chinese) [Google Scholar]
  8. The Sixth (2016) China Prospective Industry Research Report; Forward-Looking Industry Research Institute: Shenzhen, China, 2017; Available online: http://en.qianzhan.com/ (accessed on 1 March 2016).
  9. Graham, S. Telecommunications and the Future of Cities: Debunking the Myths. Cities 1997, 14, 21–29. [Google Scholar] [CrossRef]
  10. Pomeroy, S.M. City of bits: Space, place, and the infobahn. Geogr. Rev. 2000, 90, 636–645. [Google Scholar] [CrossRef]
  11. Allwinkle, S.; Cruickshank, P. Creating smart-er cities: An overview. J. Urban Technol. 2011, 18, 1–16. [Google Scholar] [CrossRef]
  12. Washburn, D.; Sindhu, U. Helping CIOs understand “Smart City” initiatives. J. Growth 2009, 23, 24–32. [Google Scholar]
  13. Giffinger, R.; Kramar, H.; Haindl, G. The Role of Rankings in Growing City Competition. In Proceedings of the XI EURA Conference, Milan, Italy, 9–11 October 2008. [Google Scholar] [CrossRef]
  14. Etzkowitz, H. The Triple Helix: University-Industry-Government Innovation in Action; Routledge: Oxon, UK, 2008. [Google Scholar] [CrossRef]
  15. Leydesdorff, L.; Deakin, M. The Triple Helix of Smart Cities: A Neo-Evolutionist Perspective. J. Urban Technol. 2011, 18, 53–63. [Google Scholar] [CrossRef]
  16. Deakin, M. A Review of City Portals: The Transformation of Service Provision under the Democratization of the Fourth Phase. In Politics, Democracy and E-Government: Participation and Service Delivery; Reddick, C., Ed.; IGI: Hershey, PA, USA, 2010. [Google Scholar]
  17. Deakin, M. SCRAN’s Development of a Trans-National Comparator for the Standardisation of E-Government Services. In Comparative E-Government: An Examination of E-Government across Countries; Reddick, C., Ed.; Springer: Berlin, Germany, 2010. [Google Scholar]
  18. Deakin, M. The Embedded Intelligence of Smart Cities. Int. J. Intell. Build. 2011, 3, 179–187. [Google Scholar] [CrossRef]
  19. Deakin, M. Smart Cities: The State-of-the-Art and Governance Challenge. Triple Helix 2014, 1, 1–16. [Google Scholar] [CrossRef]
  20. Deakin, M. (Ed.) Creating Smarter Cities; Routledge: Oxon, UK, 2012. [Google Scholar]
  21. Deakin, M. (Ed.) Smart Cities: Governing, Modelling and Analysing the Transition; Routledge: Oxon, UK, 2013. [Google Scholar]
  22. Lombardi, P.; Del Bo, C.; Caragliu, A.; Deakin, M.; Nijkamp, P. An Advanced Triple-Helix Network Model for Smart Cities Performance. In Green and Ecological Technologies for Urban Planning; Ercoskun, O., Ed.; IGI Publisher: Hershey, PA, USA, 2011. [Google Scholar] [CrossRef]
  23. Lombardi, P.; Giordano, S.; Farouh, H.; Yousef, W. Modelling the Smart City Performance. Innov. Eur. J. Soc. Sci. Res. 2012, 25, 137–150. [Google Scholar] [CrossRef]
  24. Li, R.D.; Yao, Y.; Shao, Z.F. Big data in smart city. J. Geomat. Inf. Sci. Wuhan Univ. 2015, 58, 1–12. (In Chinese) [Google Scholar]
  25. Zhou, J. Study on Evaluation System of Smart City; Huazhong University of Science and Technology: Wuhan, China, 2013. (In Chinese) [Google Scholar]
  26. Kirkman, G.S.; Osorio, C.A.; Sachs, J.D. The Networked Readiness Index: Measuring the Preparedness of Nations for the Networked World. In The Global Information Technology Report 2001–2002; Oxford University Press: Oxford, UK, 2002. [Google Scholar]
  27. World Economic Forum Annual Report 2015–2016. Available online: http://www3.weforum.org/docs/WEF_Annual_Report_2015-2016.pdf (accessed on 1 March 2016).
  28. Global Information Technology Report. Available online: https://www.insead.edu/ (accessed on 1 March 2016).
  29. Global Information Technology Report. 2015. Available online: http://reports.weforum.org/global-information-technology-report-2015/economies/ (accessed on 1 March 2016).
  30. Wang, W. 21st century ubiquitous network society—A brief discussion on Japan’s U-Japan and Korea’s U-Korea strategy. J. Inf. Netw. 2005, 11, 1–4. (In Chinese) [Google Scholar]
  31. Sawicki, D. The rise of the creative class: And how it’s transforming work, leisure, community and everyday life. J. Am. Plan. Assoc. 2003, 69, 90–91. [Google Scholar]
  32. Komninos, N. Intelligent cities: Towards interactive and global innovation environments. Int. J. Innov. Reg. Dev. 2009, 1, 62–71. [Google Scholar] [CrossRef]
  33. National Center for Intelligent Urban Research. The Fifth Annual (2015) Assessment Report on the Development of Smart Cities in China; National Center for Intelligent Urban Research: Beijing, China, 2016. [Google Scholar]
  34. Huang, G.B.; Zhu, Q.Y.; Siew, C.K. Extreme learning machine: A new learning scheme of feedforward neural networks. In Proceedings of the 2004 IEEE International Joint Conference on Neural Networks, Budapest, Hungary, 25–29 July 2004; Volume 12, pp. 85–90. [Google Scholar]
  35. Huang, G.B.; Zhu, Q.Y.; Siew, C.K. Extreme learning machine: Theory and applications. Neurocomputing 2006, 70, 489–501. (In Chinese) [Google Scholar] [CrossRef]
  36. Dinkel, H.; Michael, S.; Weatheritt, R.J. ELM—The database of eukaryotic linear motifs. Nucleic Acids Res. 2012, 40, D242–D257. [Google Scholar] [CrossRef] [PubMed]
  37. China Statistical Yearbook in 2015. Available online: http://www.stats.gov.cn/tjsj/ndsj/2015/indexch.htm (accessed on 1 March 2016).
  38. Report of Tongheng Smart City Institute. Available online: http://www.govmade.cn/index.php?m=content&c=index&a=lists&catid=111 (accessed on 1 March 2016).
Figure 1. PSF evaluation model and urban intelligent development comprehensive evaluation index category.
Figure 1. PSF evaluation model and urban intelligent development comprehensive evaluation index category.
Sustainability 10 00037 g001
Figure 2. BP neural network topology diagram.
Figure 2. BP neural network topology diagram.
Sustainability 10 00037 g002
Figure 3. Flow chart of urban intelligent development comprehensive evaluation model.
Figure 3. Flow chart of urban intelligent development comprehensive evaluation model.
Sustainability 10 00037 g003
Figure 4. The iteration error graph.
Figure 4. The iteration error graph.
Sustainability 10 00037 g004
Figure 5. Wisdom city distribution of evaluation results.
Figure 5. Wisdom city distribution of evaluation results.
Sustainability 10 00037 g005
Figure 6. The comparison of the comprehensive evaluation index of urban intelligent development based on various models.
Figure 6. The comparison of the comprehensive evaluation index of urban intelligent development based on various models.
Sustainability 10 00037 g006
Table 1. Comprehensive evaluation index system for urban intelligent development.
Table 1. Comprehensive evaluation index system for urban intelligent development.
Target Layer (A)Primary Index (B)Secondary Index (X)
Comprehensive evaluation index system for urban intelligent developmentB1 Smart individualX1 Information service industry practitioners
X2 People’s life network level
B2 Smart managementX3 Government online service level
X4 Public resource trading platform
X5 Social media engagement
B3 Smart serviceX6 Social welfare service level
X7 Open data service levels
B4 Smart economyX8 Urban innovation and entrepreneurship level
X9 Energy consumption level of economic output
X10 Level of Internet industry development
B5 Smart guaranteeX11 Development plan formulation
X12 Information publicity and training
X13 Performance appraisal
B6 Smart infrastructureX14 Basic network construction
X15 Building and sharing of basic information resources
X16 Application of urban Cloud Platform
Table 2. The comprehensive evaluation index system weight of urban intelligent development.
Table 2. The comprehensive evaluation index system weight of urban intelligent development.
Target LayerPrimary IndexWeight CoefficientConsistency Test ResultsSecondary IndicatorsWeight CoefficientConsistency Test Results
A1
Comprehensive evaluation index system for urban intelligent development
B10.1889 R C = 0.077 λ max = 6.48 X10.5 R C = 0.004 λ max = 2.0
X20.5
B20.2365X30.33 R C = 0.00 λ max = 3.0
X40.33
X50.33
B30.1156X60.5 R C = 0.00 λ max = 2.0
X70.5
B40.0771X80.25 R C = 0.00 λ max = 3.0
X90.25
X100.5
B50.1534X110.25 R C = 0.00 λ max = 3.0
X120.5
X130.25
B60.2284X140.2 R C = 0.00 λ max = 3.0
X150.2
X160.6
Table 3. BP model test error changes with the number of neurons in hidden layer nodes.
Table 3. BP model test error changes with the number of neurons in hidden layer nodes.
m89101112131415
Testing Error2.84 × 10−32.66 × 10−32.43 × 10−32.25 × 10−31.49 × 10−31.34 × 10−32.00 × 10−42.21 × 10−4
m1617181920212223
Testing Error2.40 × 10−42.31 × 10−43.73 × 10−33.74 × 10−35.00 × 10−36.44 × 10−33.21 × 10−36.35 × 10−3
Table 4. ELM model test error changes with the number of neuron nodes.
Table 4. ELM model test error changes with the number of neuron nodes.
m89101112131415
Testing Error3.46× 10−33.67 × 10−33.36 × 10−32.94 × 10−32.62 × 10−32.35 × 10−32.02 × 10−31.94 × 10−3
m1617181920212223
Testing Error1.86 × 10−31.89 × 10−38.22 × 10−49.05 × 10−48.05 × 10−59.12 × 10−49.51 × 10−49.30 × 10−4
Table 5. The simulation results and classification results of the comprehensive evaluation index of the intelligent development of part cities.
Table 5. The simulation results and classification results of the comprehensive evaluation index of the intelligent development of part cities.
Cities Y 1 Classification Y 2 Classification Y 3 Classification
Training samplesWuxi0.8054Excellent0.7977Good0.8155Excellent
Shanghai0.8113Excellent0.7952Good0.8013Excellent
Beijing0.8195Excellent0.8021Excellent0.8165Excellent
Hangzhou0.7992Good0.7824Good0.7788Good
Ningbo0.7978Good0.7806Good0.7950Good
Shenzhen0.7453Good0.7409Good0.7179Good
Zhuhai0.6698Good0.6696Good0.6722Good
Foshan0.6273Good0.6365Good0.6351Good
Huainan0.2863Pass0.2818Pass0.2948Pass
Test samplesYichun0.3078Pass0.3121Pass0.3063Pass
Hulun Buir0.3220Pass0.3225Pass0.3100Pass
Zunyi0.3431Pass0.3490Pass0.3354Pass
Lianyungang0.3556Pass0.3584Pass0.3471Pass
Xuzhou0.3009Pass0.3000Pass0.2985Pass
Baoji0.3078Pass0.3158Pass0.3084Pass
Anshan0.3192Pass0.3176Pass0.3153Pass
Shijiazhuang0.3036Pass0.2931Pass0.3148Pass
Luohe0.0533Fail0.0113Fail0.0439Fail
Table 6. AHP-BP and AHP-ELM computation time costs.
Table 6. AHP-BP and AHP-ELM computation time costs.
AHP-BPAHP-ELM
Computation Cost (s)14.11 s0.42 s

Share and Cite

MDPI and ACS Style

Shi, H.; Tsai, S.-B.; Lin, X.; Zhang, T. How to Evaluate Smart Cities’ Construction? A Comparison of Chinese Smart City Evaluation Methods Based on PSF. Sustainability 2018, 10, 37. https://doi.org/10.3390/su10010037

AMA Style

Shi H, Tsai S-B, Lin X, Zhang T. How to Evaluate Smart Cities’ Construction? A Comparison of Chinese Smart City Evaluation Methods Based on PSF. Sustainability. 2018; 10(1):37. https://doi.org/10.3390/su10010037

Chicago/Turabian Style

Shi, Hongbo, Sang-Bing Tsai, Xiaowei Lin, and Tianyi Zhang. 2018. "How to Evaluate Smart Cities’ Construction? A Comparison of Chinese Smart City Evaluation Methods Based on PSF" Sustainability 10, no. 1: 37. https://doi.org/10.3390/su10010037

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop