Construction of the Evaluation Model of Open Government Data Platform: From the Perspective of Citizens’ Sustainable Use

Abstract

Under the background of big data, citizens can freely access and use open data to create value through the open government data platform (OGDP). The sustainable use of OGDP can meet the needs of citizens. The value created by citizens can also improve quality of life, which is of great significance to the sustainable development of society. From the citizens’ perspective, we constructed an evaluation model of citizens’ sustainable use of OGDP, including 12 indicators in four dimensions: Data, platform, outcome, and citizen. We have built the complete evaluation system with the DANP (Decision-Making Trial and Evaluation Laboratory-Based Analytic Network Process) method. It explores the main influencing factors and mutual influence of citizens’ sustainable use of OGDP. Empirical research is done on four provincial OGDPs in China’s Shanghai, Zhejiang, Guizhou, and Fujian provinces. The TOPSIS (Technique for Order Preference by Similarity to an Ideal Solution) method was used to rank the OGDPs in four pilot areas in empirical research. The results demonstrate that the improvement priorities of dimensions have the following order: Citizen, outcome, data, and platform, in which data and platform are cause dimensions, and outcome and citizen are result dimensions. The satisfaction indicator has the highest weight among all evaluation indicators, followed by the quality and quantity of outcomes. The one with the lowest weight is non-discrimination. The empirical results show that the OGDP in Zhejiang ranks the highest overall, followed by the OGDPs of Shanghai, Fujian, and Guizhou provinces. In the outcome and citizen dimensions, Zhejiang provincial OGDP does the best. Fujian provincial OGDP does the best in the platform dimension. The citizens’ sustainable use of OGDP can be promoted by timely opening of data that citizens need urgently, perfecting the policy of privacy protection and user guide of OGDP, holding open data innovation competition, providing data visualization function, providing various download formats of data sets, and simplifying the download procedures for citizens.

1. Introduction

During the process of providing public services for citizens, the government has accumulated a large amount of data. On the premise of ensuring public safety and personal privacy, the data can be made open to citizens through the open government data platform (OGDP) for new value creation [1]. Citizens can freely access and use open data through the OGDP [2]. Citizens can learn the latest policy information through open data, which is helpful to improve the transparency of government work and the participation of citizens in public management [3]. By using and reusing open data, citizens can develop applications and services that benefit themselves, stimulate industrial innovation, and provide new economic opportunities, thus creating economic and social values [4]. During this process, the demand for government information can be met, and the value creation of open data and the life quality of citizens can be promoted by sustainable use of OGDP. Above all, it is one of the key paths for sustainable development to promote citizens’ sustainable use of OGDP.

By the end of April 2021, 174 provincial, sub-provincial, and prefecture-level governments in China had established OGDPs [5]. While providing open data for citizens to create value, OGDPs also have some problems, such as unbalanced development [6], uneven data quality, and low data utilization rate [7]. An evaluation can help us better find and solve the problems in the construction of OGDP. Citizens are the end users of OGDP. Only when citizens actively query, obtain, and use open data provided by OGDP can new values be generated. Thus, it is particularly important to build a sustainable use evaluation system of OGDP from the citizens’ perspective. At present, scholars mainly evaluate the OGDP from the perspective of the government. Lourenço explored whether the current organizational structure can support the transparency of the accountability system by analyzing some famous OGDPs [8]. Máchová et al. evaluated the quality of the national-level OGDP by proposing and verifying a benchmark framework [9]. Machado et al. developed an evaluation tool for the construction of OGDP from governments’ perspectives and evaluated the functions of OGDP and the free access means of information [10]. To sum up, there is little research on the overall evaluation from the citizens’ perspective. Only when citizens use and reuse open data of OGDP sustainably can new value be created. Therefore, it is important for the sustainable creation of value to clarify the influencing factors of citizens’ sustainable use of OGDP and suggest improvement views.

As an important medium for open government data, OGDP undertakes multiple tasks such as opening government data, collecting citizens’ needs, serving citizens, and communicating with citizens [1]. Thus, when evaluating the OGDP, it is necessary to adopt multiple indicators from multiple aspects to conduct evaluation research [6]. The DANP method is a method that combines the Decision-Making Trial and Evaluation Laboratory (DEMATEL) method with the Analytic Network Process (ANP) method to obtain stable limit supermatrix and element weights [11]. The DEMATEL method is suitable for analyzing the logical relationship and mutual influence relationship between the indicators in a specific system, but it cannot determine the specific impact weight of each indicator [11]. On the basis of considering the influence relationship between each indicator, the ANP method can determine the specific impact weight of each indicator in the system [12]. However, the questionnaire design of the ANP method is complicated, and the calculation process of pairwise comparison is time consuming and difficult to understand [12]. The DANP method retains the advantages of the two methods. It can determine the interdependence among the indicators and reduce the frequency of pairwise comparisons between elements when calculating the weights, thus reflecting the reality more objectively [12,13]. In recent years, the DANP method has been well applied in the research of online catering platform evaluation [14], vehicle procurement evaluation [15], electronic health record evaluation [16], and green building evaluation [17]. There are few studies on applying the DANP method to the OGDP evaluation. Meanwhile, the TOPSIS method has been widely used in empirical research, such as online catering platform evaluation [14] and vehicle procurement evaluation [15]. Based on the citizens’ perspective, this study applies the DANP method to construct the citizens’ sustainable use evaluation model of OGDP and applies the TOPSIS method to determine the comprehensive ranking of OGDPs in four pilot areas. In summary, the investigation aimed to:

• Based on citizens’ perspectives, clarify the influencing factors and determine the evaluation dimensions and indicators of citizens’ sustainable use of OGDP.
• Clarify the influence relations and weights of the evaluation dimensions and indicators using the DANP method, and establish the citizens’ sustainable use evaluation model of OGDP.
• Apply the evaluation model constructed in the study to evaluate four provincial OGDPs in Shanghai, Zhejiang, Guizhou, and Fujian provinces, and analyze the current situation of citizens’ sustainable use of OGDP in China.
• According to the research results, put forward management suggestions to promote citizens’ sustainable OGDP use.

The remainder of this paper is organized as follows. Section 2 reviews the research status of citizens’ use and the evaluation of OGDP. Section 3 elaborates the evaluation dimensions and indicators of citizens’ sustainable use of OGDP, from the perspective of citizens based on previous studies. Section 4 is the methodology, introducing the DANP method and TOPSIS method used in this study. The data analysis and results are presented in Section 5. The empirical study is elaborated in Section 6, and Section 7 presents the conclusions.

2. Literature Review

2.1. Use of OGDP

In recent years, the influencing factors and utilization outcomes of citizens’ use of OGDP have been fully studied. Zeleti et al. developed the six-value business model by open data of OGDP, enabling enterprises to realize innovative management and provide new employment opportunities to create economic value [18]. Gao et al. established the citizens’ initial adoption model of OGDP, which showed that perceived trust, performance expectation, and convenience conditions had a significant influence on the citizens’ initial adoption intention and behavior of OGDP [19]. Through building a citizens’ use willingness model of OGDP, Zhu et al. found that environmental stimulus and flow conditions positively influenced citizens’ willingness to use OGDP sustainably through citizens’ flow experience [20]. Rettore et al. established an intelligent transportation system by the vehicle data of OGDP, which improved the efficiency and safety of traffic [21]. Bulai et al. used open data of Romania to build community clusters and draw maps, which help enterprises and policymakers better estimate citizens’ income levels [22]. Analyzing the data related to the elderly of OGDP, Hou et al. argued that the matching degree between welfare services and the needs of the elderly should be strengthened for the maximum realization of social welfare services [23]. By constructing an integrated influencing factors framework for citizens’ sustainable use of open data, Chen et al. held that citizens, tasks, systems, environment, and initial acceptance behavior were the main factors promoting citizens’ sustainable OGDP use [24].

Overall, the existing research on citizens’ use of OGDP is mainly focused on the influencing factors and utilization of outcomes. However, there is a lack of research on the current situation of citizens’ sustainable use of OGDP. Only when citizens sustainably use OGDP to obtain and use open data will they create political, economic, and social values sustainably and promote the sustainable development of society. Therefore, we focus on the citizens’ sustainable use status quo of OGDP. The evaluation method and empirical method are used to study it.

2.2. Evaluation of OGDP

In recent years, the OGDP has been mainly evaluated from the overall level and specific angle level. From governments’ perspective, an overall evaluation system of OGDP was established in China Open Data Index, including 17 indicators in four dimensions: Readiness, platform, data, and utilization [6]. It is argued that the construction of OGDP can be strengthened from the aspects of policy supply, organizational guarantee, normal operation, function optimization, data utilization, and ecological cultivation [6]. Chatfield et al. evaluated the service capability of OGDP according to the strength of policy, open data provision, diversity of data formats, and entrepreneurial data services [25]. Máchová et al. selected the indicators of technology, language support, usability, accessibility, API, topic classification, keywords, and communication to evaluate the OGDP [9]. In the overall evaluation research of OGDP, it can be seen that the evaluation is mainly from the perspective of government.

Some specific perspectives have been used to evaluate the OGDP, like operation efficiency, satisfaction, participation, and data quality. Hai et al. evaluated the operation efficiency of OGDP from the point of view of government. Their study showed that the open awareness of government and rational allocation of resources should be strengthened to improve the operating efficiency of OGDP [26]. Osman et al. evaluated the satisfaction of OGDP from the perspective of citizens and found that the benefits and risks are the main influencing factors [27]. Zhu et al. evaluated the participation of OGDP from the perspective of citizens and found that citizens lacked a deeper understanding of open data [28]. Vetrò et al. constructed an evaluation framework of OGDP at the data level and applied it in Italy. They argued that the traceability and integrity of open data were poor [29]. Charalabidis et al. evaluated the data quality of OGDP from seven aspects: Data quantity, usefulness, integrity, accuracy, data theme, timeliness, and data from different countries [30].

In summary, the overall evaluation of OGDP from the perspective of the government has received a lot of research attention. However, research on the overall evaluation of OGDP from the perspective of citizens is lacking. Only when citizens use the OGDP sustainably can they promote the use and reuse of open data, the value creation of open data, and the sustainable development of society. Therefore, it is of great significance to construct the sustainable use evaluation model of OGDP from the perspective of citizens.

3. Evaluation Dimensions and Indicators

During the process of citizens using OGDP, it can be seen that the utilization effect of citizens on data is closely related to open data [31]. Data is the core of OGDP governance. Improving the quality of open data can save OGDP governance costs and promote the high-quality development of OGDP [31]. The services provided by OGDP will affect citizens’ feelings of using OGDP and whether citizens will use OGDP sustainably [30]. The platform is the foundation of OGDP governance. By optimizing the services provided by OGDP, faster and better development of OGDP can be promoted [30]. Outcomes refer to service applications generated by citizens using open data, which can provide richer services for citizens [6]. As an important part of OGDP governance, the strengthening of results utilization and transformation can promote OGDP to better serve citizens [6]. During this process, the technical literacy of citizens will affect the use effect of OGDP [32]. Citizens are the ultimate servants of OGDP governance. Understanding citizens’ satisfaction and real need for open data can help OGDP be more sustainable [32]. Therefore, this study adopts four dimensions (data, platform, outcome, and citizen) to construct the sustainable use evaluation model of OGDP from the perspective of citizens.

3.1. Data Dimension

Data quality directly affects the data utilization value of citizens [31,33]. Data quality is affected by the coverage, update frequency, and format diversification of open data [29,34]. That is the comprehensiveness, accuracy, timeliness, and flexibility in this study. Tan et al. have evaluated the data quality of OGDP from the aspects of comprehensiveness, timeliness, and diversity [35]. Máchová et al. evaluated the data quality from the aspects of update date, geographical coverage, and data format [9]. Thus, four indicators, comprehensiveness, accuracy, timeliness, and flexibility, are used to evaluate the data dimension.

3.2. Platform Dimension

When using the network platform, citizens first need to ensure security. It is necessary to ensure that the private information of citizens will not be leaked [9]. Non-discrimination refers to ensuring that citizens can normally use the functions and services of OGDP [30]. A good platform also needs a neat page design, which is convenient for citizens to operate [2]. In other words, it is the usability of OGDP. Interactivity is a very important feature of the current Internet environment. The platform can interact with citizens, thus creating a good atmosphere and shortening the distance between citizens and the platform [5]. Therefore, four indicators, security, non-discrimination, usability, and interaction, are used to evaluate the platform dimension [30].

3.3. Outcome Dimension

The type of outcome refers to the richness of outcomes, reflecting the available fields and ways of open data [6]. The quantity of outcomes is an indicator that can directly reflect the data utilization results [6]. The quality of outcomes refers to the actual application condition of outcomes [6]. The poor quality of outcomes will cause a waste of resources, and the truly useful outcomes can bring benefits [5]. Therefore, three indicators of outcome (type, quality, and quantity) are used to evaluate the outcome dimension.

3.4. Citizen Dimension

The ability of citizens to use open data will directly affect the effect of open government data [36]. The quality of outcomes and the degree of satisfaction can be reflected by the satisfaction of citizens with data or products. Satisfaction is often analyzed in combination with expectations so as to better explore the psychological state of citizens [27]. To sum up, this study uses utilization ability, expectation, and satisfaction to evaluate the citizen dimension. The final evaluation dimensions and indicators are shown in

Table 1. Citizens’ sustainable use evaluation dimensions and indicators of OGDP.

Dimension	Indicator	Label	References
Data (D1)	Comprehensiveness	C11	[29,30]
	Accuracy	C12
	Timeliness	C13
	Flexibility	C14
Platform (D2)	Security	C21	[9,30]
	Non-Discrimination	C22
	Usability	C23
	Interaction	C24
Outcome (D3)	Outcome Type	C31	[5]
	Outcome Quantity	C32
	Outcome Quality	C33
Citizen (D4)	Utilization Ability	C41	[32,36]
	Expectation	C42
	Satisfaction	C43

.

4. Methodology

4.1. DANP Method

At present, the DANP method has been well applied in the fields of online catering platform evaluation [14], vehicle purchase evaluation [15], electronic health record evaluation [16], and green building evaluation [17]. Therefore, this study uses the DANP method to calculate the weight of evaluation dimensions and indicators and construct the influential network relation map (INRM). Based on the above research results, this paper also puts forward management suggestions for the construction of OGDP in China.

The DANP method is a combination of DEMATEL and ANP. The comprehensive influence matrix of DEMATEL is directly used as the unweighted supermatrix of ANP, and the stable limit supermatrix is obtained to calculate the element weight [37]. The DANP method combines the advantages of the two methods and makes up for the deficiency of ANP. This reduces the number of pairwise comparisons between elements and can reflect reality more objectively when calculating the weight [38]. Thus, the DANP method can be used to solve real-world problems with feedback and interaction between criteria or dimensions and to determine the affecting weights [13]. In order to determine the influence weight of each evaluation dimension and indicator, the DANP method was used to process and calculate the expert questionnaire data. The specific steps are as follows.

Step 1: Build a direct impact matrix. Calculate the arithmetic average of the scoring table of direct influence degree between every two indicators scored by experts, and get the direct influence matrix A, as shown in Equation (1).

$$A={\left[a_{ij}\right]}_m$$

(1)

Step 2: Calculate the normalized matrix, as shown in Equations (2) and (3).

$$\mathrm{D}=\mathrm{s}\times \mathrm{A}$$

(2)

$$\mathrm{s}=\mathrm{m}\mathrm{i}\mathrm{n}\left\{\frac{1}{\mathrm{m}\mathrm{a}{\mathrm{x}}_{1\le \mathrm{i}\le \mathrm{n}}{\displaystyle \sum _{\mathrm{j}=1}^{\mathrm{n}}{\mathrm{a}}_{\mathrm{i}\mathrm{j}}}},\frac{1}{\mathrm{m}\mathrm{a}{\mathrm{x}}_{1\le \mathrm{i}\le \mathrm{n}}{\displaystyle \sum _{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{a}}_{\mathrm{i}\mathrm{j}}}}\right\}$$

(3)

Step 3: Calculate the comprehensive influence matrix of evaluation dimensions and indicators, respectively. Using the normalized matrix D through Equation (4), the comprehensive influence matrices of dimension and index are ${\mathrm{G}}_{\mathrm{Y}}$ and ${\mathrm{G}}_{\mathrm{y}}$.

$$\mathrm{G}=\mathrm{D}+{\mathrm{D}}^2+{\mathrm{D}}^3+\dots =\mathrm{D}{\left(\mathrm{E}-\mathrm{D}\right)}^{-1}$$

(4)

Step 4: Calculate the standardized matrix. Standardize the comprehensive influence matrices ${\mathrm{G}}_{\mathrm{Y}}$ and ${\mathrm{G}}_{\mathrm{y}}$ according to Equations (5)–(7) to obtain standardized matrices ${\mathrm{G}}_{\mathrm{Y}}^{\mathrm{c}}$and ${\mathrm{G}}_{\mathrm{y}}^{\mathrm{c}}$, in which ${\mathrm{G}}_{\mathrm{y}}^{{\mathrm{c}}_{\mathrm{ij}}}$ is the submatrix of order ${\mathrm{m}}_{\mathrm{i}}{ \times \mathrm{m}}_{\mathrm{j}}$ in ${\mathrm{G}}_{\mathrm{y}}^{\mathrm{c}}$.

$${\mathrm{G}}_{\mathrm{Y}}^{\mathrm{C}}={\left[{\mathrm{g}}_{\mathrm{Y}}^{{\mathrm{c}}_{\mathrm{i}\mathrm{j}}}\right]}_{\mathrm{m}\times \mathrm{m}}=\left[\begin{array}{ccccc}{\mathrm{g}}_{\mathrm{Y}}^{11}/{\mathrm{d}}_1& \dots & {\mathrm{g}}_{\mathrm{Y}}^{1\mathrm{j}}/{\mathrm{d}}_1& \dots & {\mathrm{g}}_{\mathrm{Y}}^{1\mathrm{m}}/{\mathrm{d}}_1\\ \dots & \dots & \dots & \dots & \dots \\ {\mathrm{g}}_{\mathrm{Y}}^{\mathrm{i}1}/{\mathrm{d}}_{\mathrm{i}}& \dots & {\mathrm{g}}_{\mathrm{Y}}^{\mathrm{i}\mathrm{j}}/{\mathrm{d}}_{\mathrm{i}}& \dots & {\mathrm{g}}_{\mathrm{Y}}^{\mathrm{i}\mathrm{m}}/{\mathrm{d}}_{\mathrm{i}}\\ \dots & \dots & \dots & \dots & \dots \\ {\mathrm{g}}_{\mathrm{Y}}^{\mathrm{m}1}/{\mathrm{d}}_{\mathrm{m}}& \dots & {\mathrm{g}}_{\mathrm{Y}}^{\mathrm{m}\mathrm{j}}/{\mathrm{d}}_{\mathrm{m}}& \dots & {\mathrm{g}}_{\mathrm{Y}}^{\mathrm{m}\mathrm{m}}/{\mathrm{d}}_{\mathrm{m}}\end{array}\right]$$

(5)

$${\mathrm{d}}_{\mathrm{i}}=\sum _{\mathrm{j}=1}^{\mathrm{m}}{\mathrm{g}}_{\mathrm{Y}}^{\mathrm{i}\mathrm{j}},\mathrm{i}=1,2,\dots ,\mathrm{m}$$

(6)

$${\mathrm{G}}_{\mathrm{y}}^{\mathrm{c}}=\left[\begin{array}{ccccc}{\mathrm{G}}_{\mathrm{y}}^{{\mathrm{c}}_{11}}& \dots & {\mathrm{G}}_{\mathrm{y}}^{{\mathrm{c}}_{1\mathrm{j}}}& \dots & {\mathrm{G}}_{\mathrm{y}}^{{\mathrm{c}}_{1\mathrm{m}}}\\ \dots & \dots & \dots & \dots & \dots \\ {\mathrm{G}}_{\mathrm{y}}^{{\mathrm{c}}_{\mathrm{i}1}}& \dots & {\mathrm{G}}_{\mathrm{y}}^{{\mathrm{c}}_{\mathrm{i}\mathrm{j}}}& \dots & {\mathrm{G}}_{\mathrm{y}}^{{\mathrm{c}}_{\mathrm{i}\mathrm{m}}}\\ \dots & \dots & \dots & \dots & \dots \\ {\mathrm{G}}_{\mathrm{y}}^{{\mathrm{c}}_{\mathrm{m}1}}& \dots & {\mathrm{G}}_{\mathrm{y}}^{{\mathrm{c}}_{\mathrm{m}\mathrm{j}}}& \dots & {\mathrm{G}}_{\mathrm{y}}^{{\mathrm{c}}_{\mathrm{m}\mathrm{m}}}\end{array}\right]$$

(7)

Step 5: Calculate the unweighted supermatrix ${\mathit{W}}_{\mathit{ij}}$. Transpose the standardized matrix ${\mathrm{G}}_{\mathrm{y}}^{\mathrm{c}}$ obtained in the previous step with Equation (8).

$$\mathrm{W}={\left({\mathrm{G}}_{\mathrm{y}}^{\mathrm{c}}\right)}^{\mathrm{T}}$$

(8)

Step 6: Calculate the weighted supermatrix ${\mathrm{W}}^{\mathrm{c}}$, as shown in Equation (9).

$${\mathrm{W}}^{\mathrm{c}}={\mathrm{G}}_{\mathrm{y}}^{\mathrm{c}}\mathrm{W}$$

(9)

Step 7: Calculate the limit supermatrix. The weighted supermatrix is multiplied until it converges so as to determine the weight of each dimension and indicator, as shown in Equation (10).

$${\mathrm{W}}^{\mathsf{\mu }}=\underset{\mathsf{\mu }\to \infty }{\mathrm{l}\mathrm{i}\mathrm{m}}{\left({\mathrm{W}}^{\mathrm{c}}\right)}^{\mathsf{\mu }}$$

(10)

4.2. TOPSIS Method

TOPSIS was first proposed in 1981 [39]. The main idea of the TOPSIS method is to first determine the positive and negative ideal values of each indicator. The positive ideal value refers to the assumed optimal value scheme, in which each attribute value reaches the optimal value of each candidate scheme as a positive ideal goal. The negative ideal solution is the worst value scheme of another hypothesis, as the negative ideal goal. Secondly, the distance between each scheme and the positive and negative ideal values is calculated by using the Euclidean distance. Finally, the approach degree of each scheme to the affirmative ideal goal is obtained. The best result is the closest to affirming the ideal goal and the farthest from denying the ideal goal. For several schemes, the distance between each evaluation goal and the positive ideal goal and the negative ideal goal is calculated, respectively, and the ranking is obtained according to the closeness of ideal solutions. The specific steps are as follows.

Step 1: Establish a decision matrix. Design the evaluation indicator scoring table, which includes qualitative indicator and quantitative indicator. There are targets of evaluation D₁, D₂, …, D_M. Each target has the evaluation indicators x₁, x₂, …, x_n. The feature matrix D is shown in Equation (11).

$$\mathrm{D}=\left[\begin{array}{ccccc}{\mathrm{x}}_{11}& \dots & {\mathrm{x}}_{1\mathrm{j}}& \dots & {\mathrm{x}}_{1\mathrm{n}}\\ \dots & \dots & \dots & \dots & \dots \\ {\mathrm{x}}_{\mathrm{i}1}& \dots & {\mathrm{x}}_{\mathrm{i}\mathrm{j}}& \dots & {\mathrm{x}}_{\mathrm{i}\mathrm{n}}\\ \dots & \dots & \dots & \dots & \dots \\ {\mathrm{x}}_{\mathrm{m}1}& \dots & {\mathrm{x}}_{\mathrm{m}\mathrm{j}}& \dots & {\mathrm{x}}_{\mathrm{m}\mathrm{n}}\end{array}\right]=\left[\begin{array}{c}{\mathrm{D}}_1\left({\mathrm{X}}_1\right)\\ \dots \\ {\mathrm{D}}_{\mathrm{i}}\left({\mathrm{X}}_{\mathrm{j}}\right)\\ \dots \\ {\mathrm{D}}_{\mathrm{m}}\left({\mathrm{X}}_{\mathrm{n}}\right)\end{array}\right]$$

(11)

Step 2: Calculate the normalized matrix. Establish the normalization matrix of the normalization vector, in which i = 1, 2, …, m, j = 1, 2, …, n, as shown in Equation (12).

$${\mathrm{r}}_{\mathrm{i}\mathrm{j}}=\frac{{\mathrm{x}}_{\mathrm{i}\mathrm{j}}}{\sqrt{{\displaystyle \sum _{\mathrm{i}=1}^{\mathrm{m}}{\mathrm{x}}_{\mathrm{i}\mathrm{j}}^2}}}$$

(12)

Step 3: Calculate the weighted decision matrix Z. Z is obtained by point multiplication of the weights of each indicator in the matrix and the elements in the normalized matrix.

Step 4: Calculate the positive ideal solution z⁺ and the negative ideal solution z⁻. They can be determined by Equations (13) and (14), in which i = 1, 2, …, m.

$${\mathrm{z}}^{+}=\mathrm{m}\mathrm{a}\mathrm{x}\left({\mathrm{Z}}_{\mathrm{i}1},{\mathrm{Z}}_{\mathrm{i}2},\dots ,{\mathrm{Z}}_{\mathrm{i}\mathrm{n}}\right)$$

(13)

$${\mathrm{z}}^{-}=\mathrm{m}\mathrm{i}\mathrm{n}\left({\mathrm{Z}}_{\mathrm{i}1},{\mathrm{Z}}_{\mathrm{i}2},\dots ,{\mathrm{Z}}_{\mathrm{i}\mathrm{n}}\right)$$

(14)

Step 5: Calculate the Euclidean distance between each evaluation factor and positive ideal solution D⁺ and negative ideal solution D⁻ by Equations (15) and (16).

$${\mathrm{D}}_{\mathrm{i}}^{+}=\sqrt{{\displaystyle \sum _{\mathrm{j}=1}^{\mathrm{n}}{\left({\mathrm{r}}_{\mathrm{i}\mathrm{j}}-{\mathrm{z}}_{\mathrm{j}}^{+}\right)}^2}}$$

(15)

$${\mathrm{D}}_{\mathrm{i}}^{-}=\sqrt{{\displaystyle \sum _{\mathrm{j}=1}^{\mathrm{n}}{\left({\mathrm{r}}_{\mathrm{i}\mathrm{j}}-{\mathrm{z}}_{\mathrm{j}}^{-}\right)}^2}}$$

(16)

Step 6: Calculate the ideal closeness, as shown in Equation (17). The evaluation targets are sorted according to the ideal paste progress value C_i. The smaller the C_i value, the closer the evaluation target is to the positive ideal solution.

$${\mathrm{C}}_{\mathrm{i}}=\frac{{\mathrm{D}}_{\mathrm{i}}^{-}}{{\mathrm{D}}_{\mathrm{i}}^{+}+{\mathrm{D}}_{\mathrm{i}}^{-}}$$

(17)

5. Data Analysis

5.1. Data Collection

In this study, the expert questionnaire was used to collect data. The questionnaire, presented in Supplementary Materials, includes three parts: Questionnaire description, expert attributes, and comparative items. When the questionnaire is distributed and explained to experts, the experts are asked to fill in the questionnaire on the spot. The completion time of each questionnaire is 1 to 1.5 h. Experts use a five-point system to score the mutual influence degree of every two indicators at the same level; 0–4 stands for “no impact”, “low impact”, “medium impact”, “high impact”, and “extremely high impact” in turn. The survey was completed in March 2021, and 14 questionnaires were collected. There were five experts from universities and research institutions, four experts from enterprises, and five experts from the government. There were nine experts who have senior professional titles and 12 experts who have worked for more than 10 years. The inconsistent rate of the questionnaire was 4.04%, less than 5%. This shows that the questionnaire can reflect the real situation, and the additional quantity will not change the overall result.

5.2. Calculation of Weight

We standardized the returned questionnaire data according to Equations (1)–(3) and obtained the normalized matrix shown in

Table A2. The normalized matrix.

	C11	C12	C13	C14	C21	C22	C23	C24	C31	C32	C33	C41	C42	C43
C11	0.000	0.069	0.055	0.071	0.050	0.055	0.052	0.057	0.064	0.071	0.057	0.062	0.067	0.071
C12	0.043	0.000	0.060	0.055	0.055	0.031	0.045	0.050	0.062	0.067	0.076	0.062	0.071	0.076
C13	0.052	0.071	0.000	0.060	0.062	0.062	0.069	0.062	0.062	0.074	0.067	0.074	0.067	0.081
C14	0.052	0.050	0.045	0.000	0.050	0.064	0.067	0.057	0.062	0.074	0.064	0.055	0.055	0.076
C21	0.062	0.074	0.064	0.064	0.000	0.057	0.052	0.060	0.057	0.064	0.067	0.069	0.071	0.060
C22	0.057	0.045	0.055	0.048	0.060	0.000	0.050	0.055	0.062	0.062	0.064	0.060	0.067	0.076
C23	0.052	0.057	0.045	0.055	0.052	0.062	0.000	0.071	0.081	0.076	0.074	0.062	0.079	0.083
C24	0.048	0.057	0.048	0.062	0.057	0.064	0.052	0.000	0.067	0.067	0.055	0.060	0.064	0.076
C31	0.067	0.045	0.040	0.050	0.050	0.045	0.050	0.043	0.000	0.074	0.076	0.071	0.069	0.083
C32	0.052	0.045	0.050	0.045	0.050	0.052	0.057	0.052	0.055	0.000	0.083	0.050	0.060	0.083
C33	0.050	0.069	0.067	0.060	0.043	0.050	0.060	0.045	0.057	0.076	0.000	0.050	0.069	0.076
C41	0.050	0.064	0.067	0.067	0.048	0.048	0.055	0.060	0.083	0.064	0.064	0.000	0.074	0.076
C42	0.060	0.060	0.067	0.052	0.055	0.050	0.050	0.052	0.064	0.052	0.060	0.050	0.000	0.081
C43	0.071	0.067	0.064	0.055	0.074	0.052	0.060	0.086	0.055	0.062	0.079	0.055	0.064	0.000

in Appendix A. Then Equation (4) was used to get the centrality and cause of each dimension indicator shown in

Table 2. Calculation results of evaluation dimensions and indicators.

Dimension	Indicator	Indicator Centrality	Indicator Cause	Dimension Centrality	Dimension Cause	Indicator Weight	Indicator Ranking	Dimension Weight	Dimension Ranking
Data (D1)	C11	7.292	0.365	2.122	0.086	0.244	13	0.233	3
	C12	7.317	−0.099			0.260	8
	C13	7.612	0.560			0.247	11
	C14	7.248	0.139			0.249	10
Platform (D2)	C21	7.320	0.515	2.095	0.124	0.247	12	0.226	4
	C22	6.969	0.319			0.241	14
	C23	7.502	0.579			0.251	9
	C24	7.321	0.103			0.261	7
Outcome (D3)	C31	7.609	−0.286	2.223	−0.124	0.319	5	0.268	2
	C32	7.728	−0.652			0.338	3
	C33	7.923	−0.535			0.343	2
Citizen (D4)	C41	7.615	0.200	2.303	−0.086	0.295	6	0.273	1
	C42	7.777	−0.544			0.330	4
	C43	8.742	−0.705			0.375	1

. Equations (5)–(9) were used to calculate the weighted supermatrix ${\mathrm{W}}^{\mathrm{c}}$, as shown in

Table A6. The weighted supermatrix W^c.

	C11	C12	C13	C14	C21	C22	C23	C24	C31	C32	C33	C41	C42	C43
C11	0.044	0.055	0.056	0.058	0.056	0.059	0.057	0.056	0.061	0.058	0.055	0.056	0.059	0.060
C12	0.062	0.049	0.063	0.061	0.062	0.059	0.061	0.061	0.060	0.060	0.062	0.062	0.062	0.062
C13	0.056	0.060	0.046	0.057	0.057	0.059	0.056	0.057	0.056	0.059	0.059	0.060	0.061	0.059
C14	0.060	0.059	0.058	0.047	0.058	0.058	0.059	0.060	0.059	0.058	0.058	0.061	0.058	0.058
C21	0.056	0.058	0.056	0.054	0.045	0.058	0.055	0.057	0.057	0.055	0.055	0.056	0.058	0.058
C22	0.056	0.052	0.055	0.056	0.056	0.043	0.056	0.057	0.054	0.055	0.055	0.055	0.055	0.053
C23	0.057	0.057	0.058	0.059	0.057	0.057	0.045	0.056	0.057	0.053	0.059	0.058	0.057	0.056
C24	0.060	0.060	0.059	0.059	0.060	0.060	0.061	0.047	0.058	0.059	0.058	0.061	0.060	0.063
C31	0.088	0.086	0.086	0.086	0.086	0.087	0.088	0.088	0.068	0.086	0.087	0.091	0.089	0.085
C32	0.094	0.091	0.093	0.094	0.092	0.091	0.091	0.093	0.094	0.073	0.096	0.090	0.090	0.091
C33	0.091	0.095	0.093	0.092	0.093	0.093	0.092	0.090	0.096	0.099	0.075	0.091	0.093	0.096
C41	0.082	0.082	0.084	0.082	0.084	0.082	0.081	0.082	0.084	0.081	0.081	0.064	0.080	0.082
C42	0.092	0.092	0.090	0.090	0.093	0.092	0.093	0.091	0.091	0.092	0.094	0.092	0.072	0.092
C43	0.103	0.103	0.103	0.106	0.100	0.104	0.104	0.104	0.105	0.108	0.106	0.102	0.106	0.084

in Appendix A. Equation (10) was used to obtain the limit supermatrix ${\mathrm{W}}^{\mathsf{\mu }}$, as shown in

Table A7. The limit supermatrix W^μ.

	C11	C12	C13	C14	C21	C22	C23	C24	C31	C32	C33	C41	C42	C43
C11	0.057	0.057	0.057	0.057	0.057	0.057	0.057	0.057	0.057	0.057	0.057	0.057	0.057	0.057
C12	0.061	0.061	0.061	0.061	0.061	0.061	0.061	0.061	0.061	0.061	0.061	0.061	0.061	0.061
C13	0.058	0.058	0.058	0.058	0.058	0.058	0.058	0.058	0.058	0.058	0.058	0.058	0.058	0.058
C14	0.058	0.058	0.058	0.058	0.058	0.058	0.058	0.058	0.058	0.058	0.058	0.058	0.058	0.058
C21	0.056	0.056	0.056	0.056	0.056	0.056	0.056	0.056	0.056	0.056	0.056	0.056	0.056	0.056
C22	0.054	0.054	0.054	0.054	0.054	0.054	0.054	0.054	0.054	0.054	0.054	0.054	0.054	0.054
C23	0.057	0.057	0.057	0.057	0.057	0.057	0.057	0.057	0.057	0.057	0.057	0.057	0.057	0.057
C24	0.059	0.059	0.059	0.059	0.059	0.059	0.059	0.059	0.059	0.059	0.059	0.059	0.059	0.059
C31	0.086	0.086	0.086	0.086	0.086	0.086	0.086	0.086	0.086	0.086	0.086	0.086	0.086	0.086
C32	0.091	0.091	0.091	0.091	0.091	0.091	0.091	0.091	0.091	0.091	0.091	0.091	0.091	0.091
C33	0.092	0.092	0.092	0.092	0.092	0.092	0.092	0.092	0.092	0.092	0.092	0.092	0.092	0.092
C41	0.080	0.080	0.080	0.080	0.080	0.080	0.080	0.080	0.080	0.080	0.080	0.080	0.080	0.080
C42	0.090	0.090	0.090	0.090	0.090	0.090	0.090	0.090	0.090	0.090	0.090	0.090	0.090	0.090
C43	0.102	0.102	0.102	0.102	0.102	0.102	0.102	0.102	0.102	0.102	0.102	0.102	0.102	0.102

in Appendix A. The results are shown in Table 2. In terms of dimension, the citizen dimension has the highest centrality, which is the key core factor. The outcome dimension and citizen dimension have negative cause values, which are the result factors. The cause value of the data dimension and platform dimension is positive, which is the cause factor. The platform dimension has the largest cause value, which can affect the other three evaluation dimensions.

In the citizens’ sustainable use evaluation model of OGDP, the weights of the four dimensions are relatively close. The citizen dimension has the largest weight, which is consistent with the highest centrality value of the citizen dimension, followed by the outcome dimension and data dimension. The platform dimension has the smallest weight value, which means it is relatively the least important, which is consistent with the lowest centrality value of platform dimension in Table 2. From the evaluation indicator, satisfaction has the highest weight among all indicators, followed by the quality and quantity of outcomes. Non-discrimination is the indicator with the lowest weight.

The satisfaction indicator has the highest weight, which is the most important indicator in the citizen dimension and the highest weight indicator among all evaluation indicators, followed by expectation and utilization ability. In the outcome dimension, the outcome quality was the most important indicator, followed by the outcome quantity, and the weight difference between them is small. Outcome type was the least important indicator under the outcome dimension. In the data dimension, accuracy was the most important indicator, followed by flexibility and timeliness. The comprehensive indicator has the lowest weight, which is relatively the least important indicator in the data dimension. In the platform dimension, interactivity is the most important indicator, followed by usability and security. Non-discrimination is the indicator with the smallest weight among all evaluation indicators. This shows that non-discrimination will not be considered emphatically when evaluating the current situation of citizens’ sustainable use of OGDP.

5.3. INRM Construction

With centrality as abscissa and cause as ordinate, the INRM of dimensions and indicators is drawn, as shown in Figure 1. In the data dimension, timeliness, as the highest cause factor, affects comprehensiveness, accuracy, and flexibility. Accuracy is the one with the lowest degree of cause and the most affected degree. In the platform dimension, interactivity is the resulting factor, which is influenced by usability, security, and non-discrimination. In the outcome dimension, the type, quantity, and quality of outcomes are all the result factors. The outcome type affects the outcome quantity and outcome quality. In the citizen dimension, the utilization ability is the cause factor, which affects the expectation and satisfaction of citizens.

6. Empirical Research

6.1. Object Selection and Indicator Quantification

One of the aims of this study was to analyze the actual situation of citizens’ sustainable use of OGDP by using the evaluation tool from the perspective of citizens so as to help managers find out the weaknesses in the current construction of OGDP and provide management suggestions. So, this study considered the data set from China and used the sustainable use evaluation model of OGDP for empirical research. In 2018, the Chinese government took Shanghai, Zhejiang, Guizhou, and Fujian provinces as pilot areas for open government data [40]. The OGDPs in these four pilot areas are all in the top ten in China Open Data Index in 2020 [6]. Therefore, the empirical data of this study comes from four OGDPs: Shanghai (https://data.sh.gov.cn/), Zhejiang (http://data.zjzwfw.gov.cn/), Guizhou (http://data.guizhou.gov.cn/), and Fujian (https://data.fujian.gov.cn/). Accessed on 15 December 2021.

To quantitatively analyze the evaluation indicators of OGDP, the meaning of the evaluation indicators in Table 1 is further elaborated. The scoring standard is set for the indicators described qualitatively, as shown in

Table 3. Indicator meaning and scoring standard.

Dimension	Indicator	Descriptions	Criterion
Data	Comprehensiveness	The number of industries or fields covered by data sets.	Actual value
	Accuracy	The proportion of data sets with complete metadata.
	Timeliness	The proportion of timely updated data sets.
	Flexibility	The proportion of data sets in all five download formats.
Platform	Security	Whether the personal privacy policy is completed.	If yes, score 1 point, otherwise score 0 points.
	Non-Discrimination	Whether the restriction for logging in to download data exist.	If yes, score −1 point, otherwise score 0 points.
	Usability	Whether the manual or guide for citizens exists.	If yes, score 1 point, otherwise score 0 points.
	Interaction	Whether the scoring function of data exists.	If yes, score 1 point, otherwise score 0 points.
Outcome	Outcome Type	The number of industries or fields covered by outcomes displayed.	Actual value
	Outcome Quantity	The number of outcomes displayed.
	Outcome Quality	The number of outcomes displayed with a score of 5.
Citizen	Utilization Ability	The proportion of data sets with non-zero downloads.	Actual value
	Expectation	The proportion of data sets applying for open independently.
	Satisfaction	The average score of citizens’ scores.

.

6.2. Data Analysis

In this study, the information of data sets, information of outcomes displayed on OGDP, and scoring information of data sets were collected by data crawling and manual observation. Each evaluation indicator was scored according to the scoring standard. The data was collected on 18 September 2021. The normalization method was used to de-dimension the original dimensional data. The TOPSIS method is used to rank and calculate four OGDPs in Shanghai, Zhejiang, Guizhou, and Fujian provinces. Based on the weights of various dimensions and indicators obtained by the DANP method, we used the TOPSIS method to calculate the collected objective data to obtain the final ranking of OGDPs of the four pilot regions. The positive and negative ideal solutions calculated according to Equations (11)–(16) are shown in

Table A10. The positive ideal solution z⁺ and negative ideal solution z⁻ of four OGDPs.

	C11	C12	C13	C14	C21	C22	C23	C24	C31	C32	C33	C41	C42	C43
z+	0.071	0.080	0.083	0.129	0.082	0.121	0.084	0.065	0.149	0.140	0.154	0.111	0.168	0.115
z–	0.040	0.039	0.029	0.000	0.000	0.000	0.000	0.065	0.034	0.023	0.000	0.019	0.028	0.075

in Appendix A. We used Equation (17) to calculate the ideal proximity of four OGDPs, as shown in

Table 4. Ranking results of four OGDPs in China.

Pilot Areas	${\mathit{D}}_{\mathit{i}}^{+}$	${\mathit{D}}_{\mathit{i}}^{-}$	${\mathit{C}}_{\mathit{i}}$	Sorting Result
Shanghai	0.2153	0.2468	0.4659	2
Zhejiang	0.1497	0.3063	0.3283	1
Guizhou	0.2975	0.1926	0.6070	4
Fujian	0.2485	0.2209	0.5294	3

. It can be seen that the comprehensive ranking of the OGDP in Zhejiang is the highest, followed by the OGDPs of Shanghai and Fujian provinces. The last one is the OGDP in Guizhou.

The weights of each evaluation dimension and indicator are calculated as shown in Figure 2 and Figure 3. In terms of data, Zhejiang province has done the best, followed by Fujian, Guizhou, and Shanghai. In terms of platform, Fujian province has done the best, followed by Shanghai, Zhejiang, and Guizhou provinces. In terms of outcomes, Zhejiang province has done the best, followed by Shanghai, Guizhou, and Fujian provinces. In terms of citizens, Zhejiang province has done the best, followed by Shanghai, Guizhou, and Fujian provinces.

The weights of indicators of four OGDPs can be seen from Figure 3. Indicators without bars in Figure 3 have an indicator value of 0, such as Shanghai and Guizhou Province’s scores for flexibility indicators. This is because there are data with a score of either 0 or 1 in the indicator scoring standard setting and data collection. It can be seen that the OGDP in Shanghai has the highest score of outcome dimension and the lowest score of data dimension. The indicators of outcome quantity, non-discrimination, and outcome quality scored the highest. The score of flexibility and safety indicator is the lowest. The citizen dimension score of OGDP in Zhejiang is the highest, while the platform dimension score is the lowest. Expectation, outcome type, and outcome quantity indicators scored the highest. The non-discriminatory indicator has the lowest score. The OGDP in Guizhou has the highest outcome dimension score and the lowest platform dimension score. The outcome quality indicator scored the highest. Flexibility, usability, and non-discrimination had the lowest scores. The OGDP in Fujian had the highest platform dimension score and the lowest outcome dimension score. Non-discrimination and flexibility indicators scored the highest. And the outcome quality indicator had the lowest score.

7. Conclusions

7.1. Results

First, from the influence factors of OGDP, data, and platform are the cause factors, and outcome and citizen are the result factors. The platform dimension has the highest degree of cause, which affects the other three evaluation dimensions. The outcome dimension has the lowest degree of cause and is influenced by the other three dimensions. Judging from the weights of evaluation dimensions, the citizen dimension has the highest weight, followed by outcome, data, and platform dimensions. Specifically, among data dimensions, timeliness is the indicator with the highest degree of cause, which affects comprehensiveness, accuracy, and flexibility. Accuracy is the indicator with the lowest degree of cause and is affected by the other three indicators. This is consistent with the research results of Vetrò et al. [29]. It is all believed that the accuracy and timeliness of data need to be observed. Among platform dimension, usability, security, non-discrimination, and interactivity are all cause factors that influence each other. Among outcome dimension, outcome type, quantity, and quality are all result factors. Among citizen dimension, utilization ability is the cause factor. Expectation and satisfaction are the result factors. Utilization ability affects the expectation and satisfaction of citizens.

Second, judging from the evaluation weights, the citizen dimension has the highest weight, followed by the outcome dimension and data dimension, and platform dimension. Satisfaction is the highest weighted proportion of all indicators, followed by the quality and quantity of outcomes. The lowest indicator is non-discrimination. This is because the OGDPs were built by governments in China. During the process of open data, the OGDP has always upheld the principle of “serving the people” and is fair, judicial, and open. Thus, when evaluating the status quo of citizens’ sustainable use of OGDP, non-discrimination is not the focus. Specifically, accuracy is the most important indicator among data dimension. This is consistent with the research viewpoint of Jiang et al. [41]. The more accurate the data provided by OGDP, the better it is for citizens to use. Among platform dimension, interactivity is the most important indicator. This is consistent with the research results of Máchová et al. [9]. This is probably due to the fact that easy communication with citizens is much more important for OGDP. Among outcome dimension, outcome quality is the most important indicator. In the process of utilization and transformation of outcomes, the better the quality of outcomes, the higher the transformation efficiency. Among citizen dimension, satisfaction is the most important indicator. For all OGDPs, the satisfaction of citizens is the ultimate goal and the most important governance driver.

Third, according to the empirical research results, Zhejiang province has the best status of citizens’ sustainable use of OGDP, followed by Shanghai and Fujian provinces. The worst one is the OGDP in Guizhou. This shows that the Zhejiang government has done the best in the comprehensive management of OGDP. Shanghai and Fujian governments are next. The Guizhou government needs to strengthen the OGDP governance as soon as possible. In detail, the OGDP in Zhejiang has the lowest score in platform construction. The OGDP in Shanghai is poor in citizen, platform, and data fields, especially in the data field. The OGDP in Fujian does the worst in terms of citizens and outcomes, especially in the outcome dimension. The status quo of citizens’ sustainable use of OGDP in Guizhou does the worst. The citizen dimension, data dimension, and platform dimension need to be strengthened.

Finally, judging from the scores of each OGDP, the OGDP in Shanghai has done the best in terms of outcomes, especially in terms of the quantity and quality of outcomes. The data and platform dimensions are the worst, especially the flexibility of data and the security of platform. The citizen dimension of Zhejiang provincial OGDP is the best; it can well meet the expectations of citizens. Outcome and data are also well done, especially in the type and quantity of outcomes and the flexibility of data. Zhejiang province has done the worst in terms of platform, especially in terms of non-discrimination. The OGDP in Guizhou has done the best in the dimension of results, especially in the quality of results. Data and platform are the worst, especially the non-discrimination and usability of platform and the flexibility of data. The OGDP in Fujian has done the best in both platform and data, especially in non-discrimination of platform and flexibility of data. The outcome dimension is the worst, especially the quality of outcomes.

7.2. Suggestions

In conclusion, we consider that the government could issue questionnaires on OGDP to understand citizens’ needs and in a timely manner open data that citizens urgently need. The results show that the citizen dimension is the key core dimension. With the lowest cause value, the data dimension is the most important influencing factor of the citizen dimension. For citizens, satisfaction is the most important thing. Therefore, we can conduct a questionnaire survey on every citizen who browses the OGDPs. It can collect the current demand of citizens for government data and help the government open the government data that citizens urgently need in time. It could meet the needs of citizens and promote the utilization and value creation of government data, thus promoting the sustainable development of society.

Second, according to the results, the policy of privacy protection and guide for citizens should be perfected so that citizens can use the OGDP sustainably with ease and convenience. The results show that the platform dimension is the cause dimension, which affects the outcome dimension and the citizen dimension. Empirical results show that the OGDP in Shanghai does the worst in terms of platform security, while the OGDP in Guizhou does the worst in terms of platform usability. Overall, the security and usability of OGDP need to be strengthened. Therefore, the government should provide and update the policy of privacy protection on OGDP in time to ensure that citizens can use it with ease. The government can in a timely manner provide and update the user guide according to the functions of OGDP so that citizens can use it more conveniently.

Third, the open data innovation competition should be held to promote the research and development (R&D) of open data outcomes. The results show that the OGDPs in Guizhou and Fujian provinces have had poor outcomes, especially in the quality of outcomes of OGDP in Fujian province. Open data innovation competition is one of the key means to promote the utilization and outcome transformation of open data [42]. The government can hold open data innovation competitions on time to encourage citizens to actively participate in the utilization of open data and R&D of outcomes, strengthening the management and promotion of R&D outcomes, and promoting citizens’ application of outcomes, thus forming a virtuous circle between open data and citizens.

Fourth, we hold that the OGDP should provide a data visualization function to strengthen citizens’ sustainable use of open data. The results show that for OGDPs in Shanghai, Guizhou, and Fujian provinces, the citizen dimension score is very low, especially in Fujian. The utilization ability of citizens is not only related to their own knowledge reserves, such as educational quality and technical application ability, but also closely related to the data visualization function of OGDP. The perfect data visualization function can make citizens more clearly and intuitively understand the metadata content and the applicable industry fields of data sets. It can save time for citizens to filter and try out data sets, thus strengthening citizens’ sustainable use of open government data. By promoting citizens’ use and reuse of open data, government data can create more value and realize the sustainable development of society.

What’s more, various download formats of data sets should be provided. This can promote citizens’ multi-way and multi-field application of open data sets. The results show that the OGDPs in Shanghai and Guizhou provinces have the worst data flexibility. Expanding the download formats can make the utilization of data sets more flexible and convenient, which helps to improve the flexibility of open data. With the sustainable development of science and technology, the government can provide various download formats of data sets in time. This can promote citizens’ multi-way and multi-field application of data sets and then promote the value creation of open government data and the sustainable development of society.

Finally, we recommend that the government could unify the login account information of all OGDPs, and simplify the operation procedures for citizens to download data sets. The results show that the OGDPs in Zhejiang and Guizhou provinces are the worst in non-discrimination. The OGDP should adhere to the tenet of “serving the people” and simplify the procedures for citizens to freely access, obtain, and use open data. Thus, for China, without a national-level OGDP, it is possible to unify the login account information of all levels of OGDPs. This can simplify the operation procedures for citizens to download data sets, thus facilitating citizens to manage their own privacy information and obtain the data they need on each platform.

7.3. Limitation of Research

From the perspective of citizens, this study constructs a sustainable use evaluation system of OGDP with four dimensions and 12 indicators. Taking the OGDPs of four pilot areas in China as examples, the usability of this evaluation system is verified. However, there are still some limitations. First, the citizens in this study are more concerned about individuals than business organizations. Second, this study does not divide citizens according to age, gender, education level, and other attributes for comparative analysis. Therefore, from the perspective of enterprise organizations, the current situation of sustainable use of OGDP can be studied in future research. Second, we could consider subdividing citizens’ attributes and conduct a detailed research on the sustainable use status of OGDP in different categories in the future. Through further research, it is expected to promote citizens’ sustainable use of OGDP and the value creation of open data so as to realize sustainable development.