1. Introduction
Citizen assessments or surveys, typically in the form of quality or satisfaction ratings, are frequently used to measure the performance of public services in Western cities [
1,
2,
3]. A significant motivation for performing such assessments is to inform policy making [
4]. In practice, however, assessment results are seldom used to guide policy making because of much debate about their validity [
5,
6]. Citizens’ assessments are influenced by their general opinions of the community and their socio-demographic backgrounds [
4]. Citizens are said to hold insufficient information about services to accurately evaluate their quality [
7] and use general views on or overall satisfaction with the city or community to assess the quality of specific services [
8,
9]. Additionally, citizen assessments are self-reported expressed views, based on personal perceptions or judgments [
3]. Personal backgrounds, such as gender, age, education, income, homeownership, and marital status, also influence assessment results [
10,
11,
12]. Further, the effects have recently been associated with the complexity of the market and thus difficulties to penetrate for citizens with specific socio-economic characteristics in Behavioral Economics. For example, the elderly, the less educated, and those not employed reported lower satisfaction with particular utility services than their peers [
13]. Because of the subjective label, citizen assessments are often treated as “a substandard, second-order category” [
6].
Furthermore, studies comparing citizen assessments with objective measures (problematically) regarded as the gold standard [
6] have presented mixed results. For example, no correlation has been found between police and fire service performance and citizen satisfaction with these services [
1] and between citizen assessments of police services and crime rates, property recovery rates, or police personnel and budgetary data per capita [
7]. Satisfaction levels and service levels are completely independent of each other [
10]. However, both measures are sometimes related [
14,
15,
16] and more recent studies also confirm this point [
6]. For example, Im and Lee [
17] found a positive relationship between both, however citizen satisfaction varies with the specific service. Citizen assessments can generate meaningful results when questions are specific and related to personal experience [
7,
18]. The appropriate part that citizen assessments should play in performance measures has been highlighted in many studies [
6].
Urban green spaces (UGSs) may belong to such services. Because of close connections with their daily lives, citizens are typically familiar with UGSs and thus easily make (reasonable) assessments as opposed to other services, such as police services [
7] and emergency medical services [
4]. Moreover, in contrast to services (e.g., public transport) that are necessary for many city dwellers, patronizing urban green spaces is an optional activity. Although the various benefits of UGSs are widely recognized, such as social, eco-environmental, economic, and health benefits [
19,
20], how to manage and provide UGSs effectively to meet the various needs of citizens challenges city managers worldwide because of rapid urbanization and increasingly diversifying urban societies. This challenge concerns the quality of urban life and urban sustainability in the long term [
21] and has attracted increasing attention [
22,
23]. Therefore, UGSs were selected as a case for analysis.
Citizen assessments can transmit accurate information about the status of community parks [
4]. Observers can easily differentiate among urban scenes with some more attractive than others, and a high agreement among various groups lies in scenic evaluations of total urban landscapes [
24]. Accordingly, local (daily) experiences of landscapes or neighborhoods are claimed as a legitimate form of knowledge [
25] or a valid source of information [
26]. However, no relationships exist between citizen assessments of parks and objective indicators (e.g., distance to a park or park expenditures per capita) [
7].
Citizen perceptions of the quality of UGSs play a key role in attracting visits [
27,
28], which are increasingly associated with health [
29,
30]. Hayward and Weitzer [
31] argued that a park’s image constitutes a substantial and consistent predictor of people’s visits to the park. Citizens’ assessments of UGSs can mirror the degree to which their needs are satisfied and reveal their preferences for UGSs [
18]. An understanding of such assessments can benefit the effective management and provision of appropriate UGSs, which have special implications for compact cities with limited UGSs [
32], particularly those in China.
Under a top-down bureaucratic administrative regime, government officials have long dominated public policy making in China but have neglected public opinion [
33]. Officials in China generally distrust the public and believe experts from universities and institutes [
34]. The policy-making approach largely depends on professional assumptions, which are increasingly criticized as patronizing or anachronistic [
35,
36]. Citizen assessments have been gradually used to evaluate the performance of Chinese local governments [
37]. Because assessments are new to most Chinese officials, however, their results are still strongly questioned [
37].
This study aims to examine the validity of citizen assessments as a policy tool of urban public services by using assessments of UGSs in China as a case. Because of the strong questioning of objective measures with contextual subjectivity [
6] and the lack of objective measures or detailed official data, the present study focused on three key debates about citizen assessments or three critical aspects of the issue in the literature mentioned above rather than directly comparing them with objective performance measures as most previous studies have done.
(1) Can citizens differentiate variations in the quality of UGSs across dimensions and types? This question examines whether citizens can make reasonable judgments of the quality of UGSs.
(2) Do the (averaged) results of citizen assessments of individual UGSs (individual assessments) agree with the overall assessment of all UGSs in the city? A comparison between these two different (indirect versus direct) assessments facilitates the examination of whether the overall assessment is influenced by citizens’ general views on or overall satisfaction with the city or community. More important, the agreement (if any) between these assessments will evidence both the reliability and validity of citizen assessments.
(3) Do socio-demographic factors influence the overall assessment? This questions concerns if and to what extent the overall assessment mirrors the quality of UGSs (i.e., its validity).
As a pioneering city in the reform and opening-up and the seat of Guangdong Province in South China, Guangzhou has a free and open political climate and increased civic consciousness compared with other Chinese cities [
33,
38,
39] and was thus chosen as the study area. Against validity challenges to the use of survey data from inexperienced or potential users, most citizen surveys, which frequently use telephone interviews, usually employ filter questions to limit assessment responses to only those having direct personal experience with the service (experienced or actual users). However, this strategy often results in a small sample size or a low percentage of experienced users of a certain service [
18], engendering the representativeness problem. To overcome this problem, the current study recruited a direct survey method, namely, on-site face-to-face interviews with actual users of UGSs.
4. Discussion
For all the three key debates on citizen assessments presented in the introduction, the assessment results offer consistently positive answers. For the first question concerning whether citizens can differentiate variations in the quality of UGSs across dimensions and types, vegetation conditions and management and maintenance captured high assessments (
Table 3).
This is consistent with another local study [
49] focusing on resident perceptions of and attitudes toward UGSs. This result shows the benefits of the specific bioclimatic condition in the tropical city and the efforts of the city government in urban greening. Landscape aesthetics gained moderately positive assessments, echoing the favorable satisfaction with landscape design in a local research [
49]. This result can be associated with the long landscaping tradition in the city originating from the royal gardens in 100 BC [
43] (p. 24); [
50], further explaining the high rating of vegetation conditions. Environmental amenities and supporting facilities were rated low. For the former, this result is expected because of the serious environmental pollution of the city and the heavy use of UGSs [
35,
42]. The unsatisfactory assessment result about supporting facilities reflects the universal inclination toward quantity over quality in the management and provision of UGSs in China because the greening ratio and the area per capita of the public UGSs of a city, rather than the quality of its UGSs, concern the political decisions of the city government.
Regarding the types of UGSs, the assessment results (
Table 4) well reflect the variations in quality of UGSs in Guangzhou associated with the hierarchical nature of the provision. Under the two-level management approach, more resources are allocated to city-level parks and then to district-level ones [
43]. Furthermore, the assessment results may explain the unusual pattern of visits to UGSs in the city that urban parks are highly preferred and nearby small gardens are largely neglected [
35].
In summary, users can differentiate the variations in quality of UGSs across both its dimensions and its types and may accordingly make reasonable assessments of UGSs. The findings lend support to citizen assessments, echoing previous studies. For example, citizen assessments can furnish accurate information on the status of community parks [
4]. Observers can readily distinguish among urban scenes, and diverse groups make highly consistent assessments of the scenic quality of total urban landscapes [
24]. Furthermore, the findings may echo local daily experiences of landscapes or neighborhoods as a valid information source [
25,
26] or “ordinary (
versus professional) modes of knowledge” [
6] (p. 561), supporting citizen assessments.
Does the assessment result of individual UGSs agree with the overall assessment? The approximately positive individual assessments (
Table 4) well matched the somewhat satisfactory overall assessment (
Figure 2). The agreement between the results from the different (indirect
versus direct) assessment methods indicates both the good reliability and validity of citizen assessments, thus verifying them as effective performance measures [
6]. Furthermore, despite the insufficient official data for full comparisons, the assessment results confirm the fact of limited UGSs in the city (
Table 1). Particularly in terms of the two widely used official assessment indicators of UGSs in China, both the green ratio and the area per capita of public UGSs of the city met the corresponding national requirements. However, this finding challenges Stipak’s [
7] argument that citizens lack sufficient knowledge about public services to make accurate judgments.
For the third question, the weak socio-demographic effects (only explaining 1% of the variance) were found on the overall assessment (
Table 5). This finding indicates that the overall assessment was only slightly influenced by personal backgrounds, echoing other studies on UGSs. For example, demographic and socio-economic variables explain only 8% of the differences in citizen satisfaction with parks [
51]. No significant effects on citizen assessments of community parks have been found in terms of age, education, and income [
4]. Variations in citizen assessments seldom result from the characteristics of respondents [
12]. The present finding may be associated with a general perception of inadequate UGSs in the compact city [
42]. Another possible reason is the generally low satisfaction with governments and the services they offer [
4], however relevant questions were not included in the survey. However, the current finding differs from those of studies on other services, such as police, fire, and street services, indicating significant effects [
8,
11,
51]. As Licari
et al. have highlighted, citizen characteristics are only sometimes (
i.e., not consistently) associated with service assessments [
4]. This relation is associated with the specific services examined [
51]. Likewise, a latest study observed that the socio-economic effects on satisfaction are complex, depending on particular (utility) services [
13]. The detachment between personal backgrounds and the overall assessment implies that citizen assessments mainly depended on the actual status of the UGSs themselves, thus indicating the validity of citizen assessments.
The positive findings about citizen assessments call for city managers to seriously consider them when formulating policies. When objective measures are unavailable, citizen assessments are a desirable alternative because, aside from their validity as evidenced in this study, they are faster and easier to perform than objective ones. Even if citizen assessments are sometimes not correlated with objective assessments [
7,
10], they can convey valuable information on the extent to which citizen demands are satisfied [
18] and still act as a complement to overcome the hardness of objective measures [
6]. Citizen assessments play an appropriate role in performance measures of public services [
6]. For example, the client’s perception of the resolution time of the dispute, rather than the objective measurement (the actual recorded time), strongly influences their satisfaction with the performance of the state agency [
52]. Further, Schachter [
6] highlighted that contextual subjectivity also equally applies to objective or expert-generated measures.
The present findings have special implications for Chinese cities where objective performance measurement systems of public services are to be established, such as where few physical data of UGSs are available. Although citizen assessments have increased, they are new and dubious to most Chinese officials [
37]. The positive findings in this study help promote and even institutionalize citizen assessments [
51] in Chinese cities as a standard management practice.
However, the quality or performance of most public services is extremely complex to measure and assess [
53,
54]. Therefore, citizen assessments should not be seen as valid in every case [
6]. This also applies to services infrequently or indirectly used by citizens, such as police services and emergency medical assistance [
4,
7]. Conversely, UGSs,
inter alia, are usually associated with the daily lives of citizens and thus make valid assessments easy for them. Im and Lee [
17] found that citizen satisfaction varies with the type of service. Accordingly, the validity of citizen assessments may rely on specific assessed services themselves or more specifically on the degree of citizen familiarity with the services [
4,
18]. More attention needs to be paid to the particular types of services when applying citizen assessments.
5. Conclusions
Through assessments of UGSs in Guangzhou, China, this research examined the validity of citizen assessments as a policy tool of urban public services. Given the debate on objective measures and data limitations, the present study concentrated on three key debates on citizen assessments or three critical aspects of the issue rather than following previous studies to directly compare citizen assessments with objective measures. Users can recognize the variations in quality of UGSs across dimensions and types, the assessment results of individual UGSs matched the overall assessment of all UGSs in the city as a whole, and the overall assessment was only slightly influenced by socio-demographic factors. The findings consistently demonstrate that user assessments are superior indicators of the quality of UGSs.
The weak effects of personal backgrounds markedly differ from findings concerning other services. Aside from the effect of a general perception of insufficient UGSs in the dense city, UGSs are free public services and their use induces almost no expenses. This is remarkably different from those (utility) services required to choose and pay in the market, such as water, electricity, and gas [
13]. Namely, UGSs may be more accessible than these services. Moreover, in contrast with other (free) services particularly seldom used, e.g., police, fire, and street services, UGSs are associated with citizens’ daily lives and familiar to them, users in particular. Such familiarity may help produce reasonable assessments, independent of personal backgrounds.
The present findings offer empirical evidence to support citizen assessments as a policy tool, positively responding to the debate regarding such assessments. In practice, these findings help promote serious consideration and implementation of citizen assessments, particularly in cities, such as Chinese ones, that lack objective performance measurement systems of public services or where citizen assessments are questioned. However, attention needs to be given to the specific types of services when performing citizen assessments. Compared with other services (e.g., police services), UGSs are familiar to citizens and thus make reasonable assessments easier for them. This point deserves caution in interpreting and generalizing the current findings.
The current study was conducted in a large municipality in China with limited UGSs. To further examine citizen assessments, similar research can be made in other (medium-size or small) cities with adequate and quality UGSs in China and abroad. Given that recent studies find a relationship between both, future efforts can also compare citizen assessments with objective measures or official data if available. Methodologically, as a complex concept, citizen assessments or satisfaction should ideally be measured through a synthetic measure with composite indicators and analyzed by more sophisticated methods as presented by Ferrari and Manzi [
3].