**2. Community Indicators in the Context of Transparency and Accountability Initiatives**

Community indicators function at the intersection of three issues of high standing on the global policy agenda and SDG framework [14]—namely, urban sustainability, transparency, and civic participation. As such, they relate to vast and overlapping academic disciplines, including policy studies, governance, communication, and management sciences [9]. To contextualize our comparative, longitudinal evaluation of Latin American initiatives, we reviewed the literature for key findings in three areas: activities of sustainability indicator projects, their organizational structure, and their achievements, including longevity; a transversal concern in each area is the influence of contextual factors.

Context differences are under-researched regarding community indicator initiatives yet are likely to play a major role in their success or failure. Informed by sectoral reviews, such as the one by Wray et al. [10], and the transparency and accountability literature, such as Grandvoinnet et al. [13], contextual factors with major relevance for community indicator initiatives include the political–legal regime, data availability and the presence of alternative, competing transparency and accountability actions at the country and city levels.

With regards to organizational set-up, a joint venture of governmental and nongovernmental organizations is often taken for granted [15] or even put forward as the "best practice [11]. A comparative review of the longevity of 82 indicator initiatives assessed the participation of governments and academic institutions and also concluded that "a broad and cross-cutting alliance of stakeholders is the most promising, because it can best ensure ongoing demand for the product, collective accountability, [...] a robustness of funding" [2] (p. 25). The same study implicitly describes barriers impairing the long-term functioning of indicator initiatives, namely funding and sustained demand for indicator information. An organization's legitimacy is also mentioned yet rarely studied comparatively; this is no surprise, as it is highly contextual and hard to operationalize [16].

Research on indicator projects has been dominated for years by studies exploring how they select which indicators. One main distinction concerns bottom-up, participatory, and context-specific versus top-down, expert-led, standardized approaches, with each having advantages and advocates [17,18]. While a set of indicators can also be merged into one index, most initiatives forgo this option, as aggregation obscures meaning and transparency [19]. Sustainability monitoring typically involves outcome indicators, often clustered in the environmental, social, and economic domains [20]. According to a recent global analysis of 67 measurement initiatives, a city's unemployment rate and green urban space are the most widely used indicators [21]. For these two examples, indicator initiatives in rich countries can obtain relevant information from official statistics or they may just need to convert raw data into more meaningful indicators, such as "green space per capita". For subjective wellbeing, however, another indicator that is widely used among community initiatives internationally [2,22], and pioneers in Latin America [23,24], all data needs to be collected through surveys, representing an expensive undertaking. In many indicator systems, political processes only receive marginal attention [21], though exceptions have been documented, such as a multi-year, voluntary benchmark among Dutch municipalities exclusively tracking sustainability policies [25]. Further, in addition to reporting outcome indicators, many initiatives deal with political processes, either through monitoring policy-related indicators—for example, government expenditures—or active community engagement, for example, by organizing round tables, public events, and so forth.

Figure 2 visualizes this array through sample activities found in this study and the literature [4]. The set is non-exhaustive, but it illustrates the spectrum of activities involving indicator reporting (concerning outcomes in the city and processes or inputs by the local government) and actions for community engagement. The latter may also be conceptualized as targeting generic processes (e.g., via community events) as well as the outcomes of deliberations, such as formalized spatial development plans. This spectrum mirrors that of sustainability reporting by local governments; a study by Niemann and Hoppe concerning the practices of European pioneers found significant divergence in terms of content along outcome and process dimensions and various degrees of citizen engagement strategies [26]. A number of North American community indicator initiatives also attempt to monitor

government performance at large [27]. On the other hand, process-focused activities may also be carried out by other organizations not working on sustainability. From this perspective, community indicator projects with their diverging scope of community engagement can be conceptualized as belonging to the broad field of transparency and accountability initiatives [13]. attempt to monitor government performance at large [27]. On the other hand, processfocused activities may also be carried out by other organizations not working on sustainability. From this perspective, community indicator projects with their diverging scope of community engagement can be conceptualized as belonging to the broad field of transparency and accountability initiatives [13].

Figure 2 visualizes this array through sample activities found in this study and the literature [4]. The set is non-exhaustive, but it illustrates the spectrum of activities involving indicator reporting (concerning outcomes in the city and processes or inputs by the local government) and actions for community engagement. The latter may also be conceptualized as targeting generic processes (e.g., via community events) as well as the outcomes of deliberations, such as formalized spatial development plans. This spectrum mirrors that of sustainability reporting by local governments; a study by Niemann and Hoppe concerning the practices of European pioneers found significant divergence in terms of content along outcome and process dimensions and various degrees of citizen engagement strategies [26]. A number of North American community indicator initiatives also

*Sustainability* **2021**, *13*, x FOR PEER REVIEW 5 of 17

**Figure 2.** Sample activities of community indicator projects or urban observatories. **Figure 2.** Sample activities of community indicator projects or urban observatories.

Assessments of effectiveness conceptually require taking objectives into account so one can measure progress against actual goals. Naturally, goals differ between indicator projects in different contexts and lifecycles [28]. Beyond the implicit notion of improving local governance and sustainability, however, initiatives often lack explicitly stated objectives [29]. To help assess outcomes in general terms, other studies about sustainability indicators [30] distinguished three clusters of use and influence: instrumental (i.e., bearing on decision and policymaking), conceptual (e.g., learning and capacity development among public servants), and political–symbolic (e.g., public discourse). These clusters overlap but help identify intended effects—such as instrumental influence on local gov-Assessments of effectiveness conceptually require taking objectives into account so one can measure progress against actual goals. Naturally, goals differ between indicator projects in different contexts and lifecycles [28]. Beyond the implicit notion of improving local governance and sustainability, however, initiatives often lack explicitly stated objectives [29]. To help assess outcomes in general terms, other studies about sustainability indicators [30] distinguished three clusters of use and influence: instrumental (i.e., bearing on decision and policymaking), conceptual (e.g., learning and capacity development among public servants), and political–symbolic (e.g., public discourse). These clusters overlap but help identify intended effects—such as instrumental influence on local government policies—as well as unintended uses and misuses.

ernment policies—as well as unintended uses and misuses. A common challenge for research on the effects of "infomediaries" [13] and multistakeholder actions is identifying causality. As all community indicator projects operate in networks (of different data providers, users, target audiences, etc.), their societal impact cannot be studied in isolation but requires a systems perspective. The method of process tracing allows for the gathering of high-quality evidence; a study applying it to three American community indicator initiatives found positive impacts regarding agenda-setting and other dimensions [29]. Process tracing and similar methods, however, require A common challenge for research on the effects of "infomediaries" [13] and multistakeholder actions is identifying causality. As all community indicator projects operate in networks (of different data providers, users, target audiences, etc.), their societal impact cannot be studied in isolation but requires a systems perspective. The method of process tracing allows for the gathering of high-quality evidence; a study applying it to three American community indicator initiatives found positive impacts regarding agenda-setting and other dimensions [29]. Process tracing and similar methods, however, require complex methodological rigour, which cannot easily be applied at a large scale. This explains why large-N comparative studies typically resort to more superficial measures of success, such as organizational survival or "staying power" [2]. Figure 3 summarizes the main factors thus identified in this literature review—various contextual factors (at country and city levels) as well as design choices (in particular, about organizational set-up and actions) that have a bearing on what community indicator projects achieve in terms of uses and influence. A further variable of interest is organizational evolution, which also serves as a reminder that none of the other constructs are static but actually co-evolve over time.

time.

complex methodological rigour, which cannot easily be applied at a large scale. This explains why large-N comparative studies typically resort to more superficial measures of success, such as organizational survival or "staying power" [2]. Figure 3 summarizes the main factors thus identified in this literature review—various contextual factors (at country and city levels) as well as design choices (in particular, about organizational set-up and actions) that have a bearing on what community indicator projects achieve in terms of uses and influence. A further variable of interest is organizational evolution, which also serves as a reminder that none of the other constructs are static but actually co-evolve over

**Figure 3.** Conceptual framework based on the literature review. **Figure 3.** Conceptual framework based on the literature review.

### **3. Population and Methods 3. Population and Methods**

The world's "battle for sustainability will be won or lost in cities", the United Nations proclaimed [31]. According to this metaphor, Latin America is an important battleground inasmuch as it is the world's most urbanized region, where about 80% of the population lives in cities. In 2011, the Latin American Network for Fair, Democratic, and Sustainable Cities (In Spanish: *Red Latinoamericana por Ciudades y Territorios Justos, Democráticas y Sustentables*) was created by community indicator organizations from across the region. Some, for example, *Bogotá Como Vamos,* had already been working for years with salaried staff. In other countries (e.g., Ecuador), all initiatives were run by volunteers, without funding or legal status, and with just a Facebook page to begin with. In conferences and discussions, a shared purpose was agreed—namely, promoting equitable development and democracy by reporting on sustainability indicators and the local government's management—as well as the stipulation that members are civil society organizations. In terms of joint activities, priority was given to communication (websites, newsletters, etc) and developing an indicator database for benchmarking. In the absence of their own financial resources, grants from philanthropic foundations (notably, AVINA) allowed the funding of a part-time communication officer in Lima, a secretary in Mexico, and consultants in Brazil developing a collective indicator database (For joint communication, the network used to run the website www.redciudades.net that following the end of project funding, has been offline since 2016). Network representatives also decided to support this longitudinal research by sharing the (contact) details of member initiatives and volunteering to be interviewed. Importantly, key informants concurred that the network included virtually all relevant, city-based Latin American initiatives and that no alternative institutional model (such as sustainability reporting by local governments) were prevalent in any country. Therefore, membership, as assessed in 2014, can be considered representative of the population of Latin American community indicator initiatives at the time. The network counted 65 initiatives on its own website, but verification of individual internet sites The world's "battle for sustainability will be won or lost in cities", the United Nations proclaimed [31]. According to this metaphor, Latin America is an important battleground inasmuch as it is the world's most urbanized region, where about 80% of the population lives in cities. In 2011, the Latin American Network for Fair, Democratic, and Sustainable Cities (In Spanish: *Red Latinoamericana por Ciudades y Territorios Justos, Democráticas y Sustentables*) was created by community indicator organizations from across the region. Some, for example, *Bogotá Como Vamos,* had already been working for years with salaried staff. In other countries (e.g., Ecuador), all initiatives were run by volunteers, without funding or legal status, and with just a Facebook page to begin with. In conferences and discussions, a shared purpose was agreed—namely, promoting equitable development and democracy by reporting on sustainability indicators and the local government's management—as well as the stipulation that members are civil society organizations. In terms of joint activities, priority was given to communication (websites, newsletters, etc.) and developing an indicator database for benchmarking. In the absence of their own financial resources, grants from philanthropic foundations (notably, AVINA) allowed the funding of a part-time communication officer in Lima, a secretary in Mexico, and consultants in Brazil developing a collective indicator database (For joint communication, the network used to run the website www.redciudades.net (accessed on 1 April 2021) that following the end of project funding, has been offline since 2016). Network representatives also decided to support this longitudinal research by sharing the (contact) details of member initiatives and volunteering to be interviewed. Importantly, key informants concurred that the network included virtually all relevant, city-based Latin American initiatives and that no alternative institutional model (such as sustainability reporting by local governments) were prevalent in any country. Therefore, membership, as assessed in 2014, can be considered representative of the population of Latin American community indicator initiatives at the time. The network counted 65 initiatives on its own website, but verification of individual internet sites showed 8 as actually inactive in 2014. As documented in the Supplementary File, these initiatives were discarded from this longitudinal study, just as 8 fledgling initiatives only started in 2013 (As also documented in the Supplementary File, at least 9 initiatives have been founded since 2014, including one in Guatemala, as an additional country). Therefore, the main sample consisted of 49 city-based community indicator initiatives started by 2012 and active in 2014.

For research on the practices, experiences, and long-term viability of these Latin American community indicator initiatives, the conceptual model derived from previous studies (cf. Figure 3) served as a starting point. For each of its main dimensions (context, set-up,

actions, use and influence, evolution), we identified the key parameters and data collection methods, as presented in Table 1. To assess an initiative's context, we relied on "hard data", such as city population numbers and the UN's Human Development Index at the country level. To explore the political–legal landscape, we further mapped country ratings from four international indices (by the World Bank, Transparency International, Reporters Without Borders, RTI). Such governance indicators are methodologically contested [32,33], unsuited to reliably capture all discontinuities, such as those arising from radically different presidencies in Brazil and Mexico, and not necessarily valid sub-nationally. However, country differences plausibly have repercussions at the city level and comparative indices may be meaningful to show overall trends and country clusters. For further validation and information at the city level, we interviewed and surveyed representatives of indicator initiatives in Spanish (and in Portuguese for Brazil).


**Table 1.** Operationalization and data sources.

The main survey contained open and closed questions about each initiative's set-up, objectives, activities, and experiences (including political interference and quality of public data). It was sent electronically in 2014 and responded to by 44 initiatives, a 90% response rate. Most answers were given by a person with high knowledge of internal processes, such as the initiative's director, and included frank statements about sensitive topics, such as relations with local governments. In response to requests for confidentiality, the survey and interview data were anonymized and summarized at the country level without identifying individual cities. Additional interviews were held with 11 representatives of indicator initiatives in 2014, 2019, and 2020.

To assess current activities, the websites of all initiatives were screened in 2021. This allowed for reliably mapping whether an initiative publishes indicators (on sustainability, quality-of-life, or government performance) but not their inherently diffuse and less visible community engagement activities; the majority of initiatives lack annual reports. Uses and influences were assessed qualitatively, with estimations (in 2014) of name recognition among various target groups serving as one quantifiable proxy indicator.

An initiative's level of continuity in 2021 was assessed through the visibility of activities on the internet. If an initiative's main website no longer existed, alternatives were explored via search engines and social media sites. If the initiative showed substantial activity in any channel (beyond a simple "like" or retweet) in the last 6 months, it was classified as "active". If no sign of life was found in more than 12 months, the initiative was classified as "discontinued", and cases in between as "unclear" (see Supplementary File for details). Average continuity or "survival rates" were subsequently compared to other factors, such as city size and country context. Comparisons based on country averages, however, are only explorative in nature due to the small number of cases involved.
