Sustainability and Quality of Life in Marginalized Areas: An Impact Evaluation of a Community Center in Santa Fe, Mexico

Abstract

The aim of this paper is to present the results of an impact evaluation of a community center in health, capacity building, and digital access, which form an approximation of quality of life, in the population of Santa Fe town in Mexico City, from 2022 to 2024. The methodology is quantitative, using an impact index and the differences in differences (DD) technique. The data were obtained from primary sources with surveys undertaken via questionnaires. The center is operated by a private university and funded by private firms. The results show a positive impact of 0.287127 out of 1 on the weighted impact index, which allows us to consider this program successful in improving the quality of life of the target population. Through impact evaluation, the effectiveness of interventions and opportunities for improvement are identified, fostering collaboration among local actors, including community members, state-run public programs, and community centers. This collaborative effort improves the quality of life, creating a sustainable community wherein each actor addresses specific needs. Impact evaluation plays a crucial role in measuring sustainability because it is a continuous improvement process that, when combined with other actions, enhances the community’s overall well-being.

1. Introduction

Evaluating the impact of non-profit social programs has been a recurrent topic in research projects over the last few years, according to the World Bank and the Inter-American Development Bank [1]. The proposal of these international organizations arises from the need to verify the fulfillment of the objectives defined and the incidence of macroeconomic issues (such as poverty and inequality), with a particular focus on the quality of life of the individuals eligible for the program.

One of the main characteristics of this type of evaluation is the construction of alliances and collaborations between operating leaders of the programs and the research team as a requisite for success. This collaboration is important given the compromise involved in improving social programs to reduce the latent deficiencies in a certain population, which also requires that researchers specializing on economics and quantitative methods work in close association with program operators [2,3].

Several countries have applied the impact evaluation methodology to research projects, always with close collaboration with the social programs’ operators, and in a very successful way. This has allowed the scaling up of this methodology to other fields, both public and private, while the method itself receives feedback. Chile, México, Uruguay, El Salvador, and Brazil [4,5,6,7] are examples of countries from Latin America that have used this methodology. The European Union [8] and the United Kingdom [4,9,10] have encouraged improvement in the practice of this process from an institutional perspective and through several open calls.

In the public sector, this methodology has been applied in large-scale programs and in small-scale projects in the private sector. The United Nations Economic Commission for Latin America and the Caribbean, the Food and Agriculture Organization of the United Nations, the Inter-American Development Bank, and the World Bank are among the institutions that use and recommend this methodology.

Some examples of empirical cases in which an impact evaluation of interventions in different sectors with different objectives has been used are women’s self-help groups working to improve health, nutrition, sanitation, and food security in 3000 households in India [1]; the productive sector in five cases in Latin America [11]; and the “Un techo para mi país” [One Roof For my Country] program in Uruguay, México, and El Salvador [5].

As for universities, there are several that promote this method through graduate programs and use it particularly in research projects and superior education programs [2,12,13,14,15]. However, although they have community centers and social programs, there are no identified studies on universities performing an impact evaluation on their own interventions.

The Community Center MAPFRE-UP (CCMUP) is in the town of Santa Fe (Pueblo de Santa Fe), which has been specified as the area of influence. It is operated by Universidad Panamericana, a private university. The objective of this center is to have a positive impact on the quality of life of its users through different services offered, based on a market study carried out in 2016. Among these areas are assistance in education, human development workshops, hospitality, legal advice, medicine, nursing, health, dental health, nutrition, psychology and digital literacy.

Assistance in education, development, hospitality workshops, and consultancy are considered part of capacity building; medicine, nursing, health, psychology, and nutrition make up health; and Internet access, owning a computer or a tablet, and access to digital platforms make up the category of technological and digital access.

The relevance of this study comes from the fact that it contributes to the improvement of the quality of life, in economic and social terms, of the people in the area of influence of the CCMUP. It is pertinent because, according to a diagnosis developed under the World Bank’s proposal [1], the Center has the maturity for this type of evaluation, and it is time to measure the proposed objective.

In terms of research, this is an empirical and methodological contribution, since no other cases have been found in Latin America or other countries in which a private university applies the impact evaluation methodology to its own social programs. It gives the Universidad Panamericana the potential to feed back into the method and other public and private universities that operate social programs the opportunity to apply the method.

In terms of social impact, the study is within the scope of the 2030 Sustainable Development Goals [16] because it analyzes physical dimensions such as physical, mental and emotional health as well as the material dimension, with respect to socioeconomic conditions in a population with latent deficiencies. Specifically, the following objectives are addressed: 1. no poverty, 2. zero hunger and 3. health and well-being.

The framework within which impact evaluation (IE) resides is the theory of change (ToC), defined as a continuous process of structured action to explore change from an intervention with a social impact [17]. It allows the establishment of a route or map of the intervention to know where it should go and how it might to get there. In other words, it is the analysis of how we might achieve change in the context of vulnerability or lack of basic needs [18]. ToC is considered to be part of local developmental sustainability with a participatory approach due to the direct influence it exerts on the area of interest in which the intervention strategies are developed [19].

The intervention considered in this study, as required by the theory of change, is the opening up of face-to-face activities at the MAPFRE-UP Community Center once the COVID-19 pandemic had ended. The objective of the CCMUP is to have a social impact on quality of life by helping in education, human development workshops, hospitality, legal advice, medicine, nursing, health, dental health, nutrition, psychology, and digital literacy.

The study period was from 2022 to 2024, which, despite being a small period, is considered enough to estimate changes derived from interventions made during the CCMUP operations after the pandemic, due to the size of the community center. As opposed to impact evaluations of public social programs where the target population is large, the size of the influenced area and the number of operations provided by the CCMUP allows shorter periods (and a higher frequency) of evaluation.

Impact evaluation is a way of analyzing the long-term sustainability of the community. In the first place, for the community to be sustainable, the quality of life of the inhabitants must improve. For this, it is necessary that local actors, individuals, the State, community centers, non-governmental organizations, companies, and local businesses, among others, intervene. Individuals can participate in interventions in their area of action and at the scale that their material, human, and financial resources allow. Secondly, collaboration and cooperation between the different actors must be maintained through local networks to join forces and not compete for the supply of services but instead satisfy the demand of the population; therefore, the measurement of the impact of the different interventions is relevant. This study could be replicated by local authorities and for public programs with the intention of measuring the effectiveness of their services. In this way, each one could direct its resources towards the general and specific needs of the population. Likewise, this study can be replicated for other communities in which interventions are present to improve the population’s quality of life, for public and private programs, and for very large and small-scale populations. By paying attention to the needs of specific populations, a comprehensive improvement in the quality of life of inhabitants may be achieved, thus ensuring that the community is sustainable.

The quality of life of inhabitants depends on several factors and dimensions, several of which must contribute to achieve the sustainability of the community. Although the main actors responsible for improving quality of life are individuals themselves and the State through its public programs, there are other actors that contribute to achieving this purpose. The CCMUP offers services that are not covered by public programs or are insufficient to meet demand. Health, capacity building, and digital access are the areas in which the center intervenes. However, it is not enough to only carry out the intervention, and it is considered necessary to evaluate how much these services have impacted the population’s life. Therefore, the question that guides this research is as follows: what is the impact of the services offered by the CCMUP on the population of Santa Fe? (as an approximation of quality of life).

In the cities, it has been observed that there are communities that have depended for many years on industries located in certain areas, and when these closed, these communities’ quality of life was affected. This is the case of the town of Santa Fe, which depended on a gunpowder factory that, when it closed, affected the workers, who had to look for another source of employment, often more precarious and unstable.

In general, in Mexico, the main universities, public and private [20,21,22,23,24], include in their mission a commitment to society. In the case of the Universidad Panamericana, one of the institutional principles is social responsibility and commitment, which is evaluated through social impact [24]. Thus, the CCMUP contributes to the social commitment of the University through the services it offers, and the evaluation that has been proposed is the measurement of impact. This study could be replicated in other universities in Mexico and in other countries for the same purpose.

2. Marginalized Areas

According to the United Nations, around 60% of the world’s urban population (one third of the global population) inhabit metropolitan areas, and it is projected that by 2035, an additional 1 billion people will become metropolitan inhabitants [25]. In Mexico, the metropolitan population changed from 75.1 million in 2015 to 80.2 million in 2020 [26]. The growth of cities is acompassed by challenges such as the proliferation of disadvantaged urban areas in which marginalization and social exclusion are present.

In Latin America, the organization of cities during the last three decades have turned to a fragmented socio-spatial structure where the boundaries between classes are not clear [27]. However, the proximity between social classes does not necessarily imply improvement of living conditions because, as stated by Ruiz-Tagle [28], in socioeconomically mixed areas, the increase in employment opportunities might be only in the informal sector. Spatial proximity has also produced micro- or small-scale segregation [29].

The concept of marginalization has been approached from different perspectives and there is still no unique definition. In sociology-related studies, marginality has been used similarly to social exclusion [30]; others have considered it an important reason for lower quality of life [31]. Ruddle and Rodinelli’s conceptualization (1983) stated that marginality is a result of geographical remoteness affecting accessibility of services, low population density, an underdeveloped economy, ethnic issues, ecological fragility, low levels of physical and social infrastructure, and little or no political influence [32].

Leimgruber [33] later considered the relevance of the scales in the dimensions, and in addition to the “spatial” scale, he also considered the “time” scale equally important. Gatzweiler et al. [34] define it as “an involuntary position and condition of an individual or group at the margins of social, political, economic, ecological and biophysical systems, preventing them from access to resources, assets, services, restraining freedom of choice, preventing the development of capabilities, and eventually causing extreme poverty” [34]. They found a relationship between this concept of marginality and poverty using Amartya Sen’s capabilities approach [35], where marginality represents the constraints that restrict the recognition of capabilities that can be transformed into functioning [34].

The Mexican National Population Council-defined marginalization as

“a structural phenomenon that originates in the style or historical pattern of development that is expressed, on the one hand, in the difficulty of spreading technical progress throughout the productive structure and in the regions of the country, and on the other, in the exclusion of social groups from the development process and from enjoying its benefits”.

[26]

Considering all the presented definitions and concepts, this research considers the Town of Santa Fe as a marginalized area in Mexico City. Even though the socioeconomic conditions of the population are higher than the level considered extreme poverty and their basic living needs are satisfied, there are dimensions in which quality of life is not optimal, such as education, health, income level, and security. The spatial location of this community causes geographic challenges in accessibility to services and resources, and its proximity to high-income areas creates inequalities in the distribution of resources and public services.

3. Context and Sociodemographic Characteristics of the Population in Santa Fe Town

The town of Santa Fe has its origins in 1532 and has official recognition as an original town of Mexico City. It is part of the biocultural heritage corridor that includes the Hermitage of Vasco de Quiroga and the Aqueduct of Santa Fe, built in the seventeenth century; there is also an area of springs, including the Santa Fe Forest and the Tacubaya River, which are natural resources being recovered by local actors in the area [36].

Up to the 1960s, the town had few inhabitants and was on the margins of the city, until industrialization made it a recipient of internal and external migrants, and the population grew exponentially [37]. From then on, expansion of popular neighborhoods began, including construction of housing units that concentrate a large number of people, without urban or territorial planning. Subsequently, in the 1980s, the town was affected by the construction of a corporate area that absorbed more resources than were being generated in the area. With the lack of planning and disproportionate population growth, shortages and problems such as lack of water, pollution, insufficient health services and access to education, unemployment, limited recreational activities, and increased insecurity and illicit activities intensified.

Given the precarious conditions that were generated in the past few decades, local actors and institutions took on the task of recovering the town of Santa Fe and improving these conditions by creating spaces for social support for the community. Currently, the community centers that are in the area are Pilares, which is part of the federal government’s social programs; Meneses, which is managed by the Universidad Iberoamericama; and Centro Comunitario MAPFRE-UP, which is operated by the Universidad Panamericana. Other actors are the Metropolitan Autonomous University, which has the Territorial Observatory of the West, in which it develops projects and studies of the Western area of Mexico City; Santa Fe Citizen Committee; Civil Associations of neighbors, settlers and the Santa Fe Market; pastoral groups; and the Council of Towns and Neighborhoods of Mexico City. Each has specific objectives and different forms of operation. At the government level, in Mexico, there are federal and local government programs created to reduce poverty, reduce marginalization, and increase employment, among other things, all of which operate in Santa Fe [36,37].

3.1. Community Center MAPFRE-UP (CCMUP)

The CCMUP was born in 2016 as a support space for the inhabitants of Santa Fe in the areas of capacity building, health and, later, digital access. The objective of the CCMUP is to influence the quality of life of the inhabitants of the town of Santa Fe through the services it offers. In the area of capacity building, support courses are offered for students at the basic, middle- and high-school levels in the subjects of English, trades (sewing, nails), workshops (cooking and hospitality), and legal advice, among others. In the area of health, medical consultations, nursing, nutrition, dental service, psychological consultations and conferences, talks, and workshops on disease prevention are offered. Regarding digital access, a space is offered in which there is access to computers and internet and digital literacy courses for older adults. of the center reaches 1500 people served annually and currently operates at 50% of its service capacity. Geographically, it is located in the west of Mexico City and is surrounded by ravines. Services are offered to residents who live within 500 m of their physical location, as ravines prevent them from connecting beyond. The CCMUP is a small-scale community center that provides services to the local community, as do many community centers that exist in the area, in Mexico City, in the country, and throughout Latin America. The size of community centers should not be a limitation when measuring their impact on people, but since it is small-scale, it is necessary to adapt the tools to their size.

3.2. Sociodemographic Characteristics of the Inhabitants of the Pueblo de Santa Fe

The 2020 Census of Population and Housing concluded that more than 2.4 thousand people live in the area where the town of Santa Fe is located; 53% are women and 47% men. 78% of the population is 18 years old or older. The average time spent in school in this region is 11.16 years, where 8.2% of the population that is 15 years old or older completed only elementary school, and 16.7% completed only middle–high school. Some 19% of people have access to health services. In terms of dwellings, the average number of occupants in inhabited private homes is 3.58, 92% of the dwellings have a floor material other than dirt or sand, 94% have access to electricity, 93% have access to water supply, 94% have sanitation coverage, 94% own a refrigerator, 85% own washing machine, 94% own television, 58% own a computer or tablet, 89% own a cellular phone, and 74% have internet access [38].

The information obtained in the surveys conducted in 2022 and 2024 describes the population of the area surrounding the MAPFRE-UP Community Center in line with the characteristics provided by the Census. The survey respondents were 67% women in 2022 and 78% women in 2024, with an average age of 39 in 2022 and 42 in 2024. On a 1 to 10 scale of quality of life (10 being the highest quality), the average perception of quality of life was 7.8 in 2022 and 7.9 in 2024. Some 83% of the household heads had an education level of high school or less in 2022, whereas in 2024, this percentage was 79%. The average number of occupants in the house was 3.79 in 2022 and 5 in 2024. In terms of services and household equipment, the results are very similar between the two periods, with more than 90% of homes having access to electricity, water supply, drainage, television, and cellular phone. Only 45% of the respondents owned a computer in 2022 (39% in 2024), and 80% had internet access in 2022, versus 85% in 2024.

In 2022, 49% of the households reported a monthly family income between MXN 3000 and 9999 (43% in 2024), and only 12% reported a monthly family income above MXN 9999 (18% in 2024). In 2022, 25% of the respondents were attending school, versus 11% in 2024. On a 1 to 10 scale, 10 being the best health situation, the average value in 2022 was 6.81, versus 8 in 2024. Some 55% of households reported having access to health services in 2022, versus 66% in 2024.

In 2024, 49.8% of the respondents were beneficiaries of the MAPFRE-UP Community Center. The socioeconomic and demographic characteristics of the beneficiaries group compared to the non-beneficiaries are very similar. Related to income, 25% of the beneficiaries reported family income in the range of MXN 3000–5999, whereas the percentage for the same range in non-beneficiaries was 22%. In terms of education level for the household head, among beneficiaries of the community center, 74% reported an education level of high school or less, and among non-beneficiaries, this percentage was 83%. In terms of household equipment, both groups show similar conditions: 36.5% of beneficiaries own a car, while 39.8% of non-beneficiaries do; 88% of beneficiaries have internet access, while 83% of non-beneficiaries do; and 37% of beneficiaries own a personal computer, while 41% of non-beneficiaries do. The average perceived level of quality of life (based on a 1–10 scale) was 8 for both groups. The average perception of health level (based on a 1–10 scale) was 7.87 for beneficiaries and 8.13 for non-beneficiaries.

4. Materials and Methods

The methodology is based on the proposal of the World Bank and the Inter-American Development Bank developed by Gertler, Martínez, Premand, Rawlings and Vermeersh in 2011 and revised in 2017. It consists of carrying out an impact evaluation [1] in a context in which scientific evaluations are integrated into program activities and determining the effect of implementation on users or beneficiaries detected worldwide.

This research has a quantitative empirical approach based on primary sources and information from the survey technique. In formal terms, it is an exhaustive analysis of causality. Unlike other quantitative methods, it defines the extent to which the intervention affects the quality of life of users, tests the causality between impact and the proposed objective, and measures this causality. The results obtained are based on the objective of the program, so it is possible to determine which areas are subject to improvement.

The operating director of the center coordinated face-to-face interviews in order to understand the socioeconomic and demographic profile of the inhabitants of the area of influence, the background of the region, the main concerns and deficiencies, and people’s perceptions of quality-of-life related issues. These results form the basis of the initial activities of the community center.

The impact evaluation aimed to identify the effects on beneficiaries compared to non-beneficiaries using indicators that show the main objective of the intervention, so it was necessary to start with a baseline, that is, the situation at a moment in time before the program began to operate or before the detected improvements were implemented, as was the case with the CCMUP. Once the CCMUP stopped operating due to the confinement of COVID-19, the baseline was constructed using the survey technique. Based on the results, it was possible to assess improvements in services and at the operational level.

First, a reconnaissance of the area was carried out on 2021 to learn about the context in which the CCMUP operates. Subsequently, the first survey was carried out in 2022 to build the baseline with which the counterfactual narrative was determined. The CCMUP then reopened its doors with a new intervention and improvements in its operational processes. With the data obtained from the baseline, indices were constructed to measure the dimensions of quality of life that the community center could improve, such as health, development capacity, and digital access. The outcome variable is a weighted index created on the indices of the different dimensions.

To assess the causal effect of the intervention, it is necessary to measure the impact that only the program has on changing an outcome. In this study, variation is measured on indices constructed from the dimension of health, capacity building, and digital access.

The selected evaluation method allows causality to be established between an intervention program and an outcome to discard the possibility that any factor other than the intervention explains the observed impact [1]. According to the characteristics of the intervention, the selected method was differences in differences.

The impact assessment measurement was carried out using the calculation proposed by Gertler [1]:

$$∆=\left(Y|P=1\right)-\left(Y|P=0\right)$$

(1)

The causal impact (Δ) of a program (P) on an outcome (Y) is the difference between the outcome (Y) with the program (P = 1) and the same outcome (Y) without the program (P = 0). The group (P = 1) is the treatment group, and the group (P = 0) is the counterfactual that describes the comparison group [1].

Since it is impossible to measure the same subject in two different realities (with and without the program) at the same time, it was necessary to develop an approximate measure. This measure is the counterfactual, defined as the result if a person had not participated in the benefits of the program. In the calculation, the counterfactual is (Y|(P = 0)|). The treatment group for this research was made up of members of the CCMUP, and the comparison or counterfactual group was non-members of the CCMUP [1].

According to the methodology, selection bias is reduced when groups are randomly chosen in two different periods and the observable characteristics are the same. In the case of this research, both groups share the same sociodemographic and cultural profiles. The observable characteristics of these profiles were obtained from the context and baseline applied in 2022.

4.1. Sample

The sample size was determined from the target population that is delimited by the Pueblo de Santa Fe in Mexico City. The population, according to the Population and Housing Census (CPyV) carried out by the National Institute of Statistics and Geography (INEGI) in 2020 [38], is 8541 inhabitants. The sample size for a finite population is calculated as follows:

$$n={\displaystyle \frac{N·Z_{\frac{a}{2}}^2·p·q}{e^2·\left(N-1\right)+Z_{\frac{a}{2}}^2·p·q}}$$

(2)

where

n: sample size;

N: population size;

Z: standard score statistic;

e: maximum accepted estimation error;

p: probability of occurrence of the event;

q = 1 − p: probability of non-occurrence of the event.

For a finite population of 8541 inhabitants, with an estimation of 95% confidence, maximum accepted error of 0.05, and probability of occurrence or heterogeneity of 0.5, the sample size is 368.

In 2022, the baseline survey was carried out through the application of 470 questionnaires of 419 questions each, while in 2024, 401 questionnaires with 427 questions were applied to the beneficiaries and non-beneficiaries of the Pueblo de Santa Fe. In the end, two standardized databases of 401 individuals were obtained, comprising 200 beneficiaries and 201 non-beneficiaries. Therefore, the sample was significant at a 95% confidence level.

The databases are published in Velazquez Salazar, Marisol; DelaTorre-Diaz, Lorena (2024), “Impact Evaluation of Community Center in Mexico”, Mendeley Data, V1, doi: 10.17632/8w52j43gcn.1.

4.2. Impact Index: An Approximation of Quality of Life

Quality of life has been measured usng different indicators, from the most general at the macro level to the most specific ones that emerge according to the needs of different areas of study or application. The most widely used approach over time has been the gross domestic product. However, this has led to much criticism because it is too general and, in heterogeneous populations, does not represent all inhabitants or actual living conditions.

There is a consensus that quality of life is a multidimensional concept. Krysk et al. [39], based on the work of Felce and Perry [40], highlight that five dimensions are used to approximate it: physical well-being, material well-being, social well-being, emotional well-being, and development and activity. These factors are derived from external, subjective, and personal factors and other measures of well-being, satisfaction, or happiness. However, there are many ways to approximate these dimensions and, therefore, quality of life. Eurostat [41] has included eight other dimensions besides GDP, based on material living conditions, leisure, social interactions, economic and physical security, governance and fundamental rights, natural and living environment, and general life experience. Other authors have included indicators that have to do with context [42,43,44], and the World Health Organization [45] has its own measurement. One of the main problems with using a created quality of life index is that the level at which they can be applied is not necessarily the level at which the study will be carried out. In the case of the CCMUP, the study was carried out at the individual level, and the gross domestic product (or any other measure generated by the statistics institute) is at the state or national level. The smallest area to which the data can be applied is the basic geostatistical area level, and even then, it is still very general for the specific population in which the CCMUP operates.

In this research, the aspects of quality of life that are measured are limited to what the CCMUP can offer given its objectives, structure, scope, and resources. It is an approximation that allows us to evaluate how much has been achieved with what we have. Since an index is not currently being used and there is flexibility in the choice of variables that are in accordance with the objective of the center, our method can be replicated in other community spaces to measure the effectiveness of programs and support offered on a small scale. This arises from the need to evaluate the effectiveness of the strategy with which local interventions are designed and whose target population is small in relation to national or international programs. The advantage of the flexibility of creating particular indices for each intervention is that it reflects the results in the specific population in which the program has been carried out.

Likewise, members of the CCMUP can be beneficiaries of different programs or be users of services from other community centers, so it is even more important to be able to do so.

The first step in creating the index is to know the objective of the community center and from there create the baseline with the application of a questionnaire that incorporates the variables defined according to the objective of the intervention of the community center or social program.

In this study, the baseline was built from the definition of the objective, which was to measure the influence of the services offered by the community center on the quality of life of the inhabitants of Santa Fe. Subsequently, the indicators that were intended to be measured before and after the intervention were chosen, which were divided into nine sections: demographic and family profile, employment status, educational profile, health, nutrition, legal advice, governmental and non-governmental support, community and environment, and religion. The indicators of each section chosen for the impact evaluation were those on which the community center has influence, and reliability was measured via Cronbach’s alpha to ensure that they were relevant and linked to the measurement carried out. The groups that were selected were the members of the community center as the treatment group and the non-members as the control group. The information was collected through a survey technique, with questionnaires supplied to both groups in the town of Santa Fe where the CCMUP is located. The application was recorded in the questionnaires, and recordings were made so no answers were omitted. Data processing was carried out through capture, exploratory analysis of data and establishment of reference lines. The application time for each survey was 40 min per questionnaire. The enumerators were previously trained to comply with the code of ethics and the appropriate survey.

The impact index (II) proposed as an approximation of quality of life is the result variable (RV) used to measure the impact of the community center on the population being influenced in the Pueblo de Santa Fe. It was constructed based on the indices of quality of life, human development, and poverty [6,7,46,47]. It was considered necessary to create a particular index due to the scale of the community center, the target population, and the services offered to improve the quality of life of the inhabitants of the town. Given that services are delimited for the population according to the needs detected in the baseline and are different from or complementary to those served by the local and federal government, a measurement index was generated that contains the health dimension (measured by 32 variables that describe diseases, chronic diseases, diet, pollution of the environment, and perception of health); capacity building (that contains 5 variables and refers to access to workshops, courses, consultancies, advices and training in general) and digital access (that contains 5 variables). To generate the general index, the three mentioned indices were constructed, and then each one was weighted to achieve a comprehensive impact index.

The questionnaire (Supplementary Materials) is divided into nine sections according to the following

Table 1. Sections of the questionnaire.

1. Demographic and family profile	Family occupations and roles Family activities Characteristics of the house Internet access
2. Employment status	Employment Occupations and trades Revenue Household income distribution
3. Educational profile	School grade Additional classes Supports for studying
4. Health	Diseases Housing environment (noise, cleanliness, water, pests) Crime, drugs Health services (health insurance)
5. Nutrition	Food Food consumption per week Food at home or away from home
6. Legal Advice	Type of needed advice, cost, type
7. Governmental and non-governmental support	Support, incentives, scholarships
8. Community and environment	Current situation of the environment, security
9. Religion	Professed religion

Source: Self-elaboration.

. Each section describes the characteristics of the population in the area influenced by the CCMUP. The questionnaire was designed based on an initial instrument that was applied before the opening of the center, which is the basic questionnaire applied to the 2020 Population and Housing Census [38] by the National Institute of Statistics (INEGI) and the National Council for the Evaluation of Social Development Policy (Coneval) [46].

The variables that were chosen for each index and the Cronbach’s alpha are shown below in

Table 2. Cronbach’s alpha.

Index	Cronbach’s Alpha	Reliability	Variable
Health	0.789	High	Physical activity
			Perception of health status
			Disease
			Pollution
			Safety
			Access to health insurance (social security or private insurance)
			Nutrition
Capacity Building	0.747	High	Legal services and assistance
			Courses and workshops
Digital Access	0.607	Medium	Internet access and use of technology

Source: Self-elaboration.

. The specific items of each variable can be consulted in Supplementary Materials. The variable and code labels can be consulted in the database published in Mendeley Data [48].

Each index was calculated by the proportion of the observed value of the variable compared to the maximum value that the variable can reach. That is, it approximates how well the individual is with respect to how well off he or she can be. For each individual, the proportions of each variable were added and divided by the number of variables answered, obtaining the average index per individual. Each dimension was measured separately and for each individual, for the comparison years 2022–2024. In all dimensions, the assumption is that the more the better (more health, more capacity buildings, more digital access is better). The ratio goes from 0 to 1, where 0 is worse and 1 is better.

4.2.1. Health Index

Although health has numerous aspects that can be measured, this study and under this methodology include the variables in which the community center can influence according to its own objective. Thirty-two variables were chosen that provide information on diseases, chronic diseases, diet, health perception and pollution of the environment. The health index is calculated as follows:

$$HI={\displaystyle \frac{\mathsf{\Sigma }H{V}_{ij}}{n}}$$

(3)

where

HI is the health index

HV is the health variable.

$$HV={\displaystyle \frac{{observed variable}_{\left(ij\right)}}{{maximum variable}_{\left(j\right)}}}$$

i = individual (1 to 401)

j = variable (1 to 32)

n = answered variables $(n\ne 0)$

4.2.2. Capacity Building Index

The capacity building index measures access to workshops, courses, consultancies and education that meet human development needs. Five variables are contemplated and, as in the case of health, they are only the variables on which the center can have an impact. It is calculated as follows:

$$CBI={\displaystyle \frac{\mathsf{\Sigma }CB{V}_{ij}}{n}}$$

(4)

where

CBI is the capacity building index;

CBV is the capacity building variable.

$$CBV={\displaystyle \frac{{observed variable}_{\left(ij\right)}}{{maximum variable}_{\left(j\right)}}}$$

i = individual (1 to 401)

j = variable (1 to 5)

n = answered variables $(n\ne 0)$

4.2.3. Digital Access Index

In principle, this dimension was not considered in the study, but it was found that access to technologies and digital literacy are relevant variables that individuals value significantly and that address the imminent inequality gap in technological development. Considering this, digital access was incorporated as a separate dimension and includes variables that were captured in the questionnaire on internet frequency and access to and possession of technological assets. It is calculated as follows:

$$DAI={\displaystyle \frac{\mathsf{\Sigma }DA{V}_{ij}}{n}}$$

(5)

where

DAI is the digital access index;

DAV is the digital access variable.

$$DAV={\displaystyle \frac{{observed variable}_{\left(ij\right)}}{{maximum variable}_{\left(j\right)}}}$$

i = individual (1 to 401)

j = variable (5)

n = answered variables $(n\ne 0)$

Once the improvements to the CCMUP were applied during the second half of 2022 and until March 2024, the following survey was carried out to measure the impact in the 2022–2024 period.

4.2.4. Impact Index of Quality of Life

The resulting variable obtained from the baseline was the weighted index of the indices of the dimensions of health, capacity building, and digital access. A weighting was assigned to each index under the assumption that a better quality of life should be balanced and contain a similar proportion of each dimension. The impact index is calculated as follows:

$$II={\displaystyle \frac{1}{3}}HI+{\displaystyle \frac{1}{3}}CBI+{\displaystyle \frac{1}{3}}DAI$$

(6)

where

II = impact index;

HI = health index;

CBI = capacity building index;

DAI = digital access index.

4.3. Impact Estimation Method: Differences in Differences

The eligibility criteria for the program or intervention determine the quantitative method to be applied to assess impact. For this research, the differences in differences (DD) method was chosen because eligibility to become a member of the community center does not have explicit criteria assigned by the CCMUP, so the methods of random assignment, instrumental variables, and discontinuous regression are not applicable [1].

The discontinuous regression method was discarded because the intervention program does not decide who is entitled to enroll and who is not. Random assignment is not chosen because the selection is not by lottery or random selection. Random promotion is not chosen by instrumental variables because there is no randomized promotion treatment.

The difference in differences method compares changes in outcomes over the study period between beneficiaries (treatment group) and non-beneficiaries (comparison group). This reduces selection bias and corrects for any differences between groups over time [1].

The first difference controls the results between the groups at a point in time, and the second difference eliminates external factors that vary over time. The difference in-differences method combines the two false estimates of the counterfactual, the one that refers to time and the one that refers to beneficiaries and non-beneficiaries. The estimation using this method is the change in the outcome variable of the comparison group. To apply DD, outcomes are measured in the treatment group, represented by the CCMUP beneficiaries, and then compared with the comparison group, which is measured by the non-beneficiaries of the CCMUP, both before and after the intervention.

The simple difference or first difference was calculated for each year and is the difference in the result between beneficiaries and non-beneficiaries. The indices of the different dimensions were the result variables, and the impact variable was whether or not the individual is a beneficiary of the CCMUP. The simple difference is calculated as follows:

$$I=\beta MC+e$$

(7)

where

I: first (or simple) difference index (0 to 1);

MC: membership at the CCMUP (no membership = 0, membership = 1);

$\beta =Counterfactual$;

e: error.

The counterfactual measures what would have happened if the individual had not been a beneficiary of the CCMUP, that is, the simple impact of each year (only the difference between the treatment group and the comparison group for each year, 2022 and 2024, defined separately) was measured.

The DD method measures impact over time by eliminating eligibility bias and bias caused by external factors over time. The outcome variables are the differences between the indices of each year. The only impact variable is membership of the CCMUP. It is calculated as follows:

$$DD=\beta MC+e$$

(8)

where

DD: differences in differences index;

MC: membership at the CCMUP (no membership = 0, membership = 1);

$\beta$: counterfactual;

e: error.

5. Results and Discussion

For the finite target population of the CCMUP, a sample of 401 individuals was selected for the years 2022 and 2024. The simple differences that measure the difference between the treatment group and the comparison group are shown below (

Table 3. Simple difference results.

Outcome	Coefficient	Standard	Statistic t	p-Value	r-Squared
Variable		(Counterfactual)		Deviation
HI2022	0.498702	0.0263531	18.92	$1.69\times {10}^{-57}$ ***	0.472373
HI2024	0.656666	0.0323643	20.29	$1.92\times {10}^{-63}$ ***	0.507193
CBI2022	0.101000	0.0148154	6.817	$3.43\times {10}^{-11}$ ***	0.104092
CBI2024	0.723000	0.0402638	17.96	$2.67\times {10}^{-53}$ ***	0.446319
DAI2022	0.538583	0.0335876	16.04	$4.8\times {10}^{-45}$ ***	0.391290
DAI2024	0.620000	0.0344746	17.98	$2.03\times {10}^{-53}$ ***	0.447081
II2022	0.379428	0.0207909	18.25	$1.43\times {10}^{-54}$ ***	0.454336
II2024	0.666555	0.0331832	20.09	$1.46\times {10}^{-62}$ ***	0.502174

*** Statistically significant. Source: self-elaboration.

).

The r-square has been included in the models, although it is worth mentioning that although it is low, the models are not intended for predicting future data. The evaluation of which model best applies was made, and the results show that linear regression had as good a fit as gradient boosting, random forest, neural network, and others. The errors of these models have minimal differences. In addition, in models with a single discrete independent variable, the model used for standard methodologies is linear regression.

In the three dimensions of health, capacity building, and digital access for both years, there was a positive coefficient, which indicates that there was improvement for the beneficiaries of the center because they belonged to it. The counterfactual measured by the coefficient indicates what would have happened if the individuals had not accessed the benefits of the CCMUP; in the case of health in 2024, being a member of the CCMUP impacted individuals’ health by 0.66 points out of 1, unlike 2022, in which it impacted health by almost 0.50 points. In other words, being a member of the CCMUP benefited individuals’ health positively by between 0.5 and 0.65 points. In the case of capacity building, the impact ranged from 0.10 in 2022 to 0.72 points out of 1 in 2024 and shows a significant difference between one year and other. Digital access was less sensitive and had 0.54 and 0.62 coefficients for 2022 and 2024. From the simple difference, we observed that individuals with access to computers or tablets with internet offered by the center had an a significant incidence and are highly valued by individuals. In a low-income population, access to technology may be an opportunity to close the inequality gap created by technological development, especially for young people and the elderly.

The following difference subtracts external factors that can be derived from the comparison between one period and another over time. Although it would be easy to calculate only the difference between one year and another from the table of simple differences, we chose to model it using several models and choose the one that shows the least error.

The simple linear regression was estimated by taking the difference in the indices between 2022 and 2024 as the result variable and belonging or not to the community center as an impact variable. The difference between the indices ranges from 0 to 1, where center membership is 1 and non-membership is 0 (

Table 4. Differences in differences results.

Outcome Variable	Counterfactual Coefficient	Standard Deviation	t-Statistic	p-Value	r-Squared
DD Health	0.1579640	0.0132946	11.88	$4.30\times {10}^{-28}$ ***	0.260872
DD Capacity Building	0.6220000	0.0400099	15.55	$5.7\times {10}^{-43}$ ***	0.376639
DD Digital Access	0.0814167	0.0263309	3.092	$2.10\times {10}^{-03}$ ***	0.023344
DDII	0.2871270	0.0185997	15.44	$1.46\times {10}^{-62}$ ***	0.373341

*** Statistically significant. Source: self-elaboration.

).

The results show that there is a positive impact of 0.16 on the index of health (DD Health), of 0.62 on capacity building (DD Capacity Building), and of 0.08 on digital access (DD Digital Access). The intervention in the period analyzed was successful and improved the dimensions of health, capacity building, and digital access.

On the other hand, the impact index (DDII), which weights each dimension in a similar way, shows a counterfactual of 0.2871. A beneficiary of the CCMUP would have missed a 28.71% improvement in his or her quality of life had they not been members of the center. In other words, health, capacity building, and digital access as a whole improved positively.

Impact evaluations that are considered successful have an average positive impact of 14% and a median of 8% [4,9,10]. These evaluations have been carried out for long-range public programs and for vulnerable populations. Among these, the “Bolsa Escola” Program in Brazil stands out, which had an impact of 8% [49]; the construction of schools in Indonesia had an impact of 3.5% [50]; and the construction of rural roads and local market development in Vietnam had an impact of 10% [51]. For these three cases, the method was differences in differences.

The most extreme cases of impact evaluation have to do with sensitive issues within populations at risk, such as police deployment to reduce the crime rate in Argentina, which had an impact of 75% [12], and the HIV and adolescent pregnancy awareness program in Kenya, which had an impact of 28% [1].

In the case of Mexico, the National Council for the Evaluation of Social Development Policy (CONEVAL by its acronym in Spanish) evaluated the impact of several programs aimed at small- and medium-sized enterprises, which had an impact of 6% on gross value added and 5% on total gross production, total sales, employment, and fixed assets. The study was carried out using the difference in differences method [52]. The United Nations Development Program (UNDP) evaluated the impact of several federal programs related to the empowerment of women in Mexico, which had an average impact of 9% [6].

Looking at these references and considering that no impact studies of small community centers of private origin such as the MAPFRE-UP Community Center could be found, it can be concluded that the impact generated was positive and comparable to other success stories worldwide.

6. Conclusions

Impact evaluation is a methodology that allows the implementation of interventions to be measured using a quantitative method that can be replicated at any scale. Therefore, the results presented are comparable with other impact measurements worldwide.

The impact is measured in relation to the objective of the community center, which is to influence the quality of life of inhabitants of the area of influence. Given that quality of life is a broad concept that is difficult to define, three indices were created that approximate different dimensions of quality of life and that the center, by definition, could impact. According to our results, quality of life, as approximated by the dimensions of health, capacity building, and digital access, improved by 0.2871 points out of 1 for those individuals who belong to the center and were part of the intervention. The result obtained can be considered a success story, since the impact has been positive and significant.

As impact assessment is part of the process of social-aid programs, it is necessary to deepen into each dimension to determine a future strategy, given we have baseline data and the impact assessment carried out in 2024. The next evaluation will be carried out in 2027 to follow up on the process and determine whether the new strategy has worked.

Health is a sensitive dimension that presents positive and almost immediate changes given the services offered by the community center. In addition, it has been observed that improving health also improves health expectations, that is, when an individual is intervened with by health services, he or she feels better and expects to continue improving. Seeing that health has improved generates expectations of greater well-being.

In terms of capacity building, the impact was positive and higher than the other dimensions. Workshops, consulting, training, and education had an immediate effect. The benefits observed by the beneficiaries in the short term were highest because they contributed to many aspects of their lives such as social activities, lessons that apply to everyday life, mental health, and nutrition, among other things.

Digital access was the dimension that was less impacted by the community center, which reflects the importance of focusing efforts so that populations with limited access can use the tools and access all available information and technology, among other benefits. It was observed that the population is aware of the inequality gap that exists between those who can access the internet and those who cannot.

The CCMUP began operations in 2016, but at that time, an impact evaluation was not contemplated and the baseline survey had not been conducted. The possibility of rebuilding it was analyzed, but confinement due to COVID-19 forced the center to stop its operations in face-to-face services; from that moment on, it was rebuilt. The intervention was considered to begin when the center reopened, in 2022, with face-to-face services and improved operations.

Although the pandemic allowed for the baseline construction, it is important to go back to the beginning and try to observe the impact the center has had since opening. To do this, it is necessary to apply different statistical techniques. For other interventions, it is advisable to develop a baseline before they begin and consider the impact evaluation as part of the operation of the program.

The results of this research will be used for decision making at the operational level of the CCMUP and for access to external financing to fund the next intervention. In future research, the next impact evaluation of the CCMUP in the population of Santa Fe is scheduled for the year 2027. Thus, the database has also been shared with the local networks of the town of Santa Fe so that they can develop studies that are pertinent to them. The replicability of the study for public programs has also been considered by the research group. For short-term or one-time interventions, other evaluation methods are recommended. However, it is important to point out the need for the evaluation of interventions on quality of life in order to meet the specific needs of communities in addition to the general needs that are met through public programs in marginalized areas.

Cooperation and work between different community centers, the local and federal government, and neighborhood organizations in the area must consider the evaluation of their interventions and develop a common strategy to cover all aspects and specific needs of the population in order to become a sustainable community. It has been observed that there have been few efforts in the area to recover natural areas and cultural heritage, so it may be necessary to analyze which actor or actors could carry out these tasks.