Changes over Time Matter: A Cycle of Participatory Sustainability Assessment of Organic Coffee in Chiapas, Mexico

Abstract

This paper discusses how to incorporate the changes within an agroecosystem into sustainability assessment. We measured the sustainability of 86 organic coffee producers located in 4 municipalities of Sierra Madre of Chiapas, Mexico. Based on the MESMIS framework, a set of indicators was selected. A sustainability index was constructed using the multi-criteria decision method known as the analytical hierarchy process in a broad participatory process with producers and cooperatives. Likewise, classification of producers was carried out through land use transitions in their lands between 1999 and 2019 using satellite imagery. A variance analysis between classes was performed to identify statistical differences. The results show that producers who expand their coffee plantations into forested areas have more sustainability levels than those producers who convert agricultural activities into coffee plantations and producers with both transitions in coffee expansion-conversion. The indicators of labor efficiency, benefit-to-cost ratio, price, remnant of primary vegetation, recovery of production from natural phenomena, family succession, and producer land area showed statistical differences between classes. These results are relevant because sustainability assessment is an action-oriented method. Therefore, the contribution of this study can help guide recommendations for specific groups of producers.

1. Introduction

The concept of sustainable development is regaining strength as social and environmental crises deepen throughout the planet [1,2]. Hence, the challenge of meeting the present needs without compromising the ability to meet future needs involves moving toward economically viable, socially just production systems that make efficient use of natural resources [3]. The intergovernmental proposal of the 17 Sustainable Development Goals (SDGs) recognizes that each country faces specific challenges for achieving sustainable development, emphasizing the special problems that prevail in the most vulnerable countries [4].

Sustainable development implies an improvement of human life, with the long-term vision of eradicating poverty and reducing inequality along with both production and consumption being more sustainable. The concept of sustainability more specifically is associated with a quality or principle applicable to systems, involving a dynamic evolution between social, economic, and environmental components to preserve their functioning and taking into account both stressors and disturbances [5,6]. However, there are discrepancies in the use of the most appropriate philosophical and ideological aspects of sustainability, as well as which are the most appropriate methodologies to assess it [7]. In this way, sustainability assessment is affected by problems inherent to the multidimensionality of the concept itself [7,8].

Assessing the level of sustainability in a given system is a crucial task for understanding which properties allow it to maintain its permanence over time [9]. According to Gallopin [10], for a system to be sustainable, it requires (1) availability of resources, (2) adaptability and flexibility to detect changes in the environment, (3) the capability of preserving essential values of internal variables for maintaining both stability and structure, and (4) the ability to respond and change strategies in the face of disturbances.

Furthermore, in the case of agroecosystems, Pretty [11] pointed out that four principles are required for achieving sustainability: (1) to integrate biological and ecological processes within the production process, (2) to minimize the use of non-renewable inputs, (3) to use the knowledge and skills of producers for improving their self-reliance, and (4) to work together in solving common problems through the collective capacities of producers.

Sustainability assessment has been developed at different scales. Studies such as that by Gromada et al. [12] have assessed development between countries using aggregated ecological, social, and economic indicators. Among the vast diversity of studies aimed at assessing the sustainability of production systems or agroecosystems, some are carried out at the production unit level through environmental, social, and economic indicators [13,14]. Assessments have been carried out at regional scales, using input indicators for assessing yields, environmental impacts, and levels of sustainability [15,16]. Other studies, for example, have applied life cycle analysis [17,18].

Concerning agroforestry systems, studies developed for evaluating sustainability have been based on analyzing livelihoods with coffee and cocoa producers using social, economic, and environmental indicators [19]. Some analyses have focused on comparing coffee production [20,21] or, more specifically, addressing nutrient balance [22].

The Natural Resource Management and Sustainability Indicators Assessment Framework (henceforth MESMIS, its acronym in Spanish) is one of the more frequently used frameworks for assessing sustainability in Latin America [23]. It consists of a systemic, multi-scalar, and participatory framework based on identifying key processes and critical points for several indicators. Most of its applications have been carried out in Latin American countries, although there are cases reported in Spain, Portugal, and the United States among other countries [24]. Such studies have focused on agricultural and agri-food products, such as lemons, cotton, and tequila [25,26,27]; conventional, organic, or silvopastoral livestock production [28,29]; or bioenergy production [30]. In the case of agroforestry systems, studies on cocoa [31] and coffee [21,32,33] are worth noting.

However, most of these studies do not delve into how to integrate the changes in the systems nor the diverse perspectives and knowledge from local stakeholders. These elements are necessary to operationalize the normative concept of sustainable development based on sustainability assessments [34]. Most sustainability assessment methods that incorporate local knowledge require longitudinal studies to identify the dynamic properties of agroecosystems [35]. In other words, the whole assessment should be repeated at different periods of time. This is a time-consuming and expensive method, which represents the main drawback. Hence, the main contribution of our study is that we propose a method which incorporates changes within the agroecosystem using a single sustainability assessment cycle by means of a modified MESMIS framework. We measured 86 organic coffee producers’ sustainability levels in four municipalities of Chiapas belonging to the El Triunfo Biosphere Reserve in Mexico. This study was performed with a deeper insertion of participatory processes at different stages. In order to incorporate the changes of the agroecosystems, we classified producers according to land use transitions in their plots. Furthermore, we identified statistical differences between the producers’ classes.

Hence, this paper is organized as follows. Section 2 shows a brief synthesis of the MESMIS framework. The local context of the area of study is described in Section 3. In Section 4, the methodology is developed. In Section 5, the results and discussion are presented, ending with conclusions in Section 6.

2. The MESMIS Framework

The MESMIS framework is a methodological proposal for assessing the sustainability of a system from indicators of seven attributes related to the social, environmental, and economic dimensions [36]. According to Masera et al. [37], it comprises six steps (Figure 1). In Step 1, both the objective and study scale are defined. Step 2 corresponds to identifying the strengths and weaknesses of the system under study by identifying critical points and systemic attributes. In Step 3, the indicators used to measure the systemic attributes are selected or constructed. In Step 4, these indicators are computed or sampled. The resulting information is integrated into a report and presented to stakeholders in Step 5, and Step 6 incorporates such feedback in the form of conclusions.

The measurement of sustainability by MESMIS is based on seven systemic attributes, which show the properties that emerge from the dynamic interaction between social, economic, and environmental processes [38]. According to García-Barrios et al. [39], the attributes are defined as follows: (1) productivity, which is the level of goods and services provided by the process per unit of time and per unit of input invested; (2) stability, which quantifies the production processes based on internal mechanisms that self-regulate the state of the critical variables (i.e., the variables should remain within the ranges that allow the process to work); (3) resilience, which is the speed with which the disturbed variable returns to its previous state; (4) reliability, which occurs when disturbances lead to variables that allow operation beyond a threshold limit; (5) adaptability, which corresponds to the capacity of a process to reorganize itself internally to continue working when it presents irreversible internal or external changes; (6) self-management, which refers to its capacity to function, regulate, and evolve, depending more on its own resources, interactions, and internal processes rather than on external conditions, disturbances, and interventions; and (7) equity, which occurs when the process allows the appropriate distribution of benefits and costs among the social agents that participate in it.

According to Quintero-Angel and González-Acevedo [40], the main drawbacks of the MESMIS framework are that, on the one side, it does not integrate the principles of the laws of thermodynamics. Moreover, on the other side, the weighting of its systemic attributes is still ambiguous. Tonolli and Ferrer [41] have pointed out other limitations, considering two main weaknesses. The first is that it has limited generalization power due to its semi-quantitative nature. The second refers to the cyclical process of assessments, since this characteristic can lead to modifications based on circumstantial situations, which might generate imprecise conclusions about the critical points of the system under study. Nevertheless, despite such drawbacks, the MESMIS is part of the efforts to have more holistic sustainability assessments. In fact, any sustainability indicator presents both advantages and drawbacks [42].

The cyclical nature of MESMIS makes it possible to measure the changes in the management system once that assessment process has been developed at least twice. In this paper, changes in the management system have been incorporated into the first cycle of the assessment process.

3. Study Area and Local Context

Shade-grown coffee is one of the most commercialized and consumed agri-food products globally [43]. Mexico is one of the primary producers of shade-grown coffee worldwide and a leader in the production of organic coffee [44]. Organic coffee production emerges as a diversification that promotes greater social and environmental benefits than its conventional production [45]. This system implies an improvement in the health and functioning of the agroecosystem derived from the elimination of the use of chemical inputs and the maintenance of vegetation cover, which acts as shade for coffee plantations [46]. Likewise, it promotes greater benefits to small producers through better prices derived from obtaining certifications and the corresponding access to export markets [45]. In Mexico, the trading system for shade-grown coffee is carried out chiefly by cooperatives, from which producers have access to technical assistance and financing. The cooperatives purchase their production for its subsequent transformation and commercialization [44].

The study area is located in the El Triunfo Biosphere Reserve (REBITRI) and its buffer area, comprising the municipalities of Ángel Albino Corzo, Capitán Luis Ángel Vidal, La Concordia, and Siltepec (Figure 2). Most of the region extends along “The Sierra Madre de Chiapas” (SMC) mountain range, where 85% of the surface corresponds to different types of forests and jungles, both primary and secondary [47], with a high proportion of cloud forests, which stand out as being of high ecosystemic importance [48,49].

The region has a long coffee-growing tradition dating back to the end of the 19th century, when the activity was introduced by European immigrants who established large farms for its production [50]. After the dissolution of the latifundia regime and the agrarian distribution in the first decades of the 20th century, the region consolidated itself as one of the country’s main shade-grown coffee-producing areas [51]. During the second half of the 20th century, a government-based agency for promoting the production of coffee, INMECAFE, was created. Hence, technological packages based on the green revolution and improved coffee varieties helped to support coffee plantations [52]. At the end of the 1980s and the beginning of the 1990s, the International Coffee Agreement collapsed, and INMECAFE was dismissed. The consequence was a halt of stockpiling at preferential prices, technical assistance, granting of credits, commercialization, and regulation that INMECAFE carried out in the region [53]. This situation, together with the work of conservation organizations with producers in the region, promoted organizational schemes among producers that sought to obtain better prices, financing, and technical assistance during the 1980s and 1990s [54]. Along this process, the REBITRI Biosphere Reserve was decreed in 1990, and its management program was implemented in 1999. Therefore, the cooperative scheme and its transition to organic production were strengthened to obtain greater social, economic, and environmental benefits since then [44].

4. Materials and Methods

The sustainability of the shade-grown coffee agroecosystem in the REBITRI Biosphere Reserve was assessed using the MESMIS framework. It is focused on integrating both the changes observed in the agroecosystem and the local knowledge. Both quantitative and qualitative methods were used in this assessment for facilitating the measurement and integration of the several perspectives. The methodology was carried out through four steps (Figure 3): (1) conceptual characterization, (2) classification of producers, (3) assessment of critical points, indicators, and weighting, and (4) sustainability measurement and differences analysis. The steps are described below.

(1)

Conceptual Characterization

The concept of sustainability applied to organic coffee production was defined by the stakeholders themselves in a participatory workshop with producers, technicians, and representatives of three coffee cooperatives. The result was a participatory description of the relationships between producers and cooperatives, the management practices at the plot level, and the possible changes that can be observed in the producer’s lands during the study period.

(2)

Classification of Producers

Once the conceptual characterization was defined, the next step was to classify the producers according to the changes in the producer’s land (i.e., transitions of land use) obtained through remote sensing analysis. Hence, three scenarios were assumed, departing from a typical initial structure. This initial structure was assumed to be a plot where coffee production, agricultural systems, and forests converge. Thus, a land use transition was generated using Landsat 5 and 8 satellite images, corresponding to the years 1999 and 2019, respectively. Six classes were obtained: agriculture, grasslands, primary forest, secondary forest, bodies of water, and other covers. The classification was carried out using the supervised method through artificial neural networks (ANNs). The ANN is an information processing model inspired by the information processing mechanism of the human brain and has been employed for many remote sensing applications [55,56]. A multi-layered feedforward ANN is used to perform nonlinear classification and uses backpropagation for supervised learning and in different comparative studies than other conventional classification techniques [57]. The calibration and verification samples were obtained from geospatial data from the Land Use and Vegetation Series II [58] and III [59], from the National Forest and Soil Inventory [60], from data on the activity of monitoring for the Mexican REDD + program [61], and tree cover reported by Townshend [62] and Hansen et al. [63].

(3)

Assessment of Critical Points, Indicators, and Weighting

A second workshop took place with representatives of each cooperative, where the indicators associated with the critical points were defined. Other sustainability assessment works [64,65] served as reference indicators for facilitating the discussions. Once the indicators of each critical point were defined, a selection was made of those that allowed establishing critical thresholds (Ct) and sustainable thresholds (St), which were defined through the traditional knowledge of the participants.

Hence, if the indicator’s value was x > Ct, then the activity ran the risk of being unviable in the future. If x > St, then the activity did not have any stress and was sustainable in the future. If St < x > Ct, the activity was assumed in the transition zone.

In order to obtain the weights for the indicators, the multi-criteria decision technique of the hierarchical analytical process, was used [66]. Workshops were held in each cooperative’s headquarters, performing a paired comparison of the indicators for each critical point and the critical points for each systemic attribute with the stakeholders. The paired comparison was carried out using the Saaty scale (

Table 1. Saaty’s scale used for the weights for the indicators (modeled after [67]).

Rating Scale	Definition	Explanation
1	Equal importance	Two elements contribute equally to the objective
2	Weak	Between equal and moderate
3	Moderate importance	Experience and judgment slightly favor one element over another
4	Moderate plus	Between moderate and strong
5	Strong importance	Experience and judgment strongly favor one element over another
6	Strong plus	Between strong and very strong
7	Very strong importance	Experience and judgment very strongly favor one element over another
8	Very, very strong	Between very strong and extreme
9	Extreme importance	The evidence favoring one element over another is one of the highest possible order or affirmation

), which shows the importance of one criterion over another [67].

Once the weights were obtained, the inconsistency indices were calculated for each set of indicators for the same critical point and, subsequently, for the critical points and systemic attributes. The inconsistency index allowed for reducing the ambiguity derived from the subjectivity of multi-criteria processes regarding the preferences of a set of actors [68]. The inconsistency index was calculated as follows:

$$CI=\frac{{\mathsf{\lambda }}_{max}-n}{n-1}.$$

(1)

where CI is the inconsistency index, which denotes the rationality of the criterion whenever CI < 0.1, λ_max is the maximum eigenvalue of the paired comparison matrix, and n is the number of criteria that are being compared [69].

Data Collection

It is important to note that the information necessary for the computation of the indicators was obtained by means of fieldwork and captured in a database (See Supplementary Materials). A semi-structured questionnaire was designed and applied to local producers. A sample of 86 from 657 producers was surveyed using the non-probabilistic convenience sampling technique [70]. The sample distribution by sex was 22% women and 78% men. Nevertheless, this study does not show results by sex-differentiated analysis because it was not part of the purpose of the research.

Given that the analysis of changes in the agroecosystem requires geospatial data, only those producers who had geo-referenced plots were surveyed. The applied questionnaire integrated questions regarding the socioeconomic characteristics of the producer and the characteristics of his production unit and productive management, as well as natural resources and commercial and income aspects.

(4)

Sustainability Measurement and Differences Analysis

The producers’ sustainability measurement was carried out in two phases: a general assessment and an assessment by groups. In the general assessment, the critical indicators were calculated, and the proportion of producers located above or below the critical and sustainability thresholds was obtained. On the other hand, the assessment by groups was carried out by calculating the sustainability index of organic coffee production from the weights obtained, from the normalization of the indicators, and from the aggregation of both elements.

The normalization of indicators was carried out through value functions [71], which transformed the values obtained from the indicators on a scale from 0 to 1, where 0 was the least desirable value and 1 was the most desirable value for the sustainability of organic coffee production. The min-max normalization method was used [72], which allowed the process to be carried out by identifying the meaning of each indicator with the goal (in this case, the sustainability of organic coffee production). Therefore, if the indicator was positive (the higher the value, the greater the desirability), Equation (2) below was used. If the indicator was negative (the lower the value, the lesser the desirability), then Equation (3) below was used:

$$X_n=\frac{X_0-X_{min}}{X_{max}-X_{min}}$$

(2)

$$X_n=\frac{X_{max}-X_0}{X_{max}-X_{min}}$$

(3)

where X_n is the normalized value of indicator n, X₀ is the indicator’s value for producer j, X_max is the maximum value of indicator n, and X_min is the minimum value of indicator n.

The sustainability index was generated by the linear combination of the weights obtained and the normalized values from the value function:

$$S_i={\displaystyle \sum }_{j=1}^{j=n}{W}_i{C}_{ij}$$

(4)

where S_i is the sustainability index, W is the weight of indicator i, and C is the value of indicator i obtained through the value function for producer j.

The analysis of differences was carried out for the types (i.e., classes) of producers obtained in a previous phase. This analysis was performed using both the Kruskal–Wallis and Mann–Whitney nonparametric analyses of variance tests through the SPSS statistics 25 program. The Kruskal–Wallis test identifies whether there is a statistical difference in k groups or independent samples for each indicator. One of the advantages of this test is that it allows categorical and continuous variables to identify the differences [73]. When the test is statistically significant, the result shows a difference in at least two groups, and therefore, it is not a random result. The statistic that was applied in this test is the following [73]:

$$KW=\left[\left(\frac{12}{N\left(N+1\right)}\right){\displaystyle \sum }_{i=1}^k\frac{R_i^2}{n_i}\right]-3\left(N+1\right)$$

(5)

where N is the total amount of data under analysis, n_i is the number of repetitions of treatment i, k is the number of treatments being compared, and R² is the sum of the squared ranges of treatment i.

The Mann–Whitney test is another non-parametric variance test that operates similarly to the Kruskal–Wallis test, with the difference that it is applied for two independent samples [74]. As in the Kruskal–Wallis test, the null hypothesis is rejected when p ≤ 0.05. This test was applied to observe statistical differences in the indicators identified with the Kruskal–Wallis test.

5. Results

5.1. Conceptual Characterization

The conceptual characterization obtained from the local actors can be summarized in three different issues: What does sustainability represent? What are the key aspects of agroecosystem sustainability? Lastly, what changes in the coffee agroecosystem have been observed?

• What does sustainability represent to local actors?

The sustainability of organic coffee production was discussed during workshops attended by coffee producers. They conceptualized sustainability as “everything that allows the activity to be maintained over time”. In order to maintain activity over time, the role of cooperatives is decisive in facilitating production and commercial aspects to producers. Hence, one of the chief results of the workshop was a general scheme of the relationship between cooperatives and producers (Figure 4).

In order to obtain an organic coffee certification label, a coffee producer must demonstrate 3 years of free agrochemical production. In the case that a producer starts to change practices (e.g., diminishing the amount of agrochemical inputs), the cooperative grants him or her a “transition status”, which means that he or she has access to several services provided by the cooperative (e.g., technical assistance, financing, or seedlings from an organic nursery). However, his or her production cannot be certified yet. To achieve an export quality, coffee producers must hire day laborers, who help in harvesting, pulping, and washing the coffee grains collected from February to April by the cooperative. The producer is paid 70% of the agreed price at the time of collection, and the remainder is delivered when the coffee is sold to the destination market. The cooperative carries out the process of converting low-valued coffee grain into export-certified coffee by drying and removing the husk.

• What are the key aspects for agroecosystem sustainability?

According to the responses given by the coffee producers during the workshops, there are four key elements for agroecosystem sustainability: shade management, tree cover, plants, and soil management.

Shade management helps to regulate the temperature and humidity of the plantation, thus preventing pests and diseases. It is carried out once a year and is maintained at 60% in the lowlands (up to 1200 m ASL) and 40% in the highlands (above 1200 m ASL).

Tree cover combines native and exotic species, with densities ranging from 80 to 100 trees per hectare. The average age of the plantations is 19 years. According to the local stakeholders, a coffee tree starts being productive at 3 years, with an optimum observed between 5 and 8 years. Most producers have densities ranging from 3500 to 4000 plants per hectare.

Soil management is carried out with individual terraces and live barriers, maintaining organic matter and avoiding bare soil. The use of compost by farmers is a frequent practice that improves soil fertility, moisture, and pH conditions (e.g., Dzung et al. [75]).

• What changes in the coffee agroecosystem have been observed?

Changes in the organic coffee agroecosystem are best explained by assuming a typical agroecosystem structure, where the production of shade-grown coffee converges with both agricultural activities and forested areas (Figure 5).

From this initial structure, it is possible to observe three main changes in the organic coffee agroecosystem: (1) conversion to coffee, which occurs when the agricultural activities are converted into coffee production; (2) coffee expansion, which occurs when coffee plantations expand into forested areas; and (3) expansion-conversion, where coffee production areas substitute both forested and agricultural areas.

5.2. Assessment of Critical Points, Indicators, and Weighting

Table 2. Critical points, indicators, and weighting obtained from the discussion during the workshops with the coffee producers.

Systemic Attributes	Weight (a)	Critical Points	Weight (b)	Indicator (Calculation)	Weight (c)	Adjusted Weight (a × b × c)
Productivity	0.3	Productive efficiency	1	Yield (quintal (57.5 kg)/hectare)	0.3	0.09
				Labor efficiency (quintal/people hired)	0.2	0.06
				Benefit-to-cost ratio (coffee income/coffee production cost)	0.5	0.15
Stability, reliability, and resilience	0.35	Resources availability	0.4	Soil conservation (producer performs soil conservation practices (yes or no))	0.3	0.04
				Shade-grown (percent of shade)	0.3	0.04
				Plantation age (average age)	0.3	0.04
				Wastewater treatment (producer performs water treatment (yes or no))	0.1	0.01
				Remnant of primary vegetation (primary vegetation area/producer’s land)	0.1	0.01
		Affectations by pests, disease, and natural phenomena	0.3	Production recovery after pest and disease affectations (production recovery after pest and disease affectations (yes or no))	0.8	0.08
				Production recovery after natural phenomena affectations (production recovery after natural phenomena affectations (yes or no))	0.2	0.02
		Market dynamism	0.3	Activity growth (perception of growth or stability)	0.1	0.01
				Change in prices (perception of growth or stability)	0.2	0.02
				Price (price received per quintal)	0.7	0.07
Adaptability	0.2	Experience	0.3	Experience (experience in coffee production (years))	0.6	0.04
				Family succession (family succession (parents, grandparents, etc.))	0.4	0.02
		Diversity options	0.7	Diversity of varieties (number of varieties used)	0.7	0.10
				Diversity of incomes (more than one source of income)	0.3	0.04
Self-management	0.05	Self-dependence	1	Dominant coffee income (coffee income ≥ 50%)	0.8	0.04
				Inputs production (producer produces his own inputs (yes or no))	0.2	0.01
Equity	0.1	Family involvement	1	Family decisions (decisions are taken by family members (yes or no))	0.3	0.03
		Access to land	1	Producer’s land (producer’s land area0	0.7	0.07

Source: authors’ own data.

shows the 9 critical points and 21 indicators that resulted from the discussion of the workshops carried out with the coffee producers.

From this data, eight indicators were selected, defining for each one both the critical threshold (Ct) and the sustainable threshold (St):

(1)

Yield: Ct = 8 quintals per hectare, St = 12 quintals per hectare;

(2)

Benefit-to-cost ratio: Ct = 2, St = 3;

(3)

Price: Ct = USD 136 per quintal, St = USD 170 per quintal;

(4)

Shade-grown: Ct = St = 40–60%;

(5)

Recovery from pests and diseases: recovered status;

(6)

Diversity of varieties: Ct = 2 varieties, St = 3 varieties;

(7)

Income from coffee activity: Ct = 50%, St = 75%;

(8)

Producer’s land: Ct = 1.2 hectares; St = 2 hectares.

Concerning the construction of the sustainability index, the paired comparison of indicators resulted in inconsistency indices below 0.1. In the case of critical points and systemic attributes, inconsistency indices of 0.09 and 0.08 were obtained, respectively.

The indicators with the highest weight (once the values were adjusted concerning the weights of critical points and systemic attributes) were the benefit-to-cost ratio, yield, recovery from pests and diseases, the price paid to the producer, varieties used, and producer’s land.

5.3. Classification of Producers

The classification of producers was realized through the association between the conceptual characterization of possible changes in the producers’ land and the map of changes in land use. We assumed that coffee production areas were located in secondary vegetation plots. Hence, the transition from agriculture or pastures to secondary vegetation was considered to be conversion to coffee land [76]. Furthermore, when traces of primary vegetation to secondary and agricultural vegetation or pastures to secondary vegetation were observed, and it was considered that there was an expansion-conversion process [77]. Finally, no changes were observed in the producers’ land.

In 43 out of 86 producer plots, no changes took place. As for the other half of the producers, 22 presented coffee expansion (25%), 11 converted their agricultural areas to coffee (13%), and 10 producers carried out expansion-conversion processes (12%).

5.4. Sustainability Measurement and Differences Analysis

The results of critical indicators reveal that the yield and benefit-to-cost ratio were the indicators in which a lower proportion of producers that exceeded the sustainability threshold was observed. Only 16% exceeded this threshold in performance and 29% in benefit-to-cost ratio. Likewise, an important proportion of producers was observed in the transition zone. However, the highest proportion was observed in those whose values resulted in being below the critical threshold. In the indicators of the price paid to the producer, shade of the coffee plantation, recovery from pests and diseases, and income from coffee growing, more than 70% of the producers exceeded the sustainable limits. For the indicators of diversity of varieties and producer’s land, it was observed that more than half of the producers surpassed the sustainable threshold. However, a significant proportion of producers was observed in the transition zone and below the critical threshold, with values of 24% and 21%, respectively, for both indicators.

Concerning the four processes of change, the highest average sustainability index was presented by coffee expansion producers, being about 56% (Figure 6).

The performance indices of the critical points for the four processes of agroecosystem change are shown in Figure 7. Among all indices, productive efficiency presented the worst performance in sustainability for all four types of changes in agroecosystems, since the average of all four classes was below 0.4 (coffee expansion (0.32), no change (0.22), conversion to coffee (0.2) and expansion-conversion (0.2)). In contrast, the index of availability of resources presented the highest scores for all four processes (coffee conversion (0.84), no change (0.74), coffee expansion (0.71), and expansion-conversion (0.69)). For the index of affectations by pests, diseases, and natural phenomena, the coffee expansion (0.82) and expansion-conversion classes (0.80) had high scores, whereas conversion to coffee performed poorly (0.55). The market dynamism index presented a high score for coffee expansion (0.88), followed by no change (0.79), expansion-conversion (0.69), and conversion to coffee (0.67). The average scores of the four classes of processes were similar for the experience in the activity index (between 0.58 for the case of expansion-conversion and up to 0.63 for conversion to coffee). For diversity options, the processes having no change, coffee expansion, and expansion-conversion showed similar averages, being 0.54, 0.51, and 0.5, respectively. However, the process of conversion to coffee displayed worse performance (0.31). The best average was recorded by the coffee conversion class for the self-dependence index, being 0.76. The coffee expansion and no change classes had similar averages, being 0.65 and 0.64, respectively. In the case of the expansion-conversion group, the average was 0.56. In family involvement, the best average corresponded to the coffee conversion class, being 0.9, while the other change processes presented values between 0.60 and 0.64. Finally, for the access to land index, the best performances were observed for the expansion-conversion and coffee expansion processes, both with a score of 0.34. The average observed in the coffee conversion class was 0.29, while the worst performance was observed for the no change class (0.15).

In performing a Kruskal–Wallis test, the indices of labor efficiency, benefit-to-cost ratio, price, remnant of primary vegetation, recovery of production from natural phenomena, family succession, and producer area were statistically significant at 5% (

Table 3. Kruskal–Wallis test for sustainability indices.

Metric	Yield	Labor Efficiency	Benefit-to-Cost Ratio	Soil Conservation	Shade Grown Percent	Plantation Age	Waste Water Treatment
Kruskal–Wallis H	6.585	12.170	9.699	2.206	4.624	1.450	2.024
Asymptotic signif.	0.086	0.007	0.021	0.531	0.201	0.694	0.567
Metric	Remnant of primary vegetation	Production recovery after pests and diseases afectations	Production recovery after natural phenomena afectations	Activity growth	Change in prices	Price	Experience
Kruskal–Wallis H	66.807	3.376	11.006	3.675	6.375	10.204	0.797
Asymptotic signif.	0.000	0.337	0.012	0.299	0.095	0.017	0.850
Metric	Family succession	Diversity of varieties	Sources of income	Dominant coffee income	Inputs production	Family decisions	Producer’s land
Kruskal–Wallis H	8.473	1.535	7.319	2.133	4.577	3.074	17.686
Asymptotic signif.	0.037	0.674	0.062	0.545	0.206	0.380	0.001

Source: authors’ own data.

). This result means that in these indicators, it was possible to establish differences between the study groups.

Likewise, the results from the Mann–Whitney test (

Table 4. Mann–Whitney tests.

Relation	Metric	Labor Efficiency	Benefit-to-Cost Ratio	Price	Remnant of Primary Vegetation	Production Recovery after Natural Phenomena Afectations	Family Decisions	Producer’s Land
Coffee expansion: conversion to coffee	Mann–Whitney U	33.00	56.50	61.50	62.00	66.00	110.00	101.00
	Asymptotic sig	0.004	0.014	0.025	0.024	0.001	0.310	0.445
Coffe expansion: expansion and conversion	Mann–Whitney U	29.00	61.00	48.00	108.00	88.00	76.00	101.00
	Asymptotic sig	0.123	0.046	0.012	0.935	0.118	0.041	0.714
Coffe expansion: no change	Mann–Whitney U	235.50	291.50	429.50	0.00	363.00	461.00	238.00
	Asymptotic sig	0.105	0.012	0.744	0.000	0.020	0.757	0.001
Conversion to coffee: expansion and conversion	Mann–Whitney U	31.00	47.00	45.50	28.00	32.50	33.00	39.00
	Asymptotic sig	0.840	0.573	0.478	0.056	0.104	0.023	0.260
Conversion to coffee: no change	Mann–Whitney U	93.50	184.00	149.50	107.50	175.50	209.00	147.00
	Asymptotic sig	0.003	0.260	0.055	0.000	0.111	0.240	0.055
Expansion and conversion: no change	Mann–Whitney U	95.00	192.00	114.50	0.00	169.50	154.00	70.00
	Asymptotic sig	0.372	0.610	0.019	0.000	0.484	0.033	0.001

Source: authors’ own data.

) show that for the groups of producers with coffee expansion and conversion to coffee, statistical differences were observed in the following indices: labor productivity, cost-to-benefit ratio, price, remnant of primary vegetation, and recovery from natural phenomena. For the coffee expansion and expansion-conversion groups, the differences were observed in the following indices: cost-to-benefit ratio, price, and family succession. In the coffee expansion and no change groups, the cost-to-benefit ratio, vegetation remnant, recovery from natural phenomena, and producer’s land indicators also showed statistical differences. In the case of the conversion to coffee and expansion-conversion groups, only the index of family succession showed a difference. In contrast, for the conversion to coffee and no change groups, a statistical difference was observed for the efficiency of labor and the remnant of primary vegetation indices. Finally, in the expansion-conversion and no change groups, the indices that showed statistical differences were price, remnant of primary vegetation, family succession, and producer’s land.

6. Discussion

The co-production of knowledge is a paramount feature of sustainability [78]. Hence, conceptual characterization was one of the most important steps in this study, because concepts, relations between actors, components of management systems, and changes in the agroecosystem were integrated. First, the definition of sustainability provided by producers implies that to maintain the system over time, it is necessary to sustain, change, or transform some of its components [10]. In this case, maintaining organic coffee production requires improving the performance of the system through management practices. Second, the advantages of cooperatives as an organizational model and organic certification were explained by local actors. In fact, certain types of certifications of coffee production can be a means of achieving local sustainability, as has been observed in Mexico and elsewhere [79,80]. Third, description of management system was useful for defining critical points and understanding possible changes that occur in producers’ land. These changes might represent different types of adaptation for coping with both economic and institutional drivers, as explained by Speelman et al. [81].

The critical points and sustainability indicators defined by local and academic knowledge were the result of a set of participative and inclusive processes. It is important to note that indicators where participatory assessments were made, such as ours, have a more robust sustainability approach, as shown by Vaidya and Mayer [82] in analyzing 13 case studies around the world. The indicators used in this study—yield, prices, diversity of varieties, affectations by pest and diseases, primary vegetation, shade-grown, source of income, and family involvement among others—have been used in different research to assess coffee production systems [32,83,84]. The indicator weighting process was developed through the analytic hierarchy process, which is one of the most often applied multi-criteria decision methods [85].

The classification that producers produced allows for including changes that occur in the agroecosystem within the assessment. As described, this is a contribution of this study, and thus no similar works have been identified in the other sustainability assessment works reviewed.

The values of the sustainability index show an interesting result, since the coffee expansion class presented higher scores than the rest. According to the values obtained for each critical point, these results were associated with the consolidation of the activity, since there was higher productive efficiency, more ecosystem benefits for control and recovery of pests, diseases, or natural phenomena, and better prices paid mainly to the producer. Authors such as Folch and Planas [86] and Vandermeer et al. [87] described how organic producers can achieve sustained yields and cost reduction from the benefits of the interaction between agroecosystem components, such as pest control, creation of moisture niches, and restoration of soil nutrients. In fact, the certification of coffee exports is an incentive for maintaining and expanding organic production, providing both environmental and social benefits in general, as demonstrated in Brazil, Costa Rica, Ethiopia, and Vietnam [88,89,90]. On the other hand, in the classes of reconversion to coffee and reconversion-expansion, the results with lower sustainability values were mainly for lower productive efficiencies, more affectations by pests, diseases, or natural phenomena, lower prices, and fewer diversification options. These results may be associated with the production costs derived from the transition to organic coffee, the prices paid to producers in transition, and the time it takes for the ecosystem’s recovery to develop functions that allow achieving higher yields and better environmental quality [21,91,92,93].

The results of the differences analysis using Kruskal–Wallis and Mann–Whitney tests allowed for identifying the statistical differences between each class. These results are relevant because MESMIS is an action-oriented framework, and this is a contribution that provides differentiated recommendations to cooperatives and their producers to move toward sustainability.

Finally, the study’s main limitation is that only the results of the first assessment cycle are presented, which does not allow comparisons to be made with the results of longitudinal studies (i.e., several stages). Another limitation is that the results presented are not differentiated by sex and age group. Furthermore, the method incorporates the changes within the agroecosystem from the first sustainability assessment cycle using the producers’ land polygons. The difficulty of obtaining such data is an important limitation. In our case, producers in the study area should calculate the land polygons as a requirement for obtaining the organic certification [94], and we had access to such data. We are aware that in other agroecosystems, this may not be the case.

7. Conclusions

One of the main objectives of this research was to make an innovative proposal that involved changes in organic coffee agroecosystems in the MESMIS evaluation framework and a greater insertion of participatory processes in each of the different stages. Therefore, the contributions of this article are multiple.

First, we co-constructed a conceptual characterization of sustainability with the stakeholders of the three coffee cooperatives to identify different interactions between the components of the production systems and the development of indicators and critical points of the activity. We showed that the co-production of knowledge between producers and the academic sector allowed the researcher to help build the framework and the research path. As a result of this effort, the integrated MESMIS index with producers’ knowledge and time-changing trajectories proved helpful in evaluating the sustainability of organic coffee plantations in Chiapas, Mexico.

Second, this methodological approach could be applied to assess the sustainability of other agroecosystems in the Latin American context. The synergy of qualitative and quantitative methods guided by local knowledge creates a collective capacity to evaluate the application of knowledge with understandable and manageable attributions between the various spheres of actors and action.

Third, diversified and consolidated coffee plantations proved to be more sustainable, even when expanding their surface area, compared with coffee plantations converted from agricultural uses. The REBITRI organizations are mainly in the process of productive diversification where the production of honey, fruit trees, and vegetables plays an important role, which allows access to different crops that contribute to their food security and some, such as honey, to family income.

Fourth, it seems that the current environmental and agricultural policy applied by both local and federal authorities has proven useful in maintaining productive levels and preserving the agroecosystems in the REBITRI, although it should be noted that the organizations are constantly strengthening their technical capacities around the management of soil fertility (vermicompost), management of soil coverage, the inclusion of young people, the revaluation of the work of coffee-growing women, and financial strengthening through credit and savings schemes for producers affiliated with the organizations.