ELECTRE-TRI Multicriteria Approach for Measuring Performance of Rural Co-Operatives in Southwest Paraná, Brazil ^†

Abstract

The maintenance of a satisfactory quality of life in rural areas is fundamental for sustainable development. One of the ways to improve quality of life indicators is through the gathering of rural workers in solidarity economy organizations as these enterprises aim to integrate development with economic, social, and environmental sustainability. Because solidarity economy organizations have a robust social purpose, their performance must be evaluated based on both social and financial indicators. The objective of this article is to propose a performance evaluation model for rural solidarity economy enterprises, aiming to support decision making in these enterprises based on multicriteria decision analysis (MCDA), particularly the ELECTRE-TRI methodology. In order to demonstrate the applicability of the developed model and to perform sensitivity analyses, the model was applied to a group of eight family agriculture co-operatives in the southwest state of Paraná, Brazil. All the participating co-ops were considered part of the solidarity economy, and they served 2500 rural producer families across at least 15 municipalities. The results showed the applicability and stability of the model, enabling us to identify the dimensions in which each co-op should concentrate their efforts to improve not only their performance but also the outcomes for the farmers that they serve. Based on these results, organizational and improvement activities can be developed and implemented. This analysis contributes to economic and social indicators by offering improvement strategies for the professionalization and strategic management of RSEEs, thus strengthening these enterprises and, consequently, family agriculture.

1. Introduction

According to data from 2023, the prevalence of moderate or severe food insecurity was 31.9 percent in rural areas compared with 29.9 percent in periurban areas and 25.5 percent in urban areas. Globally and in all regions except Northern America and Europe, the prevalence of food insecurity, at both levels of severity, is consistently higher in rural areas than in urban areas (FAO et al. 2024). In this context, one of the ways to end poverty and food insecurity includes the development of agricultural and rural livelihood through actions/movements that aim to improve the quality of life of those living in these regions, for example, through solidarity economy enterprises. The historical roots of the solidarity economy are in the mid-nineteenth century Europe, where co-operative movements developed in response to social issues resulting from the Industrial Revolution. The foundations of the solidarity economy are solidarity, the equal sharing of profits, and self-management (Singer 2002).

In the European and Latin American contexts, the concepts of the solidarity economy have developed in different ways. Da Ros (2007) notes that the term “solidarity economy” was forged in Latin America in the early 1980s, using a markedly political discourse and playing a relevant role in the struggle against the social problems intrinsic to the current economic systems (Guerra 2004). In Europe, the emergence of the solidarity economy dates to the late 1980s and developed both at the theoretical–academic and practical levels through insertion and proximity services (Da Ros 2007).

In Brazil, the first solidarity economy initiatives occurred after the economic and social crisis of the 1970s (Ribeiro and De Müylder 2014). However, it was not until 2000, with the election of a leftist president, that effective public policies aimed at supporting solidarity economy organizations became a priority for generating income and encouraging social integration (Gaiger 2013). Many of the guidelines outlined in the Declaration of the Brazilian Solidarity Economy originated from debates that occurred in the Brazilian Solidarity Economy Forum (FBES 2003). In 2003, the National Secretariat of Solidarity Economy (SENAES) was created by the left-wing government, while the National Program for Incubators of Popular Co-operatives (Proninc) was reinstated (Singer 2009). In 2013, through Decree No. 8163, the National Program for Supporting Social Associations and Co-operatives (PronaCPSocial) was founded with the purpose of “planning, coordinating, executing and monitoring actions aimed at the development of social co-operatives and social solidarity economic enterprises” (Brasil 2013b).

Co-operatives, the group addressed in this study, occupy a crucial position within numerous sectors and stand out as an important group of solidarity organizations, contributing to food security and poverty reduction in different areas around the world (Sánchez-Hernández and Castilla-Polo 2021). However, there are other types of organizations within the solidarity economy, such as mutual societies, associations, and informal groups, that follow the principles of solidarity, democracy, and self-management, with a focus on human life and the dignity of the worker (Monzon and Chaves 2008; Telles et al. 2020; Castilla-Polo and Sánchez-Hernández 2020; Sánchez-Hernández and Castilla-Polo 2021).

According to data from the national survey of SEEs carried out by the National Secretariat for Solidarity Economy (SENAES; Brasil 2013c), there were 19,708 solidarity economy organizations in Brazil in 2013, of which almost 55% (10,793) functioned exclusively in rural areas. Although these numbers are slightly lower when compared to those from 2007 (Brasil 2007) when the number of enterprises was 21,859, there has been significant growth in relation to the numbers in the 1970s, when only 139 SEEs existed in Brazil (Ribeiro and De Müylder 2014).

During the 2nd National Conference on Sustainable Rural Development and Solidarity in 2013, policies were established to stimulate the development of mechanisms that could help improve management on family farms and ensure partnerships with other farmers, including co-operatives and participation in the solidarity economy, as these mechanisms are extremely relevant for strengthening family agriculture and consequently for sustainable rural development (Brasil 2013a). In this context, Christoffoli et al. (2013) highlighted the importance of SEEs for the maintenance of family farming and as a means to support sustainable rural development; the alternatives of self-management and collective thinking offer possibilities for more economically, socially, and environmentally sustainable rural and social organization and production. Subsequently, Martínez-Campillo and Fernández-Santos (2017) argued that the growing demand for these organizations can trigger social change, yet there was a need to use indicators that measure the impacts of this change.

Other studies also showed the positive effects of SEEs on sustainable development and in combating social exclusion, demonstrating outcomes such as social innovation, natural resource preservation, economic autonomy, local development, and social transformation (França Filho 2008; Singer 2014; Martino et al. 2016; Rover et al. 2017). Additionally, Arruda et al. (2015) used a case study to highlight the relevance of solidarity economy ventures for sustainable rural development. The authors suggested that development must be evaluated through advances in human quality of life, including economic, social, environmental, political and cultural aspects.

According to the Olmedo and O’Shaughnessy (2022) study, rural-community-based social enterprises play a relevant role in contributing to rural development. The same authors stated that rural social enterprises contribute to multidimensional development (social, economic, and environmental), combining the community characteristics and local focus with the emphasis on the development of external relationships, with social entrepreneurship as well as innovative, locally focused solutions, suggesting that the way in which rural social enterprises work aligns with the principles of an endogenous rural development approach.

Thus, the activities of rural solidarity economy enterprises (RSEEs) have a concrete impact on important social indicators, especially in terms of food security, access to resources, and quality of life, not only for the associated farmers but also for consumers and the surrounding community. Thus, it is important to evaluate the performance of an enterprise by measuring the social transformation generated through their organizational activities (Stevens et al. 2015).

Because of their social role, the decision-making processes in SEEs encompass multiple points of view and a wide range of criteria. As such, decision-making tools and/or instruments must incorporate the multiple aspects presented in the decision-making process, which leads to the use of the multicriteria decision aid methodology. Lee and Nowell (2015) noted the complexity of evaluating performance in the nonprofit sector, as the outcomes are often difficult to measure since many are not quantitative. For this reason, Grieco et al. (2015) argued that the concept of performance evaluation in social organizations, including charitable enterprises, is a major challenge for scholars, mainly because of the difficulty of converting qualitative data related to the social mission into quantitative metrics.

In this context, Ebrahim and Rangan (2014) offered three different categories to evaluate the results of social enterprises: outputs (immediate results); outcomes (medium- and long-term results); and impacts (impacts on the roots of social problems or social transformation). However, it is impractical for some organizations to assess long-term results and social impacts, particularly in poorly structured enterprises or those in the early stages of development. According to Ebrahim and Rangan (2014), what is most important is developing metrics and structured measurement systems based on the organization’s mission and objectives.

Although a number of other authors have applied different methodologies to assess the performance of social organizations (Moxham 2009; Leroux and Wright 2010; Meadows and Pike 2010; Bagnoli and Megali 2011; Cançado et al. 2013; Ebrahim and Rangan 2014; Crucke and Decramer 2016; Staessens et al. 2019), performance evaluation that prioritizes the principles of the solidarity economy using the multicriteria decision analysis methodology in the management model is a novel approach in the solidarity economy. The multicriteria decision analysis (MCDA) methodology should not be considered an absolute solution but rather a tool that offers recommendations for decision makers and allows learning from the problem in question (Roy et al. 2014). According the description by Belton and Stewart (2002), MCDA is a collection of formal approaches that take into account multiple criteria to aid in individual or group decision making.

Therefore, the present study developed and applied a new instrument for the performance evaluation of rural solidarity economy enterprises (RSEEs) through the use of MCDA methodology, which can subsequently be used to support management in RSEEs. Thus, this study is novel in that it developed a new performance evaluation instrument that is applicable to solidarity economy enterprises, while also contributing to the literature on the subject, as there are few studies that have addressed the performance evaluation of SEEs. The proposed approach aims to support decision making in these enterprises based on the MCDA methodology, in particular using ELECTRE-TRI. The methodology was applied to solidarity economy co-operatives (co-ops) in the southwest state of Paraná, Brazil, to assess to the applicability of the model and conduct a sensitivity analysis. This paper is structured as follows: Section 2 discusses the ELECTRE-TRI method and presents the data collection methodology; Section 3 describes the proposed model together with the results; Section 4 discusses the application of the model; and Section 5 draws conclusions and suggestions for future research.

2. Research Methodology

2.1. ELECTRE-TRI Method

In multicriteria modeling, the type of problem generally points to the type of solution that is expected for the decision problem, making it possible to identify the desired results of the decision problem (Roy 1996). Multicriteria decision support modeling seeks to provide decision makers with elements to respond to questions arising from a process, with the aim of clarifying each decision, making the decision-making process as neutral, objective, valid, and transparent as possible, without the intention of a single, true solution (Gomes et al. 2011). In the decision support process, it is usually one of the stakeholders that is being supported. The various stakeholders involved in the process can be diverse, having different objectives and conflicting value systems. Therefore, a specific decision support application will rarely be comprehensive enough to benefit all of them. Thus, multicriteria modeling requires the participation of a decision maker, a specific stakeholder (individual, entity or community) (Roy 1996).

After the decision problem for this study was determined, the problem of classification (Pβ) was identified. This assists the decision maker in a process of classification that leads to the attribution of each action to a category, where the categories are previously defined as a function of certain norms according to the final destination of the actions. As such, the result of this type of problem is the process of classification or attribution.

According to the classification of methods for solving multicriteria decision problems proposed by Roy (1996), the problem in this study was the outranking synthesis approach, which is applied when the criteria are not compensatory and allows incomparability between alternatives (ELECTRE) and/or preference ranking organization method for enrichment evaluations (PROMETHEE).

Therefore, for the modeling, we chose the ELECTRE family of the French School, specifically the ELECTRE-TRI method, which also includes relationships of incomparability (R) between criteria. The ELECTRE-TRI method was proposed by Almeida-Dias et al. (2010) and belongs to “The Outranking Synthesis Approach”; according to Pereira and Mota (2016) this method is recommended for situations in which alternatives can be assigned to predefined classes through the evaluation of multiple criteria. With this method, the outranking relationships of alternatives and reference actions are explored; that is, the alternatives are allocated to classes through a comparison with reference actions.

Thus, in this study, the reference actions were interpreted as the performance expected for each performance level (class). According to Roy (1991), to establish the outranking relationships between alternative a and reference alternative b, one should initially obtain the following indices: concordance by criteria $c_i(a,b)$ and $c_i(b,a)$; global concordance of criteria $G(a,b)$ and $G(b,a)$; discordance by criteria $h_i(a,b)$ and $h_i(b,a)$; global discordance of criteria $H(a,b)$ and $H(b,a)$; and credibility $\sigma s(a,b)$. The global concordance indices $G(a,b)$ and $G(b,a)$, indicating “$a$ outranks $b$” for $G(a,b)$ and “$b$ outranks $a$” for $G(b,a)$, are estimated from the indices of concordance of each criterion, while the global indices of discordance $H\left(a,b\right)$ and $H(b,a)$ are obtained from the indices of discordance of each criterion (Figueira et al. 2005). The indices’ concordance by criteria $c_i(a,b)$ and $c_i(b,a)$, global concordance $G(a,b)$ and $G(b,a)$, and discordance by criteria $h_i(a,b)$ and $h_i\left(b,a\right)$ are calculated by Equations (1)–(3), respectively.

$$c_i\left(a,b\right)=\left\{\begin{array}{cc}\hfill 0 \mathrm{i}\mathrm{f}\hfill & t_i\left(a\right)\le t_i\left(b\right)-p_i\hfill \\ \hfill 1 \mathrm{i}\mathrm{f}\hfill & t_i\left(a\right)>t_i\left(b\right)-q_i\hfill \\ \hfill 0<c_i\left(a,b\right)\le 1\hfill & t_i\left(b\right)-p_i<t_i\left(a\right)\le t_i\left(b\right)-q_i\hfill \\ & \hfill {\displaystyle \frac{p_i-\left[t_i\left(a\right)-t_i\left(b\right)\right]}{p_i-q_i}}\end{array}\right.$$

(1)

$$G\left(a,b\right)={\displaystyle \frac{{\sum }_{i=1}^n{w}_i{c}_i(a,b)}{{\sum }_{i=1}^n{w}_i}}$$

(2)

$$h_i\left(a,b\right)=\left\{\begin{array}{cc}0 \mathrm{i}\mathrm{f}& t_i\left(a\right)>t_i\left(b\right)-p_i\hfill \\ 1 \mathrm{i}\mathrm{f}& t_i\left(a\right)<t_i\left(b\right)-v_i\hfill \\ 0<h_i\left(a,b\right)\le 1& t_i\left(b\right)-v_i<t_i\left(a\right)\le t_i\left(b\right)-p_i\hfill \\ & \hfill {\displaystyle \frac{\left[t_i\left(b\right)-t_i\left(a\right)\right]-p_i}{v_i-p_i}}\end{array}\right.$$

(3)

where

• $t$: criterion;
• $w$: weight;
• $p$: preference limit;
• $q$: indifference limit;
• $v$: veto limit.

ELECTRE-TRI constructs an index $\sigma \left(a,b\right)$ and $\sigma (b,a)$ that represents the degree of credibility of the assertion that $aSb$. In order to demonstrate how alternative $a$ outranks the reference alternative $b$, considering the indices of concordance $c_i(a,b)$ and discordance $h_i(a,b)$, we determine the credibility index, represented by $\sigma \left(a,b\right),$ obtained according to Equation (4).

$$\sigma \left(a,b\right)=c_i\left(a,b\right)·\pi \left[{\displaystyle \frac{1-h_i(a,b)}{1-g_i(a,b)}}\right]$$

(4)

The assumption that $aSb$ is considered valid if $\sigma (a,b)\ge \lambda$, where $\lambda =$ the cut-off level, such that $\lambda \in \left[0.5;1\right]$. According to Figueira et al. (2005), the cut-off level is the lowest value that the credibility index can assume to assert that $aSb$, and its preference relationship is obtained through comparison. Thus, the greater the value of λ, the more severe the outranking conditions of an alternative to the boundaries. The degree of credibility is the minimum acceptable value for the strength of the assertion that outranks a0, taking into consideration all criteria of the problem. For example, when a degree of credibility of 0.6 is chosen, at least 60% of the votes are needed to legitimize the outranking relationship between the alternative and the reference action (Pereira and Mota 2016). The calculation procedure for $\sigma (a,b)$ and $\sigma (b,a)$ must be repeated for each reference alternative. The number of the preference relationships between $a$ and $b$ corresponds to the number of reference alternatives of set $A$.

2.2. Application of the Model

To apply this model to the eight studied co-ops in Southwest Paraná, we developed a questionnaire (Supplementary Materials, Document S1) that addressed the research objectives and underwent a rigorous validation process. The eight co-ops were associated with a central family farming association that coordinated a network of activities among its members. They supported more than 2500 families across 15 municipalities in the southwest region and 3 in the western region of Paraná, southern Brazil. The central co-operative was initially established in 2007 by four individual family agriculture co-ops, and the group currently consisted of nine individual family agriculture co-ops. One co-op decided not to participate in the current study. We chose this group of co-ops among those included in the solidarity economy search engine due to the fact that they were organized through a central association and had similar characteristics, offering the possibility of comparing their performance. The research instrument was structured into an introduction and three sections, with a total of 72 questions:

• Section I—Characterization of the enterprise: In this section, the participating RSEE and its representative were identified, and data collected from it included activities and foundation.
• Section II—Weight of the selected criteria: In this section, the weight (importance level) of each criterion selected to assess performance in terms of the principles of the solidarity economy was identified based on the respondent’s perceptions and judgment. Using a Likert scale, the participants assigned a degree of importance to each criterion, adopting weights from 1 to 5, where 1 corresponded to the least relevant (1—not important), and 5 was the most relevant (5—extremely important).
• Section III—Performance on the selected criteria: In this section, the performance of each enterprise was evaluated.

The questionnaire was completed in March and April of 2018 by the president of each enterprise. The data were directly incorporated into Interactive Robustness Analysis and Parameters’ Inference for Multicriteria Sorting Problems (IRIS) version 2.0 Demo software (version available for testing and academic studies at www.lamsade.dauphine.fr/~mayag/links.html, accessed on 6 February 2018) (Dias and Mousseau 2002). This version did not include statistical treatments.

The set of alternatives was designated as A = {a₁, a₂, a₃, a₄, a₅, a₆, a₇, a₈}, representing the eight individual family agriculture co-ops that agreed to participate in this study. Based on previous research and the principles of the solidarity economy, 36 criteria were selected, divided into six dimensions (

Table 1. Criteria selected for evaluation in the ELECTRE-TRI method.

Dimensions	Criteria		Wa *	Wn *
Democratic management and legalization	k1	Employees as associates	4.38	0.165
	k2	Transparency	4.25	0.160
	k3	Collective decision making	4.50	0.170
	k4	Legal records and documents	4.75	0.179
	k5	Renewal of council membership	4.38	0.165
	k6	Participatory internal planning	4.25	0.160
Valuing human labor	k7	Training	4.00	0.182
	k8	Prevention of occupational accidents	3.50	0.159
	k9	General salary gap	3.63	0.165
	k10	Emergency leave for family	3.63	0.165
	k11	Training of associates (development of rural activities)	3.75	0.170
	k12	Culture and leisure	3.50	0.159
Technology and economic viability	k13	Source of revenue	4.00	0.174
	k14	Allocation of profits	4.00	0.174
	k15	Management reports	3.63	0.158
	k16	Training of associates (rural management)	4.00	0.174
	k17	Debt negotiation policy	3.13	0.136
	k18	Decapitalization	4.25	0.185
Commitment to minorities	k19	Gender equity	4.13	0.166
	k20	Salary gap by gender	4.00	0.161
	k21	Diversity	4.13	0.166
	k22	Gender equity on councils	4.50	0.181
	k23	Gender equity of associates	4.13	0.166
	k24	Combating prejudice	4.00	0.161
Environmental sustainability	k25	Environmental sustainability	4.13	0.163
	k26	Energy efficiency	4.13	0.163
	k27	Recycling and/or reuse of products	4.00	0.158
	k28	Organic/agroecological production	4.38	0.172
	k29	Promotion of organic/agroecological production	4.38	0.172
	k30	Protection of soil and water	4.38	0.172
Co-operation and solidarity	k31	Inter-co-operation	4.25	0.171
	k32	Financial institutions employed	4.00	0.161
	k33	Local initiatives	3.50	0.141
	k34	Solidarity economy, co-operation, association, and self-management	4.13	0.166
	k35	Consumer welfare	4.25	0.171
	k36	Promotion of family agriculture and solidarity economy	4.75	0.191

* Wa: assigned weight; Wn: normalized weight.

). All 36 criteria and the responses to evaluate each were presented in the research questionnaire. In questions 1 to 36, the criteria were presented, while in questions 37 to 72, the evaluation responses for each criterion were presented. The weights assigned to the criteria reflected the relative importance of each within the studied context. Equation (5) presents the formula used to calculate the normalized weight, where Wa = attributed weight, and Wn = normalized weight.

$$Wn={\displaystyle \frac{Wa}{\sum Wa}}$$

(5)

Table 1 presents the weights assigned by the decision makers (respondents) for each criterion, as well as the normalized weights.

We used the average to quantify the weights of each criterion as the result was not based solely on one decision maker but a set of decision makers (or a consensus among those involved in the analysis). Thus, the quantified result sought to ensure that the attributed weights reflected the context of all those surveyed. After identifying the criteria and assigning their respective weights, the performance classes for assessing the co-ops were defined. In the ELECTRE-TRI method, classes (or categories) are sorted from worst to best. Therefore, three performance classes $C$ = {$C_1$,$C_2$,$C_3$}, referring to poor, average and good, were established in this study.

Based on the answers to the questionnaire of the decision makers, an evaluation matrix of the alternatives in each criterion was constructed, showing the performance of each co-op in relation to the defined decision criteria. After the standardization of the evaluation matrices, the reference boundaries for each of the defined performance classes were established. These reference boundaries were represented by ($b$) and distinguished two consecutive performance classes that the analyst and the decision makers considered necessary for the distribution of the alternatives.

Table 2. Limits of the reference boundaries for the performance classes of defined criteria.

Classes (C)	Reference Boundaries (b)	Reference Values (b) for Each Criterion (k)
Democratic management and legalization
-	-	$k_1$	$k_2$	$k_3$	$k_4$	$k_5$	$k_6$
$C_1$	-	-	-	-	-	-	-
$C_2$	$b_1$	1.5	1.5	1.5	1.5	1.5	1.5
$C_3$	$b_2$	3.5	3.5	3.5	2.5	2.5	3.5
Valuing human labor
-	-	$k_7$	$k_8$	$k_9$	$k_{10}$	$k_{11}$	$k_{12}$
$C_1$	-	-	-	-	-	-	-
$C_2$	$b_1$	1.5	1.5	1.5	1.5	1.5	1.5
$C_3$	$b_2$	2.5	2.5	3.5	2.5	2.5	2.5
Technology and economic viability
-	-	$k_{13}$	$k_{14}$	$k_{15}$	$k_{16}$	$k_{17}$	$k_{18}$
$C_1$	-	-	-	-	-	-	-
$C_2$	$b_1$	1.5	1.5	1.5	1.5	1.5	1.5
$C_3$	$b_2$	3.5	2.5	3.5	2.5	2.5	2.5
Commitment to minorities
-	-	$k_{19}$	$k_{20}$	$k_{21}$	$k_{22}$	$k_{23}$	$k_{24}$
$C_1$	-	-	-	-	-	-	-
$C_2$	$b_1$	1.5	1.5	1.5	1.5	1.5	1.5
$C_3$	$b_2$	3.5	3.5	3.5	3.5	3.5	2.5
Environmental sustainability
-	-	$k_{25}$	$k_{26}$	$k_{27}$	$k_{28}$	$k_{29}$	$k_{30}$
$C_1$	-	-	-	-	-	-	-
$C_2$	$b_1$	1.5	1.5	1.5	1.5	1.5	1.5
$C_3$	$b_2$	3.5	2.5	3.5	3.5	2.5	2.5
Co-operation and solidarity
-	-	$k_{31}$	$k_{32}$	$k_{33}$	$k_{34}$	$k_{35}$	$k_{36}$
$C_1$	-	-	-	-	-	-	-
$C_2$	$b_1$	1.5	1.5	1.5	1.5	1.5	1.5
$C_3$	$b_2$	2.5	3.5	2.5	2.5	2.5	2.5

presents the reference boundaries that defined the three established performance classes.

Firstly, the number of alternatives ($A$ = 8), criteria ($K$ = 36), and performance classes ($C$ = 3) were entered into the software. The same process was performed six times, once for each dimension. This made it possible to assess the performance of the co-ops in each of the dimensions. The limits of the reference boundaries between performance classes were defined by decision makers and by the analyst and were necessary so that each alternative was securely allocated to one of the performance classes defined in the model. The indifference ($q$) and preference ($p$) thresholds were considered zero, allowing true criteria for this context, considering the difficulty experienced by decision makers in quantifying their preferences and qualitatively understanding the evaluation of the classes.

The veto limit ($v$) was not used, since all alternatives satisfied the statement aSb (alternative “a” is as good as alternative “b”). To define the limits of the criteria, the degree of importance of the weight that each criterion exerted on the problem was considered, based on the decision makers’ (respondents’) perception. For modeling, the cut-off threshold $\lambda =0.6$ was used for the degree of credibility. We chose this threshold to achieve a result with low exigency, since it was the first time that the co-ops had conducted a performance evaluation. Subsequently, the sensitivity of the model to the credibility index ($\lambda$) and the weight of the criteria were evaluated in order to assess the stability of the model.

3. Results

The results of applying the model classified the participating co-ops into three levels of performance based on the principles of the solidarity economy. The three levels were defined by the decision makers and the analyst to evaluate participants’ performance in the six dimensions, based on the principles of solidarity economy), as well as the limits of the reference boundaries between the levels. This was essential so that each alternative could be safely allocated to one of the levels defined in the model. To define the limits for the criteria, the degree of importance of the weight that each criterion exerted on the problem for the decision maker was considered. In Figure 1, the dark green indicates the allocation of the alternative to a certain category, and the light green indicates its possible reallocation into another category.

From Figure 1, we can see that for the dimension “Democratic management and legalization”, the majority of co-ops presented average performance ($C_2)$, and only co-op 4 was evaluated as good ($C_3).$ In “Valuing human labor”, no co-op achieved good performance ($C_3)$, two co-ops (3 and 6) performed poorly ($C_1)$, and the other co-ops presented average results ($C_2$). In “Technological and economic feasibility”, three co-ops (1, 4, and 8) presented a good result ($C_3$), while four (3, 5, 6, and 7) had a poor result ($C_1$). For “Commitment to minorities”, the performance was the poorest among all co-ops, since five (2, 3, 5, 6, and 7) performed poorly ($C_1$), and the others only achieved an average result ($C_2$). The results for “Environmental sustainability” also showed that no participating co-op achieved good performance: 50% were average ($C_2$), and 50% were poor ($C_1$) in this dimension. The best results were obtained in the “Co-operation and solidarity” dimension, since four co-ops (1, 2, 4 and 8) achieved good performance ($C_3$) and only one (3) showed poor performance ($C_1$).

Subsequently, we conducted a sensitivity analysis of the model in relation to the credibility index ($\lambda$), which refers to the minimum value of $\sigma s(a,b)$, to validate the outranking relationship among the alternatives. For this, $\lambda$ was given a value between 0.5 and 1. Thus, sensitivity analysis was performed for the degree of credibility by adopting $\lambda$ = 0.6, $\lambda$ = 0.7, $\lambda$ = 0.8, and $\lambda$ = 0.9. The thresholds of 0.5 and 1 were excluded because they were considered extremes, i.e., 0.5 was a very low requirement, and 1 was very high.

Table 3. Distribution of degree of credibility for participating co-ops.

Democratic Management and Legislation
Class	$\mathit{\lambda }$ = 0.6	$\mathit{\lambda }$ = 0.7	$\mathit{\lambda }$ = 0.8	$\mathit{\lambda }$ = 0.9
Poor ($C_1$)	CP7	CP1, CP3, CP5, CP7	CP1, CP3, CP5, CP7	CP1, CP2, CP3, CP5, CP6, CP7
Average ($C_2$)	CP1, CP2, CP3, CP5, CP6, CP8	CP2, CP4, CP6, CP8	CP2, CP4, CP6, CP8	CP4, CP8
Good ($C_3$)	CP4
Valuing human labor
Class	$\lambda$ = 0.6	$\lambda$ = 0.7	$\lambda$ = 0.8	$\lambda$ = 0.9
Poor ($C_1$)	CP3, CP6	CP1, CP3, CP5, CP6, CP7	CP1, CP3, CP5, CP6, CP7	CP1, CP2, CP3, CP4, CP5, CP6, CP7
Average ($C_2$)	CP1, CP2, CP4, CP5, CP7, CP8	CP2, CP4, CP8	CP2, CP4, CP8	CP8
Good ($C_3$)
Technological and economic viability
Class	$\lambda$ = 0.6	$\lambda$ = 0.7	$\lambda$ = 0.8	$\lambda$ = 0.9
Poor ($C_1$)	CP3, CP5, CP6, CP7	CP3, CP5, CP6, CP7	CP3, CP5, CP6, CP7	CP1, CP3, CP5, CP6, CP7
Average ($C_2$)	CP1	CP1, CP4	CP1, CP4	CP2, CP4, CP8
Good ($C_3$)	CP2, CP4, CP8	CP2, CP8	CP2, CP8
Commitment to minorities
Class	$\lambda$ = 0.6	$\lambda$ = 0.7	$\lambda$ = 0.8	$\lambda$ = 0.9
Poor ($C_1$)	CP2, CP3, CP5, CP6, CP7	CP1, CP2, CP3, CP4, CP5, CP6, CP7, CP8	CP1, CP2, CP3, CP4, CP5, CP6, CP7, CP8	CP1, CP2, CP3, CP4, CP5, CP6, CP7, CP8
Average ($C_2$)	CP1, CP4, CP8
Good ($C_3$)
Environmental Sustainability
Class	$\lambda$ = 0.6	$\lambda$ = 0.7	$\lambda$ = 0.8	$\lambda$ = 0.9
Poor ($C_1$)	CP3, CP4, CP6, CP7	CP1, CP3, CP4, CP5, CP6, CP7, CP8	CP1, CP3, CP4, CP5, CP6, CP7, CP8	CP1, CP2, CP3, CP4, CP5, CP6, CP7, CP8
Average ($C_2$)	CP1, CP2, CP5, CP8	CP2	CP2
Good ($C_3$)
Co-operation and solidarity
Class	$\lambda$ = 0.6	$\lambda$ = 0.7	$\lambda$ = 0.8	$\lambda$ = 0.9
Poor ($C_1$)	CP3	CP3	CP3	CP3, CP5, CP6
Average ($C_2$)	CP5, CP6, CP7	CP5, CP6, CP7	CP5, CP6, CP7	CP1, CP4, CP7, CP8
Good ($C_3$)	CP1, CP2, CP4, CP8	CP1, CP2, CP4, CP8	CP1, CP2, CP4, CP8	CP2

shows the distribution of co-operatives in the performance classes after the sensitivity analysis for each degree of credibility.

From Table 3, we can see that changes in the parameters based on the degree of credibility ($\lambda$) caused small variations in the classification of co-operatives, especially for a degree of credibility ($\lambda$) of 0.9. This was due to the fact that the credibility index ($\lambda$) in the present model had a low demand (0.6).

For the sensitivity analysis referring to the weights given to the criteria, we analyzed all criteria by assigning the same weight ($Wn$ = 0.167).

Table 4. Class distribution after sensitivity analysis: weighted criteria.

	Poor (${\mathit{C}}_1$)	Average (${\mathit{C}}_2$)	Good (${\mathit{C}}_3$)
Democratic management and legislation	CP7	CP1, CP2, CP3, CP5, CP6, CP8	CP4
Valuing human labor	CP3, CP6	CP1, CP2, CP4, CP5, CP7, CP8
Technological and economic viability	CP3, CP5, CP6, CP7	CP1	CP2, CP4, CP8
Commitment to minorities	CP2, CP3, CP5, CP6, CP7	CP1, CP4, CP8
Environmental sustainability	CP3, CP4, CP6, CP7	CP1, CP2, CP5, CP8
Co-operation and solidarity	CP3	CP5, CP6, CP7	CP1, CP2, CP4, CP8

shows the distribution of the surveyed co-ops in the models’ performance classes after sensitivity analysis with the weighted criteria.

With the changes in the parameters given the criteria weights, there were no variations in the classification of the participating co-ops. Considering the results of the two sensitivity analyses, we conclude that the model used to evaluate the performance of the RSEEs through the ELECTRE TRI-C was stable, as the alternations imposed on the parameters did not significantly affect the results. It is important to note that IRIS 2.0 software used only one pessimistic agreement variant of the ELECTRE-TRI method, so the sensitivity analyses performed here considered only the pessimistic procedure.

4. Discussion of Results

The importance of tools that convert qualitative data related to social goals into quantitative metrics was highlighted by Grieco et al. (2015). Thus, the model developed and applied in this study met this challenge by transforming the subjectivity of qualitative data in relation to RSEEs into objective and quantitative indicators. However, it is important to highlight that some subjectivity remains after this kind of transformation.

The model presented herein, consisting of 36 indicators across 6 categories, focused on evaluating performance in terms of the principles of the solidarity economy and is consistent with the argument put forth by RIPESS (2016) that the contributions of SEE organizations should be assessed based on their impact on local, national, and international development. In particular, the assessment should consider the creation of permanent jobs, development of new services, better standards of living, contributions to gender equality, protection of the environment, and the ethical creation of wealth, all of which are included in the principles of the solidary economy.

The use of the ELECTRE-TRI methodology allowed for the classification of the participating co-ops into three performance classes: poor, average, and good. Thus, decision makers have information at hand that can contribute to organizational learning and continuous improvement. Following Pereira and Mota (2016), the model offers recommendations for decision making and enables learning about the problems in question. The classification of the participating co-ops into three performance classes was made possible by applying the model in IRIS 2.0 Demo software. Ribeiro (2016) noted that based on the indicators defined with decision makers, IRIS 2.0 software allows the visualization of the results of the ELECTRE-TRI method through the classification of alternatives (in this study, the participating co-ops).

Table 5. Summary statistics of results with λ = 0.6.

Dimension/Performance Class	Poor $\left({\mathit{C}}_1\right)$	Average $\left({\mathit{C}}_2\right)$	Good $\left({\mathit{C}}_3\right)$
Democratic management and legislation	12.5%	75%	12.5%
Valuing human labor	25%	75%	-
Technological and economic viability	50%	12.5%	37.5%
Commitment to minorities	62.5%	37.5%	-
Environmental sustainability	50%	50%	-
Co-operation and solidarity	12.5%	37.5%	50%

presents the summary statistics of the survey results for $\lambda$ = 0.6.

We can see that the dimension in which half of the co-ops (50%) showed a good performance level was “Co-operation and solidarity”. However, it should be pointed out that in the dimensions “Valuing human labor”, “Commitment to minorities”, and “Environmental sustainability”, none of the co-ops were evaluated as good. Furthermore, in most of the dimensions, co-ops showed average performance, except for in “Technological and economic viability” and “Commitment to minorities”, for which at least half (50% and 62.5%, respectively) presented poor performance. The low performance of the co-operatives in some dimensions reflects the characteristics of the geographic region (southwest Paraná) where these enterprises were located. In the aforementioned region, the majority of the population was white (IBGE 2023), with conservative customs. Furthermore, local rural production was mostly conventional, with the massive use of synthetic additives, pesticides, and other chemical substances. The model pointed to those dimensions as requiring further attention. The classification of co-ops based on six evaluation dimensions comes from Bagnoli and Megali (2011) who argue that social enterprises need a multidimensional management control system, since the performance of this type of organization assumes multiple profiles.

Table 6. Classification of each co-operative for the six evaluation dimensions.

Co-operative 1				Co-operative 2
Dimension	${\mathit{C}}_1$	${\mathit{C}}_2$	${\mathit{C}}_3$	Dimension	${\mathit{C}}_1$	${\mathit{C}}_2$	${\mathit{C}}_3$
Democratic management and legislation				Democratic management and legislation
Valuing human labor				Valuing human labor
Technological and economic viability				Technological and economic viability
Commitment to minorities				Commitment to minorities
Environmental sustainability				Environmental sustainability
Co-operation and solidarity				Co-operation and solidarity
Co-operative 3				Co-operative 4
Dimension	${\mathit{C}}_1$	${\mathit{C}}_2$	${\mathit{C}}_3$	Dimension	${\mathit{C}}_1$	${\mathit{C}}_2$	${\mathit{C}}_3$
Democratic management and legislation				Democratic management and legislation
Valuing human labor				Valuing human labor
Technological and economic viability				Technological and economic viability
Commitment to minorities				Commitment to minorities
Environmental sustainability				Environmental sustainability
Co-operation and solidarity				Co-operation and solidarity
Co-operative 5				Co-operative 6
Dimension	${\mathit{C}}_1$	${\mathit{C}}_2$	${\mathit{C}}_3$	Dimension	${\mathit{C}}_1$	${\mathit{C}}_2$	${\mathit{C}}_3$
Democratic management and legislation				Democratic management and legislation
Valuing human labor				Valuing human labor
Technological and economic viability				Technological and economic viability
Commitment to minorities				Commitment to minorities
Environmental sustainability				Environmental sustainability
Co-operation and solidarity				Co-operation and solidarity
Co-operative 7				Co-operative 8
Dimension	${\mathit{C}}_1$	${\mathit{C}}_2$	${\mathit{C}}_3$	Dimension	${\mathit{C}}_1$	${\mathit{C}}_2$	${\mathit{C}}_3$
Democratic management and legislation				Democratic management and legislation
Valuing human labor				Valuing human labor
Technological and economic viability				Technological and economic viability
Commitment to minorities				Commitment to minorities
Environmental sustainability				Environmental sustainability
Co-operation and solidarity				Co-operation and solidarity

presents the individual performance of each co-op in the six dimensions. In the table, the red color is related to class ($C_1$), indicating poor performance in the aforementioned dimension. The yellow color indicates average performance ($C_2$) and the green color indicates good performance ($C_3$).

From Table 6, we can see that although these co-ops participated in the same central organization and shared the same organizational values, their performance in each of the dimensions was quite different. The dimensions that were critical for each co-op can also be observed, enabling the enterprise to direct efforts at improving performance in the areas where the outcome was poorer than expected. We highlight co-operatives 3, 6, and 7, which presented poor performance (C₁) in at least four dimensions and did not achieve a good evaluation (C₃) in the other two dimensions. These co-operatives require centralized efforts directed at improving all the dimensions evaluated herein.

Although the principles of the solidarity economy are the values that guide co-operative activities, the results for each of the evaluation dimensions demonstrated that the participating co-ops could improve their performance. Helmig et al. (2015), in their study on the managerial importance of organizational values, identified that the values that generally underpin nonprofit organizations have positive effects on their success. It is important to highlight that performance evaluation contributes to the success of these RSEEs in the short and long terms.

Consequently, our study contributes to the context of multidisciplinary sustainable development. Furthermore, according to Meramveliotakis and Manioudis (2021), in the face of recent crises (economic crisis and COVID-19), many theories, perspectives, and analyses of the workings of society are starting to be challenged. Among them are the efficiency of free markets and the accompanying conventional thinking of economic development and sustainability. In this context, solidarity enterprises that seek to develop communities and territories, as is the case of the RSEEs in this study, constitute an alternative to conventional development.

In addition, the model developed and applied in this study contributes to the self-awareness and growth of co-operatives. Dias and Mousseau (2002) noted that the IRIS methodology is a process that fosters learning and the progressive definition of variations in inputs and outputs. Thus, we can affirm that the proposed model is appropriate for addressing the problem of performance evaluation based on the principles of the solidarity economy for RSEEs. Furthermore, the methodology is useful for the enterprise to remain accountable to both internal and external stakeholders, which is in line with Bagnoli and Megali (2011), who found that organizations with social ends have multiple stakeholders, as they respond to society and not to shareholders. Finally, the sensitivity analysis showed that the model is stable.

5. Conclusions

The present study developed a performance evaluation model for rural solidarity economy enterprises based on the principles of the solidarity economy. The approach aims to support decision making in these enterprises using the MCDA methodology, particularly ELECTRE-TRI. The model was applied to solidarity economy co-operatives in the southwest state of Paraná, Brazil, to test the applicability of the model and perform sensitivity analyses.

The results of this study showed that the use of the ELECTRE-TRI method was fundamental for classifying the co-ops into three performance classes considering previously defined criteria based on the principles of solidarity economy. The model enabled an evaluation of the performance of the enterprises across six dimensions. From this, it was possible to identify the dimensions that require attention in general and those that should be addressed in particular in each of the participating co-ops. Based on these results, organizational and improvement activities and actions can be developed and implemented.

This analysis contributes to our understanding economic and social indicators by offering improvement strategies for the professionalization and strategic management of RSEEs, thus strengthening these enterprises and, consequently, family agriculture. This study also makes an important academic contribution by filling a gap in the literature, presenting a new instrument for evaluating the performance of rural organizations with social goals, based on the principles of the solidarity economy and using the MCDA methodology.

Nevertheless, this analysis is limited by the fact that the model was applied to a small group of co-ops associated with one central organization and therefore cannot be generalized across all cases. Furthermore, we highlight that the data are from 2018 and do not reflect the current situation of the co-operatives studied. However, the model can be applied to other enterprises in the solidarity economy and adapted to the particularities of each case. Future studies may apply similar analyses to other objects of study and propose new models and/or performance evaluation instruments applicable to the diverse range of solidarity economy organizations. The model can also be used continuously in the same co-operatives for current data collection and accurate results. Furthermore, the model can be adapted to consider several criteria relevant to the long-term success and sustainable development of these co-operatives. Criteria that were not addressed in this study due to the limitations of the research included innovation capacity, customer satisfaction, stakeholder relationships, and human resource management, among others.