Next Article in Journal
Large-Scale Pumped Thermal Electricity Storages—Converting Energy Using Shallow Lined Rock Caverns, Carbon Dioxide and Underground Pumped-Hydro
Next Article in Special Issue
Evaluation of Spent Grain Biochar Impact on Hop (Humulus lupulus L.) Growth by Multivariate Image Analysis
Previous Article in Journal
POU-SLAM: Scan-to-Model Matching Based on 3D Voxels
Previous Article in Special Issue
Stomatal Response of Maize (Zea mays L.) to Crude Oil Contamination in Soils
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Applied Sciences
Volume 9
Issue 19
10.3390/app9194148
applsci-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Organic Vegetable Crops Managed with Agro-Ecological Practices: Environmental Sustainability Assessment by DEXi-met Decision Support System

by
Francesco Montemurro
1,*,
Alessandro Persiani
2 and
Mariangela Diacono
2
1
Consiglio per la Ricerca in Agricoltura e l’analisi dell’economia Agraria-Research Centre for Vegetable and Ornamental Crops, Via Salaria 1, 63030 Monsampolo del Tronto (AP), Italy
2
Consiglio per la Ricerca e l’analisi dell’economia Agraria-Research Centre for Agriculture and Environment, Via Celso Ulpiani 5, 70125 Bari, Italy
*
Author to whom correspondence should be addressed.
Appl. Sci. 2019, 9(19), 4148; https://doi.org/10.3390/app9194148
Submission received: 26 August 2019 / Revised: 30 September 2019 / Accepted: 1 October 2019 / Published: 3 October 2019
(This article belongs to the Special Issue Sustainable Agriculture and Soil Conservation)
Browse Figures
Versions Notes

Abstract

:
In the last decade, there has been an increasing interest in sustainable agricultural techniques and the environmental evaluation of the effects of agricultural practices. In the present study, we evaluated both the production capacity of organic horticultural systems, and the ex-post sustainability through a new multi-attribute decision model named “DEXi-met”. This qualitative model is able to estimate the environmental sustainability of cropping systems managed with different agro-ecological approaches. In particular, we compared the following three horticultural systems: (i) ECO, an organic system with full implementation of agro-ecological strategies (agro-ecological services crops (ASC), strip cultivation, and organic amendment); (ii) GM, an organic system with the introduction of the ASC; (iii) NO ASC, an organic system without ASC. The treatments with ASC presence (ECO and GM) showed similar total energy outputs (substantially higher than the NO ASC), indicating the positive effect of this agro-ecological practice. The findings pointed out that the ECO system, which followed the principles of natural ecosystems, can contribute to building up more complex agro-ecosystems, increasing both resilience and biodiversity. This management strategy reached a good compromise between the production of vegetable cropping systems and environmental sustainability achievement. Then, it is possible to optimize the use of natural resources, support climate adaptation, and reduce greenhouse gas emissions.

1. Introduction

Sustainable development and environmental sustainability are broadly recognized as global and collective goals because of key issues such as limited resources, environmental pollution, and global warming [1]. These increasing challenges, also considering the local and the global legislative changes, have inevitably involved the agricultural sector. Therefore, agronomists, farmers, and researchers should research, design, and experiment with new agricultural systems that are environmentally friendly, economically viable, socially supportive, and efficiently adapted to a climate change context. In different farming systems, a wide range of cultivation techniques and agro-ecological management strategies that enhance biodiversity in crop fields and support the sustainability of the agro-ecosystems are already practiced and should be further promoted [2,3]. Among them, crop rotations, introduction of agro-ecological service crops (ASC; [4]), utilization of soil amendments [5], and crop/livestock mixing can increase agro-ecosystem diversity and complexity both over space and time [6].
A prerequisite to the systems sustainability implementation is the development, improvement, and/or choice of the best possible assessment methods. In fact, there is a need to determine the reliability of innovative management practices with respect to the conventional ones, and to clarify the benefits and the drawbacks of the full or partial application of an agro-ecological approach. To this end, several approaches to measure, analyze, and assess sustainability have been developed [7]. The different methodologies can be classified on the basis of: (i) the typology of indicators used, from qualitative appraisals to quantitative analytical evaluations [8]; (ii) the scale of analysis, from the single plot to the whole farm or regional scales [9]; (iii) the systems typology, from orchards [10] to arable or horticultural crops [11]; and the timing of analysis, as ex-ante or ex-post evaluation [12].
Within this large number of methodologies, a growing interest focuses on multi-method approaches, which aim at accounting for the complexity of sustainability issues [13]. In this framework, the multi-criteria analysis (MCA) decision-making methods can handle a typical decision-making problem related to sustainability assessment. The MCA are increasingly gaining importance in agriculture, since they can consider multiple and conflicting criteria and, at the same time, they are able to tackle complex decisional problems breaking them down in easily understandable elements [14,15,16]. In recent years, the scientific community has developed several qualitative MCA tools for the sustainability assessment of different agricultural systems [17,18], based on a computer program for multi-attribute decision-making, defined as “DEXi” by Bohanec et al. [19]. Among them, the DEXi-met tool was recently developed, specifically for the ex-post evaluation in organic horticulture, and it was applied to compare different crop rotations in Mediterranean conditions [20]. In this model, a cropping system was considered instead of a single cash crop, in order to have a broader idea of the environmental sustainability of the system.
There is still a lack of knowledge on the sustainability assessment when different levels of the agro-ecological approach are applied, especially in organic horticultural production in the Mediterranean environment. In light of these considerations, the aim of the present research was to evaluate the performance of different cultivation systems managed with agro-ecological practices. To accomplish this aim we evaluated both the production capacity of the systems and the ex-post environmental sustainability by using the DEXi-met qualitative multi-attribute model.

2. Materials and Methods

2.1. Study Site

The study was performed in the research farm ‘Azienda Sperimentale Metaponto’ of the Consiglio per la Ricerca in Agricoltura e l’analisi dell’economia Agraria. The farm is located at Metaponto (MT), in southern Italy (lat. 40°24′ N; long. 16°48′ E, 8 m above sea level).
The soil, classified as a Typic Epiaquert, has the following properties: low N (1.0 g kg−1) and organic matter (19.0 g kg−1) contents, 759 mg kg−1 of exchangeable potassium (K), 31.1 mg kg−1 of available phosphorus (P), pH value of 8.4, clay and silt contents of 60 and 36%, respectively, electrical conductivity of 0.48 mS cm−1 (at 0 to 30 cm depth), increasing with depth, average bulk density of 1350 kg m−3, cation exchange capacity of 27.1 meq 100 g−1 of dry soil and the soil water content (a percentage of soil-dry weight) of 34.5% and 20.1% at field capacity (−0.03 MPa) and permanent wilting point (−1.5 MPa), respectively.
The climate is classified as “accentuated thermo-Mediterranean”, considering the UNESCO-FAO classification [21], with mean monthly temperatures of 8.8 °C in the winter, and 24.4 °C in the summer. Winter temperatures can fall below 0 °C, whereas summer temperatures can rise above 40 °C. The total rainfall (on average 490 mm year−1) is concentrated mainly during the winter months and the mean annual potential evaporation rate is 1549 mm.

2.2. Experimental Field Trials, Treatments, and Measurements

The research was carried out during the 2016–2017 cropping season in two different experimental fields. The first one consists in a long-term field trial in organic horticulture, which had been planned to adapt horticultural systems to unfavorable climatic conditions (in particular, extreme rainfall events during autumn and winter periods). In this experimental field, integrated strategies are combined, i.e., soil surface shaping, crop rotations, introduction of agro-ecological services crops (ASC), ASC termination techniques, and fertilization with organic products [4]. The base layer is the soil surface shaping in a “ridge system”. Cash crops are planted both on the top of each raised bed 2.5 m wide (ridges) and in the 2.5 m flat areas (strips) between them. The crop rotation is designed to cultivate the cash crop on the ridges and the ASC in the strips during the winter-rainy period of the year. Cover crops are used to prevent soil erosion and provide N to the system via biological fixation, since on the top of the ridges, a leguminous cover crop is intercropped (as living mulch) in the winter as a vegetable crop and maintained as a ground cover. During the winter-rainy period, in the flat soil strips, mixtures of ASC species of different botanical families are cultivated between two consecutive spring–summer cash crops. The used ASC termination methods (before the subsequent cash crops transplant) are green manure (in which the cover crop is chopped and plowed at the end of flowering) vs. cover crop biomass flattening by an in-line roller-crimper, in which the mulch covers the soil surface until the vegetable crop harvest [4]. Finally, the last layer consists of organic fertilization, which is implemented into further horizontal strips, by using commercial and experimental amendments.
The second experimental field was conceived to verify the hypothesis that the use of the in-line roller crimping technology for ASC termination will improve the agronomical performances of the organically managed vegetable cropping systems. A two-year field experiment was carried out to evaluate the effect of ASC termination on tomato, by comparing green manure vs. roller-crimper, NO ASC (control) and plastic mulch (positive control) treatments. Another variability factor consists of three different fertilization treatments (i. on-farm organic fertilizer, ii. commercial organic fertilizer, and iii. unfertilized control).
The experimental design of each field trial was a strip plot with three replications, allowing the ability to calculate the standard deviation of the variables.
In the present research, the following three management systems adopting different agro-ecological approach levels are considered, which have been extrapolated from the two above-defined experimental field trials (Table 1):
  • ECO, an organic system with the full implementation of the described agro-ecological strategies, from the first experimental field. The cultivation area (1 ha) was divided into two parts (0.5 ha ridge furrow and 0.5 ha flat strip) and crops were cultivated both on the ridges and in the strips. On the ridges, with the clover as a living mulch, cauliflower during the winter period (transplanted on 20 October 2016 and harvested on 21 March 2017) and tomato crop during the spring–summer (transplanted on 24 April 2017 and harvested during August 2017) were cultivated. In the strips, the ASC (80% vetch-20% oats) were sown in November 2016 and incorporated as break crops during the following spring. Zucchini, during the spring–summer (transplanted on 27 April 2017 and harvested during July 2017), and lettuce, during the late summer–autumn (transplanted on 31 August 2017 and harvested on 26 October 2017), were then cultivated. A composted anaerobic digestate from cattle manure was used as fertilizer (i.e., on-farm organic fertilizer). The phytosanitary management followed the organic farming rules. The ECO cultivation system is under study in an experimental field in which the adaptation of horticultural systems to extreme climatic events are being tested, since these phenomena are increasing in the Mediterranean area. Consequently, we choose this experimental system to verify the hypothesis that the above-described practices may be used by the farmers as potential adaptation strategies for organic agro-ecosystems.
  • GM, an organic system with the introduction of the ASC, from the second experimental field. The ASC (80% vetch-20% oats) were sown in November 2016 and chopped and plowed into the soil in April 2017. The tomato plants were transplanted in May and harvested during July 2017. The fertilizer, a composted anaerobic digestate from cattle manure (i.e., on-farm organic fertilizer), was spread two times, before ASC sowing (70% of the total amount) and the remaining part (30%) before tomato transplanting. The phytosanitary management followed the organic farming rules. The above-described cultivation technique is becoming more utilized in organic farms, even if it needs further investigation, particularly in horticultural systems.
  • NO ASC, an organic system without ASC, from the second experimental field. The tomato crop was cultivated in the spring–summer period (transplanted on 5 May 2017 and harvested during July–August 2017). The phytosanitary approach followed the organic farming rules and a commercial organic fertilizer (NPK 4-8-12) was spread before transplanting. This cultivation system is still the most commonly used by organic horticultural farmers, even if it does not follow the agro-ecology approach.
In ECO, at the cauliflower, zucchini, and lettuce commercial maturity, five randomly selected plants in each plot were collected to determine both “production quantity” attribute and the most important quality parameters for the calculation of “production quality” attribute. Conversely, at harvest, in GM and NO ASC the tomato fruits were collected from two randomly selected plants (center of the 2 rows in each plot) and both marketable and total yields and quality parameters were recorded to calculate production attributes.

2.3. Sustainability Evaluation

2.3.1. DEXi-met Model Application

To assess the sustainability of the agro-ecological practices implemented in the experimental field trials, crops yield and energy outputs were measured. The marketable yields were multiplied by their own coefficient of equivalent energy taken by the literature, to estimate the energy outputs [22]. The data of each agricultural operation were collected in a standardized procedure. All field practices were recorded (human labor as h ha−1, fuels consumption as kg ha−1), during the cover crop management and the cash crop cycles. Moreover, to better understand the systems environmental impact and sustainability, the DEXi-met model was used, aiming at assessing the level of sustainability of each considered system [20]. DEXi-met was developed for the ex-post assessment in organic horticulture by implementing the original DEXi software, which is utilized in multi-criteria decision analysis [23]. The ex-post assessment carried out with this model includes the basic attributes derived from the field experiment (e.g., productions, organic matter, etc.). In more details, DEXi-met is based on a hierarchical decision tree structure that breaks down the sustainability into smaller modules, which can be explained and calculated. Both qualitative and quantitative basic attributes are categorized into a linguistic scale, that is from a three-value scale (“low”, “medium”, “high”), used for the basic attribute, to a seven-value scale (“very-low”, “low”, “medium-low”, “medium”, “medium-high”, “high”, “very-high”) for the “overall sustainability”. The evaluation procedure begins with the calculation of the basic attributes, that could be also calculated using a satellite tree [24]. Their homogenization into the rank-ordered qualitative scale and the pyramidal aggregation of attributes contributed to the calculation of the aggregated final sustainability. The aggregation procedure is based on decision rules and relative weightings, that were given to each attribute, according to their alleged significance and contribution to sustainability. The weightings were defined involving both decision analysts and experts (i.e., researchers, agronomists, and farmers) and considering the literature, as indicated in Montemurro et al. [20]. The DEXi-met model tree structure is reported in Figure 1. All the attributes (basic and aggregate) from the bottom to the top, their aggregation weights and the corresponding scales are presented, to understand the calculation of the final “overall environmental sustainability”.

2.3.2. DEXi-met Sensitivity Analysis

In order to identify the most significant variables that affected the sustainability of the systems, a sensitivity analysis (SA) of the DEXi-met model was also performed. According to the suggestions of Iocola et al. [17], the SA was performed utilizing the IZIEval tool (http://wiki.inra.fr/wiki/deximasc/Interface+IZI-EVAL/Accueil). The IZIEval is an interface shaped to facilitate the multi-criteria sustainability assessment of cropping systems based on models developed with the DEXi software, supplementing the existing features of DEXi. The Algdesign and XML packages, of the open-source R software [25], were used for the SA.
Through the IZIEval interface, both the sensitivity indexes (SI) and Monte Carlo (MC) analyses were performed, to gain the SA. In particular, according to Carpani et al. [15] we used the same basic attributes utilized for the “overall environmental sustainability” in the sensitivity indexes computation. The software automatically attributed an equal weight or probability of occurrence to all possible values of each variable. The SI highest values corresponded to the most important effect for a specific variable within the “overall environmental sustainability”. The SI used the hierarchical model tree structure to obtain the results. Aggregation weights and number of the basic variables at the same level, aggregation weights of the aggregated variables, and depth levels influenced the findings.
To model the probability of different outcomes when random variables are involved, the Monte Carlo simulations are a possible tool. They allow obtaining the relative frequency distribution of the output values of an aggregated variable. In our study, according to Iocola et al. [17] this analysis was carried out by using IZIeval, randomly sampling and simulating a large number of values (5000) of each variable, to obtain the frequency distribution of the overall sustainability values and their main components.

3. Results

3.1. Yields Performance and Energetic Outputs

The highest absolute value of tomato marketable yield was found in GM, whereas the ECO treatment showed the lowest one with a reduction of −58 and −24% in comparison with GM and NO ASC systems, respectively (Table 2).
The ECO treatment also showed a very low marketable yield in cauliflower cultivation. On the whole, considering that in ECO the cultivation area of each crop was 0.5 ha, while in GM and NO ASC it was doubled, the GM treatment determined higher total energy output by 70.3% and 14.4% as compared to NO ASC and ECO treatments, respectively (Table 3). Furthermore, the difference between GM and ECO was due to the low energy output occurred in cauliflower. In any case, all values of energy output were characterized by a high variation.

3.2. Environmental Sustainability Evaluation

The overall environmental sustainability of the tested cropping systems varied considering the different crop management (Figure 2). In particular, the “high” score was reached by ECO treatment, while a “medium-high” and “medium-low” score was obtained for GM and NO ASC, respectively.
The nodal aggregate attribute “production capacity” resulted in “medium” in ECO and GM treatments and “medium-low” in NO ASC (Figure 3).
The ECO treatment scored “high” value for the aggregate attributes “soil and water preservation” and “resource preservation”, while GM and NO ASC reached “medium-high” and “medium” scores, respectively, for these same aggregates. The “biodiversity conservation” ranged from “high” in ECO to “low” in NO ASC.
The sustainability evaluation of all the components (from the overall sustainability to the basic attributes) for the three scenarios is reported in Figure 4, as a comparison among the different tested systems. It is also reported the level of sustainability (from “sustainable” to “not sustainable”) of each item, according to the specific linguistic scale (from three to seven) described in Figure 1.
The “production capacity” aggregate attribute is generated from the first order aggregate attributes “control of pests and diseases”, “physical-chemical fertility”, and “production”. In the ECO management, most of these attributes scored “high”, “medium-high” or “medium-low”, with one only basic attribute (N balance) with “low” value. Conversely, in NO ASC, the most frequent score was “medium-low”, while GM showed intermediate values between the other two treatments.
The “soil and water preservation” aggregate attribute showed small differences among water management options, while the first order attribute “soil” was “high”, “medium-low” and “low” for ECO, GM, and NO ASC, respectively. These differences are generated by the basic attributes “tillage diversification” and “tillage typology and depth” (“high” in ECO and “medium” in GM and NO ASC), “soil erosion control (“high” in ECO and “medium-high” and “medium-low” for GM and NO ASC, respectively) and “organic matter balance” (“high” in ECO and GM and “low” in NO ASC).
The “resources preservation” aggregate attribute differed for the attributes related to the “energy” and “fertilization”, which scored frequently “high” and “medium” in ECO treatment, “medium” in GM and “medium-low” and “low” in NO ASC. No differences were found in the first order “phytosanitary management” attribute and in the basic attributes.
Finally, a large number of differences were recorded in the “biodiversity conservation” component. In particular, the ECO treatment scored “high” and “medium-high” in most of the basic attributes, GM showed frequently “medium-high” values, while NO ASC scored “medium-low” and “low” values.
The results of the SI calculation for the basic attributes are reported in Figure 5.
The “microorganism preservation” and the “macrofauna preservation” reached the highest (0.12 and 0.08, respectively) SI values, being the most influential variables of the first order “biodiversity conservation” attribute. Within the “resources preservation” aggregate attribute, the “fertilizer C/N” basic attribute reached the highest SI value, while in the “production capacity” attribute, the “insects and pest diseases” and the “weeds” and “production quantity” showed higher values compared with the other basic attributes. Within the “soil and water preservation” component, the only basic attribute “soil erosion control” overtakes the 0.2 sensitivity index.
Table 4 reports the frequency distributions of the 5000 simulated outputs of the Monte Carlo (MC) analysis for the overall sustainability and for the main model components (nodal attributes). The “overall environmental sustainability” showed the qualitative value “medium” more frequently (MC = 0.543) than the other values. Among the nodal attributes, the “production capacity” recorded the value “medium” (MC = 0.496) that occurred more frequently than the other modalities. The “medium-low” and “medium” values were the most frequent for the “soil and water preservation” (MC = 0.571 and MC = 0.275, respectively), while the “resources preservation” principally scored the “medium” and “medium-high” values (MC = 0.475 and MC = 0.399, respectively). In the nodal attribute “biodiversity conservation”, the “medium-low” and “medium-high” were the most frequent values.

4. Discussion

4.1. Yield Performances and Energetic Output

The values of the tomato marketable yields in GM were higher by 70 and 123% compared with NO ASC and ECO treatments, respectively. This result was probably due to higher availability of readily available N, which derives from decomposition of the aboveground biomass of the ASC plowed into the soil [4,11]. Conversely, in the ECO plots, in which the clover was used as living mulch [26,27], and in the NO ASC, the tomato yield did not reach the standard level of organic production [28]. Moreover, in the ECO system, the cauliflower marketable yield was very low, likely because adverse climatic conditions occurred. In fact, during the growing season there was an extreme adverse event, unusual in the experimental area, showing low mean month temperature (4.5 °C) and several days of values below −4 °C on January 2017, associated with high rainfall intensity (117 mm). The zucchini and lettuce marketable yields, which benefited from the residual fertility of the ASC, were comparable with other experimental results on organic crops production [29,30,31].
The total energy output per hectare was higher in GM by 14% and 70% than ECO and NO ASC, respectively (Table 3). In particular, the treatments with the presence of ASC showed similar total energy output productions (about 20,000 MJ ha−1), which were substantially different in comparison with the management without cover crops, thus indicating the positive effect of such agro-ecological practice [32]. This result was probably due to the large difference generated in the total tomato output for GM and NO ASC treatments. Conversely, the ECO treatment showed a slightly reduced energy output in comparison with GM, because of the very low productions in the cauliflower cropping cycle [33], even if the four different crops contributed to the overall production by half a hectare. According to the USDA indications [22], the coefficient of equivalent energy of the cauliflower was the highest (value equal to 1), therefore, the low production of this crop influenced the total energy output of the ECO treatment. In any case, we must take into account that the differences among treatments showed a high standard deviation, which was generated by the huge variability of the data.

4.2. Environmental Sustainability Evaluation by DEXi-met

The “overall environmental sustainability” of the cropping systems varied among the three evaluated management systems, passing from “medium-low” showed in NO ASC to the “high” in ECO. This response can be explained by the scores both of the aggregate nodal attributes (Figure 3) and the basic attributes (Figure 4), and it was a consequence of the intensification of the agro-ecological strategies adopted. In particular, the DEXi-met output showed that the ECO strategy was the most sustainable one, mainly due to the differences detected in the nodal attribute “biodiversity conservation”. In fact, the study of Depalo et al. [34] pointed out a general positive influence of the living mulch techniques on arthropods in plant/soil systems, as shown by a high level of soil biodiversity and a lack of negative impacts on the density of canopy insects. Also, the presence of ASC in the rotation as break crops enhances the “biodiversity conservation” and, at the same time, it may have impact on occurrence of weeds, diseases, and pests [4,35]. Our results confirmed these findings, being the ECO system characterized by ASC presence both as break crops and living mulch, compared with the other two treatments (Table 1).
For the “production capacity”, differently from the yield performance and the energetic output, the DEXi-met model considered not only the crop productions, but also the physico-chemical fertility and the systems control on pests and disease. The differences between ECO and GM were not perceivable by the model and scored “medium” value in both systems (Figure 3). Conversely, the NO ASC scored “medium-low”, due to the “low” value of pests and disease control (Figure 4).
The systems with the introduction of the agro-ecological service crops (ECO and GM) scored high soil erosion control, in agreement with the study of De Benedetto et al. [36]. This result was generated by a better soil cover, in particular during the winter/heavy rainy period. Therefore, the nodal attribute “soil and water preservation” was the highest in ECO, followed by the GM system. The differences in “resources preservation” was mainly due to the different fertilizers used. In fact, the composted anaerobic digestate, which was utilized in the ECO and GM plots, is a renewable, more sustainable fertilizer than the commercial organic one, and its application did not compromise the systems production capacity, thus confirming the findings of previous studies [11,37].
Finally, the DEXi-met output showed substantial differences in the aggregate nodal attribute “biodiversity conservation” among the systems, as explained above. In particular, the introduction of the ASC increased the score both in ECO and GM (“high” and “medium-high”, respectively), compared to the NO ASC management strategy. Similarly, other studies indicated that the presence of ASC enhances the insect and arthropods communities [38], as well as the soil microbial activities [39].
Even if the DEXi-met presents some aspects that should be improved, it showed some strengths and, therefore, its application gave us the possibility to analyze in detail the general structure of the overall sustainability, as well as the components and the single variables of the systems considered. We should also take into account that DEXi-met model is one of the new ex-post tools, which considers some attributes derived from the field experiment. However, to better understand how the model tree structure affects the results and to find the most significant variables that contributed most to the output variability, a sensitivity analysis was necessary. The sensitivity index results were affected by the level of complexity of each component and by the number of variables. Carpani et al. [15] indicated that a simpler component structure has a greater influence on the overall sustainability, whereas a higher number of variables, that individually could have no significant impact, become more sensitive if they are considered together. In our study, the DEXi-met produced both the “microorganism preservation” and the “macrofauna preservation” attributes as the most influential variables of our sensitivity analysis. This last result was due to the difference in the systems analyzed. In particular, it is a consequence of using agro-ecological practices, especially in the ECO system. Except for “strip cultivation with agro-ecological functions”, all the other basic attributes in the nodal aggregate “biodiversity conservation” showed the SI higher than 0.2, indicating the positive influence of the systems on the environmental sustainability [40]. The basic attributes “fertilizer C/N” (within the “resources preservation” nodal attribute), the “insects and pests diseases”, “weeds”, and “production quantity” (within the “production capacity”), and “soil erosion control” (within the “soil and water preservation”) reached the highest values. According to Carpani et al. [15], when the SI is high, the effect of each variable on the overall sustainability is more relevant.
The detailed analysis of the Dexi-met model structure through the distribution of frequencies in the overall sustainability and the nodal attributes (showed by the Monte Carlo analysis) revealed that the model adequately represents the diversity of the systems evaluated. In fact, the frequencies obtained in the “overall sustainability” showed the highest value for the “medium” modality, following a normal Gaussian pattern. This behavior is due to the use of till to seven qualitative classes at the “overall sustainability” level, allowing to distinguish the different scenarios. However, the number of the qualitative classes (from “very low” to “very high”) was not so large to generate unnecessary complications in the use of the model, and to reduce its ability to distinguish differences between systems. Besides, to avoid further complications, in agreement with Craheix et al. [24], the aggregate attributes were composed by five qualitative classes (from “low” to “high”) and the basic attributes were composed by only three classes (“high”, “medium”, and “low”).

5. Conclusions

Sustainability in agriculture is a complex concept and there are no common viewpoints among scientists about its dimension. Nonetheless, various parameters for measuring agricultural sustainability have been proposed, since the measure of the mere production capacity of an agro-system is not enough to evaluate it. This study clearly highlighted the relevance of considering different criteria, when we assess the advances in sustainability achievement that could be obtained introducing agro-ecological management practices and innovations. The findings also demonstrated that applying principles and practices which tend to follow the natural ecosystems can contribute to building up more complex agro-ecosystems, increasing resilience, and optimizing and maintaining biodiversity. In particular, the agro-ecological approach (ECO) both reduces the use of and dependency on external synthetic inputs by enabling to control pests, weeds, and improving fertility with ecological management. This management strategy could optimize and close resource loops (nutrients, biomass, etc.), by recycling nutrients and biomass in the farm. It may also support climate adaptation and resilience and contribute to greenhouse gas emissions mitigation, through lower use of fossil fuels and higher carbon sequestration in soils.
As revealed by our results, the introduction of the agro-ecological management practices such as ASC, use of on-farm produced fertilizers (composts), intercropping, etc., is an interesting way to improve the sustainability of the system. In any case, the results found here could not be fully generalized, since the Dexi-met model did not take into account some other aspects (e.g., the economic sustainability, the length of the study period, etc.). Moreover, when these strategies are applied, agronomic and productions difficulties should be kept in mind, at least in the short transition period between conventional and agro-ecological systems.
The proposed modeling approach provides a simple method of decisional support to farmers to efficiently select different crop management strategies, by assessing the environmental sustainability of the cultivation systems. An interesting topic of further research could be testing the considered agro-ecological management practices in different environmental conditions.

Author Contributions

Conceptualization, F.M., A.P., and M.D.; formal analysis, A.P.; data curation, A.P.; writing—original draft preparation, F.M., A.P., and M.D.; writing—review and editing, F.M., A.P., and M.D.; funding acquisition, F.M.

Funding

This paper is a result of the research projects RETIBIO (Attività di supporto nel settore dell’agricoltura biologica per il mantenimento dei dispositivi sperimentali di lungo termine e il rafforzamento delle reti di relazioni esistenti a livello nazionale e internazionale), funded by the Organic Farming Office of the Italian Ministry of Agriculture, and SOILVEG (Improving soil conservation and resource use in organic cropping systems for vegetable production through introduction and management of Agro-ecological Service Crops (ASC)) funded by ERA-Net CORE Organic Plus Funding Bodies partners of the European Union’s FP7 research and innovation programme under the grant agreement No. 618107.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

References

  1. Bitter, J.; Janssen, D.; Vossen, R. Requirements for multi-method approaches to sustainability assessment—A theoretical and empirical study. In The European Conference on Sustainability, Energy & the Environment. Official Conference Proceedings; The International Academic Forum: Brighton, UK, 2018; pp. 239–258. [Google Scholar]
  2. Diacono, M.; Persiani, A.; Canali, S.; Montemurro, F. Agronomic performance and sustainability indicators in organic tomato combining different agro-ecological practices. Nutr. Cycl. Agroecosyst. 2018, 112, 101–117. [Google Scholar] [CrossRef]
  3. Diacono, M.; Baldivieso-Freitas, P.; Sans Serra, F.X. Nitrogen utilization in a cereal-legume rotation managed with sustainable agricultural practices. Agronomy 2019, 9, 113. [Google Scholar] [CrossRef]
  4. Diacono, M.; Fiore, A.; Farina, R.; Canali, S.; Di Bene, C.; Testani, E.; Montemurro, F. Combined agro-ecological strategies for adaptation of organic horticultural systems to climate change in Mediterranean environment. Ital. J. Agron. 2016, 11, 85–991. [Google Scholar] [CrossRef]
  5. Diacono, M.; Persiani, A.; Testani, E.; Montemurro, F.; Ciaccia, C. Recycling agricultural wastes and by-products in organic farming: Biofertilizers production, yield performance and carbon footprint analysis. Sustainability 2019, 11, 3824. [Google Scholar] [CrossRef]
  6. Altieri, M.A.; Koohafkan, P. Strengthening resilience of farming systems: A key prerequisite for sustainable agricultural production. In Wake Up before It is Too Late: Make Agriculture Truly Sustainable Now for Food Security in a Changing Climate; UNCTAD, TER13 Report; UNCTAD: Geneva, Switzerland, 2013; pp. 56–60. [Google Scholar]
  7. Cinelli, M.; Coles, S.R.; Kirwan, K. Analysis of the potentials of multi criteria decision analysis methods to conduct sustainability assessment. Ecol. Indic. 2014, 46, 138–148. [Google Scholar] [CrossRef] [Green Version]
  8. Waas, T.; Hugé, J.; Block, T.; Wright, T.; Benitez-Capistros, F.; Verbruggen, A. Sustainability Assessment and Indicators: Tools in a Decision-Making Strategy for Sustainable Development. Sustainability 2014, 6, 5512–5534. [Google Scholar] [CrossRef] [Green Version]
  9. Rodrigues, G.A.; Rodrigues, I.A.; Buschinelli, C.C.; de Barros, I. Integrated farm sustainability assessment for the environmental management of rural activities. Environ. Impact Assess. Rev. 2010, 30, 229–239. [Google Scholar] [CrossRef]
  10. Pergola, M.; Persiani, A.; Pastore, V.; Palese, A.M.; Arous, A.; Celano, G. A comprehensive Life Cycle Assessment (LCA) of three apricot orchard systems located in Metapontino area (Southern Italy). J. Clean. Prod. 2017, 142, 4059–4407. [Google Scholar] [CrossRef]
  11. Diacono, M.; Persiani, A.; Fiore, A.; Montemurro, F.; Canali, S. Agro-ecology for potential adaptation of horticultural systems to climate change: Agronomic and energetic performance evaluation. Agronomy 2017, 7, 35. [Google Scholar] [CrossRef]
  12. Bockstaller, C.; Feschet, P.; Angevin, F. Issues in evaluating sustainability of farming systems with indicators. Oléagineux Corps Gras Lipides John Libbey Eurotext 2015, 22, 12. [Google Scholar] [CrossRef]
  13. Bond, A.J.; Morrison-Saunders, A.; Pope, J. Sustainability assessment: The state of the art. Impact Assess. Proj. Apprais. 2012, 30, 53–62. [Google Scholar] [CrossRef]
  14. Bohanec, M. DEXi: Program for Multi-Criteria Decision Making, User’s Manual, Version 5.00; IJS Report DP-11897; Jožef Stefan Institute: Ljubljana, Slovenia, 2015; Available online: http://kt.ijs.si/MarkoBohanec/DEXi/html/DEXiDoc.htm (accessed on 5 January 2018).
  15. Carpani, M.; Bergez, J.E.; Monod, H. Sensitivity analysis of a hierarchical qualitative model for sustainability assessment of cropping systems. Environ. Model. Softw. 2012, 27, 15–22. [Google Scholar] [CrossRef]
  16. Sadok, W.; Angevin, F.; Bergez, J.; Bockstaller, C.; Colomb, B.; Guichard, L.; Reau, R.; Messéan, A.; Doré, T. MASC, a qualitative multi-attribute decision model for ex ante assessment of the sustainability of cropping systems. Agron. Sustain. Dev. 2009, 29, 447–461. [Google Scholar] [CrossRef] [Green Version]
  17. Iocola, I.; Campanelli, G.; Diacono, M.; Leteo, F.; Montemurro, F.; Persiani, A.; Canali, S. Sustainability Assessment of organic vegetable production using a qualitative multi-attribute model. Sustainability 2018, 10, 3820. [Google Scholar] [CrossRef]
  18. Nikoloski, T.; Udovč, A.; Pavlovič, M.; Rajkovič, U. Farm reorientation assessment model based on multi-criteria decision making. Comput. Electron. Agric. 2017, 140, 237–243. [Google Scholar] [CrossRef]
  19. Bohanec, M.; Messean, A.; Scatasta, S.; Angevin, F.; Griffiths, B.; Krogh, P.H.; Žnidaršič, M.; Džeroski, S. A qualitative multi-attribute model for economic and ecological assessment of genetically modified crops. Ecol. Model. 2008, 215, 247–261. [Google Scholar] [CrossRef]
  20. Montemurro, F.; Persiani, A.; Diacono, M. Environmental sustainability assessment of horticultural systems: A multi-criteria evaluation approach applied in a case study in Mediterranean conditions. Agronomy 2018, 8, 98. [Google Scholar] [CrossRef]
  21. Unesco-FAO Etude écologique de la Zone Méditerranéenne. Carte Bioclimatique de la Zone Méditerranéenne. 1963. Available online: http://unesdoc.unesco.org/images/0013/001372/137255fo.pdf (accessed on 5 April 2017).
  22. USDA. United States Department of Agriculture Agricultural Research Service National Nutrient Database for Standard Reference Legacy Release. Available online: https://ndb.nal.usda.gov/ndb/ (accessed on September 2018).
  23. Bohanec, M.; Dzeroski, S.; Znidarsi, M.; Messean, A.; Scatasta, S.; Wesseler, J. Multi-attribute modeling of economic and ecological impacts of cropping systems. Informatica 2004, 28, 387–392. [Google Scholar]
  24. Craheix, D.; Bergez, J.E.; Angevin, F.; Bockstaller, C.; Bohanec, M.; Colomb, B.; Doré, T.; Fortino, G.; Guichard, L.; Pelzer, E.; et al. Guidelines to design models assessing agricultural sustainability, based upon feedbacks from the DEXi decision support system. Agron. Sustain. Dev. 2015, 35, 1431–1447. [Google Scholar] [CrossRef] [Green Version]
  25. R Core Team. R: A Language and Environment for Statistical Computing; R Foundation for Statistical Computing: Vienna, Austria, 2013; Available online: http://www.R-project.org/ (accessed on February 2018).
  26. Chase, C.A.; Mbuya, O.S. Greater interference from living mulches than weeds in organic broccoli production. Weed Technol. 2008, 22, 280–285. [Google Scholar] [CrossRef]
  27. Canali, S.; Campanelli, G.; Ciaccia, C.; Diacono, M.; Leteo, F.; Fiore, A.; Montemurro, F. Living mulch strategy for organic cauliflower (Brassica oleracea L.) production in central and southern Italy. Ital. J. Agron. 2015, 10, 90–96. [Google Scholar] [CrossRef]
  28. Adamczewska-Sowińska, K.; Kołota, E.; Winiarska, S. Living mulches in field cultivation of vegetables. Veg. Crops Res. Bull. 2009, 70, 19–29. [Google Scholar] [CrossRef]
  29. Ciaccia, C.; Montemurro, F.; Campanelli, G.; Diacono, M.; Fiore, A.; Canali, S. Legume cover crop management and organic amendments application: Effects on organic zucchini performance and weed competition. Sci. Hortic. 2015, 185, 48–58. [Google Scholar] [CrossRef]
  30. Montemurro, F.; Fiore, A.; Campanelli, G.; Tittarelli, F.; Ledda, L.; Canali, S. Organic fertilization, green manure, and vetch mulch to improve organic zucchini yield and quality. HortScience 2013, 48, 1027–1033. [Google Scholar] [CrossRef]
  31. Montemurro, F.; Ciaccia, C.; Leogrande, R.; Ceglie, F.; Diacono, M. Suitability of different organic amendments from agro-industrial wastes in organic lettuce crops. Nutr. Cycl. Agroecosyst. 2015, 102, 243–252. [Google Scholar] [CrossRef]
  32. Gomiero, T.; Pimentel, D.; Paoletti, M.G. Energy and environmental issues in organic and conventional agriculture. Crit. Rev. Plant Sci. 2008, 27, 239–254. [Google Scholar] [CrossRef]
  33. Persiani, A.; Diacono, M.; Monteforte, A.; Montemurro, F. Agronomic performance, energy analysis and carbon balance comparing different fertilization strategies in horticulture under Mediterranean conditions. Environ. Sci. Pollut. Res. 2019, 26, 19250–19260. [Google Scholar] [CrossRef]
  34. Depalo, L.; Burgio, G.; von Fragstein, P.; Kristensen, H.L.; Bavec, M.; Robačer, M.; Campanelli, G.; Canali, S. Impact of living mulch on arthropod fauna: Analysis of pest and beneficial dynamics on organic cauliflower (Brassica oleracea L. var. botrytis) in different European scenarios. Renew. Agric. Food Syst. 2017, 32, 240–247. [Google Scholar] [CrossRef]
  35. Campiglia, E.; Mancinelli, R.; Radicetti, E.; Caporali, F. Effect of cover crops and mulches on weed control and nitrogen fertilization in tomato (Lycopersicon esculentum Mill.). Crop Prot. 2010, 29, 354–363. [Google Scholar] [CrossRef]
  36. De Benedetto, D.; Montemurro, F.; Diacono, M. Impacts of Agro-Ecological Practices on Soil Losses and Cash Crop Yield. Agriculture 2017, 7, 103. [Google Scholar] [CrossRef]
  37. Montemurro, F.; Vitti, C.; Diacono, M.; Canali, S.; Tittarelli, F.; Ferri, D. A three-year field anaerobic digestates application: Effects on fodder crops performance and soil properties. Fresenius Environ. Bull. 2010, 19, 2087–2093. [Google Scholar]
  38. Navarro-Miró, D.; Caballero-López, B.; Blanco-Moreno, J.M.; Pérez, A.; Depalo, L.; Masetti, A.; Burgio, G.; Canali, S.; Sans, F.X. Agro-ecological Service Crops with roller crimper termination enhance ground-dwelling predator communities and pest regulation. In Proceedings of the 5th ISOFAR Scientific Conference “Innovative Research for Organic 3.0” 19th Organic World Congress, New Dehli, India, 9–11 November 2017. [Google Scholar]
  39. Hartwig, N.L.; Ammon, H.U. Cover crops and living mulches. Weed Sci. 2002, 50, 688–699. [Google Scholar] [CrossRef]
  40. Scherr, S.J.; McNeely, J.A. Biodiversity conservation and agricultural sustainability: Towards a new paradigm of “ecoagriculture” landscapes. Philos. Trans. R. Soc. Lond. B Biol. Sci. 2008, 363, 477–494. [Google Scholar] [CrossRef] [PubMed]
Applsci 09 04148 g001 550
Figure 1. The DEXi-met model decision tree. The model includes 30 basic attributes, aggregate attributes at different levels, four nodal attributes, and the overall sustainability. The numbers between attribute levels represent the default aggregation weights (expressed in %). For each attribute level (basic, aggregate, and overall) the scale is reported at the bottom of the figure.
Figure 1. The DEXi-met model decision tree. The model includes 30 basic attributes, aggregate attributes at different levels, four nodal attributes, and the overall sustainability. The numbers between attribute levels represent the default aggregation weights (expressed in %). For each attribute level (basic, aggregate, and overall) the scale is reported at the bottom of the figure.
Applsci 09 04148 g001
Applsci 09 04148 g002 550
Figure 2. Comparison among the different crop management strategies: evaluation results of the multi-criteria decision model DEXi-met on the overall sustainability. ECO = organic system with the full implementation of the agro-ecological strategies; GM = organic system with the introduction of the agro-ecological service crops (ASC); NO ASC = organic system without ASC.
Figure 2. Comparison among the different crop management strategies: evaluation results of the multi-criteria decision model DEXi-met on the overall sustainability. ECO = organic system with the full implementation of the agro-ecological strategies; GM = organic system with the introduction of the agro-ecological service crops (ASC); NO ASC = organic system without ASC.
Applsci 09 04148 g002
Applsci 09 04148 g003 550
Figure 3. Comparison among the different crop management strategies: evaluation results of the multi-criteria decision model DEXi-met on the four main aggregate attributes (“production capacity”, “soil and water preservation”, “resources preservation”, and “biodiversity conservation”). ECO = organic system with the full implementation of the agro-ecological strategies; GM = organic system with the introduction of the agro-ecological service crops (ASC); NO ASC = organic system without ASC.
Figure 3. Comparison among the different crop management strategies: evaluation results of the multi-criteria decision model DEXi-met on the four main aggregate attributes (“production capacity”, “soil and water preservation”, “resources preservation”, and “biodiversity conservation”). ECO = organic system with the full implementation of the agro-ecological strategies; GM = organic system with the introduction of the agro-ecological service crops (ASC); NO ASC = organic system without ASC.
Applsci 09 04148 g003
Applsci 09 04148 g004 550
Figure 4. Evaluation results from DEXi-met model from the overall environmental sustainability to the basic attributes. ECO = organic system with the full implementation of the agro-ecological strategies; GM = organic system with the introduction of the agro-ecological service crops (ASC); NO ASC = organic system without ASC.
Figure 4. Evaluation results from DEXi-met model from the overall environmental sustainability to the basic attributes. ECO = organic system with the full implementation of the agro-ecological strategies; GM = organic system with the introduction of the agro-ecological service crops (ASC); NO ASC = organic system without ASC.
Applsci 09 04148 g004
Applsci 09 04148 g005 550
Figure 5. Sensitivity index values obtained with IZIEval tool for each basic variable of DEXi-met referred to the overall sustainability. The vertical line distinguishes the more sensitive variables (right side of the line) from the others.
Figure 5. Sensitivity index values obtained with IZIEval tool for each basic variable of DEXi-met referred to the overall sustainability. The vertical line distinguishes the more sensitive variables (right side of the line) from the others.
Applsci 09 04148 g005
Table
Table 1. Description of the three different systems analyzed. ECO = organic system with the full implementation of the agro-ecological strategies; GM = organic system with the introduction of the agro-ecological service crops (ASC); NO ASC = organic system without ASC.
Table 1. Description of the three different systems analyzed. ECO = organic system with the full implementation of the agro-ecological strategies; GM = organic system with the introduction of the agro-ecological service crops (ASC); NO ASC = organic system without ASC.
ECOGMNO ASC
Year2016/20172016/20172016/2017
Total area1 ha1 ha1 ha
Soil textureClay soilClay soilClay soil
Strip cultivation for agroecological functionYes (on ridge-flat strips system)NoNo
Cash cropRidges: cauliflower/tomato 0.5 ha
Strips: zucchini/lettuce 0.5 ha		Tomato 1 haTomato 1 ha
ASC as break cropsVetch/oats 0.5 ha in strips Vetch/oats 1 haNo
ASC as living mulchClover 0.5 ha on ridgesNoNo
Phytosanitary managementOrganic (pyrethrum, Cu, S)Organic (pyrethrum, Cu, S)Organic (pyrethrum, Cu, S)
Fertilization managementOn-farm organic fertilizersOn-farm organic fertilizersCommercial organic fertilizers
Amount of N distributed with the fertilization215 kg ha−1150 kg ha−1150 kg ha−1
Soil tillageMinimum tillageMinimum tillageMinimum tillage
Irrigation system and water consumptionDrip irrigation-7320 m3Drip irrigation-3300 m3Drip irrigation-3300 m3
Table
Table 2. Effects of management strategies on marketable yields (Mg ha−1, values ± standard deviation). ECO = organic system with the full implementation of the agro-ecological strategies; GM = organic system with the introduction of the agro-ecological service crops (ASC); NO ASC = organic system without ASC.
Table 2. Effects of management strategies on marketable yields (Mg ha−1, values ± standard deviation). ECO = organic system with the full implementation of the agro-ecological strategies; GM = organic system with the introduction of the agro-ecological service crops (ASC); NO ASC = organic system without ASC.
Cash CropsECOGMNO ASC
Mg ha−1 St. dev.Mg ha−1 St. dev.Mg ha−1 St. dev.
Cauliflower0.96±0.05------
Zucchini13.21±3.73------
Lettuce24.69±1.99------
Tomat13.83±3.2330.88±13.8318.13±12.24
Table
Table 3. Crop ((MJ ha−1 values ± standard deviation) and total energy output divided by the management strategies. ECO = organic system with the full implementation of the agro-ecological strategies; GM = organic system with the introduction of the agro-ecological service crops (ASC); NO ASC = organic system without ASC.
Table 3. Crop ((MJ ha−1 values ± standard deviation) and total energy output divided by the management strategies. ECO = organic system with the full implementation of the agro-ecological strategies; GM = organic system with the introduction of the agro-ecological service crops (ASC); NO ASC = organic system without ASC.
CropEnergy EquivalentECOGMNO ASC
MJ kg−1 (USDA, 2019)MJ ha−1 St. dev.MJ ha−1 St. dev.MJ ha−1 St. dev.
Cauliflower1480±48- -- -
Zucchini0.95944±3361- -- -
Lettuce0.78643±1392- -- -
Tomato0.755185±242223,162±10,37613,599±9178
Total Energy output (MJ ha−1)20,252 23,162 13,599
Table
Table 4. Relative frequency distributions of the results of 5000 Monte Carlo simulations among the seven different qualitative values (“very low”, “low”, “medium-low”, “medium”, “medium-high”, “high”, “very high”) for the overall sustainability and among the five different qualitative values (low”, “medium-low”, “medium”, “medium-high”, “high”) for the main aggregate attributes (“production capacity”, “soil and water preservation”, “resources preservation”, and “biodiversity conservation”) obtained with DEXi-met.
Table 4. Relative frequency distributions of the results of 5000 Monte Carlo simulations among the seven different qualitative values (“very low”, “low”, “medium-low”, “medium”, “medium-high”, “high”, “very high”) for the overall sustainability and among the five different qualitative values (low”, “medium-low”, “medium”, “medium-high”, “high”) for the main aggregate attributes (“production capacity”, “soil and water preservation”, “resources preservation”, and “biodiversity conservation”) obtained with DEXi-met.
Very LowLowMedium-LowMediumMedium-HighHighVery High
Overall sustainability0.0000.0030.2570.5430.1900.0050.000
Production capacity-0.0140.2370.4960.2370.016-
Soil and water preservation-0.0200.5710.2650.1360.008-
Resources preservation-0.0040.0340.4750.3990.089-
Biodiversity conservation-0.2460.3090.0000.3380.107-

Share and Cite

MDPI and ACS Style

Montemurro, F.; Persiani, A.; Diacono, M. Organic Vegetable Crops Managed with Agro-Ecological Practices: Environmental Sustainability Assessment by DEXi-met Decision Support System. Appl. Sci. 2019, 9, 4148. https://doi.org/10.3390/app9194148

AMA Style

Montemurro F, Persiani A, Diacono M. Organic Vegetable Crops Managed with Agro-Ecological Practices: Environmental Sustainability Assessment by DEXi-met Decision Support System. Applied Sciences. 2019; 9(19):4148. https://doi.org/10.3390/app9194148

Chicago/Turabian Style

Montemurro, Francesco, Alessandro Persiani, and Mariangela Diacono. 2019. "Organic Vegetable Crops Managed with Agro-Ecological Practices: Environmental Sustainability Assessment by DEXi-met Decision Support System" Applied Sciences 9, no. 19: 4148. https://doi.org/10.3390/app9194148

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop