1. Introduction
Agriculture plays a key role in West Africa (WA), particularly in poor landlocked Sahelian countries such as Burkina Faso. The majority of smallholder farmers rely on rainfed crop production systems characterized by low productivity, low input levels and high vulnerability to climatic risks. In the past, such systems successfully adapted to changing climatic conditions. Nevertheless, climate change and variability, combined with population pressure, poverty and land degradation are reducing livelihoods’ resilience and coping capacity. Within weather and climate services (WCS), agrometeorological services (AS) are acknowledged as powerful tools for developing farmers’ resilience and sustain crop production [
1,
2,
3].
During the last 10 years, the World Meteorological Organization (WMO) actively supported WA countries, including Burkina Faso, in the transition towards Climate Smart Agriculture through the operationalization of WCS. The approach builds on the concept of Agriculture Innovation Systems (AIS), a network integrating agricultural research, extension, education agencies and farmers, as well as other private and public actors such as Meteorological Service, media and NGOs (Non-Governmental Organizations) that promote innovation in agriculture [
4]. The hypothesis is that WCS can effectively improve agricultural productivity and sustainability towards climate change and related risks, playing a critical role in meeting Sustainable Development Goals [
5].
Relevant research and investments have been deployed in earth observation and modeling with the objective of implementing WCS for agriculture. The provision of WCS is growing across Africa [
3], however, they remain almost inaccessible for a large part of smallholder farmers or, in the best case, are not very relevant to support on-the-ground decision-making [
6]. Indeed, in developing countries and particularly WA, WCS production relies mainly on National Meteorological and Hydrological Services (NMHS), which are structurally facing large capacity and competency gaps [
7]. According to many authors, besides technical capacities to develop the services, the main challenge remains the access to information by farmers [
8] and trust of farmers in WCS, ensuring operational use for risk-informed decision-making [
6,
9]. Access in WA, in terms of farmers’ capability to retrieve the information, relies mainly on local components of the AIS which are charged with bridging the gap between NMHS and local communities. Rural radios are still acknowledged as the main information channel for farmers [
2,
10] followed by the extension service and local social networks [
11]. Mobile phones and social media are gaining momentum [
12], but are still not generalized in poor rural areas. Nevertheless, trust, in terms of credibility, accuracy and source of information, affects farmers’ uptake and use of WCS [
13]. During the last years, collaborative approaches for co-development of WCS are being deployed in order to include users early in the design of the services [
2,
14]. Vincent et al. [
9] aim for a process of co-production implying an adaptation of the workflow “moving from supply-driven to demand-driven models” and applying an iterative model of co-learning establishing long-term relationships and trust between producers and users [
15,
16].
All these aspects need to be analyzed for the evaluation of WCS in agriculture. The literature is quite rich in ex-post and ex-ante evaluations of WCS impacts [
17,
18]. Ex-ante methods use models to estimate how potential uses of information could improve the productivity or socio-economic conditions of households, while ex-post methods use empirical approaches to directly estimate the benefits due to already existing WCS. All these methods do not usually deepen the causal relationship between information and outcomes. Indeed, the evaluation of WCS is particularly challenging because of the interaction of many components that vary considerably from year to year, from production system to production system, and even from community to community [
17].
According to the logic model for WCS evaluation in agriculture developed by Vogel et al. [
18], outcomes in terms of increased agricultural productivity are a cumulative indicator of the effectiveness of a complex process. Performance degradations at any of the stages of the process contribute to reducing the overall outcome of WCS. The use of such a logic model in the evaluation of WCS provides the opportunity to verify each step with users and adopt corrective measures if needed. Indeed, while the overall evaluation can be ensured by the assessment of the outcomes (e.g., yield increase), the evaluation of the process needs to be split in different steps where the first focuses on the quality of the information [
19], the second on access [
20], the third on uptake [
21] and the last on ability to act [
22].
Given the hypotheses that WCS produced by NMHS can effectively improve agricultural productivity and increase farmers’ income, this paper analyzes the access, uptake and use by farmers of WCS produced by the Burkina Faso National Agency for Meteorology (ANAM) and finally it assesses the impact on agricultural productivity in terms of both cost reduction and income generation. The paper addresses also some environmental positive impacts enabled by the adoption of WCS.
2. Materials and Methods
2.1. Study Area
The study site is the Municipality of Tenado, one of the three pilot sites of the CREWS project in Burkina Faso. Located in the West-Central Region of Burkina Faso, the area belongs to the Sudano-Sahelian climatic domain with total annual rainfall ranging from 600 to 900 mm/year. The duration of the rainy season does not generally exceed 6 months. The relief of the area is generally uneven, varying between 200 and 400 m a.s.l. as a whole; the vegetation of the area is characterized by shrub savannahs, tree savannahs and gallery forests. According to the last census in 2006, the population was 48,694, corresponding to about 4900 rural households. Cereal farming, cash crops and trade are the main economic activities, while the most produced cereals are sorghum, millet, and corn.
In line with the CREWS approach, at this site, ANAM provided training for producers and breeders in the use of WCS through roving seminars [
23,
24]. Five were held in 2018 with an average of 70 people per session, another one in 2019 and one in 2020, and rain gauges were also distributed. In addition to the roving seminars for producers, training for agents of the extension services (agriculture, animal production, environment, municipalities) and radio agents in the use and dissemination of WCS was carried out at the site level. During 2018, 2019 and 2020 cropping seasons, ANAM produced WCS, including agro-meteorological advice and weather and climate forecasts. They were disseminated to farmers, in addition to training and awareness seminars, through rural radios and text and voice messages.
2.2. Evaluation Approach
As suggested by Perrels et al. [
25], the evaluation method builds on the Weather Service Chain Analysis (WSCA) breaking down the flow from information generation to outcomes. WSCA is then applied to our specific use case: rainfed agriculture in the Municipality of Tenado. A use case is a description of what a specific user in a particular situation is looking to do or know to improve their business. The underpinning logic model represents the functional relationships between all the activities aimed at modifying farmers’ decision-making in order to improve it and obtain social, environmental and economic benefits. As suggested by Vogel et al. [
18], the logic model can be represented as follows in
Figure 1.
In the case of CREWS, the use of a logic model is particularly useful. It explicitly defines the stepwise logic according to which WCS aim to effect changes in the decision-making process. Moreover, it allows each step to be tested against observation. Indeed, verification in stages is particularly important because the contribution of each specific activity to the overall change in outcome can be difficult to discern. In our case, once it is observed that WCS (i) has a suitable quality to be disseminated, (ii) are properly accessed by farmers, (iii) are understood and trusted, the=is study aims to assess socio-economic and environmental impacts. Therefore, evaluation of the WCS chain will make it possible to interpret and establish a causal link between services and observed impacts using the explanatory variables of the logic model (
Table 1).
In the case of this study, we used an ex-post evaluation method for the 2019 and 2020 agricultural seasons in the Tenado pilot site, which benefited from the WCS provided by ANAM. The methodology used for the evaluation had already been tested in 2015 [
4] and 2016 [
14] in Niger, Mauritania, Ghana and Ivory Coast in the framework of the METAGRI (METeorological AGRIculture) project. The adopted ex-post evaluation is counterfactual, using a comparison between a treatment group and a control group, with the two groups having the same characteristics at the start of the study. The evaluation of treatment and control groups in the same system makes it possible to minimize climatic variability over time. In any case, the non-exclusivity of the information could make it difficult to compare decisions between the treated and untreated groups, because it complicates the identification of the control group without access to information, such as asked for in the randomized trial. Consequently, this approach was adopted on the assumption that the treatment is not the access to information but rather training in the use of the information itself. The control group is random and represents the entire population. The treatment group is also random, from the participants in training.
2.3. Dataset
The surveys were conducted in October 2019 and in November 2020 using structured questionnaires translated and administered in the local language by trained enumerators. Moreover, yield plots were placed in farmers’ fields and yield was measured on site. The sample at the Tenado site consisted of surveys from approximately 300 farmers (150 treated and 150 control) in 12 villages for each year.
Table 2 summarizes the quantitative variables used in the analysis for the whole sample, 2019 and 2020 considered jointly.
The market price of sorghum and the cost of seeds, fertilizer and labor were considered stable over the 2 years and in particular the sorghum price was 200 FCFA per kg, while the costs were 350 FCFA for 1 kg of seeds, and 450 FCFA for 1 kg of fertilizer. Labor cost is accounted as the opportunity costs of working time considering that 1500 FCFA is the daily wage in agriculture. Income was computed by multiplying yield by the price of sorghum, while the gross benefit is the income minus the costs.
Table 3 shows the same variables by year of the survey. Almost all variables show minimum variation over years except for the fertilizations. Despite the costs decrease, the gross benefit in 2020 is lower than that in the previous year owing to lower yields.
2.4. Statistical Methods
To assess the impact of WCS on trained farmers, we computed the Intention to Treat (ITT) effect, this allows to tackle the problem of possible imperfect compliance, or in other terms whether individuals do not follow their treatment assignment, pilot or control group. To this end, we estimate Ordinary Least Squares (OLS) regressions where the outcome variable
is in turn, yields, benefit, income, costs and the use of the following inputs: seeds, fertilizer (measured in kg), and labor (measured in working days) of the individual
i at time
t [
26].
Our variable of interest is the dummy assuming value 1 when the individual i has been selected as a participant in the training offered by ANAM at time t. Since yields, income and costs may be affected by some effects related to the year in which data were collected, we include the dummy that is coded 1 depending on whether the observation refers to 2020.
As a result, the model is the following
where
represents the variation of the outcome variable (
) when the variable of interest (
) is equal to 1. In our case, it measures the difference in the means of yield, benefit, income, costs, use of input (seeds, fertilizer, and labor) between farmers in the pilot (
) and control group (
). As a result,
indicates the impact of the treatment (the training). Finally,
is the constant term and
are the robust error terms. Regarding the outcome variables, this allows us to understand whether participation in the training has improved farmer sustainability in socio-economic and environmental terms.
The hypothesis to be tested is to find a positive and significant relationship between the training on WCS and the yields and thus between the training and income. On the contrary, we expect a negative and significant relationship between the training and costs. As a consequence, we expect an increase in the gross benefits as the overall effect.
Considering the yield, we control for other possible variables that are likely to affect it and thus we re-estimate Equation (1) including the following covariates: kg of seeds and fertilizers used and the number of working days. Since they do not have significant effects on the outcome variables, results are presented in
Appendix A.
3. Results
Results are described according to the intervention logic model developed for the case study.
3.1. Quality of WCS
ANAM conducted statistical evaluations of the produced seasonal forecasts (SF) after the end of the 2019 and 2020 seasons using forecast performance scores. In the study area, probabilistic forecasts for both periods (June–July–August and July–August–September) correctly forecasted the observed 3-month cumulated precipitation (
Table 4). In 2019, a normal season had been forecasted, and it was observed, while in 2020, an above normal season was forecasted and observed. We can therefore assume that the quality of WCS is suitable for dissemination to users.
3.2. Access
The results of the surveys show that 86% of trained farmers have regular access to understandable WCS during the 2019 and 2020 cropping seasons from May to October (
Table 5). A slight difference is nevertheless observed between the two years, with a stronger percentage of farmers having access in 2019 rather than in 2020. This difference is probably related to the nature of the rainy season: since 2020 was an above normal season, farmers felt less the need to have agrometeorological information that is generally more useful in case of a dry season. The average gender distribution is 65% males and 34% females.
Information is accessed mainly through community radios’ broadcast, followed by roving seminars, direct contact with extension agents and text messages.
3.3. Uptake
During the three years of project activity, the users’ perception of provided services evolved slightly. Indeed, at the beginning, the project faced resistance from many farmers who were used to base their decisions solely on their empirical knowledge. However, the project’s willingness to facilitate the integration of local knowledge of climate information has been very helpful in convincing reluctant farmers. Today, in the project area, farmers no longer just wait for climate or weather information, but go in search of it. Informal feedbacks from users suggests that in Tenado, AS are now considered more than an agricultural input. Seasonal forecasts are eagerly awaited at the beginning of the season, as well as climate information for monthly, ten-day, weekly and daily monitoring, and special notices and alerts. This is made possible by the very strong collaboration of ANAM with the agricultural extension services at central and regional level, farmers’ organizations and the local community radio (la voie du Sanguié).
3.4. Action
Action by farmers refers to strategic or tactical choices, the first being essential to properly plan the agricultural campaign and the latter necessary to optimize the resources used. In the sorghum rainfed system, 84% of farmers with access to the information used SF to choose crop varieties and toposequences to exploit and to prepare the land; 83% of them also declared to have used SF to choose the seeding time, and finally 82% used weather forecasts (WF) for other cropping practices. In contrast, market gardeners are not yet as prone to use AS, indeed, only 55% declared to have used SF for preparing the nurseries (
Table 6).
3.5. Impacts
Table 7 shows the means of the variables by control and pilot groups for the whole sample (2019–2020). The quantity of seeds, fertilizer and number of working days decrease drastically among farmers in the pilot group. The effective use of WCS by trained farmers is likely to have improved input management skills.
The main impacts on-farm management are summarized in
Table 8. The odds column shows results without controlling for the year, while the even column includes them. The Pilot row reports the coefficient
β2 in Equation (1), which represents the difference in the mean of the dependent variables (columns) between the control and treatment groups. The Constant row reports the coefficient
β1 in Equation (1). Considering column 1, farmers in the Pilot group show yields 408 kg/ha higher than those in the control group. As for column 3, income in the treatment group is 81,664 FCFA/ha higher than in the control one. On average, the costs incurred by farmers in the treatment group are 58,359 FCFA/ha lower than those of farmers in the control one because of a net reduction of inputs amount (
Table 9). Savings result in a reduction of production costs, while increased yield ensures an increase in income, and, as a result, benefits of pilot farmers (
Table 8, column 7) are 140,023 FCFA/ha higher than those of farmers in the control group, corresponding to an increase by 267%. Such a large difference should not be a surprise, because an important contribution comes from the opportunity cost of labor and fertilizer that pilot farmers saved thanks to WCS (
Table 9). Globally, results show that WCS impact productivity positively by improving trained farmers’ tactical and strategical decisions. Gross benefit results are higher in the Pilot group, indicating that WCS increase cropping profitability by reducing input overuse and increasing yields. Moreover, the reduction by 50% in fertilizers quantity represents an important environmental impact, reducing soil and groundwater pollution.
The impact of the treatment is stable even when introducing the second variable “year 2020” that controls for interannual variability. For example, column 2 of
Table 8 shows that the difference in yields between Control and Pilot groups remains 408 kg/ha even if, on average, yields in 2020 were 151 kg/ha lower than in 2019.
5. Conclusions
WCS cannot be evaluated as a simple product, but they need to be addressed as a process through which producers and users engage to identify needs, develop procedures, test and refine the service, including broadcasting channels and also establishing long-term relationships and trust [
9]. Defining goals, priorities, responsibilities, expectations are part of the success of such a process. Multiple actors are involved, i.e., NMHSs personnel, agriculture extension officers, media and farmers, and their collaboration and cooperation facilitate the adoption of [
21] and trust in WCS [
41]. The evidence suggests that in Burkina Faso barriers to the uptake and use of WCS have been overcome, building a collaborative environment enabling farmers to fully profit from weather and climate information. This research shows that farmers use WCS mainly for choosing crop varieties and sowing dates, and that climate information is most useful when farmers have access to appropriate response options such as improved crop varieties and quality seeds, which could further boost crop yields. WCS also help to rationalize expenses and working time. Farmers are avoiding wasting seeds or fertilizer due to predicted rainfall as well as the wrong time to plow and weed, which could lead to erosion, or an immediate re-emergence of weeds. The reduction of fertilizers use has also positive environmental effects reducing water and soil salinization. Moreover, some farmers in Tenado are trying to assess their debt capacity for the cropping season based on seasonal forecasts.
Nevertheless, in order to scale-up WCS in West Africa barriers and enablers to the use of climate information need to be further explored, particularly related to gender issues. New approaches are needed identify the role these factors play in influencing access and use across different production systems and cultural environments. According to Vaughan et al. [
3], communication strategies (e.g., messaging, visualization, etc.) and dissemination mechanisms (e.g., radio, SMS, social media, etc.) are critical factors whose role needs to be further studied. Moreover, the sustainability of WCS over time has rarely been evaluated and new business models for WCS in SSA released from the development projects approach should be explored.