Methodological Reflections on Monitoring Interactive Knowledge Creation during Farming Demonstrations by Means of Surveys and Observations

Abstract

During farming demonstrations, peer-to-peer learning is known to be more effective than technology transfer when encouraging farmers to consider adopting more sustainable farming practices. Interactive knowledge creation has the potential to create a stimulating peer-learning environment focusing on the use of hands-on activities, knowledge scaffolding, discussions and negotiation. This study investigated how insight can be gained about the interactive knowledge creation that occurs during farming demonstrations by monitoring and evaluating a diverse sample of farming demonstrations in Belgium, Spain and Denmark via surveys and observations originally designed for the AgriDemo-F2F project. The study found that the selected monitoring tools provided insight about how participants experienced specific interactive knowledge creation. However, several stumbling blocks were also identified in using the proposed tools to monitor these learning processes, including the monitoring of abstract concepts and the reluctance among farmers to respond to self-administered open-ended survey questions. Based on these learning points, several proposals were made to improve the monitoring process of interactive knowledge creation. This study confirms that the improved understanding of learning practices and their impact on actual change presents a challenge, but it is essential if the adoption of sustainable farming practices is to be increased.

1. Introduction

Climate change and resource degradation highlight the need for a transition towards more sustainable farming systems [1,2,3]. ‘Sustainable farming systems’ are a dynamic concept viewed by some as a response to so-called “wicked problems” [4] that are ambiguous, with no ‘right or wrong’ solutions, and are bound up in conflicts of interest between multiple stakeholders. Sustainability is widely acknowledged to include economic, social and environmental dimensions, but these are not rigorously theoretically described [5]. Each of these dimensions may therefore be understood in various ways, depending on the domain in which they are applied [6]. In other words, initiatives aiming to increase sustainability need to accommodate potentially conflicting values, beliefs and points of view about so-called ‘desirable’ and ‘feasible’ solutions. Furthermore, the development of sustainable farming systems that still meet current market conditions is a highly complex task. It requires the development of new solutions, as well as their practical implementation by farmers and other actors in the sector [7,8,9]. The ability of farmers to adapt to changing conditions is key to sustainable farming systems [10,11]. Learning about and teaching sustainable farming therefore poses an educational challenge [6].

It is clear that farmers’ learning is crucial to innovation and change in the agricultural sector [12], hence several projects have been funded through the Horizon2020 programme to investigate the improvement of farmer learning in demonstrations (e.g., PLAID, AgriDemo-F2F and Nefertiti). In these projects, farming demonstrations are understood to be activities or events that provide farmers with “an explanation, display, illustration, or experiment showing how something works” (Collins English Dictionary) that can be subsequently applied in their own farming practices, to bring about positive change on their farms [13,14]. The aim of this study was to investigate a specific element of farmer learning at demonstrations, namely interactive learning processes.

To substantiate its relevance, this study adopted the broad definition of learning rooted in educational science, as constructed by Illeris [15], that it is “any process that in living organisms leads to permanent capacity change and which is not solely due to biological maturation or ageing”. Illeris’s contemporary and comprehensive theory on learning has been applied in subfields such as adult learning, workplace learning and the deep level of transformative learning [16]. According to Illeris [15], “all learning includes two different processes: an interactive process between the individual and the environment, and internal mental acquisition and processing through which impulses from the interaction are integrated with the results of prior learning”. The dimension of interaction includes action, communication and cooperation, with social interactions as part of its core. This study focused particularly on the crucial dimension of interaction by investigating interactive knowledge creation (IKC) activities, as facilitated through the attendance of farming demonstrations. These activities deserve attention because there is growing awareness of the limits of transfer-of-technology, which has been the dominant method used to transfer generic knowledge and skills from farm advisors to farmers, primarily using a one-way, top-down approach. Instead, research has indicated that a more reciprocal peer-to-peer type of learning is an efficient way for farmers to learn about and adopt sustainable innovations [12,14,17,18,19].

For example, in the UN Decade of Education for Sustainable Development (2005–2014) Final Report, the authors call for “formal and non-formal education processes to develop critical analysis and decision-making skills that will help to deal with complexity and rapid change. Whole-systems thinking, together with multidisciplinary knowledge needs to be encouraged, as well as social, multi-stakeholder learning approaches” [20]. The core processes within the Education for Sustainable Development (ESD) frameworks and practices are collaboration and communication (including multi-stakeholder and intercultural dialogue), as well as applied learning, such as experiences and processes of active and participatory learning [21]. Similarly, Tilbury [21] comments on the classic sustainability model by stating that “sustainability is about challenging and transforming our mental models, policies and practices”. Individual adult learning and personal transformation can act as a foundation for encouraging environmentally sustainable behaviour [22], which the authors of the present study claim is also true in the agricultural sector. Fostering transformative learning [16] is seen as teaching for change, with learners questioning their deeply-held assumptions and being subsequently changed by the experience [23]. Taylor and Cranton [23], who undertook an in-depth investigation of transformative learning in the context of farmer field schools, which are a thoroughly organised participatory farmer education methodology, state that “core elements fostering transformative learning include individual experience, critical reflection, dialogue, engaging in other ways of knowing and awareness of context”.

Based on existing knowledge about participatory learning in agriculture, Cooreman et al. [24] developed “a conceptual framework to investigate the role of peer learning processes at on-farm demonstrations in the light of sustainable agriculture” [24]. This framework states that IKC activities represent one of three core processes in learning at farming demonstrations. IKC activities underpin the importance of interaction between the learner and the learner’s (social) learning environment, within which knowledge can be shared and constructed.

The present study particularly took a closer look at IKC activities in terms of (i) hands-on activities, a form of observable applied, active learning; (ii) knowledge scaffolding, as a method to support linking prior knowledge to the new demonstrated knowledge; (iii) discussion; and (iv) negotiation, stimulating dialogue and critical thinking. These activities represent a first step towards transformative adult learning, which is advocated in education for sustainable development. For this reason, the authors argue that an understanding of these learning processes is importan in order to improve the facilitation of transitions towards more sustainable farming.

Apart from underlying some of the broader processes defined by Tilbury [21] and Taylor and Cranton [23] as fostering transformative learning and education for sustainable development, the subcategories of IKC activities are substantiated by several other sources. Farmers identify “learning by doing” via real-life, interactive, hands-on activities as one of the most preferred and successful ways to learn [18,25,26,27,28]. Hands-on activities can include activities such as examining soil samples and plants with their own hands, or actively trying out new machinery. Being able to link new knowledge to prior knowledge is a well-known and effective strategy in adult learning [29]. Knowledge scaffolding means that a “more competent other” presents the content to be learned and helps the learner assimilate chunks that are small and clear enough to be comprehensible for the learner, but novel enough to help the learner reach a new level of knowledge or skill [30,31]. This process refers to the ability to link new knowledge to what is already known, making the new knowledge more accessible and thus facilitating the learning and retention of new knowledge. The roles of “more knowledgeable peer (or other)” and “learner” can switch at any time, depending on the topic [32]. The processes of the bottom-up negotiation of knowledge and the sharing of values through open dialogue and discussions are also widely recognised as being crucial for learning about and adopting sustainable agricultural practices [6,21,28,33]. Vandenabeele and Wildemeersch [34] similarly state that environmental learning is “above all, a process in which people are prepared to be surprised by the points of view of others and to face the ambivalences that result from this”. The guidance and facilitation of discussions, and assistance with negotiations of conflicting points of view are therefore suggested to support the learning process [6,35,36].

Despite the acknowledgment that IKC might help to foster sustainable transitions among farmers, there is very limited knowledge about whether and how such activities occur at farming demonstrations. To fill this knowledge gap, several tools were developed in the AgriDemo-F2F project based on the analytical framework [14] and conceptual framework of Cooreman et al. [24]. They suggest monitoring demonstrations through a combination of observation tools and surveys, combining quantitative and qualitative data, as well as multiple perceptions. This mixed method allows greater insight into how IKC takes place and increases the validity of the findings [37].

Building on what had already been undertaken during the AgriDemo-F2F project, the objective of the present study was to apply the tools to an even broader range of farming demonstrations, to evaluate the ability of survey and observation tools to monitor IKC activities specifically.

2. Materials and Methods

2.1. Case Study Description

Seven farming demonstrations were selected as case studies to test the survey and observation tools (

Table 1. Presentation of demonstrations in Spain, Denmark and Belgium, including the number of participants and survey respondents.

Case	Country	Topic	Description	Number of Participants (Approx.)	Respondents to Survey	Respondents (Farmers) 1
ES1	Spain	Species mixtures	Workshop held for a farmers’ group on species mixtures, including a demonstration of species mixture trials of ReMIX on one of the farmer’s fields. The group did an in-field evaluation of the trials and discussed the potential of species mixtures among farmers.	10	10	10
ES2	Spain	Farm machinery	Farm machinery demonstration, DemoAgro, to show innovation in agricultural machinery for plant production. At the demonstration, participants could visit stands to talk to manufacturers, watch a test run of machinery and try out the machinery themselves.	2000	31	28
ES3	Spain	Organic livestock farming	Part of a course in agroecology offered by the Public University of Navarra for farmers, agronomists and other interested parties. The students visited an experimental farm with organic production of sheep and dairy (cheese). Explanations were given about grazing, dairy production, commercialisation of organic products etc.	20	15	3
DK1	Denmark	Species mixtures	Workshop on species mixtures at a farm of a member of the Danish ReMIX farmers’ group. The event included a demonstration of species mixture trials on the farm, in-field evaluation and discussion on possibilities of using species mixtures among farmers.	8	5	5
BE1	Belgium	Agroforestry	Event presenting a whole farm approach on 50 hectares with agroforestry examples spread across the farmer’s land. No specific materials or tools/techniques were used/shown. There were several examples of agroforestry combinations.	40	4	4
BE2	Belgium	Mechanical weed control in maize	Demonstration of approximately eight machines on a maize field belonging to a farmer who had recently converted to organic production. The advisor guided the demonstration of the different machines by explaining them and showing the differences between them one by one.	100	22	17
BE3	Belgium	Innovative dairy farming	Presentation by advisors on a calculation tool for optimising dairy farm management in a meeting room above the barn. Afterwards, the farmer gave a guided tour of the farm and the new barn, including a demonstration of new technologies (e.g., milking robots).	40	15	14

¹ The sample is referred to below as ‘demonstration participants’, even though it does not include all the participants.

). The aim was to use a heterogeneous sample of cases to explore the usability of the tools in different types of demonstrations (regarding the number of participants, activities, topics etc.) under different conditions (political, social, economic, cultural, geographical, climatic etc.). To select the specific cases, the number of participants was chosen as the main criterion, as this is known to affect the organisation of possible IKC activities [38]; for example, organised IKC activities are usually easier to carry out in smaller groups averaging fewer than 20 participants [14,38]. For the present study, two small demonstrations with <20 participants (ES1, DK1), two medium demonstrations with 20–50 participants (BE1, BE3) and two large demonstrations with >50 participants (ES2, BE2) were monitored.

Furthermore, two demonstrations with several participants who were either not farmers or were a combination of farmers and university students (ES3 and BE1) were also monitored. This additional criterion was chosen because an initial analysis of the tools indicated that a mix of participants seemed to stimulate discussion and negotiation at farm demonstrations [38]. As this study focuses on farmers’ experiences of IKC activities, responses from non-farmers were left out of the analysis of the surveys (see last column in Table 1).

The demonstrations were made accessible for the authors through research projects in which they were enrolled at the time the study was carried out. Prior involvement in the projects allowed the researchers to undertake the monitoring process despite this requiring interrupting the participants at the demonstrations.

All the demonstrations were held on a commercial farm, an experimental farm or at field research trials located on farmers’ fields. In ES1, ES3 and DK1, the facilitator already knew the participants because they were part of an ongoing group or course. The majority of the participants in all the demonstrations were farmers, except in BE1 and ES3, as described above. The demonstrations monitored for this study are presented in Table 1.

2.2. Data Collection

The demonstrations were monitored using tools developed within the AgriDemo-F2F project. For the complete development process and the exact set of questions in each tool, reference was made to the ‘Methodological guide for data gathering and analysis: structural characteristics, functional characteristics and impact assessment’ [39], which is publicly accessible on the project website (https://agridemo-h2020.eu/publications-deliverables/). All the subjects gave their informed consent for inclusion before they participated in the study (the informed consent form template can be found in the methodological guide [39]).

All the demonstrations were investigated using (i) an observation tool completed by researchers and (ii) post-event surveys for participants that were translated into the local language and handed out after the demonstration. Surveys allow the collection of large amounts of data with limited resources, and are traditionally used for larger group sizes [40]. The observation tool allows more in-depth qualitative explanations of activities or phenomena [41]. The two monitoring tools were selected so as to be able to triangulate between the perspectives of both participants and observers, and combine quantitative and qualitative data [42].

The Belgian cases analysed in this study were monitored as part of the AgriDemo-F2F project, which investigated a range of learning processes at demonstrations. As the present study focused on IKC in particular, the demonstrations in Spain and Denmark were monitored using a modified version of the tools [39]. The survey and observation tools were shortened to focus on the particular questions which investigated elements of IKC. Thus, some questions that were not relevant to IKC activities were omitted to minimise the effort required from participants, because survey length is known to affect the response rate and answering of open-ended questions [43]. These questions focused on other aspects of the conceptual framework [24], such as the level of reflective thinking (e.g., “I thought about why I want to learn about the topic(s) of this demonstration”) or structural characteristics (e.g., “I think the day was well structured” or “Did you have to pay a fee to attend the demonstration”). All the questions analysed in this study were the same in all cases.

The observation tool was designed for one or preferably two researchers to investigate learning processes at the demonstration. The items of the observation tool were designed as a general rubric with an analytical scoring approach [44] and open questions. These items consisted of four ordinal levels, with each level containing a quality definition to ensure their validity. A fifth option, ‘not applicable’, was added. Apart from the rubric items, the observation tool included 12 open questions.

Table 2. Question on hands-on activities from the observational tool completed by the researcher.

		Circle the answer that fits best + give reasons in the box below (definitions are sometimes subjective, but are meant to be indicative of the variables on which we want data. No right or wrong answers!)					If ‘not applicable’, circle this box and give reasons for your decision in the next box
Hands-on activities (participants were asked to do something interactive with material related to the topic, other then looking and listening (e.g.， try out a machine, test a tool)	Demonstrator/ organiser	No hands-on activity was demonstrated.		A hands-on activity was demonstrated, but only very briefly.	A hands-on activity was demonstrated long enough to ensure it was clear to every participant.	More then one hands-on activity was demonstrated very clearly/instructively.	N/A
	How many? Describe them.
	Participant	No hands-on activity was carried out by participants.	Participants could take part in a hands-on activity, but did not get feedback on what they did.		Participants could take part in a hands-on activity and did get feedback on what they did.	Participants could take part in multiple hands-on activities and received some sort of feedback on what they did.	N/A
	How many? Describe them. Describe the feedback.

provides one example of the questions contained in the observation tool.

The post-event survey given to attendees was designed to measure interactive knowledge creation (IKC) activities stimulated by attending the demonstration, as well as learning processes such as engagement and reflective thinking, which fall outside the scope of this research. The survey consisted of a combination of closed questions (‘yes’ or ‘no’) and about 40 closed four-point ordinal scale questions from ‘strongly disagree’ to ‘strongly agree’, with the extra possibility of answering ‘not applicable’, with a space to add remarks.

Table 3. Examples of questions on discussion and negotiation from the post-event survey for participants.

	Strongly Disagree (--)	Disagree (-)	Agree (+)	Strongly Agree (++)	Not Applicable	Additional Comments
I asked at least one question during the event (circle).		no	yes		N/A
I shared my own point of view at least once during the event.		no	yes		N/A
I felt encouraged to ask questions during the event.	1	2	3	4	N/A
In my opinion, there were interesting discussions during the event.	1	2	3	4	N/A
When there were any discussions, I felt comfortable sharing my opinion.	1	2	3	4	N/A
I heard at least one discussion in which participants didn’t completely agree with each other.	1	2	3	4	N/A
If participants didn’t agree with each other during discussions, somebody (demonstrator/organiser/other participant) tried to reach a consensus between them.	1	2	3	4	N/A
if yes, what was the role/occupation of this person? (another farmer, advisor, the facilitator, the organisoretc.)

provides one example of questions from the post-event survey for participants.

2.3. Data Analysis

To evaluate the applicability of the monitoring tools, 12 questions in the survey and 10 rubric items in the observation tool were analysed. These included responses to questions about hands-on activities, knowledge scaffolding, discussion and negotiation. The questions in the survey tried to capture participants’ perceptions about the facilitation of hands-on activities, open discussion, conflict negotiation and knowledge scaffolding during the demonstration. All these processes include interactions through which knowledge can be shared [24], for example: “I participated in an interactive experience, (In the survey, “hands-on activity” was translated as “interactive experience” and several examples of such experiences were given to clarify the concept [45]) during the demo (e.g., trying out machinery, feeling different types of soil, etc.)” and “I asked at least one question during the event” (‘yes’ or ‘no’) etc. (example in Table 2). The ‘not applicable’ answers to the questions investigated were excluded from the analysis. The responses from the surveys (81 responses, response rate: 31%) and observations (7 observations) (see Table 1) are presented in

Table 4. Results from closed questions in the post-event survey, and observations about the hands-on activity and knowledge scaffolding.

Case	Hands-on Activity				Knowledge Scaffolding
	Participants		Observer		Participants		Observer
	I Participated in an interactive experience (on farming practices)		Hands-on activities were demonstrated *	Participants could take part in hands-on activities **	The event built on my current understanding/knowledge		The demonstrator/organizer ***	Knowledge was explained ****
	% No	% Yes			% Disagree or strongly disagree	% Agree or strongly agree
ES1	50	50	a	b	10	90	d	d
ES2	65	35	b	b	0	100	N/A	N/A
ES3	67	33	c	b	33	67	b	d
DK1	20	80	c	b	0	100	N/A	N/A
BE1	100	0	a	a	0	100	b	c
BE2	80	20	d	a	13	87	a	c
BE3	90	10	b	a	14	86	c	d

N/A = not applicable. * a: No hands-on activity was demonstrated; b: A hands-on activity was demonstrated, but only very briefly; c: A hands-on activity was demonstrated long enough to ensure it was clear to every participant; d: More than one hands-on activity was demonstrated very clearly/instructively. ** a: No hands-on activity was carried out by participants; b: Participants could take part in a hands-on activity, but did not get any feedback; c: Participants could take part in a hands-on activity, and received some sort of feedback; d: Participants could take part in multiple hands-on activities, and received some sort of feedback. *** The demonstrator/organiser a: Did not ask participants to share what they already know about the topic; b: Referred to what participants might be familiar with/know already, but did not let them talk about it; c: Asked a few questions at the beginning to let participants share what they already knew; d: Asked about, and referred back to, what participants might already have been familiar with. **** Knowledge was explained a: Unclearly, as a big unstructured new chunk of information; b: In different chunks, but unclearly about how they link together; c: In different linked chunks, but some steps to new chunks were too complicated; d: Step by step, linking to the previous step, making it a little bit more complicated each time.

and

Table 5. Results from closed questions in the post-event survey, and observations on discussion and negotiation.

Case	Discussion										Negotiation
	Participants		Observer		Participants		Observer	Participants		Observer	Participants		Observer
	I asked at least one question during the event		Time for questions *	Number of questions **	I shared my own point of view at least once during the event		Participants formulated their own point of view ***	In my opinion, there were interesting discussions during the event		Open discussions were held ****	Attempts were made to reach consensus in discussions		Negotiating conflict *****
	% No	% Yes			% No	% Yes		% Disagree or strongly disagree	% Agree or strongly agree		% Disagree or strongly disagree	% Agree or strongly agree
ES1	25	75	d	d	14	86	c	13	88	d	63	38	b
ES2	15	85	d	d	4	96	N/A	23	77	N/A	61	39	N/A
ES3	0	100	b	d	0	100	b	0	100	c	0	100	b
DK1	0	100	d	d	0	100	c	0	100	d	0	100	d
BE1	25	75	b	c	0	100	b	25	75	c	33	67	c
BE2	29	71	b	b	7	93	a	21	79	a	38	62	a
BE3	25	75	c	c	38	62	b	31	69	b	57	43	c

N/A = not applicable. * a: No time for questions; b: Little time for questions; c: Some time for questions; d: Plenty of time for questions. ** a: Nobody felt the need to ask any questions; b: Few (3–5) questions were asked; c: Some (5–10) questions were asked; d: Many (>10) questions were asked. *** a: The demonstrator did all the talking; b: A few participants tried to formulate their own points of view; c: Several participants formulated their points of view; d: Almost every participant formulated their own points of view. **** a: No open discussions were held; b: There was time for open discussion, but nobody really engaged; c: Open discussions between a few participants were stimulated; d: Open discussions were stimulated and given plenty of time, and most participants were involved. ***** a: (almost) No critical points of view on the topic were shared; b: No elaboration/further explanation on shared critical points of view; c: Shared critical points of view were clarified/rephrased so that more people could understand; d: It was ensured that everyone understood the shared critical points of view.

.

The answers given in the survey by the attendees were analysed by the percentage of ‘yes’ or ‘no’, or, in the case of the four ordinal scale questions, were clustered in the percentage of disagreements (‘strongly disagree’ (1) or ‘disagree’ (2)) and agreements (‘agree’ (3) or ‘strongly agree’(4)). Responses from the observations are represented in Table 4 and Table 5, with letters referring to the specific rubric items. The analysis is supported by quotes from the open questions of the post-event surveys and observations.

The analysis was carried out using a mixed method analysis approach with qualitative reflection based on quantitative descriptive data. For example, agreements and disagreements were sought in answers among survey responses and between survey responses and observations. A comparison between the responses from the surveys and observations allowed different perceptions about IKC activities to be accessed. Depending on the convergence, it is possible to question how accurate the tools are in capturing IKC activities taking place during farming demonstrations [37]. Apart from reflecting on the accuracy and validity of the tools, this analysis allowed any notable differences to be identified between how participants experienced the IKC activities at smaller demonstrations compared with larger demonstrations, and between demonstrations with mainly farmer participants compared with those with a mix of stakeholders.

3. Results

Selected findings from the analysis are presented below to illustrate how the participants and observers experienced IKC activities during the seven case studies.

First, Table 4 presents an overview of the responses of participants and observers about hands-on activities and knowledge scaffolding during each of the demonstrations.

3.1. Hands-on Activities

In the analysis, a distinction was made for each case between whether or not hands-on activities were demonstrated, and whether or not attendees could participate in a hands-on activity.

The results indicated that the participants took part in hands-on activities. For the two cases with a smaller number of participants (ES1 and DK1), it was evident that more participants answered yes to the question of whether they could participate in an interactive experience than in the other cases.

In ES1, ES2, ES3 and DK1, the observers found that participants could take part in hands-on activities (answer b). However, the participants gave different answers as to whether they took advantage of participating in such an activity.

In BE2 and BE3, participants replied that they were involved in a hands-on activity, despite the observer not registering that the participants were engaging in any hands-on activities (answer a). In BE2 in particular, this number was relatively high (20%). In that case, the observer found that “more than one hands-on activity was demonstrated very clearly/instructively”, which consisted of using a mechanical weeding machine in maize. Comparing the results from the survey with the observations, it can be assumed that the participants were referring to the activity of examining the soil with their own hands after the mechanical weeding machine had passed by. This conclusion was drawn because there was no other mention of any specific interactive activities in the observation of BE2. “Examining the soil with their own hands” was not recognised by the observing researcher when filling in the observation tool as “participating in hands-on activity”, but was mentioned in relation to another question in the observation tool about multisensory activities.

Finally, the observations on hand-on activities in ES3 indicated that the cheese-making demonstration gave rise to several questions from participants. In other words, it was evident that hands-on activities, even if they were only being demonstrated, triggered an element of another IKC activity: discussion.

3.2. Knowledge Scaffolding

In six of the demonstrations, with ES3 as the exception, >85% of the participants said that they felt the demonstration built on their current knowledge. The degree to which the observer thought the demonstrator/organiser referred to participants’ knowledge differed between the demonstrations (answers a–d). Looking at the results, it is striking that 33% of the participants in ES3 thought that the demonstrated content did not build on their current knowledge. Especially as the demonstration was part of an ongoing course, it would have been reasonable to assume a greater continuity between what was being learned in the different course activities. In ES1, ES3 and DK1, the participants and demonstrator/organiser knew each other beforehand (see case study description). In these demonstrations, the observer stated that the facilitator referred to discussions or content from earlier meetings, or to the concrete experiences of one or more of the participants. However, there was no clear tendency in the survey responses that more knowledge scaffolding was particularly taking place in these demonstrations.

In ES2 and DK1, the observer did not feel capable of judging how the demonstrator/organiser was facilitating knowledge scaffolding using the rubric (answer N/A). However, in DK1, the observer noticed that knowledge scaffolding was taking place among peers without being facilitated by the demonstrator/organiser. The same observer noted that knowledge was not necessarily being explained in a step-by-step manner due to the event’s dynamic format. Looking at the description and observations, it was clear that DK1 was dominated by workshop activities, with little presentation by the demonstrator/organiser. ES2 was a large farm demonstration where farmers were circulating between different demonstrators of machinery. In both cases, this explains why the circumstances made it particularly difficult for the observer to discern whether knowledge scaffolding was facilitated.

In BE1, the observer noted that the demonstrator/organiser needed to ensure there was sufficient time for participants to be asked about their current knowledge, and to create room for sharing. However, in BE3, the demonstrator/organiser did allow enough time and asked about the participants’ experiences, but the observer stated that participants were reluctant to share. The same observation was made in this case regarding discussion and negotiation, as explained below.

3.3. Discussion and Negotiation

Table 5 presents the answers from participants and observers about their experience of discussion and negotiation at each demonstration.

In all the demonstrations, it was observed that at least some time was allowed for questions (answer b, c or d), and that participants did ask questions (observer: answer b, c or d; participants: >67%) in all of the demonstrations.

Except for BE3 (62%), a large percentage of participants felt that they shared their own point of view during the event (>86%). However, observers in four of the demonstrations stated that the demonstrators did all the talking (answer a: BE2), or that only a few participants shared their point of view (answer b: ES3, BE1 and BE3). A similar tendency was found in relation to the open discussions. Despite the observer in BE2 not registering any time for open discussion (answer a), 83% of the participants felt that interesting discussions were taking place. In all cases, >69% of the participants felt that interesting discussions were taking place. In four of the cases (ES1, ES3, DK1, and BE1), this was confirmed by observers registering that discussions were being stimulated to some degree (answers c and d).

In BE2, the observer noticed that the size of the demonstration might have an influence on whether participants could share what they already knew, stating that “Since there were about 100 participants, no plenary discussions were held”. However, the number of participants who felt that they shared their own point of view was relatively high (93%) in this case. In comparison, a lower percentage of farmers in BE3 indicated that they shared their own point of view. When looking at the observer’s open answers, it seems that the participants “(…) weren’t willing; time was not the biggest issue”. Despite the lack of engagement during the plenary session, the observer stated that participants in BE3 interacted with one other a great deal while walking around the farm.

4. Discussion

The monitoring of the key aspects of IKC through surveys and observation tools revealed some interesting insight into how IKC was taking place in the case studies. From the data provided by the tools, more than half of the participants across all cases agreed that the demonstration built on their current knowledge, they asked at least one question, shared their point of view at least once and thought interesting discussions were taking place. However, incongruence among survey answers and between surveys and observations was also found regarding hands-on activities (BE2 and ES2), sharing one’s own point of view (ES3, BE1, BE2 and BE3) and whether a discussion took place (BE2). This gives the impression that some IKC elements might be interpreted differently among participants, and between participants and observers. Using the descriptions from the observations, this might be caused by different interpretations of what a hands-on activity is (BE2), for example, or because the observer did not have access to observe all of the IKC activities (ES2) or discussions and negotiations between participants (BE3). This may be to do with monitoring abstract concepts, and a discussion follows below about how this challenge could be addressed.

In the tools used, the different IKC activities were monitored separately, but observations from ES3 showed that there might be interesting correlations between the different IKC activities. This is in line with practice guides on farmer-to-farmer learning during farm demonstrations, which suggest the use of a variety of activities [46,47]. To the authors’ knowledge, exactly how these activities influence each other has not been investigated in this context. For further studies to enhance the understanding of IKC activities, questions and observations should be included about how IKC activities might trigger each other to improve knowledge of the learning processes involved.

The survey responses showed that participants in ES3 experienced less knowledge scaffolding than participants in the other cases (e.g., ES2, DK1 and BE1). This could be related to the diversity of the stakeholders in ES3. However, the mix of stakeholders in ES3 and BE1 did not seem to influence the results of other IKC activities notably. The observations from ES3 do not provide an explanation for the diverging responses in this case. Apart from when demonstrators/organisers referred to participants’ knowledge, looking at the data it is evident that observing knowledge scaffolding is quite complicated, because it concerns participants’ subjective experiences, which are not easy to observe. Open-ended questions could provide insight on this, but very few participants answered them, highlighting another interesting challenge that will be addressed below.

The data suggest that knowledge scaffolding among peers seemed to occur quite dynamically, as observed in DK1. Certain circumstances seemed to be important for IKC activities to take place, as seen in the data when demonstrators/organisers referred to the participants’ knowledge, or when participants asked questions or shared their points of view. However, the data also suggest that the right circumstances for IKC activities are a prerequisite if such activities are to occur (BE3). From other studies, it is clear that trust, acknowledgement and time for discussion are examples of prerequisites for the activation of farmer knowledge and efficient learning [25]. The study is also a reminder that the questioning and sharing of opinions also happens informally, and might be more appealing face-to-face in smaller groups with other peers or organisers than in a large group (e.g., DK1). This might explain the high level of discussions among participants in BE2.

Based on previous knowledge, it is assumed that the number of participants may influence IKC activities (see the case study description). In this study, the smaller demonstrations (ES1 and DK1) seemed more likely to have attendees participating in hands-on activities. However, no such correlations were seen between the number of participants and participation in the other IKC activities. The data obtained therefore do not offer strong indications that the size of the demonstration affected the IKC activities in the monitored cases. However, the size of the case, especially ES2, made it difficult for the observer to monitor, resulting in several ‘not applicable’ answers by the observer (‘N/A’ in 5 out of 9 of the observation rubrics analysed for this study).

Despite the insight on IKC, as shown above, the results also indicate room for improvement to increase the validity of the findings drawn from data collected using the tools. A discussion follows on the requirement and approach to overcome the two main barriers identified in the study: (i) the monitoring of abstract concepts and (ii) farmers’ reluctance to answer open-ended questions. Finally, suggestions are made as to how to improve to the tools for future research.

4.1. Overcoming the Stumbling Block of Abstract Concepts

The evaluation of the demonstrations in this paper indicated that asking about concepts such as hands-on activities and knowledge scaffolding through surveys can be interpreted and experienced differently, depending on the definition and understanding of such concepts by participants. It is crucial that there is consistent understanding of questions in a survey in order to reinforce the reliability of this type of methodology [45]. The findings suggest that asking about abstract concepts through a survey format will bias the results. For instance, the question on knowledge scaffolding “The event built on my current understanding/knowledge” required participants to reflect on and understand the dividing line between their current knowledge and new knowledge, a difference that may be less clear to some participants. Although abstract concepts such as hands-on experience and knowledge scaffolding were exemplified in the survey, different interpretations might still be attributed, which is one of the reasons why the closed questions were followed by open argumentation questions. However, most participants did not answer all the open questions in the post-event survey, which hindered the researchers’ ability to understand how participants interpreted the questions. A possible solution to this would be to avoid referring to abstract concepts in questions, or to ask about them through a semi-structured interview or during a focus group.

One way of responding to this bias in the interpretation of abstract concepts is through triangulation between the data sources using the observation tool [37]. The study revealed that observations can be a very useful way to obtain a clearer view of the dynamics at a demonstration. However, there are some difficulties associated with this. First, the completion of the observation tool requires an understanding of learning theory (such as elaborating on whether “common methods or ways of thinking on learning were questioned” or which “techniques and methods were used” to foster learning). Secondly, especially during larger demonstrations or more scattered activities in smaller groups, it was not possible for the observer to capture every conversation between participants (e.g., ES2 and BE2). This could create a gap between the answers from the observing researcher and the participants; for instance, as seen in BE2 in the answers relating to discussion and negotiation (Table 5). Using multiple researchers at the same time could provide a partial solution to this. Thirdly, some of the questions in the observation tool need to be reformulated. For example, the observer can choose to answer “The demonstrator asks about, and refers back to, what participants might already be familiar with” in the observations regarding knowledge scaffolding in the observation tool. This, among a few other questions, proved hard to judge, as the observing researcher was not likely to know the content with which participants were familiar. This might explain the lack of convergence between farmers’ responses and observations. Thus, questions that either contain abstract concepts or ask for observations that cannot be observed in practice should be rephrased or left out of the tools.

To gain deeper insight into how participants experience learning opportunities at demonstrations, in acknowledgement of the effort required to fill out written surveys, a dialogue between researcher and participant, e.g., through oral, qualitative, semi-structured interviews or focus groups, might provide added value. Such oral exchanges could be used either as a substitute for the surveys or as a supplement with selected participants to gain even more in-depth understanding of the experiences and the connection between IKC activities and learning.

4.2. Familiarity with the Target Group

Research indicates that farmers might prefer closed questions to open-ended questions [48], that the length of the survey might influence their willingness to answer the open questions [49] and that alternatives to written communication, such as video, are acknowledged as an efficient way to foster learning for farmers; for example, [50]. This supports the observation that farmers might be reluctant to formulate answers in written form, and might be less willing to spend a lot of time on filling in surveys.

Unlike the demonstrations where most participants were farmers (ES1, ES2, DK1, BE2 and BE3), most of the open-ended questions were answered at ES3, a university course on agroecology. The acknowledgement that different types of participants (including types of farmers) might be more or less willing to respond to surveys in general [49] highlights a bias regarding the representativeness of the data collected at farming demonstrations.

In this study, the open-ended questions in the surveys were supposed to validate the closed questions [51], and therefore the lack of answers presents a challenge to their possible interpretation. This indicates that an investigation of alternative ways to invite farmers to share their qualitative experiences is needed. In order to obtain a higher response rate for the open-ended questions and strive for a more representative group of respondents, a control mechanism could be included in the participant survey, or organisers/researchers could conduct the surveys as interviews with randomly selected participants.

The study also showed a tendency towards positive feedback in the surveys, meaning that participants tended to agree on the statements presented in the ordinal scale. This might indicate that participants generally had positive experiences. However it should be taken into account that social desirability could also have influenced the outcomes [52]. Although the surveys were anonymous, included closed scale answers and were self-administered, these characteristics may have been insufficient to avoid such bias. Furthermore, satisfaction about participating in what is also a social event might also influence the responses.

4.3. Towards an Improved Process for Monitoring Learning at Farming Demonstrations

After conducting the study, the observation tool and survey for participants were reviewed once more, and both were refined, based on the results and the experience of analysing the answers. By way of example, the refined post-event survey can be found in Appendix A.

Based on what was learned in the present study about the limitations of these tools, several suggestions have been made to refine the monitoring process of IKC activities at farming demonstrations. In further research, it would be relevant to study the relationship between IKC activities and critical reflection, a next crucial step in transformative learning towards change [16] which is considered to be a core process in education for sustainable development [53]. Therefore, it is proposed that some extra, different data-gathering methods are added some time after the demonstration. Increasing the amount of data might improve the understanding of the IKC activities, but at the same time increase the possible variation within responses and observations, thus making it harder for unambiguous conclusions to be drawn. However, we argue that the variation in perspectives captured by multiple tools gives a deeper and more nuanced understanding of the demonstrations [42,51], which is important in order to avoid drawing simplified conclusions about complex IKC learning processes.

Figure 1 provides a graphical illustration of a monitoring process, which can serve as inspiration for the future use and improvement of monitoring tools for learning processes at farming demonstrations. We acknowledge that resources might be limited and there might be a need to leave out some of the suggested tools or monitoring opportunities. The illustration below is provided to encourage future researchers to reflect on how each tool and data-gathering moment might affect the picture of the learning processes obtained.

The learning processes during demonstrations should be monitored through post-event participant surveys (green) (potentially carried out as interviews) and several observations (orange) to provide several perspectives on the occurrence of IKC activities at the demonstration. Ideally, these sources can be supplemented with in-depth semi-structured interviews (yellow) or a focus group (purple) at the end of (or shortly after) the demonstration, to provide a better understanding of the experience of abstract concepts in particular. A triangulation (blue) of the different sources should be made shortly after the event in order to compare and validate the different observations of researchers and participants, in order to gain a more comprehensive understanding of the IKC activities and processes taking place at the specific demonstration, and to avoid misinterpretations. If the aim is to monitor the interrelation between learning and the change of practice, an in-depth follow-up interview or focus group is proposed after six months, for example (as proposed in the original tool), because changing practices among farmers are expected to take time. However, investigating such correlations requires the recognition that changing farming practice is not just a matter of learning during farming demonstrations, but is actually driven by a complex network of opportunities [54,55]. More research should be undertaken to explore these dimensions in order to increase knowledge about how the processes of farmers’ learning are linked to sustainable transition in farming systems.

4.4. Limitations of the Study

The very limited responses to the open-ended questions in the surveys, combined with only one observation from each case study, resulted in data material that was less rich and strong than it might have been.

With further resources, based on the experiences from the current study, the evaluation of the tools could have been improved by using more case studies to increase the convergence of evidence or provide a more nuanced understanding of the cases. Furthermore, an analysis of each of the cases in greater depth, especially using the answers from the observations, could have provided an additional perspective of how the tools offer insight into how IKC occurs. If resources are available, a larger sampling size and more in-depth analysis should be included in the revised monitoring process, as presented above for future studies, to increase the knowledge of IKC and learning at farming demonstrations.

5. Conclusions

The monitoring tools provided insight into how participants experienced the specific IKC activities (hands-on activities, knowledge scaffolding, discussion and negotiation). Based on the results of this study, it would appear that the majority of the attending farmers actively participated in the interactive knowledge creation (IKC) learning activities in most of the seven different farming demonstrations monitored.

Most importantly, the study highlighted several stumbling blocks in monitoring IKC processes using the proposed tools, including the monitoring of abstract concepts and limited responses to open self-administered survey questions. In order to improve the understanding of IKC learning at farming demonstrations, as well as whether and how this leads to changes in practice among farmers, more knowledge is required. To obtain such knowledge, four specific suggestions are made for the improved use of the tools: (i) multiple observing researchers should be employed at the same time to fill in the observation tool, and triangulation between data sources should be used to avoid the misinterpretation of abstract concepts; (ii) questions that either contain abstract concepts or ask for observations that cannot be observed in practice should be rephrased or omitted (suggested refinements in Annex A); (iii) there should be greater motivation to answer open questions in the self-administered surveys, for instance through a control mechanism or organiser/researcher-conducted surveys in the form of interviews; (iv) in-depth interviews or focus groups should be organised shortly after and/or after a longer period, to increase the understanding of farmers’ experiences and link monitoring at farming demonstrations to actual changes in practice.