A Scoping Review on the Impact of Educational Technology in Agricultural Education

Reviewers’ Comments

Response

Reviewer 1

1.            Line 50-51. "and everything planned, methodical, and instructive that takes place outside of the formal system." Does ag need to be part of this statement?

Dear reviewer, thank you for your feedback. In this paragraph, we are defining agricultural education based on previous literature. The sentence quoted in lines 50-51 does not contain the word "ag" or “agriculture.”

2.            Line 133: Remove extra space at the beginning

The extra space has been deleted and can be seen on line 133. Thank you.

3.            The country/origin information is first mentioned in the research questions section. Please discuss briefly in the literature review the importance of including this information. Lines 308-309 mentioned this point, but it would be great to back it up with literature.

Thank you for this valuable contribution. We have addressed the need to include country of origin and other variables, such as context/field, educational level, and journal information, in lines 177-190. Bond et al.'s (2018) comprehensive review strongly supports this.

4.            Similarly, explain briefly the significance of revealing the trend of each methodological feature.

We have addressed this in lines 191-201, bolstering the importance of including methodological features such as research methods, sample size, data collection, and type of instrument from studies like those of Cheung and Slavin (2012), Burback et al. (1999) and Mueller et al. (2018). Thank you.

5.            I appreciate the background and definitions provided in this manuscript.

Thanks

6.            Results are well organized and presented.

Thanks

7.            Line 575-588. You need to provide justification on why the stat results show insignificant, but your observed differences are still worth noting.

Thank you for providing us with valuable feedback. Though we did not find statistical significance between the two variables, we witnessed some differences between the type of educational technology and subjects based on descriptive statistics. We provided our reasoning in lines 652-668.

Reviewer 2

1.            It is suggested that Section 4.3 and 4.4 should be swapped, because 4.4 and 4.2 are more closely related, and the operations stated in 4.3 should logically follow the contents of 4.4.

We appreciate this suggestion, and this has been done as suggested, as seen in lines 324-344.

2.             The serial number is incorrect. According to the current structure, the numbers of 4.4.2 and 4.4.3 should be 4.3.2 and 4.3.3.

Thanks for the observation. This has also been corrected, as may be seen in lines 353, 375, and 395.

3.            The figures and tables in the article can be further standardized and beautified.

Thanks for this suggestion; most of the figures are direct outputs of the analytical software (SPSS) that was used and may therefore be difficult to alter; however, Table 2 (lines 452-453) has been modified to make it more presentable.

4.   In the process of literature retrieval, in addition to gradually screening and removing documents, there should also be a process of supplementing documents that may be omitted. For example, in order to prevent the omission of important documents due to the problems of search terms or database settings, it is necessary to manually read some well-known agricultural technology journals, educational technology journals, and journals that have frequently published papers on related topic in the past 20 years to find important documents that may not be retrieved by machines. The effectiveness of the search procedure can be confirmed even if the manual search fails to find new documents that need to be added.

Thank you so much for your invaluable comment. We actually took this into consideration. The five databases we used (CAB Abstracts, AGRICOLA, ERIC, Education Source, and Web of Science) were chosen for their reputation of being comprehensive in the fields of agriculture, education, and educational technology. First, we began by manually searching reputable databases such as Google Scholar to identify as many relevant articles as possible and this helped us develop the search strategy. After this, we ran a pilot test of the literature search using the developed search terms and checked if all the articles we had identified manually were included. Those articles that were not included in the pilot test results were examined for the missing keywords which helped us refine our search terms for the search strategy. It was after this that we ran a very comprehensive search that covered our included studies. These databases also ensure that the studies we include have been peer-reviewed, which is one of our critical inclusion criteria. as noted in lines 283-293.

5.     Why is Figure 2 not plotted as an annual growth trend? The data statistics are only divided into two stages, which has rough implications for subsequent researchers. After all, in fact, the number of publications in the two stages is not very large, and also the gap is not very large. Moreover, if the data is annual, it can show more complex growth trends, such as linear, U-shaped or inverted U-shaped. Different trends have different inspirations for subsequent researchers.

Thank you for sharing your insightful observation. We were also interested in examining the annual growth trend of educational technology usage in agricultural education. However, after conducting a Chi-square cross-tabulation analysis of the years against the type of educational technology, we only observed statistical significance when the years of publication were grouped into two decades. Therefore, we thought it necessary to adhere to the two-decade growth trend analysis, supported by Table 3 (lines 650). This will indeed be an interesting angle and perspective for our future research as well as for other researchers working on this subject.

6.     In Table 2, when the number of published papers in each journal is calculated, if the topic preference and method preference of published papers in different journals can be analyzed in detail from the text, it will be more enlightening for subsequent researchers.

Thank you for your suggestion. We appreciate the detailed angles you have presented and would like to explore them further. However, our approach to this review has been influenced by the seminal work of Munn et al. (2018), who differentiate between scoping and systematic literature reviews. According to their research, scoping reviews are intended to provide an overview or map of the evidence, rather than a critically appraised and synthesized result/answer to a particular question. Therefore, we have focused on creating a comprehensive overview that aligns with our research questions.

7.     Similarly, when analyzing Figure 3, if the authors can carefully compare the thematic focus of studies in different regions and the differences in main research topics, they can provide more inspiration for future research directions for authors in regions with fewer publications.

Thank you for your suggestion. We wanted to clarify that we purposely avoided making our research findings a systematic literature review, as explained in point 7 above. Instead, we aimed to provide a comprehensive overview that meets the criteria of a scoping review but does not unintentionally become a systematic literature review. We touched this topic in our added future directions section.

8.     The analysis of Figure 4 is not only a quantitative presentation, it is suggested that the authors further analyze the research themes and focus of different disciplines to provide more inspiration.

We are grateful for these invaluable suggestions. We have worked on this as may be seen on lines 485-499.

9.     The most valuable section of this paper should be the proposal of future research directions, but it is obvious that the author's elaboration in this section is extremely insufficient. It is suggested that this section should be independent from the conclusion, making a separate section with a large number of details for summarizing potential research focuses.

We are extremely grateful for the reviewer's excellent suggestions. We have taken these concerns seriously. In lines 671-751, to address these concerns, we have separated the Conclusions from the Future Directions section and have provided more extensive elaboration on the future directions. This has resulted in a more insightful and thought-provoking piece.

Reviewer 3

1.              Title: indicate from the title that it refers to a documentary review.

We greatly appreciate the suggestions made by the reviewer. Our research is a scoping review, which was defined by Munn et al. (2018) as an effective way to determine the coverage and scope of literature on a particular topic. Scoping reviews provide a clear understanding of the amount of literature and studies available, as well as an overview of their focus. They are also useful for examining emerging evidence when it is still unclear what specific questions can be addressed by a more precise systematic review. We have meticulously followed the standards of a standard scoping review, as reflected in our research title. While we considered including a "documentary review" in our title, we ultimately decided that it would contradict the goals and objectives of our study.

2.              Abstract: it is necessary to indicate the objective of the study, why it is important to analyze it, before presenting the results. The results seem to be very limited, could you add something more? because it seems to be a mapping of literature. Therefore, indicate the methodology used, at least describe it briefly.

We would like to thank the reviewer for their excellent suggestion. While we were limited by the word count set by the journal, we were still able to include the purpose of our scoping review in the abstract. However, we have taken your feedback into consideration and have revised the purpose of the scoping review to make it more apparent as seen in line 8.

Concerning the results, we discussed our findings under three major headings which aligned with our research objectives namely: substantive features of the studies; methodological features of the studies, and characteristics of educational technology in agricultural education. Under these headings, we delved in great detail into various attributes of the 83 included studies. We discussed the article type, publication information, country/region of the study, course subject, and education level all belonging to the substantive features of the study (lines 437-506). We also discussed the methodological features of our 83 included studies covering the research methods, sample size, data collection approaches, the type of instrument (lines 507-569), and the characteristics of educational technology in agricultural education (lines 570-656). These sections are not only in tandem with the standard scoping review’s recommendations but have also been reiterated by other scoping reviews such as those of Peterson et al. (2017), AArksey &O’Malley (2005), Pham et al. (2014).

Concerning the methodology, a scoping review, which is different from a mapping review, has a standard guideline in terms of the methodology that must be followed. Arksey & O’Malley (2005), which is one of the most seminal works on scoping reviews have itemized the steps that must be followed in conducting a scoping review which are: identifying the research question; identifying relevant studies; study selection; charting the data; and collating, summarizing and reporting the results. All these steps have been duly followed and described in great detail in our manuscript.

3.              Introduction: The studies cited in the bibliography (1-8) are not recent, and the field of technology use in education has witnessed significant advancements, particularly in recent years. The background requires improvement by referencing up-to-date research and identifying the gaps in the current literature that this study aims to address.

Once again, we appreciate the reviewer for these great suggestions. The oldest study cited between 1-8 was by Reiser and Ely (1997) which however is not among the included studies. Since one of our inclusion and exclusion criteria was the date of publication of the studies (2000-2022), it was important for us to understand what the prevalent state of the use of educational technology in agricultural education was before 2000, hence, the need to go way back so as to fully capture the journey so far and that was duly covered in the literature review section (lines 267-323). The standard Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Scoping Reviews (PRISMA) flow designed by the JBI (Peters et al., 2020) which is a must for scoping reviews was also included in our manuscript (lines 335-336).

4.              Results: I suggest expanding the explanation of the type of technologies by adding application examples. I do not find the usefulness of drawing statistical correlations, what would be the significance of this analysis.

Thank you for your suggestions. We have taken care of these concerns in lines 578-592.

The main objective of our research was to determine the impact of educational technology on learning outcomes. We aimed to identify if the impact was positive or negative. Additionally, we investigated the two decades that witnessed the most significant increase in the use of educational technology and the sub-disciplines within the agricultural sciences field. Therefore, the cross-tabulation analyses were crucial in achieving our research objectives.

5.              In the discussion it is important to add who benefits from this research, what is the contribution and what are the possible lines of future research.

We are extremely grateful for the reviewer's excellent suggestions. We have taken these concerns seriously which were also raised by another reviewer. In lines 671-751, to address these concerns, we have separated the Conclusions from the Future Directions section and have provided more extensive elaboration on the future directions. This has resulted in a more insightful and thought-provoking piece.