Pedagogical Design in Technology-Enhanced Language Education Research: A Scoping Review

Round 1

Reviewer 1 Report

There are many types of technologies that are used in technology-enhanced language learning and teaching like gamification for language learning skills, Corpus, computer-supported corrective feedback, VR for language learning, computer-based speech recognition, collaborative writing tools, lexical annotations, computer-based intelligent tutoring, etc. There are a lot of publications in international journals covering many such topics. The authors should consider including some of these topics in the review.

The authors say that 51 articles published after 2012 are included in the study, however, there are some articles in the references list published before 2012, and there are only 49 total articles in the list, therefore, the reference list seems incomplete.

In the discussion of language teachers’ practice and experience of applying technology, it is not clear what the differences are between online and classroom instruction. The authors should consider including this.

51 empirical studies have been analyzed in the manuscript, currently, there are no figures to visualize the interpretation of the analysis of some of the key empirical data from these studies. The authors should consider including such figures in the discussion section.

When studying language teachers’   perceptions, motivational beliefs, attitudes, and opinions regarding TELE a comparison between teacher perceptions when teaching native and non-native students should be considered by the authors.

In the discussion of the use of technology in language learning and teaching issues such as the use of technology for grading to verify the achievement of learning objectives, to provide feedback on student work, and plagiarism detection have not been addressed. The authors should consider including this aspect. 

The authors need to highlight and indicate more avenues for future research based on their analysis.  Currently, very limited information has been provided regarding this issue.

 

 

Author Response

Dear Editor and Reviewers,

Thank you so much for your detailed comments on our manuscript entitled “Pedagogical Design in Technology-enhanced Language Education Research: A Scoping Review” (Sustainability – 2186114). Those comments are all valuable and helpful for revising our paper and making it stronger. We have studied comments carefully and made revisions as required. We hope the revised paper will meet your approval. Both tracked version and original version of revised manuscript are provided. In the tracked version of revised manuscript, the revised portions are marked in blue in the paper. The main revisions and responses to the reviewers’ comments are addressed point by point as follows.

 

Reviewer 1:

1. There are many types of technologies that are used in technology-enhanced language learning and teaching like gamification for language learning skills, Corpus, computer-supported corrective feedback, VR for language learning, computer-based speech recognition, collaborative writing tools, lexical annotations, computer-based intelligent tutoring, etc. There are a lot of publications in international journals covering many such topics. The authors should consider including some of these topics in the review.

Response: Thank you for pointing out these issues. We need to highlight the unique focus and objectives that our review has. We want to focus on language teachers designing pedagogy in technology-enhanced language education so that the review can be different from the other reviews mentioned by the reviewer. We also find that it is important for us to relate this review to the other reviews mentioned by the reviewer. For this reason, we have included some of these technologies in Introduction part. We added

Relevant research and reviews have documented that a variety of technologies are used, including gamification for language learning skills, Corpus-relevant, computer-supported automated corrective feedback, virtual reality (VR) for language learning and teaching, computer-based speech-to-text recognition applications, and online collaborative writing tools, as well as computer-based intelligent tutoring [3-6]. These types of technology, such as gamification, make learning activities more enjoyable than traditional classroom activity. As another example, Shadiev and Feng’s review of the use of  automated corrective feedback (ACF) tools highlighted the roles of ACF tools (e.g.,  grammar checkers, spelling checkers and collocation tools)  in facilitating language learners’  skill development for language learning [7].

2. The authors say that 51 articles published after 2012 are included in the study, however, there are some articles in the references list published before 2012, and there are only 49 total articles in the list, therefore, the reference list seems incomplete.

Response: Thank you for raising this issue. We cited some articles published before 2012, such as Teo [30], Kessler and Plakan [31], and Bancheri [35] in subsection 3.1, as we tried to compare our results/findings with those of previous papers. These articles (published before 2012) in the references list are not included because they belong to the studies in reviewed this research. They were included because of the necessity to referring to these studies.

3. In the discussion of language teachers’ practice and experience of applying technology, it is not clear what the differences are between online and classroom instruction. The authors should consider including this.

Response: Thank you for your advice. We have added more details regarding online and classroom instruction in this subsection, which can be shown as follows:

Studies have shown that teachers and learners prefer using technology in online and hybrid foreign language learning over traditional classroom instruction methods and materials, as students may be more actively engaged in the learning process [5,6]. Fully online, hybrid and blended courses are designed to provide flexible learning environments and individualized instruction for students of all ages. Hence, they appear to be appropriate for a range of educational and social objectives, and demand, active participation [40]. For example, studies on language learners’ online social networking have investigated the ways in which communities assist L2 learners’ learning beyond the language classroom, help them seek opportunities for L2 interaction,  and provide mutual emotional support to them[41]. The studies that have been reviewed include 5 articles regarding training language teachers for classroom instruction [42-44], as well as 4 papers about online EFL courses and projects for language teachers [45,46].

4. 51 empirical studies have been analyzed in the manuscript, currently, there are no figures to visualize the interpretation of the analysis of some of the key empirical data from these studies. The authors should consider including such figures in the discussion section.

Response: Thank you for pointing out this issue. We have addressed this issue by modifying Figure 4 and adding Table 1 in this research.

5. When studying language teachers’ perceptions, motivational beliefs, attitudes, and opinions regarding TELE a comparison between teacher perceptions when teaching native and non-native students should be considered by the authors.

Response: Thank you for raising this issue. We quite agree that there are many comparisons between teacher perceptions when teaching native and non-native students while scholars investigate teachers’ perceptions, motivational beliefs, attitudes and so on. Our review focuses more on how language teachers apply technology pedagogically in their language teaching. Thank you for your suggestion. We would definitely consider this issue in our future research.

6. In the discussion of the use of technology in language learning and teaching issues such as the use of technology for grading to verify the achievement of learning objectives, to provide feedback on student work, and plagiarism detection have not been addressed. The authors should consider including this aspect.

The authors need to highlight and indicate more avenues for future research based on their analysis.  Currently, very limited information has been provided regarding this issue.

Response: Thank you for your insightful comments and raising these issues. These issues have been addressed in the last part of this research by combining the abovementioned comments together, as follows:

Most of the studies we reviewed at focused on language teachers' positive perceptions, attitudes, and application of their technological affordances and pedagogical affordances, which is essential to promoting language learners' achievement and learning. The studies, however, did not mention having reliable, valid grading/evaluation systems to verify L2 learners’ learning achievements. They do not examine the automated corrective feedback tools in language learning, analyze the speech-to-text recognition applications, and investigate how plagiarism detection take place using technological tools [4,62,63]. Therefore, researchers may  need to focus on and explore additional facets of technology use in language teaching, such as how language teachers assess and evaluate students' learning progress and achievement using technological tools.

 

 

 

Author Response File: Author Response.docx

Reviewer 2 Report

Congratulations on a thorough much needed review. The identified gaps for more research on language teachers use of technology as pedagogical designers will open fruitful avenues of research in the future .

Author Response

Dear Editor and Reviewers,

Thank you so much for your detailed comments on our manuscript entitled “Pedagogical Design in Technology-enhanced Language Education Research: A Scoping Review” (Sustainability – 2186114). Those comments are all valuable and helpful for revising our paper and making it stronger. We have studied comments carefully and made revisions as required. We hope the revised paper will meet your approval. Both tracked version and original version of revised manuscript are provided. In the tracked version of revised manuscript, the revised portions are marked in blue in the paper. The main revisions and responses to the reviewers’ comments are addressed point by point as follows.

 

Reviewer 2:

1. Congratulations on a thorough much needed review. The identified gaps for more research on language teachers use of technology as pedagogical designers will open fruitful avenues of research in the future .

Response: Thank you for your positive comments. We are very encouraged by your positive comments.

 

Author Response File: Author Response.docx

Reviewer 3 Report

Evaluation:

 

This systematic literature review focuses on to what extent language teachers may engage in applying educational technology in foreign language teaching and learning. The current study adopted the PRISMA meta-analysis method and classifies 4 themes through reviewing and analysing the 51 relevant empirical studies. The findings of this review demonstrated that teachers’ practice, attitudes and knowledge of technological pedagogy were vastly explored, but the role of language teachers as pedagogical designers was hardly explored.

 

My comments concern ways to make the manuscript more elaborated in the sections of methods and results.

 

1. On page 2, lines 65-67, … “Given the large volume of research outputs in the field of TELE, we decided that this review should focus only on studies in the educational context published in international journals in the last decade (January 2012 to December 2022).”

I understand that the authors intended to size down the scope of the current review. However, the word “we decided” shows a subjective-oriented process for screening the studies. It seems that no clear and objective criteria for the screening process. Readers may ask why the year of 2012 is chosen as a dividing line for this review. Would the authors objectively justify why articles published in the last decade were selected as the target of this review? Is the time scope arbitrarily decided? I suggest that reference can be added to prove that the last decade is a critical period for the field of TELE.

 

2. Overall, there is an issue about the search and selection procedure.

That is, some search terms or steps used by the authors may lead to an overload work in the screening part. The keywords of both “technology-enhanced language learning” and “technology-enhanced language teaching” were mentioned on page 2 lines 73-74. Then, the subsequent content in lines 82-83 showed that, i.e.,“we excluded 800 articles, which were mainly about students/learners (491)…”. Readers may have an awareness that this follow-up screening process could be omitted if the keyword “language learning” were not used. In other words, readers may be concerned about whether the keyword “language learning” may bring about a large number of students’ use of technology in language education, instead of teachers. Would the authors explain what the differences are between the two terms of “technology-enhanced language teaching” and “technology-enhanced language learning”.

Readers with limited knowledge of pedagogy may doubt the validity of using technology-enhanced language learning” as a keyword to screen the study focused on teachers. Would the authors also justify why “technology-enhanced language learning”, to some extent a student-centred keyword, can be used as a keyword to search the study focused on teachers?

 

3. On page 2, lines 86-87, it was stated that 2 studies were irrelevant to technology use. Given that the keywords in your review actually all include the term of “technology” or technological tools such as “computer” or “mobile”. Why are the 2 studies unrelated to technology use present in the result of the search? What is the focus of the 2 studies?

 

 

4. On page 5, Figure 4 was presented to show the four themes of the finalised research. My concern is that in the preceding Figures 2 and 3, the basic information of methods in the teacher-related study was clearly and respectively displayed by year and by country/region. I would highly recommend that to keep consistent in reporting the findings of this review, the information of methods (i.e., qualitative, quantitative, and mixed) can be continuously demonstrated in Figure 4 by each theme. It would be effective to provide a Figure 4 with the integrative information of how many qualitative and quantitative studies for each theme.

 

5. On page 6, lines 145-146, “It is important to note that teachers who have positive perceptions of technology are more likely to use it and promote its use in teaching.”

A reference is needed here to support this conclusive statement at the beginning of this paragraph, to inform readers of the positive relationship between teachers’ perceptions and technology use in language teaching.

 

6. On page 10, lines 355 to 357 are irrelevant information to the current research topic.

Author Response

Detailed Responses to Reviewers

Dear Editor and Reviewers,

Thank you so much for your detailed comments on our manuscript entitled “Pedagogical Design in Technology-enhanced Language Education Research: A Scoping Review” (Sustainability – 2186114). Those comments are all valuable and helpful for revising our paper and making it stronger. We have studied comments carefully and made revisions as required. We hope the revised paper will meet your approval. Both tracked version and original version of revised manuscript are provided. In the tracked version of revised manuscript, the revised portions are marked in blue in the paper. The main revisions and responses to the reviewers’ comments are addressed point by point as follows.

Reviewer 3:

This systematic literature review focuses on to what extent language teachers may engage in applying educational technology in foreign language teaching and learning. The current study adopted the PRISMA meta-analysis method and classifies 4 themes through reviewing and analysing the 51 relevant empirical studies. The findings of this review demonstrated that teachers’ practice, attitudes and knowledge of technological pedagogy were vastly explored, but the role of language teachers as pedagogical designers was hardly explored.

My comments concern ways to make the manuscript more elaborated in the sections of methods and results.

1. On page 2, lines 65-67, … “Given the large volume of research outputs in the field of TELE, we decided that this review should focus only on studies in the educational context published in international journals in the last decade (January 2012 to December 2022).” I understand that the authors intended to size down the scope of the current review. However, the word “we decided” shows a subjective-oriented process for screening the studies. It seems that no clear and objective criteria for the screening process. Readers may ask why the year of 2012 is chosen as a dividing line for this review. Would the authors objectively justify why articles published in the last decade were selected as the target of this review? Is the time scope arbitrarily decided? I suggest that reference can be added to prove that the last decade is a critical period for the field of TELE.

Response: Thank you for pointing out these issues. We have added more information and references to justify why the year of 2012 is selected and last decade is critical for technology-enhanced language education, as shown in the following part:

According to Guichon and Hauck, for the past two decades, the use of digital technologies in language instruction has increased, but pedagogical developments and methodological research have rarely kept up [15]. In addition, several studies indicated that although technology use has increased for both personal and professional reasons, there have been few changes in actual pedagogical practices [15,16]. Publications before the year of 2012 have been examined multiple times in reviews [5]. For this reason, the year of 2012 was chosen as a dividing line from technological affordances to pedagogical affordances and practices for this review. Last decade is crucial for the field of TELE, especially related to the rapid development of technology and the need for the realization of pedagogical affordance with technological integration in practice.

2. Overall, there is an issue about the search and selection procedure.

That is, some search terms or steps used by the authors may lead to an overload work in the screening part. The keywords of both “technology-enhanced language learning” and “technology-enhanced language teaching” were mentioned on page 2 lines 73-74. Then, the subsequent content in lines 82-83 showed that, i.e.,“we excluded 800 articles, which were mainly about students/learners (491)…”. Readers may have an awareness that this follow-up screening process could be omitted if the keyword “language learning” were not used. In other words, readers may be concerned about whether the keyword “language learning” may bring about a large number of students’ use of technology in language education, instead of teachers. Would the authors explain what the differences are between the two terms of “technology-enhanced language teaching” and “technology-enhanced language learning”. Readers with limited knowledge of pedagogy may doubt the validity of using technology-enhanced language learning” as a keyword to screen the study focused on teachers. Would the authors also justify why “technology-enhanced language learning”, to some extent a student-centred keyword, can be used as a keyword to search the study focused on teachers?

Response: Thank you for raising this issue and providing insightful advice. We have addressed this issue by adding explanation of technology-enhanced language teaching and technology-enhanced language learning. In addition, we omitted language learning and re-searched the studies in technology-enhanced language education following your insightful advice. Although the selection procedure was a bit different from original search, the results remain the same. The added information is as follows:

In the review process, we used technology-enhanced language learning (TELL) to refer to the use of a computer or other device to display multimedia as a technological innovation to augment a teaching method by language teachers [17]. Several types of technology have been incorporated by teachers to support their instruction, involve students in the learning process, provide solid examples of the target culture, and connect students’ classrooms activities with technology, which links TELL and TELT inseparably in TELE.  The review of Shadiev et al. related to students’ perceptions of technology and language proficiency reveals that the majority of the technology enhanced learning activities were based on an instructor-centered approach, regardless language learning activities took place in classrooms or specific off-campus locations [3]. A number of studies on technology-enhanced language learning also examined language teachers’ pedagogical design. Therefore, the term technology-enhanced language learning was used as a keyword to screen and search studies on teachers. Furthermore, the first and second author reviewed and manually removed three articles which were non-empirical studies (N = 1) or studies unrelated to technology use (N = 2), which are relevant to technological software, such as the assessment of technological software. Only research that mentioned language teachers’ use or integration of technology was included in the scoping review. Finally, we read the remaining 51 publications to assess their applicability to the research question; all these studies met the selection criteria. Meanwhile, we omitted language learning from searching keywords and undertook another search of the literature. The search results remain the same as our initial attempt.

3. On page 2, lines 86-87, it was stated that 2 studies were irrelevant to technology use. Given that the keywords in your review actually all include the term of “technology” or technological tools such as “computer” or “mobile”. Why are the 2 studies unrelated to technology use present in the result of the search? What is the focus of the 2 studies?

Response: Thank you for pointing out this issue. We have provided the explanation as follows:

… studies unrelated to technology use (N = 2), which are relevant to technological software, such as the assessment of technological software.

4. On page 5, Figure 4 was presented to show the four themes of the finalised research. My concern is that in the preceding Figures 2 and 3, the basic information of methods in the teacher-related study was clearly and respectively displayed by year and by country/region. I would highly recommend that to keep consistent in reporting the findings of this review, the information of methods (i.e., qualitative, quantitative, and mixed) can be continuously demonstrated in Figure 4 by each theme. It would be effective to provide a Figure 4 with the integrative information of how many qualitative and quantitative studies for each theme.

Response: Thank you for your advice. We have modified the Figure 4 by keeping consistent in reporting the findings of this review as Figure 2 and Figure 3, with explicitly presenting qualitative, quantitative and mix studies for each theme.

5. On page 6, lines 145-146, “It is important to note that teachers who have positive perceptions of technology are more likely to use it and promote its use in teaching.” A reference is needed here to support this conclusive statement at the beginning of this paragraph, to inform readers of the positive relationship between teachers’ perceptions and technology use in language teaching.

Response: Thank you for raising this issue and suggestion. We have provided relevant references regarding this conclusive statement as follows:

It is important to note that teachers who have positive perceptions of technology are more likely to use it and promote its use in teaching [27,28].

6. On page 10, lines 355 to 357 are irrelevant information to the current research topic.

Response: Thank you for pointing out this. We have omitted the irrelevant information in this part.

 

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

I am satisfied with the author’s responses to my questions/issues raised in my initial review.

Reviewer 3 Report

I appreciate the thorough responses of the authors. My concerns with the earlier version of the manuscript have been addressed.