Impact of Mobile Learning on Students’ Achievement Results
Round 1
Reviewer 1 Report
I enjoyed reading your manuscript. Here’s my feedback:
+ Please check APA or respective citation method outlined by the journal. For example, in “The acquisition of new vocabulary via a mobile app was also researched by [13]…”, please consider calling out the author(s) instead of only referring to the citation number.
+ Even though obvious, please consider explaining the premise behind “mobile phone multi-tasking.”
+ You speak to how mobile phones and apps used in English language teaching contributes to the development of all four language skills (reading, listening, speaking, and writing). You then talk about some of the common pitfalls (i.e., mobile phone multi-tasking). And then you lead into a discussion about how mobile apps for learning English on the market are not suitable for specific foreign English language classes. Specifically, you state that they do not meet students’ specific needs at the moment. I didn’t find it clear as to why such mobile apps on the market today are not suitable for specific foreign English language classes and what it means that they do not meet student’s specific needs. Please consider speaking to why these mobile apps are suitable. For example, what doesn’t make them suitable? What’s missing that current classes need? Why don’t these apps meet students’ specific needs? What are these specific needs?
+ Your question is framed in the context of tailored-made mobile apps, but then your hypothesis isn’t presented in the context of tailored-made apps. Are you saying, per your hypothesis, that you are looking at mobile apps in general, both tailored-made and those developed for students in general?
+ Also, please consider defining “tailored-made” in your manuscript.
+ L93: You speak to the 19 students who had access to the app compatible with their smartphone OS, they also used a smartphone with a special mobile app tailored to their needs. I didn’t find this clear. They used two apps, one general and one specific to their needs? Please consider speaking to this further to offer clarity.
+ Also, please consider speaking to what the students’ needs were. (See my previous comments).
+ L113: You speak to features of the mobile app. Are these what the students’ need or what is needed in the classes? Again, this isn’t clear. Please consider speaking to this in further detail.
+ So, all students were exposed to the same learning content? 19 students were exposed to the instrument (mobile app), while 12 were not (control group)? All took the final achievement test? And you compared grades cross all students to determine if the mobile app resulted in better scores? Is that right? Please consider speaking to your method in greater detail to make sure this is all clear. As I’m afraid readers might have to make assumptions that aren’t accurate.
+ LN141: You speak to what notifications the students received. These were the students with the mobile app? What of the students in the control? What notifications did they receive? How did these students study the material? Please consider speaking to this for clarity.
+ You state that “The results of the final achievement tests revealed that most of the students who had not used the app for practicing after regular classes had done badly.” Does this mean that only the students who used the mobile app had an opportunity to practice after the regular class instruction? The students in the control did not? If this is the case, then couldn’t it mean that the mobile app didn’t really play a role, but rather, the significance was that the students exposed to the app had additional study/practice time? Please consider speaking to this.
+ LN156: You mention this is a pilot study. If this is the case, please consider stating this in your abstract and introduction.
+ Please consider presenting the stats in Table 1 directly in your discussion. There’s really no need for the table.
+ You didn’t offer much on the evaluation questionnaire in your method. Did you develop this on your own? Please consider speaking to this instrument as part of your method.
+ Regarding, “80% of the respondents would also implement such a mobile app in other courses taught at the faculty (Fig. 5).”, do you mean the student would like to see the app implemented in their other courses *by* the faculty. Please consider revising this statement, as it is unclear. See LN201 as well.
+ Regarding the statement, “This was evidenced by the conducted statistical analysis, through which the set hypothesis that the students who used mobile apps in their studies had significantly higher learning outcomes than the students who did not use this app was confirmed.”, please see my prior comment. Was it really the mobile app or simply a matter of exposure and practice? Is this a limitation of your study?
+ Regarding the statement, “Secondly, on the basis of the needs analysis, the relevant pedagogical methods, including methodological aids were selected.”, this is also unclear. Please consider speaking to this in more detail.
+ Regarding the statement, “…as well as extending the mobile app for the Apple's platform and iOS.”, what does this mean? Why is this important? Please consider clarifying.
+ Finally, please consider having your manuscript professionally edited for grammar and clarity.
Author Response
Dear reviewer,
Thank you very much for your constructive and justified comments. I tried hard to incorporate all the comments (please consult the comments and the attached manuscript) in a very thorough manner. All the insufficiencies have been highlighted in green. Overall, thanks to your review, the article has been significantly improved.
Many thanks.
Author
----------
Reviewer 1
I enjoyed reading your manuscript. Here’s my feedback:
+ Please check APA or respective citation method outlined by the journal. For example, in “The acquisition of new vocabulary via a mobile app was also researched by [13]…”, please consider calling out the author(s) instead of only referring to the citation number. – This has been modified. Please consult the manuscript.
+ Even though obvious, please consider explaining the premise behind “mobile phone multi-tasking.” – This has been explained. Please consult the manuscript.
+ You speak to how mobile phones and apps used in English language teaching contributes to the development of all four language skills (reading, listening, speaking, and writing). You then talk about some of the common pitfalls (i.e., mobile phone multi-tasking). And then you lead into a discussion about how mobile apps for learning English on the market are not suitable for specific foreign English language classes. Specifically, you state that they do not meet students’ specific needs at the moment. I didn’t find it clear as to why such mobile apps on the market today are not suitable for specific foreign English language classes and what it means that they do not meet student’s specific needs. Please consider speaking to why these mobile apps are suitable. For example, what doesn’t make them suitable? What’s missing that current classes need? Why don’t these apps meet students’ specific needs? What are these specific needs? – – This has been explained. Please consult the manuscript.
+ Your question is framed in the context of tailored-made mobile apps, but then your hypothesis isn’t presented in the context of tailored-made apps. Are you saying, per your hypothesis, that you are looking at mobile apps in general, both tailored-made and those developed for students in general? – No, the hypothesis was wrongly phrased. Now, it is correct.
+ Also, please consider defining “tailored-made” in your manuscript. – This has been explained. Please consult the manuscript.
+ L93: You speak to the 19 students who had access to the app compatible with their smartphone OS, they also used a smartphone with a special mobile app tailored to their needs. I didn’t find this clear. They used two apps, one general and one specific to their needs? Please consider speaking to this further to offer clarity. No, it is just one app. I tried to clarify it in the manuscript.
+ Also, please consider speaking to what the students’ needs were. (See my previous comments). – This has been explained in the part on Findings.
+ L113: You speak to features of the mobile app. Are these what the students’ need or what is needed in the classes? Again, this isn’t clear. Please consider speaking to this in further detail. – This has been explained.
+ So, all students were exposed to the same learning content? 19 students were exposed to the instrument (mobile app), while 12 were not (control group)? All took the final achievement test? And you compared grades cross all students to determine if the mobile app resulted in better scores? Is that right? Please consider speaking to your method in greater detail to make sure this is all clear. As I’m afraid readers might have to make assumptions that aren’t accurate. – OK, I tried to clarify it.
+ LN141: You speak to what notifications the students received. These were the students with the mobile app? What of the students in the control? What notifications did they receive? How did these students study the material? Please consider speaking to this for clarity. – This has been explained. Please consult the manuscript.
+ You state that “The results of the final achievement tests revealed that most of the students who had not used the app for practicing after regular classes had done badly.” Does this mean that only the students who used the mobile app had an opportunity to practice after the regular class instruction? The students in the control did not? If this is the case, then couldn’t it mean that the mobile app didn’t really play a role, but rather, the significance was that the students exposed to the app had additional study/practice time? Please consider speaking to this. – I tried to explain it. Please consult the manuscript.
+ LN156: You mention this is a pilot study. If this is the case, please consider stating this in your abstract and introduction. – This has been done. Please consult the manuscript.
+ Please consider presenting the stats in Table 1 directly in your discussion. There’s really no need for the table. – Table 1 has been removed.
+ You didn’t offer much on the evaluation questionnaire in your method. Did you develop this on your own? Please consider speaking to this instrument as part of your method. –Yes, this is described in the part on Methods.
+ Regarding, “80% of the respondents would also implement such a mobile app in other courses taught at the faculty (Fig. 5).”, do you mean the student would like to see the app implemented in their other courses *by* the faculty. Please consider revising this statement, as it is unclear. See LN201 as well. - Yes, this has been clarified.
+ Regarding the statement, “This was evidenced by the conducted statistical analysis, through which the set hypothesis that the students who used mobile apps in their studies had significantly higher learning outcomes than the students who did not use this app was confirmed.”, please see my prior comment. Was it really the mobile app or simply a matter of exposure and practice? Is this a limitation of your study? – Yes, it is.
+ Regarding the statement, “Secondly, on the basis of the needs analysis, the relevant pedagogical methods, including methodological aids were selected.”, this is also unclear. Please consider speaking to this in more detail. – This has been modified.
+ Regarding the statement, “…as well as extending the mobile app for the Apple's platform and iOS.”, what does this mean? Why is this important? Please consider clarifying. – This has been clarified.
+ Finally, please consider having your manuscript professionally edited for grammar and clarity. – The manuscript has been proofread by a native speaker teaching English at the faculty.
Reviewer 2 Report
The article tackles the important issue of MALL effectiveness and ways of adapting mobile learning to particular needs.
As you write, <<the newly developed smartphone app (Fig. 1) was targeted at and tailored to the 130 development and revision of novel English vocabulary>> and << The students had to translate the word or the phrase from their native language into English>>.
Bearing in mind that your students do have B2 level of language proficiency , do you think that the creation of the App was really worth all the invested time and effort? I hope the App contains more than was presented in the article.
There are some language/punctuation inconsistencies in lines 61.143, 199
Author Response
Dear reviewer,
Thank you very much for your constructive and justified comments. I tried hard to incorporate all the comments (please consult the comments and the attached manuscript) in a very thorough manner. All the insufficiencies have been highlighted in yellow. Overall, thanks to your review, the article has been improved.
Many thanks.
Author
---------
Reviewer 2
The article tackles the important issue of MALL effectiveness and ways of adapting mobile learning to particular needs.
As you write, <<the newly developed smartphone app (Fig. 1) was targeted at and tailored to the 130 development and revision of novel English vocabulary>> and << The students had to translate the word or the phrase from their native language into English>>.
Bearing in mind that your students do have B2 level of language proficiency , do you think that the creation of the App was really worth all the invested time and effort? I hope the App contains more than was presented in the article. – I believe it was worth since students’ performance has increased, as well as retention of new words. Currently, in the era of ICT, students seem to be lazy to retain new information, which they think they can find at any time on the Internet. However, the capacity of their long-term memory decreases. Therefore, at least in this case, the technology can help to enhance both short and long-term memory.
There are some language/punctuation inconsistencies in lines 61.143, 199 – This has been improved, please consult the manuscript. All the modifications have been highlighted in yellow. The whole manuscript has been also proofread by a native speaker teaching at the faculty.
Round 2
Reviewer 1 Report
I enjoyed reading your revised manuscript. It’s a significant improvement over the original. Here’s my feedback:
+ L6: I’m being nick picky, but please consider starting your abstract with “Today, mobile learning…” instead of “Nowadays, mobile learning…”
+ L25: “Nearly all university students in developed countries possess some kind of mobile device.” Please consider citing this statement, or does this statement belong to citation [2]?
+ L69-L73: Are you saying that the reason why learn English mobile apps aren’t suitable is because they do not target specific content and level of instruction? Meaning, the content needs to be specific to the topic in which the learner is being exposed (e.g., Management of Tourism) and that the instruction needs to be specific to their current level of knowledge? If so, I believe you’ve explained the first part, the latter is still unclear. If you are saying that the instruction needs to take into consideration the current English language knowledge of the learner, please state that. Just be aware that this opens all kinds of questions regarding intelligent tutoring and adaptive learning.
+ L152: You state that “The students in the control group were only encouraged to learn the new words and phrases during the contact classes.” Is it possible that the students exposed to the instrument (mobile app) did better because of additional practice, and the mobile app had little to do with it? Meaning, had the learners in the control been given an opportunity for additional instruction and study time, you’re results might have been different? Thus, what’s in question is a matter of exposure, not necessarily instructional delivery? I’m not suggesting this is actually the case, but rather argue that you might want to include such an argument as part of your limitations.
+ L162: For Figure 2, please consider labeling the control vs. the mobile app group. I think right now you have them coded as 0 and 1. While I think it’s obvious, it might be best to label them for clarity.
Also, while I asked to remove Table 1 in my earlier comments, you still need to speak to them. Although Figure 2 is helpful, you should speak to the actual scores between the two groups.
+ L182: Again, I’m being nick picky, and you’ll have to check with the journal, but typically, when using APA, you don’t start a sentence with a number. Instead, you’d say something like, “Eighty percent of the…”
+ L218: I agree with the statement, “In addition, the results also confirm that mobile learning can serve as an appropriate complementary method to other forms of course delivery thanks to its beneficial aspects such as the content adaptability, interactivity, accessibility, or feedback.” However, I am still concerned that while the mobile app was helpful, was the increase in scoring simply a matter of exposure, and not necessarily a factor of the app itself? Please see my previous comment.
+ Regarding the statement, “…as well as extending the mobile app for the Apple's platform and iOS, which can enable the students in the control group to use the mobile app as well.”. I guess I don’t understand this statement. Why would porting the app to the iOS platform allow the control group participants to use the app? Couldn’t they use the Android version? It’s important to speak to the practical significant of research, and offer further study items. But please consider clarify your statement.
+ Finally, while well written, your manuscript would benefit from further editing for senence structure to promote clarity.
Author Response
Dear reviewer,
Thank you very much for your relevant comments. I tried to incorporate them all. Please see the attached document about the changes and the revised manuscript.
Once again many thanks for your help.
Best wishes,
Author
--------
Reviewer
I enjoyed reading your revised manuscript. It’s a significant improvement over the original. Here’s my feedback:
+ L6: I’m being nick picky, but please consider starting your abstract with “Today, mobile learning…” instead of “Nowadays, mobile learning…” – This has been changed accordingly. Please consult the manuscript.
+ L25: “Nearly all university students in developed countries possess some kind of mobile device.” Please consider citing this statement, or does this statement belong to citation [2]? Yes, it does.
+ L69-L73: Are you saying that the reason why learn English mobile apps aren’t suitable is because they do not target specific content and level of instruction? Meaning, the content needs to be specific to the topic in which the learner is being exposed (e.g., Management of Tourism) and that the instruction needs to be specific to their current level of knowledge? If so, I believe you’ve explained the first part, the latter is still unclear. If you are saying that the instruction needs to take into consideration the current English language knowledge of the learner, please state that. Just be aware that this opens all kinds of questions regarding intelligent tutoring and adaptive learning. – I tried to explain this. Please see the main manuscript.
+ L152: You state that “The students in the control group were only encouraged to learn the new words and phrases during the contact classes.” Is it possible that the students exposed to the instrument (mobile app) did better because of additional practice, and the mobile app had little to do with it? Meaning, had the learners in the control been given an opportunity for additional instruction and study time, you’re results might have been different? Thus, what’s in question is a matter of exposure, not necessarily instructional delivery? I’m not suggesting this is actually the case, but rather argue that you might want to include such an argument as part of your limitations. – Yes, I agree.
+ L162: For Figure 2, please consider labeling the control vs. the mobile app group. I think right now you have them coded as 0 and 1. While I think it’s obvious, it might be best to label them for clarity. – I clarified it in the text.
Also, while I asked to remove Table 1 in my earlier comments, you still need to speak to them. – Since this was a non-parametric test, we cannot measure it in points, but the statistics explains that there is a difference between both groups. Although Figure 2 is helpful, you should speak to the actual scores between the two groups. – These scores have been added. Please see the main manuscript.
+ L182: Again, I’m being nick picky, and you’ll have to check with the journal, but typically, when using APA, you don’t start a sentence with a number. Instead, you’d say something like, “Eighty percent of the…” – This has been changed accordingly.
+ L218: I agree with the statement, “In addition, the results also confirm that mobile learning can serve as an appropriate complementary method to other forms of course delivery thanks to its beneficial aspects such as the content adaptability, interactivity, accessibility, or feedback.” However, I am still concerned that while the mobile app was helpful, was the increase in scoring simply a matter of exposure, and not necessarily a factor of the app itself? Please see my previous comment. – Yes, I agree and I tried to explain it more clearly.
+ Regarding the statement, “…as well as extending the mobile app for the Apple's platform and iOS, which can enable the students in the control group to use the mobile app as well.”. I guess I don’t understand this statement. Why would porting the app to the iOS platform allow the control group participants to use the app? Couldn’t they use the Android version? It’s important to speak to the practical significant of research, and offer further study items. But please consider clarify your statement. – The sentence has been rephrased, but it is also mentioned in the limitations.
+ Finally, while well written, your manuscript would benefit from further editing for senence structure to promote clarity. – Yes, once again I asked our native speaker at the university, James White, to proofread it.
Round 3
Reviewer 1 Report
Thank you making the edits. This version of the manuscript is much clearer. I have a few more minor comments:
+ L69: Please consider added ‘(upper-intermediate level of English)’ after ‘B2’ and then removing ‘(upper-intermediate level of English)’ from L93. You’ll have to verify this is correct, but I think this will offer clarity to the reader.
+ Regarding my comment and your response to the following:
L152: You state that “The students in the control group were only encouraged to learn the new words and phrases during the contact classes.” Is it possible that the students exposed to the instrument (mobile app) did better because of additional practice, and the mobile app had little to do with it? Meaning, had the learners in the control been given an opportunity for additional instruction and study time, you’re results might have been different? Thus, what’s in question is a matter of exposure, not necessarily instructional delivery? I’m not suggesting this is actually the case, but rather argue that you might want to include such an argument as part of your limitations. – Yes, I agree.
I see what you added on L153, but I didn’t see this fully spoken to as part of your limitations. By agreeing, I thought you’d speak to it. That is, you’d speak to the fact that the extra ‘practice’ time may have played a role and further research is needed to validate the findings of this investigation. Namely, that the mobile app contributed to the scores and not simply practice.
+ L219: I’m not sure that adding, “and were thus 219 enrolled into the control group”, helps explain things. See my previous comment. Essentially, it’s adding a limitation that states that the group exposed to the instrument was given the opportunity for additional study or practice, whereas the control group was not. This may have contributed to the findings. Meaning, the extra practice may explain the scores and not the mobile app. (In the future, I’d recommend that both groups get the same amount of exposure or practice time to ensure that this issue isn’t raised again.)
Author Response
Dear reviewer,
Thank you very much once again for your pertinent comments. I have made the changes accordingly. Please consult the manuscript.
Best wishes,
Author
Author Response File: 
Author Response.pdf
