Effects of Using World Indicators for Online ESD Learning
Round 1
Reviewer 1 Report
Please find them attached.
Comments for author File: 
Comments.pdf
Author Response
Response to Reviewer 1
â–ª The idea of this research is very interesting, however the method (SDCT) needs to be refined, perhaps under a different name.
I am not fully aligned with the invented concept ”Statistical data-driven concept transformation” (SDCT). I searched the extant literature and did not find any reference of it. In my opinion, this proposed concept hasn’t been well fundamented. The use of statistics ensures that the results are representative for the population, which is hardly the case here.
“Originally, SDCT was designed along with access to the world statistical database 85
as a student assignment to study world issues. Students collect data about two global 86
indicators. When some perceived disorder or discordance stems from comparing two 87
indicators, they are triggered to conduct an investigation to write a report. This moment 88
is designated in transformative learning theory as the situation of a disorienting dilem-89
ma, by which a student realizes that what they thought or believed in the past might not 90
be accurate. Instrumental learning is enacted [26].”
Response:
We appreciate the reviewer's comment on this point.
We deleted a name (SDCT) of lesson method that you recommended and rewrote to explain more about our process used. Also, the title was changed, all terms of SDCT were changed to “online learning method.”
â–ªWhat is the concept under transformation from” Statistical data-driven concept transformation” (SDCT)
Response:
We thank the reviewer for this comment. About this term, we described the following sentences in the article.
134-136: Students choose a country for which two indicators engender a discrepancy beyond the intuitive notion, common sense, or innate opinion of students, leading them to the conceptual gateway for transforming their concept.
88-91 This moment is designated in transformative learning theory as the situation of a disorienting dilemma, by which a student realizes that what they thought or believed in the past might not be accurate. Instrumental learning is enacted [26].
Summarizing these, transformation in this article is:
Disorder or discordance that they felt under the existing individual common sense to switch to a new concept that explain more about the fact.
To make this point clearer, we have added the sentences on line 91-94.
â–ª”Statistical data-driven concept transformation” (SDCT) not present in the referenced works [25], [26]
Response:
We agree that this point requires modification, and have changed to the “online learning method.”
â–ªWhich two global indicators? Four indicators are presented in Figure 2.
Response:
We appreciate the reviewer's concerns on this point.
Four sets of assignments were prepared and each student was allotted one of these. We improved explanation of this part on line 242. In addition, detailed explanation of each set is shown (line 242-244).
Additionally, I am not fully convinced that the key ESD competencies listed in Table 1 can be verified to have been achieved by students from some student messages posted on Moodle. For instance, how the achievement of Strategic competency or Anticipatory competency was assessed through messages? In my opinion this is too subjective for a ”statistically-driven” method, which theoretically is systematic, rigorous, and offers representativity to the entire population.
Response:
We appreciate the reviewer's interest in ESD competencies.
As students are first graders, it is usually difficult for them to achieved/acquired these competencies while studying at this course. However, since the lesson requires data comparison of indicators over time, it is a natural environment for them to start thinking about future trends based on concrete statistical data. Also, they were expected to look for evidence to support their predictions. Strategic competency was also counted when they represented opinions that foster the country’s sustainability by explaining their collected information. Since they are starting to learn SDGs, they struggled to refine their opinion, but we could assess that their descriptions were related to their ESD competence performed/ represented based on their described scientific evidence.
We appreciate the reviewer's comment on this point. Accordingly, we have change verb “acquire” on line 20 and 424 to “represent” for improving uncertainly.
The average number of messages/student is extremely low (307 messages appeared from 119 university students, resulting in 2.5 messages per student) to infer properties of the sample, not to mention the population.
Response:
We thank the reviewer for this comment.
We counted the total words of all 307 messages, and it was 56,644 words (in English translation, excluding words in tables, diagrams, and figures that they posted as graphics). We added this information on line 255.
Regarding score of 2.5 reports/week, the response has been a bit low. However, no post message is not no learning and no opinion, which was shown on line 346 as the result of S111. The explanation of this part proved that they read messages from others and were influenced in the messages they delivered.
It isn’t clear the relationship between collective intelligence and the ESD competencies (Figure 4).
Response:
We wish to thank the reviewer for this comment.
Both have different roots; collective intelligence is aggregated knowledge shared by people (situated) and ESD competencies are basically individual ability (attribute). However, while ESD competencies are focused in communicative environment, it is possible to discuss the relationship with the collective intelligence. In the current study, co-occurrence was examined.
To make this point clearer, we have modified explanation on line 68.
I think this SDCT method is the major drawback of this study. The traditional qualitative research method works properly here. I think the “statistically-driven” term refers to the fact that students access SDG statistical data to document their messages, not the research process per se. The strongest point, in my opinion, is the assessment of the collective wisdom, and this should be emphasized and better explained throughout the paper.
Response:
The reviewer's comment is correct. To clarify, we have changed SDCT to “online learning method.”
Reviewer 2 Report
The article's writing style confuses the reader. The English language makes the story difficult to comprehend. Authors must use the correct terminology for a variety of terms. Moreover, they should justify the usage and application of the terms in their experimental design, as they assert that it is the case. In general, authors should exert greater effort to present a logical rationale for their research and effectively communicate their findings in a manner that is compatible with their conceptual and framework.
Before final publication, the authors should have their text proofread at several points because the language is not easily comprehensible.
Methodology should be separated into its own section and expanded upon. The evaluation section seems extremely intriguing and should be clarified further. Moreover, the findings are positive, and the future work appears promising.
The authors should include more references regarding other related fields, such as game-based learning and serious games (e.g. CMX MMORPG), and how they have been designed utilizing relative design frameworks, evaluated utilizing relative evaluation frameworks, and implemented utilizing learning analytics.
Author Response
Response to Reviewer 2
The article's writing style confuses the reader. The English language makes the story difficult to comprehend. Authors must use the correct terminology for a variety of terms. Moreover, they should justify the usage and application of the terms in their experimental design, as they assert that it is the case. In general, authors should exert greater effort to present a logical rationale for their research and effectively communicate their findings in a manner that is compatible with their conceptual and framework.
Response:
We appreciate the reviewer's comment on this point. In accordance with the reviewer's comment, we have improved explanation of section 2 on line 75-137, and blue coloured letters are modified parts to improve usage of terms.
Before final publication, the authors should have their text proofread at several points because the language is not easily comprehensible.
Response:
We have had an additional manuscript rewritten by an experienced academic editor, who has improved the grammar and stylistic expression of the paper (modified words are shown as blue color).
Methodology should be separated into its own section and expanded upon. The evaluation section seems extremely intriguing and should be clarified further. Moreover, the findings are positive, and the future work appears promising.
Response:
We appreciate the reviewer's interest in terms of methodological section. We expanded and further developed Section 2 to place more emphasis on methodological features in the current study. We strongly appreciate the reviewer's comment on evaluation section.
The authors should include more references regarding other related fields, such as game-based learning and serious games (e.g. CMX MMORPG), and how they have been designed utilizing relative design frameworks, evaluated utilizing relative evaluation frameworks, and implemented utilizing learning analytics.
Response:
We wish to express our deep appreciation to the reviewer for his insightful comment on this point.
Regarding the relationship of our used method to other fields, QCA and GSEQ are already used in Learning Analytics (LA). Because QSEQ used a normal distribution when analysing and scoring behavioural sequences, we added information in line 195. SNA is in a completely different direction than game-based where the system focuses on selected users, where SNA expand involvement of users. Then SNA is frequently used in marketing research, but difficult to use in LA due to the limited processing power of workstations. However, SNA uses relational data and can make focus on users apart from their individual covariates. We added and described this analytical advantage in line 343-345. Unfortunately, we could not find any cases to instruct ESD using MMORPG.
Reviewer 3 Report
Regarding the content, I do not have any changes to recommend, it makes a good literary review to support the relevance of the problem to be studied and a good structuring of the content, it uses the correct methodology for this type of study and it is a consistent and well-detailed methodology to give significance to the results they show, makes a good discussion of the results with respect to the studies carried out previously, and marks the conclusion obtained well.
Although I advise looking at these things:
Remember to put the affiliation superscript to the author's name. Look at the template.
The “Final remarks” section calls it “6. Conclusions”. And in this section it is necessary to develop a deeper analysis of the conclusions, implications and limitations of the study. In addition to the possible future lines of research opened with this research.
And don't forget to add after the section “6. Conclusions”, the “Author Contributions” section. Look at the template.
Author Response
Regarding the content, I do not have any changes to recommend, it makes a good literary review to support the relevance of the problem to be studied and a good structuring of the content, it uses the correct methodology for this type of study and it is a consistent and well-detailed methodology to give significance to the results they show, makes a good discussion of the results with respect to the studies carried out previously, and marks the conclusion obtained well.
Response:
I wish to express my strong appreciation to the reviewers for your insightful comments on my paper.
Although I advise looking at these things:
Remember to put the affiliation superscript to the author's name. Look at the template.
Response:
I wish to thank the reviewer for this comment. After my manuscript will be accepted, I will conduct final printable version with my name with the affiliation superscript and a number.
The “Final remarks” section calls it “6. Conclusions”. And in this section, it is necessary to develop a deeper analysis of the conclusions, implications and limitations of the study. In addition to the possible future lines of research opened with this research.
Response:
I wish to thank the reviewer for this comment. Accordingly, I have changed from “final remarks” to “conclusion” and added a deeper analysis and limitation of study to this section.
And don't forget to add after the section “6. Conclusions”, the “Author Contributions” section. Look at the template.
Response:
I thank the reviewer for this comment. I added “author contributions” section.
Round 2
Reviewer 1 Report
Thank you for taking into account my recommendations.
Author Response
Response to Reviewer 1 comments.
Thank you for inviting me to submit a revised draft of my manuscript entitled, “Effects of Using World Indicators for Online ESD Learning” to Sustainability. I also appreciate the time and effort you have dedicated to providing feedback on ways to strengthen my paper. Therefore, it is my great pleasure to resubmit my article for further consideration. I have made changes that reflect the suggestions you kindly provided. I also hope that my amendments and the responses I provide below satisfactorily address all the issues and concerns that you and the reviewers have identified.
Four references have been added to describe and contextualize past and present theoretical background and empirical research.
All references have been edited and styled as appropriate.
The conclusion section was improved to be supported by the results presented in the article, and the discussion section was better referenced in secondary literature.
Also, I asked the professional academic editing service to improve readability before resubmitting.
