Designing Feedback Systems: Examining a Feedback Approach to Facilitation in an Online Asynchronous Professional Development Course for High School Science Teachers

Round 1

Reviewer 1 Report

1/ It would be interesting to refer (e.g. in the discussion) to other learning models in the face of the limits of the feedback model. 

2/ The authors describe a double-loop learning model, but in fact it is a multiple-loop feedback.

3/ The discussion is mostly a summary of the research results, it could be better to separate the results, discussions and final conclusions.

The text does not require too many corrections, but if there is still
some moment for improvement, I suggest more clear organizing the
research objectives and the whole methodology. It is not clear what
exactly the hypothesis was tested. If the goal was to understand the
role of feedback in online learning, there may be many hypotheses
depending on the questions posed. The research results shows some
questions about the effectiveness of triangle facilitation system but
also about the limits of this system. As a mixed approach was used, the
hypotheses could be justified. The discussion could also be based on a
richer and more interdisciplinary literature review.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

This manuscript attempts to understand users’ satisfaction with regards to an online course that is hosted in EdX. The manuscript needs some revision before it should be considered for publication. Below are some points that needed improvement:  

Is MOOC CSCL? Please provide justification for this claim. In a really massively online situation, how could feedback be provided at a motivating level except randomly? Perhaps the authors may want to refrain from this claim and focus on CSCL. In fact, this is just an online course mounted in EdX. It is should not be qualified as MOOC since there are only 180 participants. Although there isn’t a cutoff for number of participants, it is difficult for it to be accepted as a MOOC. Perhaps the authors can argue that it is intended to be a MOOC and draw some references.    

To facilitate readers’ understanding, I believe this statement “Despite the affordances of feedback for supporting learning, as a complex system, feedback processes are sometimes impeded by the ambiguity of the feedback, systematic misperceptions of feedback, insufficient modeling of our cognitive maps, ineffective social and emotional support, and inadequate scientific reasoning skills [24].” need to be expanded with how each concept work. It would also help to ground your design.

Line 104, Careless [31] should be Carless. It is easy to be careless about this name.

Explication of how instructors overcome the issue of providing feedback to a huge number of users would be enlightening.  How many facilitators were involved?

Section 1.2.1 to 1.2.3 are the designed feedback “system”. It is not well integrated with the literature and it should be treated as another section. It actually belongs to the pedagogical method section, which is usually part of methods.

It may be better to have clear research questions.

The response rate is lesser than 50 percent, which is a serious limitation. This should be stated.

Line 146 Despite these promise results, should be promising.    

Table 8 Report the t-test value, please.

The discussion is generally sound given the mixed results. The mixed results have however rendered the contribution of the manuscript less clear.   

Author Response

Please see the attachment

Author Response File: Author Response.pdf