Next Article in Journal
eLearning: Exploring Digital Futures in the 21st Century
Next Article in Special Issue
Investigating Gender and Racial/Ethnic Invariance in Use of a Course Management System in Higher Education
Previous Article in Journal
"E-tivities from the Front Line”: A Community of Inquiry Case Study Analysis of Educators’ Blog Posts on the Topic of Designing and Delivering Online Learning
Previous Article in Special Issue
Cultivating Reflective Practitioners in Technology Preparation: Constructing TPACK through Reflection
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Education Sciences
Volume 4
Issue 2
10.3390/educsci4020193
education-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

The Effects of Facilitating Feedback on Online Learners’ Cognitive Engagement: Evidence from the Asynchronous Online Discussion

by
Wenge Guo
1,
Ye Chen
2,
Jing Lei
2,* and
Yan Wen
3
1
Graduate School of Education, Peking University, Beijing 100871, China
2
Syracuse University, 336 Huntington Hall, Syracuse University, Syracuse, NY 13244, USA
3
Education Institute of Beijing, 2 Wenxing St. Xicheng, Beijing 100120, China
*
Author to whom correspondence should be addressed.
Educ. Sci. 2014, 4(2), 193-208; https://doi.org/10.3390/educsci4020193
Submission received: 13 December 2013 / Revised: 23 April 2014 / Accepted: 29 April 2014 / Published: 22 May 2014
(This article belongs to the Special Issue Critical Issues in Educational Technology)
Browse Figures
Versions Notes

Abstract

:
With a large-scale online K-12 teacher professional development course as the research context, this study examined the effects of facilitating feedback on online learners’ cognitive engagement using quasi-experiment method. A total of 1,540 discussion messages from 110 learners (65 in the experimental group and 45 in the control group) were both quantitatively and qualitatively analyzed. Results revealed that facilitating feedback significantly impacted learners’ cognitive engagement: (1) the level of cognitive engagement presented in treatment group was significantly higher than that of control group; (2) cognitive engagement levels of original postings increased overtime in the treatment group, while decreased in the control group; (3) the difference in discussion quantities between two groups was not significant. Effective feedback strategies and the importance of facilitating feedback in creating quality online instruction were discussed.

1. Introduction

With the fast advances in emerging web technologies, distance education has created greater opportunities for learners without constraints on geographic location, socio-economic condition, or time of their choosing [1,2], and is especially appreciated by adult learners who have fulltime work, family responsibilities, and multiple social obligations [3,4]. According to Jonnasen, the cognitive tools embedded in the online environment provide learners with more experience of knowledge construction, self-directed inquiry, and collaborative learning [5]. This constructive environment keeps alive the control-decentralized learning paradigm, in which learners have more opportunities to participate and contribute with their own voice [6,7]. In addition, many researchers advocate that distance education has great potential in reaching more students and reducing the cost (i.e., [8,9,10]). All these benefits bring great promise to distance education, and the demand for it will only continue to grow [11].
Although distance education has its strengths as mentioned above, it also faces some challenges due to physical separation [12]. This lack of physical presence often results in learners’ feeling of isolation, which is the primary reason for higher dropout rates in distance education [13]. In addition, increased chances of misunderstanding, difficulties in tracking learners’ learning, and more psychological distance between learners are some of the issues we encounter in distance education [14,15,16,17]. To address these issues, great attention has been paid to communication-orientated activities in online learning.
Learners’ opportunity to interact with other participants is an essential component in instruction [5,18], and learning is more likely to occur when such interaction is enhanced [19]. In this regard, online discussion becomes an important component of the online learning experience [20,21,22]. Laurillard advocates this point and further suggests that, in distance education, the pedagogical benefits of technology rely primarily on how successfully it supports the dialogue between students, or between teacher and students [23]. When suitably organized, such dialogue could provide a communal space for learners to socially construct knowledge, in which they are encouraged to take responsibility for sustained and collective knowledge advancement [24]. Scandamalia and Bereiter indicate that knowledge building is more likely to occur in a community that is typified by discourse [25]. This is because the dialogic mode provides more opportunities for learners to become co-creators of collective understanding and contributors of community knowledge base rather than passive information receivers [26]. They make argument, diagnose misconception, negotiating perspectives, share personal experiences, challenge accepted beliefs, and demand reason for conclusions, and all of these are important cognitive presence in deep learning [27,28]. Meyer believes that the socio-cognitive dynamics of online discourse could establish a constructive, collaborative, intentional, and authentic learning experience for learners [29], in which not only could they gain a thorough understanding of the subject content, but also obtain critical thinking skills and inquiry capacities that are prerequisite for an independent thinker.
However, the sole existence of discussion activities still cannot guarantee the effectiveness of distance instruction [28]. Researchers have identified some potential problems that might arise in online discussions. Meyer and Smith et al. found that learners are less motivated to participate in online discussions because written communication is more labor-intensive, and learners have to spend more time to read and respond to others’ postings [30,31]. Moreover, some studies reported low level of learning in online discussion [30,32,33]. Specifically, the discussions are merely of social nature and stay at the superficial level. In some cases, the conversation is even not related to the pre-determined topic. [34,35].
In situations like these, facilitating can play an important role in developing critical and in-depth discussions and improving online learning outcomes. According to Vandehaar, what matters in distance education is not the technology format, but the teaching and support from the instructor [36]. Feedback from the instructor/facilitator is one of the essential heuristics to promote learning [37]. Empirical evidence is shown by some research revealing that more effective learning occurs when learners are provided with instructors’ feedback [38,39,40]. Feedback could provide learners the outcome information regarding the correctness of their response [41], and additional information regarding the nature of outcomes and the qualities of their cognitive processing [39]. Fabro and Garrison reveal that instructor’s constructive critique into students’ thinking process significantly influences learners’ higher-order learning [42]. In an online course, the instructor’s presence by giving feedback has the potential to reduce the insecurity of lonely study [43], keep awareness of learning purpose [43], and improve self-confidence and self-esteem of students [45].
To better understand the feedback pedagogy, feedback effects have been explored by scholars and adopted in various practices. For example, Nicol and Macfarlane-Dick proposed seven principles of good feedback practice for self-regulated learning [46]. Chai reported the feedback effects on knowledge acquisition in online learning [47]. Zumbach, Hillers, and Reimann examined the feedback mechanism in problem-based online learning [48]. Butler and Winne studied the feedback in self-regulated learning from the theoretical view [39]. Effective feedback strategies in online instruction are identified including providing probing questions [45], clarifying the learning purpose/goal [40], encouraging positive motivational beliefs and self-esteem [46], provoking reflection [44,45], defining students’ roles/responsibilities, and encouraging students to explicitly confront others’ ideas [48].
A large body of research has explored the feedback effects on traditional classroom teaching (i.e., [39,49,50]). Some studies examined the feedback effects in online learning generally at test achievement level (i.e., [47,48,51]). The impact of feedback on students’ discussion activities, especially the specific intellectual aspects of discussion, in online learning environment has not been fully examined yet. Cognitive engagement is a concept proposed to describe such intellectual aspects in learning. In this study, we analyze the learner-generated data to examine the feedback effects on enhancing online learners’ cognitive engagement.
Cognitive engagement speaks to “the extent to which learners are able to construct and confirm meaning through sustained reflection and discourse” in a learning community [20]. Early studies on online education mainly focused on how to create and maintain social interaction in online learning environment (i.e., [52,53]), gradually building a better understanding of social interaction in online learning [54]. Moreover, research found that social interaction by itself is not sufficient to reflect the quality and depth of learning [44]. Actively cognitive engagement that deepens the meaning of the educational experience is also very important [55,56]. Zhu points out that, even in social learning situation, the quality of learning still primarily depends on the levels of cognitive engagement of individual learners [57]. Thus, in this study, we focus our attention on the cognitive engagement of online learners, and use it to indicate the quality of online learning.
In the present context, the facilitating feedback is the information learners receive from the instructor/facilitator about their discussion, with which learners can “confirm, add to, overwrite, tune, or restructure” their ideas and understandings [39]. In this study, the online facilitators qualitatively grade the content of learners’ postings. Formative comments are given to maintain idea exchanges/improvement, keep conversations on the right track, promote deeper inquiry, and help learners to link practices to theoretical principles. By analyzing the data collected from online discussion forum, we intended to answer the question: What are the facilitating feedback effects on the level of cognitive engagement exhibited by learners in asynchronous online discussion?

2. Experimental Section

2.1. Research Setting

This study was conducted in a National K-12 Schoolteacher Online Training Project in China [58], which was developed to teach K-12 teachers to integrate educational technologies into their classrooms. The online course in this study was run on Moodle [59], and was consisted of eight modules. Each module was open for a duration of three to five days. All learners were required to participate in three types of activities: reading and quiz, discussion, and hands-on practice [60].
The core of this course was the asynchronous online discussion activity. In each module, a discussion specification, including discussion topic, cases stories, orientation for preparation, and rules of participation, was provided to learners. With this specification, learners were guided to read the case stories and contribute their ideas and opinions afterwards. They were encouraged to develop their ideas based on the corresponding case stories and the module-learning theme. In addition, all learners were assigned to groups of 7–10 partners. Usually, learners with similar professional background were assigned to the same group to share teaching experience with each other. Learners were allowed to browse on all postings in the discussion forum, and required to at least respond to the postings in their own group. In order to create a learning community of being synchronous at model-level and asynchronous at posting-level [60], all learners were required to post at least one original posting, read other members’ postings and reply to at least two postings in each module.

2.2. Participants

A total of 110 distance learners of two classes in this online training program participated in this study. These distance learners were all K-12 in-service teachers in China who taught various subjects ranging from Math, Science, Chinese, History, Art, to Computer technology. The demographic data shows that all learners were adults, mostly over 30 years of age, and they all had a bachelor degree or above. In addition, all learners had basic computer skills, such as browsing online, uploading/downloading materials, and searching resources, yet none of them had any prior online training experience. Learners of the first class (n = 65) served as the experimental group, and learners of the second class (n = 45) constituted the control group. The first class received the training in May 2007, and the second class in January 2008. The training in both classes lasted for one month in total.

2.3. Study Design

A quasi-experiment design was used in this study. Both groups took the same training course, used the same learning materials, and participated in the same activities in eight modules. Particularly, in the discussion activities, the two groups were provided with the same readings and the discussions were initiated by the same questions. The only difference between the experimental class and the control class was whether or not the facilitating feedback was available.
For the experimental class, one online tutor was assigned to facilitate students’ online discussions by scoring and giving feedback to each posting. Such formative feedback followed these principles by encouraging learners to: (1) share their own teaching experience or use authentic information coming from their local schools or organizations; (2) revise and improve their arguments in the postings; (3) use appropriate terminologies and concepts, and to relate their ideas with principles and theories; (4) reflect on their own learning; (5) have positive motivational beliefs and self-esteem. Within 24 h after the learner submitted his/her post, the tutor graded their postings using rubrics listed in Table 1. In this course, the tutor used the comments as the personal response to the learner’s discussions and contributions. At the same time, giving comments was the strategy to foster learners’ deeper thinking and reflection. In the comments, the tutor pointed out the strength and weakness of learners’ postings, offered guidance on discussion requirements and how to improve their current performance, and employed probing strategies to move learners to higher-level cognitive engagement and to ensure continual cognitive development. The feedback was of formative nature. Learners were encouraged to revise their old postings and improve their discussion performance based on the comments given by the tutor. In order to motivate learners to re-approach discussion in a deeper way, the tutor would review and re-grade learners’ revised postings and gave bonus points to the new postings that demonstrated richer understanding and deeper thinking. The grade and the comments were provided in “score” and “teacher comment” boxes respectively as shown in Figure 1, and they were only open to the specific individual learner, and could not be viewed by others.
In addition, during the online discussion activity, the tutor employed feedback strategies, such as praising, encouraging, and appropriate interruptions, to help learners actively engage in the conversations. Postings considered as excellent would be selected by the facilitator and recommended to all learners.
The control class, by contrast, had no facilitator during the online learning process. After each module’s training, learners’ postings in the discussion forum were automatically graded by the learning manage system in terms of the posting’s length and learners’ logging times. For example, two 300-word postings were graded the same score without examining the content.
Table
Table 1. Online discussion rubrics.
Table 1. Online discussion rubrics.
Grade LevelRubric Explanation
Excellent
(35–40 points)		Express opinions in a clear and concise manner with strong connection with the given topics or readings
Have made a logical argument with premises, reasons, and conclusion
Analyze learning goals and learning activities with principles or theories
Use personal teaching experience or external resources to support the important points
Good
(25–34 points)		Express opinions in a clear manner with being related with the given topics or readings
Have made a logical argument with premises, reasons, and conclusion
Analyze learning goals and learning activities with principles or theories
Acceptable
(1–24 points)		Express opinions but without explanation
Incomplete
(0 point)		Not his/her own opinion but plagiarizes others’ idea or work
Education 04 00193 g001 1024
Figure 1. Teacher score and comment interface for discussion postings.
Figure 1. Teacher score and comment interface for discussion postings.
Education 04 00193 g001
In order to explicitly demonstrate the facilitating condition in the treatment group, the comparison of discussion facilitating between two groups is illustrated in Table 2.
Table
Table 2. Comparison between treatment group and control group.
Table 2. Comparison between treatment group and control group.
Feedback Intervention in Two GroupsTreatment GroupControl Group
Grading criteriaPostings were graded by the tutor based on the content of the posting. Postings were graded automatically by the online learning system based on the length of posting.
CommentsComments were given by the tutor within 24 h. The tutor respond by providing hints, probing questions or encouragement, reminding of overlooked steps, and directing learners’ attentions.No comments were given, no discourse development strategies are employed.
Nature of feedbackFormative. Within the three-day-length module, students were allowed/encouraged to revise/improve/reflect on their postings according to tutor’s comments. Summative. No suggestion was given to learners, and they do not have to improve or reflect on their postings.

2.4. Measurement: Cognitive Engagement Level

The Framework for Reflective Pedagogical Thinking developed by Sparks-Langer and Simmons was used as the coding scheme to assess learners’ cognitive engagement level. This framework was developed to assess schoolteachers’ ability to use concepts and principles to explain teaching activities and classroom events [59]. It has identified seven levels of thinking, based on Gagne’s hierarchy of thinking [61] and Van Manen’s idea of critical reflection [62,63].
As shown in Table 3, in this study, we focus on the five thinking levels used in this framework (the levels “No descriptive language” and “Explanation with consideration of ethical, moral, and political issues” were not included). Rubrics were also developed to assess the cognitive levels of replying postings (see Table 4). Examples of each level were also provided in the two tables.
A single posting was defined as the unit of analysis. We coded both the original and replying postings in the discussion forum, and identified the level of cognitive engagement of each one by using the indicators in coding schemas.
Table
Table 3. Cognitive coding schema for original posting.
Table 3. Cognitive coding schema for original posting.
Cognitive LevelDetailed descriptionExamples
Level 1Simple, layperson description. I used to train teachers in our school, and I like using Powerpoint best.
Level 2Events labeled with appropriate terms.We put too much emphasis on knowledge acquisition but ignored their critical thinking and their individual learning/thinking experience.
Level 3Explanation with tradition or personal preference given as the rationale. We should not simply teach our teachers to use technology, but rather we need to teach them how to integrate technology into their classrooms. Just like we always did in our own teaching, we want our students to not only acquire the knowledge, but also to apply the knowledge.
Level 4Explanation with principle/theory given as the rationale.This is an effective way, because the performance assessment method is used. This method use “task” to motivate students, and measures what the students can do during the process.
Level 5Explanation with principle/theory and consideration of context factors.Instructional media is rich in our life, but we do not have to be limited to using high-tech. All the tools, no matter multi-media or not, if appropriately used, could also improve our teaching. Recently when I taught the concepts of Angle and Edge in a 2nd grade math class in a rural area. I was told that no computers would be provided. Then I redesigned my instruction, and made an Angle model using stiff paper and nails….
Table
Table 4. Cognitive coding schema of replying postings.
Table 4. Cognitive coding schema of replying postings.
Cognitive LevelStandardExamples
Level 1Simple agreement or disagreement to the original postings.Strongly agree with you!
Level 2Explanation with tradition, personal preference, or principle/theory given as the rationale.I agree with all you guys. New media is a “truck to deliver the cargo”. However, we also need to have the ability to control the “truck”, and make this truck’s features fit into our teaching styles. Then we can have an effective instructional design.
Level 3Explanation with tradition, personal preference, or principle/theory given as the rationale, and consideration of context factors.I agree with you. Technology based teaching is a good way to improve students’ learning. But, we need to make sure that the technology is integrated appropriately. There was a lesson competition in my school, and we were required to use technology to teach. Some teachers just simply changed their materials to electronic version but did not pay attention to the pedagogical things, the results turned out to be not good. I know a very good philosophy teacher that he did not use any technology. He just used role play games to teach students abstract philosophical concepts. His teaching was very successful.

2.5. Data Collection and Analysis

There were eight modules in total in this online course, with the online discussion activity in each module. However, the discussions in module 1, 4, and 8 were not directly associated with the learning content, but were about social interaction (i.e., Icebreaking) and course evaluation. Thus, in the data collection, we excluded the postings in these three modules and finally collected 1,540 postings from the other five modules in the discussion forum.
These threaded text-based postings were the main data source in this study. Content analysis method was applied to qualitatively analyze the content of postings aiming at providing a richer view on learners’ cognitive engagement and the progression of the discussion. The researchers carefully studied all the transcripts, and used the coding schemas in Table 3 and Table 4 to identify learners’ cognitive engagement and grouped them into different levels. To increase the inter-rater reliability, negotiated coding approach was employed. The researchers coded the discussion transcripts, and later actively discussed and revised until all researchers arrived at a final agreement [64].

3. Results and Discussion

Analyses of discussion transcripts of 110 learners reveal that: (1) no significant difference in the quantity of discussion postings between two groups were found; (2) in each module the cognitive engagement levels presented in original and replying postings were much higher in experimental group; (3) cognitive engagement levels of original postings was progressively increased overtime in the experimental group, while it gradually decreased in the control group.

3.1. Comparison of the Quantity of Postings

As shown in Table 5, the experimental group posted slightly more messages in the online discussion. On average, each learner in the experimental group posted 1.06 original messages per week, while the control group posted 1.02; each learner in experimental group posted 1.77 replying messages, while the control group posted 1.71.
Although the data showed that the experimental group slightly outperformed control group in the quantity of discussion, the difference was not significant. Two reasons might account for this finding: (1) the discussion rule of “at least one original posting and two replies” ensured the control group learners’ participation; (2) mandatory grouping and students being told to discuss only within their own group limited their interactions with peers in other groups.
In each module, learners were required to contribute at least one original posting and reply to at least two people. However, we noticed that the actual numbers of original postings by each student per module in control group and experimental group were 1.02 and 1.05, respectively, and the replying ones were 1.76 and 1.77. This indicated that, on average, learners in this course were much more rubric-directed than self-directed. It seemed that they posted messages just to meet the minimum requirement in terms of the posting quantity. Once they reached the minimum requirement, the inquiry process ceased. To help learners recognize their ideas in discussion could be continually improved is important for driving them to a deeper level of inquiry [24]. Further study is very much needed to understand how to motivate learners to make more extensive use of online discussion without being limited to the minimum requirements.
Table
Table 5. Distance learners’ participation in online discussions.
Table 5. Distance learners’ participation in online discussions.
SubjectsTotal Number of Learners Total Number of PostingsTotal Number of Original PostingsTotal Number of Replying PostingsAverage Number of Postings per Learner
Control group6590433357113.9
Experimental group4563623839814.1

3.2. Comparison of the Level of Cognitive Engagement

The cognitive engagement in original postings. As Figure 2 illustrated, the starting points of two groups were very close to each other, which indicated that these two groups demonstrated almost equivalent cognitive level in online discussions before the facilitating feedback intervention. However, after facilitating feedback was provided to the experimental group, the difference between two groups grew as time went by. Moreover, the cognitive level of online discussions in the experimental group rose overtime, while the control group showed a trend of gradual decline.
Education 04 00193 g002 1024
Figure 2. The cognitive level curves of original postings over time.
Figure 2. The cognitive level curves of original postings over time.
Education 04 00193 g002
In addition, we noticed that, the experimental group’s cognitive level of discussions in every module was higher than that of the control group. On average, the mean cognitive level of experimental group was 3.84 and that of control group was 2.85. An independent sample T test showed that the difference of cognitive level of discussion between two groups was significant (t = 12.16, p < 0.001).
The results indicated that, with the facilitating feedback for their online discussions, the cognitive engagement level in the experimental group was significantly, increasingly, and stably higher than that of the control group, which had no facilitating feedback. In other words, facilitating feedback had evidently improved the cognitive engagement of learners in their online discussions. Four factors might be responsible for this: (1) With the constant feedback and guidance from the facilitators, the learners in the experimental group could in a clear way develop their skills on connecting practice with theories, reflecting on cases or teaching events, and justifying their arguments. This was also shown in the raising trend of their cognitive engagement level; (2) Learners in the experimental group were given the opportunities to revise and refine their ideas/postings, which stimulates or maintains learners’ continually cognitive engagement and development. (3) Facilitator’s presence could serve as an external stimulus to generate more monitor pressure on learners, and they were more motivated and might perform better in the discussion; (4) The interactions between the facilitator and learners could reduce learners’ feeling of loneliness and a sense of isolation, and a sense of community or belonging could make them contribute more to the online discussions. While in the control group, the feeling of freshness for online discussions might easily wane over time. Without the guidance and encouragement from facilitator, it could become harder for them to maintain their enthusiasm and to master the effective strategies for reciprocal communication and critical thinking in the online discussions. In addition, the lack of substantive feedback reduced the opportunities and motivations to rethink and further revise their ideas.
The cognitive engagement in replying postings. As shown in Figure 3, the cognitive engagement level of replying in the experimental group was consistently higher than that in the control group. Generally, the cognitive levels of most replying messages in the experimental group were above 2, while those in the control group were below 1.5. This indicated that when responding to others’ postings, learners in the experimental group presented not only their statement, but also included detailed evidences to support their argument. However, most learners in the control group simply voiced their agreement or disagreement without any further analysis.
Education 04 00193 g003 1024
Figure 3. The cognitive level curves of replying.
Figure 3. The cognitive level curves of replying.
Education 04 00193 g003
The discrepancy between two replying curves might be caused by two factors: the cognitive level of the original postings and the intervention of facilitating feedback. The learners who showed deeper thinking in the original postings were more likely to demonstrate higher level of cognitive engagement in their later responses to their peers’ comments. In additino, the ideas and opinions in high quality original messages were also more likely to inspire in-depth thinking and further discussion. Again, in the experimental group, the facilitator constantly reminded learners to provide evidence (cases, their own experience, etc.) to support their argument, and guided learners to enrich their ideas and link theories to their own experience. Such facilitating feedback helped them to reach level two or three.
It is interesting to find learners in the two groups posted almost the same number of messages but demonstrated significantly different cognitive engagement levels. The number of messages, in some extent, demonstrates the degree/frequency of social interaction among learners. In other words, learners’ cognitive engagement performance was not necessarily related with the number of interactions. This result is consistent with the findings of Kanuka and Anderson’s study [65]. They coded the students’ online discussion transcripts, and found that considerable interactions occurred but a low level of cognitive engagement was detected. Garrison et al. pointed out that, in online discussion, quantity of interaction is not enough, and social presence by itself is not sufficient to reflect the quality and depth of learning [44]. Thus, our data offers a clue for assessing the quality of online discussion, and also shows that the posting quantity hardly reflects the quality of discussions. To pursue a meaningful online discussion directed to cognitive outcomes, qualitative dimension characterized by learners’ cognitive engagement should be stressed more than quantitative measures. This point is consistent with the conclusions of several previous studies [44,65,66,67].
In addition, we also noticed that very few original/replying postings, in both group, had reached the top level of cognitive engagement in the online discussion. Could online discussion be an appropriate activity to create higher-order cognitive engagement? Many empirical distance education studies have investigated this issue, and higher-order cognitive engagement, for example, idea integration, testing, and application, are reported in computer-mediated communication than face-to-face conversation (i.e., [27,68]). The precise and permanent nature of written communication made it a different learning experience for online learners. The ideas contributed by learners are all recorded in written form, which enable learners to reflect on their previous understanding. In addition, when expressing using deliberate and explicit written words, learners are encouraged to carefully organize and justify their argument [69]. In this study, the reasons for the absence of top level cognitive engagement might be associated with the learning culture of “knowledge transmission lecture” in China. With many years of learning in such paradigm, K-12 teachers got more used to be passive information receivers rather than active contributors. When this online teachers training program was launched in 2007, many online learners-Chinese K-12 teachers-didn’t know how to “speak” in online discussion, they just posted paragraphs of learning theories on the discussion boards, but seldom shared their own teaching stories and problems, or expressed their opinions in a logical and critical way.

4. Conclusions

This study has investigated the effects of facilitating feedback on discussions within an online program of K-12 in-service teacher training in China. By quantitatively and qualitatively analyzing the content of 1,540 postings of two classes, we have investigated the effects of facilitating feedback on learners’ discussion performance in terms of cognitive engagement. The findings emerged from this study include: (1) no significant difference of the quantity of postings was found between control and experimental group; (2) facilitating feedback positively influenced the level of cognitive engagement presented in the discussion within each module; (3) with facilitating feedback, cognitive engagement levels of original postings increased overtime; while without it, the cognitive engagement levels decreased.
Based on the findings mentioned above, we drew the conclusion that such facilitating feedback has significant effects in enhancing learners’ cognitive engagement in online discussions. This kind of feedback strategies include: giving encouraging information, asking thought-provoking questions to nurture in-depth and reflective discussion, and appropriately interrupting their dialog to keep discussion on track. Our results reinforced the conclusion of previous studies in that the quantity of social interactivity (i.e., postings) does not reflect the depth of online discourse (i.e., cognitive engagement).
Unfortunately more detailed demographic information about the participants was not available, for example, the years of their teaching experience, the ratio of male and female, and specific age distribution. Thus, we had no ways to depict a richer picture of our participants. Knowing about the background information is helpful in thoroughly understanding how the formative comments facilitating works in the specific context of this study.
Future research investigating effective facilitating feedback strategies to promote critical online discussion, feedback’s role in changing online learning process, timing of such facilitating feedback, the feedback mechanisms in online discussion, and how feedback effects interact with learners’ characteristics (e.g., motivation, knowledge base) would be promising directions.

Acknowledgments

The authors would like to thank Distance Education Institute in Peking University for their valuable support and suggestions in data collection and study design.

Author Contributions

Wenge Guo and Yan Wen designed and implemented the study. Wenge Guo, Yan Wen and Jing Lei collected and analyzed the data. Wenge Guo, Jing Lei and Ye Chen interpreted the data and wrote the manuscript. All authors discussed the results and implications and commented on the manuscript at all stages.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Essel, R.; Owusu-Boateng, W. Access and Success in Learning: Technologies for scaling up open and distance learning programme in the Institute of Distance Learning, KNUST, Kumasi, Ghana. U.S.-China Educ. Rev. 2011, 3, 423–429. [Google Scholar]
  2. Kim, J. Developing an instrument to measure social presence in distance higher education. Brit. J. Educ. Technol. 2011, 42, 763–777. [Google Scholar] [CrossRef]
  3. Hiemstra, R. Computerized distance education: The role for facilitators. J. Adult Educ. 1994, 22, 11–23. [Google Scholar]
  4. Moore, M.G. University distance education of adults. Technol. Trends. 1987, 32, 13–18. [Google Scholar] [CrossRef]
  5. Jonassen, D.; Davidson, M.; Collins, M.; Campbell, J.; Haag, B.B. Constructivism and computer-mediated communication in distance education. AJDE 1995, 9, 7–26. [Google Scholar]
  6. Harasim, L. Teaching and learning on-line: Issues in computer-mediated graduate courses. Can. J. Educ. Comm. 1987, 16, 117–135. [Google Scholar]
  7. Scardamalia, M.; Bereiter, C. Technologies for knowledge-building discourse. Comm. ACM 1993, 36, 37–41. [Google Scholar] [CrossRef]
  8. Aksal, A.F. Action plan on communication practices: Roles of tutors at EMU distance education institute to overcome social barriers in constructing knowledge. TOJET 2009, 8, 33–45. [Google Scholar]
  9. Levy, Y. E-learning: An Overview of Next-generation Internet Based Distance Learning Systems. In Proceedings of the World Conference on the WWW and Internet, Orlando, FL, USA, 23–27 October 2001.
  10. Moore, M.; Thompson, M. Effects of Distance Learning: A Summary of Literature; the American Center for the Study of Distance Education: University Park, PA, USA, 1990. [Google Scholar]
  11. Beldarrain, Y. Distance education trends: Integrating new technologies to foster student interaction and collaboration. Dist. Educ. 2006, 27, 139–153. [Google Scholar] [CrossRef]
  12. Moore, M.; Kearsly, G. Distance Education: A System View; Cengage Learning: Belmont, CA, USA, 1996. [Google Scholar]
  13. Carr, S. As distance education comes of age, the challenge is keeping the students. Chron. High. Educ. 2000, 46, 39–41. [Google Scholar]
  14. Dickey, M. The impact of web-logs (blogs) on student perceptions of isolation and alienation in a web-based distance-learning environment. Open Learn. 2004, 19, 279–291. [Google Scholar] [CrossRef]
  15. Mason, R.; Lockwood, F. Using Communications Media in Open and Flexible Learning; Kogan Page: London, UK, 1994. [Google Scholar]
  16. Rovai, A.P.; Wighting, M.J. Feeling of alienation and community among higher education students in a virtual classroom. Internet High. Educ. 2005, 8, 97–110. [Google Scholar] [CrossRef]
  17. Russo, T.; Benson, S. Learning with invisible others: Perceptions of online presence and their relationship to cognitive and affective learning. Educ. Tech. Soc. 2005, 8, 54–62. [Google Scholar]
  18. Bates, A.W. Interactivity As a Criterion for Media Selection in Distance Education. In Proceedings of the Asian Association of Open Universities 1990 Annual Conference, Universitats Terbuka, Jakarta, Indonesia, 25–26 September 1990.
  19. Bandura, A. Social-learning Theory of Identificatory Processes; Rand McNally: Chicago, IL, USA, 1969; pp. 213–262. [Google Scholar]
  20. Garrison, D.R.; Anderson, T.; Archer, W. Critical thinking, cognitive presence and computer conferencing in distance education. Amer. J. Dist. Educ. 2001, 15, 7–23. [Google Scholar] [CrossRef]
  21. Lee, J. Patterns of interaction and participation in a Large Online Course: Strategies for fostering sustainable discussion. Educ. Tech. Soc. 2012, 15, 260–272. [Google Scholar]
  22. Marra, R.; Moore, J.L.; Klimczak, A.K. Content analysis of online discussion forums: A comparative analysis of protocols. Educ. Tech. Res. Dev. 2004, 52, 23–40. [Google Scholar] [CrossRef]
  23. Laurillard, D. Rethinking University Teaching: A Conversational Framework for the Effective Use of Learning Technologies; RoutledgeFalmer: London, UK, 2002. [Google Scholar]
  24. Zhang, J.; Scardamalia, M.; Reeve, R.; Messina, R. Designs for collective cognitive responsibility in knowledge-building communities. J. Learn. Sci. 2009, 18, 7–44. [Google Scholar] [CrossRef]
  25. Scardamalia, M.; Bereiter, C. Adaptation and Understanding: A Case for New Cultures of Schooling. In International Perspectives on the Design of Technology-supported Learning Environments; Vosniadou, S., DeCorte, E., Glaser, R., Mandl, H., Eds.; Routledge: Mahwah, NJ, USA, 1996; pp. 149–163. [Google Scholar]
  26. Guilar, J.; Loring, A. Dialogue and community in online learning: Lessons from Royal Roads University. J. Dis. Educ. 2008, 22, 19–40. [Google Scholar]
  27. Garrison, D.R.; Anderson, T.; Archer, W. Critical inquiry in a text-based environment: Computer conferencing in higher education. Internet High. Educ. 2000, 2, 87–105. [Google Scholar] [CrossRef]
  28. Lee, J. Patterns of interaction and participation in a large online course: Strategies for fostering sustainable discussion. Educ. Tech. Soc. 2012, 15, 260–272. [Google Scholar]
  29. Meyer, K. Quality in Distance Education; Wiley Periodicals: San Francisco, CA, USA, 2002. [Google Scholar]
  30. Meyer, K. Face-to-face versus threaded discussions: The role of time and higher-order thinking. J. Async. Learn. Network. 2003, 7, 55–65. [Google Scholar]
  31. Smith, G.G.; Ferguson, D.L.; Caris, M. Teaching college courses online versus face-to-face. J. Transform. Educ. Tech. 2001, 28, 18–24. [Google Scholar]
  32. Fahy, P.J.; Crawford, G.; Ally, M. Patterns of Interaction in a Computer Conference Transcript. Available online: http://www.irrodl.org/index.php/irrodl/article/viewFile/36/74.Garrison (accessed on 10 March 2012).
  33. McKlin, T.; Harmon, S.W.; Evans, W.; Jones, M.G. Cognitive presence in web-based learning: A content analysis of students’ online discussions. Amer. J. Dis. Educ. 2002, 15, 7–23. [Google Scholar]
  34. Bodzin, A.M.; Park, J.C. Dialogue patterns of preservice science teachers using asynchronous computermediated communications on the world wide web. J. Comput. Math. Sci. Teach. 2000, 19, 161–194. [Google Scholar]
  35. Chai, C.S.; Khine, M.S. Understanding ICT Integration in Schools. In Teaching with Technology: Strategies for Engaging Learners; Khine, M.S., Ed.; Pearson/Prentice Hall: Singapore, 2006; pp. 49–62. [Google Scholar]
  36. Vandehaar, D. Learning between Here and There: Quality Teleconference Classrooms. In Teleconferencing and Electronic Communications V: Applications, Technologies, and Human Factors; Parker, L.A., Olgren, C.H., Eds.; University of Wisconsin-Extension: Madison, WI, USA, 1986; pp. 338–346. [Google Scholar]
  37. Winne, P.H.; Nesbit, J.C. The Psychology of Academic Achievement. Ann. Rev. Psychol. 2010, 61, 653–678. [Google Scholar] [CrossRef]
  38. Bangert-Drowns, R.L.; Kulik, C.C.; Kulik, J.A.; Morgan, M.T. The instructional effect of feedback in test-like events. Rev. Educ. Res. 1991, 61, 213–238. [Google Scholar] [CrossRef]
  39. Butler, D.L.; Winne, P.H. Feedback and self-regulated learning: A theoretical synthesis. Rev. Educ. Res. 1995, 65, 245–281. [Google Scholar] [CrossRef]
  40. Meyer, L. Strategies for correcting students’ wrong responses. Elem. Sch. J. 1986, 87, 227–241. [Google Scholar]
  41. Anderson, R.C.; Kulhavy, R.W.; Andre, T. Feedback procedures in programmed instruction. J. Educ. Psychol. 1971, 62, 148–156. [Google Scholar] [CrossRef]
  42. Fabro, K.R.; Garrison, D.R. Computer conferencing and higher-order learning. Indian J. Open Learn. 1998, 7, 41–54. [Google Scholar]
  43. Reppa, I.; Leek, E.C. The modulation of inhibition-of-return by object-internal structure: Implications for theories of object-based attentional selection. Psychonomic Bull. Rev. 2003, 10, 493–502. [Google Scholar] [CrossRef]
  44. Garrison, D.R.; Cleveland-Innes, M. Facilitating cognitive presence in online learning: Interaction is not enough. Amer. J. Dist. Educ. 2005, 19, 133–148. [Google Scholar] [CrossRef]
  45. Tait, A. Reflections on Student Support in Open and Distance Learning. 2003. Available online: http://www.irrodl.org/index.php/irrodl/article/view/134/214 (accessed on 8 March 2012).
  46. Nicol, D.J.; Macfarlane-Dick, D. Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Stud. High. Educ. 2006, 31, 199–218. [Google Scholar] [CrossRef]
  47. Chai, L. To Have or Have Not: An Examination of Feedback, Learner Control and Knowledge Type in Online Learning. In Proceedings of the 36th Hawaii International Conference on System Sciences, Big Island, HI, USA, 6–9 January 2003.
  48. Zumbach, J.; Hillers, A.; Reimann, P. Distributed Problem-based Learning: The Use of Feedback Mechanisms in Online Learning. In Online Collaborative Learning: Theory and Practice; Roberts, T.S., Ed.; Idea Group Inc.: Hershey, PA, USA, 2004; pp. 86–102. [Google Scholar]
  49. Kulhavy, R.W.; Stock, W.A. Feedback in written instruction: The place of response certitude. Educ. Psychol. Rev. 1989, 1, 279–308. [Google Scholar] [CrossRef]
  50. Balzer, W.K.; Doherty, M.E.; O’Connor, R. Effects of cognitive feedback on performance. Psychol. Bull. 1989, 106, 410–433. [Google Scholar] [CrossRef]
  51. Murphy, P. Reading comprehension exercises online: The effects of feedback, proficiency and interaction. Lang. Learn. Tech. 2007, 11, 107–129. [Google Scholar]
  52. Gunawardena, C.N.; Zittle, F. Social presence as a predictor of satisfaction within a computer mediated conferencing environment. Amer. J. Dist. Educ. 1997, 11, 8–25. [Google Scholar] [CrossRef]
  53. Hackman, M.Z.; Walker, K.B. Instructional communication in the televised classroom: The effects of system design and teacher immediacy on student learning and satisfaction. Comm. Educ. 1990, 39, 196–209. [Google Scholar] [CrossRef]
  54. Akyol, Z.; Garrison, D.R. Understanding cognitive presence in an online and blended community of inquiry: Assessing outcomes and processes for deep approaches to learning. Brit. J. Educ. Technol. 2011, 42, 233–250. [Google Scholar] [CrossRef]
  55. Dewey, J. How We Think: A Restatement of the Relation of Reflective Thinking to the Educative Process; D.C. Heath and Company: Lexington, MA, USA, 1933. [Google Scholar]
  56. Stoney, S.; Oliver, R. Can Higher Order Thinking & Cognitive Engagement be Enhanced with Multimedia? 1999. Available online: http://imej.wfu.edu/articles/1999/2/07/printver.asp (accessed on 10 March 2012).
  57. Zhu, E. Interaction and cognitive engagement: An analysis of four asynchronous online discussions. Instr. Sci. 2006, 34, 451–480. [Google Scholar] [CrossRef]
  58. National K-12 Schoolteacher Online Training Project. Peking University. Available online: http://train.pkudl.cn/ (accessed on 10 March 2012).
  59. Moodle. Available online: https://moodle.org/ (accessed on 10 March 2012).
  60. Guo, W. From an online training course to a “virtual” teacher training academy: Design and implementation of Peking university asynchronous online teacher training program. LNCS 2009, 5685, 365–377. [Google Scholar]
  61. Sparks-Langer, G.M.; Simmons, J.M.; Pasch, M.; Colton, A.; Starko, A. Reflective pedagogical thinking: How can we promote it and measure it? J. Teach. Educ. 1990, 41, 23–32. [Google Scholar] [CrossRef]
  62. Gagne, R.M. Learning Hierarchies. In Instructional Design: Readings; Merrill, M.D., Ed.; Prentice-Hall: Englewood Cliffs, NJ, USA, 1968; pp. 118–131. [Google Scholar]
  63. Van Manen, M. Linking ways of knowing with ways of being practical. Curriculum Inq. 1977, 6, 205–228. [Google Scholar] [CrossRef]
  64. Simmons, J.M.; Sparks, G.M.; Starko, A.; Pasch, M.; Colton, A.; Grinberg, J. Exploring the Structure of Reflective Pedagogical Thinking in Novice and Expert Teachers: The Birth of a Developmental Taxonomy. In Proceedings of the Annual Conference of the American Educational Research Association, San Francisco, CA, USA, 27–31 March 1989.
  65. Garrison, D.R.; Cleveland-Innes, M.; Koole, M.; Kappelman, J. Revisiting methodological issues in the analysis of transcripts: Negotiated coding and reliability. Internet High. Educ. 2006, 9, 1–8. [Google Scholar] [CrossRef]
  66. Kanuka, H.; Anderson, T. Online social interchange, discord, and knowledge construction. J. Dist. Educ. 1998, 13, 57–74. [Google Scholar]
  67. Schrire, S. Knowledge building in asynchronous discussion groups: Going beyond quantitative analysis. Comput. Educ. 2006, 46, 49–70. [Google Scholar] [CrossRef]
  68. Vaughan, N.; Garrison, D.R. Creating cognitive presence in a blended faculty development community. Internet High. Educ. 2005, 8, 1–12. [Google Scholar] [CrossRef]
  69. Shea, P.; Bidjerano, T. Cognitive presence and online learner engagement: A cluster analysis of the community of inquiry framework. J. Comput. High. Educ. 2009, 21, 199–217. [Google Scholar] [CrossRef]
  70. Garrison, D.R.; Anderson, T. E-learning in the 21st Century; Routledge Falmer: New York, NY, USA, 2003. [Google Scholar]

Share and Cite

MDPI and ACS Style

Guo, W.; Chen, Y.; Lei, J.; Wen, Y. The Effects of Facilitating Feedback on Online Learners’ Cognitive Engagement: Evidence from the Asynchronous Online Discussion. Educ. Sci. 2014, 4, 193-208. https://doi.org/10.3390/educsci4020193

AMA Style

Guo W, Chen Y, Lei J, Wen Y. The Effects of Facilitating Feedback on Online Learners’ Cognitive Engagement: Evidence from the Asynchronous Online Discussion. Education Sciences. 2014; 4(2):193-208. https://doi.org/10.3390/educsci4020193

Chicago/Turabian Style

Guo, Wenge, Ye Chen, Jing Lei, and Yan Wen. 2014. "The Effects of Facilitating Feedback on Online Learners’ Cognitive Engagement: Evidence from the Asynchronous Online Discussion" Education Sciences 4, no. 2: 193-208. https://doi.org/10.3390/educsci4020193

Article Metrics

Back to TopTop