Orchestrating Teacher, Peer, and Self-Feedback to Enhance Learners’ Cognitive, Behavioral, and Emotional Engagement and Public Speaking Competence
Abstract
:1. Introduction
1.1. Peer Feedback, Self-Feedback, and Teacher Feedback on Public Speaking
1.2. Comparative Studies of Peer, Self-, and Teacher Feedback on Public Speaking
1.3. Engagement with Feedback on Public Speaking
1.4. The Present Study
2. Methodology
2.1. Participants and Context
2.2. Instruments
- Cognitive engagement (items 1–6) is indicated by students’ thinking processes: understanding and reflecting on the feedback, recognizing their strengths and weaknesses, strategizing revisions, and monitoring their progress against the feedback when preparing subsequent speeches.
- Behavioral engagement (items 7–12) is manifested through actions taken by students in response to the feedback: taking notes on the feedback, discussing feedback with peers or teachers, revising speeches, organizing speech structure, searching for supporting evidence, and practicing speech delivery.
- Emotional engagement (items 13–15) is reflected by students’ subjective feelings: the extent to which they enjoy receiving or providing feedback, find it useful, and look forward to engaging in the feedback process. The instrument was validated using Rasch measurement (see Section 2.5. Data Analysis for the validation of the PSFES).
2.3. Intervention
2.4. Procedures
2.5. Data Analysis
3. Results
3.1. Rasch–Andrich Rating Scale Model Validation of the Public Speaking Feedback Engagement Scale (PSFES)
3.2. MFRM Validation of the Ratings Assigned to Students’ Public Speaking Performance
3.3. The Impact of Feedback Sources on Student Engagement
3.4. Impact of Feedback Source on Students’ Public Speaking Performance
3.5. Correlation between Students’ Feedback Engagement and Public Speaking Performance
4. Discussion
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Appendix A
Appendix A.1. Public Speaking Feedback Engagement Scale (PSFES)
Public Speaking Feedback Engagement Scale (PSFES) | |
Directions: Please indicate in the following scale to what extent you agree with the following statements (1 = strongly disagree; 2 = disagree; 3 = not sure/neutral; 4 = agree; 5 = strongly agree) and provide your complementary comment. Cognitive engagement | |
1. I pay attention to the feedback. | (1) (2) (3) (4) (5) |
2. I understand the feedback. | (1) (2) (3) (4) (5) |
3. I reflect on and evaluate the feedback. | (1) (2) (3) (4) (5) |
4. I realize my strengths and weaknesses through the feedback. | (1) (2) (3) (4) (5) |
5. I strategize to improve my speech based on the feedback. | (1) (2) (3) (4) (5) |
6. I monitor myself against the feedback when I prepare the next speech. | (1) (2) (3) (4) (5) |
Behavioral engagement | |
7. I take careful notes of the feedback. | (1) (2) (3) (4) (5) |
8. I discuss the feedback with my teacher and classmates after class. | (1) (2) (3) (4) (5) |
9. I make revisions based on the feedback. | (1) (2) (3) (4) (5) |
10. I mark the structure of my next speech based on the feedback. | (1) (2) (3) (4) (5) |
11. I search relevant sources for evidence to support my point. | (1) (2) (3) (4) (5) |
12. I practice delivering my next speech based on the feedback. | (1) (2) (3) (4) (5) |
Emotional engagement | |
13. I enjoy receiving/providing feedback. | (1) (2) (3) (4) (5) |
14. I find the feedback helpful and valuable. | (1) (2) (3) (4) (5) |
15. I look forward to the feedback on my next speech. | (1) (2) (3) (4) (5) |
Other comments |
Appendix A.2. Feedback Sheet
Feedback Sheet |
Name: ____________ Evaluator ____________________ Please give your comment on the speaker’s performance based on the following guidelines. |
The Speech Opening |
|
|
|
The Speech Body |
|
|
|
|
The Speech Conclusion |
|
|
|
Delivery Skills |
|
|
|
|
|
|
|
Global Comment |
|
Appendix A.3. Fit Statistics of PSFES Items
Subscale | Pre-Test | Post-Test | |||
Item | Infit MnSq | Outfit MnSq | Infit MnSq | Outfit MnSq | |
Cognitive | 1 | 0.85 | 0.89 | 1.12 | 1.07 |
2 | 1.1 | 0.98 | 0.78 | 0.82 | |
3 | 0.8 | 0.8 | 0.85 | 0.88 | |
4 | 1.2 | 1.15 | 1.24 | 1.18 | |
5 | 0.96 | 0.87 | 0.94 | 0.92 | |
6 | 1.03 | 0.95 | 1.09 | 1.1 | |
Behavioral | 7 | 0.7 | 0.69 | 0.86 | 0.87 |
8 | 1.09 | 1.18 | 1.27 | 1.29 | |
9 | 0.96 | 0.94 | 0.96 | 0.97 | |
10 | 1.16 | 1.11 | 0.9 | 0.88 | |
11 | 1.23 | 1.09 | 1.16 | 1.12 | |
12 | 0.98 | 0.98 | 0.81 | 0.82 | |
Emotional | 13 | 0.73 | 0.78 | 1.05 | 1.07 |
14 | 0.74 | 0.67 | 0.98 | 0.9 | |
15 | 1.48 | 1.56 | 0.95 | 0.9 |
Appendix A.4. Rating Scale Model’s Category Structure for PSEFS
Test | Category Level | Observed Average | Expected Average | Infit MnSq | Outfit MnSq | Andrich Threshold | Category Measure |
Pre | Cognitive | ||||||
2 | −0.97 | −0.87 | 0.92 | 0.79 | NONE | −2.71 | |
3 | −0.21 | −0.17 | 0.95 | 0.88 | −1.02 | −1.57 | |
4 | 0.96 | 0.93 | 1 | 1.04 | 1.64 | 0.67 | |
5 | 2.53 | 2.58 | 1.1 | 0.99 | 2.66 | 3.77 | |
Behavioral | |||||||
1 | −0.62 | −0.82 | 1.13 | 1.02 | NONE | −4.39 | |
2 | −0.19 | −0.18 | 1 | 0.97 | −3.27 | −1.57 | |
3 | 0.51 | 0.53 | 0.85 | 0.74 | −0.54 | 0.15 | |
4 | 1.36 | 1.36 | 1.13 | 1.19 | 0.17 | 1.63 | |
5 | 2.44 | 2.42 | 0.89 | 0.97 | 2.91 | 4.04 | |
Emotional | |||||||
2 | −3.05 | −2.89 | 0.81 | 0.64 | NONE | −4.52 | |
3 | −0.9 | −0.94 | 1.03 | 1.15 | −3.31 | −2.46 | |
4 | 2.17 | 2.18 | 0.93 | 0.96 | −1.62 | 1.66 | |
5 | 5.45 | 5.44 | 0.99 | 1 | 4.92 | 6.02 | |
Post | Cognitive | ||||||
2 | −0.71 | −0.82 | 1.04 | 1 | NONE | −4.22 | |
3 | 0.14 | 0.06 | 1.09 | 1.13 | −3.07 | −1.67 | |
4 | 1.47 | 1.54 | 0.99 | 0.95 | −0.26 | 1.55 | |
5 | 3.5 | 3.42 | 0.91 | 0.91 | 3.34 | 4.46 | |
Behavioral | |||||||
1 | −2.06 | −1.2 | 0.6 | 0.74 | NONE | −5.63 | |
2 | −0.4 | −0.25 | 0.89 | 0.9 | −4.51 | −2.78 | |
3 | 1.06 | 0.98 | 1.03 | 1.03 | −1.01 | 0.08 | |
4 | 2.42 | 2.43 | 1.01 | 0.98 | 1.17 | 2.78 | |
5 | 4.07 | 4.09 | 1.05 | 1.03 | 4.35 | 5.48 | |
Emotional | |||||||
2 | −1.7 | −2.12 | 1.18 | 1.04 | NONE | −4.56 | |
3 | −0.35 | −0.36 | 1.06 | 1.02 | −3.41 | −2.08 | |
4 | 2.07 | 2.12 | 1 | 1.03 | −0.75 | 1.71 | |
5 | 4.31 | 4.24 | 0.91 | 0.85 | 4.16 | 5.26 |
Appendix A.5. Fit Statistics of the Ratings for the Pre- and Post-Tests
Student No. | Pre-Test | Post-Test | ||
Infit MnSq | Outfit MnSq | Infit MnSq | Outfit MnSq | |
1 | 1.01 | 1.01 | 1.14 | 1.14 |
2 | 0.91 | 0.91 | 0.87 | 0.85 |
3 | 1.3 | 1.3 | 0.62 | 0.61 |
4 | 0.97 | 0.97 | 0.85 | 0.85 |
5 | 1.03 | 1.03 | 1.17 | 1.16 |
6 | 1.07 | 1.08 | 1.31 | 1.31 |
7 | 1.03 | 1.04 | 1.04 | 1.04 |
8 | 0.79 | 0.79 | 0.93 | 0.93 |
9 | 0.91 | 0.91 | 1.43 | 1.42 |
10 | 0.83 | 0.83 | 0.92 | 0.92 |
11 | 0.95 | 0.95 | 1.09 | 1.1 |
12 | 0.9 | 0.91 | 0.75 | 0.76 |
13 | 1.36 | 1.35 | 1.25 | 1.24 |
14 | 1.28 | 1.26 | 1.11 | 1.11 |
15 | 1.13 | 1.13 | 1.39 | 1.37 |
16 | 0.97 | 0.96 | 0.71 | 0.7 |
17 | 1.2 | 1.2 | 1.28 | 1.26 |
18 | 1.06 | 1.06 | 1.17 | 1.16 |
19 | 1.42 | 1.41 | 0.86 | 0.86 |
20 | 1.22 | 1.22 | 0.93 | 0.93 |
21 | 1.47 | 1.45 | 1.12 | 1.11 |
22 | 1.18 | 1.18 | 0.76 | 0.75 |
23 | 0.97 | 0.97 | 0.75 | 0.75 |
24 | 0.96 | 0.96 | 1.1 | 1.08 |
25 | 1.26 | 1.27 | 1.19 | 1.19 |
26 | 1.19 | 1.19 | 1.43 | 1.43 |
27 | 0.74 | 0.74 | 1.3 | 1.3 |
28 | 1.09 | 1.09 | 1.23 | 1.23 |
29 | 1.06 | 1.06 | 0.97 | 0.98 |
30 | 1.01 | 1.01 | 1.18 | 1.17 |
31 | 1.1 | 1.1 | 1.18 | 1.17 |
32 | 0.93 | 0.93 | 0.89 | 0.89 |
33 | 1.17 | 1.18 | 0.74 | 0.72 |
34 | 0.88 | 0.88 | 0.93 | 0.93 |
35 | 0.67 | 0.67 | 1.19 | 1.17 |
36 | 1.2 | 1.19 | 0.92 | 0.92 |
37 | 1.1 | 1.1 | 1.04 | 1.04 |
38 | 0.98 | 0.99 | 0.78 | 0.78 |
39 | 1.19 | 1.19 | 0.9 | 0.9 |
40 | 1.31 | 1.32 | 1.33 | 1.32 |
41 | 0.79 | 0.79 | 0.88 | 0.86 |
42 | 0.7 | 0.71 | 0.83 | 0.83 |
43 | 1.24 | 1.23 | 1.44 | 1.43 |
44 | 0.88 | 0.88 | 0.81 | 0.8 |
45 | 0.91 | 0.91 | 0.79 | 0.77 |
46 | 1.12 | 1.12 | 1.18 | 1.16 |
47 | 1.03 | 1.03 | 1.12 | 1.13 |
48 | 0.9 | 0.9 | 0.73 | 0.74 |
49 | 1.01 | 1.01 | 1.47 | 1.45 |
50 | 0.87 | 0.87 | 0.95 | 0.94 |
51 | 0.82 | 0.82 | 1.05 | 1.04 |
52 | 1.2 | 1.2 | 0.71 | 0.7 |
53 | 1.17 | 1.17 | 1.22 | 1.21 |
54 | 1.13 | 1.13 | 0.93 | 0.91 |
55 | 0.74 | 0.74 | 0.54 | 0.54 |
56 | 0.96 | 0.95 | 0.98 | 0.98 |
57 | 0.99 | 0.99 | 0.57 | 0.56 |
58 | 0.98 | 0.98 | 0.79 | 0.79 |
59 | 0.86 | 0.86 | 0.83 | 0.83 |
60 | 1.07 | 1.06 | 0.7 | 0.69 |
61 | 0.75 | 0.74 | 1.13 | 1.14 |
62 | 0.91 | 0.92 | 1.33 | 1.33 |
63 | 0.63 | 0.63 | 0.96 | 0.96 |
64 | 0.73 | 0.73 | 1.12 | 1.11 |
65 | 1.12 | 1.12 | 0.56 | 0.56 |
66 | 1 | 1 | 1.07 | 1.07 |
67 | 1.07 | 1.07 | 0.87 | 0.87 |
68 | 1 | 0.99 | 0.71 | 0.71 |
69 | 0.84 | 0.84 | 0.62 | 0.63 |
70 | 0.65 | 0.66 | 1.03 | 1.02 |
71 | 0.83 | 0.84 | 0.77 | 0.76 |
72 | 0.73 | 0.74 | 0.77 | 0.76 |
73 | 1.09 | 1.08 | 0.98 | 0.97 |
74 | 1.13 | 1.12 | 0.9 | 0.9 |
75 | 0.89 | 0.89 | 0.94 | 0.94 |
76 | 1.08 | 1.08 | 1.37 | 1.37 |
77 | 1.02 | 1.02 | 1.13 | 1.13 |
78 | 0.91 | 0.91 | 0.97 | 0.97 |
79 | 0.95 | 0.95 | 1.29 | 1.29 |
80 | 0.83 | 0.83 | 1.36 | 1.37 |
81 | 1.05 | 1.05 | 0.63 | 0.61 |
82 | 1.24 | 1.24 | 0.8 | 0.8 |
83 | 1.12 | 1.13 | 1.28 | 1.27 |
84 | 1.02 | 1.02 | 0.77 | 0.78 |
85 | 0.92 | 0.93 | 1.1 | 1.11 |
86 | 1.05 | 1.05 | 1.09 | 1.09 |
87 | 0.57 | 0.58 | 0.96 | 0.96 |
88 | 0.87 | 0.87 | 0.72 | 0.73 |
89 | 0.82 | 0.82 | 0.95 | 0.95 |
90 | 1.16 | 1.16 | 1.05 | 1.05 |
91 | 0.95 | 0.95 | 0.96 | 0.97 |
92 | 0.85 | 0.85 | 1.43 | 1.42 |
93 | 1.17 | 1.16 | 0.66 | 0.66 |
94 | 1.12 | 1.13 | 1.1 | 1.13 |
95 | 0.78 | 0.78 | 1.04 | 1.04 |
96 | 0.97 | 0.97 | 1.35 | 1.35 |
97 | 1.02 | 1.02 | 1.21 | 1.21 |
98 | 0.97 | 0.97 | 0.67 | 0.67 |
Appendix A.6. Descriptive Statistics of Engagement and Public Speaking Competence for Four Groups in Pre-Test and Post-Test
Cognitive Engagement | Behavioral Engagement | Emotional Engagement | Total Engagement | Public Speaking Competence | |||||||
Group | Pre | Post | Pre | Post | Pre | Post | Pre | Post | Pre | Post | |
N | Group 1 | 25 | 25 | 25 | 25 | 25 | 25 | 25 | 25 | 25 | 25 |
Group 2 | 24 | 24 | 24 | 24 | 24 | 24 | 24 | 24 | 24 | 24 | |
Group 3 | 25 | 25 | 25 | 25 | 25 | 25 | 25 | 25 | 25 | 25 | |
Group 4 | 24 | 24 | 24 | 24 | 24 | 24 | 24 | 24 | 24 | 24 | |
Mean | Group 1 | 24 | 26 | 21.8 | 23.4 | 13.6 | 14 | 60.2 | 63.4 | 2.73 | 2.93 |
Group 2 | 25.50 | 23.3 | 22.7 | 22.2 | 13.8 | 12.8 | 62 | 58.2 | 3.04 | 3.11 | |
Group 3 | 23.30 | 24.4 | 20 | 22.6 | 12.4 | 13.4 | 55.6 | 60.3 | 2.82 | 3.04 | |
Group 4 | 24.40 | 23.2 | 20.8 | 21.5 | 13.5 | 12.5 | 58.6 | 57.2 | 2.91 | 3.22 | |
SD | Group 1 | 3.22 | 2.66 | 3.16 | 2.94 | 1.22 | 1.38 | 6.19 | 5.38 | 0.327 | 0.42 |
Group 2 | 2.65 | 2.49 | 3.34 | 2.99 | 1.26 | 1.26 | 6.24 | 5.96 | 0.243 | 0.289 | |
Group 3 | 2.63 | 2.43 | 2.86 | 2.43 | 1.8 | 1.55 | 6.12 | 4.31 | 0.345 | 0.366 | |
Group 4 | 2.53 | 2.96 | 3.25 | 2.83 | 1.74 | 2 | 5.83 | 6.83 | 0.281 | 0.318 | |
Skewness | Group 1 | −0.54 | −0.677 | −0.05 | −0.374 | −0.473 | −1.13 | −0.257 | −0.452 | −0.203 | −0.844 |
Group 2 | 0.09 | 1.18 | 0.492 | 0.917 | −0.338 | 0.801 | 0.477 | 1.17 | −0.81 | −1.11 | |
Group 3 | −1.60 | −0.174 | −0.118 | −0.305 | 0.108 | −0.882 | −0.327 | 0.642 | 0.306 | 0.676 | |
Group 4 | −0.06 | 0.293 | 0.003 | 0.293 | −0.621 | −0.267 | −0.008 | 0.237 | −0.405 | 0.325 | |
Kurtosis | Group 1 | −0.30 | 0.799 | −0.602 | −0.961 | −0.892 | −0.122 | −0.898 | −0.016 | −0.578 | 1.48 |
Group 2 | −0.35 | 1.23 | −0.428 | 1.24 | −1.6 | −0.521 | −0.656 | 1.28 | 0.593 | 2.36 | |
Group 3 | 3.40 | −0.644 | −0.502 | −0.154 | −0.416 | 1.01 | −0.367 | −0.129 | 1.53 | 0.54 | |
Group 4 | −1.27 | −0.795 | −1.34 | 0.156 | −1.21 | −1.15 | −1.16 | −0.192 | −0.63 | 1.19 | |
N = number; Post = post-test; Pre = pre-test; SD = standard deviation. |
References
- Lucas, S.; Stob, P. The Art of Public Speaking, 13th ed.; McGraw-Hill Education: New York, NY, USA, 2020; ISBN 978-1-259-92460-6. [Google Scholar]
- Hattie, J.; Timperley, H. The power of feedback. Rev. Educ. Res. 2007, 77, 81–112. [Google Scholar] [CrossRef]
- van Ginkel, S.; Gulikers, J.; Biemans, H.; Mulder, M. The impact of the feedback source on developing oral presentation competence. Stud. High. Educ. 2017, 42, 1671–1685. [Google Scholar] [CrossRef]
- Wu, J.G.; Miller, L. Improving English learners’ speaking through mobile-assisted peer feedback. RELC J. 2020, 51, 168–178. [Google Scholar] [CrossRef]
- Banister, C. Exploring peer feedback processes and peer feedback meta-dialogues with learners of academic and business English. Lang. Teach. Res. 2020, 27, 746–764. [Google Scholar] [CrossRef]
- Carless, D. Sustainable feedback and the development of student self-evaluative capacities. In Reconceptualising Feedback in Higher Education: Developing Dialogue with Students; Merry, S., Price, M., Carless, D., Taras, M., Eds.; Routledge: London, UK, 2013; pp. 114–122. [Google Scholar]
- Price, M.; Handley, K.; Millar, J. Feedback: Focusing attention on engagement. Stud. High. Educ. 2011, 36, 879–896. [Google Scholar] [CrossRef]
- Handley, K.; Price, M.; Millar, J. Beyond ‘doing time’: Investigating the concept of student engagement with feedback. Oxf. Rev. Educ. 2011, 37, 543–560. [Google Scholar] [CrossRef]
- Liu, N.-F.; Carless, D. Peer feedback: The learning element of peer assessment. Teach. High. Educ. 2006, 11, 279–290. [Google Scholar] [CrossRef]
- Simpson, T.; Holden, K.; Merrick, D.; Dawson, S.; Bedford, L. Does video feedback & peer observation offer a valid method of reinforcing oral presentation training for undergraduate biochemists? High. Educ. Pedagog. 2019, 4, 262–283. [Google Scholar] [CrossRef]
- Xu, Q.; Peng, H. Exploring learner motivation and mobile-assisted peer feedback in a business English speaking course. J. Comput. Assist. Learn. 2022, 38, 1033–1045. [Google Scholar] [CrossRef]
- Murillo-Zamorano, L.R.; Montanero, M. Oral presentations in higher education: A comparison of the impact of peer and teacher feedback. Assess. Eval. High. Educ. 2018, 43, 138–150. [Google Scholar] [CrossRef]
- Day, I.N.Z.; Saab, N.; Admiraal, W. Online peer feedback on video presentations: Type of feedback and improvement of presentation skills. Assess. Eval. High. Educ. 2022, 47, 183–197. [Google Scholar] [CrossRef]
- Ajjawi, R.; Kent, F.; Broadbent, J.; Tai, J.H.-M.; Bearman, M.; Boud, D. Feedback that works: A realist review of feedback interventions for written tasks. Stud. High. Educ. 2022, 47, 1343–1356. [Google Scholar] [CrossRef]
- Hung, S.-T.A. Enhancing feedback provision through multimodal video technology. Comput. Educ. 2016, 98, 90–101. [Google Scholar] [CrossRef]
- Huang, S.-C. Understanding learners’ self-assessment and self-feedback on their foreign language speaking performance. Assess. Eval. High. Educ. 2016, 41, 803–820. [Google Scholar] [CrossRef]
- Murphy, K.; Barry, S. Feed-forward: Students gaining more from assessment via deeper engagement in video-recorded presentations. Assess. Eval. High. Educ. 2016, 41, 213–227. [Google Scholar] [CrossRef]
- Christianson, M.; Hoskins, C.; Watanabe, A. Evaluating the effectiveness of a video-recording based self-assessment system for academic speaking. Lang. Res. Bull. 2009, 24, 1–15. [Google Scholar]
- Liu, T.; Aryadoust, V. The Effect of In-Class and One-on-One Video Feedback on EFL Learners’ English Public Speaking Competency and Anxiety. Stud. Lang. Assess. 2022, 11, 25–57. [Google Scholar] [CrossRef]
- De Grez, L.; Valcke, M.; Roozen, I. The Impact of an Innovative Instructional Intervention on the Acquisition of Oral Presentation Skills in Higher Education. Comput. Educ. 2009, 53, 112–120. [Google Scholar] [CrossRef]
- van Ginkel, S.; Gulikers, J.; Biemans, H.; Mulder, M. Fostering Oral Presentation Performance: Does the Quality of Feedback Differ When Provided by the Teacher, Peers or Peers Guided by Tutor? Assess. Eval. High. Educ. 2017, 42, 953–966. [Google Scholar] [CrossRef]
- van Ginkel, S.; Gulikers, J.; Biemans, H.; Mulder, M. Towards a Set of Design Principles for Developing Oral Presentation Competence: A Synthesis of Research in Higher Education. Educ. Res. Rev. 2015, 14, 62–80. [Google Scholar] [CrossRef]
- Vattøy, K.-D.; Gamlem, S.M.; Rogne, W.M. Examining Students’ Feedback Engagement and Assessment Experiences: A Mixed Study. Stud. High. Educ. 2021, 46, 2325–2337. [Google Scholar] [CrossRef]
- Ellis, R. Epilogue: A Framework for Investigating Oral and Written Corrective Feedback. Stud. Second Lang. Acquis. 2010, 32, 335–349. [Google Scholar] [CrossRef]
- Fredricks, J.A.; Blumenfeld, P.C.; Paris, A.H. School Engagement: Potential of the Concept, State of the Evidence. Rev. Educ. Res. 2004, 74, 59–109. [Google Scholar] [CrossRef]
- Han, Y.; Hyland, F. Exploring Learner Engagement with Written Corrective Feedback in a Chinese Tertiary EFL Classroom. J. Second Lang. Writ. 2015, 30, 31–44. [Google Scholar] [CrossRef]
- Storch, N.; Wigglesworth, G. Learners’ Processing, Uptake, and Retention of Corrective Feedback on Writing: Case Studies. Stud. Second Lang. Acquis. 2010, 32, 303–334. [Google Scholar] [CrossRef]
- Ferris, D.R.; Liu, H.; Sinha, A.; Senna, M. Written Corrective Feedback for Individual L2 Writers. J. Second Lang. Writ. 2013, 22, 307–329. [Google Scholar] [CrossRef]
- Wang, B.; Yu, S.; Zheng, Y.; Teo, T. Student Engagement with Teacher Oral Feedback in EFL University Classrooms. Lang. Teach. Res. 2022; advanced online publication. [Google Scholar] [CrossRef]
- Mahfoodh, O.H.A. “I Feel Disappointed”: EFL University Students’ Emotional Responses towards Teacher Written Feedback. Assess. Writ. 2017, 31, 53–72. [Google Scholar] [CrossRef]
- Zheng, Y.; Yu, S. Student Engagement with Teacher Written Corrective Feedback in EFL Writing: A Case Study of Chinese Lower-Proficiency Students. Assess. Writ. 2018, 37, 13–24. [Google Scholar] [CrossRef]
- Laranjeira, M.; Teixeira, M.O. Relationships between Engagement, Achievement and Well-Being: Validation of the Engagement in Higher Education Scale. Stud. High. Educ. 2024; Advanced Online Publication. [Google Scholar] [CrossRef]
- Fredricks, J.A.; McColskey, W. The Measurement of Student Engagement: A Comparative Analysis of Various Methods and Student Self-Report Instruments. In Handbook of Research on Student Engagement; Christenson, S.L., Reschly, A.L., Wylie, C., Eds.; Springer: Boston, MA, USA, 2012; pp. 763–782. ISBN 978-1-4614-2018-7. [Google Scholar]
- Thomson, S.; Rucker, M.L. The Development of a Specialized Public Speaking Competency Scale: Test of Reliability. Commun. Res. Rep. 2002, 19, 18–28. [Google Scholar] [CrossRef]
- Bandura, A. Self-Efficacy: Toward a Unifying Theory of Behavioral Change. Psychol. Rev. 1977, 84, 191–215. [Google Scholar] [CrossRef]
- Linacre, J.M. Winsteps Rasch Measurement Computer Program 2023, Version 5.1.0; Winstep.com: Portland, OR, USA, 2021. [Google Scholar]
- Linacre, J.M. Sample Size and Item Calibration Stability. Rasch Meas. Trans. 1994, 7, 328. [Google Scholar]
- Aryadoust, V.; Ng, L.Y.; Sayama, H. A Comprehensive Review of Rasch Measurement in Language Assessment: Recommendations and Guidelines for Research. Lang. Test. 2021, 38, 6–40. [Google Scholar] [CrossRef]
- Linacre, J.M. A User’s Guide to Winsteps: Rasch-Model Computer Program; Winstep.com: Portland, OR, USA, 2022; ISBN 0-941938-03-04. [Google Scholar]
- Aryadoust, V.; Mehran, P.; Alizadeh, M. Validating a Computer-Assisted Language Learning Attitude Instrument Used in Iranian EFL Context: An Evidence-Based Approach. Comput. Assist. Lang. Learn. 2016, 29, 561–595. [Google Scholar] [CrossRef]
- Linacre, J.M. Facets Computer Program for Many-Facet Rasch Measurement, Version 2.83.2; Winstep.com: Beaverton, OR, USA, 2020.
- Aryadoust, V. Self- and Peer Assessments of Oral Presentations by First-Year University Students. Educ. Assess. 2015, 20, 199–225. [Google Scholar] [CrossRef]
- Linacre, J.M. Many-Facet Rasch Measurement, 2nd ed.; MESA Press: Chicago, IL, USA, 1994; ISBN 978-0-941938-02-0. [Google Scholar]
- Knapp, T.R.; Schafer, W.D. From Gain Score t to ANCOVA f (and Vice Versa). Pract. Assess. Res. Eval. 2009, 14, 6. [Google Scholar] [CrossRef]
- Roever, C.; Phakiti, A. Quantitative Methods for Second Language Research: A Problem-Solving Approach, 1st ed.; Routledge: New York, NY, USA, 2017; ISBN 978-0-203-06765-9. [Google Scholar]
- Kim, Y.; Steiner, P.M. Gain Scores Revisited: A Graphical Models Perspective. Sociol. Methods Res. 2021, 50, 1353–1375. [Google Scholar] [CrossRef]
- IBM Corp. IBM SPSS Statistics for Windows, Version 26; IBM Corp.: Armonk, NY, USA, 2019.
- Carless, D.; Boud, D. The Development of Student Feedback Literacy: Enabling Uptake of Feedback. Assess. Eval. High. Educ. 2018, 43, 1315–1325. [Google Scholar] [CrossRef]
- Nicol, D.J.; Macfarlane-Dick, D. Formative Assessment and Self-regulated Learning: A Model and Seven Principles of Good Feedback Practice. Stud. High. Educ. 2006, 31, 199–218. [Google Scholar] [CrossRef]
- Xu, Y.; Carless, D. ‘Only True Friends Could Be Cruelly Honest’: Cognitive Scaffolding and Social-Affective Support in Teacher Feedback Literacy. Assess. Eval. High. Educ. 2017, 42, 1082–1094. [Google Scholar] [CrossRef]
- Nicol, D.; Thomson, A.; Breslin, C. Rethinking Feedback Practices in Higher Education: A Peer Review Perspective. Assess. Eval. High. Educ. 2014, 39, 102–122. [Google Scholar] [CrossRef]
Test | Subscale | 1st Contrast | Person Separation | Person Reliability | Item Separation | Item Reliability | Item Infit MnSq | Item Outfit MnSq |
---|---|---|---|---|---|---|---|---|
Pre | Cognitive | 1.80 | 1.57 | 0.71 | 1.53 | 0.70 | 0.80–1.20 | 0.80–1.15 |
Behavioral | 1.48 | 1.35 | 0.65 | 5.79 | 0.97 | 0.70–1.23 | 0.69–1.09 | |
Emotional | 1.79 | 1.28 | 0.62 | 3.41 | 0.92 | 0.74–1.48 | 0.67–1.56 | |
Post | Cognitive | 1.77 | 1.67 | 0.74 | 2.51 | 0.86 | 0.78–1.24 | 0.82–1.18 |
Behavioral | 1.58 | 1.51 | 0.70 | 5.68 | 0.97 | 0.81–1.27 | 0.82–1.29 | |
Emotional | 1.64 | 1.15 | 0.57 | 2.65 | 0.88 | 0.95–1.05 | 0.90–1.07 |
Test | Observed Average | Fair Average | Measure (logits) | Model SE | Infit MnSq | Outfit MnSq |
---|---|---|---|---|---|---|
Pre | 2.03–3.66 | 2.02–3.67 | −2.92–1.69 | 0.18–.21 | 0.57–1.47 | 0.58–1.45 |
Post | 1.71–4.05 | 1.72–4.05 | −4.32–3.30 | 0.18–.22 | 0.54–1.47 | 0.54–1.45 |
Sample Size | Cognitive Engagement | Behavioral Engagement | Emotional Engagement | Total Engagement | |||||
---|---|---|---|---|---|---|---|---|---|
N | M | SD | M | SD | M | SD | M | SD | |
Group 1 | 25 | 1.16 | 3.94 | 1.56 | 3.51 | 0.40 | 1.78 | 3.12 | 7.74 |
Group 2 | 24 | −2.29 | 2.16 | −0.46 | 2.64 | −1.00 | 1.02 | −3.75 | 3.77 |
Group 3 | 25 | 1.04 | 2.57 | 2.60 | 3.40 | 1.04 | 2.28 | 4.68 | 6.33 |
Group 4 | 24 | −1.17 | 3.58 | 0.71 | 3.01 | −0.96 | 1.99 | −1.42 | 6.50 |
Group | M | SD | t-Value | df | p-Value | Cohen’s d |
---|---|---|---|---|---|---|
Cognitive engagement | ||||||
Group 1 | 1.16 | 3.94 | 1.471 | 24 | 0.154 | 0.294 |
Group 2 | −2.29 | 2.16 | −5.206 | 23 | <0.001 *** | −1.063 |
Group 3 | 1.04 | 2.57 | 2.021 | 24 | 0.055 | 0.404 |
Group 4 | −1.17 | 3.58 | −1.595 | 23 | 0.124 | −0.326 |
Behavioral engagement | ||||||
Group 1 | 1.56 | 3.51 | 2.22 | 24 | 0.036 * | 0.444 |
Group 2 | −0.46 | 2.64 | −0.851 | 23 | 0.403 | −0.174 |
Group 3 | 2.60 | 3.40 | 3.82 | 24 | 0.001 ** | 0.764 |
Group 4 | 0.71 | 3.01 | 1.151 | 23 | 0.261 | 0.235 |
Emotional engagement | ||||||
Group 1 | 0.40 | 1.78 | 1.124 | 24 | 0.272 | 0.225 |
Group 2 | −1.00 | 1.02 | −4.796 | 23 | <0.001 *** | −0.979 |
Group 3 | 1.04 | 2.28 | 2.279 | 24 | 0.032 * | 0.456 |
Group 4 | −0.96 | 1.99 | −2.361 | 23 | 0.027 * | −0.482 |
Total engagement | ||||||
Group 1 | 3.12 | 7.74 | 2.015 | 24 | 0.055 | 0.403 |
Group 2 | −3.75 | 3.77 | −4.876 | 23 | <0.001 *** | −0.995 |
Group 3 | 4.68 | 6.33 | 3.697 | 24 | 0.001 ** | 0.739 |
Group 4 | −1.42 | 6.50 | −1.068 | 23 | 0.297 | −0.218 |
PSC | TE | CE | BE | EE | |
---|---|---|---|---|---|
PSC | - | ||||
TE | 0.233 * | - | |||
CE | 0.189 | 0.835 *** | - | ||
BE | 0.210 * | 0.816 *** | 0.445 *** | - | |
EE | 0.148 | 0.734 *** | 0.485 *** | 0.452 *** | - |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Liu, T.; Aryadoust, V. Orchestrating Teacher, Peer, and Self-Feedback to Enhance Learners’ Cognitive, Behavioral, and Emotional Engagement and Public Speaking Competence. Behav. Sci. 2024, 14, 725. https://doi.org/10.3390/bs14080725
Liu T, Aryadoust V. Orchestrating Teacher, Peer, and Self-Feedback to Enhance Learners’ Cognitive, Behavioral, and Emotional Engagement and Public Speaking Competence. Behavioral Sciences. 2024; 14(8):725. https://doi.org/10.3390/bs14080725
Chicago/Turabian StyleLiu, Tingting, and Vahid Aryadoust. 2024. "Orchestrating Teacher, Peer, and Self-Feedback to Enhance Learners’ Cognitive, Behavioral, and Emotional Engagement and Public Speaking Competence" Behavioral Sciences 14, no. 8: 725. https://doi.org/10.3390/bs14080725