Ascertaining the Online Learning Behaviors and Formative Assessments Affecting Students’ Academic Performance during the COVID-19 Pandemic: A Case Study of a Computer Science Course
Abstract
:1. Introduction
2. Related Works
3. Research Framework
Research Questions
- RQ1. What are the independent learning behavior variables derived from the LMS logs that are statistically significant in predicting students’ academic performance?
- RQ2. Is performance in formative assessments statistically significant in predicting performance in the summative assessment?
4. Materials and Methods
4.1. Overall Context
4.2. Context of Advanced Data Structures and Algorithms Course
4.3. Data Source
4.4. Data Analysis
5. Results and Discussion
5.1. Correlation and Regression Analysis between the Learning Behavior Variables and the Total Marks
5.2. Correlation and Regression Analysis between the Formative Assessments Marks and the Final Examination Marks
6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Celik, I.; Dindar, M.; Muukkonen, H. Distance education: A study of social media use for k-12 education during the COVID-19 pandemic. Inf. Learn. Sci. 2022, 123, 252–275. [Google Scholar] [CrossRef]
- Peterson, L.; Scharber, C.; Thuesen, A.; Baskin, K. A rapid response to COVID-19: A district’s pivot from technology integration to distance learning. Inf. Learn. Sci. 2020, 121, 461–469. [Google Scholar] [CrossRef]
- Sahu, P. Closure of universities due to coronavirus disease 2019 (COVID-19): Impact on education and mental health of students and academic staff. Cureus 2020, 12, e7541. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- You, J.W. Identifying significant indicators using LMS data to predict course achievement in online learning. Internet High. Educ. 2016, 29, 23–30. [Google Scholar] [CrossRef]
- Dhawan, S. Online learning: A panacea in the time of COVID-19 crisis. J. Educ. Technol. Syst. 2020, 49, 5–22. [Google Scholar] [CrossRef]
- Bao, W. COVID-19 and online teaching in higher education: A case study of Peking university. Hum. Behav. Emerg. Technol. 2020, 2, 113–115. [Google Scholar] [CrossRef] [Green Version]
- Aristovnik, A.; Kerzic, D.; Ravselj, D.; Tomazevic, N.; Umek, L. Impacts of the COVID-19 pandemic on life of higher education students: A global perspective. Sustainability 2020, 12, 8438. [Google Scholar] [CrossRef]
- Rasheed, R.A.; Kamsin, A.; Abdullah, N.A. Challenges in the online component of blended learning: A systematic review. Comput. Educ. 2020, 144, 103701. [Google Scholar] [CrossRef]
- Oliver, M. Online learning helps prepare pupils for university. Educ. J. 2014, 218, 12–15. [Google Scholar]
- Hong, J.-C.; Lee, Y.-F.; Ye, J.-H. Procrastination predicts online self-regulated learning and online learning ineffectiveness during the coronavirus lockdown. Personal. Individ. Differ. 2021, 174, 110673. [Google Scholar] [CrossRef]
- Li, S.; Chen, G.; Xing, W.; Zheng, J.; Xie, C. Longitudinal clustering of students’ self-regulated learning behaviors in engineering design. Comput. Educ. 2020, 153, 103899. [Google Scholar] [CrossRef]
- Mwalumbwe, I.; Mtebe, J.S. Using learning analytics to predict students’ performance in Moodle learning management system: A case of Mbeya University of science and technology. Electron. J. Inf. Syst. Dev. Ctries 2017, 79, 1–13. [Google Scholar] [CrossRef] [Green Version]
- Kim, D.; Park, Y.; Yoon, M.; Jo, I.-H. Toward evidence-based learning analytics: Using proxy variables to improve asynchronous online discussion environments. Internet High. Educ. 2016, 30, 30–43. [Google Scholar] [CrossRef]
- Kew, S.N.; Tasir, Z. Learning analytics in online learning environment: A systematic review on the focuses and the types of student-related analytics data. Technol. Knowl. Learn. 2021, 27, 1–23. [Google Scholar] [CrossRef]
- Montgomery, A.P.; Mousavi, A.; Carbonaro, M.; Hayward, D.V.; Dunn, W. Using learning analytics to explore self-regulated learning in flipped blended learning music teacher education. Br. J. Educ. Technol. 2019, 50, 114–127. [Google Scholar] [CrossRef] [Green Version]
- Gašević, D.; Dawson, S.; Rogers, T.; Gasevic, D. Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. Internet High. Educ. 2016, 28, 68–84. [Google Scholar] [CrossRef] [Green Version]
- Saqr, M.; Fors, U.; Tedre, M. How learning analytics can early predict under-achieving students in a blended medical education course. Med. Teach. 2017, 39, 757–767. [Google Scholar] [CrossRef]
- Bos, N.; Brand-Gruwel, S. Student differences in regulation strategies and their use of learning resources: Implications for educational design. In Proceedings of the Sixth international Conference on learning Analytics & knowledge, Edinburgh, UK, 25–29 April 2016; pp. 344–353. [Google Scholar]
- Ma, J.; Han, X.; Yang, J.; Cheng, J. Examining the necessary condition for engagement in an online learning environment based on learning analytics approach: The role of the instructor. Internet High. Educ. 2015, 24, 26–34. [Google Scholar] [CrossRef]
- Soffer, T.; Cohen, A. Students’ engagement characteristics predict success and completion of online courses. J. Comput. Assist. Learn. 2019, 35, 378–389. [Google Scholar] [CrossRef]
- Conijn, R.; Snijders, C.; Kleingeld, A.; Matzat, U. Predicting student performance from lms data: A comparison of 17 blended courses using Moodle LMS. IEEE Trans. Learn. Technol. 2016, 10, 17–29. [Google Scholar] [CrossRef]
- Riestra-González, M.; del Puerto Paule-Ruíz, M.; Ortin, F. Massive LMS log data analysis for the early prediction of course-agnostic student performance. Comput. Educ. 2021, 163, 104108. [Google Scholar] [CrossRef]
- López-Zambrano, J.; Lara, J.A.; Romero, C. Towards portability of models for predicting students’ final performance in university courses starting from moodle logs. Appl. Sci. 2020, 10, 354. [Google Scholar] [CrossRef] [Green Version]
- Marinoni, G.; Vant Land, H.; Jensen, T. The impact of COVID-19 on higher education around the world. IAU Glob. Surv. Rep. 2020, 23. [Google Scholar]
- Dabbagh, N.; Kitsantas, A. Supporting self-regulation in student centered web-based learning environments. Int. J. E-Learn. 2004, 3, 40–47. [Google Scholar]
- Jovanovic, J.; Saqr, M.; Joksimovic, S.; Gasevic, D. Students matter the most in learning analytics: The effects of internal and instructional conditions in predicting academic success. Comput. Educ. 2021, 172, 104251. [Google Scholar] [CrossRef]
- Figueroa-Canas, J.; Sancho-Vinuesa, T. Investigating the relationship between optional quizzes and final exam performance in a fully asynchronous online calculus module. Interact. Learn. Environ. 2021, 29, 33–43. [Google Scholar] [CrossRef]
Characteristics | Advanced Data Structures and Algorithms |
---|---|
Course duration | 12 weeks |
Year of study | Third year |
Enrolled students | 340 |
No. of lectures | 12 |
No. of workshops | 11 |
Assignment marking rubric provided to students | No |
Assessment type | In-semester assessments (40%) Final examination (60%) |
In-semester assessments (formative assessments) | 3 individual assignments (10% + 20% + 10%) |
Category | Variable Name | Description | Example |
---|---|---|---|
Time management | On time submission of each assignment | Recorded as 1 if the student submitted the assignment before the deadline. Else, it will be recorded as 0 | If a student submits the assignment 2 h before the deadline, the entry will be recorded as 1. If a student submits the assignment immediately after the deadline, the entry will be recorded as 0. |
Time spent on assignment | The duration between the student’s first access to the assignment specifications and the assignment submission, which provides a rough estimate of the time used by the student to complete the assignment. | The duration will be expressed in minutes. If a student spent 4 days between his/her first access to the assignment specifications and the time of submission, the entry will be recorded as 5760 min. | |
Accessed assignment specifications within 1 week of release | Recorded as 1 if the student accessed the assignment specifications at least once within 1 week of release. If not, it will be recorded as 0. | If a student accessed Assignment 1 specifications on the 2nd day after release, the entry will be recorded as 1. If the student does not access the materials at all, the entry will be 0. | |
Accessed assignment specifications within 24 h of the deadline | Recorded as 1 if the student accessed the assignment specifications at least once within 24 h of the due date. If not, it will be recorded as 0. | If a student accessed the assignment specifications 1 day before the assignment deadline, the entry will be 1. If the student does not access the material at all by 24 h before the deadline, the entry will be 0. | |
Number of lecture materials accessed on time | The total number of weeks before the lecture began that a student accessed the lecture materials. | If a student accessed the lecture materials for 5 weeks prior to the lecture, the entry will be recorded as 5. For n number of on-time access to the lecture materials, the entry will be n. | |
Number of workshop materials accessed on time | The total number of weeks before the workshops/tutorial commenced that a student accessed the workshop or tutorial materials. | If a student accessed workshop materials for 3 weeks prior to the workshop, the entry will be recorded as 3. For n number of on-time access to the workshop materials, the entry will be n. | |
Frequency of accessing the learning materials | Number of lecture materials accessed | The total number of lecture materials accessed, according to the lecture materials uploaded. | If a student accessed 5 out of the 12 lecture materials uploaded, the entry will be 5. For n number of access to the lecture materials, the entry will be n. |
Number of workshop materials accessed | The total number of workshop materials accessed, according to the workshop materials uploaded. | If a student accessed 5 out of the 12 workshop materials uploaded, the entry will be 5. For n number of access to the workshop materials, the entry will be n. | |
Lecture material revisits | The total number of times a student has re-visited the lecture material. | If a student has accessed an item of lecture material 8 times, the entry will be 7. n − 1 of total visits will be recorded, with n > 1. | |
Workshop material revisits | The total number of times a student has re-visited the workshop material. | If a student has accessed an item of workshop material 8 times, the entry will be 7. n − 1 of total visits will be recorded, with n > 1. | |
Frequency of accessing the LMS | Total number of clicks in the LMS | The total number of clicks within the course’s LMS page, which includes content viewing, link clicks, document downloads, assignment submissions, and content posting. | If a student has 280 clicks in the LMS, the entry will be 280. For n number of clicks in the LMS, the entry will be n. |
Average number of clicks in the LMS | The average number of clicks by the student in 1 week in the course LMS, which reflects active participation in the course. | If a student has 280 clicks per week over a duration of 12 weeks, the entry will be 23.3. For n number of clicks in the LMS, the entry will be n/number of weeks. | |
Active participation | Assignment submission | Recorded as 1 or 0. If the student submits the assignment, it will be recorded as 1, regardless of being on time or late. If not, it will be recorded as 0. | If a student submits Assignment 1 late by 2 days, the entry will be 1. If a student does not submit an assignment at all, the entry will be 0. |
Writing content in the forum posts or comments | The total number of posts and comments a student has created in the forum. | If 3 posts and 5 comments are created by the student, the entry will be recorded as 8. For n posts created and m comments posted in the forum, the entry will be n + m. | |
Reading content in the forum | The total number of posts and comments read by the student in the forum. | If a student has read 5 topics in the forum, the entry will be recorded as 5. For n topics read in the forum, the entry will be n. | |
Forum access | The total number of times the student has accessed the forum to perform read or write actions. | If a student has accessed the forum 50 times, the entry will be recorded as 50. For n number of accesses to the forum, the entry will be n. |
Category | Variable Name | Correlation (r) |
---|---|---|
Time management | Number of lecture materials accessed on time Number of workshop materials accessed on time Submitted Assignment 1 on time Submitted Assignment 2 on time Assignment 3 submitted on time Accessed Assignment 1 specifications within 1 week of release Accessed Assignment 3 specifications within 1 week of release Accessed Assignment 2 specifications within 24 h of deadline Accessed Assignment 3 specifications within 24 h of deadline Time spent on Assignment 2 Time spent on Assignment 3 | 0.322 0.153 0.182 0.336 0.273 0.180 0.189 0.218 0.133 0.216 0.254 |
Frequency of accessing the learning materials | Number of workshop materials accessed | 0.188 |
Frequency of accessing the LMS | Total number of clicks in the LMS Average number of clicks per week in the LMS | 0.121 0.121 |
Active participation | Writing content in the forum | 0.205 |
Submitted Assignment 1 | 0.238 | |
Submitted Assignment 3 | 0.231 |
Category | Predictors | Unstandardized Beta | R Square | Significance |
---|---|---|---|---|
Time management | Submitted Assignment 2 on time | 7.44 | 0.29 | 0.000 |
Submitted Assignment 3 on time | 4.39 | 0.010 | ||
Lecture materials accessed on time | 1.81 | 0.000 | ||
Active participation | Submitted Assignment 1 | 20.41 | 0.000 | |
Submitted Assignment 3 | 23.56 | 0.000 | ||
Writing content in the forum | 0.42 | 0.005 |
Variable Name | Correlation (r) | Significance (p Value) |
---|---|---|
Assignment 1 marks | 0.435 | 0.000 |
Assignment 2 marks | 0.414 | 0.000 |
Assignment 3 marks | 0.275 | 0.000 |
Predictors | Unstandardized Beta | R Square | Significance |
---|---|---|---|
Assignment 1 marks | 1.16 | 0.000 | |
Assignment 2 marks | 0.63 | 0.25 | 0.000 |
Assignment 3 marks | 0.63 | 0.029 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Ting, T.T.; Teh, S.L.; Wee, M.C. Ascertaining the Online Learning Behaviors and Formative Assessments Affecting Students’ Academic Performance during the COVID-19 Pandemic: A Case Study of a Computer Science Course. Educ. Sci. 2022, 12, 935. https://doi.org/10.3390/educsci12120935
Ting TT, Teh SL, Wee MC. Ascertaining the Online Learning Behaviors and Formative Assessments Affecting Students’ Academic Performance during the COVID-19 Pandemic: A Case Study of a Computer Science Course. Education Sciences. 2022; 12(12):935. https://doi.org/10.3390/educsci12120935
Chicago/Turabian StyleTing, Tin Tin, Shi Lin Teh, and Mee Chin Wee. 2022. "Ascertaining the Online Learning Behaviors and Formative Assessments Affecting Students’ Academic Performance during the COVID-19 Pandemic: A Case Study of a Computer Science Course" Education Sciences 12, no. 12: 935. https://doi.org/10.3390/educsci12120935