Change in Gap Perception within Current Practices in Assessing Students Learning Mathematics

Abstract

The COVID pandemic has touched many aspects of everyone’s life. Education is one of the fields greatly affected by it, as students and teachers were forced to move online and quickly adapt to the online environment. Assessment is a crucial part of education, especially in STEM fields. A gap analysis was performed by expert groups in the frame of an Erasmus+ project looking at the practices of six European countries. Specialists teaching university-grade mathematics in seven European institutions were asked about their perception of gaps in the assessment of students both before (2019) and during (2021) the pandemic. This qualitative study looks at the difference in perception of such gaps after almost one year of online teaching. The analysis of their responses showed that some gaps were present before the pandemic, as well as others that are specific to it. Some gaps, such as the lack of IT infrastructure and the need to adapt materials to an online environment, have been exacerbated by the outbreak.

1. Introduction

A vehicle employed to embed the student within a technology-enabled learning environment in the Higher Education domain is the Virtual Learning Environment (VLE) or Learning Management System (LMS); another vehicle is the use of hosted websites. Such systems may be open-source or proprietary and are increasingly being used to host material, deliver assessments, and in some cases, automatically grade and provide feedback to the student [1]. To expand the capability of the assessment systems, value may be added using tools supplied by externally hosted assessment systems.

Within a technology-enabled or mediated learning environment, it is important to maintain the concept of equity in assessment. Invalid or unfair judgements of students’ performances can have long-lasting and far-reaching negative consequences for the students [2]. Teachers’ judgements can affect short-term behavior towards a student and influence future interpretations of assessment. Teachers’ expectations of students may also contribute towards unfair decisions, particularly within systemic inequities, such as access to the internet off-campus [3,4,5]. Equity in assessment means that all students are given equal opportunities to reveal their achievements and that the instruments of assessment are not biased. The assessment process must be cognizant of cultural, behavioral, and social contexts as unfamiliarity may introduce unintended barriers. Critical to this process of assessment, whether within or outside the technology environment, is the interpretation of the students’ performance.

Open-source or proprietary VLEs/LMS’s are increasingly being used to host and deliver online assessments. Most incorporate their own versions of assessment and assessment types and will typically include—Multiple Choice Questions (MCQ), Blank Filling, Multiple Answer, Calculated Entry, Formula Entry, Short Sentence, Free Text, Hotspot, and Word List.

Within the Higher Education domain, the main systems employed are Moodle, Blackboard, Desire2Learn, and Canvas. Each has its own characteristics, but they all support the hosting of documents, communication tools, basic assessment tools, grading tools, and reporting tools. It is not within the remit of this report to describe each system in-depth, rather the lens will be solely on assessment components.

The current study was performed within an Erasmus+ project called EngiMath (reference 2018-1-EE01-KA203-047098) that was launched in November 2018, involving partners from six European countries: Estonia (project coordinator), Ireland, Spain, Portugal, Poland, and Romania. The goal of this project is the creation and development of a shared medium for materials suitable for web-based assessment of mathematics for various types of engineering education curricula in higher education. The project targets students in engineering mathematics programs, academic staff teaching engineering mathematics, research academics in Technology Enhanced Learning, and online learning, as well as other educational institutions willing to use the knowledge and materials generated within the project. The project aims to produce a report on the pedagogical analysis of the online assessment of mathematics for the partner countries, an online course in engineering mathematics translated into seven languages, and an online assessment model.

The subject of online education is not new, with the first totally online course being created as early as 1981 [6]. Still, the online education we know today is possible due to the advances in technology, such as high-speed internet and online platforms. The change in paradigm happened in 2008 with the advent of the Massive Open Online Course (MOOC) that increased access to higher education for the general public [7]. Since then, there have been many papers written on the subject of online teaching as the subject exploded. This was accelerated further by the outbreak of COVID as educational institutions all over the world were forced to move their activity online. Although there have been papers on online assessment [8,9,10], none have been found on the specific topic of assessment of university students learning mathematics.

The aim of the paper is to compare the perception of specialists teaching university-level mathematics on the gaps in the assessment of students before and during the pandemic. This study compares perceptions that were collected in two very different situations and draw conclusions on the change in paradigm.

The paper is structured in five sections. After the introduction, an overview of student assessment is made with a focus on eAssessment and the tools used. The third section presents the materials and methods used in this study, while the fourth presents the results of the analysis. The last two sections present a discussion on the results, the conclusions, and limitations of the study.

2. Assessment

Assessment is the process of forming a judgement about a student and assigning an outcome based on the judgement. The outcome of the judgement must be fair within the context of the assessment and may be made visible to the student in the form of feedback or a score. As an instrument, the assessment may occur at the unit level or in the form of an overall multi-level instrument, and depending on the form of assessment, the output will be employed for different purposes.

The form of the assessment may be determined by the purpose of the assessment. The purpose of the assessment is outlined by [11] as:

• Diagnostic feedback is used to inform a student about past learning and provides a model for the student to prepare for future learning.
• Formative feedback relates to the performance of the student within a particular recent context. It may inform the student about potential learning gaps or deficiencies.
• Summative assessment considers the culmination of a student’s work. The normal output is a score or mark used to denote competence, skill, ability, knowledge when students wish to progress.
• Evaluative assessment measures the effectiveness of the teaching or the assessment of the students. This type of assessment may contribute to quality audits.

2.1. Assessment of Mathematics

When teachers judge assessments by students, they interpret what they read using their professional judgement. The output of this judgement is an inference of what the students’ have achieved in the learning process, and the teacher will indicate the success or otherwise of the activity. A teacher will use their collective resources as well as individual resources, and these may include their [2]:

• Personal knowledge of mathematics and the curriculum.
• Beliefs about the nature of mathematics and how these relate to assessment.
• Expectations about how mathematics is communicated.
• Experience and impressions of students.
• Expectations of individual students.
• Linguistic skills and cultural background.
• Construction of knowledge about students’ mathematics.

Issues may arise that give cause for concern in the assessment process and result in inequity, namely: inconsistent application of standards, systematic bias, and poorly designed tasks.

When different assessors assess the same work, it is necessary to apply standards in a consistent manner to remove any inequity. The determination of learned capacity by an individual assessor is affected by the resources that a particular teacher may bring to the assessment. A teacher may have more experience and better linguistic skills than partner teachers and arrive at concluding judgements using different approaches. In mathematics, it is possible to arrive at the same conclusion even though the assessors have made different assessment judgements within the context of the assessment. Through shared training and rating, the potential for inequity may be reduced, but it is still subject to the vagaries and nature of human judgement.

Standard training programs for the rating and assessment of students’ learned capacity exercises address issues such as concept, procedure, and modelling. Assessment, where the nature of the activity involves different cultures, races, languages, and socio-economic groupings, may fall foul of inherent disadvantages. A student completing an assessment where the language of the assessment is not the students’ first language may experience inequity, and the assessor may not be able to take this factor into account. Differences in socio-economic grouping through variable access to resources may result in the failure by some students to participate fully or appear to participate fully, resulting in a lowering of teachers’ expectations. Race and culture may result in devaluing the performances of certain student minority groups due to the way an assessment has been conducted. The reliability of the assessments may be beyond reproach; the status of equity may be such that inequity exists due to the systematic way an assessment is judged.

A pivot on which the assessment operates is the task’s design that a teacher wishes to conduct to determine if the learning gap has been reduced and the subsequent students’ learned capacity has increased. The task must be consistently viewed by all, and the design must be in alignment with the knowledge sought by the teacher. If the task is not designed correctly, then the determination of learning is not possible. The use of poor language within a question, leading to ambiguity, has the potential to result in the students’ failure to address the true nature of the assessment. Poor design affects students and those conducting the assessment of the exercise. It may not be possible to compare the students’ mathematical approaches, practices, and procedures consistently; the meaning of the assessment may be misinterpreted. Inequity may arise in situations where poor design causes ambiguity.

2.2. E-Assessment of Mathematics

Assessment of mathematics within a technology-enhanced learning environment, or e-assessment, may be conducted using a variety of assessment methods. E-assessment includes the use of a computer as part of any assessment-related activity ([12], p. 88)—the definition of computer includes any computer device such as mobile phone, tablet, laptop, desktop computer. The degree and level of sophistication within the e-assessment may be determined by diverse “drivers” from institutional requirements to a personal choice by academics. E-assessment has developed particularly over the last twenty years to include a high degree of sophistication for mathematical questioning in the form of Computer Algebra Systems (CAS). It is recognized that the inherent embodiment of questioning and feedback is essential within any e-assessment system [1,13,14,15,16].

The opportunities afforded by e-assessment have provided scope for the expansion and development of students’ learning space [17]. The traditional learning space of the fixed bricks-and-mortar classroom has been transformed through the mediation of technology [18]. The transformation of the learning space relocates the student–teacher nexus within the continuum of physical–virtual spaces with an added temporal component. Cognizance of the shift in space must be recognized within a pedagogically sound curriculum with resultant re-imagining and consideration of the paradigm. Redecker and Johannessen [15] recognized this issue as a conflict between two assessment paradigms—computer-based testing versus embedded assessment ([15], p. 80). The predicted movement towards personalized learning involving gamification, virtual worlds, simulation, and peer assessment in the 4th generation of computer-based assessment, resulted in a call for a paradigm re-alignment in assessment. However, despite a considerable range of assessment formats on offer through fourth-generation technologies, the traditional assessment paradigm is still paramount. A conceptual shift is required to re-imagine the model of assessment to ensure it meets the expectations of all stakeholders in the learning process.

According to Hunt, cited in [1], approximately 90% of question types used within VLEs are selected response, such as MCQ. Selected response systems overcome issues of data entry where students incorrectly enter an invalid or unrecognized parameter; this has been reported by students as a common barrier to progress [4].

Invalid entry is a particular issue in the assessment of mathematics using online systems when symbolic notation is required. The assessor, in using selective response, brings a sense of objectivity to the task of assessment when using selective response questioning but must be careful that the result cannot be deduced from the options provided. It may be possible for students to guess the answers or use assessment-savvy or test-wise techniques. A major issue to be considered when using MCQ-type assessments is authenticity. Problems in real life are not solved through MCQs!

The authenticity of the assessment combined with the requirement for objectivity is fundamental to the learning process; the analysis of mathematics by e-assessment should imitate the actions of the human assessor. One avenue employed to address these requirements is the use of Computer Algebra Systems (CAS).

Each assessment system is designed to provide access to a variety of assessment tools offering different degrees of complexity. NUMBAS and STACK do not involve any financial outlay to access or use them, whereas iSpring and Maple TA require licenses to be purchased—the costing is provided on a cloud or server basis and is dependent on the number of licenses required. Assessment systems are also available within the standard VLEs employed in higher education, e.g., Blackboard, Moodle, Canvas, Brightspace.

Table 1. Comparison of virtual assessment systems.

Assessment System/Assessment Tool	Blackboard	Moodle	Canvas	BRIGHTSPACE	NUMBAS	STACK	Maple TA	iSpring
Calculated Formula	x	x	x	x	x	x	x
Calculated Numeric	x	x	x	x	x	x		x
Either/Or	x	x	x	x
Essay	x	x	x	x			x
Single Blank	x	x	x	x	x		x	x
Multiple Blanks	x	x	x	x	x		x	x
Hot Spot	x	x	x	x			x	x
Jumbled Sentence	x	x	x	x
Matching	x	x	x	x	x			x
Multiple Answer	x	x	x	x	x			x
Multiple Choice	x	x	x	x	x		x	x
Likert	x	x	x	x
Ordering	x	x	x	x				x
Quiz Bowl	x	x	x	x
Short Answer	x	x	x	x
True/False	x	x	x	x			x	x
Type In								x
Sequence								x
Text String					x	x
Reveal Steps/ Multi-Part					x	x	x
LaTeX					x	x	x
Matrix Entry					x	x	x
Symbolic Editor						x	x
2-D Plot						x	x
3-D Plot							x
App expansion						x	x

provides a comparative breakdown for each system.

Even though the partners involved in the frame project of the current study are all part of the European Union, the educational systems in each country have differences in admission criteria, curricula, and assessment. Each partner country has its own grading system and higher education admission criteria. For example, Estonia has a 6-point grading scale from 0 to 5 [19] and a minimum average national grade from secondary school of 3.4 is required for admission, while in Ireland, the average entry score is 190 of a maximum of 550 points [20]. Poland requires students to achieve at least 30% in the national examinations at the secondary level; Portugal requires 150–197 points for entry out of 200; in Spain, the average score is 10 out of 14; in Romania, students require 6 out of 10 points at the Baccalaureate exam for entry in higher education [20].

Regarding assessment, neither partner uses any diagnostic assessment, while formative assessment is done differently, either through observation (Estonia and Ireland), through seminars (Romania) or tutorials (Spain) or proposed step-by-step (Portugal) [20]. The summative assessment is done with assignments and final tests [20].

Before the pandemic, all institutions involved in the project used offline assessment, and some used online tools to evaluate students either manually (Estonia, Spain) or automatically (Estonia, Ireland, Portugal) [20]. This has created difficulties for some institutions that were only assessing students offline while others that had online assessment tools already in use had a smoother transition to online assessment.

3. Materials and Methods

The current study is a qualitative study and was performed in a two-step process. A group of 10 academics involved in teaching mathematics at a university level in the partner institutions were asked to fill in a form regarding the gaps they encountered in their practices. All respondents were involved in teaching mathematics to students at a university level. The form that was disseminated among teachers contained the following open-ended questions:

• What are the gaps within practices that you have identified in this period?
• Are the gaps known to the target audience?
• How big (serious) is the gap?
• What motivation exists to minimize the gap?
• What barriers exist to addressing the gap
• Is the gap an existing gap or an emergent gap?
• How has the gap been identified?
• How will changes in student behaviors be identified and measured?

The respondents came from all the partner institutions, and the responses were collected between February and March 2019.

In the second step, the same form was again distributed to academics from partner institutions during the period January–February 2021, after partial and full lockdowns in all partner countries. Thirteen responses were collected, aggregated, and analyzed. A brief overview of the collected data is presented in

Table 2. Description of responses by country of origin and institution.

Country of Origin	Institution	No. of Respondents
		2019	2021
Estonia	Tallinna Tehnikakorgkool (TTK UAS)	2	2
Portugal	Instituto Politecnico do Porto (IPP)	2	1
Spain	Universidad del País Vasco (UPV/EHU)	1	0
	Universitat Politècnica de Catalunya (UPC)	0	3
Ireland	Letterkenny Institute of Technology (LYIT)	2	0
Poland	Politechnika Koszalinska (PK TUK)	2	1
Romania	Universitatea Tehnica Cluj-Napoca (UTC)	1	6
	Total	10	13

. Between the two collection periods, the Spanish partner left the project, and another partner took their place.

Online word cloud generation software was used to identify the most common terms but was of limited use, as the words were not in context. Since there was a manageable amount of information, the analysis was done manually to capture the sentiment and relevant meaning of each answer. The answers were then compared with the responses from the previous period grouped by question, and conclusions were drawn based on the differences between them.

4. Results

The purpose of a Gap Analysis [21] is to support, foster, and develop the collaborative, shared, and innovative pedagogical model for teaching engineering mathematics. A gap analysis will guide the development of the pedagogical model based on the partners’ responses to a series of targeted questions.

The expert group method uses the expert knowledge of the partners to determine the salient issues, barriers, misunderstandings, and rhetoric employed in the design of engineering mathematics programs.

4.1. Identification of Gaps within Current Practices

Before the pandemic, the primary identified gap was that automated assessment systems only consider the product of the assessment. The ability to enter a correct answer whilst not demonstrating mastery or knowledge of concepts, applications, or procedures may reward a student; however, the system is not able to honestly report that the student has met the learning outcomes. An assumption of syntactic literacy is made for access to online assessment systems and that all students will have equality of access to the system. There is an insufficient spread of online assessment techniques. The reluctance of academics to accept the importance of eAssessment and expectations of digital and coding literacy. The gaps in expected knowledge of concepts, practices, and procedures from students from second-level education is a constant worry.

During the pandemic, the most hard felt gap was the lack of physical contact between students and teachers besides the deficiencies in student assessment. The lack of physical contact with the students had a big impact on the way classes are taught, mostly because of the lack of feedback from the students. Teachers cannot gauge if the students understand the concepts being taught, and they cannot rely on the non-verbal cues they are used to, and this results in lectures progressing slowly. One teacher responded: “Mainly, the blackboard explanations and the physical contact with students [are lacking]. However online resources have notably improved, they cannot cover this gap. There is also a huge difficulty for the students to immerse themselves into the theory.” The lack of a well-established assessment method that would be fair and objective and would make cheating not viable is a major worry. The lack of access to reliable technology by the students deepens this gap. Some students do not have the internet speed or computing power necessary for taking part in the class and are sometimes forced to use a smartphone instead, which do not provide the same learning experience. This is also coupled with internet or power outages that affect both learners and teachers.

A lack of digital literacy among teachers was identified, resulting in materials that are inadequate for online use. Some teachers use their offline materials without any adaptation to the online teaching method, resulting in poor materials with a low level of interactivity. This makes the students become disengaged, especially when they are not prepared or motivated enough to work independently. This prevents students from immersing themselves in the theory, and therefore, they do not understand it properly. This is worsened by the inadequate learning environment of some students that sometimes must take part in lectures from busy public places to have a good internet connection or have domestic chores such as taking care of siblings while parents are away. This directly impacts their performance during assessments. Teachers also feel some drawbacks of online teaching as it takes considerably longer to prepare for online classes, there is the assumption of permanent availability from the part of the students and that there are no incentives from the university.

4.2. Knowledge of the Gaps by the Target Audience

The gaps within the current practices were not known to the students before the start of the pandemic. Limited awareness of gaps existed for the majority of academics while the higher education institutions were generally aware of gaps in access equality. Some students were aware of gaps in their knowledge upon entering higher education and knew that this may have affected their performance. Some academics are unwilling to address their gaps in e-assessment design.

This changed in the context of the pandemic, as most respondents considered that these gaps are known to students. Even if some of these gaps were not particularly obvious before the pandemic, they are now. Ore responded said: “We think so, in some cases and at least now they know. But before pandemic lockout we are sure that several issues were not in their minds and still aren’t.” They realize that teachers need to quickly adapt to this new environment and sometimes even report some of the mentioned gaps.

4.3. Seriousness of the Gap

The gap in the quantity and quality of online assessments was being addressed and considered minimally. Expectations of digital literacy were moderately serious and could introduce barriers to the learning experience for some students. Automated assessment issues were considered moderately serious if not attended to by academics. Consideration should be given for partial credit, particularly in the early transitional stages. Expectations of equality of access were considered a barrier for some students and may have been cultural in nature. Coding literacy was required to provide solutions on some online assessment systems—concentration on coding instead of mathematics literacy may have been disadvantageous. The academic transition to e-assessment was considered slow and problematic to ensure that correct models and practices were employed.

Now, opinions about the seriousness of the gap are divided. Some feel that the gap is very serious, especially aspects related to physical interaction and assessment. Although these aspects can be managed through more tutoring classes, better materials, and access to more reliable technology, they cannot replace face-to-face teaching and assessing students. Returning to face-to-face learning will reduce some gaps but still, some others need to be addressed. Teachers feel that students are not yet aware that with online learning, more of the responsibility of learning falls on them rather than the teachers. One responded said: “The majority of the students (at least) from 1st degrees still are not ‘mature’ enough to assume the responsibility, that is automatically given to them, when switching to distance learning. The teacher remains (in their minds) responsible for their teaching and they still feel they are mere ‘receptors’ of knowledge: ‘the teacher speaks—I learn’. Research, the search for knowledge has not yet been felt by many students as a responsibility on their side regarding their academic training.” On the other hand, some feel that the identified gaps are not that serious, as the students that want to learn will quickly adapt to this new environment, and their impact can be minimized if face-to-face assessment is allowed. This was articulated by one teacher: “Not very serious. Those who wanted could learn. And the process went smoothly.” Nevertheless, access to the learning system and the materials have been mentioned as being problematic for some respondents.

4.4. Motivation to Minimize the Gap

The overall learner experience is paramount to retention in programs. The experience may also affect the psychographic components of the student profile. The reduction of the barriers or gaps may improve and address areas of contention. The learners are motivated for doable tasks, and they will be motivated toward a course that provides the appropriate level of cognitive challenge but that they perceive as achievable. The relationship between the subject and how they relate to the specialism may motivate the students. Motivation may also come from institutional support.

Returning to a face-to-face setting will minimize some of the identified gaps, and the pandemic has revealed some aspects that can be improved. The importance of a well-functioning learning platform with high-quality, interactive materials used to engage students during classes and well-developed assessment materials is recognized. Other aspects cannot be replaced through online learning, such as physical interaction and verbal and non-verbal feedback from students.

4.5. Barriers to Addressing the Gap

Before, good techniques were not available free of charge. Students were unwilling to study the “logic of using tests” in addition to the subject. The attitudes of the institution, inflexibility of academics, and access to technology were considered important barriers to addressing the gap together with the resistance by academics, lack of technicians, access to technology, recurrent financial restrictions, and commitment of the institution. The lack of self-awareness by students and academics of their abilities, knowledge, and contribution in online environments compounded this effect.

Some barriers to addressing the gap are specific to the pandemic, such as the lack of physical contact. Nevertheless, this period has outlined the need for investment in technology and reliable internet infrastructure that is a barrier for students to take part in the online learning process. The respondents also feel that teachers need training and assistance in using digital tools for both creating high-quality materials and conducting courses in an interactive and engaging manner. This is aggravated by the need for great amounts of time to develop materials and the lack of internal and external motivation of the teachers. Additionally, the socially acceptable behavior of cheating among some students is accentuated in the online environment because of the lack of anti-fraud software. One respondent said: “On the other hand, when addressing the cheating, we feel that fraud is still socially accepted among our students, and the weight of social recrimination is not felt by the fact—there are very few teachers who file cases with students for fraud, limiting themselves in most cases to cancelling the exam or work. Note that this happens due to the slowness and bureaucracy inherent in opening a fraud case to a student—the simple use of images during the exam is considered an invasion of privacy.” The use of such software should always be used only for reference, and the verdict of fraud should be given by a committee after thorough analysis, as the implications of false positives are of grave concern.

4.6. Identification of the Gap

The issues existed before the lockdown, and the issues surrounding coding have been exposed through pilot study testing. The known gaps appeared to be deepening and widening with time.

The pandemic has brought to the surface existing major deficiencies such as the lack of access to technology (especially in rural areas), but some gaps, such as lack of interactive materials, are still emerging. Some of these gaps will be diminished after returning to a face-to-face setting, but some need to be addressed.

Both before and during the outbreak, the gap was identified through different methods, such as interaction with the students during the classes, interviews with academics, staff surveys, benchmarking process of the institution, literature review, focus group discussion with students, student questionnaire, and class observations.

4.7. Identification and Measurement of Changes in Student Behavior

Changes in student behavior can be identified and measured using observation during classes, through the evaluation of their competencies and knowledge and through surveys and focus groups.

Previously teachers used a mix of both offline and online tools. All respondents used the whiteboard as the main offline tool, assisted by other physical materials, such as lecture notes and exercise, or electronic materials, such as PowerPoint slides, and tools, such as Excel, MATLAB, Geogebra, or Desmos. Online materials were also used, such as videos, weblinks, or Moodle.

During the pandemic, teachers used a much wider variety of tools both for conferencing (MS. Teams, Zoom, Google Meet) and resources in various forms (scanned notes, PDF files, PowerPoint slides, YouTube videos) shared through either third-party sharing services, such as Google Drive, or the proprietary LMS of their institution. Many have used graphical tablets and mobile devices together with virtual blackboards for explaining concepts and drawing figures and have surveyed and quizzed students in various forms (Google Forms, Moodle Quizzes, Kahoot)

Table 3. Summary of the main ideas expressed for each section.

Item	2019	2021
Gaps in current practice	automated process considers only the result expectations of digital literacy lack of access to the online system reticence of teachers to use e-assessment	lack of physical contact lack of fair and objective assessment lack of access to technology inadequate materials
Knowledge of the gaps by target audience	not known to students	known by most stakeholders
Seriousness of the gap	minimal for quantity and quality of online assessment moderate for expectations of digital literacy moderate for automated systems serious for equality of access	serious lack of physical contact divided on assessment serious lack of access to technology serious lack of materials
Motivation to minimize the gap	relatable subject appropriate cognitive challenge institutional support	returning to face-to-face setting well-functioning platforms and materials ensuring access to the materials
Barriers to addressing the gap	lack of free tools attitude of the institution inflexibility of academics access to technology	lack of physical contact lack of investment in technology lack of teacher training social acceptance of cheating
The gap is existing or emerging?	existing and widening	some are being addressed (such as access to technology) the lack of materials is still emerging
Identification and measurement of changes in student behavior	observation during classes evaluation of knowledge and competences surveys and focus groups

shows a summary of the main ideas outlined by the academic for each topic that was covered.

5. Discussion

Assessment is an important part of the educational process. It not only allows the teacher to evaluate the level of knowledge of the student, but it helps the learner reflect on the acquired knowledge, fosters critical thinking, and forces them to actively use the notions they have learned. Although the online setting poses some particular challenges to teachers evaluating students while ensuring there are no breaches of ethics, it has the potential to engage students and teachers alike in a meaningful educational experience [13]. Online formative assessments offer learners the opportunity for enhanced interactivity and formative feedback [13].

Before the pandemic, the main concern was the inability of automated assessment systems to gauge the real learning of a student, together with the assumption of syntactic literacy for access to online assessment systems and the inequality of access to the system.

The pandemic deepened the gaps identified before its start. No automated assessment system was found to compete with the face-to-face setting.

If access to these systems was optional before, it became mandatory during lockdown periods. The improper infrastructure and access to resources widened the gap. Students were forced to learn how to work in an online environment. Academics had to overcome their reluctance to accept e-assessment and were forced to adapt to this all-digital environment. Some realized the importance of having digital literacy and finding ways of adapting their teaching to the virtual environment, while others consolidated their belief that physical presence is essential in the teaching and assessment of students. Students from the second level of education were also forced to embrace online technologies and become more accustomed to using them in their next level of education. Nevertheless, this increased the disadvantage of students that lack access to technology.

The pandemic raised awareness about the gaps in online teaching and assessment. Jisc, a not-for-profit organization offering digital solutions for UK education and research, found in their spring 2020 report that there is a “fundamental and growing disconnect between the way we assess education, the value that assessment currently provides to students and the value they need and expect from it” [22]. In their opinion, assessment should be authentic, accessible, appropriately automated, continuous, and secure, and online tools can help but not without improving the data and systems infrastructure and enhancing staff skills [22]. As this change affected every stakeholder, it made students aware of gaps in current practices and academics of the need to improve both teaching materials and assessment methods. Course design and course assessment, among others, have a significant influence on the adoption of e-learning systems [23], which became even more obvious during the lockdown period. A study in Moroccan universities showed students were not satisfied with the distance learning provided by the teachers as a result of lack of technical skills on both sides and the need for teachers to overhaul their teaching materials and adapt them to online learning [24].

The switch to fully online teaching has divided academics’ opinions on the seriousness of the gaps. Some consider the gap very serious and needs to be addressed, while others consider it not that serious, with the willingness to learn being the formula for student success. Face-to-face assessment is still seen as the only viable solution that can minimize some effects of the identified gaps.

The lack of access to technology, specialized staff, and training continued to be barriers to addressing the gap, as before the pandemic. Access to technology has been cited as the main barrier in other studies as well [24,25]. Nevertheless, the reluctance of academics was shattered as they were forced into this new environment. Some have overcome it and even saw the benefits of digital technologies, while others tried to maximize the value of existing materials while minimizing the effort needed to change them. Students and academics became more aware of their abilities, knowledge, and contribution in online environments. The incentive to cheat on assessment was increased by the lack of preparedness created by the sudden onset of the pandemic.

Some gaps existed before the pandemic and are still widening. Although awareness about the lack of proper infrastructure and equipment has been raised, authorities are slow in finding and implementing solutions. The considerable effort needed to be put in by academics to adapt existing materials is a deterrent in designing and creating engaging and interactive materials and assessments.

6. Conclusions

The current study looks at how the perception of the gaps in assessing students learning mathematics at a university level changed for academics in some European institutions and shows that many of the gaps that were identified by academics during the pandemic were present before as well. With the onset of the pandemic, these gaps in practice have been recognized by other educational institutions all over the world. The pandemic deepened some of them, such as the need for proper technology infrastructure, better course, and assessment design suited for online/virtual use, and better assessment tools that do not rely on the classical pen and paper test. The lack of an automated system’s ability to correctly gauge the knowledge level of a student or the access to technology specialized staff and training still pose barriers to having fully automated, interactive, and adaptive systems.

Nevertheless, the pandemic made academics realize the importance of online learning and teaching and mastering the tools necessary for creating engaging and interactive materials that the student can use synchronously or asynchronously. Digital literacy of students and academics alike has been identified as an important factor. If before it was assumed or even expected from students, the pandemic has shown that this level of literacy needs to be increased.

The results of the analysis also suggest that there is a need for a paradigm shift in thinking about assessment, viewing technology as a facilitator, and not trying to replicate face-to-face conditions. Others have suggested solutions such as assignment/project-based evaluation, open-book exams, or online presentations [26,27].

The study also has a few limitations. Being an exploratory study, it gives a general sense of the gaps in assessment perceived by academics teaching mathematics at a university level. Although it involved seven higher education institutions from six European countries, the results cannot be deemed statistically significant because of the small sample of respondents. Nevertheless, it can serve as a starting point for larger, more complex research that involves more teaching staff. As this is a qualitative study, the collected data is mainly unstructured data in the form of text. This type of data poses particular challenges when being analyzed because of its subjective nature. There is always a risk of errors into communicating ideas and their interpretation.

Future research should look deeper in the perceptions of both teachers and students about online assessment. As some major gaps have been identified, research can be done on finding solutions and methods for bridging these gaps both during the pandemic and afterwards.