Using Integrated Multimodal Technology: A Way to Personalise Learning in Health Science and Biomedical Engineering Students

Abstract

Monitoring the learning process during task solving through different channels will facilitate a better understanding of the learning process. This understanding, in turn, will provide teachers with information that will help them to offer individualised education. In the present study, monitoring was carried out during the execution of a task applied in a self-regulated virtual environment. The data were also analysed using data fusion techniques. The objectives were as follows: (1) to examine whether there were significant differences between students in cognitive load (biomarkers: fixations, saccades, pupil diameter, galvanic skin response—GSR), learning outcomes and perceived student satisfaction with respect to the type of degree (health sciences vs. engineering; and (2) to determine whether there were significant differences in cognitive load metrics, learning outcomes and perceived student satisfaction with respect to task presentation (visual and auditory vs. visual). We worked with a sample of 31 university students (21 health sciences and 10 biomedical engineering). No significant differences were found in the biomarkers (fixations, saccades, pupil diameter and GSR) or in the learning outcomes with respect to the type of degree. Differences were only detected in perceived anxiety regarding the use of virtual laboratories, being higher in biomedical engineering students. Significant differences were detected in the biomarkers of the duration of use of the virtual laboratory and in some learning outcomes related to the execution and presentation of projects with respect to the variable form of the visualisation of the laboratory (visual and auditory vs. visual). Also, in general, the use of tasks presented in self-regulated virtual spaces increased learning outcomes and perceived student satisfaction. Further studies will delve into the detection of different forms of information processing depending on the form of presentation of learning tasks.

1. Introduction

Over the last ten years, the analysis of learning processes has taken a very important turn with respect to recording and analysing information [1]. In terms of recording, using integrated multimodal technology during the learning process helps monitor and visualise the learning process and allows for a variety of indicators to be recorded. Eye tracking technology provides several specific parameters (static—fixation, saccades and pupil diameter—and dynamic—scan path or gaze point) [1]. In addition, eye tracking technology has been accompanied in recent years by other measurement indicators such as psychogalvanic skin response (GSR) and electroencephalographic (EEG) recordings [2] during task execution. Using these resources is referred to as integrated multimodal technology [3]. Coupling this functionality with other subject-specific variables such as age, level of prior knowledge, type of degree, type of instruction, etc., will help researchers specify hypotheses related to the human learning process [4]. However, the volume of data this technology can produce is enormous and will require the application of various statistical and/or machine learning algorithms to process and treat the data [3]. Along these lines, current studies address the difficulty in centralising data. The reasons focus on data privacy, transmission costs and the analysis of large volumes of data. Also, to achieve transfer failure diagnosis with respect to data decentralisation, federated learning comes to reform transfer failure diagnosis methods, where intermediate distribution could serve as a means to indirectly evaluate the discrepancy between domains instead of the centralisation of raw data. Recent studies [5] propose a federated, semi-supervised transfer fault diagnosis method called transfer learning directed through the distribution barycentre medium (TTL-DBM). The results show that TTL-DBM could obtain similar features across domains by adapting through the distribution medium and achieve higher diagnostic accuracy than other federated adaptation methods in the presence of data decentralisation. This aspect is outside the scope of this study but is important to review for researchers focusing on the design of the software needed to tackle data fusion work.

Below, we examine each of these important milestones in the study of learning processes in 21st-century society in depth.

1.1. Simulation-Based Virtual Learning Environments

During and after the COVID-19 pandemic, teaching and learning underwent drastic change all over the world, especially in the higher-education environment. This is because there was a shift from in-person teaching to teaching in hybrid or virtual settings [3]. More specifically, Teaching Based on Training Simulation (TBTS) methodology [6,7,8,9] is being used in teaching environments in health science and engineering degrees. The instructional structure of TBTS is based on a carefully designed learning scenario where conceptual and procedural concepts are presented in a staged way through characters (avatars) that provide a metacognitive, interactive, self-regulated dialogue [10]. Furthermore, this type of instruction improves learning outcomes for students. Specifically, the effect value for this variable was found to be d between 0.45 and 0.72 [11] (d is Cohen’s d, indicating the value of the effect of a variable on other variables. The values are interpreted as follows: values of 0.20 or lower indicate no effect. Values between 0.21 and 0.49 indicate a small effect. Values between 0.50 and 0.70 indicate a medium effect value, and values greater than 0.70 indicate a high effect value). The methodology was also shown to be particularly effective for learning in health science [12] and engineering degrees [13]. These environments also increase motivation during learning [14] and student academic engagement [15].

1.2. Using Cognitive Biomarkers in the Learning Process

Technological resources that help monitor the learning process are a major advancement in understanding the learning process [16]. In this context, eye tracking technology is proving to be an important resource [17]. In addition, in recent years, eye tracking recording has been complemented with GSR and EEG sensors [2]. Specifically, pupillometry analysis has become a relevant indicator [18,19]. For example, a lower pupil magnification is related to a lower cognitive load for the learner. Similarly, higher amplitude and saccade velocity, a higher number of fixations and a shorter average fixation duration are related to information seeking intention [20]. On the other hand, gaze position (x, y or x, y, z axes) in a given task is an indicator of success or failure in solving the task [18].

Table 1. Indicators of multimodal recording in information processing.

Type of Measuring Instrument	Sensor	Meaning	Neurological Implications	Cognitive and Metacognitive Implications
Electrodermal activity	Electrodes on different parts of the body (e.g., fingers). The psychogalvanic skin response (GSR) Non-invasive.	Electrical variation in skin conductance. Changes in the level of skin sweating caused by emotional responses.	Changes in the parasympathetic nervous system.	Cognitive load and level of activation.
Eye tracking	Infrared rays record eye movement. It is non-invasive. However, it is not recommended for people suffering from epilepsy.	Various metrics can be recorded.	They are input indicators of visual, auditory and/or written information. Duration, speed, etc., in each parameter are indirect indicators of the processing of that information.
		Fixations (frequency, duration, speed, etc.).	Processing a stimulus. Fixations indicate the frequency, duration, speed, etc., of eye positioning at the x and y coordinates of the stimulus.	Selective attention. Cognitive load. They may be indicators of interest or of effort in processing.
		Saccades (frequency, duration, speed, etc.).	Processing a stimulus. Saccades indicate the passage of fixation from one stimulus to another.	A greater amplitude of the saccade may indicate less cognitive effort or problems understanding the information.
		Pupil diameter.	The dilation or constriction of the pupil of the eye.	A larger pupil diameter is associated with greater cognitive load and may denote greater problems processing stimulus information.
		Dynamic scan path or gaze point measurements (recording the spatial coordinates of fixation and saccade and ordinal positioning of these records).	Follow-up route in a learning process.	Information processing patterns are different for each learner, although similarities or differences can be found that provide information about learning styles.
	In eye tracking technology, a camera can be activated to record facial expressions while solving a task or problem. It is not invasive. However, users must be aware that their face is being recorded and must give consent.	Facial recognition software.	Emotional association with information processing.	The expression of emotions.
Electroencephalographic recording	Electrodes that collect brain activity can be dry (headband) or semi-dry (electrodes that need to be moistened).The former is less invasive.	This is a recording of the electrical activity of the brain in its different areas.	Brain activity in different areas (frontal, prefrontal, temporal, parietal). Activity can be excitatory or inhibitory.	Information processing from the analysis of brain activation in different areas. The frontal and prefrontal areas are especially related to planning and problem solving.

summarises the most representative psychophysiological metrics that can be extracted with sensors, their significance and their relationship to the development of information processing during task or problem solving.

Regarding the workload according to the type of task, for example, using simulation videos with self-regulated design seems to increase information in working memory. Related cognitive biomarkers indicators are increased pupil diameter [21], increased electrodermal activity [22], changes in facial expressions [23] and variations in positive and negative valences recorded via EEG recordings [24]. However, interpreting these cognitive biomarkers is complicated, depending on the type of learning and the phase of learning [22]. Problem solving depends on several variables, such as the learner’s metacognitive strategies, the degree of prior knowledge, how the problem is presented (structured vs. unstructured) and whether presentation includes self-regulated instruction [25]. However, the use of this methodology in the study of learning processes is just beginning. Therefore, further studies are needed in order to obtain data that can be used to develop personalised learning [17].

1.3. How to Analyse Cognitive Biomarkers in the Learning Process

Multichannel logs in learning environments that include simulation resources allow for a granular analysis of the learning process. Along these lines, studying multichannel logs is expected to enhance the development of intelligent learning environments [3,17,26]. Specifically, the Multimodal Educational Data (MED) design provides a large volume of data which must be analysed using data fusion techniques [27]. These techniques include various types of algorithms: aggregation, ensemble, statistical, similarity, prediction, probability, etc. The most fundamental data fusion technique consists of combining data in the most basic sense of aggregation or concatenation [3]. Multi-fusion data analysis involves the collection of information from different sources, such as [28] classification, and will enable the creation of explanatory models of human behaviour in the learning process. Furthermore, all of these data can be analysed in combination using statistical techniques, supervised learning techniques of prediction and classification [24,29] and data fusion techniques [3]. The techniques that can be applied are diverse (statistics, machine learning, etc.). The general idea is that different data are collected for the analysis of a specific problem or situation through different procedures [30]. These can guide teachers in designing personalised learning spaces that are expected to improve students’ academic performance [31]. However, monitoring all records in natural environments is complex. The use of multimodal models in the study and analysis of the learning process has great potential compared to traditional analysis models [32,33]. The final objective will be to improve the learning proposals [34,35]. This study will follow the analysis of users’ information processing during task solving from the cognitive load theory which is the one applied in the baseline studies used in this study. This is a theory of instruction based on the assumption that information is processed in working memory for a limited time before being stored for further processing in long-term memory. Once stored, information can be transferred back to working memory to govern action relevant to the existing environment. Working memory has no known limits of capacity or duration when it comes to information transferred from long-term memory. The main consequence for instruction is the accumulation of information in long-term memory. Each person processes information according to his or her prior knowledge of the object of learning and the cognitive and metacognitive strategies he or she applies. Cognitive load theory is a theory of instruction based on the knowledge of developmental psychology leading to human cognitive architecture. This architecture specifies individual differences due to biological or environmental factors. Information stored in long-term memory is the main source of environmentally mediated individual differences [36].

Based on previous studies, the following research questions were posed in this paper:

RQ1: Will there be significant differences in the cognitive biomarkers (fixations, saccades, GSR) depending on what degree the students are pursuing (health sciences vs. biomedical engineering)?

RQ2: Will there be significant differences in the learning outcomes depending on what degree the students are pursuing (health sciences vs. biomedical engineering)?

RQ3: Will there be significant differences in the perceived satisfaction depending on what degree the students are pursuing (health sciences vs. biomedical engineering)?

RQ4: Will there be significant differences in the in the cognitive biomarkers (fixations, saccades, GSR) depending on whether the self-regulated virtual lab uses visual and auditory self-regulation vs. only visual self-regulation?

RQ5: Will there be significant differences in the in the learning outcomes depending on whether the self-regulated virtual lab uses visual and auditory self-regulation vs. only visual self-regulation?

RQ6: Will there be significant differences in the perceived satisfaction depending on whether the self-regulated virtual lab uses visual and auditory self-regulation vs. only visual self-regulation?

2. Materials and Methods

2.1. Participants

We worked with a sample of 31 final-year undergraduates studying for health science degrees, 21 specifically pursuing an Occupational Therapy degree (20 women and 1 man) and 10 pursuing a biomedical engineering degree (8 women and 2 men). In all cases, the age range was between 21 and 22 years. Convenience sampling was applied based on students’ availability to participate in this study. The students did not receive any compensation for their participation other than their own learning in the subject. All students were informed of the objectives of this study and signed the consent form. The higher percentage of women than men is a reality in health-related degrees, as reflected in the CRUE report [37].

2.2. Instruments

(a)

Tobii pro lab version 1.194 with 64 Hz.

Table 2. Indicators in integrated multimodal eye tracking and their significance in information processing.

Metrics	Meaning	Measurement Indicator	Cognitive and Metacognitive Significance
Average duration	The duration of the virtual lab visualisation	milliseconds	The average viewing time of the virtual lab is related to the participant’s reaction times in processing the information. A longer or shorter duration will depend on the subject’s prior knowledge and way of processing information
Total Time Interest duration	Define time intervals based on recording and logged Events	milliseconds	Defines interest in the different elements of the stimulus
Fixation point X	Pixels (Display Area Coordinate System—DACS)	fixations indicate attention to stimuli that are relevant to the learner, in this case on the X-axis	These are related to each participant’s way of processing information. In this study, activity was regulated by avatars; therefore, attention to irrelevant stimuli was minimised
Fixation point Y	Pixels (DACS)	fixations indicate attention to stimuli that are relevant to the learner in this case on the Y-axis
Average pupil diameter	The average diameter of the pupil of the fixation	millimetres	This relates to the participant’s interest. In this study, it refers to the content. A larger diameter may indicate greater interest in the information or that the information is very new
Saccade direction	The angle of the straight line between the preceding fixation and succeeding fixation. This can only apply to whole saccades	degrees	This refers to the direction in the lab space in which the saccades are directed
Saccade average velocity	The average velocity across all samples belonging to the saccade, even outside the interval	degrees/second	This involves the transition from one stimulus to another. In this study, it is within the lab. It refers to the speed of information processing, and this depends on prior knowledge and an understanding of the information
Saccade peak velocity	The maximum velocity across all samples belonging to the saccade, even outside the interval	degrees/second	This refers to the peak maximum velocity of the saccade in the stimulus display
Saccade amplitude	The amplitude for whole saccades	degrees	The amplitude of the saccade refers to the participants’ cognitive effort
GSR amplitude	The average galvanic skin response (GSR) signal, after filtering, for each time of interest, with averages, medians and counts for each participant	microsiemens	The amplitude of the GSR is related to the learner’s interest. Higher scores indicate a higher cognitive load and a higher level of activation
SCR count	The number of skin conductance responses, for each interval in the time of interest with averages, medians, counts, variances and standard deviations (n − 1)	count	This refers to conductance responses to the stimulus display. Higher scores indicate a higher cognitive load and activation level

presents the measures used in this study, what they mean and their relationship to cognitive and metacognitive processes.

(b)

Shimmer3 GSR+ (galvanic skin response) single-channel galvanic skin response data acquisition (Electrodermal Resistance Measurement—EDR/electrodermal activity (EDA)). The GSR+ unit is suitable for measuring the electrical characteristics or conductance of skin. This device is compatible with Tobii pro lab version 1.194 and allows for the integration of records.

(c)

Virtual lab for the resolution of clinical cases. This lab was developed ad hoc within the European project “Specialized and updated training on supporting advance technologies for early childhood education and care professionals and graduates” No. 2021-1-ES01-KA220-SCH-000032661. In this study, the virtual laboratory consists of the visualisation of a dialogue between the two avatars, a student and a teacher. The teacher asks questions that guide the student’s reflection in the resolution of a task of elaborating a therapeutic intervention programme in a case of prematurity. The student can follow the interaction at the pace he/she needs, as he/she is the one who moves on to the next image. Therefore, it is a laboratory and not a video. The lab was controlled by the voice of an avatar. However, this functionality could be removed by the student by clicking on the sound icon (see image A in Figure 1). The lab included two avatars simulating the role of a teacher–therapist and a student (see image B in Figure 1). The student watching the lab could choose self-regulation via two routes: auditory via the voice of the avatars and visual (through reading the dialogues) or visual only (through reading the dialogues). The teacher–therapist avatar first presents the case and gives information about the clinical history (see scene C and scene D in Figure 1) before regulating the process of solving the clinical case (see scene E in Figure 1). The student avatar also asks the teacher questions (see scene F in Figure 1). The teacher’s answers follow a metacognitive dialogue that uses guiding, planning and evaluation questions (see scene G in Figure 1). Finally, the teacher avatar gives the student watching the video a comprehension task about resolving the clinical history they have been watching. The student has to arrange the answer options (see scene H in Figure 1). The answers given by the learner may be wrong, in which case a message appears telling the learner to try again (see scene I in Figure 1). The student then tries again, and if the solution is correct, they obtain feedback on the correct answer (see scene J in Figure 1). In Appendix A, each of the figures is shown separately to enlarge the display (A = Figure A1; B = Figure A2; C = Figure A3; D = Figure A4; E = Figure A5; F = Figure A6; G = Figure A7; H = Figure A8; I = Figure A9; J = Figure A10). Also, this laboratory is freely accessible after logging into the project’s Virtual Classroom https://www2.ubu.es/eearlycare_t/en/project (accessed on 7 August 2024).

(d)

Learning outcomes. A project-based learning methodology was used with both groups of students. The students were voluntarily split into small groups of 3 to 5 students, and throughout the semester, they had to solve a practical case (project) related to the therapeutic intervention in a clinical case. During the final week of the semester, the students had to present their proposed solution to the class group. Students were also required to take a test to check the conceptual content of the subject, which they had to complete individually. The test consisted of 30 multiple-choice questions with four answer options, only one of which was correct. The marks for the three tests were given on a scale of 0 to 10, with 0 being the minimum and 10 the maximum. Both groups were taught by the same teacher to avoid the effects of the teacher type variable.

(e)

Questionnaire for students’ perceived satisfaction during the learning process. This questionnaire was developed ad hoc and contained three closed questions measured on a Likert-type scale from 1 to 5: Question 1 = “Rate how satisfied you are with how the subject was delivered on a scale of 1 to 5 where 1 is not at all satisfied and 5 is totally satisfied”; Question 2 = “Rate how satisfied you are with the virtual lab on a scale of 1 to 5 where 1 is not at all satisfied and 5 is totally satisfied”; and Question 3 = “Rate whether you felt anxious using the virtual lab on a scale of 1 to 5 where 1 is never and 5 is always”. Reliability indicators were not provided for this instrument; as with only three questions, any reliability indicator would not be significant (see

Table A1. A questionnaire for students’ perceived satisfaction during the learning process.

Questions	Rating Scale
1. Rate how satisfied you are with the subject.	1	2	3	4	5
2. Rate how satisfied you are with the virtual lab on a scale.	1	2	3	4	5
3. Rate whether you felt anxious using the virtual lab.	1	2	3	4	5

).

2.3. Procedure

Before this study was carried out, approval was obtained from the Bioethics Commission of the University of Burgos (Spain), No. IO 03/2022. Subsequently, at the beginning of the semester, the students in the two groups [Group 1: health science students pursuing an Occupational Therapy degree and Group 2: biomedical engineering students] were asked to take part in this study. This was voluntary and had no compensation beyond learning through the virtual lab. All participants signed a written informed consent form. The two groups worked in similar ways. The content of the task applied in the virtual lab was novel for both groups. At the end of the fourth week of the semester, the students engaged with the virtual lab in a room with controlled light and temperature conditions. Engagement with the virtual lab was conducted individually under the supervision of an expert in the use of eye tracking technology. She gave students instructions about their posture at the table and in the chair and placed the GSR electrodes. Students’ positions varied according to their height and weight characteristics and ranged from 45 to 50 centimetres from the chair to the table. The virtual lab took varying time to complete depending on the individual student, as they were able to move through the slides depending on their understanding of the content. In addition, students could choose whether to watch the virtual lab with regulating audio from the avatars or only just by looking at the images, which contained in which the self-regulated text. At the end of the semester, the learning results were recorded in 3 tests (an individual knowledge test, a group test on the execution of a project resolving a practical case and a group test on presenting the project). Information was also collected on perceived satisfaction using the virtual lab. Figure 2 shows the process of collecting and analysing information about the learner.

In this study, we analysed the data in two registers: indicators of internal psychological response collected by eye tracking and integrated GSR were recorded and subjective indicators where students’ indicators of perceived satisfaction with using the virtual lab. Finally, the learning outcomes were collected. The measurement parameters are given in

Table 3. Multimodal Educational Data categories.

MED Spaces	Instrument	Variables
Objective psychological response indicators	Eye tracking GSR	Average duration Total Time Interest duration Total recording duration Fixation point X Fixation point Y Average pupil diameter Saccade direction Saccade average velocity Saccade peak velocity Saccade amplitude GSR amplitude SCR count
Subjective psychological response indicators	Questionnaire on perceived satisfaction with the use of the virtual lab	Satisfaction with the learning process Perceived satisfaction with the virtual lab Perceived anxiety with the use of the virtual lab
Learning outcomes	Assessment tests	Multiple-choice test Project execution test Project presentation test

.

2.4. Design

Given the choice characteristics and sample size, non-parametric statistics and a quasi-experimental design without a control group [38] were applied where the independent variables were the ‘type of degree’ (RQ1, RQ2, RQ3) and ‘type of virtual lab display’ (RQ4, RQ5, RQ6). Likewise, the dependent variables were the cognitive biomarkers (fixations, saccades and GSR) (RQ1, RQ4), ‘learning outcomes’ (RQ2, RQ5) and ‘perceived satisfaction’ (RQ3, RQ6).

2.5. Data Analysis

Given the characteristics of the sample (convenience sampling and sample size of n = 31) and based on the choice and number of the sample, non-parametric statistics were applied to test the research questions. Specifically, the Mann–Whitney U test for two independent samples and effect size test r [39] were used. Analysis was performed with the SPSS v.28 statistical package [40]. Also, we used the machine learning techniques of Linear Projection and Principal Component analyses for studying the student’s position. For this, we used Orange Data Mining v.3.35.0 [41]. Likewise, the student’s pathway and heat map were analysed with Tobii pro lab version 1.194.

3. Results

3.1. Influence of Degree Type on Cognitive Biomarker Recording, Learning Outcomes and Perceived Task Satisfaction

Looking at RQ3, we only found significant differences in the students’ perceived anxiety (Anxiety lab use) during the use of the virtual lab. The mean was higher in Group 2 (biomedical engineering students) (M = 2.18 out of 5) than in Group 1 (Occupational Therapy students) (M = 2.02 out of 5). However, in both cases, the perceived anxiety about using the virtual lab was low, with values below 2.19 out of 5 and a medium effect, r = −0.46. Likewise, no significant differences by the type of degree were found in the eye tracking and GSR parameters (RQ1). Similarly, no significant differences in learning outcomes were found either (RQ2) (see

Table 4. Mann–Whitney U test for differences in degree type.

Metric	Mean Rank		Mann–Whitney U	Z	p	r
	Group 1 n = 20	Group 2 n = 10
Average duration	15.25	16.00	95.00	−0.22	0.83	−0.04
Total Time Interest duration	15.90	16.20	103.00	−0.09	0.93	−0.02
Total recording duration	15.29	17.50	90.00	−0.63	0.53	−0.11
Fixation point X	15.62	16.80	97.00	−0.34	0.74	−0.06
Fixation point Y	15.87	16.70	98.00	−0.30	0.77	−0.05
Average pupil diameter	15.24	17.60	89.00	-0.68	0.50	−0.12
Saccade direction	14.90	18.30	82.00	−0.97	0.33	−0.18
Saccade average velocity	16.48	15.00	95.00	−0.42	0.67	−0.08
Saccade peak velocity	16.43	15.10	96.00	−0.38	0.70	−0.07
Saccade amplitude	15.67	16.70	98.00	−0.30	0.77	−0.05
GSR amplitude	16.14	15.70	102.00	−0.14	0.88	−0.03
SCR count	15.36	17.35	91.50	−0.64	0.52	−0.12
Score of multiple-choice test	13.79	20.65	58.50	−1.98	0.05	−0.36
Project execution score	13.90	20.40	61.00	−1.92	0.05	−0.35
Project presentation score	16.26	15.45	99.50	−0.25	0.81	−0.05
Satisfaction with learning	14.93	18.25	82.50	−1.05	0.30	−0.19
Satisfaction with virtual lab	14.02	20.15	63.50	−1.87	0.06	−0.34
Anxiety lab use	13.19	21.90	46.00	−2.52	0.01 *	−0.46

* p < 0.05. Note: r = ${\displaystyle \raisebox{1ex}{$Z$}\!\left/ \!\raisebox{-1ex}{$\sqrt[2]{N}$}\right.}$ where N is the total number of participants [36]. The interpretation of effect sizes is as follows: r > 0.3 small effect, r = 0.3–0.5 medium effect and r < 0.5 large effect. Group 1 = undergraduate students in Occupational Therapy; Group 2 = undergraduate students in health engineering. One experimental death occurred in Group 1.

).

3.2. Influence of Task Presentation on Cognitive Biomarker Recording, Learning Outcomes and Perceived Task Satisfaction

Looking at RQ4, we found significant differences between Group 1 (visual and auditory lab use) vs. Group 2 (visual only) in the time spent watching the virtual lab, in “Average duration” [M = 157.98 ms vs. M = 50 ms], respectively, where the effect size was medium (r = −0.40), and in “Total Time Interest Duration” [M = 307.43 ms vs. M = 268.21 ms], respectively, where the effect size was medium (r = −0.40). With regard to RQ5, we found significant differences in project presentation scores [M = 9.62 out of 10 vs. M = 9.20] out of 10, respectively, where the effect size was medium (r = −0.40). Finally, with regard to RQ6, significant differences were found in perceived satisfaction with learning [M = 5.00 out of 5 vs. M = 4.21] out of 5, respectively, where the effect size was medium (r = −0.40) (see

Table 5. Mann–Whitney U-test for type of virtual lab display.

Metric	Mean Rank		Mann–Whitney U	Z	p	r
	Group 1 n = 16	Group 2 n = 14
Average duration	18.81	11.71	59.00	−2.20	0.03 *	−0.40
Total Time Interest duration	19.24	12.07	64.00	−2.18	0.03 *	−0.40
Total recording duration	18.35	13.14	79.00	−1.59	0.11	−0.29
Fixation point X	17.82	13.79	88.00	−1.23	0.22	−0.22
Fixation point Y	17.29	14.43	97.00	−0.87	0.38	−0.16
Average pupil diameter	16.94	14.86	103.00	−0.64	0.53	−0.12
Saccade direction	14.88	17.36	100.00	−0.75	0.45	−0.14
Saccade average velocity	18.53	12.93	76.00	−1.71	0.09	−0.31
Saccade peak velocity	18.29	13.21	80.00	−1.55	0.12	−0.28
Saccade amplitude	14.71	17.57	97.00	−0.87	0.38	−0.16
GSR amplitude	15.56	16.54	111.50	−0.33	0.74	−0.06
SCR count	15.82	16.21	116.00	−0.13	0.89	−0.02
Multiple-choice test score	17.12	14.54	100.00	−0.76	0.45	−0.14
Project execution score	18.47	13.00	77.00	−1.72	0.09	−0.31
Project presentation score	18.85	12.54	70.50	−2.04	0.04 *	−0.37
Satisfaction with learning	17.09	14.68	100.50	−2.04	0.04 *	−0.37
Satisfaction with virtual lab	17.56	14.11	92.50	−1.12	0.26	−0.20
Anxiety lab use	15.68	16.39	113.50	−0.22	0.83	−0.04

* p < 0.05. Note: r = ${\displaystyle \raisebox{1ex}{$Z$}\!\left/ \!\raisebox{-1ex}{$\sqrt[2]{N}$}\right.}$ where N is the total number of participants [39]. The interpretation of effect sizes is as follows: r > 0.3 small effect, r = 0.3–0.5 medium effect and r < 0.5 large effect. Group 1 = undergraduate students in Occupational Therapy; Group 2 = undergraduate students in health engineering. One experimental death occurred in Group 1.

).

Next, a Linear Projection analysis was carried out for the positioning of each learner according to the variables in which significant differences were found between the group that had visual and auditory self-regulated virtual lab viewing (Group 1) vs. visual self-regulated virtual lab viewing (Group 2); see Figure 3 (the blue colour is Group 1, and the red colour is Group 2).

Also, as an example, Figure 4 shows a gaze point of a trainee from Group 1 (visual and auditory self-regulated virtual lab viewing) and Group 2 (visual self-regulated virtual lab viewing). As can be seen in Figure 4, there is a greater concentration of visualisation in the centre of the image (where the dialogues between the avatars take place) in Group 1. In Group 2, there is a greater dispersion of the gaze towards the edges. Figure 5 shows the data in heat map mode.

In Group 1 (health science students), 10 students chose to visualise the virtual laboratory with voice (50%) and 10 without voice (50%). In Group 2 (biomedical engineering students), six students visualised the virtual laboratory (60%) with voice and four without voice (40%).

4. Discussion

The interpretation of cognitive biomarkers is complicated by learner characteristics [25] and task characteristics [22]. However, the results of this study cannot be generalised given the characteristics of the sample. From a descriptive point of view, it can be noted that the use of simulation-based virtual laboratories seems to facilitate understanding in health science and engineering students [6,7,8,9,12,13]. Likewise, the perceived satisfaction of students from these degrees did not differ [13] except for perceived anxiety.

In sum, no significant differences were found in student learning outcomes after the use of the self-regulated virtual laboratory. This confirms what has already been found in other studies regarding the effectiveness of the use of this type of laboratory based on self-regulated simulation with respect to learning outcomes in students of health science or engineering degrees [6,7,8,9,12,13]. Regarding the perceived satisfaction with the use of the laboratories, students of both degrees were satisfied with their use. These results underline the findings of other studies on the usefulness of using self-regulated virtual laboratories for student motivation and engagement [14,15]. However, the engineering students perceived a higher degree of anxiety, and future studies will explore this further.

Regarding the workload experienced by the students depending on the modality of the visualisation of the laboratory (visual with audio vs. visualisation without audio), significant differences were found with respect to the biomarkers of Average duration (refers to the average duration of the visualisation and can be related to the form of information processing) and Total Time Interest duration (refers to the interest maintained by the learner in the stimuli). Students who chose to view the lab with audio spent more time viewing the stimuli. These results are related to information processing, possibly in working memory, depending on the way the task is presented [21]. Furthermore, it should be considered that in this study, the form of information display was voluntary. Therefore, each student chose the form that was the most appropriate for him or her. Significant differences were also found in the learning outcomes of the students. Students who chose the audio visualisation of the lab had better learning results in the project execution and project presentation tests. They also reported higher satisfaction with the learning process. These data point to a possible relationship between the type of information processing of the learner, learning outcomes and perceived satisfaction. Such a connection is probably related to the learner’s degree of commitment to learning. Therefore, further studies will address this hypothesis. In sum, all results found in this study should be taken with caution and with limitations in their generalisability. The reasons for this are the type of sampling, convenience sampling and the number of participants. However, as values of this work, we can highlight the proposed analysis of learning processes in self-regulated virtual environments monitored through integrated multichannel eye tracking technology.

5. Conclusions

The use of data from different areas (cognitive biomarkers, satisfaction perceived, learning outcome, etc.) is both a challenge and an opportunity. The challenge lies in collecting a lot of information in real time, and the opportunity lies in the personalised learning proposal. In this line, the more information we have about a learning process, the more data we will collect, which will require the use of more complex data analysis techniques. However, the collection of all this information implies highly individualised processes which, although they provide a great deal of information on each learner, limit the generalisation of the results. However, this way of working could facilitate the advancement of educational instruction towards precision in a similar way to what precision medicine conducts in order to propose the best treatment for different afflictions. Future research will delve deeper into the variables of task presentation and the study of different learning patterns (information processing type) as well as the redesign of tasks according to these patterns.

In summary, it is important to stress the difficulty that this type of study has with regard to the use of large samples, as working with integrated multichannel eye tracking technology requires individual work with each student and a high degree of sophistication of the equipment used. In addition, there is an arduous process of data extraction and data processing. This means that the generalisability of the results is limited. However, the advantages centre on a better understanding of the way of human processing, in this case applied to learning, and the possibility of offering teachers and cognitive and computational researchers procedures to continue their complex research into the workings of human cognitive processing.