Attitudes and Opinions of Biomedical Students: Digital Education Questionnaire

Abstract

(1) Background: the purpose of this study was a preliminary analysis of current methods of online teaching at the Faculty of Medical Sciences, University of Kragujevac, Republic of Serbia, in order to define the attitudes of biomedical students about education during the COVID-19 pandemic and to validate of the education medical questionnaire (eMedQ), a new tool for the assessment of the students’ perceptions about digital education. (2) Methods: this was a qualitative cross-sectional observational study that used the originally developed 45-item questionnaire (eMedQ) as an assessment instrument conducted on biomedical students (n = 209) of all study years at the Faculty of Medical Sciences, University of Kragujevac (Republic of Serbia), during winter semester or between September 2021 and January 2022. (3) Results: In this study, a Cronbach’s alpha value of 0.904 was obtained, which indicated good internal consistency; the correlation matrix revealed many coefficients greater than 0.3, denoting high correlations between the items. The Kaiser–Meyer–Olkin measure of sampling adequacy was 0.866 and Bartlett’s test of sphericity was significant (p < 0.001). The PCA revealed the presence of seven components with characteristic values over one, while three factors explained the highest percentage of variance. (4) Conclusions: this research developed and validated a new tool for evaluation of biomedical student perceptions about digital education.

1. Introduction

Changes in the world are constant and accelerated, which requires our parallel adaptation to novel modes of living, showing how humanity, although vulnerable, may also be very strong. One of the biggest health threats in human history, which has recently affected physical, psychological, and social lives, is the coronavirus disease 2019 (COVID-2019) pandemic, whose outbreak occurred in December 2019 in Wuhan, China [1,2]. On 31 January 2020, the World Health Organization (WHO) declared the global public health threat, while on March 11, the accelerated spreading of COVID-19 disease reached pandemic proportions [1]. The first COVID-19 case in the Republic of Serbia was reported on 6 March 2020, and on 15 March 2020 the Serbian government declared a state of emergency. Confrontation with the COVID-19 pandemic brought the inevitable need for immediate adaptation to the necessary restrictive measures, including the closure of international borders, suspension of education, tourism, hospitality, and other public institutions [2].

Having in mind that education is a human right [3], the negative effects of the newly emerging health risk situation at all levels of the educational system have not gone unnoticed. In fact, it has been reported that the closure of educational institutions due to the COVID-19 pandemic globally affected more than 1.5 billion students [4,5]. However, educational institutions all over the world made great efforts in order to accelerate the modification of the teaching process by switching classical teaching with online teaching [6]. From that point, the quality and efficiency of this virtual mode of education became an important subject of intensive research and continuous improvement. Various studies have pointed out numerous advantages of virtual education, including cost-effectiveness, schedule flexibility, and the absence of physical and temporal limitations, but also on many disadvantages, such as technological difficulties, low quality of teaching, and an inability to provide high-quality education in the field of applicable disciplines [6].

An additional challenge was to implement an online teaching process among biomedical students that would provide them with quality teaching content on one side but halt the face-to-face interaction with both teaching staff and patients during clinical practice. At the beginning of the pandemic, the Association of American Medical Colleges released the guidelines considering novel educational rules, including suspension of clinical rotations for medical students during the COVID-19 pandemic [7]. However, it is well known that acquiring practical knowledge and skills during clinical rotations is of crucial importance for future medical professionals [8]. A first study investigating the perceptions of medical students on this novel mode of learning, conducted in the United Kingdom during May 2020, reported that students were less satisfied with online compared to in-person teaching, with concerns about whether they will be adequately prepared for their future professions [9]. However, properly adjusted, the long-term, hybrid learning, which includes a mixed model of on-site/online teaching, appears to be adequate for teaching in biomedical sciences [10].

In the Republic of Serbia, the suspension of in-person classes forced all stages of educational institutions to adopt different modes of remote teaching. In high educational institutions, various existing platforms have been used for these purposes, such as MS Teams, Zoom, etc. Some institutions even constructed their own online platforms in order to improve the accessibility of educational materials to all students.

The core of our research is based in the fact that it gives comprehensive and valid information that can create favorable conditions for achieving more efficient online teaching methods at the faculties of medical sciences in the Republic of Serbia. In such a manner, we will provide insight into previous circumstances in students’ education in the time of the pandemic induced by the COVID-19 virus. Observed information and identified deficiencies can be used as a working directive for increasing the quality of online education in a specific area of biomedical sciences.

The purpose of this study was the examination of the current approach of learning online at the Faculty of Medical Sciences, University of Kragujevac, Serbia, in order to determine the opinions of biomedical students (medicine, pharmacy, and dentistry) about learning methods in the time of the COVID-19 pandemic, their previous experiences, and validation of the education medical questionnaire (eMedQ), a new tool for assessment of the students’ perceptions about digital education.

2. Materials and Methods

2.1. Ethical Concerns

This study’s protocol was approved by the institutional Ethics Committee, Faculty of Medical Sciences, University of Kragujevac (no. 143/19), and it is in accordance with the principles of the Declaration of Helsinki (revision 2013). The participation in this examination was voluntary and anonymous.

2.2. Research Design and Participants

This was a qualitative, cross-sectional, and observational study that used the originally developed questionnaire as an assessment instrument. The whole data were collected from biomedical students (n = 209) of all study years at Faculty of Medical Sciences, University of Kragujevac (Republic of Serbia), during winter semester or between September 2021 and January 2022. The study sample consisted of representative number of biomedical students from central part of Republic of Serbia who were active students during pandemic.

Criteria for the inclusion of respondents were as follows: students who were at undergraduate or vocational level at all biomedical faculties in Central Serbia, the presence of a study subject’s user account on the specific online platform (Licensed Moodle Platform for Online Education 27655/2), or adequate online approach and voluntary participation. For our study, there were not foreseen any special exclusion criteria.

2.3. Pilot Study for Item Generation and Targeting Population

In constructing the final instrument (45-item), we used the data from our previous pilot questionnaire (17-item) study, which was conducted between December 2020 and January 2021 at Faculty of Medical Sciences [11]. That pilot study was designed as a qualitative study, which included 332 participants (students and academics staff) at the Faculty of Biomedical Sciences, University of Kragujevac, during the pandemic of SARS-CoV-2 infection in Serbia. Pilot version of questionnaire consists of 17 items, and the aim of the first phase was to determine the share of participation in relation to the total population of potential respondents, to determine whether the questions are appropriate and understandable, what is the distribution of answers, and the suitability for their statistical analysis [11].

2.4. Moodle Open-Source Learning Platform

The Faculty of Medical Sciences had a prepared and elaborated strategy for all working conditions, including the pandemic situation in which we found ourselves globally. The Moodle platform allows students to access electronic textbooks, an electronic library, and subject pages of all study programs, as well as all other materials necessary for successful mastering of the material. Moreover, there is all the necessary information related to the faculty and schedules of lectures and exams, but also, thanks to the multifunctionality of the platform, there is the possibility of organizing online tests (colloquia and exams).

2.5. Selecting the Size of Study Sample

According to the results of pilot study [11] and the allocation of subjects regarding gender, but also gender representation in general among student and teaching population of the Faculty of Medical Sciences in January 2021, we made target research sample which presents a similar gender distribution to the official student/academic staff population at the Faculty of Medical Sciences in Kragujevac. The total number of students at Faculty of Medical Sciences is 1572, and for development and validation of eMedQ, 209 undergraduate students were selected as a representative number (7.5%) of all students from Faculty of Medical Sciences, University of Kragujevac.

2.6. Constructed eMedQ Instrument for Evaluation of Attitudes and Perceptions among Biomedical Students

The Serbian version of eMedQ is comprised of 45 closed-ended questions with graduated answers divided into 7 domains as follows: demographic characteristics, experience with online teaching, education process (teaching organization), aspects of mental functioning, clinical skills, technical aspects, and quality of life. Students filled out a questionnaire through an online learning platform, which served as educational tool during COVID-19 pandemic at this institution. Study attendance in our research was on voluntary base and entirely unidentified, and the identification details of the technical approach of the study sample were available only to the server administrator, who handles the data in line with the regular working practice of the institution. The dimensions of questionnaire were constructed according to similar research methods [12,13].

2.7. Validation of Serbian Version of eMedQ (Reliability Testing and Factor Analysis)

Internal consistency as a measure of inter-correlation of the items of the questionnaire was accessed by calculating the Cronbach alpha coefficient. Moreover, the split-half reliability testing was conducted. This method involved dividing the questionnaire into two parts and calculating Cronbach’s alpha for each part. The Spearman–Brown formula was used to determine the correlation of the scores on the scales from the two parts.

In order to discover principal factors, 33 items underwent principal component analysis (PCA). Prior to conducting the analysis, the suitability of the data for analysis was assessed by examining the correlation matrix, observing the values of the Kaiser–Meyer–Olkin indicator and the significance of the Bartlett sphericity test. Firstly, the factors were extracted without rotation based on eigenvalues greater than 1, scree plot, and factor loadings. Then, oblimin rotation was conducted.

2.8. Statistical Analysis

IBM SPSS, version 26.0, was used for statistical processing of the obtained data. The study sample was established according to the design of the survey type, based on the presumption of an error margin of 5% and confidence limits of 95%. The complete calculated obtainable study respondents at the Faculty of Medical Sciences, University of Kragujevac, were 1572 respondents, and the share of participation is estimated at 30%. Using an appropriate online calculator [14], a study pattern of 190 participants was determinate so that it was defined to the smallest sample from 200 people for the research. Descriptive statistics included the presenting of category characteristics as absolute and relative frequencies of individual categories. To test the difference between the obtained and expected frequencies (frequencies), a matching test or a chi-square test was used to examine the quality of the match. A chi-square independence test was used to examine the relationship between categorical variables. The data are presented in tabular and graphical form.

3. Results

3.1. Basic Demographic Characteristics of Participants

In

Table 1. The basic demographic characteristics of study population. Data are presented as mean ± SD, median (min–max), or frequency in percent (%). Chi-square test was used to test the differences between selected variables with the significance level of 0.05.

Characteristic	N (%)	Chi-Square Test/p
Gender
Male	42 (20.4)	72.252/0.000
Female	164 (79.6)
Study program
IASM	39 (18.8)	91.760/0.000
IASF	80 (38.5)
IASS	24 (11.5)
OSS	65 (31.3)
Year of study
I	26 (12.4)	230.321/0.000
II	115 (55.0)
III	5 (2.4)
IV	27 (12.9)
V	18 (8.6)
VI	18 (8.6)
Level of skills in using electronic devices (computers, smartphones, tablets…)
Inadequate	2 (1.0)	91.856/0.000
Acceptable	15 (7.2)
Good	57 (27.4)
Very good	64 (30.8)
Excellent	70 (33.7)
Experience with online education before COVID-19
Very great experience	15 (7.3)	58.709/0.000
Great experience	18 (8.7)
Moderate experience	42 (20.4)
Little experience	63 (30.6)
Without any experience	68 (33.0)

, the basic characteristics of the study population are shown. We have observed that the mean age of the study group was 21.83 ± 4.163, with predominantly female students present. Most of the participants were pharmacy students from the second year of study with very similar skills in the use of electronic devices (Table 1).

3.2. Validation of Serbian Version of eMedQ

By using the internal consistency as a measure of inter-correlation of the items of the questionnaire, the Cronbach alpha coefficient was calculated. Moreover, the split-half reliability testing was conducted. This method involved dividing the questionnaire into two parts and calculating Cronbach’s alpha for each part. The Spearman–Brown formula was used to determine the correlation of the scores on the scales from the two parts.

In this study, we observed the Cronbach’s alpha at 0.904, which indicates good internal consistency. When the questionnaire was divided by the split-half method into two parts, Cronbach’s alphas were 0.852 and 0.906. The value of the Spearman–Brown coefficient for the questionnaire was 0.555.

3.3. Factor Analysis

In order to discover principal factors, 33 items underwent principal component analysis (PCA). Prior to conducting the analysis, the suitability of the data for analysis was assessed by examining the correlation matrix and observing the values of the Kaiser–Meyer–Olkin indicator and the significance of the Bartlett sphericity test. Firstly, the factors were extracted without rotation based on eigenvalues greater than one, scree plot, and factor loadings. Then, oblimin rotation was conducted.

In this research, the correlation matrix revealed many coefficients greater than 0.3, denoting high correlations between the items. The Kaiser–Meyer–Olkin measure of sampling adequacy was 0.866, and Bartlett’s test of sphericity was significant (p = 0.000). The PCA revealed the presence of seven components with characteristic values over one, while three factors explained the highest percentage of variance. The scree plot also suggested a three-factor solution. After oblimin rotation was conducted, the rotated solution confirmed three factors explaining 47.7% of the variance. Two items were excluded due to having low loadings. The distribution of questions into factors after direct oblimin rotation is shown in Table 1. Cronbach’s alpha for the remaining items was 0.913. Cronbach’s alpha for the first, second, and third factor was 0.912, 0.878, and 0.781, respectively.

By using the factor analysis, we observed two negative factors in our instrument, such as questions Q35 and Q40, which are excluded from further analysis and the final form of the instrument (Figure 1).

3.4. Analyzing of the Perceptions Regarding the Online Education among Biomedical Students

Table 2. Perception regarding the online education among biomedical students. Data are presented as mean ± SD, median (min–max), or frequency in percent (%). Chi-square test was used to test the differences between selected variables with the significance level of 0.05.

Variables	N (%)	Chi-Square Test/p
Progress during the online education during the COVID-19 pandemic
Very much	30 (14.8)	72.148/0.000
Much	54 (26.6)
Moderate	81 (39.9)
Little	25 (12.3)
I have not progressed	13 (6.4)
Further improvement of the current form of online teaching
It is necessary	109 (57.4)	4.126/0.042
It is not necessary	81 (42.6)
Area in which it would be useful to have additional teaching materials compared to existing online materials
Preclinical subjects (anatomy, physiology, histology, genetics, chemistry, and other fields...)	41 (21.6)	71.568/0.000
Subjects of clinical medicine, clinical pharmacy, and clinical dentistry	118 (62.1)
Other subjects:	31 (16.3)

presents the answers from biomedical students related to the main perceptions regarding the online education during the COVID-19 outbreak. As we can see, most of the students who used the Zoom tool for online teaching expressed progress during online education (39.9%). Half of students used presentation with narration and half of them did not, and almost all students did not use online consultations with the lecturers (Figure 2). From Figure 2, we can see that there were statistically significant differences between answers regarding the forms of online teaching that students experienced during the COVID-19 pandemic (p < 0.001, chi-square test).

Interesting, about 57.4% of biomedical students think that the use of online education must be improved, while 42.6% of students do not think that. Perceptions about the online education modalities that could improve education and additional material for teaching are similar among all included students (Figure 3).

3.5. Analyzing of the Attitudes Regarding the Online Education among Biomedical Students

The characteristics of the study population related to attitudes toward online education from different aspects are presented in

Table 3. The attitudes regarding the online education among biomedical students. Data are presented as mean ± SD, median (min–max), or frequency in percent (%). Chi-square test was used to test the differences between selected variables with the significance level of 0.05. The full names of questions are presented in Supplementary File S1.

Question	Strongly Disagree N (%)	Somewhat Disagree N (%)	Neither Agree nor Disagree N (%)	Somewhat Agree N (%)	Strongly Agree N (%)	Chi-Square Test/p
EDUCATION PROCESS (TEACHING ORGANIZATION)
Q13	8 (3.8)	16 (7.7)	44 (21.1)	70 (33.5)	71 (34.0)	82.794/0.000
Q14	9 (4.3)	31 (15.0)	49 (23.7)	59 (28.5)	59 (28.5)	44.329/0.000
Q15	6 (2.9)	17 (8.1)	28 (13.4)	62 (29.7)	96 (45.9)	129.971/0.000
Q16	6 (2.9)	19 (9.1)	30 (14.4)	36 (17.3)	117 (56.3)	183.394/0.000
Q17	12 (5.7)	18 (8.6)	47 (22.5)	64 (30.6)	68 (32.5)	63.656/0.000
Q18	8 (3.8)	12 (5.7)	37 (17.7)	55 (26.3)	97 (46.4)	126.191/0.000
Q19	9 (4.3)	19 (9.1)	51 (24.4)	67 (32.1)	63 (30.1)	66.144/0.000
Q20	11 (5.3)	12 (5.8)	36 (17.3)	52 (25.0)	97 (46.6)	120.702/0.000
Q21	28 (13.6)	15 (7.3)	50 (24.3)	42 (20.4)	71 (34.5)	44.340/0.000
Q22	87 (41.6)	23 (11.0)	27 (12.9)	32 (15.3)	40 (19.1)	64.947/0.000
ASPECTS OF MENTAL FUNCTIONING
Q23	28 (13.6)	21 (10.0)	49 (23.4)	39 (18.7)	72 (34.4)	38.153/0.000
Q24	35 (16.8)	21 (10.1)	37 (17.8)	42 (20.2)	73 (35.1)	35.462/0.000
Q25	36 (17.2)	24 (11.5)	53 (25.4)	41 (19.6)	55 (26.3)	15.569/0.004
Q26	54 (26.1)	34 (16.4)	43 (20.8)	30 (14.5)	46 (22.2)	8.870/0.064
Q27	64 (30.6)	24 (11.5)	56 (26.8)	33 (15.8)	32 (15.3)	28.344/0.000
Q28	64 (30.8)	25 (12.0)	53 (25.5)	30 (14.4)	36 (17.3)	25.798/0.000
Q29	82 (39.2)	20 (9.6)	52 (24.9)	18 (8.6)	37 (17.7)	66.622/0.000
CLINICAL SKILLS
Q30	47 (22.6)	27 (13.0)	66 (31.7)	47 (22.6)	21 (10.1)	31.038/0.000
Q31	34 (16.4)	40 (19.3)	67 (32.4)	40 (19.3)	26 (12.6)	22.976/0.000
Q32	10 (4.8)	9 (4.3)	36 (17.4)	45 (21.7)	107 (51.7)	154.135/0.000
Q33	33 (15.9)	23 (11.1)	51 (24.6)	46 (22.2)	54 (26.1)	16.454/0.000
Q34	115 (55.8)	25 (12.1)	31 (15.0)	16 (7.8)	19 (9.2)	168.466/0.000
Q35	11 (5.4)	7 (3.4)	39 (19.1)	48 (23.5)	99 (48.5)	134.137/0.000
Q36	6 (2.9)	17 (8.3)	58 (28.2)	75 (36.4)	50 (24.3)	80.748/0.000
TECHNICAL ASPECTS
Q37	10 (4.9)	19 (9.2)	54 (26.2)	64 (31.1)	59 (28.6)	59.874/0.000
Q38	29 (13.9)	27 (13.0)	66 (31.7)	49 (23.6)	37 (17.8)	25.077/0.000
Q39	57 (27.5)	29 (14.0)	71 (34.3)	28 (13.5)	22 (10.6)	44.184/0.000
Q40	19 (9.1)	23 (11.1)	66 (31.7)	43 (20.7)	57 (27.4)	40.654/0.000
QUALITY OF LIFE
Q41	49 (23.6)	25 (12.0)	49 (23.6)	41 (19.7)	44 (21.2)	9.404/0.052
Q42	19 (9.1)	10 (4.8)	36 (17.3)	41 (19.7)	102 (49.0)	124.740/0.000
Q43	15 (7.2)	15 (7.2)	48 (23.2)	36 (17.4)	93 (44.9)	99.739/0.000
Q44	68 (32.9)	29 (14.0)	46 (22.2)	37 (17.9)	27 (13.0)	26.792/0.000
Q45	62 (30.0)	25 (12.1)	53 (25.6)	30 (14.5)	37 (17.9)	23.604/0.000

. Regarding the education process (questions 13–22), most of the biomedical students were satisfied with the organization and type of learning (Table 3). This type of education induced some type of changed mental functioning, as we can see from the next part of the questionnaire (questions 23–29). A large number of students had a drop in motivation and concentration (Q23–34.4%; Q24–35.1%) and difficulties memorizing lectures (Q25–26.3%). On the other hand, online education during the COVID-19 outbreak did not induce mental stress in a large number of students (Q26–26.1%), but we could not diminish the significant number of the students with some type of stress during digital education (Q26–22.2%). Depressive emotion was not dominant amount biomedical students, and we also have an indicative number of students who reported that they had emotions that contributed to depression and anxiety (Q27–15.3%; Q28–17.3%). Moreover, the occurrence of insomnia in students was very significant, so about 39.2% of biomedical students reported the presence of insomnia during the COVID-19 outbreak and studying (Q29) (Table 3).

3.6. Multifactorial Analysis of Different Variables on the Perception and Attitudes among Biomedical Students

Table 4. The total score for each factor of eMedQ. The score for each factor was obtained by summing the points of all questions that are classified into a certain factor.

Factor	Mean ± SD	Median (Min–Max)
Factor I	40.48 ± 13.12	40 (14–68)
Factor II	38.36 ± 7.80	40 (15–50)
Factor III	21.34 ± 6.28	22 (7–35)

shows the mean scores and median of scores of factors one to three. As we can see, the means of the first two factors are similar, as well as the medians, while the third factor has the lowest mean. In that sense, to analyze which variable has the most significant impact on the factor separately, we carried out the analytical statistical test shown in

Table 5. Comparison of each variable related to the factor. Statistically analysis was conducted by using Mann–Whitney U test or Kruskal–Wallis test with the significance level of 0.05.

Characteristic	Factor I Median (IQR)	p	Factor II Median (IQR)	p	Factor III Median (IQR)	p
Gender *
Male	41.5 (18)	0.708	38.0 (12)	0.644	20.5 (10)	0.219
Female	39.5 (21)		40.0 (12)		22.0 (9)
Study program **
IASM	34.0 (21)	0.001	40.0 (18)	0.006	20.0 (9)	0.000
IASF	45.0 (20)		42.0 (11)		23.5 (8)
IASS	36.5 (15)		36.5 (14)		18.5 (9)
OSS	38.0 (17)		37.0 (9)		20.0 (7)
Year of study **
I	38.0 (14)	0.005	44 (11)	0.004	21.5 (10)	0.006
II	40.0 (20)		39.0 (10)		21.0 (9)
III	47.0 (23)		39.0 (9)		21.0 (5)
IV	52.0 (23)		43.0 (9)		24.0 (6)
V	40.0 (17)		38.0 (16)		20.0 (7)
VI	29.5 (20)		31.5 (14)		17.0 (10)
Level of skills in using electronic devices (computers, smartphones, tablets…) **
Inadequate	33.5 (NA)	0.545	32.5 (NA)	0.102	13.5 (NA)	0.249
Acceptable	35.0 (14)		39.0 (13)		19.0 (10)
Good	42.0 (14)		39.0 (12)		22.0 (8)
Very good	38.5 (22)		38.0 (11)		21.0 (9)
Excellent	38.5 (27)		42.5 (13)		22.0 (10)
Experience with online education before COVID-19 **
Very great experience	46.0 (26)	0.129	40.0 (13)	0.277	26.0 (8)	0.074
Great experience	43.0 (23)		37.0 (12)		21.5 (12)
Moderate experience	42.0 (15)		42.5 (9)		21.5 (10)
Little experience	38.0 (21)		40.0 (12)		21.0 (10)
Without any experience	37.0 (18)		38.5 (14)		21.0 (8)
Progress during the online education during the COVID-19 pandemic **
Very much	45.5 (23)	0.011	43.5 (9)	0.000	25.0 (9)	0.000
Much	44.5 (21)		41.0 (10)		23.0 (6)
Moderate	38.0 (18)		39.0 (11)		21.0 (8)
Little	32.0 (11)		35.0 (12)		18.0 (8)
I have not progressed	42.0 (19)		30.0 (11)		17.0 (9)
Form of online teaching experienced during the COVID-19 pandemic
Zoom/Teams/Webex/Google Meet meeting *
Yes	41.0 (21)	0.021	40.0 (12)	0.084	20.0 (8)	0.046
No	38.0 (15)		36.0 (14)		22.0 (8)
Pre-recorded videos *
Yes	36.0 (21)	0.104	37.0 (14)	0.213	22.0 (9)	0.944
No	42.0 (20)		40.0 (11)		21.0 (8)
Digital platform (Moodle, etc.) *
Yes	39.0 (23)	0.995	36.0 (14)	0.195	22.0 (10)	0.597
No	40.0 (20)		40.0 (10)		21.5 (9)
Presentation with narration *
Yes	38.5 (20)	0.963	40.0 (14)	0.993	22.0 (9)	0.580
No	42.0 (22)		40.0 (11)		21.0 (9)
Short online consultations in writing (chat consultations) *
Yes	46.0 (33)	0.881	44.0 (25)	0.741	24.0 (14)	0.544
No	40.0 (20)		40.0 (11)		21.5 (9)
Test questions *
Yes	42.5 (23)	0.262	40.0 (13)	0.291	22.0 (6)	0.384
No	40.0 (20)		40.0 (11)		21.0 (10)
Further improvement of the current form of online teaching *
It is necessary	37.0 (19)	0.006	37.0 (12)	0.000	21.0 (8)	0.017
It is not necessary	44.0 (21)		43.0 (9)		23.0 (10)
Online educational modalities that would significantly improve the acquisition of practical knowledge and skills in biomedical sciences *
Virtual Classroom
Yes	42.0 (23)	0.966	42.0 (12)	0.093	21.0 (9)	0.170
No	39.0 (21)		39.0 (12)		22.0 (7)
System Simulations of practical skills *
Yes	37.0 (18)	0.249	39.0 (12)	0.798	21.0 (9)	0.304
No	42.5 (24)		40.0 (12)		22.0 (9)
Educational Games/Gamification *
Yes	42.0 (22)	0.956	35.0 (14)	0.093	21.0 (8)	0.443
No	39.0 (21)		40.0 (11)		22.0 (9)
Clinical Scenarios, Virtual Patients, Clinical Vignettes *
Yes	38.0 (21)	0.995	40.0 (13)	0.591	22.0 (9)	0.811
No	41.5 (22)		39.0 (13)		21.0 (8)
Multimedia content/Educational multimedia streaming *
Yes	48.0 (18)	0.022	40.0 (10)	0.280	23.0 (10)	0.092
No	38.0 (19)		39.0 (12)		21.0 (9)
Area in which it would be useful to have additional teaching materials compared to existing online materials **
Preclinical subjects	38. 0 (15)	0.272	37.0 (14)	0.319	21.0 (9)	0.103
Subjects of clinical medicine, clinical pharmacy, and clinical dentistry	39.0 (22)		40.0 (11)		21.5 (8)
Other subjects:	42.0 (21)		40.0 (11)		23.0 (12)

* Mann–Whitney U test; ** Kruskal–Wallis test.

. Study program and year of study are two statistically significant variables that influenced factors one to three. There are differences between IASM vs. IASF: Factor I (0.023) and Factor III (0.003), IASF vs. IASS: Factor I (0.000), Factor II (0.020), and Factor III (0.002), and IASF vs. OSS: Factor I (0.006), Factor II (0.001), and Factor III (0.001) (Table 5). Moreover, the year of study determines the factors as follows: II vs. IV: Factor I (0.003), Factor II (0.025), and Factor III (0.010); IV vs. VI: Factor I (0.008), Factor II (0.001), and Factor III (0.000).

Moreover, progress during online education during the COVID-19 pandemic was observed as significant variables in determining the scores of each part of eMedQ. Interestingly, further improvement of the current form of online teaching is something that is also observed as a significant variable. Finally, the type of online application/platform was also something that statistically influenced factors one and three, where students choose one of the mentioned (Zoom/Teams/Webex/Google Meet meeting) (Table 5).

From our point of view, the most important variable extracted after statistical analysis was progress in education among biomedical students, where it is statistically confirmed that for the students it is important how much they can progress through digital education in this specific pandemic environment and in medical education (Figure 4). We observed the next significant differences: Very much vs. Moderate: Factor I (0.037), Factor II (0.002), and Factor III (0.002); Very much vs. Little: Factor I (0.010), Factor II (0.000), and Factor III (0.000); Very much vs. I have not progressed: Factor II (0.000) and Factor III (0.014); Much vs. Moderate: Factor I (0.016) and Factor III (0.039); Much vs. Little: Factor I (0.005), Factor II (0.002), and Factor III (0.001); Much vs. I have not progressed: Factor II (0.002); Moderate vs. Little: Factor II (0.027) and Factor III (0.040); Moderate vs. I have not progressed: Factor II (0.004). On the other hand, between the next answers there were no differences: Very much vs. Much: no difference; Little vs. I have not progressed: no difference (Figure 4).

4. Discussion

The purpose of this questionnaire study was an evaluation of current teaching approaches at the Faculty of Medical Sciences, University of Kragujevac, Serbia, in order to determine the attitudes of biomedical students about learning methods in the time of the COVID-19 pandemic, as well as their previous experiences. Moreover, the second aim was to establish the tool for determining the attitudes and perceptions of students from medical faculties regarding digital education in pandemic conditions.

All over the world, the impacts of the pandemic in recent years have been observed. In higher education, institutions responded innovatively and rapidly to the unusual conditions in each region. However, the impact varied according to the particular system, and for each of them this pandemic was a chance to answer social needs. How each region responds to the unexpected and provides the necessary medical education focused on students in health care professions are excellent representations of how the regional and national higher-education communities responded to the pandemic. Moreover, it is still important to find a reason for barriers to student learning that could be related to the learning environment [15,16,17,18].

The first part of the study is designed to develop and validate the eMedQ, a 45-item global questionnaire for determining the students’ perceptions of the learning environment. Our results suggested that eMedQ is an acceptable instrument and provided the evaluation of different aspects of students’ learning and medical education, such as demographic characteristics, experience with online teaching, education process (teaching organization), aspects of mental functioning, clinical skills, technical aspects, and quality of life. Developing the eMedQ in these seven dimensions, we represented a tool for measuring the overall perception of the learning environment. Initially, the target population for this study was undergraduate students and academic staff. However, during the development process, we expanded the previous scope related to academic staff, considering the large differences between previous skills, experience, and age among academic staff [4,19]. Furthermore, we included only students, but undergraduate, since that level of study requires more intensive, continuous, and practical education. Moreover, this eMedQ is focused on students from biomedical faculties and this tool could be relevant for students of health care professions (e.g., medicine, pharmacy, dentistry, and nursing sciences).

There are some other constructed surveys that were used in the measuring of the impact of COVID-19 in higher-education institutions. The first global survey conducted by The International Association of Universities under UNESCO was conceived in order to better understand the disruption caused by COVID-19 on higher education and to investigate the first measures undertaken by higher-education institutions around the world in response to the crisis. This instrument provides only the information about the learning but not the psychological aspects of ours.

The second aim of this research was to evaluate the study programs in the form of digital education related to biomedical sciences in the Republic of Serbia. Definitely, the COVID-19 pandemic suddenly transformed the ways of working, living, and relating to each other on a global level. This research was focused on education in medical faculties, and we showed how higher-education institutions transformed their own educative and training processes. We constructed the self-assessment survey, which examined and founded barriers and challenges that students have. Based on that information, we plan to transform higher education and apply in full the digitalization of education in the foreseeable future. According to our results, remote learning did not disrupt the quality of education in a greater manner, even in this last short time. Moreover, the system posed unprecedent challenges for students, but they had to keep assisting and learning in the same manner as before the pandemic. This technological revolution in higher education in the Republic of Serbia was very fast, but young people such as students from our research led these fundamental changes and accelerated them. The motivation of the most students was not decreased and even increased in some students with the requirements for new educational material and methods (e.g., chats, interaction with professors, recorded lectures, gamification, quizzes, etc.).

In the last two years, there have been many studies that examined the impact of technology on higher education during the age of COVID-19 [20,21,22,23,24,25].

Carrillo and Flores [26] carried out a literature review between January 2000 and April 2020 on online teaching and learning practices in teacher education to explore how and why online teaching and learning in teacher education occur and also to discuss its implications in the context of the pandemic. The authors emphasized the complicated character of the teaching method, point out specific elements such as social, cognitive, and teaching issues and the need for a comprehensive view of the pedagogy of online technology-based education used to support teaching and learning.

Di Pietro et al. [27] made a work that examined the direct and indirect ways in which the COVID-19 pandemic may influence education. Based on the existing literature and pre-COVID-19 data, it made predictions about the impact on and future of education. The results observed four major findings: (1) learning, on average, is expected to experience stagnation; (2) the impact on academic performance is likely to vary with socio-economic levels; (3) different socio-economic positions may manifest in an emotional response, as those from less privileged backgrounds may be under more environmental stress; (4) the broadening social gap may persist and have permanent consequences.

Since that phenomenon is still recent, there are just a few studies that discuss the direct effects of digital education in higher education caused by the pandemic, especially on pros, cons, and future consequences. Up to today, there are just literature reviews that systematically report about the transition from face-to-face education or traditional classroom education to the new remote or digital education [28,29,30].

Remote medical education is a specific situation and required the multidisciplinary approach. Undergraduate medical students have a lot of practical education, and the aim of their study is archiving skills. For example, radiology residents have to see many patients and radiographs in order to learn how to recognize different types of pneumonia. A very interesting previous report about the clinical skills of residents during the COVID-19 pandemic examined how the learning process was organized [31]. Mc Roy and coworkers describe a novel cloud-based HIPAA-compliant and accessible education platform that simulates a live radiology workstation for continued education of first-year radiology (R1) residents, with an emphasis on call preparation and peer-to-peer resident learning. Methods such as this and many others must be developed, because remote medical education must closely mimic face-to-face learning [31].

In Spain, a prospective study was conducted to examine the learning experiences in education during pandemic. The authors concluded that the imposition of e-learning provides limitations for older students and for students from rural areas, based on technical limitations [32].

In China, Huang et al. examined the effects of online education on providing the skills from chemistry. In this research, 56 teachers and 432 students were tested, and the authors concluded that the new way of education was a huge challenge for both students and teachers and that teachers must be more familiar with new technologies [33].

Moreover, previous authors examined the effects of the COVID-19 pandemic on the learning process among 242 students in Nigeria. They concluded that this pandemic has affected students’ well-being, behavior, and learning [34]. Similarly, research conducted by Russian teachers tended to examine the effects of the pandemic on the learning process but from the other point of view [35]. From the answers from 87 university teachers, the study provided the conclusion that many professional development programs at universities helped to minimize the negative impact of the rapid changes of the educational process [35]. Definitely, the COVID-19 pandemic has induced multilevel changes in the learning process among universities in different countries.

This study was the first research from this region that examined the impact of the COVID-19 pandemic on the perceptions and attitudes of biomedical students. Definitely, we observed the most important variables that influenced the global perception of remote medical education, such as type of study, year of study, progression in education, and striving for further improvement. Other difficulties have been solved, as we observed in this research.

The limitation of this study could be a lack of viewpoints from learners or educators regarding the switch from offline to online education among medical students. Moreover, a limitation is the unicentric placement of the study and including students from one institution, such as the University of Kragujevac. That will be the plan of the next research, when we will evaluate the lecturers’ acceptance of online education.

5. Conclusions

This research developed and validated a new tool (eMedQ) for the evaluation of biomedical students’ perceptions about digital education. It goes without saying that the COVID-19 pandemic has had profound impacts on society and the way humans organize themselves in the real world. The real consequences will be seen in the distant future. Our higher-education institutions invested heavily in technical infrastructure to remove the many barriers in shifting from conventional to a blended education model in biomedical sciences. Moreover, learning assessment and other approaches must be observed in order to comply with online medical teaching and learning pedagogy. Many other high-quality remote tools must be implemented for students of health professions who require practical skills.