*Article* **Development of an Online Asynchronous Clinical Learning Resource ("Ask the Expert") in Dental Education to Promote Personalized Learning**

**Rohit Kunnath Menon 1,\* and Liang Lin Seow <sup>2</sup>**


**\*** Correspondence: rohitkunnath@imu.edu.my

**Abstract:** This article describes the development and testing of an online asynchronous clinical learning resource named "Ask the Expert" to enhance clinical learning in dentistry. After the resource development, dental students from years 3 and 4 were randomly allocated to two groups (Group A—"Ask the Expert" and L—"lecturer-led"). All the students attempted a pre-test related to replacement of teeth in the anterior aesthetic zone. Group A (33 students) underwent an online case-based learning session of 60 minutes' duration without a facilitator, while Group L (27 students) concurrently underwent a case-based learning session of 60 minutes' duration with a lecturer facilitating the session. An immediate post-test was conducted followed by a retention test after one week. Student feedback was obtained. There was a significant increase in the test scores (maximum score 10) for both groups when comparing the pre-test (Group A—5.61 ± 1.34, Group L—5.22 ± 1.57) and immediate post-test scores (Group A—7.42 ± 1.34, Group L—8.04 ± 1.22; paired *t*-test, *p* < 0.001). However, no significant difference was observed in the test scores when comparing Group A to Group L for both the immediate post-test as well as the retention test (Group A—5.36 ± 1.29, Group L—5.33 ± 1.39 (independent sample *t*-test, *p* > 0.05). To conclude, adequately structured online asynchronous learning resources are comparable in their effectiveness to online synchronous learning in the undergraduate dental curriculum.

**Keywords:** e-learning; online learning; dentistry; dental education

#### **1. Introduction**

Learning from clinical cases or case-based learning provides an opportunity for students to demonstrate application of knowledge, thus augmenting the relevance of their learning [1]. Case-based learning promotes inherent motivation to learn, encourages self-directed learning and enhances clinical decision making abilities by repeated experiences [2,3], leading to a profounder understanding and reflection [4]. However, clinical case discussions are usually conducted between a clinical supervisor or lecturer and a group of students in a clinical setting. Clinical learning from clinical cases may also occur during case-based learning sessions conducted by a lecturer for a cohort. In view of the current pandemic, these sessions are routinely being conducted as online synchronous sessions between a lecturer and a group of students. These discussions are usually isolated bundles of learning between a faculty and a group of students. This approach provides restricted opportunity for feedback from other faculty who are not involved in the primary discussion and also precludes the participation from students who are undergoing clinical learning in other cohorts.

Harden and Hart have explained the benefits of e-learning in removing constraints for learning and expanding possibilities [5]. Computer-assisted learning (CAL) provides flexibility for students and teachers by enabling students to choose the time for learning

**Citation:** Menon, R.K.; Seow, L.L. Development of an Online Asynchronous Clinical Learning Resource ("Ask the Expert") in Dental Education to Promote Personalized Learning. *Healthcare* **2021**, *9*, 1420. https://doi.org/10.3390/ healthcare9111420

Academic Editors: Luís Proença, José João Mendes, João Botelho and Vanessa Machado

Received: 13 June 2021 Accepted: 24 September 2021 Published: 22 October 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

and freeing the time for teachers to focus on topics needing more close supervision [6]. Enhanced accessibility, diminished costs and effective time management have been cited as significant advantages that e-learning may offer as compared to other modes of learning [7,8]. Educational benefits of e-learning have been previously demonstrated in multiple areas including knowledge acquisition, assessment, development of professionalism and also acquisition of physical skills [9–12]. The concept of developing a "reusable learning package" [13,14] is advantageous in clinical learning, since it provides collaborative learning (learning across semesters/years and disciplines) available anytime and anywhere.

In addition to the development of new e-learning resource to enhance learning, evaluation and comparison of these resource to conventional/traditional methods of learning is equally important. In dentistry, e-learning has been previously found to be equally [6,15–17] or more effective [18–20] than traditional methods. However, some studies have significant limitations with respect to the method of assessment employed [18], and none of the aforementioned studies have investigated the impact of clinical case-based learning in dentistry on the knowledge acquisition and retention among dental students by employing methodology with minimal bias, thus ensuring reproducibility.

This study describes the development and evaluation of an asynchronous online clinical learning resource and the subsequent evaluation of its effectiveness by a randomized study. This study aimed to compare the knowledge acquisition and retention amongst dental students who utilized the online asynchronous clinical learning resource to those who underwent a lecturer-led learning session with the same content.

#### **2. Materials and Methods**

#### *Development of the Online Asynchronous Learning Resource: "Ask the Expert"*

We developed an online clinical learning resource named "Ask the Expert". The portal contains video-recorded clinical case discussions between a clinical supervisor/lecturer and a student. Students are encouraged to contribute clinical cases of interest in a previously provided case template. The case template is a PowerPoint presentation where the areas to enter the relevant patient details and the required photographs and radiographs are indicated. This is provided to ensure a relatively standardized format for case presentations in the learning resource (Supplementary Material Figure S1). The student is required to prepare the case as per the template and store it in a mobile device. Each clinical case is discussed with a clinical supervisor/lecturer using the student's mobile device (with screen recording) with two additional cameras capturing the discussion (Figure 1a). One camera focuses on the conversation between the student and the lecturer, whereas the other camera focuses on any study models used during the discussion. The discussion between the student and the lecturer is enhanced with the capability to draw on the clinical images and radiographs shown in the student's mobile device. Upon completion of discussion, the editing team combines the data from the two cameras and the student's recording on the mobile device to create an interactive video-based learning resource (Figure 1b). Self-assessment components are incorporated into each clinical case in the form of single best answer questions (Figure 1c). At the end of each recorded session, the student is asked to reflect briefly on the discussion with the expert with respect to what they learned. This is included at the end of each video. Further, a forum is created for each case, which is accessible to other students and internal experts for review and discussion. Students and faculty are able to access this anywhere and anytime by scanning a QR code.

**Figure 1.** (**a**) Video recording of the case discussion; (**b**) interactive video-based learning resource; (**c**) self-assessment. **Figure 1.** (**a**) Video recording of the case discussion; (**b**) interactive video-based learning resource; (**c**) self-assessment.

At the preliminary phase, the videos are shared with a group of 20 students to acquire preliminary feedback. The videos are re-edited as per student feedback into shorter segments of 1–5 min each. Self-assessment questions are provided in the initial segment before the clinical case discussion commences for each case. When answered incorrectly, students are directed to the section of the video where the correct answer is discussed. The learning resource is a learning bank for clinical cases covering a variety of cases in restorative dentistry. Each case has different learning outcomes, and the self-assessment questions are created from the recorded discussion and then incorporated into the resource. At the preliminary phase, the videos are shared with a group of 20 students to acquire preliminary feedback. The videos are re-edited as per student feedback into shorter segments of 1–5 min each. Self-assessment questions are provided in the initial segment before the clinical case discussion commences for each case. When answered incorrectly, students are directed to the section of the video where the correct answer is discussed. The learning resource is a learning bank for clinical cases covering a variety of cases in restorative dentistry. Each case has different learning outcomes, and the self-assessment questions are created from the recorded discussion and then incorporated into the resource.

To evaluate the educational impact of this learning resource, students in years 3 and 4 at the School of Dentistry at the International Medical University, Kuala Lumpur, Malaysia, were invited to participate in the study. The primary outcome of the study was to identify any difference in test scores between the two modes of learning. Ethical approval To evaluate the educational impact of this learning resource, students in years 3 and 4 at the School of Dentistry at the International Medical University, Kuala Lumpur, Malaysia, were invited to participate in the study. The primary outcome of the study was to identify any difference in test scores between the two modes of learning. Ethical approval for the

for the study was obtained from the Joint Committee on Research and Ethics at the

study was obtained from the Joint Committee on Research and Ethics at the International Medical University (Project ID: IMU 480/220). A study information sheet was provided to the students, and the students were given a period of one week to carefully study the information sheet. Students who participated in the creation of the online content were excluded from the study.

The topic covered for the clinical learning session was "Aesthetic restorative dentistry" and, specifically, restoration/replacement of teeth in the anterior aesthetic zone. The learning levels of both the year 3 and year 4 students were assumed to be similar for this topic.

#### Step 1: Pre-Test

A pre-test comprised of 10 one best answers (OBAs) of one mark each (based on the learning outcomes) were answered by the students who gave written informed consent to participate in the study.

#### Step 2: Randomization

Subsequently, the students were randomized into two groups, namely Group A (Ask the Expert group) and Group L (Lecturer-led group), by a simple cluster randomization technique. The allocation ratio was 50:50; however, only students who were interested in participating were asked to enroll. The random sequence was computer generated from a random number table. Hence, there was a difference in the number of groups. Consent to participate is an important consideration, especially in education research where students are vulnerable, which was maintained in this context. The learning outcomes and content for the topic were kept standard for both groups to eliminate bias.

#### Step 3: Intervention

Both the groups underwent the test concurrently during a commonly scheduled time. Group "Ask the Expert" (A; Online asynchronous learning)—30 students

Students in this group were able to access the online asynchronous learning resource "Ask the Expert" by using a login ID and password, which were provided for each student in the group for 60 min. The resource was uploaded with three clinical cases for the test. The students used the source independently and were not facilitated by a lecturer.

Group "Lecturer" (L; Lecturer-led learning)—27 students

An online synchronous session over Microsoft TEAMs was conducted by a single lecturer with the same clinical cases and content as for Group A for 60 min. The lecturer shared the cases as static PowerPoint slides with the group. After the case was presented, the lecturer instructed the students to answer questions in an OBA format for self-assessment (same as those included in the self-assessment for Group A). This was followed by a discussion between the students and the lecturer regarding the clinical case. The lecturer maintained the discussion similar to the content in Group A, ensuring that the content delivery was standardized. The session was recorded.

Both the online asynchronous session for Group A and the online synchronous session for Group L were conducted concurrently.

#### Step 4: Immediate post-test

Upon completion of the sessions, an online test was conducted for both groups concurrently, where 10 OBAs were to be answered in 20 min. This was the immediate knowledge acquisition test. The questions used in the immediate knowledge acquisition test were the same as in the pre-test.

#### Step 5: Retention Test

Both groups were provided with additional reading material including journal articles related to the topic covered. One week after the immediate test, a retention test was conducted for both groups (10 OBAs in 20 min). The questions in the retention test were new questions that included content discussed in the earlier session and information from the shared reading material. However, the newly prepared questions were aligned with the learning outcomes. The primary outcome of the study was to identify any difference in test scores between the two modes of learning.

After the completion of the retention test, all the students were provided access to the asynchronous clinical learning resource "Ask the Expert" and the recorded synchronous sessions to ensure fairness.

Step 6: Student feedback and evaluation of the "Ask the Expert" resource

Student feedback was obtained using a previously validated questionnaire [16]. Various Likert scales were used to test the students' beliefs about acceptability (Q1), effectiveness (Q2–5) and learning preferences (Q6–7). A section was provided for open comments.

#### **3. Results**

The average time taken for the development of a clinical case as a learning resource was calculated to be 180 min. The time calculated included the contribution by the student, the lecturer and the personnel involved in editing and uploading the content (Supplementary Material, Figure S2).

The mean and standard deviation of the scores in the pre-test, immediate post-test and the retention test obtained by the students with the pertinent analysis are depicted in Table 1.


**Table 1.** Test scores from the pre-test, immediate post-test and retention test.

The distribution of the scores for both groups are provided in Supplementary Material, Figures S3 and S4.

There was no significant difference in the test scores at baseline (pre-test) between the two groups (Group A—5.61 ± 1.34, Group L—5.22 ± 1.57; independent sample *t*-test, *p* = 0.406). There was a significant increase in the test scores for both groups when comparing the pre-test and immediate post-test scores (Group A—7.42 ± 1.34, Group L—8.04 ± 1.22; paired *t*-test, *p* < 0.001). No significant difference was observed in the test scores when comparing Group, A to Group L for the immediate post-test scores (independent sample *t*-test, *p* = 0.395).

We did not find a significant difference when comparing the pre-test scores to the scores of the retention test (Group A—5.36 ± 1.29, Group L—5.33 ± 1.39; paired *t*-test, *p* > 0.05). No significant difference was observed in the test scores when comparing Group A to Group L for the scores in the retention test (independent sample *t*-test, *p* = 0.788). The distribution of the scores for both groups for the pre-test and immediate post-test are depicted in (Supplementary Material Figures S1 and S2).

The questions used for all the tests are provided as Supplementary Material, Figure S5.

Student feedback was obtained in the domains of acceptability of the learning resource, and its effectiveness and the learning preferences of the students are depicted in Table 2. A total of 52% of the students (30/57) responded to the questionnaire. All the respondents found the method to be acceptable; 93% of the respondents rated the resource as good/very good, and 87% of the respondents indicated that the resource stimulated them to explore the topic further. A total of 60% of the respondents found the method to be timeefficient, 30% were neutral in relation to this question and 10% did not find the resource to be time-efficient.


#### **Table 2.** Student feedback.

A total of 63% of the respondents indicated that they would recommend the resource, while the rest remained neutral on this question; 60% of the respondents mentioned that they prefer learning from books, while 37% indicated online resources as the preferred method. A total of 70% of the respondents mentioned lectures as the preferred method, with the remaining indicating private study, e-learning and other methods.

#### **4. Discussion**

The need for the development of an online asynchronous clinical learning resource emerged from the inability of faculty and students who were not participants in a clinical case discussion to learn from and more importantly contribute to the discussion. A key factor which dictated the demand was feedback from students regarding lack of opportunities to learn from clinical cases being treated by their peers in different cohorts.

Provision of a standard case template was deemed necessary to enable standardization of presentation of cases and minimize preparation time. The students were encouraged to volunteer and share their own cases for discussion. Clinical learning may become more meaningful for dental students when they delve into their own experiences or clinical cases and learn from the content. This approach aligns with the theory of constructivism initially worked on by John Dewey, which proposes that learning is inherently related to action-knowledge, and when students extract learning from their own experiences, it may provide more meaning and significance to the learning [21]. Moreover, learning from one's own cases and cases treated by peers and faculty in the institution may lend a dimension of authenticity to the learning process, which may be absent in routine learning from the internet or textbooks.

The video-recorded case discussion with the expert marks the next step in the development of resource for clinical case learning. Interaction with the experts contributes to the learning process, where students are exposed to the thinking process of the expert during decision making. This mode of learning aligns with the concept of social constructivism emphasized by Jean Piaget and Lev Vygotsky. Profounder understanding may be achieved by the discussion, increasing the ability of the students to test their own ideas and synthesize and analyses the ideas of others [22,23]. Expert–student dialogue has been previously shown to enhance retention of knowledge and stimulate thinking in undergraduate dental students [24]. With respect to competency assessment in dentistry, expert–student dialogue has been previously shown to result in higher confidence and preparedness, leading to diminished uncertainty and stress. The aforementioned have been reported to contribute to the development of higher-order thinking and a broader clinical experience [25].

Self-assessment in the form of one best answers was incorporated at the commencement of each clinical case, and the same questions re-appeared after the segment of the video in which the answer to the question was discussed by the expert. Self-assessment has been previously established as an integral component of student learning through various studies conducted in dentistry [26–32]. Self-assessment may enable the students to understand and gauge their thinking and devise strategies to improve in this domain.

Another key element of each clinical case recording was a section on student reflection, where the student reflects on the learning after the completion of the discussion with the expert. After the video recording, the student summarizes and reflects on the discussion with the expert briefly. Reflective learning enables the student to critically review their own experience [33] and connect their current experience with previous learning and build on deeper learning. The incorporation of reflection as a component in the video segment is likely to facilitate deeper learning and critical thinking [34]. Observing a peer performing a reflective discourse (when other students watch the video) gives an opportunity for other students to reflect on and compare their own thought process while critically evaluating the clinical case. Further, the students and faculty may utilize the interactive forum to contribute to a discussion on the clinical case and share their views and experience. This helps to create an avenue for transparency in decision making in the institution and sharing of evidence-based resources in support of the decision or otherwise. Apart from internal faculty, external faculty when visiting as external examiners were also invited to participate in the clinical case discussion. This facilitated collaborative learning with faculty from an external university and hence provided a unique opportunity for the students.

It takes time, effort and money to generate computer-assisted learning (CAL) tools [35,36]. The development of a completed clinical case video takes 3 h. This includes contribution time from all the contributors, students, faculty and the e-learning department. The reusable learning object thus developed can be used by students and faculty anywhere and at any time and provides unique advantages. Sharing of learning resources and cooperation between universities can lead to economic advantage in the long run. CAL enables standardization of learning material delivery as compared to traditional methods of teaching, which involve different lecturers. Further, improving the interactivity, repeatability and feedback in the CAL program may increase their effectiveness. Real-time feedback and increased interactivity has previously been shown to enhance learning [37,38]. Interactivity incorporated into a CAL program might even be better in holding a student's attention when compared to traditional methods. Considering the advantages of CAL, new strategies to incorporate these into the curriculum and hence augment/replace conventional teaching should be deliberated.

The randomized study was conducted to evaluate the effectiveness of the current resource in teaching a topic in aesthetic restorative dentistry: "Replacement of teeth in the anterior aesthetic zone". Two cohorts were invited to participate in the study, and the current learning levels of both the cohorts were assumed to be similar for the specified learning outcome. There may be differences in the knowledge levels of year 3 and 4 students; however, the scope of the learning resource is aligned for all clinical semesters and

hence addresses topics with considerable overlap. The scores from the pre-test were not significantly different for both groups, and hence the assumption of baseline comparability was confirmed. Previous studies in dentistry comparing an e-learning intervention to a traditional method of learning have practiced this approach of conducting a pre-test to ensure homogeneity between the groups being compared [15,19,20,39], and out of these three studies used the same questions for the pre-test and immediate post-test as in the current study [15,20,39]. Ensuring the comparability between the groups at baseline is important to ensure homogeneity, particularly since the students belong to two different cohorts. Clinical learning during year 3 and year 4 involve topics which may be of interest and aligned with the learning outcomes for students across the semesters/years. Conventionally, clinical learning may inadvertently be restricted to a particular semester/year due to the allotment of clinical sessions or case-based learning sessions as per the scheduled timetable. This creates a situation where learning may occur in isolated bundles with inaccessibility for the other cohorts, even though the learning may be relevant to them. The creation of an online asynchronous learning resource was hence aimed at creating unbundled learning, which spans across faculty/students in the institution and beyond. There was a significant increase in the test scores at the immediate post-test, which was conducted immediately upon completion of the session for both groups. Hence, we concluded that both the asynchronous learning resource and the synchronous session with a lecturer were equally effective in delivering the learning outcomes for the session. However, no significant difference was found when comparing the scores between the two groups. Previous studies that have evaluated the effectiveness of e-learning and compared it with other forms of learning have yielded mixed results. Overall, e-learning is either equally [6,15–17] or more [18–20] effective than traditional methods of teaching. However, the use of different methods of assessment for the two groups as undertaken by Eitner et al. is a significant limitation of the study [18]. It is interesting to note that two [18,20] out of three studies in which e-learning had significantly better outcomes than traditional learning had an element of enhanced interactivity in the e-learning tool in the form of assessments and feedback. It is beneficial to assess both short-term knowledge acquisition and long-term retention in the same cohort, as the examinations and real test of what the student has learned is spaced out by time [6,19,39]. The finding from the study conducted by Silveira is also significant, as it indicates that knowledge retention regarding identification of cephalometric landmarks are significantly better after two weeks when compared to conventional learning [19]. Contrary to this finding, we did not find a significant increase in test scores for both groups at the retention test, which was conducted after one week. This may be explained by the fact that we used newly framed questions that were also based on the reading material provided to both groups after the immediate post-test. Nevertheless, the key finding was that there was no significant difference between the two groups when comparing the scores of the retention test, hence suggesting that the online asynchronous learning resource performed at par with the online synchronous learning session. It is interesting to note that the retention test scores were almost the same as the pre-test scores. This can be attributed to the fact that the questions in the retention test were formulated from topics included in the additional learning material shared with both groups. The students may not have adequately covered the learning material provided, leading to the drop in scores. This may reflect the real-life situation in education, where students need to fortify their learning with additional reading; however, they seldom do so. Over-dependence on content from learning tools alone or lecture notes may not be the best way to develop life-long learning skills. This could be considered as a limitation of the study, and in the future, reminders to refer to the additional learning material and feedback based on the exam performance may be used to motivate the learner to refer to the material.

The response rate for the feedback survey was low and may be attributed to survey fatigue for e-learning courses and other feedback requested by the school. The feedback on the e-learning resource was taken after the resource was made available to both groups

after the completion of the study to ensure fairness. Hence, intergroup comparisons were not made. The online asynchronous resource was acceptable and received good ratings from the respondents to the feedback survey. Most of the students gave feedback that the resource stimulated the students to explore the topic further. There is previous evidence that structured educational resources may develop desirable habits, linking curiosity and inquisitiveness in the minds of learners, leading to reflection and mindfulness [40]. Interestingly, when queried about learning preferences, most students still prefer to learn from books and through lectures. This may explain the fact that 63% of the respondents indicated that they would recommend the learning resource and the remaining remained neutral in their response. There is previous evidence on the superior effectiveness of lectures over e-learning in dental education [39]. Even though significant effort is being put into the development and validation of digital and e-learning resources, students may use these resources more when employed as augmentation to conventional methods of teaching and learning.

#### **5. Conclusions**

The online asynchronous clinical learning resource "Ask the Expert" was as effective as online synchronous teaching by a lecturer for clinical case discussions in restorative dentistry. The resource augments other modes of teaching in delivering learning outcomes related to clinical dentistry. This offers a "reusable learning environment" that provides unbundled learning, self-assessment, opportunity for reflection, discussion among peers and opportunity for collaboration with collaborating universities.

**Supplementary Materials:** The following are available online at https://www.mdpi.com/article/ 10.3390/healthcare9111420/s1, Figure S1: Case template, Figure S2: Time distribution for video resource generation, Figure S3: Distribution of test scores for Group A, Figure S4: Distribution of test scores for Group L, Figure S5: Questions used for pre-, post- and retention test.

**Author Contributions:** R.K.M. contributed to conception, design, data acquisition and interpretation and drafted the manuscript; L.L.S. contributed to conception and data interpretation and critically revised the manuscript. All authors gave final approval and agreed to be accountable for all aspects of the work. All authors have read and agreed to the published version of the manuscript.

**Funding:** The APC was funded by IMU.

**Institutional Review Board Statement:** The study was approved by the Institutional Review Board (or Ethics Committee) of the International Medical University (Project ID: IMU 480/220).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The data presented in this study are available on reasonable request from the corresponding author.

**Acknowledgments:** We would like to acknowledge the support from the e-learning department at the International Medical University, Malaysia, for their support in the development of the learning resource in the form of video recording and editing.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


### *Article* **Social Media Usage among Dental Undergraduate Students—A Comparative Study**

**Eswara Uma <sup>1</sup> , Pentti Nieminen 2,\*, Shani Ann Mani <sup>3</sup> , Jacob John <sup>4</sup> , Emilia Haapanen <sup>5</sup> , Marja-Liisa Laitala <sup>6</sup> , Olli-Pekka Lappalainen <sup>7</sup> , Eby Varghase <sup>1</sup> , Ankita Arora <sup>1</sup> and Kanwardeep Kaur <sup>1</sup>**


**Abstract:** Social media use among students has infiltrated into dental education and offers benefits but may also cause problems. The aim of this study was to explore and compare current social media usage among dental undergraduate students from two countries—Malaysia and Finland. A self-administered structured online questionnaire was used. WhatsApp, YouTube, Instagram, Facebook and Snapchat were the services that were most familiar to the respondents from both countries. There were differences between the students from the two countries among the most preferred platforms. The most frequently used applications were WhatsApp (91.1% of students in Malaysia and 96.1% in Finland used it very frequently) and Instagram (74.3% of students in Malaysia and 70.0% in Finland used it very frequently). Students in Malaysia spent significantly more hours per week using the platforms as study tools than students in Finland. Over 80% of the Finnish dental students reported that lack of knowledge was not an issue in social media usage, while 85% of Malaysian students felt that lack of knowledge prevented them from using social media platforms frequently. The findings offer evidence that dental students used social media extensively.

**Keywords:** social media use; dental students; social media platforms; dental training; Malaysia; Finland

#### **1. Introduction**

In today's digital world, most people are logged in perpetually and always connected. Our devices have ensured that technology is "always on us and always on"! Never has it become so easy to access information, and social media has become a major tool for communication and seeking information. We have a multitude of interactions with others, on topics that can be varied, and this has fused our professional and personal lives [1].

Social media is defined as "websites and applications that enable users to create and share content or to participate in social networking." Social media includes social networking platforms including Facebook and Twitter and media sharing sites, for example, YouTube and Instagram. In addition, there are other platforms like blog sites and microblogging sites. Healthcare professionals use social media extensively and it was reported that up to 90% of practicing doctors use Facebook accounts for professional or personal use [2]. Social media platforms offer different approaches to content sharing, and this

**Citation:** Uma, E.; Nieminen, P.; Mani, S.A.; John, J.; Haapanen, E.; Laitala, M.-L.; Lappalainen, O.-P.; Varghase, E.; Arora, A.; Kaur, K. Social Media Usage among Dental Undergraduate Students—A Comparative Study. *Healthcare* **2021**, *9*, 1408. https://doi.org/10.3390/ healthcare9111408

Academic Editors: Luís Proença, José João Mendes, João Botelho and Vanessa Machado

Received: 1 September 2021 Accepted: 18 October 2021 Published: 20 October 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

has wide-ranging uses in dentistry. Social media is useful not only for education and networking, but also for marketing and recruitment. Studies in health professionals' education have found benefits in the use of social media tools in clinical education [3]. Social media has become pervasive in society and is playing an important role in the personal and professional lives with dentistry being no exception.

In remote study conditions during the COVID-19 pandemic, the use of digital tools and social media platforms became imperative for medical and dental medical education for information retrieval, sharing of learning materials, and video meetings and discussions [4,5]. These tools were often used for both academic and non-academic purposes among students and teachers. Social media platforms were found to enhance ways of studying, allow for learning new skills, enhance performance, foster social relationships and social support and strengthen organizational identification [6,7].

Reports indicate that dental students use more than one social media application with Facebook being the most used platform among students in the United States and the United Kingdom [2–8]. The second most favored platforms were YouTube and Instagram. Skype and YouTube were used to improve dental skills while Twitter and blogging sites for interactions with the faculty and also to enhance communication [9].

A bibliometric study of articles published in journals indexed by the Web of Science database found 41 studies related to social media and dentistry during the period of 2010 –2016 [10]. Most of these studies focused on the impact of social media on dental education and professional practice. These studies emphasized the extension of the dental curriculum to involve the teaching and learning using social media platforms. However, these studies also highlighted the concern towards educational preparedness of future generations in the academic community, and understand the limitations of discourse produced by social media platforms. These studies also noted the concern regarding information mediation through social media platforms and its impact on dental education [10].

Significant differences in social trends, cultural beliefs and perceptions between Asian and European countries can affect the use of different social media applications and other technologies. This may also influence how these technologies are utilised by undergraduate students in their academics.

There are some previous studies about social media and dentistry in Malaysia [11–14]. Rani et al. [13] have described how dental undergraduates were trained to use social media for promoting oral health in the community, while Affendi et al. [11] evaluated the use of social media for marketing by dentists. See et al. [14] investigated the support, exposure and use of social media technologies among students, academics and administrators from both informatics and non-informatics undergraduate programs in Malaysia. In addition, internet addiction among dental students was also studied [12]. To our knowledge, there are no published studies about social media usage among Finnish dental or medical students.

The main purpose of our study was to compare the social media usage among dental undergraduate students from two countries Malaysia and Finland. This comparative study between similar cohorts of students from two different countries from two continents will help to estimate the extent and nature of social media use among undergraduate dental students. In addition, the study will provide suggestions for social media training in the dental curriculum. In this article, empirical data focused on the following research questions: How familiar are students with social media platforms? How often do students use social media services? How competent are the students at using social media services? How many hours do students spend using social media platforms as part of dental education? What factors encourage or prevent students to use social media?

#### **2. Materials and Methods**

#### *2.1. Study Design and Data Collection*

This was a cross-sectional online questionnaire survey conducted among dental undergraduates of the academic year 2020–2021 from two dental schools in Malaysia (Manipal Melaka Medical College and University of Malaya) and Finland (the University of Helsinki

and University of Oulu) each. Ethical approval to conduct this study was obtained for the institutions in Malaysia; Medical Ethics Committee, Faculty of Dentistry, University of Malaya [DF CD2105/0015 (L)] and Research Ethics committee, Faculty of Dentistry, Melaka Manipal Medical college [MMMC/FOD/AR/E C-2021(F-01)] prior to commencement of the study. According to the guidelines of the Ministry of Education and Culture in Finland, survey studies with anonymous questionnaires do not need approval from an ethics committee.

The instrument used was a questionnaire modified from a previous study among dental students in the USA [15]. The validated questionnaire assesses social media usage among medical undergraduates. Twelve items from the questionnaire were used to assess social media usage and the perceptions of social media usage in relation to dentistry/dental practice. Questionnaire items had three sections; Part A consisted of five questions regarding the demographic characteristics of the participants. Part B had six items regarding their familiarity, competence, time spent on various social media platforms and factors that encouraged and discouraged students from using social media. Part C had one item which addressed practice and perceptions regarding social media use in dentistry. Most questions required responses on a 4-point or 5-point Likert scale. The updated version of the questionnaire is included as the Supplementary Materials.

In Malaysia, the English questionnaire was pre-tested on a sample of five students in different years of the dental undergraduate programme at the Manipal Melaka Medical College and University of Malaya to check for semantic comprehension. Only minor modifications were made to the questionnaire following feedback from the pre-test. The questionnaire was translated into Finnish. In Finland, five dental students also pretested the first Finnish version. Based on their feedback, minor changes were made to improve the language and to clarify the purpose of the questions. The final questionnaire was administered using Google Forms from March 23rd, 2021, to April 11th, 2021, in both countries, the link being circulated via email and WhatsApp to student representatives of each year of study who then forwarded it to their classmates.

All dental undergraduates who received the online survey link were invited to participate in this study. In the online Google form, all the participants were asked to declare that they had read the participant information sheet (PIS) and voluntarily give consent for data collection and processing. If they refused consent, the questionnaire was closed. Inclusion and exclusion criteria were specified in the PIS. The survey was anonymous and did not include personal sensitive data.

For the estimation of sample sizes, we selected the time spent using social media as the outcome variable. The following formulas with finite population correction for proportions were used to estimate the minimum sample sizes in Malaysia and Finland:

$$m\_0 = \frac{z\_{\alpha/2}^2 p(1-p)}{e^2} \tag{1}$$

and

$$m = \frac{n\_0 N}{n\_0 + (N - 1)}\tag{2}$$

where *n* = required minimum sample size, *n*<sup>0</sup> = Cochran's sample size for large populations, *N* = available number of students (years 1–5) (population size), *e* = maximum error in estimation, *p* = proportion of the outcome variable (more than 15 h a week), *p*(1 − *p*) = variance of the outcome variable, *z* 2 *<sup>α</sup>*/2 = 1.96 for 95% confidence limit [16,17].

Setting maximum error to 5%, presuming that 50% of the student population have the outcome proportion of using more than 15 h a week social media and estimating that the population size *N* = 3250 in Malaysia, the minimum number of participating students should be at least 344 in Malaysia. Currently, there are about 1000 dental students (years 1–5) in Finland. So, the required sample size was 278 students in Finland.

#### *2.2. Data Analysis*

Tabular and graphical displays of data were used as the main tools of data presentation and analysis. The frequency and percentage distributions of participant characteristics (age, sex, year of study, and hours a week using social media) were presented for students from Malaysia and Finland. Percentage distributions were used to estimate the proportions of responses to questions "How familiar are you with each of the following social media services?", "How often do you use each of the following social media services?", "Approximately how many hours a week do you spend using the following social media services as part of your dental education?", and to question "How competent are you with each of the following social media services?" by country. We also compared the percentage distributions of students using very frequently the five most popular social media platforms and how competent they felt using these platforms by age, sex and year of dental school. In addition, frequency and percentage distributions of factors that encouraged or prevented students to use social media platforms were presented. Statistical significance of differences between Malaysian and Finnish student groups and basic characteristics were evaluated using a chi-square test with exact *p*-values. Among applications, we also evaluated the relationship between overall high use, competence at using and use in dental education using Spearman's rank correlation coefficient (rho). The data satisfactorily fulfilled the underlying assumptions and preconditions of the applied analysis methods. All statistical analyses were performed using IBM SPSS Statistics software (version 26) and Origin 2020 graphing software.

#### **3. Results**

#### *3.1. Participants*

A total of 613 students participated in this study. Table 1 shows the distribution of age, sex, year of dental school and hours per week using social media by country. Most of the participants were female in both countries. The student groups were quite different in terms of age and year of dental school. In particular, the Malaysian students were younger than the Finnish dental students. Most of the students spent more than 11 h per week using social media. Almost 75% of the Finnish dental students reported that they used social media at least 11 h per week.

**Table 1.** The frequency and percentage distributions of basic characteristics among dental students from Malaysia (*n* = 440) and Finland (*n* = 203).


#### *3.2. Familiarity with Social Media Platforms*

Most of the students were familiar with several applications (Figure 1). WhatsApp, YouTube, Instagram, Facebook and Snapchat were the most familiar services to the respondents from both countries. All Malaysian and Finnish students were familiar or very familiar with WhatsApp. In addition, all Finnish students were at least familiar with YouTube and Facebook. The number of students not familiar with YouTube or Instagram was also minimal. Only two students from Malaysia reported that they were not familiar with YouTube. Seven Malaysian students and one Finnish student reported that they had heard about Instagram but were not sure of its purpose. Students in Malaysia were more aware of Telegram, WeChat and Weibo than students in Finland. Respectively, the Finnish students were more familiar with Facebook, Snapchat, Jodel and LinkedIn. *Healthcare* **2021**, *9*, x 6 of 18

**Figure 1.** Percentage distributions of responses to question "How familiar are you with each of the following social media services?" by country. Data include dental undergraduate students from Malaysia (*n* = 440) and Finland (*n* = 203). Statistical significances between countries evaluated by exact chi-square test are as follows: 0.059 (Instagram), 0.054 (Tumblr), 0.002 (YouTube), <0.001 (all other services). **Figure 1.** Percentage distributions of responses to question "How familiar are you with each of the following social media services?" by country. Data include dental undergraduate students from Malaysia (*n* = 440) and Finland (*n* = 203). Statistical significances between countries evaluated by exact chi-square test are as follows: 0.059 (Instagram), 0.054 (Tumblr), 0.002 (YouTube), <0.001 (all other services).

*f Social Media Use*

Figure 2 shows that all students reported using more than one social media platform at least regularly. WhatsApp was the most commonly used, all students used it at least regularly, 91.1% very frequently in Malaysia and 96.1% in Finland. Instagram was the second most frequently used platform, 74.3% used it very frequently in Malaysia and

ter, Google+ frequently or regularly than those in Finland. The Finnish students were more likely to use Snapchat and Jodel compared to students from than in Malaysia. Most of the respondents in our survey had never used WeChat, Tumblr, LinkedIn, or Weibo.

o

*3.3. Reported Frequency* 

#### *3.3. Reported Frequency of Social Media Use*

Figure 2 shows that all students reported using more than one social media platform at least regularly. WhatsApp was the most commonly used, all students used it at least regularly, 91.1% very frequently in Malaysia and 96.1% in Finland. Instagram was the second most frequently used platform, 74.3% used it very frequently in Malaysia and 70.0% in Finland. Students in Malaysia were more likely to use YouTube, Telegram, Twitter, Google+ frequently or regularly than those in Finland. The Finnish students were more likely to use Snapchat and Jodel compared to students from than in Malaysia. Most of the respondents in our survey had never used WeChat, Tumblr, LinkedIn, or Weibo. *Healthcare* **2021**, *9*, x 7 of 18

**Figure 2.** Percentage distributions of responses to question "How often do you use each of the following social media services?" by country. Data include dental undergraduate students from Malaysia (n = 440) and Finland (n = 203). Statistical significances between countries evaluated by exact chi-square test are as follows: 0.304 (LinkedIn), 0.294 (WhatsApp), 0.118 (Pinterest), 0.014 (Tumblr), 0.002 (Facebook), <0.001 (all other services). **Figure 2.** Percentage distributions of responses to question "How often do you use each of the following social media services?" by country. Data include dental undergraduate students from Malaysia (*n* = 440) and Finland (*n* = 203). Statistical significances between countries evaluated by exact chi-square test are as follows: 0.304 (LinkedIn), 0.294 (WhatsApp), 0.118 (Pinterest), 0.014 (Tumblr), 0.002 (Facebook), <0.001 (all other services).

We also analysed the associations of background characteristics with the frequency of social media usage. For this analysis, we included only the five most popular platforms

students was not associated with the frequent use of these services in Malaysia. However, younger students used Instagram and Snapchat more frequently than older students in

Finland.

We also analysed the associations of background characteristics with the frequency of social media usage. For this analysis, we included only the five most popular platforms presented in Figure 2 and the analyses were stratified by country. Table 2 shows the proportions of respondents using frequently these basic student characteristics. The age of students was not associated with the frequent use of these services in Malaysia. However, younger students used Instagram and Snapchat more frequently than older students in Finland.

In Malaysia, male participants were more likely to report very frequent use of Facebook than female participants (45.6% vs. 27.9%). In Finland, male students were more likely to report using very frequently YouTube than females (51.1% vs. 12.8%) and female students used Instagram (77.6% vs. 44.7%) or Snapchat (55.1% vs. 38.1%) more often than males. Year of study was not associated in either country with the very frequent use of the platforms (Table 2).

**Table 2.** Percentage of students using very frequently the five most popular social media platforms by age, sex and year of dental school. Data include dental undergraduate students from Malaysia (*n* = 440) and Finland (*n* = 203).


Statistical significances evaluated by exact chi-square test for each platform.

#### *3.4. Perceived Competence of Social Media Use Healthcare* **2021**, *9*, x 9 of 18

Both Malaysian (99.6%) and Finnish (100.0%) students reported that they were highly competent or competent in using WhatsApp (Figure 3). We found some substantially significant differences in the reported competencies between students from Malaysia and Finland. More Malaysian students declared that they were highly competent in using YouTube, Instagram, Telegram, TikTok, Twitter and Google+. On the other hand, the Finnish respondents declared more often than they were highly competent using Facebook, Snapchat and Jodel. Figure 3 also shows that the majority of the students do feel that they are beginners or not at all competent in using Tumblr, LinkedIn or Weibo. YouTube, Instagram, Telegram, TikTok, Twitter and Google+. On the other hand, the Finnish respondents declared more often than they were highly competent using Facebook, Snapchat and Jodel. Figure 3 also shows that the majority of the students do feel that they are beginners or not at all competent in using Tumblr, LinkedIn or Weibo.

**Figure 3.** Percentage distributions of responses to question "How competent are you with each of the following social media services?" by country. Data include dental undergraduate students from Malaysia (n = 440) and Finland (n = 203). Statistical significances between countries evaluated by exact chi-square test are as follows: 0.426 (WhatsApp), 0.105 (Pinterest), 0.080 (Tumblr), 0.003 (Snapchat), <0.001 (all other services) **Figure 3.** Percentage distributions of responses to question "How competent are you with each of the following social media services?" by country. Data include dental undergraduate students from Malaysia (*n* = 440) and Finland (*n* = 203). Statistical significances between countries evaluated by exact chi-square test are as follows: 0.426 (WhatsApp), 0.105 (Pinterest), 0.080 (Tumblr), 0.003 (Snapchat), <0.001 (all other services).

Table 3 shows the proportions of self-reported high competence in using the most preferred social media platforms by basic student characteristics and stratified by country. Sex was associated with the reported competence of using the most commonly used ap-

Table 3 shows the proportions of self-reported high competence in using the most preferred social media platforms by basic student characteristics and stratified by country. Sex was associated with the reported competence of using the most commonly used applications. Female students in both countries are more likely reported to be highly competent using popular social networking sites than males. In Malaysia, no substantial difference between males and females was found only in the self-reported skill of using Facebook. In Finland, males and females reported similar highly competent only in the use of YouTube. Year of study had little influence on the self-reported competence of using the most common platforms (Table 3). However, our analysis indicated that age was associated with the perceived high competence differently in Malaysia and Finland. In Malaysia, younger dental students felt less often highly competent with YouTube and Facebook. In Finland, younger students reported more likely that they were highly competent in applying Instagram and Snapchat.

**Table 3.** Percentage of students reporting high competence in using the five most commonly used social media platforms by age, sex and year of dental school. Data include dental undergraduate students from Malaysia (*n* = 440) and Finland (*n* = 203).


Statistical significances evaluated by exact chi-square test for each platform.

#### *3.5. Reported Frequency of Social Media Use for Dental Education*

Participants were also asked about the time (hours per week) they spent using social media sites as part of their dental education (Table 4). Frequency and percentage distributions by country show that there were statistically significant differences between the countries among the most preferred platforms. For most applications, students in Malaysia spent more hours per week using the platforms as study tools than students in Finland. Jodel was the only social media application where the Finns spent more hours per week to manage their study assignments or tutorials.

**Table 4.** Frequency and percentage distributions of responses to question "Approximately how many hours a week do you spend using the following social media services as part of your dental education?" by country. Data include dental undergraduate students from Malaysia (*n* = 440) and Finland (*n* = 203).


We also analysed whether students used the same social media platforms for their personal use as well as for their educational purposes and if their perceived competence correlated with the use of these specific platforms for dental education. Figure 4 shows a strong association between personal and educational use of platforms in both countries. In addition, perceived knowledge of the use of the applications was associated with their use in educational purposes.

**Figure 4.** (**a**) Scatter plot showing correlation between percentage frequency of use of social media platforms (frequent and very frequent) for personal use vs. use in dental education in Malaysia and (**b**) in Finland; (**c**) Scatter plot showing correlations between perceived competence (highly competent) of use of social media platforms vs. use in dental education in Malaysia, and (**d**) in Finland. Data include dental undergraduate students from Malaysia (n = 440) and Finland (n = 203). rho = Spearman's rank correlation coefficient. **Figure 4.** (**a**) Scatter plot showing correlation between percentage frequency of use of social media platforms (frequent and very frequent) for personal use vs. use in dental education in Malaysia and (**b**) in Finland; (**c**) Scatter plot showing correlations between perceived competence (highly competent) of use of social media platforms vs. use in dental education in Malaysia, and (**d**) in Finland. Data include dental undergraduate students from Malaysia (*n* = 440) and Finland (*n* = 203). rho = Spearman's rank correlation coefficient.

#### *3.6. Encouraging or Preventing Factors to Use Social Media*

Table 5 reports factors that encouraged students to use social media platforms. To stay in touch with friends and family members was very important in both countries (77.0% in Malaysia vs. 86.2% in Finland). However, Malaysian students valued platforms more in connecting with old friends they had lost touch with than Finnish students. Malaysian students found social media platforms also more encouraging in communicating about issues related to dental training (Table 5).

**Table 5.** Frequency and percentage distributions of responses to questions "How important is each of the following factors in encouraging you to use social media?" by country. Data include dental undergraduate students from Malaysia (*n* = 440) and Finland (*n* = 203).


We also asked students about factors that prevented them from using social media. Students from Malaysia reported more often reasons that they experienced important not to use social media than students from Finland (Figure 4). Over 80% of the Finnish dental students reported that lack of knowledge was not an issue in social media usage, while 85% of Malaysian students felt that lack of knowledge prevented them from using social media platforms somewhat or very much. Similar differences were observed with lack of time, lack of interest, lack of perceived value and concern about harm to professional image (Figure 5).

**Figure 5.** Percentage distributions of responses to question "How much the following reasons (factors) prevent you from using social media?" by country. Data include dental undergraduate students from Malaysia (n = 440) and Finland (n = 203). There were statistically significant differences (p < 0.001) between the countries among all reasons (factors). Significances were evaluated by chi-square test. **Figure 5.** Percentage distributions of responses to question "How much the following reasons (factors) prevent you from using social media?" by country. Data include dental undergraduate students from Malaysia (*n* = 440) and Finland (*n* = 203). There were statistically significant differences (*p* < 0.001) between the countries among all reasons (factors). Significances were evaluated by chi-square test.

#### **4. Discussion 4. Discussion**

The present comparative study between Malaysian and Finnish universities was conducted to evaluate the social media usage among dental undergraduate students. We found that the same top five platforms (WhatsApp, YouTube, Instagram, Facebook and Snapchat) were the most familiar services to the respondents from both countries. There were country-specific differences among students in the familiarity and usage of the other platforms. For most applications, students in Malaysia spent more hours per week using the platforms as study tools than students in Finland. Malaysian students found social media platforms more encouraging in communicating about issues related to dental training. Finnish dental students reported more often that lack of knowledge was not an issue in social media usage, while the majority of Malaysian students felt that lack of knowledge prevented them from using social media platforms somewhat or very much. Age and year of dental studies were not clearly associated in either country with the frequent use of the The present comparative study between Malaysian and Finnish universities was conducted to evaluate the social media usage among dental undergraduate students. We found that the same top five platforms (WhatsApp, YouTube, Instagram, Facebook and Snapchat) were the most familiar services to the respondents from both countries. There were country-specific differences among students in the familiarity and usage of the other platforms. For most applications, students in Malaysia spent more hours per week using the platforms as study tools than students in Finland. Malaysian students found social media platforms more encouraging in communicating about issues related to dental training. Finnish dental students reported more often that lack of knowledge was not an issue in social media usage, while the majority of Malaysian students felt that lack of knowledge prevented them from using social media platforms somewhat or very much. Age and year of dental studies were not clearly associated in either country with the frequent use of the platforms.

platforms. Most students from both countries used social media for more than eleven hours per week. In addition, 25.3% of the respondents used social media more than twenty hours per week. This is not surprising but reflects the current habits of communication among young adults. Recent studies from different fields of study have reported that undergrad-Most students from both countries used social media for more than eleven hours per week. In addition, 25.3% of the respondents used social media more than twenty hours per week. This is not surprising but reflects the current habits of communication among young adults. Recent studies from different fields of study have reported that undergraduate students spent on average about three to four hours per day using social media platforms [18–21]. These services include mainly entertainment and communication, but also learning and searching for general information.

Our findings reflect the global popularity of the same platforms in contemporary society and show that most students are now heavy users of these platforms in the two culturally different countries. Most of the participants from both countries were familiar with WhatsApp, YouTube, Instagram, and Facebook. In a global world, students are also aware of other several social media services, but do not necessarily use all of them. There was a difference in the familiarity with some of the social media applications by the students in both countries. Malaysian students were more familiar with Telegram, WeChat, and Weibo while Finnish students were more familiar with Snapchat, Jodel and LinkedIn. This difference could be attributed to the students being familiar with the social media applications that are more commonly used in their region. Probably most of the students use those applications that their peers use so that there is an ease in communication and sharing. Jodel is a community-based social media application and is limited to its use in Europe and part of the Middle East [22]. LinkedIn is a social platform that mainly caters to people involved in business and has so far been very popular with the older age group [23] and among dental professionals looking to develop connections [24]. Finnish students being slightly older could account for their greater familiarity with LinkedIn. Familiarity with Telegram and its usage has grown exponentially in Malaysia as it is a popular medium to disseminate official information from Government [25]. There was no statistically significant difference in the usage of social media based on age, sex, and years of study in both the groups of students, though female students showed a greater tendency to use Instagram than the male which is similar to the worldwide data [26].

When considering the prevalence of using social media services among dental students, it is worth noting that these services have different purposes and target audiences. For example, the students in our sample may be too old to use Snapchat, a popular chat and communication channel for children and young people, for daily communication. On the other hand, they are still too young and without sufficient work experience to seek new work contacts through LinkedIn. The popularity of WhatsApp among study participants reflects the importance of keeping in touch with friends, family members or members of other restricted messaging groups in a convenient way (whether by text, picture or voice). Instagram and Twitter offer channels to increase internet online visibility. Using YouTube involves sharing your own videos and watching and commenting on other users' videos.

It was observed that all the students had greater competency in the social media applications they used most frequently which was primarily WhatsApp. Between both the groups of students, the difference observed regarding competence in using social media was the same as observed related to social media usage. Application usage by individual increases as he/she becomes more familiar with its features and thereby feels competent in using the application. Age of the student, year of study did not show any significance regarding the perception of competence for using social media applications, though females showed statistically significant competence in using Instagram, which seems to be very popular among females [26].

Our research clearly showed that students prefer to use for educational purposes the same platforms as they use for personal communication and online visibility. An interesting finding was spending a greater number of hours per week on the most popular social media applications like WhatsApp, YouTube, Instagram, and Facebook by the Malaysian students for purposes related to their dental education. This was statistically significant compared to Finnish students who used more hours on Jodel for the assignments or tutorials. This contrasts with a study from Saudi where dental students preferred to use Facebook for their learning [21].

The literature is not conclusive about the effects of social media on healthcare education. Some studies have reported a positive impact of social media on education while some have mentioned the reluctance of students to use social media for their education due to perceived negative impacts [27–30]. There is a study on the use of social media for dental health promotion [13], for dental marketing [11] and for dental education [31], as well as a review on what has been carried out so far in literature [10]. Literature also has

studies on the usage of social media by dental students and its effects [21,29,32], however, in Malaysia and Finland, it is not known how dental students are using social media. It is known that the dental students of the present cohort are digital natives, they use social media universally and their dental education includes information retrieval studies using online platforms [33]. In Finland, all study programs in the participating faculties require the use of social media in different forms now. Social media is an integral part of modern pedagogy along with other methods. The dental schools of this study in Malaysia (Manipal Melaka Medical College and the University of Malaya) and Finland (the University of Helsinki and the University of Oulu) follow the official guidelines for social media use and learning these is also an elementary study content of every dental student.

Dental students from both countries considered keeping in touch with family and friends as a very important factor that encouraged them to use social media. This is probably why WhatsApp as a communication tool is so popular. It should be noted that the effect of social media tools may affect soft skills such as communication and confidence in spoken language in the future among these students.

For Malaysian students, the reasons for refraining from the use of social media applications such as lack of knowledge about a particular application of interest, lack of time, lack of interest, lack of perceived value and concern about harmful effects on professional image, were more important than to Finnish students. In Malaysia, as per the MCMC report, one of the top reasons for Malaysians not using the internet in 2020 was a lack of interest [25]. The same report also mentioned that while the majority of Malaysians' online activity is sending text messages to communicate and visit social media sites, sharing of content using social media has gone up since 2018.

The extensive use of social media may have several drawbacks. Dental students should be made aware of the quality of information that they obtain from social media [33,34]. Social media content related to education may not be evidence-based or referenced from reliable sources since anyone can upload content that can affect not only their learning but also the profession [21]. Using social media can be time-consuming, addictive and distract from studying [19–21]. The social media activity of some healthcare students has also resulted in unanticipated ethical consequences [6]. In our study, questions were not specific to dental education and most of the questions were regarding the general use of social media. So, it would be inappropriate to extrapolate the questions of our study to social media use in dental learning. We will explore the disadvantages of using social media, long-term consequences, and perceptions of e-professionalism among dental students in our further study.

The present study used only a self-completed questionnaire and may suffer from the disadvantages of a cross-sectional online survey. The questionnaire failed to explain the underlying reasons for using specific platforms. In addition, the cross-sectional nature of our study made it impossible to assess possible rapid changes in students' use and preferences of social media. We did not study how social media communication has affected study engagement before and during the COVID-pandemic. Thus, we do not know whether and how the pandemic changed students' social media use. Future studies will need to study trends and priorities of social media use among dental students. Another limitation of this study should be noted. It was performed in local settings in Malaysia and Finland. Each regional setting is unique. Despite its limited scope, our findings might be helpful in considering the use of social media in different forms, and especially when considering students' attitudes, preferences, and experiences with various platforms.

#### **5. Conclusions**

This multi-institutional study provides useful information on the usage of social media during the COVID-pandemic among dental students in two culturally different countries. The findings offer evidence that dental students used social media extensively in both countries. A few apps, which are popular worldwide, are widely used by students for both personal communication and education. Regionally popular platforms bring variety

to the social media toolbox of dental students. Extensive use of social media can also be a distraction, especially when used for non-educational purposes. Students should be guided if they have specific interests or lack knowledge in some respects.

**Supplementary Materials:** The following are available online at https://www.mdpi.com/article/10 .3390/healthcare9111408/s1, Questionnaire.

**Author Contributions:** Conceptualization, E.U., P.N. and S.A.M.; methodology, E.U., P.N. and S.A.M.; formal analysis, P.N.; data curation, P.N. and E.H.; writing—original draft preparation, E.U.; writing review and editing, P.N., S.A.M., J.J., E.H., M.-L.L., O.-P.L., E.V., A.A. and K.K.; supervision, P.N. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Ethical approval to conduct this study was obtained for the institutions in Malaysia; Medical Ethics Committee, Faculty of Dentistry, University of Malaya [DF CD2105/0015 (L)] and Research Ethics committee, Faculty of dentistry, Melaka Manipal Medical college [ MMMC/FOD/AR/E C-2021(F-01)] prior to commencement of the study. According to the guidelines of the Ministry of Education and Culture in Finland, survey studies with anonymous questionnaires do not need an approval from an ethics committee.

**Informed Consent Statement:** Informed consent was obtained from all students involved in the study.

**Data Availability Statement:** The data presented in this study are available on request from the corresponding author.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


### *Article* **Pharmacy Student Challenges and Strategies towards Initial COVID-19 Curriculum Changes**

**Luyao Liu, Suzanne Caliph, Claire Simpson, Ruohern Zoe Khoo, Geenath Neviles, Sithira Muthumuni and Kayley M. Lyons \***

> Faculty of Pharmacy and Pharmaceutical Sciences, Monash University, Parkville, VIC 3052, Australia; lliu0007@student.monash.edu (L.L.); Suzanne.Caliph@monash.edu (S.C.); csim0005@student.monash.edu (C.S.); rkho0002@student.monash.edu (R.Z.K.); gnev0001@student.monash.edu (G.N.); smut0004@student.monash.edu (S.M.)

**\*** Correspondence: Kayley.lyons@monash.edu

**Abstract:** Due to COVID-19, tertiary institutions were forced to deliver knowledge virtually, which proposed challenges for both institutions and students. In this study, we aimed to characterize pharmacy students' challenges and strategies during COVID-19 curriculum changes, therefore developing a comprehensive understanding of students' learning, wellbeing, and resilience in the ever-changing situation. Data were collected from student written reflections across four year levels at one school of pharmacy from March–May 2020. In addition, data were collected from written responses of second-year pharmacy students responding to prompted questions. The data were qualitatively analyzed inductively by five coders using NVivo 12. For each piece of data, two coders independently coded the data, calculated the inter-rater agreement, and resolved discrepancies. The most coded challenges were 'negative emotional response' and 'communication barrier during virtual learning'. The most coded strategies were 'using new technology' and 'time management'. This study allows researchers and education institutions to gain an overview of pharmacy students' experiences during COVID-19, therefore helping universities to provide students with necessary support and techniques on how to self-cope with COVID-19 as well as stressful events in the future.

**Keywords:** e-learning; healthcare education; clinical teaching

#### **1. Introduction**

At the beginning of the COVID-19 global crisis, healthcare education institutions and their students underwent transformative change. Overnight, institutions cut placements, moved small-group learning to Zoom®, and delivered education virtually. The self-isolation and new virtual learning systems influenced students' study and daily life, potentially resulting in negative impacts on some students' well-being [1,2]. For example, a recent study in China reported that 24.9% of their medical student cohort experienced anxiety to some degree due to social distancing and a lack of interpersonal communication during the COVID-19 pandemic [3]. Students have reported several challenges and low satisfaction with engaging in virtual learning during COVID-19 [1,4]. In contrast, other authors have reported that health professions students have adapted well from the challenges and virtual learning has resulted in better attendance, engagement, and feedback as both teaching staff and students have created various ways to cope [5,6].

Due to these varied responses to online learning, scholars have been interested in exploring the factors that influence student satisfaction during COVID-19. Chen and colleagues have found that student satisfaction with online learning during COVID-19 was explained by the quality of the online platform, emotional changes, and communication with students [4]. Chiu proposes that self-determination theory can help explain student engagement during COVID-19 [7]. Self-determination theory encourages educators to support student autonomy, competence, and relatedness in online learning. Satisfaction of

**Citation:** Liu, L.; Caliph, S.; Simpson, C.; Khoo, R.Z.; Neviles, G.; Muthumuni, S.; Lyons, K.M. Pharmacy Student Challenges and Strategies towards Initial COVID-19 Curriculum Changes. *Healthcare* **2021**, *9*, 1322. https://doi.org/10.3390/ healthcare9101322

Academic Editors: José João Mendes, Vanessa Machado, João Botelho and Luís Proença

Received: 18 August 2021 Accepted: 28 September 2021 Published: 4 October 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

three basic needs will, therefore, improve students' behavioural, emotional, and cognitive engagement [7,8].

The purpose of this study is to gain a comprehensive understanding of pharmacy student benefits, challenges and strategies during the first few months of the COVID-19 pandemic. By examining the fallout from the immediate transition from face-to-face learning to distance virtual learning, this study contributes to a better understanding healthcare students' adaptability and well-being throughout the pandemic. There is limited research on what types of strategies students have implemented in response to COVID-190 s effect on their learning, well-being, and motivation. For example, we are extending the work by Chiu into higher education. Additionally, this study will add to the emerging literature of the types of benefits that COVID-19 has had on their learning and development as future health care professionals. Insights from this study will allow healthcare education institutions to identify necessary student supports for this current pandemic and any future interruptions. This study will also provide an insight for future researchers during this time of unprecedented stress on the students.

Research questions for this study as follows:

RQ1: What types of benefits did pharmacy students experience during COVID-19? RQ2: What types of challenges did pharmacy students experience during COVID-19? RQ3: What types of strategies did pharmacy students use during COVID-19?

#### **2. Materials and Methods**

*2.1. Study Design*

The study design was a qualitative case study at one university institution. Our aim was to characterize pharmacy students' experience during the first few months of the COVID-19 pandemic. The study was conducted through the analysis of students' written reflections relevant to COVID-19 curriculum changes. In total, 774 responses from March 2020 to May 2020 were analyzed using Nvivo 12. Our six coders were guided by the research questions to independently code students' written responses, and a codebook was developed based on several theories.

Previous studies have utilized software log data, questionnaires, interviews, and more [3,4,6,7]. Though these data sources are beneficial for understanding our studied phenomenon, we add a new perspective by analyzing students' naturally occurring reflections. We mined students' written reflections for what they spontaneously say affected them during COVID-19. These responses may differ than if we asked students directly in a questionnaire or interview. Other researchers have also mined written reflections for insights on the effects of COVID-19 but these were conducted with a focus on educators [9] and graduate students [10]. Findyartini and colleagues also explored health professions students' written reflections by looking at medical students in Indonesia [11]. Our study provides another case-study in this area to triangulate Finyartini and colleagues' results.

#### *2.2. Participants and Study Context*

The participants of this study included first, second, third, and fourth-year pharmacy students studying a Bachelor of Pharmacy (Honors) and/or Master of Pharmacy at Monash University in Melbourne, Australia. The program follows the Pharmaceutical Society of Australia's National Competency Standards Framework for Pharmacists in Australia [12].

The data collection occurred from March to May in 2020 following emergency education changes due to COVID-19. A summary of changes we made are outlined in a recent commentary [13]. Overall, the curriculum followed a standardized flipped classroom model explained in recent investigations of our program [14,15]. For on-campus learning, each week the students complete the following activities: (1) one day of self-directed online learning, (2) a day of large-class interactive lectures, (3) two days of small group workshops (one facilitator for 30 students in teams of five), and (4) a final day of "close-the-loop" lectures to answer any question and solidify the material. Before the first few months of COVID-19, the students attended large lectures and small group workshops in person. After COVID-19, these activities were shifted to Zoom conferencing technology. Some of the lectures and workshops were shortened or replaced with interactive online modules [10]. Before COVID-19, students attended placements in hospitals and community pharmacies, taking more responsibility each year they progressed in the program. After the first few months of COVID-19, some placements were shifted to virtual placements and others granted students more responsibility to assist with the increased workload at the pharmacy.

At Monash University, all pharmacy students complete a skills coaching program. The data for this study were collected in the context of a skills coaching program. The purpose of the skills coaching program is for students to develop their professional skills by discussing and reflecting upon eight professional skills: problem-solving, oral communication, written communication, empathy, reflective practice, integrity, teamwork, and inquiry. Students are required to attend skill coach meetings every few weeks. Before every skill coach meeting, students write Personalized Learning Plans (PLPs) on an ePortfolio. After submitting to the ePortfolio, a skill coach (e.g., faculty member, practicing pharmacist) provides feedback on the students' PLP.

#### *2.3. Data Collection*

Data were collected from two sources. The first data source was the students' PLPs across all year levels. The second source of data was second-year students' written answers to written prompts during a skill coach meeting discussing student challenges and strategies related to COVID-19.

For the PLPs, students have the liberty to talk about incidents that have affected their skills, and therefore COVID-19 was a frequent and naturally emerging topic. The PLPs were collected from March 2020 to May 2020, following the start of the COVID-19 outbreak. From all available PLPs, we only selected PLPs which contained the following words for this study: 'corona', 'COVID-190 , 'online', 'zoom', 'virtual', 'virus', 'lockdown', 'quarantine', and 'pandemic.' Of the many written PLPs, there were a total of 879 PLPs that met this inclusion criteria (i.e., contained certain words). A further 67 PLPs were excluded by coders due to the lack of relevance with our study. Each PLP varied in length from 100 to 400 words.

In addition to the PLPs, we collected data from an in-class learning activity. During a skills coach meeting, second-year pharmacy students were asked prompts on a shared Google Doc® to facilitate discussion in groups of 10-12 students and a skills coach. The prompts were "What impacts are you seeing from COVID19?" and "How are you supporting your well-being during this time?" We collected all available student responses— 197 responses from the first prompt and 97 responses from the second prompt.

#### *2.4. Ethical Considerations*

The Monash University Human Ethics Low Risk Review Committee approved this study (Project ID 24477). At the Monash Faculty of Pharmacy and Pharmaceutical Sciences, the pharmacy students are enrolled in an education research registry. Each year all students were informed of education research projects and presented with the opportunity to opt out of having their student data used for education research. For this study, we removed any PLPs from students who have opted-out of the education research registry.

#### *2.5. Data Analysis*

To analyze the data, we used a directed content analysis approach according to our three research questions [16,17]. First, we developed a codebook based on relevant theoretical frameworks and previous literature on self-determination theory, motivational strategies, learning strategies, and time management [18–21]. Research question two was guided by self-determination theory. Research question three was guided by frameworks in motivational, learning, and time management strategies. We open-coded responses for research question one.

Self-determination theory. Self-determination theory (SDT) is an empirically based theory of human motivation, development and wellness [18]. In our research, SDT served as the primary theoretical framework for coding students' challenges, investigating how the pandemic has been causing frustration on their three basic needs: competence, relatedness, and autonomy [22].

Motivational strategies. Motivation refers to a student's willingness to engage and persist in a task. Motivational regulation strategies may be triggered when students experience problems with their ongoing level of motivation, learning, and performance [19]. The coding was developed based on seven motivation regulation strategies identified by Maenpaa including environment structuring, self-consequating, goal-oriented self-talk, efficacy management, emotion regulation, regulation of value, and interest enhancement [19].

Learning strategies. The codes for learning strategies were primarily developed from ten different types of learning strategies examined by Dunlosky and colleagues including elaborative interrogation, self-explanation, summarization, highlighting (or underlining), the keyword mnemonic, imagery use for text learning, rereading, practice testing, distributed practice, and interleaved practice [20]. Ten learning techniques were either summarized from literature indicating they might improve student performance or surveys that students reported using them most frequently. As we aimed to identify strategies in virtual learning, the techniques were intentionally selected as students should be able to practice them without assistance and supervision.

Time management. Self-learning regulation models mainly focus on three perspectives of managing time: planning, monitoring, and regulating [23]. Uzir et al. clearly demonstrated that students who actively and consistently use time management strategies were associated with higher academic performance, established the relationship between the use of time management strategies and learning performance in blended learning of a health science course [21].

Six coders (LL, CS, RK, GN, SM, and KL) applied the codebook to the data using Nvivo12®. First, the six coders underwent a calibration phase by coding the data together. During the calibration phase, the coders added emerging codes and built group agreement for accurately coding the data. Then, five coders coded all of the second-year pharmacy student written responses together. After the calibration phase, the PLP data set (879 PLPs) were divided into teams of two coders. For each PLP, two coders independently coded the PLP. During this process, pairs of coders met to calculate inter-coder reliability and resolve their coding disagreements. The overall team of six coders met frequently to discuss changes to the codebook and KL frequently audited all of the team's coding output.

After coding 480 out of the 879 PLPs, the team decided it had reached a saturation point and ceased coding. The saturation point was defined as the time when coding did not produce any new codes and the data coded under each code was often redundant [24]. After coding had ceased, the team calculated a final inter-rater agreement rate. Following coding, we identified themes and categories according to the established research questions. Frequent and salient themes were reported.

#### **3. Results**

The simple inter-rater agreement rate for all codes was 80.3%. The final codebook is included in Table 1. The results are organized according to research question.

#### *3.1. What Types of Benefits Did Pharmacy Students Experience during COVID-19?*

Although to a lesser extent than the challenges, many students discussed how COVID-19 had resulted in positive impacts on their studies and lives. Although we did not originally plan to investigate the benefits of COVID-19, it became an important and emerging research question. The benefits pharmacy students gained during COVID-19 depended on their experiences. In general, students on placement during COVID-19 shared positive practice experiences, whereas students completing remote learning benefited from 'more time for themselves, family, friends, and Uni' and 'less travel commuting'.


**Table 1.** The types of benefits that pharmacy students experienced during initial COVID-19 changes.

Overall, of the benefit codes, having positive placement experiences was the most frequently coded theme (n = 27 instances). For some students, their placements during COVID-19 were the most experiential of their learning experiences. For example, one student said, "in my time at [hospital] amidst the COVID-19 pandemic, I had my most productive & growth-driven placement that has fundamentally changed how I approach my patients even as a student pharmacist." Students discussed various reasons for why their placements were valuable during this time. Several students said they were "contributing", "helping the pharmacy" and "a part of the team". Students also thought their placements were interesting as they witnessed firsthand how pharmacies handled the fallout from pandemic.

The second most frequently coded theme was less travel commuting (n = 14). Transitioning to virtual learning meant some students "can manage [their] study at [their] own time" and "catch up with uni work" because they "spend less time travelling to uni". Some students appreciated staying at home to enjoy more family time (n = 14), more friend time (n = 4) and more self-time (n = 7).

#### *3.2. What Types of Challenges Did Pharmacy Students Experience during COVID-19?*

Similar to the benefits, the challenges of COVID-19 depended on the setting. We identified three main themes under challenges using SDT: Autonomy frustration, relatedness frustration, competence frustration. However, challenges also varied whether students were participating in virtual learning, placements, part-time pharmacy job, and a group inquiry (i.e., research) project (See Table 2).

Autonomy frustration refers to the feeling of no choice when students are carrying out an activity [22]. For example, students discussed in their reflections about how they were negatively impacted by the travel ban. The travel restriction (n = 12) has particularly impacted on the students coming from/planning to travel to the regions on the travel ban list, resulting in ending their trips early or cancelling their future trips, especially international students.

Competence frustration refers to feeling incapable of carrying out an activity [22]. For example, some students were struggling with managing their time (n= 22) and keeping up to date with their study (n = 22). Without the accountability of physically attending lectures and workshops, some of them "found [themselves] tend to procrastinate the work and the work [kept] piling up day by day", and sometimes even worse, "[they] tend to forget that [they] actually still have classes and accidentally plan [ned] something on the time slot when [they] should be attending classes". Students who were unable to manage their time generally felt "rushed", "stagnant", "overwhelmed" and eventually "lost in [their] study".

**Table 2.** The types of challenges that pharmacy students experienced during initial COVID-19 changes.


Relatedness frustration refers to a lack of sense of belonging and connection to others [22]. As one of most coded themes, social isolation was coded 49 times (n = 49). Students were upset with entertainment restriction (n = 5) and travel restriction (n = 12). As one student illustrated, "while I love my alone time, finding anything to look forward to is a challenge". Unfortunately, a few students reported observing or personally experiencing racism (n = 6) during their part-time job in pharmacies. Students described "unfair", "feel attacked", and "upset" about the distressing incidents.

Emotional responses (n = 147). As one of the most coded themes, emotional responses refer to any negative emotions such as stress and anxiety. The emotional responses were related to curriculum changes, social isolation, placement and working environment changes and the risk of getting infected during the pandemic. Some students "lost [a] sense of routine" in virtual learning; some students "[felt] terrible about staying at home only"; some students were "overwhelmed to attend the placement during a pandemic" and some students were worried about serving an infected patient.

For those students participating in virtual learning, the most frequently coded theme was communication barriers (n = 68), followed by difficulty adapting to the virtual environment (n = 43). For some students, communication with their peers via virtual meetings was much more difficult than face-to-face communication. One of the barriers was the lack of body language (n = 11). The students said they "can't see visual cues" and it was "hard to make eye contact", making it harder to communicate virtually. Lack of engagement

(n = 26), motivation (n = 26), and teammate contribution (n = 19) have worsened students' virtual learning experiences, especially with group assignments. Students complained that they "[feel] oddly disconnected during group meetings because people tend to stay silent" and "[they] don't feel like [they] are learning or contributing".

For the students that were undertaking a placement at the time, many challenges were posed against them. This mostly included fourth-year pharmacy students who were assigned two 4-week placement blocks. Many other students also reflected on their parttime job in the pharmacy. The most frequently coded challenge was a busy pharmacy (n = 63) and angry and difficult customers (n = 61). An example of a busy pharmacy response from a student includes "this was the most busiest and strangest experience for me in pharmacy". In addition, they were required to supply limited stock to patients who tried to stockpile medications (n = 45) due to stock shortage (n = 32). Many students witnessed "the rush of medication hoarding", which raised their concern for continuing care as "regular elderly patients [unable] to get their medications due to low stock". In addition, some placement students had very limited placement exposure (n = 37) due to COVID-19.

*3.3. What Types of Strategies Did Pharmacy Students Use during COVID-19?*

We identified four main categories under strategies: mental and physical wellbeing, learning strategies, time management, and motivational strategies (See Table 3).


**Table 3.** The types of strategies that pharmacy students used during initial COVID-19 changes.

Physical and mental well-being. In terms of mental and physical wellbeing, COVID-19 preventative measures was the most coded theme (n = 84). Students emphasize that they have been practicing preventative and one student reflected, "As a pharmacist in training, this is an important lesson to be particularly vigilant and be role models for other students,

friends and families". Following by COVID-19 preventative measures, virtual social activities (n = 55) and leisure activities (n = 51) were the next two most coded themes. Some students have planned to "reach out family and friends" every day because they believe "stay connected" is very important during the 'isolation period'. Meanwhile, doing leisure activities such as "pick[ing] a new hobby", "baking", and "watch[ing] TV shows" took students' attention away from the current situation and their stress.

Learning strategies. Not every one of Dunlosky's learning strategies was coded from students' reflection [20]. However, students have developed their own learning strategies in response to virtual learning. The most frequently coded strategy from Dunlosky was distributed practice (n = 25); implementing a schedule to spare their learning activities evenly over time. For example, one student commented, "I just have to get into a routine of studying during study time, while giving myself breaks so that I don't wear out too soon." Most students have also employed their own learning strategies to assist with their continued learning. Most coded strategy was the use of new technology (n = 169) (e.g., Google Calendar) to adapt themselves to the virtual learning environment, especially for group communication and collaboration.

Time management. Most of the students acknowledged the importance of managing their time appropriately, especially in independent learning. Planning time ahead (n = 107) was one of the most coded themes in the codebook. Students implemented different tactics of planning their time ahead of study to help them keep on track. However, only one student mentioned evaluating use of time after the task was completed (n = 1).

Motivational strategies. Out of motivational strategies, the most frequently coded theme was goal-oriented self-talk (n = 21), which refers to the process that students think about various reasons for persisting or completing a task. For example, one student said, "If I ever get into a low head space with little motivation, I will remember my goal of wanting to help people which will put me back on track." Students also reported that they were motivated by peer support. Students appreciated the support from friends and team members, motivating them to "keep [themselves] accountable in studying". On the other hand, some students have also employed the same strategy to encourage their team members by "sharing how well [they] have worked so far and appreciating each member's contribution to the team.

#### **4. Discussion**

Our study characterized pharmacy students' benefits, challenges and strategies during the COVID-19 pandemic. Although previous literature has focused on students' mental and learning challenges during pandemic [1–3], our research results provide an overview of both positive and negative pharmacy students' experiences in specific contexts (e.g., placement, virtual learning). Overall, the transition to virtual learning was welcomed by some students and challenging for others. In placements, students experienced novel situations. However, students working and learning at community pharmacies experienced work demands, racism and difficult patient encounters. Nonetheless, students have implemented various strategies in order to overcome these difficulties and adapt to the new learning environment.

Similar to previous research [1], we found that the emergency implementation of virtual learning created new dynamics and posed new challenges for the students. In terms of self-determination theory, many students experienced competence frustration due to the abrupt transition to virtual learning. The competence frustration may have been attributed to a lack of structure [25]. In this case, students no longer have the structure of the campus, going to the library, starting their day, and hallway conversations with peers. As a potential result, many students described their struggles with personal accountability, procrastination, and time management. Deci and Ryan also outlined that the frustration of basic psychological needs can thwart the autonomous motivation therefore negatively affect the academic performance and achievement [18,26,27]. In addition to motivational strategies, the educators might consider implementing strategies to support students' basic

psychological needs in virtual learning. For example, educators might consider including students in decision-making of emergency changes to encourage student empowerment.

In particular, students cited poor-quality teamwork in virtual settings due to a lack of effective communication, motivation, and active engagement of all team members. The overwhelming descriptions of poor-quality teamwork is especially important considering that these students will work in healthcare in their career. These poor-quality teamwork experiences may discourage students from working collaboratively in the future. Therefore, educators may need to modify their approach. Just as multiplayer games and sport teams, group assignments requiring a strong team spirit can enhance students' relatedness satisfaction [8]. For example, educators may design the virtual group assignments to be more competitive to ensure effective engagement and contribution. Additionally, it might be necessary to consider liaising with students regarding expectations of active engagement in a virtual learning environment, such as switching on the webcam during lessons. Although previous researchers have shown that the characteristics of educators do not significantly influence students' learning outcomes in computer supported collaborative learning, they are still pivotal in relieving students' anxiety therefore improving students' academic performance [28].

Our research findings suggest that the majority of pharmacy students did not tend to employ learning strategies that were summarized from previous literature [20]. In a study conducted by Wolters et al., focusing on motivational regulation strategies, Wolters also found out that "students do not use all the types of motivational strategies equally" [29]. Although it is inspiring to see many students report developing their own strategies to overcome challenges and adapt to the new learning environment, this may reveal the need to educate the students on evidence-based approaches and introduce more advanced learning and motivational strategies. Due to the nature of the data source, the effectiveness of these self-developed strategies was not assessed. Therefore, future research that evaluates these strategies in the context of pandemic may be required.

In contrast, some students perceived some aspects of virtual learning as advantageous due to ease of accessibility and flexibility. Universities should apply what they learned from the emergency delivery of virtual learning to their future offerings. For example, our results suggest that students may prefer a combination of virtual learning on-campus learning. However, our research did not explore the correlation between virtual learning and academic learning outcomes and future research should continue to explore the affordance, effectiveness and barriers of blended learning models in the new post-COVID-19 reality. Specifically, future researchers could explore the impacts on educators, resources, and other types of students.

In addition to the findings that are corresponding to previous literature, our results provide an insight into the exclusive challenges faced by pharmacy students who went on placement or were participating in part-time jobs at local pharmacies. During the COVID-19 pandemic, some pharmacy students encountered limited placement exposure due to COVID-19 regulations in hospitals. On the other hand, students in community pharmacies felt that they were expected to handle excessive and unreasonable demands from customers in community pharmacy. In these cases, students may require individualized support from the faculty, in response to the rapidly changing placement and working environment. For example, educators might consider assigning additional assessment to students with limited exposure or providing additional training to students regarding how to handle these tough situations [13]. Our research is only limited to students' experiences, and future research might explore faculties' and preceptors' perspectives.

However, some students reported only the benefits of COVID-19 on their placement experience. These future pharmacists were able to contribute more in their placements than if COVID-19 had never occurred. This increased participation in practice may then contribute to students' professional identity formation [30] and sense of belonging [31]. When students participate in real-world communities of practice, they learn theories and skills in ways that are more readily applied in future careers [32].

Pharmacy students in our study reported heightened negative feelings and emotions during the pandemic, similar to a recent study by Zhai and colleagues involving medical students [2]. Factors that contributed to the negative emotions might be related to unfamiliar learning environments and placement experience and universal uncertainty during the pandemic. Many students have employed different tactics to support their mental health throughout the hard time. However, our results show that only a handful of students have self-reported utilizing the counselling service available at the University. Accordingly, it may be seen as an opportunity for the educators to encourage the pharmacy students to take advantage of the supporting services provided by the tertiary institutions. In addition to current existing services, educators may consider developing new programs to support students' mental health during COVID-19 pandemic.

Our research finding suggests that autonomy and relatedness frustrations were related to the international students and students who live far away from their families. It is well established that the fulfilment of each basic psychological need is determinative with regard to an individual's daily well-being [8]. While a large number of students reported experiencing negative emotion towards the pandemic as per our research results, specific plans may be tailored for individuals, specifically for international students and those who live far away from home. For example, the educators may consider scheduling group meetings for those students and a sense of belonging can be enhanced by sharing their experience with people in similar situations [33].

Our study was not without limitations. A limitation of our study was that we were unable to identify the year level of each student from the data source. To better aid student development, future researchers could evaluate the impacts and needs for each stage of student development. Although this research provides an in-depth view of one pharmacy cohort, any national or international policies should also account for related research in other cohorts, professions, institutions, and countries. Further, the data source used in this study is limited. Although analyzing student reflections reveals topics of central importance to students, some students may have written their reflections as an academic exercise, thereby "going through the motions" while writing their PLPs. In addition, it is possible that some students may be more challenged to express themselves in writing than in an interview or focus group study, whereas others may be more apt to express themselves through written reflections. Furthermore, the research provides first-hand information on pharmacy students' experience in the early stage of the pandemic. However, due to the time constraint, we were not able to follow up any possible changes over time. The future research might compare students' experiences in the early stage and the experiences in the later stage to better understand how students adapt to the COVID-19 pandemic. In addition, since the data were collected for other purposes (i.e., teaching and learning), there was a missed opportunity to comprehensively address all of the theoretical frameworks. For example, we were unable to explore social support factors for self-determination theory.

#### **5. Conclusions**

This study provides a high-level overview of pharmacy students' benefits, challenges and strategies during the initial months of the COVID-19 pandemic. Our result demonstrated that the most coded challenges were "negative emotional response" and "communication barrier during virtual learning". The most coded strategies were "using new technology" and "time management". These research findings may help researchers and educators better understand students' well-being and adaptability during the COVID-19 pandemic. Future researchers should investigate the long-term effects of COVID-19 on health professions students. Overall, tertiary institutions and educators may use the research findings to provide support that better suits the pharmacy students' needs in the ongoing pandemic and any future emergency events.

**Author Contributions:** Conceptualization, K.M.L.; methodology, K.M.L.; formal analysis, C.S., R.Z.K., G.N., S.M., K.M.L.; writing—original draft preparation, L.L., C.S., R.Z.K., G.N., S.M., K.M.L.; writing—review and editing, S.C.; supervision, K.M.L., S.C. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received funding from the Faculty of Pharmacy and Pharmaceutical Sciences at Monash University.

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Monash University Human Ethics Low Risk Review (Project ID 24477, 2020).

**Informed Consent Statement:** Participant consent followed an opt-out consent process due to the low-risk study. At the Monash University Faculty of Pharmacy and Pharmaceutical Sciences, the pharmacy program runs an education research registry. Each year all students are informed of education research projects and presented with the opportunity to opt-out of having their student data used for education research. For this study, we removed any PLPs from students who have opted-out of the education research registry.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**

