Enhancing Final-Year Medical Students’ Clinical Examinations Performance via a Transdiagnostic ACT-Based Intervention

Abstract

Introduction: Performance anxiety is a transdiagnostic construct similar in both sports and clinical medical examinations. Given that acceptance and commitment therapy (ACT) and other similar therapies have transdiagnostic evidence in improving the performance of athletes, the same approach can be adopted for medical students in improving their performance during clinical examinations. This pilot study aimed to assess the efficacy of a brief ACT-based intervention in improving transdiagnostic performance anxiety in clinical medical students through assessing changes in psychopathology scores (depression, anxiety, and stress) and psychological process variables (psychological flexibility, cognitive fusion, and mindfulness). Methods: Final-year medical students were randomized into intervention and control groups. A one-day ACT-based intervention was delivered to an intervention group, with the control group crossing over one month post intervention. Both groups filled in sociodemographic questionnaires and questionnaires measuring psychological flexibility, cognitive fusion, mindfulness, depression, anxiety, and stress at three time points: T1 (before intervention), T2 (immediately after intervention), and T3 (one month post intervention). Repeated measures ANOVA was employed to assess the change between the intervention and control groups over time. Results and Conclusions: There was a significant change in anxiety scores from T1 to T3. In addition, there were significant improvements in mindfulness, cognitive fusion, and psychological flexibility scores over time in the intervention group compared with the control group. This pilot study builds on small single-sample evidence bases for the efficacy of an ACT-based intervention in non-sports performance enhancement, suggesting that larger-scale randomized trials of similar interventions in clinical medical students may prove equally efficacious.

1. Introduction

Mindfulness-based therapies have increasingly gained currency as a panacea for treating multiple disorders across the mood and neurotic spectrum. In the specific group of university student populations, which overlaps with the age of presentation of both clusters of disorders, there is evidence both for individual [1] and group mindfulness [2] for both psychological distress and academic results [3,4]. However, mindfulness-based therapies can be employed as experiential avoidance tools if not coupled with effective action and behavior change. ACT (Acceptance and Commitment Therapy) is an offshoot of pure mindfulness focusing on two aspects: acceptance (mindfulness techniques) coupled with committed action (identification of values and goals and plans in line with them) [5]. It is transdiagnostic and has been used for various conditions, including depression, anxiety, and substance use [6,7,8].

An exciting new field of interventions is in the field of performance enhancement. There are existing interventions that share similar concepts with ACT per se, including the Mindfulness Acceptance and Commitment (MAC) protocol [9] for increasing “flow” [10] and the Mindfulness Sports Performance Enhancement (MSPE) protocol [11,12]. There is substantive evidence that such transdiagnostic mindfulness-based therapies are efficacious, even for one intervention, in terms of boosting performance by increasing psychological flexibility [13]. Some of the fields that have benefited from this include springboard diving [14], tennis [15], basketball [16], and weightlifting [17]. Apart from sports, military life has been associated with benefits from mindfulness in the sense that perceived stress on hopelessness among soldiers is inversely proportional to the level of mindfulness [18]. A recent study also demonstrated that subjective well-being and self-compassion are positively associated with mindfulness [19].

Nonetheless, there is little research on using ACT-based methods to tackle the specific issue of performance anxiety in a clinical performance setting. This research project proposes using a transdiagnostic ACT-based approach to tackle performance anxiety in clinical examinations for medical students. Clinical medical examinations generally do not merely assess discrete knowledge or skills per se, but rather are very cross-sectional assessments that reward individuals who have the ability to remain calm under pressure [20]. Hence, there is a very large component of performance anxiety that can impair the performance of a previously reasonable candidate. A key part of this is intervening in a way that can increase psychological flexibility [21].

Psychological flexibility is defined as the “ability to contact the present moment more fully as a conscious human being, and to change or persist in behavior when doing so serves valued ends” [22]. This would and should be able to improve performance in medical students, as they are able to do a few things better after attending. Firstly, they focus on the goals and values that drove them to study medicine and identify “away moves” that lead them away from their desired values [23]. Then, they learn the necessary skills of acceptance and being present to help them manage their responses to anxiety rather than pushing it away or internalizing it unproductively. An athlete receiving ACT learns the skills of mindfulness, cognitive defusion, acceptance of difficult internal experiences, and values clarification [9]. These skills will then further enhance behavior changes that facilitate performance enhancement.

The research question is hence whether ACT can improve psychological flexibility and reduce markers of psychopathology in medical students undergoing clinical-based performances. The objective of this study was to assess the efficacy of the ACT-based intervention in improving performance in clinical medical examinations. Thus, the hypothesis was that an ACT-based intervention would result in significant improvements in psychological flexibility, state mindfulness, and cognitive failures, as well as significant decreases in psychopathology, namely depression, anxiety, and stress.

2. Materials and Methods

2.1. Intervention and Instructional Program Followed

A one-day ACT-based intervention tailored to clinical medical students was created by adopting methods described in the Mindfulness Acceptance Commitment model and condensing the methods into the triflex model described by Russ Harris. Author EK also adjusted the model on the basis of his clinical experience as an ACT practitioner to ensure that there was a continual coherence between each session within the module. There was a total of 5 sessions during the one-day intervention, comprising of an introduction, being present, being open, being committed, and a closure session added toward the end to encourage students to exchange experiences and skills for performance. This session was added to ensure that the other sessions prior to that were absorbed and utilized. Because of pandemic travel restrictions, the intervention was delivered by EK online while the students were grouped together in the same room. Authors NP and MAMK, who were both experienced in ACT, remained physically in the group to assist and provide feedback to students, which was consistent with ACT principles. Details of the program are explained as per

Table 1. Contents of the Program.

Contents of the Program
The sessions given	Two sessions were conducted one month apart.
Modality	Author EK delivered the intervention online. Author NP and MAMK were in the room to facilitate intervention. Recipients of intervention were all together in a room face-to-face.
Total duration and by sessions	Each intervention was one day long.
Instructional procedure followed	There was a total of 5 sessions during the one-day intervention, comprising of an introduction, being present, being open, being committed, and a closure session added toward the end to encourage students to exchange experiences and skills for performance.
Fidelity of the treatment
Prior written protocol	A written protocol was conducted prior to the intervention by the instructor.
Training of instructors	As there was only one instructor, the protocol did not need to be replicated. The two facilitators did not deliver the intervention but facilitated responses.
Follow-up of their application	Participants were followed up one month later.
Guarantee of application to all in the same way	As there was only one group; there were no fidelity issues as such.
Number of groups	One intervention group and one control group.
Characteristics of intervention
Who were the applicants?	Final-year medical students undergoing professional clinical examinations.
Strategies and techniques implemented	Acceptance and Commitment techniques were adapted using the “triflex” of being present, being open, and being committed.
Phases	There was only one phase.

.

2.2. Participants

The study was performed in the medical faculty of a public university on Borneo island, which accepts students on the basis of merit in a nationwide examination and a battery of interviews and psychometric testing. To meet the inclusion criteria, participants in both groups were required to be final-year medical students who were experiencing anxiety about performing clinical examinations and volunteered to participate in the one-day intervention. The exclusion criteria were acute medical or psychiatric illness and non-consent. All medical students were in the same phase of their program and were three months from sitting for the major exit examination (Professional II examination); hence, each participant had equal representativeness. Convenience sampling was employed.

2.3. Procedure and Intervention Design

The number of experimental groups was two—an intervention and a control group—which students were randomized into. The control group crossed over to the intervention after one month. Pre–post monitoring measures were as follows: the intervention group (Group I) completed the questionnaires at three time points: T1 (prior to the intervention as a baseline), T2 (immediately after the intervention), and T3 (one month post intervention). The control group (Group C) completed the questionnaires at three time points too, similar to the intervention group: T1 and T2 (at the same time as the intervention group) and T3 (one month later, prior to their cross-over intervention).

2.4. Variables and Instruments Used

Both groups were given a set of questionnaires to complete at certain predetermined time points, which measured psychological process variables, namely psychological flexibility, state mindfulness, self-reported failures in perception, memory, and motor function, as well as psychopathology, focusing on depression, anxiety, and stress scores. For psychological flexibility, the AAQ-II was used. For state mindfulness, the Mindfulness Attention and Awareness Scale was used. For self-reported failures in perception, memory, and motor function, the Cognitive Failures Questionnaire (CFQ) was used. The DASS-21 was used to measure depression, anxiety, and stress. All measurements utilized English scales and all instruments used are included in Supplementary Materials in this study.

The DASS-21 questionnaire is a set of three self-report scales measuring the emotional states of depression, anxiety, and stress [24]. Respondents were requested to complete the DASS-21 on the basis of the presence of a symptom over the previous week. They rated each item on a four-point Likert scale ranging from 0 (did not apply to me at all over the last week) to 3 (applied to me very much or most of the time over the past week), with higher scores indicating greater severity. It is a very widely used composite measure of the three psychopathologies measured and is routinely used in clinical practice as well as research. The internal consistency was between 0.85 and 0.89 in this study, while the average variance extracted was between 0.46 and 0.55. Furthermore, discriminant validity was achieved with the square root of AVE ranging from 0.82 to 0.86.

The Cognitive Failures Questionnaire (CFQ) is a 25-question instrument designed to represent cognitive errors in daily life [25]. These questions reflect different aspects of cognitive functioning, including attention (“Do you fail to notice signposts on the road?”), perception (“Do you fail to see what you want in a supermarket (although it’s there)?”), memory (“Do you find you forget appointments?”), impulsivity (“Do you say something and accomplish afterward that it might be taken as insulting?”), and language (“Do you find yourself suddenly wondering whether you’ve used a word correctly?”). Each question is answered on a frequency scale, ranging from never (0 points) to very often (4 points). The total scale ranges from 0 to 100 points. The CFQ is one of the unique scales illustrating this dimension and is used primarily in research settings with robust psychometric properties, including high internal consistency and test–retest stability [26,27]. The internal consistency was 0.96 in this study, while the average variance extracted was 0.77. Furthermore, discriminant validity was achieved with a square root of AVE of 0.94.

The Acceptance and Action Questionnaire (AAQ-II) is one of the most widely accepted instruments in ACT studies and ascertains experiential avoidance and psychological inflexibility [23]. Experiential avoidance refers to the attempt of the individual to ignore unpleasant thoughts, feelings, and physical sensations, which leads to measures against one’s values, causing continual harm [5]. Psychological inflexibility is defined as firm psychological action against one’s value to avoid distress and uncomfortable feelings or thoughts, ignoring the present moment [22]. AAQ-II is a unidimensional scale with 7 items and is rated using a 7-point Likert scale. AAQ-II has good internal consistency (α = 0.88) and good test–retest reliability over 3 and 12 months at 0.81 and 0.79, respectively [28]. Higher AAQ-II scores reflected significant psychological inflexibility and have also been found to be associated with greater levels of depressive symptoms, anxiety and stress, thought suppression, and psychological distress. The internal consistency was 0.92 in this study, while the average variance extracted was 0.64. Furthermore, discriminant validity was achieved with a square root of AVE of 0.89.

The Mindful Attention Awareness Scale (MAAS) is a popularly used research instrument that measures the level of awareness and attention to the present-moment experience [29]. It consists of 15 items, and participants respond to each item using a six-point Likert scale ranging from 1 (almost always) to 6 (almost never) [30]. Higher scores indicate higher levels of mindfulness. The internal consistency was 0.94 in this study, while the average variance extracted was 0.53. Furthermore, discriminant validity was achieved with a square root of AVE of 0.85.

2.5. Data Analysis

Data were analyzed according to the intention-to-treat principle. All participants were analyzed according to the condition (Group I or Group C) to which they were initially randomized. Multiple imputation by the SPSS 26.0 software package was adopted for missing data. Data were analyzed with descriptive statistics, and analyses of normality were performed for all continuous data.

Repeated measures ANOVA was employed to analyze the repeated measurement data, as the normality analysis suggested a normally distributed data set. Mauchly’s test of sphericity was calculated for all variables to establish whether there were significant differences between the variances of the differences, and if violated, Greenhouse–Geisser corrections were used if ε < 0.75, and Huyhn Feld corrections were used if ε > 0.75. Subsequently, the estimated marginal means for each measured parameter were plotted on a graph for three time points—pre-intervention (T1), post intervention (T2), and one month post intervention (T3).

This study was conducted according to the Helsinki Declaration and was approved by the Institutional Review Board of Universiti Malaysia Sabah (IRB serial number: 110606). Written informed consent was obtained from all the participants involved in this study. Data protection was ensured by utilizing paper forms, which were not stored electronically on any devices, and the paper forms were shredded at the end of the study after a period of two years.

3. Results

A total of 22 participants volunteered for the one-day intervention out of 83 students enrolled in the final year of the medical program. They were randomly divided into an intervention group and a control group of 11 students each. All continuous data of depression, anxiety, stress, CFQ, MAAS, and AAQ-II scores were normally distributed with skewness and kurtosis less than or equal to ±2, as per normality requirements [31].

For depression scores, Mauchly’s test of sphericity indicated that the assumption of sphericity was violated, χ²(2) = 7.970, p = 0.019, epsilon > 0.75, and therefore, a Huynh–Feldt correction was used. There was, however, no significant effect of time*group on the subjects, F(1.662, 33.248) = 9.898, p = 0.062. As Figure 1 suggests, the raw score trend was that there was a sustained decrease in scores for the intervention group compared with the control group, whereby the scores decreased marginally at T2 but increased at T3.

For anxiety scores, Mauchly’s test of sphericity indicated that the assumption of sphericity was not violated, χ²(2) = 1.792, p = 0.408, epsilon > 0.75. There was a significant effect of time*group on subjects, F(2, 2) = 5.109, p = 0.011. As Figure 1 suggests, the raw score trend was that there was a sustained decrease in scores for the intervention group compared with the control group, whereby the scores decreased marginally at T2 but increased at T3.

For stress, Mauchly’s test of sphericity indicated that the assumption of sphericity was violated, χ²(2) = 6.243, p = 0.044, epsilon > 0.75, and therefore, a Huynh–Feldt correction was used. There was, however, no significant effect of time*group on the subjects, F(1.756, 35.117) = 10.838, p = 0.104. As Figure 1 suggests, the raw scores showed that the intervention group had a drop in scores at T2 and then had a small increase at T3, which was still beneath the baseline scores at T1. This differed from the control group, in which there was an increase over time from T1 to T3.

For levels of mindfulness, Mauchly’s test of sphericity indicated that the assumption of sphericity had not been violated, χ²(2) = 5.073, p = 0.079, epsilon > 0.75. There was a significant effect of time*group on the subjects, F(2, 40) = 2.208, p = 0.003. As Figure 2 suggests, the mean scores increased from T1 to T3 in the intervention group, whereas in the control group, they decreased over the course of T1 to T3.

For psychological flexibility, Mauchly’s test of sphericity indicated that the assumption of sphericity was violated, χ²(2) = 10.684, p = 0.005, epsilon < 0.75, and therefore, a Greenhouse–Geisser correction was used. There was a significant effect of time*group on the subjects, F(1.398, 27.970) = 7.620, p = 0.005. As Figure 2 suggests, the scores decreased for the intervention group (indicating lower levels of psychological inflexibility) from T1 to T3. This differed from the control group, in which the scores increased from T1 to T3 (indicating higher levels of psychological inflexibility).

On measures of cognitive fusion, Mauchly’s test of sphericity indicated that the assumption of sphericity was violated, χ²(2) = 7.118, p = 0.28, epsilon > 0.75, and therefore, a Huynh–Feldt correction was used. There was a significant effect of time*group on the subjects, F(1.171, 34.125) = 10.327, p = 0.001. As Figure 2 suggests, cognitive fusion decreased for the intervention group from T1 to T3. In the control group, however, after an initial decrease from T1 to T2, it rose significantly after T3.

4. Discussion

This study answered the research question in that it showcased that ACT could improve psychological flexibility and reduce markers of psychopathology in medical students undergoing clinical-based performances. Specifically, the study demonstrated that the ACT-based intervention had beneficial effects on all three psychological process variables that were expected to respond—psychological flexibility, mindfulness, and cognitive fusion. There were also reductions in depression and stress scores; however, the effect of time*group on both was not significant. However, for the main construct that the intervention was intended to deal with, namely anxiety, there were significant score reductions for the intervention group compared with the control group, with significant time*group effects. These data thus add value to the literature, as the objective of the study was achieved in that we were able to ascertain the efficacy of an ACT-based intervention in improving performance on clinical medical examinations. The hypothesis was also confirmed in that there were three psychological process variables that demonstrated improvement; on the other hand, depression and stress demonstrated non-significant improvements, whereas anxiety demonstrated significant improvements.

From a theoretical standpoint, this study significantly boosts the existing evidence for ACT-based intervention in performance enhancement, as its sample size was larger than those of the previous single-sample studies focusing on performance enhancement in non-medical populations [10,11,12,13]. This is hence the first documented case in the literature looking at the efficacy of ACT-based intervention for medical student-related performance anxiety. In comparison with cognitive-based psychotherapies, which focus on Socratic questioning to reduce negative or irrational thoughts and hence curtail negative emotional responses, ACT approaches may be useful because they promote a more acceptance-based approach to what is essentially a natural response to a stressful situation.

In terms of its application for practice, as an online intervention, it is highly transferable and can hence be conducted with exponentially larger groups. The cohort of final-year medical students in this research study had their final examinations postponed two times, and this created a high level of anxiety due to the uncertainties of delayed employment, possible forestalled scholarships for university studies, and high financial implications for flights to hometowns because of examination cancellations. Hence, it is hoped the lessons obtained from the ACT-based intervention performance anxiety course can be translated to other anxiety-inducing situations in real life.

In comparison with previous studies, mindfulness approaches reduce anxiety amongst health profession students in general [1], with comparable efficacy for group mindfulness [2]. There have also been comparable studies that have found efficacy for mindfulness practices for academic results [3,4]. Consequently, it has been adopted widely for use in medical schools to improve psychological health and well-being, with the intended effect of reducing the student dropout rate [32]. This, however, was a unique result in that it showcased the efficacy of mindfulness specifically for the purpose of performance enhancement. Moreover, this is the first study that specifically used the ACT modality, which emphasizes committed action with value- and goal-setting on top of merely using conventional mindfulness or meditation practices.

This study has several limitations that limit the generalizability of the results. Firstly, the sample size of this study is rather small because of the low uptake rate by volunteers; a higher-powered study would ideally incorporate all 83 medical students in the same batch. However, as this was the first ACT-based performance enhancement intervention in a non-athlete group, it was felt that a pilot study would be best to redress any larger issues with the ACT-based intervention before proceeding to a larger-scale intervention. The other limitation was that we did not use students’ actual grades on clinical examinations pre and post intervention. However, this would be inherently highly subject to bias, as all final-year students were undergoing one of four different clinical postings, which have inherently differing difficulty levels and high degrees of variability in end-of-posting assessment formats. For instance, in the surgery posting, one-third of students were reliably expected to fail, whereas for the pediatrics posting, all students were predicted to receive an A or A minus grade. Hence, falsely unobjective results would be obtained if the intervention group had gone from a difficult clinical posting to a comparatively easier one, which would have artificially boosted their post-intervention grade. As a consequence, the researchers decided to use objective markers of performance anxiety, e.g., DASS scores and psychological process variables, as these were unaffected by the wide variances in posting examinations.

5. Conclusions

In conclusion, this pilot study provides limited but preliminary evidence for the use of the ACT-based intervention as part of an online intervention that may be able to alleviate performance anxiety in clinical medical students. Such ACT skills are highly transferable across the career of a medical student, as doctoring, whether as a resident, a medical officer, or a specialist trainee, inherently implies participation in multiple assessments of one’s clinical abilities, such as formalized examinations; workplace-based assessments; or by a jury of one’s own peers, subordinates, and patients. It is hoped that a larger-scale fully randomized study involving a statistically significant population of clinical medical students can be employed to further assess the veracity of the significant findings obtained so far in this smaller-scale pilot study.