Does the Brain Drain Effect Really Exist? A Meta-Analysis

Abstract

Smartphones have become an indispensable part of everyday life. Given the current debate about the use of smartphones in classrooms and schools, it seems appropriate to examine their effects on aspects of cognitive performance in more detail. Ward and colleagues not only demonstrated the negative effect of smartphones on cognitive performance but also showed that the mere presence of these devices can have this effect—this is known as the Brain Drain effect. In the present article, a meta-analytic approach was adopted in order to verify these findings. Here we show a significant overall negative effect of smartphone use and presence. In a database search we identified 22 studies with a total of 43 relevant effects that could be assigned to the categories “memory”, “attention”, and “general cognitive performance”. A subgroup analysis suggests that not all cognitive domains are equally affected by the negative effect of smartphones. The heterogeneity of the effects reinforces this finding. The nationality of the test subjects or the origin of the studies was identified as a further key variable. Our findings also indicate that the distracting effect of smartphones varies on the area studies and further research is necessary. In view of the present research results, it seems important that people in general, and especially children and adolescents in schools and classrooms, learn how to deal with the distracting potential of smartphones.

1. Introduction

Smartphones are a part of most people’s lives, and increasingly so for children and young people. The devices are also being used more and more frequently in educational institutions. According to the Bavarian State Ministry of Education and Cultural Affairs, smartphones are used for teaching purposes (e.g., quizzes, digital learning environment, online research). Permission to use the digital devices privately is also being discussed [1]. However, there are countries that have completely banned smartphones from schools after years of testing. In France, smartphone use has been banned in elementary schools and colleges since September 2018 [2]. It is debatable whether the constant presence of this kind of media does more harm than good. Studies such as that by Ward and colleagues indicate that the mere presence of smartphones has negative effects on learning performance and attention [3]. Their study named this phenomenon the Brain Drain effect and it has led to a great deal of research activity in this area, with mixed results [4]. Thus, the question arises whether the Brain Drain effect really exists. Here we try to answer this question using meta-analytical techniques and deliberately choose a narrow focus. Based on the findings of the Brain Drain study, we examined studies that report effects of smartphones (tablets, smartwatches, and the like are not included in this meta-analysis) on cognitive processes. We focused on domains of cognitive performance that could be distinguished on the basis of the primary studies found.

Effects on the functioning of working memory seem to be at the centre of the impact of smartphone use on cognitive capacities. Study results indicate that the duration of smartphone use correlates with the availability of working memory [5]. On the one hand, the partially automatic control of attentional processes helps maintain attention, but on the other hand, it can be detrimental to cognitive performance if a stimulus (e.g., the receipt of an important message) is perceived as relevant to one’s goals but is not relevant to the current task [6]. General cognitive performance includes test scores to be completed in, for example, mathematics, native language, or science. This shows why smartphones have a distracting effect even if they are not being used but are in spatial proximity during the completion of a task. Smartphones can also disrupt focused attention when the user is trying to ignore them [7]. Interfering with automatic attentional processes demands attentional resources. Therefore, performance on tasks that rely on these capacities may also decline when attention is not consciously paid to the phone, but the mere presence or absence of the smartphone has an effect [8]. In light of the fact that Ward et al. (2017) [3] also investigated these forms of distraction, studies that refer to the mere presence of the smartphone are also relevant for the present study.

The country of origin of the studies or the nationality of the subjects examined in studies seems to play a role regarding the Brain Drain effect. This finding is supported by a study in which the distracting effect of smartphones was investigated. In this study, Asian students were able to concentrate better without smartphones than European students [9]. For this reason, the region of the studies is given an attention in the present meta-analysis.

2. Materials and Methods

The meta-analysis follows the PRISMA statement. A prerequisite for inclusion in this meta-analysis was the report of a measure of the effect of smartphones (e.g., means and standard deviations) on cognitive performance. Accordingly, the focus of the search was on those studies selected that were methodologically similar to the study design of the Ward et al. (2017) [3] review and aimed to confirm, or refute, the Brain Drain effect. We searched the databases PSYNDEX, Web of Science, and ResearchGate using the search pattern “(phone OR smartphone OR cellphone OR nomophobia) AND (attention OR awareness OR distraction OR vigilance OR concentration OR memor* OR concentrat* OR perform* OR “cognitive capacity” OR “Brain Drain” OR mere)”. This resulted in 802 hits. In addition, we searched the study by Ward et al. (2017) [3] using the “cited by” function of the Google Scholar search engine, which resulted in an additional 672 hits. The study search was completed on 10 December 2022. Subsequently, we eliminated 94 duplicates. We screened the remaining 1350 studies and narrowed them down to 121 after reading the abstracts. Only studies that employed quantitative-empirical approaches, investigating the impact of smartphones on cognitive performance and thereby establishing a direct connection to the Brain Drain effect, were included. Studies that had a qualitative-empirical design, lacked a connection to the Brain Drain study, or focused on other digital devices such as tablets and smartwatches were excluded. Finally, a full text analysis led to the exclusion of a further 109 studies due to a lack of fit between the research question and the research design. Finally, 22 studies met all search criteria (Figure 1).

Some studies have reported effects of smartphones or their presence that were mediated by the anxiety disorder nomophobia [9]. These studies also provide findings regarding the effects of smartphones on cognitive performance. Studies on smartphone-related addiction [10], the effects of electromagnetic fields on the human brain and its functioning [11], or effects on social skills and emotions [12] or the like, were not included. Some studies have reported multiple effects of smartphones or their presence on various cognitive functions (e.g., attention, fluid intelligence). Consequently, the number of effects (n = 43) exceeds the number of selected studies (n = 22) (Appendix A).

We performed all calculations using the Open Meta Analyst program (http://www.cebm.brown.edu/openmeta/#, accessed on 8 September 2023). Information on effect sizes and standard deviations was taken from the primary studies listed in Appendix A and entered into the calculation program.

3. Results

Hedge’s g was selected as the effect size measure. The sample sizes across studies vary and, at times, are small. By utilizing the small sample bias correction, it becomes feasible to address potential biases in small samples, consequently yielding a more consistent and precise estimation of effect sizes. We calculated the pooled effect size using a continuous random effects model applying the DerSimonian and Laird method [13]. The pooled effect across all studies was g = −0.14, with a 95% confidence interval (CI) of −0.24 to −0.03, at p < 0.05 (Figure 2). The included effects show a significant heterogeneity (Q (df = 42) = 110.06, I² = 61.84, p < 0.001).

An examination of the effect sizes as a function of the underlying sample size reveals that the largest and smallest effects in each case were found among relatively small study groups. At the same time, the majority of positive effects were detected in comparatively small samples, while negative effects were also reported in samples that tended to be larger. The funnel plot does not indicate a possible bias or possible outliers, as we also found and took into account studies in which low effect sizes were reported for a small number of cases (Figure 3). In addition to the visual inspection of the funnel plot, which does not display any conspicuous indications of systematic bias, the False Discovery Rate (FDR) was controlled using the method proposed by Benjamini and Hochberg. Applying the FDR correction to the employed statistical tests helps mitigate the risk of random discoveries in the examination of multiple hypotheses [32,33].

The studies included in the present meta-analysis examined various aspects of cognitive performance. In terms of content, it was possible to distinguish the areas of “memory”, “attention”, and “general cognitive performance” (Appendix A). In the course of subgroup analyses, we examined these factors in more detail. Only the pooled effect of “memory” was significant (g = −0.23, 95% CI = −0.36; −0.10, p < 0.001). The “attention” effects (g = −0.07, 95% CI = −0.21; 0.06, p = 0.29) and “general cognitive performance” effects (g = 0.10, 95% CI = −0.52; 0.72, p = 0.76) were not. Despite the distinction of subgroups, significant heterogeneity was evident when we examined all three factors, just as it was when we calculated the overall effect (

Table 1. Heterogeneity of the factors.

Factor	Q (df)	I2
Memory	36.14 ** (17)	52.96
Attention	32.30 * (18)	44.29
General cognitive performance	37.37 ** (5)	86.62

Notes. Q = Cochran’s Q. df = degrees of freedom, I² = according to DerSimonian and Laird. * = p < 0.05. ** = p < 0.01.

).

To determine possible causes for heterogeneity, we conducted another subgroup analysis in which the effects were grouped on the basis of the region from which the study originated. This approach is based on the findings of Mahsud et al. (2021), in which Asian subjects were influenced by smartphones to a different extent than European subjects [9]. A significant pooled effect could only be determined for the studies from the Asian region (g = −0.39, 95% CI = −0.57; −0.21, p < 0.001). The effects determined for the European (g = −0.20, 95% CI = −0.46; 0.06, p = 0.12) and North American (g = −0.03, 95% CI = −0.15; 0.09, p = 0.60) regions did not reach the significance threshold. The effects from the Asian region showed no signs of heterogeneity. In the case of North America and Europe, the heterogeneity was significant (

Table 2. Heterogeneity of effects depending on origin.

Factor	Q (df)	I2
North America	48.96 ** (24)	50.98
Europe	30.73 ** (9)	70.71
Asia	10.08 (5)	30.58

Notes. Q = Cochran’s Q. df = degrees of freedom, I² = according to DerSimonian and Laird. ** = p < 0.01.

).

4. Discussion

The present meta-analysis essentially confirms the Brain Drain effect, which is negative regarding cognitive performance. The results vary with respect to the different domains that were examined here. They are strongest for memory and lower for attention and general cognitive performance. Some studies showed that both smartphone use and the presence of the device result in negative effects on aspects of cognitive performance. However, not all studies reached this conclusion [16]. Consequently, heterogeneous findings may also emerge with respect to smartphone use and its presence. Moreover, variables such as smartphone dependence, which are discussed in some primary studies, may have a crucial influence on the distracting effect of these devices [9]. Moreover, this current meta-analysis encompassed studies employing various instruments to examine diverse cognitive performance indicators. While we took into account a range of results, it is important to acknowledge the presence of some variability in the observed effects. Due to the number of available effects, survey instruments, and possible implications, it seems obvious that subgroups that are more homogeneous still show signs of heterogeneity.

The present meta-analysis includes studies and effects from different countries. As the nationality of the subjects may cause heterogeneity, we performed a subgroup analysis that took into account the regions in which the respective surveys were conducted. It showed that the overall negative effect of smartphones was more pronounced and significant in the Asian region. Unfortunately, we did not find a satisfactory explanation for this result. It seems obvious that more research is needed both to provide further evidence for the Brain Drain effect and to further explain existing differences in terms of domain and region. Due to the small number of primary studies, further calculations, for example based on different measurement instruments or subjects, was not possible at this stage, but may be of interest in the future.

Certain limitations need to be addressed: As the inclusion criteria for the selection of studies based on the Brain Drain study, a total of 22 studies were included in the review, making an expansion of the data pool desirable in the coming years. Another limiting factor was the nature of outcome reporting in the various studies. Means and standard errors were not always available. In addition, the overall effect and the effects in some subgroups showed significant heterogeneity. In dealing with this heterogeneity, we applied the steps proposed by Petitti [34]. It seems reasonable to assume that the heterogeneity of the effects found can be attributed in part to the variance in the specific subjects included in the primary studies. Various aspects of cognitive performance were considered in the studies, for example, the effects of smartphone presence and smartphone use in different skill domains, such as mathematics, language comprehension, and spelling [14]. Another study focused on the “stroop effect” and the “switch-cost effect” [21]. For this reason, we calculated separate meta-analyses for “memory”, “attention”, and “general cognitive performance”. We again detected significant heterogeneity in each of these subgroup analyses. It is possible that the variance that continues to exist is due to the use of different measurement instruments. For example, some study designs used “Raven’s Standard Progressive Matrices Test” or the “Spanboard Test” [16], while others used a “trail making test” or a test to capture “digit cancellations” [30]. An analysis of possible moderators suggests that the nationality of the individuals studied contributes to this variance. The investigation of further possible influencing factors was omitted, because the small number of studies did not allow for the extraction of any further, theoretically significant moderators. Furthermore, not all of the studies reported potentially important personal characteristics, such as age, social origin, educational background, gender, and degree of smartphone dependence. These factors could be crucial when it comes to the distracting effect of smartphones. We recommend that further studies should address the identification and investigation of possible moderators.

Despite the heterogeneous findings, the results of the meta-analysis indicate a negative effect of smartphone use or smartphone presence on various aspects of cognitive performance. Therefore, we confirmed the tendencies found in the Brain Drain study. Our findings indicate a distracting effect of smartphones, even if they are not actively used. For this reason, two aspects of media education are important from an educational point of view [35]. On the one hand, it is necessary to protect children in particular from the use of smartphones that is not controlled in terms of content and time. To this end, it will also be necessary to discuss bans, especially in schools, as recently outlined by UNESCO in its Global Education Monitoring Report (2023) [36]. On the other hand, we need school concepts that introduce young people to the use of smartphones, with a high degree of self-reflection at the centre. This requires not only knowledge of technical aspects, but also, and above all, knowledge of the distraction potential of smartphones and their influence on memory, attention, and general cognitive performance. A clear distinction should be made between the smartphone as a work tool in the classroom and the use of the device for private purposes. The available research findings show that smartphones should not be used without reflection. Due to their distracting potential, their use should always be accompanied by a pedagogical-didactic added value, which should always be examined. We cannot support calls for smartphones to be widely used in schools with little regulation. Based on the available research findings, it seems advisable that smartphones should not even be near learners during periods of learning.

Nevertheless, smartphones are an important part of our living environment. Consequently, it is necessary to empower people, especially children and young people, to take advantage of smartphone opportunities while avoiding the dangers. The knowledge of the Brain Drain effect is essential for this.