The Exploration of Combining Hologram-like Images and Pedagogical Agent Gesturing

Abstract

The split-attention principle suggests that separating onscreen information sources can overburden working memory and impede learning. While research has traditionally focused on the separation of images and text, relatively little is known about the impact of multiple simultaneous visual inputs. This study examined the split-attention principle in a multimedia environment featuring a pedagogical agent performing gestures, with hologram-like images either integrated centrally with the agent or spatially separated. A within-subjects design (N = 80) investigated the impact on satisfaction, cognitive load, and cued recall. The quantitative findings revealed no significant differences between the two spatial conditions. Preliminary qualitative insights from a limited sample of six individual interviews suggested that some participants may employ strategies to simplify complex designs and manage perceived cognitive load. Based on these limited qualitative observations, this research tentatively proposes the “pruning principle”, a metacognitive strategy where learners actively “prune” extraneous information to optimize cognitive resources. These findings underscore the importance of considering individual differences and metacognitive strategies in multimedia design.

1. Introduction

Designing multimedia environments presents significant challenges due to the complexities involved in information presentation. To address cognitive load, the Cognitive Theory of Multimedia Learning (CTML) has introduced fifteen principles over the past two decades [1]. For example, the multimedia principle states that combining words and pictures in presentations enhances learning more effectively than using words alone. Similarly, the signaling principle highlights the importance of visual cues, such as arrows or highlighting, to guide attention toward key information. While many CTML principles are well-established [2], certain nuances warrant further exploration to ensure their applicability across diverse multimedia designs. For instance, the redundancy principle originally suggested that presenting identical information through written text and speech alongside pictures could impede learning [3]. However, subsequent research by Mayer and Johnson [4] demonstrated that displaying key terms, rather than verbatim text, alongside narration can enhance learning. This finding underscores the importance of nuanced interpretations and adaptations of CTML principles.

Although CTML offers a robust framework for designing multimedia environments, additional design principles complement and extend its guidelines, often with subtle but important differences, particularly regarding cognitive load. For example, CTML’s spatial contiguity principle closely parallels the split-attention principle [5]. Both principles advocate for reducing extraneous cognitive load by ensuring that related information is presented in close proximity. The spatial contiguity principle states that that separating text from related images increases cognitive load, as it forces users to mentally integrate the two elements in working memory [6]. A classic example of the spatial contiguity principle is a map of Europe with the country names listed below the map rather than integrated within it. In such cases, users unfamiliar with the countries must repeatedly shift their focus, increasing cognitive demand on working memory. In contrast, the split-attention principle extends beyond spatial considerations to include factors such as mismatched timing between animations and narration or presenting information through conflicting modes (Ayres & Sweller, 2014). Thus, while CTML addresses certain temporal and modality aspects through its temporal contiguity and modality principles, the split-attention principle provides a broader framework for managing cognitive load across diverse information formats. This is particularly relevant in multimedia environments that incorporate pedagogical agents, where multiple channels of information must be effectively integrated.

While the spatial contiguity and split-attention principles have a significant overlap, the split-attention principle provides deeper insight into multimedia environments with multiple graphical elements, such as pedagogical agents combined with visual aids. Although Mayer [6] argued that a speaker’s image does not significantly enhance learning, the embodiment of pedagogical agents introduces an important nuance, as their design and integration can influence learner engagement and cognitive processing. Mayer and DaPra [7] found that highly embodied agents—featuring lip synchronization, eye gaze, facial expressions, gestures, and body movements—significantly enhance social perception and learning outcomes compared to minimally embodied agents with only lip synchronization. However, incorporating highly embodied agents can increase the complexity of the multimedia environment, potentially contributing to split-attention effects that may hinder learning if not carefully managed. Learners must divide their attention between the agent’s nonverbal cues, speech, and other visual elements, potentially increasing the processing demand on working memory. While research has examined split-attention effects between text and images, there remains a gap in understanding how multiple simultaneous visual inputs—such as pedagogical agent gestures combined with visual aids—affect satisfaction, cognitive processing, learning outcomes, and viewing strategies. This gap is particularly significant for understanding how learners allocate attention and integrate complex visual information during multimedia presentations.

This study aims to investigate whether split-attention effects arise in multimedia environments when multiple simultaneous visual inputs are presented simultaneously. The design features a pedagogical agent performing gestures at double frequency, with static images either positioned to the side of the agent or integrated centrally within the display. This within-subjects study investigates participants’ satisfaction, cognitive load, learning outcomes, and viewing strategies when engaging with declarative information on the topic of Australia. Furthermore, individual interviews were conducted to explore how learners allocate attention to and integrate complex visual information during multimedia presentations, offering insights into how split attention may have influenced their interactions with the visual elements.

2. Literature Review

2.1. Split-Attention Principle in Multimedia Learning

The split-attention principle suggests that the spatial separation of multimedia information sources, such as images and text, requires users to mentally integrate these elements. This process can divert essential cognitive resources, thereby increasing cognitive load [8]. Grounded in cognitive load theory (CLT), which acknowledges the limited capacity of working memory, the split-attention principle emphasizes that instructional materials should be designed to minimize extraneous cognitive load—the mental effort required to process information that is not directly relevant to the learning task [9,10,11]. Dividing attention between separate sources of information can hinder efficient processing and comprehension, particularly in multimedia learning environments.

Similarly, the split-attention principle builds upon the concept of dual-channel processing, which suggests that working memory operates through separate channels for processing auditory and visual information [12]. When learners encounter spatially separated information, they are required to continuously shift their visual attention, dividing their focus and potentially hindering learning and comprehension. This process risks exceeding the limited capacity of working memory. Such a division of attention and its impact on working memory aligns with the active processing assumption, which emphasizes the importance of meaningful engagement with material for effective learning [1]. The active processing assumption suggests that there is an active cognitive engagement, where learners select relevant information, organize it into coherent representations that can be assessed by working memory, and integrate these representations into existing schemas within long-term memory. This type of processing allows learners to experience germane cognitive load, which supports learning by fostering meaningful connections. In contrast, overloading one or both of the dual channels can lead to extraneous cognitive load, which hinders learning by consuming cognitive resources unnecessarily [13]. While the active processing assumption underscores the importance of engaging with relevant information, Schroeder and Cenkci’s [14] systematic review raises questions about the role of extraneous cognitive load. Their findings suggest that learners may employ preemptive strategies, such as selective attention, to manage the cognitive demands placed on working memory, potentially mitigating the negative effects of extraneous cognitive load. The authors suggest that integrating graphics and text is an effective design strategy, as it reduces the cognitive burden on working memory and minimizes the need for selective attention, allowing learners to focus more efficiently on the learning task.

Theories and the majority of the research on split attention have primarily focused on the integration of text and images. However, the concept of split attention extends beyond this scope. It also applies to multiple visuals, such as multiple visuals, such as multiple static images presented simultaneously [15], images paired with animations [16,17], multiple animations displayed concurrently [18], or combinations of images and a gesturing pedagogical agent [19]. The findings from these studies suggest a potentially complex interaction between visual information and the spatial design of information. For example, some research [15] indicates that spatial distance alone does not necessarily hinder the integration of information. In contrast, other studies suggest that spatial separation can increase cognitive load and alter viewing strategies, potentially impacting learning outcomes [17]. Individualized strategies for managing split-attention can vary widely, ranging from ignoring one source of information entirely [17] to actively altering focus between multiple sources [19]. These strategies are often influenced by the learner’s perception of each source’s usefulness and its potential contribution to learning. Furthermore, individual cognitive abilities, such as mental rotation, can affect whether split attention negatively impacts performance [18]. This indicates that processing spatially separated information requires not only the integration of different content but also the effective management of cognitive resources and the application of metacognitive strategies.

2.2. Gestures

Human-to-human communication often involves multiple strategies to convey information. This can occur verbally, through spoken or written language, or nonverbally, through human displays such as facial expressions, tone of voice, body position, and gestures, which are interpreted by the listener [20]. Although research indicates that listeners typically spend more time focusing on the speaker’s face (84.9%) than on gestures (2.1%), gestures still play an important role in communication. It is suggested that attention to gestures occurs unconsciously, enabling the concurrent processing of verbal and gestural information [21]. Similarly, Gullberg and Holmqvist [22] found that listeners are more likely to focus on gestures that extend into the peripheral vision, include holds (pauses in movement), or are accompanied by the speaker’s direct gaze toward the gesture. This suggests that gestures may be processed unconsciously as part of the visual background. This is consistent with Kendon’s [23] argument that gestures aid comprehension, even when listeners do not consciously focus on them.

However, there is nuance in how attention is directed, or not directed, toward gestures. For example, listeners with weaker verbal skills, such as children or second-language users, tend to benefit significantly from gestures during communication, as these provide additional contextual and semantic cues that support comprehension [24]. In the context of foreign language acquisition, Vandergrift [25] observed that gestures help learners conceptualize speech and support working memory. Similarly, Church et al. [26] demonstrated that both English and Spanish speakers performed better in math learning tasks when watching videos that combined speech and gestures, compared to videos with speech alone. The Spanish-speaking students had no prior knowledge of the English language, leading the authors to suggest that the gestures played a crucial role in helping these students link the visual information to their own language, thereby enhancing their comprehension of the concepts. Beyond aiding comprehension, gestures can be viewed as a form of language production. Neuroscientific evidence from fMRI and PET scans indicates that Broca’s area, the brain’s primary language center, is actively engaged in processing meaningful gestures. This suggests that gestures and speech function are integrated modalities of communication [27,28]. However, neither pure movement nor meaningless gestures activate Broca’s area [29,30] suggesting that meaningful gestures convey information that transcends basic motor skills or visual stimuli. Since gestures occur in space, they are classified as spatial movements that engage spatial working memory [31,32]. Alibali [33] suggests that gestures provide benefits not only for their producers but also for their observers. For the producer, gestures offer the ability to emphasize spatial information, support spatial descriptions, and integrate spatial details into verbal communication. For the viewer, spatial gestures provide visual reinforcement for complex topics involving spatial relationships or convey unspoken spatial information essential for understanding. Thus, gestures can be viewed as sophisticated communication strategies that play a crucial role in enhancing both the transmission and comprehension of information, especially when spatial concepts are involved.

2.3. Gesturing Pedagogical Agents

Early researchers theorized that designing pedagogical agents with gestures could not only give the participants the perception that the agent possessed a persona but also enhance the learning process [34]. Lester et al. [35] proposed that pedagogical agents should be designed with deictic believability, meaning they should incorporate speech, movement, and pointing gestures to emulate human behavior. Similarly, social agency theory suggests that designing agents with verbal (e.g., human voice) and visual (e.g., facial expressions and gestures) social cues can activate the user’s social schema, akin to those used in human-to-human communication [36,37]. Mayer and Dapra [7] expanded upon social agency theory by proposing the embodiment principle, which suggests that pedagogical agents should be designed to exhibit common human behaviors, such as lip synchronization, eye gaze, facial expressions, body sway, and gestures. A number of meta-analyses and review studies have provided empirical evidence that embodying an agent positively impacts both learning outcomes and the perception of the agent’s persona [38,39,40]. A meta-analysis by Davis [41] examining the impact of agent gestures found that gestures significantly enhanced retention and transfer measures, as well as improving the perception of the agent’s persona. However, most studies included in this analysis primarily investigated agents using deictic gestures. Craig et al. [42] suggested that while deictic gestures are effective in directing the learner’s spatial attention to important onscreen information, researchers should investigate other gesture types that might convey information valuable for learning. Since then, only a limited number of studies have examined the use of all gesture types (iconic, metaphoric, deictic, and beat) and the frequency rates at which gestures are performed in relation to learning outcomes. The findings from these studies suggest that increasing gesture frequency (doubling the average rate) enhances the recall of both procedural knowledge [19,43] and declarative knowledge [44]. However, these studies have exclusively focused on advanced non-native speakers, who may derive greater benefits from the additional input compared to other learner groups. More research is needed with native-speaking populations to determine whether full gesturing and increased gesture frequency provide the same benefits observed with non-native speakers.

2.4. Research Purpose

Effective multimedia learning environments require careful considerations of how users’ view and perceive visual inputs that convey information. This research employs a counterbalanced repeated measures design with two distinct spatial conditions: (1) gestures and hologram-like images centrally integrated and (2) gestures and hologram-like images spatially separated. Individual interviews were conducted to gather insights into participants’ perceptions and viewing patterns. This mixed-methods design investigates the concept of split attention, examining how the integration or separation of visual inputs impacts satisfaction, cognitive load, and learning outcomes.

This study addresses the following research questions:

RQ1: How do gesturing agents and image placement influence satisfaction in a multimedia learning environment?

RQ2: How does image placement and agent gesturing affect the type of cognitive load experienced by participants?

RQ3: To what extent does cued recall vary based on the use of gesturing agents and the spatial positioning of images (middle vs. side)?

RQ4: How do participants’ perceptions of spatially integrated and separated learning visuals influence their satisfaction, perceived difficulty, and viewing strategies?

3. Methods

3.1. Research Design and Participants

The research intervention employed a repeated measures design with two different scripts about Australia. Participants viewed hologram-like images presented in two spatial configurations: positioned to the side of the gesturing agent and centrally aligned with the gesturing agent. The scripts were presented in the same order, but the placement of the hologram-like images was alternated to control for order effects. Participants were randomly assigned to one of two conditions: S1M2 (side images for video one, middle images for video two) or M1S2 (middle images for video one, side images for video two). For the purposes of this exploratory study, the middle hologram-like image configuration serves as the experimental condition. Participants were given both oral and written informed consent prior to completing demographic information. For the order of the intervention and data collection, please see

Table 1. Order of the research intervention and data collection.

1. Prior knowledge test

2. First video with side or middle image placement

3. Satisfaction survey 4. Cognitive load survey 5. Cued recall assessment

6. Second video with the remaining image placement

7. Satisfaction survey 8. Cognitive load survey 9. Cued recall assessment

.

An a priori power analysis was conducted using G*Power 3.1 to determine the required sample size for a repeated measures ANOVA. The analysis was based on an alpha level of 0.05, a desired power of 0.80, and a medium effect size (f = 0.25). The results indicated that a minimum of 34 participants was needed to achieve sufficient power to detect meaningful within-subject effects. The participants in this study were 80 advanced foreign language learners at a foreign language university in Seoul, South Korea. The participants came from diverse native language backgrounds, with 52 identifying their native language as Korean, 8 identifying their native language as Chinese, and 12 identifying other native languages such as French (4), Russian (3), Spanish (3), English (2), in addition to single speakers of Romanian, Portuguese, Indonesian, Ukrainian, Uzbek, German, Afrikaans, and Slovenian. To ensure the results were not skewed, the data from the two native English speakers were analyzed separately to assess their potential influence on the study’s findings. No significant impact was observed across any of the measures, so their data were retained. The average age of the participants was 22.36 years (SD = 2.43), with 56 participants identifying as female and 24 as male. A majority of the students were seniors (27) or juniors (35), with the remaining participants being freshmen (13) and sophomores (5). The participants were either majoring or taking a double major in an English language concentration, or they were exchange students taking English-only courses offered by the university.

3.2. Instructional Content

The content for this study was based on Discovering Australia [45,46] which provides general information (declarative knowledge) about Australia through 22 individual text snippets, each ranging from 74 to 80 words in length. Two of the snippets serve as introductions, while the remaining 20 snippets contain two questions to assess the participant’s understanding of the content. This research utilized one introduction and ten informational topics, which covered the following subjects: Northern coastal wetlands, Perth, sea life in Australian waters, wild rabbits, Wolf Creek Crater, Aboriginal people, MacDonnell Ranges, Great Sandy Desert, Timor Sea, and morning glory clouds. The content has a Flesch–Kincaid grade level of 10.7 and a Flesch reading ease score of 46.7, indicating that it corresponds to college-level reading in English.

3.3. Multimedia Environment and Gesture Design

The agent was created using iClone 8.3 ^TM with a black background to enhance contrast for the hologram-like images, which were subsequently integrated using Camtasia 2023.3.13. To address the complexity of the images used in the previous studies, the hologram-like images used in this research were sourced from royalty-free websites and modified using an online PNG image converter and editor. The PNG enabled the removal of color, adjustments to line width, and the addition of glowing of lines to produce a hologram-like effect. The learning environment with hologram-like images and location can be viewed in Figure 1.

The pedagogical agent’s gestures were created using the timeline of iClone 8.33. The gesture frequency was established by doubling the average number of gestures observed in human-to-human communication involving declarative information [47], where speakers typically produced 6.51 representational gestures (iconic, metaphoric, deictic) and 1.32 nonrepresentational gestures (beat) per 100 words. This approach is consistent with previous agent gesture research, which found that increasing gesture frequency (doubling) significantly improved cued recall compared to other experimental conditions [44]. Accordingly, this study employed only an enhanced gesture frequency, with the agent performing 13.44 representational gestures and 2.97 nonrepresentational gestures per 100 words.

3.4. Instruments

Demographic data were collected from participants after they provided informed consent and before completing a prior knowledge test. Participants were asked to provide their international age (to account for Korea’s distinct age counting system), gender, university major, year classification, and native language. Participants were also asked if they had lived or had extended stays in Australia and, if so, for how long. This information was collected to assess whether any prolonged experiences could potentially influence the data. No participants reported having prior experience of living in or staying in Australia.

3.4.1. Prior Knowledge Test

Previous research using this instructional content [45,48] did not include a prior knowledge test, as it was assumed that American participants would not possess substantial knowledge about Australia. Previous research involving participants in Korea employed a 5-point Likert scale to allow participants to self-assess their knowledge of Australian history, geography, and wildlife [44]. For this experiment, prior knowledge was assessed using 10 generalized questions covering four broad areas—geography, ecology, culture and society, and the environment—related to the instructional content. Questions included “What is the most dangerous sea creature found in Australian waters?” and “What is the name of the period between November and December in Australia?” Analysis revealed no significant differences in prior knowledge between the two conditions.

3.4.2. Satisfaction

The satisfaction survey was adapted from Ritzhaupt et al’s. [48] time compression study, which measured satisfaction with the presentation of information. This survey consisted of 15 questions, each scored on a 5-point Likert scale. The first ten questions utilized bipolar adjective pairs, ranging from negative (1) to positive (5), and the final five questions were rated on a scale from strongly disagree (1) to strongly agree (5). This survey demonstrated excellent internal consistency, with a Cronbach’s alpha of α = 0.95.

3.4.3. Cognitive Load

Cognitive load was measured using ten questions based on the framework by Leppink et al. [49]. The questions addressed intrinsic cognitive load (3 items), extraneous cognitive load (3 items), and germane cognitive load (4 items). The questions were rated on a 9-point Likert scale, ranging from extremely disagree (1) to extremely agree (9). Intrinsic cognitive load items were negatively worded, meaning higher scores indicated greater perceived inherent difficulty. In the original survey, extraneous cognitive load items were negatively worded. However, for this study, the questions were rephrased positively to reduce potential comprehension difficulties for participants. As a result, lower scores indicate higher extraneous cognitive load. Conversely, higher scores on germane cognitive load items reflect an increase in beneficial cognitive load, which supports learning. The Cronbach’s alpha values for intrinsic cognitive load (α = 0.87), extraneous cognitive load (α = 0.83), and germane cognitive load (α = 0.85) were all considered good, indicating reliable internal consistency for each measure.

3.4.4. Cued Recall

Learning was assessed through two cued recall questions for each of the ten topics. The questions were identical to those used in previous research [44,45,48]. The open-ended responses were independently scored by two reviewers, and any scoring discrepancies were resolved through discussion. The weighted Cohen’s Kappa interrater reliability was calculated at k = 0.98, indicating almost perfect agreement.

3.5. Interviews

Interviews were conducted to explore participants’ perceptions and viewing patterns regarding the placement of hologram-like images in the center and at the sides of the gesturing agent. At the conclusion of the study, participants were invited to volunteer for a 20 min one-on-one interview with a Korean researcher, fluent in both English and Korean, by providing their email address. Participants who completed the interview were compensated with a $10 coffee gift card.

The interviews focused on questions related to satisfaction, cognitive load (perceived difficulty), spatial placement preferences, and viewing strategies. To facilitate recall, screenshots of the two conditions were shown during the interviews. The interviews were conducted within one week of participants’ exposure to the research intervention. While ten participants initially expressed interest in being interviewed, only six completed interviews. The other four participants did not respond to either the initial or follow-up interview email requests.

A five-phase thematic analysis [50] was conducted to examine the interview responses. First, all the Zoom interviews were recorded and transcribed for a thorough review. Second, preliminary codes and sub-codes were generated from the transcripts. Third, specific coding themes were identified within broader thematic categories. Fourth, the themes were refined by analyzing similarities and differences in the participant experiences. Finally, the qualitative findings were integrated with the quantitative results to generate the final report.

4. Results

A repeated measures ANOVA was conducted to analyze the effect of image placement (S1M2: side images first, middle images second; M1S2: middle images first, side images second) on each measured outcome. Additional tests (t-tests) were conducted to compare the image conditions (S1 vs. M1; S2 vs. M2) for each video and to assess potential order effects (S1 vs. S2; M1 vs. M2). For this exploratory study, the middle hologram-like image position served as the experimental condition. Please see

Table 2. Group means and standard deviations.

		Cognitive Load
Group	Satisfaction	IL	EL	GL	Cued Recall
S1	45.65 (12.39)	12.67 (5.88)	17.15 (5.59)	18.52 (7.47	1.25 (1.74)
S2	45.88 (12.78)	14.20 (4.78)	16.75 (4.88)	18.70 (6.47)	2.18 (2.36)
M1	48.80 (11.42)	11.35 (4.50)	17.83 (4.77)	19.95 (7.16)	1.60 (1.87)
M2	43.38 (11.28)	14.95 (5.28)	15.70 (3.66)	17.73 (7.61)	1.52 (2.05)

for the satisfaction, cognitive load, and cued recall means and standard deviations.

4.1. Satisfaction

A repeated measures ANOVA revealed a significant difference between the two conditions (F(1,78) = 5.192, p = 0.025). However, Bonferroni post hoc tests revealed no significant pairwise differences (p < 0.05).

Between-subjects analysis for the side and middle hologram-like images based on the video order (S1 vs. M1; S2 vs. M2) revealed no significant differences (p < 0.05).

Between-subjects order effects analysis (S1 vs. S2; M1 vs. M2) found no significant order effects for the side hologram-like images. However, there were significant order effects for the middle hologram-like images. Participants reported significantly lower satisfaction with the M2 video compared to the M1 video, with a small effect size (p = 0.036, d = 0.48). See Figure 2 for the satisfaction descriptive plots.

4.2. Cognitive Load

4.2.1. Intrinsic Cognitive Load

A repeated measures ANOVA indicated that there was a significant difference between the two conditions (F(1,78) = 21.759, p < 0.001). Bonferroni post hoc tests suggested that increased intrinsic cognitive load may be attributed to order effects. In the M1S2 group, S2 scored significantly higher on intrinsic cognitive load than M1, with a medium effect size (p = 0.003, d = 0.55). Similarly, in the S1M2 group, M2 scored significantly higher on intrinsic cognitive load than S1, with a small effect size (p = 0.027, d = 0.42).

Additionally, the between-subjects analysis indicated an order effect for the middle hologram-like images. The M2 video was perceived as significantly increasing intrinsic cognitive load compared to the M1 video, with a medium effect size (p = 0.013, d = 0.70).

Between-subjects analysis comparing side and middle hologram-like images based on video order (S1 vs. M1; S2 vs. M2) revealed no significant differences (p < 0.05).

4.2.2. Extraneous Cognitive Load

Extraneous cognitive load was not reverse-scored, meaning that lower scores indicate higher extraneous cognitive load. A repeated measures ANOVA indicated that there was a significant difference between the two conditions at F(1,78) = 7.152, p = 0.009. However, Bonferroni post hoc tests revealed no significant pairwise differences (p < 0.05).

Between-subjects analysis comparing the side and middle hologram-like images based on video order (S1 vs. M1; S2 vs. M2) revealed no significant differences (p < 0.05).

Between-subjects order effects analysis (S1 vs. S2; M1 vs. M2) revealed no significant order effects for the side hologram-like images. However, significant order effects were found for the middle hologram-like images. Participants reported significantly higher extraneous cognitive load in the M2 video compared to the M1 video, with a medium effect size (p = 0.028, d = 0.50).

4.2.3. Germane Cognitive Load

A repeated measures ANOVA indicated that there was no significant difference between the two conditions (F(1,78) = 2.181, p = 0.144). Between-subjects analysis also found no order effects for any of the image conditions (p < 0.05). See Figure 3 for the cognitive load descriptive plots.

4.3. Cued Recall

A repeated measures ANOVA revealed a significant difference between the two conditions (F(1,78) = 5.478, p < 0.022). However, Bonferroni post hoc tests revealed no significant pairwise differences (p < 0.05). Assumption checks for the side hologram-like image data (S1 vs. S2) indicated a violation of Levene’s test of equality of variances. A between-subjects Mann–Whitney U test indicated that the higher score of the S2 video was approaching significance against the S1 video (U = 620.50, p = 0.072, d = 0.224). No other pairwise comparisons were significant. See Figure 4 for the cued recall descriptive plots.

4.4. Interviews

4.4.1. Data Analysis

This study utilized thematic analysis to systematically analyze the qualitative data. Thematic analysis is a method used to identify, analyze, and report patterns (themes) within the data [51]. It offers a structured approach for organizing and describing the dataset in detail. By using thematic analysis, the study was able to gain a comprehensive understanding of the underlying patterns and meanings in the qualitative data, facilitating a robust interpretation of the findings [52]. The data were first transcribed verbatim and thoroughly reviewed to ensure familiarity with the content. Open coding was then performed to generate initial codes by identifying key features within the data. These codes were then organized into potential themes based on their relationships and relevance to the research questions. The themes were systematically reviewed and refined to ensure they accurately represented the data and captured the nuances of participants’ experiences. This process involved iterative reading and re-reading of the data, accompanied by collaborative discussions with co-researchers to reach consensus on the thematic structure. The final themes were carefully defined and named to encapsulate the core essence of the data, creating a coherent narrative that effectively addresses the study’s objectives.

4.4.2. Interview Findings

Participants were asked to watch two videos, each featuring a speaker accompanied by hologram-like images. In one set of videos, the holograms were placed in front of the speaker, while in the other, they were positioned to the side. The identified themes were categorized as satisfaction and viewing strategy. The satisfaction category encompassed cognitive load, as the participants often associated their satisfaction with the condition that imposed less cognitive load. Similarly, cognitive load was also identified as a key influence on participants’ viewing strategies. See

Table 3. Interview participants.

	Gender	Major	Grade	Condition	Preferred Placement
1	Female	International Relations and English Literature	Senior	S1M2	Middle
2	Male	ELLT	Senior	M1S2	Side
3	Female	EICC	Sophomore	M1S2	Side
4	Female	EICC	Junior	M1S2	Middle
5	Male	ELLT	Senior	S1M2	Middle
6	Female	EICC	Sophomore	S1M2	Side

for information on the interview participants. Please see Appendix A for the full quotes by the participants.

4.4.3. Satisfaction

The preferences for the placement of the hologram-like images were evenly divided, with three participants preferring the side placement and three preferring the middle. Among those who preferred the images at the side, the middle placement of the image in front of the gesturing agent overwhelmed them visually. Participants two and six preferred side placements because the overlapping hologram-like images and gestures were distracting, making it difficult to focus on either the images or the gestures.

However, the other participants found the middle placement more satisfying. They mentioned that the gestures played a significant role in their understanding, and having the hologram-images centrally positioned enhanced their learning experience. Participant one felt having the images and gestures in one central location allowed her to attend to all the information because both elements could be seen simultaneously. Participant five suggested that the central location was preferred because the gestures were important for learning, and looking towards the side meant he could potentially miss the gestures.

The level of satisfaction with the placement of the hologram-like images was influenced by the difficulty each design presented. Participants who preferred the side placement reported being distracted by the overlapping visual input in the middle position, which they found overwhelming. Conversely, those who preferred the middle placement emphasized the importance of observing the gestures during the presentation. For them, the side placement was more challenging, because it diverted their attention away from the gestures.

4.4.4. Viewing Strategy

Side Viewing Strategy

Although the students were evenly divided in their satisfaction with the hologram-like images in the middle or to the side, their viewing strategy was notably similar when the images were positioned to the side. All the participants, except one, reported briefly looking at the image first before returning their focus to the agent.

However, within the strategy of switching between the hologram-like images and the gesturing agent, there were strategies for directing attention. Participant one (middle viewing preference) would try to memorize the image and then direct attention back to the gesturing agent. Participant five (middle viewing preference) indicated that he would briefly look at the hologram-like images and then view the agent like he views humans—by watching the agent’s mouth and hands. Conversely, participant three (side viewing preference) mentioned that she would switch back and forth between the images and the agent, but that the agent was only briefly viewed and her attention was mostly focused on the images because the agent was something extra.

Regardless of their satisfaction, all the participants, except one, employed a similar strategy when the images were at the side. Most preferred to glance briefly at the hologram-like images before shifting their attention back to the agent. In contrast, participant three prioritized the images.

Middle Viewing Strategy

The participants who preferred the middle image placement reported using a similar viewing strategy. Since the hologram-like images and gestures were in a central location, it was easy for the participants to direct their attention to either visual input. For example, participant four suggested that it was easier to concentrate on the images or gestures with a slight visual adjustment.

However, the participants who were more satisfied with the side placement of the hologram-like images reported using different viewing strategies for the middle image placements. Participant six stated that she was distracted by the middle placement and had to change strategies a couple times. At first, she tried to focus only on the images, but when that was still overwhelming, she directed her attention to a blank space and listened to the audio. However, she did not totally abandon the images, as she would briefly look at the images when they changed before returning to the blank spot.

Notably, not all the participants who preferred side placement were able to implement a useful viewing strategy. Participant two indicated he stopped concentrating on the narration because he was overwhelmed trying to mentally separate the visual inputs.

The viewing strategies suggest that, regardless of their satisfaction with the placement of the hologram-like images, the participants generally adopted similar strategies when the images were positioned to the side. This suggests that the design may present a binary choice for the viewer: either focus more on the agent or on the image, depending on which is perceived as more beneficial for retaining information. Since the static images were simple and easy to remember, most participants felt the gesturing agent provided more valuable input to help them retain the information. The lone participant who primarily focused on the static images perceived the agent perceived the agent as non-essential, describing the agent’s gestures as merely “extra”.

Conversely, the middle image placement prompted distinct viewing strategies that were closely tied to satisfaction. The participants who preferred the middle placement expressed satisfaction with having all the information in one location, allowing them to access it with only minor glances up or down. They did not find the combination of multiple informational inputs distracting. However, the participants who were not satisfied with the middle placement appeared to be overwhelmed, leading to negative viewing strategies. Some participants turned the presentation into a listening session by avoiding visual engagement altogether, while others tuned out of the presentation entirely. The middle placement design restricted participants’ ability to adopt alternative viewing strategies to reduce stress, because all the information was centrally concentrated, limiting flexibility in how they engaged with the content.

5. Discussion

This research investigated whether the spatial arrangement of hologram-like images and gestures provided evidence supporting the split-attention principle and examined how separating or combining multiple visual inputs influenced satisfaction, cognitive load, and learning. The quantitative findings revealed no significant differences attributable to the spatial placement of hologram-like image and gestures. Any significant results observed were likely attributable to order effects rather than the spatial arrangement itself.

To explain these findings, it is important to acknowledge that the original concept of split attention primarily addressed the integration of images and text [8]. Over time, research expanded to examine the effects of multiple visuals presented simultaneously [15,16,17,18,19]. The foundation of split attention is that spatially separating information sources forces learners to engage in mental integration, which can overload working memory and hinder efficient processing [8,9,10,11]. This additional processing demand contributes to extraneous cognitive load [13].

The results of this study did not provide evidence supporting the split-attention principle, which is consistent with Schroeder and Cenkci’s [14] analysis of 41 studies. Their findings indicated that integrated graphic designs do not significantly impact measures of cognitive load compared to spatially distant designs. Intrinsic cognitive load was the only significant finding, with the results influenced by order effects rather than the spatial placement of visual information. This suggests that the second video required more inherent processing, which is likely due to the change in the display of visual information from the first video. Unfortunately, the quantitative data provides limited insight beyond drawing parallels to similar findings in previous research. Therefore, the qualitative data will be used for a deeper analysis to explore how these findings may relate to existing theories and research.

The individual interviews were conducted to explore how different visual presentations affected participants’ attention, focusing on whether presenting information in a central location reduced the need to split their attention between separate locations. The interviews indicated that the participants were evenly divided in their design preferences. Satisfaction was higher in conditions that imposed less viewing difficulty (cognitive load), which influenced their viewing strategies. In other words, satisfaction and viewing strategies appeared to be closely linked to the participants’ perception of extraneous cognitive load experienced during the presentation.

These interviews corroborated the findings from previous research, indicating that when participants experience increased cognitive load, they tend to either ignore one source of information [17], or actively switch between sources to focus on the one they perceive as most beneficial for learning [19]. This is consistent with the active processing principle [1], which suggests that learners actively engage with relevant information, organizing it in working memory to integrate it into long-term memory. The data suggests that Schroeder and Cenkci’s [14] assertion that students may employ preemptive strategies could also be correct. This means that the active processing principle might need to be slightly expanded to account for individual differences in cognitive processing that affect viewing choices. If multimedia presentations include numerous visuals, what is considered relevant information may be subjective and vary between individuals, depending on their prior knowledge, cognitive preferences, and learning goals. We propose the “pruning principle”, a metacognitive strategy that prompts learners to actively evaluate information when the processing demands challenge the limits of working memory. This principle suggests that learners will identify and prioritize content they deem relevant while “pruning” information they perceive as extraneous, thereby optimizing their cognitive resources. This principle is consistent with the information foraging theory [53], which suggests that information sources vary in “profitability” and assessments are based on the cost of processing the information. Thus, optimizing cognitive resources for effective learning can be observed in examples such as participant three, who ignored the agent, perceiving it as “something extra”, or participant six, who stared at a blank space on the screen and focused solely on the narration, feeling overwhelmed by having all the visuals concentrated in one place. This may explain why extraneous cognitive load was not significant in this research, as the participants were likely assessing their mental load after implementing their pruning strategies. The pruning principle does not suggest that participants are making optimal choices for learning, but rather that they are concentrating on the information they personally deem relevant for effective learning. Future research could examine the pruning principle using eye-tracking and think-aloud protocols with materials designed to induce varying cognitive load levels, potentially revealing how individuals prune information during the learning process.

Regarding the preference for visual inputs being separated or combined in one location, future research should examine whether satisfaction and cognitive load are influenced by individuals’ inherent spatial ability. Spatial ability is often categorized as high or low, where high spatial ability indicates sufficient working memory capacity to construct mental models, and low spatial ability suggests limited available resources for constructing such models [54]. Spatial ability encompasses multiple skills, one of which is perceptual speed. Perceptual speed refers to the ability to process multiple visual inputs simultaneously [55]. It is plausible to suggest that viewing preference may be influenced by an individual’s spatial perceptual speed. Participants with high spatial perceptual speed might feel comfortable with information presented in a central location, easily making simple up-and-down head movements to integrate the hologram-like images and gestures matching the narration. Conversely, participants with low spatial perceptual speed may have felt overwhelmed due to their reduced ability to process multiple sources of information in working memory. Further research is needed to evaluate whether spatial ability significantly impacts the choices participants make when viewing videos. Additionally, examining how specific spatial skills, such as perceptual speed, can be utilized to design multimedia environments that accommodate diverse learning preferences would provide valuable insights.

Limitations

This study has some limitations. First, the repeated measures design indicated that the only significant outcome was likely influenced by order effects. Future research could address this limitation by employing a between-subjects design to eliminate the potential of order on the results. Second, the results may not be generalizable to all contexts, as the participants were advanced second-language users of English. Factors such as language proficiency level or whether the language is native or a second language could affect the outcomes. Further exploration with different populations is encouraged.

6. Conclusions

The findings from this research suggest that learners employ metacognitive strategies, such as the proposed “pruning principle”, to manage extraneous cognitive load. This principle offers a new perspective on how learners prioritize and process visual inputs in complex multimedia designs, enabling them to focus on information they deem most relevant.

Further qualitative research, such as employing a think-aloud protocol, might be useful to provide valuable insights into how participants make decisions about what to focus on when engaging with multimedia presentations. This approach would allow researchers to capture real-time cognitive processes and decision-making strategies.

Although eye-tracking can provide detailed data on what participants view during the learning process, it does not reveal the underlying strategies guiding individual viewing behaviors. Specifically, it cannot capture conscious decision-making processes or the attention allocation strategies learners employ.

Despite the absence of significant differences in quantitative measures, the qualitative data indicated a relationship between satisfaction, cognitive load, and viewing strategies. Learners’ preferences for image placement were closely tied to their satisfaction and the perceived difficulty of the image location. This suggests that individual differences could impact experiences in multimedia environments. Consequently, further research is needed to assess spatial skills, such as perceptual speed, to better understand how these factors can inform and optimize multimedia design. For instructional designers, this study underscores the importance of creating environments that accommodate the diverse skills and capacities of learners. For example, providing options to customize the placement of visual inputs could improve both learner satisfaction and learning outcomes by addressing individual preferences and cognitive needs.