Creating an Immersive Virtual Reality Game Space for Multiuser, Synchronous Co-Located Collaboration: Design Considerations and Influencing Factors

Abstract

The last decade has witnessed the rapid development of immersive virtual reality (IVR) and its application in various contexts. However, its application in supporting real-time virtual collaboration has been quite rare due to technical barriers and the lack of validated design principles. To address this research gap, this study designed and developed an IVR space to enable multiuser synchronous co-located collaboration to complete a fantasy game. An evaluation study (N = 95) was conducted to explore its useful design considerations and the influencing factors for collaboration experience in the game. The IVR space was enabled by the simultaneous localization and mapping (SLAM)-based inside-out tracking technique and was informed by four essential design considerations for promoting effective collaboration in IVR, namely, the role script, learning task, collaboration mechanism, and communication design. The study results revealed that students in general were satisfied with their collaboration experience in IVR, with social presence and collaboration competency as significant predictors of collective efficacy and social experience. Based on both quantitative and qualitative results, this study proposes four validated principles for designing effective IVR spaces to support synchronous co-located collaboration.

1. Introduction

Immersive virtual reality (IVR) is considered a promising space for delivering authentic and vivid experiences [1,2]. IVR enables students to gain a presence in a highly immersive virtual environment with the capacity to afford high-fidelity simulations, rich sensory stimuli, and multimodal interaction [3]. The sense of presence describes the illusive perception of being present in an artificial or remote environment [4], and immersion is often referred to as a feature of VR that provides a convincing illusion [5]. The past decade has witnessed IVR being applied in various domains, such as entertainment [6], medicine [7], engineering [8], consumption [9], and education [10]. However, previous studies on IVR have mainly focused on individual experience, and little is known regarding users’ collaboration experience in IVR [11].

The implementation of collaboration in IVR is hampered by several challenges, such as geographical distance, temporal distance, and perceived distance [12]. Collaborative work can be classified according to the modalities of the space and time taxonomy as follows: synchronous co-located, asynchronous co-located, synchronous distributed, and asynchronous distributed collaboration [13,14]. Both verbal and nonverbal interactions, which are pivotal for effective group work, are diminished by the asynchrony of time and space during collaboration [15]. This reduction in interaction negatively affects trust between team members and increases the perceived distance between individuals, ultimately undermining the collaborative experience and effectiveness [16,17]. For instance, distributed virtual collaboration has been explored by several studies [18,19], where communication primarily takes place through formal texts. However, this mode of communication still poses challenges to establishing confidence and trust within the virtual space.

Synchronous co-located virtual collaboration is an innovative mode characterized by users entering the virtual space while being physically present in the same location, which enables direct communication between users without any mediation [14]. The social and emotional experiences between collaborators are enhanced through unmediated interaction, which are strongly associated with learners’ engagement and performance [20]. For example, Liang et al. [21] designed a virtual mall where two players stayed in the same physical space and wore VR head-mounted displays (HMDs), which helped them communicate naturally to find goods collaboratively or competitively. Positive affective outcomes, such as increased enjoyment, intrinsic motivation, and social presence, could be facilitated by immersive and interactive experiences [22], which can benefit learning performance by focusing learners’ attention on the task and enabling generative cognitive processing through more frequent and natural social dialogue [23].

However, synchronous co-located virtual collaboration has rarely been explored owing to its additional technological requirements and a lack of design principles. First, the commonly used outside-in tracking technology that enables multiuser co-presence in IVR has proved to be problematic [24,25]. It is susceptible to glitches caused by occlusion and lighting issues, and it also requires many cameras and sensors to be mounted in advance, which increases the total cost and preparation time. Second, a list of validated design assumptions for promoting effective collaboration in IVR is still lacking owing to insufficient design cases and intervention studies in the literature. It remains unknown whether effective design principles for collaboration in brick-and-mortar settings still apply to IVR spaces. It is imperative to overcome these two challenges to enable collaboration in IVR so that students can harness its pedagogical benefits, including social dialogue [26], shared inquiry [27], and participatory equity [28].

To address this research need, we conducted a design case that used a simultaneous localization and mapping (SLAM)-based inside-out tracking technique to design and develop an IVR collaboration game space. By searching the literature on collaboration design, we identified the following four design considerations for the IVR space: role script, task, collaboration mechanism, and communication design. Based on both quantitative and qualitative evidence, these considerations were empirically validated. Additionally, a series of psychological factors were examined as possible predictors of collaboration experience in IVR, enriching the understanding of instructional design in such a context. According to Boling [29], a good design case should offer in-depth explanations of design rationales, multidimensional descriptions of experiences, and documentation of factors that influence the design and implementation process. Consequently, we sought to answer the following three research questions:

• What are the overall collaboration experiences in the IVR space?
• What are the effective design features of the IVR space that support collaboration experiences?
• What are the factors that influence collaboration experiences in the IVR space?

2. Literature Review

2.1. Key Constructs of Collaborative Experience

Most scholars are concerned with the effects of collaboration on learning outcomes, such as cognitive improvement, behavioral skill development, and work completion, but less research has been conducted on collective efficacy and social experience, which are strongly related to collaborative effectiveness [30,31]. For example, Elms et al. [32] investigated collective efficacy as a key predictor of team effectiveness, which was consistent with the argument of Goncalo et al. [30] that collective efficacy contributes to team performance owing to its ability to motivate members and provide direction for cohesive effort. Additionally, learners’ cognitive processes and learning outcomes tend to be profoundly affected by social experience, like social–emotional experiences [33].

Collective efficacy is defined as “a group’s shared belief in its conjoint capabilities to organize and execute the course of action required to productive given levels of attainment” [34]. In other words, collective efficacy reflects a team’s shared confidence in its ability to perform an assigned task well to meet common goals [31]. Collective efficacy has a strong and positive association with team effectiveness (e.g., cognitive skills in computer-mediated communication) [35], performance in programming [36], and sports team performance [37]. Social experience is about socializing, relationship formation and teamwork, and a feeling of real or virtual connectedness [38], which has significant effects on group collaboration effectiveness.

Although collaboration pedagogy is widely used in multiple contexts owing to its effectiveness, the collective efficacy and social experience of collaboration are influenced by many factors, including an individual’s social ability, extroversion, social presence, and past team performance [39]. Social ability usually refers to a person’s capacity to interact, communicate, and use shared resources with peers to accomplish complex tasks [40]; it consists of three key prominent constructs: social presence, social navigation, and social connectedness [41]. Social presence refers to the degree to which one perceives the sense of being in a community and belonging to a group [40,42], and it can predict students’ collaboration experience and satisfaction [43]. Additionally, individuals high in extroversion are likely to be gregarious, assertive, and sociable, as well as to enjoy participating in groups and to believe that group work is a productive use of class time [39].

2.2. Collaboration in IVR

A highly immersive, interactive, and authentic virtual space is created by IVR technology, which completely separates users from the physical world using devices such as HMDs or mobile viewers (e.g., Google Cardboard) [44]. IVR may benefit collaboration by enabling higher levels of representational fidelity and embodiment, (non-)verbal communication, and avatar realism [45]. Several studies have explored collaboration effectiveness in an IVR environment; the results indicate that IVR can support communication, understanding, and teamwork for medical training and education [46] and that it can be used to effectively assess social communication skills [47]. Other benefits include enhanced creativity compared with conventional tools and systems [48], empowered empathetic behavior [49], and improved knowledge acquisition [50].

According to the computer-supported cooperative work time–space matrix, most existing collaboration types in IVR environments are categorized as asynchronous co-located, synchronous distributed, and asynchronous distributed collaboration; there has been relatively less research conducted on synchronous co-located collaboration [13]. In the first three collaboration types, individuals typically use different devices and engage with the virtual content at different times [11,51]. These patterns of IVR collaboration present unequal perspectives that can lead to communication difficulties and poor collective efficacy, especially when different devices are used. Contrarily, synchronous co-located collaboration enables the use of the same devices and equal interaction among peers, which is essential for enhancing presence and facilitating seamless communication [11,52]. For example, Ghoshal et al. [53] developed a co-located multiplayer game and found that co-located collaboration in VR could evoke a strong sense of presence. Drey et al. [11] obtained similar results, finding that symmetric and co-located collaboration led to higher presence and immersion than other collaboration types and also positively influenced the cognitive load.

However, limited research has been conducted on synchronous co-located IVR collaboration, particularly involving more than three users. Possible reasons for this include the requirement for additional technology and space, as well as the absence of design principles for symmetric IVR collaboration [12,21].

2.3. Design Considerations for Collaboration in IVR

By searching the existing literature on collaboration design, we identified four key design considerations for IVR spaces supporting effective collaboration, as summarized in

Table 1. The description of four design considerations.

Collaboration Design Principle	Description	References
Role script design	create vivid and diverse function avatars establish rules that encourage learners to collaborate	[50,51]
Task design	narrative-driven attractive challenging	[1,11,54]
Collaboration mechanism design	equal access in virtual world for all users permit collaboratively discussing strategies	[14,50]
Feedback and communication design	provide virtual space feedback allow real-time communication	[18,55]

. The first is role script design. Considering the imaginative features of VR, designers can create vivid and diverse function avatars and establish rules that encourage multiple learners to collaborate toward a shared goal [50]. For example, Thompson et al. [51] developed a Cellverse virtual space, established roles (explorers and navigators), and distributed resources between players. The explorers were responsible for completing tasks that involved identifying protein structures and tracking processes within the cell, and the navigators guided the explorers and provided helpful clues for working with other team members. This sharing of responsibilities in design builds positive interdependence and individual accountability between team members.

The second is task design. As emphasized by Won et al. [54] and Ferguson et al. [1] regarding the characteristics of the IVR story structure, multiuser collaborative tasks should be narrative-driven, attractive, and challenging to engage users emotionally and intellectually in common objectives. For instance, Drey et al. [11] constructed a VR forest environment where students could roam freely, see three forest animals, and learn their typical characteristics. This IVR task demonstrated an authentic and interesting virtual scenario to attract users to find animals located in different areas. However, the approach of introducing the characteristics of animals mainly focused on content in a direct presentation, lacking challenge and exploration.

The third is collaboration mechanism design. This principle refers to design content and instructions that are equally accessible to all users [50]. This can be achieved through task design, role design, or by allowing students to collaboratively discuss strategies. Several studies have been designed to allow collaborators to access the VR world through different devices [14,50]—for example, two collaborators who physically stayed in the same space, one of them wearing an HMD and the other using a tablet to experience VR separately. However, there existed unequal involvement among students because these IVR collaboration rules allowed one to experience a highly immersive virtual world and the other merely to access information through a 2D or 3D flat screen, which potentially influences effective and worthwhile collaboration.

The fourth is feedback and communication mechanism design. This involves integrating virtual space feedback design with real-time communication design while aligning real and virtual locomotion. This aligns with the “actional” and “social” features of IVR-based learning environments proposed by Dede et al. [55]. Eiris et al. [18] established a 360° panoramic virtual learning space for masonry and architectural knowledge learning, where students can use their keyboard and mouse interfaces to direct a color-coded virtual laser pointer. This IVR space enabled students to point to 360° panoramic content while describing their thoughts using a third-party voice system (Zoom). Students were more engaged in collaborative tasks when interactions were varied and feedback was immediate.

3. Design and Development of an IVR Collaboration Space

The IVR collaboration space developed by the research team was a multiplayer collaborative fantasy game called The League of Castle Defenders. The game provides opportunities for five players to collaboratively complete interactive missions. It features three roles with different functions that help players build shared responsibilities and positive interdependence between team members. According to the collaboration mechanism and task design considerations, the game scenarios are inspired by ancient Chinese culture and present heroes fighting against enemies to defend their homeland, which requires players to defend a castle with their unique role functions. Additionally, this game was designed specifically so that players situated in co-located physical space can communicate strategies and provide timely feedback when players’ conditions have changed. Figure 1 shows a prototype of the game, which includes three game roles: enemies, a castle, and a warship, among other elements. The focus of the game is that the three roles collaboratively resist the attacking enemies.

3.1. Mission and Role Design

The mission of defending a castle is accomplished after the players slay five enemies before the castle gates fall and while at least one player remains alive. During the game, five enemies fire random artillery at 5 s intervals from the cannons, which cause 100 damage points to either the castle or the players. The castle has 1000 health points and can withstand up to 10 artillery attacks. Each enemy is set to 1000 health points, while the defender is initially set to 500 health points, and the other two roles are set to 50. As shown in

Table 2. Functions and missions of the three different roles.

Role Type	Role Functions	Role Missions
Defender	Block artillery fire Initially has 500 health points	Anticipate the flight path of the artillery and move quickly to block it.
Attacker	Attack the enemies Initially has 50 health points Initially has 10 arrows (an arrow = 100 damage)	Slay all the enemies by shooting arrows while avoiding the enemies’ artillery.
Collector	Initially has 50 health points Open medical supply boxes that restore 100 health points to defenders and ammunition supply boxes that replenish two arrows to attackers	Find and open the supply boxes at the back of the group to provide the defender with blood or the attacker with arrows.

, the defender can hold a shield against the enemies’ artillery fire to protect teammates or the castle from attack; the attacker can shoot arrows at the enemy. Each attacker initially has 10 arrows, and one arrow can generate 10 points of damage; the collector is responsible for collecting supplies by opening boxes, including an ammunition supply box to replenish arrows for the attackers and a medical supply box to restore a defender’s health by 100 points. Players are eliminated from the game when their health points fall below zero.

The players need to collaboratively complete the missions of the game to achieve victory. Figure 2 shows the prototype design of the three roles. While executing missions, the defender needs to stand at the front of the team to block the artillery attack and, at the same time, needs to anticipate the flight path of the artillery and pay attention to the changes in health points. The attacker needs to pay attention to the line of the artillery while ensuring accurate shooting. The collector needs to move quickly in a large area at the back of the team near the walls to find supply boxes and prioritize the correct supply boxes for the team’s specific situation.

3.2. Human–Computer Interaction Design

The researchers designed interactions between the player and the virtual space or virtual objects to enhance realism and the users’ immersion. The researchers classified the interactions into visual and action interactions to highlight the large spatial multiplayer collaboration in VR. For visual interaction, players can check their health points or arrows by observing the status bar. Players who select the defender role can observe their health points through the status bar, while players who select the attacker role can observe their health points and remaining arrows, as shown in Figure 3. Additionally, people outside the game field can see the status of each player in real time by looking at the right side of the computer interface. For example, when a player has zero health points, the status bar appears grayed out.

In terms of action interactions, players can run and move fully in space to complete missions. The players who choose the attacker role pick up the in-game bow and shoot arrows in the virtual space by raising their arms and pressing the button on the VR controller. The players who choose the defender role also lift their shields by raising their controllers. The players who choose the collector role roam in the virtual space to collect and open the supply boxes by wielding their controllers.

3.3. Collaborative Mechanism Design

This game requires three roles to use their character features and equipment, thus interdependently collaborating to achieve game success. The defender, standing at the front of the team, needs to try to block each launch of the enemy’s artillery fire, thus ensuring the safety of other teammates. The attacker, hiding behind the defender, needs to accurately attack the enemy as fast as possible, which is the key to achieving victory. The collector, at the end of the team, needs to communicate with the other players to obtain supplies to ensure that the defender has enough health points to withstand enemy fire and that the attacker has enough arrows to inflict damage on the enemies. Overall, without the defender on the team, the other two roles would soon be injured by artillery fire; without the attacker, the enemy would continually attack the castle, causing its gates to collapse. Furthermore, without the collector, the attacker would not be able to obtain weapons, and the defender would have difficulty blocking artillery fire owing to insufficient health points. Therefore, to achieve victory and complete the game together, the three roles need to have positive interdependence, prompt communication, and effective collaboration.

3.4. Technical Features

3.4.1. Hardware

The device used in this study was an Oculus Quest 2 (Manufactured by Facebook Technologies, LLC, Wilmington, NC, USA), an all-in-one VR device developed by Meta. It offers high resolution and strong refresh-rate capabilities, resulting in immersive and realistic visual effects. The Oculus Quest 2 simplifies the integration of the VR headset and Leap Motion device by providing 360° head-movement tracking and gesture tracking without the need for controllers. We still used controllers in our IVR game for steadier performance and prolonged battery usage. Consequently, users can engage in VR experiences without being limited by physical space, making the device well suited for large-scale multiplayer VR activities.

3.4.2. Software

The collaboration space for this study was created using Unreal Engine 4 (UE4), version 4.26.2. UE4 offers a comprehensive visual scripting system that enables researchers to build diverse virtual environments and programmable visual representations. The underlying layer of UE4 is implemented in the C++ programming language, which effectively improves the programming quality of large-scale programs. At the same time, UE4 provides a complete visual scripting system: the Blueprint tool. To make development more efficient, the Blueprint tool and C++ tools in UE4 can be used in tandem. The game was also developed using Microsoft Visual Studio, which is a largely complete set of development tools. UE4 works perfectly with Visual Studio, enabling users to write project code quickly and easily.

3.4.3. Algorithm

To facilitate real-time online interactions among multiple participants, this virtual collaboration game relies on a spatial-positioning technology called “inside-out tracking”. Inside-out tracking uses the cameras on the HMD to detect changes in the external environment. It also leverages SLAM algorithms and depth cameras to calculate the precise position of the HMD in space. This SLAM-based inside-out tracking technology enables multiuser VR spatial localization by sharing SLAM data and ensuring real-time synchronization among users. Combining the rendering mechanism of virtual scenes allows for the co-presence of multiple users in virtual collaboration spaces, leading to a heightened level of social immersion.

Consequently, The League of Castle Defenders is a multiplayer VR game that was designed to provide visual, auditory, and physical interactive experiences. Figure 4 depicts the complete process, starting from the beginning of the game and leading to victory. Although the process is not overly complex, successful completion requires effective communication and collaboration between all players involved.

4. Evaluation Study

4.1. Participants

The participants were 95 undergraduate students (30 men and 65 women) recruited from a Chinese university via advertisements posted on a social platform. The minimum sample size calculated by the software G*Power 3.1.9.4 was 92 (f² = 0.15, α err prob = 0.05, power = 0.8, number of predictors = 5), which was adequate to conduct the linear multiple regression in this study according to the approach of Faul et al. [56]. The participants’ ages ranged from 18 to 21 years (Mean = 21.68, Standard Deviation [SD] = 1.93). Based on the information that they provided in a formal questionnaire before the experiment, participants came from a wide range of academic programs (e.g., history, journalism, education, computer science, and chemistry) and reported being unfamiliar with playing VR games. All the participants had normal vision or corrected-to-normal vision and received a small reward to compensate them for participating. Informed consent was obtained from all the participants before conducting this study. This study was approved by the Institutional Review Board of Central China Normal University (IRB No. CCNU_IRB_202111_047).

4.2. Procedure

Before the experiment, participants were required to complete a questionnaire on a scale that included basic information, personality, collaboration ability, and VR experience. The participants were then randomly divided into 19 groups of five participants and informed of the accurate time and location of the formal experiment by message. The experimental site chosen was in the school’s indoor gymnasium, which provided a spacious, safe environment.

The formal experiment was divided into three phases. The first phase was for observing and generating a collaboration strategy. Before a group entered the IVR collaboration space to complete the collaborative task, the researcher introduced the task scenario, game mechanism, role functions, and individual responsibilities. All members of the group observed the performance of the group before them that was playing on a laptop and then chose an assignment strategy for the three roles through face-to-face negotiation. The second phase was the formal IVR experience phase. Five research assistants assisted participants in wearing the HMD to ensure that they could clearly see the objects in the IVR environment, guided them on how to use the VR equipment, and explained the rules of the game. Then, under the guidance of the research assistants, the participants chose their roles according to what they had discussed, and then they were moved to the specific location to realize an exact match between the physical and virtual locations. During the experience phase, participants could communicate and exchange information by shouting aloud. Finally, participants played the VR games under the supervision of three research assistants, who ensured their safety in the physical tracking space. Figure 5 illustrates a game screenshot and the corresponding physical space as the students collaborated to complete the game task.

After the VR game, each participant was required to complete a questionnaire on a scale related to their VR experience. Additionally, seven groups were selected to participate in semi-structured interviews to express understanding of the collaboration experience through the following questions: Can you briefly describe your gaming experience this time? What designs and experiences impressed you? Did anything interesting or difficult happen during the game? The interview duration for each group ranged from 13 to 25 min, and the recorded interviews were later partially transcribed, generating a text content of 11,244 Chinese words for qualitative analysis.

4.3. Data Collection and Analysis

4.3.1. Quantitative Data Collection and Analysis

This study included quantitative and qualitative data. The quantitative data were collected through a five-point Likert questionnaire after the IVR game experience. The questionnaire comprised three parts. Part 1 of the questionnaire featured basic information about the participants, including their personality, collaborative competence, and VR readiness (see Appendix A). Collaborative competence measured participants’ communication skills, teamwork ability, leadership, and collaboration experiences; these items were modified from the scales designed by Radziej et al. [57] and Duong [58].

Parts 2 and 3 of the questionnaire assessed the IVR collaboration experience (collective efficacy and social experience) and the IVR presence (spatial presence and social presence) (see Appendix A). The items related to IVR collaboration experience were based on the scale for digital collaborative games proposed by Vidergor [38], and the items related to IVR presence were based on the scale for collaborative augmented-reality games proposed by von der Pütten et al. [59]. As shown in

Table 3. Structure of questionnaire and its reliability and validity.

Constructs	Number of Items	Cronbach’s Alpha	CR	AVE
Basic information
Personality	5	0.708	n/a	n/a
Collaborative competence	19	0.944	n/a	n/a
VR readiness	8	0.809	n/a	n/a
Collaboration experience
Collective efficacy	8	0.942	0.922	0.748
Social experience	6	0.902	0.914	0.641
IVR presence
Spatial presence	5	0.836	0.831	0.632
Social presence	3	0.763	0.770	0.408

, the results of Cronbach’s alpha coefficients are considered acceptable when the number of items is less than 10, according to Bland and Altman [60], and the results of composite reliability (CR) and average variance extracted (AVE) meet the criteria in general, according to Campbell and Fiske [61].

A descriptive statistical analysis and regression analysis were applied to the quantitative data. The former was used to understand the participants’ experience based on IVR experience and basic information. The latter was used to explore the factors that influenced students’ collaboration experience in IVR. Consequently, collective efficacy and social experiences were considered outcome variables, and all other variables (e.g., personality, collective competency, social presence, and spatial presence) were entered into regressions as predictors.

4.3.2. Qualitative Data Collection and Analysis

The qualitative data comprise mainly interview data. We used various coding techniques (e.g., Structural, In Vivo, and Evaluation coding) described by Saldana [62] to qualitatively analyze the interview transcripts, aiming to assist in the meaningful interpretation of participants’ collaboration. Finally, 136 codes were identified from the transcribed interviews. Then, we sorted the codes for further analysis and theme generation, as suggested by Bazeley [63]. A total of 32 nodes emerged during our qualitative coding process and can be classified into eight categories, as shown in

Table 4. Results of themes, categories, and nodes for VR experiences.

Themes	Categories (Frequency Count)	Nodes (Frequency Count)
Theme 1	Game experience (25)	Novel and interesting (9)
		Challenging (8)
		Intense (3)
		Enjoy (3)
		Competing and exploring (2)
	Immersion (24)	Spatial presence (5)
		Exquisitely crafted visuals (5)
		Realistic special effects (5)
		Natural feedback (4)
		Authentic story (5)
Theme 2	Co-presence (14)	Observing teammates’ positions (2)
		Seeing avatars’ actions (3)
		Interacting with teammates (6)
		Moving realistically (1)
		Face-to-face negotiation (2)
	Co-located (11)	Safety issues (6)
		Real and warm (2)
		Form strategies quickly (3)
	Pre-training (22)	Lack of understanding of game rules (9)
		Unfamiliarity with VR devices (1)
		Effective observation and learning (11)
		Prior experience (1)
Theme 4	Technological issues (7)	Unstable real-mapping technology (5)
		Unstable game functionality (2)
	Communication issues (14)	Channels of communication (5)
		Timeliness of communication (5)
		Flexibility in communication (4)
	Game issues (19)	Lack of dynamic information cues (10)
		Lack of level challenges (3)
		Lack of audio cues (2)
		Lack of key content prompts (3)
		Lack of personalized choices (1)

. Additionally, Appendix B lists the complete qualitative coding results.

4.4. Quantitative Findings

4.4.1. Overall Experience of Collaboration in IVR

To understand participants’ experience during IVR-based collaboration, we conducted a descriptive analysis. As shown in

Table 5. Descriptive analysis of the collaboration experience in IVR.

Item Statement a	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
Social presence (Mean = 3.84, SD = 0.64)
Presence of virtual objects in the game scene	1 (1.1%)	0	9 (9.5%)	55 (57.9%)	30 (31.6%)
Realistic feeling of virtual things	0	5 (5.3%)	27 (28.4%)	42 (44.2%)	21 (22.1%)
Forgot that it was computer-generated	0	13 (13.7%)	22 (23.2%)	36 (37.9%)	24 (25.3%)
Awareness of teammates’ presence	0	6 (6.3%)	10 (10.5%)	50 (52.6%)	29 (30.5%)
Easy communication with teammates	3 (3.2%)	21 (22.1%)	25 (26.3%)	31 (32.6%)	15 (15.8%)
Spatial presence (Mean = 4.21, SD = 0.66)
A feeling of entering a new world	0	2 (2.1%)	4 (4.2%)	48 (50.5%)	41 (43.2%)
A feeling of being part of the game	0	3 (3.2%)	13 (13.7%)	41 (43.2%)	38 (40.0%)
All my senses were stimulated in the game	0	4 (4.2%)	15 (15.8%)	43 (45.3%)	33 (34.7%)
Collective efficacy (Mean = 3.98, SD = 0.72)
My group could solve difficult situations.	0	2 (2.1%)	16 (16.8%)	52 (54.7%)	25 (26.3%)
Group members worked harder than expected.	0	4 (4.2%)	11 (11.6%)	50 (52.6%)	30 (31.6%)
Group members worked hard to complete task.	0	1 (1.1%)	5 (5.3%)	51 (53.7%)	38 (40.0%)
My group was effective in finishing the task.	1 (1.1%)	4 (4.2%)	26 (27.4%)	38 (40.0%)	26 (27.4%)
My group did a good job in getting things done.	0	3 (3.2%)	23 (24.2%)	36 (37.9%)	33 (34.7%)
My group effectively fulfilled task requirements.	0	3 (3.2%)	23 (24.2%)	40 (42.1%)	29 (30.5%)
My group accomplished its goals successfully.	0	9 (9.55)	28 (29.5%)	35 (36.8%)	23 (24.2%)
My group completed its task successfully.	2 (2.1%)	14 (14.7%)	22 (23.2%)	32 (33.7%)	25 (26.3%)
Social experience (Mean = 4.11, SD = 0.66)
I felt I was not alone.	1 (1.1%)	4 (4.2%)	13 (13.75%)	46 (48.4%)	31 (32.6%)
I felt the group members supported me.	0	2 (2.1%)	9 (9.5%)	57 (60.0%)	27 (28.4%)
I felt I had someone to work with in my group.	0	4 (4.2%)	6 (6.3%)	52 (54.7%)	33 (34.7%)
I felt like a member of my group.	0	1 (1.1%)	6 (6.3%)	50 (52.6%)	38 (40.0%)
I felt connected to others in my group.	0	4 (4.2%)	12 (12.6%)	52 (54.7%)	27 (28.4%)
I felt I could discuss with members of my group.	2 (2.1%)	13 (13.7%)	11 (11.6%)	40 (42.1%)	29 (30.5%)

^a. Certain item statements are condensed to fit in the table column. See Appendix A for complete statements.

, participants were satisfied with the IVR collaboration experience, which can be seen from the high evaluation of all dimensions (all Mean > 3), especially in spatial presence (Mean = 4.22, SD = 0.66) and social experience (Mean = 4.11, SD = 0.66).

4.4.2. Factors Influencing the Collaboration Experience in IVR

Next, to determine the relationships among all of the variables, a correlation analysis was performed.

Table 6. Correlation analysis between predictors (Pearson’s correlation coefficient).

	EP	RD	CC	SpaP	SocP	CE	SE
Extroverted personality (EP)	1
VR readiness (RD)	0.476 **	1
Collaboration competence (CC)	0.612 **	0.373 **	1
Spatial presence (SpaP)	0.226 *	0.153	0.375 **	1
Social presence (SocP)	0.277 **	0.154	0.302 **	0.706 **	1
Collective efficacy (CE)	0.303 **	0.174	0.297 **	0.459 **	0.521 **	1
Social experience (SE)	0.275 **	0.119	0.365 **	0.577 **	0.679 **	0.627 **	1

Note: ** p < 0.01 and * p < 0.05.

shows the preliminary results for the descriptive statistics and Pearson’s correlation coefficients. First, an extroverted personality and collaboration competence were weakly positively correlated with spatial presence (r = 0.226 and 0.375, respectively) and social presence (r = 0.227 and 0.302, respectively). In addition, spatial presence and social presence had a medium correlation (r = 0.706). Additionally, VR readiness was not correlated with collective efficacy, while an extroverted personality (r = 0.303) and collaboration competence (r = 0.297) were weakly correlated with collective efficacy, and spatial presence (r = 0.459) and social presence (r = 0.521) were moderately positively correlated with collective efficacy. Moreover, an extroverted personality and collaboration competence were weakly correlated with social experience, while spatial presence and social presence were moderately correlated with social experience.

To find the most parsimonious model for predicting collective efficacy and social experience, we used the backward-elimination multiple linear regression analysis approach. Using collective efficacy as the dependent variable, the final model included two independent variables (IVs): extroverted personality and social presence. The regression analysis suggested a significant final model (F _{(2, 92)} = 19.577, p < 0.001) in which the IVs explained 28.3% of the variance in collective efficacy (R² = 0.283), and only social presence (β = 0.473, p < 0.001) was a significant predictor of the collective efficacy. In terms of social experience, the results showed that in the final significant model (F _{(2, 92)} = 44.161, p < 0.001), the IVs explained 47.9% of the variance (R² = 0.479), and collaboration competence (β = 0.176, p < 0.05) and social presence (β = 0.626, p < 0.001) were significant predictors. The collinearity analysis further reported an acceptable multicollinearity level among the independent variables [64].

Table 7. Key statistics of the backward-elimination multiple linear regression analysis (N = 95).

Dependent Variable	Predictors	B	SE	β	Collinearity		Adjusted R2
					Tolerance	VIF
Collective efficacy	Extraversion	0.197	0.104	0.172	0.923	1.083	0.283
	Social presence	0.529	0.102	0.473 ***	0.923	1.083
Social experience	Collaboration competence	0.214	0.095	0.176 *	0.909	1.100	0.479
	Social presence	0.642	0.080	0.626 ***	0.909	1.100

Note: SE is an abbreviation of Standard Error; VIF is an abbreviation of variance inflation factor; *** p < 0.001 and * p < 0.05.

shows the regression model summary.

4.5. Qualitative Findings

We identified four main themes through the qualitative analysis of interviews. The first three describe the specific aspects of participants’ experiences with the VR game and reveal the possible influencing factors, such as technical affordances and individual attributes. The themed findings provide data triangulation and an explanation of the quantitative results, and they are detailed in this section. The fourth theme identifies several issues regarding the current game design and technologies, which improved our understanding of participants’ collaboration experiences in VR and provides a basis for future VR game refinement. Table 4 lists the themes, categories, and nodes.

4.5.1. Themed Finding 1: Spatial Presence and Natural Feedback Afford a Sense of Immersion in the IVR Collaboration Space

Although the students admitted that they had limited past VR experience, their unease and self-doubt rapidly disappeared after entering the virtual world, and they engaged with the created virtual environment immediately. A possible explanation is that the virtual world was highly authentic, with the same spatial and temporal rules as the physical world, thus greatly reducing the time needed for adaptation and adjustment. As one student explained, “As the artillery fire flew toward me, I felt the danger and needed something to block”. Additionally, the fluency and details of the virtual scenarios enhanced the immersion level of students. Especially when seeing smoke appearing after the explosion of artillery or the enemy fall, they found it extremely “playful”, “exciting”, and even “intimidating”.

The immersion was further enhanced when various learner actions were triggered in the virtual space, with effects similar to those in the real world; for example, the speed of running in reality matched the speed of moving in the virtual space. Additionally, the gesture interaction was an important factor in enhancing immersion; as explained by a student playing the role of the attacker, “I can press the button on the handle to shoot arrows to attack the enemy, which is the same as pulling the trigger to actually shoot an arrow”.

4.5.2. Themed Finding 2: Co-Presence in IVR Is a Key Feature That Improves Social Presence and Collaboration Experience

Compared with computer-mediated communication in VR worlds, the synchronous co-located collaboration space enabled a high level of co-presence through real-time negotiation and reflection among the students. According to Strojny et al. [65], the perceived presence of group members affects one’s behavior and attitude directly or vicariously, thus facilitating the social dynamics and decisions within the group. This phenomenon is also known as social facilitation. The responsibility and interdependence designed into the roles can promote communication and enhance co-presence. One student who played the role of attacker stated that “the Collectors were able to give ample resources”, and with the “Defender in front against the artillery fire”, he could go forward without fear.

Additionally, the co-presence and social presence in the VR collaboration were further enhanced through real-time mapping of the physical and virtual space. When activities in the virtual space are comparable to those in the physical world, students will carefully consider individual skills and role assignments. The sense of co-presence is further enhanced through the overlapping of virtual and physical tasks, which leads to greater social presence and collective efficacy. For example, those who run faster in the real world than other participants are better suited for the role of collector, while those with better communication skills may be better suited for the role of attacker, who needs to constantly command the team during game play.

4.5.3. Themed Finding 3: Individual Competence Rather Than Personality Influences the Collaboration Experience in IVR

Although an extroverted personality is a vital factor that affects collaboration in traditional face-to-face environments, its influence was diminished in VR environments. When wearing the HMD and entering the VR environment, students create an avatar to collaborate with others, instead of acting as themselves, which makes it more comfortable for introverted students to be involved in collective tasks. For example, a self-reported introverted student immersed in IVR would keep trying to talk to his teammates to ask for more supplies when he realized he was low in health points. However, the other two extroverted students did not collaborate as well in the IVR collaboration space, noting that

“… We discussed the division of labor before game started, but there were still some unexpected issues when artillery fire appeared. Sometimes two of us bumped into each other trying to block the same artillery fire, and sometimes no one blocked a particular fire …”

Moreover, compared with their individual personalities, students’ collaborative competencies, such as communication, negotiation, and decision-making, were crucial factors that influenced the social experience of VR environments. The groups that completed the game tasks preferred to negotiate and develop an effective strategy and also find ways to contribute to teamwork based on their abilities. For example, an introverted student who achieved victory chose the role of collector because she was used to roaming in a mobile game. Similarly, as an extroverted student from the winning group explained,

“I’ve always been a novice at playing games, so I thought I might not be able to shoot in this game, but the Defender only needs to run back and forth, and I thought I’d be better able to do it, so I chose the Defender …”

5. Discussion

This paper presented a design case of a highly immersive, naturalistic, and multiuser IVR collaboration space enabled by SLAM-based inside-out tracking technology. To explore the students’ experience in this virtual space and the factors that affected collective efficacy and social experience, an evaluation study was also conducted. Consequently, this design case provided important empirical evidence that instantiates and validates the four design considerations for creating effective synchronous, co-located collaboration in IVR. In this section, we provide an in-depth discussion of the four design considerations, with elaborations on their theoretical underpinnings and pragmatic implications.

5.1. Role and Collaboration Mechanism Design

The fact that social presence can significantly predict students’ collective efficacy and social experience in IVR highlights the importance of designing a proper role script and collaborative mechanism for the IVR learning space because these two design features are known to cultivate a strong social presence during group collaboration [51,66]. Our interview and observation results showed that the role script (featuring three distinctive functions) and collaborative mechanism were necessary and useful in structuring collaboration and delivering co-presence in the novel context of IVR. Such role design allowed the students to engage in meaningful negotiation and joint actions, which enabled the whole collaborative system to accomplish the common goal in a more effective and standardized way, as argued by Strijbos and Weinberger [67]. The reciprocal relationship between the role design and collaboration mechanism was also emphasized by Laal [68], who argued that well-designed role scripts can promote group awareness and participation and thus benefit the collaborative process with increased interdependence. Additionally, IVR can enable role scripts and collaboration mechanisms that are more engaging, creative, and complex through virtual avatars and fantasy worldviews, leading to enhanced social experiences.

5.2. Task Design

According to the qualitative findings of this study, the collaboration task of the IVR game The League of Castle Defenders was well received by the students owing to three design features. First, the collaboration task allowed the students to freely explore an authentic virtual world characterized by an authentic environment, action, and narratives [69]. IVR allowed students to experience imaginary environments as active explorers, rather than as passive observers. According to Thompson et al. [51], such affordance of IVR is ideal for the construction of inquiry-based collaboration tasks. Second, the collaboration task engaged the students and helped them actively participate in the process of discussion, decision-making, and problem-solving to create positive interdependence. As Rodríguez Illera [42] concluded, tasks that create genuine interdependence have three features: “(1) to share the resources; (2) the division of work between the members of the team; and (3) to share the cognition through a joint activity” (p. 485). In this study, the castle defense task was divided into collecting resources, attacking enemies, and defending against artillery fire such that the dependence between the subtasks and the main task was clearly and intuitively established. Third, the collaboration task allowed for real-time mapping of the virtual and physical space, which made the students’ cognition and decisions in IVR more intuitive and reflective of their skills and preferences in the physical world. The researchers favor such a design as it maximizes individual strengths and can potentially enhance students’ independent thinking, communication, and problem-solving skills [70].

5.3. Feedback and Communication Design

A feedback design for The League of Castle Defenders game in this study was generated through two types of interaction between the learner and the IVR environment: visual interaction and action interaction. The interview results revealed that the natural feedback of the VR game scenes (e.g., smoke dispersal) and the same spatial and temporal rules as the physical world generated a high level of spatial presence and immersion. The game was designed to provide a high degree of interaction between students and the virtual objects included in the IVR environments, as well as ongoing feedback to the players, which has been linked to deeper learning and assists in the learning process [71]. Additionally, the communication mechanism in this study was designed based on the mapping of virtual and real space, which allowed students to synchronously communicate and perceive the presence of others more intensely. We believe that this mechanism can generate greater social presence and thus lead to enhanced social experience and collective efficacy, as indicated by the regression model. Such a finding supports previous research findings that link social presence with enhanced collaboration experience and performance in online learning contexts [43].

However, there is still room for improvement in the feedback and communication design. In our study, the students regretted not being able to keep track of their teammates’ status in the virtual game because the physical game field was extremely large, which was one of the key factors affecting the sense of co-presence. As Bulu [72] argued, co-presence consists of two dimensions: “having a sense of feeling of other individuals and having a sense of feeling that others were actively perceiving us and being part of a group” (p. 155). This collaborative space lacked dynamic perception among peers owing to poor communication. Thus, future research can consider elaborating on the internal communication channel to facilitate verbal interaction.

5.4. Pre-Training Design

Despite students’ general positive ratings of their collaboration experience in IVR, few groups completed the task in this study. We believe that the lack of individual collaboration competency for certain students is a possible reason. The regression results suggest that one’s social experience in IVR collaboration can be negatively affected by low collaboration competency, which, in turn, can lead to decreased cognitive engagement and performance [33]. Therefore, it is necessary to conduct pre-training to develop students’ collaboration skills prior to their IVR experience for enhanced social experience and group performance. The importance of adding a pre-training session in IVR interventions is also supported by previous studies, reporting benefits such as enhanced self-efficacy [73] and reduced cognitive load [73,74].

5.5. Limitations and Future Research

This study has several limitations. First, the IVR collaboration space developed in this study lacks stability owing to technical glitches, such as the loss of the player’s position in virtual space, which had negative effects on the collaboration experience. Future research needs to improve the stability of the IVR space to boost the relevance of study results on collaboration. Second, because the empirical investigation was conducted only once in this study, students’ experiences in the IVR collaboration space may have been influenced by the novelty effect. Future researchers are recommended to conduct longitudinal studies to enhance the credibility of the empirical findings. Third, this study measured collaboration experience by collecting data only through a self-reported questionnaire, which tends to be subjective and less accurate. Subsequent studies could use more diverse data-collection instruments, such as observation protocols, location heatmaps, or the Kinect sensor to collect multimodal data for a more accurate and comprehensive analysis of students’ collaboration behaviors and experiences. Fourth, note that the overall prediction capacity of the regression models was low, suggesting the absence of key influencing factors. Future studies should extend the scope of the investigation to include a more comprehensive list of predictors for collaboration experience in IVR. Lastly, although we believe our research findings based on the collaborative IVR game can still inform real work collaboration in IVRs, future research studies are needed to systematically investigate their applicability with more authentic collaboration tasks.

6. Conclusions

In conclusion, our study provides tentative answers to the research questions raised in this study. First, this study presented a design case of a synchronous co-located IVR game space that supports multiuser, real-time collaboration, which achieved high satisfaction among players with IVR collaboration experience. Second, we explicated and empirically validated four design considerations for IVR collaboration spaces by documenting the design, development, and evaluation of a collaborative IVR game. Third, the study results revealed that social presence and collaboration competence positively predicted social experience and collective efficacy. Additionally, we further identified spatial presence and natural feedback as two important factors that affect the immersion of IVR space through qualitative data analysis, finding that co-presence was essential for enabling social presence in IVR collaboration activity. Moreover, our findings revealed that synchronous, co-located virtual collaboration was found to place high demands on certain individual competencies, such as motor ability.

We believe this study makes several contributions to the existing literature. The first contribution is that we innovatively created a synchronous co-located IVR space using a SLAM-based inside-out tracking technology that supports multi-player presence, and thus the research findings extend our contextual understanding of computer-mediated collaboration. The second contribution is that we demonstrated a complete design case of an IVR collaboration space, which serves as an exemplar showing how different design considerations can be integrated to enable group collaboration in IVR environments. The third contribution is that we utilized both quantitative and qualitative data to empirically explore the effects of the IVR design on promoting positive collaboration experiences as well as the potential influencing factors. The empirical findings provide valuable implications for designing and implementing synchronous co-located collaboration in IVR.