A total of 53 university students, 27 males and 26 females, participated in the experiment. Their ages ranged from 20 to 29 (mean: 23.8, standard deviation: 2.3).
Table 2 is the summary of the participants.
4.1. User Analysis
The participants were categorized into the four groups as shown in
Table 2 based on their level of immersion and positive or negative valence in VR with appreciation or depreciation of new technologies. Each participant’s answers were simply added as scores in each category, and the average of the participant’s scores was compared to those of the other participants. High and low immersion levels were evaluated, then the means of positive or negative valence in VR were compared after. As a result, we found three types of user groups; high immersion group with positive valence (HIPV), low immersion group with positive valence (LIPV), and low immersion group with negative valence (LINV). High immersion with negative valence (HIPV) was not found among the participants. The participants in HIPV are deeply immersed when they are watching TV or movies, and they believe VR could make people have diverse experiences through an extended sense. The participants in LIPV have a low immersion level, but still they enjoy and appreciate new technologies. The participants in LINV have a low immersion level like those in LIPV, but they have many concerns about being in a virtual world and the society with unverified new technologies.
Table 3 shows the distribution of the four groups identified and interestingly, no one was identified as LINV.
As a result of the pre-questionnaire, out of 53 participants, 12 were assigned to HIPV, 39 to LIPV, and 2 to LINV. In addition, the data were analyzed in terms of gender. The distribution of genders of the participants in the three groups is shown in
Table 4. Overall, two participants in the LIPV group were eliminated from the eye-gaze data results due to lack of data for statistical analysis, but they were included in the gender analysis.
4.2. Eye-Gaze Data
Selective attention, along with perceptual fidelity and other sensory factors, affects how much presence is reported [
14]. Eye movement data of the participants’ right eye (slightly different with their left eye but not significant for this analysis) were analyzed for observing their attention, then the average value of both eye gaze directions was calculated for each frame. By calculating the standard deviation (SD) using these averages of data from the subjects in each group, it is possible to grasp how much the participants’ eye gaze deviated in each frame. In the Unity engine, 1 unit of grid corresponds to 1 m in the real world, so the scale of the SD shows how much a participant’s eye focus deviates from the average focus of all the participants. SD closer to 0 indicates a higher level of shared attention from the participants.
Figure 6,
Figure 7,
Figure 8 and
Figure 9 show the SD of the users’ eye foci in each group for every frame. The tables on the right show the items that were seen by the participants. The numbers in these tables are the mean hits counts of the objects that the participants see. These numbers were acquired by detecting the collisions of the items and rays that were cast from the participants’ eyes in each frame using the Unity engine’s raycasting function.
The SD fluctuated (min 0.084, max 0.452, mean 0.229) most of the time and dropped for a short time (d) in Climax. The ‘back (a soldier running out from a player’s back)’ and ‘tripod (the most important moving object in the story)’ were the items that were seen most by the participants in Rising Action and Climax, respectively. The interest of LIPV group is scattered quickly toward the environment.
The SD was low in the most range except for some spots in Rising Action and highly stable in Climax (min 0.048, max 0.462, mean 0.135). The ‘back’ was most seen by participants in Rising Action, but the number of collisions was low compared to the number of ‘back’ in
Figure 6. During ‘Rising Action’, the people in this group were distracted by the falling ‘meteo (meteorite)’ and ‘two soldiers run out of the building’.
The SD was bumpy and unstable most of the time (min 0.073, max 0.530, mean 0.243), but the appearance of ‘tripod’ (d) captured their attention in the first part of Climax.
The SD was low and stable compared with other cases (min 0.064, max 0.332, mean 0.162), and the tendency line goes lower as time goes by. Including the moment when the tripod was coming out from the building (d), their attention follows the moving stimulus as expected.
Although the HIPV group had only 12 participants, the results of eye gaze analysis in the climax (
Figure 7) significantly demonstrates that their attention was very close and focused. In
Figure 8 and
Figure 9, it is shown that the female group’s concentration goes high along the story, but the male group’s attention does not. The female group has a more stable attention to the story overall.
Along with the SD graphs that show the different levels of attention between the user types and genders, the elements that contributed to the participants’ attention were also investigated. The following is an analysis based on the rate of eyeball collisions for each type of LIPV, HIPV, male and female in the scenes, Rising Action and Climax. Particularly, Rising Action and Climax are where the most information is delivered to the user in many media contents.
In Rising Action, one of the closest soldiers, tagged with ‘back,’ stopped and saw the consequence of a fallen meteorite as shown by most of the users in (a) and (b). When the soldier tagged with ‘back’ jumped into the scene and stopped to see what was happening at the 1150 frame in
Figure 6,
Figure 7,
Figure 8 and
Figure 9, it gained a lot of attention from the HIPV and female groups as shown in (c). At frame 2300, all participants’ eyes were focused on the ‘tripod’ which is the alien that comes out in front (d). In Climax, the participants’ eyes were getting busier almost four times of the usual movement of the user’s focus for the object that newly appeared in the field of view in (a) and (b). There is a considerable gap between the most focused object and others in Climax. The female group’s concentration shown in
Figure 9 is remarkable in both the SD graph and the collision detection from the ray of eye-gaze.
4.3. Enjoyment and Memory
To investigate and compare the level of enjoyment, subjective answers were collected from the question ‘(Post-Q.1) How much did you enjoy yourself during the experience?’ in the post-questionnaire. As
Figure 10 suggests, all four groups of user characteristics showed similar averages, and most of the participants answered higher than ‘4’, which corresponds to ‘Agree’ and ‘Strongly agree’, on the enjoyment level. The low immersion (LIPV) and concentration (Female) groups showed slightly higher levels of enjoyment. The male group had relatively low levels of enjoyment and high variations of SD values on each frame number.
Another question regarding the participants’ enjoyment was ‘(Post-Q.4) Did you find anything that you are interested in?’ The answers in
Figure 11 show the participants’ willingness to remain engaged in the VR story. In particular, the HIPV and female groups had a strong willingness to remain engaged in the VR story.
The questions about memory were ‘(Post-Q.6) How many soldiers were in the scene? (exclude the player)’ and ‘(Post-Q.10) How many legs the alien has?’. (Post-Q.6) and (Post-Q.10) required simple memorization, and the soldiers and the aliens were the items that were seen the most.
Figure 12 shows that the overall rate of correct answers is higher in the HIPV and female groups. In the case of the male group, the rate of correct answers for (Post-Q.6) is low compared with the other groups, but the rate for (Post-Q.10) is higher than that of the female group.
Furthermore, we conducted a correlation analysis between enjoyment and memory. The results indicated no meaningful relationship between the variables (corr = −0.132).