Could You Evaluate Sounds in a Virtual Environment? Evaluation Components of Auditory Experience in a Metaverse Environment

Abstract

The study aims to develop an auditory experience evaluation questionnaire to improve metaverse environments’ presence and derive evaluation components considering the acoustic and auditory user experience (AUX) through a survey. After conducting a survey with a total of 232 participants, five evaluation components were extracted from auditory presence and AUX evaluation factors through principal component analysis (PCA) and reliability analysis (RA): ‘realistic auditory background’, ‘acoustic aesthetics’, ‘consideration of acoustic control and accessibility’, ‘auditory utility and minimalist design’, and ‘auditory consistency’. In particular, although AUX evaluation factors such as ‘ease of access to sound control’ have limitations in improving the sense of presence, negative factors of presence such as ‘distraction due to sound’ can be improved by utilizing AUX evaluation factors, so it is judged that the sense of presence in the metaverse environments can be improved by enhancing the auditory sense of presence and AUX evaluation factors according to the composition of the five evaluation components derived in the study. The study can be used as a basis for developing an auditory experience evaluation questionnaire for the metaverse platform, creating sound design guidelines, and identifying sound development priorities.

1. Introduction

Metaverse technology is expected to bring many innovations to society, culture, and the economy by providing realistic services in various fields such as manufacturing, construction, education, healthcare, and social networking while suppressing time and space constraints [1,2,3,4]. Especially in metaverse fields such as education, gaming, and healthcare, research on presence and immersion has been actively conducted to provide more realistic experiences, and such presence is an essential element of a metaverse environment [5,6,7,8,9,10]. There is an ongoing discussion on how to integrate sensory information such as the visual and auditory information of users in realizing presence, and up to now, auditory factors have been used mainly as a supplement to visual information. However, recently, the realization of enhanced presence has become an important task, and the realization of a lifelike auditory environment is becoming important [11]. In general, questionnaire evaluation methods such as the presence questionnaire (PQ) and the Slater–Usoh–Steed presence questionnaire (SUS-PQ) are used to identify presence, and recently, auditory factors have been treated as a major factor to improve presence, and the need to evaluate them has emerged [12,13].

Existing questionnaires for evaluating presence focus on visual presence and cover various factors such as emotion, ease of use, and familiarity arising from the user’s sensory interaction with the avatar and the system, but there is little reflection of auditory factors [14,15,16]. In particular, considering that the sense of presence is evaluated through factors such as the illusion created by the user’s sensory information, the cumulative user experience (UX) of using the system, and the experience formed after using the system, it seems that it can be used if evaluation factors related to the sense of presence are added to the existing UX evaluation factors [10,17,18]; however, the existing UX evaluation contents have a limitation: it is difficult to evaluate the auditory part due to its general characteristics [19].

Against this background, since UX evaluation factors affect the understanding of presence, and presence evaluation factors act to understand UX [17,18], extracting common components of AUX evaluation factors and auditory presence evaluation factors may help to identify auditory experience evaluation factors considering presence enhancement in metaverse environments. Therefore, the study aims to develop an auditory experience evaluation questionnaire considering presence by conducting a literature review and primary and secondary expert interviews to improve the presence of the metaverse environment, and to derive the evaluation components considering auditory presence and AUX through questionnaires, PCA, and RA.

2. Literature Review

2.1. Presence and Auditory Characteristics of a Metaverse Environment

An aspect that should be considered to provide a more realistic experience in the field of the metaverse is the sense of presence, which is considered an essential prerequisite for visual, tactile, and auditory factors [12]. The sense of presence considered in metaverse environments is related to sensations such as perceiving oneself as existing in a virtual space [20] or being transported to a virtual space [21], and the feeling of ‘being there’, i.e., the subjective experience of the receiver focusing on and immersing themselves in the virtual world [22]. In metaverse environments, presence is also related to the perception of the user and others with whom they interact, including the experience of perceiving virtual objects as if they were real [23], the degree to which the user identifies with the avatar’s body [24], the degree to which they feel they are interacting with others [25,26], and the degree to which the recipient attributes personality to virtual humans, making them perceive them as social actors [27,28]. Therefore, the sense of presence can be categorized as the feeling of being transferred and immersed in a virtual world [20,21,22], the feeling of similarity to a real person [23,25], and the feeling of similarity in social behavior [25,26,27,28].

Table 1. Definition and classification of presence by researchers.

Researchers	Definition of Presence	Classification of Presence
Kim et al. (1997) [20]	Subjective sensations, such as being transported to a virtual space	A sense of transference and immersion in the virtual world
Heeter (1992) [21]
Witmer et al. (1998) [22]	The cognitive process of focus and engagement and the subjective experience of the receiver
Rizzo et al. (1998) [23]	How much the avatar’s body identifies with the user	Similarity to real people
Slater et al. (1997) [24]	The experience of feeling that a person exists in a virtual environment
Schubert et al. (2001) [25]	How much you feel connected to others	Similarity to social behavior
Short et al. (1976) [26]
Lemish et al. (1982) [27]	Recognizing virtual humans as social actors
Lombard et al. (1997) [28]

shows the definitions and classification of presence by researchers.

In a metaverse environment, presence is interrelated with spatial, temporal, and acoustic properties related to hearing. Spatial characteristics refer to where the sound is coming from, temporal characteristics refer to when the sound is heard, and acoustic characteristics refer to the pitch or intensity of the sound. These characteristics are based on the concept of the binaural effect, in which the brain analyzes factors such as the time difference between the sound source and the sound pressure difference between the two ears to determine where the sound is coming from within the sound field. This means that there are differences in the spatial perception of sound due to the diffraction, reflection, and absorption of sound from the sound source to the ear and body characteristics. Based on this, there are three main criteria for determining the location of a sound source in three-dimensional space: the time difference between the two ears (ITD, interaural time difference), the level difference between the two ears (ILD, interaural level difference), and the frequency characteristics (spectral cue) [29,30,31].

The auditory evaluation factors of a metaverse environment can be derived by identifying the auditory factors that make the metaverse environment feel real [21]. In addition, since the equipment elements of the metaverse environment, subjective factors considering the user’s personality characteristics and immersion tendencies, social factors arising from interactions with users, emotional factors, movement in the virtual environment, and familiarity can affect the sense of presence of the metaverse environment [16,32], the UX factors of existing accumulated digital devices, including the newly emerging metaverse environment, can be considered together [18]. Therefore, it is desirable to consider the auditory factors of the metaverse environment together with the AUX evaluation factors of digital devices that are familiar to the user and the auditory presence evaluation factors of the newly emerging metaverse environment.

2.2. Auditory Evaluation Factors in the Presence Evaluation Questionnaire for a Metaverse Environment

Since most of the evaluations of auditory presence in metaverse environments are technical evaluations of engineering sensors, and it is difficult to find data in the form of questionnaires for auditory presence evaluation, a literature review of auditory evaluation factors was first conducted in general presence evaluation questionnaires. In order of development date, the reviewed questionnaires include the presence questionnaire (PQ), the Slater–Usoh–Steed presence questionnaire (SUS-PQ), the reality judgment presence questionnaire (RJPQ), the igroup presence questionnaire (IPQ), the Independent Television Commission sense of presence inventory (ITC-SOPI), the engagement, enjoyment and immersion questionnaire (E2IQ), and the measurement, effects, conditions spatial presence questionnaire (MEC-SPQ).

PQ categorizes presence into three characteristics: involvement, immersion, and presence, and emphasizes sensory stimulation, participation, immersion, and interaction. The auditory presence evaluation factors include QN 6 (How much did the auditory aspects of the environment involve you?), QN 15 (How well could you identify sounds?), and QN 16 (How well could you localize sounds?), which are all about presence [22]. SUS-PQ assesses the sense of presence, which is the degree to which a person feels that he or she exists in a virtual environment rather than in an actual physical space. In particular, the degree to which a person is convinced that fictional information is real is viewed as presence, and it measures the degree of presence caused by the relevance of the user and the avatar. SUS-PQ consists of five evaluation factors: the experience and degree of ‘being there’, a comparison of virtual and real images, the association of the visual structure of the virtual environment, and the frequency of detecting virtual reality [32,33].

RJPQ categorizes presence into nine dimensions: reality judgment, presence, involvement, interaction, control, attention, realism, perceptual congruence, and expectations in virtual environment, and emphasizes natural interaction. The auditory presence dimension consists of three questions: QN 4 (How clear were the sounds in the virtual world?) on realism, QN 73 (To what extent did what you hear and the quality of the sound in the virtual world influence how real the experience seemed to you?) on reality judgment, and QN 76 (To what extent did the sounds influence how deep into the virtual world you went?) on presence [34]. IPQ categorizes presence into three factors: spatial presence, involvement, and realism, and covers a wide range of subjective experiences, immersion, and interaction, with no questions related to hearing [25]. ITC-SOPI categorizes the sense of presence into four factors: sense of physical space, engagement, ecological validity, and negative effects, and finds that not only the implementation of the physical environment, but also the overall usability, such as the attractiveness, naturalness, and trustworthiness of the content, could affect the sense of presence, and there are no questions related to hearing [35].

E2IQ categorizes the sense of presence into six factors: the sensory factor, distraction factor, realism factor, control factor, pleasure factor, and satisfaction factor, and considers various UX aspects such as the attractiveness of the visual scene, the degree of distraction caused by noise outside the device, the realism and consistency of the sensation of virtual movement, and the immersion of the task. The auditory dimension is covered by QN 2 (To what extent did events such as noise occurring outside Crayolaland distract your attention from Crayolaland?), which is about distraction [36]. MEC-SPQ approaches the evaluation of presence from a UX-centered perspective rather than a technology-centered perspective, and its contents are important for redefining spatial presence, which has not been systematically addressed in existing studies. In the MEC-SPQ, there are questions that consider the user’s state, behavior, and sense of presence, and, in particular, the questionnaire on the sense of presence consists of questions about the sense of location of the user and the objects surrounding them, and questions about whether the impression (feeling) received in the virtual environment allows the user to control their behavior [37]. As shown above, the evaluation factors of auditory presence in the questionnaire for evaluating presence in metaverse environments were examined and are shown in

Table 2. Characteristics of presence evaluation questions and auditory presence evaluation factors in metaverse environments by evaluation questionnaire.

Type of Questionnaire	Characteristics of Presence Evaluation Questions	Auditory Presence Evaluation Factors	Classification of Presence
PQ [22]	Sensory stimulation, engagement, immersion, and interaction	Whether you focus on content due to hearing (QN 6)	Presence
		Identification of sounds (QN 15)
		Location of the sound recognized (QN 16)
SUS-PQ [32]	Comfort, naturalness, and immersion	No auditory presence factor
RJPQ [34]	Natural interactions	Auditory clarity (QN 4)	Realism
		Similarity to real-world sounds (QN 73)	Reality judgment
		Focus on content due to hearing (QN 76)	Presence
IPQ [25]	Subjective experience, immersion, and interaction	No auditory presence factor
ITC-SOPI [35]	Appeal, naturalness, and credibility	No auditory presence factor
E2IQ [36]	Attractiveness, distraction, realism, consistency and immersion	Distractions from noise outside the metaverse environments (QN 2)	Distraction
MEC-SPQ [37]	User state, behavior, and presence	No auditory presence factor

.

As shown in Table 2, the characteristics of the presence questionnaires include sensory stimulation, participation, immersion, interaction, attractiveness, distraction, and consistency, but there is very little information about auditory presence, or it plays a supporting role to visual aspects. This suggests that sensory realization technology is becoming important for improving the presence of metaverse environments, and given the literature review indicating that auditory factors contribute significantly to presence, especially with the development of audio technology, auditory factors should be expanded to evaluate sensory stimulation, participation, immersion, interaction, attractiveness, and consistency, which can affect presence [13,14,15]. In addition, it is difficult to evaluate the auditory features of the interface (keyboard, mouse, display, etc.) of a metaverse platform that is familiar to real users, as the existing perceived presence evaluation questionnaires consider VR devices such as head-mounted display interfaces. In other words, VR devices are the iconic interface of the metaverse environment, but the most popular metaverse platforms used by real users are ‘ranking 1, Roblox’, ‘ranking 2, Minecraft’, and ‘ranking 3, Fortnite’, which utilize existing interfaces [38]. Considering the utilitarian aspect of the auditory presence evaluation factors, it is believed that a presence study based on the interfaces of the most popular metaverse platforms used by current users should be conducted first, rather than a presence study on VR interfaces. Therefore, the study was conducted to identify auditory factors that can enhance the sense of presence in a metaverse environment, to componentize auditory evaluation factors for auditory presence and AUX, and to determine the relationship between auditory evaluation factors and the enhancement of presence.

2.3. Prior Research on Auditory Presence in a Metaverse Environment

In addition to the implementation of acoustic technologies such as soundscapes that embody real-world sounds [39,40,41,42,43,44], studies on the quality of sound sources [39,44], studies on the stereophonics of sounds [39,44], studies on the spatial characteristics of sounds [43,44,45], studies on auditory cues that help with navigation tasks [44], and studies on auditory factors that match visuals [40,42] and factors such as coherence between sounds [14] have been identified as influencing the sense of presence.

Table 3. Auditory presence evaluation factors for metaverse environments identified by researchers.

Researchers	Contents of the Study	Auditory Presence Evaluation Factors	Classification of Presence
Hirose (1993) [39]	Theory and technology on the five senses that enhance the sense of reality in the metaverse environment	Implementing real-world sounds	Presence
		Creating a three-dimensional space for sound
		Sound source quality
Serafin et al. (2004) [40]	The function and role of soundscapes in a metaverse environment	Delivering the sound that you expect visually	Presence
		Auditory experiences that match motion but do not loop
		Recreating soundscapes
Baharin et al. (2010) [41]	How everyday sounds affect the sense of presence in a metaverse home environment	Everyday soundscapes	Social presence
Larsson et al. (2010) [14]	A conceptual framework for the relationship between sound and immersion, illusions of place, illusions of plausibility, and virtual body ownership in a metaverse environment	Spatial properties of sound	Presence
		Presence with and without auditory backgrounds
		Consistency between sounds
		Sound quality
Lee et al. (2017) [42]	How sound matches the visual experience in a metaverse environment to create a sense of presence	Three-dimensional representation of sound	Immersion
		Ambience sounds to match the visual experience	Presence, immersion
Hendrix et al. (1995) [43]	Elements of presence related to sight and sound in a metaverse environment	Spatialized sounds	Presence
Hendrix et al. (1996) [44]	The role of visual and auditory cues in perceived presence in a metaverse environment	Spatialized sound sources	Realism
		Auditory cues to help you navigate	Presence
Kiridoshi et al. (2022) [45]	How auditory spatial information affects users in a metaverse environment	Binary audio (spatial acoustics)	Social presence

shows research examples of the auditory presence evaluation factors in metaverse environments identified by researchers.

As shown in Table 3, the research cases of auditory presence evaluation factors in metaverse environments presented that even the same kind of auditory presence evaluation factors were classified as different presence factors by individual researchers. In the case of ‘spatial acoustics’, some studies classified presence as presence [39,43,44], while others classified presence as immersion [42], presence [44], and social presence [45]. Studies on ‘soundscapes’ classify presence as presence [40] and social presence [41], and studies on ‘auditory factors related to visual experience’ classify presence as presence [40] and presence and immersion [42]. In other words, auditory presence factors have not yet been systematically organized as they were classified differently by different researchers. This result is consistent with a literature review that found it difficult to generalize about auditory presence [12]. Against this background, the study focuses on auditory presence, which is the most basic and essential element of virtual worlds, i.e., the feeling of ‘being there’, and the auditory sensation such as the transfer of sounds from the physical world to the virtual space, among various concepts of presence.

2.4. Reflections on the Evaluation Factors of AUX for Products and Services

UX is a branch of interaction research in human computer interaction (HCI) that refers to the total experience, including usability and emotions, experienced in interactions with products, systems, and services [46,47]. AUX factors are specifically about the auditory aspects of UX and include musical elements (melody, rhythm, harmony, timbre, etc.), auditory user interface (AUI) such as notifications that occur while using products and services; auditory backgrounds such as ambient sounds and noises, and voice user interfaces (VUI) that assist in using the product [48]. AUX is also considered very important in the sense that it provides differentiated emotions and usability through hearing and imprints a company’s identity, and is often described as the counterpart of the graphic user interface (GUI) [49].

Since the evaluation of UX is characterized by avoiding formalized formats and reconfiguring them to fit the unique features of the target audience [50], AUX factors can be described as evaluating the usability and emotions that users experience in auditory interactions with products, systems, and services [51]. However, research on auditory UX factors does not have the same standard as the existing visual UX evaluation, ‘heuristic evaluation of user interfaces’ [52], so a literature review was conducted to identify auditory UX factors. The literature review includes a study on deriving a meaningful auditory attractiveness measure for UX [53], a study on proposing an auditory interface UX scale [19], a study on evaluating acoustic measurements and noise abatement in general [54], a study on delivering consistent and differentiated experience and value in products and services [55], a study on AUX in UX evaluation of glasses-type hearable products [56], and an expert interview on designing AUX in a web or app environment [57]. From these results, the existing studies on AUX evaluation factors were mainly distributed in products and services, and AUX evaluation factors were derived according to the researchers, as shown in

Table 4. AUX factors identified by researchers.

Researchers	Research Summary	Evaluation Factors for AUX	Research Categories
Boos et al. (2017) [53]	Deriving meaningful auditory attractiveness measures from UX	Loudness of sound	AUX Scale
		Unpleasantness of hearing
		Degree of sound echoing
		Degree of sound softness
Tomlinson et al. (2018) [19]	Auditory interface proposal of UX scale (interpretation, meaning and enjoyment)	How much the sound helps	AUI Scale
		How interesting the sound is
		How good the sound is
		How easy it is to understand the sound
		Relevance of sound and ideas
		Matching sound and meaning
		How difficult it is to understand the sound being varied
		How pleasant the sound is to listen to
		How boring it is to hear the sound
		How confusing it is to hear the sound
		How easy it is to understand what the sound represents
Aletta et al. (2019) [54]	Acoustic measurements and noise abatement in general using ISO standard (Method B of the ISO/TS 12913-2:2018)	Loudness of sound	Soundscape evaluation
		Unpleasantness of sound
		Degree of harmony between surroundings and sound
		How likely you are to stay again
Namkung (2019) [55]	Delivering consistent and differentiated experiences and value across products and services	Consistency between sounds within products and services	Sound identity structural diagram
Seok et al. (2020) [56]	AUX evaluation factors for hearable type of glasses	Ease of access to sound control	AUX evaluation
		The right amount of sound resolution
		How clear the sound is
		How much is enough to resize the volume
		How rich the range of the sound is
Blackburn et al. (2023) [57]	Interviewing an expert for designing AUX design in web or app environments	Accessibility for those with sensory sensitivities	AUX design guideline
		Blending the organic and digital
		Keeping sounds short and smooth
		Attracting attention without being distracting

.

As shown in Table 4, a literature review of AUX evaluation factors shows that they deal with the accessibility and intuitiveness of the UX, such as ‘ease of access to sound control’ [56] and ‘how easy it is to understand what the sound represents’ [19], while the characteristics of auditory presence evaluation factors differ in that they aim to realize the sound of reality, such as the ‘spatial properties of sound’ [14,43,44,45], ‘similarity to real-world sounds’ [34], and ‘implementing real-world sounds’ [39].

3. Methods

In the study, three steps were conducted in addition to the literature review to derive auditory experience evaluation components in a metaverse environment. In Step 1, auditory factors were grouped through primary expert interviews; in Step 2, an auditory experience evaluation questionnaire that considers presence was created so that it is easy for the public to understand through secondary expert interviews; and in Step 3, surveys and statistical analysis were conducted. The primary and secondary expert interviews were a qualitative rather than a quantitative evaluation, and thus were conducted with three and five experts, respectively, as it was suggested that a larger number of people could add to the confusion. The overall research process is shown in Figure 1.

3.1. Step 1: Primary Expert Interviews for Grouping Auditory Evaluation Factors

The primary expert interviews were conducted to group the auditory evaluation factors derived from the literature review. The participants in the primary expert interviews were selected as experts with a level of understanding of the existing literature review and at least three years of experience in the field of sound design (sound design, planning, composition, etc.) in a metaverse environment. The primary expert interview method was conducted in the form of listening to the opinions of experts using an Excel sheet; the participant information is shown in

Table 5. Participant information from Step 1.

Participants	Gender	Age	Work Experience	Activity Industry	Profession/Occupation
P1	Male	45	15 years	About AUX university labs	Professor and sound designer
P2	Male	26	5 years	Game company	Sound designer
P3	Male	30	10 years	Entertainment company	Service planning and sound designer

.

3.2. Step 2: Secondary Expert Interviews to Create a Questionnaire to Evaluate Auditory Experiences for Presence

The purpose of the secondary expert interviews was to define auditory presence in a way that the public can understand, and to create an evaluation questionnaire that the public can also easily understand.

Experts for the secondary expert interviews were recruited with at least three years of experience in the sound design field and at least three years of experience in the music education and music therapy field to help explain music terminology to the public as shown in

Table 6. Participant information from Step 2.

Participants	Gender	Age	Work History	Activity Industry	Profession/Occupation
P4	Male	31	6 years	Doctoral graduate students	Music educator and sound designer
P5	Female	33	3 years	Music therapy practice in a university hospital	Music therapist and composer
P6	Male	32	7 years	Game company	Sound designer
P7	Female	31	10 years	Healthcare company	Composer and sound designer
P8	Male	26	3 years	Consumer electronics company	Composer and sound designer

. This multidisciplinary approach to expert recruitment was based on research showing that a panel of experts with diverse perspectives can help improve deliverables [58].

3.3. Step 3: Survey and Statistical Analysis

To derive an evaluation component of auditory experience that takes into account the presence of the metaverse environment, a survey was conducted using a five-point Likert scale on the auditory factors that enhance the presence of the metaverse environment. The participants were recruited using the following criteria: they had used a VR device at least once to understand their familiarity with the virtual world, played games and metaverse platforms at least once or twice a week, and listened to music or media content at least two or three times a week to be able to discriminate auditory stimuli. The total number of participants was 232 (139 males and 93 females), with 67 participants aged 10–19, 60 participants aged 20–29, 60 participants aged 30–39, 25 participants aged 40–49, and 20 participants aged 50–59, with 80.6% of participants aged 10–39. The survey was also subjected to PCA to reduce the data with as little loss of information as possible. The PCA was performed in IBM SPSS^® Statistics, with a varimax factor rotation, and an RA was performed on the derived values to confirm the appropriateness of the newly derived evaluation components of auditory experience.

4. Results and Discussion

4.1. Results of Grouping for Auditory Evaluation Factors

From the primary expert interviews, the auditory evaluation factors identified in the literature review were grouped and are presented in

Table 7. Grouping of auditory evaluation factors from primary expert interviews.

Grouped/ Non-Grouped	Grouping within an Auditory Presence Factor	Grouping within AUX Factors	Grouping between Auditory Presence and AUX
Grouped factors	Focusing the metaverse environment with sound	-	-
	Implementing the spatial properties of sound
	Implementing realistic soundscapes in a metaverse environment
	Three-dimensional perception of sound
	-	Willingness to return due to sound
		-	The quality of the sound source
			How distracting the sound is
			Clarity of sound
			Sound experiences that live up to visual expectations
			How consistent the overall sound is in a sound variation situation
			Auditory cues to help you navigate
Non-grouped factors	Sound to match visual motion timing *	-	-
	Non-looping auditory experiences *
	-	Ease of access to sound control
		Relating sounds to ideas
		Harmonize between digital and natural sounds
		Accessibility for those with sensory sensitivities
		Distribution of the sound range
		Sufficiency of volume size control

* ‘Auditory experience matched to motion but not looped’ [34] was split into two factors by experts P1 and P2.

.

As shown in Table 7, the factors of ‘the concentration of contents due to hearing’ [22], and ‘presence with or without auditory backgrounds’ [34] were grouped as ‘focusing the metaverse environment with sound’ because they are classified as the same concept of ‘sense of presence’ in the reviewed literature [32,34,44]. The factors of ‘the spatial characteristics of sound’ [14], ‘spatialized sound’ [43], ‘spatialized sound source’ [44], and binary audio (spatial acoustics) [45] were grouped into ‘implementing the spatial properties of sound’ because they use the common concept of ‘space’ according to P1’s opinion. ‘Similarity to real-world sounds [34], ‘realization of real sounds’ [39], ‘reproduction of soundscapes’ [40], and ‘everyday soundscapes’ [41] were grouped as ‘implementing realistic soundscapes in a metaverse environment’ because they use the common auditory concept of ‘real soundscapes’ according to P1’s opinion. ‘Perceived location factor of sound’ [22], three-dimensional spatial representation of sound [39], and ‘three-dimensional representation of sound’ [42] were grouped into ‘three-dimensional perception of sound’ because they use the common concept of ‘location’ according to P2’s opinion.

The factors of ‘good sound’ [19], ‘interesting sound’ [19], ‘boring sound’ [19], ‘unpleasant sound’ [53], ‘wanting to stay again’ [54], and ‘unpleasant sound’ [54] were grouped under the common term ‘willingness to return due to sound’ because of their common meaning of measuring the impression of sound, according to all expert opinions. The factors of ‘quality of sound’ [14], ‘sound source quality’ [39], ‘degree of sound echoing’ [53] ‘degree of sound softness’ [53], ‘loudness of sound’ [53,54], and ‘keeping sounds short and smooth’ [57] were grouped into ‘the quality of the sound source’ because they use the common auditory concept of ‘quality of sound’ according to P2’s opinion. The factors of ‘how confusing it is to hear the sound’ [19], ‘distractions from noise outside the metaverse environment’ [36], and the ‘attracting attention without being distracting’ [57] can be grouped into ‘how distracting the sound is’ as they use the common auditory concept of ‘distraction’ according to P1 and P3’s opinions. The factors of ‘auditory clarity’ [34], ‘the right amount of sound resolution’ [56], and ‘how clear the sound is’ [56] can be grouped into ‘clarity of sound’ because they use the common auditory concept of ‘resolution of sound’ in P2 and P3’s opinions. The factor of ‘delivering the sound you expect visually’ [40], ‘ambience sounds to match the visual experience’ [44], and ‘degree of harmony between surroundings and sound’ [54] could be grouped together as ‘sound experiences that live up to visual expectations’ as they use a common auditory concept in the sense that ‘sight and sound should match’, according to P3’s opinion.

The factors of ‘consistency between sounds’ [14], ‘how difficult it is to understand the sound being varied’ [19], and ‘consistency between sounds within products and services’ [55] are believed to use a common auditory concept of ‘ how consistent the overall sound is in a sound variation situation’, and, thus, can be grouped into ‘the degree to which the overall sound is consistent in the context of sound variation’, in P3’s opinion. The factors of ‘how much the sound helps’ [19], ‘how easy it is to understand the sound’ [19], ‘how easy it is to understand what the sound represents’ [19], ‘matching sound and meaning’ [19], ‘identification of the sound’ [22], and ‘auditory cues to help you navigate’ [44] can be grouped under ‘auditory cues to help you navigate’ as they share a common auditory concept of the role of sound in metaverse environments as an alternative to data navigation and visual interfaces based on a literature review [19]. As mentioned above, primary expert interviews were conducted and it was found that the grouping within the auditory presence evaluation factor was dominated by the tendency to realize a realistic auditory experience, while the grouping within the AUX evaluation factor was dominated by acoustic characteristics, such as the sufficiency of volume size control, the distribution of the sound range, and the ease of access to sound control. In the case of grouping between the auditory presence and AUX evaluation factors, the factors covered by the existing AUX evaluation factors, such as ‘auditory utility to help navigation’, were also covered by auditory presence, confirming that the scope of auditory presence can be expanded.

4.2. Creating an Auditory Experience Evaluation Questionnaire That Considers the Public Understanding of Presence

Secondary expert interviews were conducted to explain the auditory presence and AUX evaluation factors in the public’s terms, and to create evaluation questionnaires that make it easier for the public to understand presence. As a result of the secondary expert interviews, the definition of auditory presence in a metaverse environment is shown in

Table 8. Questions and expert responses to the definition of auditory presence.

Interview Question	Participants	Expert Answers
How can you define auditory presence in a metaverse environment?	P4	Auditory stimuli that make you feel ‘real in space, time, and emotion in a virtual space’.
	P5	Make it feel like a real-world auditory experience.
	P6	Auditory stimuli in the virtual world that reflect the sounds or emotionally evocative elements that listeners expect in the real world.
	P7	To make a virtual environment sound and feel like it is real.
	P8	The emotional and socially relevant auditory experience of everyday life is embodied in a virtual world.

, the considerations for creating an evaluation questionnaire for auditory experiences with presence are shown in

Table 9. Questions and expert responses for creating an auditory experience evaluation questionnaire that considers the public understanding of presence.

Interview Question	Participants	Expert Answers
What should you consider when asking the public about auditory presence and AUX factors in a metaverse environment?	P4	Strongly open to subjective interpretation and should be explained with examples.
	P5	Avoid technical terminology.
	P6	Use simple, intuitive terminology.
	P7	Reflects the words used to express sounds verbally.
	P8	Organize your survey questions so that they are clearly separated and do not look like the same thing.

, and the terminology for the evaluation questionnaire is shown in

Table 10. Questions and expert responses for deriving questionnaire terms from auditory evaluating factors that consider the public understanding of presence.

Interview Question	Auditory Evaluation Factors	Expert Answers
		P4	P5	P6	P7	P8
Please create terminology for the public to understand auditory presence and AUX metrics in a metaverse environment.	Implementing the spatial properties of sound	Location	Sound environment	Sense of space	Spatial awareness	Realism
	Three-dimensional perception of sound	Distance	Object location	Distance	Location	Source
	Quality of the sound source	Broken	Bad	Torn	Sound quality	Reverberation
	Clarity of sound	Mushy	Clean	Cloudy	No response	Frustrating
	Focusing the metaverse environment with sound	Background music	Immersion	Background sounds	No response	Music
	How distracting the sound is	Varies	Complexity	Select a sound	Recklessness	Sound control
	Sounds that match visual motion timing	Matching	Gestures	Gaze	Movement	As it changes
	Sound experiences that live up to visual expectations	Matching	Guess	Blending with the background	Fits the mood	Expectations
	Non-looping auditory experiences	Naturalness	Regular	Constant	Repetitive	No response
	Implementing realistic soundscapes in a metaverse environment	Natural sounds	Routine sounds	Daily sounds	Realistic sounds	Household Noise
	Auditory cues to help you navigate	Environmental awareness	Situational judgment	Navigation help	When you need it	Intuitive
	How consistent the overall sound is in a sound variation situation	Consistent	Content identity	Change of Scene	Change of mood	Sound identity
	Ease of access to sound control	Volume sounds	Easy access	Volume icon	menu	Volume control
	Accessibility for those with sensory sensitivities	Adjusting sound effects	Acoustic control	Specific sounds manipulation	Desired sound	Sound texture
	Harmonize between digital and natural sounds	Artificial	Natural	Realistic	The actual	Created
	Relating sounds to ideas	Imagined	No response	Thought-related	Image	Semantics
	Distribution of the sound range	Contextualized	Suddenly getting bigger or smaller	No volume control	Negative pitch	All notes
	Sufficiency of volume size control	Adjust at will	Max volume	The desired as many	Volume control	Volume size
	Willingness to return due to sound	Memorable	No response	An impressive	Music, sound effects	Good memory

.

Based on the secondary expert interviews shown in Table 8, it was confirmed that the experts’ opinions on the question ‘How can you define auditory presence in a metaverse environment?’ were generally described as relating to the auditory sensation of ‘being there’, i.e., the transfer of sounds from the physical world to a virtual space, which is consistent with the theories advocated in the literature review [20,21], and these were considered when creating the presence evaluation questionnaire.

As shown in Table 9, it was confirmed that the auditory presence and AUX factors are subject to subjective interpretation and should be accompanied by examples, and that the examples should be described in as intuitive and everyday terms as possible, rather than in technical terms.

As shown in Table 10, it was found that existing terms such as ‘three-dimensional’ can be concretized into ‘location and distance identification’, and terms such as ‘approach to sound control’ can be simplified to be understood by the public, such as ‘sound you want to reduce’. After completing the expert interviews in Step 1 and Step 2, the questionnaire for evaluating the auditory experience with a final sense of presence was compiled and is shown in

Table A1. Questionnaire to evaluate auditory experiences for presence according to primary and secondary expert interviews.

Auditory Evaluation Factors	QN	Questionnaire Contents for Evaluating Auditory Experiences
Implementing the spatial properties of sound	1	Being able to distinguish places in a virtual world based on sound alone. (e.g., you can tell if you’re in a library or a cave by the echoes of the sounds.)
Three-dimensional perception of sound	2	You can orient yourself in a virtual world by listening to sounds. (e.g., I can tell forward/backward/left/right/up/down by the sounds I hear)
Quality of the sound source	3	Less ‘bad sound’ in the virtual world (e.g., poor sound quality, cracking, tearing, reverberant, loud, harsh, etc.)
Clarity of sound	4	Sound is clear in the virtual world. (e.g., crisp sound, clean sound)
Focusing the metaverse environment with sound	5	Auditory backgrounds are immersive in virtual worlds. (e.g., background music, everyday noises, etc.)
How distracting the sound is	6	Different types of sounds can be turned on and off in the virtual world (on/off). (e.g., turn on and off notifications, button clicks, the other person’s voice, background music, and more)
Sound to match visual motion timing	7	The timing of motion and sound in the virtual world is correct. (e.g., in a hypothetical running situation, the sound of footsteps matches the motion of running)
Sound experiences that live up to visual expectations	8	The virtual world plays music that matches the background. (e.g., grandiose music on a grandiose background, cutesy music on a cutesy background, etc.)
Non-looping auditory experiences	9	Notice that the sound is looping in the virtual world. (e.g., when you notice the sound of waves repeating in an imaginary ocean.)
Implementing realistic soundscapes in a metaverse environment	10	The noise of everyday life is heard in the virtual world. (e.g., keyboard sounds, air conditioning blowing, copier sounds, etc. from a fictional office space.)
Auditory cues to help you navigate	11	There are sounds that can help you navigate in virtual worlds. (e.g., if you’re wandering around in a virtual world trying to find your school, look for the school in the direction of the bell.)
	12	Hear notifications in the virtual world and know what to do. (e.g., message notifications, alert notifications, quest notifications, etc. and know what they are by just hearing them)
How consistent the overall sound is in a sound variation situation	13	Just by listening to music or certain sounds in the virtual world, you can tell which company made it.
ease of access to sound control’	14	When you need to adjust the volume in the virtual world, you can easily find the volume icon.
Accessibility for those with sensory sensitivities	15	Adjust the effect of sounds in the virtual world (high, low, soft, ringing, etc.).
Harmonize between digital and natural sounds	16	Artificial and natural sounds blend together in the virtual world (e.g., artificial music blends with the natural sound of the wind as you run through a virtual forest).
Relating sounds to ideas	17	You can hear sounds and see images in the virtual world. (e.g., when you hear thunder, you think of rain.)
Distribution of the sound range	18	Sound is offensive in virtual worlds. (e.g., during a relaxing playthrough, the sound suddenly gets louder)
Sufficiency of volume size control	19	When you adjust the volume (sound) in the virtual world, you can adjust it as much as you want.
Willingness to return due to sound	20	The music and sounds of the virtual world are memorable and I want to play them again.

in the Appendix A.

4.3. Results of the Survey and Statistical Analysis

The Kaiser–Meyer–Olkin (KMO) and Bartlett’s test of sphericity indicators indicate the degree to which the correlation between variables is well explained by other variables, and low values suggest that the variables are not appropriate for PCA. The KMO value for the survey in the study is 0.830, and the probability of significance (p) is less than the significance level (0.001), so the model fit is high. RA shows that a Cronbach’s α value of 0.6 or higher is generally accepted as reliable, and the reliability of the study is very high with a total component of Cronbach’s α = 0.896; the Cronbach’s α of Component 1, Component 2, Component 3, Component 4, and Component 5 was 0.867, 0.832, 0.829, 0.903, and 0.744, respectively. In addition, the eigenvalue of the extracted components indicates the amount of variance in the component, with a value greater than 1 being significant, and all five components were found to be the main auditory components considered in the metaverse environment. Commonality values below 0.4 are subject to removal, but all of the components were above 0.5, so no components were removed. The results are shown in

Table 11. Results of principal component analysis.

KMO = 0.837, Bartlett’s Test: Approximate χ2 = 2615.319, Degrees of Freedom (df) = 190, p < 0.001
QN	Commonality	Components					Cronbach’s α
		1	2	3	4	5	Each	Overall
QN 1	0.721	0.794	−0.049	0.245	0.018	0.167	0.867	0.896
QN 2	0.716	0.763	0.353	0.073	0.017	0.053
QN 10	0.710	0.754	0.190	0.235	0.224	−0.014
QN 16	0.681	0.733	0.360	0.002	0.086	0.086
QN 5	0.664	0.691	0.134	0.262	0.162	0.269
QN 3	0.697	0.151	0.799	0.032	0.174	0.066	0.832
QN 4	0.689	0.169	0.761	0.128	0.092	0.237
QN 18	0.714	0.173	0.713	0.063	0.414	0.029
QN 20	0.625	0.263	0.695	0.155	0.175	0.136
QN 6	0.809	0.211	−0.034	0.869	0.024	0.091	0.829
QN 19	0.777	0.154	0.210	0.834	0.089	0.072
QN 15	0.650	0.255	−0.021	0.738	0.015	−0.199
QN 14	0.621	0.005	0.246	0.683	0.063	0.300
QN 11	0.814	0.077	0.160	0.023	0.884	0.027	0.903
QN 12	0.859	0.089	0.266	0.079	0.873	0.112
QN 9	0.809	0.149	0.166	0.056	0.861	0.118
QN 13	0.629	−0.025	0.035	−0.058	−0.073	0.786	0.744
QN 17	0.670	0.149	0.215	0.147	0.187	0.738
QN 8	0.660	0.368	0.125	0.087	0.182	0.684
QN 7	0.576	0.334	0.319	0.305	0.300	0.424
	Component Name	Realistic auditory background	Acoustic aesthetics	Consideration of acoustic control and accessibility	Auditory utility and minimalist design	Auditory consistency	−
Division
Eigenvalue		7.064	2.466	1.718	1.570	1.274
Variance (%)		35.320	12.330	8.590	7.848	6.371
Accum. (%)		35.320	47.650	56.240	64.087	70.458

Remarks: Analysis method—principal component analysis, rotation method—varimax with Kaiser normalization.

.

The results in Table 11 show that the cumulative variance of the five components, including ‘realistic auditory background’ and ‘acoustic aesthetics’, is 70.458%, which can explain more than 70% of the auditory experience of the metaverse environment, so it is believed that improving these components can improve the sense of presence. Component 1, which is explained by 35.320% (variance) of the total factors, consists of QNs 1, 2, 10, 5, and 16, and has the characteristics of ‘realistic auditory background implementation’ in that it considers the positional and spatial characteristics of sounds in the metaverse environment and the implementation of everyday soundscapes and auditory backgrounds (such as background music and everyday noises) that can evoke a place. In particular, considering that ‘Component 1’ is derived from the AUX factors ‘harmonization between digital and natural sounds’ (QN 16) and ‘auditory presence factors related to constructing a realistic sound environment’ (QNs 1, 2, 10, 5), it can be interpreted that the auditory environment of the metaverse should not be implemented as it is in reality, but should seek harmonization with existing digital sounds, which can enhance the sense of presence.

Component 2, described as 12.330% of the total factor, consists of QNs 3, 4, 18, and 20. Component 2 is characterized as ‘acoustic aesthetics’ in terms of fewer bad sounds, clearer sounds, auditory discomfort, and user revisits due to sounds in the metaverse environment. The reason why factors such as QNs 3, 4, 18, and 20 were derived as ‘Component 2’ is that the auditory presence factor (QNs 3, 4), which is related to acoustic quality, considers ‘willingness to return due to sound’ (QN 20), which tends to be a strong AUX evaluation factor These results suggest that users’ ‘willingness to return due to sound’ (QN 20) does not directly increase presence, but can function as a checklist for presence by ensuring that ‘Component 2’ has no issues with sound quality if metaverse use is low (QNs 3, 4). Comprising QNs 6, 19, 15, and 14, ‘Component 3’, which explains 8.590% of the total factor, can be described as ‘consideration of acoustic control and accessibility’ in terms of the granularity of sound volume and sound effect control in the metaverse environment and the ability to turn different kinds of sounds on and off. In particular, since ‘Component 3’ is the component with the most AUX evaluation factors, including ‘sufficiency of volume control’ (QN 19), ‘ease of access to sound control’ (QN 14), and the auditory presence evaluation factor ‘distraction from sound’ (QN 6), it is judged that the AUX evaluation factor itself does not improve the sense of presence, but the sense of presence can be improved by controlling ‘distraction from sound’ (QN 6) through acoustic control and accessibility considerations. Comprising QNs 11, 12, and 9, ‘Component 4’, explained as 7.848% of the total factor, is characterized by ‘auditory utility and minimalist design’ in the sense that AUI such as notifications and signals that help navigate in the metaverse environment should be designed in such a way that users do not notice the sound looping. In particular, ‘Component 4’ states that ‘signals that help with navigation’ (QNs 11, 12) are not just AUI, but auditory signals that are experienced in the real world (such as car horns), and if the auditory signals are mechanically repeated, the sense of presence is diminished. Therefore, the auditory design of the metaverse environment should be minimalist, with sounds that can be useful to the user, rather than importing all the sounds of the real world.

Comprising QNs 13, 8, 17, and 7, ‘Component 5’ explains 6.371% of the total factor and can be described as ‘auditory consistency’, considering the coherence of sound and visual motion and atmosphere in the metaverse environment, as well as the coherence of sound changes in the content. ‘Component 5’ is grouped into ‘sound matching visual motion timing’ (QN 7), ‘sound experience meeting visual expectations’ (QN 8), ‘relevance of sound to ideas’ (QN 17), and ‘degree of consistency of overall sound across sound variations’ (QN 13), which is a strong tendency of AUX evaluation factors, suggesting that the unified experience of sight and sound covered by traditional auditory presence should be harmonized with the auditory experience across the metaverse environments.

Based on these results, it was identified that five main components, including ‘realistic auditory background’ and ‘acoustic aesthetics’, explained more than 70% of the variance in auditory presence in metaverse environments. While previous studies have mainly focused on the spatial characteristics of auditory presence in metaverse environments or the congruence of auditory and visual atmospheres [36,37,38,39,40,41,42], the study focuses on the importance of AUX and its impact on presence. As expected from the above research questions, it was confirmed that auditory presence and AUX factors can enhance presence when they act complementarily. These findings suggest that users do not experience a sense of presence solely through the reproduction of the auditory environment, but that it can be enhanced by preventing factors that inhibit the sense of presence and amplifying acoustic quality through AUX factors. A limitation of the study is that it focuses on virtual worlds, which are the most popular metaverse platforms, and therefore does not consider aspects of the other three types of metaverses (augmented reality, lifelogging, and mirror worlds), especially auditory interaction through VR devices. As VR devices reflect the unique characteristics of metaverse environments, further research on the auditory presence of VR devices should be conducted, and such research will have a significant impact on improving the presence of metaverse environments in the future.

5. Conclusions

As the realistic experience of users in a metaverse environment becomes important, an evaluation questionnaire for auditory experience considering presence was developed to improve the sense of presence in the metaverse environment, and derived evaluation components considering auditory presence and AUX through a survey, PCA, and RA. As a result of grouping the auditory evaluation factors through primary expert interviews (Step 1), it was confirmed that the tendency to realize a realistic auditory experience was strong in the grouping within an auditory presence evaluation factor, and that the grouping within the AUX evaluation factor had strong acoustic characteristics such as ‘sufficiency of volume size control’, ‘distribution of the sound range’, and ‘ease of access to sound control’. In the case of grouping the auditory presence and AUX evaluation factors, it was found that the factors covered by the existing AUX evaluation factors, such as ‘auditory cues to help you navigate’, are also covered by auditory presence, confirming that the evaluation area of auditory presence can be expanded.

Through secondary expert interviews (Step 2), an evaluation questionnaire that can be easily understood by the public was created, and it was confirmed that the following should be included in an auditory experience evaluation questionnaire: consideration of auditory sensation such as the transfer of sounds from the physical world to the virtual space, attachment of auditory examples to the questionnaire, and use of intuitive and everyday terms. It was also identified that existing broad terms such as ‘three dimensions’ could be refined to ‘location and distance identification’ and that difficult terms such as ‘ease of access to sound control’ could be simplified to make them more understandable to the public, such as ‘sounds you want to reduce’. Among the five components extracted from the results of survey, PCA, and RA (Step 3), the eigenvalue for ‘realistic auditory background’ was the highest, so it is a prioritized evaluation component of auditory experience in the metaverse environments.

In particular, it was found that factors that hinder presence, such as ‘distraction due to sound’, which were addressed in auditory presence, can be improved through the AUX evaluation factor as they are componentized with the AUX evaluation factor ‘ease of access to sound control’. Therefore, by considering the relationship between auditory evaluation factors (auditory presence and AUX) and the improvement of presence in the metaverse environment, which is the research question of the study, it was confirmed that auditory presence and AUX evaluation factors are complementary, and that the presence of the metaverse environment can be improved if the contents of the five evaluation components devised in the study are applied.