Sounds of History: A Digital Twin Approach to Musical Heritage Preservation in Virtual Museums

Abstract

Musical cultural heritage, as an important component of cultural heritage, possesses significant cultural value and inheritance significance. With the development of society and the passage of time, these precious traditional musical cultural heritages inevitably face the dilemma of gradual depletion or even disappearance. In the digital age, effectively protecting and inheriting these musical cultural heritages has become an urgent problem to be addressed. Therefore, this paper proposes an application method based on digital twin technology, exploring how to protect and inherit musical cultural heritages through digital twin technology. By leveraging digital twin technology, a virtual museum dedicated to showcasing the richness and historical connotations of music cultures is created, preserving and simulating the soundscapes of historical music eras. Through the integration of audio archives, 3D modeling, and interactive displays, users can immerse themselves in the experience of historical music in the digital space. This paper evaluates the feasibility and cultural preservation value of this digital music history museum through the creation of music digital twin technology instances and user survey feedback and discusses the prospects of digital twins in the field of musical cultural heritage.

1. Introduction

Music, as an art form that transcends eras and cultures, carries profound historical and cultural value. Musical cultural heritage embodies a significant manifestation of human history and cultural diversity. It represents not merely an exhibition of art but also a crucial component of social and cultural identity. Musical cultural heritage encompasses, but is not limited to, traditional music, folk songs, ceremonial music, and related cultural expressions. As witnesses to history, these heritages record the evolution of human civilization and serve as vital channels for the transmission of cultural and societal values [1]. However, the preservation of musical cultural heritage faces numerous challenges. With the acceleration of globalization in politics, economy, culture, and changes in modern lifestyles, many traditional forms of music and expressions are gradually disappearing [2]. For instance, specific ethnic instruments and performing arts face the threat of extinction due to a lack of successors or audience. Additionally, the ongoing development of science and technology poses challenges to the preservation of musical cultural heritage, as the authenticity of music forms and traditional performance methods are impacted by modern media [3].

In this context, the protection and transmission of musical cultural heritage become increasingly critical. Due to its multifaceted information and diverse presentation methods, existing preservation approaches—such as manual recording, documentary collection, digital capture, and information documentation—often face limitations due to resources and technology, affecting their effectiveness. Therefore, in the digital age, finding effective ways to protect and transmit this heritage to ensure its comprehensive, in-depth understanding and appreciation by a broader audience is an urgent issue to address [4].

Digital twin technology [5], as an emerging digitalization technique and exhibition concept, offers a new perspective and potential for the preservation of musical cultural heritage [6]. By creating a virtual world as a novel platform for cultural dissemination, it presents musical cultural heritage in digital form to the public. The construction of digital twins for musical cultural heritage not only allows for more comprehensive recording and preservation of musical works, instruments, and methods of performance but also provides researchers and the public with an interactive and immersive experience [7]. This approach not only enhances public awareness and interest in musical cultural heritage, allowing for interaction and experience in digital spaces, but also offers new perspectives and tools for its preservation [8]. This has significant implications for the protection and dissemination of musical cultural heritage.

This paper introduces an innovative method that employs digital twin technology to preserve and disseminate musical cultural heritage within virtual museums. It aims to investigate and evaluate the practical application of digital twin technology in protecting musical cultural heritage within virtual museum settings. The paper delves into the capability of digital twin technology to faithfully replicate and reproduce musical cultural heritage in virtual museums, addressing specific applications, challenges encountered, and potential solutions.

To demonstrate the effectiveness of this method, this article takes Chinese Kunqu Opera as an example and attempts to use digital twins and other related metaverse technologies to build a virtual museum and virtual music interactive platform. The interactive platform is guided by a virtual character, “Kunkun”, which aids users in exploration and enhances their experience. Additionally, the platform integrates artificial intelligence technology, enabling users to participate in virtual creative activities.

Through a comprehensive analysis of user experience and dissemination effects, our approach not only provides an effective preservation of musical cultural heritage but also provides the public with a highly interactive and participatory experience. This innovative method offers a new perspective for the inheritance and protection of musical cultural heritage, as well as a valuable reference for the construction of the metaverse in the intangible cultural heritage of the music domain. Our work demonstrates the outstanding achievements of advanced technologies in the display and dissemination of cultural heritage, offering new insights and practical guidance for the preservation and inheritance of musical cultural heritage.

2. Related Research Work

2.1. Analysis on the Current Situation of Music Cultural Heritage Protection

Musical cultural heritage, broadly defined, encompasses all cultural and historical information conveyed through music. This includes not only various musical compositions, such as traditional songs, ethnic music, and religious music, but also related practices of performance, musical instruments, music production techniques, and the role of music in social and cultural activities. As a form of cultural expression, music is an integral part of human history and social structure, reflecting the diversity, creativity, and understanding of the world across human societies [9].

The importance of musical cultural heritage lies in its unique cultural and educational value. Firstly, music serves as a powerful medium for conveying and preserving history, culture, and traditions. Through music, individuals can learn about and experience the historical narratives, lifestyles, and values of different cultural backgrounds [10]. Moreover, musical cultural heritage is a significant marker of ethnic and regional cultural identities, fostering connections between individuals and communities and promoting an understanding and respect for cultural diversity [11]. However, many traditional musical forms have been neglected or forgotten, especially those transmitted orally. Thus, the protection of musical cultural heritage is not only about preserving the past but also about maintaining cultural diversity and promoting the sustainable development of future cultures [12].

The preservation of musical cultural heritage currently faces multifaceted challenges and limitations, stemming from changes in the external environment as well as the inherent limitations of preservation methods. In the process of globalization and modernization, traditional musical cultural heritage is at risk of marginalization and neglect. The rise of mainstream culture and commercial entertainment forms in modern society makes it difficult for traditional music to compete, reducing opportunities for its dissemination and performance and impacting the younger generation’s interest and transmission of traditional music [13]. The rapid development of technology has, to some extent, altered the ways in which music is produced and disseminated. Although modern technology has facilitated the recording and preservation of music, it may also affect the authenticity of musical expression. For example, music recorded through digital means might not fully capture the nuances and emotional expressions of live performances [14]. Many traditional musical forms rely on oral transmission, which, despite its personalization and flexibility, is extremely vulnerable. Once the number of transmitters decreases or breaks, the related musical cultural heritage may be lost permanently [15]. The protection and transmission of musical cultural heritage require corresponding resources and financial support, including training transmitters, organizing musical events, and recording and preserving music. However, in many cases, these activities have not received sufficient attention and financial support, especially in economically disadvantaged regions [16]. Currently, methods used for the preservation of musical cultural heritage are often limited to the physical recording of music and the collection of documentary materials. Although these methods have played a role in preserving musical compositions, they are insufficient in transmitting the skills and expressive methods of musical performance [17]. Given these challenges and limitations, exploring new preservation methods and technological applications for the protection of musical cultural heritage is particularly important and urgent.

2.2. Application Analysis of Digital Twin Technology

Digital twin technology, as an innovative information technology, has begun to be widely applied across various industries. The concept of digital twins was first introduced by the United States Air Force in 2002. Digital twin technology involves creating a high-fidelity virtual replica of a physical entity to simulate, monitor, and optimize the entity, providing support for decision making, performance optimization, future trend prediction, and necessary interventions [18].

Over time, with technological advancements, the diversity and flexibility of digital twin technology have led to its broad application across different fields. Particularly under the influence of Industry 4.0 and the Internet of Things (IoT), digital twins have become a key technology in manufacturing, urban planning, healthcare, and more [19]. In the manufacturing sector, digital twins are utilized to create virtual copies of products to optimize design, test performance, and predict maintenance needs [20]. In urban planning, the application of digital twins aids city administrators in more effectively monitoring and managing urban infrastructure [21]. In the medical field, digital twin technology is employed to simulate human organs or entire physiological systems, optimizing treatment plans and pre-operative procedures [22]. The energy industry uses digital twins to monitor and optimize the production and distribution of energy [23]. In recent years, with improvements in computing power and advancements in data analysis techniques, digital twin technology has become more efficient and precise in simulating complex systems and processes.

In the field of cultural heritage preservation, digital twin technology offers a novel perspective. By constructing accurate digital models of historical buildings, artworks, or even entire historical scenes, it not only provides rich resources for research and education but also scientific foundations and methods for the preservation and restoration of cultural heritage. Notably, digital twin technology can capture and reproduce the intricate features and historical contexts of cultural heritage, which is particularly important for protecting heritage that is vulnerable to environmental and temporal degradation. Furthermore, the application of digital twins in cultural heritage preservation extends beyond the replication and archiving of physical characteristics. By integrating virtual reality (VR), augmented reality (AR), and other interactive technologies, digital twins can offer the public immersive cultural experiences, allowing people to transcend the boundaries of time and space to directly experience and learn about different cultures. In this process, public participation and experience become active parts of cultural transmission, which is significant for enhancing societal recognition and awareness of the value of cultural heritage. Therefore, this paper aims to explore the potential and application of digital twin technology in the preservation of musical cultural heritage, particularly in the context of virtual museums. Through this research, we seek to reveal how digital twin technology can help us better understand, preserve, and transmit precious musical cultural heritage.

3. Music Cultural Heritage Protection Method Based on Digital Twin Technology

3.1. Analysis on the Application of Digital Twin Technology in the Protection of Musical Cultural Heritage

Digital twin technology, with its capacity to transcend geographical and physical barriers, offers a novel avenue for enhanced access and engagement with musical cultural heritage. By creating virtual spaces, this innovative approach renders a musical heritage that was once inaccessible readily available and provides platforms for public participation and interaction. Through the employment of virtual reality (VR) and augmented reality (AR) within the framework of digital twin technology, it is possible to present musical cultural heritage in an environment that is not only more immersive but also highly interactive [24,25]. Such an experiential paradigm shift not only augments the vividness and engagement of musical heritage but also substantially enriches the understanding of its cultural and historical context.

Digital twin technology introduces an avant-garde method for the education and exhibition of musical heritage [26,27]. Within these virtual confines, it is feasible to reconstruct musical scenes from diverse historical epochs, thereby enabling users to engage with and comprehend the history and culture of music in an interactive milieu. Furthermore, digital twin spaces facilitate the digital restoration and preservation of musical heritage, offering a lifeline to musical forms that are either endangered or on the brink of extinction [28]. This capability not only serves the purpose of safeguarding cultural heritage but also equips music scholars and researchers with innovative analytical tools.

Moreover, digital twin technology fosters a shared space for individuals from various cultural backgrounds, promoting exchange and comprehension amongst different musical heritages [29]. This cross-cultural dialogue and exchange are paramount for the advocacy of global cultural heritage protection and understanding. In light of the significant contributions of digital twin technology to the preservation of cultural heritage, the exploration of effective strategies for integrating musical cultural heritage within digital twin technology emerges as a compelling research topic. This inquiry not only promises to enrich the field of cultural preservation but also to enhance the global appreciation and understanding of musical heritage.

3.2. Design of Virtual Music Museum Based on Digital Twin Technology

The construction of a virtual music museum based on digital twin technology provides a feasible means for the effective preservation and inheritance of intangible world musical cultural heritage such as Chinese Peking opera, Kunqu opera, and Guqin art. This virtual museum aims to seamlessly integrate technologies such as virtual reality (VR), augmented reality (AR), blockchain, and artificial intelligence (AI) with the features of musical cultural heritage preservation and exhibition. The goal is to create a digital space that is not only educational but also entertaining. This paper realizes the vision of building a virtual Kunqu museum by constructing a Kunqu metaverse. Continuing this idea, we delve into the implementation and development of this virtual museum.

The Kunqu virtual universe is a digital representation of the rich cultural heritage of Kunqu, offering visitors an immersive interactive experience. Utilizing advanced digital twin technology, we have meticulously recreated historical venues, exquisite costumes, traditional music, and iconic performances within this virtual environment.

Furthermore, our approach transcends mere replication by integrating innovative features and functionalities. Users can explore the virtual museum at their own pace, interact with virtual guides, or access detailed information about each exhibit. Additionally, interactive elements such as virtual performances, workshops, and educational courses are integrated to attract visitors and enhance their understanding and appreciation of Kunqu.

Moreover, the virtual museum serves as a dynamic platform for cultural exchange and collaboration. Artists, scholars, and enthusiasts from around the world can come together to share their expertise, collaborate on research projects, and contribute to the ongoing protection and promotion of Kunqu. The technological framework is illustrated in Figure 1.

In constructing the Kunqu Metaverse virtual museum, nine modules collaboratively create an interactive and immersive cultural experience. Here, an analysis of each module’s role in building the virtual museum is given.

Entrance Module: Provides multiple platforms and interfaces for accessing the virtual museum, ensuring that diverse user groups can visit the museum using their chosen devices, such as smartphones, PCs, and VR equipment.

Ecological Application Module: Demonstrates how the Kunqu Metaverse can be integrated into everyday life through various mediums and applications such as gaming, education, and business. These applications promote Kunqu culture and increase user engagement.

Digital Twin Technology Development Direction: Serves as the technical foundation of the virtual museum. For instance, real-time simulation can dynamically display Kunqu performances, and cloud-based simulation supports broad access.

Original Elements Module: Constitutes the core content of the museum. These elements ensure a precise and thorough presentation of all aspects of Kunqu, from basic knowledge to in-depth content such as scripts, music, and performing arts.

Related Technology Module: Provides the tools needed to create and maintain the museum, such as digital archiving and cultural heritage modeling, which help preserve historical data and present it virtually to the public.

Interaction Module: Enhances visitor engagement with Kunqu through interactive learning experiences, such as games and educational activities, which enhance participation and knowledge transfer.

Digital Twin Management Module: Ensures effective operation and user experience of the virtual museum, covering aspects like data management, content delivery, and technical support.

Perceptual Interaction Technologies: Allow users to interact with museum exhibits through natural human–machine interaction methods, such as gesture control and voice commands.

Virtual Human Types: Features various forms of virtual guides or performers, from cartoon pets to high-fidelity robots, to increase the exhibitions’ entertainment and educational value.

The integration of VR and AR technologies in this museum provides an immersive experience, bringing visitors closer to the nuances of musical performances and traditions. Additionally, our use of AI and blockchain technologies offers innovative solutions for the cataloging, analysis, and secure sharing of cultural data. We have also designed a photorealistic virtual intelligent character, “Kunkun” (as shown in Figure 2; this picture was generated by ChatGPT 4.0 [30]), which significantly enhances the overall user experience and interactivity in the Kunqu virtual universe. “Kunkun” is not merely a digital avatar; it embodies AI technology to provide personalized guidance and assistance to visitors. Equipped with natural language processing capabilities, “Kunkun” can engage in real-time conversations, answer questions, provide historical context, and offer insights into various exhibits and performances. Moreover, “Kunkun’s” design is adjusted based on user preferences and interests. Through machine-learning algorithms, it can analyze user behavior and engagement patterns to tailor its recommendations and suggestions. For example, if a visitor shows particular interest in a specific period of Kunqu history or a type of performance, “Kunkun” can curate personalized tour itineraries or recommend related exhibits and content. Beyond guiding, “Kunkun” also facilitates interactive experiences within the virtual museum, initiating virtual performances or workshops that allow visitors to witness authentic Kunqu performances or engage in activities such as costume design or traditional music composition. Furthermore, as a bridge between the virtual and real worlds, “Kunkun” connects visitors to additional resources and opportunities beyond the digital realm, providing information about upcoming live performances, workshops, or exhibitions at physical locations and facilitating online interactions with artists, scholars, and other enthusiasts. By adopting this integrated technological framework, the virtual music museum not only protects intangible musical heritage but also enhances accessibility and understanding among global audiences. This initiative highlights the potential of digital twin technology as a transformative tool for cultural preservation, offering new avenues for appreciating and continuing the world’s musical heritage.

The development of a virtual museum primarily encompasses key technologies such as 3D modeling, audio archive integration, and user interface design. In June 2023, the Ministry of Culture and Tourism of the People’s Republic of China published the “ Digital safeguarding of the intangible cultural heritage—Digital resources collection and description (WH/T 99.5-2023)”, with Part 5 specifying clear standards for the digital resource collection and cataloging of traditional music within intangible cultural heritage [31] (Figure 3). This paper will fulfill the design for the collection and cataloging of digital resources for cultural heritage, based on the requirements set forth by these standards.

Three-dimensional Modeling Technology: Utilizes laser scanning, among other techniques, for the digital scanning of physical artifacts, capturing precise three-dimensional model data. Computer graphics technology is then employed to model and render the scanned data, creating lifelike virtual exhibits and spaces. Animation and interactive features are added to exhibits and scenes to enhance user experience and engagement, akin to the process for a classical guitar. For instance, Artec Studio could be used for laser scanning a real classical guitar to obtain high-precision 3D model data. These data are then imported into Blender for further modeling and editing to repair any potential defects or damage and to add more details and textures. Realistic exhibition scenes are created in virtual showrooms using Unity3D, placing the classical guitar appropriately. Finally, interactive features are added using Unity3D, such as enabling users to click on the guitar to access more information or play audio samples, thereby enhancing engagement and experience.

Audio Archive Integration Technology: Digital processing of actual musical artifacts’ audio materials includes recording, editing, and format conversion. An audio database is established for the integration, management, and retrieval of audio archives, facilitating their display and playback in the virtual museum. Audio playback engines or media player technologies are applied for smooth audio playback and control, using software like Adobe Audition (Version 2024 v24.4.0.45), Soundminer (Version V5PRO), Pro Tools (Version 2023.12), VLC Media Player (Version 3.0.18), and HTML5 Audio Player.

User Interface Design Technology: Designs an intuitive and user-friendly interface, including navigation menus, exhibit browsing, and information viewing functions, enabling easy operation and navigation by users. It integrates 3D models, audio, video, and other multimedia content within the user interface to provide rich display effects and interactive experiences. Responsive interface designs adapt to different devices and screen sizes, including PCs, tablets, and smartphones, meeting the access needs of users across various platforms. Application software includes Adobe XD (Version 56.0), Sketch (Version 100), Unity3D (Version 2023.1.4), HTML/CSS/JavaScript, Bootstrap (Version 5.3.0), and Media Queries.

Designing a virtual museum based on digital twin technology also requires adherence to a generalized methodology to ensure the museum’s design can adapt to diverse needs and scenarios. This includes establishing clear goals and audiences, designing for interactive experiences, and continuously improving and optimizing based on user feedback and technological advancements. This approach ensures the museum’s design remains competitive and appealing while fulfilling its core objectives of education and entertainment.

3.3. Construction and Implementation of Digital Music History Museum

This project, based on the China Kunqu Museum in Suzhou (Figure 4), China, aims to utilize digital twin technology to replicate the essence of the museum. It represents not only a reverence for traditional Kunqu art but also an innovative application of modern technology. In this virtual space, global audiences will have the opportunity to immerse themselves in the charm of Kunqu.

Digital twin technology provides robust support for this vision. It can accurately recreate every corner, exhibit, and historical story of the physical museum in the virtual world. Audiences can experience the museum’s unique charm through their screens without being physically present.

In this virtual space, tradition and modernity are intertwined. Audiences can not only appreciate the exquisite skills of traditional Kunqu performances but also gain a deeper understanding of the art’s connotations and historical background through interactive experiences and other modern technological means. This fusion not only makes traditional culture more vivid and interesting but also provides a new way for modern audiences to experience it.

In addition to attracting domestic audiences, this project can also capture the attention of global audiences. Regardless of their location, as long as they have an internet connection, anyone can easily access this virtual museum and intimately engage with Kunqu art. Such a globalized experience not only contributes to the dissemination of Chinese traditional culture but also promotes cultural exchange and understanding.

Overall, this project, based on digital twin technology, brings the China Kunqu Museum into a new digital era. It not only preserves the essence of traditional culture but also provides a unique cultural feast for global audiences, allowing the charm of Kunqu to spread worldwide.

The project construction begins with digital scanning and model reconstruction. High-precision scanning of the interior and exterior of the China Kunqu Museum in Suzhou is conducted using laser scanners and photogrammetry techniques to capture the details and structure of the architecture. Special attention is paid to intricate decorations such as carvings, murals, and ceramics to ensure their accurate representation in the digital version. Based on the scan data, the structure of the museum is reconstructed in Unity3D (Version 2023.1.4), and meticulous processing is applied to the models of the architecture and exhibits. Detail sculpting is carried out using tools like Zbrush (Version 2021.6) and Autodesk Maya (Version 2022.1) to ensure a high level of realism without sacrificing performance.

Virtual environment design and development involve referencing the original museum’s layout to design the flow of the virtual space, ensuring smooth navigation between exhibition areas while setting interactive points to guide users in exploring Kunqu. Interactive experiences are designed for exhibits and displays, such as allowing users to listen to audio narrations of Kunqu or watch virtual reenactments of performances by clicking. Characters modeled in stage two are integrated into the virtual environment, ensuring consistency in lighting and shadow effects between characters and the environment. The virtual museum is ensured to run on various devices including PCs, mobile devices, and VR headsets to cater to different user needs, as shown in Figure 5.

System testing and optimization involve inviting Kunqu enthusiasts, historians, and general audiences to conduct system tests, collecting feedback to optimize the tour experience and interface design. Issues identified during testing, including user interface, interaction logic, and performance issues, are rectified.

Upon ensuring all functionalities operate smoothly, the virtual museum is launched, and its access address is publicly announced, with promotion carried out on relevant platforms. Online support and documentation are provided to assist users in troubleshooting during their visits. Regular updates to exhibit content are conducted to maintain freshness and appeal while monitoring the system’s operation status and providing necessary technical support and maintenance.

Through this comprehensive process of construction and implementation, the virtualization of the Kunqu Digital Twin Museum is realized. It represents not only a replication of the physical museum but also a new cultural dissemination method, allowing people to transcend physical boundaries and experience the unique charm of Kunqu, an ancient Chinese art form.

3.4. Digital Twin Display of Kunqu Opera “The Peony Pavilion”

The project is based on a performance of “The Peony Pavilion” produced by the Shanghai Zhang Jun Kunqu Art Center at the Zhu Jiajiao Kezhi Garden, specifically focusing on the water-stage theater version (Figure 6). It aims to digitally replicate the performance scene to create a virtual performance environment, allowing global audiences to experience this unique Kunqu performance through digital platforms. Below is a detailed explanation of the digital twinning process and the relevant software and methods used (Figure 7):

Environment scanning and data acquisition: Detailed 3D scanning of the Zhu Jiajiao Kezhi Garden’s water-stage theater is conducted using high-precision laser scanning equipment such as Leica Geosystems AG (Part of Hexagon, Heerbrugg, Switzerland). to capture accurate terrain and architectural details. Special attention is paid to the relationship between water bodies, vegetation, and architectural structures during scanning to ensure the realism of the digital environment. High-resolution photography is also employed to document the theater’s lighting, colors, and textures for texture creation in later stages.

Model reconstruction and environment design: Three-dimensional model reconstruction and environment design are achieved using SketchUp Pro 2021.1 (Part of Trimble Inc, Sunnyvale, CA, USA). This involves basic geometric modeling of the venue as well as refinement of complex structures. Built-in tools and plugins in SketchUp Pro 2021 are used for texture processing to ensure the realism of natural and architectural elements such as water surfaces, stone materials, and wooden structures. Reflective and shadow properties of materials are adjusted to match the effects of both natural and stage lighting.

Integration of dynamic elements and interactive design: Built-in dynamic water effects in SketchUp Pro 2021 and additional plugins are utilized to create the specific water-stage effects seen in Image 2. Parameters of the water wave simulation are adjusted to realistically reflect the interactive dynamics of the water surface during performances. Additionally, the extension capabilities of SketchUp enhance the scene’s liveliness with dynamic vegetation, including trees and flowers swaying in the wind. Interactive scripts are developed to allow users to navigate the virtual environment freely, observe performances from various angles, and interact with elements within the scene.

Lighting and rendering: The advanced rendering tools of SketchUp Pro 2021 are used to simulate natural light and the unique lighting design of the theater. The program ensures that the light accurately illuminates key areas of the performance while creating the atmosphere of the performance time, such as dusk. The rendering features of SketchUp Pro 2021 are utilized to ensure clarity of every detail on high-resolution display devices.

Testing and optimization: The virtual environment is tested on devices with varying configurations to ensure a smooth experience across systems ranging from low- to high-end. Model complexity and texture resolution are optimized based on test results. Kunqu enthusiasts and general audiences are invited to experience the virtual environment, and their feedback is collected to adjust the interactive design and visual effects for the best user experience.

Through this process, the digital twinning demonstration of Kunqu’s “The Peony Pavilion” not only reproduces the beauty of the Zhu Jiajiao Kezhi Garden’s water-stage theater, but it also provides a new way for global audiences to experience and appreciate traditional Kunqu performances without the need to travel, thus opening a window to the artistic charm of Kunqu, as shown in Figure 8.

4. Example Verification and Effect Evaluation

4.1. Construction of Chinese Kunqu Opera Virtual Music Community and Interactive Platform

The Chinese Kunqu Opera virtual music community and interactive platform integrate and apply digital twin technologies such as virtual reality (VR), augmented reality (AR), blockchain, and artificial intelligence (AI), aiming to provide users with an immersive Kunqu opera experience and communication platform. The platform uses virtual reality (VR)/augmented reality (AR) technology, blockchain technology, artificial intelligence (AI) technology, cloud service platforms (such as AWS (Seattle, WA, USA), Google Cloud), and other services. Through these design and technical solutions, we were able to create a virtual music community and interactive platform that not only retains the cultural essence of Kunqu Opera but also incorporates modern technology. This platform provides users with a brand new cultural experience and learning space. The platform will also provide an intuitive, easy-to-use and colorful virtual environment, allowing users to enjoy a personalized and interactive experience while exploring the art of Kunqu Opera. The virtual platform VR display page is shown in Figure 9.

User Interface Design Proposal. This design proposal aims to ensure an intuitive and user-friendly interface while integrating visual elements of Kunqu Opera to enhance the cultural experience. The homepage employs a visual design that fuses traditional Kunqu elements, such as costumes, masks, and stages, with a modern, minimalist interface. The primary color palette consists of tranquil blue-green hues, creating an atmosphere that melds classical and contemporary influences. The homepage is designed to feature modules such as performance previews, selected videos, and the latest news, showcased through scrolling banners and dynamic modules.

The virtual experience zone leverages VR/AR technology to provide users with an immersive experience in viewing Kunqu performances. Users can experience performances in a 3D virtual space via virtual reality headsets or smartphones. Interactive elements, including virtual interactive seat selection and role-playing game elements, are incorporated to enhance engagement.

The learning area offers tiered tutorials ranging from basic knowledge to advanced techniques, including lessons on singing styles and body movement training, presented in various formats such as videos, images, and audio. Utilizing AI technology, the system provides personalized recommendations and feedback based on the user’s learning progress and preferences, including the generation of customized practice content and an evaluation of user performances, thereby facilitating AI-assisted learning.

An interactive discussion area features a dedicated forum where users can initiate discussions, share experiences, and upload original content. This area supports live teaching and performance by artists and enthusiasts, with capabilities for real-time comments and tipping by the audience.

This proposal integrates traditional cultural elements with cutting-edge technology to create a user interface that not only enhances the accessibility and enjoyment of Kunqu Opera but also promotes interactive learning and community engagement within a modern digital framework.

User Experience Design Proposal. This proposal presents a streamlined user journey with an easy-to-navigate interface to ensure rapid access to content and features of interest. The introduction of beginner tutorials via guided instruction aids new users in familiarizing themselves with the platform’s functionalities.

Personalized Experience: Utilizing user behavior data analysis, the platform recommends content, performances, and learning resources that align with user interests. It allows for customization of personal profile pages, including selection of favored Kunqu Opera roles, plays, and artists.

Interaction and Community Building: The enhancement of community interaction features, such as forums, user-generated content (UGC), live commentary, and interactive Q&A, encourages user sharing and communication. A points and rewards mechanism is designed to recognize active users and quality content creators, fostering community participation.

Accessibility Design: The platform ensures inclusivity for all users, including those with visual, auditory, or other disabilities. Features such as text-to-speech and speech-to-text conversion, high contrast modes, and magnification tools are implemented to guarantee content accessibility.

Technical Design Proposal. Virtual Reality (VR)/Augmented Reality (AR) Technology: VR/AR content is developed using Unity or Unreal Engine to ensure high-quality visual effects and smooth interaction. Coupled with 3D modeling and animation, these technologies reproduce the nuances of Kunqu performances, enhancing realism through:

Content Creation: High-quality 3D scanning and motion capture technologies record the performances of Kunqu artists, ensuring accurate representation of movements and expressions. Virtual environments, characters, and interactive elements are built using Unity or Unreal Engine development platforms.

Platform Compatibility: Applications compatible with mainstream VR headsets (e.g., Oculus Rift, HTC Vive) and AR platforms (e.g., ARKit for iOS, ARCore for Android) are developed. For users without VR/AR devices, a web-based 3D experience is provided, accessible via mouse and keyboard interactions.

User Interaction: Interactive tutorials and gamified learning elements, such as role-playing and performance guidance, are designed to increase user engagement. Multi-user virtual environments support online user interaction and collaboration.

Blockchain Technology: A secure and transparent transaction system built on blockchain technology is utilized for ticketing and art sales. Smart contracts automate transactions, ensuring fairness and security through:

Smart Contracts: Automated transaction processing and contract term enforcement, such as ticket sales, copyright management, and art trading, are facilitated by smart contracts. All transaction records are transparent and traceable, increasing user trust.

Token System: Platform-specific tokens are issued for internal transactions, rewarding contributors, and purchasing virtual goods. A token economic model is designed to regulate supply and demand, maintaining token value.

Digital Collectibles (NFTs): Unique digital art pieces and collectibles, such as limited edition Kunqu videos, audios, or virtual items, are offered through NFT technology. An NFT marketplace enables users to buy, sell, and exchange these digital collectibles.

Artificial Intelligence (AI): Machine learning analyzes user data to provide personalized learning content and recommendations. Voice and image recognition technologies are developed for the learning area’s automatic evaluation functions, such as assessing singing accuracy and body posture, including:

Recommendation System: A machine-learning-based recommendation algorithm analyzes user behavior and preferences, offering personalized content recommendations, such as performances, tutorials, and articles. The recommendation strategy is adjusted in real time to match changing user interests.

Content Generation: Deep-learning models, such as Variational Autoencoders (VAE) or Generative Adversarial Networks (GAN), automatically generate Kunqu music and performance scripts for user exploration and learning. AI-assisted creation tools help users produce Kunqu-related music and textual content.

Learning Assistance: Natural language processing (NLP) and voice recognition technologies develop intelligent tutoring robots for voice-interactive learning and feedback. Computer vision analyzes user performance videos to provide movement correction and performance improvement suggestions.

Other Technologies: Cloud service platforms (e.g., AWS, Google Cloud) deploy applications, ensuring stability and scalability. NoSQL databases (e.g., MongoDB) support extensive user interaction data and personalized content. The latest network security protocols and encryption technologies protect user data from unauthorized access and leaks. Transparent privacy policies allow users to control their data and preferences. Responsive design frameworks (e.g., Bootstrap) ensure good platform performance across various devices and screen sizes. JavaScript frameworks (e.g., React or Vue) create dynamic user interfaces and single-page applications (SPAs), enhancing the responsiveness and smoothness of user interactions. Server-side technologies like Node.js or Python develop APIs for user requests and data exchanges. A microservices architecture ensures platform scalability and modularity, facilitating future feature additions and maintenance.

4.2. Effectiveness Evaluation and Feedback Analysis of the Chinese Kunqu Virtual Music Community and Interaction Platform

The assessment of effectiveness and analysis of user feedback are crucial for the Chinese Kunqu Virtual Music Community and Interaction Platform. Various methods were employed to gather user feedback, aiming to comprehensively understand the platform’s performance and user experience.

Initially, user surveys and feedback analyses were conducted. Through online questionnaires and real-time feedback devices, data on user satisfaction with the museum experience were collected, encompassing evaluations of exhibits, interactive elements, and educational content. For users with educational purposes, a specific assessment of learning outcomes was conducted, collecting data on users’ mastery of Kunqu Opera history before and after visiting, through tests and interviews. Special feedback was also gathered on interactive experiences, such as virtual reality performances and music games. Furthermore, user behaviors within the museum were tracked and analyzed using data analysis tools, including visit duration, interaction frequency, and browsing paths, to identify interest points and behavior patterns. These surveys and analyses enabled the platform to better understand user needs, continuously optimize experiences, and enhance its effectiveness in preserving and disseminating Kunqu cultural heritage. A Virtual Music Community Feedback Questionnaire is shown in

Table 1. Virtual Music Community Feedback Questionnaire.

Section	Question	Options
Basic Information	Gender	Male, Female, Other, Prefer not to say
	Age Range	<18, 18–24, 25–34, 35–44, 45+
	How do you usually learn about Kunqu?	Internet, Books, Education, Friends, Other
Platform Usage Experience	How would you rate the ease of use of the community platform?	Very Dissatisfied, Dissatisfied, Neutral, Satisfied, Very Satisfied
	Did the interactive elements enhance your experience?	Strongly Disagree, Disagree, Neutral, Agree, Strongly Agree
	Are you satisfied with the educational content provided?	Very Dissatisfied, Dissatisfied, Neutral, Satisfied, Very Satisfied
	Did you encounter any technical issues?	Yes, No (If yes, please specify)
Education and Cultural Transmission	How has your understanding of Kunqu changed after using the platform?	Significantly Decreased, Slightly Decreased, No Change, Slightly Increased, Significantly Increased
	Would you recommend this virtual community to others?	Definitely, Probably, Not Sure, Probably Not, Definitely Not
Additional Feedback	Do you have any suggestions for improving this virtual community?	(Open-ended response)

.

Instructions for Completion:

• Please select the most appropriate option for each question.
• For any open-ended questions, provide detailed feedback where possible.
• Submit the completed questionnaire via the provided submission method on the platform.

We conducted a survey of 2000 people, and below is a table of possible survey results and an analysis table, which will show the statistical results of key issues and related analysis. The results are shown in

Table 2. Virtual Music Community Feedback results.

NO.	Problem Description	E	D	C	B	A	Yes	No	Detailed Description/Remarks
4	Ease of use of virtual music community	100	150	250	800	700	-	-	-
5	Interactive elements enhance experience	50	100	300	900	650	-	-	-
6	Satisfaction with educational content	80	120	400	850	550	-	-	-
7	Whether technical problems are found	-	-	-	-	-	360	1640	Most of the problems are about slow loading speed
8	Changes in understanding and knowledge of Kunqu Opera	40	110	350	900	600	-	-	-
9	Willingness to recommend	30	70	300	950	650	-	-	-

Note: In the table, E: Very Dissatisfied, D: Dissatisfied, C: Neutral, B: Satisfied, A: Very Satisfied.

.

In terms of user satisfaction and educational effectiveness, high levels of usability have been widely acknowledged, with approximately 70% of users indicating satisfaction or high satisfaction. Similarly, satisfaction with educational content is also notable, with the same proportion of users deeming the educational content to be generally in line with their expectations. The introduction of interactive elements has been perceived as effective in enhancing the user experience by 77.5% of users, thus reflecting the platform’s appeal and interactivity to a considerable extent. Regarding the willingness to recommend the platform, around 80% of users express a possibility or definite intention to recommend it, suggesting an overall positive user experience. A visualization of the statistical results is shown in Figure 10.

However, technical issues persist, with approximately 18% of users reporting problems such as slow loading speeds, which have affected the experience of some users. Nonetheless, the majority of users have not encountered major impediments to usage.

In terms of educational outcomes, platform usage has significantly increased users’ understanding and knowledge of Kunqu, as approximately 75% of users indicate that their understanding and knowledge of Kunqu have been enhanced through platform usage, with 60% of users reporting a significant or slight increase.

User feedback and suggestions primarily focus on accelerating loading speeds and improving system performance, adding more interactive elements and educational content, and enhancing the platform’s accessibility and usability. These suggestions are instrumental in further refining the platform to enhance user experience and educational effectiveness.

Additionally, evaluations and suggestions from cultural heritage experts were solicited, including music historians, cultural heritage preservationists, and digital technology experts. The results are shown in

Table 3. Expert opinion statistics table.

Expert Category	Position	Main Suggestions and Comments
Government agency cultural tourism manager	Ministry of Culture and Tourism	Increase policy support, promote the linkage between digital platforms and physical exhibitions, and provide financial and resource support.
Intangible cultural heritage expert	National Cultural Heritage Administration	Add interactive teaching content and improve educational functions, especially for teenagers, to increase their participation and interest.
University researcher	Director of Cultural Research Center	Cooperate with academic circles to conduct in-depth research on the history and artistic value of Kunqu Opera, enrich platform content, and provide academic support.
Museum manager	Museum Director	Exhibition design should be more attractive and educational, support the complementarity of virtual exhibitions with physical exhibitions, and improve public accessibility.
Famous heritage bearer	Kunqu Opera Artist	It emphasizes the combination of traditional art and modern technology, uses virtual technology to inherit art, and innovates while maintaining the original flavor of art.

. Most acknowledged the platform’s effectiveness in preserving musical cultural heritage, particularly through high-quality digitalization, facilitating the preservation and dissemination of valuable musical works and cultural information. Experts also provided suggestions on balancing technology and content and enhancing diversity and inclusivity.

Combining these evaluation methods and expert recommendations, it is concluded that the Chinese Kunqu Virtual Music Community and Interaction Platform plays a significant role in the preservation, education, and cultural exchange of musical cultural heritage. As a new form of protection and dissemination for musical cultural heritage like Kunqu, its value in cultural preservation is demonstrated in multiple aspects. Through digital technology, ancient and rare musical works are preserved for the long term, traditional music is revitalized and popularized, and the virtual music community and interaction platform offers a new platform for music education and cultural heritage. By breaking geographical and physical barriers, it enables people worldwide to access and experience musical cultural heritage. User feedback and expert evaluations have provided valuable insights and directions for improving the platform. Efforts will continue to optimize the platform, enhance user experience, and act as protectors and promoters of musical cultural heritage.

5. Further Discussion

The future development of digital twin technology in the field of musical cultural heritage is trending towards diversification and intensification. Its potential lies in providing richer, more interactive, and personalized cultural experiences, while offering new possibilities for the preservation and research of musical cultural heritage. With the ongoing advancement of technology, the potential impact of digital twin technology on the preservation of musical cultural heritage is becoming increasingly significant. The following analysis highlights key areas of impact: enhancing preservation accuracy and quality, expanding dissemination and education pathways, fostering international cooperation, supporting complex data analysis, and the coexistence of challenges and opportunities. The impact of technological progress on the preservation of musical cultural heritage is profound and complex, necessitating a cautious approach to addressing accompanying challenges while fully leveraging technological advantages.

As the application of digital twin technology in the musical cultural heritage field deepens, future research directions and the challenges faced become more evident. These include the study of multimodal data fusion, research on optimizing user experience, ethical issues in digitalizing cultural heritage, sustainability and accessibility of technology, and the promotion of cross-cultural understanding and communication. Through these research directions and addressing challenges, the future may see further expansion of digital twin technology applications in the musical cultural heritage field, providing increased support for the protection and dissemination of global cultural heritage.