1. Introduction
Architectural heritage is a spatial representation of local, traditional characteristics and a spatial witness of the origin of identity. It is not only a cultural asset endowed by history but also a cultural resource required for future development. Ancient opera stages, as examples of unique architectural heritages, are the carriers of Chinese traditional opera culture and are also witnesses to the history of the formation, development, and prosperity of Chinese opera. Historically, there were more than 20,000 ancient opera stages in China. The unique heritages of these stages reflect both the traditional architectural style and the historical opera culture [
1]. Alongside the decline in opera culture, these ancient stages are seriously threatened and may even disappear. The reasons for this decline include the continuous erosion of the natural environment and deliberate or unintentional damage and destruction caused by people. Therefore, there is an urgent need to protect and inherit these precious architectural heritage sites.
Fortunately, the rapid development and adoption of digital technology has provided opportunities to resolve this bottleneck. Remote sensing, geographic information systems (GIS), and virtual reality (VR) have been widely used in cultural heritage protection [
2], bringing about a new wave of technological revolution in the protection and utilization of cultural heritage with 3D reconstruction, virtual restoration, and digital display at the core [
3]. The documentation and inheritance of cultural significance, as one of the major challenges in the field of heritage conservation, is essential in the design of preventive conservation strategies. Since the 1990s, the theory, method, and application of digital protection and visualization in cultural heritage have gained strong interest and become research hot spots. Academically, various methods have been developed for the documentation of heritage and the inheritance of culture through multidisciplinary approaches and the use of digital technologies.
In the documentation of heritage, the main measures used have been reality-based 3D digitization and modeling. Virtualization restoration enabled a highly complex and endangered sacred heritage site to be brought back to public view [
4]. The combination of 3D scanning and photogrammetry effectively captures the uniqueness of architectural heritage sites and record them flexibly and accurately [
5,
6,
7]. The Dunhuang Mogao Grottoes in China form a world cultural heritage site which has been digitally protected for more than 30 years through methods such as the digital recording of existing grottoes, the construction of virtual grottoes, the nondestructive detection of cave diseases, etc. [
8,
9,
10]. About 230 caves have been completely recorded in the digital collection, accounting for 30% of the total number, basically realizing the original intention of “permanent preservation and sustainable utilization” [
11]. Furthermore, in the Church St. Peter, documents were recorded and preserved using 3D scanning and modeling techniques, showing that documents of high historical value can be preserved permanently [
12]. In the 3D modeling of Malay Cities, the effective fusion of laser scanning and UAV photogrammetry data was proposed for the rapid realization of 3D modeling of cities [
13]. The digital paradigm was exemplified by the 3D virtual reconstruction of the Shukhov hyperboloid tower in Moscow [
14], the Mevlana Museum in Konya [
15], the Iconostasis of Saint Nicholas in the Serbian Orthodox Cathedral [
16], etc. In short, digital technology gives new life to architectural heritage and makes its reconstruction possible.
In the inheritance of culture, 3R technology, namely virtual reality (VR), augmented reality (AR), and mixed reality (MR), represents and interprets cultural heritage via visualization, so this technology has become a powerful ally of sociology, art, the humanities, and archeology [
17,
18]. A mass of cultural heritage digital content has been developed and has become accessible online and offline. This digital content enables visitors to access culture heritage sites of interest without the risk of damaging or destroying them, which can not only improve their understanding of history culture, but also increase their participation and experience [
19,
20,
21]. Argentine has created a museum called “Virtual Collections”, which digitally displays collections and includes a virtual tour of real sites. The creation of this virtual museum effectively integrates cultural heritage resources, realizes visualization and interaction in a virtual environment, and thus disseminates cultural heritage [
22]. The VR/AR application of JejuView allowed visitors to immerse themselves in the heritage of Jeju Island, providing a perfect visual experience and interactive experience [
23]. Similarly, immersive bridge diving at Stari Most [
24], a virtual tour of Jeddah in Saudi Arabia [
25], etc., are good examples that exhibit the possibilities and importance of virtual technology. To a certain extent, 3R technology breaks through the traditional perception of cultural heritage, fully demonstrates the historical and cultural charm of heritage, and enables deep cross-temporal and cross-regional communication.
Most of the existing studies targeted a particular cultural heritage site. Little research has systematically investigated scattered cultural heritage sites, which would be more meaningful and challenging. Academics of different nationalities have carried out research in succession, for instance, the creation of a Qatar historic buildings information modeling platform via Q-HBIM [
26], a simulation model for Digital Twin applications in historical masonry buildings [
27], the protection and utilization of Great Wall Cultural Heritage using information technology [
28], etc.
In view of previous concerns, in this research, we focused on the creation of a comprehensive method to protect and inherit the scattered ancient stages with a multidisciplinary approach and digital technology. This method included digital documentation, digital culture inheritance, and intelligent management, involving TLS, VR/AR, knowledge visualization, GIS, etc. A detailed description of the method developed is presented in the second section. In the third section, the results of this method are discussed, with a practical application to the case study of the ancient opera stages in Shanxi, China, acquired by our team.
2. Materials and Methods
We considered this research from the perspective of protection, utilization, and management. The problem to be solved in terms of protection is how to quickly and accurately digitally record ancient stages. Utilization is how to transform digital resources into digital culture to display and inherit it. Management is how to commendably manage and use a large quantity of heterogeneous data. Thus, this method was divided into three parts: digital documentation, digital culture inheritance and intelligent management.
The technical flowchart of this study is shown in
Figure 1.
2.1. Digital Documentation
Digital documentation was used to enable the digital protection of ancient stages. There are different methods of digital collection and model construction for different types of cultural heritage sites, different materials, and different geographical environment characteristics [
29]. At present, many methods have been developed, such as aerospace remote sensing, airborne lidar, three-dimensional laser scanning, unmanned aerial vehicle (UAV) remote sensing, close-range photogrammetry, and so on [
30].
In this study, the work was completed by data acquisition, data processing, model construction, and texture mapping. The data of ancient stages were acquired by terrestrial laser scanning (TLS), close-range photogrammetry, and satellite remote sensing from different macro-, meso-, and microscales, as well as in 2D and 3D. The recorded data included original point clouds and images, CAD drawings, a three-dimensional model, and a digital restoration model, etc.
- (1)
Terrestrial Laser Scanning
TLS was used to gain point clouds for the principal parts of the ancient stages. In our study, a Leica C10 was used to obtain point clouds from the external and indoor surfaces of a stage, shown in
Figure 2a. There were multiple scanning stations, which contained the spatial information of the stage and its surrounding environment, shown in
Figure 2b. The expected accuracy was less than 5 mm at a distance of 100 m, and the angular resolution was 0.0005◦. The full scanning field angle was 360° × 270°. The signal intensity received by the sensor system was recorded in 8 bits [0 255]. The point cloud accuracy was 6 mm, and the point spacing was 3 mm. The point cloud information from any target point was
, where
is the 3D coordinate,
is the color information, and
is the echo intensity.
A Leica Cyclone was used to register multisite original point clouds, eliminate the noise of surrounding buildings and trees, extract point cloud contours and slices, etc. AutoCAD was used to draw the plane, elevation, and sections of the CAD drawings of the stage building via the point clouds. The 3ds Max was used to construct the 3D prime model of the whole stage building and building components according to the CAD drawing. Mudbox was used to map the texture photos into the 3D prime stage model, which was turned into a color 3D stage model.
- (2)
Close-Range Photogrammetry
UAV photogrammetry and Ground HD photography was mostly used for close-range photogrammetry. A DJI inspire2 UAV was used to carry out UAV photogrammetry, which was equipped with a camera to obtain images of the top of the stage and the surroundings of the stage from one vertical and four inclined directions and generate point clouds. It clearly measured the spatial position, surrounding terrain, and architectural shape of the stage, photographic images of which also clearly recorded the texture image on the top of the stage. A Canon 5D4 camera with a 24–70 mm lens was used for Ground HD photography, which acquired the texture images of the stage surface and the stage panoramic and local images. The resolution of the photographs was 4464 × 2976 pixels, at 400 to 800 ISO depending on the intensity of sunlight. According to different sets of images, the lens aperture varied from f5.6 to f12, and the shutter speed varied from 1/250 to 1/640 s. In order to obtain high-quality images, it was necessary to set the white balance and color correction and shoot under good lighting conditions, preferably in a sunny and smooth environment.
Context Capture was used to build the large space model around the stage with the UAV images. Adobe Lightroom was used to process images in batches, so that the texture image had a unified tone. Adobe Photoshop CC 2017(San Jose, CA, USA) was used to process the images in terms of aspects such as distortion correction, color correction, and splicing and fusion to form high-definition global and local images of the ancient stages. According to the texture image obtained using a camera and the stage-top image acquired using a UAV, the texture of the prime model was mapped to construct a textured 3D model of the stage.
- (3)
Satellite Remote Sensing
Any architectural heritage site is always situated on a real and complex three-dimension geographical environment, which can be depicted using topography, geomorphology, vegetative cover, etc. In our study, various satellite remote sensing images were used to obtain the large-scale three-dimensional environment of the area surrounding ancient stages, such as landform, vegetation, rivers, and surrounding buildings, to create the three-dimensional landform environment model. The basic geographical data, such as the Digital Raster Graphic (DRG), Digital Line Graphic (DLG), Digital Orthophoto Map (DOM), and Digital Elevation Model (DEM), were used to describe the 3D geographical environment in the area where the stage was located. The data came from a GIS in 2019 with high resolution, which was enough to position the 3D models.
2.2. Digital Cultural Inheritance
Digital culture inheritance was mainly realized with digital displays. The abovementioned digital protection achievements provided us with a large amount of stage opera information. We examined how digital technology can be used to excavate cultural connotations and reconstruct knowledge, making it easier for people to understand abstract and complex stage knowledge and inherit excellent Chinese traditional culture.
In our study, we used the knowledge visualization theory to inherit the stage culture from digital short films, VR/AR, and digital humanities in order to better reflect its historical, scientific, and artistic value. The most important task was to build a knowledge visualization model to excavate the cultural connotations of ancient stages.
Knowledge visualization examines the use of visual representations to improve the creation and transfer of knowledge between people, which designates all graphic means that can be used to construct and convey complex insights [
31]. It aims to transfer facts, insights, experiences, attitudes, values, expectations, perspectives, opinions, and predictions in a way that enables someone else to reconstruct, remember, and apply these insights correctly [
32]. On the basis of the abovementioned theory, we built a knowledge transformation model of the ancient stages to present invisible knowledge in an explicit way via new media technologies such as AR/VR, animation, etc. Various digital displays transferred ancient stage culture, such as architecture, decoration, opera, and art, from multisensory channels such as vision, hearing, and feeling, so that the public could more systematically and comprehensively experience the charm of traditional culture.
This model realized the transformation of invisible knowledge through “knowledge acquisition → knowledge transformation → knowledge innovation” and transformed the abstract ancient stage knowledge space, “academic language”, into the intuitive visual space, “public language”, to excavate the hidden cultural connotations of ancient stages. As shown in
Figure 3, the knowledge visualization model of ancient stages is as follows:
- (1)
Knowledge Acquisition
From experts, museums, and networks, the knowledge of ancient stages, such as images, point clouds, documents, and documentation, was acquired and built into a knowledge database of ancient stages.
- (2)
Knowledge Transformation
Based on the knowledge database of ancient stages, the implicit knowledge of ancient stages was extracted, absorbed, and then integrated into popular knowledge using visualization technology to form a knowledge visualization material library of ancient stages.
- (3)
Knowledge Innovation
Using visualization technology, the ancient stage knowledge was transformed into digital forms such as animations/videos, VR/AR, graphics/motion, games, images, etc., to express the deep-rooted culture. Additionally, these were disseminated through mobile phones, websites, museums, and so on. Furthermore, the digital innovation works were in the knowledge visualization database of ancient stages.
2.3. Intelligent Management
Intelligent management was used to manage and utilize the heterogeneous data generated via digitization. Shanxi ancient stages were large in quantity and are widely distributed; hence, digital documentation and the inheritance of culture produce a great quantity of heterogeneous data. It was difficult to effectively manage and utilize complex data, which require the cooperation of multiple professionals from different fields, extensive resources, accurate organization, and a robust structure.
Therefore, an innovative digital platform, the 3D geographic information system of Shanxi ancient stages, was designed and developed to effectively integrate the stage architecture, surrounding environment, and the opera, culture, history, society, and art involved, which had the characteristics of digitization, visualization, intelligence, and openness. The dynamic real-time preservation and protection of such invaluable heritage is highly desirable to facilitate interaction between users from different specialties.
The structure of the system was divided into four layers, as shown in
Figure 4. From bottom to top, these were the technical layer, data layer, platform layer, and the application layer.
- (1)
Technical Layer
The key technologies were GISs, big data, remote sensing, photogrammetry, three-dimensional modeling, AR/VR technology, etc.
- (2)
Data Layer
The data layer was used to store system data, including the basic geographic database, point cloud database, image database, 3D model database, video database, interactive database, humanities database, tourism database, and user database.
- (3)
Platform Layer
The platform layer consisted of three parts: the Core Platform, Visualization, and Space intelligence. The Core Platform included a digital protection service platform, digital research service platform, and cultural tourism service platform. Visualization included security monitoring visualization, analysis visualization, and virtual simulation visualization. Space intelligence included space analysis, virtual simulation, security monitoring, and planning analysis.
- (4)
Application layer
The application layer was mainly oriented to users, such as the administrator, researchers, and the public. Each user was permitted to access different applications. The applications included stage GIS retrieval, stage information retrieval and stage digital display, stage culture exhibition, stage space analysis, smart tourism services, data maintenance, system maintenance, etc.
3. Results and Discussion
Based on the abovementioned methods, we began to conduct the experiment in 2019. Our team carried out research regarding the digital protection and cultural inheritance of Shanxi ancient stages. At present, this work has been completed for more than 10 stages, including the Erlang Temple stage, the earliest existing stage.
3.1. Case Study Introduction
The case study chosen in order to better illustrate the method focused on the ancient opera stages in Shanxi, China.
Chinese opera is a highly comprehensive art, which integrates music, dance, rap, performance, acrobatics, etc. Opera characters usually include a Sheng (male lead), Dan (female lead), Jing (painted face), Mo (old role), and Chou (clown), which are identified by facial makeup and clothing [
33]. Early operas were performed by two to five people, which were watched from three sides of the stage. The sound was echoed through a sunk panel of the stage or a big vat around the stage. Shanxi, located in North China, is known as the “Cradle of Chinese Opera”. The earliest Chinese stage appeared in Shanxi, which was made to carry out ancient sacrifices. According to the stone tablet records of Houtu Temple, Wanrong County, Shanxi, a dance stage was built between 1004 and 1007, which was the earliest stage recorded [
34]. The earliest existing stage, the Erlang Temple Stage, was also in Shanxi, which was built in Gaoping in 1183 [
35].
Ancient stages were used to not only to reward gods, but also for people’s entertainment and celebrations, carrying the dual value of material and intangible cultural heritage. The development process of these ancient stages can be divided into three periods: the sprouting period, developing period, and heyday period. Firstly, in the sprouting period (1004–1368), stages appeared in temples. About 13 stages existed in this earliest period, mainly distributed in the south and southeast of Shanxi [
36]. In recent years, there have been archaeological discoveries of new stage remains, such as the Dance Temple Stage in Lucheng [
37]. The developing period was between 1368 and 1644. Opera stages were spread all over the Chenghuang temples in villages, counties, and states. According to historical records, there were 187 stages in this period, which were also mainly distributed in the south and southeast of Shanxi [
38]. Thirdly, in the heyday period (1644–1840), traditional opera and the accompanying stages were flourishing. There was a stage in almost every temple and village, and the number grew to almost ten thousand [
39]. The types of stage developed from temple stages to ancestral-hall stages, courtyard stages, teahouse stages, etc. The stage shapes became more magnificent and exquisite, and the decoration of the stages was also particularly exquisite in this period.
At present, most of the ancient opera stages are scattered in ancient villages. Due to natural and social factors, the number has decreased. There are about 3515 ancient stages in Shanxi, which were the first to be constructed in China. The overall distribution of stages in Shanxi is still characterized by there being more in the south and less in the north, with 1887 in the southeast of Shanxi, accounting for 53% of all of the stages and ranking first in distribution. Therefore, ancient stages and their culture urgently need to be protected and inherited. Fortunately, the development of digital technology has provided us with new concepts and methods.
3.2. Digital Documenting
Digital protection is mainly used for the storage of ancient stage data. These data are divided into the following types: point clouds, images, CAD drawings, and 3D models.
3.2.1. Point Cloud and Image
Point clouds and images are the most original data of the ancient stages in digital protection, which provided strong scientific support for the research and construction of ancient stages. Point clouds were obtained through three-dimensional scanning, which accurately recorded buildings’ geometric shapes. Images were obtained through close-range photography, which accurately recorded the texture information. In our study, point clouds were saved as two kinds of data; one was the original datum, which was opened by the scanner supporting software. The other was the preprocessed datum after preprocessing such as denoising, registration, and fusion, the format of which was .ply, .dxf, .pcd, etc.
Figure 5 shows the point cloud of a station in the Niuwang Temple Stage, which was built in 1283.
Images were also divided into two types. One was a .raw file, which recorded the metadata of original information, including the ISO setting, shutter speed, aperture value, white balance, etc.
The other is made up of .JPG, .PNG and other image files.
3.2.2. CAD Drawing
CAD drawings are engineering drawings drawn using CAD software, such as mechanical drawings, electrical drawings, and architectural drawings. In the digital protection of architectural heritage, CAD drawings can accurately draw and record architectural structures and dimensions. In our study, using CAD drawings in the draw stage, the plane and elevation were drawn by measuring the point clouds. In
Figure 6, the CAD drawings of the plane, facade, and side elevation of the Niuwang Temple Stage are shown.
Many ancient stages in Shanxi are wooden structures, which are interspersed with columns and beams to form a framework with diverse styles and stable structures. They have beautiful forms, flexible shapes, and a strong stereoscopy, and have highly aesthetic designs and the characteristics of Oriental Art and culture. However, their complex structures, such as bucket arches, systems of brackets inserted between the top of columns and crossbeams, also posed challenges to digital recordings. Therefore, we recorded these components in detail and drew CAD drawings. In
Figure 7, the plan drawing, elevation drawing, and section drawing of a bucket arch on the Niuwang Temple Stage are shown.
3.2.3. Three-Dimensional Model
The 3D models were composed of polygons, including a prime model and textured model. They accurately expressed the geometric and texture features of the architectural heritage sites through visualization. In
Figure 8, the model of the Houtu Temple Stage, built in 1345, is shown, and each building component can be disassembled and has accurate dimensions. All the 3D models of the other stages have the same structure as this one, so we will not show them one by one here.
3.3. Digital Cultural Inheritance
In the digital cultural inheritance of ancient stages, we should not only comprehensively and objectively describe the architecture, but also emphasize the excavation and presentation of the cultural and social value. We adopted an answer-oriented design by ancient stage knowledge visualization. According to the knowledge visualization model, the profound stage knowledge was transformed into visual language, which is used to express and explain answers to finally construct the key points and frameworks of ancient stage knowledge. Especially for the general public, who do not have professional knowledge, this design enhanced the transmission of knowledge through visual expression in the form of images, digital short films, and AR/VR, and helped them establish the knowledge essential framework and internalize it into individual experience to inherit stage cultural heritage.
3.3.1. Digital Short Film
Digital short films were one of the expressions of knowledge, produced using digital media methods such as 2D animation, 3D animation, and live action, using point cloud, CAD drawings, 3D models, and so on. Films interpreted profound knowledge, such as knowledge regarding architectural features, opera culture, and historical background, into simple content through knowledge visualization, so that the general public could quickly and easily understand the culture surrounding the stages. For example, when analyzing the structure of the stages, architectural growth animations were used, which gradually presented the structure from the stage base and column to the bucket arch, beam frame, and roof. In addition, the growth animation changing from “point cloud” to “prime model” and then to “texture model” also fully represented the process of digital protection.
Figure 9 shows the growth animation of the Erlang Temple Stage.
3.3.2. VR Interaction
In our study, we used HTCVR interaction and developed it with Unity3D. Although the ancient stages were scattered in terms of location, the experimenters wore VR helmets and held VR handles, meaning they could “walk” to any corner of the stage and “touch” any part of a stage without going anywhere. They could observe and research it carefully without worrying about damaging it. Furthermore, the visual presentation of the historical background, architectural structure, and traditional opera culture of the ancient stage, including 3D opera roles, virtual stage scenes, and the virtual opera culture, enabled users to experience the historical context behind the stage. This collision between VR technology and culture broke through the limitations of time and space and enabled the public to immerse themselves anytime and anywhere and understand various stages in different regions, which enhanced the visiting experience.
In
Figure 10, the VR interactive experience of the Houtu Temple Stage is shown.
3.3.3. AR Interaction
AR interaction is usually applied through mobile phones. We designed a mobile APP with AR and developed it with unity3D. In this APP, the stage knowledge was divided into three levels: cognitive visualization, experience visualization, and research visualization (
Figure 11). In simple terms, the stage knowledge was introduced, understood, and consolidated, so that the participants gradually cognitively processed the senses and behavior of the stage culture and realized the transformation from the primary schema to advanced schema.
AR was used to present the static stage culture in a three-dimensional, interactive, and virtual–real way. For instance, when the stage picture was scanned, the 3D stage model appeared, which was scaled, rotated, and moved, so that the user could intuitively and accurately understand the structure, components, and carvings of the stage. Interest in AR scenes can promote the cultural connotations of the stages, which can not only attract public visits but also promote the development of cultural communication.
3.4. Intelligent Management
This system development mainly included the establishment and maintenance of a front-end (web) and background cloud-enabled database management module. The front page was developed in visual studio 2018 and C# Script, and the access to background data was based on the web service Description Language of Microsoft.NET Framework 4.7.1. The background database management module was developed in MYSQL Server. The secondary development of ArcGIS with a visual studio was used to deploy geographic graphic data and display related geographic information in the front end. The main features of the system are summarized as follows:
4. Conclusions
The conservation and use of architectural heritage sites, especially scattered ancient stage heritage sites, are significant and valuable. This work requires interdisciplinary collaboration. This research, combining technological and humanistic knowledge, proposed a new method with which to inherit and protect ancient stage culture with digital technology, and the method was employed on ancient stages in Shanxi, China.
The results show that the multidisciplinary approach is effective in the preservation and use of ancient stage culture. It is also applicable to other scattered architectural heritage sites. We could even provide a 3D geographic information service system on the heritage culture of stages. VR/AR and digital films on the heritage can be displayed in museums and enable visitors to personally “visit” scenes. Accordingly, this method is helpful in the promotion of the public’s awareness to protect the Chinese ancient stage culture.
The integration of multidisciplinary knowledge and cutting-edge technologies dealt with in this study is still in the process. More advanced methods are needed to explore the culture of ancient stages and present this culture using AR/VR. Intelligent management has not yet enabled the dynamic monitoring and simulation of the stages. The number of stage digitalizations in analysis is far from the actual number of stages in China. Therefore, further studies are needed to improve this method, such as in the dynamic monitoring, virtual repair, interactive simulation, and intelligent control of stages. In particular, future research should focus on how to further transform the “tacit knowledge” of stage culture into “explicit knowledge” using knowledge visualization. In addition, we have only completed this work for less than 20 ancient stages. We will continuously document and inherit others to form an enormous data platform to provide support for researchers by nationality. This is also an important task for our future work.