Remote Practice Methods of Survey Education for HBIM in the Post-Pandemic Era: Case Study of Kuiwen Pavilion in the Temple of Confucius (Qufu, China)

Featured Application

This study introduces remote practice methods for a heritage documentation course, representing alternative solutions in the post-pandemic era. The aims are to help students to observe and understand the cultural heritage through virtual tours, and achieve HBIM deliverables as the basis for scoring.

Abstract

The Field Trip for Measured Survey of Built Heritage carried out by Tianjin University every summer aims to educate students in each practice session, i.e., on-site data acquisition, condition investigation, classification of the component library, taking observation notes, and accomplishing HBIM deliverables. Under the risks posed by the ongoing COVID-19 pandemic, the students could not leave the campus as a provisional arrangement. Only a team of five people was allowed to visit the Kuiwen Pavilion in the Temple of Confucius (Qufu, China). Therefore, the field trip for students had to be replaced with remote solutions, which consists of the following methods: on-site data acquisition; post-processing; online education, observation, modeling, delivery. Kuiwen Pavilion in the Temple of Confucius is a library with the official architectural style of the Ming and Qing Dynasties (14th to 20th century), for which building regulations are commonly recognized, and are suitable for survey education. In this context, this article focuses on the remote practice methods applied and tested throughout the case study. During the practice of the course, students who managed to finish the course, through the virtual tours and other online methods, finally achieved delivery of the HBIM models.

1. Introduction

1.1. Structure of the Article

This article begins with a review of related studies, the background information of the Kuiwen Pavilion, and an overview of survey education. Then, the materials and methods describe preparations for the remote practice methods and propose a rebuild workflow of HBIM. The results showed how students finish the HBIM models through the proposed methods. Solutions for the issues that appeared during the HBIM session were discussed. Finally, the conclusion proposes a procedure covering the whole process.

1.2. Review of Related Works

The concept of HBIM (historic building information modeling) was initially proposed by Murphy et al. (2007) [1], suggesting a workflow of correlation and mapping of parametric objects onto scan data. Later, Murphy et al. (2009) [2], further developed the HBIM workflow as a reverse engineering process: surface model meshing from the point cloud provides a framework onto which HBIM objects can be mapped. In summary, HBIM objects could be regarded as recording regularity, while the surface model meshing from the point cloud could be regarded as recording differentiation. However, the process of mapping HBIM objects onto the point cloud relies on subjective judgments, bringing difficulties for repetition.

Wu et al. (2016) [3] suggested that an HBIM model is essentially an “index frame” for the heritage building’s information, does not have to represent the “real object” 100%. So Li et al. (2019) [4] continued to suggest that the fundamental issue of HBIM is the balance between regularization modeling and differentiation modeling. Therefore, the algorithmic modeling of HBIM could be a better solution because it is easy to repeat with node-visible modeling programs [5,6,7,8,9,10].

Traditional methods of digital documentation have accumulated a large number of two-dimensional CAD drawings, survey reports, and survey photographs. These data need to be processed with some degree of presumption, relying on experiences to avoid mistakes. Data registration needs subjective-based judgments to unify many dimensions; therefore, it is hard to repeat. The solution for the utilization of existing archives needs to be discussed. López et al. (2018) proposed that the development of useful plug-ins could be a future focus [11].

Bruno et al. (2019) [12] suggested a solution for information management that links external databases into Navisworks Manage through the Datatools plug-in. Through the SQL (Structured Query Language) statement, the databases and the models are well associated.

Achille et al. (2020) [13] suggested a web-based information system, which uses a remote server to install external SQL databases, making it possible to automatically link information to models through the web interface.

Wang et al. (2021) [14] summarized the regularized rebuild workflow of HBIM, as one possible solution for the utilization of existing archives. The workflow consists of the classification of components while on-site investigations, the summary of the building regulations, the development of algorithm modules using Rhino and Grasshopper, the association of the model in IFC (industry foundation class) format and data in Excel format, the selection of algorithm modules, and multi-disciplinary collaboration in Navisworks. According to IFC schema, IfcAttribute describes the directly contained data, whereas IfcProperty describes the linked data. This workflow aims at creating regularity models, unifying the parameters that are slightly different in similar components, with algorithm modeling using Grasshopper. The whole process can be repeated and adjusted at any node (Figure 1). This workflow was applied for the HBIM modeling session during the survey education.

Research on the solutions for modeling issues and information management issues remained ongoing, although the COVID-19 pandemic brought new challenges.

Figure 2 shows that almost one-quarter of studies have used the term “HBIM” in their subjects, abstracts, or titles since 2008 [14]. It seems clear that the COVID-19 pandemic has significantly affected the research, and made distant traveling and personnel-intensive activities difficult. Therefore, online platforms and cloud storage services were developed to meet various kinds of requirements [5,15,16,17,18], such as PetroBIM [19], Web BIM3D [20], and ARK-BIM [21].

Remote practice approaches were favored, e.g., Mandelli et al. (2017) [22], who proposed an integrated online system, BIM3DSG, accessible through the web browser, supporting management of customized information with various model formats; Murphy et al. (2017) [23] presented on-site survey education for students, and discussed system architecture for the virtual learning platform; Napolitano et al. (2018) [24] compiled a virtual tour for observation and then wrote investigation reports; Banfi et al. (2019) [25] developed a VR solution for built heritage, using UE4 and its Blueprints (visual programming tool) to establish virtual tours based on a detailed HBIM; Lee et al. (2019) [26] evaluated the effectiveness of on-site VR application and remote VR application for risk management of the built heritage; Murphy et al. (2019) [27] developed a workflow to disseminate the built heritage using game engine platform and semantic web platform, enabling educational applications to the public; Cantatore et al. (2020) [28] added technical information in a virtual tour environment; Diara and Rinaudo (2021) [29] introduced work-in-progress research on ideal cloud-based BIM platform for data sharing and exchange.

1.3. Background Information of Kuiwen Pavilion

“Confucius did not speak of strange events, violence, riots, and supernatural things.” [30]; therefore, the word “Kui” (魁, means “chief, great”, with a ghost in its strokes) was replaced by the word “Kui” (奎, originally means “legs”, share the meaning of “chief, great” as interchangeable homophones), and the name “Kuiwen” (奎文), generally accepted to mean great literature, scholar, and academic achievement, was given to the Kuiwen Pavilion.

Many Temples of Confucius were found across East Asia, including in China, Korea, Japan, Vietnam, Indonesia, and Malaysia. Since 1994, the Temple of Confucius in Qufu was a UNESCO World Heritage Site: the “Temple and Cemetery of Confucius and the Kong Family Mansion in Qufu”. These three sites are collectively known in Qufu as San Kong as The Three Confucian (sites) [31].

The Kuiwen Pavilion (Figure 3) in Qufu was firstly built in 983 AD as a library in the largest and most renowned Temple of Confucius in Qufu, Shandong Province, China, which was the hometown of Confucius.

Teenage students that are subjected to many exams would like to pray for good grades; therefore, the hometown of Confucius has become the best destination on summer vacations. Kuiwen Pavilion symbolizes both the great literature and success in exams, without performing the functions of a library. Consequently, the online virtual tour built for survey education would also provide access to the tourist that could not visit this attraction.

1.4. Overview of Survey Education

The Field Trip for Measured Survey of Built Heritage, i.e., the survey education course, was carried out by Tianjin University every summer vacation for grade two undergraduates. Through on-site practice, students can learn to acquire 3D data, investigate conditions, take photographs of component details, classify component libraries, take observation notes, and finally, accomplish HBIM deliverables.

In the summer vacation of 2001, the Kuiwen Pavilion was surveyed for the first time, documented in 2D drawings and film photographs. For survey education and updating the data, it was surveyed again in 2021 with lots of new methods applied. The challenges posed by the pandemic also brought opportunity, because there were fewer visitors to this popular tourist attraction. To avoid the risks of personnel gathering, large numbers of students could not attend the on-site practice. Instead, they could access an online platform, which included a virtual tour and cloud storage, to finish the survey education course. The HBIM lab sent a team of five people for the on-site investigation. During two days of work, the survey team acquired data including aerial and ground 3D panorama photographs, orthophotos of textures on the architecture component, mesh models of details by photogrammetry, and a point cloud with a laser scanner. The online platform could be built from these data.

This article thus proposes remote practice methods as alternative solutions for HBIM survey education, to overcome the spatial and temporal barriers created in the post-pandemic context, and to share valuable cultural heritage with the public.

2. Preparations: Materials and Methods

2.1. On-Site Data Acquisition

The spacing of each site for panoramic photograph acquisition should be as dense as possible, aiming to replace on-site observations to a certain extent, by generating a virtual tour without blind spots. For example, when acquiring photographs in the corridor, one site per bay or one site per beam was selected; when acquiring data under outer eaves, one site per one bay was selected; for special locations, such as a stone tablet, surround acquisition was essential (Figure 4).

During ground acquisition, a countdown of 3 s to 8 s was set so that the photographer could move to the appropriate position to avoid occupying too many areas for image acquisition. If possible, GPS functions were activated to auto-link hotspots in the virtual tour.

During UAV acquisition, the stability needed to be ensured during exposure to avoid stitch problems.

Three-dimensional scanning and detailed photo shooting could also use these sites.

To avoid overexposure in bright parts, or no images in dark parts, if the light contrast is strong, 3D panorama photographs were taken in raw format. Raw files could be post-processed in Camera Raw, a plug-in of Photoshop. As a result, both the bright parts and dark parts preserved solid images (Figure 5).

In the completely dark environment of the blindstory, raw photographs were taken with a Canon 6D camera with an 8 mm fish-eye prime lens, a 720 panoramic camera pan, and a group of fill lights (Figure 6).

In all conditions, for both ground devices and UAVs, raw formats are most suitable because they are faster, saving on-site time and processing in the lab, i.e., stitching panoramas, adjusting the brightness.

2.2. Standardized Classification with Cloud Storage

The students did not practice on-site investigation; therefore, the photos and point cloud were important for observations. As a result, an online classified directory of photos was developed, i.e., “/BimProjects/KuiwenPavillion/4Node/4-5WoodenEnvelope/4-5-1Ceiling/4-5-1-1Front” (Figure 7).

The photographs were taken according to the classification. The point cloud, however, was post-processed to match the classification. During the HBIM modeling session, students could find the right folder by searching keywords, then possibly create shapes by referencing both the photographs and the point cloud.

2.3. Online Education

During the on-site investigation, the teacher started the panoramic live streaming of the research on a short-video mobile application (i.e., https://www.kuaishou.com/, accessed on 17 August 2021), using Insta360 OneX2. They would also learn from the online course platform (Figure 8).

The students were free to rotate the directions on their screens while learning. The investigation methods were taught through demonstration by the on-site team. The online course platform provides a virtual experiment of survey skills, as well as video clips (http://www.ilab-x.com/details/v4?id=4931&isView=true, accessed on 16 November 2021).

2.4. Virtual Tour Generation

A total of 149 photographs captured in 3D panorama format were imported to Pano2vr (Figure 9) to generate a virtual tour. Notably, the panorama photographs could be auto-linked with hotspots to become tour nodes if they contained GPS data, which were supported by the DJI drone and not supported by Insta 360 Pro or Canon 6D (but available by plugging in a third-party GPS USB-disk).

While building the tour, each node was linked to other nodes by as many hotspots as possible, or at least the nearest hotspots surrounding the site. Some shortcuts could be created, such as visible access to the door of the second floor, the corridor, a birds-eye view, etc. The tour map and the floor plan were included.

For VR devices, the FoV (field of view) of each tour node needs to be 25.00°, the same as the human eye, for a better experience; for PCs, the FoV should be 85.00°, for a better view on computer monitor and control by mouse.

2.5. Platform of Cloud Service

A NAS (network-attached storage) cloud service developed on Synology.com (accessed on 6 July 2020) was deployed. With the Web Station toolkit for Synology, the virtual tour could be freely accessed. First, the Web Station was used to create a long Virtual Host on any possible Port, as long as it was simple to remember, such as 8000, 8001, etc. The default document root was the web folder. Secondly, the File Station was used for document management and was the output folder for uploading the virtual tour. Finally, index.html was modified with the path of each tour folder. The process is shown below in Figure 10, Figure 11 and Figure 12.

The Synology NAS service also provided a file sync function, i.e., Synology Drive, enabling all students to collaboratively build the same Revit project, the Kuiwen Pavilion.

2.6. The Schema of the HBIM Component Library and Suggested Workflow

The HBIM component library includes two categories according to the modeling and assembly method: automatic generative modeling then automatic assembly; direct modeling then manual assembly.

(1)

Automatic generative modeling and automatic assembly. For components with better regularity and large quantities, such as roof tiles (also including roof-like components such as a canopy made of polished, transparent seashell tiles), wood roofing base, floor tiles, wall tiles, ceilings, indoor wood partitions. Stored as GH and 3DM files.

(2)

Direct modeling then manual assembly. Components with little difference from modern buildings, such as stairs, steps, doors, and windows, can be modeled and assembled directly with the basic functions of BIM software. Alternatively, special-shaped components, or “artistic components”, which lack regularity, such as ridge ornaments, strange stone steps, plum branch window frame, etc., need to be automatically generated as mesh surface models according to laser scanning or photogrammetry, then assembled manually. Stored as 3DM, DGN, and RFA files.

When the students started modeling, they reviewed the observation notes and condition information, to decide which method category should be browsed, and which level of detail should be applied. The level of the model definition (LOD1.0~4.0) had to be consistent with the level of investigation, i.e., the same level of information detail (N1~4). The level of geometric detail (G1~4) was free to choose, with different levels for different sub-models, according to the requirement of delivery (

Table 1. Examples of LODs.

	Roof Tiles	Roof Tile Rows	A Flat Tile	The Top Surface
Surface	LOD1.0/G1/N1	LOD2.0/G1/N2	LOD3.0/G1/N3	LOD4.0/G1/N4
Solid	LOD1.0/G2/N1	LOD2.0/G2/N2	LOD3.0/G2/N3	LOD4.0/G2/N4
True shape	LOD1.0/G3/N1	LOD2.0/G3/N2	LOD3.0/G3/N3	LOD4.0/G3/N4
True shape with patterns	LOD1.0/G4/N1	LOD2.0/G4/N2	LOD3.0/G4/N3	LOD4.0/G4/N4

).

The rebuild workflow of HBIM could be concluded in Figure 13.

2.7. Event Log

As shown in

Table 2. The survey event log by timeline.

Event	Data	Session	Software	Hardware
Laser scan	Point cloud	On-site		FARO Focus S350
Laser scan	Point cloud	Post-processing	Cyclone	Workstation
Laser scan	Mesh model	Post-processing	Geomagic Wrap	Workstation
Photogrammetry	Photo	On-site		Canon M2
Photogrammetry	Mesh model	Post-processing	Context Capture	Workstation
Photography	Photo	On-site		Sony RX100
Photography	Photo database	Post-processing	Synology Manager	Synology NAS
Virtual tour	3D Panorama photo	On-site		DJI Mavic 2 Pro
Virtual tour	3D Panorama photo	On-site		Insta360 Pro
Virtual tour	3D Panorama photo	On-site		Canon 6D
Virtual tour	3D Panorama photo	Post-processing	Camera Raw	Workstation
Virtual tour	3D panorama tour	Post-processing	Pano2VR	Workstation
Virtual tour	Website page	Post-processing	Visual Studio	Workstation
Virtual tour	VR experience	Online	Firefox reality	HTC Vive Focus 3
Cloud Service	Website database	Online	Synology Manager	Synology NAS
Observation	Notes	Online	Firefox explorer	PC
Collaboration	HBIM model	Online	Revit	PC
Optimization	Algorithm	Post-processing	Rhino, Grasshopper, VisualARQ	Workstation

, during the survey education, different software was applied for different events.

Laser scanning is slower and suitable for general investigations, whereas photogrammetry is faster and suitable for capturing details. All preparation work was processed on a high-performance workstation, while the students could experience the virtual tour, take observation notes, and then collaboratively develop the HBIM model on their PCs.

3. Results: Remote Practice on Kuiwen Pavilion

3.1. Virtual Tour Experience

After the preparation finished, students began to test the remote practice methods, try to achieve good grades for the HBIM survey course.

The virtual 3D experience tour was free to access for 24 h through Internet Explorer from any device. To start a quick tour, most students chose to use Google Cardboard with a personal smartphone for a 3D experience, because they could access it anytime and anywhere. Images from the tour are shown in Figure 14.

For VR devices such as HTC Vive Focus 3 (All-in-One VR device) and HTC Vive Pro 2 (PC and VR systems), the 3D virtual tour was accessible if Firefox Reality Internet Explorer was installed. The All-in-One VR device was chosen for the virtual tour because it was not convenient to experience a virtual tour with numerous wires, sensors, and a high-performance PC taking up space in the room.

3.2. Observation Notes

However, to take thorough observation notes while taking the virtual tour, most students preferred to use a PC through Internet Explorer for the convenience of handwriting while observing the screen.

The standard observation notes taken from the virtual tour consisted of several parts: the level of model definition (LOD) [32], the classification of the architecture component, mathematical logic in the structure, and the spatial relationship of each component. The component library was summarized by classifications from their geometric features and tectonic logic from the observation notes, e.g., Figure 15.

Combining the observation notes with the condition investigation text reports from the historical archives, a database including all kinds of damage states and interventions could be summarized (Figure 16). When students observed damages to the heritage site in the virtual tour, they could check the list line by line, to clarify whether the damage was pre-existing and whether interventions had been undertaken.

3.3. HBIM Modeling and Information Association

To start the regularized rebuild workflow for HBIM modeling, the grid system was generated initially. Due to its Chinese traditional wooden carpentry architectural system, all of the components in the Kuiwen Pavilion were found to have deviations to some degree, e.g., Figure 17.

Therefore, the first step of the regularized rebuild is data registration of the grid system, i.e., creating a regular grid system from a deviated one. To make the solution computer-readable, a finite element analysis model of the grid system was created.

The finite element analysis model of the grid system simplifies the problem to several unknown quantities, known quantities, and a goal. The unknown quantities are axis spacing and the angle of the first horizontal axis. The known quantity is that the grid is rectangular, i.e., the horizontal and vertical axes are orthogonal. The goal is, the average value of the distance between every rebuild grid point and original grid point is expected to be minimum. Using the genetic algorithm to build an automatic solution, the whole process only takes 2~3 min.

The students used this self-developed algorithm in Grasshopper to generate a regular grid system [33]; as a result, all of the columns and beams were linearly arranged instead of randomly distributed, e.g., Figure 18.

With the rebuilt axis, all of the components could be located, meaning that each student could rebuild one part of the building at the same time, with no need to wait for each other’s progress. Each component in the Revit model (Figure 19) that contained condition data could be linked to the Damage States Database by component name, with visual illustration in Navisworks, e.g., Figure 20.

3.4. IFC Delivery of Mass-Produced Components

During the HBIM modeling session, one issue was frequently asked by the students: how to mass-produce the regularly assembled model elements with auto-numbering, i.e., the big roof, and how to lightweight the model for IFC delivery? The solutions were evaluated and have proved to be effective, detailed as follows.

Due to the uniqueness of traditional Chinese architecture, the roof unit (IfcRoof) consisted of two parts: the wood roofing base (IfcRafter) and the roof tiles (IfcCovering). Both are mass-produced components; hence, there were at least 10,000 elements. However, the roof unit of the Kuiwen Pavilion, similarly to other main buildings in an architectural group, was built in the shape of a double-curved surface. As a result, a simple modeling method such as the array was no longer appropriate for the roof unit; additionally, the large size of the model brought about difficulties to the students when operating their PCs.

Therefore, an optimization method was proposed by the authors, where all mass-produced components should be classified into several prototypes, as shown in Figure 21. Prototypes are linked to the Block Definition generator from VisualARQ (a Grasshopper plugin). Then, use the Block Instance generator to call each Block Definition with processing through a group of generators including Move and Rotate (three rotations in the XY, XZ, and YZ planes), viewed in xBim Xplorer (Figure 22).

Before optimization, a total of 27,692 polysurfaces in the 3 dm model occupied 1284 MB of hard drive storage. After optimization, 26,364 Block Instances in the IFC model occupied 70.4 MB of the hard drive storage. Thus, the proposed optimization method succeeded in reducing the file size, which was applied to the case study.

4. Conclusions

In the post-pandemic era, a decrease in tourist flow has led to a decrease in ticket revenue for all heritage sites. Meanwhile, online virtual activities have thrived, and are no longer regarded as a supplement to offline real-world activities. Therefore, digital documentation was highlighted as important as part of the fundamental work for heritage virtual visiting, as well as online education. This paper has proposed the procedure of remote practice methods (Figure 23).

The On-Site and Post-Processing sessions provided preparation materials, whereas the Online session provided remote practice.

Students first accessed the Synology cloud service, and then took observation notes by learning from the mesh model and the investigation photographs stored in the classified directory, and the virtual tour. They summarized the classification of components, along with the selection of modeling platforms. By selecting the suitable modeling method for different kinds of components, they could test if the selection was right, and the teacher gave scores on their selection. If the selection was wrong, the student was assigned to fail, and there was no need to check the model details, saving a lot of time. After the selection, the students inputted the suitable parameters to the Grasshopper algorithms, such as the number of roof tile rows, the height of the beam, the type of the column, etc (Figure 24). The teacher gave scores according to the parameters rather than the models, which was much more efficient. As for some irregular or “artistic” components (Figure 25), manual discrimination from the model detail was still required. Finally, the HBIM deliverables were finished.

The HBIM deliverables and virtual tour of the Kuiwen Pavilion as digital twins represent access to online education, VR experience, and remote worship, for both researchers and the public. During the process of survey education, issues occur in the modeling and assessing of condition information; the procedure was proven as an efficient solution, overcoming spatial and temporal barriers.

This process is transplantable, repeatable, and could be applied to most World Heritage Sites; we hope to share this exploration technique with fellow researchers.