Four-Dimensional Digital Monitoring and Registering of Historical Architecture for the Preservation of Cultural Heritage

Abstract

Preserving cultural heritage through monitoring, registering, and analyzing damage in historical architectural structures presents significant financial and logistical burdens. Developed approaches for monitoring and registering 4D (4-dimensional)-scanned range and raster images of damaged objects were investigated in a case study of historical Baron Palace in Egypt. In the methodology, we first prepared and observed the damaged historical models. The damaged historical models were scanned using a laser scanner at a predetermined date and time. Simultaneously, digital images of the models were captured (by a calibrated digital camera) and stored on a researcher’s tablet device. By observing and comparing the scanned models with the digital images, geometric defects and their extent are identified. Then, the observed data components were detected on the map. Then, damaged statue materials were investigated using system of energy dispersive (SEM; scanning electron microscope, Gemini Zeiss-Ultra 55) and XRF (X-ray fluorescence) spectroscopic analysis to identify the statue’s marble elements, and the results indicate that SEM-EDX and XRF analyses accurately identify major and minor compositions of the damaged statue. Then, the damaged models were registered in two stages. In the registration stages, the corresponding points were determined automatically by detecting the closest points in the clouds and ICP (iterative closest point) algorithm in RiSCAN. The point clouds (of the Palace and damaged statues) gave very detailed resolutions and more realistic images in RiSCAN, but it is a costly program. Finally, the accuracies of the registration tasks were assessed; the standard deviations are within acceptable limits and tend to increase irregularly as the number of polydata observations used in the registration calculations increase.

1. Introduction

Historical structures such as palaces and statues are valuable monuments of cultural heritage that play an integral role in fostering sustainable development. However, they are susceptible to various disasters, climate change, problematical topsoil, Earth’s water fluctuations, and human intervention. These factors can cause significant damage to their structural integrity, rendering them unsafe for visitors and threatening their preservation.

Therefore, it is imperative to promptly identify and assess potential structural damage of historical structures to prevent catastrophic failures. This motivates the need for a reliable method to detect structural damage in historical edifices and anticipate any potential damage that can happen at an early stage. Premature detection can help prevent further damage and ensure the long-term preservation and safety of historical structures.

Cultural heritage sites are invaluable assets that must be protected and maintained. Accurate and updated documentation is essential for this purpose and can be achieved through comprehensive structure documentation. This documentation is crucial for reconstruction efforts in the event of damage, as well as for extracting geometrical data and monitoring the integrity of the structure. Many methods have been used to detect structural defects in historical edifices and anticipate any potential damage that can happen at an early stage. Advanced imaging and scanning technologies, such as laser scanning and photogrammetry, have engendered a paradigm shift in the documentation and analysis of historical structures. These sophisticated tools provide high-resolution 3D models that are crucial for detecting structural defects, monitoring changes over time (the fourth dimension), and planning restoration of the damaged edifice [1,2]. Techniques like synthetic aperture radar (SAR) imaging, especially through systems like COSMO-SkyMed, have revolutionized the study of archaeological landscapes, prospection, and condition assessment of cultural heritage sites [1].

Digital elevation models (DEMs) derived from SAR data provide precise topographic maps that are essential for surveying surface archaeological features. These methods enable archaeologists to uncover and map previously unknown sites, monitor environmental impacts, and assess damage from humans. Key computational methods for simulating structural collapses include the finite element method (FEM) and discrete element method (DEM), and physics engines have also been used in examining collapse resistance and predicting pre-collapse structural damage of historical buildings [2,3,4,5].

These models help in planning effective preservation strategies and ensuring the longevity of these structures. Hybrid architectural visualization and computational simulation, which combine numerical unit blocks and a physics engine (i.e., bullet constraints builder), were utilized to explore the process of failure development, its consequences, and potential resolutions for the Ming Great Wall of China [6]. While commercial FEM programs are highly effective in analyzing collapse resistance and accurately predicting pre-collapse structural damage, they fall short in simulating post-collapse behavior.

One of the effective methods is 3D modeling, which enables accurate measurements and reconstructions [7,8]. Three-dimensional laser scanning is a new measurement technique for gathering topographic data points on the surfaces of objects quickly and precisely. The acquired data points are specified in general using 3D (x, y, and z) coordinates together with the intensity of the laser ray [9].

In Egypt, the application of laser scanning technology for archaeology and heritage protection is a relatively recent development. The Belgian millionaire, Baron Empain, made the decision to build a private home in a desert region that became known as “Heliopolis”, or the city of the sun. What makes Baron Palace so unique is that it started out as a standing monument in Egypt that was constructed by a foreigner in 1905. Because fieldwork is non-destructive, this is a unique instance of the nation’s house heritage where the geophysical survey has the potential to image the urban and suburban arrangements of ancient and modern history. Different photos of Baron Palace are shown in Figure 1. A is the oldest photographic image from around 1924 [10]; B is an old photographic image from around the middle of the 20th century [11]; C is a digital image captured by a Canon digital camera in 2010 by the first author; D is a digital image of the Palace after the restoration project in 2023 [12].

Tracking changes to famous buildings throughout their history underscores the critical need for accurate and reliable data in building monitoring. Adding to building inventory and retaining pertinent data are two benefits of using building information modeling (BIM). The precision of 3D point cloud models was evaluated and data registration using the iterative closest point (ICP) algorithm was explained in [13] that emphasized the benefits, limitations, and possibilities of combining multiple sources of LiDAR (light detecting and ranging) point clouds for building modeling, stressing the importance of precise and effective data collection and processing.

An intriguing problem that many countries face is the preservation of some old and historic buildings [14]. As the need to monitor the technical state of structures is established to avert crises, the subject of evaluating the state and functional capability of historic structures has gained importance in recent years [15]. Reliable inspection methods are required in order to identify certain important structures early [14].

Remote sensing by TLS (terrestrial laser scanner) is a significant instrument to detect three-dimensional data to become valuable information [16]. According to [17], 28 studies using laser waves to capture geometrical and spatial data approved the use of laser scanners [16]. Forming a model or 3D shape, such as meshes, NURBSs (non-uniform rational basis surfaces), or solids, is known as data modeling. The deduction of geometric information and data reduction are terms used to describe the processing of a point cloud. It is also a crucial step in the data transfer process for CAD or GIS systems. Post-processing, however, might take a long time, depending on the modeling techniques and required levels of details [16,18].

This research presents a novel method of monitoring and registering of 4D historical building defect information model (HBDIM). It gives parameters that should be established for monitoring, modeling, and fixing historical defects to preserve culture heritage. The paper examines the results of applying a 4D digital modeling technique to a case study of the historical Baron Empain Palace in Egypt.

The key novelty of this study is the creation of a 4D historical building defect information model (HBDIM). This model leverages real-world 3D data (including x, y, and z coordinates) to capture changes over time, which is the fourth dimension. This eliminates the need for time-consuming and expensive reconstruction of irregular models of damaged historical elements, facilitating cultural heritage preservation.

The second research purpose will focus on examining the key parameters of detecting, registering, and analyzing defects in historical building, and this will be based on 4D data with different accuracies in Baron Palace. To achieve this objective, a range of sensors such as laser scanner and digital cameras, SEM-EDX, and XRF will be used for 4D monitoring, registering, and analyzing the defects of the historical structure. The outcomes of this study through the addition of 4D LiDAR flaw data to HDIM and the simplification of the data presentation process will enhance historical building defect observations, as illustrated in the study methodology in the next section.

2. The Methodology

Figure 2 illustrates the typical activities performed on the historical Baron Palace, outlining the sequential order of the work. More details and discussions about the proposed method follow in the next sections.

2.1. Preparation and Calibration for Historical Observations

In first step of the methodology, the defective historical models of the Palace and statue are scanned by the sensor (laser scanner) at a scheduled date and time, and the same model’s digital images (that are captured at the same date and time) were prepared in the researcher’s tablet device. Then, we observed and compared the two models to detect the geometric defects and their limits.

The calibration of the digital cameras involved gathering the necessary data to calculate the camera model parameters, which is the initial step. Following this, the image coordinates of the object were determined. This involved identifying the image points, linking the objects to these points, and establishing the correspondences. The utilized tools and instruments are mentioned in the next sections.

2.2. Detection of Observed Data Components

In second step of the methodology, the observed components of the defective historical models of the Palace were identified on the maps and recognized on the 4D model of the point clouds of the Palace, which assisted the researchers to detect the defective components, as mentioned in detail in the discussion section.

2.3. Historical Defect Data and Materials Investigation

In the third step of the methodology, the defective historical models of the Palace and statue were imported into the modeling software. Then, the defects were identified by their characteristics in the modeling program. Subsequently, the identified defect characteristics were cross-referenced with supplementary data sources, including defective digital images of the palace and statue. A more comprehensive elaboration on this process can be found within the discussion section.

Material and Morphological Analyses

The surface morphology of the marble samples of defective statue was investigated using SEM attached with system of energy dispersive (SEM; scanning electron microscope, Gemini Zeiss-Ultra 55). The elemental composition of the marble was determined using EDX.

X-ray fluorescence spectroscopic (XRF) analysis was employed to identify the marble elements without causing any damage to the samples [19,20,21]. The results of the materials investigations are discussed in detail in the analysis section.

2.4. Defective Model Registration

In the fourth step of the methodology, the system was planned to develop the registration between the 4D geometric information of the historical model from the point clouds and the digital images of the Palace and statue and improve the performance and reality of the 4D information models of the defective historical objects. The registration task was based on matching points. Matching points are defined by artificial target viewing in the scans and are precisely located using automated total station. The defective model will be aligned in its real position and direction in the 4D historical object. The registering procedure was performed in 2 stages:

• Coarse Registering: Registers roughly the point cloud datasets of the two models with each other.
• Precise Registering: Exactly locates the finest alignment and the fine registration to align the point clouds of the two models to have a 4D historical model from the aligned scanned as as-built scenes by using principal component analysis (PCA), which is suitably fast and strong [22,23].

The discussions and results of the automated fine registration of the Baron Palace scans by applying multi-station registration adjustment in RiSCAN with different parameters were analyzed in Section 4.4 of defective model registration. These parameters are the search radius and the calculation mode; for more details see [24].

• The calculation mode which defines the error calculation using two fitting methods:
  • The least squares method: the square distances of the pairs of points were used.
  • The robust fitting method: the absolute distances of the point’s pairs were used.
• Automatic points registration using iterative closest point (ICP) algorithm:

Corresponding points were determined automatically by detecting closest points of the scans for each point of the clouds (ICP algorithm) [23]. In an attempt to improve scan registrations, the alignment and image data were adjusted using much iteration to compute the finest whole fitting of them, until the errors reached minimum values.

2.5. Defect Information Modeling (DIM)

In fifth step of the methodology, the defect data were modeled to replicate their characteristics to a real-world system. The modeling of laser-scanned points cloud included data processing, which is set of procedures to process and develop the suitable laser scans; this will be studied in the next research paper.

2.6. Accuracy Assessments

In sixth step of the methodology, in order to evaluate the modeling and the registration tasks of the defective historical models, the total standard deviation S was used to show the geometric precision of the modeling and registration of the models:

$$S=\pm \sqrt{\frac{{{\displaystyle \sum }}_{i=1}^n{v}_i^2}{n-1}}$$

where n is no. of distances of the matching points of the models and v is the residual (i.e., the difference between the measured distance and the most probable value for that distance). The modeling software programs were used to automatically calculate the standard deviations during the process of modeling and the registration of the models.

2.7. Conservation of the Defective Models

In last step of the methodology, the defective models of the Palace were repaired and rehabilitated. So, samples were taken from damaged part below the base of the statue to characterize it to provide the information about the composition of elements. The plan for the statue of Baron Palace presents a strategy aimed at safeguarding it. Grounded in research and practical considerations, the plan incorporates interdisciplinary methods to achieve a nuanced and well-balanced approach to the conservation and will be researched in the next paper.

The tools and instruments that were used in the practical study are illustrated in the following.

3. The Utilized Tools and Instruments

• Canon EOS 5D digital camera was used (to take the photos of the Palace and the two statues, by the free-hand camera method, without being attached in the laser scanner). Leica automated total station was used in the calibration of Canon digital camera by observing the artificial targets that were placed on the control field as shown in Figure 3. (Z + F) imager5006i laser scanner, as shown in Figure 4, was used in the laser scans of Baron Palace and the statues in fieldwork.

2.

The captured digital images by Nikon (Tokyo, Japan) D300 digital camera were used in the calibration task of the RiSCAN. Nikon digital camera was mounted on the top of Riegl LMS-Z620 laser scanner (in the laboratory test), as shown in Figure 5.

3.

Laptop computer, as shown in Figure 5, and 3D modeling software programs that were used in the laboratory test works, like RiSCAN-Geomagic Studio-AutoCAD 3D.

4.

Artificial paper targets (were placed in Baron Palace) as white circles with black background, as shown in Figure 3; black and white squares, as shown in Figure 6, at right; and circular reflective targets on the front defective statue, as shown in Figure 6 at left.

5.

Captured homogenous digital photos of Baron Palace and the front defective statues by Canon (Nagasaki, Japan) 5D digital camera with a fixed focal length (20 mm) are shown in Figure 6 at right; these are discussed in the results, analysis and discussions in the next section.

4. Results, Analysis, and Discussions

The results of the detecting, monitoring, and registering methods of the spatial data of the historical building and the damage to the two statues using defect information models (DIMs) and laser scanning to preserve the culture heritage (as mentioned in the methodology) are in the following subsections.

4.1. Observation Data Preparation and Calibration Results

As mentioned in the first step of the methodology, the 3D models of the defective Baron statue (without its head and its arm) were scanned at a specified date and prepared in RiSCAN, as shown at the left of Figure 6, and the digital images of the statue were captured on the same date, as shown at the right of Figure 6, with some artificial paper targets that were observed and detected.

The 4D analysis of the two statue models was matched and compared to identify geometric defects in the head, arm, and other body parts, as well as to determine the extent of these defects. From the left of Figure 6, it is observed that the 3D point cloud gave very detailed resolution and a more like-like image of the palace and the defective statue in RiSCAN than in other modeling programs, because RiSCAN has a high detection of the point’s intensity values of the laser scans. However, the point cloud navigation and handling in RiSCAN were hard because of the high number of point clouds (tens of millions of points).

Digital Camera Calibration Results

As mentioned in the camera calibration in the methodology section, RiSCAN software (version 1.6.4) was utilized by the first author to calculate the following: In Baron fieldwork, a Canon digital camera was calibrated; the calibration field (as shown in Figure 3) was used for the calibration process. The calibration field consisted of 228 tie points in the form of white circles with a black background. In the calibration process, more than 18 photos were captured for the calibration field in different levels, horizontally and vertically.

In the laboratory test work results, a Nikon D300 digital camera was mounted on top of a Riegl LMS-Z620 scanner as shown in Figure 5, and the lab area was scanned through a Riegl scanner by operation of tie point images on robotically identified reflectors. After the image acquisition had finished, the camera was calibrated. The main benefits are that the calibration field was built easily and no total station was required. The calibration results were the following: The Nikon D300 camera’s internal calibrations (as interior orientation parameters, IOPs and distortion parameters) are as shown in Figure 7, and the mounting calibration matrix (as exterior orientation parameters, EOPs) were calculated in RiSCAN.

The benefits of the Nikon D300 camera’s calibration parameter results and the exterior orientation matrix were used in the point’s registration between the Riegl laser scans and the Nikon digital images. That was useful in the registration and the accuracy calculation task results; more information is in Section 4.4.

4.2. Detection of Observed Model

The observed components of the defective model of Baron Palace were identified on the map during the time and recognized on the 4D models of the point clouds. So, the observed model of Baron Palace was detected during the time with its 3D coordinates (30°05′11″ N, 31°19′43″ E and 61 m Z elevation) and identified in Google Earth maps, in 2017 before restoration, as shown in Figure 8.

4.3. Defect Data and Materials Investigation Results

Some defect types in Baron Palace were investigated, as spalling damage and cracks in the inner walls, which were constructed of sandstone. The crack measurements were measured by tapes. The defective historical models were imported into the modeling software; then, the defect properties were identified and determined; e.g., in a crack defect, the width of the crack defined its severity. The defect characteristics were determined and compared with the other data sources such as the defect digital images, as shown in Figure 9 and Figure 10. Figure 9 shows

A.

Damage in the upper beam cover and appearance and corrosion of inner reinforcement bar.

B.

Erosion in the column cover and in the roof cover.

C.

Water seepage defect and some cracks and flaking of the slab paints.

4.3.1. Assessment of Materials Condition Results

It is noted that the quality of the marble in Baron Palace and the defective side of the statue is medium and that it has suffered from different weathering forms and conditions, like cracks, micro cracks, and loss of strength as a result of the harsh climate in Cairo. The statues are exposed to different high-value temperatures (38–45 °C) in summer and low temperatures (8–10 °C) in winter. Based on the visual observations of the statue, many vertical cracks were observed, as well as micro cracks and cracks in the statue due to deterioration factors, as shown in the results in Figure 10.

4.3.2. Results of Materials Observation and Components Analysis by SEM-EDX

The characterization of the marble samples was performed using SEM. The SEM-EDX images shows that the samples are rich in quartz and calcite as the major components, as shown in the results in Figure 11 and Figure 12.

4.3.3. Results of XRF Analysis

The sample of the defective side of the statue consists of calcite (Ca) calcium carbonate CaCO₃, as the major component, chloride (Cl), sodium (Na), sodium chloride (Nacl), sulfate (S) sulfate dehydrate CaSO₄·2H₂O (gypsum), and quartz (Si) as shown in the results in Figure 13 and

Table 1. The composition of elements from the sample analysis of the defective side of the statue.

Sample	Statue
Element	Na	Al	Si	S	Cl	Fe	Ca
Con.%	16.27	0.56	2.6	2.09	15.96	2.7	59.82

.

4.4. Results and Analysis of Defective Model Registration with Parameter Study

This subsection analyzes the results of applying RiSCAN’s multi-station adjustment for fine registration of the Baron Palace laser scan images. The analysis explores the relationships between key registration parameters and evaluates the geometric precision achieved through these registration tasks.

Figure 14 shows the fine registration as (the multi-station adjustment) in RiSCAN of two laser scan images of the Baron Palace façade with some registration parameter results in the right window of the figure.

The first author aligned the matching defective historical models on the proper locations and orientations. The laser-scanned defective models were defined and registered in the same coordinate systems by using the well-defined artificial target points. That was performed to compare the laser-scanned models and the digital image models to detect and flag the geometric defects in them.

The laser scans of Baron Palace were registered by some matching points (e.g., the natural targets and well-identified features like edges, corners, and high-amplitude points). To accelerate this process, the point clouds were filtered (were reduced). Figure 15 is the result of the filtration and the registration of 25 polydata laser scans (the reduced point clouds) of 12,802,102 points as high-resolution real images of the Baron Palace façade in RiSCAN.

The filtration and the registration of five polydata laser scans of 605,066 points as high-resolution real images of the defective front statue in RiSCAN resulted and are shown in Figure 16.

Results of Fine Registration and Parameter Analysis

In the fine registration adjustment results in RiSCAN, the total standard deviation parameters of the corresponding distances are displayed among the sets of the polydata scans (the reduced point clouds after filtration) to study the geometric precision parameters of the points registration (given in this section). The registration distance was computed as the mean normal distance between both registered surfaces.

The first author displayed some of the registration parameter results about the used polydata in the charts as illustrated in Figure 17, Figure 18, Figure 19, Figure 20 and Figure 21. The paint of the points is visualized based on the total distance between the two surfaces of the observation (red = minor errors and blue = huge errors). In these diagrams, the histogram of the remaining errors (residues) displays how several observations had a convinced space between the two planes (surfaces) of the observations.

Once the analysis was performed (it’s meaning that the search for the corresponding point pairs and the analysis of the errors, without any modifications to the positions and the orientations of the scans images). That in order to study the relation between the input of the numbers of the observations as polydata numbers and the output of the standard deviation values of the spaces (the distances) among the sets of the polydata scans. For example, Figure 17, shows that the polydata point number count is up to 2500 at a distance = 0.125 m between the two registered planes (the surfaces) of the observations (the registered point clouds of the two scans images of the Baron Palace façade).

From these figures, the search for the corresponding point sets and the investigation of the error parameters with the modification of the positions and the orientations of the scans images were performed, and it was found that the standard deviation error values (of the registration spaces among the sets of the registered polydata scans) increase irregularly with an increase in the polydata numbers of the observations that were used in the registration calculations, as shown in the results in

Table 2. The standard deviations error values (of the registration spaces among the sets of the registered polydata scans) increase irregularly with increasing polydata numbers of the observations.

Polydata No.	Errors (Std. Dev.) (m)
8950	0.023
10,371	0.035
12,488	0.036
13,372	0.036
35,187	0.038
69,567	0.042

.

5. Future Works

5.1. Results of the Defect Information Modeling DIM

From the defect data modeling proposed in the methodology section, the historical data are processed to acquire the modeling results as will be investigated and analyzed in the next research paper.

5.2. Results of the Conservation of the Defective Models

The defective models of Baron Palace and the two statues are repaired and rehabilitated. Grounded in research and practical considerations, the plan incorporates interdisciplinary methods to achieve a nuanced and well-balanced conservation approach. This will be further discussed in our next research paper.

6. Conclusions and Recommendations

The analysis of the outcomes demonstrates that the existing techniques of data detection, investigation, and registration in the case study can be effective to identify and register the potential defects within historical objects of Baron Palace and the defective statues as discussed in Figure 6, Figure 9, Figure 10, Figure 11, Figure 12 and Figure 13, thus ensuring the preservation of cultural heritage monuments promptly and precisely. These defects were inspected visually in the historical models, and this was the simplified method that was proposed in the research objectives. A 4D historical building defect information model (HDIM) was created reflecting its real-world condition based on various data sources. This significantly reduces the effort, time, and cost required to remodel irregular models of damaged historical elements. It has effectively recognized minor scales defects, which were not able to be detected without using the proposed methodology.

The point clouds give a very detailed resolution and more realistic images (of the Palace and the defective front statue) in RiSCAN. That is because RiSCAN has high detection abilities and displays the intensity of the laser scan point clouds. On the other hand, the point navigating and handling in RiSCAN were not easy, because of the many scanned points (tens of millions of point clouds). While the commercial software RiSCAN gave better results in the combinations between the coarse and the fine registrations, we recommend it with some concerns about its high cost, which presents a challenge. Furthermore, it is used with limitations for advanced options with only a licensed dongle.

The corresponding points were determined automatically by detecting the closest points in the point cloud (ICP) algorithm in RiSCAN. The standard deviations of the registration tasks were acceptable compared with the laser scanning precision and compared with Baron Palace and the dimensions of the statue. The standard deviation error values (of the registration spaces among the sets of the registered polydata scans) increase irregularly as the polydata number of observations increase, which are used in the registration calculations.

The XRF analysis accurately identifies the major and minor compositions of the defective side of the statue. The sample of the defective side of the statue consists of calcite (Ca) calcium carbonate CaCO₃, as the major component, chloride (Cl), sodium (Na), sodium chloride (Nacl), sulfate (S) sulfate dehydrate CaSO₄·2H₂O (gypsum), and (Si) quartz.

Future research will build upon these findings, utilizing nanomaterial for strengthening and strategies for treating the statues’ defects. Laser scanners are the viable technology to replace the other past measuring devices for defect measurements in historical buildings. Because of the advantages of the laser scanning techniques, it is recommended to apply them in data restorations and in historical defect documentation.

Referring to future studies, we recommend concentrating on the usage of artificial intelligence. Progressive AI processes allow for the computerization of the development of investigating laser point cloud data and building 4D defects information models 4D DIM. This significantly speeds up full digitization developments, which are particularly vital when investigating greater and extra composite historical monuments for the preservation of cultural heritage.