Next Article in Journal
An Enhanced Methodology for Creating Digital Twins within a Paleontological Museum Using Photogrammetry and Laser Scanning Techniques
Previous Article in Journal
A Systematic Literature Review of Gamification in/for Cultural Heritage: Leveling up, Going Beyond
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Multiscale 3D Documentation of the Medieval Wall of Jaén (Spain) Based on Multi-Sensor Data Fusion

by
José Luis Pérez-García
,
Antonio Tomás Mozas-Calvache
*,
José Miguel Gómez-López
and
Diego Vico-García
Department of Cartographic, Geodetic and Photogrammetric Engineering, University of Jaen, 23071 Jaen, Spain
*
Author to whom correspondence should be addressed.
Heritage 2023, 6(8), 5952-5966; https://doi.org/10.3390/heritage6080313
Submission received: 28 July 2023 / Revised: 15 August 2023 / Accepted: 16 August 2023 / Published: 19 August 2023

Abstract

:
The medieval wall of Jaén is a historical monument that has suffered from the apathy of institutions in recent years, causing its calamitous current status. This study focuses on the previous geomatic tasks developed to obtain a 3D documentation of this site in order to manage future restoration works. The methodology included the integration of data obtained using several geomatic techniques, such as LiDAR and photogrammetry at three scale levels, from the general to the particular. Therefore, data fusion is the main aspect of this methodology, developed in an attempt to take advantage of the benefits of each technique. The application demonstrated the feasibility of using the methodology in order to obtain a complete documentation, including 3D models, orthoimages and DEMs, at several scales with different resolutions and point densities. In addition, we also considered the accuracy of data and products with respect to the project requirements. Another aspect to highlight is the reduction in acquisition time by minimizing the necessity of conducting classic surveying to obtain georeference data. The results show reliable products for supporting restoration tasks and allowing the development of a BIM application to manage them, but also for the dissemination of knowledge regarding this unknown monument.

1. Introduction

The current availability and capabilities of sensors, acquisition techniques and processing algorithms based on remote sensing allow for quality 3D documentation of heritage, and more specifically of historic buildings, even in cases of complex scenes. Examples of these possibilities are the widespread use of photogrammetric and light detection and ranging (LiDAR) technologies, both aerial and terrestrial, independently or combined, to obtain complete 3D models of historic buildings that supposes, in most cases, a reliable modeling of reality due to their great level of detail and accuracy. These products allow heritage modeling for multiple purposes, such as documentation, restoration and reconstruction projects, augmented reality and virtual tours. In this sense, digital twins [1] are emerging as a valuable tool to connect reality and virtual models due to the synchronic capacity of these systems. Luther et al. [2] described these models and their application in the heritage field considering some dimensions of digital twins, such as those based on sensors and geometry. Considering geomatic techniques, the evolution of the two main technologies commonly used in recent years (photogrammetry and LiDAR) has been impressive. We highlight the following causes:
  • Development of data acquisition systems. The use of non-metric (conventional) and low-cost cameras and 360-degree multicameras, for example, [3,4,5,6,7,8] has increased in these types of studies due to their efficiency in data acquisition and image processing. There are a large number of examples of these applications, but we highlight those related to medieval fortresses [9,10,11] and others focused on complex scenes [5,12,13]. Therefore, the use of photogrammetric techniques is highly contrasted in this type of study mainly due to the geometric and radiometric advantages of the products obtained. On the other hand, scanning devices based on LiDAR have undergone an important evolution allowing their common use thanks to, among other reasons, their efficiency in data acquisition. For instance, a simple conventional scan acquired in a few minutes can document the scene, obtaining a point cloud of high geometric accuracy composed of millions of points. This aspect has contributed to the increase in the number of applications developed for heritage documentation using these techniques. In this context, we highlight those applications mainly based on LiDAR focused on fortresses and other complex scenes [14,15,16,17]. However, most of the studies published in recent years are based on the combination of photogrammetry and LiDAR. In this sense, data fusion based on both technologies allows us to take advantage of their potentialities and avoid their inconveniences, for example, the geometric aspect of LiDAR data and the radiometry of photogrammetry. We highlight some examples related to fortresses [18,19,20,21] and other complex scenes [22,23]. In addition to these techniques, recent studies [24,25,26,27,28] have used portable mobile mapping systems (MMS) to survey complex spaces. In these systems, data acquisition is based on images and/or LiDAR, and navigation and positioning are based on Global Navigation Satellite System (GNSS) and/or Inertial Measurements Unit (IMU) sensors. In some cases, the trajectory of the MMS is based on Simultaneous Localization and Mapping (SLAM) algorithms using images or point clouds (Visual SLAM and LiDAR SLAM). MMS improve the efficiency of data acquisition with an accuracy of several centimeters [26], which can be considered sufficient for most cases. A comprehensive review of current MMS was provided by Elhashash et al. [29]. In this context, we highlight the MMS application regarding the medieval wall of Avila (Spain) [28].
  • New platforms. The widespread use of new platforms to capture the scene from adequate points of view in many cases elevated with respect to the ground, such as Remotely Piloted Aircraft System (RPAS) [30,31,32] and masts [33,34,35,36], have improved acquisition conditions, providing better configurations for documenting complex scenes. These platforms can support different sensors (e.g., cameras and LiDAR). They are remotely controlled and, in most cases, can follow a previously designed flight plan, even in complex scenes [37]. The availability of RPAS platforms (both fixed-wing and rotatory) is continuously increasing, with systems that can fly at low and medium altitude to develop a wide range of applications (e.g., from several meters to 120 m). In contrast, we must consider that their use is not always possible due to safety restrictions. In these cases, the usual sizes of historic buildings can allow the use of masts and elevated platforms to lift the sensors.
  • Processing algorithms and hardware capabilities. The development and application of improved image-based processing algorithms, such as the Structure from Motion (SfM) [38,39,40,41] and the dense Multi-View Stereo (MVS) [41,42,43,44], together with the increase in hardware capacities, have made possible the development of a greater number of applications due to improved processing procedures, such as image orientation. Moreover, recent software applications have democratized the use of these techniques even for non-professional users [45]. On the other hand, improved point cloud registration algorithms, such as the Iterative Closest Points (ICP) [46,47], have also supported applications based on these techniques.
All these improvements related to data acquisition and processing have also been supported by the development of information systems, such as the Historic Building Information Modelling (HBIM) [48], which allows managing historic building data considering other purposes in addition to the 3D documentation (such as virtual reconstructions and structural analysis).
Considering the current possibilities of geomatics, the selection of the technique to be implemented to document historic buildings will depend on several aspects related to the project itself and the availability of instruments. In the first case, we highlight some considerations to be taken into account, such as the size of the scene (which is directly related to the scale and resolution), the products demanded, the accuracy requirements and the complexity of the scene. In the case of the scale and resolution, Lambers and Remondino [49] categorized the geomatic techniques taking into account three scales: regional, local and object. Historical buildings should be included mainly in the local scale category. There is a wide range of geomatic techniques that can be applied considering this scale level. However, the local scale can be subdivided into other subscales considering, for example, the size of the site. This aspect is probably the main variable to take into account in the selection of the data acquisition technique to be applied. For instance, the documentation of small buildings (e.g., a small chapel) will involve the selection of different sensors to those considered for larger buildings (e.g., a high wall). In the first case, terrestrial techniques supported by low-height flight (RPAS) or even using masts may be sufficient to cover the entire scene, while in the second case, medium-height flight is likely to be needed, using RPAS to perform a general survey of the area in addition to the specific and detailed surveys of each part. This is the case of this study, where the characteristics of the scene and the project requirements suggested the development of a methodology that integrates data acquired from various geomatic techniques considering three levels of the local scale.

1.1. The Medieval Wall of Jaén

The province of Jaén (Spain) has the highest concentration of castles and fortresses in Europe [50]. This is a consequence of the large number of battles fought throughout history in this territory (e.g., the battle of Navas de Tolosa between the Cristian and the Almohad forces in 1212) and the fact that the border with the Kingdom of Granada (Nasrid dynasty from 1238 to 1492) was settled in this region for a period of almost three centuries (1212–1492). In the case of the city of Jaén, the conquest of the Cristian forces was dated to 1246 after several sieges carried out from 1225. As a consequence of these circumstances, the city of Jaén was surrounded by different defensive systems based on several walls and towers. Although there are some remains of the medieval wall at various points of the urban area and the slope of the hill of the Santa Catalina (elevated about 200 m above the town), the main remains are located in the northern section of this elevated area wherein the castle is located at the top. In this context, this study focuses on this section of the wall, which has a total length of more than 300 m (Figure 1) and contains about 12 towers and various gates. From a conservation point of view, we must consider that some parts of this wall were repaired in 1980, although the interventions were reduced to some sections without considering the complete wall in the restoration works. The wall currently shows a much degraded state. Except in a few areas, it is very deteriorated by the lack of interventions since it was abandoned several centuries ago. As a consequence of this state, the City Council has finally initiated a consolidation project in 2023, which starts with a previous geometric documentation of the wall, the subject of this study.
The location of the wall on a steep slope (height difference of about 80 m between the lowest and the highest point, which is more than 25% of mean slope) and other circumstances, such as the presence of trees and a power line situated next to the wall (Figure 1), and especially the geometry of the wall, conditioned the documentation work to be developed (Figure 1). In this sense, the wall presents heights of more than 5 m in several sections, some secondary walls attached to the main one, towers located outside the wall that caused discontinuities in the linear geometry and the presence of gates. We have to note the great presence of occlusions caused by these circumstances that conditioned the acquisition work. In addition to these issues, the requirements of the project were the most important aspects in defining the objectives of the 3D documentation to be carried out and the methodology to be developed in order to achieve them.

1.2. Objectives

The project requirements defined the products to be obtained and their resolutions and accuracy. Thus, the need for a complete documentation of the wall at several scale levels, representing it in plans, 3D models and orthoimages (complete zone and for any linear wall section), was the first aspect to be considered. The full area products will be used to analyze the context of the building with respect to the surrounding elements. The detailed products will be used to document the wall in order to plan the restoration procedure, based on building archaeology (stratigraphic survey). This survey will analyze the construction procedures based on the materials and possible modifications and alterations that affected the wall.
Considering this context, the main objective of this study is the obtaining of a complete documentation of the wall based on data fusion obtained from different sensors considering the three scale levels of this complex lineal scene, from the general to the particular. In addition, some secondary objectives are considered, such as the obtaining of 3D documentation of the wall and its surrounding areas in order to contrast hypotheses about the constructive process of the wall and the access path that could exist to communicate the town center with the castle located at the top of the hill.
In this sense, the complexity of the scene characterized by the geometry of the wall itself, a great slope of the terrain and the presence of trees constituted a great challenge compared to other cases related to walls (e.g., [9,18,19]). Obviously, these aspects caused the development of a new methodology adapted to this specific case based on the combination of different geomatic techniques at several scales.
This document is structured as follows: First, a description of the proposed method-ology developed to obtain the products demanded in this study is presented. Second, we present a description of the results obtained and the discussion where these results are analyzed; finally, the main conclusions of this study are summarized.

2. Materials and Methods

The project requirements, based on the objective of obtaining 3D documentation, defined the products to be obtained and their levels of accuracy. These values are shown in Table 1. In addition to these requirements, the complexity of the scene conditioned the selection of the methodology to be developed. In this sense, we chose different geomatic techniques to achieve the objectives of this documentation, attempting to take advantage of their potentialities and reducing their most negative aspects. We defined three scale levels: full area (SL1), complete wall (SL2) and wall section (SL3), considering a minimum resolution and point density related to them (Table 1). The accuracy requirements were in line with these values, following the positional accuracy recommendations for orthoimages and DEMs (e.g., ASPRS standard [51]). Considering the amount of data to be managed at scales SL2 and SL3, the wall was divided into three zones (SL2) and about 70 linear sections (SL3).
Considering the products to be obtained related to the three scale levels, the methodology proposed in this study is shown in Figure 2, where the different geomatic techniques are displayed considering the acquisition time and from the general to the particular study. In this sense, we divided the procedure considering the use of surveying techniques, LiDAR and photogrammetry. After acquisition, data preprocessing included the verification of the surveyed area to ensure the complete coverage of the wall using one or several techniques at each scale level. After processing the data acquired with each technique at each scale level, we developed a data fusion procedure in order to obtain the final products by integrating data from several sources. The selection of the data source considered some aspects related to density, geometry, accuracy and radiometric quality, although, depending on the complexity of the wall in some zones, we could only use a specific technique (e.g., the highest areas of the wall were only surveyed using RPAS photogrammetry).

2.1. Surveying

This stage was developed in order to georeference all products to an official coordinate reference system (CRS). Considering the full area and the wall sections, we measured some control and check points that were well-distributed throughout the study area and along the wall. These points were materialized on the ground using targets. The determination of the coordinates of these targets was developed using GNSS/RTK positioning with accuracies of about 2 cm. The coordinates of these targets were used to georeference those data obtained in the case of SL1 and to georeference (XYZ transformation) the TLS point cloud (high density) obtained in SL3 (Figure 2).

2.2. LiDAR

The LiDAR technique was applied in different ways (aerial and terrestrial) considering the three scale levels, SL1, SL3 and SL2 (Figure 2):
  • SL1: We used an aerial LiDAR survey using a DJI L1 to cover the full area (SL1). This device was mounted on a DJI Matrice 300 RTK RPAS (Figure 3a). The flight was planned previously, considering the terrain and the scene to be surveyed. In this sense, we used the application developed by Gómez-López et al. [37] for planning block flights on a sloped zone. Thus, the flight was conducted at about 60 m over the terrain, considering seven trips. We used the coordinates of several targets well-distributed throughout the area to georeference the LiDAR point cloud. The point density of LiDAR was about 820 points per square meter with a point spacing of 3 cm. The RMS after the LiDAR processing was about 2.5 cm (height accuracy). We used DJI Terra and Lastools software to process and control these data. All points were classified for bare earth extraction (using the Lasground module). This procedure allowed us to detect ground and non-ground points. In addition, we also included those points related to the wall. As a product, we obtained a complete point cloud with color information.
  • SL3: We developed a TLS survey of the wall considering a high-density point acquisition of about 7 mm at ten meters. We used a Faro Focus X130 scanner (Figure 3b). In order to improve the efficiency of acquisition, we developed the capture without considering the color mode. In this sense, the texture of final products depends on the photogrammetry focusing the TLS data on the geometry. The scanning stations were distributed along the wall considering some overlapping between adjacent scans. In addition, the scans were placed considering the geometry of the wall, aiming to capture it completely (except the highest zones, where there was no accessibility). From each scanning station, we obtained a point cloud. All point clouds were registered relatively using cloud-to-cloud algorithms (e.g., ICP [46,47]). The RMS of the registering was about 5 mm. After that, several targets located throughout the scene were used to georeference the final point cloud and to check the results of the TLS procedure. The RMS of this 3D transformation was about 27 mm. From the final point cloud, we obtained the coordinates of additional targets, which were used for orientation and for checking the low and very low flight height photogrammetry (SL2 and SL3) and the close range photogrammetry (SL3). As a product, we obtained a high density point cloud. We used Faro Scene and Maptek Point Studio software to develop these procedures.
  • SL2: The point cloud obtained in the previous stage was filtered in order to obtain a simplified one. In this regard, we used a basic distance threshold of 10 mm. This implied a reduction in the number of points of about 80%.

2.3. Photogrammetry

Photogrammetric surveys (aerial and terrestrial) were also applied considering the three scale levels (Figure 2). All procedures were performed using the Agisoft Metashape software. The main aspects of these survey were:
  • SL1: The image acquisition at this scale level was developed with an RPAS survey at medium flight height and, more specifically, using a DJI Matrice 300 with an RTK integrated module. The flight plan included vertical images following a block flight on a sloped zone [37], with seven strips perpendicular to the slope and maintaining the flight height of each strip with respect to the terrain. The mean flight height was 60 m and the mean GSD was about 2 cm. We used several targets, whose coordinates were obtained from a GNSS survey, to improve the previous orientation of images (camera coordinates obtained by the GNSS-RTK module integrated in the aircraft) and to check the orientation. The RMS after the orientation procedure was about 1.9 cm. As products, we obtained a set of oriented photographs, a point cloud and a texture of the zone.
  • SL2: In this case, the scene covered the wall and its surrounding areas. Therefore, we developed an RPAS survey at a low flight height using a DJI Phantom 4 Pro with an integrated RTK module (Figure 4a). The selection of this aircraft, in contrast to the one used in SL1, was related to the greater maneuverability and efficiency and the need to carry less weight. The flight plan included vertical and oblique images following a combined flight (block and corridor flights) [37] in order to cover the entire wall. The average flight height was about 50 m. This supposed an average GSD of about 1.5 cm. As in the previous case, image orientation was developed using several targets that were well-distributed throughout the scene, although camera positions were pre-calculated from the RTK module. The coordinates of these targets were obtained from the high-density TLS point cloud. Therefore, we limited the GNSS survey to obtaining those targets used for orientation and checking purposes in the case of the RPAS at a medium flight height (SL1) and the high density TLS survey (SL3). The RMS after the orientation procedure was about 1.9 cm. As products, we obtained a set of oriented photographs, a point cloud and a texture of the scene.
  • SL3: At this scale level, we developed two photogrammetric surveys. The first one was implemented in order to obtain high resolution products of the highest areas of the wall. It was performed through an RPAS survey undertaken at a very low flight height using a DJI mini. This aircraft allowed us to acquire closed images of the wall avoiding those issues generated by the presence of trees and other objects (such as poles and lights and power lines). The great maneuverability of this aircraft allowed us to cover the scene completely because of its capacity to position itself in complex and narrow spaces. On the other hand, the lower areas of the wall were surveyed using close-range photogrammetry (CRP) with a conventional camera (Sony A6000) mounted on a mast that allowed us to raise the sensor up to 5 m (Figure 4b). In both cases, we followed the CIPA recommendations for architectural photogrammetric projects using non-metric cameras (called 3 × 3 rules) [52]. In this sense, we obtained normal and convergent images covering the scene from different viewpoints. The orientation of the images was obtained from several targets that were well-distributed along the wall. The coordinates of these targets were obtained from the high-density TLS point cloud. The average RMS after the orientation stage was 1.4 cm. As products, we obtained a set of oriented photographs, a point cloud and a texture of the scene.

2.4. Data Fusion

One of the main points to highlight considering the methodology proposed in this study is related to data fusion. As described previously, we obtained partial products from several geomatic techniques and resolutions. These products showed specific characteristics, including advantages and disadvantages. Our aim was to take advantage of the improvements provided by their combination. Therefore, we considered the project requirements for developing a data fusion stage to obtain the final products. The procedure included the checking of errors using a distribution of independent targets, whose coordinates were obtained from a more accurate source (GNSS and the high-density TLS point cloud). In this sense, we followed the recommendations of the ASPRS standard [51] to develop this assessment. The main characteristics of the data fusion developed in this study are:
  • SL1: Data fusion included LiDAR point cloud and photogrammetry. The LiDAR data were used to obtain the geometry of the terrain avoiding the presence of trees, while photogrammetry was used to provide a high-quality texture. As final products, we obtained a 3D model of the area including the wall, an orthoimage of 1.5 cm of spatial resolution, a DTM (5 cm of spatial resolution) and a topographic map at a scale of 1:1000.
  • SL2: Data fusion of SL2 integrated the geometry of the wall derived from the TLS (lower areas) and photogrammetry (higher areas) and the texture obtained from the photogrammetry. The selection of points from the photogrammetric point cloud was related to gaps in the TLS point cloud (Figure 5a). Thus, we selected those points from the photogrammetric point cloud that were at a certain minimum distance from the TLS point cloud. We therefore obtained a detailed 3D model of the complete wall.
  • SL3: In the case of photogrammetry, we merged two projects that were previously processed independently in order to generate a complete project, including both RPAS and conventional camera images (Figure 5b). After that, we integrated the geometry of the wall obtained using the TLS point cloud and some points selected from the photogrammetric point cloud representing those areas where the TLS had gaps. As in the previous case, the texture was obtained from the photogrammetry. As final products, we obtained an orthoimage (2 mm of spatial resolution) and a DEM (15 mm of spatial resolution) of each section of the wall.

3. Results and Discussion

The results obtained include several products considering each scale level. Figure 6 shows some examples of the documentation conducted in this study. In the case of SL1, we obtained a general orthoimage and a DTM. From the second one, the contour lines of the terrain were extracted and integrated into a topographic map. Finally, we also obtained a general 3D model of the wall and the area to be documented. Considering SL2, we obtained three 3D models (Figure 6). A video of these 3D models is available at https://youtu.be/PaEGSNe3Ovc (accessed on 15 August 2023). Finally, at SL3, we obtained three vertical orthoimages related to the zones analyzed and one orthoimage and one DEM for each wall section (Figure 6). This means about 70 orthoimages and 70 DEMs. All products were referred to the same CRS, and their accuracy was checked using a set of independent checkpoints.
The methodology proposed in this study allowed us to obtain two point clouds from TLS and the photogrammetry of a large part of the wall. As described previously, the geometry of the 3D documentation included in our approach was based on TLS due to the better behavior of this technique considering the characteristics of the object. In this sense, we took advantage of the better definition of the geometry of the wall provided by the TLS due to the simplification in the photogrammetric model caused in internal spaces between the stones (dark areas in the photographs). This effect is clearly shown in the example in Figure 7, where the photogrammetric mesh shows greater distances with respect to the TLS mesh in areas with mortar erosion or holes in the masonry. On the other hand, the photogrammetric survey revealed its feasibility for surveying narrow and complex spaces of the wall. Thanks to the use of RPAS and the mast to lift the camera. the documentation of the wall was achieved in its entirety, albeit mainly in the higher zones.
Another advantage of aerial acquisition is related to the LiDAR data obtained at the SL1 of the area surrounding the wall. Thanks to the data obtained in our study, we identified the old path that was historically used to access to the castle by Muslims. This path was known from historical references, although it had not been mapped to date. Our approach allowed the identification of this path (Figure 8) from the DTM obtained at SL1 using LiDAR data, despite the presence of trees in this area (see Figure 1). This path is clearly identifiable, despite the presence of other more recent paths, which are visible in the DTM representation. Therefore, the use of these techniques is justified in this type of study where vegetation is considerable. This identification has made it possible to analyze the interaction of the old path and the wall, including the location of the gates.
Therefore, data fusion revealed a great potential considering the goal of obtaining a complete coverage of the scene. In this sense, an important aspect to be considered is related to accuracy. Based on the result of the assessment of our products, we can assure that TLS and photogrammetry demonstrated their feasibility for providing accurate products, and they are perfectly complementary in providing a complete 3D documentation of this type of heritage monument. Regarding this documentation, the next steps of the project will consist of the development of a BIM application to manage the restoration of this building. The application will be based on the 3D models developed in this study, and each element of the wall model will contain attributes about materials, construction and restoration history, current state and collapse risk. After the restoration, the 3D documentation obtained in this study will become an important tool for future generations to understand the evolution of this monument. This will confirm how the apathy of institutions put the wall at risk for too many decades, despite being one the main historical examples of the past of the city. In addition, the comparison between the current 3D model and the one obtained after the restoration will allow researchers to calculate the magnitude of the restoration works, facilitating the management of future protection tasks.
Another important aspect to be considered in this study is that related to field efficiency. Obviously, a reduction in field work is always desirable, although this may imply a low increase in office workload. In this sense, the use of TLS to provide the coordinates of the targets to be used for the orientation purposes of photogrammetry greatly reduced the time spent in surveying, avoiding the use of the total station.
Our approach allowed us to obtain a complete documentation of the wall at several scale levels. The use of aerial LiDAR data allowed us to determine the ancient structures that remained unmapped to date. In this sense, we recommend the use of these types of techniques, at least in general and preliminary studies. The combined use of RPAS and terrestrial photogrammetry demonstrated its feasibility for obtaining a complete documentation of the site.

4. Conclusions

The methodology developed in this study allowed us to obtain a complete 3D documentation of the medieval wall of Jaén, which has been seriously damaged by the passage of years without any intervention. This documentation provides a complete tool for modeling the current status of the wall and designing the restoration tasks to be conducted in the coming years. Therefore, the main objective of this study was fully achieved.
The proposed methodology was mainly based on the fusion of data from several sources. The use of different geomatic techniques based on LiDAR and photogrammetry allowed us to take advantage of their principal benefits. We also tried to minimize their disadvantages. In summary, we always had in mind the goal of reducing field work. For example, the use of TLS to obtain the coordinates of the targets located on the wall showed a great increase in efficiency, avoiding the use of the total station in this complex topography. The division into three scale levels is a main aspect to be highlighted due to the different products required at each level and the large amount of data to be considered at the local scale. In this sense, each product at each scale level can be used for specific purposes (for instance, SL1 for general study and SL3 for the specific analysis of a wall section). The combination of data acquired from the aerial and terrestrial platforms allowed us to survey all parts of the wall, including those located in inaccessible places or in narrow and complex areas. In addition, the accuracy achieved with these techniques largely meets the project requirements.
The methodology developed in this study demonstrated its feasibility to document this type of complex historic walls, allowing us to obtain a complete 3D documentation to facilitate heritage conservation and dissemination. In our opinion, it can be easily adapted to other similar historical sites where the complexity of the scene requires the integration of different geomatic techniques at several scale levels.
Future work will include the development of a BIM application related to this monument based on the documentation obtained in this study. This application will allow the management of the restoration works. Once the restoration is completed, a new 3D model will be developed allowing the comparison of models. It will also make it easier for users to visualize the evolution of these works through VR. From a geomatic point of view, we will include new sensors to improve the efficiency of field acquisition, such as MMS and 360-degree multicameras, aiming to guarantee data accuracy.

Author Contributions

Conceptualization, all authors; methodology, all authors; software, J.M.G.-L. and D.V.-G.; validation, all authors; formal analysis, all authors; investigation, all authors; resources, all authors; data curation, all authors; writing—original draft preparation, A.T.M.-C.; writing—review and editing, A.T.M.-C. and D.V.-G.; visualization, all authors; supervision, J.L.P.-G.; project administration, J.L.P.-G.; funding acquisition, J.L.P.-G. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Data Availability Statement

Not applicable.

Acknowledgments

The authors would like to thank the City Council of Jaén and the staff of the Municipal Board of Culture, Tourism and Historical Heritage for their support in this project.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Grieves, M. Digital Twin: Manufacturing Excellence through Virtual Factory Replication, White Paper. 2014. Available online: http://www.apriso.com/library/Whitepaper_Dr_Grieves_DigitalTwin_ManufacturingExcellence.php (accessed on 30 May 2023).
  2. Luther, W.; Baloian, N.; Biella, D.; Sacher, D. Digital Twins and Enabling Technologies in Museums and Cultural Heritage: An Overview. Sensors 2023, 23, 1583. [Google Scholar] [CrossRef] [PubMed]
  3. Ogleby, C.L.; Papadaki, H.; Robson, S.; Shortis, M.R. Comparative camera calibrations of some “off the shelf” digital cameras suited to archaeological purposes. Int. Arch. Photogramm. Remote Sens. 1999, 32, 69–75. [Google Scholar]
  4. Cardenal, J.; Mata, E.; Castro, P.; Delgado, J.; Hernandez, M.A.; Pérez, J.L.; Ramos, M.; Torres, M. Evaluation of a digital non metric camera (Canon D30) for the photogrammetric recording of historical buildings. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. 2004, 35, 564–569. [Google Scholar]
  5. Covas, J.; Ferreira, V.; Mateus, L. 3D reconstruction with fisheye images strategies to survey complex heritage buildings. In Digital Heritage 2015; IEEE: Granada, Spain, 2015. [Google Scholar]
  6. Fiorillo, F.; Limongiello, M.; Fernández-Palacios, B.J. Testing GoPro for 3D model reconstruction in narrow spaces. Acta IMEKO 2016, 5, 64–70. [Google Scholar] [CrossRef]
  7. Barazzetti, L.; Previtali, M.; Roncoroni, F. Fisheye lenses for 3D modeling: Evaluations and considerations. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. 2017, 42, 79–84. [Google Scholar] [CrossRef]
  8. Barazzetti, L.; Previtali, M.; Roncoroni, F. 3D modeling with 5K 360° videos. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. 2022, 46, 65–71. [Google Scholar] [CrossRef]
  9. Arias, P.; González-Aguilera, D.; Riveiro, B.; Caparrini, N. Orthoimage-based documentation of archaeological structures: The case of a mediaeval wall in Pontevedra, Spain. Archaeometry 2011, 53, 858–872. [Google Scholar] [CrossRef]
  10. Drap, P.; Merad, D.; Boi, J.-M.; Seinturier, J.; Peloso, D.; Reidinger, C.; Vannini, G.; Nucciotti, M.; Pruno, E. Photogrammetry for Medieval Archaeology: A Way to Represent and Analyse Stratigraphy. In Proceedings of the 2012 18th International Conference on Virtual Systems and Multimedia, Milan, Italy, 2–5 September 2012; IEEE: Milan, Italy, 2012; pp. 157–164. [Google Scholar]
  11. Sabina, J.A.R.; Valle, D.G.; Ruiz, C.P.; García, J.M.M.; Laguna, A.G. Aerial Photogrammetry by drone in archaeological sites with large structures. Methodological approach and practical application in the medieval castles of Campo de Montiel. Virtual Archaeol. Rev. 2015, 6, 5–19. [Google Scholar] [CrossRef]
  12. Martínez, S.; Ortiz, J.; Gil, M.L.; Rego, M.T. Recording complex structures using close range photogrammetry: The cathedral of Santiago de Compostela. Photogramm. Rec. 2013, 28, 375–395. [Google Scholar] [CrossRef]
  13. Pérez-García, J.L.; Mozas-Calvache, A.T.; Barba-Colmenero, V.; Jiménez-Serrano, A. Photogrammetric studies of inaccessible sites in archaeology: Case study of burial chambers in Qubbet el-Hawa (Aswan, Egypt). J. Archaeol. Sci. 2019, 102, 1–10. [Google Scholar]
  14. Teza, G.; Pesci, A. Geometric characterization of a cylinder-shaped structure from laser scanner data: Development of an analysis tool and its use on a leaning bell tower. J. Cult. Herit. 2013, 14, 411–423. [Google Scholar] [CrossRef]
  15. Castellazzi, G.; D’Altri, A.M.; de Miranda, S.; Ubertini, F. An innovative numerical modeling strategy for the structural analysis of historical monumental buildings. Eng. Struct. 2017, 132, 229–248. [Google Scholar] [CrossRef]
  16. Guarnieri, A.; Fissore, F.; Masiero, A.; Vettore, A. From TLS survey to 3D solid modeling for documentation of built heritage: The case study of Porta Savonarola in Padua. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. 2017, 42, 303–308. [Google Scholar] [CrossRef]
  17. Sánchez-Aparicio, L.J.; Del Pozo, S.; Ramos, L.F.; Arce, A.; Fernandes, F.M. Heritage site preservation with combined radiometric and geometric analysis of TLS data. Autom. Constr. 2018, 85, 24–39. [Google Scholar] [CrossRef]
  18. Stal, C.; De Wulf, A.; Nuttens, T.; De Maeyer, P.; Goossens, R. Reconstruction of a medieval wall: Photogrammetric mapping and quality analysis by terrestrial laser scanning. In Proceedings of the 31th EARSeL Symposium 2011, Prague, Czech Republic, 30 May–2 June 2011; pp. 54–65. [Google Scholar]
  19. Mateus, L.; Fernández, J.; Ferreira, V.; Oliveira, C.; Aguiar, J.; Gago, A.S.; Pacheco, P.; Pernão, J. Terrestrial laser scanning and digital photogrammetry for heritage conservation: Case study of the Historical Walls of Lagos, Portugal. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. 2019, 42, 843–847. [Google Scholar] [CrossRef]
  20. Zaragoza, I.M.E.; Caroti, G.; Piemonte, A. The use of image and laser scanner survey archives for cultural heritage 3D modelling and change analysis. Acta IMEKO 2021, 10, 114–121. [Google Scholar] [CrossRef]
  21. Fabris, M.; Fontana Granotto, P.; Monego, M. Expeditious Low-Cost SfM Photogrammetry and a TLS Survey for the Structural Analysis of Illasi Castle (Italy). Drones 2023, 7, 101. [Google Scholar] [CrossRef]
  22. Colonnese, F.; Carpiceci, M.; Inglese, C. Conveying Cappadocia. A new representation model for rock-cave architecture by contour lines and chromatic codes. Virtual Archaeol. Rev. 2016, 7, 13–19. [Google Scholar] [CrossRef]
  23. Mozas-Calvache, A.T.; Pérez-García, J.L.; Gómez-López, J.M.; de Dios, J.M.; Jiménez-Serrano, A. 3D models of the QH31, QH32 and QH33 tombs in Qubbet el Hawa (Aswan, Egypt). Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. 2020, 43, 1427–1434. [Google Scholar] [CrossRef]
  24. Zlot, R.; Bosse, M. Three-dimensional mobile mapping of caves. J. Cave Karst Stud. 2014, 76, 191–206. [Google Scholar] [CrossRef]
  25. Farella, E.M. 3D mapping of underground environments with a hand-held laser scanner. Bollettino Della Società Italiana di Fotogrammetria e Topografia 2016, 2, 1–10. [Google Scholar]
  26. Di Stefano, F.; Torresani, A.; Farella, E.M.; Pierdicca, R.; Menna, F.; Remondino, F. 3D surveying of underground built heritage: Opportunities and challenges of mobile technologies. Sustainability 2021, 13, 13289. [Google Scholar] [CrossRef]
  27. Fassi, F.; Perfetti, L. Backpack mobile mapping solution for DTM extraction of large inaccessible spaces. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. 2019, 42, 473–480. [Google Scholar] [CrossRef]
  28. Rodríguez-Gonzálvez, P.; Jiménez Fernández-Palacios, B.; Muñoz-Nieto, Á.L.; Arias-Sanchez, P.; Gonzalez-Aguilera, D. Mobile LiDAR System: New Possibilities for the Documentation and Dissemination of Large Cultural Heritage Sites. Remote Sens. 2017, 9, 189. [Google Scholar] [CrossRef]
  29. Elhashash, M.; Albanwan, H.; Qin, R. A Review of Mobile Mapping Systems: From Sensors to Applications. Sensors 2022, 22, 4262. [Google Scholar] [CrossRef]
  30. Colomina, I.; Molina, P. Unmanned aerial systems for photogrammetry and remote sensing: A review. ISPRS J. Photogramm. Remote Sens. 2014, 92, 79–97. [Google Scholar] [CrossRef]
  31. Nex, F.; Remondino, F. UAV for 3D mapping applications: A review. Appl. Geomat. 2014, 6, 1–15. [Google Scholar] [CrossRef]
  32. Campana, S. Drones in Archaeology. State-of-the-art and Future Perspectives. Archaeol. Prospect. 2017, 24, 275–296. [Google Scholar] [CrossRef]
  33. Mozas-Calvache, A.T.; Pérez-García, J.L.; Cardernal-Escarcena, F.J.; Delgado, J.; Mata de Castro, E. Comparison of Low Altitude Photogrammetric Methods for Obtaining Dems and Orthoimages of Archaeological Sites. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. 2012, 39, 577–581. [Google Scholar] [CrossRef]
  34. Ortiz, J.; Gil, M.L.; Martínez, S.; Rego, T.; Meijide, G. Three-dimensional Modelling of Archaeological Sites Using Close-range Automatic Correlation Photogrammetry and Low-altitude Imagery. Archaeol. Prospect. 2013, 20, 205–217. [Google Scholar] [CrossRef]
  35. Blockley, P.; Morandi, S. The recording of two late Roman towers, Archaeological Museum, Milan 3D documentation and study using image-based modelling. In Digital Heritage 2015; IEEE: Granada, Spain, 2015. [Google Scholar]
  36. Pérez-García, J.L.; Mozas-Calvache, A.T.; Gómez-López, J.M.; Jiménez-Serrano, A. Three-dimensional modelling of large archaeological sites using images obtained from masts. Application to Qubbet el-Hawa site (Aswan, Egypt). Archaeol. Prospect. 2018, 26, 121–135. [Google Scholar] [CrossRef]
  37. Gómez-López, J.M.; Pérez-García, J.L.; Mozas-Calvache, A.T.; Delgado-García, J. Mission Flight Planning of RPAS for Photogrammetric Studies in Complex Scenes. ISPRS Int. J. Geo-Inf. 2020, 9, 392. [Google Scholar] [CrossRef]
  38. Ullman, S. The interpretation of structure from motion. Proc. Royal Soc. B 1979, 203, 405–426. [Google Scholar]
  39. Koenderink, J.J.; Van Doorn, A.J. Affine structure from motion. J. Opt. Soc. Am. A 1991, 8, 377–385. [Google Scholar] [CrossRef] [PubMed]
  40. Lowe, D.G. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 2004, 60, 91–110. [Google Scholar] [CrossRef]
  41. Szeliski, R. Computer Vision: Algorithms and Applications; Springer: London, UK, 2011. [Google Scholar]
  42. Scharstein, D.; Szeliski, R. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 2002, 47, 7–42. [Google Scholar] [CrossRef]
  43. Seitz, S.M.; Curless, B.; Diebel, J.; Scharstein, D.; Szeliski, R. A comparison and evaluation of multi-view stereo reconstruction algorithms. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, New York, NY, USA, 17–22 June 2006. [Google Scholar]
  44. Furukawa, Y.; Hernández, C. Multi-view stereo: A tutorial. Found. Trends Comput. Graph. Vis. 2015, 9, 1–148. [Google Scholar] [CrossRef]
  45. Westoby, M.J.; Brasington, J.; Glasser, N.F.; Hambrey, M.J.; Reynolds, J.M. ‘Structure-from-Motion’ photogrammetry: A low-cost, effective tool for geoscience applications. Geomorphology 2012, 179, 300–314. [Google Scholar] [CrossRef]
  46. Fitzgibbon, A.W. Robust registration of 2D and 3D point sets. Image Vis. Comput. 2003, 21, 1145–1153. [Google Scholar] [CrossRef]
  47. Sahilioglu, Y.; Kavan, L. Scale-Adaptive ICP. Graph. Models 2021, 116, 101113. [Google Scholar] [CrossRef]
  48. Murphy, M.; McGovern, E.; Pavia, S. Historic building information modelling (HBIM). Struct. Surv. 2009, 27, 311–327. [Google Scholar] [CrossRef]
  49. Lambers, K.; Remondino, F. Optical 3D measurement techniques in archaeology: Recent developments and applications. In Proceedings of the 35th International Conference on Computer Applications and Quantitative Methods in Archaeology, Berlin, Germany, 2–6 April 2007. [Google Scholar]
  50. Jaén Paraiso Interior. Site Castillos y Batallas. The Route. Available online: https://www.jaenparaisointerior.es/en/castillos-y-batallas/la-ruta (accessed on 30 May 2023).
  51. American Society for Photogrammetry and Remote Sensing. ASPRS Positional Accuracy Standards for Digital Geospatial Data. Photogramm. Eng. Remote Sens. 2015, 81, 277. [Google Scholar]
  52. CIPA Heritage Documentation. The Photogrammetric Capture. The ‘3 × 3’ Rules. Available online: https://www.cipaheritagedocumentation.org/ (accessed on 16 February 2023).
Figure 1. Views of the section of the wall documented in this study that begins just below the castle of Santa Catalina and ends next to the town center of Jaén. Detailed views of some sections of the damaged walls and towers.
Figure 1. Views of the section of the wall documented in this study that begins just below the castle of Santa Catalina and ends next to the town center of Jaén. Detailed views of some sections of the damaged walls and towers.
Heritage 06 00313 g001
Figure 2. Methodology proposed in this study.
Figure 2. Methodology proposed in this study.
Heritage 06 00313 g002
Figure 3. Examples of data acquisition techniques: (a) aerial LiDAR using a DJI Zenmuse L1; (b) TLS using a Faro Focus x130 scanner.
Figure 3. Examples of data acquisition techniques: (a) aerial LiDAR using a DJI Zenmuse L1; (b) TLS using a Faro Focus x130 scanner.
Heritage 06 00313 g003
Figure 4. Examples of data acquisition techniques: (a) aerial photogrammetry using a DJI Phantom 4 Pro; (b) CRP using a conventional Sony A6000 camera mounted on a mast.
Figure 4. Examples of data acquisition techniques: (a) aerial photogrammetry using a DJI Phantom 4 Pro; (b) CRP using a conventional Sony A6000 camera mounted on a mast.
Heritage 06 00313 g004
Figure 5. Examples of data fusion: (a) data fusion between TLS point cloud and a selection of points from the photogrammetric point cloud; (b) fusion of photogrammetric projects, including images obtained from RPAS and terrestrial CRP.
Figure 5. Examples of data fusion: (a) data fusion between TLS point cloud and a selection of points from the photogrammetric point cloud; (b) fusion of photogrammetric projects, including images obtained from RPAS and terrestrial CRP.
Heritage 06 00313 g005
Figure 6. Examples of the results obtained in this study.
Figure 6. Examples of the results obtained in this study.
Heritage 06 00313 g006
Figure 7. Comparison of the meshes obtained from TLS and photogrammetry.
Figure 7. Comparison of the meshes obtained from TLS and photogrammetry.
Heritage 06 00313 g007
Figure 8. Determination of the old path route using LiDAR data.
Figure 8. Determination of the old path route using LiDAR data.
Heritage 06 00313 g008
Table 1. Summary of the requirements of the project.
Table 1. Summary of the requirements of the project.
ProductsScale LevelDescriptionArea/DimensionGSDPoint Density
Three-dimensional model, orthoimage, DEMSL1Full zone20,000 m2>10 mm>10 cm
Three-dimensional modelSL2Complete wall300 m1–10 mm1–5 cm
Orthoimages, DEMsSL3Wall section<20 m0.1–1 mm<1 cm
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Pérez-García, J.L.; Mozas-Calvache, A.T.; Gómez-López, J.M.; Vico-García, D. Multiscale 3D Documentation of the Medieval Wall of Jaén (Spain) Based on Multi-Sensor Data Fusion. Heritage 2023, 6, 5952-5966. https://doi.org/10.3390/heritage6080313

AMA Style

Pérez-García JL, Mozas-Calvache AT, Gómez-López JM, Vico-García D. Multiscale 3D Documentation of the Medieval Wall of Jaén (Spain) Based on Multi-Sensor Data Fusion. Heritage. 2023; 6(8):5952-5966. https://doi.org/10.3390/heritage6080313

Chicago/Turabian Style

Pérez-García, José Luis, Antonio Tomás Mozas-Calvache, José Miguel Gómez-López, and Diego Vico-García. 2023. "Multiscale 3D Documentation of the Medieval Wall of Jaén (Spain) Based on Multi-Sensor Data Fusion" Heritage 6, no. 8: 5952-5966. https://doi.org/10.3390/heritage6080313

Article Metrics

Back to TopTop