Next Article in Journal
Quality Control Methods Using Quality Characteristics in Development and Operations
Next Article in Special Issue
Decoding the Relationship of Artificial Intelligence, Advertising, and Generative Models
Previous Article in Journal
The Effects of Augmented Reality on Very Young Learners’ Motivation and Learning of the Alphabet and Vocabulary
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

An Improved Approach for Generating Digital Twins of Cultural Spaces through the Integration of Photogrammetry and Laser Scanning Technologies

by
Markos Konstantakis
1,*,
Georgios Trichopoulos
1,
John Aliprantis
1,
Nikitas Gavogiannis
2,
Anna Karagianni
2,
Panos Parthenios
2,
Konstantinos Serraos
3 and
George Caridakis
1
1
Department of Cultural Technology and Communication, University of the Aegean, University Hill, 81100 Mytilene, Greece
2
School of Architectural Engineering, Technical University of Crete, 73100 Chania, Greece
3
School of Architecture, National Technical University of Athens, Heroon Polytechniou 9, 15780 Athens, Greece
*
Author to whom correspondence should be addressed.
Digital 2024, 4(1), 215-231; https://doi.org/10.3390/digital4010011
Submission received: 20 December 2023 / Revised: 5 February 2024 / Accepted: 8 February 2024 / Published: 16 February 2024
(This article belongs to the Special Issue Digital in 2024)

Abstract

:
The paper introduces an innovative methodology that combines photogrammetry and laser scanning techniques to create detailed 3D models of historic mansions within the Kifissia region of Attica, Greece. While photogrammetry excels in capturing intricate textures, it faces challenges such as lighting variations and precise image alignment. On the other hand, laser scanning offers precision in capturing geometric details but struggles with reflective surfaces and large datasets. Our study integrates these methods to leverage their strengths and address limitations, resulting in comprehensive and accurate digital twins of cultural spaces. The methodology section outlines the step-by-step process of integration, emphasizing solutions to specific challenges encountered in the study area. Preliminary results showcase the enhanced fidelity and completeness of the digital twins, demonstrating the effectiveness of the combined approach. The subsequent sections of the paper delve into a detailed presentation of the methodology, provide a comprehensive analysis of obtained results, and discuss the implications of this innovative approach in cultural preservation and broader applications.

1. Introduction

In recent years, the fusion of cutting-edge technologies has opened new routes for groundbreaking advancements in the recreation and preservation of cultural spaces. Among these innovations, the integration of photogrammetry and laser scanning has emerged as a transformative approach in the creation of digital twins for such significant environments. This paper delves into an enriched methodology that exploits the combined potential of these two techniques, aiming to enhance the fidelity, accuracy, and completeness of digital twin generation for cultural spaces. Photogrammetry, leveraging the capabilities of computational algorithms, offers a versatile means to reconstruct three-dimensional models from a multitude of photographic inputs. Its capacity to capture intricate details and textures offers virtual replicas a level of realism that mirrors the authenticity of the original environment. However, inherent challenges such as lighting variations, optical obstructions, and the need for meticulous image alignment pose critical considerations for its implementation. In parallel, laser scanning technology operates on principles of precision and depth, generating highly accurate point clouds that describe spatial attributes with exceptional detail. Its ability to capture intricate geometric details demonstrates its exceptional precision. Nonetheless, limitations in scanning reflective or transparent surfaces, as well as the complexity of managing vast datasets, underscore the need for a nuanced approach when employed in isolation.
Through the amalgamation of these methodologies, the Mansion project aims to create an innovative augmented reality platform, accessible from portable and fixed devices (smartphones, tablets, computers, etc.), through which the visitor will be able to browse the urban fabric with the guidance of a digital map and be guided around the mansions of Kifissia (a region in Athens, Greece) with the help of three-dimensional architectural diagrams that will provide condensed architectural information (history, typology, form, construction), rendered in an intelligible way. To achieve our objectives in this project and create the accurate 3D replicas of the selected building in the area, we focus on combining the detailed visual data from photogrammetry with the accurate geometric data from laser scanning. This will allow us to create digital twins for cultural spaces that are more complete and accurate. The subsequent sections outline the comprehensive methodology employed in this study, detailing the integrated approach, utilizing both photogrammetry and laser scanning techniques. Following this methodology, the attained results are presented. Additionally, a discussion ensues, focusing on the efficacy and implications of this approach within cultural preservation, heritage studies, and broader applications.
The rest of this paper is structured as follows: Section 2 provides an overview of the related work in the field of digital twin generation through photogrammetry and laser scanning technologies. Section 3 describes the materials and methods used in the Mansion project. Section 4 describes in detail the Mansion methodology. Finally, Section 5 concludes the paper by summarizing the results and outlining future directions for research and implementation.

2. Related Work

The utilization of digital image processing techniques has become widespread across various fields. Numerous studies have shifted their focus towards methodologies that integrate imaging with computer vision techniques, including photogrammetry and laser scanning. Those two prominent methodologies have emerged as powerful tools in the realm of cultural heritage preservation.
Three-dimensional (3D) laser scanning methodologies excel at creating detailed point cloud files, prioritizing comprehensive coverage, but they may overlook finer details such as corners and edges. On the other hand, photogrammetry techniques utilize high-resolution images to generate dense and colored point clouds, emphasizing the complex features of the subject. As shown below, many research studies have seen the benefits of both techniques and have tried to combine them to improve geometric accuracy. This is especially important when it comes to cultural heritage, where accuracy is very important [1,2,3].
More specifically, Hassan et al. [4] proposed a methodology for the integration of both technologies through case studies of a large Neo-Manueline church in Portugal and a small Romanesque church in Spain. They focused on creating geo-referenced, enriched digital models and supporting the design of effective preventive conservation plans, considering the diverse shapes and sizes of the buildings, which was the main challenge they faced.
This was also the main challenge of another study by Liang H. et al. [5], regarding the digital documentation of Huanxiu Shanzhuang (HXSZ) in Suzhou, China. The authors combined terrestrial laser scanning and terrestrial/unmanned aerial vehicle digital photogrammetry to create a two-step method for 3D reconstruction. This method produced high-accuracy 3D models and detailed drawings of HXSZ’s complex landscape features. They claimed that their methodology can be used in various cultural heritage sites, especially those with diverse elements and complex spatial features. Moreover, data acquisition in a valid format and data registration from the two technologies were the main challenges for many other relevant studies [6,7].
Furthermore, another project by Fawzy H.E.D [8] delved into a study of 3D digital documentation methods applied to the mosque of the Kafrelsheikh University campus. Specifically, the author explored close range photogrammetry (CRP) as a means of generating a digital smart point cloud mesh, comparing its effectiveness with laser scanning. The objective was to develop a hybrid approach that would combine these techniques to achieve more accurate 3D feature documentation. The total station served as a tool for surveying the geometric properties of the building surface, acting as the foundation for this research. Computational methods and operational procedures were elucidated within the manuscript. The hybrid technique emerged as the optimal approach, demonstrating enhanced accuracy in 3D digital documentation.
Additionally, during a 3D modeling study by Rzonca A. [9], an evaluation of the current state of proposed methodologies for the integration of photogrammetric and laser scanning data was conducted. The study acknowledged the lack of a universal data format for vector and imagery data that was crucial for data registration from both technologies. Thus, the research advocated for an automated approach, aligning with the need for extracting common features from both photogrammetric and laser scanning data and ensuring that the basic elements of the buildings were aligned smoothly and effectively.
Jie Tan et al. [10] combine laser scanning and photogrammetry into their case study to extract historic information from archaeological sites. The authors created a digital replica of Xuanluo Hall and presented their methodology. With a similar approach, Kong X. et al. [11] present a framework for monitoring the deterioration of historic structures. They integrate photogrammetry technologies and point cloud processing algorithms to health monitor buildings and constructions like arches and bridges. Their case study is on the Taleyfac Spanish Bridge, a stone arch bridge that can be found on the island of Guam. On the same path, Park E.K.J. [12] uses a method that combines laser scans with photogrammetry to reconstruct digitally several cultural sites in South Korea.
Monterisi C. et al. [13] try a different approach to accurately reconstruct a digital twin of cultural heritage buildings. The authors use a smartphone and a commercial compact camera and apply the close-range photogrammetric technique, based on the combination of structure for motion and computer vision approaches. The study focused on the Ognissanti Church in Valenzano, Italy. Open-source software like MicMac Graphic User Interface and CloudCompare were utilized for camera calibration, preprocessing, and processing phases. After comparing the collected databases based on various factors like image matching, resolution, and processing times, they found that despite differences in resolution and stability, both cameras produced similar picture matching after several orientation repetitions. Although the smartphone’s point cloud was less dense than the compact camera’s, the resulting clouds were nearly identical, with an average distance between them of 0.01 m.
For Shabani A. et al. [14], the initial step in creating digital twins involves three-dimensional (3D) geometric documentation, which can be achieved through various equipment and methodologies, such as aerial and terrestrial close-range photogrammetry, 3D laser scanning, and geodetic methods. The focus during data processing lies in generating precise and detailed 3D models suitable for finite element (FE) modeling. This is primarily achieved by combining 3D point clouds obtained from laser scanning and photogrammetric methods. These 3D models serve as the basis for developing FE models using specialized software packages. Additionally, some FE software allows direct import of the 3D volumes obtained in the previous step. The study explores the challenges encountered at each stage of this process, offering examples of surveyed heritage structures to illustrate these challenges and the strategies used to address them.
The “Holy Transfiguration” Church in Pomorie, Bulgaria, recognized as the oldest operational church on the Bulgarian Black Sea coast, serves as the focal point for the study of Marinov I. et al. [15]. The authors explore the utility of digital twin models in managing immovable cultural heritage, emphasizing their role in informed decision-making regarding maintenance, management, interpretation, and precise documentation. The focus is on outlining an approach to creating a digital twin through diverse spatial data acquisition methods: close-range terrestrial digital photogrammetry, aerial digital photogrammetry, and terrestrial laser scanning. To ensure data accuracy, distance measurements were cross-verified using a measuring tape and classical land surveying methods employing a total station. The research concludes by delving into the strengths and weaknesses of each approach used to gather spatial data for creating digital twins of immovable cultural heritage. It also discusses the practical applications derived from these approaches.
In addition, Cruz Franco P.A. et al. [16] aim to establish and authenticate a process for creating physical models, termed “physical twins”, from digital replicas obtained from unique architectural structures within our cultural heritage. These physical twins serve the dual purpose of preserving and sharing cultural assets while introducing innovative educational methods for heritage dissemination. Moreover, these physical replicas serve as effective tools for ensuring universal accessibility, as they offer low-cost and swift distribution of typological models. By transcending physical limitations, they enable broader access for users, irrespective of their economic, physical, or geographical constraints. The workflow involves utilizing digital twins of architecturally significant elements obtained through photogrammetry or terrestrial laser scanning. These digital twins undergo optimization through various platforms to create an optimized exchange file, facilitating dissemination and printing. A proposed digital platform ensures the unrestricted distribution of these digital twins to any interested user. Finally, physical twins are created using 3D resin printers employing SLA technology for their efficiency and high-quality output.
Continuing with projects that combine the methods of photogrammetry with laser scanning to render 3D representations, Atik M. et al. [17] combined the above techniques to model historic instruments, which are on display at the Istanbul Technical University (ITU) Geomatics Engineering Department Historic Measurement Instruments Museum. Specifically, they modeled three instruments called a Theodolite, a Sextant, and a Graphometer, which have complex geometry, but managed to render them in great detail. Giżyńska J. et al. [18] proceed to a comparison of the aforementioned techniques for the representation of a cultural heritage small construction, the water pumping station building located in the Museum of King John III’s Palace at Wilanów, Poland. The authors describe their methodology, present the settings used both for photography and laser scanning, and discuss their results. Similarly, Melendreras R. et al. [19] compare the same techniques in religious imagery. Their purpose was to digitize artwork with the highest possible quality and precision. They describe their workflow for both methods, and at the end, they proceed with a comparative analysis of the results. They conclude that digital photogrammetry and structured light techniques are the most accurate techniques for documenting heritage.
In a research project by Tausch R. et al. [20], the authors use a 3D laser scanner to render small museum objects. They present their workflow leading to virtual replicas, i.e., online 3D renderings, AR realizations, and 3D prints of the replicas. Laser scanners excel in reproducing high-precision 3D models, as supported by various references. For instance, Wujanz et al. [21] propose an intensity-based stochastic model for terrestrial laser scanners, underscoring precision and accuracy. Digne et al. [22] showcase high-precision 3D datasets obtained from laser scanners, emphasizing their capability to capture accurate raw data. Arayici Y. [23] discusses the rapid acquisition of high-density point data using 3D laser scanning technology, reinforcing precision. Villa et al. [24] focus on curvature changes and morphological analysis, indirectly supporting precision by demonstrating detailed and accurate morphological feature capture. Kurban [25] highlights laser scanners’ ability to obtain large 3D point clouds for constructing precise 3D models. Rodriguez-Quinonez et al. [26] discuss 3D laser scanners’ precision and accuracy in applications like building and city modeling. In conclusion, these references collectively affirm that laser scanners can achieve high-precision 3D models through accurate data capture, detailed morphological feature representation, and the generation of large, dense point clouds.

3. Mansion Project: Materials and Methods

3.1. Introduction

The proposed project aimed to develop an innovative digital platform that offered digital services and interactive tour tools within cultural spaces, particularly mansions, in the broader region of Kifissia Municipality, which is marked in red on a Google map in Figure 1. The area is located to the north of Athens, the capital of Greece, and is a suburb of the city. This platform incorporated cutting-edge technologies like Augmented Reality (AR), 3D architectural models, and personalized guided tours, with the primary objective of creating novel and immersive cultural experiences.
The main focus of this project was to create an innovative AR platform accessible through various portable and stationary devices such as smartphones, tablets, and computers. Through this platform, visitors will have the opportunity to explore the urban landscape with the assistance of a digital map and receive guidance on the historic mansions of Kifissia. These mansions will be presented using three-dimensional architectural diagrams that offer concise architectural information, including their history, architectural style, structure, and construction techniques, in a comprehensible manner.
The selection of Kifissia Municipality as the project’s location was primarily due to the rich diversity of architectural styles represented by its historic mansions. These architectural styles mirror the economic and social conditions prevalent during the period in which they were constructed (approximately 1880–1935) (Figure 2 and Figure 3). Documenting, dating, and digitally preserving these mansions on a map will provide valuable insights into the historical evolution of Kifissia and enhance our understanding of the settlement’s history. A total of 51 historic mansions have been identified and tagged on a Google map (Figure 4).
The project was executed in several phases, beginning with documentation, architectural analysis, and categorization of the buildings. Subsequently, selected mansions undergo three-dimensional modeling using 3D scanning technology, and the necessary three-dimensional diagrams will be created. The platform’s development followed, culminating in the implementation of the augmented reality application. Following the platform’s completion, an evaluation phase was conducted that used appropriate usability assessment techniques with a focus on the user experience.

3.2. Digitization Methods and Tools

For precise and comprehensive digitization, during the Mansion project, we gathered data from diverse sources and strived to effectively integrate this information to create a three-dimensional representation of spaces. Throughout the implementation of our methodology, we utilized the FARO M70 laser scanner for laser scanning results, along with a DJI Mavic 2 Pro drone and professional cameras (Sony FDR-AX53) with the capability to capture images up to 20MP resolution, and modern iPhone smartphones such as the iPhone XS to facilitate the photogrammetry process, capturing both aerial and ground photographs.
When capturing aerial photos, we aimed to minimize strong shading on surfaces. Ideally, we seek cloud cover to provide diffused and non-intense lighting, although achieving this condition was challenging. Therefore, we preferred midday hours when the sun was at its peak, creating minimal shadows. Additionally, gentle or, ideally, non-existent wind conditions were essential to prevent violent drone movements. Unfavorable weather, including rain, made photography impractical. During image capture, the drone completed at least three full rotations around the building at varying heights. This approach ensured coverage of the roof, which might be challenging to photograph otherwise, and captured elevated aspects of the building such as porches, antennas, railings, recesses, or overhangs. Each rotation involved taking 300–400 photos, approximately one photo per circular degree, at a distance from the building ranging between 5 and 20 m. The distance was adjusted based on factors like vegetation, power cables, poles, and other obstacles in the area. This strategy ensured an overlap between photos greater than or equal to 80 percent. The three rotations included height deviations of approximately 3–5 m, depending on the external morphology complexity of the building.
Simultaneously, whenever possible, ground-level photographs were captured alongside the aerial photography, utilizing any available means. The aim was to achieve consistent lighting and shading conditions across the photos for seamless stitching. While maintaining a fixed distance from the building, continuous shots were taken, striving for over 80 percent overlap. The focus was on capturing every detail and aspect of the building, including exterior stairs, architectural embellishments, intricate details, small decorative elements, railings, carvings, artworks, entrances to basements, portals, arches, and other architectural features. This complexity in photographing required special attention and time. Facilitating the process, we used the Polycam Pro 2023 software (https://poly.cam/ accessed on 15 January 2024), which automates the capture of multiple images, eliminating the need for the user to constantly press the capture button on the smartphone. This allowed the photographers to concentrate more easily on their trajectory and objective, ensuring a high degree of overlap between images.
Table 1 shows the settings for the cameras used during photography. The aperture for the drone was alternated between the two values f/2.8 and f/5.6 depending on the lighting and the distance between the drone and the mansion. The shutter speed was set to automatic on all shots, so there is a large variation in values from photo to photo depending on the intensity of the light. All cameras photographed in high resolution, with the highest possible image quality. The GPS system on the drone was enabled on all shots.
As for the laser scanner, it has an inclinometer, an altimeter, a compass, and a GPS geolocation system, all of which were enabled during the scans. Additional settings made during the scans are listed in Table 2. The choices of settings for the laser scanner were made based on finding a balance between scan quality, scan times, and the size of the created files. The scan quality can theoretically reach much higher levels (1/1 Resolution, 9× Quality), but this would mean extremely long waiting times for scanning and then huge file sizes, with a substantial difficulty in processing. Two different scanning profiles were used, one for longer distances where the general shape of the building was of interest and one for closer scans, where architectural details (carvings, ornaments, etc.) had to be rendered.
Initially, regarding the laser scanning with the FARO M70, a thorough analysis of the outdoor environment was conducted to determine the placement positions of the scanner, along with deciding on the number of scans required. Areas with complex morphology necessitate closer scans, while factors like vegetation and various natural obstacles around buildings compel the scanner’s repositioning, adding complexity to the process. Stitching between scans involved the use of specific reflective spheres strategically placed in space, which must be removed from the 3D model upon completion of the process. Increasing the density of points recorded by the laser enhances the reliability of capturing the building’s morphology. However, even a minimal increase in this density results in a proportional rise in the time required for each scan, the volume of generated files, and their processing time. Special attention was also crucial for activating the geographic coordinate system, as its absence leads to scans lacking spatial data, rendering them incompatible with the rest of the media, such as photos.
The decision to digitize certain mansions was influenced by multiple factors, such as the physical tours offered to cultural visitors. During these tours, particular architectural features were emphasized, aligning with the significance of each mansion and accommodating the available tour duration. Table 3 below presents the number of selected mansions per shot, while Figure 5 illustrates the 3D modeling process for the mansions.

4. The Mansion Methodology

The overall methodology includes the alignment of both laser scans and images, the registration of point clouds, the calculation of a 3D model (mesh), and lastly the simplification of the latter and the production of high-res textures. Initially, the scans from the FARO M70 were aligned, using the FARO® 3d SCENE v2023 Software, with control points that were clearly identified in every pair of adjacent scans. The aligned component was exported in .e57, a non-proprietary format that can be accessed with different software systems.
The images of each villa, as previously mentioned, were retrieved from 3 different sources: aerial photography, handheld cameras, and smartphones using Polycam software, as a background assistance to achieve the optimal overlap. The registration of all inputs into point clouds was made with Reality Capture (RC), a state-of-the-art technology for photogrammetric visualization of real environments. RC uses Structure from Motion (SfM), a very common technique in image-based modeling, explained thoroughly by Rahaman, H. and Champion, E. [27]. As discussed by Kingsland K. [28] and by Hellman, T., and Lahti, M. [29], RC is one of the easiest and fastest among similar software solutions, even though it did require manual work in the alignment of the images.
The alignment of point clouds, when undertaken automatically by the software, almost always showed errors. The reason was the use of a different coordinate system between the two devices (drone and scanner), which led the generated clouds to two different locations with a large deviation between them, as shown in Figure 6. Even after manually trying to move the two clouds to the same coordinates, it was found that there were discrepancies not only in the position but also in the rotation of the objects, and even worse, in the size scale of the objects. Figure 7 shows the existence of two representations from Zoura mansion, after trying to place the two point clouds in the same position. The presence of a roof indicates the drone photography, while the second cloud comes from the scans. Only the tower of the house can be distinguished in both representations. Objects are shown rotated in all 3 axes, and there is little deviation in size scale. It turned out that, the only solution to the problem was to ignore the geographic data in the images and scans during the process of merging the point clouds, and manually align them based on control points.
The images were firstly split by means of capturing into different folders. The surroundings of the villas did not often allow for continuous aerial photography, creating gaps in areas that were covered by vegetation or other visual obstacles. The latter resulted in inability to produce a single registration automatically. Therefore, at least 3 common control points were placed in every set of produced registrations, aiming to cover a wide range of XYZ coordinates. Often more than 3 were needed to have a correct aerial photography registration, like in Villa Zoura, where a total of 10 control points were used to merge successfully the 3 automatically produced components to one, seen bottom right (Figure 8). The user needs to manually identify and mark each control point in images from the initial input set, while RC retrieves more images containing the control point, increasing, thus, the final accuracy.
Additionally, the Polycam software divided the images into the various sections that represented the villa (Figure 9). The separate registrations were exported separately and merged consecutively into the aerial photography component, again using manually placed control points, as described previously.
The Polycam components added necessary information about the lower parts of the villas that were hidden behind vegetation, while the aerial photography captured clearly the roofs and balconies (Figure 10). Lastly, the final registration was merged with the FARO component, which was imported separately.
The next step in the tested workflow is the calculation of the final registration for a 3D model. Within this phase of the model creation, it was finally decided not to include the FARO component, derived from the FARO M70 scans, mainly because the level of detail was significantly higher than what was needed based on the project description and desired outcomes, and also because it was only in selected areas of the villas (visible from ground level), resulting in a non-uniform output. Therefore, the final calculation for all the villas was based solely on the imagery.
The simplification of the geometries using the decimation function within RC was fundamental for the scope of the Mansion project since the final platform required the minimum possible size of models. Thus, the geometries were carefully cleaned from irrelevant triangles (vegetation or remains from neighboring man-made structures or objects) using the selection functions and then simplified to 300 K triangles. On the contrary, the textures were required to be as large as possible, and the final outcome was either 1 × 16 K or 2 × 16 K texture maps per villa. The combination of low triangle quantity and high resolution texture maps generated models that were small in size, ranging from 10 to 40 MB each, but detailed in texture, which allowed their flawless and smooth navigation within the final platform, as well as easy dissemination in other software solutions.
Lastly, every simplified and textured model was imported in Blender, a free and open source software (FOSS) for manual removal of damaged detailed geometries (railings, decorative elements, and low vegetation) and for the remodeling of valuable architectural details. When possible, identical areas of the villas were copied and pasted from areas that were scanned and modeled in RC correctly to others where the final geometry or texture was damaged because of visual obstacles. This process results in a fully featured 3D version of the villas, giving more complete 3D models to the users, who can also view the real state of the villas by altering the available layers of the 3D model. In this way, users can display both the real and the ideal version of the building, focusing on the differences and the damaged areas.
Finally, the models were exported both in .obj and .glTF format to be implemented and tested in Unreal Engine. The latter was chosen since, as a binary format including textures and shading, it shows strong advantages in rendering time and overall volume size (LeeGeon-hee [30]).

5. Results

The transformation of architectural exhibits through digitization has significantly impacted architectural heritage, offering novel avenues to showcase and access architectural content. The emergence of digital twins, involving the creation of virtual representations of physical objects, stands out as a powerful strategy to engage broader audiences and harness cutting-edge technologies and digital tools. Leveraging advanced technologies like laser 3D scanning, photogrammetry, and 3D design software allows for the meticulous capture of intricate three-dimensional data from architectural structures, resulting in precise and immersive digital replicas. These digital twins find applications in diverse fields such as research, education, and entertainment, providing captivating experiences for architectural enthusiasts.
However, the digitization process is not without its challenges. Addressing these challenges and establishing clear standards and frameworks for architectural digitization efforts are crucial. The Mansion project exemplifies a pioneering initiative in this realm, presenting a methodology that can be adapted to various contexts. This approach facilitates the creation of digital twins with applications in research, education, and entertainment. By leveraging existing resources, the materials provided by the Mansion project offer a foundation for identifying scenarios and content suitable for educational initiatives centered around architecture and cultural heritage.
More specifically, regarding Villa Zoura, for example, as well as most of the villas in the project, the most challenging parts of the overall methodology were the scheduling of the scanning and photography of the monument, which, because of the ownership status, was only able to happen in times of the day with plenty of sunlight and shadows; the capturing of the aerial photography in a continuous and not disruptive way because of the physical obstacles; and lastly, the manual alignment of the separate components within RC. The latter is a direct outcome of the first two constraints, which slowed down the procedure but did not affect the quality of the final result.
The villas that were scanned are usually privately owned, and access to the site is upon arrangement only, which limits the time windows for shadeless aerial photography, as it would ideally be required for an automatic alignment. Furthermore, the buildings were usually placed in the middle of big plots with plenty of vegetation in the surrounding area, limiting the direct views from all the necessary angles. Figure 11 shows the 3D rendering of a mansion, while Figure 12 and Figure 13 show corresponding photographs of the same mansion. By comparing the images, the detail in the depiction of the roof and other architectural details can be seen, as well as the strong shadows created by the presence of harsh light.

6. Conclusions

In conclusion, the fusion of photogrammetry and laser scanning technologies, as explored in this study, presents a promising and enriched methodology for the recreation and preservation of cultural spaces. By leveraging the complementary strengths of these techniques, our approach aims to overcome the inherent limitations of individual methods, offering a comprehensive solution that addresses the challenges associated with each technology.
The integration of photogrammetry, with its computational algorithms enabling versatile three-dimensional reconstructions from photographic inputs, provides a visually realistic representation enriched with intricate details and textures. However, challenges such as lighting variations and optical obstructions necessitate careful consideration. On the other hand, laser scanning technology, with its precision and depth principles, generates highly accurate point clouds depicting spatial attributes with exceptional detail. Despite its precision, limitations in scanning reflective or transparent surfaces and managing vast datasets underscore the need for a nuanced approach.
Our study successfully navigates these challenges by combining the detailed visual data from photogrammetry with the accurate geometric data from laser scanning. This integrated approach seeks to create digital twins for cultural spaces that are not only visually immersive but also spatially accurate and complete. The subsequent sections of this paper delineate the comprehensive methodology employed, presenting the intricacies of the integrated approach. The results obtained from this study underscore the effectiveness of our methodology, opening avenues for applications in cultural preservation, heritage studies, and beyond.
As outlined in the subsequent sections, the comprehensive methodology, materials and methods used in the Mansion project, and the detailed Mansion methodology contribute to the understanding of our approach. Our study not only contributes to the evolving field of digital twin generation but also provides valuable insights for researchers, practitioners, and stakeholders engaged in the preservation and exploration of cultural spaces.

Author Contributions

Conceptualization, M.K. and G.T.; methodology, M.K., G.T. and N.G.; software, M.K. and N.G.; validation, G.T., J.A. and N.G.; formal analysis, M.K.; investigation, G.T. and J.A.; resources, N.G. and M.K.; data curation, M.K.; writing—original draft preparation, M.K., G.T., J.A. and N.G.; writing—review and editing, P.P., A.K. and G.C.; visualization, M.K., G.T. and N.G.; supervision, P.P., A.K., K.S. and G.C.; project administration, M.K.; funding acquisition, K.S. and M.K. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Data are contained within the article.

Conflicts of Interest

The authors declare no conflicts of interest.

Abbreviations

The following abbreviations are used in this manuscript:
3DThree Dimensional
ARAugmented Reality
RCReality Capture

References

  1. Konstantakis, M.; Trichopoulos, G.; Aliprantis, J.; Michalakis, K.; Caridakis, G.; Thanou, A.; Zafeiropoulos, A.; Sklavounou, S.; Psarras, C.; Papavassiliou, S.; et al. An Enhanced Methodology for Creating Digital Twins within a Paleontological Museum Using Photogrammetry and Laser Scanning Techniques. Heritage 2023, 6, 5967–5980. [Google Scholar] [CrossRef]
  2. Moraitou, E.; Konstantakis, M.; Chrysanthi, A.; Christodoulou, Y.; Pavlidis, G.; Caridakis, G. Supporting conservation and restoration through digital media modeling and exploitation-the example of the Acropolis of Ancient Tiryns. In Proceedings of the 2022 17th International Workshop on Semantic and Social Media Adaptation & Personalization (SMAP), Online Event, 3–4 November 2022; pp. 1–4. [Google Scholar]
  3. Moraitou, E.; Konstantakis, M.; Chrysanthi, A.; Christodoulou, Y.; Pavlidis, G.; Alexandridis, G.; Kotsopoulos, K.; Papastamatiou, N.; Papadimitriou, A.; Caridakis, G. Supporting the Conservation and Restoration OpenLab of the Acropolis of Ancient Tiryns through Data Modelling and Exploitation of Digital Media. Computers 2023, 12, 96. [Google Scholar] [CrossRef]
  4. Hassan, A.T.; Fritsch, D. Integration of Laser Scanning and Photogrammetry in 3D/4D Cultural Heritage Preservation—A Review. Int. J. Appl. 2019, 9, 16. [Google Scholar]
  5. Liang, H.; Li, W.; Lai, S.; Zhu, L.; Jiang, W.; Zhang, Q. The integration of terrestrial laser scanning and terrestrial and unmanned aerial vehicle digital photogrammetry for the documentation of Chinese classical gardens–A case study of Huanxiu Shanzhuang, Suzhou, China. J. Cult. Herit. 2018, 33, 222–230. [Google Scholar] [CrossRef]
  6. Alshawabkeh, Y.; Baik, A. Integration of photogrammetry and laser scanning for enhancing scan-to-HBIM modeling of Al Ula heritage site. Herit. Sci. 2023, 11, 147. [Google Scholar] [CrossRef]
  7. Mohammadi, M.; Rashidi, M.; Mousavi, V.; Karami, A.; Yu, Y.; Samali, B. Case study on accuracy comparison of digital twins developed for a heritage bridge via UAV photogrammetry and terrestrial laser scanning. In Proceedings of the 10th International Conference on Structural Health Monitoring of Intelligent Infrastructure, SHMII, Porto, Portugal, 30 June–2 July 2021; Volume 10. [Google Scholar]
  8. Fawzy, H.E.D. 3D laser scanning and close-range photogrammetry for buildings documentation: A hybrid technique towards a better accuracy. Alex. Eng. J. 2019, 58, 1191–1204. [Google Scholar] [CrossRef]
  9. Rzonca, A. Review of Methods of Combined Orientation of Photogrammetric and Laser Scanning Data. Meas. Autom. Monit. 2018, 64, 57–62. [Google Scholar]
  10. Tan, J.; Leng, J.; Zeng, X.; Feng, D.; Yu, P. Digital Twin for Xiegong’s Architectural Archaeological Research: A Case Study of Xuanluo Hall, Sichuan, China. Buildings 2022, 12, 1053. [Google Scholar] [CrossRef]
  11. Kong, X.; Hucks, R.G. Preserving our heritage: A photogrammetry-based digital twin framework for monitoring deteriorations of historic structures. Autom. Constr. 2023, 152, 104928. [Google Scholar] [CrossRef]
  12. Park, E.K.J. Using Digital Twins for Architectural Heritages. In Proceedings of the 41st eCAADe Conference, Graz University of Technology, Graz, Austria, 20–23 September 2023; Volume 2, pp. 681–690. [Google Scholar]
  13. Monterisi, C.; Capolupo, A.; Tarantino, E. Crowdsensing Close-Range Photogrammetry for Accurately Reconstructing a Digital Twin of a Cultural Heritage Building Using a Smartphone and a Compact Camera. In Proceedings of the Computational Science and Its Applications—ICCSA 2023 Workshops, Athens, Greece, 3–6 July 2023; Gervasi, O., Murgante, B., Rocha, A.M.A.C., Garau, C., Scorza, F., Karaca, Y., Torre, C.M., Eds.; Springer: Cham, Switzerland, 2023; pp. 234–250. [Google Scholar]
  14. Shabani, A.; Skamantzari, M.; Tapinaki, S.; Georgopoulos, A.; Plevris, V.; Kioumarsi, M. 3D simulation models for developing digital twins of heritage structures: Challenges and strategies. Procedia Struct. Integr. 2022, 37, 314–320. [Google Scholar] [CrossRef]
  15. Marinov, I.; Simeonova, G.; Mickrenska, C.; Antova, G. APPROACH TO DIGITAL TWIN OF IMMOVABLE CULTURE HERITAGE. Int. Multidiscip. Sci. GeoConf. SGEM 2023, 23, 193–200. [Google Scholar]
  16. Cruz Franco, P.A.; Rueda Márquez de la Plata, A.; Pérez Sendín, M. Investigating a Workflow for Obtaining Physical Models from Digital Twins Obtained through Photogrammetry and TLS: New Ways for a Sustainable Dissemination of Heritage. Appl. Sci. 2023, 13, 1057. [Google Scholar] [CrossRef]
  17. Atik, M.E.; Duran, Z.; Yanalak, M.; Seker, D.Z.; Ak, A. 3D modeling of historical measurement instruments using photogrammetric and laser scanning techniques. Digit. Appl. Archaeol. Cult. Herit. 2023, 30, e00286. [Google Scholar] [CrossRef]
  18. Giżyńska, J.; Komorowska, E.; Kowalczyk, M. The comparison of photogrammetric and terrestrial laser scanning methods in the documentation of small cultural heritage object—Case study. J. Mod. Technol. Cult. Herit. Preserv. 2022, 1, 20–28. [Google Scholar] [CrossRef]
  19. Ruiz, R.M.; Torres, M.T.M.; Allegue, P.S. Comparative Analysis Between the Main 3D Scanning Techniques: Photogrammetry, Terrestrial Laser Scanner, and Structured Light Scanner in Religious Imagery: The Case of The Holy Christ of the Blood. J. Comput. Cult. Herit. 2021, 15, 1–23. [Google Scholar] [CrossRef]
  20. Tausch, R.; Domajnko, M.; Ritz, M.; Knuth, M.; Santos, P.; Fellner, D. Towards 3D digitization in the GLAM (Galleries, Libraries, Archives, and Museums) sector: Lessons learned and future outlook. IPSI Trans. Internet Res. 2020, 16, 45–53. [Google Scholar]
  21. Wujanz, D.; Burger, M.; Mettenleiter, M.; Neitzel, F. An intensity-based stochastic model for terrestrial laser scanners. ISPRS J. Photogramm. Remote Sens. 2017, 125, 146–155. [Google Scholar] [CrossRef]
  22. Digne, J.; Audfray, N.; Lartigue, C.; Mehdi-Souzani, C.; Morel, J.M. Farman Institute 3D Point Sets - High Precision 3D Data Sets. Image Process. Line 2011, 1, 281–291. [Google Scholar] [CrossRef]
  23. Arayici, Y. An approach for real world data modelling with the 3D terrestrial laser scanner for built environment. Autom. Constr. 2007, 16, 816–829. [Google Scholar] [CrossRef]
  24. Villa, C.; Gaudio, D.; Cattaneo, C.; Buckberry, J.; Wilson, A.S.; Lynnerup, N. Surface Curvature of Pelvic Joints from Three Laser Scanners: Separating Anatomy from Measurement Error. J. Forensic Sci. 2015, 60, 374–381. [Google Scholar] [CrossRef] [PubMed]
  25. Besdok, E.; Kurban, T. Design, Implementation and Comparison of LowCost Laser Scanning Systems for 3D Modeling. In Proceedings of the ICIT 2015 The 7th International Conference on Information Technology, Bhubaneswar, India, 21–23 December 2015; pp. 540–544. [Google Scholar]
  26. Rodríguez-Quiñonez, J.; Sergiyenko, O.; Hernandez-Balbuena, D.; Rivas-Lopez, M.; Flores-Fuentes, W.; Basaca-Preciado, L. Improve 3D laser scanner measurements accuracy using a FFBP neural network with Widrow-Hoff weight/bias learning function. Opto-Electron. Rev. 2014, 22, 224–235. [Google Scholar] [CrossRef]
  27. Rahaman, H.; Champion, E. To 3D or Not 3D: Choosing a Photogrammetry Workflow for Cultural Heritage Groups. Heritage 2019, 2, 1835–1851. [Google Scholar] [CrossRef]
  28. Kingsland, K. Comparative analysis of digital photogrammetry software for cultural heritage. Digit. Appl. Archaeol. Cult. Herit. 2020, 18, e00157. [Google Scholar] [CrossRef]
  29. Hellman, T.; Lahti, M. Photogrammetric 3D Modeling for Virtual Reality; Technical Report; WindCoE project; Springer: Helsinki, Finland, 2018. [Google Scholar]
  30. Lee, G.-H.; Choi, P.-H.; Nam, J.-H.; Han, H.-S.; Lee, S.-H.; Kwon, S.-C. A Study on the Performance Comparison of 3D File Formats on the Web. Int. J. Adv. Smart Converg. 2019, 8, 65–74. [Google Scholar]
Figure 1. The area of Kifissia, in Attica, Greece.
Figure 1. The area of Kifissia, in Attica, Greece.
Digital 04 00011 g001
Figure 2. A mansion under study.
Figure 2. A mansion under study.
Digital 04 00011 g002
Figure 3. A mansion under study.
Figure 3. A mansion under study.
Digital 04 00011 g003
Figure 4. The historic mansions tagged on a Google map.
Figure 4. The historic mansions tagged on a Google map.
Digital 04 00011 g004
Figure 5. Mansions of Kifissia and 3D scanning process.
Figure 5. Mansions of Kifissia and 3D scanning process.
Digital 04 00011 g005
Figure 6. Deviation of the point clouds after the automatic alignment from Reality Capture.
Figure 6. Deviation of the point clouds after the automatic alignment from Reality Capture.
Digital 04 00011 g006
Figure 7. Double view of the same mansion, from the divergence of the point clouds.
Figure 7. Double view of the same mansion, from the divergence of the point clouds.
Digital 04 00011 g007
Figure 8. Aerial photography registrations and manually inserted ground control points.
Figure 8. Aerial photography registrations and manually inserted ground control points.
Digital 04 00011 g008
Figure 9. Registration of Polycam images, showing the main entrance of Villa Zoura.
Figure 9. Registration of Polycam images, showing the main entrance of Villa Zoura.
Digital 04 00011 g009
Figure 10. Component including both aerial and terrestrial registrations.
Figure 10. Component including both aerial and terrestrial registrations.
Digital 04 00011 g010
Figure 11. 3D model of a mansion using a drone for aerial photogrammetry.
Figure 11. 3D model of a mansion using a drone for aerial photogrammetry.
Digital 04 00011 g011
Figure 12. Aerial photograph of the mansion.
Figure 12. Aerial photograph of the mansion.
Digital 04 00011 g012
Figure 13. Aerial photograph of the mansion.
Figure 13. Aerial photograph of the mansion.
Digital 04 00011 g013
Table 1. Technical specifications for photography.
Table 1. Technical specifications for photography.
CameraF-StopISOResolutionBit Depth
DJI Mavic 2 Prof/2.8–f/5.61004056 × 3040/75 dpi24 bit
Sony FDR-AX53f/41003840 × 2160/350 dpi24 bit
iPhone XS/13 Prof/1.5502124 × 2832/72 dpi32 bit
Table 2. Settings used for the FARO M70 scanner.
Table 2. Settings used for the FARO M70 scanner.
Scanner ProfileResolutionQualityScan Time (s)Scan Size (MB)
Scan with objects of interest at a larger distance1/4137138
Scan where the details of objects are of interest1/2969538
Table 3. Selected mansion for digitization per shots.
Table 3. Selected mansion for digitization per shots.
Mansion NameFARO ScansDrone ShotsGround Shots
Depferld22495556
Kazouli511522047
Pinelopis Delta187081319
Xrysanthemo15629796
Stathatou28861150
Zoura18606610
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Konstantakis, M.; Trichopoulos, G.; Aliprantis, J.; Gavogiannis, N.; Karagianni, A.; Parthenios, P.; Serraos, K.; Caridakis, G. An Improved Approach for Generating Digital Twins of Cultural Spaces through the Integration of Photogrammetry and Laser Scanning Technologies. Digital 2024, 4, 215-231. https://doi.org/10.3390/digital4010011

AMA Style

Konstantakis M, Trichopoulos G, Aliprantis J, Gavogiannis N, Karagianni A, Parthenios P, Serraos K, Caridakis G. An Improved Approach for Generating Digital Twins of Cultural Spaces through the Integration of Photogrammetry and Laser Scanning Technologies. Digital. 2024; 4(1):215-231. https://doi.org/10.3390/digital4010011

Chicago/Turabian Style

Konstantakis, Markos, Georgios Trichopoulos, John Aliprantis, Nikitas Gavogiannis, Anna Karagianni, Panos Parthenios, Konstantinos Serraos, and George Caridakis. 2024. "An Improved Approach for Generating Digital Twins of Cultural Spaces through the Integration of Photogrammetry and Laser Scanning Technologies" Digital 4, no. 1: 215-231. https://doi.org/10.3390/digital4010011

Article Metrics

Back to TopTop