Digitization of the Built Cultural Heritage: An Integrated Methodology for Preservation and Accessibilization of an Art Nouveau Museum

Abstract

The emergence of new technologies has dynamized the way in which cultural heritage is documented, preserved, and passed on to new generations; something that determines a paradigm shift in terms of research in this field. Most operations now also have access to the virtual component. In this context, the current study aimed to make accessible through virtual and augmented reality one of the most interesting objectives belonging to the Jewish cultural heritage built in Art Nouveau style in the municipality of Oradea (Romania), which currently functions as a La Belle Epoque Museum. In the study, the techniques of terrestrial, aerial photogrammetry, and terrestrial laser scanning were used to remodel, in a three-dimensional format, as faithful as possible and usable in different applications, the special architecture of the exterior of the monument. This information was doubled by making the interior of the monument accessible through a complete and complex series of panoramic images interconnected within a virtual tour that will be made available to tourists interested in discovering the Darvas-La Roche House. The virtual tour, which includes both graphic, textual, and audio information, represents an innovative approach for the buildings built in Art Nouveau style in the municipality of Oradea, representing a virtual bridge for better promotion of the tourist destination and for the awareness of the local people regarding the importance of preserving and appreciating the local cultural heritage. This is all the more important as this is the first initiative to make the Art Nouveau buildings in Oradea Municipality accessible to the general public in an innovative way.

1. Introduction

Tourism is one of the expanding activities at a spatial level with still-unknown effects on the components of the natural and anthropic framework. The impact of tourism on heritage buildings is a specific reality nowadays in the context where, all over the world, a large part of these heritage buildings, some with the status of a historical monument, are exploited through tourism [1,2].

In the field of tourism, buildings belonging to the Jewish heritage have a representative role in terms of European multiculturalism [3,4]. They are imposed on tourism through spectacular architecture, construction methods, and grandeur. It should be noted that these architectural jewels are the expression and imprint of the existence of a large Jewish community at the beginning of the 20th century, after which, unfortunately, their number decreased sharply due to the events that shook humanity during World War II [4]. However, the architectural creations and testimonies are proof of the vitality and creativity of the Jews in this multi-millenary geographical space at the confluence of civilizations and interests. Thus, in this space of contact between the Pannonian Plain and the Carpathian Mountains, over time, several ethnic groups have settled and coexisted, developing specific civilizations that are also reflected in the form of the architectural heritage specific to the place, among which are the Romanians, Jews, Hungarians, Roma, etc. [3].

Against this background, the particularly rich and diverse Jewish cultural heritage has the role of guiding the present and the future in the direction of avoiding the errors of the past. In addition to this historical symbolism, the Jewish architectural heritage also provides us with essential information about the glory and greatness of the ancient society, marking the technical and technological progress registered by that society, symbolism, myths, and beliefs.

On the other hand, Jewish architectural heritage, through its qualitative and quantitative characteristics, represents a latent tourist resource with real possibilities of capitalizing on it through tourism. Among the characteristics of the architectural heritage, age, originality, aesthetics, size, significance, and functionality are worth noting. All these characteristics constitute a motivational factor of prime importance in shaping and affirming tourist destinations, generating real tourist flows from emission areas to tourist reception areas [5,6,7]. Located mainly in large urban centers, a large proportion of Jewish heritage buildings have begun to be exploited (actively and passively) by tourism. Their valorization requires an awareness of their value among all the factors involved, doubled by a commensurate financial effort. These aspects arise from the need to know, restore, and preserve for future generations these architectural jewels in a form as little altered as possible from the original and from the way they were conceived in the initial phase.

Unfortunately, the problems faced today by the elements that belong to the Jewish cultural heritage are among the most diverse, but among the most serious is the fact that they suffer damage due to human activity and environmental factors. In this way, unique buildings are gradually being destroyed, and not all objects can be restored for various reasons. Today, digital technologies make it possible to describe, capture, or virtually reconstruct architectural monuments and heritage buildings, and restore lost cultural monuments. This makes it possible to recreate the primitive image of the heritage building and give virtual life to the object. Many countries are already moving to full 3D scanning of heritage buildings and creating a database for virtual viewing in 3D models for free access by the general public.

Digitization of heritage buildings in the urban environment is an urgent necessity and is difficult to achieve due to the high density of buildings in cities [8,9]. In the present study, the capture of 3D spatial information was achieved through the 3D laser scanning technique, data processing through the point cloud modeling method, etc. [10], and the presentation of the 3D model of a heritage building from the historical center of the Oradea Municipality is presented in detail in what follows.

The paper emphasizes the importance of 3D scanning of heritage buildings and the creation of a single 3D database for virtual viewing in the public domain. Computer technology can simulate virtual reality not only for cultural heritage but also for other objects in the form of visual images. Currently, 3D models are actively used in three main industries: In entertainment—movies, computer games; in medicine—3D modeling helps to model prostheses and implants, and much more; in industry—3D modeling allows spatial visualization; remote sensing—for viewing and obtaining quantitative and qualitative data about objects in the real world; as well as in many other applied fields [11]. In these fields, for the creation of three-dimensional models, the use of both separate and combined three-dimensional laser scanning and photogrammetry is considered most of the time [12,13]. The wide applicability of the photogrammetry technique resides in the fact that it is cheap, portable, and flexible, giving rise at the same time to three-dimensional models of superior quality in terms of geometry and texture [14,15,16]. In addition to the many advantages it offers, photogrammetry also has some disadvantages, especially compared to the laser scanning technique. Thus, in some applications, it does not have the necessary precision to obtain qualitative results (for example, regarding surfaces without texture or with homogeneous texture, transparent materials, or with high reflectivity) [17], it can be quite slow (considering that the number of photos is directly proportional to the level of detail that is desired to be obtained), and the risk of making a mistake is quite high. Thus, the combination of the two digitization methods favors obtaining superior results from a qualitative point of view, exploring the advantages of each technique, and thus helping to reduce the shortcomings generated by the separate application of each technique.

Due to the ability to create accurate digital copies of objects, 3D scanning is often used in scientific research and education projects for the preservation of historical and cultural monuments and heritage buildings. Three-dimensional databases of cultural heritage, created with the help of contact-free 3D scanning technology, using specific software and hardware, provide a collection of 3D models and 2D, and ensure the preservation of cultural heritage for future generations with unhindered free access to exhibits for everyone and the possibility of being integrated into other tourist or conservation projects [18,19,20,21,22,23]. When conducting various studies, 3D scanning helps to compare the shapes and parameters of objects quickly and accurately. In the field of virtual reality, this technology allows for creating scenes with the effect of full immersion [24,25,26,27].

Batyrbaeva [24] published recently the virtual reconstruction of medieval architectural monuments on the territory of Kyrgyzstan (e.g., the fortress city of Koshoy-Korgon of the X–XII centuries) because most of the architectural monuments testifying to the high medieval urban culture have not been preserved, so this country is perceived as a region of nomadic civilization. The results of 3D modeling of the heritage building also reveal the distortion of the heritage building over time and timely reconstruct the object for a long existence. The results of a study that analyzed the Golden Mosque of Tillya Kori Madrasah (Samarqand, Uzbekistan) through digitization techniques influenced the decision regarding the reconstruction of the exterior dome of the monument in order to reduce the maximum weight and preserve the monument in the best conditions [25]. Scientists at the Perm Polytechnic University of the Russian Federation are creating a database for analyzing and collecting information about historical and architectural heritage objects in 3D formats [26]. This database will be useful both for the reconstruction of buildings and for the preservation and study of architecture.

Regarding the analysis of Jewish cultural heritage through digital means, the work of Matoušková et al. [27] analyzes using 3D point clouds the former homes of the Jews in Morocco; Aguilera et al. [28] use a combined method of terrestrial laser scanning and low-cost aerial photogrammetry to model a Jewish tannery located in Avila (Spain); Abdelhamid [29] analyzes by means of photogrammetry and laser scanning the synagogues of Alexandria (Egypt) in order to create a complete and easily accessible digital database, which can be used by researchers and tourists; Gram et al. [30] use terrestrial laser scanning for the extraction of quantitative and qualitative data, as well as for the tourist promotion of a Jewish house of cultural heritage in Romania; while Prechtel et al. [31] use 3D models and geo-information to present a chapter of Jewish prosecution in Nazi Germany, the final goal being that of viewing by tourists.

The 3D models obtained by laser scanning and high-resolution photogrammetry add value to heritage objects, making them accessible on a large scale and for a wide range of users. The perspective offered to the user is new for interactive use, based on an interactive interface (free) for exploring in detail certain parts of the object and obtaining the most detailed and accurate data and information; it has been successfully used for data processing in architecture, construction, conservation, archiving, etc. The tools used can be open source and free software, a very cost-effective solution for both the administrator and the client, and the transparency of the data processing methodology and methods is maximum. The size of the managed and processed data was also carefully taken into consideration, using segmentation and simplification, to maintain optimal access speed [32,33,34].

Accurate documentation of cultural heritage objects is essential for their documentation, prevention, and conservation activities. New methods offer new opportunities: Automatic measurement and orientation procedures, generation of 3D vector data, digital ortho-images, and digital surface models [35,36]. Three-dimensional mapping of cultural heritage is essential for undertaking decisions for management and for the community. At the same time, these techniques can form the basis for shaping the image of the destination in the tourism sector, determining a greater flow of tourists towards the elements of interest following a remote visit to them [37,38].

In this context, the purpose of this study is to analyze and transmit information over time in a form that is as little altered as possible through the complementary use of two assessment methods, namely the laser scanning method and the photogrammetry method. This approach was chosen primarily to use the advantages of both digitization methods: Laser scanning is very efficient in sampling with great accuracy the structure of the monument, while photogrammetry has the advantage of rendering with great efficiency the structure of the site. At the same time, the two methods combine very well to capture the whole scene because the upper part of the monument can only be rendered using aerial photogrammetry.

The current study is also necessary due to the nomination of the municipality of Oradea by European Best Destination as the most beautiful Art Nouveau destination in Europe in 2023. In this context, due to its historical-architectural importance, doubled by the need for security, the current study stands as an example of good practices for the three-dimensional modeling of Art Nouveau-style objectives in order to preserve their beauty and make them accessible to the general public. The success of this project can determine the opening of new horizons regarding the virtualization of the monuments built during the “Belle Epoque” to fulfill the trinomial so important for the objectives of architecture: conservation, promotion, and restoration.

2. The Study Objective

The Darvas-La Roche House is an architectural monument of particular historical importance, being a testimony of Bihor society and of the small and middle bourgeoisie from the end of the 19th century and the beginning of the 20th century. It is one of the most representative buildings of Jewish cultural heritage in Oradea Municipality, along with others, among which stand out the synagogues, the Jewish palaces, and other buildings of Jewish relevance [3,4].

The location in the central part of Oradea Municipality, on Iosif Vulcan Street, in the old city center, on the right bank of the Crișul Repede River, made the access of potential visitors relatively easy (Figure 1). Thus, from the reopening of the house as a museum until now, the number of tourists has been continuously increasing, due to the beauty of the house and the activities that took place inside.

Built in two stages, 1909–1910 (main facade) and 1911–1912 (secondary facade), according to a project by the architects László Vágó and József Vágó, the building offers an exceptional architectural spectacle, embellished by numerous Viennese elements with simplified, geometric shapes that are zoomorphic and anthropomorphic [39,40,41,42,43]. The composition of the main facade stands out through the volumes that compose it, which are punctuated with decorative Zsolnay ceramic knobs. A special element of the facade is the balcony guarded by a statuary personage who holds in his arms a vessel decorated with star-shaped ceramics. It presents a rich openwork decoration, with the symbolic motifs being pinecones, hearts, and vegetal spirals [44,45,46]. Its artistic value makes it considered one of the most original architectural projects from the beginning of the 20th century.

Regarding the history of the Darvas-La Roche House, the first owners were Imre Darvas and the Swiss banker Alfred La Roche. The construction served as an office building for the “La Roche and Darvas” company, which had as its object of activity the exploitation of forests, the purchase of timber exploitation rights, and the processing of wood and artificial wood materials. After the building passed to several owners over time, during the years of communism, a large part of the building was nationalized by the authorities. The nationalized part was changed into a sports center, then into the headquarters of a football team. During this period, numerous changes and destructions of the structure and ornaments took place; and starting in 2010, the local administration purchased the entire building.

Starting in August 2020, Darvas-La Roche House opened its doors to tourists as the first Art Nouveau Museum in Romania, establishing itself as one of the few such museums operating in Eastern Europe. It includes exhibitions dedicated to “La Belle Époque” Oradea interiors (recreation of the bourgeois environment of Oradea), video mapping projections in the children’s room, spaces for manufacturing workshops of local artists, conference rooms, spaces for events, and theater [38]. This was possible following the implementation of a rehabilitation program with European funding of the “Regional Operational Program 2014—2020”, which had as its objective, among others, the spiritual rehabilitation of the building to regain its former splendor and grandeur.

Darvas-La Roche House was chosen as a case study for the application of this work methodology due to the fact that, although for the municipality of Oradea, it represents an element of indisputable value, being one of the most interesting Art Nouveau buildings in the city and the only museum dedicated to this style of construction from Romania, for tourists it is too little known. Therefore, the three-dimensional representations and the creation of a website for the promotion of cultural heritage are intended to add value to Darvas-La Roche House and therefore aim to dynamize the way in which the monument is visited, not only by attracting new tourists to visit the house physically but also through the dissemination of information in the online environment so that the monument will be better known in the country and abroad.

3. Materials and Methods

To carry out the current study, three main stages of creating the necessary products were considered. Thus, the first stage focused on the realization of the external three-dimensional model of the Darvas-La Roche House; the second stage concerned the sampling of 360° images showing the interior, because in the last phase, the products obtained from the previous stages of creation were combined to make virtual tours (Figure 2).

3.1. Realization of the Three-Dimensional Model of the Exterior of Darvas-La Roche House

In order to obtain this product, two data capture techniques were used, namely photogrammetry and laser scanning. The use of both different capture techniques was considered to obtain different types of information, which was then concentrated in a single comprehensive database. Such an approach helps to obtain a larger database and, therefore, a more faithful model of the object. This is of real value for the project in question, considering that the Darvas-La Roche House is defined by surfaces that are very difficult to capture, such as the Art Nouveau motifs and decorations that enrich the building, the stained-glass windows, and the rich vegetation in the summer garden.

3.1.1. Photogrammetry Technique

The photogrammetry technique was used by applying both terrestrial and aerial photogrammetry (drone imagery), both at short distances. The first one was based on the use of Canon EOS R handheld cameras (Canon Inc., Tokyo, Japan) with 30.3 MP resolution, equipped with full-frame RF 24-105 f4 L IS USM lenses (Canon Inc., Tokyo, Japan). This camera is a mirrorless SLR with a high resolution and the ability to handle high ISO. The mount was chosen considering the fact that the type of camera and lenses can drastically change the quality of the scan, so the camera was sought to have as many megapixels per image as possible, the best quality of the images, and the option to play images in RAW format. The camera settings considered that the shutter speed should be set to 50 or more so that the photos are not blurred and thus unusable; the camera’s ISO was set to 500 units or lower so as not to affect the accuracy and placement of the tie points; and the camera’s aperture was set to be as large as possible to capture the finest details. The photos with the terrestrial camera were captured in portrait orientation, capturing as many details as possible from the ground and from the ceiling. Circular and elliptical paths were made to ensure as much overlap as possible (at least 70% in all directions), and every fine detail of the building was thus photographed from several angles and approaches. In this way, 2416 different photos were captured, with an average resolution of 2933 × 4400 pixels and 300 dpi, both in .JEPG format and in the RAW format of .CR3 cameras.

The aerial photogrammetry technique was implemented by using an unmanned aerial vehicle (UAV), the DJI Mavic 2 Pro (SZ DJI Technology Co., Ltd., Shenzhen, Guangdong, China), with a one-inch CMOS sensor and 20 MP and a camera featuring ND16/PL, ND8/PL, and ND4/PL filters, depending on the time of day when the photographs concerned were taken and on the degree of brightness that was present in the outdoor environment. The photos acquired with the UAV were taken in RAW format (.DNG), this being an uncompressed and unprocessed format, therefore more suitable for creating 3D models [47], using the manual capture mode and a fixed white balance. When operating the manual capture mode, care was taken not to have over/underexposed shots in the process; this was moderated using the histogram in the drone controller. The UAV photos were acquired in the form of a grid as well as in the form of sets of images. These involved pointing the sensor at 90°, 45°, and 15° to the target object, keeping the distance between the sensor and the object between 20 and 30 m as much as possible. The photography was performed both using the 3D Survey Pilot 2.18 automatic flight mission planning software and by manually controlling the UAV. Both ways of control were necessary, considering that some parts of the monument could not be captured by an automatic flight due to the fact that the object is in the center of the city, the obstacles present in the scene, and the fact that it steamed at a lower altitude so as not to lose the important details of the building. In 3D Survey Pilot 1.3, the overlapping was set to 80%, both frontal and lateral, and in terms of manual flight, the overlapping of the images was sought to be maintained at approximately ¾ of the successive images. Through this technique, 756 individual photos were captured at a size of 5472 × 3648 pixels, both in .JPEG and .DNG format.

Both for aerial and terrestrial photogrammetry, obtaining the photos was planned with great care so that the unwanted influence of external factors was removed as much as possible. Thus, in order to avoid the situation in which the light reflectivity on the optical surfaces is too high, the photos were captured on days with the sky completely or partially covered by clouds, so that the light is diffused. At the same time, a big problem for the acquisition of photos is the shading effect, and to remove it, the choice of the middle of the day, when the sun is at its nadir, was taken into account in order to obtain the most faithful photos. In order to render the monument in its entirety, the emphasis was placed on the hidden and remote parts of the monuments, to limit the self-occlusion effect, and to finally obtain a complete and complex 3D model (Figure 3). In order to obtain compliant photographic databases, several trips to the field for photography were necessary; before each field trip, the available data were aligned and interpreted so as to identify the areas of the monument where there is a discontinuity in the data or where the photographs completely cover the scene.

After obtaining all the photos, their pre-processing was performed both manually and automatically within dedicated software programs. The first step was to manually check the databases and remove photos that had distortions, blurred areas, or noisy areas. Furthermore, all the data were entered into Agisoft Metashape 2.0.4 to check the internal parameters of the images and their quality by using the image quality estimation mode; photos that obtained scores lower than 0.5 units were considered non-conforming and therefore eliminated [8,9,10]. The second automatic filtering method considered was the use of the open-source computer program Digital Lab Notebook Inspector—PG Version 1.0 Beta, developed by Cultural Heritage Imaging. This program is based on an automatic calculation algorithm, which aims to identify photos whose internal and external parameters record disorders, as well as photos that are broadly different from the others that make up the data set [48]; photos with errors deemed irreparable have been removed. Finally, the photos considered compliant were processed in the mold in the Adobe Lightroom Classic 13 program, where they were corrected in terms of geometric and textural distortions, exposure, highlights, profile correction, and vignetting. This is a method by which the user can minimize or correct the mismatches among image capture, display, and viewing parameters. After completing these processes, all the corrected photos were exported in compressed image format in order to be aligned as easily as possible in the dedicated software.

3.1.2. Laser Scanning Technique

As for the laser scanning, this involved the use of a Leica C10 (Leica Camera AG, Wetzlar, Germany) fixed terrestrial scanner. It has an accuracy of ±4 mm and is equipped with a built-in Smart X-Mirror video camera that allows the possibility of quick and high-quality acquisition of both the afferent point cloud of the Darvas-La Roche House as well as its texture.

In order to obtain a point cloud as faithfully as possible, it was considered necessary to scan the objective using a network made up of eight different scanning points. Three scanning stations were used for the front facade, respectively, in the center of the building on its left and right. The other five scanning stations were used to render the garden and the back of the monument. Considering that they record small details, it was quite difficult to render with a smaller number of points or using only photogrammetry. The scanning stations for the back of the monument were positioned both in the garden (2), on the balconies of the building (2), and in the street in the immediate vicinity of the house (1) (Figure 4). In order to combine the resulting point clouds as well as possible, target points were used for each scene, placed directly on the object to be scanned; they were placed so as to be visible to the scanner used, and their position was accurately determined from any of the three scan points.

The eight-point clouds were entered to be processed within the Leica Cyclone 2023.1 program. This is a software program that specializes in the processing of point clouds and offers a wide set of options for processing the results obtained after laser scanning in fields such as engineering, construction, or topography. The first action implemented here was the georeferencing of the clouds of points and the identification of relationships between them based on the control points through the scan registration process. The procedure aims to unite the eight clouds of resulting points and render them in the form of a single compact entity. A group of control points needs to be added to ensure the ability to merge the different components. In this case, a total of 100 control points were used to combine all the data. The process of merging the different components takes significant time due to the possibility of misalignments. Finally, the aligned point cloud was exported in .e57 format, compatible with the Reality Capture 1.2.1 computer program, in which future analyses were performed. Thus, a high-quality model was obtained, approximately 5 cm by 10 m, which is also suitable for being published and run in an online environment.

3.2. Creation of Three-Dimensional Models of the Interior of Darvas-La Roche House

In order to obtain coherent and integrated three-dimensional models of the interior of the Darvas-la Roche House, it was decided to create a 3D structure based on the architectural sketches of both floors, and within it, the panoramic images taken from the interior were inserted to give a touch of realism to the whole model. In this way, the aim was to obtain a complete model of the studio object, which can also be accessed during tours that use virtual and augmented reality.

To obtain the 360° panoramic images, only the rooms that are interesting from an artistic, historical, and tourist point of view were chosen, leaving undocumented those rooms that serve as storage, cloakrooms, or have other similar uses. On the other hand, regarding the recreation of the 3D structure based on the sketches, this aimed at rendering the entire house without discounting any room.

In order to acquire photographic databases for the creation of panoramas, the field work followed a methodology adapted for the accessibility of heritage elements through virtual tours [49,50,51,52,53]. The acquisition methodology was planned in such a way as to take into account the characteristics of each scene to be photographed and the internal and external parameters of the camera. Thus, the same camera (Canon EOS R; Canon Inc., Tokyo, Japan) was used as in the case of obtaining the data for the terrestrial photogrammetry technique; as a difference, instead of the full-frame RF 24-105 f4 L IS USM lens (Canon Inc., Tokyo, Japan) used in the previous case, an object with a wide aperture of the full-frame RF 15–35 mm F2.8 L IS USM (Canon Inc., Tokyo, Japan) type was now chosen. This choice was based on the fact that panoramas require high quality and a wide field of view, and the most common lenses used for this purpose, the fisheye type, suffer from distortions and low quality, even if they are easier to use and faster to purchase. The camera was attached to an adjustable panoramic head of the Nodal Ninja Ultimate M2 type, which includes rotating elements with gradations, which helps to optimally position the camera for capturing photos. At the same time, the panoramic head helps to reduce the parallax that most panoramic images acquired at cultural heritage elements suffer from [54,55,56]. To reduce the unwanted influence of parallax, in addition to using the panoramic head, the internal characteristics of the device were calibrated in the laboratory, and the external characteristics (mounting mode on the panoramic head) were adjusted on the spot using between 2 and 3 dedicated tripods. Considering that in some rooms, the degree of illumination was too low to leave the possibility of achieving panoramic images of sufficient quality, it was decided to implement a partially or totally controlled light environment. This was achieved by using between four and six powerful 45 W Fancier LED-SLH4A lamps (Ningbo Fancier Photographic Equipment Co.,Ltd, Shanghai, China) with bicolor light between a temperature range of 2700 K and 5500 K, with diffusers made of textile material installed, to reduce the amount of direct light received by the photographed surfaces. In order for the scenes to record approximately the same amount of light and a smooth transition, the LEDs were directed directly towards the scene to be photographed, keeping a constant distance between them.

In order to best represent the entire analyzed objective, 11 panoramic images were chosen to be captured. For each targeted room, one or two panoramas were purchased, depending on the level of interest the room represents and its dimensions. Thus, between 92 and 168 individual photos were captured for each panorama, positioned on 6–10 rows and 12–16 columns, with a distance between pairs of photos between 15° and 30°; all images were taken in uncompressed and unmodified format, .CR3. After sorting on the spot, removing, and recapturing the scenes that recorded errors, all the photos were inserted into the Adobe Lightroom Classic program, where the database was homogenized by eliminating geometric, textural, and exposure distortions.

The databases, once formed and pre-processed, were inserted into PTGui 12.20, a program that considers panoramic image stitching, through semi-automatic processes. Within this software, masks were applied to the images; they were cropped and optimized to obtain the most faithful results. At the same time, where the pairs of images obtained had too few connecting elements, control points and connections were manually inserted. Finally, the panorama sampling point was masked, superimposed on the tripod and panoramic head, the colors and brightness of the scenes were balanced by the multi-band blending technique, and the final panorama was exported in .JPEG format at a size of 4000 × 2000 pixels.

3.3. Making Darvas-La Roche House Accessible through Virtual and Augmented Reality

In order for all the three-dimensional elements created within this study to be actively promoted, it was decided to create an integrated virtual tour. Such an approach aims to transmit advanced knowledge, both visual and informational, about the studied object through virtual and augmented reality. The three-dimensional platform is mainly addressed to tourists and those interested in Art Nouveau architecture and Jewish heritage for an exhaustive off-site analysis as well as to all those who cannot visit this architectural monument in person, for various reasons.

The virtual tour created included both the three-dimensional model of the exterior created based on the combination of photogrammetry and laser scanning results, as well as the 3D graphic model of the interior and the 360° panoramic images taken in each interior room. Considering the different materials in terms of structure and format, one of the hardest tasks, as indicated by Büyüksalih et al. [57] and Koeva et al. [58], was to find a balance between all these materials so that they could work together in a smooth and unified way. Thus, as support for the virtual tour, the Bihor360° platform was used, developed in a previous study by Caciora et al. [50]. This is a minimalist platform for the virtual promotion of the cultural and natural heritage of Bihor County, created using the WordPress CMS. Within it, the Woo3D Viewer Pro commercial plug-in was used to run the 3D models; for 3D models to correspond to that plug-in, they had to be processed so that they were compressed to reach a maximum size of 50 MB without losing their quality as much as possible, with the most efficient extension being glb.

The iPanorama 360° plug-in was used to integrate the panoramas within the 3D models, which were reduced to 4000 × 2000 pixels, a maximum of 20 MB, and a .JPEG extension. The plug-in allows the possibility of adding hot spots for additional information, as well as guitar arrows for interactive navigation through the 360° image sets. The resizing of the materials was performed to shorten as much as possible the loading times and their running mode, but at the same time without greatly reducing their quality. This makes them available to be accessed from any device with the best possible performance.

To complement and energize the visual materials, historical and social information in audio format in English was inserted in the virtual tours of the Darvas-La Roche House. These descriptions cover both the three-dimensional models of the exterior, where the descriptions cover the general architectural component, and the 360° panoramic images of the interior, where the audio recordings have the role of presenting the user with information about each room viewed separately. For this approach, the text-to-speech NaturalReader 16 software was used, which has the property of converting a written text into MP3 format in different languages. These files can be successfully integrated both in the Woo3D Viewer Pro 1.3.1 plug-in and in iPanorama 360°.

4. Results

4.1. The Results Obtained in Terms of Creating 3D Models

Regarding the databases obtained for the implementation of the photogrammetric technique, they contained a total of 3326 images, of which 2517 were captured with terrestrial cameras, and the remaining 809 had UAV as a capture device. Not all photos were further processed to obtain the rare cloud of points, considering that some of them were considered inappropriate or of low quality, following the processes of manual and automatic filtering of the databases to determine the possible discontinuities. Thus, 154 photos were considered useless for the photogrammetric process, suffering from problematic exposure, lack of texture, lack of key points for alignment, or other internal or external shortcomings of the camera; the problems of deleted photos could not be fixed by processing them in different dedicated programs.

After preparing a suitable photographic database, the images were imported into the Reality Capture 1.3 software for their detailed alignment and to obtain the final sparse point cloud. The chosen software is one that uses the SfM algorithms to determine the intrinsic and extrinsic parameters of the camera and to align all 3172 photos in a single reference space, as it was presented for the first time by Triggs et al. [59]. At the same time, Reality Capture has various capabilities to easily combine the data obtained from photogrammetry with those determined by laser scanning, something of great importance for the present scientific endeavor.

Despite the fact that the data acquired with the UAV benefits from a well-defined coordinate system, thanks to the mountain GPS on the drone, the first step in the process of creating the three-dimensional model of the Darvas-La Roche House was the identification of the target control points on the ground in order to guide the computerized program in the data alignment process and finally achieve a very high level of precision. According to the theory (to be quoted), a number of at least three connection points must be created on the component to be able to align different images and use the coordinate system. Cycling between the images manually and adding control points fixes most alignment issues and helps connect as many photos as possible. In the present case, 97 control points were used (Figure 5), and the results obtained were somewhat consistent, generating clouds of very voluminous points (Figure 6).

Figure 7 shows the reprojection errors recorded by each Cartesian coordinate (X, Y, Z) of the photos after applying the corrections based on the measurements of the target points in the field. The average value of the error of the X coordinate was 1.5 m, with absolute values between −0.67 m and 0.51 m. Regarding the Y coordinate, the average of the reprojection errors amounts to 1.6 m, with absolute values between 0.2 and 4.4 m. The lowest values were integrated in the case of the Z coordinate, which had average values of 1.2 m and absolute values that deviated between 0.2 and 3.9 m. About 53.9% of the data recorded reprojection errors smaller than 1.5 m after applying the corrections.

The first result obtained through the photogrammetry technique is the rare point cloud of Darvas-La Roche House. It is characterized by the presence of a large number of raw points, directly proportional to the very large volume of data entered into the software (Figure 8a). The cloud of raw points was manually and automatically filtered to eliminate unwanted points and those that were outside the area of interest. Manual filtering involved the selection and removal of points from the cloud, while for automatic filtering, analyses aimed at determining reprojection errors, projection accuracy, and scene reconstruction uncertainties were applied.

Based on the sparse point cloud, the dense point cloud was determined by means of the MVS calculation algorithm [60], amounting to approximately 310 million points. The principle on which it is based is that of estimating the positions of the cameras within the scene and generating additional point clouds within the three-dimensional model through matching, expanding, and filtering processes [61]. As in the case of the sparse point cloud, the dense point cloud was filtered to remove unwanted points, both by manual and automatic processes (especially in terms of confidence and color). At the same time, to make it easier to work with the 3D model made of points, it was decimated by approximately 97%, leaving only approximately 3% of the initial points in the final model.

After obtaining the clouds of dense points with the help of photogrammetry, we went on to add the clouds of physical points obtained on-site with the help of the laser scanning technique. These were initially recorded and processed in the Leica Cyclone 2023.1 software (considering that they are BLK360-type data) based on the 12 target points applied to the building. These target points were intended to provide a solid basis for the detailed interconnection of the data obtained from the 8 scanning points. This is all the more important as the scans aimed to scan both the facade of the building and the back garden, as well as the decorative elements in the yard.

The scanning of the main facade of the monument resulted in obtaining a cloud of points in the amount of 2,856,429 (Figure 9a), while the garden and the rear facade were represented by a cloud of points containing 9,391,958 individual coordinates (Figure 9b).

The final dense cloud of points was created by inserting the results of photogrammetry and laser scanning into a single reference space in the Reality Capture computer program and aligning them based on the control points. In the end, a cloud of points was obtained representing Darvas-La Roche House, containing approximately 21.4 million individual points.

This last three-dimensional model made up of points was the basis for the further creation of the polygonal model of the case study by interconnecting all the individual coordinates, determining the triangulation, and arranging a continuous surface over the points making up the cloud. This stage is a very important one, considering that the three-dimensional structure is an indispensable criterion that gives quality and correctness to any 3D representations. Therefore, it was decided that the resulting model should have as much triangulation as possible, represented by a high number of faces and vertices. Considering that the obtained polygonal model also includes parts that are considered useless or that are not part of the area of interest of the digital reconstruction, the operator’s interventions in this phase considered their manual filtering and elimination, where appropriate. The process is also useful for creating fixed boundaries in the case of the polygonal model.

After decimating the mesh data, some issues appear in the mesh due to bad alignment or reflective/translucent surfaces. In order to fix these issues, the model was exported into 3D modeling software, such as Autodesk Meshmixer 3.5 and Autodesk 3ds MAX 2023. Within these computerized programs, the cleaning of the polygon model and the elimination of problems related to wrong forms, pointy edges, surface bumps, and holes in the mesh were taken into account. Due to the versatility of the computer programs chosen for cleaning the mesh, all these shortcomings have the possibility of being solved by means of smoothing, filling, cutting, and remodeling options.

Some examples are relevant to the current situations, the most well-known being the problems of the lack of texture recorded by the transparent and translucent surfaces within the three-dimensional models composed (fully or partially) with the help of the photogrammetry technique, especially regarding the windows. These issues are very difficult to repair because they involve a special remodeling technique that involves replacing the wrongly rendered surfaces with a new surface with the characteristic of being smooth. In Figure 10a, the curved window resulted in the wrong model due to the shape and translucency of the glass, issues that were fixed in Autodesk Meshmixer through the surface reshaping options. Within the model, multiple surfaces of the windows were identified that registered rendering problems and the wrong shapes, requiring interventions for remodeling. At the same time, the surfaces with cylindrical geometry required smoothing and some retouching on the edges, while some of the flat surfaces registered bumps, which required flattening and smoothing to achieve better and higher-quality results (Figure 10b,c).

After all the texture problems of the polygonal model were solved and the cleanest textures were obtained (Figure 11), it was re-imported into the Reality Capture software for the implementation of texturing processes. The information that was the basis of the implementation of this stage was that of the original colors and textures of the monument, captured from the field, both by means of photogrammetry and laser scanning. This process, as indicated by [62] in their work, aims to calculate, based on the available information, the individual coordinates of each vertex within the three-dimensional surface so that the RGB values can then be projected within them.

Depending on the desired size of the model, the model UVs need to be unwrapped to a specified size, which can significantly increase the number of texture parts or lower them. The result was a fully colored 3D model incorporating a texture identical to that of the original monument (Figure 12).

For all the results obtained within the photogrammetric processes and those related to the processing of data obtained from laser scans, the desired quality of the models was average; these were the options selected for running the operations. This took into account the fact that the final digital model obtained is to be published on the Internet in a virtual tour, and obtaining a high or very high quality of the model would have generated a file with a much too large size, well over the 30–35 MB recommended [50], thus being unsuitable for publication. Although the model can be decimated so that its size can still be reduced as much as needed, such a process implemented on a much too large scale would have inevitably led to a drastic loss of the model’s geometry and texture. Therefore, it was desired that the final model obtained should include the maximum quality of the structure and texture with minimal costs in terms of size. The average quality preselected from the beginning generated a model that required limited operator intervention for its decimation, having a good quality and a final size of 33 MB.

4.2. The Results Obtained in Terms of Creating Panoramic Images

In order to make accessible the most important and interesting features inside the Darvas-La Roche House, it was decided to obtain 11 panoramic images, acquired from different parts of the monument. They show only the rooms where exhibitions are currently organized or rooms that are important to visitors due to their construction, design, or story. The 11 panoramic images created thus lead to the accessibility of nine different rooms, four of which are located on the ground floor of the monument and five on the upper floor (Figure 13). Most of the rooms are permanent exhibitions, which include period furniture, pieces of clothing, jewelry, and various accessories that belong to the “Belle Epoque” era. But within the accessible rooms, there are also some that host only temporary exhibitions, for which a new set of data will be taken, which will be updated in the virtual tour every time the exhibition changes.

In addition to the nine rooms that are of real value for tourism, for a holistic inclusion of the monument in a virtual tour, their interconnection must also be taken into account, so as to ensure the best transition from one to another [53]. In this sense, it was also considered to take panoramic images that capture only the connecting elements of the museum’s interior, such as parts of the hallways, access areas, stairs, or intermediate rooms (which are not of particular interest to the visitor). The final goal of these steps was to ensure a more realistic and uninterrupted transition to the museum’s hot spots. Thus, visitors will experience a visit as natural as possible, in which they have to cross corridors and access areas to reach the exhibitions. It was not necessary to implement a virtual tour in which those interested “jump” from one room to another without the visit having a well-planned logic and itinerary.

4.3. The Results Obtained in Terms of Creating the Virtual Tour

The three-dimensional and panoramic models created in the present study were the basis for the accessibility through virtual reality and augmented reality of Darvas-La Roche House. They provide visitors with a complete and complex remote visiting experience, through which visitors can move freely, interact with the exhibits, and find out valuable information about them, thus creating a feeling of physical presence inside the monument.

As the platform for uploading the virtual tour, www.bihor360.agts.ro (accessed 16 December 2023) was chosen, developed in a previous study [50]. This is a site with a simple and user-friendly interface, in which only markers have been inserted that have the role of guiding the visitor in the simulated environment without requiring the user to familiarize himself beforehand with how to run the online platform.

The interface makes a clear separation between cultural heritage sites and natural heritage ones, with Darvas-La Roche House being placed in the first category. After opening the menu dedicated to these buildings and selecting the one that is the object of the current study, users start by viewing the three-dimensional model made by integrating the data obtained from photogrammetry and laser scanning. This visual information is complemented by a descriptive section (both in visual and auditory format), which has the role of giving users some general information about the history of Darvas-La Roche House, the style of construction, and what this represents for Oradea Municipality, as well as an accurate location of the monument within the city, the pin being shared from Google Maps (Figure 14).

From the viewing window of the three-dimensional model, users have the opportunity to view the different panoramas of the interior of the monument by accessing the “view panel” button. Navigation through the interior of the virtual structure is performed with the help of the pop-up and guide buttons. The sequence of panoramas within the virtual tour respects the sightseeing routes that tourists usually follow. Thus, the virtual tour begins with the visualization of the panoramas from the ground floor of the building, with the first four panoramas being dedicated to the rooms that include temporary or permanent exhibitions, while the fifth panorama aims to transition to the upper floor of the building. The second floor was made accessible through a number of six panoramas, thus being exposed to the users to be viewed: The access hall, the great hall, the men’s room, the ladies’ room, and the bedroom; the visiting routes following a logical sequence in this case as well (Figure 15). For easier guidance in the virtual environment, a cartogram of the level the user is at and the camera he is viewing at the time has been arranged for viewing each panorama.

The created virtual tour can be accessed from all types of devices, this being one of the most important aspects in order to promote the heritage object, as suggested by Mah et al. [63], Pasquaré Mariotto and Bonali [64] and Choi et al. [65]. At the same time, so that the created virtual tour can be accessed by as many interested people as possible, it does not involve the use of sophisticated devices related to augmented reality, but only an output device connected to the Internet.

5. Discussions

The use of a combined approach of terrestrial and aerial photogrammetry with terrestrial laser scanning has multiple advantages. It is mainly highlighted that the results obtained are superior to those obtained with the use of a single method, considering that both the resolution of the data and the texture of the final results are of superior quality. No single system can do them all, but their combination can introduce a new level in terms of acquiring cultural heritage data. At the same time, the combination of the two methods makes even the most difficult-to-access areas of heritage buildings accessible (roof, etc.), and the translucent, reflective, or glossy surfaces are reproduced in a very accurate way. Among the disadvantages, we must mention the very high costs of the equipment dedicated to the three data acquisition techniques, the fact that the use of a combined method limits the portability and scanning and processing times, and the fact that acquisition and processing of the results require the experience of an operator. At the same time, the limitations regarding the acquisition of images in the case of the present monument were due to the fact that the house is framed by other buildings, the acquisition of images being difficult, and the final result needing to be partially reproduced together with the neighboring buildings. Moreover, the retrieval of the images required several trips to the field, during which the setting in the courtyard of the Darvas-La Roche House was changed several times. Therefore, during the processing and preprocessing of the image sets, the optimization of the packages was considered so that the unwanted influence of these changes was minimal.

One important reason to scan details from Art Nouveau buildings is to better understand the style and its unique characteristics. This can help us better appreciate the artistry and craftsmanship of the movement, as well as provide inspiration for contemporary artists and designers. Further, scanning details from Art Nouveau buildings can help with preservation efforts. Many Art Nouveau buildings are historic landmarks, and documenting their intricate details can aid restoration and maintenance efforts. Overall, scanning details from Art Nouveau buildings is an important way to gain insight into this influential artistic movement and preserve its legacy for future generations.

As indicated by Kantaros et al. [66] and Smaczynski and Horbinski [67] in their works, the quality of the three-dimensional model obtained is of real importance for architectural objects, considering that every fine detail of the scene must be represented as faithfully as possible. In order to overcome these challenges, the use of high-quality equipment with high-resolution settings in terms of data capture [68], as well as three-dimensional rendering techniques that took into account the correction of problem surfaces of the mesh [69].

Thus, the obtained model stores data in one place in multiple formats, leaving the possibility of performing technical analyses on the architectural details and efficient monitoring of the degradations that may occur in order to organize the most effective restoration and conservation strategy [70,71]. In this sense, the combination of a terrestrial laser scanner and photogrammetry led to obtaining a mesh of very high quality, which includes on a very large scale all the architectural details of the Darvas-La Roche House (Figure 16). This is an important starting point for a superior analysis of them and for the preparation of long-term conservation strategies.

For a more intuitive and complete experience, the virtual tour created within the online platform can be adapted to be accessed through augmented reality within the museum using specialized devices, such as VR glasses and controllers. This approach has the advantage of offering users the opportunity to perceive what is visualized in an improved way by superimposing computer-generated information in a real environment [72,73]. In the case of Darvas-La Roche House, the Oculus Quest 2 virtual reality headset was used so that tourists could view the panoramas made inside the monument in an augmented environment. Through it, those interested can freely explore the interior of the museum; the movements of turning the head and viewing the points of interest are very natural; therefore, the visiting experience is much improved compared to the use of classic virtual reality (Figure 17) [74]. This is also indicated by authors such as Cruz et al. [75], Malik et al. [76], or Sebastiani [77], who mention augmented reality as a means by which the general public can understand and experience cultural heritage more intuitively and interactively.

The virtual experiences created in the present study can be used as an educational tool, which can assist in disseminating information about the cultural heritage built in Art Nouveau style in the municipality of Oradea [78]. This is all the more important, as currently, no building in the city built in this emblematic style has a well-prepared virtual tour.

Pasquaré Mariotto and Bonali [64] indicate that these virtual approaches can also stimulate the young generation and attract them in an interactive way to discover and be interested in the local cultural heritage, which thus becomes transposed into a kind of video game. At the same time, Bec et al. [79] and Xiao et al. [80] consider virtual and augmented reality as a sustainable alternative to classic virtual experiences, especially where they are difficult to obtain due to environmental and social conditions or due to the fact that the site is positioned in a remote area, where access is very difficult to achieve. In the case of some disadvantaged communities, such an approach can induce added value and development opportunities due to the superior promotion of the destination among individuals interested in discovering the cultural heritage of a certain region.

6. Conclusions

Virtual reality applied to monuments belonging to cultural heritage is one of the most up-to-date technologies, considering that it has the potential to offer multiple results, which can be the basis for the promotion of the monuments, their visualization, as well as for conservation and restoration of the site. It is important for tourism because it allows users to perceive the surrounding world in an improved way, dynamizing the user experience by superimposing computer-generated information within scenes taken from the environment.

In this study, the accessibility through virtual reality of a heritage monument built in the Art Nouveau style in the municipality of Oradea was carried out by using a complex and complete methodology to obtain highly accurate virtual products. This combined methodology proved to offer much better results than using one alone, both in terms of making a large-scale monument, such as the Darvas-La Roche House, accessible as well as for rendering the subtle motifs that adorn the buildings in Art Nouveau style.

The innovative products obtained were used to be integrated into a virtual tour dedicated to people interested in discovering the cultural heritage of this municipality in Romania. Through the use of photogrammetry (aerial + terrestrial) and terrestrial laser scanning, combined with the creation of panoramic models and obtaining descriptive information, a voluminous and complete database on the targeted objective was obtained. The inclusion of all the virtual products obtained within a tour that can be accessed through virtual reality, as well as through augmented reality, has generated an interactive and integrative tool that has the characteristics of being a sustainable virtual environment for the promotion, visualization, and even preservation of the heritage element targeted.

The results obtained in this study are a multilateral information tool that has the characteristics of dynamizing the already-existing (traditional) databases with a view to expanding the existing knowledge of the monument and acting as a marketing tool for the Oradea tourist destination. This is all the more important as the municipality of Oradea does not currently have such digitization and accessibility initiatives for the general public of the heritage elements that define and individualize it among other cities in Romania and Eastern Europe. But in essence, this synthetic environment is not intended to replace traditional tourism but rather aims to provide those interested with an additional computer base, intended to contribute to a better understanding of the cultural-historical objective.

Based on the current methodology and considering that the municipality of Oradea is a multicultural city, future research will consider the digitization and accessibility of as many emblematic buildings as possible for the history of the local community. All this will be brought together on the tourist promotion website of the destination image, and the quantitative and qualitative data will be made available to the decision-making bodies for the purpose of promoting and preserving the monuments, as well as to the interested public. The generalization and reapplying of the working methodology within the present study to other buildings belonging to the Art Nouveau cultural heritage in the municipality of Oradea has the potential to be easy to apply, considering the fact that most of these monuments have common characteristics: being in the center of the city, being decorated with zoomorphic motifs, and being painted in light colors.