A Comprehensive HBIM to XR Framework for Museum Management and User Experience in Ducal Palace at Urbino

Abstract

Digitization of Cultural and Museum Heritage represents one of the most engaging challenges that would ensure a sustainable and ethical approach for next generations; digital technology’s pervasiveness imposes a comprehensive management of architectural heritage by producing facsimiles of buildings and artworks and by testing robust methodologies, with the final result of providing effective multipurpose models. In this context, the main goal of the present paper is to develop a semantically aware HBIM model that includes an intelligent objects parametrization, leveraging Extended Reality (XR) technologies and digital curation of contents to pursue the preservation of Cultural Heritage (CH) as a whole. This research is implemented in the case study of the Ducal Palace of Urbino that houses the National Gallery of Marche. It was chosen as a remarkable example of a museum located in an architectural complex with a relevant historical background and fine detail of shapes and mouldings. In Italy, as in other European scenarios, museums and their collections need suitable dissemination and management systems that take advantage of the recent digital paradigms. The challenging approach is to exploit existing platforms and software and to adopt a cognitive modelling process, able to develop tools supporting managers and museum curators while enabling user experiences using immersive and interactive features. In order to stress the workflow, this work proposes the use of families with high Level of Detail (LOD) and high Level of Information (LOI). The present article provides, as well, an accurate data enrichment process specifically designed for a gallery’s artworks such as paintings and sculptures, in line with the national and international policies. The study presents a robust and reproducible methodology for digital musealization and management, focusing, as future overall objectives, towards a greater merging between the HBIM approach and XR technologies, also facilitated by training new professional figures with more in-depth digital skills.

1. Introduction

The management of Cultural and Architectural Heritage is a complex activity that requiress a number of extended competencies, both in humanities and informatics. In fact, in the last decades, massive processes of digitisation and digitalisation of artworks have begun in order to preserve, enhance and disseminate the CH. This phenomenon enables a variety of opportunities that represent an interesting challenge for digital heritage research. The proliferation and the growing quality of geometrical and aware 3D models are paired with the need to guarantee interoperability, reliability of information and multidisciplinary collaboration within the same environment. HBIM models provide accurate morphological features and represent a suitable database where each architectural object includes all information, with a hierarchical structure. Currently, Heritage Building Model (HBIM) platforms are considered the most well-performing tools for historical architecture management, addressing all purposes.

For a long time, many HBIM based works focused on representing and mapping peculiar geometric features of the architectural complex according to point cloud data.

More recently, HBIM models of an architectural complex have been focused on a heterogeneous amount of information concerning geometry, materials, structural behaviour, historical events, construction phases, stratigraphic analysis, microclimatic features, energetical performance and restoration works, enabling the definition of Level of Knowledge (LoK) [1].

Nonetheless, despite the presence of many studies and funded international projects concerning DCH, a considerable amount of data are, as matter of fact, addressed only to specialized users [2]. Furthermore, the lack of interoperability between software [3] and the lack of technical know-how of practitioners for the application of the technologies for HBIM [4] are relevant weaknesses.

Extended Reality represents an effective solution to this issue. It provides a number of immersive technologies which include Virtual Reality, Augmented Reality and Mixed Reality, currently widely used by creative industries such as video entertainment and gaming. XR refers to the spectrum of experiences that blur the line between the real world and virtual environments [5]. A key application of HBIM will be the use of MR or VR to enhance user community interest in cultural tourism [6].

The present work provides a comprehensive methodology of HBIM to XR (see Figure 1), where two immersive applications have been developed.

The first is a Mixed Reality App that concerns the optimisation of museum access and management, addressed to both professional and non-professional users, based on collected BIM data. The second is a Virtual Reality App that proposes a remote user experience leveraging the high level of details of the HBIM model. Furthermore, this study becomes part of HBIM for museum line developing and testing a semantic ontology for museum management and digital curation of contents. All of these topics have been implemented in a real case of study, the Ducal Palace at Urbino, which houses the National Gallery of Marche. Due to the size of the complex, this work specifically focused on the Piano Nobile, which can be considered the most representative floor of the Ducal Palace, thanks to the architectural complexity, the historical stratifications and the quality of the artworks collected.

2. Background

2.1. HBIM Approach

In the last several years, Building Information Modeling (BIM) has been affected by growing employment in the Facility Management–Architecture Engineering Construction (FM–AEC) industry. It permits an effective and synchronized exchange of information among stakeholders [7], and it represents a considerable reduction in terms of costs and time compared to a non-BIM [8]. Moreover, different European countries have adopted a number of policies that promote the adoption of a BIM based framework in order to establish a univocal reference for all the figures involved [9].

For more than a decade, the BIM approach has been extended to Cultural Heritage, enabling a new action line labelled as HBIM [10]. Many studies focused on the opportunity to include survey data within BIM platforms, reaching robust results with the Scan to BIM process, as reported in [11]. Hence, a HBIM framework usually starts from the acquisition of built heritage with sophisticated technological tools such as laser scanning photogrammetry and drones [12]. HBIM concerns the creation of novel libraries of reusable parametric objects, based on survey and historic architectural data [13,14], in order to develop a semantically aware 3D model.

Chiarabrando et al. [15] presented a H-BOMs (Historical Building Object Modeling) methodology where teamwork imported a points cloud into a BIM platform to achieve a semantically aware 3D object.

For the management and representation of such a large dataset, some ontology-based strategies are adopted to define building information organization. Nevertheless, even today, as regards Facility Management, a standard ontology, able to convert BIM data into a semantic format, has not yet been developed [16]. Authors in [17] adopted an ontology-based approach in the development of augmented reality (AR), leveraging semantic annotation of building information models.

The degree of geometric accuracy and the quantity of information related to a BIM model are defined through different denominations: Level of Detail, Level of Geometry, Level of Information Need and Level of Development. A generalized consensus has been established as regards the AIA’s (American Institute of Architects) proposal that states five levels of development: LOD100, LOD200, LOD300, LOD400, LOD500 [18].

The main source in Italy is provided by the UNI committee; in particular, we refer to UNI 11337, which divided LOD in level of geometry (LOG) and level of information (LOI) and defined seven specific levels in order to classify BIM models: from A to E for architectural drawing and F, G for As-Built representations [19].

In a 2018 work [20], the authors introduced the concept of GoG (Grade of Generation) protocols to describe different Levels of Geometry (LOG) in function of the geometry resulting from the laser scans, in the line of traditional Level of Detail (LOD) to the case of architectural heritage restoration. This reflection is upgraded and validated by the already mentioned UNI’s definition.

Castellano-Román et al. (2019) [1] proposed a general characterization of HBIM, transferring BIM concepts originally related to the production and management of new site architecture to the specific context of cultural heritage; the concept of LOD is thus upgraded as Level of Knowledge (LOK) for heritage application. According to this conceptual proposal, the HBIM model of the Charterhouse of Jerez has been generated, primarily focusing on the generation of knowledge that should be incorporated in a digital replica.

In a recent study, authors discuss the theoretical issues related to the unsolved discrepancy between the rigidity of modelling tools and the high flexibility needed by heritage elements. The outcomes of the research led to a clearer understanding of features of the HBIM model through the adoption of an additional parameter, the level of reliability for digital objects.

Through the study of the Architectural Order, it is possible to include formulas and proportions within HBIM objects. Similar approaches enable the paradigm of aware models and libraries. The fundamental text for this approach is “Renaissance Revit” [21].

Using, as a case study, a Doric column modelling, authors showed the interest in architecture’s geometrical rules also for dissemination and learning by developing VR applications based on the HBIM model [22].

The trace of parametric thinking in Palladio’s designs, as highlighted by several scholars, provided Spallone and Calvano [23] the opportunity to develop an instrument to interpret the compositional logic of the villas Palladio. Using treatise as a source, they created 3D parametrical models through the combination of VPL and BIM tools.

In the HBIM field, researchers and technicians often dealt with artworks or unique architectures with no chance of parametrization. Nonetheless, due to the importance of the parametric intelligence, some cases aimed at achieving the right balance between HBIM features and the nature and needs of the buildings [24].

Barrile et al. (2022) [25] faced the issue of including specific information in the model through a segmentation of a historical manufact and by focusing on the implementation of a 4D method that provides the building’s evolution over time.

Recent works leveraged the HBIM approach to develop a suitable management system for museums.

In the case study of the Egyptian Museum of Turin, Giovannini and Lo Turco [26] deal with the relationship between the container (building) and the content (collection’s artworks). They realized a data enrichment of 3D BIM objects categorized as a “Collection Information Model”. The authors showed a BIM-oriented approach for the context of Museum Management, dedicated exclusively to professional users.

The research conducted in the National Archaeological Museum of Naples—MANN [27] aims to present an HBIM workflow that is general in scope and supports the planning, implementation of maintenance and conservation activities of built heritage. The outcomes show that the workflow is sufficiently flexible to accommodate a heterogeneous range of objectives.

Tucci et al. [28] proposed a holistic information management system, labelled as Museum Building Information Modeling (M–BIM), for the Florence Academy’s Gallery, where 3D mesh objects of artworks have been decimated, segmented and included in BIM model.

The present work, also framed in the M–BIM line, aims to develop a comprehensive management system for museums in line with the present state of the art and according to policies and decrees provided by public institutions. Moreover, it broadens the usage of HBIM features for non-professional users’ engagement through immersive virtual reality and mixed reality applications.

2.2. XR for Historical Architecture

Interoperability is facilitated through the conversion in the IFC interchange format, developed by BuildingSMART [29], that enables the use of BIM models in other software without loss of information.

A study on the Egyptian Museum of Turin [30] employed IFC parameters, exported from a HBIM model, to carry out some evaluation about microclimatic data of museum’s objects, using VPL algorithms.

Alizadehsalehi et al. [31] presented a complete methodology implementing BIM models for Extended Reality applications. The study provided, as well, a taxonomy and a comparison of the ultimate technologies and devices available.

In the work concerning the “Basilica di Sant’Ambrogio” in Milan [32], authors started from the building’s points cloud and realized a HBIM model for developing XR applications. Despite its complexity and actual potential, the outcomes reveal a considerable separation between the several workflow parts, a problem that the present study aims to overcome.

In a work based on the EU-funded Project INCEPTION [33], authors realized a HBIM model of the Museum of Hydra and developed a Web/Mobile App (AR/VR) to offer an innovative alternative to conventional Museum/CH Site Tour Guiding.

In a non-BIM based study [34], authors developed a Mixed Reality application, in order to enhance the intangible dimensions, that includes the practitioner’s gestural know-how in craft practice.

An increasing number of research teams are adopting these new technologies, even if some studies highlighted the lack of proper levels of knowledge and the complete and open interoperability between HBIM and XR [35]. The outcomes of this research work provide some tools in order to clarify these issues.

2.3. The Ducal Palace of Urbino and the CIVITAS Project

The Ducal Palace of Urbino, seat of the National Gallery of the Marche Region, is one of the most emblematic architectural and artistic examples of the Italian Renaissance. The author and diplomat Baldassare da Castiglione (1478–1529) described it as a “city in the form of a Palace” [36].

The building reached its peak in the XV century, during the dukedom of Federico da Montefeltro. He involved the most remarkable artists of his time, such as Luciano Laurana, Francesco di Giorgio Martini, Bramante and many others, for the creation of prestigious architectural elements, paintings, sculptures and furniture that make the Ducal Palace a work of art of its own.

Since 1912, the National Gallery of Marche was established there, including a rich collection of paintings and other finds among the most significant artworks of Italian Cultural Heritage.

For many years, the Polytechnic University of Marche (UNIVPM) worked in partnership with the museum through many initiatives and projects. Some of the outcomes are the development of the “Ducal App” [37], that enhances access to three masterpieces from the museum’s collection, CIVITAS project [38], which provided the comprehensive points cloud of the Ducal Palace, and the deep learning research works [39,40], where a semantic automatic segmentation of the Palace’s court has been implemented.

The UNIVPM strategic project CIVITAS (ChaIn for excellence of reflectiVe societies to exploit dIgital culTural heritAge and museumS) has, as its general goal, making the Ducal Palace of Urbino and the Galleria Nazionale delle Marche into places of experimentation and research for excellence in the digitization of cultural heritage, allowing the massive three-dimensional digitization of the Ducal Palace of Urbino (see Figure 2). Through the TLS technique, the spaces of the Ducal Palace were surveyed with the integration of three different laser scanner instruments chosen according to the characteristics of the portions: Leica Backpack, Leica ScanStation C10 and Leica ScanStation P40. The survey operations generated an overall digital model of the Ducal Palace composed of 1.790 million points with a maximum alignment error of 0.006 m [12]. The CIVITAS project and its outputs were the basis for the implementation of the methodology presented here.

3. Scan-to-HBIM

Among the main objectives of this work, there is the creation of a multipurpose HBIM model that can match different approaches, strategies and needs of the stakeholders. Therefore, our group developed a semantic ontology to manage suitable geometric (LOG) and information (LOI) layers in order to achieve a high level of development (LOD) according to the UNI 11337 standard.

The first step was to process the Ducal Palace’s point cloud in order to make it more usable in computational terms through random decimation and segmentation work. Specifically, within the Cyclone software, we extrapolated and subsequently divided the “Piano Nobile” (first floor) into six apartments (see Figure 3) according to the literature from the 15th century [41].

3.1. Making of the Piano Nobile’s Smart Model

For the creation of the BIM data, the modelling software Revit was used. This software allowed the point cloud model of the main floor to be linked directly to the HBIM environment so it could be used as an accurate reference of the building’s existing conditions, which is essential for obtaining an as-it-is model. In addition, five fundamental principles guided the entire workflow, namely:

• adaptability of the parametric object;
• modelling as a cognitive process;
• automation of work processes;
• declination in the three levels of detail;
• development of a semantic ontology.

3.1.1. Parametrical Families Modelling

In this case study, due to the complexity of the historical building, such as walls with uneven thicknesses, deviations, lack of squareness, non-serial decorative elements and unique artworks, diversified modelling strategies were adopted.

The walls were created according to the “direct modelling” technique [42,43], using the point cloud as a basis to define the guiding lines and to identify the thicknesses with a low level of approximation. We defined fourteen families of walls, enriching them with the stratigraphic information identified in the literature about the interventions on the buildings.

The portals, windows, chimneys and other decorative elements were created through “parametric object-oriented modelling”. These objects were conceptualized into their generative matrices, from which we created the “source families”, declined into their variations and adaptations in the HBIM model according to three levels of representation: coarse, medium and fine.

To better understand the operational process, the modelling process and strategy concerning the 86 portals in the Piano Nobile are explained as exemplary (Figure 4 and Figure 5). They were summarized into two source families containing all the morphological and decorative combinations necessary to cover the entire knowledge domain, with parametrical rules about mouldings and geometries.

The study of the historical data, particularly the architectural order [44] and the method to implement it in an HBIM parametric modelling environment [21], allowed the inclusion of information, proportions and equations characterizing each architectural element. Since this is a Renaissance building, this process is particularly suited to the case study.

The vaults were modeled according to different techniques to automate the process of insertion into the model. The geometrically simpler vaults (barrel, groined, schifo) were created as adaptive parametric families in the Revit family editor (Figure 5). This process made it easier to fit them into the geometric complexity of the HBIM model. The most complex vaults were created using VPL Dynamo software (Revit plug-in). Specifically, our group, with the collaboration of the University of Genova, developed an algorithm able to generate a trough vault with lunettes by choosing the walls of room, the height of the arrow and the number of lunettes. This process is theoretically based on [45].

Some of the artworks housed in the museum were modelled, including paintings, tapestries and furnishing. The frames of the paintings were modelled in Revit’s family editor as a parametrical object in the “furniture” category, whereas the images of the paintings in .jpeg format were included as high-resolution textures. In addition, three showcase types were created and embedded in a single family.

The constructive elements and their materials had been defined from the study of historical written sources [46] which describe properties and location in the Ducal Palace. Information such as name, taxonomy, quarry of origin and geological period for the stone elements was included in the model as BIM data. Due to the absence of flooring description and representation in the previous surveys and/or in literature, they had been captured through the photogrammetry process and inserted in the BIM model as textures.

The comprehensive HBIM model is constructed by the assembly and the combination of all implemented parametrical families (Figure 6 and Figure 7).

3.1.2. Ontology Based Approach

In order to handle the huge amount of graphical data and related information, it was mandatory to create an unambiguous reference system. In this specific case, a single ontology (Figure 8) was developed covering the entire knowledge domain related to the Ducal Palace, from the massive wall structure to the single moulding of the interior decorations, characterized by a shared vocabulary of terminologies.

The result was an accurate taxonomic description (Figure 8) of all architectural elements coinciding with related Revit families. The Level of Detail achieved can be likened overall to a “LOD F” (Executed Object) according to UNI 11337 standard.

3.1.3. Data Enrichment

Our research followed several normative references for the implementation of data enrichment, carefully selecting general national and international requirements for museum management [28]: The Ministerial Decree of 1 May 2001, Annex A, (art. 150, comma 6, del D.L. n. 112 del 1998) [47] regarding details of technical–scientific criteria and standards for the operation and development of muses; the ICOM Code of Ethics [48], which defined requirements for proper working of a museum; the Confidential Facility Report [49] inspired by International best practices; and the D.M. 21 Febbraio 18 [50], that introduced the National Museum System. Based on these norms, we implemented all required parameters useful for the maintenance and improvement of management systems throughout the museum life cycle (see Figure 9).

Each artwork was enriched through the following information:

• Catalogue and/or inventory number;
• Name and author;
• Low-res photo;
• Type of object (painting, chimney, tapestry, furnishings, etc.);
• Geolocation of the work (in the museum or in case of a loan);
• Components (e.g., frame, canvas, painted board, base, etc.);
• Metric and overall data (length, width, volume etc.);
• Microclimatic parameters (admissible range of temperature, illuminance and relative humidity).

The realization of this strategic data enrichment of the HBIM model allowed us to discretize this amount of data into a single schedule, one that is not a simple inventory, but a real detailed catalogue of the artworks, paying special attention to microclimatic parameters. This step is a fundamental one in the digitization of curatorial practices, exploiting the HBIM approach and developing a sustainable data management strategy.

3.2. Focus on “Sala Degli Angeli”

Considering the huge amount of data collected and the size of the entire Piano Nobile (5073 m²), we decided to focus the work on the Hall of Angels of the Duke’s Apartment (Figure 10) in order to further stress the modelling pipeline and developing a reality-based pipeline for HBIM. The hall is a significant example of the Palace, both in terms of the architectural elements present and the collection of artworks that can be visited there. In this case, the main objective was to further implement the HBIM model by equipping it with high-definition 3D models, based on the SfM technique. In fact, here, the chimney, considered a highlight, was not sufficiently detailed via parametric modelling approaches.

Through a photogrammetric survey, the painting “Communion of the Apostles” by Jodochus van Ghent accompanied by predella “The Miracle of the Desecrated Host” by Paolo Uccello and Il Camino Degli Angeli by Domenico Rosselli were digitized, representing the works collected in the hall.

3.2.1. Angeli’s Chimney Photogrammetry

In particular, the photogrammetric survey of the Camino Degli Angeli was carried out by considering the morphology and dimensions of the artifact. The acquisition design included a taking distance of 1 m and a grid of 11 × 8 taking points, 60 cm apart. Nadiral photos were taken for each point of the grid, plus necessary tilted photos from an angle of −30° down toward the basement to +60° toward the vaulted ceiling. The 198 photos obtained were edited and processed in Agisoft’s Metashape software. The point cloud so obtained was scaled and geolocated using the coordinates of homologous points belonging to the room model previously made with TLS techniques in order to have a univocal spatial reference system. The obtained 3D model was further processed in Blender software to complete the parts that could not be easily detected, such as the chimney hood.

3.2.2. Reverse Engineering

Thanks to this 3D sculpture, an additional field of experimentation was opened for the HBIM model: the integration and dialogue between the parametric families and the complex photogrammetric model. The 3D model of the chimney, obtained via SfM, was inserted; this step enables the implementation of fourth LOG, called “as built”, beside the already discussed 3 LODs. The McNeel Rhinoceros plug-in, rhinoInside, was used to import the mesh model of the chimney into Revit. All the steps were made in a common and synchronized Revit–Rhino working environment (Figure 11) created through the VPL Grasshopper software (Rhinoceros plug-in). Unfortunately, during the import phase, a loss of texture-related information was observed due to the undue complexity of the model. With the purpose of solving this problem, the mesh model was segmented into more basic parts in the Rhinoceros dashboard and, through an additional Grasshopper visual script, the new textures were reassigned correctly to the simpler models.

4. HBIM to XR

After the first methodological step, that is, the development of the 4D information system, the research aimed to investigate how these data could be used and enjoyed through portable and wearable devices. In order to pursue this task, a mandatory step is to import the informed models into a cross-platform development environment, Unity 3D, with the aim of generating two outputs (see Figure 12):

• A Mixed Reality (MR) application exploiting the Microsoft Hololens2 device: the application deals with the simulation of some user experiences for museum management and enjoyment based on the semantically aware model, focusing on thermo-hygrometric parameters.
• A Virtual Reality (VR) application for Oculus Quest: the goal was, in this case, to verify the LOG achieved in the HBIM environment and to enable user experiences, such as storytelling and/or gaming interaction.

The open-source Unity 3D Game Engine platform has been used for the lower computational cost required compared to Unreal Engine.

One of the main issues was to import the complete BIM model within the Unity 3D, limiting the loss of information. Interoperability and secure and/or error-free data interchange between BIM and any other working environment are provided by the IFC format, developed by BuildingSMART.

To ensure proper export to the interchange format, it was necessary to associate the internal BIM parameters with the external IFC parameters. The Revit “furniture” family was implemented with the corresponding IFC thermo-hygrometric parameters, regulated by DM 10 May 2001, through the IFC 2019 for Revit plug-in from BuildingSmart. The chimney was also enriched with the same data at the four created LODs.

Once these associations were completed, the BIM model was exported to the IFC format. The geometry was exported with the maximum level of detail, while an export template was defined to maintain only the thermo-hygrometric data useful for the next stage of application development.

The open-source Tridify plug-in, after automated processing, allows us to import the IFC model of the Hall of Angels and read and operate with all the families and data of the HBIM model (Figure 13) within the Unity Inspector, as an intrinsic property of the 3D object, the game object.

4.1. MR Experience

Two Unity plug-ins were used for the development of Mixed Reality applications:

• Vuforia, for the development of MR functionality; specifically, the development kit allows the creation of targets based on different types of sources (Image, 3D Model, Point Cloud);
• MRTK (Mixed Reality ToolKit). Microsoft’s asset required for software operation of the Hololens eyewear.

Initially, our research project aimed to develop an application based on “target area”, using the point cloud of the Hall of Angels as reference data, so the eyeglass would be able to recognize the space from the 3D geometric information of the point cloud. This innovative method would have provided an extended georeferenced basis that ensures a more stable and accurate display of holograms than other types of targets [51]. Vuforia, being a commercial platform, has limitations and constraints, allowing us to work only with point clouds generated by a small number of devices. Unfortunately, having scans generated by a non-compatible laser scanner (Leica P40), this method could not be pursued.

Therefore, the workflow was redefined by adopting a different solution: the idea was to develop an application to simulate a case of museum management based on the “model target” of the Camino Degli Angeli and another to simulate a case of museum enjoyment based on the “image target” of Paolo Uccello’s painting “The Miracle of the Profaned Host”. Both masterpieces are conserved and shown in the Hall of Angels.

In order to develop and test the apps in the lab environment, the painting was printed on paper at a scale of 1:1, whereas, not having a real physical representation of the chimney in the lab, it was replaced with a 1:100 scale 3D print, already available, of the Torrione di Cagli by Francesco di Giorgio Martini. This step was necessary in order to test the functionality of the model target. The choice of a 3D print model was because it enabled matching between a physical object and the model source, as a digital copy of it. The image target was generated in Vuforia’s online platform using the image of the painting (obtained from the photographic survey) in .jpeg extension. Concerning the model target, the corresponding 3D model (obj. format) was loaded into Vuforia’s Model Target Generator, and the “guide views”, representing the guidelines that the algorithm needs to recognize the real object, were scaled and generated. Once the virtual scene was completed, it was exported from Unity 3D and installed via Microsoft Visual Studio 16.11 software.

4.1.1. Management Application Demo

The first app, addressed to technical users, concerns the evaluation of the microclimatic parameters required for proper artwork preservation in an exhibition environment, according to UNI 10829 standard.

The scene proposes a series of panels, which can be displayed once the reference object is framed, allowing the comparison of the hypothetical environmental room’s conditions with the thermo-hygrometric parameters of the object (Figure 14a,b).

4.1.2. User Experience Application

The second app is addressed to visitors of the exhibition hall, and it concerns the narration of the story illustrated in the predella “The Miracle of the Profaned Host”.

Specifically, the application offers a series of panels placed below the painting where the short story inspired by the illustrations is featured. Through buttons, in addition to the simple audio-visual narration, the scene is corroborated by the creation of 3D models which include the architecture, furniture and characters of the painting (Figure 15a,b).

The information panels (of both demos) and the 3D models of the scene are equipped with a selection box that allows them, through freehand gestures, to be scaled or moved freely in space.

4.2. VR Experience

The last relevant point of our research was the creation of a Virtual Reality (VR) App. The development of this app exploits all the potential of the IFC model with LOD G in order to have a complete and realistic representation of the hall. For this purpose, the geometries, including the Chimney of Angels, were successfully imported and visualized. The original materials were reassigned with Tridify’s Material Replacer tool to the various objects in the Unity scene, which remain subdivided and organized according to original families by Revit. To enable interaction with the objects in the scene, they were equipped with “grabber” and “raycast” components found in the Oculus Integration plug-in for Unity. Once the implementation of the scene in Unity was finished, through the “build and run” command, the app was installed directly into the Oculus Quest2 device.

Remote User Experience

The scene interface proposes the same features experienced in the lab through the MR App. Equipped with the Oculus Quest2 device and joystick, the user can move freely in the virtual environment of the Hall of Angels, grab objects and, thanks to a raycaster, press buttons. It is possible to compare environmental data with the thermo-hygrometric parameters of the Chimney of Angels (Section 4.1.1) and watch the narration and 3D reconstruction of the illustrated events of the painting “The Miracle of the Profaned Host” (Section 4.1.2). The perfect portability of the data enrichment in HBIM used for the MR application was here further assessed by changing the interaction and visualization device (Figure 16).

5. Results and Conclusions

The research outcomes provide a comprehensive HBIM to XR workflow that shows how a HBIM model, with suitable levels of information and geometry, enable innovative paths for museum management and user experience, based on Virtual and Mixed reality.

The workflow has been tested by using some of the most widespread and proven software for HBIM, according to the scientific literature [22,26,28,30]. Autodesk Revit is the first software that allowed the implementation of 3D parametrical objects using point cloud as a basis and was already used by our team for previous works at the Ducal Palace of Urbino [38,39,43]. In similar studies [32], authors opted for Revit and Unreal Engine to develop a XR experience, but without directly exploiting BIM data. In [52], Franczuk implemented a virtual point cloud tour using Unreal Engine based on ArchiCAD. Compared to the last two works, our workflow demonstrated high reliability, considering LOG/LOIN portability and the right balance on computational costs.

A major achievement is the assessment of new tools and knowledge for digital curation of content, specifically addressed to museum professionals, through the adoption of an ontology-based semantic modelling of Piano Nobile of Ducal Palace at Urbino.

A HBIM model has been developed at the LOD F level, according to UNI 11337, which responds to the graphic and information requirements of an “as built” construction and is specifically required for preservation and management activities for Building Heritage. Indeed, the high level of geometry (LOG) is provided by the combination of accurate parametrical elements (Figure 17) and reverse engineering acquisitions, and the high level of information need (LOIN) is obtained thanks to massive and multidisciplinary data enrichment.

As the major goal of the present paper was to exploit the HBIM model for VR/MR solutions, it is possible to conclude that the workflow gives rise to fully operative applications already assessed in our Lab.

During some educational visits, the XR applications were tested by students and other researchers. They received very positive feedback, especially the predella’s app, thanks to the high quality of contents such as the 3D reconstruction of the scenes and the narratives provided by a recorded voice. In addition, the possibility to freely move and scale the panels and the holograms provides users with a high level of engagement and suggests pursuing the gamification approach. On the other hand, we observed that HoloLens eyewear has a low usability, especially in the first human–device interaction, due to a less intuitive UI compared to Oculus Quest2 or a common smartphone. For these reasons, we decided to not conduct user studies on museum personnel for the present work. This first phase of observation led us to conclude that the Technology Readiness Level (TRL) of Microsoft Hololens2 must be improved. Although it undoubtedly represents an innovative and promising technology that offers a number of remarkable possibilities for museums’ on-site enjoyment, the tech acceptance of similar devices (see-through glasses) has to be further tested in the domain. On the other hand, Oculus Quest2 is a proven tool with a higher TRL for remote user experience.

After a final evaluation, this approach, despite the significant investment of time, will succeed in the future thanks to the HBIM model’s adaptability for heterogenous purpose. Furthermore, the affordability of these adopted technologies and tools will be higher and, on the other hand, the computational cost will be gradually lower.

6. Future Works

The research presents some issues and future needs for implementation. Due to the complexity and the size of the building, the system was tested in one floor of the Ducal Palace. Despite this strategic choice, the obtained BIM file is quite heavy, and it requires a high computational cost. In the future, management and visualization could possibly be facilitated by dividing the model into linked parts and by producing lower size files.

The lack of a real and open interoperability in software environments and the presence of commercial restrictions currently represents one of the main obstacles. Due to this issue, the present workflow has been affected by a change in strategy as regards the choice of MR targets. In the light of that, our work team will test indoor localisation for Mixed Reality, using point cloud as an area target, to develop more accurate and realistic user experiences for museum management and enjoyment.

Another challenge is the development of a tracking system object to pursue the concept of automation. A more sophisticated application will be implemented, able to perform real time BIM data visualisation according to a user’s query. This requires an in-depth knowledge of programming languages and a higher compatibility between software that enable a synchronized work environment.

Another issue concerns the ICT skills requested for the usage of XR technologies and, besides that, the personnel training. On the one hand, there is a need to evaluate how users respond to these digital experiences. Through some assessment questionnaires, it is possible to examine four experience dimensions [53]: aesthetics, escapism (related to level of immersion), education and entertainment (related to levels of absorption). In particular, we aim to validate our applications in a relevant environment with museum personnel. Plus, we will conduct some evaluations as regards the users’ preferences between XR and VR by measuring the level of engagement for the same contents (CH artefact) and interactions (features available). On the other hand, professionals must embrace technological innovation to pursue museums’ cultural and social mission. After years of lacking policies in the European scenario, a series of reforms and decrees have been provided, and many research projects [54,55], funded by the EU, have faced the need to train for these new museum job profiles.

The general objective is to broaden the knowledge and the comprehension of digitisation phenomenon in order to preserve and enhance Cultural Heritage by merging humanistic and technological perspectives.

The research described in this article has dealt with the creation of an HBIM model, taking into account the principal features of the historical buildings, especially the parametric rules in the Renaissance age architecture, and the main information useful for museum management. Thus, a high complexity model has been created tailoring modelling strategies and reaching a high level of detail (geometric level and information needed level). In relationship with the information management, a new knowledge-based ontology and data enrichment for HBIM environment have been developed, enabling us to incorporate knowledge in the research line of M–BIM approaches. Besides this, different workflows for developing MR and VR experiences based on the HBIM information system have been tested and assessed.