Next Article in Journal
Matching the Ideal Pruning Method with Knowledge Distillation for Optimal Compression
Previous Article in Journal
Towards Unlocking the Hidden Potentials of the Data-Centric AI Paradigm in the Modern Era
Previous Article in Special Issue
Elevating Academic Advising: Natural Language Processing of Student Reviews
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
ASI
Volume 7
Issue 4
10.3390/asi7040055
asi-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Enhanced and Combined Representations in Extended Reality through Creative Industries

by
Eleftherios Anastasovitis
* and
Manos Roumeliotis
Computer and Network Systems Technologies Laboratory, Department of Applied Informatics, University of Macedonia, 156 Egnatia Street, 546 36 Thessaloniki, Greece
*
Author to whom correspondence should be addressed.
Appl. Syst. Innov. 2024, 7(4), 55; https://doi.org/10.3390/asi7040055
Submission received: 18 April 2024 / Revised: 15 June 2024 / Accepted: 21 June 2024 / Published: 26 June 2024
(This article belongs to the Special Issue Advanced Technologies and Methodologies in Education 4.0)
Browse Figures
Versions Notes

Abstract

:
The urgent need for research and study with nondestructive and noninvasive methods and the preservation of cultural heritage led to the development and application of methodologies for the multi-level digitization of cultural elements. Photogrammetry and three-dimensional scanning offer photorealistic and accurate digital representations, while X-rays and computed tomography reveal properties and characteristics of the internal and invisible structure of objects. However, the investigation of and access to these datasets are, in several cases, limited due to the increased computing resources and the special knowledge required for their processing and analysis. The evolution of immersive technologies and the creative industry of video games offers unique user experiences. Game engines are the ideal platform to host the development of easy-to-use applications that combine heterogeneous data while simultaneously integrating immersive and emerging technologies. This article seeks to shed light on how heterogeneous digital representations of 3D imaging and tomography can be harmoniously combined in a virtual space and, through simple interactions, provide holistic knowledge and enhanced experience to end users. This research builds on previous experience concerning the virtual museum for the Antikythera Mechanism and describes a conceptual framework for the design and development of an affordable and easy-to-use display tool for combined representations of heterogeneous datasets in the virtual space. Our solution was validated by 62 users who participated in tests and evaluations. The results show that the proposed methodology met its objectives. Apart from cultural heritage, the specific methodology could be easily extended and adapted for training purposes in a wide field of application, such as in education, health, engineering, industry, and more.

1. Introduction

The digital transformation in every aspect of everyday life is taking place at a rapid pace [1,2,3,4,5,6,7,8]. On the one hand, the rapid technological developments in digitization are leading in this direction. On the other hand, the special conditions that arise in socio-economic life accelerate the whole process [4]. The sector of culture is a typical example that was significantly affected by the recent pandemic [9,10]. At the same time, it is also a sector that has benefited significantly from the digital transition [11,12,13,14]. The traditional ways of digitizing cultural heritage and the testing of innovative methodologies contribute to the documentation, conservation, maintenance, restoration, and better communication of cultural assets [15]. This may concern ancient artifacts, works of contemporary art, archival collections, buildings, archaeological sites, and more [16,17]. Thus, repositories and infrastructures host various collections of digital cultural heritage assets accompanied by their metadata and paradata [18].
Specific types of data can provide specialized information for a range of disciplines. Tomographs provide the ability to study invisible areas that are inside three-dimensional geometries [19]. The use of computed tomography scans (CT scans) in medicine enables the doctor to draw safe conclusions or estimates about a patient’s condition, such as the early diagnosis of a cancerous tumor or the healing process of a broken bone [20]. In cultural heritage, the use of CT scans can help to document the findings holistically without applying deductive methods to investigate their internal structure [21]. It has characteristic fields of application in Egyptology and papyrology [22,23]. In specific cases, analysis of the scans revealed information that shed light and revised our knowledge [24,25] of ancient technology [26,27,28,29,30,31]. Increased computing resources and specialized software and hardware packages are required to analyze and study the tomographs. One of the most important things is the interdisciplinary collaboration between the research groups and the stakeholders when working with CT scans.
In recent years, the development of immersive technologies has been remarkable [32,33]. Their adoption, across a wide spectrum of socio-economic life, is expected to increase steadily, especially as we enter the age of virtual worlds and metaverses [34,35,36,37,38,39]. Extended reality (XR) is characterized by augmented reality (AR), virtual reality (VR), and mixed reality (MR). Extended reality offers the possibility to develop innovative products and useful applications that were unimaginable just a few years ago [14,40]. The fact that extended reality applications can be developed on popular game engines also contributed to their widespread use [41]. The characteristic of game engines is that they can incorporate heterogeneous immersive technologies but, at the same time, benefit from emerging technologies such as the broad field of artificial intelligence or haptic technologies [42,43].
The key component of extended reality applications is their content. Primarily, it concerns 3D geometries that are overlaid with material, with a photorealistic texture, and placed in virtual space to simulate an environment. The creation of extended reality content is based on automated digitization methods, semi-automated processes, manual methodologies, or their combination [15,44,45,46,47,48,49]. Often, the assistance of artificial intelligence (AI) is used to speed up the process or to optimize the result [50]. However, a critical factor for the smooth execution of extended reality applications is the degree of complexity of the geometry of the objects and, specifically, the number of polygons of which they are composed [51,52,53,54]. In several cases, additional optimizations are required to simplify the geometry, targeting two factors. Through the decimation of the 3D mesh, we try to ensure a reasonable balance between the quality of the representation result and the complexity of the geometry, depending on the target platform on which the XR application is to be executed [55].
This article proposes a methodology for designing and developing meaningful and multilayered representations of cultural heritage. Through the combination of decimated three-dimensional meshes and the reuse of their relevant representations from computed tomography scans, the suggested methodology considers the advantages of the game engines and the immersive technologies for the integration of CT scan images in immersive virtual environments. The combined representations could be applied and used in various domains, such as health and engineering, apart from cultural heritage, for training purposes and better understanding. Section 2 presents the background of this research. Section 3 depicts the methodology that was followed and analyzes the combination of the decimated three-dimensional meshes and the reuse of their CT scans. Section 4 presents the results for the end users of the virtual display and provides a discussion space for different aspects and challenges of this approach. Section 5 concludes with highlights of the research in advanced representations in extended reality through creative industries.

2. Related Work

2.1. Computed Tomography Imaging

Cultural heritage is a field that has benefited significantly from the use and study of tomographies related to elements of archeology and cultural heritage in general [56]. The ability to provide important information through nondestructive and noninvasive methods on non-visible areas of cultural heritage objects contributes to holistic study and documentation [19,27]. In a recent study, a sample consisting of 96 CT scans of skulls from contemporary Egyptians and 54 3D models of skulls from the Egyptian Old Kingdom Period (2700–2180 BC) were used in the sector of bioarchaeology to validate the reliability of sex classification for geographically and temporally distant skull series [57]. In a different study, Takabuti’s mummy was re-examined to evidence the scope of combining a computerized examination of CT imaging and an image intensifier-guided needle biopsy (MIP), coupled with analytical technologies and existing Egyptological data, to extend previous understanding of her background, life events, and preparation for the afterlife [58].
Another study provides a rigorous evaluation of the practice of producing 3D-printed replicas of animal anatomy from archaeological remains and CT scans and has shown the utility of the practice for species identification [21,59]. Virtual unwrapping allows textual artifacts to be read completely and noninvasively, and it was applied in the case of the extremely fragile En-Gedi scroll, the oldest Pentateuchal scroll in Hebrew outside of the Dead Sea Scrolls [60]. Moreover, X-ray tomography for virtual unfolding and nondestructive access to hidden texts was applied in ancient papyrus packages from the papyrus collection of the Musée du Louvre [23]. EduceLab-Scrolls combines volumetric CT scans with aligned 2D labels. Together with a software pipeline that combines machine learning (ML) with a novel geometric framework, the interconnected data and method can detect the carbon ink inside Herculaneum scrolls [61].
Advances in 3D graphics and the capabilities offered by sophisticated processors have greatly aided the development of software packages for the study and processing of X-ray CT scans. A wide range of available software packages are available for the analysis of tomographic datasets, whether they are commercial products [62,63,64,65,66,67,68] or free [69,70,71,72,73,74] software (Table 1). Different types of tomography can be imported into the specific programs, and multi-level analyses can be performed on the volumetric representations. Users are also given the ability to export 3D digital representations of joined areas or the entire object in ideal formats for further use in 3D editing programs [75]. In several cases, it is also possible to produce videos of predefined processes through the volumetric analysis program, such as for the internal overview of the geometry of an object along its axes. However, in most cases, it is necessary to use computer systems equipped with powerful graphics cards. At the same time, the handling of volumetric analysis programs requires familiarity with and basic knowledge of 3D graphics. Thus, the data related to computed tomography scans remain in limited access behind the doors of laboratories.

2.2. Three-Dimensional Graphics

Digitization offers the possibility to capture and store the digital instance of an entity at a specific moment in time. Digitization applies to two-dimensional (2D) and 3D objects of any size, whether they are simple objects, excavation findings, artifacts, buildings, open spaces, or landscapes [76]. Digitization is achieved through different technologies and is applied on a case-by-case basis. Photogrammetry is an easy-to-use and economical way to digitize an object using affordable means, such as a smartphone camera or the integration of a high-resolution camera into an unmanned vehicle [77,78]. As a next step, by choosing an image processing software package, we can easily compose the 3D digital representation of the object and export it to suitable file formats for further use. Accordingly, an advanced digitization method is 3D scanning via special equipment, offering enhanced possibilities and improved quality to the result [76,79]. However, digitization is no longer limited to tangible objects and to intangible concepts and attributes, such as olfaction.
Mesh decimation is a useful procedure in the domain of graphics that offers flexible representations of surfaces and objects for further use in advanced applications [51,80,81]. Through a variety of algorithms, the mesh decimation produces simplified and lighter models in terms of their number of polygons [53,82,83,84,85,86,87,88]. However, users who wish to apply the decimation methodology to their cases do not need to have in-depth knowledge of the algorithms. This is achieved by incorporating the decimation methodology into almost all software packages for processing 3D graphics. The users have to import their 3D model, parameterize the algorithm through user-friendly graphical interfaces, and wait for the export of the final result. Users can specify the degree to which they wish to reduce the number of polygons in the overall geometry [54]. Furthermore, this procedure can be applied in multiple steps or test decimation scenarios with a gradual reduction rate.

2.3. Cutting-Edge Technologies

The degree of immersion offered by XR technologies is determined by the user’s level of isolation in the extended worlds, as well as the number of senses with which the user participates in those experiences [14,40]. In AR, the user’s simultaneous interaction with the real world and augmentation elements through their portable device has a limited degree of immersion. However, AR solutions are usually the most affordable and readily accessible due to the equipment, as displayed in [89]. In MR, the degree of immersion is greatly enhanced as the user interacts with the elements of the augmented space through the head-mounted display and gestures. The cost of acquiring MR equipment has, until recently, been a factor limiting the spread of their use. However, in recent years, significant progress has been made in incorporating mixed reality elements into devices that provide highly immersive experiences in VR environments [90,91,92,93]. The current trend seems to lead to the development of XR devices that will include haptics and special devices for reproducing smells. In other words, multi-sensory XR devices will lead to the pinnacle of immersion [94]. However, the ease of use and the usefulness of the applications to be developed should be a high priority.
Artificial intelligence is treated with caution by many and offers additional possibilities in everyday life and the field of research. In the sector of cultural heritage, leveraging AI, machine learning (ML), and advanced algorithms could be used for digitization, reuse, and knowledge enhancement [95,96,97]. AI algorithms, including deep learning models like convolutional neural networks (CNNs) and recurrent neural networks (RNNs), could be applied for automated artifact recognition, textual analysis, and contextual understanding [95,98]. In other words, these technologies enable the efficient and accurate categorization, indexing, and annotation of digital repositories, facilitating easy access and reuse [99]. In addition, ML algorithms contribute to data mining, trend analysis, and predictive modeling, empowering heritage professionals to derive valuable insights and trends from extensive datasets [100]. Integrating AI with 3D scanning and reconstruction techniques allows for the generation and preservation of artifacts in digital spaces [101]. Moreover, AI is used to provide effective training and upskilling for users. Explainable AI (XAI) is suitable for providing reasonable explanations to non-technical users [102].

2.4. Game Engines

The link that unites the aforementioned technologies is game engines. Game engines are used as platforms that can orchestrate the integration of heterogeneous technologies and the development of universal applications that can be used on a wide variety of target devices [94]. Game engines host and process 2D and 3D graphics to develop virtual environments [103]. They integrate heterogeneous equipment as input and output units to develop interactions between the system and the user. They provide advanced simulation capabilities of natural phenomena and realistic mechanisms, giving users the feeling that the virtual world behaves according to the laws of physics. Additionally, artificial intelligence can be applied, layered through game engines, with a significant impact on their system and application usability. There are many game engines available that can meet almost all of the requirements and needs of development teams [104].

2.5. Contribution of This Research Work

This research proposes a methodology to enhance the democratization of heterogeneous datasets and their derivatives, such as 3D meshes and CT scan images, through combined representations in interactive and immersive virtual environments. Several research questions (RQs) need to be addressed in this effort.
RQ1: How could CT scans be repurposed and reused for widespread use by the general public?
RQ2: How could heterogeneous representations be combined and reused to better understand concepts and artifacts?
RQ3: How could immersive technologies and the creative industry of games combine to develop affordable and easy-to-use digital tools for advanced artifact representation?
RQ4: How might emerging technologies contribute to this endeavor, and what ethical dilemmas are raised?

3. Materials and Methods

This section describes the material and the methodology that was followed in our effort to design and develop the combined easy-to-use and affordable representations for the fragments of the Antikythera Mechanism in a fully immersive virtual space. Specifically, in the first subsection, the overall methodology (Figure 1) and the concept for multimodal representations in XR are depicted, while in the second subsection, the implementation of the interactive X-ray viewer full immersive virtual space is described in detail.

3.1. Methodology

3.1.1. Research in Graphics and Immersive Technologies

The research was based on the study and processing of the available datasets related to the Antikythera Mechanism that were kindly provided by the Hellenic Ministry of Culture through Prof. Moussas. The data refer to high-resolution images and X-ray CT scans of fragments related to the Antikythera Mechanism. In the first stage, we investigated the available software packages for the volumetric study of tomographies and used two of them extensively [105]. In the second stage, we extracted the 3D meshes and proceeded to decimate the fragments, as well as the application of textures and materials for the usability and photorealism of the digital representations for further use (RQ1). In the third step, we extracted several instances from inside of the fragments of the Antikythera Mechanism along a specific axis of their 3D geometry, using the CT scans dataset and the combination of two specific software packages (RQ1), namely Drishti [70] and VGSTUDIO [68].
The analysis of the challenges and the requirements of the immersive technologies was our next step. In particular, we studied and compared the possibilities offered by augmented and virtual reality regarding the integration of heterogeneous representations in a common digital space. We also put a lot of emphasis on the interactivity that each of the immersive technologies offers. In addition, we explored the possibilities of game engines regarding the integration of heterogeneous elements and their easy handling that could offer meaningful experiences [106] (RQ2).

3.1.2. Development and Assessment of an Experimental Virtual Museum

We designed an experimental virtual museum and developed it using the Unity game engine. The virtual museum was dedicated to the Antikythera Mechanism, and it hosted the fragments of the artifact. The 3D digital representations were artificially reconstructed by using the CT-scans dataset. In addition, we designed, developed, and integrated the functionality of the X-ray viewer [107] (RQ2). By using the virtual reality head-mounted display with its touch controllers, the users can interact with the 3D representations, as well as with the internal structure of the fragments of the Antikythera Mechanism [54] (RQ3).
As part of the evaluation of the virtual museum for the Antikythera Mechanism, in 2019, we conducted a series of pilot tests with end users from different backgrounds. Specifically, in the context of four pilot tests, 135 users experienced the virtual reality application on an Oculus Rift S device with second-generation controls. Voluntarily, 62 of our participants (female: 42, male: 20) provided anonymous ratings on specific factors through a survey of 25 questions. The factors evaluated were primarily usability, usefulness, and user satisfaction [49]. Regarding the integrated functionality of the CT scans in the virtual reality application, we included two questions related to its direct evaluation (RQ1). Simultaneously, the effectiveness of the proposed combined representation was also evaluated indirectly with specific questions that related to the wider user experience (RQ2). A further analysis of the results of the integrated functionality is presented for the first time in the current research (RQ3).

3.1.3. Concept for Multimodal Representations in Extended Reality

The X-ray functionality in the virtual museum for the Antikythera Mechanism offers an additional modality for the representation of the seven largest fragments of the artifact. On the one hand, the 3D digital representations, which are placed as exhibits on special showcases at the virtual exhibition, give information about the external appearance of the fragments. On the other hand, the X-ray viewer offers an opportunity for users to understand the internal structure of the key parts of the ancient mechanism (RQ3). The common starting point for the multimodal representation is the processing of the Antikythera Mechanism X-ray CT scans (RQ1).

3.2. Implementation

3.2.1. Three-Dimensional Mesh Decimation and Texturing

We designed and applied a pipeline for the artificial reconstruction of the fragments of the Antikythera Mechanism, using the CT-scans dataset and a couple of images in a high level of detail for each one of the fragments [54]. The result of this process was the production of digital 3D representations for the seven largest fragments (A–G in Table 2) of the Antikythera Mechanism (RQ1).
In our effort to achieve an easy-to-use digital application, we should ensure that the elements we would integrate into the virtual space would be easily manipulated. Specifically, both the geometries of the associated objects and the complex 3D digital representations of the fragments should be simplified. Therefore, we applied the 3D mesh decimation of each fragment, applying the saving of the number of surfaces by 95% of their original number in two steps. Specifically, in the first step, we decimated the initial geometries by 90%, while in the second step, we applied the savings by 50%. Through this way, from the initial 71,477,933 polygons, we were led to a total number of 3,451,031 for the selected fragments [54] (RQ1). The validation of the quality of the final digital 3D representations was ensured during the testing of the virtual reality application by an interdisciplinary team that was involved in the study of the Antikythera Mechanism.

3.2.2. Instances from CT-Scans

Generating the Instances

In our attempt to democratize tomography through immersion technologies, we relied on the possibility offered by specialized software for the analysis of volumetric data. Specifically, we took advantage of the possibility of extracting a finite number of snapshots of the internal structure of the seven major fragments (A to G) of the Antikythera Mechanism. For each of the artifact fragments, starting from one edge of the bounding box, we selected the number (N = 100) of snapshots that we deemed necessary to provide end users with a complete picture of the internal structure of the fragments. The software, traversing the volumetric representation along a selected axis, produced the snapshots in image form (in .png file format with opacity, except fragment D) until they reached the edge of the bounding box (RQ1). The resolution of the images was set to full high definition (1920 × 1080 pixels, apart from fragment D) (Table 2). Figure 2 depicts indicative instances for each fragment (A to G).
We investigated and tested two possible scenarios for the integration of the information from the tomographies in the virtual museum, namely (a) switching of the snapshots on a static surface and (b) switching of the snapshots on a moving surface. In the virtual museum for the Antikythera Mechanism, the first selection was applied (RQ2).

Display on a Static Surface

In the case of switching the sequence of snapshots (RQ1) to a static surface, we use a surface in the game engine that is permanently located at a fixed point. The content of the fixed surface is determined by the dynamic rotation of the content, which consists of the serial stack of images. The sense of perspective regarding the internal structure of the fragment results from the illusion that will be created for the user by the alternating images. Essentially, we rely on the basic principle of the art of animation to communicate the information of the scans in an easy-to-use and understandable way (RQ2).

Display on a Moving Surface

In the case of switching the sequence of snapshots on a moving surface, we use a surface in the game engine that is permanently in motion. The moving surface is moved along the axis on which the snapshots were taken, following the same displacement step. The content of the moving surface is determined by the dynamic rotation of the content, which consists of the serial stack of images. However, each distinct position on the surface corresponds to a single instance. The sense of perspective regarding the internal structure of the fragment is greatly enhanced by the simultaneous displacement of the viewing surface. It could be described as a movable projection screen.

3.2.3. Multimodal Interaction in Virtual Reality

The interactions developed for the user’s experience with the information produced by the CT scans were based on the user interface (UI) graphical element of the panel offered by the Unity game engine and on the options of the controllers of the Oculus Rift S virtual reality device (RQ3). Primarily, users could interact with the 3D representations of the artifacts physically, as they could pick them up and hold each fragment with their virtual hands through the touch controllers (RQ2). Regarding the interactions with the inner structure of the fragments (RQ2), two functionalities were developed: (a) the discrete linear exploration and (b) the fast linear exploration. Both functionalities were active when the user was standing in the predefined area of a specific fragment, which was marked through a triggered collider (Figure 3a,b). The pipeline (RQ3) of multimodal interactions in the virtual museum of the Antikythera Mechanism is depicted in the flowchart (Figure 4).

Discrete Linear Exploration

When selecting discrete linear exploration, the user can use the A and B buttons to study the interior of the fragments. Snapshots are stored and sorted in a table. Each time the user presses the A key, the function OnClickNext is executed, so the display surface shows the next snapshot. Conversely, whenever the user selects the B button, the function OnClickPrevious is executed, so the display surface shows the previous snapshot. For each fragment, the range in which the user can interact is from 0 to 100 (see Algorithm 1). Through this interaction, the study of the internal structure is made instance by instance (RQ1, RQ3).
Algorithm 1. Script in C# for discrete linear exploration on the fragment A of the Antikythera Mechanism.
public void OnClickNext(string nameImage)
{
if (imgNumber < 100)
   {
   imgNumber++;
   nameImage = imgNumber.ToString ();
   imageXrayA.sprite = Resources.Load<Sprite> (“FragmentA/x_fragment_A_” + nameImage) as Sprite;
   }
}
public void OnClickPrevious(string nameImage)
   {
   if (imgNumber > 0)
      {
      imgNumber--;
      nameImage = imgNumber.ToString();
      imageXrayA.sprite = Resources.Load<Sprite> (“FragmentA/x_fragment_A_” + nameImage) as Sprite;
   }
}

Fast Linear Exploration

When selecting fast linear exploration, the user can use the Fire 1 and Fire 2 triggers to study the interior of the fragments. Each time the user presses the SecondaryIndex trigger, a sequence of consecutive snapshots is displayed on the display surface in ascending order as a natural stream. Conversely, whenever the user selects the SecondaryHand trigger, a sequence of consecutive snapshots in descending order is displayed on the display surface as a natural stream. As long as one of the triggers is pressed, the user is given the illusion that the fragment comes to life (RQ1, RQ3). These interactions are controlled by the Update function in Unity (see Algorithm 2).
Algorithm 2. Script in C# for fast linear exploration on fragment A of the Antikythera Mechanism.
void Update ()
{
string nameImage;
if((Input.GetButtonDown(“Fire1”)) ||
(OVRInput.Get(OVRInput.Button.SecondaryIndexTrigger)))
   {
   if (imgNumber < 100) {
      imgNumber++;
      nameImage = imgNumber.ToString ();
      imageXrayA.sprite = Resources.Load<Sprite> (“FragmentA/x_fragment_A_” + nameImage) as Sprite;
   }
}
if((Input.GetButtonDown(“Fire2”)) ||
(OVRInput.Get(OVRInput.Button.SecondaryHandTrigger)))
   {
   if (imgNumber > 1) {
      Debug.Log (imgNumber);
      imgNumber--;
      nameImage = imgNumber.ToString();
      imageXrayA.sprite = Resources.Load<Sprite> (“FragmentA/x_fragment_A_” + nameImage) as Sprite;
         }
   }
}

3.2.4. Advanced Interaction

The proposed solution was developed exclusively for a virtual reality application. However, with proper customization, it could easily be adapted for other forms of extended reality as well (RQ3). Specifically, in the context of augmented reality, the activation of the multimodal representation of the fragments of the Antikythera Mechanism could be triggered by recognizing the corresponding fragment in the physical exhibition area of the museum. Through the camera of the mobile device, whether it is a smartphone or a tablet, the physical fragment would be identified, with the corresponding collection of instances of its CT scans. By interacting with a user interface element, such as a slider, the users could augment different regions of the fragment’s internal structure on their device’s screens in real time (RQ3). Accordingly, through a mixed reality device, an archeology student could be trained in heritage documentation based on a multimodal representation of an artifact or objects at risk (RQ3).

4. Results and Discussion

4.1. Results

Among the questions of the survey [49], for the assessment of the virtual museum for the Antikythera Mechanism, question Q15 explored the extent to which the integration of the specific functionality for the combined representation of the fragments was informative. Accordingly, question Q18 investigated the degree of ease of use when handling the X-ray function of studying the interior of fragments. Using a Likert scale from 1 (not informative at all) to 4 (very informative), we received feedback from the end users regarding the usefulness of the X-ray functionality. According to the results, 90.3% (N = 56) of the users expressed their positive opinion toward the combined representations, while 9.7% (N = 6) expressed their doubts (RQ1, RQ2, RQ3). Similarly, we received users’ feedback regarding the usability of the proposed functionality. According to the results, 83.9% (N = 52) of the users found the functionality comfortable to use, while 16.1% (N = 10) expressed their doubts [49] (RQ1, RQ2, RQ3).
In the present research, we analyze further the negative feedback in the specific questions. However, it should be noted that of the total participants, 19.4% (N = 12) had no previous experience with a virtual reality application, while 12.9% (N = 8) had no previous experience with video games.
We further analyzed the profile of the participants who expressed a negative opinion in question Q15 (N = 6). Five of them mentioned that they had never played any computer game before their testing in the virtual museum for the Antikythera Mechanism, while the sixth participant used to play once per year (Q5). Moreover, regarding their previous experience in virtual reality applications, two of them had no experience at all, while one participant had at least one experience in the last five years, and three of them had tested at least one VR application in the last year (Q7).
Furthermore, we analyzed the profile of the participants who expressed a negative opinion in question Q18 (N = 10, female: 9, male: 1) regarding the ease of use when handling the X-ray. Three of them had no previous experience in computer games, while five of them had played at least one in the previous year, and only one had played in the last month (Q5). In addition, two of them had no previous experience in VR applications. Two of them had at least one experience in the last five years. Five participants had at least one VR experience during the last year, while one had tested a VR application during the previous month (Q7).

4.2. Discussion

The proposed methodology for the multimodal representation of objects, combining the three-dimensional geometry of the shell with the internal structure, enhances the democratization of the use of tomography in a wide range of applications. Thus, CT scans could be established as a basic form of object representation that offers additional and more complete information about an object [49]. CT scans could be stored in repositories for easy access and reuse in applications corresponding to the proposed one, applying the usage policies defined by their owners. For example, a 3D digitized artifact stored in Europeana could be reused [18], together with its CT scans, in applications similar to that of the proposed methodology, as well as more advanced ones. In this way, this form of representation will be able to be used by a wider range of users outside the confines of laboratories for more practical reasons, such as better communication and understanding of complex objects or artifacts at risk.

4.2.1. Training Purposes

The proposed solution could be a useful interpretive tool in the educational process (RQ3). In specific specialties, the study of tomography is a basic means of providing information. Integrating digital multimodal applications into the education of future professionals is likely to better prepare them as scientists (RQ3). For example, in medicine, a diagnosis can be based on and validated according to the correct interpretation of a patient’s CT scans [20]. The application of the proposed digital solution in the education of medical students could simulate the process of diagnosis of diseases based on tomography (RQ3). Accordingly, in archaeology, the analysis of CT scans of burial remains [22] or excavation finds at risk through friendly and easy-to-use digital tools for an audience unfamiliar with volumetric graphics technologies could provide useful directions on how to preserve and manage a rare and fragile object. The same digital multimodal application could be used to train students in archaeology (RQ3).
Industry is a field in which the multimodal representation of 3D objects through the proposed digital solution could be widely applied [32]. On the one hand, most machines are usually three-dimensional and, in several cases, have a complex internal structure. On the other hand, there is a constant need for initial training of workers in the industry to update their knowledge in new production processes or for training in the use of a new industrial system. In some cases, training can be undertaken by demonstrating the machine’s functionalities, while in other cases, this is impossible or dangerous to implement [33]. Therefore, easy-to-use digital applications based on the widespread immersive technologies of extended reality can holistically represent a machine, both for its external form and internal structure and function, offering safe and effective training to its operators (RQ3).

4.2.2. Advanced Cultural Heritage Representation

The proposed representation could be applied during visits to physical cultural heritage spaces (RQ3). As this solution offers the possibility to observe the non-visible areas of the exhibits, such as their internal structure hidden behind their external surface, it could be used extensively in museums. Exhibits that are in advanced deterioration because of time and environment can be made accessible to the public by providing information about their internal structure through the combined digital representation. Accordingly, complex artifacts that are exhibited in glass cases, where any physical interaction with visitors is forbidden, can communicate with the public through this approach. Also, the case of exhibits related to burial remains is a category of cultural heritage objects that could benefit from the complex representation of the proposed methodology.
Metaverses and virtual worlds are digital spaces that can adopt this methodology, as they rely on game engines, 3D graphics, and immersive technologies. The proposed solution could be applied during the virtual visit of the user to cultural heritage sites or when studying complex representations in virtual laboratories (RQ3). In addition (RQ4), the solution could be optimized and extended by incorporating explainable artificial intelligence to provide additional real-time information related to the objects [102], as well as to assist with interaction usage. Furthermore, an extension that could be added to the proposed solution would be the possible integration of haptic devices for enhanced users’ immersion when interacting with the combinatorial representation. In this way, they could feel the texture of the individual elements through haptic interactions, while at the same time, they would also strengthen the inclusion of people with impaired vision [94,108].

4.2.3. Ethics

However, some ethical questions arise (RQ4) regarding the management and exploitation of digital solutions that directly or indirectly use representations based on CT scans.

Ownership

A key question that arises concerns the ownership of the CT scans and the generated representations. The answer is not simple, as the process involves several groups of people from different disciplines who justifiably have a stake in the property. The complexity of the situation is better understood by using the example of an archaeological find. Specifically, the team of archaeologists is excavating a complex artifact that needs further investigation into its internal structure for better documentation. The cultural organization that hosts or undertakes its temporary conservation needs the specialized services of a group of experts who will undertake the process of producing the tomography records. A team of experts will likely need to analyze the scans with special software and high-end computing systems, producing datasets that are more understandable and usable for culture professionals. Finally, a creative team of programmers will undertake the development of an interpretive tool for the multimodal representation of the artifact based on different types of data, including CT scans. Therefore, the precise ownership framework for each phase of the life of the data and its derivatives should be explored in depth, prioritizing the principles of open access to research.

Use of Data

The second critical point that raises extensive discussion is the use of CT scans, indirectly or directly, especially in cases where the finding to be studied concerns human remains, whether it is an archaeological excavation or a patient in the case of medicine. The ethical question that arises is the way to use information and its representations. This debate is ongoing, as it has already developed around the use of digitized 3D representations of bones and skeletons in the field of archaeoanthropology. Therefore, the parameter that needs to be added here is the ethical way of handling and using the scans and their derivatives related to humans. Extending to the field of medicine and health, the discussion should center on ensuring the privacy and protection of sensitive personal data of patients, as well as the ethical use of data for research and educational purposes.

Authenticity

An equally important ethical question that concerns all digitized representations is the authenticity of the generated products to the originals. This debate has developed more in the field of culture, where the transition of cultural heritage from the physical to the digital space takes place. Therefore, the discussion should be extended to the digital representations and resulting digital products based on the CT scans. In particular, the parameters that could ensure that the result of the digital representation approximates the primary physical reference object should be investigated. Therefore, distinct boundaries should be set at which a digital representation of an object is acceptable so that it can be identified with the original. Certainly, enhancing interdisciplinarity will make a significant contribution to ensuring the authenticity of representations and replicas.

4.3. Limitations

The research could have benefited from advances in artificial intelligence and machine learning. For example, AI could integrate algorithms directly into the game engine to automatically generate the geometry of the interior regions of the Antikythera Mechanism fragments based on the volumetric data. However, the absence of available datasets to train suitable algorithms to automatically fill in geometries or missing regions prevented us from using artificial intelligence and machine learning. Even though the proposed methodology can be applied to the whole range of extended reality, in the context of the research, it was developed, evaluated, and validated in a virtual reality environment. It would be of great interest to assess it through augmented reality or mixed reality in the physical museum space where the Antikythera Mechanism is exhibited by museum visitors. Accordingly, extended reality applications that would incorporate the methodology for training purposes would be useful to be evaluated in the context of a real training program by the medical students themselves.

5. Conclusions

This research presented a methodology for the effective use of heterogeneous digital representations for the holistic understanding and communication of cultural heritage objects. This research, combined in a virtual reality environment, uses 3D digital representations of the fragments of the Antikythera Mechanism and snapshots from its interior through a digital tool for their multimodal and interactive representation for the general public. The proposed methodology was based on virtual reality technologies and 3D graphics, which can be empowered by emerging technologies and applied to the whole of extended reality in a large field of everyday life as an advanced representational, interpretive, educational, and affordable medium. The results confirmed the effectiveness of the suggested methodology and the research questions. Limited doubts and concerns were expressed only by a small group of participants who had limited or no experience with immersive technologies and/or video games. Moreover, through the research, new challenges for the future have emerged.
As for future work, the proposed methodology could be improved by the advantages of artificial intelligence. AI and computer vision advancements are synergizing with immersive technologies by enabling realistic interactions with cultural heritage artifacts. AI algorithms, such as classification algorithms, could be utilized to classify the artifacts based on certain characteristics. In addition, regression algorithms can be used to predict missing or incomplete information about cultural heritage assets from vast and diverse datasets, allowing for the dynamic generation of historically accurate environments and artifacts within the virtual realm. Moreover, deep learning techniques in AI can be utilized for shape completion on 3D range scans, as well as on tomographies. Furthermore, the CT scans dataset could be imported directly into the game engines and offer new challenges to the developers of the extended reality applications for even more advanced functionalities to the end users. In addition, regarding the 3D mesh decimation, different scenarios for the applied intermediate steps in a variety of CT scan datasets could be tested. In this way, useful suggestions could emerge for the optimization of the decimation process.

Author Contributions

Conceptualization, E.A. and M.R.; methodology, E.A. and M.R.; software, E.A.; validation, E.A. and M.R.; investigation, E.A.; resources, E.A. and M.R.; data curation, E.A.; writing—original draft preparation, E.A.; writing—review and editing, M.R.; visualization, E.A.; supervision, M.R.; project administration, E.A.; funding acquisition, E.A. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The original contributions presented in this study are included in this article; further inquiries can be directed to the corresponding author.

Acknowledgments

The authors would like to thank John Hugh Seiradakis, Magdalini Anastasiou, and Xenophon Moussas for their advice and guidance. The authors would like to thank the Hellenic Ministry of Culture for providing access to any dataset of the Antikythera Mechanism. The authors would like to thank Nikon Metrology (now Nikon X-Tek Systems Ltd.), as its equipment was kindly loaned and was used to collect the specific X-ray data acquired during an inspection of the Antikythera Mechanism in October 2005 at the National Archaeological Museum in Athens, Greece.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Alnasser, N.S.; Yi, L.J. Strategies applied by different arts and cultural organizations for their audience development: A comparative review. Heliyon 2023, 9, e15835. [Google Scholar] [CrossRef] [PubMed]
  2. Chen, X.; Zou, D.; Xie, H.; Wang, F.L. Technology-enhanced higher education: Text mining and bibliometrics. Heliyon 2024, 10, e25776. [Google Scholar] [CrossRef] [PubMed]
  3. Brozovsky, J.; Labonnote, N.; Vigren, O. Digital technologies in architecture, engineering, and construction. Autom. Constr. 2024, 158, 105212. [Google Scholar] [CrossRef]
  4. Wu, J.; Qu, X.; Sheng, L.; Chu, W. Uncovering the dynamics of enterprises digital transformation research: A comparative review on literature before and after the COVID-19 pandemic. Heliyon 2024, 10, e26986. [Google Scholar] [CrossRef] [PubMed]
  5. Yin, Y.; Zheng, P.; Li, C.; Wang, L. A state-of-the-art survey on Augmented Reality-assisted Digital Twin for futuristic human-centric industry transformation. Robot. Comput.-Integr. Manuf. 2023, 81, 102515. [Google Scholar] [CrossRef]
  6. Rowan, N.J. Digital technologies to unlock safe and sustainable opportunities for medical device and healthcare sectors with a focus on the combined use of digital twin and extended reality applications: A review. Sci. Total Environ. 2024, 926, 171672. [Google Scholar] [CrossRef] [PubMed]
  7. Myshko, A.; Checchinato, F.; Colapinto, C.; Finotto, V.; Mauracher, C. Towards twin transition in the agri-food sector? Framing the current debate on sustainability and digitalisation. J. Clean. Prod. 2024, 452, 142063. [Google Scholar] [CrossRef]
  8. Liberty, J.T.; Sun, S.; Kucha, C.; Adedeji, A.A.; Agidi, G.; Ngadi, M.O. Augmented reality for food quality assessment: Bridging the physical and digital worlds. J. Food Eng. 2024, 367, 111893. [Google Scholar] [CrossRef]
  9. Liu, Z.; Orr, S.A.; Kumar, P.; Grau-Bove, J. Measuring the impact of COVID-19 on heritage sites in the UK using social media data. Humanit. Soc. Sci. Commun. 2023, 10, 537. [Google Scholar] [CrossRef]
  10. Parker, M.; Spennemann, D.H.; Bond, J. The Effect of the COVID-19 Pandemic on Heritage Festival Soundscapes—A Critical Review of Literature. Curr. Pollut. Rep. 2024, 10, 277–285. [Google Scholar] [CrossRef]
  11. Ryder, B.; Zhang, T.; Hua, N. The Social Media “Magic”: Virtually Engaging Visitors during COVID-19 Temporary Closures. Adm. Sci. 2021, 11, 53. [Google Scholar] [CrossRef]
  12. YiFei, L.; Othman, M.K. Investigating the behavioural intentions of museum visitors towards VR: A systematic literature review. Comput. Hum. Behav. 2024, 155, 108167. [Google Scholar] [CrossRef]
  13. Povroznik, N. Museums’ digital identity: Key components. Internet Hist. 2024, 8, 1–16. [Google Scholar] [CrossRef]
  14. Rodriguez-Garcia, B.; Guillen-Sanz, H.; Checa, D.; Bustillo, A. A systematic review of virtual 3D reconstructions of Cultural Heritage in immersive Virtual Reality. Multimed. Tools Appl. 2024, 1–51. [Google Scholar] [CrossRef]
  15. Salleh, S.Z.; Bushroa, A.R. Bibliometric and content analysis on publications in digitization technology implementation in cultural heritage for recent five years (2016–2021). Digit. Appl. Archaeol. Cult. Herit. 2022, 25, e00225. [Google Scholar] [CrossRef]
  16. Perino, M.; Pronti, L.; Moffa, C.; Rosellini, M.; Felici, A.C. New Frontiers in the Digital Restoration of Hidden Texts in Manuscripts: A Review of the Technical Approaches. Heritage 2024, 7, 683–696. [Google Scholar] [CrossRef]
  17. Lovell, L.J.; Davies, R.J.; Hunt, D.V.L. The Application of Historic Building Information Modelling (HBIM) to Cultural Heritage: A Review. Heritage 2023, 6, 6691–6717. [Google Scholar] [CrossRef]
  18. Europeana Collections. Available online: https://www.europeana.eu/en/collections (accessed on 6 April 2024).
  19. Giovannetti, G.; Guerrini, A.; Minozzi, S.; Panetta, D.; Salvadori, P.A. Computer tomography and magnetic resonance for multimodal imaging of fossils and mummies. Magn. Reson. Imaging 2022, 94, 7–17. [Google Scholar] [CrossRef] [PubMed]
  20. Lam, S.; Bai, C.; Baldwin, D.R.; Chen, Y.; Connolly, C.; de Koning, H.; Heuvelmans, M.A.; Hu, P.; Kazerooni, E.A.; Lancaster, H.L.; et al. Current and Future Perspectives on Computed Tomography Screening for Lung Cancer: A Roadmap From 2023 to 2027 From the International Association for the Study of Lung Cancer. J. Thorac. Oncol. 2024, 19, 36–51. [Google Scholar] [CrossRef]
  21. Bibb, R.; McKnight, L. Identification of bird taxa species in ancient Egyptian mummies: Part 2, a qualitative evaluation of the utility of CT scanning and 3D printing. J. Archaeol. Sci. Rep. 2022, 46, 103668. [Google Scholar] [CrossRef]
  22. Sokiranski, R.; Faltings, D.; Sokiranski, S.; Pirsig, W.; Mudry, A. Probable fatal mastoiditis by the around 2300 year old Heidelberg's Egyptian mummy Djed-Hor. Eur. Ann. Otorhinolaryngol. Head Neck Dis. 2024, 141, 107–112. [Google Scholar] [CrossRef] [PubMed]
  23. Mahnke, H.E.; Arlt, T.; Baum, D.; Hege, H.C.; Herter, F.; Lindow, N.; Manke, I.; Siopi, T.; Menei, E.; Etienne, M.; et al. Virtual unfolding of folded papyri. J. Cult. Herit. 2020, 41, 264–269. [Google Scholar] [CrossRef]
  24. de Solla Price, D. Gears from the Greeks. The Antikythera Mechanism: A Calendar Computer from ca. 80 B. C. Trans. Am. Philos. Soc. 1974, 64, 1–70. [Google Scholar] [CrossRef]
  25. Roumeliotis, M. Antikythera Mechanism, Multimedia CD (In Greek, English, and German); Technology Museum of Thessaloniki: Thermi, Greece, 1999. [Google Scholar]
  26. Freeth, T.; Bitsakis, Y.; Moussas, X.; Seiradakis, J.H.; Tselikas, A.; Mangou, H.; Zafeiropoulou, M.; Hadland, R.; Bate, D.; Ramsey, A.; et al. Decoding the ancient Greek astronomical calculator known as the Antikythera Mechanism. Nature 2006, 444, 587–591. [Google Scholar] [CrossRef] [PubMed]
  27. Ramsey, A. X-ray Tomography of the Antikythera Mechanism. In Proceedings of the Antikythera to the Square Kilometre Array: Lessons from the Ancients, Kerastari, Greece, 12–15 June 2012; Volume 170, p. 22. Available online: https://pos.sissa.it/170/022/pdf (accessed on 15 April 2024).
  28. Roumeliotis, M. Are the Modern Computer Simulations a Substitute for Physical Models? The Antikythera Case. In Proceedings of the Antikythera to the Square Kilometre Array: Lessons from the Ancients, Kerastari, Greece, 12–15 June 2012; Volume 170, p. 36. Available online: https://pos.sissa.it/170/036/pdf (accessed on 15 April 2024).
  29. Anastasiou, M. The Antikythera Mechanism: Astronomy and Technology in Ancient Greece. Ph.D. Thesis, Aristotle University, Thessaloniki, Greece, 2014. Available online: http://go.nature.com/2Af8uxy (accessed on 15 April 2024). (In Greek).
  30. Roumeliotis, M. Calculating the torque on the shafts of the Antikythera Mechanism to determine the location of the driving gear. Mech. Mach. Theory 2018, 122, 148–159. [Google Scholar] [CrossRef]
  31. Seiradakis, J.H.; Edmunds, M.G. Our current knowledge of the Antikythera Mechanism. Nat. Astron. 2018, 2, 35–42. [Google Scholar] [CrossRef]
  32. Feddoul, Y.; Ragot, N.; Duval, F.; Havard, V.; Baudry, D.; Assila, A. Exploring human-machine collaboration in industry: A systematic literature review of digital twin and robotics interfaced with extended reality technologies. Int. J. Adv. Manuf. Technol. 2023, 129, 1917–1932. [Google Scholar] [CrossRef]
  33. Rivera, M.L.; Mora-Serrano, J.; Oñate, E. A Critical Review of How Extended Reality (XR) has Addressed Key Factors Influencing Safety on Construction Projects (fSCPs). Arch. Comput. Methods Eng. 2024, 31, 2015–2048. [Google Scholar] [CrossRef]
  34. Stephenson, N.; Pushparajah, K.; Wheeler, G.; Deng, S.; Schnabel, J.A.; Simpson, J.M. Extended reality for procedural planning and guidance in structural heart disease—A review of the state-of-the-art. Int. J. Cardiovasc. Imaging 2023, 39, 1405–1419. [Google Scholar] [CrossRef]
  35. Kanade, S.G.; Duffy, V.G. Exploring the effectiveness of virtual reality as a learning tool in the context of task interruption: A systematic review. Int. J. Ind. Ergon. 2024, 99, 103548. [Google Scholar] [CrossRef]
  36. Wong, E.Y.C.; Lee, P.T.Y. Virtual reality in transportation and logistics: A clustering analysis of studies from 2010 to 2023 and future directions. Comput. Hum. Behav. 2024, 153, 108082. [Google Scholar] [CrossRef]
  37. Woodland, M.B.; Ong, J.; Zaman, N.; Hirzallah, M.; Waisberg, E.; Masalkhi, M.; Kamran, S.A.; Lee, A.G.; Tavakkoli, A. Applications of extended reality in spaceflight for human health and performance. Acta Astronaut. 2024, 214, 748–756. [Google Scholar] [CrossRef]
  38. Mudička, Š.; Kapica, R. Digital Heritage, the Possibilities of Information Visualisation through Extended Reality Tools. Heritage 2023, 6, 112–131. [Google Scholar] [CrossRef]
  39. Tukur, M.; Schneider, J.; Househ, M.; Dokoro, A.H.; Ismail, U.I.; Dawaki, M.; Agus, M. The Metaverse digital environments: A scoping review of the techniques, technologies, and applications. J. King Saud Univ. Comput. Inf. Sci. 2024, 36, 101967. [Google Scholar] [CrossRef]
  40. Innocente, C.; Ulrich, L.; Moos, S.; Vezzetti, E. A framework study on the use of immersive XR technologies in the cultural heritage domain. J. Cult. Herit. 2023, 62, 268–283. [Google Scholar] [CrossRef]
  41. Chueca, J.; Verón, J.; Font, J.; Pérez, F.; Cetina, C. The consolidation of game software engineering: A systematic literature review of software engineering for industry-scale computer games. Inf. Softw. Technol. 2024, 165, 107330. [Google Scholar] [CrossRef]
  42. Unity Real-Time Development Platform. Available online: https://unity.com/ (accessed on 7 April 2024).
  43. Unreal Engine, the Most Powerful Real-Time 3D Creation Tool. Available online: https://www.unrealengine.com/ (accessed on 7 April 2024).
  44. Yastikli, N. Documentation of cultural heritage using digital photogrammetry and laser scanning. J. Cult. Herit. 2007, 8, 423–427. [Google Scholar] [CrossRef]
  45. Kingsland, K. Comparative analysis of digital photogrammetry software for cultural heritage. Digit. Appl. Archaeol. Cult. Herit. 2020, 18, e00157. [Google Scholar] [CrossRef]
  46. Acke, L.; Corradi, D.; Verlinden, J. Comprehensive educational framework on the application of 3D technologies for the restoration of cultural heritage objects. J. Cult. Herit. 2024, 66, 613–627. [Google Scholar] [CrossRef]
  47. Di Angelo, L.; Di Stefano, P.; Guardiani, E. A review of computer-based methods for classification and reconstruction of 3D high-density scanned archaeological pottery. J. Cult. Herit. 2022, 56, 10–24. [Google Scholar] [CrossRef]
  48. Calantropio, A.; Chiabrando, F. Underwater Cultural Heritage Documentation Using Photogrammetry. J. Mar. Sci. Eng. 2024, 12, 413. [Google Scholar] [CrossRef]
  49. Anastasovitis, E.; Roumeliotis, M. Transforming computed tomography scans into a full-immersive virtual museum for the Antikythera Mechanism. Digit. Appl. Archaeol. Cult. Herit. 2023, 28, e00259. [Google Scholar] [CrossRef]
  50. Schönfelder, P.; Aziz, A.; Faltin, B.; König, M. Automating the retrospective generation of As-is BIM models using machine learning. Autom. Constr. 2023, 152, 104937. [Google Scholar] [CrossRef]
  51. Campagna, S.; Kobbelt, L.; Seidel, H.P. Enhancing digital documents by including 3D-models. Comput. Graph. 1998, 22, 655–666. [Google Scholar] [CrossRef]
  52. Marovic, B.; Jovanovic, Z. Visualization of 3D fields and medical data and using VRML. Future Gener. Comput. Syst. 1998, 14, 33–49. [Google Scholar] [CrossRef]
  53. Brett, A.D.; Hill, A.; Taylor, C.J. A method of 3D surface correspondence and interpolation for merging shape examples. Image Vis. Comput. 1999, 17, 635–642. [Google Scholar] [CrossRef]
  54. Anastasovitis, E.; Roumeliotis, M. Virtual Museum for the Antikythera Mechanism: Designing an Immersive Cultural Exhibition. In Proceedings of the 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), Munich, Germany, 16–20 October 2018; pp. 310–313. [Google Scholar] [CrossRef]
  55. Bilskie, M.V.; Hagen, S.C.; Medeiros, S.C. Unstructured finite element mesh decimation for real-time Hurricane storm surge forecasting. Coast. Eng. 2020, 156, 103622. [Google Scholar] [CrossRef]
  56. Mahnke, H.E. Radiation studies of items of cultural heritage. Radiat. Phys. Chem. 2022, 200, 110323. [Google Scholar] [CrossRef]
  57. Meinerová, T.; Šutoová, D.; Havelková, P.B.; Velemínská, J.; Dupej, J.; Bejdová, Š. How reliable is the application of the sex classifier based on exocranial surface (Musilová et al., 2016) for geographically and temporally distant skull series. Forensic Sci. Int. 2023, 352, 111850. [Google Scholar] [CrossRef]
  58. White, K.N.; Chiasserini, D.; Loynes, R.; David, A.R.; van Dongen, B.E.; Drosou, K.; Forshaw, R.; Fraser, S.; Causey-Freeman, P.; Metcalfe, J.; et al. Enhancing mummy ‘palaeobiographies’ through the use of multidisciplinary techniques and approaches. J. Archaeol. Sci. Rep. 2023, 47, 103784. [Google Scholar] [CrossRef]
  59. McKnight, L.; White, J.; Rosier, A.; Cooper, J.; Bibb, R. Identification of avian remains contained within wrapped ancient Egyptian mummies: Part 1, A critical assessment of identification techniques. J. Archaeol. Sci. Rep. 2022, 45, 103585. [Google Scholar] [CrossRef]
  60. Seales, W.B.; Parker, C.S.; Segal, M.; Tov, E.; Shor, P.; Porath, Y. From damage to discovery via virtual unwrapping: Reading the scroll from En-Gedi. Sci. Adv. 2016, 2, e1601247. [Google Scholar] [CrossRef]
  61. Parsons, S.; Parker, C.S.; Chapman, C.; Hayashida, M.; Seales, W.B. Educelab-scrolls: Verifiable recovery of text from herculaneum papyri using X-ray ct. arXiv 2023, arXiv:2304.02084. [Google Scholar] [CrossRef]
  62. Avizo Software: Materials Characterization Software. Available online: https://www.thermofisher.com/gr/en/home/electron-microscopy/products/software-em-3d-vis/avizo-software.html (accessed on 11 April 2024).
  63. DigiM Image to Simulation Software. Available online: https://www.digimsolution.com/technology/digim-i2s-software (accessed on 11 April 2024).
  64. Dragonfly: 3D Visualization and Analysis Solutions for Scientific and Industrial Data. Available online: https://www.theobjects.com/dragonfly/index.html (accessed on 11 April 2024).
  65. GeoDict: Modular Software Solution for Digital Material Design. Available online: https://www.math2market.com/geodict-software/geodict-base-modules.html (accessed on 11 April 2024).
  66. Imaris Microscopy Image Analysis Software. Available online: https://imaris.oxinst.com/packages (accessed on 11 April 2024).
  67. Tescan Applications. Available online: https://www.tescan.com/applications/ (accessed on 11 April 2024).
  68. VGSTUDIO: The Simple Solution for the Visualization of CT Data. Available online: https://www.volumegraphics.com/en/products/vgstudio.html (accessed on 11 April 2024).
  69. 3D Slicer Image Computing Platform. Available online: https://www.slicer.org/ (accessed on 11 April 2024).
  70. Drishti. Available online: https://github.com/nci/drishti (accessed on 11 April 2024).
  71. Fiji Processing Package. Available online: https://imagej.net/software/fiji/ (accessed on 11 April 2024).
  72. ImageVis3D: Volumetric Rendering Program. Available online: https://www.sci.utah.edu/software/imagevis3d.html (accessed on 11 April 2024).
  73. InVesalius Open-Source Software for Reconstruction of Computed Tomography and Magnetic Resonance Images. Available online: https://invesalius.github.io/ (accessed on 11 April 2024).
  74. volBrain: An Automated MRI Brain Volumetric System. Available online: https://www.volbrain.net/ (accessed on 11 April 2024).
  75. Limaye, A. Drishti: A volume exploration and presentation tool. In Proceedings of the Developments in X-ray Tomography VIII, San Diego, CA, USA, 17 October 2012; SPIE: Bellingham, WA, USA, 2012; Volume 8506, pp. 191–199. [Google Scholar] [CrossRef]
  76. Dell'Unto, N.; Leander, A.M.; Dellepiane, M.; Callieri, M.; Ferdani, D.; Lindgren, S. Digital reconstruction and visualization in archaeology: Case-study drawn from the work of the Swedish Pompeii Project. In Proceedings of the 2013 Digital Heritage International Congress (DigitalHeritage), Marseille, France, 28 October–1 November 2013; pp. 621–628. [Google Scholar] [CrossRef]
  77. Obradović, M.; Vasiljević, I.; Đurić, I.; Kićanović, J.; Stojaković, V.; Obradović, R. Virtual Reality Models Based on Photogrammetric Surveys-A Case Study of the Iconostasis of the Serbian Orthodox Cathedral Church of Saint Nicholas in Sremski Karlovci (Serbia). Appl. Sci. 2020, 10, 2743. [Google Scholar] [CrossRef]
  78. Obradović, M.; Mišić, S.; Vasiljević, I.; Ivetić, D.; Obradović, R. The Methodology of Virtualizing Sculptures and Drawings: A Case Study of the Virtual Depot of the Gallery of Matica Srpska. Electronics 2023, 12, 4157. [Google Scholar] [CrossRef]
  79. Papas, N.; Tsongas, K.; Karolidis, D.; Tzetzis, D. A comparison of laser and structured light scanning technologies for archaeological applications. Int. J. Mod. Manuf. Technol. 2021, 13, 111–116. [Google Scholar] [CrossRef]
  80. Liu, S.Q.; Ong, S.K.; Chen, Y.P.; Nee, A.Y.C. Real-time, dynamic level-of-detail management for three-axis NC milling simulation. Comput.-Aided Des. 2006, 38, 378–391. [Google Scholar] [CrossRef]
  81. Li, M.; Nan, L. Feature-preserving 3D mesh simplification for urban buildings. ISPRS J. Photogramm. Remote Sens. 2021, 173, 135–150. [Google Scholar] [CrossRef]
  82. Oh, S.; Koo, B.K. Data perturbation for fewer triangles in marching tetrahedra. Graph. Models 2007, 69, 211–218. [Google Scholar] [CrossRef]
  83. Narayanaswamy, A.; Dwarakapuram, S.; Bjornsson, C.S.; Cutler, B.M.; Shain, W.; Roysam, B. Robust Adaptive 3-D Segmentation of Vessel Laminae from Fluorescence Confocal Microscope Images and Parallel GPU Implementation. IEEE Trans. Med. Imaging 2010, 29, 583–597. [Google Scholar] [CrossRef]
  84. Maglo, A.; Courbet, C.; Alliez, P.; Hudelot, C. Progressive compression of manifold polygon meshes. Comput. Graph. 2012, 36, 349–359. [Google Scholar] [CrossRef]
  85. Pagani, L.; Jiang, X.; Scott, P.J. Investigation on the effect of sampling on areal texture parameters. Measurement 2018, 128, 306–313. [Google Scholar] [CrossRef]
  86. Qiao, Z.; Lu, T.; Luo, H.; Liu, Q.; Klasky, S.; Podhorszki, N.; Wang, J. SIRIUS: Enabling Progressive Data Exploration for Extreme-Scale Scientific Data. IEEE Trans. Multi-Scale Comput. Syst. 2018, 4, 900–913. [Google Scholar] [CrossRef]
  87. Ghazanfarpour, A.; Mellado, N.; Himeur, C.E.; Barthe, L.; Jessel, J.P. Proximity-aware multiple meshes decimation using quadric error metric. Graph. Models 2020, 109, 101062. [Google Scholar] [CrossRef]
  88. dos Anjos, R.K.; Roberts, R.A.; Allen, B.; Jorge, J.; Anjyo, K. Saliency detection for large-scale mesh decimation. Comput. Graph. 2023, 111, 63–76. [Google Scholar] [CrossRef]
  89. Fang, W.; Chen, L.; Zhang, T.; Chen, C.; Teng, Z.; Wang, L. Head-mounted display augmented reality in manufacturing: A systematic review. Robot. Comput.-Integr. Manuf. 2023, 83, 102567. [Google Scholar] [CrossRef]
  90. Apple Vision Pro. Available online: https://www.apple.com/apple-vision-pro/ (accessed on 11 April 2024).
  91. Meta Quest 3: New Mixed Reality VR Headset. Available online: https://www.meta.com/quest/quest-3/ (accessed on 11 April 2024).
  92. Varjo XR-4 Series: Mixed Reality Headset for Professionals. Available online: https://varjo.com/products/xr-4/ (accessed on 11 April 2024).
  93. Vive XR Elite—Convertible All-in-One XR headset. Available online: https://www.vive.com/us/product/vive-xr-elite/overview/ (accessed on 11 April 2024).
  94. Anastasovitis, E.; Georgiou, G.; Matinopoulou, E.; Nikolopoulos, S.; Kompatsiaris, I.; Roumeliotis, M. Enhanced Inclusion through Advanced Immersion in Cultural Heritage: A Holistic Framework in Virtual Museology. Electronics 2024, 13, 1396. [Google Scholar] [CrossRef]
  95. Thiel, S.; Bernhardt, J.C. AI in Museums: Reflections, Perspectives and Applications; Volume 74 in the series Edition Museum; Transcript Verlag: Bielefeld, Germany, 2023. [Google Scholar] [CrossRef]
  96. Cotella, V.A. From 3D point clouds to HBIM: Application of Artificial Intelligence in Cultural Heritage. Autom. Constr. 2023, 152, 104936. [Google Scholar] [CrossRef]
  97. Reshma, M.R.; Kannan, B.; Jagathy Raj, V.P.; Shailesh, S. Cultural heritage preservation through dance digitization: A review. Digit. Appl. Archaeol. Cult. Herit. 2023, 28, e00257. [Google Scholar] [CrossRef]
  98. Mylonas, G.; Kalogeras, A.; Pavlidis, G.; Lalos, A.; García-López, A. Digital Twins for Protecting Cultural Heritage Against Climate Change. Computer 2023, 56, 100–104. [Google Scholar] [CrossRef]
  99. Münster, S. Advancements in 3D Heritage Data Aggregation and Enrichment in Europe: Implications for Designing the Jena Experimental Repository for the DFG 3D Viewer. Appl. Sci. 2023, 13, 9781. [Google Scholar] [CrossRef]
  100. Jin, X.; Wang, X.; Cao, X.; Xue, C. Construction and recognition of acoustic ID of ancient coins based on deep learning of artificial intelligence for audio signals. Herit. Sci. 2023, 11, 46. [Google Scholar] [CrossRef]
  101. Sarithadevi, S.; Rajesh, R. Character recognition for Malayalam palm leaf manuscripts: An overview of techniques and challenges. In Proceedings of the AIP Conference, Kochi, Kerala, India, 17–22 January 2022; AIP Publishing: Melville, NY, USA, 2022; Volume 2773, p. 020003. [Google Scholar] [CrossRef]
  102. Alonso, J.M.; Casalino, G. Explainable artificial intelligence for human-centric data analysis in virtual learning environments. In Proceedings of the International Workshop on Higher Education Learning Methodologies and Technologies Online (HELMeTO 2019), Novedrate, Italy, 6–7 June 2019; Springer International Publishing: Cham, Switzerland, 2019; Volume 1091, pp. 125–138. [Google Scholar] [CrossRef]
  103. Ververidis, D.; Migkotzidis, P.; Nikolaidis, E.; Anastasovitis, E.; Papazoglou Chalikias, A.; Nikolopoulos, S.; Kompatsiaris, I. An authoring tool for democratizing the creation of high-quality VR experiences. Virtual Real. 2022, 26, 105–124. [Google Scholar] [CrossRef]
  104. Marín-Vega, H.; Alor-Hernández, G.; Bustos-López, M.; López-Martínez, I.; Hernández-Chaparro, N.L. Extended Reality (XR) Engines for Developing Gamified Apps and Serious Games: A Scoping Review. Future Internet 2023, 15, 379. [Google Scholar] [CrossRef]
  105. Anastasovitis, E.; Roumeliotis, M. Software tools for analysis and visualization of the Antikythera Mechanism. In Proceedings of the 2017 3DTV Conference: The True Vision-Capture, Transmission and Display of 3D Video (3DTV-CON), Copenhagen, Denmark, 7–9 June 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 1–4. [Google Scholar] [CrossRef]
  106. Anastasovitis, E.; Roumeliotis, M. Creative Industries and Immersive Technologies for Training, Understanding and Communication in Cultural Heritage. In Digital Heritage, Progress in Cultural Heritage: Documentation, Preservation, and Protection (EuroMed 2020), Lecture Notes in Computer Science; Ioannides, M., Fink, E., Cantoni, L., Champion, E., Eds.; Springer: Cham, Switzerland, 2021; Volume 12642, pp. 450–461. [Google Scholar] [CrossRef]
  107. Anastasovitis, E.; Roumeliotis, M. Virtual museum for the Antikythera Mechanism. In Proceedings of the Demos track of the 14th EuroVR International Conference (EuroVR 2017), Laval, France, 12–14 December 2017; pp. 8–9. [Google Scholar]
  108. Dudley, J.; Yin, L.; Garaj, V.; Kristensson, P.O. Inclusive Immersion: A review of efforts to improve accessibility in virtual reality, augmented reality and the metaverse. Virtual Real. 2023, 27, 2989–3020. [Google Scholar] [CrossRef]
Asi 07 00055 g001
Figure 1. The overall methodology.
Figure 1. The overall methodology.
Asi 07 00055 g001
Asi 07 00055 g002
Figure 2. The 3D representations of the fragments A–G of the Antikythera Mechanism and five relevant indicative instances of their internal structure.
Figure 2. The 3D representations of the fragments A–G of the Antikythera Mechanism and five relevant indicative instances of their internal structure.
Asi 07 00055 g002
Asi 07 00055 g003aAsi 07 00055 g003b
Figure 3. Multimodal interactions with the combined representations of the fragments in the virtual museum for the Antikythera Mechanism. In (a), the user holds objects with her left hand and examines the 3D representation of fragment A, while the right-hand controller interacts with the inner structure of the artifact. Similarly, in (b), the user examines, in a deeper layer, the non-visible parts of the corroded fragment.
Figure 3. Multimodal interactions with the combined representations of the fragments in the virtual museum for the Antikythera Mechanism. In (a), the user holds objects with her left hand and examines the 3D representation of fragment A, while the right-hand controller interacts with the inner structure of the artifact. Similarly, in (b), the user examines, in a deeper layer, the non-visible parts of the corroded fragment.
Asi 07 00055 g003aAsi 07 00055 g003b
Asi 07 00055 g004
Figure 4. The flowchart of the user experience for the combined display of 3D representations and X-ray toolkit in virtual space.
Figure 4. The flowchart of the user experience for the combined display of 3D representations and X-ray toolkit in virtual space.
Asi 07 00055 g004
Table 1. Indicative software packages for the analysis of tomographic datasets and imaging.
Table 1. Indicative software packages for the analysis of tomographic datasets and imaging.
SoftwareDistribution
3D Slicer [69]free/open-source
Avizo [62]commercial
digiM I2S [63]commercial
Dragonfly [64]commercial
Drishti [70]free/open-source
Fiji [71]free/open-source
GeoDict [65]commercial
ImageVis3D [72]free/open-source
Imaris [66]commercial
InVesalius [73]free/open-source
TESCAN FIB-SEMs [67]commercial
VGSTUDIO [68]commercial
volBrain platform [74]free for research only
Table 2. Details for the captured instances from the CT scans of the selected fragments of the Antikythera Mechanism.
Table 2. Details for the captured instances from the CT scans of the selected fragments of the Antikythera Mechanism.
FragmentFormatOpacityResolutionInstances per Fragment
A.pngyes1920 × 1080100
B.pngyes1920 × 1080100
C.pngyes1920 × 1080100
D.pngno648 × 736100
E.pngyes1920 × 1080100
F.pngyes1920 × 1080100
G.pngyes1920 × 1080100
Total700
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Anastasovitis, E.; Roumeliotis, M. Enhanced and Combined Representations in Extended Reality through Creative Industries. Appl. Syst. Innov. 2024, 7, 55. https://doi.org/10.3390/asi7040055

AMA Style

Anastasovitis E, Roumeliotis M. Enhanced and Combined Representations in Extended Reality through Creative Industries. Applied System Innovation. 2024; 7(4):55. https://doi.org/10.3390/asi7040055

Chicago/Turabian Style

Anastasovitis, Eleftherios, and Manos Roumeliotis. 2024. "Enhanced and Combined Representations in Extended Reality through Creative Industries" Applied System Innovation 7, no. 4: 55. https://doi.org/10.3390/asi7040055

Article Metrics

Back to TopTop