Harnessing the Potential of the Metaverse and Artificial Intelligence for the Internet of City Things: Cost-Effective XReality and Synergistic AIoT Technologies

Abstract

The Metaverse represents an always-on 3D network of virtual spaces, designed to facilitate social interaction, learning, collaboration, and a wide range of activities. This emerging computing platform originates from the dynamic convergence of Extended Reality (XR), Artificial Intelligence of Things (AIoT), and platform-mediated everyday life experiences in smart cities. However, the research community faces a pressing challenge in addressing the limitations posed by the resource constraints associated with XR-enabled IoT applications within the Internet of City Things (IoCT). Additionally, there is a limited understanding of the synergies between XR and AIoT technologies in the Metaverse and their implications for IoT applications within this framework. Therefore, this study provides a detailed overview of the literature on the potential applications, opportunities, and challenges pertaining to the deployment of XR technologies in IoT applications within the broader framework of IoCT. The primary focus is on navigating the challenges pertaining to the IoT applications powered by VR and AR as key components of MR in the Metaverse. This study also explores the emerging computing paradigm of AIoT and its synergistic interplay with XR technologies in the Metaverse and in relation to future IoT applications in the realm of IoCT. This study’s contributions encompass a comprehensive literature overview of XR technologies in IoT and IoCT, providing a valuable resource for researchers and practitioners. It identifies challenges and resource constraints, identifying areas that require further investigation. It fosters interdisciplinary insights into XR, IoT, AIoT, smart cities, and IoCT, bridging the gap between them. Lastly, it offers innovation pathways for effective XR deployment in future IoT/AIoT applications within IoCT. These contributions collectively advance our understanding of synergistic opportunities and complementary strengths of cutting-edge technologies for advancing the emerging paradigms of urban development.

1. Introduction

The realm of immersive experiences has been dramatically reshaped by the rapid strides in Extended Reality (XR) technologies, an umbrella term for Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR) technologies. These dynamic advancements have unveiled a wealth of captivating possibilities for crafting interactive and spellbinding encounters. These immersive technologies, when combined with the Internet of Things (IoT) and Artificial Intelligence (AI), hold the potential to revolutionize the way we interact with the digital world as part of everyday life [1,2]. In particular, the emergence of the Metaverse, a digitally interconnected and immersive urban environment where virtual and physical realities converge, presents exciting opportunities for Extended Reality (XR) and Artificial Intelligence of Things (AIoT) technologies, which intersects with “the distinctive features of platform-mediated everyday life experiences in cities” [1] in the realm of the Internet of City Things (IoCT). As an emerging concept (e.g., [3]) reflects the integration of numerous IoT devices, sensors, and technologies with VR/AR/MR within urban environments to create smart cities, offering promising avenues for creating intelligent interconnected cities. The idea revolves around what has been identified as virtual forms of smart cities of the future, depicted in the Metaverse as speculative fiction (e.g., [1,4,5,6]). Urban scholars have long explored the role of fictional representations of the city and urban life in shaping urban change (e.g., [7,8]). The Metaverse plays a role in shaping alternatives to the socio-technical imaginaries of smart cities [9,10].

However, IoCT enables the collection, analysis, and sharing of data from different aspects of urban life, such as mobility, transportation, energy, healthcare, education, and more. By connecting and interlinking smart IoT devices, cities can optimize resource management, enhance public services, and create a more efficient and livable urban environment. IoCT plays a crucial role in the development of smart cities, where data-driven decision making and technology integration are utilized to address urban challenges and improve the overall quality of life for citizens [3]. The role of advanced technologies, such as AIoT and Big Data, 5G, Digital Twin (DT), blockchain, and edge computing, is crucial for IoT deployments in the context of IoCT. These technologies bring various capabilities and advantages that enhance the efficiency, functionality, and overall performance of smart city implementations [11,12,13]. The integration of these advanced technologies in the realm of IoCT empowers cities to become more intelligent, sustainable, and efficient, offering a wide range of benefits to both the city administration and its residents.

Additionally, AR/VR/MR technologies have enormous potential to unleash the power of IoT in a diversified set of applications in smart cities. These technologies make full use of data generated in IoT deployments in an interactive way, leading to an immersive user experience with useful and attractive visualizations and end applications [14]. The use of these technologies in IoT applications is on the rise in several applications. Companies and businesses are investing more and more in the duo of technology to explore the full potential of IoT devices and generated data. The interest is mainly due to the visualization capabilities of AR and VR applications enabling operators/employees to better understand IoT data by providing an intuitively three-dimensional view. These technologies also help to better understand and diagnose an underlying problem by merging and combining data from different sources in a single view [3].

Despite the outstanding capabilities of exploring the potential of IoT devices and enormous growth in the market, VR and AR technologies face several challenges affecting their adaptation in several IoT applications. Some key challenges include the lack of hardware and regulations/documentation on its usage and suitability in different applications, the unavailability of creative content for novel applications, public skepticism, and risks associated with physical safety [15]. More importantly, the cost associated with VR and AR hardware and software components is one of the main hurdles in its widespread adaptation.

There are several factors, such as the type of VR environment, instructional design and programming, type of content, and VR headsets, contributing to the overall cost of AR and VR solutions. The overall cost of the solutions could be reduced by making appropriate choices in the selection of different components. For instance, several types of VR headsets are available in the market, providing a wide range of prices. Similarly, the choice of off-the-shelf or custom content also has a direct impact on the cost of VR and AR solutions.

The scope of the study revolves around the combination of cost-effective XR and synergistic AIoT technologies. Cost-effective XR entails the utilization of XR technologies in a manner that is efficient and economical. It involves deploying XR solutions that offer high value and immersive experiences without incurring excessive costs, making these technologies more accessible and feasible for various IoT applications. Synergistic AIoT is about the integration of AI and IoT as powerful technologies to create a collaborative and mutually reinforcing environment. AIoT leverages AI’s capabilities in analyzing and interpreting IoT-generated data to enhance decision-making, efficiency, and automation. This synergy results in an ecosystem where IoT devices gather data, and AI algorithms process and interpret it, leading to more informed actions and better outcomes in relation to IoT applications.

Within this scope, the study provides a detailed overview of the literature on the potential applications, opportunities, and challenges associated with deploying VR/AR/MR technologies, including their XR versions, in IoT applications within the broader framework of IoCT. Additionally, it explores the emerging computing paradigm of AIoT and its synergistic interplay with XR technologies in the Metaverse and in relation to future IoT applications in the realm of IoCT. From these objectives, we derive the following key research questions:

1.

RQ1: What roles do virtual digital models play in IoCT applications, and how do the derived XR variants contribute to IoCT applications?

2.

RQ2: What are the latest successful IoCT applications realized through the integration of XR models, and how can researchers customize virtual digital models to cater to specific IoCT application needs?

3.

RQ3: What are the current challenges, insightful observations, and prospective avenues for future research when it comes to harnessing XR within the IoCT framework?

4.

RQ4: How does the amalgamation of AIoT with XR contribute to the seamless development of the Metaverse, and what influence does this integration exert on user experiences and interactivity as well as future IoT applications?

By answering these research questions, we contribute the following insights to the existing knowledge:

• Employing a comprehensive approach, the study offers a detailed overview of key concepts underlying VR-, AR-, MR-, and XR-related technologies.
• Enriching the discourse, the study delves into a thorough examination of cost-effective devices suitable for VR and AR applications.
• By examining recent IoT use cases through the prism of cost-effective VR/AR devices, the study provides a comprehensive review.
• Identifying untackled challenges and potential avenues for further research, the study contributes to the advancement of cost-effective VR/AR-driven IoCT applications.

The remainder of the study is organized as follows. Section 2 discusses the key components and the enabling technologies of VR and AR and provides a strong platform for choosing them for IoCT applications. Section 3 summarizes the related works in the prescribed domain of discussion. Section 4 outlines the methodology used in the systematic collection, analyses, and syntheses of existing work to facilitate a deep understanding of the multifaceted topic of focus. Section 5 provides a summary of the factors impacting the cost of VR and AR frameworks for Metaverse-enabled IoCT applications. Also, this section discusses XR technology for IoCT applications and its impact. In Section 6, we provide key insights and lessons learned from the literature and future research directions. Finally, Section 7 concludes the paper.

2. Conceptual Background

In this section, we provide a delineation of the pivotal conceptual constructs that compose the essence of our study, intricately highlighting their interconnections and synergies. This elucidation is poised to serve as a guiding compass for readers, illuminating the significance of IoT, XR, and AIoT technologies, the underlying components of IoT applications, and smart cities contextualized within the realm of IoCT.

2.1. IoT, IoCT, and the Metaverse

In the dynamic landscape of technological advancement, two related concepts have risen to prominence, each wielding the power to reshape our digital realm: the IoT and the IoCT. These interconnected frameworks have ushered in a new era of connectivity and intelligence, revolutionizing how we interact with both our physical surroundings and the virtual world. While IoT has initiated a transformative shift by interlinking devices and enabling data exchange, IoCT, with its focus on urban environments, holds the potential to fundamentally enhance how we experience and manage cities using IoT. According to the Annual Internet Report (2018–2023), Cisco [16] states that machine-to-machine connections will reach 14.7 billion by 2023. IoT Analytics [17] predicts a leap to up to 27 billion IoT-connected devices globally by 2025. This expansion is increasingly spanning almost all urban and industrial spheres through a variety of applications. The distinction between IoT and IoCT lies at the heart of their impact, influencing applications across various domains. Moreover, as the digital horizon extends to encompass the Metaverse, these two concepts take on renewed significance in shaping immersive experiences that bridge the tangible and the virtual within urban landscapes [1]. To fully grasp their implications and potential, it is essential to explore their definitions, commonalities, differences, and their intricate relationship with the emerging concept of the Metaverse.

IoT refers to the interconnected network of physical devices, vehicles, buildings, and other objects embedded with sensors, software, and network connectivity that enables them to collect and exchange data. These devices can communicate with each other and with central systems, enabling them to perform tasks and make decisions based on the data they collect. IoT is commonly used in various domains, including smart urbanism, platform urbanism, and the Metaverse [18]. In essence, IoCT’s technical underpinnings involve a complex ecosystem of sensors, data integration mechanisms, computing resources, analytics tools, and communication infrastructure [19,20]. By effectively harnessing these components, IoCT transforms cities into smart, interconnected entities that can respond intelligently to challenges and opportunities. From a technical perspective, IoCT represents a sophisticated framework that leverages digital connectivity and data integration to transform urban environments into intelligent and responsive entities. IoCT harnesses a wide array of technologies to facilitate seamless communication, data collection, analysis, and decision making across diverse sectors within a city’s infrastructure. Below we present a breakdown of how IoCT operates at a technical level while drawing on many studies on the use of IoT in smart cities (e.g., [3,18,21,22]).

1.

Sensor Networks and Data Collection: IoCT relies heavily on sensor networks strategically placed throughout the city to capture real-time data. These sensors can monitor various parameters such as temperature, air quality, traffic flow, energy consumption, waste management, and more. These devices play a pivotal role in gathering data points that reflect the current state of the urban environment.

2.

Data Integration and Interoperability: IoCT involves integrating data from a multitude of sources across different sectors. This requires establishing interoperability standards and protocols to ensure seamless communication between various devices, systems, and platforms. This integration enables a holistic view of the city’s operations and helps in making informed decisions.

3.

Edge and Cloud Computing: The vast volume of data generated by IoCT sensors demands efficient processing and analysis. Edge computing, where data are processed closer to the data source, ensures real-time insights and reduces latency. Cloud computing is also utilized for more complex analytics, storage, and long-term data aggregation.

4.

Data Analytics and Insights: IoCT employs advanced data analytics techniques, including Machine Learning (ML) and AI, to extract meaningful insights from the collected data. These insights help identify patterns, trends, and anomalies, enabling city planners and administrators to make informed decisions for optimizing urban operations and services.

5.

Smart Decision-Making: IoCT enables data-driven decision making by providing real-time and predictive information. For instance, real-time traffic data can optimize traffic signal timings to reduce congestion, or energy consumption patterns can help adjust lighting and HVAC systems in public spaces. Predictive analytics can anticipate maintenance needs, preventing infrastructure failures.

6.

Communication Infrastructure: IoCT relies on robust communication infrastructure, such as high-speed internet connectivity, wireless networks (e.g., 5G), and communication protocols, to ensure reliable data transmission between devices and systems.

7.

Security and Privacy: Given the sensitivity of urban data, security measures including encryption, access controls, and data anonymization are paramount for protecting both citizen privacy and the integrity of the system.

8.

User Interfaces and Visualization: IoCT systems often provide user-friendly interfaces and visualizations that display real-time data and insights. These interfaces allow city officials, administrators, and citizens to monitor and engage with the city’s various aspects, enhancing transparency and participation.

Both IoT and IoCT involve the interconnection of physical objects with digital systems, enabling data exchange and automation. They both contribute to the concept of a more connected and intelligent world, where devices and city components work together to improve efficiency, convenience, and decision making. The main difference lies in the scope of the application. While IoT encompasses a wide range of applications across various industries, IoCT specifically focuses on the urban environment and its infrastructure, spanning applications across various domains, such as smart transportation and mobility, environmental monitoring and sustainability, smart energy management, smart waste management, smart water management, smart healthcare, smart infrastructure and building management, connected public service, and data-driven urban planning and management. Furthermore, IoT and IoCT play a role in shaping the Metaverse by providing the necessary connectivity and intelligence to bridge the gap between the physical and digital worlds. As the Metaverse incorporates diverse digital and physical entities, the data generated by IoT and IoCT systems can contribute to creating richer and more immersive experiences within this virtual realm. For example, real-world urban data collected through IoCT systems can influence the virtual cityscapes within the Metaverse, enhancing realism and interactions for users exploring these digital realms.

From a general perspective, the Metaverse represents a digital universe comprising interconnected virtual worlds, environments, and spaces where users can socialize, interact, work, play, and conduct various activities through digital representations of themselves, called avatars. From a technological perspective, the Metaverse is an advanced iteration of the internet, incorporating VR, AR, and MR as immersive technologies. It provides a persistent and immersive digital environment where users can seamlessly transition between different experiences and platforms, blurring the lines between the physical and digital realms. It offers opportunities for new forms of entertainment, social interaction, education, and business. Companies are beginning to explore the Metaverse’s potential to craft virtual events, interactive conferences, dynamic marketplaces, and other innovations, to shape novel urban digital economies and cultural landscapes. Similarly, cities are integrating the Metaverse into various aspects of urban life to create immersive digital experiences, create virtual events and spatial forms, and offer interactive urban planning and design simulations. The Metaverse is being utilized to reimagine urban spaces and experiment with new forms of living within city environments, though with profound ethical and social implications [4]. In a recent study, Kuru [23] examines the integration of the Metaverse within the smart city ecosystem to enhance the immersive quality of experiences. The proposed urban Metaverse framework aims to transform data-driven smart cities into virtually inhabited spaces with shared urban experiences. Through this framework, cities can establish granular virtual societies and dissolve boundaries between real and virtual realms. This advancement holds potential for global immersive integration, connecting digitally enabled cities beyond physical constraints. The framework anticipates expediting the adoption of immersive urban experiences, expanding citizens’ interaction with digitally connected urban landscapes. All these socio-technical imaginaries of smart cities are being actualized by the recent advances in AIoT and XR technologies as well as their integration [1,4]. Still, the Metaverse has been evolving, and its exact nature might change over time as technology and ideas progress, and visions may not actualize as they are imagined.

2.2. Extended Reality (XR)

XR is the conglomeration of VR, AR, and MR, where these three main realities compose the XR technology. XR devices encompass all three realities, instead of having different devices for each realm.

There exists quite a large category of VR/AR devices capable of providing rich immersive experiences for the user. A few of the types of VR/AR devices are as follows:

• Mobile devices: Smartphones and tablet PCs are playing a leading role in the VR/AR market by providing a rich experience for industrial tasks, business, entertainment, gaming, and social networking.
• Special VR/AR devices: These devices are special devices designed only to provide a rich VR/AR experience. Head-mounted displays (HMDs) are one category of special VR/AR devices, which makes the data transparent to the view of the users.
• VR/AR glasses: See-through wearable glasses supporting VR/AR functionalities are capable of displaying information from smartphones directly in the VR/AR glasses, providing hands-free operations. These types of VR/AR glasses are capable of assisting workers in industries, to gain quick hands-free access to the internet and gain valuable information.
• VR/AR contact lenses: Paving the way for new VR/AR experiences, VR/AR contact lenses fixed to human eyes can interface with smartphones and are capable of performing actions similar to a digital camera and providing enhanced VR/AR experiences.

2.2.1. Virtual Reality (VR)

In general, VR creates a whole new environment and provides a completely immersive experience for the users. It uses computer technology to create a simulated experience that may be similar or completely different from the real world. Standard VR systems use either headsets or multi-projected environments to generate realistic sounds and visuals. The following are the most vital elements of VR.

• Virtual world: Independent from the real world, the virtual world is an imaginary space with a real world of digital objects. Simulation and computer graphic models are used to create such a virtual world by rendering digital objects. The designers establish the link between the digital objects by a predefined set of rules.
• Immersion: Specially designed VR headsets provide a better field of vision for providing an immersive experience for the users. The users will be detached from the real world on the sensory level and will be immersed in the virtual space. Apart from the immersive visual aid, VR headsets also support audio facilities for the users.
• Sensory feedback: Changes in user positions and movement of the head and other body parts provide sensory feedback to the VR headsets to track the scenario and provide appropriate changes in the virtual world. This provides a perfect illusion for the users of VR headsets that they are moving in a virtual world.
• Interactivity: An interactive experience could be attained by users using VR headsets, which provide a real feel of the digital objects in the virtual world. They could pick any virtual object, use it in the virtual environment, and subsequently use it in the digital world.

The major design principles of VR devices are to employ comfort and better user interactions for the end users. Based on the experience it leverages from the graphical simulation, the following are popular types of VR simulations.

• Fully immersive: With the appropriate HMD or VR glasses, a more realistic immersive experience could be gained with complete sight and sound inputs to the users. Fully immersive experiences encompass a wide view of the field with high resolution and sound effects of the digital content.
• Semi-immersive: With the realistic environments created using 3D graphics, semi-immersive VR provides a partial virtual environment for the users. It ensures physical connectivity with the physical scenario as well as focuses on the digital models of objects. They are most used for training and educational activities since they replicate the functions and design aspects of real-world mechanisms.
• Non-immersive: Non-immersive experiences do completely fall under the VR category, since most of them include everyday common usage of computer-generated environments. It allows the users to control the virtual environment projected in the console or computer with the aid of keyboards, mice, and controllers.

2.2.2. Augmented Reality (AR)

AR, on the other hand, keeps the real-world objects as such and adds digital objects to the real world. AR systems integrate three different features: (1) the combination of the real and virtual world, (2) a real-time interaction, and (3) accurate 3D registration of virtual and real objects. The following are the most vital components used for providing rich AR experiences for end users.

• Sensors and Cameras: For imparting successful AR performance, the role of cameras and sensors is very critical. It helps to locate objects in the environment, measure their features, and assist in creating equivalent 3D models.
• Processing modules: Conversion of the captured real-life images into augmented ones is performed by processing units, such as RAM, CPU, and GPU modules. The rich specification of the processing modules helps to understand the reality of AR applications in the deployed environments.
• Projection areas: They are mostly present in AR devices or headsets used or AR applications. It helps to provide interactive visualization of the environment and changes the views as necessary. The surface for the projection of a visualization could be a wall or floor.
• Reflection: For a pleasant view of the 3D augmented images, the reflections from the environment to the user’s eye provide a path for the graphically modified digital images. Curved, double-sided mirrors are used in AR devices to reflect the light, separate the images for both eyes, and reflect the RGB color components as well.

Different technological approaches are being used by manufacturers to provide different AR experiences. The following are popular types of AR technologies.

• Simultaneous Localization and Mapping (SLAM): For rendering augmented real-life images, SLAM provides one of the most effective approaches. It assists in mapping the complete structure of the environment considered for visualization by localizing the sensors present in the AR devices that support SLAM functionality.
• Recognition-based: This is a marker-based AR technology, which uses a camera to locate the objects or visual markers. The recognition-based method depends on the camera to distinguish between real-world objects and markers. Here, 3D virtual graphics are immediately replaced, while a marker is recognized by the device.
• Location-based: Unlike the recognition-based technology, the location-based approach uses a compass, GPS, etc., to obtain the data from the locations for the implementation of AR based on the location information. Deployment of this technology could be comfortably performed using smartphones along with location-based AR applications running on smartphones.

2.2.3. Mixed Reality (MR)

MR is the merging of real and virtual worlds to produce new environments and visualizations. Here, the physical and digital objects coexist and interact in real time. Unlike AR, users can interact with virtual objects.

To provide different user experiences from fully immersive to light information layering of environments, MR developers have provided robust tools to bring virtual experiences to life. The following are popular types of MR apps that integrate HCI, perception, and conventional reality.

• Enhanced environmental apps: As contextual placement of digital objects in virtual environments is becoming popular, enhanced environmental apps could facilitate this feature with the support of HoloLens HMDs. The placement of digital content in the world-of-view environment of the users is one of the key features imparted through enhanced environmental apps.
• Immersive environmental apps: These apps completely change the perspective of users’ view with respect to time and space, driven through an environment-centered approach. In this approach, the context in the real-world environment might not play a significant role in providing immersive experiences for the users.
• Blended environmental apps: The complete transformation of an element into a different digital object is supported through blended environmental apps. It helps to map and recognize the environment of the users and build a digital layer to completely overlay the space of the users. Even though the complete transformation of digital objects is enabled through this blended environmental app, it retains the dimension of the base object.
• MR headset-based apps: Most of the leading semiconductor manufacturers have initiated the making of MR headsets that could provide inside-out tracking and six degrees of freedom of movement across the field-of-view environment. This kind of headset supports plug-and-play features with MR-enabled PCs and thereby provides an amazing immersive experience for the users.

2.3. Integration of VR/AR Technologies and IoT

Immersive visualization experiences have made major contributions to the growth of IoT, but in integration, there is an inherent need to use cost-effective VR and AR solutions to meet all possible end use cases of IoT. Through VR/AR integration with IoT devices, we gain incredible insights into their operation, services, and outcomes. This allows us to visualize the digital model of sourcing data from all sensors interfaced with the IoT devices involved in perceiving the environment and using them all in one place and see how the real-world data impact its services and operations, right on the interactive screen.

Achieving an absolutely immersive experience in the IoT systems using VR/AR technology is challenging, considering the vastly varying types, features, and costs of VR/AR HMDs. Few of the VR/AR headsets and devices have resource constraints and most of them might not meet the economic constraints. Therefore, it is vital to decide on the choice of VR/AR devices based on their roles, compatibility with the IoT application, and economic factors.

The choice of integration of IoT devices with VR/AR solutions depends on the requirements of the IoT applications. For instance, when there is a need for home automation and appliance control on IoHT, cost-effective VR/AR solutions with simple HMDs make sense. In applications that require high performance such as image or video processing, using economic VR/AR devices may not be apparent, and it would make sense to use hybrid solutions. In the following sections, we will see how cost-effective VR/AR technologies can play a relevant role in addressing the challenges and presenting better immersive experiences in IoCT applications.

2.4. Artificial Intelligence of Things (AIoT) and Its Relation to the Metaverse and IoCT

AIoT is the incorporation of AI technology into the IoT infrastructure, enabling real-time data processing, advanced analytics, improved human–machine interaction, and enhanced decision making. AIoT brings together AI and IoT, relocating AI capabilities closer to the data generated by IoT devices and systems. This integration empowers intelligent and autonomous behavior, enhancing the overall performance and capabilities of IoT-based solutions. AIoT acts through control and interaction to respond to the dynamic environment, a process where ML/DL has shown value in enhancing control accuracy and facilitating multimodal interactions [24]. The resurgence of AI is driven by the abundance and potency of IoT-enabled Big Data, thanks to enhanced computing storage capacity and real-time data processing speed. IoT produces Big Data, which in turn requires “AI to interpret, understand, and make decisions that provide optimal outcomes” [25] pertaining to a wide variety of practical applications spanning various urban domains [26].

The role of AIoT in advancing XR technologies within the context of the Metaverse is a catalyst for transformative experiences, revolutionizing how we interact with our digital and physical surroundings [1,2]. This advancement not only propels XR’s capabilities but also infuses AIoT’s intelligence to amplify the potential of both technologies. AIoT’s synergy with XR technologies transforms the Metaverse and elevates XR’s role in the current IoT applications within the IoCT. The marriage of AIoT and XR enriches user experiences, optimizes resource utilization, and enhances data visualization, fostering more intelligent, immersive, and connected urban environments.

2.5. Smart Cities and Their Relationship to the Metaverse

Smart cities are urban areas that leverage digital technologies, data analytics, IoT, and AIoT to enhance the quality of life, sustainability, and efficiency of their services and citizens. These cities employ advanced technologies and data-driven strategies to manage resources, infrastructure, transportation, and public services more effectively. The underlying components of smart cities include [27]:

• Digital infrastructure: Smart cities have robust digital infrastructure, including high-speed internet connectivity and data networks that enable seamless communication between devices, sensors, and citizens.
• Big data analytics: They collect vast amounts of data from various sources, such as sensors, smartphones, and public services. These data are analyzed to gain insights, optimize operations, and improve decision-making processes.
• IoT: They rely on sensors and connected devices to monitor and manage various aspects of urban life such as traffic flow, mobility patterns, energy consumption, air quality, water management, and waste management.
• Enhanced public services: They improve public services such as healthcare, education, and safety by using technology to enhance access and efficiency.
• Citizen engagement: They encourage citizen participation through digital platforms, enabling residents to provide feedback, access services, and engage in decision-making processes.
• Sustainable practices: They incorporate sustainable development in their strategies by implementing practices and initiatives that support and advance the environmental, economic, and social goals of sustainability.

While smart cities primarily exist in the physical world, they share common goals with the Metaverse in terms of leveraging advanced technology for enhanced experiences, connectivity, and sustainability [11]. In terms of digital overlap, the physical and digital worlds may become more intertwined in a future scenario. Smart city data could be integrated into the Metaverse, providing real-time information on urban life to virtual city inhabitants. Regarding simulation and modeling, smart cities can benefit from the Metaverse’s ability to create digital twins or simulations of real-world environments. These simulations can be used for urban planning and testing sustainability strategies. Concerning data integration, smart cities can tap into Metaverse data to gain insights into virtual representations of urban spaces, potentially influencing real-world decisions and optimizations. Overall, while smart cities and the Metaverse are distinct concepts, they share common objectives related to digital technology, data, connectivity, and sustainability. As technology continues to advance, there is potential for increased synergy between smart cities and the Metaverse, leading to more immersive and data-driven urban experiences [9].

3. A Survey of Related Works

In recent years, several surveys and tutorials have been published covering different aspects of IoT applications. However, these works either focus on physical objects, traditional wireless network-based connectivity, protocols, or security-related issues. The majority of existing surveys and tutorials focus on explaining the end IoT use cases in a generalized way, overlooking the detailed explanation of physical objects, protocols, communication, and security threats. However, the generalized discussion and categorization of IoT applications presented in prior works cannot illustrate the detailed implication of using virtual digital objects that could be realized using VR, AR, MR, and XR technologies.

In a recent survey compiled by Hu et al. [28], the authors concisely review the VR-enabling technologies used in IoT applications. The authors analyzed several VR-based IoT use cases and evaluated the streaming quality, prediction, computation, and user experience. In the AR domain, several emerging technological solutions with architectures and networks are summarized with the vision of AR-based IoT applications [29]. The survey by Minerva et al. [30] presents the research efforts in the digital twin IoT application using VR and AR solutions, which helps to identify an extensive set of digital twin features for the transformation of physical objects into digital objects. Coburn et al. [31] summarized the design process involved in the hardware capabilities of cost-effective VR devices. However, they lack the focus on IoT devices and eliminate the discussion on cost-effective AR devices.

Given the evolving nature of AIoT, only few literature review studies have been carried out in relation to the Metaverse, AIoT applications, and smart cities. Huynh et al. [2] explore the role of AI in advancing the Metaverse, investigating its application across diverse technical domains such as natural language processing and machine vision. The authors delve into AI’s potential to enhance immersive experiences and enable the human-like intelligence of virtual agents. They also discuss AI-aided applications in areas such as healthcare, manufacturing, smart cities, and gaming. Rahi et al. [32] center on the synergy between AIoT and the Metaverse in enhancing brain health, mental well-being, and overall health. The authors elucidate how AI, along with data sourced from interconnected sensors, can fortify the Metaverse’s support for mental health. The study also probes the ethical and social implications of the Metaverse on mental health and introduces AIoT-enabled standards for brain and mental healthcare policy. Bibri [1] provides a more holistic exploration, unearthing the convergence of AIoT, XR, neurotechnology, and nanobiotechnology in the Metaverse context. The main focus of this study is to develop a novel conceptual framework for the Metaverse, envisaging it as a virtual model of platform urbanism. The author undertakes a thematic analysis of various themes including platform urbanism, virtual urbanism, XR and AIoT technologies, neurotechnology, and nanobiotechnology. Importantly, the study scrutinizes the challenges and risks posed by these converging technologies in the realm of the Metaverse and their broader implications. These three studies collectively contribute to a comprehensive understanding of the intersection between AI, AIoT, the Metaverse, and XR technologies. While the first and second studies delve into specific applications, the unique contribution of the third study lies in its comprehensive exploration of the underlying technologies and their societal and ethical implications, offering a more holistic perspective on the Metaverse’s multidimensional landscape. This comparative summary underscores the distinctive research objectives and contributions of each study within the Metaverse domain.

Concerning AIoT in industrial and urban settings, Mukhopadhyay et al. [33] emphasize the pivotal role of sensors within IoT systems and their integration with AI, advocating for intelligent, interconnected sensors capable of autonomous decision making and collaborative communication. The authors also highlight the emergence of advanced AI technologies enhancing sensor capabilities, pattern detection, and innovation. Shi et al. [34] explore the convergence of IoT and AI, contrasting knowledge-enabled AI and data-driven AI while exploring their integration across IoT layers—sensing, networking, and applications. Addressing AIoT’s potential to empower IoT, Zhang and Tao [24] present a comprehensive survey showcasing how AI techniques, particularly Deep Learning (DL), augment IoT’s speed, intelligence, sustainability, and safety. The authors discuss AIoT’s architecture across cloud, fog, and edge computing, along with applications and challenges. Mastorakis et al. [35] adopt a broader perspective on the convergence of AI and IoT, covering various AI methods in IoT, research trends, industry needs, and practical implementation. Their work offers a balance between theoretical concepts and real-world applications, serving as a comprehensive resource for researchers and practitioners alike. Collectively, these studies contribute valuable insights into the integration of AI and IoT technologies, with a thorough exploration of AIoT’s applications and implications in specific contexts and the broader IoT landscape. Focusing on specific urban sustainability applications, Bibri et al. [26] conducted a comprehensive systematic review of emerging smarter eco-cities and their AI and AIoT solutions. Through a unified evidence synthesis approach, encompassing configurative, aggregative, and narrative methods, the authors address questions on the core conceptual underpinnings of emerging smarter eco-cities, foundational drivers and enablers, primary AI and AIoT solutions, the role of AI and AIoT technologies in fostering environmental sustainability, and the challenges in their implementation within smarter eco-cities. The findings holistically illuminate the potential of AI and AIoT in promoting sustainable urban development, serving as a crucial resource for policymakers, practitioners, and researchers. By offering insights into successfully applied solutions, the study empowers stakeholders to make informed decisions, implement effective strategies, and prioritize environmental well-being within the framework of smarter eco-cities.

The aforementioned studies have provided fertile and critical insights into the convergence of AI, IoT, and the Metaverse and their varied applications. However, the current study further advances this understanding by comprehensively examining the interplay between XR technologies, AIoT, and the Metaverse in relation to current IoT applications and emerging AIoT applications within the framework of IoCT. By synthesizing diverse perspectives and contributing to the holistic understanding of these technologies’ potential, challenges, and applications, this study underscores their significance in shaping the future landscape of IoCT and paves the way for further research and innovation.

With respect to the first strand of this study, it is worth pointing out that understanding the current challenges in the existing literature is necessary for the research community to design reliable solutions for IoT-based use cases with rich user experience.

Table 1. Summary of existing surveys related to VR, AR, MR, XR, and IoT.

Refs. (Author)	Year Review	VR Review	AR Review	MR Review	XR Review	IoT and Application	Low Cost Solution	End Use Case	Overview
Hu et al. [28]	2021	✓	✗	✗	✗	✓	✗	Enabling technologies	Use cases of VR for IoT and its future research directions.
Lanka et al. [29]	2017	✗	✓	✗	✗	✓	✗	Architecture, and technology networks	Emerging technologies for integrating IoT with AR.
Blanco-Novoa et al. [36]	2020	✗	✓	✓	✗	✓	✗	Smart power socket	Monitoring and evaluating the performance of linking AR and MR through AR/MR glasses.
Pirmagomedov et al. [37]	2019	✗	✓	✓	✗	✗	✗	Augumented Human	Design principles, connectivity issues, and security challenges are addressed.
Alam et al. [38]	2017	✓	✓	✓	✗	✓	✗	Safety system	Simplifies network operations and assists maintenance task in complex environments.
White et al. [39]	2018	✗	✓	✗	✗	✓	✗	context-aware applications	Contextual information to service providers and end users are summarized.
Shafique et al. [40]	2020	✗	✓	✗	✗	✓	✗	5G-IoT	Key enabling technologies for standarizing 5G-enabled IoT.
Andrade et al. [41]	2019	✓	✓	✗	✓	✓	✗	Smart buildings and cities	Translation of IoT data into XR objects using the data communication model.
Makolkina et al. [42]	2017	✓	✓	✗	✓	✓	✗	Vehicular ad hoc network	Maximization of information for the service delivery models.
Minerva et al. [30]	2020	✓	✓	✗	✗	✓	✗	Digital twin	Identifies an extensive set of digital twin features for transformation of physical objects into digital objects.
Carneiro et al. [43]	2018	✓	✓	✗	✗	✓	✗	Road networks	Geographic information systems and building information modelling are integrated with IoT.
Fernandez et al. [44]	2018	✗	✓	✗	✗	✓	✗	Smart Clothing and E-textiles	Business model impacts and requirements of smart IoT-enabled garments.
Guo et al. [45]	2021	✓	✓	✗	✗	✓	✗	6G-enabled massive IoT	Core 6G requirements for IoT and new network architecture to enable massive IoT.
Jo et al. [46]	2016	✗	✓	✗	✗	✓	✗	Scalable AR for object tracking	User tracking, fast recognition, and interactive contents augmentation are explored.
Mylonas et al. [47]	2019	✗	✓	✗	✗	✓	✗	Educational Activities	Design and current status of using AR for education are highlighted.
Norouzi et al. [48]	2019	✓	✓	✗	✗	✓	✗	Disruptive technology	Collective strengths of intelligent virtual agents and IoT are explored.
Cao et al. [49]	2019	✗	✓	✗	✗	✓	✗	Robot Task planning	Interactive task authoring and navigation of robots are studied.
Bacco et al. [50]	2020	✓	✗	✗	✗	✓	✗	Monitoring Ancient Buildings	WSN and real-time data are explored to observe structural patterns.
Simiscuka et al. [51]	2019	✓	✗	✗	✗	✓	✗	Cloud synchronization	Local network testbed and cloud testbed are tested and analyzed.
Our survey	2023	✓	✓	✗	✗	✓	✓	cost-effective VR/AR solutions for IoT applications	Convergence of cost-effective VR/AR devices for potential IoT use cases and their research challenges are explored.

✓—Covered, ✗—Not covered

shows the summary of existing surveys targeted for IoT applications through VR, AR, MR, and XR technologies. In comparison to these, we present an updated review of IoT use cases from ten different streams classified and deployed through the utilization of cost-effective VR/AR modules. Other reviews focus on specific IoT applications or environments of deploying VR/AR technology and none of them focused primarily on economic VR/AR solutions’ targets for the diversified categories of IoCT applications. Finally, to the best of the authors’ knowledge, this is the first review that provides a complete overview of cost-effective VR/AR solutions for IoCT applications classified under different streams.

4. Methodology

This study employs a thematic review method to explore the potential applications, opportunities, and challenges of deploying VR/AR/MR technologies in IoT applications, as well as the emerging paradigm of AIoT and its synergistic interplay with XR technologies within the Metaverse and future IoT applications in the realm of IoCT. A thematic review focuses on identifying and analyzing themes, patterns, and trends within a specific area of research. Instead of providing a comprehensive overview of all available literature on a topic, a thematic review seeks to synthesize and organize existing research around key themes that emerge from the literature [52]. This approach allows researchers to gain a deeper understanding of the concepts, theories, and issues within a specific field by examining how different studies relate to and contribute to these themes. Thematic review is often used to identify gaps in research, highlight areas of consensus or disagreement, and provide insights for future studies. Our thematic review consists of 9 key stages: (1) definition of research objectives and scope, (2) selection of key themes, (3) thorough literature search, (4) selection of documents, (5) data extraction and synthesis, (6) thematic analysis, (7) integration and interpretation, (8) discussion, and (9) conclusion (see Figure 1).

Regarding stages 3 and 4, we followed the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) approach for literature search and selection [53]. Figure 2 shows the four-phase flowchart of the literature search and selection process related to this approach. Among the available pool of academic research databases, SCOPUS and Web of Science were selected given their broad coverage of high-quality peer-reviewed studies related to the topic that meet strict standards for rigor. They are considered to be a more reliable and trustworthy source of academic literature. To retrieve the sought literature, we developed a search string covering the different topics of the study and their links. Accordingly, the search string included “virtual reality”, “augmented reality”, “mixed reality”, “extended reality”, “Metaverse”, “IoT applications”, “AIoT applications”, and “Internet of City Things”. These were used to search against the title, abstract, and keywords of articles to produce initial insights prior to screening, data extraction, synthesis and analysis, and discussion.

As a result, the conducted thematic review serves as a thorough exploration of the existing body of knowledge pertaining to the research questions raised. This comprehensive overview not only establishes the bedrock for subsequent data analysis and synthesis but also discerns existing research gaps. Moreover, it serves as a springboard for delineating unexplored directions warranting future investigation. Ultimately, this endeavor enables the derivation of meaningful insights, the formulation of substantiated conclusions, and the cultivation of novel knowledge within the field.

5. Results

In this section, we delve into the outcomes of our comprehensive review, shedding light on key aspects related to the integration of VR and AR technologies within the framework of the IoCT. The exploration encompasses several significant themes, including the factors influencing the cost of VR and AR solutions, the economic technologies and devices driving these immersive experiences, and the applications of VR and AR technologies in IoCT scenarios. Furthermore, we explore the impacts of XR, AI, IoT, and AIoT technologies, examining their unique characteristics and how they are interconnected. Lastly, we delve into the valuable ways AI and XR technologies work together, showcasing how they can significantly enhance IoT applications and create new possibilities for emerging AIoT scenarios in the extensive realm of IoCT.

5.1. Factors Contributing to the Cost of VR and AR Solutions

5.1.1. VR and AR App Development

The digital virtual environment to be visualized through the headsets is completely developed by VR App developers, which involves proper consideration of the choice of headsets used, tools, rendering engines, and development kits. On the other hand, the same set of tools is used by the AR developers, and in addition, they are also involved in generating additional layers for providing a digital experience for augmenting the digital models in the virtual environment.

5.1.2. VR and AR Hardware

The HMDs used for VR/AR-driven applications present in the current market are bulky and few of them use a wired interface to the PC of the users that runs the supporting software for the VR/AR devices. Such VR/AR headsets require processors with exceptional computation compatibility and a Graphic Processing Unit (GPU) with low power consumption capabilities. Subsequently, the cost factors involved in the design of such headsets also increase.

5.2. Economic Technologies and Devices for VR and AR

In this subsection, we present an overview of several common cost-effective VR and AR devices with a particular focus on the cutting-edge features and applications that have been introduced in recent years. The literature already contains several works in this direction. For instance, Tham et al. [54] compared and evaluated the performance of high-end to low-end VR devices that have been reported in the literature in terms of immersing experience, sensibility, and autonomy. Based on their discussion, the design of professional and technical VR devices and integration for potential deployments were highlighted. Coburn et al. [31], on the other hand, undertake an encyclopedic review of capabilities of cost-effective VR technologies, likely the most comprehensive so far, covering the cost-effective, high-quality VR experiences and enhancements in the hardware design processes.

A comparative analysis of high-cost and cost-effective VR solutions for using IoT in the educational sector is performed by authors in [55]; the authors evaluated the active engagement of students through various delivery mediums, facial emotions, tolerance for ambiguity, competence, and a combination of electromyography (EMG) and galvanic skin response (GSR) signals. Further, this data triangulation provides an immersive and interactive learning experience for students. A similar application was reported in [56], where the authors tested and validated different VR devices considering various scenarios for enhancing the learning experience of students. Further, in [57], various VR applications are summarized, aiming to provide an immersive learning experience for higher education.

The Things to Cloud architecture [58] developed with nine layers provides the most prominent communication protocol between IoT devices and the cloud environment. It helps to integrate IoT devices by employing fog computing in the background for imparting significant low-power, scalable solutions mapped to the developed Things to Cloud architecture. In another interesting application presented in [59], the authors adopted VR, AR, and IoT for library services. It provides an interactive interface for the usage of library services.

From the perspective of hardware and software tools, the market of VR and AR/MR technologies is expected to reach USD 16.3 billion and 8.2 billion, respectively, by the end of 2023 [60]. Figure 3 shows the growth in the hardware and software consumer market of VR, mobile AR, and MR. As apparent from the figure, VR and AR markets increased significantly over the last few years. Mobile AR technology also received great attention, and significant growth in its market is expected in the future.

There are already some efforts towards the development of cost-effective VR and AR solutions [31]. For instance, Ahuja et al. [61] equipped a cost-effective VR headset with motion- and user-sensing capabilities through a pair of hemispherical mirrors and a smartphone camera. There are also some efforts for the deployment of cost-effective VR and AR devices for different applications. For instance, Junior et al. [62] propose a framework allowing cost-effective devices to visualize and interact with photorealistic scenes. Moreover, the need for choosing the appropriate tools to impart the functionalities of VR and AR in IoT devices is also an important focus of the literature [63]. The literature also demonstrates the ability of cost-effective mobile VR and AR solutions in a wide range of IoT applications. Also, there has been substantial research interest, since cost-effective VR/AR devices and services provide potential economical benefits and interactive immersive experiences for IoT use cases.

Some commonly used cost-effective VR and AR devices are described below.

5.2.1. Google Cardboard

The Google Cardboard VR headset uses a smartphone to create the headset. A piece of cardboard can be transformed into a state-of-the-art VR headset. It gives a cost-effective VR experience, supports interactive gaming, and has an immersive viewing experience of the video contents.

GestOnHMD [64] employs gesture-based interaction and gesture classification techniques by leveraging scratching gestures on the surfaces of Google Cardboard VR headsets using smartphones and stereo microphones. In this work, gesture data and acoustic data collected from users are trained using DL models for gesture recognition, detection, and classification.

Cyber-Physical–Social Eco-Society (CPSeS) [65] aims to establish social networking with a blend of real and virtual worlds through Google Cardboard VR, robots, smartphones, IoT, and visual games. This work experimented with a virtual museum scenario for connecting the objects in the museum and the visitors and established social meeting points among them.

Through the digital cloning of gastronomic things and with the support of IoT, AR, and VR technologies, the authors in [66] presented the concept of eGastronomic things. They demonstrated this concept on an ice cream machine as a case study with digital twin technology. It secures the system with the IoT gateway embedded into the data acquisition board and ensures protection against unauthenticated access to the ice cream machine.

From the consumer value perspective, in [67], the authors adopt VR technology to evaluate tourist destinations from the response obtained from the tourists. The obtained value was one of the vital predictions for adopting VR for the estimation of tourist attractions by tourism companies.

5.2.2. Google Daydream View

Similar to Google Cardboard, the Google Daydream View is powered by smartphones. With the ergonomic design and head strap, it could be worn for a longer period. It comes with a miniature motion controller, which lets the user have more interactive visualization. Further, it supports interactive gaming features with a full-fledged experience for the users. The smartphone should launch the Daydream application and the controller pairs with the smartphone through Bluetooth. In [54], the authors considered using Google Daydream View as one of the VR devices for comparing its performance with other equivalent VR devices.

Suen et al. [68] developed VR services for academic libraries to establish interactive experiences for the users. The authors explored site visits and interviews to assess the demands of VR library services. With two practical applications of VR technologies, the authors in [69] used the Google Daydream View VR device. The first one is for providing a virtual online shopping experience for the users with virtual shows and fitting rooms, and salespersons are involved in the scenario. Secondly, it was targeted at brick-and-mortar retailers and provides managerial implications of virtual experiences for adding value to the customers.

5.2.3. Homido VR Headset

The Homido VR Headset is a cheap, small, easy-to-use, and extremely portable headset for mobile-based VR devices. Unlike the Google Cardboard and Daydream View, it does not require any large boxy cardboard structures, straps, or flaps. It will be comfortable for people wearing glasses. The challenge in using the Homido VR headset is that one of the user’s hands needs to hold the smartphone, which may not allow the user to control other features.

In [70], the authors discussed the visualization of the British Waterways network using the Homido VR Headset, which observes the 360-degree visualization of the videos of the journey of the boat in the waterways. Herman et al. [71] focused on cartographic visualization and analyzed and tested its possibilities by using Homido as one of the cost-effective VR headsets. Here, the pilot environment for various orientations of the headset and lighting effects are assessed for cartographic visualization.

Further, the Homido VR Headset helps to provide more authentic learning experiences [72]. It was established in a classroom environment with smartphones considering space and time constraints. The Homido VR Headset also helps to facilitate immersive non-verbal communication in terms of a better interpretation of body language, tones, and facial expressions [73]. In [74], the authors used the Homido VR Headset for establishing coordination among multiple UAVs integrated with a Robot Operating System (ROS)-based virtual environment.

5.2.4. Samsung Gear VR

For an immersive experience with content, including images, videos, and games, the Samsung Gear VR provides the most portable VR experience. It is very lightweight at around 0.62 pounds and provides a more cushioned effect to fit on the face of users. It has a trackpad, which is easier to use, with no tactile sensation while sliding and tapping the surface.

The eGastronomic things implemented in [66] also used Samsung Gear VR as one of the vital VR devices to provide 360-degree visual experiences. Usage of different VR HMDs is investigated for evacuation research applications in [75], where the evaluation is made by assessing the answers of the questionnaire survey to a group of people. In another similar work [57], Samsung Gear VR is used as one of the VR devices for providing higher education with an immersive experience. In another work focusing on construction education [76], authentic learning environments were ensured using Immersive VR (IVR) through various VR devices. In this work, single- and multi-user IVR environments are assessed, and they make this an interactive launch pad for further research in the construction sector. In [67], assessing the customer value experience, the usage of Samsung Gear VR was found to play a supportive role for the users.

VR devices also play a crucial role in surgical training for medical experts. One such work based on Samsung Gear VR was reported in [77], where the authors used it along with IoMT, providing a better Cyber-Physical Framework for orthopedic surgery training applications. In another interesting application presented in [78], the authors promised to provide 360° video streaming and interactive access for HMD clients. Further, this work leverages the usage of edge servers and provides lower network latency during the streaming process. It achieves 62% consumption of bandwidth with a high-quality visual experience, reduced computation overhead, and extended battery life for the device.

5.2.5. Merge VR Goggles

Merge VR Goggles are made of incredibly soft form material and provide good comfort for hands with an adjustable strap for a satisfying fit. With proper ventilation and an additional overhead strap, it has custom adjustable lenses for providing a 360-degree view. With the accelerometer in the smartphone, true interaction with the environment is supported.

Virtual gaming applications using a variety of VR devices are compared by the authors in [79]. In this work, cost-effective alternate VR solutions are compared to select the best device for interactive gaming applications. Further, the authors in [80] evaluated the high-end and low-end VR devices considering the visual experiences achieved through the scores accessed from the participants involved in the study. The

Table 2. A summary of key characteristics of the cost-effective VR and AR devices.

Device	Pros	Cons	Key Features	Cost
Google Cardboard	Very cheap Better lenses Lots of content	No head strap Comfort is decent but not high-end	Durable materials used in high-quality lenses, supports motion tracking and stereoscopic rendering.	USD 10.00
Google Daydream View	Simple High-quality Breathable head strap	Trouble with gap between nose and headset Initial setup process is challenging	Easy to use and comfortable headset, ensures low-latency and high-quality visualization.	USD 10.00
Homido VR	Very cheap Adjustable distance Ergonomic headstrap	Only basic functionalities Need to hold in hand always	Provides 100° Field of View with comfort ergonomics and adjustable lenses adaptable to eyes.	USD 54.00
Samsung Gear VR	High functionality Comfortable Sleek	Price comparatively higher Fingerprints visible on screen	It is a well-designed HMD with responsive controls available at cost-effective price, providing rich app options and ease of use.	USD 29.00
Merge VR Goggles	Soft material Easy to use Works for both IOS and Android	Hard to get a device in and out of the foam material Limited to higher-end mobile phones.	It includes anti-fog ventilation channels, audio ports, camera access for AR, adjustable lenses, and comfort strap.	USD 50.00

shows the summary of key characteristics of the cost-effective VR and AR devices along with its key features.

5.3. VR, AR Technologies for IoCT Applications

In this subsection, we introduce the key features of various IoCT applications driven by VR and AR technology and discuss the related literature in detail.

Next-generation cities could integrate millions of sensors with IoT devices, employed for collecting data from massive infrastructures. IoCT helps to make such infrastructure much smarter, making it more accessible. Smart cities are aimed at making each part of the city smarter through IoT devices and AI and interfacing them all through the cloud to operate seamlessly together. IoT devices could be employed in traffic lights to implement intelligent transport ion systems (ITS), which could assist the drivers in optimizing routes and timing as well as help decrease congestion at peak hours. In addition, waste management through smart garbage collectors and water management in smart cities helps to make a better experience for the citizens and government to be better able to accomplish their goals. Most of the smart city projects focus on the enhancement of the environment, transportation, economy, government, living, and people in those cities.

The work in [81] by Jo et al. provides a survey of AR-enabled IoT for interactive and smart environments. In that work, the authors identified three vital components, namely IoT-guided object tracking, seamless interaction, and object-centric data management, that integrate AR and IoT technology. Badouch et al. [82] highlighted the development of AR for smart city applications and before this conducted a comprehensive review of examples of implementing AR using IoT infrastructure, and synthesized the research challenges and the limiting factors involved. Sanaeipoor et al. [83] have reviewed three international AR-placemaking projects used for smart city applications. In this work, the authors studied the implementation strategies of AR with IoT components for framing layouts and placemaking and summarized the impact of AR infusion with the advances in science and technology. In 2018, Kaji et al. [84] presented a summary of using AR in smart cities focusing on the enhancement of the environment, living style, and transportation. This work summarizes the key aspects of improving tourism and the maintenance of smart cities. The authors in [85] envisaged a town disaster management system using IoCT devices and AR-based smart buildings. Here, they introduced an inter-operable test bed capable of operating across large categories of IoT devices used in smart cities for testing the AR-based disaster management service. Further, the system also provides safe and quick rescue guidelines for the residents in the smart buildings during emergencies.

As illustrated in [86], the accessibility to improve living standards for people with disabilities can be imparted in smart cities through AR and IoT. In this case, the RFID-based technique is integrated with the system for all wheelchair users to gain a rich experience. Further, studies were performed on fourteen wheelchair users and various degrees of impairment were evaluated, and more independence was incorporated into the system. In [87], the authors introduced a 3D visualization technique based on a Microsoft HoloLens AR device to help to understand the planning and management of Toronto City. In this work, the created 3D city model contained attractive user interactions that include geographic data of the city acquired through IoT devices. Improved accessibility for better smart city maintenance was presented in [88], where the authors tagged 2D visual codes on the objects and identified them with dynamic AR markers. This technique renders sensor data to the IoT devices employed and enables visualization through cameras and smartphones.

ARvatar [89], a serious gaming interface based on AR, integrates the real-time environmental data from IoCT devices employed in smart cities. Here, this interactive gaming application provides awareness of the environment in smart cities beyond its core entertaining capability. In [90], a framework dealing with energy management in smart cities based on AR was developed, and this work aims to provide better human–computer interaction. This work finally summarizes the required future energy demands of smart cities.

The enabling technologies for the digital identity of the objects are explored in [91], which allows dependent social identification with the support of big data, IoT, cloud computing, and VR. Digi-log [92], an interactive shopping experience, was implemented using AR, IoT, and pervasive environments. The authors of Digi-log identified three architectural components that include interoperability, object-centric data management, and mechanisms for interacting as well as controlling the environment. In one of the recent works by Singh et al. [93], the authors explored the implementation of green communication technology driven by the IoT, VR, and AR in smart city applications. This work emphasized the optimization of intrusion detection systems (IDSs) that handle the data acquired from the IoT devices employed in smart environments.

5.4. XR, AI, IoT, and AIoT Technologies: Characteristics and Relationships

The evolution of the Metaverse as a feasible and operational computing platform is progressively shaped by advancements in XR technologies. Embracing a seamless integration of immersive technologies, the Metaverse embodies a comprehensive, synchronous, interconnected, interactive, and ever-present virtual–physical realm. Users’ entry into the Metaverse is facilitated by XR technologies, harmonizing virtual and physical realms alongside human–machine interactions, culminating in the emergence of what is known as the post-reality universe. Encompassing all immersive technologies, XR amalgamates experiences to such an extent that users grapple with distinguishing the boundary between reality and virtuality, experiencing a fusion of sensory immersion that encompasses visual, auditory, and even tactile dimensions [26].

VR, as an alternative reality, absorbs users entirely into a digitally simulated environment. It imbues a sense of being transported to a distinct world, where actions mirror real-world behaviors [94]. The VR headset, a head-mounted display, serves as the conduit for user interaction, facilitated through head tracking or tangible control mechanisms [95]. This setup provides a 360-degree panoramic view of the virtual world, effectively deceiving the user’s cognitive faculties. Augmented through modalities such as vision, sound, touch, and movement, users are enveloped in virtual landscapes. The avatars, which users manipulate through movements and gestures, driven by ML/DL models, engage with intricate nuances like facial expressions, body language, eye movement, emotional states, physical interactions, and speech recognition, enabled by AI for precise and swift perception, learning, decision making, and behavioral responses. This confluence in technology aims to reshape the user experience by erasing the boundaries between the tangible and virtual realms, fostering engagement and satisfaction through technological convergence, in line with the aspirations of the Metaverse.

AR introduces a realm where virtual entities are seamlessly integrated with the physical space, marked by users’ perception of these entities coexisting with the real world. AR strives to enrich reality by superimposing digital artifacts and information into the physical environment [96]. This overlay involves a computer-generated layer, encompassing visuals, graphics, video feeds, text, animations, and more, all blended into the tangible world. AR experiences are facilitated through devices like AR glasses, smartphones, screens, and tablets, providing users with interactive insights into their surroundings. The anticipated infusion of AR into VR headsets, featuring pass-through capabilities, further extends this merger by leveraging integrated camera sensors, thereby enhancing the virtual ambiance [97]. With AR’s pervasive integration into daily living environments, the Metaverse’s fusion with urban settings becomes feasible, enabling the projection of digital entities onto physical urban objects [12]. However, the ongoing oversight of sensory elements like smell and haptics in lightweight AR headsets remains an issue.

MR, often synonymous with hybrid reality, amalgamates virtual and real dimensions through digital overlays, crafting a dynamic user experience. MR operates in the interstice of VR and AR, introducing virtual objects into the real world to create an intensified immersion beyond what VR and AR achieve separately. This intricate interplay necessitates specialized headsets or glasses that empower users to interact with digital entities, manipulating, positioning, and interacting with them as they wish. MR encompasses immersive devices for interacting with virtual elements and holographic devices enabling manipulation of physical objects via transparent displays [2]. Going beyond VR and AR, MR’s complexity reflects the evolving technological trajectory, influencing discourse and trends in the field [98]. Worth highlighting is AI’s pivotal role in fueling the XR domain, which is instrumental in providing a seamless MR experience.

The integration of AI, IoT, and Big Data into XR technologies has precipitated the harmonization of formerly disparate virtual environments, owned by diverse platform corporations, into a network of interconnected 3D virtual worlds [4]. Lee et al. [12] advocate for a holistic framework featuring user interactivity, XR, AI, computer vision, blockchain, IoT, edge cloud, wireless networks, and hardware infrastructure as cornerstones for the Metaverse [12]. Multiple studies [11,12,13] delve into the technical aspects of XR-AIoT convergence, blockchain, DT, and 5G/6G networks. The amalgamation of these technologies has the potential to underpin a dependable, always-on 3D platform, facilitating scalable, secure, and realistic virtual environments within the Metaverse. AI’s core role lies in reinforcing the Metaverse’s foundational infrastructure and augmenting its performance. Advanced ML/DL algorithms, incorporated into 5G and 6G systems, tackle spectrum monitoring, resource allocation, channel estimation, traffic off-loading, attack prevention, and network fault detection [2]. Notably, Meta’s introduction of the AI research supercluster (RSC) as a premier AI supercomputer is poised to expedite AI research and the Metaverse’s development, encompassing diverse applications through enhanced DL models [2].

Further amplifying this landscape, AIoT streamlines IoT processes, enhances human–machine interactions, augments data management and analytics, and elevates decision-making prowess [26]. AIoT merges AI’s cognitive capabilities with IoT’s expansive network, addressing challenges rooted in data deluge and intricate processing demands. The synergy of AI, encompassing ML/DL, computer vision, and natural language processing, is harnessed in shaping the Metaverse, culminating in interactive experiences between human users and virtual assistants through text and speech interactions [2]. Within the realm of Metaverse applications, XR-AIoT products and services span diverse everyday life domains. These encompass education, entertainment, leisure, tourism, business, e-commerce, manufacturing, finance, healthcare, medicine, and governance, embodying the breadth of XR-AIoT technologies’ influence [4].

5.5. The Clear Synergies between AI and XR for Advanced IoT and Emerging AIoT Applications within IoCT

The integration of AI and XR technologies holds immense potential to propel the utilization of VR, AR, and MR in future IoT or emerging AIoT applications within the framework of IoCT. This symbiotic convergence enhances the capabilities of immersive experiences, visual analytics, and visualization capabilities [1,3,99] and augments IoT’s intelligence and utility in urban contexts.

From a technical perspective, AI can significantly enrich XR experiences by enabling real-time processing and analysis of vast data streams generated by IoT sensors and devices. This synergy empowers XR systems to offer contextually relevant information, adapt to dynamic environments, and provide personalized interactions. For instance, AI algorithms can process data from smart city sensors to enhance the accuracy of AR overlays, making them more informative and tailored to individual users. The convergence of AI and XR also paves the way for the advancement of existing IoT applications in the context of the IoCT. By employing AI-driven insights obtained from XR-enhanced data, cities can optimize urban infrastructure, resource allocation, and services. ML models can discern patterns from XR data, allowing for predictive analysis related to city assets. Additionally, AI-powered XR can transform city planning and management by simulating and visualizing scenarios, thereby aiding decision-makers in evaluating the potential outcomes of urban interventions before implementation. Moreover, there are many applications of VR, AR, and MR technologies [1,11,32,58,100] within the context of IoCT, where AI can serve as a transformative force, including the following:

• Urban planning and design: VR/AR/MR can be used to create immersive simulations for urban planning, enabling city planners to visualize and optimize infrastructure layouts.
• Tourism and cultural heritage: These technologies can offer virtual tours of historical sites and landmarks, enhancing the tourist experience and preserving cultural heritage.
• Remote maintenance and repairs: Technicians can use AR/MR to access real-time information and instructions while performing maintenance tasks in various urban systems.
• Real estate and property management: VR can offer virtual property tours, while AR can provide real-time property information when viewing physical locations.
• Interactive city navigation: AR applications can provide navigation guidance, points of interest, and real-time information layered onto the user’s view of the city.
• Smart retail and marketing: AR can be used to enhance shopping experiences by providing interactive product information, promotions, and recommendations.
• Healthcare and well-being: VR/AR/MR technologies can support telemedicine, therapy, and healthcare education within urban environments.
• Entertainment and events: AR/MR can offer enhanced experiences during urban events, such as festivals, concerts, and exhibitions.
• Education and learning: VR/AR/MR can bring immersive educational experiences to classrooms and urban learning environments.

XR technologies enrich various aspects of urban life by seamlessly integrating digital information with the physical environment. Furthermore, the integration of AI and XR catalyzes the development and deployment of AIoT applications, where AI-driven insights are seamlessly woven into XR experiences. Imagine an AR-based navigation system that not only guides pedestrians but also utilizes AI algorithms to suggest efficient routes based on real-time traffic conditions and personal preferences. This integration transforms XR devices into intelligent companions, enhancing user engagement. From another perspective, Rahi et al. [32] delves into the synergy between AIoT and the Metaverse to enhance mental healthcare through VR and related techniques. The authors spotlight brain and mental healthcare technologies, potential applications, and associated research. The outcome of their study will yield policy recommendations for AIoT-driven standards in brain and mental healthcare, poised to shape the future landscape. In essence, the fusion of AI and XR technologies amplifies the potential of VR, AR, and MR in future IoT applications within IoCT. This synergy empowers cities to harness data-driven insights and enhance urban experiences, ultimately leading to the realization of advanced AIoT applications that leverage XR’s immersive capabilities and AI’s cognitive prowess for the betterment of urban living.

In addition, the integration of AI and XR technologies presents an intriguing avenue for boosting the efficiency and cost-effectiveness of XR reality technologies in advanced IoT and emerging AIoT applications within IoCT. By leveraging AI’s data analytics capabilities, XR devices can intelligently adapt and optimize their functionalities based on real-time environmental conditions and user behaviors. For instance, AI algorithms can predict the optimal times for deploying AR overlays to provide crucial information, thereby minimizing energy consumption and maximizing user engagement. Also, AI-driven predictive maintenance can extend the lifespan of XR hardware by identifying potential issues before they escalate, ensuring longer operational periods and reducing replacement costs. As a result, the synergy between AI and XR not only enhances the value proposition of XR technologies but also elevates their cost-effectiveness in delivering advanced IoT and AIoT solutions within the dynamic realm of IoCT.

With respect to the Metaverse, Soto and Leon [101] explore the impact of AI on its evolution and its potential to enhance user immersion in virtual environments. The authors delve into key technology areas, encompassing NLP, computer vision, blockchain, networks, DL, and neural interfaces. They further examine diverse application domains, ranging from smart cities and healthcare to manufacturing. Through AI-driven solutions, the research underscores how AI can bolster system design, enrich virtual world services, and elevate 3D immersive experiences. The study concludes by evaluating prominent Metaverse projects integrating AI to enhance services and foster Metaverse ecosystems. Within the framework of AIoT, AIoT-driven XR technologies enhance the realism and interactivity of virtual experiences in the Metaverse. By integrating IoT devices and AI models with XR, AIoT enriches sensory input, allowing users to experience virtual environments more authentically [1]. For instance, AIoT can dynamically adjust XR simulations based on real-world data, altering lighting, sound, and environmental conditions to match users’ surroundings. This creates a more immersive and believable experience, blurring the lines between the physical and virtual realms.

The benefits of AIoT-driven XR technologies extend to current IoT applications within IoCT. In terms of visualizations, AIoT enhances data representation and understanding. Complex city data can be visualized through XR interfaces, leveraging AI to transform raw data into intuitive, interactive 3D models. This enables urban planners and decision-makers to gain holistic insights into city dynamics, making informed choices for urban development and resource allocation. Moreover, AIoT-driven XR technologies optimize resource utilization within IoCT. AI algorithms analyze data from IoT sensors and devices, enabling XR platforms to adapt to real-time conditions. This dynamic responsiveness translates to energy-efficient usage of IoT resources. For instance, smart lighting controlled by AIoT-driven XR interfaces can adjust brightness based on occupancy patterns, contributing to energy savings in urban environments. Another dimension is the enhancement of situational awareness and emergency response. AIoT-driven XR technologies can combine real-time sensor data with AI-powered analytics to simulate crisis scenarios in XR, aiding in preparation. During emergencies, XR visualizations can overlay critical information onto real-world views, facilitating more effective decision making and responses. At the practical level, VR/AR/MR technologies can enhance various AIoT applications pertaining to smart cities and environmental sustainability. Bibri et al. [26] provide a comprehensive systematic review on smarter eco-cities and their leading-edge AIoT applications for environmental sustainability in the context of urban planning and management, including energy conservation and renewable energy, sustainable transportation and mobility, traffic management, water management and conservation, waste management for efficient resource utilization, biodiversity and ecosystem services, pollution control, climate change adaptation and mitigation, climate monitoring and early warning systems, and disaster resilience and management. Accordingly, the authors identify various benefits and opportunities that AI and AIoT technologies offer in promoting environmental sustainability, including optimized resource management, increased energy efficiency, improved waste management, enhanced transportation and mobility management, reduced environmental impacts, increased resilience to environmental challenges, and improved decision making in urban management and planning.

6. Discussion: Challenges, Open Issues, and Future Research Directions

The preceding sections of this comprehensive review have illuminated the potential of VR/AR technologies in addressing various challenges across diverse domains of IoT applications. These technologies have showcased their capability to enhance interactions and experiences. As we transition from envisioning to implementing these advancements, the nuances of hardware selection, software compatibility, and computation resources come to the fore with respect to IoCT applications. However, as we delve deeper, it becomes evident that a new frontier is emerging at the intersection of AIoT and XR technologies within the Metaverse. In this evolving landscape lies a spectrum of uncharted territories and unresolved queries. The integration of AIoT and XR technologies offers exciting prospects, yet it also presents intricate challenges within the context of IoCT applications. In this section, we embark on a dual journey. Initially, we explore the nascent areas of concern from the vantage point of AR/VR/MR as well as the integration of XR and AIoT, examining how they apply to the realization and advancement of IoCT applications within different contexts. Subsequently, we illuminate pathways for future research that hold the potential to bridge the current gaps and lead us into unexplored horizons specific to smart cities.

6.1. XR-Enabled Virtual Digital Models in IoCT Applications

6.1.1. Open Issues

• Economic Impact of Hardware for VR/AR: Despite the recent advancement in hardware technology with miniaturization targeted for mobile devices, IoT, data collection with sensors, and other hands-on interactive applications, there are still significant challenges that need to be addressed to mature this technology. Apart from the significant internal characteristics of VR/AR devices, the cost involved in deploying them for IoT applications imposes a challenge for the developers. In the following, we highlight these challenges.

  -

  The massive impact on the cost factors involved in VR/AR devices that pose a great challenge is especially due to the branding of the devices.

  -

  The integration of powerful hardware before the consumer market is ready for its incorporation poses the challenge of the increased price of VR/AR modules.

  -

  As the developments of wearable, headsets, and accessories for VR/AR are labor-intensive to develop, it raises challenges for an increase in the cost of the product.

  -

  Finally, a main challenge for the developers is to integrate optical mechanics with the devices to provide an immersive experience, which involves a huge investment.

• Availability of useful Content: As the world of VR/AR solutions is advancing at a faster rate than ever before, the technological advances, by converging their end use cases with IoT, try to keep on adding more vital and critical data (e.g., healthcare, industrial data, transport, smart cities, etc.) apart from gaming and entertainment uses. For instance, VR-based treatments help to overcome fear and phobia, thereby assisting the treatment of mental health issues such as anxiety and depression. Such crucial data from IoT devices can be ethically collected and analyzed to deliver even better experiences. In a few cases, the volume of useful content collected from IoT devices could also drive personalized experiences for the users. However, VR/AR devices need to cope with the technological advances for handling the threats towards crucial content to come up with improved, standardized, and proactive IoT applications.
• Computational Resources: Processing of IoCT data at the VR/AR nodes requires more resources and for performing data analytics, consideration of the processing power, memory, and power requirements are vital aspects. However, from the practical perspective, the IoT and VR/AR devices will have to pay substantially more for processing the data. The sustained delay in communication, pre-processing, and excessive energy consumption, mostly due to the existing form factor and footprint of IoT devices, needs to be extended well beyond the existing needs of IoT applications to suit the integration of VR/AR devices. Hence, an open issue is to maintain a balance between computation resources and performances, so that the IoT devices and VR/AR modules do not consume too many resources and are capable of providing a better increase in performance.
• Display and Power consumption issues: Intuitively, the richer the demand for interactive user experience from VR/AR devices, obviously the higher the processing power and the power required for driving the display units. Although the power consumption for integrating and visualizing data from simple home appliance IoCT devices is relatively low, for healthcare, manufacturing, and gaming applications it is relatively difficult to manage the power consumption of VR/AR devices. Advances in the optical features and near-eye display systems for VR/AR displays demand more power consumption [102]. However, it drives towards addressing the potential shortcomings of VR/AR devices, and the development of recent innovations in optical display holds the base for deploying XR displays.
• Cyber-security: Ensuring security and privacy in IoT data is a major concern in many applications since the IoCT data need to be analyzed and thus can be presented for better visualization through VR/AR devices. While data exploitation is happening in many IoT applications, those exploitation techniques may anonymously have vital data acquired from the IoCT devices. Moreover, VR/AR devices are also subjected to malicious attacks, which in turn may affect the functional and non-functional requirements of the IoCT devices [103]. Apart from them, the following challenging issues need to be solved when using VR/AR devices to ensure cybersecurity.
• Mobility: The lack of mobility of VR/AR devices due to cords attached to the devices for being connected to PC or large machines. When they are focused on IoCT applications they lead to a major hurdle for incorporating them for gaining a dynamic 360-degree immersive experience in the environment. Moreover, when multiple users are sharing a single design space for visualizing the data, the presence of cords leads to an annoying experience and also raises safety hazards.
• High Speed Connection: High-speed processing of IoCT data is necessary for most of the applications with the best performance in different environments. For example, in healthcare applications, numerous medical devices, the smart wearable can send their data such as patient healthcare data, their body parameters, etc., [104] to nearby edge devices, and eventually, reach the cloud servers and healthcare professionals for better interactive visualization using VR/AR devices. The high-speed processing for enhanced performance of VR/AR devices in IoCT applications is a challenging task for the following reasons.

6.1.2. Future Research Directions

Although cost-effective VR/AR solutions are vividly portrayed, there are still many uncertainties and unexplored areas. The following section addresses the most substantial ones among them.

• Edge Computing: One remarkable part of handling IoCT data comes from the usage of edge computing. Investigation of efficient ways of utilizing IoCT data in conjugation with edge computing platforms is a way to come up with better and more interactive services for VR/AR-driven IoCT applications. In [105], the implication of AI approaches on multiple-access edge computing frameworks is assessed on the data acquired from IoT devices using 5G services. Further, here AR/VR technology is deployed for real-time reporting and monitoring of the data. However, the estimation of content distribution among the set of edge devices in the network is required. Also, the hierarchical caching architecture for managing the data from massive heterogeneous IoCT data from edge computing scenarios needs attention. Hence, a more effective way of edge computing and its enabling frameworks are required to ensure that the performance of the VR/AR devices for IoCT applications is more consistent during their practical deployment.
• 5G Private Networks: With the deployment of high-speed transmission of IoT data for driving high-speed computation and data analytics, 5G private networks could provision to handle the stream of data to the processing units. This network lies in between the mobile core network and the base station for providing transparent inline services. More flexible and diverse services could be supported for IoT devices in both real-time and online scenarios. Despite the streaming nature of IoT data and its huge volume, sequencing the transmission by 5G services requires awareness of the IoT data acquired from the environment. Recently, the 5G Intelligent A+ network based on the 5G Private Networks has been introduced and adopted in edge computing solutions [106], which has shown to be an effective and flexible solution. However, the issues targeted for IoT network access management, and bandwidth allocation on-demand from IoCT systems depend on many design factors and resources. This will help the IoT systems to establish high-speed secure communication and consequently increase the level of dependability on the 5G private networks.
• Trade-off between Display quality and Cost: As we rely more on the sophisticated user experience of the VR/AR devices in IoCT end applications, the need for mechanisms to ensure the trade-off between the economic aspects of the VR/AR devices and their display quality becomes more crucial. Zhan et al. [107] reported the presence of ghost images present in VR displays due to chromatic aberration. By pre-processing of images, chromatic aberration can be reduced in the images but at the cost of excess processing time, power consumption, and memory space. In [102], the advances in display technologies, optical elements, signal processing, and VR/AR devices may increase the cost of the products. However, more investigation is required to maintain the balance between the cost involved and the quality of the display in VR/AR devices.
• Mitigate VR Training Costs: The usage of VR/AR is a promising step that can assist in training employees, athletes, students, etc., and it reduces the training costs for hard-reaching services or in unavoidable situations. Numerous factors are involved to reduce the costs involved in VR training, which are based on the type of training, the start point of training, the filming site, and the computer-generated environment. Another way of mitigating the VR training costs is by partnering with the existing data repository sources. VR environment built using the blueprint data eliminates the need of building the framework from scratch. Also, by proper way of filming using 360-degree cameras, the raw footage could be effectively used for post-production of the visualization videos. Collaborative development from the data streaming IoCT application developers and the design team of VR/AR applications could consider the trade-offs involved to subsequently reduce the investments in training.

6.1.3. Insights and Lessons Learned

At their core, few commercial variants of VR/AR devices are capable of providing cost-effective solutions for IoT applications. In contrast to high-cost VR/AR solutions, the inherent features of a few cost-effective VR/AR devices have minimum processing capability and less form factor. However, simply using them for low-processing-capability IoCT applications makes them a viable solution. Therefore, for mission-critical IoCT applications, there is room for development in terms of providing cost-effective VR/AR solutions. Moreover, from recent research efforts, we also gain insights on the consideration of vital contents for the applications, managing effective computation resources with reduced power consumption. Also, this section outlines the recent research efforts carried out in leveraging to provide mobility, high-speed connection, and security of IoCT devices. In addition to this, many VR/AR-based IoCT application domains demand rich virtual experience in a time-critical manner, therefore providing cost-effective VR/AR solutions considering the constraints and capabilities of IoT devices are essential.

6.2. The Synergy of AIoT and XR within the Metaverse for Future IoT or Emerging AIoT Applications

The challenges and open issues discussed here have been distilled from a diverse array of studies that collectively address the realm of AIoT, the seamless integration of AIoT and XR technologies, and AIoT applications for smart cities and other domains [1,2,3,32,108]. These insights have been synthesized from a wide spectrum of research endeavors, each delving into distinct facets of this interdisciplinary convergence. By drawing from these multifaceted studies, we gain a comprehensive understanding of the complexities and opportunities inherent to the fusion of AIoT, XR, and IoT/AIoT applications within IoCT, contributing to a holistic view of the intricate landscape that lies at the intersection of these transformative domains. Next, we focus on those open issues that are more relevant to this study.

6.2.1. Open Issues

• Data fusion and processing: The fusion of heterogeneous data sources within AIoT poses challenges in terms of real-time processing and integration. Addressing the intricacies of combining data from IoT sensors, AI algorithms, and XR interfaces is imperative for achieving seamless interactions within the Metaverse.
• Contextual intelligence: The effective integration of AIoT and XR requires contextual awareness, where real-time environmental data interacts with XR interfaces. Developing algorithms that leverage AI to enhance XR experiences based on real-time context remains an open challenge.
• Privacy and security: As AIoT-driven XR experiences capture and process vast amounts of personal and environmental data, ensuring user privacy and data security becomes paramount. Developing robust protocols and frameworks to safeguard user information within the Metaverse is crucial.
• Environmental impact: The ecological footprint of AIoT and XR technologies, including energy consumption and electronic waste, is a growing concern. Minimizing the environmental costs associated with these technologies is a critical consideration for sustainable development.

6.2.2. Future Research Directions

• Dynamic XR environments: Exploring the potential of AIoT to dynamically adapt XR environments based on real-time data feeds holds promise. Research into AI algorithms that modify XR elements to suit changing environmental conditions could lead to more immersive and contextually relevant experiences.
• Cognitive XR interfaces: Investigating AI-powered XR interfaces that understand user intentions and adapt interactions accordingly is a captivating avenue. The development of AI models capable of interpreting user gestures, emotions, and context can enhance user engagement and immersion.
• Ethical AIoT-XR integration: Delving into the ethical implications of AIoT-XR integration within the Metaverse is vital. Research can focus on establishing guidelines for responsible data usage, privacy protection, equitable access, and mitigating the environmental impact.

6.2.3. Insights and Lessons Learned

• Enhanced user experiences: The amalgamation of AIoT and XR enriches user experiences by tailoring content to a real-time context. This fusion enables users to interact with data-rich environments intuitively, fostering deeper engagement and understanding.
• Dynamic IoT applications: The integration of AIoT with XR expands the horizons of IoT applications within smart cities. Dynamic data visualization, real-time analytics, and immersive interfaces redefine how urban data are harnessed for enhanced decision making and resource management.
• Sustainable development: The iterative development of AIoT-XR integration emphasizes the importance of sustainable practices. Balancing technological innovation with environmental responsibility will play a pivotal role in ensuring the long-term benefits of these advancements.

In sum, the symbiotic relationship between AIoT and XR within the Metaverse presents a novel landscape for IoT applications in urban environments. Addressing open challenges, charting new research directions, embracing insights from this integration journey, and mitigating the environmental impact will collectively drive the realization of a transformative IoT ecosystem that seamlessly intertwines the virtual and physical realms of the connected city while fostering sustainability.

7. Conclusions

This comprehensive review encompasses two primary areas of investigation. Firstly, the study examined the integration of VR, AR, and MR technologies into IoT applications within the broader context of IoCT. The review emphasizes the positive impact of VR/AR on human lives through IoT use cases in smart cities, where these technologies allow for interactive visualization and interpretation of raw data from IoCT devices. It delves into cost-effective VR/AR solutions, highlighting key concepts, vital features of devices, contributions in various application domains, and challenges. Additionally, the review explored the harmonization of AIoT and XR technologies within the evolving Metaverse, underlining their transformative potential for enhancing user experiences, urban efficiency, and the future of IoT applications within IoCT. The review stresses the need for interdisciplinary collaboration and customization, outlining future research prospects and the exciting possibilities of XR in the IoCT landscape.

In addressing the first area, the study identifies the crucial role of virtual digital models in IoCT applications, accentuating their importance in enhancing visualization and interaction. Extended XR variants are recognized for their continuum of experiences that blend physical and digital realms, enriching IoCT applications. The research explores successful IoCT applications achieved through XR integration, ranging from industrial scenarios to healthcare applications, while also discussing challenges such as data integration and user-centric security and privacy concerns.

The second focal point involves the amalgamation of AIoT and XR in the Metaverse, enabling unprecedented realism and dynamism, enhancing user engagement, and revolutionizing urban experiences. The integration is seen as a transformative force with substantial advantages for AIoT-driven XR applications within IoCT. The review also highlights the potential of XR technologies, advocates for interdisciplinary collaboration, underscores the need for customization, and envisions a future where the seamless convergence of AIoT and XR reshapes urban living. Ultimately, the study provides valuable insights for future research, emphasizing the role of technology, user-centered design, and strategic implementation in shaping the transformative potential of XR in the IoCT landscape.

In light of these findings, this study underscores several implications for both researchers and practitioners. First, it highlights the immense potential of XR technologies in amplifying the capabilities of IoCT applications, offering new avenues for innovation and user engagement. Second, it underscores the need for interdisciplinary collaboration between experts in XR, IoT, and related fields to overcome the multifaceted challenges associated with the seamless integration of these technologies. Third, the study prompts the research community to delve deeper into the customization of virtual digital models to ensure that XR experiences align with the specific requirements of diverse IoCT applications. It will also help the readers to understand the state-of-the-art features of VR/AR devices and their cost-effective variants which meet the demands of IoCT applications, as well as list the future research prospects covering some promising directions in the field.

As the boundaries between physical and digital realms continue to blur, the marriage of XR and IoCT presents an exciting trajectory of possibilities. The insights provided by this study pave the way for future research endeavors that can further refine the integration of immersive technologies within the intricate fabric of IoCT. Ultimately, it is the synergy between technological innovation, user-centered design, and strategic implementation that will shape the transformative potential of XR in the IoCT landscape.

Furthermore, the seamless convergence of AIoT and XR within the Metaverse paves the way for innovative solutions that can revolutionize urban experiences, transcending physical constraints. As cities embrace smart technologies, the synthesis of AIoT and XR opens avenues for efficient urban management, enhanced user participation, and novel applications across various domains. This study underscores the significance of continued research and development in this direction, which will be instrumental in shaping the future of urban living within the context of the IoCT.