Designing an Educational Metaverse: A Case Study of NTUniverse

Abstract

An up-and-coming concept that seeks to transform how students learn about and study complex systems, as well as how industrial workers are trained, metaverse technology is characterized in this context by its use in virtual simulation and analysis. In this work, a virtual environment is created that duplicates real-world situations and enables immersive and interactive learning in the educational metaverse. For this purpose, we built a digital twin of the Nanyang Technological University (NTU) campus as a foundation, called NTUniverse. It is designed as an educational metaverse in which various academic and analytical applications are digitized as 3D content embedded within this virtual campus. The approach to digitally twinning educational systems and embedding them within virtual campuses enables remote and collaborative learning as well as professional technical skills training. It also makes feasible the analysis of abstract concepts, complicated structures, dynamic processes, and sensitive industrial procedures virtually, which is otherwise challenging if not impossible to perform in the real world. The work offers important insights into the behaviors and interactions of systems in the metaverse by evaluating design choices and user interests. NTUniverse is an attempt to explore a novel approach that addresses remote education and training challenges. Three efforts with NTUniverse will be discussed in this work, including (1) digitalization of the NTU campus; (2) campus train modelling and simulation; and (3) science, technology, engineering and mathematics education.

1. Introduction

The term metaverse technology describes a virtual environment within which users can engage in a variety of activities, including socializing, learning, gaming, and content creation while interacting with one another through avatars. Recent years have witnessed the substantial progress of metaverse and digital twin technologies and their applications in various areas, such as education in different contexts, including academia and industry [1], cultural heritage [2], the building and construction industry [3], social media and gaming [4], etc.

In comparison to conventional approaches, the use of metaverse technology for education and industrial training by companies to improve the skills, knowledge, and competencies of employees relevant to their job activities has grown in popularity recently. The metaverse is an ideal context for the development of educational and skill-development materials for two reasons. Firstly, its collaborative approach allows for remote learning and the assimilation of skills without geographical or space constraints. For instance, in the context of industry, the metaverse enables workers undergoing training from several industrial units to interact concurrently in shared virtual worlds, even when these facilities are geographically separated. Secondly, its highly visual nature substantially reduces the amount of time required to visualize, interpret, and understand challenging topics not limited to abstract concepts, complex structures, and dynamic processes [5]. Students and workforce members can “dive into” a secured virtual environment with the aid of a digital twin, practicing and learning theories at an accelerated rate with lowered barriers to entry that may otherwise be raised by physical limitations. This approach also makes it feasible to analyze complex systems and processes virtually in 3D, which would be challenging, if not impossible, in the real world. For example, industrial orientation and onboarding, as well as technical skills training, aim to ensure that personnel understand industrial procedures and settings, and machinery operation and maintenance, among other considerations. While traditional training methods rely on physical facilities and equipment, the use of the metaverse enables a wide range of training and assessment relevant to factory operations, including setting/guidance, sorting, picking, keeping, assembling, and installation. These applications are independent of physical facilities, reducing facility investment, economic losses, and potential equipment damage. Without this type of training, training the workforce would disrupt productivity. The metaverse can also benefit safety training. Worker safety entails having the abilities and tools required to assess workplace dangers, such as safety regulations, emergency protocols, and information on the effective use of personal protective equipment (PPE). The immersive and metaverse-based training experience enables users to practice, for an unlimited number of times, how to operate important industrial equipment virtually, reducing potential hazards when users operate the real equipment. It reduces the possibility of destroying important industrial equipment due to the lack of experience. Using the metaverse to train operators on robots and heavy machinery in a virtual setting, for example, allows for training without the risk of physical harm. As a result, multiple failures can be simulated, including emergency scenarios that do not occur in normal day-to-day operations, preventing losses, hazards to health, and asset destruction. However, it is essential to evaluate both the benefits and the potential limitations of metaverse training. While immersive training effectively teaches a range of skills relevant to emergency responses, there is a concern that it might not fully represent the real-world consequences of dangerous situations, a gap that could lead to trainees underestimating risks in real-life scenarios. Despite this, research supports the idea of immersive training’s effectiveness and its ability to generalize skills from virtual to real settings, confirming its value and applicability across diverse professional domains [6].

Previously implemented designs of digital twins and applications of the metaverse in the healthcare industry and medical training are well reported; these aim to transform healthcare domains [7], including breast cancer monitoring and the training of new doctors [8]. Designs based on the Unity 3D development engine, which are used in a variety of industry verticals including automotive, manufacturing, energy, and aerospace [9], are discussed in terms of different applications, such as smart cities [10], robotics control [11], and guided tour [12].

Online and remote education and training have maintained their popularity in colleges and industrial organizations since the COVID-19 pandemic. Embodying education within the metaverse to provide an immersive learning and skill-development environment offers many advantages, such as effective collaboration with other learners and augmented training experiences. The use of virtual digitalization and simulations in a shared 3D virtual space in real time for educational and tutoring purposes can improve workflow efficiency and reduce testing costs. Conventional approaches that bear little physical resemblance to the hazardous environment are frequently employed, with the result that the sensory data obtained is insufficient to generate the impression that one is truly there. Because metaverse and digital-twin technologies can recreate complex representations of the work environment which would be challenging, costly, or dangerous to replicate in the real world, they can achieve strong physical fidelity and effectively place trainee workers in a highly realistic scenario. Knowledge transmission can therefore be enhanced via training simulations that make use of these technologies. Therefore, building a digital campus or industrial site can potentially maximize the effectiveness of knowledge acquisition and production capabilities, generating better learning outcomes, while limiting the downside in terms of the equipment costs for initial installation, and maintenance needs.

The objective of this research is to explore the implementation of a practical use case in education and training for the metaverse and digital twin technology; this implementation is powered by the Omniverse platform developed by NVIDIA, which is utilized by enterprises like Amazon Robotics, whose intricate operations necessitate over 500,000 mobile drive robots to support warehouse logistics, and PepsiCo, whose digital twin improves supply-chain efficiency and environmental sustainability [13]. NTUniverse is a campus digital twin of Nanyang Technological University (NTU) in Singapore, which is constructed mostly according to the real-world campus settings. A future train transportation system on campus has also been designed virtually [14]. As the base of the educational metaverse, NTUniverse provides immersive and interactive learning activities in NTU.

The remaining part of this paper is organized as follows. The overall process of the system design and methodology of NTUniverse is presented in Section 2. Section 3 focuses on the digitalization of the NTU campus. Section 4 discusses the design of the future campus-train system. The application of NTUniverse for virtual education is shared in Section 5. Section 6 concludes this paper.

2. Overall System Design

2.1. System Architecture

NTUniverse, a digital twin of the NTU campus, is designed to support collaborative and virtual learning. Overall, NTUniverse is for educational use, and currently consists of three modules, namely, Design Education, Heritage Education, and STEM (Science, Technology, Engineering, and Mathematics) Education (shown in Figure 1). For this purpose, efforts are made as to (1) the digitalization of the NTU campus, referencing the real-world NTU campus; (2) the design and modelling of the future campus-train–transportation system; and (3) virtual education using NTUniverse. NTUniverse is built on top of NVIDIA Omniverse, a platform that unifies disparate 3D workflows and data sources into one comprehensive interactive 3D environment.

To explore the incentives relevant to utilizing the metaverse, we first develop the System Design Architecture (SDA), as shown in Figure 2. This SDA can serve as a blueprint for the system structure, components, and interactions. Universal Scene Description (USD) serves as a foundation of the Omniverse architecture, making possible smooth integration and interoperability across many platforms within the USD format.

2.2. Design Tools

To support collaboration, simulation, and real-time 3D graphics design, the NVIDIA Omniverse platform provides a collection of workflow-focused foundation applications, e.g., Omniverse Create/USD Composer for scene composition, and Isaac Sim for robotics, etc. [15]. Omniverse enables collaboration across various development tools by different users. Universal Scene Description, a file format first introduced by Pixar and now adopted natively by Omniverse, is an extensible 3D interchange standard utilized for the sharing of 3D data between various applications and tools. USD allows 3D models from other software, such as Unreal Engine (version 5) or Unity 3D (version 2022.3.0), to be integrated into Omniverse. Nucleus is the database and collaboration engine in the Omniverse platform. It offers storage and database capabilities, and as a collaboration engine, it helps users connect with multiple apps and allows them to establish bidirectional file connections between Omniverse and their custom application via Connectors [13].

A user interface for accessing the virtual environment rendered on the Omniverse platform and viewing project-related data is provided via the NTUniverse webpage, which is created using React. This online interface improves accessibility and offers users a central location to communicate with the platform and receive pertinent information. By fostering interoperability, scalability, and seamless collaboration, this system design approach enables users to take advantage of their favorite tools and collaborate effectively when building and exploring virtual environments within Omniverse. The SDA for NTUniverse is enabled in the Omniverse architecture for visualization and analytical applications. Real-time physics simulations and visualizations are available for the creation and modification of intricate models based on USD across different platforms.

SolidWorks (version 2023) is a popular tool widely used in design, and it forms the basis of our campus’s CAD model. Currently, there is no connector between SolidWorks and Omniverse; thus, we designed our campus model in SolidWorks and converted the data into a 3D data format (OBJ file type) before importing it into Omniverse as a unified USD file.

The first module to build is the NTU Campus Digitalization. This involves constructing a photorealistic digital twin of the physical NTU campus, with a focus on improvements in campus navigational accuracy and efficiency. In this construction, 3D directional signs and navigational aids are strategically created and placed to outline locations of significant structures and key installations. This increases the overall accessibility of NTUniverse.

The second module to construct is the design and modeling of the future campus-train system that is currently under construction. The 3D models of trains, train stations, and tracks are incorporated into NVIDIA Omniverse for dynamic simulations. Both first-person and third-person views are available for the train system in NTUniverse. Possible applications include the analysis and behavior testing of the campus train models. From a system design standpoint, Omniverse USD Composer provides useful scene composition tools for VR and 3D applications, including simulation of physics-based effects. Furthermore, action graphs and PhysX are features that enable users to simulate and view realistic physical activities in the virtual world using push subgraphs to produce physics simulations [16].

The third module is for Virtual Heritage, Design, and STEM education. Action Graphs can be applied to streamline the experiment process, providing an immersive and educational experience. Figure 3 summarizes the design and development of NTUniverse.

3. Digitalization of NTU Campus

From an overarching point of view, we aim to establish the NTU campus as the foundation and base of our research, as which it will be expanded with various additional objectives. The first step in this project would be to create the digital twin of the NTU campus by using the Omniverse platform. This would involve planning and designing the digital twin accurately based on real-world conditions and the parameters of the NTU campus. With NTUniverse virtually created, the project can move on to the next step, namely, adding the virtual future campus-train system.

3.1. Campus Modeling

Covering about 200 hectares of land, the main campus of NTU hosts the College of Engineering; College of Science; Nanyang Business School; Lee Kong Chian School of Medicine; National Institute of Education; and the College of Humanities, Arts, and Social Sciences. There are 23 Halls of Residence on the NTU campus. As such, NTUniverse has a large number of schools and halls to be modelled.

Various software tools are used in research on 3D modelling, such as SolidWorks and Maya, among others; these are used to elaborate the details of buildings, and to provide texture, labels, models, etc. These models are then imported to Omniverse after USD formatting. An alternative solution has been developed based on the Scan-to-BIM technology, in which laser scanners or cameras are used to digitally capture the physical space in the form of a 3D point cloud, followed by the creation of a building-information-models (BIM) model [17]. AI technology can then be applied to reconstruct the 3D models in the BIM format from the point cloud. In general, modeling of NTUniverse is primarily implemented using two methods: (i) 3D modeling for the external aspects of entire existing buildings, and (ii) Scan-to-BIM conversion for individual classrooms or laboratories, from laser scanned point clouds. For those structures or objects not yet in physical existence, 3D design is employed to create them (e.g., the future train system of the NTU campus).

The modeling flow is described as follows: (1) modeling (FBX file) of buildings on the NTU campus or reconstruction of classrooms or laboratories via LiDAR scanning; (2) compiling the models for direct import into Omniverse (USD file); (3) exploring the as-is state of the NTU campus via Omniverse navigation features, i.e., panning and rotation of camera angles; (4) separating or ungrouping layers of objects, e.g., building components; (5) improving texture of objects, features, and buildings in the campus model based on their respective materials, e.g., roof tiles, aluminum cast, and bricks; (6) importing botany models across the campus as generic greeneries and adding additional shapes for better replication accuracy, i.e., missing building objects/components, texture maps, and geometry; (7) creating labels for key/important buildings within the campus, consisting of naming boards created using USD Composer, and the text on the naming boards using the Omniverse Extension “3D Text”; and (8) creating interactive avatars augmented with motion capabilities that simulate real-world scenarios. Figure 4 shows the created digital twin of the entire NTU campus in NTUniverse, and Figure 5 illustrates two selected buildings modelled in NTUniverse.

3.2. Classroom Modeling

Classrooms in NTUniverse can be modelled using design software like SolidWorks and Unity. Interactive avatars are created with augmented motion capabilities. Each student can have their personalized avatar for immersive and interactive learning use (Figure 6). A virtual classroom is also designed to accommodate both avatars and video-based participation of students, in a hybrid manner (Figure 7).

3.3. Reconstruction of Laboratory

Digitalization of the existing laboratories, classrooms, tutorial rooms etc., can also be performed via laser scanning. Point-cloud representations of the laser-scanned internal spaces can then be converted to BIM. BIM is widely used in digital-twin construction. Figure 8 shows the reconstruction of a laboratory in NTU. A Faro camera shown in Figure 8a is used to scan the laboratory, generating a point cloud as shown in Figure 8b, which is then converted to a BIM-based digital twin with reconstructed structure and mechanical or electrical appurtenances as shown in Figure 8c. The BIM model so reconstructed can be used for energy simulation, indoor climate management, and other sustainability studies [18].

4. Design of Future Campus-Train System

Currently, the future train system on the NTU campus is under construction and is expected to be completed in 2029. It will be connected to the Mass Rapid Transit (MRT) network of Singapore. There will be three MRT stations crossing the NTU campus: JW3—Nanyang Gateway, JW4—Nanyang Crescent, and JW5—Peng Kang Hill [14].

As there are no publicly available blueprints of the future campus-train, in this module, the design and modelling for NTUniverse are only based on images of artists’ impressions depicting the trains, stations, and train tracks. Figure 9 shows trains modelled with a two-carriage design (total length of 18.6 m). Two variations of the future campus-train stations are developed using SolidWorks (Figure 10). These models in STEP format are converted into an OBJ file and imported into Omniverse as a USD file using APOSE. The motion path and vehicle dynamics features in Omniverse are utilized to create a dynamic train simulation along the track on campus. First-person and third-person perspective views are available with NTUniverse.

The building of NTUniverse in itself provides a great opportunity for NTU students to learn design-thinking through real problem solving. Students have a chance to learn relevant design skills, including CAD design, parametric and variational design, dynamic simulation, data exchanges, etc. They are exposed to various commercial design and software tools including SolidWorks (version 2023), Maya (version 2023), 3D Max Studio (version 2023), Unity (version 2022.3.0), Unreal (version 5), Blender (version 4.0), and NVIDIA Omniverse (version 2022.3.1).

The design flow of the second module is described in the following steps: (1) creating the train and track models, using SolidWorks, based on references (STEP file); (2) converting the STEP file into an OBJ file and then USD via an online file converter; (3) importing the USD file into Omniverse; (4) creating a new file in Omniverse USD Composer; (5) scaling the model to the appropriate size and adjusting the camera and lighting settings accordingly; (6) utilizing the “Motion Path” feature in Omniverse to create a simulation along a path; (7) utilizing the “Vehicle Dynamics” feature in Omniverse to perform a simulation based on vehicle dynamics; (8) saving the simulation as a USD file; and (9) importing and integrating the respective simulations into the general file, with the NTU Campus and other projects.

5. NTUniverse for Virtual Education

With the digital twin of the NTU campus having been created, students will gain immersive learning experience via NTUniverse.

5.1. Design Education

Design is an interdisciplinary yet challenging subject which plays a fundamental role in the training of engineers within the Industry 4.0 era. Luckily, the NTUniverse project integrates the design element within the education metaverse by using the example of the NTU campus train system, which is currently under construction. In a manner different from the conventional curriculum of CAD design, students have the chance to design and assemble the campus train systems before they are inputted in the Omniverse platform for simulation application.

The module of a future campus-train system design offers a great opportunity for students’ design education. For instance, students can learn to simulate the designed train system by enabling the motion of the train following a track or path. They can learn vehicle dynamics, as well as potential train faults, with the modelled train system. Section 4 provides all the details of the future campus-train system (Figure 9 and Figure 10), which fits well with design education. Apparently, NTUniverse itself heavily involves design education relevant to the NTU campus (Figure 4, Figure 5, and Figure 11).

Apart from the design of the campus train system, NTUniverse provides both a new way for students to learn about heritage-related matters and STEM education.

5.2. Heritage Education

NTUniverse has photorealistically represented most of the campus buildings as 3D models. As such, students can perform a “virtual walkthrough” with iconic buildings like the School of Art Design Media (ADM), the HIVE, and the ARC, among others. This navigation function allows students to walk through the virtual campus anytime, and anywhere. It facilitates the navigation of users and their experience in exploring the campus by improving the accessibility of information. Heritage-based educational experiences within NTUniverse can be developed to augment students’ experiences, developing the capability for a better understanding of the history of NTU. With the aid of interactive avatars, they can engage in Heritage Walks, a traditional activity for all NTU freshmen during their orientations. These Heritage Walks can help students develop their identities and promote intercultural understanding and the preservation of cultural heritage. Specifically, Yunan Garden and the Chinese Heritage Centre are identified areas for the virtual Heritage Walk. Interactive avatars are created based on Omniverse, enabling immersive experiences through the simulation of real-world scenarios with guided gestures and actions (Figure 11).

5.3. Science, Technology, Engineering, and Mathematics Education

VR approaches have been adopted in fundamental education [19]. STEM education is a core activity in higher education. NTUniverse serves as a STEM education platform allowing students to engage in virtual hands-on, online discussions, and collaborative team projects in virtual or hybrid classrooms.

For instance, NTUniverse can be used for science education to learn physics theory. Students can use the visualization and interactive functions within NTUniverse when learning abstract concepts such as gravity and collisions. We designed a virtual experiment with a ball and a ramp to demonstrate the effects of collisions. Different parameters (e.g., varying x and y coordinates of the ball) of experiments can be interactively changed, helping students to better understand the laws of physics and energy conservation principles.

Section 4 is devoted to the design of the future campus-train system. Obviously, with the aid of the campus train system, NTUniverse can be used to allow students to learn design technology. The same NTUniverse train system can also be used for engineering education. For instance, students can assemble or disassemble the train components of the campus train system.

NTUniverse is used by students, not only for remote learning virtually, but also for immersive and interactive learning physically. Figure 12 shows students doing immersive and interactive learning in vector geometry. This is a novel approach developed to help students to learn abstract mathematics concepts through interactive manipulation. Abstract concepts, like Vector Cross Product, Skewline Distance, Left-hand Rule, etc., become much more tangible when students are able to interact with virtual objects during their learning. This immersive and interactive approach will help students, improving their learning in terms of self-efficacy [5,20,21,22,23,24], situational interest [25,26], and so on.

A small-scale survey involving eight participants was conducted based on the Technology-Rich Outcome-Focused Learning Environment Inventory (TROFLEI) [27].

Table 1. Survey TROFLEI questions for each participant.

No.	Questions	Detailed Descriptions	Responses
Q1	Student Cohesiveness	The extent to which you are able to support and help other students in NTUniverse environment, (e.g., I am able to work with others effectively in NTUniverse).	1—Strongly Disagree; 2—Disagree; 3—Neutral; 4—Agree; 5—Strongly Agree
Q2	Student Involvement	The extent to which you are attentive, participate in discussion, are willing to do additional work, and enjoy the experience in NTUniverse, (e.g., I am able to discuss ideas and give my opinions while utilizing the NTUniverse platform).	1—Strongly Disagree; 2—Disagree; 3—Neutral; 4—Agree; 5—Strongly Agree
Q3	Student Investigation	The extent to which you are able to use skills and processes of enquiry to help with problem solving and investigation in NTUniverse, (e.g., I am able to carry out useful functional analysis within NTUniverse).	1—Strongly Disagree; 2—Disagree; 3—Neutral; 4—Agree; 5—Strongly Agree
Q4	Student Cooperation	The extent to which you are able to cooperate with one another rather than compete on learning tasks in NTUniverse environment, (e.g., I can cooperate with other students effectively and build teamwork in NTUniverse).	1—Strongly Disagree; 2—Disagree; 3—Neutral; 4—Agree; 5—Strongly Agree
Q5	Differentiation	The extent to which teachers can cater for students and provide curated experiences of different levels, (e.g., I can work at my own pace within my own ability, rate of learning, and interests in NTUniverse).	1—Strongly Disagree; 2—Disagree; 3—Neutral; 4—Agree; 5—Strongly Agree
Q6	Equality	The extent to how equally you are treated by Teachers in the NTUniverse environment, (e.g., I have the same amount of say and capabilities as other students in the NTUniverse environment).	1—Strongly Disagree; 2—Disagree; 3—Neutral; 4—Agree; 5—Strongly Agree
Q7	Creativity	The extent to which you are able to utilizing creativity thinking and designing during the VR activities in NTUniverse, (e.g., I can freely utilizing my creativity in NTUniverse).	1—Strongly Disagree; 2—Disagree; 3—Neutral; 4—Agree; 5—Strongly Agree
Q8	NTUniverse	The extent of usefulness to which you find utilizing VR as a tool for analysing real life applications or use cases, (e.g., I think this NTUniverse is useful in analysing problems in the real world).	1—Strongly Disagree; 2—Disagree; 3—Neutral; 4—Agree; 5—Strongly Agree

lists the eight survey questions used to collect the opinions of each participant as to NTUniverse. Each question is associated with a five-point Likert scale: (1) Strongly Disagree, (2) Disagree, (3) Neutral, (4) Agree, and (5) Strongly Agree.

Table 2. TROFLEI Results collected from the eight participants.

	Q1	Q2	Q3	Q4	Q5	Q6	Q7	Q8
Strongly Disagree	0	0	0	0	0	0	0	0
Disagree	0	0	0	0	0	0	0	0
Neutral	0	0	0	0	0	0	0	0
Agree	3	3	4	4	3	2	3	3
Strongly Agree	5	5	4	4	5	6	5	5
Scores (out of 5)	4.625	4.625	4.5	4.5	4.625	4.75	4.625	4.625

shows that the initial TROFLEI results are promising, with most questions receiving scores of 4.5 out of 5.

6. Conclusions

This work outlines the design process for an educational metaverse, called NTUniverse, that functions as the digital twin of the NTU campus. It has been designed with the aim of enabling students to learn design, heritage, and STEM proficiency. We have used NVIDIA Omniverse as a central unified platform upon which to base NTUniverse. Three modules are designed in this work. (1) For the NTU Campus Digitalization, we have developed a comprehensive strategy that produces a realistic and immersive experience for users exploring the campus by adding important building components and material textures, including labelling significant structures and installations, improving the educational botany experience, and incorporating interactive avatars. (2) For the future campus-train modelling using the Omniverse platform, a realistic simulation of land rail transportation has been created to include several viewpoint views, motion path features, and realistic vehicle-dynamics. Students can compare the model to other projects and evaluate its technical performance by comparing its frame rate, rendering quality, and the effectiveness of the physics and motion systems, all of which have been examined. (3) Within the virtual education of NTUniverse, heritage-based knowledge, design and STEM proficiency are developed.

There are many opportunities for further elaboration and extension of NTUniverse. Future directions for research include the creation of increasingly precise and up-to-date campus buildings and installations. It will be beneficial to extend the integration of NTUniverse with other information technology (IT) systems associated with universities, in order to display real-time building information in the digital twin. For future campus-train modelling, in order to generate more dynamic and realistic train simulations that better reflect real-world events, real-time data may be collected using sensors, including weather conditions and train schedules. For virtual education, a large number of STEM modules can be incorporated in NTUniverse. In addition, efficient user interfaces should be designed to enhance student engagement and learning outcomes. The implementation of user interface enhancements is able to address the accessibility issues for all user profiles and improve engagement. Students could also track and visualize their learning progress more easily. Furthermore, NTUniverse may be expanded to provide industrial training scenarios, including onboarding and orientation, to acquaint new hires with the policies and workplace of a company; technical skills training covers topics like operating machinery, maintaining equipment, and performing other job-specific tasks; safety training covers safety protocols, emergency procedures, and how to properly use personal protective equipment; additionally, soft-skills training covers soft skills like leadership, problem-solving, communication, and teamwork—all of which are crucial for a productive industrial setting. Evaluations of NTUniverse with bigger sample size will be performed as part of the future work.