UniRoVE: Unified Robot Virtual Environment Framework

Abstract

With robotics applications playing an increasingly significant role in our daily lives, it is crucial to develop effective methods for teaching and understanding their behavior. However, limited access to physical robots in educational institutions and companies poses a significant obstacle for many individuals. To overcome this barrier, a novel framework that combines realistic robot simulation and intuitive control mechanisms within a virtual reality environment is presented. By accurately emulating the physical characteristics and behaviors of various robots, this framework offers an immersive and authentic learning experience. Through an intuitive control interface, users can interact naturally with virtual robots, facilitating the acquisition of practical robotics skills. In this study, a qualitative assessment to evaluate the effectiveness and user satisfaction with the framework is conducted. The results highlighted its usability, realism, and educational value. Specifically, the framework bridges the gap between theoretical knowledge and practical application in robotics, enabling users to gain hands-on experience and develop a deeper understanding of robot behavior and control strategies. Compared to existing approaches, the framework provides a more accessible and effective alternative for interacting with robots, particularly for individuals with limited physical access to such devices. In conclusion, the study presents a comprehensive framework that leverages virtual reality technology to enhance the learning and training process in robotics. By combining realistic simulations and intuitive controls, this framework represents a significant advancement in providing an immersive and effective learning environment. The positive user feedback obtained from the study reinforces the value and potential of the framework in facilitating the acquisition of essential robotics skills. Ultimately, this work contributes to flattening the robotics learning curve and promoting broader access to robotics education.

1. Introduction

Industrial robotics has witnessed significant growth in recent decades, revolutionizing various sectors and optimizing production processes. However, the use of traditional industrial robots introduces safety concerns, demanding the implementation of stringent safety measures [1]. Furthermore, the installation and start-up procedures associated with these robots can be time-consuming. In contrast, collaborative robots, which share similarities with traditional robots but are designed for direct interaction with humans, offer enhanced safety features [2]. The intuitive nature of the user–robot interaction, such as physically dragging the robot to the desired position, contributes to the rising adoption of collaborative robots in industrial settings.

However, programming and testing current and new features, as well as modifying existing robot programs, often require access to the robot software core, which may not be available. While commercial simulators exist as potential solutions, their complexity necessitates a significant investment of time for learning and expert use [3]. In this context, virtual reality (VR) technology emerges as a promising approach to simulate the experience of operating real robots, enabling easier understanding and programming for operators and students alike. The primary objective of this project is to introduce VR simulation to industrial manipulators and demonstrate its potential in overcoming the limitations of traditional approaches.

Integrating robotics into VR environments has gained traction, with major companies like ABB and Kuka incorporating Extended Reality (XR) functionalities into their frameworks [4,5]. However, the high license costs and privacy constraints imposed by these companies limit the accessibility of their environments, especially for small businesses and educational institutions [6]. This underscores the importance of developing an open-source framework that enables interaction with virtual robots through VR. Previous research has explored applications for interacting with industrial manipulators as though they were collaborative robots using VR technology [7,8,9,10]. However, existing applications have typically been limited to individual robots within specific environments, overlooking the vast range of possibilities offered by VR projects. Consequently, there is a need for an open framework that overcomes these limitations and emulates the experience of interacting with both real and virtual robots, different robot manufacturers, and fulfilling to the needs of industry and education.

This paper introduces UniRoVE (Unified Robot Virtual Environment), an open-source VR framework designed to enhance users’ comprehension of various robot behaviors, while also evaluating the effectiveness and user satisfaction with VR-enabled robotic interactions. This paper is organized as follows: Section 2 presents an analysis of the current state of the art, highlighting strengths and weaknesses of previous studies. In Section 3, a detailed overview of the equipment and methods employed in implementing the UniRoVE system and its evaluation is provided. Additionally, the section presents a performance analysis based on user feedback, demonstrating the effectiveness of UniRoVE. The findings from the user feedback are further examined in Section 4. Finally, Section 5 summarizes the project’s results and conclusions, emphasizing the invitation to researchers to leverage the UniRoVE framework as a foundation for future robotics applications.

2. Related Work

In this section, the related work is analyzed and compared, focusing on similarities, differences, and common shortcomings.

One prominent software for simulating a wide range of robots in virtual reality (VR) is RobotStudio developed by ABB. It offers realistic simulations using actual robot programs and configuration files identical to those used on the shop floor. However, being a private software limits its scalability and imposes an annual software license cost, which hinders its accessibility and usability for non-ABB robots [4,6].

In contrast, MinRend’s VR Robotics Simulator provides a low-cost entry point for industrial robot simulation using HTC Vive headsets. Although it offers a variety of tools for building robotic cells and programming in a VR environment, its main focus is not on generating programs for real robots, but rather on demonstrating the potential of robot cells and aiding users in understanding how to control different simulated robots [5].

From an academic perspective, several simulators have been developed. One such example is a VR-based framework proposed by Bolano et al. This framework allows intuitive offline programming of robots by dragging the Tool Center Point (TCP) and visualizing joint value evolution; nevertheless, it is limited to the UR10 robot, which restricts its applicability [10].

Similar research projects, such as Holubek’s project, explore using VR as a supporting tool for studying robot-end-effector movement in a virtual workstation; however, it focuses solely on simulating the ABB IRB 120 robot and lacks direct interaction with the robot joints [7].

Su et al. developed a software with two main modules: real-time 3D reconstruction and 3D manipulation using controllers. The software allows accurate control of a physical robot through a virtual environment, resulting in improved learning rates and reduced training time [8].

Rodriguez implemented a VR environment for programming the Yaskawa Motoman HP20D robot using the MYO Armband sensor. While it aimed to ease interaction without deep programming knowledge, issues arose from sensor selection, highlighting the importance of choosing appropriate tools [9].

Another VR environment previously developed by Alberto Zafra allows intuitive offline programming of the UR10 robot by controlling each joint separately or by dragging the end effector. However, there is a risk of reaching singularity positions, and a singularity detection functionality was developed to address this issue [11].

These projects indicate a growing trend of connecting robotics with VR, as supported by the effectiveness of VR-based training. The use of VR in learning tasks reduces training time and enhances learning quality [12]. Intuitive human–robot interaction and support for a variety of robots are desirable factors in these virtual environments.

In conclusion, while the reviewed articles demonstrate the potential and benefits of using virtual reality (VR) in the context of robotics, they also highlight several limitations. The use of proprietary software like RobotStudio by ABB limits scalability and accessibility for non-ABB robots, and the associated annual license cost poses a barrier. Similarly, MinRend’s VR Robotics Simulator, while cost-effective, focuses more on demonstrating robot cells rather than generating programs for real robots. Academic frameworks, such as Bolano et al.’s VR-based framework and Holubek’s project, have limitations in terms of applicability to specific robot models. Rodriguez’s VR environment using the MYO Armband sensor encountered issues related to sensor selection. Despite these limitations, the projects collectively indicate the growing trend of integrating VR with robotics, supported by the effectiveness of VR-based training in reducing training time and enhancing learning quality. This paper acknowledges the importance of intuitive human–robot interaction and aims to address limitations by providing an intuitive offline programming environment for a variety of robots while considering singularity detection to ensure safe robot control. By addressing these limitations, the paper contributes to the advancement and practicality of VR in the field of robotics. In the following section, the equipment and methods used for the development of the UniRoVE are presented.

3. Equipment and Methods

The development of the project involved several phases and tasks, which are described in detail below. The analysis of previous projects in the literature revealed common functionalities and limitations. To overcome these limitations and incorporate the strengths of previous works, several key functionalities were identified for implementation. These included the simulation of multiple robots in a VR environment, the implementation of a robot library to support different robot models, the inclusion of robot movement tools (TCP dragging and joint control), the creation of a singularity detector, and the development of an object library for customization of the virtual environment.

3.1. System Development

The system proposed in this project includes a VR framework which allows the user to intuitively understand how different robots interact with their environment; and to create personalized case studies for presenting the immeasurable amount of possibilities that the combination of these technologies provides. One of the crucial factors which determined the selection of the VR technology over the rest of the technologies that encompasses the XR was the development of this technology over the other visualization techniques. Due to the early start of its commercialization, the VR technology has been constantly under development since long ago, becoming able to reach more affordable prices [13,14]. Additionally, the high development status of the VR technology gives the user the impression of being within a realistic environment where it is possible to interact in a natural way with the simulation via the use of a VR headset and its controllers.

First of all, it was necessary enable the VR functionality, for doing this the XR Legacy Package and Oculus Integration Package were installed in the Unity project. The Oculus Integration Package provided prefabs and components for easier VR implementation. It was primarily used for locomotion control, allowing users to move within the environment using joystick controls or teleportation. The Grabbing Controller feature enabled users to interact with objects and facilitated robot–human interaction. The program incorporated different working modes (Creation, Selection, Move, Deletion) managed by the RobotSelection script, which interacted with the PointableObject_Custom script to detect and provide visual feedback on selected elements.

Afterwards, a virtual environment was created. Using the HQ Hangar Free scenario, a free scenario designed by Purple Sky that can be downloaded from the Unity Asset Store the main scenario was configured. Additionally, four interactive objects: a conveyor, a custom platform, a cardboard box, and an adjustable working table, were included in the environment to form an object library. This object library allowed users to customize their industrial environment.

Later, the main menu of the user interface of the UniRoVE framework was implemented. The main objective of this menu was to improve the lack of intuitiveness of the present industrial system architectures, reducing the amount of time needed for training to use these frameworks appropriately. Designing the Human Machine Interaction (HMI) simple and practical, the created system tries to be as intuitive as possible in order to address this shortcoming, flattening the workers learning curve. This is accomplished by the creation of a simplistic radial menu as represented in Figure 1, which allows the user to choose between the various functionalities introduced in the framework, as well as by including special functionalities aimed at simplifying the HMI, such as visual feedback or the environment’s menu customization.

On the other hand, this implementation had to be among the first frameworks, which gives the user the freedom to select which kind of robot to interact with, including real and custom robots. Due to this reason, one of the most important developed functionalities is the creation of an easy customizable robot library, which allows the introduction of diverse robots into the virtual environment through the use of URDF files. Nevertheless, there are also pre-imported robot models that can be used without further configuration. If the user wants to employ one of these framework’s already-introduced robots, they can accomplish this by choosing it from the robot library shown in Figure 2. This example library, attached initially to the left hand of the VR controller, is composed of four robots (two industrial manipulators, a collaborative robot, and a Scara robot).

The pre-imported robots selected for simulation include the UR10 robot, ABB IRB 6640, Stäubli RX160, and Denso Robotics HS-045 SCARA robot. To ensure a realistic simulation, a comprehensive understanding of their behavior was necessary. Thus, prior to integrating them into the framework, their kinematics were simulated and represented using the RobotDK simulator and Matlab with the Robotics Toolbox by Peter Corke. Finally, the Denavit–Hartenberg (DH) parameters were calculated for each robot, and their behavior was further simulated in the Unity platform using URDF files obtained from the official GitHub repositories of each robot.

Nevertheless, it is not possible to say that the user has a complete freedom, in means of personalization, if the working environment is not fully customizable. For this reason, in addition to the robot library, the previously mentioned object library was created. Giving the user the opportunity to generate virtually limitless case studies. The object library has been implemented with the same functionalities as the robot library and it is composed by four elements (a carton box, a conveyor, a working table and a robot platform). However, for introducing new objects to the library in this case, instead of using URDF files, it is necessary to introduce the 3D models of the objects to the framework. An example is shown in Figure 3 where the user is deleting a platform that was placed over a working table.

On the other hand, as previously stated, this project not only aims to be a tool for creating industrial environments, but also to be a framework for actually simulating the robot’s behavior in different situations, while testing its feasibility. For this reason, a fully functional kinematics controller was designed, allowing the user to jog the virtual robots both, by the use of a menu, or by dragging the end-effector of the robot itself. This is presented in the right-hand menu of the VR framework controller.

The implementation of the kinematics controller was accomplished through the use of Unity scripts. These scripts consist of two main components: the “URDF Robot” script, which facilitates the configuration and movement of the robot joints, and the “UR Controller” script, responsible for conducting forward and inverse kinematics calculations based on manually inputted Denavit–Hartenberg (DH) parameters.

When the user chooses to move the robot through the use of the joints menu represented in Figure 4, he or she can move each joint of the robot independently of the different sliders shown in the menu. However, in order to jog a robot, the user has to select first which robot they want to move, something which is accomplished with the pointer attached to the right controller. The selected robot can easily be differentiated from the rest of the robots, because, once a robot is selected, an indicator appears at the base link of the selected robot.

Additionally, for jogging the robot in the case of a collaborative robot, by dragging its end-effector as shown in Figure 5, it is not necessary to previously select the desired robot, the interacted-with robot being selected automatically. However, by using this kind of movement, the robot can reach a singularity more easily, something that makes it quite difficult to control. This can result in undesired configurations, which can block various parts of the robot. For avoiding these situations, the user can return the robot to its original configuration at any time by pressing the ”Go Home” button on the radial menu.

Additionally, to ensure smooth operation, a sophisticated singularity management system was integrated into the framework. This system warns the user when a singularity position is reached or whether the robot is near to a singularity. The singularity management system is based on the use of the Denavit–Hartenberg (DH) convention for calculating the Jacobian matrix of the robot in real time [15]. Additionally, this kinematic controller is also used for calculating the direct and inverse kinematics of the robot, making possible to follow the variation of the joint values of the robot while moving it by dragging its end-effector.

Finally, a state machine to move between four different working modes: the “Selection Mode” or also named “Robot Control”, where the user is able to select independently the robot to be controlled; the “Creation Mode”, which can be used to create or add more components to the environment (in order to make from this a personalized experience for the user); a “Move Mode”, where the user can modify the position and orientation of the different elements placed in the VR environment; and the “Deletion Mode”, which is used for reducing the number of available components disposed in the workstation. This facilitates the expansion of the implemented system, which is available on GitHub since July 2022 (https://github.com/albertozafra7/VR_MultiRobot_Sim, accessed on 6 July 2022), by other researchers. The next sections presents the performance analysis of the VR framework.

3.2. Performance Analysis

In order to validate the developed UniRoVE framework, a complete performance analysis has been carried out. This analysis also covers a qualitative evaluation, which is prevalent in the assessment of VR systems.

The qualitative evaluation is based on the user experience interacting with the virtual environment and its elements, assessing the intuitiveness, immersion, realism and other qualitative aspects described in

Table 1. User evaluation questionnaire.

ID   Question
Aspect 1: Interaction Realism
Q1:   I did understand the purpose of each menu without further explanation.
Q2:   I felt as I was in a real industrial environment.
Q3:   I did not get confused with the buttons used of the controllers.
Q4:   It was really easy for me to find the functionallity that I was looking for.
Aspect 2: Robotics Motor
Q5:   When I was dragging the end-effector of the robot, the robot followed perfectly my
hand.
Q6:   I feel like I have a greater comprehension of how the robots behave.
Q7:   I think that I can expect when a robot is going to pass by a singularity.
Q8:   I was able to understand how the movement of an individual joint would affect to the
end-effector.
Aspect 3: Personalisation
Q9:  I felt like I could move with total liberty inside the environment.
Q10: I did not feel like any functionality was missing.
Q11: I feel that I can create my own case of study with ease by using this framework.

(Section 3.2.1), which addresses the qualitative evaluation in detail.

3.2.1. Qualitative Evaluation

The measurement of the realism and immersion of most VR experiments through the use of qualitative studies is common practice. Additionally, questionnaires are the default method to qualitatively assess the different parts of a project [16,17]. However, there is an absence of a standardized set of questions for different experiments that allows fair comparisons [18]. In addition, the different nature of the VR systems makes it almost impossible to find a set of evaluation questions that fits every framework [19]. Nevertheless, during the evaluation of the system for easing the process of analysis, the questions were categorized depending on their evaluation target in this project. The following categories resulted from the clustering of the questions.

• Interaction Realism: this category contains all the related questions about the realism of the environment and the qualitative feeling of the user after making simple interactions with the framework. An example of these questions could be whether the user found any difficulty while interacting with the radial menu.
• Robotics Motor: this aspect is related to the robotics simulation, aiming to analyze the experience obtained from working with the simulated robot. An example question of this category could be if the user obtained a better knowledge about how robots move in reality.
• Personalisation: this category evaluates the sense of freedom that the user had during the assessment of the system. An example of a question of this category could be if the user felt that some functionality was missing, or whether if the user wanted to accomplish something that was not implemented yet.

The questionnaire, shown in Table 1, is composed of 11 questions related to the previously described aspects. Following that, the users of the study were asked to answer the questions right after the end of the experience in a randomized order to limit context effects.

In addition, the questions must be answered following the 7-point Likert-scale: (+3) strongly agree, (+2) agree, (+1) somewhat agree, (0) neutral, (−1) somewhat disagree, (−2) disagree, and (−3) strongly disagree. The results will be presented as a single embodiment score using Equations (1) and (2):

$$\begin{array}{c}\hfill InteractionRealism=(Q1+Q2+Q3+Q4)/4\\ \hfill RoboticsMotor=(Q5+Q6+Q7+Q8)/4\\ \hfill Personalisation=(Q9+Q10+Q11)/3\end{array}$$

(1)

$$Score=(InteractionRealism+RoboticsMotor+Personalisation)/3$$

(2)

These questions were introduced to a set of participants after testing the UniRoVE framework. This is presented in the following sub-section.

3.2.2. Participants

For the evaluation of the system, diverse participants were recruited and classified into different profiles. Overall, 80% of them had previous experience with VR applications, while the remaining 20% were inexperienced in using VR technology. Among the participants, 65% were male, and 35% were female. Additionally, all participants had at least a bachelor’s degree, and the average age was 26 years.

Regarding their background in robotics, 30% of the participants had a formal education in robotics, but did not have any practical experience interacting with real robots. Another 30% had a background in robotics and had interacted with real robots in practical settings. The remaining participants did not have any prior experience or background in robotics and had not interacted with robots in reality.

It is worth noting that the participants in this study did not receive any compensation for their participation.

Table 2. Different profiles of the recruited participants.

	Profile 1	Profile 2	Profile 3	Profile 4
Knowledge in robotics	X	X
Interacted with real robots	X
Previous experience with VR	X	X	X
Group participants:	30%	30%	20%	20%

provides a detailed breakdown of the participant groups recruited for the performance analysis based on their background in robotics, prior interactions with real robots, and experience with VR applications. The participants were classified into different categories according to their knowledge and experience:

• Profile 1: Participants with a background in robotics, practical experience with real robots, and experience with VR applications.
• Profile 2: Participants with a formal education in robotics and had previous experience with VR applications, but no practical experience with real robots.
• Profile 3: Participants without any prior background in robotics and no experience with real robots, but with previous experience with VR applications.
• Profile 4: Participants without any prior background in robotics, no experience with real robots, and without previous experience with VR applications.

The diversity of participant profiles allowed for a comprehensive evaluation of the UniRoVE framework’s effectiveness and user experience.

The performance analysis procedure is described in the following subsection, indicating the concrete tasks to be performed by each participant.

3.2.3. Procedure

The performance analysis starts with the user introduction to a pre-defined environment, where each one of the default robots of the robot library are already disposed, as is shown in Figure 6.

In this scenario, users were required to interact with at least one of the available robots. Subsequently, they were given the freedom to personalize the environment according to their preferences. The users were given ample time to familiarize themselves with the system before proceeding to complete a series of assigned tasks. The time taken to accomplish each task was recorded for further analysis.

The tasks assigned to the users were as follows:

• Teleport within the scenario.
• Perform a Cartesian jog with the pre-imported UR10 robot.
• Introduce a new robot into the scenario using the robot library.
• Perform a joint jog with the newly placed robot.
• Place 5 different objects within the scenario, utilizing at least 2 distinct types.
• Delete 1 robot and 1 object from the scenario.

Upon completing the tasks, the user is required to respond to the evaluation questionnaire outlined in Table 1. The questionnaire aims to assess various aspects of user experience in VR environments, including the nature of the human–robot interaction, the obtained robotics understanding, degree of freedom perceived by the user, and other qualitative aspects related to their experience. Additionally, participants are encouraged to provide any further comments or feedback they may have regarding the tested framework. The next section summarizes and discusses the results obtained from the performance analysis.

4. Results and Discussion

The feedback obtained by the interviewed individuals is presented in

Table 3. Average score for each qualitative aspect of the evaluation and group of participants.

	Profile 1	Profile 2	Profile 3	Profile 4
Interaction Realism	1.83	1.75	1.88	−0.13
Robotics Motor	2.33	2.42	2.00	1.88
Personalisation	1.89	1.67	2.33	1.83
Embodiment Score	2.02	1.94	2.07	1.19

, in which it is possible to determine that totally inexperienced users, in both robotics and VR, found out that the initial interaction with the system was pretty difficult. The evaluation of the UniRoVE system encompassed three key aspects: Interaction Realism, Robotics Motor, and Personalization. These aspects were crucial in understanding the overall effectiveness of the system and its impact on user satisfaction; being 3.0 the maximum score and −3.0 the minimum.

Regarding Interaction Realism, the results determined that participants generally responded positively to the VR-based interaction with the robots and the virtual environment. The high scores in this aspect indicate that the system provided an immersive and realistic experience for the users who had worked previously with VR. On the other hand, the low score provided by the users who never experienced VR before indicate that the framework is not as intuitive as desired. Additionally, some users determined that there would be a need for an initial tutorial to show the user how to use this framework correctly, suggesting that a good approach to it could be to make an introductory video about the controls and functionalities of the system. Furthermore, there was some misunderstanding with the use of different menus, such as the “Go Home” button, which was usually confused with a teleportation to the initial position or with a reset of the whole system. Moreover, some users stated that it was difficult for them to say in which working mode they were, because they expected a little change on the pointer color or something similar. This has been considered as improvement of the system.

The Robotics Motor aspect evaluates the accuracy and fidelity of the robot’s motion representation within the VR environment. The positive feedback and scores in this aspect reflect that the implemented kinematics controller, based on the Denavit–Hartenberg parameters, effectively captured and simulated the robotic movements. This functionality allowed users to control the robots smoothly and precisely, contributing to a more authentic and valuable learning experience. Additionally, the singularity management system aided in preventing unrealistic movements and enhancing the overall realism of the robot behaviors. Being really helpful for understanding how robots behave and to consolidate previous theoretical knowledge in robotics. However, despite of being a good approximation to the reality, this Robotics Motor can be improved by adding more functionalities specifically designed for teaching and programming robotics. Something that also affects it directly is customization assessment, due to the lack of these specific functionalities for robot programming are seen as a constraint to the possibilities of personalization, and its direct application to the industry.

Personalization, the third aspect of evaluation, focuses on the system’s flexibility and adaptability to accommodate user preferences and create a customized virtual environment. The ability to customize the industrial environment by adding various interactive objects and robots from the object library was well-received by the participants. They appreciated the freedom to design and tailor their case studies, and this feature greatly influenced their satisfaction with the system. However, we acknowledge that there is room for improvement in expanding the range of objects in the library and enhancing the level of customization options to provide users with even more extensive possibilities. In general, the different participants of the experiment gave a positive feedback related to the customization of the environment, as most of them felt that the predefined robots and objects that are included in the default libraries are more than enough for creating basic case studies. Moreover, the possibility of including custom elements to the system was seen as one of the best functionalities of the framework for customizing the environment. In addition, most of the participants stated that in the current state of the environment, this system could be used for verifying some industrial applications of the robots, and analyze a huge variety of industrial case of studies.

Finally, having an overall embodiment score of 1.81 out of 3.0, it is possible to state that the majority of testers have a positive opinion of the project, with a small difference in adaptation to the system between users with previous experience in working with VR and first-time users of a VR headset. This situation indicates that the developed framework could significantly reduce learning time applying small changes to the current presented system. Being the resulting system capable of delivering an immersive experience to the user, where users can understand the behavior of a robot, flattening the learning curve of robotics, and reducing the training time in industry.

5. Conclusions

This project aims to enhance the comprehension of robot behavior while simultaneously reducing the learning curve for workers, laying the groundwork for the advancement of novel VR Robotics frameworks. Although the newly developed multi-robot simulation framework, UniRoVE, has areas that can be further refined, it enables direct interaction with diverse simulated robots, allowing users to create personalized environments. Beyond its primary objective of facilitating the learning process for robot–environment interactions, this system possesses the potential to establish virtual workplaces where the realm of robotic engineering in the industrial domain can be explored.

Furthermore, the UniRoVE framework improves previous research by providing an intuitive means to interact with various industry robots. Its scalability and flexibility offer a valuable approach to analyze the impact of robot integration in industrial settings, facilitating performance comparisons between different robot types within a simulated environment. Such capabilities prove indispensable when selecting robots for distinct production lines in a company. Moreover, the framework being open-source creates opportunities for the development of VR and robotics-related projects.

Several aspects of this project could benefit from future work and additional resources, particularly the implementation of offline robot programming, a feature observed in most of the analyzed related works. This functionality would significantly enhance the program’s usability by enabling not only the teaching of robot–environment interactions, but also robot programming.

Finally, the inclusion of new functionalities such as virtual commissioning, as well as extending this project to other XR technologies like mixed reality (MR) or augmented reality (AR), would further enhance its usability. These advancements would enable real-time control of physical robots through virtual simulations. Consequently, this project has been developed as open-source software and has been made available on GitHub, serving as a foundation for future endeavors.