A Modular XR Collaborative Platform for Occupational Safety and Health Training: A Case Study in Circular Logistics Facilities

Abstract

Over the past few years, safety and health have become major concerns in the warehouse and logistics sectors. Each year, warehouse fatalities, injuries, and accidents cause unrecoverable losses and huge financial costs. In spite of all the advancements in methods, tools, equipment, and regulations, the number of accidents, especially fatal ones, has not subsided significantly. As a result, safety professionals and researchers have explored new and innovative ways to combat this problem. In the circular logistics facility (CLF) industry, located inside warehouses and providing human muscle-oriented services to maintain pallets, both short-term safety incidents and long-term health concerns are present. Long-term health training is rarely discussed in the literature compared to short-term safety training. This is because health issues are more complex than safety issues, since biological outcomes may take time to develop, are affected by multiple resources, and cumulative injuries may occur. This paper contributes to warehouse health and safety by designing and developing a modular XR collaborative training and testing platform (MXC-P). The co-design process is applied to design each module in the MXC-P. Three main modules related to health and safety training for CLF were considered, namely personal protection equipment, pallet handling, and pallet repairing. On this platform, a virtual interactive world provides a solid hands-on training environment and generates syntactic data for evaluating long-term health risks. On the other hand, collaborative and modular environments provide a solution to geographically distributed systems, allowing employees to connect and train remotely. The effectiveness of the MXC-P is compared with traditional safety training in a pilot study. Based on the results, we can establish that the MXC-P is effective in teaching and testing hazard identification situations, especially those relating to short-term health. The results also indicate that trainees’ recall of knowledge would improve with the MXC-P. In addition to this, the MXC-P can also be used to test and evaluate a new system and generate syntactic data for evaluating long-term health.

1. Introduction

Pallets are a common part of any warehouse, and the handling, repairing, and recycling of them are mostly performed within the warehouse by a third party or a special section that is commonly referred to as a circular logistics facility (CLF). The development of CLFs provides an efficient way for companies to manage their pallets [1]. Third-party management companies combine pallet manufacturers with pallet repair and recycling companies to provide a complete pallet management system. Third-party companies either manage a user’s existing pool of pallets or rent pallets to the user on a per-trip basis. The CLF tracks pallets throughout their usage and retrieves them for reuse or refurbishing. The CLF uses higher-quality pallets, increasing the number of trips per pallet by an average of 20 to 40 times compared to 5 times when managed by the user. This style of pallet management reduces the amount of lumber used to make new pallets, promotes reuse and repair, reduces pallet waste via recycling, and saves the pallet user money and time [2,3]. Due to the nature of the warehouse and pallets, CLFs are primarily a human-originated and muscle-based industry. In recent years, the issue of warehouse safety, specifically regarding CLFs, has become more frequent in practice and more important because of the high hiring rate of non-expert workers during the high seasonal periods for CLFs [4,5].

The rates of illness and injury at warehouses were described by the U.S. Department of Labor Office of Inspector General (OIG) as “consistently high”. In 2021, the rate was 5.5 per 100 employees, more than double the rate across all U.S. industries. About 1.7 million people work in the nation’s roughly 20,000 warehouses, and one-third of these warehouses have an active CLF. A total of 19,709 facilities are classified as warehouses; an additional 38,785 are listed as electronic shopping and mail-order houses, which the OIG classified as “online retailers”. Online retailers employed 447,059 workers in 2021 [6,7]. The OIG noted that warehouse workers can face hazards resulting in serious injury due to powered industrial trucks, loaded pallets, and repetitive movements. These dangers can be compounded when time-based delivery quotas and production goals increase because of the rise in e-commerce. According to the U.S. Census Bureau, during the first year of the COVID-19 pandemic in 2020, there was a 43% increase in e-commerce sales, with sales rising from USD 571.2 billion in 2019 to USD 815.4 billion in 2020 [8]. This causes the hiring of non-expert workers, which dramatically increases the rate of injury in the warehouse industry.

Figure 1 illustrates the representative operational workflow of a CLF in a warehouse, which is started by unloading pallets from a truck via a forklift in groups of ten or eight. Once the pallets are stacked in the buffer zone, workers manually sort them into three categories indicating that they need repair, should be recycled, or are ready to use. The pallets belonging to the repair category move to the repair table to undergo the repair process, the pallets to be recycled move to the recycle station, and the ready-to-use pallets move to the buffer zone to be sent to customers. The key factors that contribute to injury are awkward lifting, reaching, or twisting, the pace of work, and repetitive motion. Regarding this point, Amazon recently published the main factors that contribute to injury in its warehouses [9]. Awkward lifting, reaching, or twisting is the highest factor at 28%, followed by heavy packages at 27%, pace of work/workload at 21%, and repetitive motion at 19%. In addition, the U.S. Department of Labor reported CLF occupational injuries, illnesses, and fatal injuries from 2016 to 2018 [10]. Figure 2 shows occupational injuries and illnesses by type of case for the warehouse and CLF industries. Overexertion and bodily reactions (OBR), contact with objects (CB), and slips, trips, and falls (STP) are the main factors involved in occupational injuries and illness.

Improved safety performance in the industry can best be achieved by preventing incidents from occurring in the first place [11]. A better education and training program for CLF workers is one way to achieve this goal. Historically, different approaches have been used to train workers on safety, such as video recordings, handouts, and hands-on training. Each of these methods has its advantages and disadvantages. For example, while video recordings and handouts are easy to produce and relatively inexpensive, they often fail to fully engage workers during the training process. Additionally, in all of the previous cases, keeping the information updated is a time-consuming process. Hands-on education, with the highest impact on training, has the highest cost and risk associated with it [12]. In particular, because CLFs are geographically dispersed, sending officers to conduct hands-on safety education is very costly, and due to a limited number of available officers, this could reduce the educational impact [13]. This current scenario, where the absence of modern technology is notorious, also leads to missed opportunities. In this setting, it is challenging to obtain key information that can be used to improve training methods, such as capturing typical errors and pain points.

In this context, immersive technologies such as augmented reality (AR), virtual reality (VR), and mixed reality (MR) are becoming increasingly common in industrial applications under the umbrella of extended reality (XR). The increasing availability of XR devices has led to the possibility of various forms of employee training. This training includes safety training, machine and equipment operator training, and maintenance training, as well as remote repair training in collaboration with outside experts [14]. Recently many researchers and industries have highlighted the advantages of XR in safety training, such as Rafael Sacks et al. [15], who examined the hypothesis that safety training in VR construction sites would be more effective in terms of workers’ learning and recall than equivalent training with conventional methods in identifying and assessing construction safety risks. Arachchige, Chander et al. [16] provided a brief overview of the design and development of a VR tool for physiological and subjective measures of anxiety with repeated exposure on construction sites. An industry-based expert flow was used to design and develop the VR application. Ankit Shringi et al. [17] proposed an interval type-2 fuzzy Delphi method to understand the effectiveness of different XR technologies in imparting important construction safety training to construction workers in a virtual environment compared to conventional classroom training sessions. Zuzhen Ji et al. [18] utilized VR and diminished quality of life (DQL) to develop a training method for occupational health and safety (OSHA). Both hazard identification and risk perception were improved through the development of the platform.

Safety training using VR, AR, and XR provides significant improvements in occupational injuries and illnesses and has been widely studied by many researchers. These training programs are mainly focused on educating about sequential operations, standardizing working procedures, and short-term safety risk assessment [19,20,21]. The majority of developed platforms seem promising for specific industrial aims without adapting training materials for hands-on and practical learning in the XR environment with a geographically distributed system.

Table 1. Comparative studies of XR technologies and traditional training methods by domain and industry.

XR-Technologies vs. Alternatives	Domain of Operation	Industry	Key Results	Reference
VR vs. lecture-based training	Safety training	Construction	VR has provided statistically significant results, being at least as effective as or better than traditional methods for immediate learning and over time. A key element for an effective learning experience in VR is maintaining smaller groups of trainees. Further research is needed to determine the optimal group size.	[15]
VR study only	Safety training	Construction	Stressful environmental conditions can negatively affect cognitive processing. In such cases, VR has been demonstrated to be an excellent tool to reduce the affected cognition.	[16]
VR vs. AR vs. lecture-based training	Safety training	Construction	Conventional methods of training are, by far, an inefficient method of training compared to XR alternatives (VR or AR).	[17]
VR vs. lecture-based training	Safety training	Chemical	The study shows that, for the homogeneous sample used, VR training outperforms conventional methods in both short-term and long-term hazard identification and risk perception.	[18]
VR vs. lecture-based training	Safety training	Water treatment	The VR alternative took significantly less time to share the same content without compromising the knowledge acquired. Other positive findings of the VR alternative are an overwhelming positive reaction of trainees and the easier accessibility that it provides.	[19]
VR vs. video training VR vs. paper-based training VR vs. lecture-based training	Safety training	Construction	This work showed that VR safety training is better than traditional methods in three dimensions: behavior, experience, and skills. Young workers with fewer years of experience benefit more from VR training than experienced workers.	[20]
VR vs. desktop training (verbal + presentation)	Safety training	Mining	While the obtained knowledge was comparable for both methods, VR training significantly improves the user confidence to perform the evaluated task.	[21]
VR vs. lecture-based training	Safety training	Industrial Electrical	VR improves engagement and enjoyment of training, strengthening the learning experience. Score evaluations immediately after and four weeks later were significantly better compared to traditional methods.	[22]
VR vs. video training	Operation training and best practices	Medical	VR simulation enhances operative performance and shortens operative times. Immediate feedback also boosts training quality	[23]
VR vs. video training VR vs. paper-based training VR vs. lecture-based training	Safety training	Construction, Fire Safety, Aviation, Mining	Construction and fire safety training are the most studied industries in the literature. VR safety training methods are more effective than traditional methods for both knowledge acquisition and retention.	[24]
AR vs. video training AR vs. paper-based training AR vs. lecture-based training	Safety training	Construction, Manufacturing, Transportation	AR outperforms traditional methods in providing safety training and demonstrates equivalent efficacy in knowledge acquisition.	[25]
AR vs. on-site training	Operation training and best practices	Forensic science	Easy accessibility to training and re-training material is fundamental for information retention, a feature that AR offers but on-site training cannot. AR is a highly customizable tool that enables the generation of multiple training scenarios in virtually no time.	[26]
AR vs. on-site training	Tactical and warfare operations training	Military	AR can serve as a platform to train military forces without compromising the effectiveness of the training. While the results seem promising, more AR studies are required, as only a few countries are currently exploring this area.	[27]
MR vs. simulated environments	Operation training and best practices	Aerospace	A multi-module approach has shown that MR can create suitable environments for astronauts to conduct their training at a fraction of the actual cost and with high customization. Results also suggest that combining MR with digital twins can help to obtain relevant KPIs immediately.	[28]
MR vs. on-site training	Operation and maintenance training	Maritime	MR technology enables the feasible and accessible generation of virtual training scenarios as well as the enhancement of physical settings. No significant differences in the gained knowledge, but MR has been proven to close the gap in training accessibility, providing an ‘everyone, everywhere’ experience.	[29]
MR study only	Operation training and best practices	Medical	MR technology is able to provide a significant training experience, even in complex scenarios like neurosurgery. MR has been proven to help identify relationships between complex variables that were difficult to grasp with 2D and 3D images	[30]

synthesizes the key findings mentioned previously and introduces additional insights from other relevant studies. This table’s structure is designed to facilitate the comparison of the specific XR technology used with traditional alternatives when training workers in different domains of operations and across several industries.

This paper presents the development of a modular virtual collaborative training and testing platform (MXC-P) utilizing a co-design process for designing each module for CLFs which can be transferable to other industrial cases. It provides a methodology to create an effective safety learning environment and scenarios in collaboration with safety professionals, CLF managers, and a team of researchers. The MXC-P is an educational software package which runs on the cloud and can be used by people with little prior computer or VR experience. It is composed of three modules, namely the personal protection equipment (PPE) module, the pallet handling (PH) module, and the pallet repairing (PR) module, which utilize a VR-based software capability to assist the workers in “visualizing” the concepts and to provide an immediate collaborative virtual environment between workers and safety officers, giving users the opportunity to graphically provide feedback during the learning or evaluating processes.

Through our study, we aim to make several theoretical and practical contributions. First, we propose a modular virtual collaborative platform for CLFs aimed at overcoming the challenges of OSHA compliance training. This platform offers cost-effective and hands-on learning experiences. Second, we design and develop each module through a co-design process, which ensures better educational content for end-users and addresses customization issues. Third, the proposed platform is implemented through a cloud-based and collaborative environment, enabling the capture of synaptic data for subsequent analysis of long-term health concerns.

The rest of this paper is organized as follows. Section 2 provides an overview of the proposed MXC-P platform. Section 3 presents the implementation approach. Section 4 explains the evaluation method and the results, while Section 5 provides the limitations and discussion. Finally, Section 6 concludes this paper and outlines future research directions.

2. The Overview of the Proposed Architecture

The proposed MXC-P was designed and developed as a software package. Figure 3 shows the overall proposed architecture of the MXC-P. It consists of four main parts, namely an educational content engine, which is used to co-design and create educational content, the users and developers, which include all players in the system, long- and short-term analysis engines, which provide data for analysis and evaluation, and a cloud-based immersive interactive environment, which is the main location of the MXC-P. During the development of the system, a set of fundamental functionalities and a set of software requirements for the system were identified to ensure coherent software generation, resulting in a highly modular and editable system. A new module can be added to the system by following sets of defined requirements in the development of new modules. There are some technical challenges involved in MXC-P development, such as the exchange of data among modules and users, which is resolved by assigning a database to each module and user profile and considering a data server with direct access to all the databases through an authentication and verification process to manage all the shared data in the program. Each layer is explained as follows.

2.1. Users and Developers

The user and developer parts include all direct and indirect individuals who engage with a CLF. This encompasses workers responsible for performing processes such as unloading, sorting, and repairing pallets; they are the main users of training materials, as well as the MXC-P. Managers are responsible for managing the CLF and facilitating the training materials. Officers are responsible for providing and administering training materials to workers. Safety officers are responsible for evaluating and validating the training materials. Researchers are working on improving training materials, in conjunction with policy makers.

2.2. Educational Content Engine

The educational content engine (ECE) is responsible for providing educational content for each module, such as objectives and scenarios. To design each module, a step-by-step co-design process is applied. A co-design approach is defined as a process of collaborative design thinking, involving joint inquiry and imagination in which diverse people jointly explore and define a problem and jointly develop and evaluate solutions [31]. Co-design is emerging as the best methodological approach for designing health services and educational games [32], and the solutions designed through this process appear more likely to be successful and sustainable. Following the co-design approach, the ECE has four sequential main steps, namely to empathize, define, ideate, and prototype. Each step is defined as follows.

• Empathize: This step involved a mixed-method study to describe and capture the existing safety training methods and materials for CLFs. It helped to capture data from files by applying a time study method and an interview with workers and domain expert. This approach provided information regarding the process, needs, and safety index related to each activity. This step helped us to define proper requirements for safety education based on the existing activities and process. It had three main phases. The preparation phase captured data from the field by applying the time study method and the interview process with workers and experts in the field and provided data regarding the process and safety index for each process. The investigative phase aimed to provide insights on the data and identify short-term and long-term safety aspects related to each activity through data which were delivered by the preparation phase as well as existing training content and materials. In addition, it was responsible for understanding the experiences and empowerment needs of people. The development phase aimed to achieve three objectives: first, to prioritize addressable safety needs based on short- and long-term aspects; second, to propose examples and possible scenarios for hands-on training of content for each module; and third, to assess the acceptability and usability of each module.
• Define: This step assisted us in discussing and analyzing the information we obtained from the empathize step to create actionable problem and objective statements. It enabled us to prioritize the safety training needs for a specific audience. All end users, such as workers, managers, and safety officers, played direct roles in formulating priorities and problem statements.

  Table 2. Key actionable problem statements.

  Level of Action	Key Actionable Problem Statements
  Worker level	Inadequate hand-on learning materials Poor training material and delivery method Limited confidence of workers in communication with safety officers and managers Poor self-learning practices
  Manager level	Poor communication between officers and managers Limited resources and contents Lack of integration between learning materials and manager’s level High cost of training Low satisfaction
  Safety officer level	There is a limited number of officers available. Unavailability of training long term health issues Lack of information, educational, communication training materials

  shows the key actionable problem statements.
• Ideate: Within the context of the problem statements and prioritizing the safety training needs, we generated conceptual design alternatives [12]. In this process, we utilized brainstorming and mind-mapping exercises, followed by convergent thinking, to synthesize and refine collections of ideas into cohesive module concepts. We shared the generated concepts with officers, workers, researchers, and policymakers to improve the modules based on their feedback. The entire ideate step led to the development of a conceptual model of modules and training content for each module. In this step, we proposed three main modules to represent key concepts and training materials for CLFs, namely the PPE module, the PH module, and the PR module.
• Prototype: In this step, the model of each module was validated for its conceptualization and appropriateness and subsequently refined. The objective of this stage was to initiate evaluation, reflection, and learning and typically to develop a single prototype of each module, which was required for the testing or implementation phase. Based on the ideate step, three modules were defined and, in the prototype step, each module was designed and developed.

2.3. Modular Virtual Collaborative Platform

The MXC-P was designed and developed as a software package in order to satisfy multiple users in a geographically distributed system. To enable users to connect to the platform remotely, we adopted a cloud network-based approach. Figure 4 shows the network properties and topology. To implement the proposed network topology, we utilized the Fusion API, which is designed to resemble regular Unity MonoBehaviour code [33]. Developers and officers are defined as authorized users on the platform who can play key roles in modifying the modules or activating specific modules for workers, who are defined as normal users on the platform.

The MXC-P has three main modules which were developed in the ECE. Each module is explained as follows.

The PPE module is focused on providing immersive, audio-based training regarding gloves, shoes, glasses, high-visibility vests, and hearing protection to CLF workers. Training materials related to each piece of personal equipment were designed based on the first three steps of the co-design method: empathize, define, and ideate. This approach allowed us to fully customize this information for CLF workers. Each user has the capability to immerse themselves and interact with each personal protection resource in order to improve their understanding of them. Figure 5 shows the key training materials and the characteristics of the PPE module.

Figure 6 illustrates the key characteristics of the PH module, which serves as a conductor for safety training spanning short-term and long-term dimensions, leveraging immersive and experiential pedagogical methodologies. The core instructional content encompasses pallet lifting, stacking, and sorting techniques. This module comprises three principal instructional modules, employing a hybrid instructional approach for knowledge dissemination. Initially, trainees engage with instructional videos pertinent to each task, subsequently transitioning to practical application within a virtual reality milieu, thus facilitating heightened cognitive assimilation and skill mastery.

Figure 7 depicts the PR module, concentrating on both tool and process safety in the context of pallet repair. This module is entirely hands-on, with activities such as nailing, board removal, and pallet cleaning being conducted within a virtual reality environment. Its primary objective is to facilitate worker comprehension of the repair processes while adhering to safety regulations. This module adopts an approach that integrates short-term and long-term safety considerations, thereby offering an optimal method for tool handling. Each tool is designed and developed to emulate real-world mechanisms, providing users with an authentic experience conducive to skill acquisition and safety awareness.

2.4. Analysis Engine

The analysis engine (AE) interfaces with the MXC-P to furnish training data to the CLF and a researcher in order to create the user portal, which includes the result of each module and personal progress. The AE gives CLF managers the ability to access the training data of workers across geographically distributed CLFs. Researchers could utilize these data to assess long-term safety aspects, particularly focusing on the PH and PR modules, within each process. Officers can use this engine to follow up training for work or evaluate the work based on the training performance.

3. Implementation

To implement the proposed platform, Unity3D was used as the main platform to create the interactive environment, Blender open source 3D modeling software was utilized to create 3D assets and textures, and we utilized the XR Interaction Toolkit framework, which is a high-level, component-based interactive system for creating VR and AR experiences. Photon Fusion 2 was used for networking and the C# language was used as the main programing language. The choice of C# as the primary programming language ensured robust control over the development process, allowing us to implement complex interaction systems, user interfaces, and data management functionalities. The platform was developed for OpenXR API, which supports both Windows and Android devices.

Avatars are one of the most important components of tools. They are virtual representations of humans, performing social roles in the MXC-P, corresponding to their roles and personas. Based on different topologies and abstractions, each service user can have different types of avatars. For example, Figure 8 shows the main factors of avatars in the MXC-P. There is more emphasis on ergonomics and human factors in MXC-P avatars than on facial representation. This is because through full-body avatars, we can capture long-term health information. In this study, we developed two different avatars based on abstraction and topology. The first one is based on high abstraction and low topology, which is used by officers or managers. The second one has low abstraction and high topology. This avatar contains 82 joints, Figure 8 shows the kinematic configuration of the avatar’s body and hands. Kinematic equations are used to obtain the positions of each joint according to the joint angles [34]. The kinematic equations applied followed the Denavit–Hartenberg (D-H) parameters [35]. This convention is commonly used for mechanism and robotics modeling. In this research, we based the human body kinematics on two main equations, the kinematic equation for the hand and the four-segment human body kinematic equation. Figure 8 shows the hand and body models which we used in the avatar.

Regarding the analysis of long-term health risks, the MXC-P platform incorporates an analysis engine (AE) designed to capture and process real-time data from users during their interaction with the virtual environment. The AE is integrated with a cloud-based system, allowing for the aggregation and analysis of data from multiple sessions and users. The ergonomic evaluation within the platform is conducted using a full-body avatar model, which simulates human kinematics with a focus on critical joints such as the knees, ankles, and shoulders. These data are then processed to generate ergonomic risk scores, identifying areas where the user may be at risk of long-term injury due to repetitive or awkward movements. For the analysis of these data, statistical methods are employed to assess the potential for cumulative injuries. The ergonomic risk scores are plotted against operation sequences, allowing for the identification of tasks that pose a higher risk to specific joints or muscle groups. In the case study presented, it was observed that certain pallet handling tasks led to “urgent to change” scores for the knees and ankles, indicating a significant risk for long-term health issues in these areas. This insight allows us to propose adjustments to training modules to focus on safer handling techniques and reduce the risk of chronic injuries.

4. Evaluation

4.1. Learning Evaluation

For the evaluation and verification of the proposed architecture, we utilized a quasi-experimental study [36]. We recruited 24 volunteers for this case study, aged 21 to 35. None of the participants had any prior scaling experience related to CLFs. For the purpose of comparative case studies, all the participants filled out an online questionnaire (see Appendix A), then we divided them into two separate groups, namely control and experimental groups. Each group comprised 12 participants. The control group were trained through existing safety training materials which were provided by our industry collaborator.

The participants in the experimental group were asked to use MXC-P to complete their safety training. First, they received guidance and experienced the operations of virtual reality devices to ensure that they were familiar with the virtual environment and virtual operations. Training started first with the PPE module, then the PH module and finally the PR module. Additional evaluation based on defined VR-specific scales was performed separately using the survey results, and the results are presented in Figure 9. We utilized an heuristic evaluation method [37] to identify design problems in a user interface. The applied approach involved gathering input from users to evaluate their experience using a Likert scale, ranging from strongly agree to strongly disagree, across various categories relevant to the immersive experience on the platform. Figure 9 presents the mean score of the sample for each category. Notice that in all categories, the score was 4 or above, meaning that on average, the group agreed that they felt conformable interacting in the proposed user interface. While this is a positive result, it also provides vital insights into what can be improved for a future design of this experiment. For example, enhancements could be applied to the visual design of objects and task, as well as to the interaction with these objects in the task, to improve the categories of ‘faithful viewpoint’ and ‘realistic feedback’. By ensuring a coherent standard across the platform with a clear interface that helps the user to complete the task, we will address the categories of ‘consistent departures’ and ‘navigation and orientation support.’

After training, both groups were asked to fill out the same Likert-type questionnaire with responses from strongly agree to strongly disagree (see Appendix B) to evaluate their own learning outcomes.

Table 3. Demographic information of the participants.

Category	Details	Male	Female	Total (n)
Age	19–25	8	4	12
	26–30	4	4	8
	31–35	2	2	4
Occupation	Full-time	2	2	4
	Part-time	8	4	12
	Student	4	4	8
Level of Knowledge on Warehouse Safety	Novice	6	8	14
	Intermediate	5	2	7
	Advanced	3	0	3
Experience with Virtual Reality	Novice	6	4	10
	Intermediate	5	5	10
	Advanced	4	0	4

shows the demographic information of the participants; the majority of participants were between 19 and 25 years old, with minimum CLF experience, which is highly representation of the majority of workers in CLFs.

In order to evaluate our participants’ learning outcomes using the MXC-P, statistical software was used to analyze our data. In the first step, an independent sample t-test was conducted to compare the learning outcomes between the control and experimental groups. The results indicated statistically significant differences between the two groups (p-value < 0.05), indicating that the MXC-P provides a more effective learning environment for the experimental group compared to the control group.

Table 4. Descriptive statistics for both the experimental and control group.

Question	Group	Mean	Std. Deviation	t-Value	p-Value	Cohen’s d/Standardizer
Q1	Experimental	4.170	0.835	3.331	0.003	0.919
	Control	2.920	0.996
Q2	Experimental	4.670	0.492	13.140	<0.001	0.590
	Control	1.500	0.674
Q3	Experimental	3.580	0.793	6.680	<0.001	0.733
	Control	1.580	0.669
Q4	Experimental	4.330	0.778	7.088	<0.001	0.749
	Control	2.170	0.718
Q5	Experimental	4.500	0.674	6.780	<0.001	0.696
	Control	2.600	0.750
Q6	Experimental	4.170	0.718	9.120	<0.001	0.694
	Control	1.480	0.669
Q7	Experimental	4.670	0.492	3.083	<0.001	0.587
	Control	1.580	0.669
Q8	Experimental	3.830	0.835	2.600	<0.001	0.651
	Control	1.170	0.389
Q9	Experimental	4.080	0.669	2.667	<0.001	0.597
	Control	1.420	0.515
Q10	Experimental	4.500	0.522	3.000	<0.001	0.522
	Control	1.500	0.522

summarizes our results. It is worth mentioning that participants’ understanding of long-term safety (Q8) displayed the lowest mean within the experimental group (M = 3.83), although this is still higher than the control group (M = 1.17). This suggests that while the MXC-P approach improves understanding of long-term safety, there is still room for improvement in this area. In the control group, the questions Q2 (1.5), Q3 (1.58), Q6 (1.48), Q7 (1.58), Q8 (1.17), Q9 (1.42), and Q10 (1.5) had mean scores less than the acceptable level, indicating that learning in the control group is not at an acceptable level.

Cohen’s d effect size for all questions was greater than the medium effect size (d = 0.5), indicating that the effect sizes for these questions are substantial.

4.2. Long-Term Safety Evaluation

The real-time ergonomic evaluation of the PH module via the full body avatar is shown in Figure 10, plotted with the operation sequence on the horizontal axis and the ergonomic evaluation score on the vertical axis. As Figure 10 shows, there are potential long-term health issues with the knees and ankles. During pallet handling, the knees twice reached the “urgent to change” area in the ergonomic evaluation score, indicating the need to propose better pallet handling approaches to reduce the long-term health impact on the knees. This occurred once for the ankles, indicating that the ankles in the pallet handling module could develop long-term issues in the future. We can also conclude that the shoulders would experience minimal impact regarding long-term health issues and would mostly remain in the acceptable range, which is a good sign for shoulder health. Figure 11 shows the ergonomic evaluation of the PH module based on the left and right hands. We can conclude that the right hand performed more movements than the left hand, which might cause long-term health issues, as the score crossed into the “urgent to change” area.

While the initial results from our study demonstrate the effectiveness of the MXC-P platform in improving safety training outcomes, it is crucial to consider the long-term efficacy of the training provided. The effectiveness of safety training can diminish over time if the skills and knowledge acquired are not regularly reinforced. Therefore, a refresher course is mandatory every six months at CLFs. The MXC-P platform, developed as a cloud-based system with a user portal, has the capacity to incorporate these refresher courses and continuous learning modules.

These refresher courses can be scheduled at regular intervals based on the specific needs of CLF workers and the nature of the tasks they perform. By integrating periodic training updates, the platform ensures that workers’ knowledge remains current and that safety practices are consistently reinforced. Additionally, the platform can track individual progress and identify areas where further training may be required, allowing for a personalized approach to long-term training efficacy.

The long-term health risks identified through the ergonomic evaluation of tasks, such as those related to knee and ankle strain during pallet handling, can also be mitigated by incorporating targeted training modules that focus on proper body mechanics and safe handling techniques. These modules can be revisited periodically to reinforce safe practices and reduce the risk of cumulative injuries.

5. Limitations of the Work

This study has several limitations. One significant limitation is the need for a larger and more diverse sample size, as the current number of participants and their range are limited. Existing research in the field [16,22] demonstrates that small sample sizes are often accepted as sufficient for preliminary stages of evaluation, even when the sample is divided into several groups [29]. Furthermore, in [15], it is stated that for VR training, it is desirable to use small samples so that all of the trainees can benefit from the high degree of customization that XR technologies allow. This acceptance of small samples in the literature is explained, since the goal is to explore feasibility or gather initial insights [38,39]. In early-stage studies like ours, the primary focus is on identifying potential trends, testing methodologies, and refining experimental designs, rather than achieving broad generalizability. This approach allows researchers to make informed decisions about whether to pursue larger-scale studies and to finetune the study design for subsequent, more comprehensive research. The data capturing method and avatar enrichment could be enhanced by incorporating more sensitive sensors and advanced video analysis algorithms. The bias in data regarding joint movements could be mitigated by using a more sophisticated video analysis system. The case study and platform were developed with industrial workers in mind, especially in CLFs, and thus how well the method might apply to other areas has not been determined. The long-term safety evaluation can be improved by integrating predication methods using effective machine learning approaches.

6. Conclusions

This study addresses the critical need for improved safety and health training in the warehouse and logistics sectors, particularly within CLFs. Despite advancements in safety protocols and training methods, warehouse accidents and injuries remain significant concerns, necessitating innovative solutions. Our research presents the development of an MXC-P designed to enhance safety training through immersive and interactive virtual environments. The MXC-P is structured into three primary modules: personal protection equipment (PPE), pallet handling (PH), and pallet repairing (PR). These modules were developed using a co-design process involving safety professionals, CLF managers, and researchers to ensure relevance and effectiveness. The platform provides a collaborative and remote training environment, offering hands-on experience and generating data for long-term health risk evaluation.

A pilot study comparing the MXC-P with traditional safety training methods demonstrated the platform’s effectiveness in improving safety learning and knowledge recall. Trainees using the MXC-P showed better engagement and comprehension, indicating that virtual collaborative training can significantly enhance learning outcomes. The MXC-P also offers the potential for widespread application across geographically distributed systems, making it a cost-effective solution for training large workforces. The ability to generate and analyze data on long-term health risks further underscores the platform’s value in promoting occupational health.

The results from the quasi-experimental study indicate that the MXC-P significantly enhances the learning outcomes for participants compared to traditional training methods. This was evidenced by the statistically significant differences observed in the independent sample t-tests between the experimental and control groups. The experimental group, which used the MXC-P platform, showed higher scores in all evaluated dimensions, including the understanding of short-term safety practices, engagement with training materials, and knowledge retention. Specifically, the data demonstrated that the MXC-P was particularly effective in areas such as the proper use of personal protective equipment (PPE) and pallet handling techniques. This suggests that immersive and interactive environments can better reinforce critical safety practices.

Another important point that emerges from the results mentioned above is that the use of immersive environments for safety training opens up encouraging avenues of research for other areas within the company, in particular how the MXC-P can be used in different scenarios, such as the risk assessment of load manipulation while loading/unloading a container, the appropriate and safe use of forklifts and other vehicles within a warehouse, and proper crane manipulation. This will allow researchers to create more studies for the logistics industry, a sector where the literature for safety training and virtual training is scant. Moreover, introducing a well-functioning modular, virtual, and collaborative platform, with promising preliminary results, represents a step forward in how companies, from different industries, can think, design, and deliver safety training and training in general and leverage MXC platforms.

However, the conclusions drawn from this study must be considered in the context of its limitations. The small sample size and the preliminary nature of the study mean that further research is necessary to confirm these findings. Future studies should involve larger and more diverse participant groups, as well as a longer follow-up period to assess the long-term impact of the training.

In conclusion, the MXC-P represents a significant advancement in safety training for warehouse environments. By leveraging virtual reality and a collaborative design process, the platform provides a robust and scalable solution to address both immediate safety concerns and long-term health risks. Future research will focus on expanding the dataset and refining the platform with advanced sensors and video analysis to enhance training efficacy and health monitoring.

As future work, we plan to address this limitation by increasing the sample size in subsequent studies. This will enable more robust statistical analyses and enhance the reliability and generalizability of our findings, ultimately strengthening the validity of our proposed approach.