Development and Usability Evaluation of Augmented Reality Content for Light Maintenance Training of Air Spring for Electric Multiple Unit

Abstract

The air spring for railway vehicles uses the air pressure inside the bellows to absorb vibration and shock to improve ride comfort and adjust the height of the underframe with a leveling valve to control stable driving of the train. This study developed augmented reality content that proposes a novel visual technology to effectively support the training of air spring maintenance tasks. In this study, a special effect algorithm that displays the dispersion and diffusion of fluid, and an algorithm that allows objects to be rotated at various angles, were proposed to increase the visual learning effect of fluid flow for maintenance. The FDG algorithm can increase the training effect by visualizing the leakage of air at a specific location when the air spring is damaged. In addition, the OAR algorithm allows an axisymmetric model, which is difficult to rotate by gestures, to be rotated at various angles, using a touch cube. Using these algorithms, maintenance personnel can effectively learn complex maintenance tasks. The UMUX and CSUQ surveys were conducted with 40 railway maintenance workers to evaluate the effectiveness of the developed educational content. The results showed that the UMUX, across 4 items, averaged as score of 81.56. Likewise, the CSUQ survey score, consisting of 19 questions in 4 categories, was very high, at 80.83. These results show that this AR content is usable for air spring maintenance and field training support.

1. Introduction

Railways are a primary mode of public transportation, and reliable maintenance is essential to prevent inconvenience and traffic congestion for passengers due to train failures during operation. Therefore, the maintenance of railway vehicles is a crucial factor in preventing performance degradation and failures. Maintenance of railway vehicles involves tasks to restore the vehicles to their original state due to issues such as performance degradation, aging, and failures, requiring high adaptability and expertise. Maintenance costs account for 70% of the total lifecycle cost, unlike automobiles [1]. Hence, railway operating agencies are considering specialized maintenance training to minimize personnel errors and prioritize reliability while minimizing maintenance costs.

Recently, many industrial sites are emphasizing predictive maintenance using condition-based maintenance (CBM) and smart maintenance. CBM monitors the actual condition of equipment using sensors to determine maintenance needs. Smart maintenance integrates technologies like artificial intelligence (AI) and the Internet of Things to predict and prevent equipment failures, transitioning industries towards smart factories—highly digitized and connected environments [2,3,4]. AI involves creating systems capable of tasks requiring human intelligence, such as learning and problem solving. Sensor technologies detect and measure physical properties, converting these into data for analysis. This digital transformation enhances efficiency and productivity in manufacturing and maintenance. Core technologies of the Fourth Industrial Revolution include big data, AI, augmented reality (AR), and virtual reality (VR). Big data involves analyzing large volumes of data for insights. AR overlays digital information onto the real world, while VR immerses users in a virtual environment. Both AR and VR provide valuable information without time and space constraints, improving remote assistance, visualization, and interactive training in maintenance and manufacturing.

1.1. Related Work

1.1.1. AR-Based Maintenance Content

AR content related to maintenance and repair is being researched in various fields to enhance the efficiency of maintenance operations. AR training is extensively studied in education, industry, vocational training, and healthcare fields. It is frequently used in medical training, industrial maintenance management, and assembly fields [5,6].

Yazdi investigated methods to integrate AR and VR technologies into maintenance training programs [7]. They proposed overlaying digital information onto physical equipment to enhance education, provide real-time guidance, and offer interactive simulations for trainees. Akulov et al. [8] developed computer education tools, such as AR, to enhance the practical skills of students in railway universities and provide hands-on experience for designing, implementing, and improving tools. Zhong et al. [9] created a customized VR system for elevator maintenance practical training to address issues like inherent risks, high educational costs, and spatial constraints. They employed interactive methods to help students master technical skills related to elevator structure, operational principles, and governor adjustment. Darmawan et al. [10] analyzed the application of AR in vocational education to complement standard curricula and provide specific learning experiences.

In the industrial maintenance sector, Brown et al. [11] applied AR and VR in the context of ergonomics across education, aviation, and maintenance domains, offering a new approach to education through immersive environments and interactive simulations. Garcia et al. [12] introduced a customized AR Holorailway system to support assembly operations within the railway industry. By integrating AR technology into assembly processes, they provided workers with real-time guidance and visual data, increasing efficiency, accuracy, reducing errors, and enhancing productivity. Prathibha et al. [13] proposed integrating AR and VR technologies into maintenance and repair workflows for automotive and machinery equipment to enhance the efficiency, accuracy, and reliability of technicians, emphasizing the potential for innovation in maintenance and repair practices. Kwon et al. [14] developed AR learning content focusing on pneumatic flow in railway brake operation devices. They described the development process, including considerations for AR content design and implementation, and evaluated the effectiveness of AR learning content.

1.1.2. AR Content Evaluation

Research on AR content not only involves development but also continues to explore the validation of AR content effectiveness. Studies on usability evaluation, such as user experience assessments, involve conducting surveys to evaluate how users interact with and perceive AR prototypes, providing recommendations for improvement based on user feedback [15]. Martin-Gonzalez et al. [16] evaluated the usability and learning potential of an AR system developed for grading Euclidean vectors in physics and mathematics through a system usability scale (SUS) assessment. Wang et al. [17] evaluated the usability scale and NASA-Task Load Index of an AR-based instruction system developed for maintenance tasks, presenting the ARBI methodology with the shortest task completion time and the fewest errors. Lauer et al. [18] investigated the usability of HoloLens 2 using standardized tutorials on AR’s multimodal interaction among 47 elementary school students (grades 2–6). While the overall system usability resulted in a “good” outcome, various behavioral metrics revealed differences in the efficiency of specific interaction modes. Dutta et al. [19] developed an MAR application, utilizing keypad- and marker-based interactions, to teach Karnaugh mapping in digital electronics courses. Evaluation with 90 engineering students using the SUS and HARUS revealed that the keypad-based MAR application yielded superior evaluation results, indicating better user interaction. De Boer et al. [20] developed augmented-reality-based robot control content and conducted an objective usability evaluation using the Usability Metric for User Experience (UMUX) survey to assess remote presence. Mitaritonna et al. [21] developed an augmented-reality-based software architecture for military scenarios, which effectively responded to situational awareness, and obtained usability evaluation scores through UMUX. Criollo et al. [22] created augmented reality content for engineering education via mobile applications to facilitate new learning experiences and conducted an objective usability evaluation using the Computer System Usability Questionnaire (CSUQ) survey. Caria et al. [23] developed augmented-reality-based industrial operation content for performance and precision machining industries and performed an objective usability evaluation of the developed content through the CSUQ usability assessment.

1.2. Research Objectives

The application of AR technology for railway vehicle maintenance is currently limited to key mechanical equipment such as the bogie frame and braking system [24,25]. Existing AR content utilizes touchscreen gestures to control position, size, and rotation, achieving intuitive learning effects by developing algorithms to represent pneumatic flow diagrams related to compressed air systems, which are challenging to identify with the naked eye. However, when developing AR for air springs, the use of existing gestures was limited in usability due to the characteristics of the axisymmetric model. Additionally, while the existing Stream Line Matching Variable Calculation (SLMVC) algorithm and Continuously Emission Property Correction (CEPC) algorithm can represent compressed air system flow, these algorithms struggle to depict diffusion phenomena such as leakage [14].

This study aims to develop AR training content for maintenance, connected to a database and storyboard, centered on the 3D model of an air spring device that enhances vehicle ride comfort by adjusting air pressure. Additionally, the study aims to facilitate onsite maintenance of air spring devices and deliver efficient training by accomplishing the following objectives:

First, it is necessary to develop MAR content that includes specifications, detailed structures, actual work procedures, and supplementary photos and videos to assist with onsite tasks. This will replace the inefficient paper-based theoretical and practical training. The content should take into account the restricted maintenance environment of the railroad vehicle air springs installed under the vehicle and ensure easy portability and visibility.

Second, to improve user visibility and facilitate maintenance tasks, it is crucial to utilize the current gesture method and create a 3D object auto-rotation algorithm. Furthermore, new algorithms should be developed by integrating existing ones to replicate fluid flow and visualize air leakage. This will enhance maintenance support and training efficiency.

Third, the maintenance process of the air spring should be analyzed to include visual inspection, height measurement, leakage inspection, and repair. The essential maintenance tools should be readily available during maintenance tasks, and procedures should be presented in an animated format to enhance the practical maintenance training scope.

The usability of AR content can be evaluated through various methods. Types of surveys include SUS, CSUQ, UMUX, and Questionnaire for User Interface Satisfaction (QUIS). Previous studies have used UMUX and CSUQ survey methods for user experience evaluation to objectively assess the effectiveness, efficiency, and usability of AR systems developed across various fields. The UMUX and CSUQ are widely used metrics for evaluating the usability of systems, including educational content developed using AR. UMUX provides a quick assessment of perceived usability, while CSUQ offers a more detailed evaluation of system usefulness, information quality, and interface quality. Combined, these tools offer a comprehensive framework for assessing the usability of AR-based educational products and can serve as objective measures in usability evaluation [26,27,28,29].

This study similarly employs UMUX and CSUQ surveys to conduct an objective evaluation of user experience [20,21,22,23]. The usability of the developed air spring AR content and document manual was evaluated by surveying 80 professionals involved in railway vehicle maintenance using the UMUX and a 19-question CSUQ tool. The UMUX score of the air spring AR content was 81.56, which is approximately 10% higher in learning efficiency than the 73.66 scored by the document manual, and the CSUQ results also demonstrated very high usability, with an average score of 80.83 out of 100 across 19 questions, which is 7.54 scores higher than the 73.29 scored by the document manual. Therefore, the developed AR content combines the previously used algorithms, SLMVC and CEPC, with the newly developed Fluid Diffusion Generation (FDG) and Object Auto-Rotation (OAR) algorithms. This allows for the sequential visualization of the flow of compressed air and the diffusion phenomena caused by leaks, not only depicting fluid flow but also diffusion. This systematic learning approach enhances the accuracy of fault detection and determination of fault locations, improves the troubleshooting skills of maintenance personnel, and significantly boosts the educational effectiveness of training maintenance personnel to perform maintenance tasks more proficiently.

2. Implementation Devices

2.1. Industrial AR

Industrial AR enhances productivity and efficiency by combining real-world and virtual information, as shown by the SUS and systematic literature reviews covering AR applications, industry interest, development, benefits, and challenges [30,31]. These systems primarily deliver necessary data to workers visually through AR glasses or mobile devices. These data serve as advanced instructions and guidance, helping workers perform tasks more efficiently. For instance, during machine assembly, real-time display of installation instructions or work procedures for actual parts can minimize errors and maximize productivity. During maintenance tasks, it can similarly assist in recognizing faulty parts and guiding repair procedures. Industrial AR technology is recognized as a crucial part of digital transformation, and, consequently, the demand for it continues to grow. It is expected that this technology will be more widely adopted across various industries, including manufacturing, automotive, shipbuilding, and energy, in the future. Many domestic and international companies are developing such industrial AR technologies, providing various solutions tailored to the Fourth Industrial Revolution era. These technologies not only contribute to improving companies’ productivity and competitiveness but also play a significant role in enhancing the safety and convenience of workers. The air spring AR content developed in this paper was created to provide work guidelines necessary for the maintenance of railway vehicle air springs by applying industrial AR.

2.2. AR Implementation Device

AR technology mainly utilizes wearable head-mounted display (HMD) devices and mobile devices. Representative examples of wearable HMD devices include Microsoft’s HoloLens, Google’s AR Glasses, and Magic Leap [32,33]. These devices provide users with a realistic experience by offering a perspective similar to that of actual glasses. However, the display brightness is low, the field of view is limited, and there are issues with the accuracy and speed of hand-tracking functionality. Additionally, the high cost of each device presents challenges for widespread adoption. On the other hand, MAR does not require separate HMD devices but is implemented through individual mobile devices. This eliminates constraints on the field of view and provides users with portability and accessibility. Furthermore, it is relatively affordable and easily accessible. In terms of technological trends, it has been observed that companies like Google and Apple have recently discontinued their development of AR HMD devices. This indicates the challenges in commercialization due to the inconvenience and low utility of HMD-based AR technology in real-life scenarios. Moreover, there are technical limitations concerning the field of view.

Table 1. Examples of overseas research content.

Division	Mobile Device	HMD Device
Interaction	Touch gesture and button-based interaction.	Hand tracking-based interaction.
Point of view	Experience with perspective applied like an actual camera.	Experience with perspective provided like actual glasses.
Usability	Adjustable display brightness, no restriction on field of view.	Low display brightness, narrow field of view.
Deployment environment	Easy distribution on personal smartphones and tablets.	Difficult distribution due to high cost.

provides a comparison between HMD devices and mobile devices.

AR technology applied to railway vehicle maintenance requires extensive development of maintenance manuals, involving long-term time, high costs, and constrained environments. Ensuring safety amidst limited visibility and a surge in inexperienced personnel necessitates advanced technical support. Therefore, this study focuses on designing AR content based on MAR, universally deployed on cost-effective, portable smartphones and tablets [34,35]. This approach ensures a broad field of view, stability, and real-time high-level technical support.

3. AR Content Development Process of the Air Spring

In order to create maintenance AR content for air springs, the process involves 3D modeling through scanning, converting the extension, creating a mind map and storyboard based on the manual, and developing a user interface to implement the work. Figure 1 shows the development and design process of AR content.

3.1. Air Spring 3D Modeling

For railway vehicles, the air spring is installed at both ends of the car frame center, as shown in Figure 2. It receives compressed air at 4–6 bar pressure from the air compressor, which is applied through the brake operating unit. The air spring serves as a suspension device with smooth damping performance by compressing the gas and applying fluid resistance. Optimizing damping performance with pneumatic systems requires meticulous design of the air spring as a 3D model, tailored to its stiffness characteristics. The stiffness characteristics are analyzed using innovative methods, including advanced 3D finite element analysis techniques. These techniques comprise the alternating direction implicit finite difference scheme for the three-dimensional time-fractional telegraph equation, and the second-order backward differentiation formula alternating direction implicit Galerkin finite element method for handling evolution equations with nonlocal terms in three-dimensional space [36,37]. It improves ride comfort through vertical and lateral motion damping and adjusts vehicle height through leveling valves.

The maintenance process for air spring assembly and disassembly is straightforward and involves external and dimensional inspections. Maintenance personnel conduct visual inspections of the external surfaces and contact areas of bellows, upper plates, lower plates, and skid plates for cracks and leaks caused by friction. Additionally, dimensional inspection measures the standard height between the bottom of the car body frame and the bogie frame to check whether it meets the specified tolerance and performs height adjustment. To measure the height of these air springs, information such as the tools provided in the maintenance manual, height measurement criteria, and height adjustment procedures are required. Since such external inspections and dimensional checks are difficult to identify visually, relying solely on 2D drawings poses a problem. Therefore, it is important to develop AR content that displays the hierarchical structure of the air spring and the air diffusion due to leakage in a 3D format for the workers.

Figure 3 shows the process of developing a 3D model of the air spring for the AR maintenance training content. To provide accurate model information for the 3D model’s shape, the manufacturer’s 2D drawings and prototypes were gathered using a 3D scanner (Shining 3D, Hangzhou, China), as shown in Figure 4, in ASC format. The 3D CAD model was generated using the Geomagic Design X program [38] and CATIA V5 [39,40] based on the scanned shape data, saved as an XRL extension file. The XRL format 3D model was converted to STP for integration into Unity [41], and the final model file was produced as an FBX format Unity 3D model file.

3.2. Storyboard

Figure 5 shows a mind map that summarizes the information from the maintenance manual necessary for creating mobile-based AR educational content for air spring maintenance. It is based on the existing document-based maintenance manual performed by the operating company, with additional information added through interviews with maintenance technicians performing related maintenance tasks at the operating agency. The top-level categories consist of general, structure, and maintenance. These top-level categories are further divided into a total of 13 mid-level items according to the three classifications. Among these, in the maintenance section, the inspection confirmation items for four air spring components (bellows, upper plate, lower plate, sliding plate) are structured, and steps such as tools for inspection, height measurement, and height adjustment for checking the air spring height are included. Therefore, various options were provided to specify the tasks required for the production of air spring AR content.

Figure 6 shows a sample storyboard of AR content based on the mind map created in the process of developing the air spring maintenance content in Figure 5. The left part (Screen) of Figure 6a represents the AR environment screen UI interface and text, while the right part (Scenario) represents the description of the scenario that each button constituting the UI performs when the function is activated. Figure 6b shows a storyboard example for creating visualized animations for tasks that can be represented by step-by-step learning within the content. It elaborately describes the sequential process, time-based settings, and additional functions displayed on the mobile screen. The storyboard is continuously modified and updated based on worker interviews and the updated manual.

3.3. User Interface

When applying AR technology to mobile devices, the implementation of the user interface (UI) should allow users to quickly and efficiently acquire maintenance-related information through their mobile devices. In particular, the components of the UI within limited space should be appropriately arranged to consider spatial utilization.

Table 2. UI components and descriptions of AR content.

Component	Mobile Device
Text	Explains long and complex maintenance details in small amounts of text.
Color	Maintains color unity and promotes visual perception.
Icon	Provides visual unity so that even beginners can easily understand.
Typography	Balanced and intuitive design that saves as much interface space as possible.
Image	Photos and language information are provided in real-life situations.

represents the UI components and descriptions of the air spring AR content. This study establishes guidelines for the UI components of the air spring AR content to ensure efficient information delivery.

Railway maintenance procedures are highly specialized and complex. Therefore, information needs to be summarized into concise text through the editing process by experts, enabling users to quickly and systematically acquire information. Colors should maintain consistency throughout the screen to facilitate visual perception for users. Icons should employ flat designs standardized for easy understanding by new or novice maintenance personnel, providing visual coherence. Typography should conserve space on the small screens of mobile devices and ensure a consistent application of font type, size, and intuitive experience. Images should offer users accurate and swift information, depicting actual railway incidents or device photos, considering the relatively unfamiliar characteristics of the railway sector, accompanied by relevant language information.

Railway vehicles vary widely depending on the route and operator, and each type is composed of numerous components. Furthermore, each component consists of a complex structure composed of numerous subcomponents according to the bill of materials. Therefore, developing educational content for the railway vehicle sector results in complex and diverse menu structures due to the richness and diversity of related components. Creating menus using conventional uniform methods requires significant time and effort due to the repetitiveness of the task.

Figure 7 is structured in a three-step format for implementing AR content UI. Firstly, the Global Navigation Menu Bar (GNMB) on the left side of the AR screen, as shown in Figure 7a, constitutes a large category menu composed of the top-level items from Figure 5 (General, Structure, Maintenance). Next, the Local Navigation Menu Bar (LNMB), as shown in Figure 7b, appears as a submenu when one of the categories from GNMB is clicked, containing the detailed content of the top-level items (General, Structure, Maintenance) from Figure 5. Finally, the Side Navigation Menu Bar (SNMB), as shown in Figure 7c,d, includes a detailed text pop-up window and other inspection item table formats to display detailed information through text, photos, etc. Additional functionalities like “Hide Menu” are also available. Such organization enhances menu readability and increases user convenience in operation.

4. Development of the Algorithm and Implementation Process

4.1. Fluid Diffusion Generation Algorithm

Since air springs are systems that use compressed air, it is visually challenging to detect diffusion phenomena such as leaks in emergency situations. To address this issue, the FDG algorithm has been developed to visualize air leakage inspection, which involves the use of spray-gun particle and bubble particle processes.

Unity’s particle system allows for controlling small images or meshes to render and animate them, enabling the creation of various visual effects by setting parameters such as size, direction, and shape [42]. This system is particularly useful for implementing dynamic objects that are difficult to depict in 2D or 3D, such as fire, smoke, or liquids. This study utilized a particle system to represent the flow of air necessary for the operation of the air spring, which is not visible to the naked eye, through special effects.

The FDG algorithm was implemented using Unity’s particle system and prefabs. This algorithm aims to visually represent the dispersion and diffusion of fluids effectively in AR. Using a particle system, the flow of spray-gun and bubble particles is visually represented. The particle system module includes 24 properties that affect the particle system. To simplify the complexity of these attributes, the algorithm primarily adjusts four key properties: the initial velocity, size, lifespan, and emission rate of the particles. Additionally, specially designed prefabs are employed to mimic the movement of actual fluids, allowing for the repeated use of objects like particles. Through these configurations, the diffusion and dispersion of fluids are visually and intuitively conveyed to the user. Figure 8 shows the flowchart of the FDG algorithm.

Figure 9 shows the mechanism of the flow of compressed air using particles until leakage occurs and spreads. As shown in Figure 9a, in Step 1, the particles according to the SLMVC algorithm proceed linearly as a continuous flow of compressed air, and in Step 2, they change into point-shaped particles. As shown in Figure 9b,c, bubble-shaped particles diffuse in the direction of flow, and the particle size gradually increases. The FDG algorithm enhances the stability of the output, allowing for the intuitive learning of air leakage phenomena, which are difficult to identify with the naked eye.

4.2. Object Auto-Rotation Algorithm

Due to the axisymmetric nature of the air spring model, manipulating rotation using finger gestures for specific directions, such as rotation or tilt tests, is challenging. To address this issue, an OAR algorithm was developed using a three-dimensional coordinate system to automatically rotate objects in a certain direction. Figure 10 shows the flowchart of the OAR algorithm. The OAR specifies the rotation area when designating a three-dimensional model to obtain the rotation reference coordinates. By clicking on rotation buttons (Top, Bottom, Left, Right, Main), users can rotate the object in the desired direction. However, implementing a rotation system within the AR space leads to the problem of gimbal lock, where the rotation axes overlap. To address this, the OAR algorithm introduces constraints to mitigate gimbal lock by allowing a slight error of 0.01° for angles where rotation axes overlap (90°). By using this OAR algorithm, axially symmetric models implemented as 3D objects, which are difficult to rotate using gestures, can be automatically rotated with button controls, providing a viewpoint transition effect for inspecting parts during learning. Additionally, this algorithm can be usefully applied as a means of viewpoint transition along with gestures in the creation of other content. Figure 11 shows the mechanism of automatic object rotation using a cube object developed through the OAR algorithm. Figure 11a shows the initial 3D model implemented on the AR screen. As shown in Figure 11b, clicking the front button on the bottom cube displays the front view, and clicking the side button rotates the 3D model to show the right side view, as shown in Figure 11c. Clicking the UP button on the top of the cube displays the top view of the 3D model, as shown in Figure 11d.

5. Training Support Air Spring Assembly AR Contents

5.1. General

The general section of the air spring AR content includes general information, inspection cycle, and specifications as intermediate items. As shown in Figure 12a, the air spring’s location within the railway carriage operating device can be inspected. Moreover, by clicking the button at the bottom right of the GNMB’s general section, users can access information about the operating principles of the air spring device and precautions for handling, as shown in Figure 12b. Additionally, separate pop-up windows provide access to the inspection cycle and specifications through buttons in the GNMB. The inspection cycles pop-up window presents criteria and checklist items for inspections (daily, periodic, intermediate, comprehensive) in a tabular format, including criteria for inspecting the exterior condition and height of the air spring. Similarly, the specifications pop-up window displays a table summarizing the specifications of the air spring.

Operation Principal

Figure 13 shows the airflow adjustments to account for variations in height resulting from vertical, lateral, and torsional movements of the air spring due to air pressure. The utilization of a particle system effectively demonstrates the airflow passing through the leveling valve as the air spring experiences loading, offering a three-dimensional representation.

5.2. Structure

The hierarchical structure section includes seven intermediate items, corresponding to the seven subcomponents of the air spring: bellows, upper plate, lower plate, upper seat, lower seat, auxiliary spring, and slide board. This structure provides an effective learning framework for understanding the overall combined structure of these components and subcomponents. Figure 14a shows the complete exploded view of the air spring assembly. Users can utilize touch gestures to rotate the view at any angle to observe the connections and detailed hierarchy between components and use the hide menu button located at the top of the screen. By clicking on any of the seven components on the left, users can access additional information such as sectional information on the 2D drawings of each component and functional descriptions, as shown in Figure 14b. Thus, users can effectively learn the hierarchical structure of the air spring assembly before dismantling the device, which was previously unknown before starting the operation.

5.3. Maintenance

The maintenance structure section consists of three intermediate items for inspecting the appearance and air spring height, as well as conducting a leakage test. The appearance inspection includes checking the bellows, upper plate, lower plate, and sliding plate for defects. As shown in Figure 15a, the areas requiring inspection are simultaneously displayed on the 3D model and 2D cross-sectional view, with highlighted animations to guide users to the specific parts that require attention. Moreover, visual inspections for air spring damage are supported by pop-up windows displaying photos and descriptions of actual damage cases, as shown in Figure 15b, to prevent human errors during the inspection process.

The height measurement of the current device is represented by a separate pop-up window with real-life photos of the tools used in the inspection process, as shown in Figure 16a. Before performing the height measurement operation, an animation, as shown in Figure 16b, explains the specification of the adjustment plate and the method for calculating the reference height. The actual height measurement is displayed by creating a reference plane using punching marks on the lower chassis and the side of the main frame, as indicated by yellow and orange texts in Figure 16c, respectively. The numerical values are presented in red text using dimension reference lines.

During the height adjustment process, separate buttons are placed at the bottom right to allow users to learn the actions based on the reference height measurement values. Additionally, to improve the realism of the work process animation, the specifications of the tools are clearly presented in a separate text, as shown in Figure 17a. Figure 17b shows how to grip and utilization of the tools using finger models. Moreover, animations were created to reflect the maintenance timing of mechanics during actual operations. As a result, users can experience a lifelike atmosphere similar to that of an actual working environment.

In the leak test process, the high-pressure air generated by the air compressor flows into the interior of the air spring, as depicted simultaneously in the 2D cross-section and 3D model in Figure 18a to visualize the airflow. Leakage can occur due to external factors like rocks, as shown in Figure 18b, or improper fastening during air spring assembly, resulting in air leakage. The leak test process, shown in Figure 18c, utilizes the FDG algorithm to represent the spraying of soapy water by a spray gun. Furthermore, Figure 18d shows the dispersion of soapy bubbles generated by air leakage, also depicted using the FDG algorithm. Figure 18c,d compare the rotated state of the object by the “RIGHT” command on the bottom cube, generated through the OAR algorithm. This way, the OAR algorithm provides functionality for rotation in various directions to enhance the visibility of fault locations during visual inspections of symmetrical air spring models and for leak testing using the FDG algorithm. Lastly, in the event of leaks, inspection procedures for defects and fastening conditions are demonstrated, as shown in Figure 18e,f.

5.4. Pilot Test

In this study, a pilot test was conducted on the produced digital content. The goal of the test was to verify the operational functionality and evaluate its real-world applicability and performance. The test subject was the air spring device, and the procedure involved the researchers directly running the installed app, generating objects by touching the plane, performing air spring height measurement, and pressing the leakage test button to execute the particle system functionality. Subsequently, the researchers compared the actual air spring device with the AR object. Figure 19 shows the researchers conducting the pilot test at the actual workshop. Figure 19a shows the calculation of the reference height for height measurement and the actual height measurement, while Figure 19b shows the execution of the air pressure flow in the air spring to compare the performance of the leakage test with the actual air spring device while learning. The pilot test results showed that the AR object could be matched with the actual air spring device, and the particle system functioned smoothly. Additionally, the menus of the learning interface, such as GNMB, LNMB, and SNMB, were accurately displayed according to the designed size and ratio.

6. AR Content Usability Evaluation and Results

6.1. Survey Tool

6.1.1. UMUX

UMUX, introduced by Finstad in 2010, is a survey questionnaire designed to meet the need for shorter surveys to evaluate system usability scales. It provides an effective and reliable method for measuring perceived usability. Berkman and Karahoca conducted a thorough investigation into the structure, validity, and reliability of UMUX and discovered that the scores were correlated with system usability scales [43]. The UMUX survey consists of two positive and two negative questions with seven response options ranging from 1 (strongly disagree) to 7 (strongly agree). The response options are scored on a Likert scale, and

Table 3. UMUX questions.

Item	Questions
U1	This content’s capabilities meet my requirements
U2	Using this content is a frustrating experience
U3	This content is easy to use
U4	I have to spend too much time correcting things with this system

showing the survey questions for UMUX evaluation.

UMUX survey evaluation can be calculated using the following method:

• For positively framed questions, subtract 1 from the response number.
• For negatively framed questions, subtract the response number from 7.
• Sum up the scores for all questions.
• Multiply the sum by 4.1667 (or 100/24).

This process can be formally represented by Equation (1):

$$\mathrm{U}\mathrm{M}\mathrm{U}\mathrm{X}=[{\sum }_{i=1}^2\left(P_i-1\right)-(N_i-7)]\times {\displaystyle \frac{100}{24}},$$

(1)

where P_i represents positively framed questions, and N_i represents negatively framed questions. The UMUX score calculated from this formula will receive a score between 0 and 100.

6.1.2. CSUQ

For the evaluation of the air spring AR content, the CSUQ method, another commonly used technique, was also employed. Four concepts related to usability, including usability, information quality, interface quality, and overall satisfaction, were selected, comprising a total of 19 items. The detailed content of the CSUQ survey referring to the four concepts is presented in

Table 4. CSUQ questions.

Category	No	Questions
System Usefulness	1	Overall, I am satisfied with how easy it is to use this system
	2	It was simple to use this system
	3	I can effectively complete my work using this system
	4	I am able to complete my work quickly using this system
	5	I am able to efficiently complete my work using this system
	6	I feel comfortable using this system
	7	It was easy to learn to use this system
	8	I believe I became productive quickly using this system
Information Quality	9	This system gives error messages that clearly tell me how to fix problems
	10	Whenever I make a mistake using this system, I recover easily and quickly
	11	The information (such as online help, on-screen messages, and other documentation) provided with this system is clear
	12	It is easy to find the information I needed
	13	The information provided for this system is easy to understand
	14	The information is effective in helping me complete the tasks and scenarios
	15	The organization of information on this system screens is clear
Interface Quality	16	The interface of this system is pleasant
	17	I like using the interface of this system
	18	This AR system has all the functions and capabilities I expect it to have
Overall Usability	19	Overall, I am satisfied with this system

. The CSUQ survey utilizes a 7-point response scale ranging from 1 (strongly agree) to 7 (strongly disagree), providing participants seven response options [27]. After users experienced AR content for air springs, an experience survey was conducted and analyzed using a survey tool.

CSUQ survey evaluation can be calculated using the following method:

• Add up the scores from 1 to 7 for all questions 1–19.
• Divide the total score of the 19 questions by 19 to calculate the average, then subtract 1 to determine the score based on the 0-point scale.
• Convert the value you subtracted in step 2 to 100 by multiplying it by 16.6667 (or 100/6).
• Since a score of 1 represents a strongly agree value, subtract the value of step 3 from 100.

This process can be expressed as Equation (2):

$$\mathrm{C}\mathrm{S}\mathrm{U}\mathrm{Q}=100-\left({\displaystyle \frac{\sum _{j=1}^{19}{Q}_i}{19}}-1\right)\times {\displaystyle \frac{100}{6}},$$

(2)

where Q_i is each question in the CSUQ survey. The CSUQ scores calculated from this formula will range from 0 to 100.

6.2. Evaluation Condition

For the usability evaluation in this study, 80 practitioners involved in railway vehicle maintenance were recruited through individual contacts. The final participants consisted of individuals who had not experienced the maintenance tasks related to the air spring of electric trains and who agreed to undergo the preassessment.

Table 5. The general information of survey participants.

Classification		Detailed Values	Remarks
Total participants		80
Average age		39.9	Standard deviation: 11.7
Average years of employment		14.8	Standard deviation: 10.6
Age distribution	20–29 Years	21 people
	30–39 Years	18 people
	40–49 Years	20 people
	50–59 Years	21 people
Gender	Male	65 people
	Female	15 people
Rail vehicle classification	High-speed rail	57 people
	General rail	23 people
Maintenance part	Light	41 people
	Heavy	39 people

shows the age distribution of the survey participants. The average age of the participants was 39.9 years (standard deviation: 11.7), enabling the collection of opinions from all age groups and ensuring reliability.

In addition, the average years of employment was 14.8 years (standard deviation: 10.6 years), and by age group, there were 21 people in their 20s, 18 people in their 30s, 20 people in their 40s, and 21 people in their 50s. There were 65 men and 15 women, and the railway vehicle classification was 28 people in high-speed trains and 12 people in general trains. In the maintenance part, there were 41 people in light maintenance and 39 people in heavy maintenance.

The usability evaluation experiment was conducted on separate dates, with 40 participants divided into the A group (document manual(booklet/e-file)) and the B group (AR content). The research team informed the participants that the collected data would be used anonymously for research purposes only. Before the evaluation, participants were briefed on the purpose of the usability evaluation. Then, the participants studied the assigned document manual or air spring AR training content for 20 min, following the instructions provided to each group. After completing the training, the participants were asked to fill out the UMUX and CSUQ questionnaires. Following the completion of the questionnaires, the research team collected feedback from the participants regarding any issues and areas for improvement.

6.3. Result

Figure 20 shows the results of the UMUX survey conducted using the document manual and AR content for the air spring, where group A had an overall average score of 73.66 across the four items, and group B scored 81.56, resulting in a 7.9-score increase for the AR content, which indicates approximately a 10% improvement in learning efficiency; specifically, Group B scored 85.00 for U1, “This content’s capabilities meet my requirements”, which is 10.83 scores higher than group A’s 74.17, reflecting a 14.6% improvement in meeting requirements; 80.83 for U2, “Using this content is a frustrating experience”, which is 5.83 scores higher than group A’s 75.00, indicating that the AR content is approximately 7% less frustrating to use; 78.75 for U3, “This content is easy to use”, which is 9.17 scores higher than group A’s 69.58, suggesting a 13.1% improvement in usability; and 81.83 for U4, “I have to spend too much time correcting things with this system”, which is 5.95 scores higher than group A’s 75.88, showing a 7% reduction in the time required to correct issues, though it is worth noting that U3 received the lowest score among the AR content items, likely due to users being unfamiliar with the operation and touch controls of AR content in a mobile environment, as well as the brief instructions and lack of a detailed user manual.

Figure 21 presents a graph showing the average scores of categories from the CSUQ results conducted using the document manual and AR content for the air spring, where the overall average score across the four categories and 19 items of the CSUQ survey was 73.29 for group A and 80.83 for group B, indicating that the AR content scored 7.54 scores higher than the document manual; specifically, in the system usefulness category, group B scored 84.95, which is 9.74 scores higher than group A’s 75.21, reflecting an approximate 10% increase in usefulness; in the information quality category, group B scored 80.48, which is 6.55 scores higher than group A’s 73.93, showing an approximate 8% improvement in information quality; in the interface quality category, group B scored 70.42, which is 2.78 scores higher than group A’s 67.64, indicating an approximate 4% enhancement in interface quality; and, finally, in overall usability, group B scored 81.67, which is 11.25 scores higher than group A’s 70.42, representing the largest increase and confirming that the AR content led to an approximate 16% improvement in overall usability.

Table 6. Average and standard deviation for each CSUQ question between the document manual and AR content.

Category	No	Document Manual		AR Content
		Average	Standard Deviation	Average	Standard Deviation
System Usefulness	1	79.58	19.42	86.25	13.55
	2	77.08	16.32	82.50	14.10
	3	77.08	18.75	85.83	14.40
	4	73.75	19.20	85.83	13.89
	5	69.58	24.43	81.25	12.62
	6	73.33	19.90	87.08	13.86
	7	75.42	23.57	85.00	12.96
	8	75.83	22.31	85.83	12.83
Information Quality	9	77.92	20.46	80.83	16.69
	10	75.83	20.31	73.33	15.92
	11	73.75	20.63	86.25	9.16
	12	71.25	21.01	83.33	9.99
	13	74.58	21.01	83.33	8.44
	14	70.83	20.93	76.25	14.07
	15	73.33	20.95	80.00	16.96
Interface Quality	16	67.50	21.00	69.17	22.50
	17	68.75	18.94	67.92	21.81
	18	66.67	24.17	74.17	21.67
Overall Usability	19	70.42	18.29	81.67	13.50

presents the results of the CSUQ survey, showing the average and standard deviation of the 19 items across the 4 categories. When using this AR system, users are expected to feel comfortable (87.08) and be able to quickly complete tasks and work efficiently (85.83), leading to a rapid improvement in productivity (85.83). Overall, users are likely to experience satisfaction (81.67) with the usability. However, concerning information quality, it might be difficult to recover from mistakes in the AR system (73.33). Regarding the interface, there are issues with technical explanations of interface usage methods and insufficient functionality and performance (74.17), leading to discomfort (69.17). This is because the system does not align with the users’ preferences (67.92).

Figure 22 presents the traditional Absolute Grading Scale (AGS) system for measuring usability evaluation scores, as proposed by Bangor et al. [31]. In this system, an “A” grade corresponds to a score of over 90 and up to 100, a “B” to a score over 80 and up to 90, a “C” to a score over 70 and up to 80, a “D” to a score over 60 and up to 70, and anything below that is graded as an “F”. The use of this absolute grading system makes achieving an “A” grade, which requires a score above 90, virtually impossible.

As an alternative, the scores obtained from the proposed usability evaluation were compared and analyzed using the curved rating scale (CGS) table proposed by Lewis and Sauro [44,45]. This CGS grading system stems from the preference for assigning grades based on a curve where a usability evaluation score of 68 is positioned at the center of the “C” range. Sauro et al. reviewed 241 studies and identified only 2 instances where the score exceeded 90. To provide a fairer evaluation allocation, they developed a curved grading scale for the average score using percentiles calculated as shown in

Table 7. The Sauro–Lewis CGS.

Score Range	Grade	Percentile Range
84.1–100.0	A+	96–100
80.8–84.0	A	90–95
78.9–80.7	A−	85–89
77.2–78.8	B+	80–84
74.1–77.1	B	70–79
72.6–74.0	B−	65–69
71.1–72.5	C+	60–64
65.0–71.0	C	41–59
62.7–64.9	C−	35–40
51.7–62.6	D	15–34
0–51.6	F	0–14

. Additionally, a total of 446 survey and usability study data points were included to analyze this grading system. According to James, although the CGS table is designed for SUS, it is also compatible with UMUX and CSUQ [46].

The AR content average scores for the UMUX and CSUQ surveys are 81.56 and 80.83, respectively. These scores correspond to a “B” grade on the AGS system, which is categorized as good and considered acceptable. The CGS system assigns an “A” grade, placing the scores within the 90–95th percentile. This result confirms, as shown in Figure 22 of the usability evaluation of the air spring AR content, that the content is acceptable and falls within the 90–95th percentile, indicating that it is highly useful.

Based on the participants’ feedback on the user experience evaluation, the researchers identified two main issues, as summarized in

Table 8. AR content issues according to user experience evaluation.

Item	Detail
Feature documentation	In using the UI, the functions are not explicitly described.
touch gestures	Interaction is not smooth when controlling position, size, and rotation simultaneously.

. Participants had difficulty understanding the interface’s usage, often resorting to repeated touches to learn. Participants learned how to use the AR system but experienced significant inconvenience because commands were not executing properly along the way. Therefore, it is necessary to provide separate user manuals or instructions to ensure that the AR content is easily understandable for everyone.

Additionally, there were frequent instances of instability in the interaction with the modeling. Many participants experienced the model rotating in the opposite direction due to misinterpreting subtle finger movements, or buttons did not respond due to unrecognized finger touches. To address this, it is essential to create tutorials to adequately train users on touch gestures before starting the training and to increase touch sensitivity during design.

7. Discussion

Railway vehicles are composed of various devices, making their mechanisms highly complex and difficult to understand individually. This complexity extends to the numerous maintenance processes required for each device. Therefore, this study proposes two algorithms for efficient maintenance training in the railway vehicle field.

The FDG algorithm utilizes a particle system to implement fluid diffusion, such as air pressure in air springs, enabling efficient learning. This algorithm can be applied in the development of AR content for various devices in the railway vehicle industry that perform fluid-flowing operations, in addition to air spring tilt tests.

The OAR algorithm provides a touch cube that allows users to move objects at various angles during task execution using the air spring AR content. Users can learn maintenance processes performed from various directions during maintenance operations. As a result, the proficiency of maintenance tasks can be increased, enabling precise operations.

The document manual and developed content were subjected to user experience evaluation and the results were analyzed to compare and verify usability. The evaluation consisted of responses to the UMUX evaluation of four items and the CSUQ survey of 19 items. Analysis of the evaluation data resulted in identifying improvement scores for two repeatedly presented issues, such as function manuals and touch gestures. In future research, the research team plans to develop an improved version of the AR content based on the behavior patterns and feedback of the evaluation participants. This includes addressing and correcting all identified shortcomings.

In the field of railway vehicles, AR content using algorithms to depict complex mechanisms, such as electricity and fluids, which are difficult to observe with the naked eye, is being developed. However, existing algorithms that illustrate flow lines do not provide educational systems capable of detecting air leaks caused by failures in air springs. By utilizing the implementation method of the FDG algorithm developed in this study, the sequential visualization of the flow of compressed air and the diffusion phenomena caused by leaks can be achieved. This systematic learning approach not only enhances the accuracy of fault detection and determination of fault locations but also improves the troubleshooting skills of maintenance personnel. Furthermore, it significantly enhances the educational effectiveness of training maintenance personnel to perform maintenance tasks more proficiently.

8. Conclusions

In this study, AR content was developed to maximize the efficiency of maintenance personnel training for air springs. A screen rotation algorithm was used to enhance the visibility of the content, while the FDG algorithm provided interaction between the user and the leakage test. The user experience evaluation of the developed content was conducted through task performance assessments and post-survey tools. The conclusions drawn from the research are as follows:

• For practical training, a document-based maintenance procedure for an air spring, which is not easily disassembled and maintained, was developed into highly detailed and immersive content on mobile devices using the general-purpose Unity program. This content was designed to be easily implemented offline, closely resembling actual maintenance work. The content was created to allow feedback on the maintenance process and evaluation. It was also designed with an MAR system considering the maintenance work environment, ensuring that there are no restrictions in the railway vehicle maintenance setting.
• To simulate the rubber air spring’s operation driven by compressed air, the previously developed SLMVC and CEPC algorithms were supplemented with the newly added FDG and OAR algorithms. The FDG algorithm is used to implement the effects of air spring damage and leakage inspection, while the OAR algorithm is employed to enhance visibility for maintenance tasks, such as visual inspection of damaged areas.
• After subdividing the air spring maintenance work by creating a mind map, a storyboard was created, and based on this scenario, an external condition inspection, air spring height inspection, and leakage test were produced. First, the maintenance tools required for maintenance work were displayed through images, and each maintenance step was implemented as an animation, with highlights applied to areas that required inspection or were for worker safety. In addition, standard dimensions or contents for maintenance explanations were visualized and delivered in text.
• To analyze the effectiveness of the content incorporating these algorithms, two surveys were conducted targeting railway vehicle maintenance personnel. As a result, the UMUX survey received an average score of 81.56 across four items, with the most positive response being for U1, “This AR content’s capabilities meet my requirements” (85.00). This indicates that not only the touchscreen gesture method but also the OAR algorithm improved the ease of object manipulation.
• Additionally, the CSUQ received an average score of 80.83 across the 19 items in the four categories, corresponding to a grade of A. Among the categories, “System Usefulness” had the highest score of 84.95. This reflects that the combined use of the newly added FDG and OAR algorithms with the existing SLMVC and CEPC algorithms provides an efficient, comfortable, and quick way to enhance productivity during tasks.
• The UMUX and CSUQ survey results indicate that the AR content significantly improves learning efficiency and usability compared to the traditional document manual. Group B, using the AR content, showed a 10% improvement in learning efficiency and a 16% enhancement in overall usability. However, while the AR system demonstrated clear advantages in ease of use, system usefulness, and information quality, there are areas needing improvement, particularly in information recovery and interface functionality. These findings suggest that the AR content is highly effective, though further refinement is needed to fully meet user expectations.

In conclusion, the developed content represents an innovative educational system that deepens and visualizes the specific characteristics of railway vehicle maintenance beyond conventional manuals. Future research should focus on preserving critical physical properties and structures to enhance the robustness of numerical methods. Practical application will be further improved through precise finite element simulation techniques, such as the recent positivity-preserving finite volume schemes for subdiffusion equations and time-fractional Fokker–Planck equations, by developing AR content that accurately reflects the stiffness characteristics and physical structure of air springs using compressed air [47,48].