How to Make Augmented Reality a Tool for Railway Maintenance Operations: Operator 4.0 Perspective
Abstract
:1. Introduction
1.1. AR Technology
1.2. AR Application Fields
1.3. Scope
2. Theoretical Background
2.1. AR in Maintenance Operations
- Step 1: Select equipment. In the first step, the operator must decide what to analyze. Each system component has a unique combination of failure modes and failure rates [22]. When a failure occurs in a system, the operator should prioritize and analyze the impact each failure has on the process. High impact failures have high priority.
- Step 2: Determine the functions. The function of a system determines the action that it will perform. AR spatial mapping and tracking systems can be used for finding all major and less obvious failures in a system [23]. An operator can overlook less obvious failures, whereas the AR tool can capture and report all failures.
- Step 3: Describe failures. Overlapping virtual information to physical components, according to their real-world position, ensures identification of the failure. The operator can see that the virtual image and the real object are not in the same place.
- Step 4: Describe failure modes. A failure mode indicates how the system fails to perform its function [22]. Maintenance interventions such as checking, changing, and condition monitoring can be performed using AR.
- Step 5: Select maintenance action. Based on predefined actions and instructions, the operator can address the failure using tailored AR guidance and contextualization.
- Step 6: Document results. Technical manuals often recommend a maintenance method for certain equipment and systems. However, manuals or work descriptions are not often tailored to a particular operating environment and actual environmental conditions. The technology can capture the time needed for addressing the failure and what sequence of tasks has been performed. Hereafter, the technology provides a periodic intervention to eliminate failure to occur.
- AR solution: Select a flexible tool. This tool should assist the operator by systemizing the maintenance procedure. Besides this, the tool should contextualize and customize the information supply to the need and skills of the operator. This support tool should easily be embedded in everyday maintenance operations.
2.2. Operator 4.0: Augmented Operator
2.3. AR Capabilities
2.4. The Need for a Dynamic Tool
3. Methods
3.1. Decision Support System
3.2. Boundary Conditions
3.3. Architectural Requirements
- Provide the augmented operator with real-time feedback and augmented reality content on tasks/procedures execution. Operators are guided by the supplying of visual and audible instructions to give tangible feedback.
- Based on expertise, experience, external factors, current conditions of the component, it is required to ensure the operator has a personal tailored digital knowledgeable assistant to interact with. Depending on the operator’s ability, noncritical information can be supplied using subtle instructions in different visible frequencies.
- By capturing the knowledge of the operator and procedural steps, the system can learn from previous maintenance procedures. The time and sequence of steps used to perform a maintenance task can be captured and reported. This can indicate how much time is needed in the future to perform the task. Moreover, failure rates can be compared to this information, providing insight on the most sustainable procedure. Maintenance planning and schedules can be adapted to these accurate findings. Therefore, the efficiency of operation support will be increased.
- Step 1: Select equipment. The goal to be achieved is formulated. The operator will be guided by visual and audible instructions which also give tangible feedback.
- Step 2: Determine functions. The task that needs to be selected to reach the goal is stated. Besides this, the adaptive AR tool provides all failure causes and digital information on all potential solutions. It will let the operator be aware of the context to gather relevant information and/or services, relevancy depends on the operator’s tasks [37]. Using context awareness systems, such as AR, accurate access to maintenance information is provided such that the operator’s performance efficiency can be increased.
- Step 3: Describe failures. Initiation and evaluating the operator’s expertise is currently based on the operator’s or manager’s perspective. The level of expertise varies from having no clue what is going on up to being an expert and able to train others. In this framework, initiating the level of expertise is performed manually but can become an automized process in the future. Operators can be equipped with sensors to activate psychomotor and cognitive responses that are beyond what operators can verbalize. Capturing gestures of experts can improve interactions with AR and ensures future knowledge capturing [33].
- Step 4: Describe failure modes. Dynamic behavior capturing is required to perform a successful fault diagnosis [33]. Based on time and process tracking, the operator should know what initiated the fault, what the current situation is, what is needed to solve the issue, and what time is required to solve the task. Varying business demands changes in work routines, resource availability, and environmental conditions. Depending on the complexity and nature of the maintenance task, the operator adapts his/her maintenance concept.
- Step 5: Select maintenance action. The tool presents the task that aims to restore the functionality of a system. The actions that can be performed to restore the functionality of the product can be technical, administrative, and managerial [30]. Continuous assessment takes place of the operator’s performance, task condition, and other external conditions. Besides this, the tool will send warning messages of improper maintenance operation execution. When the task or business demand increases, mental demand increases resulting in negative effects on physiological variables [38]. The likelihood that the operator fails in performing his/her task becomes subsequently larger, it is therefore needed to have a control or monitoring system that alarms the operator when tasks are not performed adequately.
- Step 6: Documents results. Documentation can support the detection of schedule derivations or the search for sources of defects and the responsibility of the operator [39]. Adequate process monitoring methods help managers and operators to document the current status of the maintenance work as well as to understand origins and defects. Some maintenance tasks require inactive input, for instance, to leave comments on specific objects. AR allows storing these annotations directly in relationship to the real environment.
- AR solution: Select a flexible tool. Incorporating different types of data, interfaces, visualization systems and sensors makes the adaptive tool applicable to multiple solutions.
4. Case Study
4.1. Case Study Characteristics
4.2. Investigation of the Retractable Step
4.3. Failure Description Retractable Step
4.4. Application Adaptive Architectural Framework
- Decisions are made based on the operator’s perspective on his/her level of expertise. Let the amount of AR information and frequency of information supply be adapted to the specific user, task demand, and business demand.
- Knowledge is captured from expert operators to use their experience to assist a novel operator to solve a problem. General and specific domain knowledge should be gathered to provide an incremental learning method. The time needed and sequence of steps of the procedures involved can be derived from the task performance. Hereby, the operator and the company capture detailed knowledge on the procedure to become more efficient and adequate problem solvers.
- Safety and security measures should be taken into account more consciously. The framework ensures sending warning messages if procedures or tasks are not performed according to procedures or safety standards.
5. Discussion of the Main Results
5.1. Technology Adoption Patterns
5.2. Support Requirements for Operator 4.0
5.3. Future Trends in Maintenance Operations
6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Lima, E.S.; McMahon, P.; Costa, A.P.C.S. Establishing the relationship between asset management and business performance. Int. J. Prod. Econ. 2021, 232, 107937. [Google Scholar] [CrossRef]
- Maletič, D.; Maletič, M.; Al-Najjar, B.; Gomišček, B. An analysis of physical asset management core practices and their influence on operational performance. Sustain. J. Rec. 2020, 12, 9097. [Google Scholar] [CrossRef]
- Gavrikova, E.; Volkova, I.; Burda, Y. Strategic aspects of asset management: An overview of current research. Sustain. J. Rec. 2020, 12, 5955. [Google Scholar] [CrossRef]
- Infoholic Research LLP. Augmented Reality for MRO (Maintenance, Repair and Overhaul) Market—Global Forecast to 2024. Available online: https://www.infoholicresearch.com/report/augmented-reality-for-mro-market/ (accessed on 14 December 2020).
- Nee, A.; Ong, S.; Chryssolouris, G.; Mourtzis, D. Augmented reality applications in design and manufacturing. CIRP Ann. 2012, 61, 657–679. [Google Scholar] [CrossRef]
- Ratnayake, D.; Lohit, P.; Singh, B.; Mishra, V.P. Analysis of machine learning algorithms in smart manufacturing. In Proceedings of the 8th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO), Noida, India, 4–5 June 2020; IEEE: New York, NY, USA, 2020; pp. 707–712. [Google Scholar]
- Mekni, M.; Lemieux, A. Augmented reality: Applications, challenges and future trends. Appl. Comput. Sci. 2014, 20, 205–214. [Google Scholar]
- Keil, J.; Edler, D.; Dickmann, F. Preparing the HoloLens for user studies: An augmented reality interface for the spatial adjustment of holographic objects in 3D indoor environments. KN J. Cartogr. Geogr. Inf. 2019, 69, 205–215. [Google Scholar] [CrossRef] [Green Version]
- Velazco-Garcia, J.D.; Shah, D.J.; Leiss, E.L.; Tsekos, N.V. A modular and scalable computational framework for interactive immersion into imaging data with a holographic augmented reality interface. Comput. Methods Programs Biomed. 2021, 198, 105779. [Google Scholar] [CrossRef]
- Hansen, L.H.; Wyke, S.S.; Kjems, E. Combining reality capture and augmented reality to visualise subsurface utilities in the field combining reality capture and augmented reality to visualise subsurface utilities in the field. In Proceedings of the 37th International Symposium on Automation and Robotics in Construction (ISARC 2020), Kitakyushu, Japan, 26–30 October 2020. [Google Scholar]
- Stylianidis, E.; Valari, E.; Pagani, A.; Carrillo, I.; Kounoudes, A.; Michail, K.; Smagas, K. Augmented reality Geovisualisation for underground utilities. PFG J. Photogramm. Remote Sens. Geoinf. Sci. 2020, 88, 173–185. [Google Scholar] [CrossRef]
- Del Amo, F.I.; Erkoyuncu, J.A.; Roy, R.; Palmarini, R.; Onoufriou, D. A systematic review of Augmented Reality content-related techniques for knowledge transfer in maintenance applications. Comput. Ind. 2018, 103, 47–71. [Google Scholar] [CrossRef]
- Saidy, C.; Valappil, S.P.; Matthews, R.M.; Bayoumi, A. Development of a Predictive Maintenance 4.0 Platform: Enhancing Product Design and Manufacturing; Springer International Publishing: Cham, Switzerland, 2020; Volume 166, pp. 1039–1049. [Google Scholar]
- Masoni, R.; Ferrise, F.; Bordegoni, M.; Gattullo, M.; Uva, A.E.; Fiorentino, M.; Carrabba, E.; Di Donato, M. Supporting remote maintenance in industry 4.0 through Augmented Reality. Procedia Manuf. 2017, 11, 1296–1302. [Google Scholar] [CrossRef]
- Ruiz-Sarmiento, J.-R.; Monroy, J.; Moreno, F.-A.; Galindo, C.; Bonelo, J.-M.; Gonzalez-Jimenez, J. A predictive model for the maintenance of industrial machinery in the context of industry 4. Eng. Appl. Artif. Intell. 2020, 87, 103289. [Google Scholar] [CrossRef]
- Nardo, M.; Forino, D.; Murino, T. The evolution of man–machine interaction: The role of human in Industry 4.0 paradigm. Prod. Manuf. Res. 2020, 8, 20–34. [Google Scholar] [CrossRef] [Green Version]
- Hermann, M.; Pentek, T.; Otto, B. Design Principles for Industrie 4.0 Scenarios. In Proceedings of the 2016 49th Hawaii International Conference on System Sciences (HICSS), Koloa, HI, USA, 5–8 January 2016; Institute of Electrical and Electronics Engineers (IEEE: USA): New York, NY, USA, 2016; Volume 2016, pp. 3928–3937. [Google Scholar]
- Kaasinen, E.; Schmalfuß, F.; Özturk, C.; Aromaa, S.; Boubekeur, M.; Heilala, J.; Heikkilä, P.; Kuula, T.; Liinasuo, M.; Mach, S.; et al. Empowering and engaging industrial workers with Operator 4.0 solutions. Comput. Ind. Eng. 2020, 139, 105678. [Google Scholar] [CrossRef]
- del Amo, I.F.; Erkoyuncu, J.A.; Roy, R.; Wilding, S. Augmented Reality in maintenance: An information-centred design framework. Procedia Manuf. 2018, 19, 148–155. [Google Scholar] [CrossRef]
- Holm, M.; Danielsson, O.; Syberfeldt, A.; Moore, P.; Wang, L. Adaptive instructions to novice shop-floor operators using Augmented Reality. J. Ind. Prod. Eng. 2017, 34, 362–374. [Google Scholar] [CrossRef] [Green Version]
- Kaczmarek, M.J.; Gola, A. Maintenance 4.0 Technologies for Sustainable Manufacturing—An Overview. IFAC PapersOnLine 2019, 52, 91–96. [Google Scholar] [CrossRef]
- Campbell, J. The Reliability Handbook, from Downtime to Uptime—In No Time! Ed. Glob. Lead. Phys. Asset Manag. Price Waterhouse Coopers LLP 1999, 23, 23–36. [Google Scholar]
- Bellucci, A.; Ruíz, A.; Díaz, P.; Aedo, I. Investigating augmented reality support for novice users in circuit prototyping. In Proceedings of the 2018 International Conference on Big Data and Computing, New York, NY, USA, 25–30 June 2018; Association for Computing Machinery (ACM): New York, NY, USA, 2018; p. 35. [Google Scholar]
- Peruzzini, M.; Grandi, F.; Pellicciari, M. Exploring the potential of Operator 4.0 interface and monitoring. Comput. Ind. Eng. 2020, 139, 105600. [Google Scholar] [CrossRef]
- Romero, D.; Stahre, J.; Wuest, T.; Noran, O. Towards an operator 4.0 typology: A human-centric perspective on the fourth industrial revolution technologies. In Proceedings of the International Conference on Computers and Industrial Engineering, Tianjin, China, 29–31 October 2016; pp. 1–11. [Google Scholar]
- Longo, F.; Nicoletti, L.; Padovano, A. Smart operators in industry 4.0: A human-centered approach to enhance operators’ capabilities and competencies within the new smart factory context. Comput. Ind. Eng. 2017, 113, 144–159. [Google Scholar] [CrossRef]
- Segura, Á.; Diez, H.V.; Barandiaran, I.; Arbelaiz, A.; Álvarez, H.; Simões, B.; Posada, J.; García-Alonso, A.; Ugarte, R. Visual computing technologies to support the Operator 4. Comput. Ind. Eng. 2020, 139, 105550. [Google Scholar] [CrossRef]
- Bigliardi, B.; Bottani, E.; Casella, G. Enabling technologies, application areas and impact of industry 4.0: A bibliographic analysis. Procedia Manuf. 2020, 42, 322–326. [Google Scholar] [CrossRef]
- Egger, J.; Masood, T. Augmented reality in support of intelligent manufacturing—A systematic literature review. Comput. Ind. Eng. 2020, 140, 106195. [Google Scholar] [CrossRef]
- Palmarini, R.; Erkoyuncu, J.A.; Roy, R.; Torabmostaedi, H. A systematic review of augmented reality applications in maintenance. Robot. Comput. Manuf. 2018, 49, 215–228. [Google Scholar] [CrossRef] [Green Version]
- Małachowski, B.; Korytkowski, P. Competence-based performance model of multi-skilled workers. Comput. Ind. Eng. 2016, 91, 165–177. [Google Scholar] [CrossRef]
- Smith, J.; Yazdanpanah, F.; Thistle, R.; Musharraf, M.; Veitch, B. Capturing expert knowledge to inform decision support technology for marine operations. J. Mar. Sci. Eng. 2020, 8, 689. [Google Scholar] [CrossRef]
- Illankoon, P.; Tretten, P. Collaborating AI and human experts in the maintenance domain. AI Soc. 2020, 1–12. [Google Scholar] [CrossRef]
- Kambhampati, S. Explicable Planning and Replanning for Human-in-the-Loop Decision Support; NASA: Washington, DC, USA, 2017.
- Crowder, J.A.; Carbone, J.N. Collaborative Shared Awareness: Human-AI Collaboration; Springer International Publishing: New York, NY, USA, 2004. [Google Scholar]
- Bottani, E.; Vignali, G. Augmented reality technology in the manufacturing industry: A review of the last decade. IISE Trans. 2019, 51, 284–310. [Google Scholar] [CrossRef] [Green Version]
- Erkoyuncu, J.A.; del Amo, I.F.; Mura, M.D.; Roy, R.; Dini, G. Manufacturing technology improving efficiency of industrial maintenance with context aware adaptive authoring in augmented reality. CIRP Ann. Manuf. Technol. 2017, 66, 465–468. [Google Scholar] [CrossRef]
- Huang, T.-L.; Liao, S. A model of acceptance of augmented-reality interactive technology: The moderating role of cognitive innovativeness. Electron. Commer. Res. 2015, 15, 269–295. [Google Scholar] [CrossRef] [Green Version]
- Gattullo, M.; Scurati, G.W.; Fiorentino, M.; Uva, A.E.; Ferrise, F.; Bordegoni, M. Towards augmented reality manuals for industry 4.0: A methodology. Robot. Comput. Manuf. 2019, 56, 276–286. [Google Scholar] [CrossRef]
- Cousin, G. Case study research. J. Geogr. High. Educ. 2005, 29, 421–427. [Google Scholar] [CrossRef]
- Narasimhan, R.; Jayaram, J. Reengineering service operations: A longitudinal case study. J. Oper. Manag. 1998, 17, 7–22. [Google Scholar] [CrossRef]
- Guillemette, M.G.; Mignerat, M.; Paré, G. The role of institutional work in the transformation of the IT function: A longitudinal case study in the healthcare sector. Inf. Manag. 2017, 54, 349–363. [Google Scholar] [CrossRef]
- Demeter, K.; Losonci, D.; Nagy, J. Road to digital manufacturing—A longitudinal case-based analysis. J. Manuf. Technol. Manag. 2020. [Google Scholar] [CrossRef]
- International Union of Railways (UIC). Railway Statistics—Synopsis. Available online: https://uic.org/support-activities/statistics/ (accessed on 10 February 2021).
- NS. Expected Future Developments. Available online: https://www.nsjaarverslag.nl/jaarverslag-2020/inleiding/onze-strategie/verwachte-ontwikkelingen-op-lange-termijn (accessed on 26 February 2021).
- Malterud, K.; Siersma, V.D.; Guassora, A.D. Sample Size in Qualitative Interview Studies: Guided by Information Power; Qualitative Health Research: Salt Lake City, UT, USA, 2016. [Google Scholar]
- Martinetti, A.; Hart, K.; Damgrave, R.; van Dongen, L.A.M.; Turkenburg, R.; Nouwens, A. There is no spoon: Applying virtual reality for maintenance training of rolling stock technicians. Int. J. Syst. Syst. Eng. 2018, 8, 398–415. [Google Scholar] [CrossRef] [Green Version]
- Stadler. Electric Low-Floor Multiple-Unit; Stadler: Pierce Brendan Brosnan, Switzerland, 2016; Volume 41. [Google Scholar]
- Mertens, P.; van Hal, B. Rapport Onderzoek Schuiftrede FLIRT HRN; Nederlandse Spoorwegen: Utrecht, The Netherlands, 2020. [Google Scholar]
- Bode. Technische Documentatie Schuiftrede met Elektrische Aandrijving; Bode: Hamburg, Germany, 2016. [Google Scholar]
- Funk, M.; Bachler, A.; Bachler, L.; Kosch, T.; Heidenreich, T.; Schmidt, A. Working with augmented reality? A long-term analysis of in-situ instructions at the assembly workplace. ACM Int. Conf. Proc. Ser. 2017, F1285, 222–229. [Google Scholar]
- Masood, T.; Egger, J. Adopting augmented reality in the age of industrial digitalisation. Comput. Ind. 2020, 115, 103112. [Google Scholar] [CrossRef]
- Romero, D.; Stahre, J.; Taisch, M. The Operator 4.0: Towards socially sustainable factories of the future. Comput. Ind. Eng. 2020, 139, 106128. [Google Scholar] [CrossRef]
- Bortolini, M.; Faccio, M.; Gamberi, M.; Pilati, F. Motion Analysis System (MAS) for production and ergonomics assessment in the manufacturing processes. Comput. Ind. Eng. 2020, 139, 105485. [Google Scholar] [CrossRef]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Scheffer, S.; Martinetti, A.; Damgrave, R.; Thiede, S.; van Dongen, L. How to Make Augmented Reality a Tool for Railway Maintenance Operations: Operator 4.0 Perspective. Appl. Sci. 2021, 11, 2656. https://doi.org/10.3390/app11062656
Scheffer S, Martinetti A, Damgrave R, Thiede S, van Dongen L. How to Make Augmented Reality a Tool for Railway Maintenance Operations: Operator 4.0 Perspective. Applied Sciences. 2021; 11(6):2656. https://doi.org/10.3390/app11062656
Chicago/Turabian StyleScheffer, Sara, Alberto Martinetti, Roy Damgrave, Sebastian Thiede, and Leo van Dongen. 2021. "How to Make Augmented Reality a Tool for Railway Maintenance Operations: Operator 4.0 Perspective" Applied Sciences 11, no. 6: 2656. https://doi.org/10.3390/app11062656