Digital Twin of Experimental Smart Manufacturing Assembly System for Industry 4.0 Concept

Abstract

This article deals with the creation of a digital twin for an experimental assembly system based on a belt conveyor system and an automatized line for quality production check. The point of interest is a Bowden holder assembly from a 3D printer, which consists of a stepper motor, plastic components, and some fastener parts. The assembly was positioned in a fixture with ultra high frequency (UHF) tags and internet of things (IoT) devices for identification of status and position. The main task was parts identification and inspection, with the synchronization of all data to a digital twin model. The inspection system consisted of an industrial vision system for dimension, part presence, and errors check before and after assembly operation. A digital twin is realized as a 3D model, created in CAD design software (CDS) and imported to a Tecnomatix platform to simulate all processes. Data from the assembly system were collected by a programmable logic controller (PLC) system and were synchronized by an open platform communications (OPC) server to a digital twin model and a cloud platform (CP). Digital twins can visualize the real status of a manufacturing system as 3D simulation with real time actualization. Cloud platforms are used for data mining and knowledge representation in timeline graphs, with some alarms and automatized protocol generation. Virtual digital twins can be used for online optimization of an assembly process without the necessity to stop that is involved in a production line.

1. Introduction and Relevant Previous Works

The Industry 4.0 concept is referred to as the fourth industrial revolution and is the current trend in automation, monitoring, and data mining from manufacturing processes. The main technologies to realize these tasks are [1]:

• IoT (internet of things);
• CPS (cyber-physical systems);
• CP (cloud platforms);
• CC (cognitive computing);
• AR/VR (augmented reality/virtual reality) devices; and
• other related disciplines.

The main tasks are digitization of data, analysis, and knowledge extraction. In these areas many papers have been published with research results aimed towards the use of cyber-physical systems within the Industry 4.0 concept, big data processing [2], and the combination of CPS with IoT systems [3]. The next task of the Industry 4.0 concept is product customization, because current customers want unique products. This requirement is a big challenge for the Industry 4.0 concept to ensure production with minimal cost and high customization.

Generally, manufacturing systems, production processes, and products are nowadays suitable for full digitalization. The first task in process digitalization is the specification of suitable technologies. There can be defined two groups of devices, the first for permanent installation in product, which must be low cost and the second group for machine and production process monitoring.

Perspective technologies for data digitization and collection from machines, processes, and products are:

• RFID (radio-frequency identification) labels for wireless part identification, and RFID transceivers for fixture monitoring in the production process;
• MEMS sensors integrated into the product for contactless data measuring and also integrated in the production process;
• IoT devices with independent wireless communication for data upload to cloud platforms for next processing; and
• Cloud platforms with data mining for knowledge extraction and data representation in timelines and alarms.

A modern approach to product identification is using RFID UHF (ultra high frequency) technology, because it reaches higher distances and reliability like LF and HF (low and high frequency) RFID technology. Some research results concerning RFID systems were published in articles about security of tags [4], detection of missing tags [5], and new searching protocols [6].

MEMS sensors have minimal power consumption and can be powered from battery during a product’s lifetime. They can be used, for example, for monitoring of product overheating and vibration during operation by the customers, and are generally used for continuous monitoring of the product environment. Some research on the usability of MEMS sensor devices was published in the articles about the monitoring of mechatronic systems by MEMS [7], vibration analyses by MEMS [8], and structural monitoring by MEMS [9].

Modern IoT systems are based on specialized communication technologies for data isolation from standard Wi-Fi (Wireless-Fidelity) or Bluetooth networks. The main IoT communication standard is LPWAN (low-power wide-area network) and includes solutions such as LoRa/LoRaWAN and Sigfox. Other technologies are using modifications of GSM (Global System for Mobile Communication) networks for low data transfer. Some research results in this area of interest are described in articles about energy-efficient communications in the internet of things [10], novel chaining encryption algorithm for LPWAN IoT networks [11], evaluation of next-generation low-power communication technology to replace GSM in IoT applications [12], performance assessment of long-range and Sigfox protocols with mobility support [13], and a new effective way to boost LoRaWAN network capacity [14].

Knowledge extraction (data mining) and data analysis are the main tasks of a cloud system. Cloud platforms can provide data in user-friendly form by timelines, day/weeks, or monthly automated reports. There can be also integrated an alarm system for critical production status, usually as a message by email or SMS (short message service). Some new research trends in cloud platforms and data mining are described in the papers concerning clouds and big data connection [15], cloud robotics data [16], clouds in industrial automation [17], and chaos theory combined with cloud systems [18]. The practical aspects are described in the papers about industrial process simulation and data digitization in Tecnomatix software [19,20].

Applying a virtual model for remote monitoring is a new trend of the Industry 4.0 concept and can represent the real manufacturing system, production process, or product. Such virtual models digitally replicate all aspects of real devices and they are called digital twins, which continuously learn and update themselves from multiple data sources. The articles and case studies aimed at digital twins are, for example, about digital twins with ergonomic optimization [21], digital twin methodology to commentary [22], the learning of new experiences by digital twins [23], automatic generation of simulation for digital twins [24], digital twins’ implementation for legacy systems [25], new possibilities of digital twin technology [26], and its application for Smart Industry [27].

Our research idea was to create a unified methodology for full data transfer from an experimental manufacturing assembly system to its digital twin, with real-time simulation and long-time data storage in a cloud platform with adequate synchronization.

The main purpose of this article is to present an architectural concept for a digital twin design with cloud platform and methodology of full data digitalization for smart manufacturing, which was implemented and tested on the experimental assembly system based on a belt conveyor system and an automatized line for quality production check.

2. Problem Specification and Proposed Work

The basic problem in implementation of an Industry 4.0 concept is data acquisition from the manufacturing environment. So, the first task is to select suitable technologies for data collection from production machines, production processes, and products.

Data from production processes can be provided by integrated position check sensors, for example: inductive proximity sensors, optical sensors, and laser sensors. These sensors can provide the actual status of the production, but no customized data about products; for example, they cannot provide information about where an exact order is currently positioned within the production process.

The next task is implementation of technology for exact part localization in production. The useful technology for exact part localization in production is RFID technology. An RFID reader, with an RFID transceiver on fixture and RFID labels on final assembly or parts, can provide exact information about the product status and its history.

The next important task in production is quality control. It can be divide into these levels:

• standard part identification;
• part presence check-in;
• dimension check in defined tolerance; and
• assembly check for product completeness.

If there is need for automatic feedback from a realized product, some autonomous monitoring equipment has to be implemented. The solution is implementation of IoT devices combined with MEMS sensors. This provides a reliable and low-cost solution with long-term operation. The data collection must be separated from standard communication networks like Wi-Fi or Bluetooth. The new IoT communication technologies provide this feature also with roaming possibilities.

The last task involes data storage and knowledge representation in some readable form. The standard databases cannot be used, because industrial systems now usually produce big data. The solution is to use industrial cloud platforms for data storage and visualization from all production processes. Open source timeline databases with graphical visualization are also suitable for data collection from IoT devices.

The proposed work is divided into four phases:

• Creation of minimalized physical experimental assembly system with technologies for data collection;
• Design of data digitization from all integrated technologies;
• Synchronization of the actual status of physical devices with a digital twin; and
• Data accumulation and representation in a cloud platform.

The main novelty of the research is its methodology for full digitalization of different inspection and identification technologies used in the experimental smart manufacturing assembly system. The basis is an architectural concept design of the digital twin of the experimental system and its synchronization with the cloud platform.

2.1. The Concept of Experimental Smart Manufacturing Assembly System

The concept of the experimental smart manufacturing assembly system is shown in Figure 1 and partial physical realization for research and educational purposes is shown in Figure 2. This concept includes all necessary technologies for digital data acquisition: RFID technology, vision systems, and IoT devices. All data from these technologies must be transformed to standardized industrial format. The PLC (programmable logic controller) is the main data collector for the next data processing and transfer to cloud system. Standardization of data is reached by the open platform communications (OPC) server, which also provides data distribution to the digital twin based on Siemens Tecnomatix plant simulator and the cloud platform MindSphere.

2.2. RFID Part Identification System

The RFID reader (Siemens SIMATIC R685R with one integrated and second external antenna SIMATIC RF615A) was used for product identification. The first antenna checked the part presence by RFID labels tags. The second antenna checked the assembly completeness and wrote data for the fixture’s RFID tag transponder. Implementation of RFID technology created an identification gate for input parts and output for finalized assembly. Any data acquired from vision system can be written to a part or assembly RFID tag as unique data. The used RFID system with two antennas, tested tags, and RFID tag implementation for some assembling parts and the fixture is shown in Figure 3.

Data from the RFID system were acquired as EPCID (Electronic Product Code ID) numbers and collected in PLC system, as shown in Figure 4.

2.3. Vision Systems for Part and Assembly Inspection

The used industrial vision systems, and specialized vision system with integrated trained convolutional neural network for standard parts (nuts, screws, washers) recognition [28], are shown in Figure 5. Examples of using vision systems for part checks, dimension measuring, assembly completeness, and standard part (nut) identification are shown in Figure 6.

An example of digital data output from a vision system for measured dimension and its transfer to the OPC server is shown in Figure 7.

2.4. IoT Devices for Long-term Product Monitoring by MEMS Sensors

Three IoT communication technologies (GSM technology, Sigfox, and LoRaWAN) which can provide isolated data transfer were selected for long-term product monitoring. The experimental system combines these IoT technologies to one module with data collection to an open source system, as shown in Figure 8.

Examples of the tested components for such combined system are shown in Figure 9. The software was programmed by Pymakr IDE inside Visual Studio and transferred to an FiPy module. The multiple IoT communication was operated by the FiPy module, which was connected to a MEMS sensor to collect vibration information.

These devices were used for experimental signal quality measurements, and for real implementation they must be reduced in dimension for integration to final assembly product.

An example of digital data collected from the hybrid IoT device by LoRaWAN network user interface “The things network” (TTN) is shown in Figure 10.

The limitations of the tested IoT communication technologies for industrial use are summarized in

Table 1. Comparison of IoT technologies for industrial tasks.

Technology	Nr. of Uplink MSG/Day	Nr. of Downlink MSG/Day	Bytes	Delay
LoRaWAN (TTN)	1440	max 10	10	30s
Sigfox	140	max 4	12	30s
GSM	not defined	not defined	UDP/TCP	-

.

The simple principle of MEMS system calibration in production is to acquire the reference vibration of product during normal working condition. Then it is possible to identify increased vibration or critical states during operating status. An example of the calibration and the critical status identification is shown in Figure 11. The reference status in this case is vibration amplitude in a stepper motor running at nominal speed. So, there is need to acquire this reference vibration status as the value range and write the offset value to the MCU (MicroController Unit) as a critical level. Data acquisition from MEMS sensors has been tested in Arduino IDE (Integrated Development Environment). An example of the vibration with filtration based on Kalman filters in x-axis and cumulated data from MEMS sensor decoded by Node Red interface is shown in Figure 12.

3. Digital Twin

A digital twin is a digital replica (a virtual model) for several hierarchy levels, like a sensor, an actuator, a production unit, a plant. It consists of 3D models grouped to assemblies with the possibility of remote monitoring and data synchronization with a real system and offline simulation.

The basis for digital twin creation is 3D virtual models of empty fixture, a fixture with parts, a fixture with assembly (Figure 13), and a 3D virtual model of a belt conveyor system with equipment for data acquisition (Figure 14).

Optimization of the production systems and their processes have so far usually been executed offline, mainly before the production starts or before it had to be stopped for optimization. The main improvement in the suggested approach is online optimization within the digital twin and synchronization with the real system. In the presented setup, a Python OPC UA (Unified Architecture) server (FreeOpcUa framework) is responsible for data transfer from the digital twin to the PLC system (Siemens PLC S7-1200 with HMI 4,3” TFT KTP 400).

So, it is possible to synchronize simulation (digital twin of the system) with control system of the real manufacturing system and update process variables in a very short time by user interface. The designed 2D and 3D digital twins of the experimental assembly system are shown in Figure 15.

The digital twin was transferred to a virtual reality device—HTC Vive Pro virtual reality glasses. This equipment can be used in quality control to remotely monitor the actual status of production. The main task is the synchronization of data from real production to the 3D virtual digital twin with simple visualization. An example of a digital twin of product identification and customization (a) and transfer (b) of Technomatix simulation to virtual reality by HTC Vive Pro is shown in Figure 16.

VR devices (HTC Vive Pro, Oculus Rift, Valve Index VR) for communication with virtual scenes use hand controllers, but in assisted assembly there is the need to have both hands free. This problem was solved by using a Leap motion sensor to detect hand and finger position. Another problem in assisted assembly is that recognized parts must be detected in any position, rotation, and scale, but this is unsolvable by standard industrial vision systems. This task we solved by using deep learning techniques; all assembly parts were trained in deep neural networks by Google TensorFlow frameworks [29]. The trained network was transformed to OpenCV library DNN (Deep Neural Network) for Android and executed in an AR device inside a Unity 3D engine. An example of hand detection by Leap motion installed on a VR device (HTC Vive Pro with SteamVR software) is shown in Figure 17a, and standardized assembly parts recognized by the AR device (Epson Moverio BT350 with the development software Unity engine) in Figure 17b [28].

4. Data Transfer to Cloud System

To isolate the plant from the internet, we used IoT gateway MindConnect (connected to Amazon Web Services Cloud access) with separated network interfaces: the first one for internet connection, the second one for data transfer from the isolated manufacturing process by a Python OPC Server. So, security of all critical digital data transfer from the experimental manufacturing system to the cloud platform was solved by Mindsphere network unit MindConnect, which works like a hardware firewall and isolates the production process (plant) from the internet. The extracted data sent to the cloud platform Mindsphere were encrypted. The encryption for OPC server data is possible, but all OPC data were transferred in local networks, so there was no problem with security.

The used communication devices for data transfer to the MindSphere cloud platform are shown in Figure 18.

An example of data timelines’ visualization from the vision system for all measured dimensions in the MindSphere cloud platform is shown in Figure 19.

Some typical IoT data from MEMS sensors as acceleration in two axis X and Y in the cloud platform Thinger.io, and signal quality monitoring by Grafana visualization, are shown in Figure 20.

5. Discussion

All data acquisition technologies, which were integrated into the experimental assembly system for identification (RFID tags) and inspection (vision systems, MEMS sensors, IoT devices) have their own limitations, which must be solved before data is stored into a cloud platform.

Identification systems used in manufacturing based on UHF RFID devices cannot be used for very small parts, because the tag size is limited by antenna length. RFID readers working with low frequency (LF) or high frequency (HF) tags are the solutions for tagging smaller parts, but they are not primarily developed for industrial parts (mainly for the food industry) and they reach lower distances for stable communication. The main limitation of UHF RFID technology in experimental assembly systems is UHF RFID tag size. The small parts—for example, screws, washers, and nuts—cannot be tagged. One of the possible solutions could be the combination of UHF with LF or HF RFID technologies, but this would have the disadvantage of shorter distance detection of assembly parts.

Smart industrial vision systems are usually closed source systems without the capability to modify an algorithm. This limitation arises especially in surface errors in recognizing very complicated parts. The next limitation of industrial vision systems for assembly operation is that detected parts must be in the fixed distance and position from camera, with only limited rotation. A new approach and solution could be the use of convolutional neural networks (CNN) with deep learning training techniques described in details in research articles about automated training of deep learning networks by 3D virtual models [28] and recognition of assembly parts by convolutional neural networks [29]. This new approach was also tested in the experimental smart manufacturing assembly system using the Tensor Flow framework, with good results for its application in real production. The main advantage of these approaches is a relatively simple integration into assisted assembly operation with AR/VR devices.

The data acquisition from MEMS sensors is based on some physical theorems which provide data with limited long-term precision (LTP), because they are physically isolated from measured objects and therefore must acquire data contactless. This is main reason why the MEMS sensors’ data from accelerometers or gyroscopes have to be filtered by some filters, for example a Kalman filter, because their output signal is very noisy. The position of MEMS sensors in the manufacturing system must be with minimum interferences from RFID system. MEMS sensors integrated into the product are not influenced by RFID signal because they start to operate after the production process, during the lifetime of products. The next problem is high frequency output bandwidth generated by MEMS sensors, which is not compatible with currently used cloud platforms. A standard cloud platform minimal data acquisition time is one second. So, there is a need to use some buffers for data accumulation. The problem of how to accumulate more data in the one second interval is described in the research article about data optimization for communication between wireless IoT devices and Cloud platforms in production process [30].

The significant problem in the usability of clouds platforms is the lack of support to store a digital twin and its simulation and visualization. Industry 4.0 uses both technologies: cloud platforms for simple data storage, and digital twins for actual status visualization and simulation. So, there is a need to have some connection between these technologies.

Cloud platforms are primarily focused on data collection and basic graphical representation via timelines and data knowledge extraction, with critical status alarm systems or user-defined periodical reports: daily, weekly, monthly, and yearly. Available cloud platforms currently offer for storage simple variables like integer, float, bool, data time, and string. All digital data are usually stored in predefined frames and defined only by strings or numbers. It is not possible to include advanced structures with combined blob (Binary Large OBject) data, for example images, or binary data combined with numerical/string variables.

In the currently available cloud platforms, it is not possible to store customized 3D virtual digital twin models of products with adequate digital variable data.

The first solution could be to create a separate database for the customized digital twin 3D model, with a hyperlink to standard digital data stored in a cloud platform. This solution could link data from a digital twin to a cloud platform in one direction of synchronization. The main advantage of this approach would be increased security for critical data of products, because customized digital twin models of products would be stored in local databases.

The second solution could be the modification of an open source cloud platform for visualization and storage of customized digital twin models and image data, with simple variables, in one place. The main advantage of this approach would be very low operating costs, because whole solutions could be integrated into factory server infrastructure.

The main disadvantage of the designed and tested architecture, with the digital twin and cloud platform after implementation to real device, is the necessity of trained staff to modify, optimize, and extract knowledge, which is specific to every production process.

6. Conclusions

Full digitalization of production processes is a very important part of the Industry 4.0 concept to increase the effectivity of SME (Small and Medium-sized Enterprise) manufacturing. Suitable technologies for fast digitalization, data transfer, data storage, and finally for data mining have to be used. In this paper, we described an approach for how to capture data with contactless technologies in a smart manufacturing assembly process, and to transfer this data from the production process to the extended digital twin 3D model.

To build a bridge between the real production system and its digital twin, an OPC technology for data synchronization in both directions was implemented using OPC UA Server (OPC DCOM –Distributed Component Object Mode). The OPC server must ensure three communications: the first to the digital twin model, the second to the cloud platforms, and the third to the PLC system. For that reason, the customized OPC server written in Python Programming Language was designed and implemented. The IoT gateway MindConnect was used for data transfer to the MindSphere cloud platform.

The vision system, consisting of three camera modules, was implemented into the experimental manufacturing assembly process to check the shapes and surfaces of assembly parts and to measure their dimensions. An RFID system was used to localize parts on the conveyor line by RSSI (radio signal strength indication) signal from tags. The RFID gate wrote unique information to the main assembly RFID tag label of every product, as for example, acquired dimension data and quality data. So, in this way a smart identification system was applied to the experimental manufacturing assembly system.

Furthermore, the MEMS sensor data acquisition in combination with IoT communication technologies was tested. The product vibrations had been measured by an integrated accelerometer. The IoT data were processed by Node Red data conversion technology to the specialized NoSQL (Non-Structured Query Language) database with Grafana visual interface. Independent IoT communication technologies Sigfox and LoRaWAN were used for data transfer to a cloud platform.

After the implementation of the architectural concept of the digital twin with the cloud platform to the real manufacturing system with full digitalization, the finding is that there are some limitations which arise from currently available technologies. The main limitation is the cloud platform because it does not support storage of customized digital twins and it provides a minimum delay of about one second in data transfer.

The realized experimental smart manufacturing assembly system will serve for further research (for example, to extend the digital twin via data from articulated assembly robot) and also for educational purposes. Using advanced technologies based on the Industry 4.0 concept by educated students and workers can help to develop sustainable production of SMEs. The designed and tested digital twin architecture on the experimental manufacturing system with full data digitalization provides a universal digital model which can be used as a template for real manufacturing of SMEs. Implementation of digital twins into manufacturing is a necessary condition for product lifecycle management (PLM) to ensure sustainable production.