Virtual Simulation and Experiment of Quality Inspection Robot Workstation

Abstract

(1) Background: Quality inspection robots are widely used in automated production lines. However, the design cycle is long, iteration costs are high, and algorithm development is challenging. It is difficult to perform effective validation during the design phase. Applying virtual reality technology to simulate quality inspection robot workstations offers a new approach to addressing the issues. (2) Methods: The research creates a simulation platform for quality inspection robot workstations based on a virtual reality architecture. The platform creates an immersive quality inspection robot workstation operation interface and conducts testing of the inspection process, thereby validating the rationality of the quality inspection robot workstation design. Building upon this foundation, we conducted experimental comparisons of various defect detection algorithms. (3) Results: Compared to the traditional YOLOv7 algorithm, the improved YOLOv7 algorithm achieved an 18.1% increase in recognition precision. Experimental results demonstrate that the quality inspection robot workstation simulation platform can be applied to validating workstation design proposals. (4) Conclusions: It has a positive impact on reducing the research and development costs of quality inspection robot workstations and shortening the defect recognition algorithm development cycle.

1. Introduction

With the rapid development of industrial automation and next-generation information technology, robots have become a key research area in intelligent manufacturing [1]. Quality inspection robot workstations offer advantages such as high detection efficiency, accuracy, and flexibility, playing a significant role in improving quality, inspection efficiency, and accuracy on production lines. However, the long design cycle and high construction costs of quality inspection robot workstations make it challenging to perform effective simulation validation during the design phase.

The introduction of virtual reality presents new opportunities for industrial simulation. Virtual reality technology precisely simulates industrial production environments and processes, including machine operations and changes in physical conditions. It creates an immersive, three-dimensional virtual environment. Engineers can accurately simulate the operation process of robot workstations in this virtual environment, assisting designers in rapidly constructing product models and evaluating and optimizing the performance of robot workstations [2,3,4,5]. Applying VR technology for simulation testing is a cost-effective and efficient approach, providing an effective means for evaluating industrial quality inspection robot workstations.

Currently, virtual reality technology is widely applied in the industrial sector. Scholars have conducted research in areas such as production and manufacturing. Zhu Zhaoju [6] constructed a digital twin of a robot using the Unity3D platform. Through the use of virtual-to-real mapping techniques, the simulation achieved real-time visualization and monitoring of robot milling motion. Elisa Prati [7] introduced an integrated approach to designing human–robot collaborative workstations in industrial workshops. By leveraging virtual reality (VR) technology, this method enables designers to create interactive workstation prototypes and validate design outcomes during the early stages of development. Wei Xia [8] devised an image acquisition technique for a 6-DOF serial robot, utilizing industrial cameras. The detection system’s digital twin unit was established, and the proposed UR10 robot kinematics model and acquisition trajectory planning method were implemented. The Unity3D software facilitated the configuration of the digital twin environment for the detection system. Kamali [9] proposed a deep reinforcement learning (DRL) approach to address the motion planning problem for robot arms. This method enables the reflection of human hand movements onto the robotic arm. By constructing a virtual reality environment platform, users can smoothly control the position and orientation of the robot’s end effector through hand movements. Gustavo Caiza [10] proposed the use of immersive digital twin (DT) to implement an industrial process monitoring and control architecture for manufacturing execution systems (MES). Cyberphysical systems, MES, robotics, the Internet of Things, virtual reality, and Open Platform Communications Unified Architecture (OPC UA) communication protocols are used to integrate these technologies and enhance the capabilities of the DT by providing higher performance. However, research on virtual reality technology in the field of quality inspection robot workstations is relatively scarce.

To address the aforementioned issues, we propose a method that utilizes VR technology to develop a simulation platform for quality inspection robot workstations. Through this virtual simulation platform, we conduct testing of quality inspection robot workstations, validate the rationality of workstation designs, and accelerate the design, testing, and deployment process of quality inspection robot workstations.

Our key contributions are as follows:

• Design of a virtual reality framework for quality inspection robot workstations: We develop a virtual reality framework for quality inspection robot workstations. With the virtual reality framework, we achieve accurate and robust reconstructions suitable for a variety of industrial applications.
• Development of a virtual simulation platform: We build an immersive interactive virtual simulation platform for quality inspection robot workstations, and apply the defect detection algorithm to the virtual simulation platform for the first time.
• Defect detection experiments in the virtual environment: We conduct experiments on different defect detection algorithms on the virtual simulation platform, and confirm the effectiveness of the improved YOLOv7 algorithm.

2. Virtual Reality Framework for Quality Inspection Robot Workstation

A quality inspection robot workstation is an operational system primarily centered around quality inspection robots. It utilizes machine vision to perform quality inspections on workpieces. The virtual simulation platform for quality inspection robot workstations can be divided into five levels: User Service, Mechanism Model, Decision Algorithm, Data Process, and Physical Device, as shown in Figure 1 [11].

Physical devices primarily include components such as robotic arms, mobile rails, vision systems, sensors, and AGVs used in quality inspection workstations. The mechanism model is constructed through geometric modeling and model rendering to create three-dimensional representations of the physical equipment. This includes kinematic models and topological structures. By simulating geometric features, motion models, and decision systems, the digital model achieves a comprehensive mapping of the physical equipment information. The User Service layer, leveraging the effective virtual-to-real mapping mechanism between physical and digital spaces, enables process simulation, solution validation, and algorithm optimization for quality inspection workstations in the virtual environment. Digital space and physical space interact through a data management system, enabling data collection, transmission, computation, and feedback for devices. Quality inspection robots capture images of the workpiece surface using cameras. The decision system invokes defect detection algorithms to analyze the quality of the workpiece surface. After processing, the results are fed back to the user service end, thus achieving the simulation-driven quality inspection robot workstation system.

3. Development of Virtual Simulation Platform

3.1. Digital Model Construction

Developing a simulation platform for quality inspection robot workstations based on virtual reality platform technology architecture starts with constructing the digital model of the quality inspection robot workstation, as shown in Figure 2 [12,13,14]. Digital model construction involves integrating physical information to map virtual units to the geometry, structure, properties, behavior, rules, logic, and entire product lifecycle of physical entities. This process includes three main components: setting up workstations, constructing geometric models, and defining physical rules for robots. Workstation scene construction involves setting up virtual scenes for production areas, inspection zones, and finished product areas by configuring scene cameras, UI elements, scripts, and lighting. This process aims to achieve a highly accurate representation of factory environments. Geometric model construction involves building the geometric model of a quality inspection robot using the 3D modeling software CATIA, ensuring that the model components have the same structure, shape, size, and assembly relationships as the actual robot. The model is then imported into 3Ds Max, a 3D rendering software, to apply high-fidelity texture materials and color attributes, enhancing the visual effect of the model. Robot physical rule construction involves loading forward and inverse kinematics models onto the three-dimensional model of the robot within the virtual simulation engine Unity3D. This process enables the robot to simulate movement in digital space, ensuring a genuine constraint relationship between the behavior of the physical entity and the established rules.

3.2. Robot Simulation Drive

In the virtual reality environment, the industrial quality inspection robot is driven by simulation. Based on the kinematics theory of the robot, we establish the formula relationship among the relevant kinematic parameters and impose the rotation angle and speed constraints of each joint through the script to simulate its physical behavior characteristics. The forward kinematics analysis of the robot structure is carried out to determine the pose of the robot’s end connecting rod coordinate system in the fixed base coordinate system. According to the forward kinematics of the robot and its transformation principle, the D–H coordinate system of the industrial quality inspection robot is established, and the mechanical structure of the robot is transformed into an analytical mathematical model through the D–H coordinate system, so as to realize the drive of the industrial quality inspection robot in the intelligent virtual simulation platform. The established robot D–H coordinate system and its original state are shown in Figure 3.

The pose transformation between the local coordinate systems of adjacent links requires two rotational homogeneous transformations and two translational homogeneous transformations. The change matrix between {$i-$1} system transformation and {$i$} system transformation is expressed as:

$$T_i=Rot\left(z,{\theta }_i\right)Trans\left(\mathrm{0,0},d_i\right)Trans\left(a_i,\mathrm{0,0}\right)Rot\left(x,{\alpha }_i\right)$$

(1)

where ${\theta }_i$, $d_i$, $a_i$ and ${\alpha }_i$ represent multiple linkage parameters of the robot, and ${\theta }_i$ is the Angle between axis $X_{i-1}$ and axis $X_i$. $d_i$ is the directional distance between axis $X_{i-1}$ and axis $X_i$ along axis $Z_{i-1}$; $a_i$ is the directional distance between axis $Z_{i-1}$ and axis $Z_i$ along axis $X_i$; ${\alpha }_i$ is the angle between axis $Z_{i-1}$ and axis $Z_i$. Then, multiply the adjacent connecting rod transformation matrix successively to obtain the coordinate transformation relationship between the robot’s sixth axis and the end coordinate system, as shown in Equation (2):

$$T_6=T_1\left({\theta }_1\right)T_2\left({\theta }_2\right)T_3\left({\theta }_3\right)T_4\left({\theta }_4\right)T_5\left({\theta }_5\right)T_6\left({\theta }_6\right)$$

(2)

The industrial quality inspection robot is independently driven by six independent servo motors, which are successively divided into levels from the fixed end according to 6 degrees of freedom [15,16]. In the Unity 3D platform Hierarchy, the virtual reality model is divided into the Base, Shoulder, Elbow, Wrist1, Wrist2, Wrist3 hierarchy. On the basis of completing the hierarchical division of the equipment structure, the joint entity is added to the joint and connecting rod of the robot. In the Hierarchy attribute panel, assign the “Articulation Body” component to the robot, adding rigid body and collision properties to the joints and links. In a six-axis robot, the joints are limited to rotating around one axis, so set the Shoulder, Elbow, Wrist1, Wrist2, and Wrist3 parts to “Revolute” rotation constraints, allowing them to rotate around the parent anchor point’s axis. After assigning the kinematic and controller attributes, drive the robot’s motion control by writing a C# script, ultimately achieving the simulation drive of an industrial inspection robot on a virtual reality platform.

3.3. Human-Computer Interaction System Development

In the virtual reality environment, the user interacts with the quality inspection robot in various forms and controls the robot to carry out quality inspection tasks [17,18,19]. By adding ‘Button’ and ‘Slider’ components to the Canvas panel, users can operate the industrial quality inspection robot through virtual buttons, sliders, and other controls. This includes adjusting the robot’s running speed, selecting inspection items, and viewing inspection results. By importing plugins such as the SteamVR Plugin SDK, users can control the system using handheld controllers. By adding an Audio Source component to set up a real-time feedback system, users can obtain information such as the status of the robot, inspection progress, and results based on auditory feedback. In the VR environment, users can also inspect quality assessment results and closely examine defect details, including specific information about non-conforming products, defect types, and their locations.

4. Workflow Simulation Testing

4.1. Experiment Environment Setting

The simulation testing equipment primarily includes the Vive Pro head-mounted device, Vive controllers, base stations, the quality inspection robot workstation simulation platform, and a computer (with Windows 11, AMD Ryzen 5 5600 G CPU, NVIDIA GeForce RTX 3060Ti GPU, and 16 GB RAM). When using virtual reality devices, it is necessary to set up a virtual interaction area of at least 2 m × 1.5 m. Each base station has a 120° field of view, ensuring it tilts downward between 30° and 45° to fully cover the designated virtual interaction area.

4.2. Quality Inspection Robot Workstation Test Process

The simulation tests are carried out according to the operation process of quality inspection robot workstation in real work. The workflow can be broken down into the following steps: First, the Automated Guided Vehicle (AGV) picks up parts from the inspection area using an automated loading and unloading system. The AGV then transports these picked parts to the quality inspection robot. At the quality inspection robot workstation, the detection robot utilizes a high-resolution camera and sophisticated image processing algorithms to perform defect inspection on the parts. Once the inspection is complete, the AGV separately transports the qualified parts and the non-conforming parts to their respective locations. Qualified parts are sent to the finished product area, while non-conforming parts are routed to a specialized processing area for further handling. During the inspection process, data such as part inspection results and inspection time are automatically stored and uploaded to the Manufacturing Execution System (MES).

4.3. Test Results and Analysis

In this experiment, the workflow of the quality inspection robot workstation was verified in a virtual environment, and the simulation results are a shown in Figure 4.

From the simulation results, the quality inspection robot is capable of defect detection on the parts carried by the AGV. After identifying the defect type, the data are uploaded to the Manufacturing Execution System (MES). The AGV receives this data and transports the parts to their respective units, achieving the expected outcome. Through experimentation, it has been verified that the quality inspection robot workstation provides a favorable human–computer interaction environment, and the workflow operates seamlessly. The design proposal for the inspection robot workstation is feasible.

5. Defect Detection Experiments in the Virtual Environment

5.1. Defect Detection Algorithm Model

The industrial quality inspection robot utilizes machine vision to recognize defect types on workpieces and employs the improved YOLOv7 algorithm for part defect detection. YOLO series object detection algorithms have been widely used in the field of computer vision because of their high efficiency and real-time performance. YOLOv7 is the most advanced algorithm in the YOLO series, which has been improved in accuracy and speed [20,21]. In the aspect of defect detection, the YOLOv7 algorithm still has the disadvantage that it is difficult to detect because the detected target is small and the features are not obvious. On the foundation of the YOLOv7 algorithm, the improved YOLOv7 algorithm incorporates the Convolutional Block Attention Module (CBAM) attention module into the feature fusion network. The CBAM incorporates both spatial attention and channel attention mechanisms, gradually refining feature maps. This approach emphasizes important features while suppressing less relevant information. The model architecture is illustrated in Figure 5. The improved YOLOv7 algorithm effectively solves the problems of small defect areas on the metal surface and significant distribution changes in the image and greatly improves the accuracy of defect detection.

After incorporating the Convolutional Block Attention Module (CBAM) into the feature fusion network of the improved YOLOv7, the structure of the defect detection algorithm is depicted in Figure 6.

5.2. Experiment’s Data Processing

The defect detection experiment utilized the GC10-DET dataset as the data source for algorithm testing. GC10-DET is a collection of metallic surface defect data gathered from real-world industrial scenarios [22]. It encompasses four types of surface defects: oil spots, water spots, welding lines, and inclusions, as illustrated in Figure 7. The dataset comprises 1584 grayscale images, randomly divided into training, validation, and test sets in an 8:1:1 ratio.

5.3. Evaluation Indicators

To better compare the experimental performance of improved network models on datasets and describe and analyze the experimental results, we select commonly used evaluation indicators in the field of object detection as the evaluation criteria for this experiment [23]. These indicators include precision (P), recall rate (R), average precision (AP), and mean average precision (mAP). The relevant calculation formulas are shown in the figure below.

$$\mathrm{P}=\frac{TP}{TP+FP}$$

(3)

$$\mathrm{R}=\frac{TP}{TP+FN}$$

(4)

$$\mathrm{A}\mathrm{P}={\int }_0^{1}P\left(R\right)dR$$

(5)

$$\mathrm{m}\mathrm{A}\mathrm{P}=\sum _{i=1}^N\frac{\mathrm{A}\mathrm{P}\left(i\right)}{N}$$

(6)

5.4. Model Training and Deployment

Using the PyTorch 1.13.1 + CUDA 11.7 deep learning framework, we train the algorithmic model and subsequently deploy it to the quality inspection robot’s workstation’s virtual simulation platform. Within the virtual environment, we conduct defect detection experiments. First, we export the model’s weight file (.weights) and configuration file (.cfg) to the ONNX format. Next, in Unity3D, we utilize the Barracuda library from ML-Agents to load the ONNX model. By writing C# scripts, we load the model, process input images captured by the camera, and invoke the model for inference. The output of the YOLO model includes detected object bounding boxes, class labels, and confidence scores. By writing scripts, we parse these outputs and transform the detection results into visual elements within the Unity scene. This is achieved by creating UI elements in the scene to display the detection boxes and category labels, enabling defect detection for workpieces.

5.5. Experimental Results and Analysis

When conducting defect detection experiments for parts on the quality inspection robot workstation’s virtual simulation platform, we utilize machine vision to identify part defects. Under the same dataset split, we compare the currently popular YOLO series algorithms with the proposed improved YOLOv7 algorithm in this study. These comparative algorithms include YOLOv5s, YOLOv5m, YOLOv5l, YOLOv5x, YOLOv7, and YOLOv7-X. The comparative experimental results are shown in

Table 1. Comparison of detection performance with different models.

Method	P (%)	R (%)	mAP (%)
YOLOv5s	51.2	67.6	65.6
YOLOv5m	55.8	69.6	67.9
YOLOv5l	61.5	59.2	60.8
YOLOv5x	60.6	63.4	68.0
YOLOv7	59.8	70.9	71.9
YOLOv7-X	73.1	50.1	65.7
Ours	70.6	72.7	73.3

, with the optimal values highlighted in bold.

Compared with the experimental results in the table, it is found that the precision (P) of the YOLOv7-X algorithm is the highest, reaching 73.1%. The improved YOLOv7 algorithm has the highest recall rate (R) and mean average precision (mAP) are, which are 72.7% and 73.3%, respectively. Compared with the YOLOv7 algorithm, the precision (P) of the improved YOLOv7 algorithm has increased by 18.1%, the recall rate (R) has improved by 3.6%, and the mean average precision (mAP) has in-creased by 1.4%. It can be seen from the test results in Figure 8 that the robot can accurately detect surface defects and has a high precision rate for various types of defects. The proposed method in this paper is superior to the previous algorithm in terms of precision and recall rate, thus proving the effectiveness of the improved YOLOv7 algorithm for defect detection.

6. Conclusions

In response to the high development costs, lengthy design cycles, and challenges in validating algorithm effectiveness for quality inspection robot workstations, we developed a virtual simulation platform for quality inspection robot workstations based on virtual reality technology. Within this virtual environment, we rapidly constructed product models, performed performance analysis, and optimized the system. We created virtual simulation work scenarios, designed an improved YOLOv7 algorithm, and tested the quality inspection robot workstation workflow within the virtual simulation environment. Building upon this foundation, we conducted defect recognition experiments for the quality inspection robot using virtual reality technology. By comparing experimental results, we demonstrated the effectiveness of the improved YOLOv7 algorithm. Compared to the YOLOv7 model, the precision (P) has increased by 18.1%, the recall rate (R) has improved by 3.6%, and the mean average precision (mAP) has increased by 1.4%. Through experiments, we validated the feasibility of the quality inspection robot workstation design. Modeling, simulating, and optimizing complex systems of quality inspection robot workstations using virtual reality technology holds significant potential for industrial production.

Our future work includes lightweight improvements to the object detection algorithm. We also plan to improve the model fidelity of the industrial quality inspection robot between the VR interface and the physical systems.