Automated Evaluation Method for Risk Behaviors of Quay Crane Operators at Ports Using Virtual Reality

Abstract

Currently, the operational risk assessment of quay crane operators at ports relies on manual evaluations based on experience, but this method lacks objectivity and fairness. As port throughput continues to grow, the port accident rate has also increased, making it crucial to scientifically evaluate the risk behaviors of operators and improve their safety awareness. This paper proposes an automated evaluation method based on a Deep Q-Network (DQN) to assess the risk behaviors of quay crane operators in virtual scenarios. A risk simulation module has been added to the existing automated quay crane remote operation simulation system to simulate potential risks during operations. Based on the collected data, a DQN-based benchmark model reflecting the operational behaviors and decision-making processes of skilled operators has been developed. This model enables a quantitative evaluation of operators’ behaviors, ensuring the objectivity and accuracy of the assessment process. The experimental results show that, compared with traditional manual scoring methods, the proposed method is more stable and objective, effectively reducing subjective biases and providing a reliable alternative to conventional manual evaluations. Additionally, this method enhances operators’ safety awareness and their ability to handle risks, helping them identify and avoid risks during actual operations, thereby ensuring both operational safety and efficiency.

1. Introduction

Container handling is a core aspect of port operations, requiring precise handling and strong safety awareness from operators, which directly impacts the successful delivery of goods. However, the frequent use of heavy machinery at container terminals often leads to accidents, resulting in severe economic losses, injuries, and even fatalities [1]. Since absolute safety cannot be guaranteed in these operations, and given the significant risks involved, safety training is particularly crucial. One of the main causes of frequent accidents is improper handling by operators. Traditional port operator training relies on real equipment operation and experience-based guidance, which is costly and risky. Moreover, it cannot comprehensively simulate various emergency situations or recreate accident scenarios, thus limiting the effectiveness and scope of training. Researching and developing effective safety training methods and techniques to improve operators’ safety awareness and precision can reduce the frequency and severity of accidents, as well as significantly lower economic losses and personal injuries.

Traditional port operator training relies on real equipment operation and experience-based guidance to develop the skills necessary for crane operation. However, this method is costly and risky. On-site crane operation training often unavoidably hinders normal construction processes and exposes novice operators to danger, as even a minor mistake can lead to serious incidents, such as injuries, delays, and budget overruns [2].

The advent of virtual reality (VR) technology has effectively replaced the traditional hands-on training model by offering a realistic experience while reducing costs and risks [3,4]. VR-based safety training not only provides a comprehensive and long-lasting platform to develop perceptual experiences and cognitive abilities in solving problems and making decisions under complex and stressful situations but also eliminates real-time exposure to danger or environmental conditions [5]. In crane training, VR technology is often used to train operators [2,6].

Lin et al. [7] demonstrated the use of 4D-BIM technology to create real-time augmented reality (AR) visualizations for crane operations, thereby optimizing both operational safety and efficiency. Fang and Teizer [8] designed a multi-user virtual 3D training environment to improve teamwork between crane operators and ground personnel. Juang J. R. et al. [9] developed a crane simulator named SimCrane 3D, which applies kinesthetic feedback and stereoscopic vision to the virtual crane simulator to enhance operator training. B. Patrão and P. Menezes [10] proposed an immersive virtual reality simulator that uses a stereoscopic head-mounted display to deliver visual and auditory stimuli, thereby training tower crane operators. Noda Y et al. [11] researched a training system that allows operators to safely and effectively master the operation of electric overhead cranes while suppressing load sway. Additionally, a distributed virtual reality bridge crane simulator system based on a 3D engine has been developed [12].

Although many existing virtual crane training systems can develop operators’ practical skills, they mainly focus on enhancing performance and lack the simulation of potential accidents during operations. These systems also do not include the design of corresponding risk simulation modules, which makes them ineffective in enhancing operators’ safety risk awareness. Since container handling at container terminals is one of the most accident-prone activities in port operations, enhancing drivers’ risk awareness and providing comprehensive training for port operators is particularly important.

Safety risk assessment is a crucial part of safety management and an essential component of safety system engineering [13]. Safety risk assessments mainly focus on areas such as coal mining [14], subway stations [15], construction sites [16], and ports [17,18,19,20,21,22,23]. In the context of ports, researchers have primarily studied loading and unloading operations, port equipment, and dangerous goods. For instance, Chen-Yu Lin et al. [18] developed a novel region-specific risk assessment model for container ports. This model divides the container port into four areas—the loading and unloading area, internal transportation area, storage area, and gate area—based on the container transport process. By combining Failure Mode and Effect Analysis (FMEA) with a quantitative risk analysis model, it assesses the risk of accidents in each segment of the process. Alyami H. et al. [19] proposed an advanced FMEA method based on a fuzzy rule-based Bayesian network (FRBN) to evaluate the severity of hazardous events (HEs) at container terminals. Sunaryo and Hamka [20] developed a risk assessment model for container ports using hazard identification, risk assessment, and fault tree analysis methods. Y.L. Yang et al. [21] analyzed the risk factors affecting safe loading and unloading operations, their causal relationships, and interconnections. They used the Decision-Making Trial and Evaluation Laboratory (DEMATEL) method to evaluate the core risk factors for container handling operations at Taiwan’s Kaohsiung Port. Other methods include safety assessment models for dangerous goods within the port [22,23].

In maritime navigation, various types of risk assessments are employed. Previous studies on quantitative risk analysis of maritime navigation have utilized different methods for risk evaluation, each with distinct applications, such as event trees (ETs), fault trees (FTs) [24], Bayesian networks (BNs) [25,26], and the Analytic Hierarchy Process (AHP) [27,28]. ETs and FTs are particularly useful for analyzing the causal effects of specific risks and are easy to understand. However, they rely heavily on historical data and can become extremely time-consuming as the number of factors increases. BNs, on the other hand, are quantitative tools applied to maritime traffic modeling and have indeed extended into the field of maritime traffic safety. While BNs allow for the integration of data with expert knowledge and are suitable for complex systems with probabilistic uncertainties, their complexity and the difficulty in determining expert probabilities can present challenges [29]. A summary and analysis of the literature are presented in

Table 1. Literature summary and analysis.

Classification	Literature	Literature Analysis
Application of Virtual Reality Technology in Crane Training	He, F. et al. [6], Lin et al. [7], Fang and Teizer [8], Juang, J. R. et al. [9], B. Patrão and P. Menezes [10], Noda, Y. et al. [11], Dong, H. et al. [12]	These studies mainly focus on improving operational skills but lack the simulation of potential accidents that may occur during operations. Additionally, they do not include corresponding risk simulation modules, making these systems ineffective in improving operators’ safety risk awareness.
Application Domains of Safety Risk Assessment	Coal mining [14], subway stations [15], construction sites [16], container ports [17,18,19,20], port loading and unloading operations [21], dangerous goods [22,23]	Current risk assessment methods primarily focus on evaluating container port and quay crane operations in real operational settings, with limited application in virtual environments.
	Maritime navigation at ports [24,25,26,27,28,29]	Fault trees (FTs) and event trees (ETs) rely on historical data, and as the number of factors increases, they become extremely time-consuming. Bayesian networks (BNs) combine historical data with expert knowledge, but their complexity and the probabilistic nature of expert judgments present challenges. The Analytic Hierarchy Process (AHP) introduces subjective assumptions when assigning weights to individual factors but does not account for the interdependent relationships between the various structural elements.

.

Currently, few risk assessment methods directly target quay crane operators at container ports, and they mostly focus on real operating scenarios, with limited research on risk assessment in virtual environments. Therefore, this paper proposes an automated evaluation method for the risk behaviors of quay crane operators at ports. Building upon the existing automated quay crane remote operation simulator, a risk simulation module is designed, and a Deep Q-Network (DQN) model is constructed to learn the operational methods and specifications of skilled operators. This model, trained using a large amount of simulated operational data, accurately reflects the operational behaviors and decision-making processes of skilled operators. Based on this model, a method for evaluating operators’ risk behaviors is proposed. Through interaction with the baseline model, the system can simulate various operational scenarios, monitor and evaluate operators’ behaviors in real time, and provide an objective and effective assessment of their risk behaviors.

2. Risk Simulation Module Design

In container port operations, improper handling by drivers can lead to severe property damage and casualties, making the standardization of driver operations and the evaluation of operational risks crucial. To ensure the scientific rigor and effectiveness of operational training and risk assessment methods for remote quay crane operators at ports, this study adds a risk simulation module to the existing automated quay crane remote operation intelligent simulation system. Figure 1 shows the original automated quay crane remote operation simulator, which comprises a model physics engine module, a model motion control module, and a remote crane monitoring system data management module. This simulator not only accurately replicates the real operational scenarios of automated quay cranes but also enables the training of novice operators’ operational capabilities [30].

The risk simulation module is developed based on one of the most significant risks that remote quay crane operators face during operations—collisions. It identifies and collects key data that could lead to collision risks during operations. These data help determine which operational methods and strategies employed by drivers are effective and safe. Therefore, in the automated quay crane remote operation simulator, this study uses the simulator’s sensor system to capture environmental information and record key data at every time point during the skilled drivers’ operations, including actions, speed, position, and the safety distance between the crane and the operational target. The specific data recorded are listed in

Table 2. Calculation parameters.

Symbol	Physical Quantity	Parameter Type
m	load mass	constant
l	length of wire rope	control variable
$v_l$	wire rope lifting speed	control variable
$v_c$	trolley speed	control variable
$\phi$	swing angle of the load (angle between the wire rope and the vertical direction)	control variable
$v_s$	swing speed of the load	control variable
${\omega }_s$	swing angular velocity of the load	control variable
t	time to execute each operation	passive measurement
$d_p$	spreader position	passive measurement
$d_c$	trolley position	passive measurement
$d_1$	relative distance between the load and the target position	passive measurement
$d_2$	relative distance between the container and the truck pallet	passive measurement
$d_3$	relative distance between the left and right sides of the operational path along the ship’s side and obstacles	passive measurement
$d_4$	relative distance between the load and obstacles	passive measurement
$d_5$	relative distance between the load and the top of the target container	passive measurement

, and Figure 2 illustrates the parameters corresponding to Table 2. The physical quantities recorded in this study are classified into constants, control variables, and passive measurements. Control variables are those that the driver can manually control via the joystick, while passive measurements are the variables measured by the system during the driver’s operation. All variables in this study use the International System of Units (SI), and the ranges of other variables are specified in Equations (10) and (11).

These data help identify the potential risks that drivers may encounter under different operational conditions. For example, the speed of the trolley enables drivers to learn how to control it in various operating environments, particularly when approaching obstacles and target positions, to avoid collisions. The swinging angle and speed of the load are used to train drivers on their sensitivity to the swinging angle and how to effectively control the swinging speed, thereby improving their skills in maintaining load stability. The position of the spreader allows drivers to grasp the real-time location of the spreader, enhancing their spatial awareness of both the spreader and the surrounding environment, thus avoiding collisions. The relative distance between the load and the target position trains drivers to accurately judge and operate toward the target, reducing the risk of collisions caused by errors in distance judgment. The relative distance between the container and the truck pallet helps drivers precisely control the load position during loading and unloading, preventing collisions between the container and the pallet. The relative distance between the left and right sides of the load’s operational path along the side of the ship and obstacles is intended to train drivers to operate in complex environments, improving their perception and avoidance of obstacles on both sides. The relative distance between the load and obstacles is designed to keep drivers vigilant about the distance between the load and obstacles during operations, thereby avoiding collisions. The relative distance between the load and the top of the target container helps drivers accurately control the height of the load, preventing collisions with the top of the target container and avoiding container collapse.

3. Baseline Model Design Based on a DQN

This study uses the DQN algorithm to establish a baseline model for the operational behaviors of skilled quay crane operators. The DQN is a value-based deep reinforcement learning (DRL) method that combines deep neural networks with the Q-learning algorithm to train an agent (in this case, the simulated port operator) to make optimal decisions in a specific environment.

3.1. State Space

The state space represents the relevant characteristics of the port operation environment and is typically used as input information for DRL. In this model, the input state information is collected from the virtual operation simulator’s sensors, including data on the position and speed between the spreader and the container. The specific state information includes the following: the load mass m, wire rope length l, wire rope lifting speed $v_l$, trolley speed $v_c$, swing angle of the spreader $\phi$, swing speed of the load $v_s$, swing angular velocity ${\omega }_s$, time taken to perform each operation t, spreader position $d_p$, trolley position $d_c$, relative distance between the load and the target position $d_1$, relative distance between the container and the truck pallet $d_2$, relative distance between the operational path along the ship’s side and obstacles on both sides $d_3$, relative distance between the load and obstacles $d_4$, and relative distance between the load and the top of the target container $d_5$, totaling 15 states.

The final state space is defined as $s=\left(m,l,v_l,v_c,\phi ,v_s,{\omega }_s,t,d_p,d_c,d_1,d_2,d_3,d_4{d}_5\right)$, which can represent the status of the spreader at any given moment in the environment.

3.2. Reward Function

In the process of learning and training the operational model of skilled drivers, the design of the reward function plays a crucial role, as it determines the effect and efficiency of the neural network’s training. The reward function serves as an indicator to evaluate the effectiveness and safety of the operator’s actions with a results-oriented approach. The reward function in this study is designed as a combination of operational safety, operational efficiency, and load stability.

3.2.1. Operational Safety

In terms of safety, the spreader needs to avoid collisions with obstacles and successfully complete the lifting task. This paper sets up different safety rewards according to the actual operating conditions and increases the danger zone near the obstacles so that the spreader can avoid obstacles to reach the target point more quickly.

If the spreader collides with an obstacle, the driver receives a significant penalty to strongly discourage dangerous operations. At each time step, if no collision occurs, a small positive reward is given to the driver, encouraging them to cover as much distance as possible without collisions. The specific design is as follows:

$$r_{\mathrm{safe}}=\left\{\begin{array}{c}+100,\mathrm{When}\mathrm{the}\mathrm{spreader}\mathrm{collides}\hfill \\ +0.01,\mathrm{other}\hfill \end{array}\right.$$

(1)

The danger zone is defined as the area where the spreader (whether unloaded or loaded) approaches an obstacle but has not yet collided with it. To help the spreader quickly move away from obstacles, the penalty around obstacles is intensified. However, to avoid trapping the spreader in local situations, the penalties in the danger zone should not be too concentrated and need to be adjusted according to the distance from the nearest obstacle. The reward function for the safety distance is designed as follows:

$$r_{\mathrm{distance}}=\left\{\begin{array}{c}+5,d_4\ge 2m\hfill \\ +10\left(d_4-0.5\right),0.5m\le d_4\le 2m\hfill \\ -20,d_4<0.5m\hfill \end{array}\right.$$

(2)

where $r_{distance}$ is the distance between the container and the obstacle at time t. The value of the safety distance is set based on the safety regulations of remote-controlled quay crane operations and the average operating values of skilled drivers.

The overall operational safety reward function is given by

$$R_{\mathrm{safe}}=r_{\mathrm{safe}}+r_{\mathrm{distance}}$$

(3)

3.2.2. Operational Efficiency

In terms of operational efficiency, the objective of the efficiency reward is to encourage the model to complete tasks as quickly as possible while maintaining safety and accuracy. Therefore, the efficiency reward function consists of time efficiency, operational accuracy, and safety rewards.

The time efficiency reward is designed based on the time required to complete an operation. For every 5 s reduction in operational time, a reward of (+1) point is given. The specific design is as follows:

$$R_{efficiency}=max\left(0,{\displaystyle \frac{T_{target}-T_{actual}}{5}}\right)$$

(4)

where $T_{\mathrm{target}}$ is the target operational time for completing a loading or unloading task and $T_{\mathrm{actual}}$ is the actual time taken to complete the operation.

Operational accuracy rewards are designed based on the distance between the ideal deviation of container placement and the actual deviation during the actual operation to encourage high accuracy upon completion of the task, and are designed as follows:

$$r_{precision}=K_{precision}·e^{-\lambda (d_{target}-d_{actual})}$$

(5)

where $K_{\mathrm{precision}}$ is the reward coefficient for accuracy, $\lambda$ controls the rate of the reward decay, $d_{\mathrm{target}}$ is the actual deviation distance from the target position, and $d_{\mathrm{actual}}$ is the ideal deviation for placing the container.

To prevent compromising safety while pursuing efficiency, a safety reward is also set:

$$r_{\mathrm{safety}}=\left\{\begin{array}{c}{K}_{\mathrm{safety}},iftherearenosecurityviolations\hfill \\ +0,other\hfill \end{array}\right.$$

(6)

where $K_{\mathrm{safety}}$ is the safety reward coefficient, and its range of values is [0,1].

The total operational efficiency reward is given by

$$R_{\mathrm{efficiency}}=r_{\mathrm{time}}+r_{\mathrm{precision}}+r_{\mathrm{safety}}$$

(7)

3.2.3. Load Stability

When moving the spreader and container, the load’s maximum swing angle is measured to assess the operational smoothness. A smaller swing angle indicates better load stability. The reward function for load stability is designed as follows.

$$R_{\mathrm{stability}}=\left\{\begin{array}{c}+15,\phi \le a\hfill \\ +10,a<\phi \le b\hfill \\ -15,\phi >b\hfill \end{array}\right.$$

(8)

According to the site environmental conditions and the actual operation of the driver, the values of a and b in the above formula are $a=8^{\circ }$ and $b={20}^{\circ }$, which are more in line with real site conditions and operational safety requirements. Let the weights of operational safety, operational efficiency, and load stability be ${\omega }_{\mathrm{safe}}$, ${\omega }_{\mathrm{efficiency}}$, and ${\omega }_{\mathrm{stability}}$, respectively. The final design of the reward function is expressed as

$$R={\omega }_{safe}·R_{safety}+{\omega }_{efficiency}·R_{efficiency}+{\omega }_{stability}·R_{stability}$$

(9)

3.3. Network Design

In this study, the DQN network model for driver operations is structured as shown in Figure 3. The input layer takes the current state of the spreader in the simulator as input. The loss function is set to the Mean Squared Error (MSE) loss, and the activation function used is the ReLU function. The optimizer is the Adam optimizer, which ensures stable training. The weights of the neural network are initialized using the Xavier initialization method, which helps mitigate the issues of vanishing and exploding gradients by keeping the variance of each layer’s output close to one. The evaluation network and the target network have the same structure, both using the network structure shown in Figure 3, but their update frequencies differ. The evaluation network is updated at every step, while the target network is updated only after a certain period, meaning that the parameters of the evaluation network are periodically copied to the target network.

3.4. Model Training and Learning Process

The DQN-based skilled driver training model consists of the following stages: First, before training and learning, it is essential to correctly design the environmental states, skilled operator actions, action strategies, and reward functions. Next, the model learns to obtain a strategy that maximizes future rewards through interaction and continuously adjusts the value function to achieve the optimal strategy. Finally, after the training and learning process is completed, the optimal control strategy can be derived based on state information. Novice drivers can then use this model to learn the operational methods of skilled drivers, guiding them to safely and efficiently complete the entire loading and unloading process of containers.

Figure 4 shows a flowchart of the model training using the DQN algorithm based on skilled driver data. First, the sensor system is used to collect environmental information from the simulated port operation scenario. Then, the current environmental state information of the port operation scene is input into the evaluation network, which outputs the Q-values of each action in the action space. Based on the policy, an action a is selected, and the spreader executes it to obtain a reward r, transitioning the spreader to a new state $s_{t+1}$. The current state $s_t$, action a, next state $s_{t+1}$, and reward are stored as historical experience information in the experience replay pool.

At each time step, a random sample of data is selected from the experience pool to train the evaluation network, allowing it to approximate the optimal action value. Finally, the DQN loss function is computed by combining the Q-values of the evaluation network and the target network, and the network parameters are updated using historical experience data. When the evaluation network has been sufficiently trained, its weights will approximate the optimal parameters. Additionally, the evaluation network periodically copies its parameters to the target network every N time steps to reduce the correlation between the two networks.

4. Evaluation Method Design for Driver Risk Behaviors

In this section, the trained baseline model is used to evaluate the basic skills and operational safety risk behaviors of novice drivers in the remote operation of quay container cranes. Novice drivers interact with the trained baseline model, comparing their operational behaviors with the model’s expected behaviors to assess their capabilities and risk behaviors.

4.1. Operational Difference Measurement

To quantify the disparity between the operation sequences of novice drivers and the optimal ones recommended by the benchmark model, this paper assesses whether the novice drivers’ operations can effectively maneuver the vehicle to reach the desired state stably and rapidly by comparing the differences between the state transitions resulting from their operations and those suggested by the model. The Euclidean distance is employed to compare the distances between the actual state transitions and the expected ones.

The Euclidean distance is a method for calculating the straight-line distance between two points in a multi-dimensional space. In this context, it can be used to measure the difference between the state achieved by the novice operator’s actions, denoted as $S_i^{novice}$, and the expected state derived from the model’s optimal actions, denoted as $S_i^{model}$. Each state consists of multiple dimensions, such as position, speed, and other variables. The Euclidean distance between the novice operator’s state and the model’s optimal state is given by

$$d=\sqrt{\sum _{i=1}^n{(S_i^{model}-S_i^{novice})}^2}$$

(10)

In formula (10), the Euclidean distance is used to measure the differences between state vectors. Each state vector consists of multiple dimensions, with each dimension representing a different variable $x_i$ (e.g., position or speed). To avoid certain variables dominating the distance calculation due to their larger ranges, all variables (i.e., each dimension $x_i$ in the state vector) are normalized before the distance calculation. This normalization ensures that all variables contribute equally to the distance metric, regardless of their original scales. For each dimension $x_i$ in the state vector, the normalization process adjusts the variable to the range $[0,1]$. The normalization for a variable $x_i$ with a range of $[a,b]$ is given by

$$x_{i,\mathrm{norm}}={\displaystyle \frac{x_i-a}{b-a}}$$

(11)

This normalization ensures that all dimensions are scaled equally, preventing variables with larger ranges from disproportionately affecting the distance metric. If the variables are not normalized, variables with larger ranges may dominate the calculation, leading to biased results that do not accurately reflect the true differences in the state vectors.

4.2. Safety Evaluation Indicators

In the remote control quay crane operation simulator, it is very important to establish safety evaluation indices, which can help evaluate the operational safety of novice drivers and guide training and improvement. Based on the operational standards followed by quay crane operators and the qualitative manual evaluation criteria, this paper quantifies the safety evaluation indicators. Manual scoring often relies on experiential judgment to assess operators’ risk behaviors, such as visually estimating whether the operating distance is within the safe range, which typically results in considerable deviation. By establishing these safety evaluation indicators, risk behaviors can be quantified, enabling a comprehensive evaluation of novice operators’ operational safety and providing targeted improvement measures accordingly. The specific safety evaluation indicators are described below.

4.2.1. Number of Approaches to Obstacles

The number of times the spreader’s distance from the obstacle is below the safe distance during operation is counted, reflecting the operator’s ability to maintain a safe distance between the equipment and the obstacle.

This paper sets a safe distance $d_{safe}$. Every time the distance between the spreader and the obstacle $d_{actual}<d_{safe}$, the count increases, and the total number of such occurrences during the entire operation is recorded.

4.2.2. Speed Control Consistency

Speed control consistency is evaluated by calculating the standard deviation of the speed changes of the trolley and the spreader, reflecting the driver’s ability to maintain consistent and smooth speed control. For a series of recorded speed values $(v_1,v_2,\dots ,v_n)$, the average speed $\overline{v}$ is calculated as

$$\overline{v}={\displaystyle \frac{1}{n}}\sum _{m=1}^n{v}_m$$

(12)

where n is the total number of recorded speed values, and $v_m$ represents the speed at the $m-th$ recorded moment. The standard deviation of speed changes $\sigma$ is then calculated as

$$\sigma =\sqrt{{\displaystyle \frac{1}{n-1}}\sum _{m=1}^n{(v_m-\overline{v})}^2}$$

(13)

A smaller standard deviation indicates better consistency in speed control.

4.2.3. Operational Accuracy

Operational accuracy assesses the novice driver’s ability to perform specific tasks (e.g., short-distance movement, long-distance movement, and alignment of the spreader and container). The Euclidean distance is used to calculate the deviation between the target point and the actual stopping point to measure the accuracy of these specific operations.

Let the target point be $p_{actual}$ and the actual stopping point be $p_{actual}$. The difference in operational accuracy can be expressed as

$$d_{accuracy}=\sqrt{{(p_{actua{l}_x}-p_{targe{t}_x})}^2+{(p_{actua{l}_y}-p_{targe{t}_y})}^2}$$

(14)

Here, $p_{actua{l}_x}$ and $p_{actua{l}_y}$ are the horizontal and vertical coordinates of the actual stopping point, while $p_{targe{t}_x}$ and $p_{targe{t}_y}$ are the horizontal and vertical coordinates of the target point. $d_{accuracy}$ represents the difference in operational accuracy, where a smaller $d_{accuracy}$ value indicates higher operational accuracy.

4.2.4. Emergency Response Time

The emergency response time measures the driver’s reaction time in the event of sudden incidents (e.g., container tilting during lifting). Sensors record the exact moment when an emergency occurs and the moment when the driver initiates a response. The average emergency response time for a novice driver is calculated as

$$T_{response}={\displaystyle \frac{1}{N}}\sum _{j=1}^N(t_{response,j}-t_{event,j})$$

(15)

where $t_{response,j}$ is the time when the driver responds to the j-th event, $t_{event,j}$ is the time when the j-th emergency occurs, and N is the total number of events.

4.2.5. Load Stability

Load stability is assessed by measuring the swing angle of the load during movement. A smaller swing angle indicates better stability.

From the recorded position data, the spreader position $d_p(x,y,z)$ and the trolley position $d_c(x_c,y_c,z_c)$ are identified. Let the swing angle of the load be $\phi$ and define the unit vector $d_v(0,0,-1)$ pointing vertically downward. The difference vector between the two positions d is then

$$d=d_p-d_c=(x-x_c,y-y_c,z-z_c)$$

(16)

The swing angle $\phi$ is then computed as

$$\begin{array}{cc}\hfill \phi & ={cos}^{-1}\left({\displaystyle \frac{\mathbf{d}·{\mathbf{d}}_{\mathbf{v}}}{\left|\mathbf{d}\right||{\mathbf{d}}_{\mathbf{v}}|}}\right)\times {\displaystyle \frac{180}{\pi }}\hfill \\ \hfill & ={cos}^{-1}\left({\displaystyle \frac{-(z-z_c)}{\sqrt{{(x-x_c)}^2+{(y-y_c)}^2+{(z-z_c)}^2}}}\right)\times {\displaystyle \frac{180}{\pi }}\hfill \end{array}$$

(17)

$${\phi }_{final}=\left\{\begin{array}{cc}-\phi ,\hfill & \mathrm{if}y-y_c<0\hfill \\ \phi ,\hfill & \mathrm{if}y-y_c\ge 0\hfill \end{array}\right.$$

(18)

In the formula, $\left|d\right|$ is the length of the position difference vector between the spreader and the trolley, and ${\displaystyle \frac{180}{\pi }}$ is the factor used to convert radians to degrees. Equation (18) adjusts the sign of the angle based on the sign of the y component of the difference vector d to ensure the correct direction of the final angle. If $y-y_c\ge 0$, it indicates that the spreader is in front of the trolley (relative to the negative y-axis), and the angle is taken as positive; otherwise, it is negative.

5. Experiment

5.1. Preparation and Experimental Content

This experiment was categorized into a control group and an experimental group. The control group used the traditional manual scoring evaluation method, while the experimental group used the evaluation method proposed in this paper. The operational risk behaviors of 10 novice drivers who had undergone benchmark model training were evaluated and scored using the two different evaluation methods. The average score obtained was the average of the scores obtained by these 10 people after multiple operational evaluations. In addition, the average scores obtained for the same driver when evaluated using the two different methods were compared, i.e., the average scores of the same driver in the control group and the experimental group. Furthermore, the members of the control group and the experimental group were all male, aged between 18 and 27, and had no experience in crane operation. Among them, 70% had no experience using virtual reality, 20% had operated in virtual reality 1–3 times, and 10% had operated in virtual reality 4–6 times. All the drivers participating in this experiment received basic induction training, the main content of which included theoretical knowledge of pre-operation, in-operation, and post-operation procedures; theoretical knowledge of stable operation of the lifting gear; accident cases; and various rules and regulations. Subsequently, they all underwent a 10-day training course.

In the control group, the operational safety risk behaviors of novice drivers during the remote operation of the shore-based container crane were scored using the traditional manual scoring method. In the experimental group, the operational safety risk behaviors of novice drivers were evaluated by employing the trained reference model. The novice drivers interacted with the trained reference model by comparing their operational behaviors with the expected behaviors of the model, and their operational safety risk behaviors were thereby evaluated. The specific evaluation content was derived from the actual assessment content for the remote operation of the shore-based container crane workers at the port. The specific evaluation content and scoring criteria are presented in

Table 3. Scoring Criteria.

Items	Specific Requirements	Deduction Criteria	Scores
Pre-operation and Post-operation	Before the operation, check that each monitoring display is normal and give a confirmation command. Sound the horn before starting and after ending.	If the driver fails to verbally confirm the display status before starting the main control and fails to sound the horn at the start and end of the operation, 5 points will be deducted for each failure.	—
During Operation	1. During the alignment practice of the sea and landside girders, there should be no obvious swaying of the lifting gear.	1. If there is obvious swaying of the lifting gear when loading or unloading empty or loaded containers, 10 points will be deducted.	—
	2. Place the 20-foot container on the designated operation lane on the wharf.	2. If the designated container is not placed on the designated operation lane on the wharf, 5 points will be deducted.	—
	3. Use the spreader guide plates properly during operations.	3. If the guide plate is used for loading and immediately retracted, 5 points will be deducted. If the guide plate is not retracted after the lifting hook is raised and the operation is completed, 5 points will be deducted for each occurrence.	—
	4. When lowering the container onto the truck, slow down and stop the spreader 20–25 cm above the pallet.	4. If the guide plate is not retracted after placing the container, 5 points will be deducted.	—
	5. When lowering the spreader above the container, slow down and stop 20–25 cm above the container top.	5. If the lifting gear is not stopped and not slowed down when the container is lowered toward the truck pallet when it has descended 20–25 cm from the pallet, 5 points will be deducted for not stopping and 5 points for not slowing down.	—
Computer-Based English Fault Alerts	Identify the alerts from the computer, accurately report faults, and provide a detailed fault history.	If the driver fails to report faults accurately, in a timely manner, and in detail, 10 points will be deducted.	—
Negative Items	1. Violation of safety operating procedures resulting in equipment accidents.	If a collision occurs or there is a violation of safety operating procedures resulting in equipment accidents, 100 points will be deducted.	—
	2. A severe collision occurs, causing the container to shift.	100 points will be deducted for each occurrence.	—
Total Score	—		—

.

The operating system used in this experiment was Ubuntu 20.04, with 16 GB of memory, an Intel Xeon Gold 5218 CPU, an NVIDIA GeForce GTX 1080 GPU, TensorFlow 1.4 as the deep learning framework, and Python 3.5 as the programming language. The specific parameter settings for the DQN benchmark model are shown in

Table 4. DQN benchmark model training parameters settings.

Parameter	Value
Episodes	20,000
$ϵ$	0.9
$ϵ$ decay	0.9995
$\gamma$	0.9
Learning Rate	0.0001
Replay Buffer	40,000
Batch Size	32

.

5.2. Experimental Results and Analysis

In this experiment, there were a total of 10 drivers, numbered 1, 2, …, 10, with a total score of 100 points. A score of 70 points indicated that the driver had adequate operational safety risk awareness. The results were analyzed and compared using three descriptive statistical methods: (1) mean (M), (2) standard deviation (SD), and (3) coefficient of variation (CV). Table 3 presents the evaluation results of the operational safety risk behaviors of the 10 drivers under different scoring methods.

From

Table 5. Operational safety risk awareness evaluation results for the same driver using different scoring methods.

Group	1		2		3		4		5
	M	SD	M	SD	M	SD	M	SD	M	SD
Experimental group	86	1.31	77.12	3.64	76.38	5.18	78.75	4.89	91.75	4.53
Control group	84.75	9.19	72.12	4.02	77.75	8.83	80.38	7.39	87.62	6.61
Group	6		7		8		9		10
	M	SD	M	SD	M	SD	M	SD	M	SD
Experimental group	87.63	4.17	72.12	2.95	82.75	3.11	87.13	3.18	78.38	4.87
Control group	85.25	9.98	72.12	3.80	85.12	7.75	86.50	10.07	79.00	8.54

and

Table 6. Comparison of coefficient of variation results.

Group	1	2	3	4	5	6	7	8	9	10
Experimental group	0.015	0.047	0.068	0.062	0.049	0.048	0.041	0.038	0.037	0.062
Control group	0.108	0.055	0.114	0.092	0.075	0.117	0.053	0.091	0.116	0.108

, it can be observed that the mean scores of the control group and the experimental group for the same driver were relatively close, and all drivers participating in the experiment passed the operational safety risk awareness test. However, in terms of the statistically significant standard deviation and coefficient of variation of the two groups, the evaluation method based on the reference model adopted by the experimental group was more objective and stable than the traditional manual scoring method followed by the control group and was less susceptible to human subjective judgment. Particularly, for Drivers 1, 3, 6, 8, 9, and 10, when the mean scores in the control group were relatively close to those in the experimental group, the standard deviation and coefficient of variation differed significantly, indicating the instability of manual scoring and its susceptibility to influence.

Figure 5 and Figure 6 show the distribution of scores for Drivers 1 and 6. In both figures, (a) are box plots and (b) are line graphs. From these figures, it can be seen that, compared to the control group, the score distribution in the experimental group was more concentrated. The score fluctuations for most drivers in the experimental group were smaller, while the score fluctuations in the control group were larger, demonstrating greater dispersion. Compared to the control group, although the final results for the novice drivers were the same, the evaluation method based on the reference model was more stable and objective than the traditional manual scoring method and better reflected the deficiencies of the novice drivers during the scoring process.

6. Discussion

With the improvement of automation levels in ports and the growth of cargo volumes, the requirements for drivers’ operational skills and safety awareness are also increasing. However, traditional training methods not only have high costs and risks but also exhibit problems of unstable training effects and insufficient coverage. This study innovates based on this and proposes an automated evaluation method utilizing virtual reality and the DQN model. This method not only effectively reduces training costs and risks but also improves drivers’ abilities to deal with complex environments and sudden events.

The experimental results of this study show that the automated evaluation method based on the DQN model has higher objectivity and consistency in evaluating operational risk behaviors. The standard deviation and coefficient of variation of the experimental group are significantly lower than those of the control group, indicating that the system can effectively reduce the fluctuations caused by subjective human judgments. This consistency is particularly important for port operations, as the operating environment in ports is complex and variable. A highly stable evaluation method can more accurately reflect drivers’ operational abilities, reduce accidents, and improve overall operational efficiency.

Although only 10 operators participated in our experiment, the scores presented are the averages of multiple evaluations, ensuring the validity of the experimental results for verifying this method. In the future, we will consider incorporating more safety risk factors related to quay crane operations and recruiting more participants to further enhance the accuracy of the risk behavior assessment for operators from multiple perspectives.

7. Conclusions

This paper first designs a risk simulation module based on the existing automated quay crane remote operation simulator. This module can simulate potential risks in various real operating scenarios, filling the gap in the current simulator’s capability to model driver operational risks. A DQN-based baseline model is then constructed to learn the operational methods and standards of skilled drivers. This model accurately reflects the operational behaviors and decision-making processes of experienced drivers. Based on this baseline model, an automated evaluation method for drivers’ risk behaviors is proposed. Novice drivers interact with the model to simulate various operational scenarios, allowing the system to monitor and assess their behaviors in real time and score their risk-related actions. This approach not only makes the evaluation process more objective and scientific but also enhances the drivers’ safety risk awareness, thereby reducing errors and accidents during actual operations.

Currently, this experiment has been authorized at only one terminal, resulting in a relatively small sample size limited to the drivers of a single port. Future research can expand to more ports and broader areas to validate the applicability of the model across different operational environments. Additionally, this study can be enhanced by incorporating more sensor data, such as drivers’ physiological states (e.g., fatigue levels and attention levels), to improve the accuracy of the evaluation from multiple dimensions. This will contribute to establishing a more comprehensive port safety evaluation system, providing new perspectives and practical guidelines for the development of the industry.