6-Dimensional Virtual Human-Machine Interaction Force Estimation Algorithm in Astronaut Virtual Training

Abstract

It is necessary to conduct virtual training for astronauts on the ground to improve the efficiency and safety of astronauts carrying objects in space. Cooperation between the two astronauts is required when handling massive objects. During this process, it is necessary to obtain the operating force of each astronaut. The research purpose of this paper was to propose an algorithm to map the astronaut’s operation on the VR handle to the human–machine interaction force without the robot’s participation, thereby saving costs. In this paper, a virtual robot simulation model is established, while the controller is designed based on the inverse system method. The virtual human–machine interaction force was obtained based on the inverse dynamics method. The influence of different parameters on the final position of the virtual object was analyzed. The physical engine was integrated into the virtual force sensor to ensure that the human–machine interaction forces of multiple astronauts can be coupled. The results showed that the virtual human–machine interaction force is similar to the real one and has a low output noise (approximately 5.5 N). This force can be applied to astronaut collaborative virtual training.

1. Introduction

When astronauts carry out missions in space, they cannot feel the existence of gravity, but the inertial force still exists. While astronauts carry large objects in space, it is difficult to ensure they reach the target point at zero speed, which poses certain potential safety hazards. Therefore, astronauts must receive simulation training on the ground [1,2].

A virtual astronaut operation training system is used in simulation training to reduce training costs and improve efficiency. In this system, in the single-astronaut task training option, the astronauts use a rope-driven robot with realistic force feedback to train to carry the objects in a virtual environment [3,4]. In the multi-astronaut cooperative operation training option, one of the astronauts operates the real robot. In contrast, the other astronauts use VR handles to simulate the process of carrying objects, reducing the cost, and improving the authenticity of the training. Therefore, the astronaut collaborative virtual training system requires a human–machine interaction algorithm for mapping the VR handle movement to the object motion.

At present, many virtual training systems support multi-person collaboration. The China Astronaut Research and Training Center [5] has developed a simulation system for astronaut space cooperative operation training based on virtual reality technology. Y. R. Zhang [6] et al. used data gloves and position trackers to achieve human–machine interaction (HMI) in the virtual training system. Leoncini P. [7] et al. describe devising a real-time full-body motion-capture system for a multi-user collaborative virtual environment (CVE), combining Kinect-based multi-body tracking with head-tracking and head-mounted leap motion controllers data to realize body-tracking enhanced with full hand detail. Desai K [8,9] et al. used low-cost cameras to capture and generate a 3D model of the trainee, allowing users to interact with the virtual objects in the scene.

Currently, multi-person collaborative virtual training systems mostly focus on solving the conflict problem in multi-person collaboration, that is, the ownership of objects. There are a few cases where more persons carry the same object [10]. In the HMI mode, the external posture is often used as the input to the virtual environment, whereas the human–machine interaction force (HMIF) is rarely calculated. This HMI mode is feasible when only one person carries an object. However, when many people carry the same object together, this prior art HMI mode can hardly produce good results. The virtual force sensor offers a good solution. The pose of the VR handle is used to estimate the force exerted by the astronaut on the virtual object.

Currently, virtual force sensors are mainly used to estimate robot joint moments [11,12,13,14] or contact forces [15,16,17] to reduce costs. The force or moment is obtained by establishing the dynamic model of the robot [18,19,20,21]. Haritha S. [22] et al. proposed a method to estimate force from SEMG signals using an artificial neural network. Muhammad A. [23] et al. designed a 3-DOF wrist force sensor for a haptic human–machine interface. Based on fingertip images, contact force estimation is also an estimation method for human–machine interaction [24,25,26]. The above methods are applicable under the premise that the human body does output the corresponding force and thus can be detected. In the collaborative virtual training of astronauts, those who operate the VR handles do not need to output the force that makes the virtual objects move accordingly. Based on the principle of real force sensors, a virtual force sensor, suitable for the virtual collaborative training of astronauts, is proposed in [10]. However, it is only applicable in the case of a single degree of freedom.

In this paper, a virtual robot simulation model was established. At the same time, the controller was designed based on the inverse system method, decoupling the robot dynamics. In contrast, the virtual human–machine interaction force (VHMIF) was obtained by the inverse dynamics method. The physical engine was integrated into the virtual force sensor so that the human–machine interaction forces of multiple astronauts can be coupled and influenced by the physical engine during collaborative training, thereby realizing the interaction between the astronauts operating the real robot and the VR handle. The paper is divided into the following sections. Section 2 presents the estimation method of VHMIF, including the establishment of the virtual robot, rigid body dynamics analysis in the space station, and inverse dynamics. Section 3 introduces experimental and simulation results. Section 4 discusses the results and compares them with other methods in the same section. Finally, Section 5 concludes the paper.

2. 6-Dimensional VHMIF Estimation Algorithm

The real HMIF is the interaction force between the end handle of the real robot and the human hand, so it is related to the control effect of the robot. In actual operation, there is always an error in the control of the robot, i.e., there is always a deviation between the position of the virtual object and the position of the end handle of the real robot. At the same time, because isomorphic robots are more suitable for dual-robot collaboration, virtual robot models can be used to simulate the motion errors of real robots, thus achieving pseudo-isomorphism. Furthermore, rigid body dynamics analysis of space station loads station is essential to obtain the virtual robot’s end effector’s command pose. After determining the pose of the virtual robot’s end effector and the command pose of the astronaut, the VHMIF can be estimated through the inverse dynamics method.

2.1. Virtual Robot Kinematics and Dynamics Model

Establishing a virtual robot model is necessary to make the virtual force sensor simulation more realistic.

A schematic diagram of the mechanism of the virtual robot (Figure 1) can lead to the pose of the end coordinates system {b}:

$${\mathit{T}}_{\mathrm{b}}{=}^{\left[s_1\right]d_1}{e}^{\left[s_2\right]d_2}{e}^{\left[s_3\right]d_3}{e}^{\left[s_4\right]{\theta }_1}{e}^{\left[s_5\right]{\theta }_2}{e}^{\left[s_6\right]{\theta }_3}{\mathit{M}}_{\mathrm{b}0},$$

(1)

where ${\mathit{M}}_{\mathrm{b}0}$ is the initial pose of the end coordinate system and ${\mathcal{S}}_i$ is the screw coordinate of joint i that can be expressed as:

$$s_1=(0,0,0,1,0,0),s_2=(0,0,0,0,1,0),s_3=(0,0,0,0,0,1),$$

$$s_4=(0,0,1,a_1,0,0),s_5=(0,1,0,a_2,0,0),s_6=(1,0,0,0,-a_2,-a_1).$$

According to [27] and by considering joint damping, the positive dynamic equation can be expressed as:

$$\mathit{M}\left(\mathit{\theta }\right)\ddot{\mathit{\theta }}=\mathit{\tau }-\left(\mathit{c}\left(\mathit{\theta },\dot{\mathit{\theta }}\right)+\mathit{g}\left(\mathit{\theta }\right)\right)-{\mathit{J}}^{\mathit{T}}{\mathit{𝓕}}_{\mathit{t}\mathit{i}\mathit{p}}-\mathit{B}\dot{\mathit{\theta }},$$

(2)

where $\mathit{\theta }$ is the joint angle matrix, $\mathit{\tau }$ is the driving torque, $\mathit{M}\left(\mathit{\theta }\right)$ is the mass matrix, $\mathit{c}\left(\mathit{\theta },\dot{\mathit{\theta }}\right)$ represents Coriolis force and centrifugal force, $\mathit{g}\left(\mathit{\theta }\right)$ indicates the moment of gravity, ${\mathit{𝓕}}_{\mathrm{tip}}$ indicates the external force screw, $\mathit{J}$ represents the Jacobian matrix of the forward kinematics, B is the damping coefficient matrix.

After obtaining $\ddot{\theta }$, numerical integration methods can be adopted to obtain $\mathit{\theta }$.

2.2. Virtual Robot Controller Design

The virtual robot dynamic system is multivariable and nonlinear, exhibiting a dynamic coupling relationship between the robot’s joints. Further study of the virtual force sensor shows that the dynamic coupling greatly impacts the control effect, so the robot dynamics model needs to be decoupled. In this paper, the inverse system method was used to decouple the dynamic model.

Considering the motors of each joint as a part of the robot, the input of the original virtual robot system is the voltage U of the motors of each joint. In contrast, the system’s output is the torque of the external environment, as exerted by the end link of the robot ${ℱ}_{\mathrm{tip}}$ and the angle of each joint. Since the input of the inverse system is the output of the original system, it is the angle of each joint and the torque applied by the end link of the robot to the external environment ${\mathit{𝓕}}_{\mathrm{tip}}$, which acts as the input of the inverse system.

The original system can prove to be reversible, as shown by the proof process in Appendix A. The original virtual robot system can be transformed into an integral decoupling system. According to Appendix A, the relative degree of the original system can be expressed as ${\alpha }_1={\alpha }_2={\alpha }_3={\alpha }_4={\alpha }_5={\alpha }_6=3$, ${\alpha }_7={\alpha }_8={\alpha }_9={\alpha }_{10}={\alpha }_{11}={\alpha }_{12}=1$. Therefore, the original system is divided into six independent third-order integral subsystems and six independent first-order integral subsystems. The third-order integral subsystem outputs the angle and position of each joint. In contrast, the first-order integral subsystem outputs the force screw exerted by the end link of the robot on the external environment ${\mathit{𝓕}}_{\mathrm{tip}}$. Therefore, the input of its inverse system is the third-order derivative of the given six joint angles and positions and the first-order derivative of the force screw exerted by the end link of the robot on the external environment. The output of the inverse system is each joint motor voltage, divided into two parts: ${\mathit{U}}_{\mathrm{d}1}$ and ${\mathit{U}}_{\mathrm{d}2}$. The expression of the inverse system is:

$$\{\begin{array}{ll}{\mathit{U}}_{\mathrm{d}1}& =\frac{\mathit{L}\mathit{a}}{\mathit{i}\mathit{C}\mathit{m}}\left(\mathit{M}\left(\mathit{\theta }\right)+{\mathit{J}}_{\mathrm{m}}{\mathit{i}}^2\right)\stackrel{⃛}{\mathit{\theta }}+\frac{1}{2}\mathit{C}\mathit{e}\mathit{i}\dot{\mathit{\theta }}+\\ & \frac{\mathit{R}\mathit{a}}{\mathit{i}\mathit{C}\mathit{m}}\left(\mathit{M}\left(\mathit{\theta }\right)\ddot{\mathit{\theta }}+\mathit{c}\left(\mathit{\theta },\dot{\mathit{\theta }}\right)+\mathit{g}\left(\mathit{\theta }\right)+\mathit{B}\dot{\mathit{\theta }}+{\mathit{J}}_{\mathrm{m}}{\mathit{i}}^2\ddot{\mathit{\theta }}+{\mathit{B}}_{\mathrm{m}}{\mathit{i}}^2\dot{\mathit{\theta }}\right)+\\ & \frac{\mathit{L}\mathit{a}}{\mathit{i}\mathit{C}\mathit{m}}\left(\mathit{𝕄}\left(\mathit{\theta },\dot{\mathit{\theta }}\right)\ddot{\mathit{\theta }}+ℂ\left(\mathit{\theta },\dot{\mathit{\theta }},\ddot{\mathit{\theta }}\right)+\mathbb{G}\left(\mathit{\theta },\dot{\mathit{\theta }}\right)+\mathit{B}\ddot{\mathit{\theta }}+{\mathit{B}}_{\mathrm{m}}{\mathit{i}}^2\ddot{\mathit{\theta }}\right)\\ {\mathit{U}}_{\mathrm{d}2}& =\frac{\mathit{L}\mathit{a}}{\mathit{i}\mathit{C}\mathit{m}}{\mathit{J}}^{\mathrm{T}}\left(\mathit{\theta }\right){\dot{\mathit{𝓕}}}_{\mathrm{tip}}+\frac{1}{2}\mathit{C}\mathit{e}\mathit{i}\dot{\mathit{\theta }}+\left(\frac{\mathit{R}\mathit{a}}{\mathit{i}\mathit{C}\mathit{m}}{\mathit{J}}^{\mathrm{T}}\left(\mathit{\theta }\right)+\frac{\mathit{L}\mathit{a}}{\mathit{i}\mathit{C}\mathit{m}}\mathbb{J}\left(\mathit{\theta },\dot{\mathit{\theta }}\right)\right){\mathit{𝓕}}_{\mathrm{tip}}\end{array},$$

(3)

where $\mathit{L}\mathit{a}$ is the motor inductance matrix, $\mathit{C}\mathit{m}$ is the motor torque constant matrix, ${\mathit{J}}_{\mathit{m}}$ is the motor moment of inertia matrix, $\mathit{i}$ is the transmission ratio matrix, $\mathit{C}\mathit{e}$ is the back electromotive force constant matrix of the motor, $\mathit{R}\mathit{a}$ is the motor armature resistance matrix, ${\mathit{B}}_{\mathrm{m}}$ is the damping coefficient matrix of the motor, $𝕄\left(\mathit{\theta },\dot{\mathit{\theta }}\right)$ is the first derivative of $\mathit{M}\left(\mathit{\theta }\right)$, and $ℂ\left(\mathit{\theta },\dot{\mathit{\theta }},\ddot{\mathit{\theta }}\right)$ is the first derivative of $\mathit{c}\left(\mathit{\theta },\dot{\mathit{\theta }}\right)$, $\mathbb{G}\left(\dot{\mathit{\theta }}\right)$ is the first derivative of $\mathit{g}\left(\mathit{\theta }\right)$. $\mathit{L}\mathit{a}$ is constructed as follows, and the rest of the matrices are constructed similarly.

$$\mathit{L}\mathit{a}=\left[\begin{array}{cccc}L{a}_1& 0& \cdots & 0\\ 0& L{a}_2& \cdots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \cdots & L{a}_6\end{array}\right]$$

(4)

where $L{a}_i$ is the inductance of the i-th joint motor.

According to Equation (3), the order of the original system is increased by one order as a whole due to the motor inductance $\mathit{L}\mathit{a}$.

However, in actual situations, for a single motor, the only acceptable input is ${\mathit{U}}_{\mathrm{d}\_1}\sim {\mathit{U}}_{\mathrm{d}\_6}$, where:

$$\begin{array}{l}{U}_{\mathrm{d}\_1}=U_{\mathrm{d}1\_1}+U_{\mathrm{d}2\_1}\\ ⋮\\ U_{\mathrm{d}\_6}=U_{\mathrm{d}1\_6}+U_{\mathrm{d}2\_6}\end{array}$$

(5)

In the analysis of this paper, ${\mathit{𝓕}}_{\mathrm{tip}}$ is regarded as the output of the original system. However, in the actual simulation model, ${\mathit{𝓕}}_{\mathrm{tip}}$ is the external load force of the environment acting on the robot, and it is a known quantity. In the case where the input of the original system is in the form of $U_{\mathrm{d}\_1}\sim U_{\mathrm{d}\_6}$, the simulation model will automatically calculate $U_{\mathrm{d}2\_1}\sim U_{\mathrm{d}2\_6}$ according to ${\mathit{𝓕}}_{\mathrm{tip}}$, thus calculating $U_{\mathrm{d}\_1}\sim U_{\mathrm{d}\_6}$ achieves decoupling.

The model is composed as follows by introducing the PD closed-loop controller. The motor voltage of each joint is:

$${\mathit{U}}_{\mathrm{d}}={\mathit{U}}_{\mathrm{d}1}\left({\stackrel{⃛}{\mathit{\theta }}}_{\mathrm{d}},\ddot{\mathit{\theta }},\dot{\mathit{\theta }},\mathit{\theta }\right)+{\mathit{U}}_{\mathrm{d}2}\left(\dot{\mathit{\theta }},\mathit{\theta },{\mathit{𝓕}}_{\mathrm{tip}},{\dot{\mathit{𝓕}}}_{\mathrm{tip}}\right)+\left(\mathit{M}\left(\mathit{\theta }\right)+{\mathit{J}}_{\mathrm{m}}\right)\left({\mathit{K}}_{\mathrm{p}}\left({\mathit{\theta }}_{\mathrm{d}}-\mathit{\theta }\right)+{\mathit{K}}_{\mathrm{d}}\left({\dot{\mathit{\theta }}}_{\mathrm{d}}-\dot{\mathit{\theta }}\right)\right)$$

(6)

where ${\theta }_{\mathrm{d}}$ is the angle/displacement command matrix, ${\mathit{K}}_{\mathrm{p}}$ is the proportional coefficient matrix of the PD controller, and ${\mathit{K}}_{\mathrm{d}}$ is the derivative coefficient matrix of the PD controller.

2.3. Rigid Body Dynamics Analysis of Loads in the Space Station

The coordinates system involved in this section is shown in Figure 2. Where {i} is the earth-equatorial coordinates system, {b} is the coordinates system of the space station body, {lc} is the mass center coordinates system of the load, {l} is the centroid coordinates system of the load, {p1} and {p2} are two proxy point coordinates systems.

According to a force analysis of the loads in the space station, the single rigid body dynamic equation is as follows:

$${\mathit{𝓕}}_{\mathrm{lc}}={\mathcal{G}}_{\mathrm{lc}}{\dot{\mathcal{V}}}_{\mathrm{lc}}-{\left[{\mathrm{ad}}_{{\mathcal{V}}_{\mathrm{lc}}}\right]}^{\mathrm{T}}{\mathcal{G}}_{\mathrm{lc}}{\mathcal{V}}_{\mathrm{lc}}$$

(7)

where ${\mathit{𝓕}}_{\mathrm{lc}}=\left[\begin{array}{c}{\mathit{m}}_{\mathrm{lc}}\\ {\mathit{f}}_{\mathrm{lc}}\end{array}\right]$ is the force screw of the object acting on the load, ${\mathcal{G}}_{\mathrm{lc}}=\left[\begin{array}{cc}{ℐ}_{\mathrm{lc}}& 0\\ 0& m{\mathit{I}}_{\left(3\times 3\right)}\end{array}\right]$ is the space inertia matrix, m is the mass of the load, and ${\mathcal{V}}_{\mathrm{lc}}$ is the motion screw of the object at the center of the mass coordinates system of the load.

The total object force screw is divided into two parts, that is, the object force screw ${\mathit{𝓕}}_{\mathrm{ni}}$ produced by the non-inertial system and the object force screw ${\mathit{𝓕}}_{\mathrm{lp}}$ exerted by the astronauts:

$${\mathit{𝓕}}_{\mathrm{lc}}={\mathit{𝓕}}_{\mathrm{ni}}+{\mathit{𝓕}}_{\mathrm{lp}}$$

(8)

The parameter ${\mathit{𝓕}}_{\mathrm{lp}}$ can be directly obtained by the six-dimensional force sensor and converted to the force screw of the object, expressed in the coordinates system of the load center of mass by the following formula.

The screw force exerted by the i-th astronaut on the load is denoted as ${\mathit{𝓕}}_{\mathrm{p}i}$ in the corresponding proxy point coordinates system, while the pose of the load’s centroid coordinates system {lc} in the i-th proxy point coordinates system is denoted as ${\mathit{T}}_{\mathrm{p}i\mathrm{lc}}$; so the total body force screw exerted by the i-th astronaut on the payload is:

$${\mathit{𝓕}}_{\mathrm{lp}}={\displaystyle \sum _{i=1}^n\left({\left[{\mathrm{Ad}}_{{\mathit{T}}_{\mathrm{p}i\mathrm{lc}}}\right]}^{\mathrm{T}}{\mathit{𝓕}}_{\mathrm{p}i}\right)}$$

(9)

Substituting the force screw ${\mathit{𝓕}}_{\mathrm{lc}}$ into the rigid body dynamics relation, as expressed in Equation (7), can obtain ${\dot{\mathcal{V}}}_{\mathrm{lc}}$ and then integrate to obtain ${\mathcal{V}}_{\mathrm{lc}}$.

Moreover, ${\mathcal{V}}_{\mathrm{lc}}$ is regularized to obtain the screw axis $\mathcal{S}$, whereas ${\mathcal{V}}_{\mathrm{lc}}=\left[\begin{array}{c}{\mathit{\omega }}_{\mathrm{lc}}\\ {\mathit{v}}_{\mathrm{lc}}\end{array}\right]$. If $\Vert {\omega }_{\mathrm{lc}}\Vert \ne 0$, then $\mathcal{S}=\frac{{\mathcal{V}}_{\mathrm{lc}}}{\Vert {\mathit{\omega }}_{\mathrm{lc}}\Vert }$, the angular velocity $\dot{\theta }=\Vert {\omega }_{\mathrm{lc}}\Vert$ of the screw axis; if $\Vert {\omega }_{\mathrm{lc}}\Vert =0$, then $\mathcal{S}=\frac{{\mathcal{V}}_{\mathrm{lc}}}{\Vert {\mathit{v}}_{\mathrm{lc}}\Vert }$, the angular velocity of the screw axis $\dot{\theta }=\Vert {\mathit{v}}_{\mathrm{lc}}\Vert$. Therefore, the pose of the center of mass coordinates system of the load at the time i in the center of mass coordinates system of the load at a time (i − 1) is: ${\mathit{T}}_{i-1,i}=e^{\left[\mathcal{S}\right]\left(\dot{\theta }\mathsf{\Delta }t\right)}\in \mathrm{SE}\left(3\right)$. Here, $\Delta t$ is the sampling time of the system.

Therefore, at time t, the pose of the payload’s centroid coordinates system {lc} in the coordinates system of the space station body {b} can be expressed as follows, noting that the multiplication in the formula is right multiplication.

$${\mathit{T}}_{\mathrm{blc}}\left(t\right)={\mathit{T}}_{\mathrm{blc}0}{\displaystyle \prod _{i=1}^{t/\Delta t}{\mathit{T}}_{i-1,i}}$$

(10)

where ${\mathit{T}}_{\mathrm{blc}0}$ is the initial pose.

2.4. Design of Virtual Force Sensor

The robot coordinates system is shown in Figure 3. Suppose the pose of the load center of mass coordinates system in the coordinate system of the space station body {b} is ${\mathit{T}}_{\mathrm{blc}}$, while the poses of the agent point coordinates system {p1} and {p2} in the load center of mass coordinates system {lc} are ${\mathit{T}}_{\mathrm{lcp}1}$ and ${\mathit{T}}_{\mathrm{lcp}2}$, respectively. Therefore, the poses of the agent point coordinates system {p1} and {p2} in the coordinate system of the space station body {b} are: ${\mathit{T}}_{\mathrm{bp}1}={\mathit{T}}_{\mathrm{blc}}{\mathit{T}}_{\mathrm{lcp}1}$ and ${\mathit{T}}_{\mathrm{bp}2}={\mathit{T}}_{\mathrm{blc}}{\mathit{T}}_{\mathrm{lcp}2}$, respectively. In general, there will be a certain difference between the position of the two agent points and the position of the robot’s end effector because the position of the agent point is considered the position of the force sensor. On the other hand, the position of the force sensor and the position of the robot’s end effector are not necessarily the same. Assuming that the poses of the two force sensors in the end effector coordinates systems of the two robots are $T_{\mathrm{E}1\mathrm{p}1}$ and $T_{\mathrm{E}2\mathrm{p}2}$, the poses of the end effector of the robot in the body coordinates system of the space station are ${\mathit{T}}_{\mathrm{bE}1}={\mathit{T}}_{\mathrm{bp}1}{\mathit{T}}_{\mathrm{E}1\mathrm{p}1}^{-1}$ and ${\mathit{T}}_{\mathrm{bE}2}={\mathit{T}}_{\mathrm{bp}2}{\mathit{T}}_{\mathrm{E}2\mathrm{p}2}^{-1}$.

Furthermore, if the poses of the base coordinates systems {s1} and {s2} of the two robots in the body coordinates system {b} are: ${\mathit{T}}_{\mathrm{bs}1}$ and ${\mathit{T}}_{\mathrm{bs}2}$, respectively, they are also fixed values in a training task. Then, the pose of the coordinate system of the robot end effector in the corresponding robot base coordinates system {s} is:

$$\begin{array}{l}{\mathit{T}}_{\mathrm{s}1\mathrm{E}1}={\mathit{T}}_{\mathrm{bs}1}^{-1}{\mathit{T}}_{\mathrm{bE}1}={\mathit{T}}_{\mathrm{bs}1}^{-1}{\mathit{T}}_{\mathrm{blc}}{\mathit{T}}_{\mathrm{lcp}1}{\mathit{T}}_{\mathrm{E}1\mathrm{p}1}^{-1}\\ {\mathit{T}}_{\mathrm{s}2\mathrm{E}2}={\mathit{T}}_{\mathrm{bs}2}^{-1}{\mathit{T}}_{\mathrm{bE}2}={\mathit{T}}_{\mathrm{bs}2}^{-1}{\mathit{T}}_{\mathrm{blc}}{\mathit{T}}_{\mathrm{lcp}2}{\mathit{T}}_{\mathrm{E}2\mathrm{p}2}^{-1}\end{array}$$

(11)

This pose is needed to solve the robot’s inverse kinematics solution and motion control.

Regarding the virtual force sensor and virtual robot, to simplify the calculation, it is considered that the handle coordinates system {h}, the agent point coordinates system {p}, and the coordinates system of the robot end effector {E} are the same, namely, ${\mathit{T}}_{\mathrm{Ep}}={\mathit{E}}_{\left(4\times 4\right)}$. It should be noted that, at this time, the proxy point coordinates system {p} is not the same as the one used as the control command, but the proxy point coordinates system feedback after the control of the virtual robot; that is, the feedback value of the coordinates system of the robot end effector. In other words, the purpose of virtual robot control is to enable the virtual coordinates system of the robot end effector {E} and the agent point coordinates system {p} to follow the command values of the handle coordinates system {h} and the agent coordinates system {p}. The purpose of the virtual force sensor is to calculate the force screw that needs to be applied to the agent point coordinates system {p}, during this process. The proxy point coordinates system {p} is the medium connecting the virtual load and the virtual robot.

At time t, the pose of the handle coordinates system in the base coordinates system of the virtual robot is denoted as ${\mathit{T}}_{\mathrm{sh}t}$, while the pose of the virtual robot’s end effector coordinates system {E} in the virtual robot’s base coordinates system is denoted as ${\mathit{T}}_{\mathrm{sE}t}$. Then, the pose of the handle coordinates system in the end effector coordinates system of the virtual robot is: ${\mathit{T}}_{\mathrm{Eh}t}={\mathit{T}}_{\mathrm{sE}t}^{-1}{\mathit{T}}_{\mathrm{sh}t}={\mathit{T}}_{\mathrm{lcE}t}^{-1}{\mathit{T}}_{\mathrm{lch}t}$.In the virtual robot, the coordinates system of the agent point and the coordinates system of the robot end effector are considered to be the same coordinates system, so the pose of the handle coordinates system in the coordinates system of the agent point {p} is actually: ${\mathit{T}}_{\mathrm{ph}t}={\mathit{T}}_{\mathrm{Eh}t}$. Specifically, to enable the coordinates system of the agent point to follow the coordinates system of the handle, it is necessary to move the coordinates system of the handle from the pose ${\mathit{E}}_{\left(4\times 4\right)}$ that coincides with the coordinates system of the agent point to ${\mathit{T}}_{\mathrm{ph}t}$, within a sampling period Δt, while ${\mathit{T}}_{\mathrm{ph}t}=\left[\begin{array}{cc}\mathit{R}& \mathit{p}\\ 0& 1\end{array}\right]=e^{\left[\mathcal{S}\right]\left(\dot{\theta }\Delta t\right)}$, where $\mathcal{S}=\left(\omega ,\mathit{v}\right)=\left({\omega }_1,{\omega }_2,{\omega }_3,v_1,v_2,v_3\right)$. The next step is to solve for the value of $\mathcal{S}$.

If $\mathit{R}={\mathit{E}}_{\left(3\times 3\right)}$, then $\omega =0$, $\mathit{v}=\frac{\mathit{p}}{\Vert \mathit{p}\Vert }$, $\theta =\dot{\theta }\Delta t=\Vert \mathit{p}\Vert$. If $\mathit{R}\ne {\mathit{E}}_{\left(3\times 3\right)}$, first $\omega$ and $\theta$ are calculated according to the matrix logarithm of $\mathrm{SO}\left(3\right)$, followed by the calculation: $\mathit{v}=\left(\frac{1}{\theta }\mathit{E}-\frac{1}{2}\left[\omega \right]+\left(\frac{1}{\theta }-\frac{1}{2}\cot \left(\frac{\theta }{2}\right)\right){\left[\omega \right]}^2\right)\mathit{p}$.

The procedure for solving $\omega$ and $\theta$ is as follows:

If the trace of the rotation matrix R $\mathrm{Trace}\left(\mathit{R}\right)=-1$, then $\theta =\pi$, while $\omega$ can take any value of the following three cases:

$\omega =\frac{1}{\sqrt{2\left(1+R_{33}\right)}}\left[\begin{array}{c}{R}_{13}\\ R_{23}\\ 1+R_{33}\end{array}\right]$ or $\omega =\frac{1}{\sqrt{2\left(1+R_{22}\right)}}\left[\begin{array}{c}{R}_{12}\\ 1+R_{22}\\ R_{32}\end{array}\right]$ or $\omega =\frac{1}{\sqrt{2\left(1+R_{11}\right)}}\left[\begin{array}{c}1+R_{11}\\ R_{21}\\ R_{31}\end{array}\right]$

Otherwise, $\theta =\mathrm{acos}\left(\frac{1}{2}\left(\mathrm{Trace}\left(\mathit{R}\right)-1\right)\right)\in \left[0,\pi \right)$, $\left[\omega \right]=\frac{1}{2\sin \left(\theta \right)}\left(\mathit{R}-{\mathit{R}}^{\mathrm{T}}\right)$.

Therefore, the motion screw from ${\mathit{E}}_{\left(4\times 4\right)}$ to ${\mathit{T}}_{\mathrm{ph}t}$ is expressed in the proxy point coordinates system as ${\mathcal{V}}_t=\mathcal{S}\dot{\theta }$. More specifically, within time $\Delta t$, the average value of the object motion screw in the agent point coordinates system needs to reach ${\mathcal{V}}_t$, so that the agent point coordinates system can follow the handle coordinates system. In this paper, considering that the object motion screw based on the object coordinates system does not depend on the fixed coordinates system selection, the virtual robot’s base coordinates system is selected as the fixed coordinates system. At the end of the time step $\Delta t$, the object motion screw in the agent point coordinates system or the coordinate system of the virtual robot end effector is: $\left[{\mathcal{V}}_t^{\prime }\right]={\mathit{T}}_{\mathrm{sE}t}^{-1}{\dot{\mathit{T}}}_{\mathrm{sE}t}$. Therefore, the acceleration is ${\dot{\mathcal{V}}}_t=2\frac{{\mathcal{V}}_t-{\mathcal{V}}_t^{\prime }}{\Delta t}$. The space inertia matrix expressed in the proxy point coordinates system is ${\mathcal{G}}_{\mathrm{p}}={[{\mathrm{Ad}}_{{\mathit{T}}_{\mathrm{lcp}}}]}^{\mathrm{T}}{\mathcal{G}}_{\mathrm{lc}}\left[{\mathrm{Ad}}_{{\mathit{T}}_{\mathrm{lcp}}}\right]$. Substituting ${\mathcal{G}}_{\mathrm{p}}$, ${\dot{\mathcal{V}}}_t$ and ${\mathcal{V}}_t^{\prime }$ into the rigid body dynamics equation can obtain the force screw $\mathit{𝓕}$.

The block diagram of the system in this process is shown in Figure 4. Among them, $M_{\mathrm{h}}$ is the pose of the handle coordinates system given by the VR handle, ${\mathit{𝓕}}_{\mathrm{ni}}$ is the non-inertial force screw and ${\mathit{𝓕}}_{\mathrm{p}2}$ is the human-computer interaction force of the astronaut operating the real robot.

Theoretically, the $\Delta t_1$ and $\Delta t_1$ of the virtual force sensor component in Figure 4 should both be the sampling period $\Delta t$. Parameters $\Delta t_1$ and $\Delta t_2$ can be set at different values to improve the performance of the control system. At the same time, a filter link is connected in series to the output of the virtual force sensor to attenuate the medium and high-frequency signals and limit the output. A differential link is connected in parallel with the virtual robot’s terminal pose output to improve the system’s stability. To select the appropriate time constant $\Delta t_1$, $\Delta t_2$, filter parameter $t_3$, and differential constant $t_4$, the motion of one of the robot’s degrees of freedom is first selected for analysis.

Since the robot controller adopts the inverse system method, it is theoretically possible to cancel the transfer functions of the robot controller and the virtual robot. However, there is still a difference between the transfer function of the robot controller and the virtual robot. Especially after the PD controller is introduced, the two cannot cancel each other out.

The expression of the PD controller is:

$$U_{\mathrm{d}3}\left(t\right)=\left(\tilde{J}e\left(\mathit{\theta }\right)Kp+\tilde{J}e\left(\mathit{\theta }\right)Kds\right)\left({\theta }_d\left(s\right)-\theta \left(s\right)\right)$$

(12)

Among them, $\tilde{J}e\left(\mathit{\theta }\right)$ is the estimated value of the equivalent moment of inertia of the joint, which is only related to the joint angle behind the current joint but not to the current joint itself. In the case of only one single degree of freedom being considered, it can be treated as a constant. Considering that, $\tilde{J}e\left(\mathit{\theta }\right)Kp=Kp\left(\mathit{\theta }\right)$, $\tilde{J}e\left(\mathit{\theta }\right)Kd=Kd\left(\mathit{\theta }\right)$, the block diagram of the system function of a single joint is shown in Figure 5.

The input of the original system $U_{\mathrm{d}}\left(s\right)$ can be expressed as:

$$\begin{array}{ll}{U}_{\mathrm{d}}\left(s\right)& =\frac{\tilde{L}a}{i\tilde{C}m}\left(i^2\tilde{J}e\left(\mathit{\theta }\right){\theta }_{\mathrm{d}}{s}^3+{\tilde{\tau }}_{\mathrm{Load}}\left(\mathit{\theta },\dot{\mathit{\theta }},\ddot{\mathit{\theta }}\right)s+i^2\tilde{B}e\theta s^2\right)+\\ & \frac{\tilde{R}a}{i\tilde{C}m}\left(i^2\tilde{J}e\left(\mathit{\theta }\right)\theta s^2+{\tilde{\tau }}_{\mathrm{Load}}\left(\mathit{\theta },\dot{\mathit{\theta }},\ddot{\mathit{\theta }}\right)+i^2\tilde{B}e\theta s\right)+\\ & \tilde{C}ei\theta s+Kp\left(\mathit{\theta }\right)\left({\theta }_{\mathrm{d}}-\theta \right)+Kd\left(\mathit{\theta }\right)\left({\theta }_{\mathrm{d}}-\theta \right)s\end{array}$$

(13)

where ${\tilde{\tau }}_{\mathrm{Load}}\left(\mathit{\theta },\dot{\mathit{\theta }},\ddot{\mathit{\theta }}\right)=\tilde{c}\left(\mathit{\theta },\dot{\mathit{\theta }}\right)+\tilde{g}\left(\mathit{\theta }\right)+{\tilde{\tau }}_{\mathrm{F}}\left(\mathit{\theta }\right)+\tilde{m}\left(\ddot{\mathit{\theta }}\right)$. This variable represents the influence of other joints on this joint. When considering a single degree of freedom, the following applies: $\dot{\theta }=\ddot{\theta }=0$.

In Equation (13), the symbol ~ above a variable indicates that the variable is an estimated value. The parameter $\tilde{J}e\left(\mathit{\theta }\right)$ is the equivalent moment of inertia equal to $\tilde{J}e\left(\mathit{\theta }\right)={\tilde{J}}_{\mathrm{m}}+\frac{1}{i^2}\tilde{J}\left(\mathit{\theta }\right)$ and $\tilde{B}e$ is the equivalent damping coefficient equal to $\tilde{B}e={\tilde{B}}_{\mathrm{m}}+\frac{1}{i^2}B$.

The output of the original system is:

$$\begin{array}{cc}\theta \left(s\right)& =\frac{\frac{Cm}{\tilde{C}m}\left(\tilde{L}as+\tilde{R}a\right){\tilde{\tau }}_{\mathrm{Load}}\left(\mathit{\theta },\dot{\mathit{\theta }},\ddot{\mathit{\theta }}\right)-\left(Las+Ra\right){\tau }_{\mathrm{Load}}\left(\mathit{\theta },\dot{\mathit{\theta }},\ddot{\mathit{\theta }}\right)}{s\left(Je\left(\mathit{\theta }\right)s+Be\right)\left(Las+Ra\right)i^2}+\\ & \frac{\frac{Cm}{\tilde{C}m}\left(\tilde{L}a\left(\tilde{J}e\left(\mathit{\theta }\right){\theta }_{\mathrm{d}}{s}^3+\tilde{B}e\theta s^2\right)+\tilde{R}as\left(\tilde{J}e\left(\mathit{\theta }\right)\theta s+\tilde{B}e\theta \right)\right)}{s\left(Je\left(\mathit{\theta }\right)s+Be\right)\left(Las+Ra\right)}+\\ & \frac{Cm\left(\tilde{C}e-Ce\right)\theta }{\left(Je\left(\mathit{\theta }\right)s+Be\right)\left(Las+Ra\right)}+\frac{Cm\left(Kp\left(\mathit{\theta }\right)\left({\theta }_{\mathrm{d}}-\theta \right)+Kd\left(\mathit{\theta }\right)\left({\dot{\theta }}_{\mathrm{d}}-\dot{\theta }\right)\right)}{s\left(Je\left(\mathit{\theta }\right)s+Be\right)\left(Las+Ra\right)i}\end{array}$$

(14)

When all estimates are equal to actual values, $\theta \left(s\right)={\theta }_{\mathrm{d}}\left(s\right)$, the PD controller does not work. Therefore:

$$\theta =\frac{La\left(Je\left(\theta \right){\theta }_{\mathrm{d}}{s}^3+Be\theta s^2\right)+Ras\left(Je\left(\theta \right)\theta s+Be\theta \right)}{s\left(Je\left(\theta \right)s+Be\right)\left(Las+Ra\right)}={\theta }_{\mathrm{d}}$$

(15)

The above relation proves to be always established. The validity of the design of the inverse system can be verified.

In case the estimated value is not equal to the actual value, to obtain the order of the transfer function, the external load moment of the motor is not considered, or it is considered as such: $\frac{Cm}{\tilde{C}m}\left(\tilde{L}as+\tilde{R}a\right){\tilde{\tau }}_{\mathrm{Load}}\left(\theta ,\dot{\theta },\ddot{\mathit{\theta }}\right)-\left(Las+Ra\right){\tau }_{\mathrm{Load}}\left(\theta ,\dot{\theta },\ddot{\mathit{\theta }}\right)=0$. Hence, Equation (14) can be written in the following form:

$$G_1\left(s\right)=\frac{\theta \left(s\right)}{{\theta }_{\mathrm{d}}\left(s\right)}=\frac{b_0\left(\theta \right)s^3+b_1\left(\theta \right)s^2+b_2\left(\theta \right)s+b_3\left(\theta \right)}{a_0\left(\theta \right)s^3+a_1\left(\theta \right)s^2+a_2\left(\theta \right)s+a_3\left(\theta \right)},$$

(16)

where

$$\{\begin{array}{l}{a}_0\left(\theta \right)=\tilde{C}miLaJe\left(\theta \right)\\ a_1\left(\theta \right)=\tilde{C}miJe\left(\theta \right)Ra-Cmi\tilde{J}e\left(\theta \right)\tilde{R}a+\tilde{C}miBeLa-Cmi\tilde{L}a\tilde{B}e\\ a_2\left(\theta \right)=\tilde{C}miBeRa-Cmi\tilde{B}e\tilde{R}a-Cm\tilde{C}m\left(\tilde{C}e-Ce\right)+Cm\tilde{C}mKd\left(\theta \right)\\ a_3\left(\theta \right)=Cm\tilde{C}mKp\left(\theta \right)\\ b_0\left(\theta \right)=Cmi\tilde{L}a\tilde{J}e\left(\theta \right)\\ b_1\left(\theta \right)=0\\ b_2\left(\theta \right)=Cm\tilde{C}mKd\left(\theta \right)\\ b_3\left(\theta \right)=a_3\left(\theta \right)\end{array}$$

The transfer function shows that the following is true $\theta \left(s\right)={\theta }_{\mathrm{d}}\left(s\right)$ when all the estimated values are equal to the actual values.

The single joint model of the virtual force sensor is shown in Figure 6.

3. Results

The proposed algorithm was simulated and verified on a single person carrying an object and two people cooperating to carry an object. Furthermore, two virtual robot models were established, and the proposed algorithm was experimentally validated via a two-person antagonism.

3.1. Simulation Results When a Single Person Carries Objects

The controller parameters are set as: $\Delta t_1=0.042$, $\Delta t_2=15$, $t_3=0.1215$, $t_4=0.15$, while MATLAB is used to establish the simulation model, and the system sampling period is set to 0.001 s.

The pose $M_{\mathrm{h}}$ of the VR handle is given by the position $\left(x_{\mathrm{d}},y_{\mathrm{d}},z_{\mathrm{d}}\right)$, and the Euler angle $\left({\varphi }_{\mathrm{d}},{\theta }_{\mathrm{d}},{\psi }_{\mathrm{d}}\right)$ of the VR handle. The given signals are all sine waves, and the position $\left(x,y,z\right)$ and Euler angle $\left(\varphi ,\theta ,\psi \right)$ of the end effector of the robot are shown in Figure 7.

Figure 7 shows that the robot’s end effector can better follow the given position of the VR handle while the system is in a stable state.

The output of the virtual force sensor ${\mathit{𝓕}}_{\mathrm{p}1}=\left(M_{\mathrm{x}},M_{\mathrm{y}},M_{\mathrm{z}},F_{\mathrm{x}},F_{\mathrm{y}},F_{\mathrm{z}}\right)$ during this process is shown in Figure 8.

Figure 8 illustrates that the output of the virtual force sensor will oscillate during the first period and enter a stable state after a single cycle due to the sudden change of the force screw at the initial moment.

3.2. Simulation Results When Two People Cooperate in Carrying Objects

The virtual force sensor proposed in this paper is mainly used in virtual training when two people cooperate in carrying objects. Matlab is used to establish a simulation model of two people cooperatively carrying objects to verify the effect of the virtual force sensor when two people cooperate, as shown in Figure 9.

The pose output of the virtual object considering two sets of the same sine wave signals as the pose input is shown in Figure 10. Compared to the case of a single-person handling objects, the overshoot of the pose-following output is slightly larger in the case of two people carrying objects together. However, the system is still in a stable state, which is consistent with the daily life experience. Compared to single-person handling, the accuracy of two people cooperating to carry objects is slightly worse and easy to overshoot.

When two people carry objects, the output of the virtual force sensor 1 ${\mathit{𝓕}}_{\mathrm{p}1}=\left(M_{\mathrm{x}},M_{\mathrm{y}},M_{\mathrm{z}},F_{\mathrm{x}},F_{\mathrm{y}},F_{\mathrm{z}}\right)$ is shown in Figure 11, i.e., the output is exactly half of that of a single person carrying objects. This is consistent with the common daily life experience where, compared to single-person handling objects, it is more effortless when two people cooperate in carrying objects.

The above simulations are based on the assumption that the given positions of the two astronauts are the same; that is, the two astronauts have the same goal. In actual situations, the goals of the two astronauts are not necessarily the same, and they may even be in a state of mutual antagonism. Next, the simulation of two astronauts cooperatively carrying objects is carried out according to the following scenarios and conditions.

(1)

At the initial moment $\left(x_0,y_0,z_0\right)=\left(0,0,0\right)$, the initial Euler angles of the virtual object are: $\left({\varphi }_0,{\theta }_0,{\psi }_0\right)=\left(0,0,0\right)$;

(2)

The goal of Astronaut 1 is to transport the virtual object to the pose corresponding to $\left(x_{\mathrm{d}1},y_{\mathrm{d}1},z_{\mathrm{d}1}\right)=\left(0.4,0.4,0.4\right)$ and $\left({\varphi }_{\mathrm{d}1},{\theta }_{\mathrm{d}1},{\psi }_{\mathrm{d}1}\right)=\left(0.4,0.4,0.4\right)$. The goal of Astronaut 2 is to keep the pose of the virtual object constant.

(3)

Astronaut 1’s minimum and maximum output force screw, that is, the output limit is: (±60 Nm, ±60 Nm, ±60 Nm, ±400 N, ±400 N, ±400 N), Astronaut 2’s minimum and maximum output force screw, that is, the output limit is: (±30 Nm, ±30 Nm, ±30 Nm, ±200 N, ±200 N, ±200 N).

The simulation results are shown in Figure 12, where $\left(x,y,z\right)$ and $\left(\varphi ,\theta ,\psi \right)$ represent the position and Euler angle of the virtual object, respectively.

According to Figure 12, the final position of the virtual object is neither the target position of astronaut 1 nor the target position of astronaut 2, but a position somewhere in between. Moreover, the final position of the virtual object appears closer to the target position because astronaut 1’s maximum output force of astronaut 1 is larger. Based on Figure 12, under the same controller parameters, the system’s instability appears to increase in the state of mutual antagonism compared to the single-person handling. The output graphs of the virtual force sensor 1 and virtual force sensor 2 (Figure 13) show that, after a period of oscillation, the output of virtual force sensor 1 is in a relatively stable state, while the output of the virtual force sensor 2 is in a saturated state. After stabilization, the output of virtual force sensor 2 is in a saturated state. In contrast, the output of virtual force sensor 1 is close to the negative value of the output of the virtual force sensor 2, which, coupled with the object force screw generated by the non-inertial system, sums up approximately to zero. The virtual object is finally in a relatively static state, but there will still be slight oscillations with continuously declining amplitudes.

To verify the influence of $\Delta t_1$ and $\Delta t_2$, first $\Delta t_2=15$ is set, while $\Delta t_1$ is set to different values, producing simulation results in the x direction, as shown in Figure 14. It is evident that the smaller $\Delta t_1$ is, the closer the final position $x$ of the virtual object is to $x_{\mathrm{d}1}$. Moreover, $\Delta t_1=0.042$ is set, while $\Delta t_2$ is set to different values, producing simulation results in the x direction as shown in Figure 15. Evidently, the smaller $\Delta t_2$ is, the closer the final position of the virtual object $x$ is to $x_{\mathrm{d}1}$. For simplicity, the results of other directions are not shown, but they follow similar rules.

3.3. Experimental Results

Due to limiting conditions, this paper does not conduct a collaborative experiment between real and virtual robots. Instead, two virtual robots are used for experiments. The virtual robot runs on the dSpace semi-physical simulation platform, a real-time system that can ensure the real-time operation of the virtual robot. Pose settings are provided by the VR handle of HTC VIVE pro. The VR handle’s pose is obtained through Unity on the PC’s Windows system. In contrast, the pose of the two handles is sent, through RS422 communication, to the dSpace semi-physical simulation platform, where the pose of the virtual object and the VHMIF are calculated.

In this paper, two experiment sets are set up, and the experimental conditions are shown in

Table 1. Experimental setup.

No.	Tester 1	Tester 2	$\Delta {{\mathit{t}}_1}^{ }$	$\Delta {\mathit{t}}_2$
1st	Move 1	Still 2	0.02	0.5
2nd	Still	Move	0.02	0.5

¹ “Move” means that the tester initially moves and freely rotates the VR handle in space but remains stationary at the end of the experiment; ² “Still” means the tester maintains the VR handle static during the test.

. Furthermore, tester 1’s minimum and maximum output screw forces are (±60 Nm, ±60 Nm, ±60 Nm, ±400 N, ±400 N, ±400 N), and tester 2’s minimum and maximum output screw forces are (±30 Nm, ±30 Nm, ±30 Nm, ±200 N, ±200 N, ±200 N).

The results of the first experiment are shown in Figure 16, where $x_{\mathrm{d}1}$, $y_{\mathrm{d}1}$, $z_{\mathrm{d}1}$, ${\varphi }_{\mathrm{d}1}$, ${\theta }_{\mathrm{d}1}$, and ${\psi }_{\mathrm{d}1}$ represent the pose of tester 1′s VR handle. Parameters $x_{\mathrm{d}2}$, $y_{\mathrm{d}2}$, $z_{\mathrm{d}2}$, ${\varphi }_{\mathrm{d}2}$, ${\theta }_{\mathrm{d}2}$, and ${\psi }_{\mathrm{d}2}$ represent the pose of the tester 2′s VR handle. Lastly, parameters $x$, $y$, $z$, $\varphi$, $\theta$, and $\psi$ represent the pose of the virtual object. According to Figure 16, the final pose of the virtual object is closer to the pose of tester 1’s VR handle.

The VHMIF, as measured by the virtual force sensor during this process, is shown in Figure 17, where ${\mathit{𝓕}}_{\mathrm{p}1}=\left(M_{\mathrm{x}1},M_{\mathrm{y}1},M_{\mathrm{z}1},F_{\mathrm{x}1},F_{\mathrm{y}1},F_{\mathrm{z}1}\right)$ represents the output of the virtual force sensor 1 and ${\mathit{𝓕}}_{\mathrm{p}2}=\left(M_{\mathrm{x}2},M_{\mathrm{y}2},M_{\mathrm{z}2},F_{\mathrm{x}2},F_{\mathrm{y}2},F_{\mathrm{z}2}\right)$ represents the output of the virtual force sensor 2. According to Figure 17, the virtual force sensor 2 is finally saturated.

The results of the second experiment are shown in Figure 18. Variables in Figure 18 are the same as those in Figure 16. According to Figure 18, the final pose of the virtual object is closer to the pose of the tester 1’s VR handle.

The VHMIF, as measured by the virtual force sensor, during this process, is shown in Figure 19. Variables in Figure 19 are the same as those in Figure 17. According to Figure 19, the virtual force sensor 2 is finally in a saturated state.

4. Discussion

According to Figure 14 and Figure 15, the smaller $\Delta t_1$ and $\Delta t_2$ are, the closer the final position of the virtual object $x$ is to $x_{\mathrm{d}1}$. The final position of the virtual object is determined by the astronaut whose maximum output force spin is the highest. According to the scenario set in this paper, the final position of the virtual object is determined by astronaut 1. According to Figure 6, the output of the virtual force sensor in the case of a single joint can be expressed as:

$$F_1=M\frac{\frac{x_{\mathrm{d}1}-x_1}{\Delta t_1}-{\dot{x}}_1}{2\Delta t_2},$$

(17)

where $M$ is the mass of the virtual object and $x_1$ is the actual position of virtual robot 1.

Therefore,

$$x_{\mathrm{d}1}-x_1=\left(\frac{2F_1\Delta t_2}{M}+{\dot{x}}_1\right)\Delta t_1$$

(18)

In the case where the control effect of the robot controller is good, $\underset{t\to \infty }{\lim }{x}_1\left(t\right)=x$, where x is the position of the virtual object, $\underset{t\to \infty }{\lim }{\dot{x}}_1\left(t\right)=0$ and $\left|\underset{t\to \infty }{\lim }{F}_1\left(t\right)\right|=F_{\min }$. Therefore:

$$x_{\mathrm{d}1}-x=\frac{2F_{\min }\Delta t_1\Delta t_2}{M}$$

(19)

The above formula shows that the smaller $\Delta t_1\Delta t_2$ is, the smaller $F_{\min }$ is. In contrast, the larger M is, the smaller $\left|x_{\mathrm{d}1}-x\right|$ is, and the deviation between the final position of the virtual object $x$ and the given position $x_{\mathrm{d}1}$ is smaller. Setting: $\Delta x=x_{\mathrm{d}1}-x$, then:

$$F_{\min }=\frac{M}{2\Delta t_1\Delta t_2}\Delta x=\mathcal{K}\Delta x$$

(20)

where $\mathcal{K}$ is called the generalized stiffness of the virtual force sensor, and its meaning is equivalent to the stiffness of the deflection structure of the real force sensor. When $\mathcal{K}$ increases, the difference between the position of the virtual object and the handle is smaller, and vice versa. The results of the theoretical analysis agree with the simulation.

According to Figure 16, although the pose obtained by the VR handle has much noise, the system can still maintain a relatively stable state, while the final pose of the virtual object is determined by tester 1 with greater strength. Although there is a certain deviation between the pose of the virtual object and the given pose of tester 1, considering that the generalized stiffness of the virtual force sensor is not infinite, this deviation is within an acceptable range. If this difference needs to be further reduced, the generalized stiffness of the virtual force sensor can be increased. However, the output fluctuation of the virtual force sensor will also increase, placing the system in an unstable state and amplifying the noise.

In Figure 17, in the first 2.5 s, the outputs of virtual force sensors 1 and 2 are not saturated, while the pose of the virtual object is not determined by tester 1 nor tester 2 but is basically in the middle of the two positions. After 2.5 s, the output of the virtual force sensor 2 is in a saturated state. In contrast, the pose of the virtual object gradually follows the given pose of tester 1′s VR handle, meaning the virtual object’s pose will not be saturated. Although tester 2 did not push the object, under the influence of tester 1, tester 2 also felt the HMIF, as illustrated in Figure 17.

Figure 18 and Figure 19 also lead to the same conclusion. The final pose of the virtual object is determined by the more powerful tester 1, while the object does not move. Similarly, although tester 1 did not push the object, under the influence of tester 2, tester 1 also felt the HMIF.

Compared to the method proposed in [10], the presented method in this paper is suitable for the situation of six degrees of freedom in space. Since the method proposed in [10] does not apply to the case of six degrees of freedom, the following comparison is only made with a single degree of freedom.

A command signal of 0.4 m is provided in the x direction. Simultaneously, noise is superimposed. For the same controller parameters of the virtual robot, the effects of the virtual force sensor under the two methods are shown in

Table 2. Effect comparison of the virtual force sensor.

Work	Rise Time 1	Noise Value 2	Solution Time 3
Proposed	1.3795 s	5.5 N	0.1852 s
[10]	1.4412 s	200 N	0.1190 s

¹ Rise time is the time required for the position response of the virtual object to rise from 10% to 90% of the position command value; Due to the saturation effect of the virtual force sensor, this value is not very small; ² Noise value refers to the difference between the maximum and the minimum value of the virtual force sensor output once the system reaches a steady state; ³ Solution time refers to the time required for the virtual force sensor to simulate one second; When the value is less than one second, the algorithm has real-time performance.

.

According to Table 2 and compared with the algorithm in [10], the proposed algorithm has a smaller output noise of the virtual force sensor for a similar rise time. However, the proposed algorithm requires a longer solution time, but the real-time performance is still satisfied.

Figure 13, Figure 14 and Figure 15 show that, even in the absence of noise interference, the output of the unsaturated virtual force sensor still exhibits a certain oscillation. At the same time, this overshoot is difficult to avoid, contributing to the overall. Since the quadratic integral of the force screw of the object is positive, the virtual object finally follows the instruction of trainee 1. One of the next research directions is minimizing the overshoot and the number of oscillations.

As proposed in this paper, the virtual force sensor is a complex, nonlinear, multiple-input and multiple-output, time-varying control system. However, its stability was not analyzed in this paper. At the same time, the appropriate selection of controller parameters so that the whole system has improved stability, speed, and stronger robustness, is also a key research direction in the future.

5. Conclusions

This paper presented a human–machine interaction force estimation algorithm suitable for astronaut collaborative virtual training to drive the physics engine. According to the experimental results, the final position of the virtual object is determined by the tester with greater strength. This is consistent with real-life experience and can demonstrate the model’s effectiveness. Moreover, in a realistic antagonistic situation, even if a single person needs to hold the object still, that person can feel the force due to the other person’s action on the object. In other words, when two people carry an object, the force exerted by a single person on the object is affected by the other person. This is consistent with experimental results. For the traditional method, the acceleration is obtained by directly calculating the second derivative of the position of the VR (virtual reality) handle, obtaining the input force. Since the position of the VR handle does not change, the calculated force is zero. This is not realistic in the case of two-person antagonism. However, the algorithm proposed in this paper is more in line with the actual situation, i.e., even if the position of the VR handle does not change, the obtained human–machine interaction force is still not zero.

Although the two people in collaborative virtual training are not connected physically, the two people interact through the physical engine to affect each other in the algorithm proposed in this paper. The virtual human–machine interaction forces of the two testers are similar in value but opposite in sign, which can be understood as action and reaction forces. In summary, compared to directly mapping the pose of the VR handle to the HMIF, integrating the physical engine and the virtual robot is more suitable for the collaborative virtual training of astronauts, as proposed in this paper.

This paper mainly focused the ground training of two astronauts jointly handling a very heavy object in the space station. Without the participation of the real robot, two trainees each used a VR handle as an input device. However, it is not necessary to use a VR handle; any device that can recognize the posture of the hand is satisfactory. In the author’s project, using the VR handle will not increase other costs since the VR system is employed as the display device. The trainee uses the VR handle to simulate the pose of a certain agent point of the object to be transported. Subjectively, the trainee feels as though he is carrying an object, but there is no force feedback. The trainee operates the VR handle as if he is operating a real object. During this process, the force that the trainee applies to the virtual object can be obtained, which is the purpose of the algorithm proposed in this paper. If heightened presence is required, one of the trainees can use a robot with force sensors to create force feedback. Therefore, a virtual robot was introduced in this paper for structural symmetry. In other words, this paper proposed a human–computer interaction method to facilitate collaborative virtual training of astronauts by the computer.