1. Introduction
Master–slave teleoperated robotic systems are being applied to a wide range of fields, from the deep sea to space exploration and from mining to medical industries. Robotic minimally invasive surgery (RMIS) is one such field benefiting from the improved levels of control offered by teleoperated systems. Teleoperated robotics are capable of delivering improvements in motion scaling, hand tremor filtering, and hand-eye co-ordination, as well as enabling the automation of specific quantifiable actions, such as suturing [
1,
2,
3,
4].
Bilateral teleoperators are teleoperated systems capable of providing force feedback or haptic feedback to the operator [
5,
6]. Presently, commercial RMIS systems do not include haptic feedback [
7,
8], which has been reported as one of the disadvantages associated with RMIS systems [
9,
10,
11,
12]. The lack of an intuitive force feedback mechanism contributes to the technique’s steep learning curve and slow completion time.
The difficulty in providing haptic feedback for surgical applications stems from the fact that traditional bilateral controllers are unable to provide high levels of transparency and stability simultaneously [
13]. Ideal or perfect transparency is defined as the matching between the impedance felt by the operator and that experienced at the slave. Typically, this is achieved via kinematic and force correspondence between slave and master subsystems. First realised in [
14] by Lawrence and extended in [
15] by Hashtrudi and Salcudean with the inclusion of modelled time delays, transparency and stability have been shown to occupy opposing ends of a spectrum. Improving transparency degrades stability while increasing stability mutes system transparency [
16,
17,
18]. Additionally, perfect transparency is not possible in traditional feedback/feedforward bilateral controllers due to this innate communication time delay between slave and master subsystems [
19].
The literature contains a wide range of algorithms and approaches designed to address the issue of communication time delay in bilateral teleoperations. The majority of these approaches are based on the passivity control theory [
20,
21]. Using this approach, the communication network, which is a source of non-passivity among the other teleoperation control subsystems (including the operator and environment, both of which are considered passive components), is transformed into a passive element, resulting in a stable and passive system as a whole. Among the most commonly used passivity-based techniques is wave variable control (WVC) [
22,
23], which involves transforming force and motion into wave variables, resulting in the passivity of the communication network being limited to an unknown and constant delay. There is also the time domain passivity control (TDPC) [
24] technique that dissipates the additional and nonpassive energy by injecting adaptive damping into the system in the event that there is an unknown and variable time delay. As an alternative to passivity-based control, there is the small gain control theory [
25], which does not require the assumption of passivity of system components since it deals with the overall loop gain of the system. However, all of the algorithms above alter the flow of force and motion data in some form in order to mitigate the excess energy that adversely affects the system’s transparency and results in an infeasible haptic feedback system for surgical applications [
13].
These restrictions have prompted the use of predictive control in place of the approaches mentioned above and direct force feedback methodologies [
26,
27,
28,
29]. Predictive control strategies are able to compensate for the system time delay by using local models at the master and/or slave to generate operator feedback forces. A majority of these predictive controllers use Smith predictors [
30,
31,
32,
33] and neural networks [
34,
35,
36] to compensate for communication time delays. An overview of the various predictive control approaches can be found in [
30].
A similar predictive approach involves modelling the environment on-line, using estimation methods such as the Recursive Least Squares (RLS) or Exponentially Weighted Recursive Least Squares (EWRLS) [
37,
38,
39]. By developing an estimation of the environment, a virtual environment can be created with which the haptic device can interact. Force-feedback can then be tied to the operator’s own hand movements, effectively bypassing the time delay present between the slave and master subsystems and the need to wait for the slave system to measure the response. Further, provided the estimated parameters are a good representation of the environment, then stability is guaranteed under the assumption that the contact environment is passive.
The Kelvin–Voigt (KV) force model, consisting of the parallel connection of a linear spring and damper, has been widely used as an underlying environment force model [
40,
41]. However, the KV model has several inconsistencies in relation to power exchange and restitution during contact [
42], which makes it unsuitable for modelling for soft-body contact. Further, the Kelvin–Voigt model is a linear model, whereas most biological soft tissues are non-linear [
43,
44].
The Hunt–Crossley (HC) force model demonstrates more consistency with the dynamic behaviour of soft bodies [
42]. Palpation exercises conducted in [
45], comparing a number of different force models, found that the Hunt–Crossley model demonstrated superior distinction between regions of varying stiffness. However, human-in-the-loop operation was not evaluated, and force feedback was limited to a position–position architecture. Estimating the parameters of the Hunt–Crossley force model has been previously conducted by partially decoupling the estimation of the model parameters via separate processes [
42], which results in relatively slow parameter convergence. In [
46], a single-staged identification method was developed whereby the Hunt–Crossley model was linearised via a logarithmic approximation. With the log-linearised form, parameter estimation could then be conducted using the robust family of RLS algorithms. In [
47], parameter estimation was conducted on-line and demonstrated comparatively faster parameter convergence when compared to the two-stage identification process. However, these estimations were performed with a computer-generated trajectory and did not include human-in-the-loop teleoperation.
In cases where remote environments included materials with different rigidity, the researchers also investigated a hybrid approach using a combination of KV and HC models called the threshold contact switching model (TSCM) [
48]. For adapting to different contact environments, a threshold was used to switch between each model. While this approach had some advantages, it was limited by undesired inconsistency and oscillation when switching between different models. To resolve this issue, a continuous switching contact model (CSCM) based on energy loss was developed [
49,
50,
51]. However, further improvements are still required in terms of accuracy and transparency.
Human-in-the-loop teleoperation presents additional challenges, particularly when an estimation of the environment is desired. When the slave mechanism’s motion is driven by a human-controlled device (i.e., a haptic-feedback device), the question of persistency of excitation becomes important. Does the environment estimator receive enough varying information (force, position, or velocity) to accurately estimate the contact environment’s parameters? Additionally, the feedforward/feedback nature of the bilateral controller means that teleoperator performance needs to be evaluated holistically. The goal of this work is to experimentally verify the performance of an on-line environment estimation–force prediction methodology; to evaluate teleoperator performance as a whole. It also aims to demonstrate that transparent teleoperation can be achieved with an estimated virtual environment, which is interacted with by the haptic master device.
The main contributions of this research are as follows: first, development, characterisation, and experimental validation of a high-transparency predictive force-feedback methodology. In this study, an environment estimation–force prediction control architecture is developed and implemented into a three-channel bilateral teleoperated system. The use of a force predictor means that neither kinematic nor force correspondence is required to be strictly maintained. Secondly, investigation of the human-in-the-loop teleoperation transparency performance of the bilateral robotic system by studying various force modelling approaches in isolation, including a spring-based adaptor, KV, and HC force models. Lastly, a metric for transparency is developed that accounts for the positional lag between slave and master systems. By using the current slave position as a reference, and comparing forces at a prior comparable master position, an accurate measure of error can be determined on the fly.
The remainder of this paper is organised as follows:
Section 2 describes an overview of the proposed system the environment models chosen, and the parametrisation method used for each sample.
Section 3 outlines the metric used to evaluate the transparency performance.
Section 4 describes the experimental platform and procedure.
Section 5 presents the experimental results of a palpation exercise into soft polyurethane foam. Finally,
Section 6 presents the conclusions and suggestions for future work.
3. Transparency Performance
Figure 2 represents an equivalent circuit diagram of the bilateral teleoperator system shown in
Figure 1. Here,
V,
F, and
Z refer to velocity, force, and impedance, respectively. The subscript
s and
m refer to the slave and master, respectively. In this figure,
is the operator’s force, and
is the environment’s exogenous input force.
is the master feedback, and
is the measured slave force.
and
are impedances representing the dynamics of the operator’s hand and remote environment, respectively.
is the master velocity, and
is the slave velocity.
is the impedance perceived by the operator. Impedance,
Z, encompasses physical mass, damping, and stiffness properties, and each quantity is the Laplace transform of their respective variable. It is generally assumed that the operator and environment are passive (and thus stable), as they do not act in such a way as to produce or inject additional energy into the system; thus,
and
[
53].
The linear time-invariant (LTI) dynamics of the above system are:
The impedance experienced by the operator is defined as:
Lawrence [
14] defines the transparency condition as:
Equation (
13) translates to the operator (
) experiencing the same environmental behaviour as the slave (
).
This approach is known as
impedance matching and is a consequence of the sought-after
kinematic correspondence (
14a) and
force reflection (14b) between the master and the slave:
With these definitions, a
hybrid matrix can be developed, as shown below:
Using the kinematic and force correspondence conditions in (14a,b), and the hybrid matrix of (
15), the
ideal transparency is defined as (with no force or position scaling):
The proposed bilateral controller has two definitions of error that can be used to evaluate teleoperator transparency performance: estimation error and prediction error, each with a definition describing the accuracy of different processes. The estimation error is localised to the slave–environment interaction and indicates the accuracy of the estimated parameters in reference to the measured forces at the slave. The prediction error relates the interaction between the master and virtual environment to the interaction of the slave with the measured environment. The prediction error encompasses the master’s dynamics, which typically extends beyond the range of the slave–environment dynamics. As such, the prediction error provides an indication of system transparency, whereas the estimation error indicates estimator transparency.
3.1. Prediction Error
Prediction error is the error between the predicted master force and the measured slave forces at a given position inside the real and virtual environments. It is used as a metric for transparency, given that it evaluates how well the predictive force feedback mechanism is operating in terms of the teleoperation system as a whole.
A method for calculating the prediction error involves comparing the measured current force with the predicted master force at an unknown time in the past. This is because, at each time-step, the slave and the master are typically at different positions within the real or virtual environment and, thus, are expected to experience different forces and cannot be compared directly. Instead, the forces are compared when the slave and the master kinematics are as similar as possible.
The prediction error,
, at time-step
is presented as
for
This is accomplished by:
Considering the slave position () and force () at each time-step (), and using these as the slave reference;
Searching through the dataset of the master positions, backwards from the slave reference time-step, for the first closest master position () to .
Comparing the direction of motion of the slave and master to ensure the two systems were travelling in the same direction. If the directions differ, then the next closest slave and master positions are used.
The time-step when the positions are closest, and the direction of travel is consistent, is , and is used as the master reference;
Finding the error between the slave force () and the master force () at their respective reference time-steps.
The above method attempts to determine how accurate the predicted force was once the slave has passed through the previous master position; the error analysis is aposteriori, as opposed to current.