1. Introduction
Given that simple end-effectors cannot meet the demands of complex manipulation, multi-fingered dexterous hand grasping has become one of the most important topics in humanoid robot research. Research on the multi-fingered dexterous hand covers multiple aspects, including structural design [
1,
2,
3], grasping mechanisms [
4], grasping control [
5,
6], and intelligent operation through multi-sensor information fusion [
7,
8]. The core of these research topics is to ensure that the multi-fingered dexterous hand applies an appropriate grasping force to objects for stable grasping while avoiding damage [
9,
10]. There are three main reasons for damage during the grasping process: instability in the dexterous hand’s grasp, causing the object to fall; excessive grasping force from the dexterous hand, resulting in crushing damage to the object; slippage displacement between the dexterous hand and the object causing surface damage. The first two damage mechanisms primarily relate to grasping force, whereas the third involves both grasping force and displacement. Therefore, accurate calculation of both the grasping force and the corresponding displacement is critical.
Because the relationship between the multi-fingered dexterous hand and the object presents variable contact conditions during the grasping process [
11], it poses a challenge to the calculation of grasping forces and displacements in the robot multi-fingered grasping problem. To solve this challenge, researchers have introduced complementarity relationships. The complementarity methods for the two-dimensional (2D) multi-fingered grasping problem are relatively mature. Various complementarity methods, such as the Pivot method [
12,
13], can be used to solve it efficiently. However, the complementarity method for the three-dimensional (3D) grasping problem is far from mature. Because the 3D multi-fingered grasping problem can be modeled into different models based on different treatments of the friction contact relationship, complementarity methods of these models vary significantly.
Due to the nonlinearity of frictional contact, some researchers have modeled the multi-fingered grasping problem as an optimal problem with nonlinear complementarity constraints for its solution. For instance, the Bullet simulator [
14] proposes a Projected Gauss–Seidel method to solve this optimization problem. Carpentier [
15] combines the Alternating Direction Method of Multipliers with the Proximal Method to solve it. However, these methods all require solving subproblems in each outer iteration.
By using a polyhedral pyramid to approximate the Coulomb friction cone, some researchers have modeled the multi-fingered grasping problem as an LCP. For instance, Fahed modeled frictionless multi-fingered grasping [
12], friction-considered hard finger grasping [
16], and soft hand grasping [
17] as LCPs. Panagiotopoulos [
18] decomposed the multi-fingered grasping problem into a contact problem with known contact force, then modeled it as an LCP. Maximilian [
19] modeled the multi-fingered grasping problem as an LCP and then transformed the LCP into an optimization problem for numerical solution. Wu and Li [
20] transformed the elastic contact mechanics problem into an LCP with the Toeplitz coefficient matrix.
Numerous methods can be used to solve LCPs in numerical computation fields. For example, iterative methods, such as the projected method [
21], the modular iteration method [
22], and the Newton method [
23,
24], can be used to solve large-scale problems. The projected method and the modular iteration method have rapidly developed since Bai [
25] introduced the idea of matrix splitting. But the convergence of these iterative methods requires that the coefficient matrix has a special structure, such as H-matrices [
26], H
+-matrices [
27], and L-matrices [
28]. However, the coefficient matrix of LCP arising from the 3D multi-fingered grasping problem is a saddle-point matrix, and existing numerical methods are unable to solve the problem; meanwhile, the Pivot method is only suitable for special cases.
Addressing the aforementioned issues, we utilize a merit function such that solving the LCP is equivalent to solving the non-smooth system of equations. In the process of seeking methods to solve non-smooth system of equations, it was found that the Kaczmarz method has advantages. On the one hand, Kaczmarz-type methods have demonstrated superior performance in solving linear [
29,
30] and nonlinear [
31,
32] systems of equations. On the other hand, the Kaczmarz method can just find the solution closest to the initial iteration value. These advantages motivated us to apply the Kaczmarz method to the solution of grasping problems. Based on this, we propose the NSNGRK method. If the NSNGRK method converges and the zero vector is set as the initial value, the result must be the solution to the multi-fingered grasping problem, even though there are multiple satisfactory grasping forces. Notably, in practical applications, state-of-the-art multi-fingered robotic systems can be equipped with embedded 6-DOF F/T sensors at each fingertip for real-time contact force monitoring. This sensory feedback can serve as critical input to the NSNGRK method for the online computation of external wrenches.
A brief overview of this paper is presented below. Firstly, the LCP model of the robot multi-fingered grasping problem is formulated based on the existing model. Secondly, the NSNGRK method is designed. Thirdly, the convergence of NSNGRK is proven. Fourthly, numerical examples are used to verify the feasibility and effectiveness of the NSNGRK method in solving multi-fingered grasping problems. The conclusions are briefly presented in the final section.
2. LCP Model of Multi-Fingered Grasping
Figure 1 illustrates a simple scenario in which a multi-fingered hand, equipped with n two-jointed fingers, grasps an object within a 3D space. Many more complex grasping scenarios can also be simplified into the form illustrated in this figure for further analytical studies. Consider a grasping case where finger
i of the multi-fingered hand can exert a fingertip force on the object within the friction cone at the contact point, Fahed [
16] and Panagiotopoulos [
18] transformed the grasping problem into a linear complementarity problem. For the grasped object, when a finger applies a normal force to the object, a friction force arises between the finger and object under external loading. This friction force combines with the normal force to balance external forces. On the finger, a tangential reaction force develops due to friction. When the grasp achieves force closure, there is
where
denotes the normal force,
the tangential force,
the grasping matrix for the normal force,
the grasping matrix for the tangential force, and
the external wrench. According to the principle of virtual work [
33], we have
where
,
, and
denote the displacement, the normal displacement, and the tangential displacement of the object. According to the unilateral contact condition, in the normal direction, there exists
where
denotes the normal slippage displacement,
the normal displacement of the fingertip, and
the initial normal distance vector between the fingers and the object. The compatibility conditions of unilateral contact behavior can be described as in
Figure 2. The complementarity relationship between the normal slippage displacement
and the normal force
can be written as follows:
Similarly, the tangential boundary condition can be obtained as follows:
where
denotes the tangential slippage displacement, and
denotes the tangential displacement of the fingertip. According to Coulomb’s friction law, when contact occurs, there exists
where
denotes the frictional coefficient, and
n denotes the number of times contact is made. The frictional condition can be described as follows:
If the tangential force is less than the friction force (i.e., there exists ), the tangential slippage displacement equals zero.
If the tangential force is equal to the friction force (i.e., there exists ), the tangential slippage displacement is non-zero.
The direction of the slippage displacement is opposite to that of the tangential force. Equation (
6) represents the Coulomb friction cones, each of which is a three-dimensional cone. To linearize the friction cone, a polyhedral cone is used to approximate the friction cone. The equation for the friction cone is
when the friction cone is approximated by a pyramid with
m faces, the edges of the pyramid can be expressed as
in the contact coordinate system
. The piecewise-linearized form of Equation (
7) becomes
where
is the frictional coefficient. By substituting (
8) into (
6), we obtain
If
, then Equation (
9) can be rewritten as
where
in which
denotes a block diagonal. Given that the normal force is orthogonal to the tangential motion, let
be a coefficient related to the tangential motion satisfying the following relationship:
Then,
and
are orthogonal, that is,
If linear elastic behavior exists between the finger and the object during the grasping, there is
where
and
denotes the flexibility matrix.
Since the grasping force cannot be unbounded, to further constrain the solution set we impose an upper bound
on the normal force component, yielding
where
. By introducing a non-negative relaxation factor
, Equation (
14) can be rewritten as
At the same time, the normal unilateral contact conditions (
3) can be rewritten as
where
and
are complementarity variables. When the normal forces are lower than the upper bound (i.e.,
), we have
; the unilateral contact conditions remain unchanged. When the normal forces reach the upper bound (i.e.,
), we have
; the unilateral contact conditions will depend on
. The unilateral contact condition will change in this situation, which makes disables the normal force from reaching the upper bound. Let
If
is a nonsingular matrix, then
where
A standard LCP can be obtained from the relationship between (
1), (
2), (
4), (
5), (
10)–(
13), (
15), and (
16), which can be written as
where
, .
It is apparent that the coefficient matrix
M in Formula (
17) is neither symmetric nor skew-symmetric and the element in the bottom right corner is a zero matrix. This type of matrix is also known as a saddle-point matrix. Since the coefficient matrix
M is a saddle-point matrix, methods like modulus based splitting methods and projected methods cannot be applied, as a saddle-point matrix does not satisfy their convergence conditions. As mentioned before, the Pivot method, as detailed in Algorithm 1, provides an effective solution for linear complementarity problems arising from 2D grasping problems. When reducing 3D grasping problems to 2D equivalents, the resulting linear complementarity problem’s coefficient matrix becomes a skew-symmetric saddle-point matrix. As the Pivot method’s convergence criterion requires copositive-plus matrices [
13], convergence is guaranteed in this configuration. However, for 3D grasping problems where the coefficient matrix, often fails to satisfy copositive conditions, the Pivot method remains applicable only in specific scenarios.
Algorithm 1 Pivot method [13] |
- Require:
Coefficient matrix , vector - Ensure:
Solution vector pair - 1:
Check Initial Condition: If , stop the calculation, is a complementary basic feasible solution; otherwise, Otherwise, a column vector of ones and an artificial variable must be introduced to construct a new system - 2:
Formulate Tableau and Initialize Pivot: Represent the system in a tabular format as . Determine
take row s as the pivot row and the column corresponding to as the pivot column, perform pivot elimination, and set . - 3:
Determine Pivot Direction: In the current tableau, let the column corresponding to variable be . If , stop the calculation, obtaining an extreme direction of the feasible region of the problem; otherwise, determine the index r by the minimum ratio test such that If the basic variable in row r is , proceed to Step 4; otherwise, proceed to Step 3. - 4:
Pivot Operation for Basic Variables: Let the basic variable in row r be or (). Variable enters the basis, with row r as the pivot row and the column corresponding to as the pivot column, perform pivot elimination. If the leaving variable is , set ; if the leaving variable is , set , then return to Step 2. - 5:
Final Pivot to Remove Artificial Variable: Variable enters the basis, and leaves the basis. Take row r as the pivot row and the column corresponding to as the pivot column, perform pivot elimination. Obtain a complementary basic feasible solution and stop the calculation.
|
3. Newton–Subgradient Non-Smooth Greedy Randomized Kaczmarz Method
In this section, we attempt to construct a new method to solve the LCP with a saddle-point coefficient matrix. Firstly, we transform the LCP into a non-smooth system of equations. There are two transformation ideas. One is to transform the LCP into absolute value equations (AVEs) like the modular matrix splitting iterative method. However, as shown in [
34], determining the existence of a solution to AVEs is NP-hard. We choose the second transformation, that is, transforming the LCP into an equivalent non-smooth system of equations by introducing the Fischer–Burmeister (F-B) function. The F-B function, being semi-smooth and continuously differentiable except at the origin, yields the following non-smooth system of equations:
where the
n-dimensional real vector
,
. Since the F-B function
is locally Lipschitz-continuous [
35], the subgradient of
at
meets
where
denotes the subdifferential. Thus, Equation (
18) has a generalized Jacobian matrix
in the sense of Clarke [
36] at
, denoted as
where
h is a small disturbance. We can deduce that the iterative format of the Kaczmarz method for the non-smooth system of Equation (
18) is
To solve the non-smooth system of equations by using the idea of the greedy randomized Kaczmarz method, it is necessary to construct a non-empty indicator set
that guarantees a higher probability of selecting a row with larger residuals at the
kth iteration. With the generalized Jacobian matrix
, the idea of solving linear equations in reference [
32] can be used to construct a non-empty set
by setting a threshold
related to the current iteration in a non-smooth system of equations
It is easy to prove that the indicator set
is non-empty. Let us define the residual as follows:
Then, the hyperplane is selected according to the probability
. Now, we give the NSNGRK method for solving the LCP (please see Algorithm 2). Facchinei and Soares [
37] demonstrated that although the F-B function is semi-smooth, its natural merit function is continuously differentiable. This allows the selection of any initial point during iteration. Specifically, when the initial iterate for the Kaczmarz method is set to the zero vector, the NSNGRK method, if convergent, will certainly converge to the point closest to the origin.
Algorithm 2 Newton–subgradient Non-smooth Greedy Randomized Kaczmarz (NSNGRK) |
- Require:
Coefficient matrix , vector , precision threshold , initial value - Ensure:
Solution vector - 1:
Initialize residual - 2:
for to n do - 3:
- 4:
end for - 5:
while do - 6:
Compute subgradient matrix: - 7:
Determine threshold ( 21) and active index set ( 22) - 8:
Build probability distribution: - 9:
- 10:
Compute new residual - 11:
end while
|
5. Numerical Experiments
We present the numerical solutions of our algorithm in five robot grasping scenarios: a 2D grasping problem, a 3D three-finger grasping problem with partial known finger forces, a 3D three-finger grasping problem with all unknown forces in two different configurations, and a 3D five-finger grasping problem.
All numerical experiments were conducted on a computer with a CPU of 2.30 GHz and 16.0 GB of memory. For the NSNGRK method, the zero vector was used as the initial value, with the termination criteria set to either exceeding 20,000 iterations or achieving a RES
, where RES is defined as the norm of the residual norm vector [
23]
To conduct a comparison with the NSNGRK method’s numerical solutions, we also provide results from the Pivot method in cases where they are applicable.
Example 1. Consider a 2D grasping problem: a rigid planer object grasped by six frictionless elastic fingers [12]. The global coordinate system and finger contact locations are shown in Figure 3. The flexibility of the fingers is . The external wrench is . The results of the Pivot method and the NSNGRK method are presented in
Table 1 and
Table 2. From both tables, it can be observed that the numerical solutions show complete agreement with the exact solutions within the error range. The results demonstrate that the NSNGRK method exhibits excellent performance in solving 2D grasping problems.
Example 2. Consider a 3D grasping problem: an isosceles right-angle prism grasped by three fingers [38]. The global coordinate system and finger contact locations are shown in Figure 4. The positions of the contact points are , , . The friction coefficient is . The flexibility of the springs are, for the normal , the tangential . Finger exerts a normal force on the object. When approximating the friction cone with a pyramid, engineering standards require the pyramid to have at least four faces [
39]. The normal and the tangential forces of fingers
and
, the slippage displacements, and the object displacement calculated by the Pivot method and the NSNGRK method when approximating the friction cone with a four-face pyramid are given in
Table 3 and
Table 4, respectively. From these two tables, we can see that the results of the two methods are of the same order of magnitude. To improve friction cone representation, polyhedral cones with increased face counts were implemented. However, the Pivot method fails to converge when the face counts exceed eight, whereas the NSNGRK method successfully handles these cases.
Table 5 shows the results of polyhedral approximations with 16 faces. Compared with
Table 4, we find that normal force and tangential force of two fingers become closer, which shows that the NSNGRK method can obtain a more accurate solution with more faces and the NSNGRK method has more advantages than the Pivot method in solving larger-scale problems. On the other hand, we find that the grasping forces are enlarged with fewer pyramid faces.
Example 3. Consider a 3D grasping problem: a cube grasped by three hard fingers [16]. The global coordinate system and finger contact locations are shown in Figure 5. The positions of the contact points are , , . The friction coefficient is 0.6. The gravity of the cube is 5. For the polyhedral approximations of a Coulomb friction cone with eight faces, the results calculated by two methods are shown in
Table 6 and
Table 7. When the number of faces is larger than eight, the Pivot method cannot solve the problem. The results of the NSNGRK method with 16 polyhedral faces are shown in
Table 8.
In the existing three examples, the Pivot method can still yield results when there are fewer pyramid surfaces. However, in the following example, even with fewer pyramid surfaces, the Pivot method fails to produce results.
Example 4. Consider another grasping configuration of a cube grasped by three hard fingers, as shown in Figure 6. The finger and object contact locations are set to , , . The other parameters are set to the same values as in Example 3. In this example, the Pivot method fails to solve the problem. The results of the NSNGRK method are shown in
Table 9 when the polyhedral approximation is set to six faces.
Example 5. Consider a cube grasped by five fingers, as shown in Figure 7. Finger and finger are frictionless, and the three others are hard fingers. The finger and object contact locations are set to , , , , . The other parameters are set to the same values as in Example 3. The Pivot method also fails to solve the problem in this example. The results of the NSNGRK method are shown in
Table 10 when the polyhedral approximation is set to six faces. The five numerical experiments rigorously validate the proposed method’s computational feasibility and effectiveness across different multi-fingered grasping configurations. Although physical experiments are currently lacking, these numerical experiments are based on existing multi-fingered grasping problems, ensuring the numerical results can be realistically interpreted.