A Modified Structured Spectral HS Method for Nonlinear Least Squares Problems and Applications in Robot Arm Control

Abstract

This paper proposes a modification to the Hestenes-Stiefel (HS) method by devising a spectral parameter using a modified secant relation to solve nonlinear least-squares problems. Notably, in the implementation, the proposed method differs from existing approaches, in that it does not require a safeguarding strategy and its Hessian matrix is positive and definite throughout the iteration process. Numerical experiments are conducted on a range of test problems, with 120 instances to demonstrate the efficacy of the proposed algorithm by comparing it with existing techniques in the literature. However, the results obtained validate the effectiveness of the proposed method in terms of the standard metrics of comparison. Additionally, the proposed approach is applied to address motion-control problems in a robotic model, resulting in favorable outcomes in terms of the robot’s motion characteristics.

1. Introduction

Most classical iterative algorithms for handling nonlinear least-squares (NLS) problems and their formulations demand computation of the gradient along with the Hessian matrix of the objective function. However, the computation and storage of the entire Hessian matrix is time-consuming, because the accurate second derivatives of the residuals function are not accessible at an affordable cost [1]. In solving NLS problems, there are two classes of iterative methods: general unconstrained optimization methods that include Newton’s and quasi-Newton methods. On the other hand, the special iterative methods that take the problem’s structure into account include the Gauss–Newton, Levenberg–Marquard, and structured quasi-Newton methods [2]. The NLS problems form a category of unconstrained optimization problems determined by the following equation:

$$\underset{x\in R^n}{\min }f\left(x\right),f\left(x\right)=\frac{1}{2}{\left|\left|C\left(x\right)\right|\right|}^2=\frac{1}{2}{{\displaystyle \sum }}_{i=1}^m{\left[C_i\left(x\right)\right]}^2,$$

(1)

where $C\left(x\right)={\left(C_1\left(x\right),C_2\left(x\right),\dots ,C_m\left(x\right)\right)}^T$ and each residual $C_i:R^n\to R,i=1,2,\dots ,m$ is twice a continuously differentiable function. $\mathfrak{J}\left(x\right)\in R^{m\times n}$ stands for the Jacobian of the residual function $C\left(x\right), g\left(x\right)$ and ${\nabla }^{2}f\left(x\right)$ represents the gradient and Hessian of the objective function, f, respectively. However, $g\left(x\right)$ and ${\nabla }^{2}f\left(x\right)$ of the NLS problem stated in Equation (1) have unique structures, which are determined by the following equations:

$$g\left(x_k\right):={{\displaystyle \sum }}_{i=1}^m{C}_i\left(x\right)\nabla C_i\left(x\right)=\mathfrak{J}{\left(x\right)}^{T}C\left(x\right).$$

(2)

$$\begin{gathered} {\nabla }^{2}f\left(x_k\right):={\displaystyle \sum }_{i=1}^m\nabla C_i\left(x\right)\nabla C_i{\left(x\right)}^T+{\displaystyle \sum }_{i=1}^m{C}_i\left(x\right){\nabla }^2{C}_i\left(x\right) \\ =\mathfrak{J}{\left(x\right)}^T\mathfrak{J}\left(x\right)+T\left(x\right), \end{gathered}$$

(3)

where $\mathfrak{J}\left(x\right)=C^{\prime }\left(x\right)$ is the Jacobian matrix of the residual function and $T\left(x\right)={{\displaystyle \sum }}_{i=1}^m{C}_i\left(x\right)S_i\left(x\right);$ $C_i\left(x\right)$ is the $i$th of the residual vector’s components $C\left(x\right)$ and $S_i\left(x\right)$ is the Hessian matrix of $C_i\left(x\right)$.

Due to the cost associated with calculating the entire Hessian matrix, certain methods have been developed that utilize only the information from the first derivative. For instance, the Levenberg–Marquardt approach and the Gauss–Newton method are famous [3]. However, in solving non-zero residual problems, both seem to manifest low performance [4]. In [5], using the weighted Frobenius norm, a universal diagonal updating for DQN techniques was built through least-change weak secant diagonal updates. Furthermore, based on the structure of the NLS problem, several improvements have been recorded in quasi-Newton algorithms. The reader may refer to the surveys provided by [4,6] for detailed information about NLS algorithms. When iteratively addressing optimization problems with no constraints, conjugate-gradient (CG) methods are particularly well-known for problems with higher dimensions [7,8], due to their minimal memory usage and straightforwardness [9]. Some of the classical CG methods include Hestenes–Stiefel (HS) [10], Polak–Ribiere and Polyak (PRP) [11], and Fletcher–Reeves (FR) [12], with their formulas stated as follows:

$${\beta }_k^{HS}=\frac{g_k^T{y}_{k-1}}{d_k^T{y}_{k-1}},$$

(4)

$${\beta }_k^{PRP}=\frac{g_k^T{y}_{k-1}}{\Vert g_{k-1}{\Vert }^2},$$

(5)

$${\beta }_k^{FR}=\frac{\Vert g_k{\Vert }^2}{\Vert g_{k-1}{\Vert }^2}$$

(6)

where $y_{k-1}=g_k-g_{k-1}$, and $||.\; ||$ stands for the vectors’ Euclidean norm. The CG approaches previously outlined inspired researchers to develop several CG variations for unconstrained optimization problems; for more information, see [13,14,15,16,17,18,19].

Recently, CG algorithms have been appealing in solving the nonlinear least squares problems. For instance, Kobayashi et al. [20] developed a nonlinear CG method by fusing the structured secant relation and the Dai–Liao concept. Later, Dehghani and Mahdavi-Amiri [21] proposed a scaled CG approach by employing the modified structured secant relation to address NLS problems. A diagonal-based estimation using the Hessian matrix’s first and second terms was put forth by Muhammad and Santos [22], although, their search direction demanded a safeguarding strategy in order to attain the satisfaction of the sufficient descent condition. To overcome some of the problems they encountered in [22], Yahaya et al. [23] developed structured, quasi-Newton-based methods with the second term being approximated using higher-order Taylor’s series expansion. In subsequent efforts, Muhammad and Waziri [24] suggested a couple of BB-like algorithms for handling NLS problems and, later, Yahaya et al. [25] proposed a modified version by improving the numerical performance of [24]. However, the two algorithms demand several safeguarding techniques, especially when the spectral parameter is non-positive at some iteration points.

The objective of this research was to modify a CG-like method that utilizes a spectral parameter to improve the approximation of the Hessian matrix of the objective function. This is achieved by incorporating modified secant updates in the second part of the Hessian matrix. The motivation behind this work was the expectation that our matrix-free approach, with the inclusion of the spectral parameter update, could capture more information from the objective function and eliminate the need for a safeguarding strategy, unlike the method proposed in [25].Below is an overview of the key contributions of this study:

• A new structured spectral quasi-Newton method was designed via devising a spectral parameter and combining the modified Hestenes–Stiefel formula with the modified secant equation.
• Using a non-monotone line search, the suggested method was shown to be globally convergent under some conditions.
• Numerical experiments were conducted to compare the performance of the proposed method with other methods found in the literature.
• The proposed method was applied to solve the motion-control problem of a three-degrees-of-freedom robotic arm, showcasing its practicality.

The remaining sections are organized as follows: Section 2 provides the description of the proposed method and its algorithm. Section 3 verifies the convergence properties of the proposed algorithm under standard conditions. In Section 4, the effectiveness of the proposed method is tested through numerical experiments, comparing it with other methods in the literature. Section 5 applies the new algorithm to a robotic problem with three degrees of freedom. Section 6 presents limitations and directions for future work. Finally, Section 7 presents the conclusion, based on these comparisons.

2. Motivation and Formulation of the Modified SSHS Method

To approximate the Hessian of (1), we derive a structured spectral quasi-Newton approximation. Considering Equation (3) at a specific iteration, say $k$, for $k>0$.

$$T\left(x_{k-1}\right)={{\displaystyle \sum }}_{i=1}^m{C}_i\left(x_{k-1}\right){\nabla }^2{C}_i\left(x_{k-1}\right).$$

(7)

For the matrix $T\left(x_{k-1}\right)$ to satisfy the following secant equation, we constructed a structured vector, e.g., ${\hat{\mathsf{\Omega }}}_{k-1}$:

$$T\left(x_k\right)s_{k-1}= {\hat{\mathsf{\Omega }}}_{k-1}$$

(8)

where $s_{k-1}=x_k-x_{k-1}$; utilizing and simplifying the Taylor’s series expansion on $\nabla C_i\left(x_{k-1}\right)$ yields the following equation:

$$\nabla C_i\left(x_{k-1}\right)\approx \nabla C_i\left(x_k\right)-{\nabla }^2{C}_i{\left(x_k\right)}^T{s}_{k-1}, i=1,2,\dots ,m.$$

(9)

Therefore, pre-multiplying (9) by $C_i\left(x_k\right)$ and simplifying yields the following equation:

$$C_i\left(x_k\right)\nabla C_i\left(x_{k-1}\right)\approx C_i\left(x_k\right)\nabla C_i\left(x_k\right)-C_i\left(x_k\right){\nabla }^2{C}_i{\left(x_k\right)}^T{s}_{k-1} i=1,2,\dots ,m$$

(10)

and Equation (10) becomes

$$C_i\left(x_k\right){\nabla }^2{C}_i{\left(x_k\right)}^T{s}_{k-1}\approx C_i\left(x_k\right)\nabla C_i\left(x_k\right)-C_i\left(x_k\right)\nabla C_i\left(x_{k-1}\right) i=1,2,\dots ,m$$

(11)

Post-multiplying Equation (7) by $s_{k-1}$ and summing over Equation (11), we obtain

$$\begin{gathered} T\left(x_k\right)s_{k-1}\approx C_i\left(x_k\right)\nabla C_i\left(x_k\right)-C_i\left(x_k\right)\nabla C_i\left(x_{k-1}\right), i=1,2,\dots ,m. \\ T\left(x_k\right)s_{k-1}\approx \left(\nabla C_i\left(x_k\right)-\nabla C_i\left(x_{k-1}\right)\right)C_i\left(x_k\right), i=1,2,\dots ,m. \\ T\left(x_k\right)s_{k-1}\approx {\left(\mathfrak{J}\left(x_k\right)-\mathfrak{J}\left(x_{k-1}\right)\right)}^{T}C\left(x_k\right). \end{gathered}$$

(12)

Post-multiplying the first term of (3) by $s_{k-1}$ and adding (12), we obtain

$${\nabla }^{2}f\left(x_k\right)s_{k-1}\approx \mathfrak{J}{\left(x_k\right)}^T\mathfrak{J}\left(x_k\right)s_{k-1}+{\left(\mathfrak{J}\left(x_k\right)-\mathfrak{J}\left(x_{k-1}\right)\right)}^T C\left(x_k\right).$$

(13)

Assuming that $P_k\approx {\nabla }^{2}f\left(x_k\right)$, such that

$$P_k{s}_k\approx {\hat{\mathsf{\Omega }}}_{k-1}$$

(14)

where $P_k$ is some $n\times n$ positive definite and symmetric matrix obeying the quasi- Newton Equation (14), where $s_{k-1}=x_k-x_{k-1}={\alpha }_k{d}_k$ we obtained ${\hat{\mathsf{\Omega }}}_{k-1}$ from Equation (13), as follows:

$${\hat{\mathsf{\Omega }}}_{k-1}=\mathfrak{J}{\left(x_k\right)}^T\mathfrak{J}\left(x_k\right)s_{k-1}+{\left(\mathfrak{J}\left(x_k\right)-\mathfrak{J}\left(x_{k-1}\right)\right)}^T C\left(x_k\right).$$

(15)

Taking inspiration from [25], the modified method is coined to solve Equation (1), by incorporating the modified secant relation (15) into the CG formula, and we determined the search direction in the following manner:

$$d_k=\left\{\begin{array}{c}-g_k, k=0,\\ -{\lambda }_k{g}_k+{\beta }_k^{SSHS}{d}_{k-1}, k\ge 0.\end{array}\right.$$

(16)

The CG coefficient was defined as follows:

$${\beta }_k^{SSHS}=max\left\{\frac{g_k^T{\hat{\mathsf{\Omega }}}_{k-1}}{d_{k-1}^T{\hat{\mathsf{\Omega }}}_{k-1}},0\right\}$$

(17)

where ${\beta }_k^{SSHS}$ is the new CG coefficient, which we named the structured spectral Hestenes–Stiefel method (SSHS). The spectral parameter is defined as:

$${\lambda }_k=\frac{s_{k-1}^T{s}_{k-1}}{s_{k-1}^T {\hat{\mathsf{\Omega }}}_{k-1}}$$

(18)

where ${\lambda }_k$ is the spectral parameter and ${\hat{\mathsf{\Omega }}}_{k-1}$ is the modified secant equation defined in (15). One of the advantages of this method is its CG search direction, which ensures fast convergence and efficient memory usage, unlike the methods discussed in [24,25], where the direction follows the steepest descent method. These methods require storing the entire history of gradient vectors, which can be memory-intensive for large-scale problems. We employed the non-monotone line search developed by Zhang and Hager [26] to guarantee the global convergence of the proposed algorithms. Below is a description of the non-monotone line search. If the direction $d_k$ is sufficiently descending, the selection of step length $\alpha$ fulfills the Armijo-type non-monotone line-search conditions as follows:

$$f\left(x_k+\alpha d_k\right)\le V_k+\delta \alpha g_k^T{d}_k, \delta \in \left(0,1\right), g_k={\mathfrak{J}}_k^T{C}_k$$

(19)

where

$$\left\{ \begin{array}{c}{V}_0=f\left(x_0\right),\\ V_{k+1}=\frac{n_k{Q}_k{V}_k+f\left(x_{k+1}\right)}{Q_{k+1}},\\ Q_0=1,\\ Q_k=n_k{Q}_k+1,\end{array}\right.$$

(20)

and ${\mathfrak{J}}_k=\mathfrak{J}\left(x_k\right), C_k=C(x_k$). The parameter $n_k$ regulates the degree of non-monotonicity. The non-monotone line search shown in Equation (19) typically simplifies to a Wolfe- or Armijo-type line search if for every $k,n_k=0$.

Remark 1

([26]). The sequence $\left\{V_k\right\}$ is located amidst $f\left(x_k\right)$ and $D_k$, where

$$D_k=\frac{1}{k+1}{\displaystyle \sum }_{i=1}^{k}f\left(x_i\right), \forall k\ge 0.$$

(21)

Now, the sequential steps for solving Equation (1) are described in Algorithm 1 as follows:

Algorithm 1: Structured Spectral Hestenes-Stiefel (SSHS)

Inputs: Given $x_0\in R^n, 0<n_{min}\le n_{max}, \delta \in \left(0,1\right),$ and $k_{max}\in \mathbb{N}$. Let $ϵ>0$ be tolerance. Set $k≔0, V_0=f_0, {\lambda }_0=1, Q_0=1.$

Step 1: Compute $C_k, f_k$ and $g_k.$

Step 2: While $\Vert g_k\Vert >ϵ$ and $k\le k_{max}$ perform

Step 3: Compute

$d_k$ using (16)

Initialize $\alpha =1$;

while $f\left(x_k+\alpha d_k\right)>U_k+\delta \alpha g_k^T{d}_k$ carry out

$\alpha =\frac{\alpha }{2}$.

End

Compute the scalar ${\beta }_k$ and ${\lambda }_k$using

${\beta }_k^{SSHS}=max\left\{\frac{g_k^T{\hat{\mathsf{\Omega }}}_{k-1}}{d_{k-1}^T{\hat{\mathsf{\Omega }}}_{k-1}},0\right\}$;

${\lambda }_k=\frac{s_{k-1}^T{s}_{k-1}}{s_{k-1}^T{\hat{\mathsf{\Omega }}}_{k-1}}$. Compute the next iteration $x_{k+1}=x_k+{\alpha }_k{d}_k$.

Update

$n=min\left\{max\left\{ {\lambda }_k,n_{min}\right\},n_{max}\right\}$.

Choose $n_k\in \left[n_{min},n_{max}\right]$ and compute $Q_{k+1}$ and $V_{k+1}$ using Equation (19).

Set $k≔k+1$.

End

3. Convergence Analysis

This section presents the convergence analysis of the modified SSHS method. To begin, we introduce some valuable assumptions.

Assumption 1.

(a) 

The level set   $\mathcal{L}=\left\{x: f\left(x\right)\le f\left(x_0\right)\right\}$  is bounded, where$x_0$ denotes the starting point of Algorithm 1.

(b) 

There are non-negative constants   $c_1$,  $c_2,c_3, d_1,d_2,d_3$ in such a way that

$$\Vert \mathfrak{J}\left(x\right)-\mathfrak{J}\left(y\right)\Vert \le c_1\Vert x-y\Vert , \mathrm{for} \mathrm{all} x,y\in \mathcal{L}$$

(22)

$$\Vert C\left(x\right)-C\left(y\right)\Vert \le c_2\Vert x-y\Vert , \mathrm{for} \mathrm{all} x,y\in \mathcal{L}$$

(23)

It is obvious that (21) and (22) imply

$$\Vert \nabla f\left(x\right)-\nabla f\left(y\right)\Vert \le c_3\Vert x-y\Vert , \mathrm{for} \mathrm{all} x,y\in \mathcal{L}$$

(24)

and

$$\Vert C\left(x\right)\Vert \le d_1, \Vert \Im \left(x\right)\Vert \le d_2,\Vert g\left(x\right)\Vert \le d_3\forall x\in \mathcal{L}$$

(25)

Assumption 2.

On an open convex set that contains the level set   $\mathcal{L}$, the gradient of problem (1),$g\left(x\right)=\mathfrak{J}{\left(x\right)}^{T}C\left(x\right)$ is uniformly continuous.

Lemma 1

([25]). Let  $f$ satisfy Assumption 1 and let (15) be the definition of   ${\hat{\mathsf{\Omega }}}_{k-1}$. Then, a constant   $M\in \left(0,1\right)$ exists such that

$$\Vert {\hat{\mathsf{\Omega }}}_{k-1}\Vert \le M\Vert s_{k-1}\Vert , \forall k\ge 0.$$

(26)

Proof of Lemma 1.

Keep in mind that ${\hat{\mathsf{\Omega }}}_{k-1}$ is defined in (14) as follows:

$$\Vert {\hat{\mathsf{\Omega }}}_{k-1}\Vert =\Vert \Im {\left(x_k\right)}^T\Im \left(x_k\right)s_{k-1}+{\left(\Im \left(x_k\right)-\Im \left(x_{k-1}\right)\right)}^T C\left(x_k\right)\Vert$$

$$\le \Vert \mathfrak{J}{\left(x_k\right)}^T\mathfrak{J}\left(x_k\right)s_{k-1}\Vert +\Vert {\left(\mathfrak{J}\left(x_k\right)-\mathfrak{J}\left(x_{k-1}\right)\right)}^T C\left(x_k\right)\Vert .$$

Applying the triangular inequality, we obtain

$$\le \Vert \mathfrak{J}{\left(x_k\right)\Vert }^2\Vert s_{k-1}\Vert +\Vert \mathfrak{J}\left(x_k\right)-\mathfrak{J}\left(x_{k-1}\right)\Vert \Vert C\left(x_k\right)\Vert .$$

By using the inequalities in Assumption (1), we obtain

$$\le d_2^2\Vert s_{k-1}\Vert +c_1\Vert x_k-x_{k-1}\Vert \Vert C\left(x_k\right)\Vert .$$

By using the inequality in Equation (25), we obtain

$$\le d_2^2\Vert s_{k-1}\Vert +c_1{d}_1\Vert s_{k-1}\Vert ,$$

$$=\left(d_2^2+c_1{d}_1\right)\Vert s_{k-1}\Vert .$$

Hence, by setting $M≔d_2^2+c_1{d}_1$, the inequality (26) holds. □

Lemma 2.

Assume that Algorithm 1 generated the sequence   $\left\{x_k\right\}$ and the search direction   $d_k$. Let   $m_1$ and   $m_2$ be two constants with positive values. The following inequalities hold for every   $k\ge 0$:

(a) 

$g_k^T{d}_k\le -m_1\Vert g_k\Vert {}^2$,

(b) 

$\left|\left|d_k\right|\right|\le m_2\left|\left|g_k\right|\right|$.

Proof of Lemma 2.

To show the inequality in (a), if the direction $d_k$ is defined by (16), then

$$g_k^T{d}_k=g_k^T\left(-{\lambda }_k{g}_k+{\beta }_k^{SSHS}{d}_{k-1}\right),$$

$$=g_k^T\left(-\frac{s_{k-1}^T{s}_{k-1}}{s_{k-1}^T {\hat{\mathsf{\Omega }}}_{k-1}}{g}_k+\frac{g_k^T {\hat{\mathsf{\Omega }}}_{k-1}}{d_{k-1}^T {\hat{\mathsf{\Omega }}}_{k-1}}{d}_{k-1}\right),$$

$$\le -\frac{\Vert s_{k-1}{\Vert }^2}{\Vert s_{k-1}\Vert \Vert {\hat{\mathsf{\Omega }}}_{k-1}\Vert }\Vert g_k\Vert { }^2+\frac{\Vert g_k{\Vert }^2 \Vert {\hat{\mathsf{\Omega }}}_{k-1}\Vert }{\Vert d_{k-1}\Vert \Vert {\hat{\mathsf{\Omega }}}_{k-1}\Vert }\Vert d_{k-1}\Vert ,$$

$$\le -\frac{\Vert s_{k-1}\Vert }{ \Vert {\hat{\mathsf{\Omega }}}_{k-1}\Vert }\Vert g_k{\Vert }^2+\Vert g_k{\Vert }^2,$$

$$\le -\left(\frac{1}{M}-1\right)\Vert g_k\Vert {}^2.$$

Below is the outcome, which sets $m_1:=\left(\frac{1}{M}-1\right)$. To show the inequality in (b), we proceed as follows:

$$\Vert d_k\Vert =\Vert -{\lambda }_k{g}_k+{\beta }_k^{SSHS}{d}_{k-1}\Vert ,$$

$$=\Vert -\frac{s_{k-1}^T{s}_{k-1}}{s_{k-1}^T {\hat{\mathsf{\Omega }}}_{k-1}}{g}_k+\frac{g_k^T {\hat{\mathsf{\Omega }}}_{k-1}}{d_{k-1}^T{\hat{\mathsf{\Omega }}}_{k-1}}{d}_{k-1}\Vert ,$$

$$\le \frac{\Vert s_{k-1}{\Vert }^2}{\Vert s_{k-1}\Vert \Vert {\hat{\mathsf{\Omega }}}_{k-1}\Vert }\Vert g_k\Vert +\frac{\Vert g_k\Vert \Vert {\hat{\mathsf{\Omega }}}_{k-1}\Vert }{\Vert d_{k-1}\Vert \Vert {\hat{\mathsf{\Omega }}}_{k-1}\Vert }\Vert d_{k-1}\Vert ,$$

$$\le \left(\frac{1}{M}+1\right)\Vert g_k\Vert .$$

The results follow by letting $m_2≔1+\frac{1}{M}$, and the proof is completed. □

Lemma 3.

Let  $x_k$ be the iteration sequence produced by Algorithm 1 with$\Vert g_k\Vert \ne 0$, and let Assumption 2 hold. Moreover, using Lemma 2, where  $d_k$ was shown to be in descent, then there exists a small positive step length  ${\alpha }^{*}$ such that

$f\left(x+{\alpha }^{*}{d}_k\right)\le V_k+\delta {\alpha }^{*}{g}_k^T$holds, indicating a clearly defined line search.

Proof of Lemma 3.

To show that we always find a small-enough step length ${\mathsf{\alpha }}^{*}$, we suppose, by way of contradiction, that

$$f\left({\alpha }_k+{\alpha }_j{d}_k\right)>V_k+\delta {\alpha }_j{g}_k^T{d}_k, \forall {\alpha }_j\ge 0, \mathrm{holds},$$

(27)

Such that $\underset{j\to \infty }{ \mathit{lim}}{\alpha }_j=0$.

However, since $f\left(x_k\right)=\frac{1}{2}{\left|\left|C\left(x\right)\right|\right|}^2$ as defined in (1), this implies that

$$f\left({\alpha }_k\right)=\frac{1}{2}{\left|\left|C\left({\alpha }_k\right)\right|\right|}^2\ge 0, \forall k$$

In addition, as stated in (21), $V_k$ is between $V_{k-1}$ and $f\left(x_k\right),$ i.e., it is a convex combination of $f\left(x_k\right)$ and$V_{k+1}$. Further, $V_0=f\left(x_0\right)$ at $k=0$; then, we obtain, for all $k$, $V_k\ge 0$.

Using the boundedness of$d_k$, (19) becomes

$f\left({\alpha }_k+{\alpha }_j{d}_k\right)>V_k-\delta {\alpha }_j{n}_j\Vert g_k\Vert {}^2$; thus, using $\underset{j\to \infty }{\mathit{lim}}{\alpha }_j=0$, we obtain

$$f\left(x_k\right)\ge V_k.$$

(28)

However,

$$V_k=\frac{n_{k-1}{Q}_{k-1}{V}_{k-1}+f\left(x_k\right)}{n_{k-1}{Q}_{k-1}+1},$$

indicating that $V_k$ is convexly combining $f\left(x_k\right)$ and $V_{k-1}$, therefore, combining the above statement and (28), yields $f\left(x_k\right)\ge V_k$; hence, this means that $n_{k-1}=0$, because $Q_{k-1}$ and $V_{k-1}$ cannot be zero.

Thus, the non-monotone line search becomes monotone; i.e., Equation (27) becomes $f\left(x_k+{\alpha }_k{d}_k\right)>f\left(x_k\right)+\delta {\alpha }_j{g}_k^T{d}_k$, indicating that

$$\frac{f\left(x_k+{\alpha }_j{d}_k\right)-f\left(x_k\right)}{{\alpha }_j}>\delta g_k^T{d}_k.$$

Now, by assuming the limit as $\mathrm{j}\to \infty$ and using a uniform continuity assumption of gradient, we obtain $g_k^T{d}_k\ge \delta g_k^T{d}_k$.

However, since $g_k^T{d}_k<0$, clearly this indicates that $\mathsf{\delta }>1$, which is a contradiction. Thus, the proof. □

Theorem 1

([26]). Suppose $f\left(x\right)$ as given by Equation (1), and also that Assumptions 1 and 2 are true. Then, the produced sequence  $\left\{x_k\right\}$ by SSHS is included in the level set  $\mathcal{L}$ and

$$\underset{k\to \infty }{Lim} inf\Vert g_k\Vert =0.$$

Furthermore, if$n_{max}<1$, then

$$\underset{k\to \infty }{Lim}\Vert g_k\Vert =0.$$

Proof of Theorem 1.

The proof can easily be obtained from [26], with slight modification. □

4. Numerical Experiments and Discussion

This section presents the outcomes of the numerical experiment of the performance of our newly proposed method, the SSHS algorithm for handling NLS problems. We verified the effectiveness of the proposed approach in contrast with structured spectral gradient method (SSGM1) [24] and alternative structured spectral algorithm (ASSA1) [25], which are derived based on spectral updates parameters. This experiment’s set of test problems consists of the 15 large-scale benchmark test NLS functions described in

Table 1. List of benchmark test functions with reference and initial points.

Problems	Function Name	Initial Point	Reference
P1	Penalty function I	${\left(\frac{1}{3},\frac{1}{3},\dots ,\frac{1}{3}\right)}^T$	[27]
P2	Variably dimensioned	${\left(1-\frac{1}{n},1-\frac{2}{n},\dots ,0\right)}^T$	[27]
P3	Trigonometric function	${\left(\frac{1}{n},\frac{1}{n},\dots ,\frac{1}{n}\right)}^T$	[28]
P4	Discrete boundary value	${\left(\frac{1}{n+1}\left(\frac{1}{n+1}-1\right),\dots ,\frac{1}{n+1}\left(\frac{n}{n+1}-1\right)\right)}^T$	[28]
P5	Linear full rank function	${\left(1,1,\dots ,1\right)}^T$	[28]
P6	Linear rank-1 function	${\left(1,1,\dots ,1\right)}^T$	[28]
P7	Problem 202	${\left(2,2,\dots ,2\right)}^T$	[29]
P8	Problem 206	${\left(\frac{1}{n},\frac{1}{n},\dots ,\frac{1}{n}\right)}^T$	[29]
P9	Exponential function I	${\left(0.5,0.5,\dots ,0.5\right)}^T$	[27]
P10	Singular function 2	${\left(1,1,\dots ,1\right)}^T$	[27]
P11	Singular Broyden	${\left(-1,-1,\dots ,-1\right)}^T$	[29]
P12	Broyden tridiagonal function	${\left(-1,-1,\dots ,-1\right)}^T$	[27]
P13	Function 27	${\left(100,1/n^2,\dots ,1/n^2\right)}^T$	[27]
P14	Linear rank 2	${\left(\frac{1}{n},\frac{1}{n},\dots ,\frac{1}{n}\right)}^T$	[27]
P15	Zero Jacobian function	$for i=1 ,\frac{100\left(n-100\right)}{n},$ $for i \ge 2,\frac{\left(n-1000\right)\left(n-500\right)}{{\left(60n\right)}^2}$.	[27]

. We considered distinct dimensions for each of the problems. The dimensions comprise eight different dimensions: 1000, 3000, 5000, 7000, 9000, 11,000, 13,000, and 15,000. For more detail on the problems considered, see Table 1.

The experiment was carried out on a computer with an Intel CORE-i7-3537U processor operated as 2.00 GHz and 8 GB RAM, using MATLAB R2022a. The proposed algorithm, ASSA1, and SSGM1 were implemented with the parameters listed below.

• SSHS algorithm: $t_{min}={10}^{-30},t_{max}={10}^{30}, \delta ={10}^{-4}, n_{min}=0.1, n_{max}=0.85$ and tolerance, $ϵ={10}^{-4}$.
• SSGM1 and ASSA1: These approaches continued to use the same settings with no changes, as in [24,25], respectively.

Furthermore, at the time of execution process, the stopping criteria, $\Vert g_k\Vert \le {10}^{-4}$, was employed. The failure was reported by using ‘#’ if either of the following scenarios occurred:

(i)

There were more than 1000 iterations, or

(ii)

There were more than 5000 function evaluations.

To assess the SSHS method’s effectiveness, we considered the following metrics for comparison: (a) the number of iterations (NI); (b) the number of function evaluation (NF); (c) the CPU (TIME) taken; (d) the number of gradient evaluations (NG); and (e) the function value (FV). The results of the experiment are illustrated in

Table 2. Numerical results of the SSHS algorithm and the ASSA1 and SSGM1 algorithms on large-scale problems 1–15 in terms of NI, NF, TIME, NG, and FV, respectively. The symbol ‘#’ denotes failure of a function to solve a problem.

(a)
Problem	Dimension	SSHS	ASSA1	SSGM1
		NI/NF/TIME/NG/FV	NI/NF/TIME/NG/FV	NI/NF/TIME/NG/FV
	1000	2/4/0.011358/10/0.03612	3/5/0.012429/13/0.004316	#/#/#/#/#
	3000	2/4/0.013405/10/0.04654	3/5/0.022454/10/0.048782	#/#/#/#/#
	5000	2/4/0.020036/10/0.05777	3/5/0.058253/10/0.048677	#/#/#/#/#
P1	7000	3/4/0.015607/10/0.06985	3/5/0.047486/10/0.048491	#/#/#/#/#
	9000	3/4/0.02476/10/0.08278	3/5/0.026428/10/0.048244	#/#/#/#/#
	11,000	3/5/0.017349/7/3.73 × 10−1	4/5/0.014667/7/0.37321	#/#/#/#/#
	13,000	3/5/0.014613/7/3.78 × 10−1	4/5/0.014776/7/0.3783	#/#/#/#/#
	15,000	3/5/0.055171/7/3.83 × 10−1	4/5/0.028917/7/0.38325	#/#/#/#/#
	1000	29/513/0.064544/88/2.52 × 10−19	#/#/#/#/#	#/#/#/#/#
	3000	33/820/0.21943/100/1.37 × 10−19	#/#/#/#/#	#/#/#/#/#
	5000	38/987/0.38171/115/1.49 × 10−19	#/#/#/#/#	#/#/#/#/#
P2	7000	37/1065/0.57431/112/2.22 × 10−19	#/#/#/#/#	#/#/#/#/#
	9000	29/778/0.76929/88/1.55 × 10−19	#/#/#/#/#	#/#/#/#/#
	11,000	55/1379/1.5488/166/2.40 × 10−17	#/#/#/#/#	#/#/#/#/#
	13,000	46/1066/1.3319/139/1.85 × 10−19	#/#/#/#/#	#/#/#/#/#
	15,000	44/1032/1.3834/133/1.50 × 10−18	#/#/#/#/#	#/#/#/#/#
	1000	16/34/0.076156/106/4.72 × 10−7	20/40/0.033097/61/3.04 × 10−7	#/#/#/#/#
	3000	18/38/0.2165/169/1.02 × 10−5	18/42/0.063206/55/1.26 × 10−5	#/#/#/#/#
	5000	18/38/0.85469/175/5.00 × 10−7	23/48/0.14509/70/5.32 × 10−6	#/#/#/#/#
P3	7000	19/41/0.64686/145/1.02 × 10−6	23/49/0.2846/70/8.52 × 10−6	#/#/#/#/#
	9000	20/42/0.86614/163/6.64 × 10−6	21/48/0.40443/64/6.26 × 10−6	#/#/#/#/#
	11,000	21/43/0.86565/148/8.79 × 10−6	24/51/0.40143/73/5.87 × 10−6	#/#/#/#/#
	13,000	21/44/0.95234/166/6.50 × 10−7	22/50/0.3723/67/1.06 × 10−5	#/#/#/#/#
	15,000	21/44/1.1937/154/4.34 × 10−6	25/53/0.51278/76/1.12 × 10−6	#/#/#/#/#
	1000	6/5/0.030069/7/3.30 × 10−8	10/14/0.024906/13/4.44 × 10−8	21/23/0.075393/34/4.28 × 10−8
	3000	6/5/0.033187/9/9.24 × 10−9	12/15/0.01883/15/9.74 × 10−9	22/25/0.065956/34/1.15 × 10−8
	5000	5/7/0.029342/9/6.03 × 10−9	13/15/0.039985/27/4.22 × 10−9	23/26/0.0204/48/6.21 × 10−9
P4	7000	6/5/0.025398/8/3.08 × 10−9	13/16/0.03135/28/2.78 × 10−9	23/26/0.02165/51/3.17 × 10−9
	9000	5/5/0.028674/11/2.51 × 10−9	15/18/0.037032/29/2.51 × 10−9	25/31/0.03047/54/2.51 × 10−9
	11,000	5/5/0.037101/12/1.68 × 10−9	15/18/0.025466/31/1.68 × 10−9	27/35/0.035078/61/1.68 × 10−9
	13,000	5/5/0.038462/13/1.20 × 10−9	16/21/0.036502/32/1.20 × 10−9	29/37/0.030798/64/1.20 × 10−9
	15,000	5/8/0.041678/16/9.03 × 10−10	17/23/0.053914/31/9.03 × 10−10	31/43/0.038641/76/9.03 × 10−10
	1000	2/3/0.014686/7/5.00 × 10−1	4/5/0.004563/7/0.50018	#/#/#/#/#
	3000	2/3/0.010132/7/5.00 × 10−1	4/5/0.006245/7/0.50029	#/#/#/#/#
	5000	2/3/0.021818/8/5.00 × 10−1	5/5/0.012006/7/0.50029	#/#/#/#/#
P5	7000	3/3/0.054611/10/5.00 × 10−1	5/7/0.015672/7/0.50029	#/#/#/#/#
	9000	3/3/0.034913/10/5.00 × 10−1	5/7/0.012716/7/0.50022	#/#/#/#/#
	11,000	2/3/0.03168/7/5.00 × 10−1	5/7/0.01547/7/0.50018	#/#/#/#/#
	13,000	3/3/0.064686/10/5.00 × 10−1	5/7/0.015508/7/0.50015	#/#/#/#/#
	15,000	2/3/0.038598/7/5.00 × 10−1	5/7/0.017924/7/0.50013	#/#/#/#/#
	1000	12/59/0.011921/7/1.26 × 10+2	14/61/0.010744/7/125.5626	#/#/#/#/#
	3000	12/69/0.018244/7/3.76 × 10+2	14/71/0.023509/7/375.5625	#/#/#/#/#
	5000	12/73/0.047106/7/6.26 × 10+2	14/75/0.034067/7/625.5625	#/#/#/#/#
P6	7000	12/76/0.041661/7/8.76 × 10+2	14/78/0.040503/7/875.5625	#/#/#/#/#
	9000	14/79/0.185367/10/1.13 × 10+3	15/161/0.082234/13/1125.563	#/#/#/#/#
	11,000	14/88/0.23688/31/1375.563	17/234/0.10084/49/1375.563	#/#/#/#/#
	13,000	15/82/0.20939/10/1625.563	17/193/0.092382/22/1625.563	#/#/#/#/#
	15,000	15/84/0.16251/10/1875.563	17/158/0.10326/13/1875.563	#/#/#/#/#
(b)
	1000	4/5/0.006943/14/1.95 × 10−14	5/6/0.005326/16/1.70 × 10−9	6/7/0.027251/19/8.90 × 10−11
	3000	4/5/0.01982/14/2.29 × 10−14	5/6/0.00863/16/5.16 × 10−9	6 /7 /0.014115/19/5.17 × 10−8
	5000	4/5/0.021647/14/3.09 × 10−14	5/6/0.049918/16/8.62 × 10−9	6/8/0.015656/19/3.55 × 10−11
P7	7000	4/5/0.033112/14/3.95 × 10−14	5/6/0.014603/16/1.21 × 10−8	5/7/0.019737/19/1.10 × 10−5
	9000	4/5/0.049036/14/4.82 × 10−14	5/6/0.078329/16/1.55 × 10−8	5/8/0.03029/19/1.57 × 10−6
	11,000	4/5/0.056278/14/5.70 × 10−14	5/6/0.029075/16/1.90 × 10−8	6/9/0.036986/21/6.53 × 10−9
	13,000	4/5/0.056559/14/6.58 × 10−14	5/6/0.04135/16/2.25 × 10−8	7/8/0.043513/21/2.11 × 10−6
	15,000	4/5/0.067435/14/7.47 × 10−14	5/6/0.045155/16/2.59 × 10−8	7/8/0.03234/21/8.21 × 10−6
	1000	4/6/0.032058/4/6.60 × 10−8	10/8/0.01048/8/8.63 × 10−8	12/15/0.014342/14/5.02 × 10−9
	3000	6/6/0.032955/5/1.84 × 10−8	14/14/0.02164/13/1.94 × 10−8	15/20/0.014741/16/3.36 × 10−9
	5000	7/9/0.022651/5/1.21 × 10−8	14/15/0.019703/10/8.42 × 10−9	11/23/0.024378/13/2.40 × 10−9
P8	7000	8/9/0.021604/7/6.15 × 10−9	15/17/0.013349/11/5.55 × 10−9	19/27/0.025558/24/1.81 × 10−9
	9000	9/12/0.041471/7/5.02 × 10−9	17/18/0.034446/13/5.02 × 10−9	#/#/#/#/#
	11,000	10/18/0.018886/13/3.36 × 10−9	19/22/0.015397/17/3.36 × 10−9	#/#/#/#/#
	13,000	10/19/0.036625/14/2.40 × 10−9	25/29/0.03634/23/2.40 × 10−9	#/#/#/#/#
	15,000	11/19/0.019079/14/1.81 × 10−9	32/36/0.018665/31/1.81 × 10−9	#/#/#/#/#
	1000	2/3/0.013945/4/1.20 × 10−6	3/4/0.005307/5/4.04 × 10−6	3/4/0.022246/5/5.39E-07
	3000	2/3/0.003967/4/4.07 × 10−6	3/4/0.005192/5/4.07 × 10−6	3/4 /0.009165/5/4.07 × 10−6
	5000	2/3/0.008789/4/2.44 × 10−6	3/4/0.009438/5/2.44 × 10−6	3/4/0.013304/5/2.44 × 10−6
P9	7000	2/3/0.038487/4/1.75 × 10−6	3/4/0.005782/5/1.75 × 10−6	3/4/0.005297/5/1.75 × 10−6
	9000	2/3/0.011686/4/1.36 × 10−6	3/4/0.009688/5/1.36 × 10−6	3/4/0.007591/5/1.36 × 10−6
	11,000	2/3/0.004065/4/3.79 × 10−6	3/4/0.004049/5/3.79 × 10−6	3/4/0.004384/5/3.79 × 10−6
	13,000	2/3/0.001971/4/3.21 × 10−6	3/4/0.003526/5/3.21 × 10−6	3/4/0.003947/6/3.21 × 10−6
	15,000	4/5/0.004646/7/2.78 × 10−6	5/5/0.02421/7/2.78 × 10−6	5/5/0.003753/7/2.78 × 10−6
	1000	25/517/0.19922/76/2.5275	#/#/#/#/#	#/#/#/#/#
	3000	31/619/0.63201/94/1.0418	#/#/#/#/#	#/#/#/#/#
	5000	35/329/0.53717/106/36.0174	#/#/#/#/#	#/#/#/#/#
P10	7000	45/921/1.5763/136/1922.027	#/#/#/#/#	#/#/#/#/#
	9000	47/1012/2.3182/142/127.9268	#/#/#/#/#	#/#/#/#/#
	11,000	49/973/2.9744/148/4778.505	#/#/#/#/#	#/#/#/#/#
	13,000	34/925/2.7605/103/4778.505	#/#/#/#/#	#/#/#/#/#
	15,000	41/739/2.644/124/9932.661	#/#/#/#/#	#/#/#/#/#
	1000	3/139/0.095251/10/515	#/#/#/#/#	#/#/#/#/#
	3000	3/138/0.10986/10/1515	#/#/#/#/#	#/#/#/#/#
	5000	4/229/0.32759/13/2515	#/#/#/#/#	#/#/#/#/#
P11	7000	4/229/0.37494/13/3515	#/#/#/#/#	#/#/#/#/#
	9000	5/362/0.99275/16/4515	#/#/#/#/#	#/#/#/#/#
	11,000	5/362/1.1212/16/5515	#/#/#/#/#	#/#/#/#/#
	13,000	5/362/1.1222/16/6515	#/#/#/#/#	#/#/#/#/#
	15,000	5/362/1.2672/16/7515	#/#/#/#/#	#/#/#/#/#
	1000	131/224/0.15723/394/1.1629	147/410/0.12585/442/1.1629	#/#/#/#/#
	3000	66/95/0.23273/199/0.70391	82/235/0.16029/247/0.88148	#/#/#/#/#
	5000	77/106/0.36204/232/0.70391	77/227/0.35681/232/0.83612	#/#/#/#/#
P12	7000	65/91/0.76641/196/0.70391	141/431/0.35773/424/0.97255	#/#/#/#/#
	9000	67/101/0.81341/202/0.83612	99/317/0.49921/298/0.97255	#/#/#/#/#
	11,000	88/263/0.91256/457/0.83612	152/282/0.82151/265/1.2744	#/#/#/#/#
	13,000	110/183/1.9178/331/0.94217	191/599/1.303/574/1.4545	#/#/#/#/#
	15,000	142/388/1.6325/649/0.94217	206/467/2.0717/427/1.4545	#/#/#/#/#
(c)
	1000	9/32/0.036114/28/5.91 × 10−8	17/45/0.022348/52/6.37 × 10−7	#/#/#/#/#
	3000	9/32/0.030503/28/5.89 × 10−8	17/45/0.029784/52/6.37 × 10−7	#/#/#/#/#
	5000	9/32/0.043114/28/5.89 × 10−8	17/45/0.097405/52/6.37 × 10−7	#/#/#/#/#
P13	7000	9/32/0.071886/28/5.89 × 10−8	17/45/0.11269/52/6.37 × 10−7	#/#/#/#/#
	9000	9/32/0.097217/28/5.89 × 10−8	17/45/0.10614/52/6.37 × 10−7	#/#/#/#/#
	11,000	9/32/0.10795/28/5.89 × 10−8	17/45/0.1248/52/6.37 × 10−7	#/#/#/#/#
	13,000	9/32/0.14875/28/5.89 × 10−8	17/45/0.15258/52/6.37 × 10−7	#/#/#/#/#
	15,000	9/32/0.13755/28/5.89 × 10−8	17/45/0.28263/52/6.37 × 10−7	#/#/#/#/#
	1000	19/594/0.2163/58/418461	#/#/#/#/#	#/#/#/#/#
	3000	19/629/0.69794/58/7.25 × 10+8	#/#/#/#/#	#/#/#/#/#
	5000	28/1091/0.59477/85/3.39 × 10+8	#/#/#/#/#	#/#/#/#/#
P14	7000	34/1094/0.85736/92/1.00 × 10+10	#/#/#/#/#	#/#/#/#/#
	9000	40/1585/1.1221/121/1.05 × 10+10	#/#/#/#/#	#/#/#/#/#
	11,000	40/1589/1.1221/121/1.05 × 10+10	#/#/#/#/#	#/#/#/#/#
	13,000	41/1743/1.3302/124/9.02 × 10+9	#/#/#/#/#	#/#/#/#/#
	15,000	46/1847/1.4432/139/3.28 × 10+9	#/#/#/#/#	#/#/#/#/#
	1000	8/27/0.039893/25/1.42 × 10−10	16/31/0.020794/49/2.45 × 10−7	#/#/#/#/#
	3000	9/28/0.019763/28/6.66 × 10−7	17/32/0.033332/52/4.65 × 10−7	#/#/#/#/#
	5000	9/28/0.043256/28/3.17 × 10−8	17/32/0.067191/52/2.19 × 10−7	#/#/#/#/#
P15	7000	9/29/0.074132/28/2.25 × 10−6	17/32/0.090807/52/3.04 × 10−7	#/#/#/#/#
	9000	11/29/0.099276/34/5.55 × 10−7	17/32/0.12987/52/3.62 × 10−7	#/#/#/#/#
	11,000	11/29/0.13154/34/3.25 × 10−7	17/32/0.14986/52/4.03 × 10−7	#/#/#/#/#
	13,000	12/30/0.1364/37/1.50 × 10−7	18/32/0.1706/54/4.34 × 10−7	#/#/#/#/#
	15,000	13/32/0.15359/40/1.10 × 10−6	21/35/0.15893/64/4.57 × 10−7	#/#/#/#/#

a–c for the metrics NI, NF, TIME, NG, and FV, respectively.

Considering the results obtained in Table 2a–c, it is evident that each algorithm is competitive. Interestingly, algorithm ASSA1 failed to solve Problems P2, P10, P11, and P14 and the SSGM1 failed to solve problems P1, P2, P3, P5, P6, P10, P11, P12, P13, P14, and P15. All the test problems considered in this experiment were successfully solved by the novel SSHS algorithm. The results reported in Table 2a–c are encapsulated in Figure 1a–c, utilizing a description of Dolan and Moré’s performance [30]. In terms of NI, NF, and TIME, these figures demonstrate that the new algorithm SSHS performs better than algorithms ASSA1 and SSGM1. Obviously, the effectiveness and efficiency of the new SSHS algorithm can be identified. However, our method demonstrates slow convergence, particularly in problems P2, P10, and P14, but the other methods, in comparison, failed to solve the problems. The comparison between these algorithms was achieved using the Dolan and Moré [30] performance profile. First, we provided a summarized description of this tool; it involved a performance ratio, which entailed comparing a solver’s (s) performance on problem $p$ with the better results obtained by any solver for this problem, as provided by [31].

$$r_{p, s}=\frac{{\tau }_{p,s}}{min\left\{{\tau }_{p,s}:s\in S\right\}}$$

(29)

Suppose we select a parameter, denoted as $M$, such that $r_M\ge r_{p,s}$ is satisfied for all pairs of $p$ and $s$. This condition implies that $r_M=r_{p,s}$ holds true only when solver $s$ fails to solve problem $p$. Although we might be interested in the performance of solver $s$ on individual problems, our goal is to obtain a comprehensive evaluation of the solver’s overall performance, as defined by

$${\rho }_s\left(\tau \right)=\frac{1}{n_p}size\left\{p\in P:r_{p,s}\le \tau \right\}$$

(30)

Therefore, ${\rho }_s\left(\tau \right)$ is the likelihood, for solver $s\in S$, that a performance ratio $r_{p,s}$ is inside a component $\tau \in R$ of the finest feasible ration [31]. Moreover, the finest solver has high values of $\rho \left(\tau \right)$ and is visible in the figure’s upper-right corner.

The results shown in Figure 1d,e illustrate the performance profiles of the SSHS, ASSA1, and SSGM1 techniques, as measured by the median of the number of gradient evaluations and the function value, respectively. Furthermore, as Figure 1e shows the comparative errors of the SSHS and other methods, the convergence of the SSHS approach provides a good approximation of the solution with relatively less error. Based on the findings shown in Table 2a–c and Figure 1a–e, we can draw the conclusion that the suggested method is more successful than the ones that were compared for handling complicated nonlinear least-squares problems.

5. Application in Robotic Arm Motion Control

In this section, the proposed method utilizes Algorithm (1) to address the real-time motion control problem of a three-degrees-of-freedom planar robotic manipulator. This problem involves optimizing a time-varying nonlinear system. The majority of unconstrained optimization methods, such as [32,33], were specifically designed for addressing static nonlinear optimization problems. As a result, they may not possess the capability to effectively handle time-varying nonlinear optimization (TVNO) problems [34]. The fundamental distinction between TVNO problems and static nonlinear optimization problems is evident in the fact that TVNO problems undergo changes over time. This difference highlights the crucial role played by the time derivative in obtaining precise real-time solutions for TVNO problems. The description of the problem begins with a discrete-time kinematic representation at a specific position level, as mentioned in [34,35].

$$C\left({\theta }_k\right)=c_k,$$

(31)

where the joint angle vector ${\theta }_k\in {ℝ}^2$ is the parameter and $C\left({\theta }_k\right)$ denotes the kinematic mapping function with a familiar characteristic that can be formulated as follows:

$$C\left(\theta \right)=\left[\begin{array}{c}{l}_1\cos \left({\theta }_1\right)+l_{2}cos\left({\theta }_1+{\theta }_2\right)+l_{3}cos\left({\theta }_1+{\theta }_2+{\theta }_3\right)\\ l_1\sin \left({\theta }_1\right)+l_{2}sin\left({\theta }_1+{\theta }_2\right)+l_{3}sin\left({\theta }_1+{\theta }_2+{\theta }_3\right)\end{array}\right].$$

(32)

The function $C\left(.\right)$ is used to map the position and orientation of a robot’s end-effector or any part of the robot to the active joint displacements ${\theta }_k\in {ℝ}^2$. The length of each link is denoted by $l_i$ (for$i=1,2,3$). In the context of robotic motion control, $C\left(\theta \right)$ represents the end-effector position vector. Let $c_{t_k}\in {ℝ}^2$ be the preferred path vector at a specific moment $t_k$. Then, a least-squares problem is formulated and addressed at each interval $t_k\in \left[0,t_f\right]$, described as follows:

$$\underset{\theta \in {ℝ}^2}{\min }\frac{1}{2}\Vert C\left(\theta \right)-c_{t_k}{\Vert }^2$$

(33)

where the required Lissajous curve route at time $t_k$, as described in [33], is expressed as follows:

$$c_{t_{k1}}=\left[\begin{array}{c}1.5+0.2\sin \left(2t_k\right)\\ \frac{\sqrt{3}}{2}+0.2sin\left(t_k\right)\end{array}\right]$$

(34)

$$c_{t_{k2}}=\left[\begin{array}{c}1.5+0.2\sin \left(4t_k\right)\\ \frac{\sqrt{3}}{2}+0.2sin\left(3t_k\right)\end{array}\right]$$

(35)

Equation (33) presented above has a similar structure to that of Equation (1). Consequently, the application of the SSHS algorithm becomes viable for assessing its solution. The steps for solving the problem of robotic motion control are outlined in Algorithm 2.

Algorithm 2: SSHS for solving (33)

Inputs: initialize parameters $t_0, {\theta }_{t_0}, t_{max}, g and K_{max}$.

for $k=1:$ $K_{max}$   perform

$t_k=kg$;

Evaluate $c_{t_k}$, using (34) and (35).

Compute ${\theta }_{t_0}$, using SSHS (${\theta }_{t0},c_{t_k})$ from Algorithm 1.

Set ${\theta }_{new}=\left[{\theta }_{t_0};{\theta }_{t_k}\right]$;

end for

Output: ${\theta }_{new}$

Remark 2.

The Lissajous curve$c_{t_k}$depends on the following scenarios: (i) if$c_{t_k}=c_{t_{k1}}$, we solve the Lissajous curve route using Equation (34) and (ii) if$c_{t_k}=c_{t_{k2}}$, we solve the Lissajous curve route using Equation (35).

The following are the parameters defined as part of the SSHS Algorithm’s implementation for the model: $t=0$ is the time instant by ${\theta }_{t_0}=\left[0,\frac{\pi }{3},\frac{\pi }{2}\right]$. In addition, we chose a maximum time factor of 10 s.

Upon careful examination of the depicted figures showcasing the outcomes, the effective utilization of the SSHS algorithm in solving Equation (33) becomes evident. Figure 2a–d provides graphical representations showing a clear visual illustration of the findings. These representations played a crucial role in showcasing the impact of the SSHS algorithm in solving the equation. Specifically, Figure 2a was highlighted to emphasize the successful synthesis of the robot trajectories. The figure vividly portrays the model of the robot’s end effector closely adhering to the desired trajectory. This indicates that the SSHS algorithm effectively produced trajectories that aligned closely with the intended path, demonstrating its ability to generate accurate and precise results. Additionally, Figure 2c,d provides compelling evidence of a residual error rate approaching 10⁻⁶. The residual error rate measures the discrepancy between the desired and achieved outcomes. In this case, the near 10⁻⁶ residual error rate signifies an exceptionally high level of precision achieved by the SSHS algorithm. This result further supports the effectiveness of the algorithm in obtaining accurate and reliable solutions to Equation (33).

Furthermore, the outputs of the second Lissajous curve of Equation (35) are depicted in Figure 3. Figure 3e provides a visual representation of the successful synthesis of robot trajectories accomplished by the SSHS algorithm. It is evident from this figure that the robot effectively tracks the movement through its end effectors. Additionally, Figure 3g,h presents a residual error rate of approximately 10⁻⁶, thereby confirming the convergence of the SSHS algorithm. This convergence assures the accuracy and effectiveness of the algorithm in achieving the desired outcomes.

6. Limitations and Direction for Future Research

Our method demonstrates slow convergence, particularly in problems P2, P10, and P14. Our future research aims to address this limitation. One avenue we will explore is extending the method to a three-term spectral conjugate gradient approach by leveraging the conjugacy condition discussed in [34]. This extension will allow us to apply the new method to tackle more intricate robotic motion-control problems because, in a three-term approach, we have three subspaces: the previous gradient, the previous direction, and the previous structured modified secant relation. Furthermore, we are intrigued by the potential of utilizing the concepts presented in this paper to solve systems of nonlinear equations and their applications in data fitting and imaging problems. This opens exciting possibilities for employing our method in various domains where accurate modeling and optimization are crucial. By expanding the scope of our research, we hope to contribute to advancements in these areas and to unlock new opportunities for practical applications.

7. Conclusions

In this research paper, a modified structured spectral Hestenes–Stiefel method was suggested for solving nonlinear least-squares problems. The algorithm was developed to satisfy the modified secant equation by introducing a CG-like parameter ${\beta }_k$, that approximates the action of a vector on the Hessian of the objective function. The implemented algorithm does not require the construction or storage of matrices during the iteration phase, making it suitable for large-scale problems. Additionally, a safeguarding strategy, as mentioned in [25], is not necessary. Numerical experiments were conducted on benchmark test functions to demonstrate the effectiveness of the novel approach. It was found out that the SSHS method outperformed ASSA1 and SSGM1 methods, based on the five metrics used in the numerical results. Finally, the SSHS algorithm was applied to address motion-control problems in a three-degrees-of-freedom robot; this indicated the applicability of the suggested method.