A Review on Inverse Kinematics, Control and Planning for Robotic Manipulators With and Without Obstacles via Deep Neural Networks

Abstract

Robotic manipulators are highly valuable tools that have become widespread in the industry, as they can achieve great precision and velocity in pick and place as well as processing tasks. However, to unlock their complete potential, some problems such as inverse kinematics (IK) need to be solved: given a Cartesian target, a method is needed to find the right configuration for the robot to reach that point. Another issue that needs to be addressed when dealing with robotic manipulators is the obstacle avoidance problem. Workspaces are usually cluttered and the manipulator should be able to avoid colliding with objects that could damage it, as well as with itself. Two alternatives exist to do this: a controller can be designed that computes the best action for each moment given the manipulator’s state, or a sequence of movements can be planned to be executed by the robot. Classical approaches to all these problems, such as numeric or analytical methods, can produce precise results but take a high computation time and do not always converge. Learning-based methods have gained considerable attention in tackling the IK problem, as well as motion planning and control. These methods can reduce the computational cost and provide results for every situation avoiding singularities. This article presents a literature review of the advances made in the past five years in the use of Deep Neural Networks (DNN) for IK with regard to control and planning with and without obstacles for rigid robotic manipulators. The literature has been organized in several categories depending on the type of DNN used to solve the problem. The main contributions of each reference are reviewed and the best results are presented in summary tables.

1. Introduction

A robotic manipulator can be defined as a series of links connected together by joints [1]. They are commonly used in pick and place or processing tasks and can be governed directly by an operator, or a logical device. Manipulators are used throughout the industry in a variety of sectors, such as manufacturing, agriculture, health and medicine, space, exploration, military, defense, and service.

One of the main challenges regarding robotic manipulators is solving the inverse kinematics (IK) problem: given a desired pose for the end-effector $X_d\in SE\left(3\right)$, find joint space positions $\theta$ which achieve said pose. This problem may have multiple solutions, a unique solution, or no solution at all [2]. Furthermore, some of the solutions could lead the robot to reach a singularity, causing operational issues. The problem can become more complicated if there are obstacles present in the workspace. In this case, given a desired pose for the end-effector $X_d\in SE\left(3\right)$ and a set of constraints, a sequence of configurations could be devised for the manipulator so that the robot can follow them and reach the target without colliding (motion planning [3]). The alternative to motion planning would be to design a controller so that, given the current state of the manipulator and the obstacles, the best immediate configuration is chosen so that the robot gets closer to the target while staying away from the obstacles (motion control [4]).

The IK problem can be classically solved using analytical, numerical, or meta-heuristic algorithms. Analytical solutions are generally the most precise, so advances are being made in this field to find new methods [5,6,7]. The computing time of the analytical IK solutions is faster than the numerical approach and real-time capable, but it is only available for specific robot kinematics. Numerical methods can be divided into three categories: Jacobian, Newton and heuristic methods [8]. They are the preferred option when dealing with redundant manipulators. However, they often reach singularities and have high computational costs (except for heuristic methods, which tend to be faster but cannot adapt to global constraints and sometimes end in unnatural positions). New advances regarding numerical methods can be seen in [9,10]. Finally, meta-heuristic algorithms [11] such as genetic algorithms [12] and particle swarm optimization [13] represent another possible means to solve the IK problem. However, these methods are unsuitable for dynamic environments and require high computation time.

Motion control takes the manipulator and environment state as an input and decides the best immediate action according to a function. In the presence of obstacles, this function will contain the distance to the target as an objective to minimize and the distance to the obstacles as an objective to maximize. Motion control for obstacle avoidance can be achieved by means of the Jacobian pseudo-inverse [14]. On the downside, this method can lead to singularities and relies on a distance algorithm such as [15]. Artificial potential fields [16] represent another option, which consist of a set of forces that attract the manipulator toward the goal and repel it from obstacle regions. They can be used in real-time controllers. A more recent alternative is the Jacobian transpose [17], which does not need to calculate inverse Jacobian matrices or to know the dynamics of manipulators.

Motion planning takes the manipulator and environment state as an input and outputs a sequence of actions according to a set of parameters and constraints. In the presence of obstacles, these constraints will make sure that the robot does not make movements that would take it close to colliding with them or with itself. Motion planning can be achieved via potential fields [18] as well. However, the preferred options to plan trajectories while avoiding collisions are random sampling based methods such as Rapidly Exploring Random Trees (RRT) [19]. RRT has probabilistic completeness, perfect expansion, and a fast exploring speed. However, it cannot deal with dynamic unstructured environments in which many obstacles are distributed randomly. Another sampling based method is Probabilistic Roadmaps (PRM) [20]. The algorithm abstracts the configuration space by selecting a number of points in it and connecting them to form an undirected graph. The path search problem in the free space is transformed to a graph search problem.

In this article, the approach to the IK, motion control and motion planning problems using Deep Neural Networks (DNN) is studied. Neural networks are capable of learning complex functions from data, and then generalize to new points outside of it. Therefore, their use in inverse kinematics and obstacle avoidance yields multiple solutions within an acceptable error range, and far enough from possible obstacles. Additionally, once the training has been completed, the inference time is independent of the configuration of the robot, proving useful for real-time controllers.

In the following sections, different DNN architectures will be reviewed, studying their use and precision when dealing with inverse kinematics, motion control and planning for robotic manipulators, with and without obstacles. More specifically, the key challenges addressed by the reviewed approaches are their precision, architectural and computational complexity, time cost and real-world applicability. To select the references included in this paper, Google Scholar has been used as the main scientific search engine, filtering by year and prioritizing on well-established publishers such as MDPI, IEEE, Elsevier, and Springer. The search has been conducted using keywords such as inverse kinematics, motion control, motion planning, robotic manipulator, DNN and each specific type of DNN architecture (e.g., MLP, CNN). Other relevant reviews devoted to the application of neural networks in robotics are [8,21,22,23,24,25].

The sections are organized as follows: In Section 2, solutions to the IK problem involving DNN are presented. They are classified in supervised learning, unsupervised learning and Deep Reinforcement Learning, and also by the type of DNN used. This is a conventional classification within the machine learning community, which focuses on the problem statements before delving into the set of techniques or algorithms that serve to solve them. The obstacle avoidance problem is studied next, first from a motion control perspective in Section 3 and then from a motion planning one in Section 4. Section 6 also includes a brief review on the use of transformers. The results and trends found are discussed in Section 5, and some concluding remarks are presented in Section 6.

2. Deep Neural Networks for Inverse Kinematics

This section deals with the inverse kinematics (IK) problem in its simplest form. Given a Cartesian target (a Cartesian space position, or a complete pose composed by a Cartesian space position and orientation), a Deep Neural Network (DNN) is used to learn the joint space positions that reach it, e.g., the network architecture seen in Figure 1. The following studies focus on solving the IK problem for rigid manipulators without considering obstacles. Soft robots or dynamic control may be mentioned as examples in some cases, but are not in the scope of this review.

2.1. Supervised Learning

Supervised learning is a type of machine learning algorithm designed to learn how to make predictions based on certain input parameters. The training needs a set of input data accompanied by the desired outputs. The algorithm can learn to identify patterns and correlations in the data and then predict the correct output for new data based on what it has learned [26]. This framework is ideal for the IK problem, as a set of data can be easily composed that contains the Cartesian end-effector positions for a given set of joint space positions using forward kinematics.

An example of such a dataset would be [27], which contains 10,000 input–output data pairs in which the end-effector position and orientation are the inputs and a set of joint angular positions are the outputs. A different dataset is provided in [28] for a 7-degrees-of-freedom (DOF) manipulator in both 2D and 3D space. The dataset can be generated by sampling random joint space positions and applying forward kinematics or by using motor babbling on a simulated robot or directly on the real one. Data generation and preprocessing is more extensively discussed in [29]. The effect of different datasets in the training of neural networks is studied in [30,31].

2.1.1. Feed-Forward Multilayer Perceptron

The feed-forward multilayer perceptron (MLP) is one of the most commonly used DNNs. For IK, it is a model representing a nonlinear mapping between an input vector (in this case, the desired Cartesian target for the end-effector $X_d$) and an output vector (the corresponding joint space positions $\theta$). It contains an input layer, an output layer, and several hidden layers, each of them containing a number of neurons with an activation function [32].

Feed-forward MLPs have been proved to be useful for solving the IK problem, competing with analytical and numerical methods [33], and heuristic methods such as FABRIK [34]. In [35], an MLP is compared to other neural network approaches, achieving small error values for targets where the orientation is not considered. A study on the effect of different arm lengths, datasets, hyperparameters and optimizers is conducted in [36], comparing performance in estimating IK solutions. In [37], a training dataset is virtually generated considering only the $x,y$ and z coordinates as inputs, and different hyperparameters and architectures are tested for MLP. The achieved distance error is low, but it proves that orientation information is needed for a better performance of the end-effector joint.

Because of their simplicity, computational robustness and low computational cost, feed-forward MLPs are widely used. Still, when compared to other types of DNNs such as recurrent neural networks, they are proved to be less accurate [38]. Some possibilities to reduce the error are discussed in [39,40], such as the division of the workspace in different subspaces so that a different MLP can be used for each of them. A quality function is added in [41] to detect the best solutions between all those generated for a pose in the case of a redundant robot. This idea can also be applied to manipulators with a lower number of DOF [42]. A genetic algorithm is used in [43] to improve the orientation precision of an MLP, outperforming a Generative Adversarial Network (GAN) in small workspaces.

A recent trend for feed-forward MLPs in this area are physically inspired data-driven learning methods for inverse dynamics of a robotic manipulator [44]. However, as ref. [45] shows, they cannot yet achieve the necessary precision.

2.1.2. Convolutional Neural Network

Convolutional Neural Networks (CNN) are conventionally composed by convolutional and pooling layers [46]. Fully-connected layers (also called dense layers) are typically also placed towards the output of the network. Convolutional layers are composed of input data, a filter, and a feature map. This allows the CNN to extract significant features from the input database. Pooling layers conduct downsampling operations, reducing the number of parameters in the input. Finally, any fully-connected layer assures each node in the next layer is directly connected to a node in the previous layer. CNNs are typically used for image processing and recognition tasks [47], but they can also serve other purposes such as solving IK.

In [48], a network composed by a CNN connected to shallow neural network of dense layers was tested against a simpler MLP, providing better accuracy and highlighting the importance of data scaling. Similar results are obtained in [49] for a 6-DOF industrial manipulator. A similar manipulator is used in [50], improving the precision of the CNN by tuning its hyperparameters. The network is then compared to recurrent neural networks and analytical methods with different types of noise. In general, it obtains better results than the analytical method and recurrent neural networks (RNNs) with Gated Recurrent Units but performs worse than RNN with Long-Short Term Memory units and its bidirectional version. The effect of the manipulator’s number of DOF is also studied, proving that precision is reduced when the number of DOF grows for every network. To reduce the pose error, ref. [51] proposes the use of a CNN combined with a Bilateral Long-Short Term Memory and a squeeze-and-excitation network [52].

2.1.3. Recurrent Neural Network

Recurrent neural networks (RNN) are DNN models that are capable of managing sequential data [53]. This makes them well suited for processing time-series. In this case, inputs and outputs are not independent, but can affect the way new data is processed. RNNs possess a recurrent unit that acts as a form of memory, allowing the network to learn from past inputs. Three main types of RNN units can be selected: Gated Recurrent Units (GRU), Long-Short Term Memory (LSTM) and Bilateral Long-Short Term Memory (BiLSTM). As noted previously, multiple studies prove RNNs to be more suitable than other options for the IK problem [38,50].

In [54], a RNN is used to solve the IK problem for a 6-DOF manipulator. This network is then compared to the approach followed in [55], where a Backpropagation Neural Network (BPNN) with particle swarm optimization (PSO) is used. The RNN achieves remarkable results and is proven to outperform the BPNN, even when this one uses optimization. An LSTM is used in [56] to solve IK for a 5-DOF manipulator. This type of RNN has the ability to remember long-term information and avoid the problem of gradient vanishing or explosion. The results prove it has high efficiency and satisfies the demands of real-time computing.

To reduce the complexity of the problem for high-DOF manipulators, ref. [57] builds several network architectures taking into account the dependencies between the different joints. The RNN model proves useful for this task, as it takes the coordinate of the target point as the initial state and outputs one joint space value based on the previous joint space positions at each time step. In fact, it achieves better results than an MLP and Hierarchical Neural Networks. More examples of RNNs for the IK problem can be seen in [58,59,60].

2.2. Unsupervised Learning

As was mentioned before, supervised learning is an excellent framework to deal with the IK problem, as it only requires a training dataset that can be obtained using forward kinematics or motor babbling. Nevertheless, other options exist that allow to forego this step and the costs associated to it.

2.2.1. Generative Adversarial Networks

Generative Adversarial Networks (GAN) consist of two interlinked neural networks: the generative model captures the data distribution, and the discriminative model tries to differentiate between genuine and generated data [61]. The objective of the generative model is to maximize the probability of the discriminative model making a mistake. In [62], four types of GAN are compared to the performance of a Fully Connected Neural Network for both IK and inverse dynamics, achieving similar performances in every case. The idea is to approximate the real model globally using a limited real-world dataset, which is augmented with fake data generated using GANs. As happened with other DNNs, the performance is lower when the number of DOF grows. To solve this problem, ref. [63] uses GANs to generate initial solutions to the IK problem that are later refined using an iterative numeric approach. In large workspaces, GANs have also been shown to perform better than MLP [43].

2.2.2. Autoencoders

Autoencoders (AE) are generative models composed of an encoder that learns how to extract “latent” variables from training data and a decoder that uses those variables to reconstruct the input data [64]. Different variants of this model exist, one of the most famous being Variational Autoencoders (VAE), which are extended from AEs by introducing a variational inference [65]. A goal-conditioned VAE-based kinematics model is developed in [66] for expressing the redundant forward/inverse kinematics of the class-1 tensegrity manipulator. In [67], a framework is introduced that is capable of concurrently solving both forward and inverse kinematics for diverse models and datasets using a predefined and differentiable structure for forward kinematics. The performance of a classic AE is compared to that of its variational version, showing the VAE’s greater precision. VAEs are also used in [68] to model the IK solution space and find multiple distinct joint space solutions for a given pose target.

2.2.3. Normalizing Flows

Normalizing Flows (NF) [69] represent another generative approach, which were created to solve some of the problems of GANs and AEs, such as vanishing gradients or mode collapse. This approach is followed in [70], proving computationally very efficient, especially when compared to numeric methods. It is also used in [71], providing a representative set of solutions, quickly and with acceptable error. A new variation on this approach is tested in [72] and proved to learn faster and more accurately than the previous one with the same amount of data passed to the network during the training process.

2.2.4. Graph Neural Networks

Graph Neural Networks (GNN) [73] are a type of Deep Neural Network specialized in processing data that can be represented as graphs. The main element of GNNs is the use of message passing for graph nodes to update their information by communicating with their neighbors. In the case of IK, this method presents the possibility of creating one neural network to solve the problem for every robot provided it is described as a graph. This way, the training does not depend on the manipulator’s specific parameters. This can be seen in [74,75,76]. It also allows for motion constraints to be introduced as part of the robot’s graph description. It is to be noted that this approach to IK, although promising, is still not as developed as others.

2.3. Deep Reinforcement Learning

Deep Reinforcement Learning (DRL) combines reinforcement learning (RL) [77] and deep learning. RL deals with the problem of learning through interaction by trial and error, allowing the agent to make decisions from unstructured data by trying to maximize the reward received for its actions on each state. DRL techniques aim to solve RL via DNNs and have a clear advantage over supervised learning in that they do not require a previously generated labeled training dataset, which reduces the time and computation cost. Compared to unsupervised learning, DRL can be architecturally simpler and less sensitive to hyperparameters. Additionally, they have a notion of objective via the reward signal, which can explicitly map to reaching a target and/or attempt to enforce a certain degree of collision avoidance.

DRL is used in [78] to solve the IK of a high-DOF robotic arm using the Deep Q-Network (DQN) algorithm resulting in a maximum error in end-effector position of 1 mm. In [79], Deep Deterministic Policy Gradient (DDPG) is used to solve the problem, at the same time focusing in possible issues with data privacy. A modified version of said algorithm is used in [80]. In [81] the Multi-Agent Proximal Policy Optimization (MAPPO) algorithm is successfully applied to the problem, overcoming the shortcomings of traditional methods and learning autonomously based on the environment. An actor–critic multi-agent approach is followed in [82] where each joint is a separate agent. The results are compared to standard Jacobian-based and neural network methods and proved to converge faster. DDPG, Twin Delayed DDPG (TD3) and Soft Actor–Critic (SAC) are compared in [83], with TD3 obtaining the best results. The approach followed in this reference is to integrate the manipulator’s forward kinematics model into the control framework and conceptualize the control task as a Markov decision process.

2.4. Performance Summary

In

Table 1. Relevant results for each DNN model for inverse kinematics.

Reference	Model	Year	Metric	Performance	DOF	Time (s)	Advantages	Limitations
[43]	MLP	2023	Accuracy	98.49%	8	0.243	Simple architecture.	Does not work well in large workspaces.
[50]	CNN	2023	Accuracy	94.66%	6	0.637	Extracts significant features from the input dataset.	Complexity similar to RNN but worse performance.
[50]	RNN	2023	Accuracy	95.34%	6	0.659	Deals with dependencies in time series.	More complex, higher time cost.
[63]	GAN	2021	Accuracy	88.6%	7	0.0125	Learns distribution of valid robot configurations.	Useful for initial guesses but lacks further precision.
[68]	VAE	2023	Distance	0.5 cm	7	0.0272	Models IK solution space.	Not as developed as other methods.
[75]	GNN	2022	Accuracy	90%	6	0.0097	One network for every robot.	Not as developed as other methods.
[83]	DRL	2024	Accuracy	98%	4	-	No training dataset. Simpler and less sensitive to hyperparameters.	Sim2real transfer.

, a summary of the performance of each method is presented. The examples have been chosen by the relevance of the experiments conducted, discarding the measures taken in training time. Of those remaining, one result is presented for each type of DNN mentioned, selected by the amount of relevant information provided in the corresponding reference. It is to be noted that, although they are indicative of how precise each method can be, these results cannot be interpreted as a rigorously fair comparison, as each measure has been taken under different circumstances.

3. Deep Neural Networks for Motion Control

This section deals with the problem of controlling a manipulator’s movements to reach a Cartesian target from its current position while considering different constraints. These constraints can be obstacles present in the environment, or limitations that need to be respected. In this case, the input for the DNN will be the Cartesian target and some additional information, such as the position of obstacles, as can be seen in Figure 2. With this information, the DNN will decide the best immediate movement at each given moment to bring the manipulator closer to the target while respecting the constraints. The following references are focused on rigid serial robotic manipulators; however, these techniques can also be applied to other types of robots, such as continuum manipulators [84] or robotic fish [85].

3.1. Supervised Learning

To apply supervised learning to motion control problems with obstacle avoidance, the position of the obstacles has to be known beforehand so it can be incorporated to the training dataset information. In refs. [41,86], for example, the points used for training are specifically selected by a quality function [87] so that they are close to the desired trajectory and a safe distance away from the obstacles. An MLP can be later trained with this data to control the manipulator and force it around the obstacles. The issue of data generation is further discussed in [88], where the issue of multiple solutions to the problem for redundant manipulators is avoided and obstacles are incorporated.

RNNs are the best supervised method for obstacle avoidance, as the problem can be considered a time series. Although they are often used for motion planning, motion control can also be solved with RNNs, as can be seen in [89].

3.2. Unsupervised Learning

Unsupervised learning is better suited to perform motion control while avoiding collisions, as it does not require the addition of obstacles to the training dataset. Instead, soft constraints can be integrated in the training scheme, e.g., computing the distance between joints and obstacles and adding it to the error being minimized. This is performed from a motion control perspective in [90], where an unsupervised scheme is proposed involving an MLP and auxiliary loss functions to add constraints. In [91], an AE framework for kinematic control of manipulators is proposed, and both non-sparse and sparse AE controllers are developed. In this case, however, no obstacles are included.

3.3. Deep Reinforcement Learning

As can be seen by the amount of results and ideas developed, DRL is the preferred technique when dealing with motion control and obstacle avoidance in deep learning. Firstly, it does not need pre-generated data for training, so it is faster than supervised alternatives. Additionally, most unsupervised approaches rely on formulating the problem as an optimization one and adding the obstacles to the loss function. However, it seems more intuitive and practical to add them to a reward function.

Different software tools and frameworks for DRL-based control for robotic manipulators in simulation are reviewed in [92], including simulators, application programming interfaces (APIs), libraries, and methods, and their interactions with each other.

For this problem, a ground-truth model of the environment is usually not available to the agent, meaning model-free DRL algorithms are preferred. Different algorithms have been tested to achieve joint control of a manipulator to reach a Cartesian space target while avoiding possible obstacles. Deep Deterministic Policy Gradient (DDPG) is used in [93], where it is applied to a 3-DOF manipulator to capture space objects and avoid collisions. Proximal Policy Optimization (PPO) is used in [94,95], where the distance to obstacles is considered instead of the collisions themselves. In these two cases, the importance of curriculum learning [96] to obtain better results is highlighted. Soft Actor–Critic (SAC) is used in [97], obtaining better results than DQN and DDPG, in a sim-to-real (sim2real) framework. SAC, DDPG and TD3 are compared in [98], with SAC achieving the best results again.

3.4. Performance Summary

In

Table 2. Relevant results for motion control with obstacle avoidance via DNN.

Reference	Model	Year	Control	Dynamic Environment	DOF	Orientation Considered	Advantages	Limitations
[88]	MLP	2024	Position	No	5	No	Simple architecture.	Requires dataset limited to points far from obstacles.
[89]	RNN	2023	Velocity	No	7	No	Problem can be considered a time series.	Better suited for motion planning.
[90]	Unsupervised	2023	Position	No	7	Yes	Does not need to add obstacles to dataset.	Not many examples.
[95]	DRL	2022	Both	Yes	7	Yes	Simplest way to add obstacles.	Sim2real transfer.

, a summary of the most relevant aspects of each method is presented. As discussed in the previous section, the examples have been chosen by the relevance of the experiments conducted and the amount of information provided in the corresponding reference. In the case of similar references, we chose the newest.

4. Deep Neural Networks for Motion Planning

Similarly to the previous section, the objective now is to reach a Cartesian target while avoiding collisions with possible obstacles or self-collisions. In this case, instead of deciding the best current move given the manipulator’s state and the environment information, a DNN will be used to produce the sequence of actions or trajectory that the robot will have to follow to reach the target from beginning to end, as seen in Figure 3. The following references are focused on rigid serial robotic manipulators; however, these techniques can also be applied to other types of manipulators, such as parallel robots [99].

4.1. Supervised Learning

To apply supervised learning to the motion planning problem, the obstacles need to be known beforehand so they can be added as constraints to the function that needs to be minimized. A way to achieve this is to formulate the problem as a quadratic programming one.

CNNs are not directly used for this particular case. An indirect approach uses CNN for tasks such as obstacle detection so that a cost map can be generated, while the best path is found using other methods such as RRT [100].

RNNs are the best supervised method for motion planning with obstacle avoidance via DNNs, as the problem can be considered a time series. By choosing a proper objective function to be optimized and formulating physical constraints as inequality constraints, the problem can be formulated as a quadratic programming one [101] and solved using recurrent neural networks, as is performed in [102,103]. In [104], a GRU network is used to compute the trajectory between two states avoiding obstacles.

RNNs can also be used alongside other methods to boost their performance. For example, in [105], the SAC algorithm is used to learn obstacle avoidance by reinforcement learning, with both the actor and critic networks being LSTM, which allows them to use historical information and the current state to improve the agent’s decisions.

4.2. Unsupervised Learning

Equivalently to supervised learning, the key to adding obstacles to a motion planning scheme using unsupervised DNNs relies on the problem being formulated as an optimization problem with constraints.

From the motion planning point of view, the IK problem can be formulated as an optimization problem such as [106], including collisions with the environment and self-collisions. This is directly used to update the network weights via backpropagation, finally achieving great precision in a much shorter time than the supervised approach, which needs generating data previously. The approach followed in [63] to solve IK using GANs can also accept constraints. In both approaches, the IK results are used in a larger scheme to find the best motion planning paths.

In [107], a VAE is used to learn a generative latent-variable model of a dataset containing joint space positions and end-effector positions. A loss function is then minimized that takes into account the latent representations of the target, joint positions and obstacles. VAEs are also used in [108] to capture information about the high-dimensional robotic space in a lower-dimensional manifold, so that collision avoidance solutions can be found using Topological Manifold Learning [109].

4.3. Deep Reinforcement Learning

Deep Reinforcement Learning is, again, the best framework for motion planning in the presence of obstacles. In this case, the problem does not need to be reformulated or adapted, it is enough to add the target as a positive reward and the obstacles as negative ones to compute the best path between two points.

From the motion planning perspective, different solutions have been devised: Deep Deterministic Policy Gradient (DDPG) is used in [110,111], where the reward takes into account the distance to the target, a penalty for collisions and the joint effort. PPO is used in [112], where a success reward is added, as well as an out-of-workspace penalty. This approach is proved to outperform traditional RRT methods. A SAC-based algorithm is used in [113] with the addition of the Hindsight Experience Replay (HER) technique to use the training data efficiently. A prophet-guided actor–critic (PAC) structure is designed in [114] by integrating the policies trained in other similar scenes as prophets. The DRL expert memory is expanded using Generative Adversarial Imitation Learning and the sparsity of the reward is reduced by adding an attraction reward to the goal and a repulsion reward to the obstacles.

4.4. Transformer

A transformer [115] is a deep learning architecture for transforming one sequence into another one using an encoder and a decoder. They include a parallel multi-head attention mechanism, which allows them to consider the importance of each token in the sequence. Transformers have no recurrent units, therefore requiring less training time than RNNs. Transformers have unique advantages over other DNN architectures in the motion planning scope when the task at hand is described not as set of states and goals but as spoken commands, human actions or with images. Some examples of this are [116,117,118,119].

A transformer is employed in [59] to learn how to map a sequence of end-effector poses to a sequence of joint space positions. In [120], a dataset is generated randomizing link length, configuration, and dimension of robotic manipulators with the intent of learning general inverse kinematic solutions via a generative pre-trained transformer. Transformers can also be used to learn the kinematic models of soft robotic manipulators, as can be seen in [121,122].

4.5. Performance Summary

In

Table 3. Relevant results for deep learning obstacle avoidance via motion planning.

Reference	Model	Year	Planning	Dynamic Environment	DOF	Orientation Considered	Advantages	Limitations
[102]	RNN	2021	Online	Yes	6	Yes	Problem can be considered a time series.	Problem must be reformulated.
[63]	GAN	2021	Offline	Yes	7	No	Learns a distribution of valid robot configurations.	Problem must be reformulated and added to a larger scheme.
[107]	VAE	2022	Offline	No	7	No	Models latent space of target, joint positions and obstacles.	Problem must be reformulated and added to a larger scheme.
[114]	DRL	2024	Online	No	7	Yes	Simplest way to formulate the problem.	Sim2real transfer.

, a summary of the most relevant aspects of each method is presented. As discussed in the previous section, the examples have been chosen by the relevance of the experiments conducted and the amount of information provided in the corresponding reference.

5. Discussion

In this study, we have explored the application of Deep Neural Networks (DNNs) for addressing the inverse kinematics (IK), motion control, and motion planning problems, all in the presence and absence of obstacles. These are long-standing problems, with diverse well-proved numerical and analytical solutions, but none of them are definitive.

Deep Neural Networks have gained increasing popularity in recent years, and appear to be a great tool to tackle these problems. First of all, the IK problem can be easily framed as a supervised learning task. Once that is performed, the capabilities of DNNs as function approximators should make them ideal for the task of finding a suitable solution. The problem can become harder to formulate with unsupervised learning, but, if performed correctly, the computational and time cost can be reduced greatly.

We have seen that simple DNNs, such as MLPs, can provide good accuracy in specified environments or settings. Newer architectures such as GANs, VAEs or GNNs are also being introduced and present acceptable results under certain conditions, though they still have considerable room to improve. DRL appears to be a good alternative as well, and is better suited for control and path planning around obstacles. Our research shows that studies are tending to focus on hybrid methods, using DNNs for fast initial IK guesses and then optimizing the solution through numerical methods, genetic algorithms or meta-heuristics such as PSO.

The obstacle avoidance problem is harder to pose in a supervised learning framework. The positions of the obstacles need to be known beforehand and the trained DNN cannot adapt to changes in the environment. Additionally, introducing the constraints in the training dataset is not a trivial task. This is why there are not many results in this field, with the exception of RNNs, as they deal with time series. This model is mainly used for motion planning. In the case of motion control, the best strategy would be to select points for training that are already far from obstacles and train a simpler model, such as an MLP. Unsupervised learning appears to be a convenient option. However, constraints are still difficult to add to the framework. From both the motion control and planning perspectives, the idea is to reformulate the problem as an optimization one and then add constraints. This is why Deep Reinforcement Learning shines in this environment, as obstacles can be added intuitively to the reward function to be taken into account when training. This set of techniques are mostly used for control, but can also learn to plan trajectories. The results look promising, but more progress can still be made. Additionally, there is a lack of validation tests in real robots, as the learning is usually conducted in simulated environments. More recent tools such as transformers have yet to make an impact in this field.

The majority of presented results have been achieved in simulated environments, where the structure and behavior of the manipulator and obstacles can be set by the user, as well as the physics. All models thus trained will have to face an additional issue when transitioning from simulations to real-world applications. In the first place, while high-quality 3D scene models exist, real-world scenes usually differ from those in which the robot was trained. Additionally, collecting real-world data to reconstruct scenes is time consuming. When obstacles are involved, the distances between them and the robot have to be calculated, and the transition is limited to its capacity to generalize in new scenes where objects have new positions and geometries. Secondly, real-world scenarios are unpredictable, as the physics behavior cannot be set by hand. Additionally, the manipulator may not act in an ideal way, and sensor noise and hardware limitations have to be considered. All of these factors will have to be taken into account when trying to deploy one of the methods presented in this review to real-world manipulation scenarios.

6. Conclusions

DNNs hold considerable promise for improving the computational cost and precision of long-standing solutions to the inverse kinematics (IK) and obstacle avoidance problems. Theoretically, deep learning provides an ideal framework to tackle these complex problems, given its ability to model high-dimensional, non-linear relationships, and the significant progress made in this field in the latest years. DNNs, similar to traditional numeric methods, can be implemented in a much more generic and robot-agnostic manner than analytical methods. However, they can potentially achieve faster and more accurate solutions than traditional numeric methods, which often rely on iterative processes and approximations.

Our findings prove that these expectations are well-founded, as interesting results are being achieved even with relatively simple DNN architectures. However, there is still a gap between the expectation and the practical outcomes. This is evidenced by the lack of experimental testing in real robots, with most advances being made in simulated environments. While simulation provides valuable insights, it cannot fully account for the complexities and unpredictability of real-world scenarios, such as sensor noise, environmental variations, and hardware limitations.

Further investigation is required to refine the methodologies and enhance their robustness, ensuring that DNNs can be effectively deployed in real-world robotic systems. This will involve not only optimizing network architectures and training techniques but also developing hybrid approaches that combine deep learning with traditional control theory. Additionally, testing on actual robots under a range of dynamic and challenging conditions is essential to fully understand the practical limitations and potential of deep learning for these applications. By addressing these challenges, we can unlock the full capabilities of deep learning in solving robot IK, motion control, and motion planning, paving the way for more efficient and autonomous robotic systems.