Automated Magnetic Microrobot Control: From Mathematical Modeling to Machine Learning

Abstract

Microscale robotics has emerged as a transformative field, offering unparalleled opportunities for innovation and advancement in various fields. Owing to the distinctive benefits of wireless operation and a heightened level of safety, magnetic actuation has emerged as a widely adopted technique in the field of microrobotics. However, factors such as Brownian motion, fluid dynamic flows, and various nonlinear forces introduce uncertainties in the motion of micro/nanoscale robots, making it challenging to achieve precise navigational control in complex environments. This paper presents an extensive review encompassing the trajectory from theoretical foundations of the generation and modeling of magnetic fields as well as magnetic field-actuation modeling to motion control methods of magnetic microrobots. We introduce traditional control methods and the learning-based control approaches for robotic systems at the micro/nanoscale, and then these methods are compared. Unlike the conventional navigation methods based on precise mathematical models, the learning-based control and navigation approaches can directly learn control signals for the actuation systems from data and without relying on precise models. This endows the micro/nanorobots with high adaptability to dynamic and complex environments whose models are difficult/impossible to obtain. We hope that this review can provide insights and guidance for researchers interested in automated magnetic microrobot control.

1. Introduction

Microrobots, characterized by their small size, remote wireless drivability, and adaptability to confined spaces, have shown widespread applications in fields such as biomedicine, healthcare, and micro-manipulation [1,2,3,4,5,6,7]. These miniature machines, with dimensions ranging from hundreds of nanometers to micrometers, possess unique advantages over conventional robots, including enhanced maneuverability and accessibility in confined spaces, especially in complex and narrow regions within the human body, such as the gastrointestinal (GI) tract [8,9], vasculature [10,11], brain [12,13], and eye [14,15]. Especially in the biomedical field, in contrast to conventional surgical interventions or non-targeted chemical and radiation therapies, their small size enables the use of far less invasive procedures. This remarkable capability significantly facilitates the diagnosis and detection of diseases, while also mitigating the risks of side effects, complications, infection, and prolonged recovery times for patients [16].

Due to the small size of microrobots, self-contained power sources pose a significant challenge. Consequently, microrobots necessitate external control and propulsion mechanisms. Existing propulsion methods can be classified into self-propulsion and external field propulsion based on their driving mechanisms. Self-propulsion refers to the ability of robots to obtain energy from the surrounding fluid, such as self-electrophoresis and chemical fuel propulsion [17,18,19,20]. External field propulsion, on the other hand, requires the robots to obtain driving energy only under the influence of an external field. This category includes electric field propulsion [21], optical field propulsion [22], acoustic field propulsion [23], thermal propulsion [24], magnetic field propulsion [25], and various hybrid field propulsion methods. Among these, magnetic field-driven microrobots have gained significant attention worldwide due to the following advantages [26,27]: (1) the use of low-frequency and low-intensity magnetic fields enables these microscale devices to navigate through biological tissues without causing harm to living organisms; (2) the reconfigurable and programmable nature of magnetic materials grants a high degree of control and flexibility in the design and operation of magnetic microrobots; (3) magnetic fields provide a non-invasive control mechanism for micro/nanorobots, eliminating the need for onboard fuel consumption; (4) the ability of magnetic fields to generate substantial torques and forces at the microscale facilitates precise 2D and 3D motion and orientation control of these miniature systems. Additionally, the direction, density, and gradient of magnetic fields can be precisely controlled with the assistance of electromagnetic equipment.

One notable example of the impact of magnetic microrobots is their use in precise delivery of cells. Wei et al. [27] developed a degradable, magnet-driven microrobot for the precise delivery of stem cells to liver tumors. This microrobot, designed with a porous sphere structure, ensures degradability, mechanical strength, and magnetic control capability. Guided by photoacoustic imaging and actuated by a magnetic field, it was tested on mice, showing significant tumor growth inhibition after cell release. This study is the first to use degradable microrobots for targeted cell delivery in vascular tissues, demonstrating promising preclinical results. In another application, magnetic slippery micropropellers have been used to perform precise intravitreal delivery of therapeutic agents to the retina [28]. These helical magnetic microrobots, coated to minimize adhesion, can actively navigate through the vitreous humor, overcoming its dense macromolecular matrix. Clinical optical coherence tomography confirmed their successful arrival at the retina. This innovation promises targeted and efficient ocular treatments. Therefore, magnetic field-driven microrobots hold great significance in biomedical applications [16,27]. In this review, our attention is directed toward the motion control of microrobots using magnetic actuation.

These magnetic actuated miniature microrobots typically comprise biocompatible materials embedded with magnetic nanoparticles [29], allowing for controlled manipulation under the influence of external magnetic fields [30,31,32,33,34]. Central to the operation of magnetic microrobots are magnetic control systems capable of generating precise and tailored magnetic fields. Electromagnets or permanent magnets serve as the primary components of these systems, providing the necessary magnetic flux density to manipulate microrobots with precision [35]. Researchers have employed diverse magnetic actuation systems to accommodate varying workspace sizes and degrees of freedom [36].

To make magnetic microrobots automatically and precisely execute tasks, a spectrum of motion control strategies has been implemented, progressing from conventional approaches to those rooted in learning-based methodologies [34]. Traditional control techniques such as linear control and nonlinear control offer stability, adaptability, and robustness in controlling microrobot behavior in dynamic environments [37,38]. Additionally, emerging learning-based control approaches, such as deep learning and reinforcement learning, enable microrobots to learn optimal control policies through trial-and-error interactions with their environment [39,40,41]. These learning-based approaches offer adaptability and autonomy, enhancing microrobot performance in complex tasks.

In this review, we aim to provide a comprehensive overview of automated control methodologies for magnetic microrobots, spanning from theoretical foundations to recent advancements in learning-based control approaches. The subsequent sections of this paper are organized as follows:

Section 2 will formulate the theoretical underpinnings of magnetic microrobot control, exploring the generation and modeling of magnetic fields, as well as the magnetic field-actuation modeling of magnetic microrobots. Building upon the theoretical foundation, Section 3 will explore various motion control methods, including traditional control techniques as well as more recent advances in learning-based control approaches. Finally, Section 4 will synthesize the key findings from the preceding sections and discuss future directions for research in this field.

2. Theoretical Foundations

2.1. Generation of Magnetic Fields

Different types of magnetic fields can be generated using permanent magnets and electromagnetic coils [35].

In terms of permanent magnets, when the distance between the source and microrobots greatly exceeds the dimensions of the magnetic source, a dipole model becomes applicable for delineating the distribution of a magnetic field. Therefore the magnetic strength acting on a microrobot is determined by Equation (1) [42]:

$$\mathit{B}=\left(\frac{{\mu }_0}{4\pi {\mathit{r}}^5}\left(3\mathit{r}{\mathit{r}}^T-{\mathit{r}}^2\mathbb{I}\right)\right)V\mathit{M}$$

(1)

where B represents the magnetic field produced by the dipole magnet; ${\mu }_0$ denotes the permeability of air; $\mathbb{I}$ stands for the identity matrix; V is volume; M represents the magnetic moment of the source, consistently positioned perpendicular to the rotation axis; and r represents the vector from the source to the microrobots. As per the Equation above, magnetic intensity diminishes cubically as distance increases, correlating with V and M. Once the magnetic source is chosen, adjustments to its orientation and position enable the creation of a desired magnetic flux density at the microrobot’s location.

In contrast to magnetic fields produced by electromagnet coils, a permanent magnet offers energy-efficient and remarkably effective means for manipulating small entities. Figure 1a shows the magnetic field of a permanent magnet. Furthermore, it occupies a smaller footprint than electromagnetic coils while generating equivalent magnetic field intensities [1]. However, a significant drawback of permanent magnet-based approaches is their inability to deactivate magnetic strength after operation, which could pose potential safety hazards. Additionally, changes in magnetic field direction within these systems are achieved mechanically, resulting in continuous changes and mechanical noise. Therefore, it is unfeasible to generate a discontinuous magnetic field [36], like a square wave oscillating field.

Electromagnetic coils offer an alternative method for generating adjustable magnetic fields. Unlike permanent magnets, they allow for adjustments in the strength of the magnetic field without requiring the relocation of magnetic sources, offering a highly manageable approach for magnetic actuation [36]. However, electromagnetic coils typically yield lower magnetic force and torque compared with permanent magnets. Strategies to enhance force and torque output involve increasing current or coil quantity, yet these approaches may escalate heat generation and energy consumption concerns. For electromagnets, their coils generate spatially varying magnetic fields that act on magnetic particles within microrobots, enabling multi-degree-of-freedom motion. The configuration and parameters of these coils determine magnetic field characteristics such as strength, directionality, and gradient, with precise control being pivotal for achieving desired microrobot motions and interactions. In general, the magnetic field produced by a cylindrical coil can be calculated using the Biot–Savart Law, which is related to the current density I, the number of coil layers N, the unit vector of the coil l, and the height of the coil h [42], as shown in Figure 1b [43]:

$$\mathit{B}=\frac{N{\mu }_{0}I}{4\pi }{\int }_h{\int }_{\mathit{l}}\frac{d\mathit{l}\times \mathit{r}}{{\mathit{r}}^3}dh$$

(2)

According to Equation (2), once an electromagnetic system is constructed with a set size and a fix number of coils, the strength of a magnetic field at a specific point relies on both the alignment of the cylindrical coil and the current flowing through the coils [42,44,45].

2.2. Modeling of Magnetic Fields

Due to the non-adjustable nature of permanent magnet fields, achieving controlled propulsion for microrobots may necessitate more intricate methods for controlling the position of permanent magnets. Therefore, electromagnetic coils are often employed to achieve controlled motion in microrobots, enabling magnetic field control by altering the current flowing through the coils.

Common electromagnetic system setups include the Helmholtz coil as well as the Maxwell coil, which, respectively, produce uniform and gradient fields [42,46]. Helmholtz coils are a special type of coil arrangement capable of generating a uniform magnetic field at the center of the coil. They consist of one pair of coaxially placed coils, with the coil distance h matching the coil radius a, and the currents flowing in the same direction [47]. The magnetic field within the Helmholtz coil is uniformly distributed near its axial midpoint, as illustrated in Figure 2a. When the coil distance h is $\sqrt{3}$ times the coil radius a, and both coils carry currents in opposite directions, this configuration is known as the Maxwell coil, it is capable of maximizing the space between coils and producing a gradient uniformly distributed magnetic field [47]. The magnetic field within the Maxwell coil exhibits a gradient distribution near its axial midpoint, as depicted in Figure 2b.

For these two single-axis coils, the magnetic induction intensity at any point P in space is generated by two energized coils: one in the negative half-axis and the other in the positive half-axis. Equation (3) describes the magnetic field produced by the single-axis Helmholtz coil in the x-direction. According to Equation (3), it can be inferred that the magnetic induction strength generated by a single-axis Helmholtz coil in a uniform area increases proportionally with the applied electric current [47].

$${\mathit{B}}_{HX}(x)={\left(\frac{4}{5}\right)}^{\frac{3}{2}}\frac{{\mu }_{0}NI}{a}$$

(3)

where a is the average radius of the coil and I is the coil input current. Three pairs of orthogonally nested Helmholtz coils can generate a three-dimensional uniform magnetic field within their working area, as depicted in Figure 2d [49]. This setup is capable of creating a magnetic field that rotates uniformly. This magnetic field is represented as Bn and surrounds any axis n within the three-dimensional space. This magnetic field is produced through the modulation of currents flowing within the coils, which is related to the magnetic flux density at the central point of the Helmholtz coils B₀ and the rotational frequency f [1]:

$${\mathit{B}}_n(t)=B_0\mathit{cos}\left(2\pi ft\right)\mathit{u}+B_0\mathit{cos}\left(2\pi ft\right)\mathit{v}$$

(4)

where the vectors (u, v) serve as the fundamental vectors for the plane that is perpendicular to the axis n. By controlling the magnitude, direction, and frequency of the currents in each coil pair, various types of three-dimensional magnetic fields such as rotating, conical, oscillating, and square-wave switching fields can be produced [51,52,53,54,55].

Equation (5) describes the magnetic field produced by the single-axis Maxwell coil in the x-direction.

$${\mathit{{\rm B}}}_{MX}\left(x\right)=\frac{16}{3}{\left(\frac{3}{7}\right)}^{\frac{5}{2}}\frac{{\mu }_{0}NI}{a^2}x=g_{m}x$$

(5)

According to Equation (5), it can be observed that a single-axis Maxwell coil generates a certain uniform gradient magnetic field along the x-axis direction at x = 0 [56]. Similarly, a single-axis Maxwell coil also generates a certain uniform gradient magnetic field along the y- and z-axis directions, with the gradient magnitude being half of that along the x-axis, but in the opposite direction. The magnetic field produced by a single-axis Maxwell coil along the x-axis direction can be expressed by Equation (6):

$${\mathit{{\rm B}}}_M={\left[\begin{array}{ccc}{g}_{m}x& -0.5g_{m}y& -0.5g_{m}z\end{array}\right]}^T$$

(6)

To achieve controllable motion of microrobots under the influence of a magnetic field, independent, controllable, and sufficiently strong and gradient three-dimensional spatial magnetic fields are required. A three-dimensional coil model can be formed by combining Helmholtz coils and Maxwell coils. In the immediate vicinity of the center of all coils, the Helmholtz coils are capable of creating a consistent rotating magnetic field, while the Maxwell coils can produce a uniform gradient magnetic field within the specified range [47,57]. To enhance the accuracy of magnetic field generation for micro/nanorobot motion control, Fan et al. [48] developed a new drive circuit using a bipolar linear power amplifier. They also introduced an advanced control strategy that combines a neural network algorithm and a proportional resonant differential feedforward method. This approach optimizes control parameters and addresses coil temperature disturbances. Experiments on Maxwell and Helmholtz coils (Figure 2c,f) showed the scheme’s adaptability and excellent performance in both dynamic and steady-state conditions. Therefore, these innovations in magnetic field generation have significantly advanced microrobot control, offering improved precision and responsiveness. In addition, researchers have been working on optimizing electromagnetic systems to generate stronger magnetic fields within confined spaces. These coils enable the precise manipulation of microrobots with higher levels of control and efficiency. For example, Gervasoni et al. [58] reported a portable electromagnetic navigation system “Navion” for minimally invasive surgery. Designed for seamless integration into clinical workflows, Navion generates magnetic fields across diverse spatial directions, ranging from 5 to 50 mT, with gradients of 250 mT/m and rotating fields up to 10 Hz. Utilizing a triangular arrangement of high-current electromagnets, Navion preserves fluoroscope image quality for neurovascular procedures.

2.3. Magnetic Field-Actuation Modeling of Microrobot

2.3.1. Magnetic Actuation Principle

For a magnetic object, its overall average magnetization strength can be represented by the total dipole moment M. The magnetic force F_m and torque ${\mathit{\tau }}_m$ experienced by the magnetic object in a magnetic field with flux density B can be represented as follows

$${\mathit{F}}_m=\left(\mathit{M}\cdot \nabla \right)\mathit{B}$$

(7)

$${\mathit{\tau }}_m=\mathit{M}\times \mathit{B}$$

(8)

From Equation (7), it can be inferred that magnetic force only exists when there is a gradient in the magnetic field. Similarly, from Equation (8), it can be deduced that magnetic torque only exists when there is a relative angle between M and B.

According to Equation (7), the gradient magnetic field can generate a magnetic force that drives micro-nanorobots to move in the direction of increasing field strength. Therefore, in a gradient magnetic field, due to variations in the magnetic flux density along a specific direction, magnetic objects experience a magnetic force that propels them so that move along the gradient direction. Based on this mechanism, researchers propose utilizing a combination of Maxwell coil or Gradient Saddle Coil to construct a magnetic field generation device, driving magnetic microrobots to achieve gradient propulsion [59]. However, controlling robot motion using gradient magnetic fields is challenging. If the magnetic force is too weak, friction or viscous forces dominate, preventing any movement. Conversely, if the magnetic force overcomes friction and viscosity, the robot may undergo high acceleration, resulting in unpredictable behavior.

According to Equation (8), when the magnetic microrobot is oriented at an angle relative to the external magnetic field, the microrobot experiences torque under the action of the uniform magnetic field, leading to rotation. Therefore, a magnetic object experiences no net magnetic force in a uniform magnetic field; it is capable of attaining the anticipated rotational movement under the control of the external magnetic field. Based on this mechanism, researchers propose using a Helmholtz coil or a Uniform Saddle Coil to construct a magnetic field generation device for controlling microrobots to achieve directional motion [60,61]. However, when the orientation of the magnetic dipole moment of the magnetic object is not aligned with the external magnetic field, the object undergoes a magnetic torque. It rotates until the magnetic dipole moment aligns with the external field. Therefore, in a uniform magnetic field, only time-varying magnetic fields, such as rotating, oscillating, or pulsed magnetic fields, can provide controlled magnetic torques for driving microrobots. However, in low Reynolds number environments, inertia can be neglected, resulting in the symmetric reciprocating motion of the robot. To achieve directed motion, specialized robot shapes and gaits are necessary.

In addition, utilizing oscillating magnetic fields can also induce motion in microrobots, facilitating the aggregation of magnetic particles into various configurations such as strips or chains. However, compared with rotating magnetic fields, the application scope of oscillating magnetic fields is relatively limited, and they are less convenient for controlling robot motion.

When multiple microrobots coexist in the workspace in close proximity, their interactions can profoundly influence their behaviors. Due to the substantially higher amplitude of the external magnetic field compared with the local field produced by the microrobots under control, there is a tendency for these devices to orient itself with the external magnetic field. Nonetheless, owing to the close proximity of neighboring microrobots, the local interaction force cannot be disregarded [62,63]. The magnetic interaction force between two controlled devices is represented by Equation (9) [42],

$$\mathit{F}=-\frac{{\mu }_0{\mathit{M}}_1{\mathit{M}}_2}{4\pi }\nabla \left(\frac{1-3{\mathit{cos}}^2\theta }{{‖\mathit{r}‖}^3}\right)$$

(9)

where M₁ and M₂ are the magnetic moments of the field-generating unit and the force-receiving unit, respectively, and θ is the angle formed between the direction of the global magnetic field and the line connecting the two controlled devices.

2.3.2. Magnetic Actuation System

The magnetic drive system can be roughly divided into two categories: stationary magnetic actuation systems and mobile large-space magnetic actuation systems [36]. Among them, the stationary magnetic actuation system includes systems driven by conventional electromagnetic coils and non-conventional electromagnetic coils (Helmholtz coils and Maxwell coils), while the mobile magnetic actuation system includes systems driven by mobile permanent magnets and mobile electromagnetic coils. The characteristics comparison of different electromagnetic coil systems is illustrated in

Table 1. Classification of magnetic actuation systems.

Types	Composition Structure	Characteristics
Stationary	Conventional electromagnetic coils	The magnetic field is confined to a specific small area within a limited space, and optimization is not achieved in terms of energy consumption, magnetic field intensity, and distribution density.
	Helmholtz coils and Maxwell coils	Maximize the workspace where uniform distribution of magnetic field strength or gradient can be achieved.
Mobile	Mobile permanent magnets	Generate a relatively large magnetic field, but it cannot quickly open or close the magnetic field in the workspace or generate high-frequency periodic changes in the magnetic field.
	Mobile electromagnetic coils.	Quickly generate, modify, or turn off magnetic fields within the workspace while activating the moving device. But the maximum achievable magnetic field strength and gradient intensity are relatively small and have limitations.

.

Electromagnetic coils in a stationary space can be further divided into four subclasses: ordinary electromagnetic coils, Helmholtz coils, Maxwell coils, and combinations of multiple types of electromagnetic coils. The normal electromagnetic coil system is a type of electromagnetic system designed and customized according to specific requirements. It consists of one or multiple coils and iron cores, capable of generating the required gradient magnetic field and/or uniform magnetic field within a specific area in space. While extensively studied [64,65,66,67,68,69,70,71,72], these normal electromagnetic coil systems are constructed simply by combining coil sets or iron-core coils, lacking optimization in workspace size, energy consumption, and magnetic field distribution. Therefore, Helmholtz coils and Maxwell coils have been widely utilized. Both are optimized coil arrangements capable of maximizing the workspace for achieving either uniformly distributed magnetic fields or gradient distributed magnetic fields. Zhang et al. [73] achieved spiral propulsion of artificial bacterial flagella microrobots by rotating a magnetic field with three orthogonal Helmholtz coil pairs. Du et al. [74] utilized three-dimensional Helmholtz coils to generate diverse magnetic fields, enabling various motion modes for membrane microrobots with magnetic heads and soft tails, including rolling, swinging, crawling, and spiral propulsion. Combining various coil configurations like ordinary coils, Helmholtz coils, and Maxwell coils enables the generation of a wider range of magnetic field types, offering increased versatility. Choi et al. [56] integrated two pairs of nested Helmholtz coils (inner) and two pairs of Maxwell coils (outer) orthogonally to create an electromagnetic coil system, facilitating two-dimensional steering and the translational motion control of microrobots through current magnitude adjustment. The group of Ferreira [50,75] proposed an electromagnetic coil arrangement that includes three sets of Helmholtz coils and one set of Maxwell coils to achieve comprehensive three-dimensional motion control. Future advancements may extend their capabilities to generate uniform or gradient magnetic fields in larger spaces, alongside diverse magnetic field types with increased degrees of freedom. These characteristics of electromagnetic coils in compact static environments are pivotal for advancing the widespread adoption of magnetically driven microrobots.

The magnetically driven system in mobile spaces consists of mobile devices such as mechanical arms, electric translation stages, etc., coupled with electromagnetic coils or permanent magnets, and can be classified into mobile permanent magnet systems and mobile electromagnetic systems. Permanent magnets offer stable, power-free magnetic field generation, ideal for multi-degree-of-freedom robotic arm manipulation across larger spatial domains. Mahoney et al. [76] demonstrated this with a 6-degree-of-freedom robotic arm featuring a permanent magnet at the end effector, enabling precise magnetic field control within its reach. Kim et al. [77] and Pittiglio et al. [78] employed similar setups for navigating magnetic guidewires and soft catheters in vascular models, respectively. Zhang et al. [79] utilized vertically symmetric three-dimensional electric translation devices for magnetic guidewire control. However, while permanent magnet systems offer larger magnetic fields, they lack rapid field switching capabilities. In contrast, mobile electromagnetic systems, like ARMM by Sikorski et al. [80] and DeltaMag by Yang et al. [81], provide flexible control over magnetic field direction, gradient, and intensity in large spaces. In addition, Yang et al. [82] utilized mobile ultrasound tracking and magnetic control to enable the long-distance navigation of miniature robots within the human body, overcoming limitations of two-dimensional ultrasound imaging and magnetic field decay. This scheme integrates ultrasound probes with electromagnetic coils for simultaneous tracking and control, enabling robust navigation in pulsatile flow conditions and generating rotating fields for helical propulsion based on real-time tracking feedback. Although electromagnetic systems can swiftly alter fields, they have limitations in maximum strength and gradient due to coil geometry and current constraints. Future research must focus on magnetically driven systems capable of programmable magnetic fields in larger spaces, particularly for biomedical applications.

2.4. Dynamics Modeling of Microrobots

In addition to magnetic field-driven models, microrobots are subject to various physical constraints and environmental factors that influence their motion. Dynamic models are required to represent the interactions between microrobots and their surroundings. These models serve as valuable tools for motion planning, control algorithm design, and system optimization.

In the movement of magnetically driven microrobots, besides being influenced by the driving magnetic force F_m, the microrobots also experience their own weight F_g and liquid buoyancy F_b in the vertical direction within the liquid environment. Additionally, in the opposite direction of motion, there is viscous resistance F_d from the liquid acting on the microrobots [83,84,85], as shown in Figure 1c. The dynamic equation of the microrobot is shown in Equation (10):

$${\mathit{F}}_m+{\mathit{F}}_d+{\mathit{F}}_g+{\mathit{F}}_b=m\mathit{a}=m\ddot{\mathit{p}}$$

(10)

The magnetic force F_m is the force acting on the microrobot due to the external magnetic field generated by the superposition of several pairs of coils.

Let B_m(P) denote the magnetic field produced by the m-th electromagnetic coil at any given point P within the spatial domain, with the coil carrying a current variable i_m. Assuming that a current of 1A flowing through the coil generates a magnetic field ${\overline{\mathit{B}}}_m\left(\mathit{P}\right)$, the magnetic field produced by the combination of these coils can be represented as follows [86]:

$$\mathit{B}(\mathit{P})=\sum _{m=1}^n{\mathit{B}}_m(\mathit{P})=\sum _{m=1}^n{\overline{\mathit{B}}}_m\left(\mathit{P}\right)i_m=\left[\begin{array}{ccc}{\overline{\mathit{B}}}_1\left(\mathit{P}\right)& \cdots & {\overline{\mathit{B}}}_n\left(\mathit{P}\right)\end{array}\right]\left[\begin{array}{c}{i}_1\\ ⋮\\ i_n\end{array}\right]=\beta (\mathit{P})\mathit{I}$$

(11)

Three components of magnetic field gradient are as follows:

$$\left\{\begin{array}{c}\frac{\partial \mathit{B}(\mathit{P})}{\partial x}=\left[\begin{array}{ccc}\frac{\partial {\overline{\mathit{B}}}_1(\mathit{P})}{\partial x}& \cdots & \frac{\partial {\overline{\mathit{B}}}_n(\mathit{P})}{\partial x}\end{array}\right]\left[\begin{array}{c}{i}_1\\ ⋮\\ i_n\end{array}\right]={\beta }_x(\mathit{P})\mathit{I}\\ \frac{\partial \mathit{B}(\mathit{P})}{\partial y}=\left[\begin{array}{ccc}\frac{\partial {\overline{\mathit{B}}}_1(\mathit{P})}{\partial y}& \cdots & \frac{\partial {\overline{\mathit{B}}}_n(\mathit{P})}{\partial y}\end{array}\right]\left[\begin{array}{c}{i}_1\\ ⋮\\ i_n\end{array}\right]={\beta }_y(\mathit{P})\mathit{I}\\ \frac{\partial \mathit{B}(\mathit{P})}{\partial z}=\left[\begin{array}{ccc}\frac{\partial {\overline{\mathit{B}}}_1(\mathit{P})}{\partial z}& \cdots & \frac{\partial {\overline{\mathit{B}}}_n(\mathit{P})}{\partial z}\end{array}\right]\left[\begin{array}{c}{i}_1\\ ⋮\\ i_n\end{array}\right]={\beta }_z(\mathit{P})\mathit{I}\end{array}\right.$$

(12)

Combining Equations (7) and (8) allows us to calculate the torque and magnetic force acting on the microrobot within the magnetic field [87]:

$$\left[\begin{array}{c}{\mathit{\tau }}_m\\ {\mathit{F}}_m\end{array}\right]=\left[\begin{array}{c}Sk(\mathit{m})\beta (\mathit{P})\\ {\mathit{m}}^T{\beta }_x(\mathit{P})\\ {\mathit{m}}^T{\beta }_y(\mathit{P})\\ {\mathit{m}}^T{\beta }_z(\mathit{P})\end{array}\right]\left[\begin{array}{c}{i}_1\\ ⋮\\ i_n\end{array}\right]={\mathit{\Lambda }}_{T,F}(m,P)\mathit{I}$$

(13)

The gravitational and buoyant forces acting on the microrobot in the vertical direction can be expressed as Equations (14):

$${\mathit{F}}_g+{\mathit{F}}_b=V\left(\rho -{\rho }_f\right)\mathit{g}$$

(14)

where V corresponds to the volume of the microrobot, and ρ and ρ_f denote the densities of the microrobot and the surrounding liquid, respectively. The viscous drag experienced by the microrobot in the liquid is primarily dependent on the microrobot’s velocity v, the density of the liquid ρ_f, the cross-sectional area A of the microrobot, and the drag coefficient C_d [88]. For a microrobot with a spherical diameter D and a fluid dynamic viscosity of μ_f, the viscous resistance experienced can be calculated using the following formula:

$${\mathit{F}}_d=3\pi {\mu }_{f}D\mathit{v}$$

(15)

The dynamic equation of the microrobot (10) can be expressed in the following form:

$$\left[\begin{array}{c}{F}_{mx}\\ F_{my}\\ F_{mz}\end{array}\right]+\left[\begin{array}{ccc}-k_x& 0& 0\\ 0& -k_y& 0\\ 0& 0& -k_z\end{array}\right]\left[\begin{array}{c}{\dot{\mathit{P}}}_x\\ {\dot{\mathit{P}}}_y\\ {\dot{\mathit{P}}}_z\end{array}\right]+\left[\begin{array}{c}0\\ 0\\ -V(\rho -{\rho }_f)\mathit{g}\end{array}\right]=m\left[\begin{array}{c}{\ddot{\mathit{P}}}_x\\ {\ddot{\mathit{P}}}_y\\ {\ddot{\mathit{P}}}_z\end{array}\right]$$

(16)

where, k_x, k_y, and k_z are constant coefficients calculated based on the shape of the microrobot.

3. Motion Control Methods

Achieving precise and adaptive control of magnetic microrobots requires the implementation of advanced motion control strategies. This section explores traditional control paradigms as well as emerging learning-based approaches, discussing their principles, advantages, and limitations.

3.1. Traditional Control

Early studies on motion control of magnetic-driven microrobots primarily employed open-loop control [73,89]. However, with a deeper understanding of the motion of helical microrobots, it was realized that under external disturbances and modeling uncertainties, open-loop control exhibited instability and limited precision. Consequently, researchers shifted their focus towards closed-loop control [34]. Traditional control methods encompass a range of techniques, including linear control, nonlinear control, observer-based control, fuzzy control, and model predictive control (MPC), etc.

Linear control techniques, exemplified by the widely used Proportional–Integral–Derivative (PID) control, are valued for their simplicity and stability in regulating microrobot motion. For instance, Li et al. [90] employed a visual-based PID controller to stabilize the motion of magnetic microrobots during target tracking tasks in an ex vivo scenario. Similarly, Tang et al. [91] utilized a micro visual-based PID controller combined with a Kalman filter to achieve robust target tracking, and employed the Dynamic Window Approach for intelligent obstacle avoidance control of spherical microrobots driven by rotating magnetic fields. Pawashe et al. [92] successfully implemented visual servoing for a magnetic microrobot by employing a PI controller tailored for diverse micro-manipulation tasks. Xu et al. [37] devised a dual closed-loop path-following PI controller system. They defined reference paths and error states in collected images and experimentally verified path-following and obstacle avoidance for various trajectories (Figure 3a). Subsequently, addressing the issue of manually setting reference paths, they employed a PID controller along with the Rapidly-exploring Random Tree (RRT) path planning algorithm to automatically compute the shortest path. In addition, Ryan et al. [93] constructed a magnetic drive platform utilizing permanent magnets, aimed at maximizing magnetic field strength and force. They optimized the spatial arrangement of rotating permanent magnets and achieved three-dimensional path tracking under the influence of a PID controller. However, these studies are based on simplified kinematic or dynamic models, with insufficient consideration for external disturbances, posing a challenge to the adaptability of the controller. Therefore, observer-based controllers are proposed, which utilizes sensor feedback to estimate system states and compensate for uncertainties, enhancing control performance in dynamic environments. Khalil et al. [94] introduced inner loop control using external disturbance observers to propel magnetic particles with gradient magnetic fields from electromagnetic coils, analyzing system dynamics. In contrast, Yang et al. [38] proposed model-free path following control, treating microrobots dynamics and external disturbances as generalized perturbations and utilizing extended state observer for position and velocity estimation. A linear trajectory tracking controller was designed using the estimated states. Meanwhile, implementing a visual servoing control system, they conducted real-time experiments, demonstrating high-precision tracking for TPMM samples with varying body lengths and fluids. (Figure 3b).

However, the intricate nature of microrobots, marked by pronounced nonlinearity and uncertainty [95], presents formidable obstacles for linear control methods. Factors such as Brownian motion [40,96,97], external fluid forces [98,99], and inaccuracies in actuation systems [61] collectively undermine the effectiveness of traditional linear controllers like PID in addressing certain control tasks. To overcome these limitations, nonlinear control approaches have emerged, such as sliding mode control [99], offering enhanced adaptability to nonlinear behaviors and disturbances, thereby bolstering the performance and robustness of microrobot control systems. Marino et al. [100] demonstrated the application of an H_∞ controller to address force and localization uncertainties in microrobot control (Figure 3c). Compared with a PID controller, the H∞ controller exhibited superior performance under higher uncertainties, as evidenced by simulations and experiments conducted on a prototype electromagnetic control system. Researchers have explored various nonlinear control strategies beyond adaptive and robust control, aiming to achieve precise manipulation of microrobots. Zhang et al. [101] employed a position-based impedance controller with a dual-axial interaction force determination mechanism, utilizing magnetic flux measurement instead of onboard force sensors, to actively track the reference force of a magnetically navigated microrobot in uncertain environments. Additionally, an adaptive control algorithm minimized force tracking errors, demonstrating potential applications in biomedical microsurgery. Dong et al. [102] utilized predictive proportional control to reproduce major crawling behaviors of a genetically engineered nematode worm, achieving highly controllable locomotion with real-time visual feedback (Figure 3d). This approach enables the closed-loop regulation of movement direction and destination, facilitating scientific studies on crawling locomotion and neural basis in nematode species. Liu et al. [103] proposed a proxy-based sliding mode control (PSMC) method for creating stable controllers tailored to magnetically driven helical microswimmers in three-dimensional autonomous manipulation tasks, ensuring precise path following despite weight and lateral disturbances (Figure 3e). The PSMC strategy utilizes an online updating scheme to enhance the adjustment of compensatory angles between the magnetic field direction and the swimming direction, enabling submillimeter accuracy in 3D space. Zheng et al. [104] developed a prescribed performance controller with disturbance observer to ensure the continuous visual servoing of the microrobot within the view of microscopic cameras. This controller guarantees satisfactory transient and steady-state performance, enabling automatic three-dimensional navigation in complex microenvironments and reliable field-of-view tracking at high speeds.

In addition, optimization-based control methods offer significant advantages. These algorithms ensure system output tracking to reference signals while minimizing input efforts, making them particularly effective in the closed-loop manipulation of microrobots, even in the presence of disturbances and uncertainties. Optimization-based control strategies encompass a variety of techniques such as linear quadratic regulation (LQR) and model predictive control (MPC). For instance, MPC considers a predictive model of the system and optimizes control actions over a finite time horizon to minimize a cost function. Pieters et al. [105] employed an MPC approach for the precise motion control of a nonholonomic rolling microrobot, enabling it to follow a rotating magnetic field and navigate obstacles (Figure 3f). Yang et al. [106] utilized a robust model predictive trajectory-tracking controller, integrated with particle swarm optimization-based path planning, to automate the precise targeted delivery of multifunctional magnetic spores (Mag-Spores) under fluorescence imaging guidance. This approach enhances recognition, tracking, and obstacle avoidance, ensuring efficient delivery in complex environments. Xu et al. [107] introduced an independent manipulation approach for magnetic-driven helical miniswimmers, incorporating an LQR controller to enable precise path following and directional control in the horizontal plane, validated through simulations and experiments with submillimeter accuracy.

Figure 3. Traditional control: (a) Image-based PI controller, reproduced with permission from Xu et al. [37]; (b) Obstacle avoidance task based on PI controller, the tracked TPMM sample is marked in the dashed rectangular, the yellow line is the reference trajectory, the blue line is the actual trajectory, reproduced with permission from Yang et al. [38]; (c) H_∞ controller, the gray circle is microrobot, the blue point is the reference trajectory, the red line is the actual trajectory, reproduced with permission from Marino et al. [100]; (d) Three paths under Predictive proportional controller, reproduced with permission from Dong et al. [102]; (e) Proxy-based sliding mode controller, reproduced with permission from Liu et al. [103]; (f) MPC controller, reproduced with permission from Pieters et al. [105].

Figure 3. Traditional control: (a) Image-based PI controller, reproduced with permission from Xu et al. [37]; (b) Obstacle avoidance task based on PI controller, the tracked TPMM sample is marked in the dashed rectangular, the yellow line is the reference trajectory, the blue line is the actual trajectory, reproduced with permission from Yang et al. [38]; (c) H_∞ controller, the gray circle is microrobot, the blue point is the reference trajectory, the red line is the actual trajectory, reproduced with permission from Marino et al. [100]; (d) Three paths under Predictive proportional controller, reproduced with permission from Dong et al. [102]; (e) Proxy-based sliding mode controller, reproduced with permission from Liu et al. [103]; (f) MPC controller, reproduced with permission from Pieters et al. [105].

3.2. Learning-Based Control

While advancements have been made in path tracking, further research is needed to enhance control robustness. In recent years, learning-based control methods have gained traction in microrobotics, leveraging techniques such as deep learning and reinforcement learning. Wu et al. [108] utilized Radial Basis Function (RBF) neural network methods to adaptively compensate for disturbances caused by gravity and external perturbations in three-dimensional space (Figure 4a). They achieved this by fitting mapping models to offline-collected data. In addition to realizing three-dimensional trajectory tracking, they proposed online updatable models to address the limitations of offline model fitting precision and generalization capabilities. This enabled automated trajectory planning, tracking, and obstacle avoidance [103]. For magnetotactic bacteria, Ou et al. [109] employed the least squares method to establish a discrete motion model, and subsequently designed a model predictive controller based on image feedback to track the trajectory of magnetotactic bacteria in a planar space. Dai et al. [110] employed deep learning for the precise orientation control of spherical oocytes, mitigating variances in polar body shape and size alongside cytoplasmic interference. This involved integrating a rotational stage with visual servoing and implementing robotic manipulation using a standard micropipette, with force modeling and optimal control to minimize oocyte deformation. Syntakas et al. [111] employed deep learning to approximate system dynamics where traditional models may fail, integrating it with a model predictive control (MPC) law. The algorithm estimates future uncertainty in MPC by combining a nominal model with a neural network-learned model, enhancing safety-critical applications such as obstacle avoidance in navigation tasks. Yang et al. [112] using Deep Neural Network (DNN)-based methods to enable micro/nanorobot swarms to autonomously learn optimal distributions for navigating through complex, unstructured environments, demonstrating real-time autonomous swarm navigation for targeted delivery and cargo transport, thereby showing that computational intelligence can enhance control capabilities in micro/nanorobot swarms (Figure 4b). Wang et al. [113] developed a data-driven optimal integrated controller (D²-OIC) to address microrobots that struggle with tracking control due to environmental disturbances and changes in helical body structures. The D²-OIC combines a nonlinear feedforward controller with a linear feedback controller. This system can adapt to different microrobot types by updating the feedforward controller with new data.

Although deep learning methods can achieve target tracking and localization, their accuracy in localization is not high, and the computational speed is not fast enough. Especially when applying localization to closed-loop control circuits, higher demands are placed on both the accuracy of localization and the speed of localization solving. Introducing deep reinforcement learning can provide benefits in controlling microrobots, offering improved adaptability, robustness, and efficiency in navigating complex environments with varying conditions. Reinforcement learning frameworks, such as deep Q-learning and policy gradient methods, enable microrobots to learn optimal control policies through trial-and-error interactions with their environment. In terms of perception and decision-making, Behrens et al. [114] used deep reinforcement learning to develop a smart helical magnetic hydrogel microrobot capable of autonomously swimming through uncharacterized fluidic environments under the control of a time-varying magnetic field (Figure 4c). This approach enables the microrobot to adapt its behavior without the need for complex system dynamics modeling, potentially revolutionizing microrobot control capabilities. Zou et al. [115] introduced a novel approach combining context detection with deep reinforcement learning to enhance microrobot performance in changing environments. Experiments show that this method enables the microrobot to detect environment changes effectively, suggesting the potential of this integration for robust microrobotic locomotion. In terms of advancements in reinforcement learning for adaptive control, Marino [116] introduced a significant advancement by employing model-free reinforcement learning for the adaptive control of a magnetically manipulated endoscope in the complex environment of the human gastrointestinal tract. Through experimental validation, the approach demonstrates effective navigation, obstacle avoidance, and optimal tissue contact without relying on motion planning algorithms. This pioneering application lays the foundation for the further development of advanced navigation techniques. In addition, Salehi et al. [41] applied model-free deep reinforcement learning to develop two control systems for guiding a disk-shaped magnetic microrobot in fluid environments (Figure 4d). Both Trust Region Policy Optimization (TRPO) and soft actor-critic (SAC) algorithms successfully learn optimal paths for reaching random target positions, with TRPO exhibiting superior sample efficiency and training stability. Muiños-Landin et al. [96] proposed a motion control method for spherical microrobots in grid environments. Due to the limited motion space, they employed classical reinforcement learning (Q-learning) by establishing a state-action table in the finite motion space and training to obtain the state transition probabilities in the table, achieving motion navigation between two points. Yang et al. [40,97] utilized a reinforcement learning approach employing an actor-critic network to facilitate targeted locomotion across diverse microrobot variants (Figure 4e). Validating the proposed control scheme, three microrobots with varying degrees of controllable degrees of freedom (DOF) achieved robust targeted locomotion in both obstacle-free and obstructed environments following training. Abbasi et al. [39] employed reinforcement learning to enable precise three-dimensional positional control of magnetic microrobots within a defined workspace using electromagnetic coil-generated gradient fields, surpassing conventional control methods in accuracy and efficiency. This approach offers adaptability to varying environments and microrobot properties while minimizing training time through simulation-based exploration, ultimately enabling fully autonomous control integrated with path planning algorithms.

Based on the analysis in Section 3, we found that a key limitation of traditional methods is that the complex dynamics modeling of micro/nanoscale robots and their interactions with the environment hinders the model-based precise navigation control. In contrast, learning-based approaches overcome this bottleneck, as they enable the micro/nanorobots to autonomously learn and evolve the optimal navigation strategies through self-learning, all without the need for explicit mathematical models [117]. This capability enables microrobots to navigate and operate effectively in environments characterized by uncertainties and practical disturbances, offering significant potential for addressing complex tasks and playing pivotal roles in enhancing microrobot control, particularly in medical applications where precision and adaptability are paramount. Therefore, a learning-based controller is the better controller solution for microrobot control in complex environments.

The comparisons between tradition control methods and learning-based control methods are shown in

Table 2. Comparisons between tradition control and learning-based control.

Control Strategies	Advantages	Limitations
PID control	Easy to implement and widely used in industrial automation; General applicability to various systems.	Limited precision due to linearity assumptions; Requires parameter tuning for optimal performance.
Nonlinear control	Widely applicable to real-world systems; Offers flexibility, faster response, and better accuracy.	Complex design process due to nonlinear system models; Limited to specific types of systems.
Optimal control	Seeks optimal solutions over a specified time horizon.	Complex design process and high computational cost.
Learning-based control	Data-driven and adaptable to uncertainty and nonlinearities. Simplifies sensor requirements by learning from data.	Requires substantial training data for good performance; Lack of interpretability (black-box models).

.

4. Conclusions and Outlook

In this review, we have explored the theoretical foundations underpinning magnetic microrobotics, including the generation and modeling of magnetic fields, as well as the dynamics governing microrobot motion. Moreover, we have discussed a spectrum of motion control methods, ranging from traditional approaches like linear control and fuzzy control to emerging learning-based techniques such as deep learning and reinforcement learning. These breakthroughs offer significant potential across various domains, including precise drug delivery, less invasive surgical procedures, and microassembly tasks. By harnessing the capabilities of magnetic microrobots, researchers can overcome existing limitations in precision and adaptability.

However, several challenges still require further investigation. One significant challenge lies in the development of robust control strategies capable of addressing complex environments, motion constraints, and dynamic scenarios. Traditional control methods may struggle to accommodate the nonlinear and time-varying dynamics often encountered in real-world magnetic microrobot applications. To address this, researchers can explore advanced control frameworks, such as adaptive control, model predictive control, and hybrid control schemes that combine traditional and learning-based approaches. These methodologies can enhance the adaptability of magnetic microrobot control, enabling them to navigate and perform tasks in unpredictable and unstructured environments. Additionally, the integration of machine learning into microrobot control presents both opportunities and challenges. While data-driven approaches hold great potential for developing more adaptive and autonomous systems, the reliable deployment of these techniques requires careful consideration of aspects such as sample efficiency, safety, and interpretability. Strategies like transfer learning, active learning, and the incorporation of domain knowledge can help address the data scarcity and computational challenges often associated with machine learning in microscale robotics.

The future of automated magnetic microrobot control is promising, with several avenues for research and development. Firstly, advancements in materials science and fabrication techniques may lead to the design of more sophisticated microrobots capable of complex maneuvers and interactions at the microscale. Furthermore, the integration of sensor technologies, such as microscale imaging and biosensors, could enhance the perception capabilities of magnetic microrobots, enabling real-time feedback and adaptive control. Additionally, interdisciplinary collaborations between robotics, nanotechnology, and biomedicine may pave the way for innovative applications in healthcare, including targeted therapy delivery and minimally invasive surgical procedures. Moreover, the continued exploration of machine learning algorithms, coupled with advancements in computational power, holds the potential to revolutionize microrobot control. By leveraging data-driven approaches, researchers can develop more robust and adaptive control strategies, capable of learning from and adapting to dynamic environments.