Improved Performance for PMSM Sensorless Control Based on the LADRC Controller, ESO-Type Observer, DO-Type Observer, and RL-TD3 Agent
Round 1
Reviewer 1 Report
This research presents the global FOC control structure of a PMSM in which the outer rotor speed control loop is implemented using a LADRC controller.
1. The paper is well prepared. However, I found a reference paper similar to your research
Nicola, M.; Nicola, C.-I.; Ionete, C.; Șendrescu, D.; Roman, M. Improved Performance for PMSM Sensorless Control Based on Robust-Type Controller, ESO-Type Observer, Multiple Neural Networks, and RL-TD3 Agent. Sensors 2023, 23, 5799. https://doi.org/10.3390/s23135799
So the contribution of the paper should focus on the LADRC controller only. The author should cite this reference and include a comparison or provide some discussion to explain the advantage of ADRC compared to PID.
2. The similar score of the manuscript is too high (32% exclude your preprint)
Author Response
Dear reviewer, thanks for your recommendations.
1. This paper presents the global FOC control structure of a PMSM in which the outer rotor speed control loop is implemented using a LADRC controller. Typically, the operation of the LADRC controller requires rejection of the disturbance acting on the system, which is estimated using an ESO-type observer. Similar performance for a LADRC controller can be obtained by replacing the ESO-type observer with a DO-type observer. In addition, numerical simulations even show a slight improvement by using DO-type observer instead of ESO-type observer in the LADRC controller structure. In summary we can say (it can be seen from the structure of the LADRC controller given by equation (11)) that the LADRC controller is of medium complexity, but that it is based on an accurate estimation of the perturbations by using ESO-type observer or DO-type observer.
Indeed in the article referred by you, and added to the reference to position [11], the PMSM control problem is presented in which the control is of the robust type. We point out that in this case, regardless of the type of observer, the success of the control is mostly due to the high complexity of the robust controller.
The comparison between LADRC and PI controllers is made at the beginning of Section 6. Thus, Figures 19 and 20 show the evolution of the parameters of interest of the PMSM control system following the numerical simulations performed. The PI type controller ensures a good performance of the PMSM rotor speed control system when the load torque is close to the nominal value. Therefore, Figure 19 shows the evolution of the PMSM rotor speed, electromagnetic torque, stator currents, and id and iq currents in the d-q reference frame. The PMSM rotor speed reference sequence is as follows: ωref = [1000 1200 1400 900]rpm for a load torque TL = 0.5Nm. Figure 19 (a) shows the evolution of the parameters of PMSM control system based on PI speed controller, Figure 19 (b) shows the evolution of the parameters of PMSM control system based on LADRC controller with ESO-type observer, and Figure 19 (c) shows the comparison between the performance of PMSM control system based on classical type PI controller and LADRC controller with ESO-type observer. It can be seen that the difference in performance between the two control systems is minimal. By increasing the load torque TL to 2Nm, it can be seen in Figure 20 that the PMSM control system based on the classical PI controller manages to stabilize the system, but with a steady-state error and a longer response time compared to the case where the load torque is greater than 0.5Nm. However, it can be seen that the control performance remains the same when using the LADRC controller with an ESO-type observer.
2. We will need the similarity report to reduce the percentage.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
In order to suppress the load torque variation in the control system of permanent magnet synchronous motor, a linear adaptive disturbance rejection controller based on interference observer is proposed.Ant colony algorithm is used to optimize the gain of interference observer.The Reinforcement Learning Twin-Delayed Deep Deterministic Policy Gradient is trained to provide correction signals and further estimate external perturbations.The analysis results verify the correctness of the theoretical analysis and the effectiveness of the control strategy. Give advice in the following areas:
1.The application of ant colony algorithm is not mentioned in the abstract.
2.Some variables in Formula 1 are not explained.
3.The stability of the ESO observer should be demonstrated.Improve the stability proof of the DO observer.
4.Figure 3 is marked incorrectly, please modify and improve.
5.The DO-type observer in Figure 9 is wrong, please modify and improve it.
6.The name of Table 1 is not centered.
7.In order to make readers more convenient and intuitive to read the paper, the ACO algorithm and Figure 18 should be explained.
8.In the case of wref=1000rpm ,TL= 4Nm with uniform distributed noise, and an increase of 100% for J parameter, there is no comparison between LADRC controller with DO-type observer and LADRC controller with ESO-type observer to show the superiority of DO-type observer.
Author Response
1. We mentioned in the abstract the use of the Ant Colony Algorithm.
2. We have also explained the rest of the variables in equation (1).
3. About the stability of the ESO-type observer is indicated the reference [30].
In summary the equation of an ESO-type observer is given in equation (14) of linear form. For an ESO-type observer of order 3 the characteristic polynomial is given in equation (17), and for the choice of the values of the L-amplifications in equation (18) the stability of the ESO-type observer system is obtained by placing the poles in the left half-plane. Also, for an ESO-type observer of order 2 equations (23), (24), and (25) demonstrate the stability of the system.
About the stability of the DO-type observer is indicated the reference [30].
Starting from the nonlinear DO-type observer equation described in equation (35), by choosing the amplification factor l(x) in equation (37) we obtain the differential form of the error in equation (38). The rest of the calculations are completed by obtaining the linear form of the DO-type observer given in equation (43) and the differential equation of the error given in equation (46).
By choosing a Lyapunov function and calculating its derivative , using equation (46) one obtains the strict negativity of the derivative which implies the stability of the DO-type observer.
4. We have modified and improved Figure 3.
5. We have modified and improved Figure 9.
6. We have corrected and centered the name of Table 1.
7. From Figure 18 it can be seen that after 200 iterations the best cost indicator has a convergent evolution, which is an clue of the success of the training phase of the ACO algorithm. Indeed the optimized value obtained in this case for the gain of DO-type observer, leads to improved performance compared to the basic version of DO-type observer.
8. In case of ωref = 1000 rpm and a load torque TL = 4Nm with uniform distributed noise, and an increase of 100% for J parameter, we performed numerical simulations when using DO-type observers in the LADRC controller structure for comparison with the control structure of LADRC controller with ESO-type observer to prove the superiority of DO-type observer. Thus, we inserted a new figure (Figure 24), and the other figures were incremented by one position.
Author Response File: Author Response.pdf
Reviewer 3 Report
This article focusses on PMSM speed regulation using a PD controller with DO, second order ESO and third order ESO observing the disturbance. The working principle as well as control structure are well presented. And the comparative study is conducted by simulation. However, sensorless control is not involved in the content which does not comply with the title.
Other comments are listed below:
1. Sector 2 should not be named as sensorless control of PMSM since there is nothing related to sensorless control.
2. The notation in Figure 3 should be changed to DO instead of ESO.
3. The item Bw/J is included in the disturbance observation using ESO while it is separated in DO application, any reason behind?
4. In Figures 12, 13, 15, and 16, rotor position is from motor sensor, position/speed observer should be added to achieve sensorless control.
5. RL-TD3 is used in this article, however its working principle has not been explained.
6. The meaning of using the index DF is missing.
7. In Table 2, the rotor speed ripple is near 120 rpm, any reason behind?
Author Response
Dear reviewer, thanks for your recommendations.
1. The name of the Section2 was changed into "Proposed Structures for Sensorless Control of PMSM". Section 2 describes the operating equations of the PMSM, from which the FOC control strategy in Figure 1 can be developed. Figure 2 shows the diagram of the proposed sensorless control system for PMSM based on LADRC controller with ESO-type observer and RL-TD3 agent. Similarly, Figure 3 shows the diagram for the proposed PMSM sensorless control system based on LADRC controller with DO-type observer and RL-TD3 agent. Thus, Figures 1, 2, and 3 contain the sensorless version with ESO-type and DO-type observers respectively, which justifies the appearance of the word "sensorless" in the name of Section 2.
2. The notation in Figure 3 was changed to DO-type observer instead of ESO-type observer.
3. The term appears in equation (27) in the generalized perturbation component f of the ESO-type observer description.
Note that in Figure 10, given the estimated d-term ( ) (given in equation (47) describing the DO-type observer), by adding the term , we can obtain exactly the term denoted by z3, which represents the generalized perturbation denoted by f in relation (27), which characterizes the ESO-type observer. This shows the equivalence of the use of the term in the two types of observations.
4. In Figures 12, 13, 15, and 16 the variables "thetaestim" and "westim" appear as input parameters in the described blocks, which are taken as reaction quantities (taken from the output of these blocks and brought to their input).
5. The implementation of an RL-TD3 agent first requires a training phase, during which the RL-TD3 agent [11,36], based on the signals collected from the system and by maximizing a reward that can be considered as an integral performance criterion, will provide a correction signal for the current reference iqref. Next, the inner current control loops are those used in the FOC control strategy and can be implemented as ON/OFF controllers with hysteresis for fast response. The usual expression of reward is given in relation (50). The first four terms refer to the actual error due to the d-q reference frame terms, id and iq currents, and the speed and position of the PMSM rotor ω and θ relative to their references. The last term in the relation (50) contains the actions that were generated at the previous times.
Figure 14 shows the evolution of the performance for the RL-TD3-type agent for improvement of LADRC with ESO-type observer for PMSM control.
6. It can be specified that the box-counting method is used to calculate DF according to [11,38,39].
In principle, it starts from a square that can contain the signal, and the length of one side of this square is used as the unit of measurement. The side of the square is chosen so that it contains the signal, and the size of the side of the square is chosen as a power of 2 to speed up the calculations. By dividing the unit of measurement by 2, the algorithm is repeated until the value is below a predefined threshold. We can write that the form of each division has the expression 1/2k, where k is the current step. The division on the second axis is of the form where the total number of occupied domains on the scale given by the initial signal is denoted by nk. In the current step, the values obtained for each of the two axes using the algorithm described above are retained, namely the coordinates of a point Mk(x,y), represented in the Cartesian system by logarithmic coordinates. The sequence of points M1, M2, ..., Mn is calculated sequentially at each step. The stop condition is when the set threshold value has been exceeded. The slope of a line closest to points M1, M2, ..., Mn is calculated by least squares, and the value obtained represents the DF of the initial signal. In the MATLAB environment, using the command “[n,r] = boxcount(signal,‘slope’)”, the vectors of the two dimensions are obtained according to the algorithm described above.
Moreover, by using the commands “df = −diff(log(n))./diff(log(r))” and “[‘DF = ’num2str(mean(df(1:length(df)))) ‘+/− ’num2str(std(df(1:length(df))))])” the coordinates of the Mk points in logarithmic coordinates are obtained, as well as the slope of the line closest to the points provided by the algorithm representing DF.
7. please see the attachment.
Author Response File: Author Response.pdf
Round 2
Reviewer 2 Report
In order to suppress the load torque variation in the control system of permanent magnet synchronous motor, a linear adaptive disturbance rejection controller based on interference observer is proposed. Ant colony algorithm is used to optimize the gain of interference observer. The Reinforcement Learning Twin-Delayed Deep Deterministic Policy Gradient is trained to provide correction signals and further estimate external perturbations.The analysis results verify the correctness of the theoretical analysis and the effectiveness of the control strategy. Give advice in the following areas:
1. The drawing title was marked incorrectly, jumping from FIG. 7 to FIG. 9 without FIG. 8, reordering the drawing notes, and modifying the drawing notes in the article.
2. In order to make the paper more convenient and intuitive for readers, please explain how it can be concluded from fig. 14 and fig. 17 that RL-TD3 agent can improve the control system.
3. As shown in Figure 18, it only iterates 100 times, but it is written in the article that it iterates 200 times, please correct it.
4. Comparison of LADRC controller with DO-type observer with other two control methods under the same conditions should be added in fig. 19 and fig. 20 to demonstrate the superiority of DO-type observer.
Author Response
Dear reviewer, thanks for your recommendations.
- We have renumbered all the figures in the article.
- The implementation of an RL-TD3 agent first requires a training phase, during which the RL-TD3 agent, based on the signals collected from the system and by maximizing a reward that can be considered as an integral performance criterion, will provide a correction signal for the current reference iqref. Next, the inner current control loops are those used in the FOC control strategy and can be implemented as ON/OFF controllers with hysteresis for fast response. The usual expression of reward is given in relation (50). The first four terms refer to the actual error due to the d-q reference frame terms, id and iq currents, and the speed and position of the PMSM rotor ω and θ relative to their references. The last term in the relation (50) contains the actions that were generated at the previous times.
equation (50)
From Figure 13 (formerly Figure 14) and Figure 16 (formerly Figure 17), which show the evolution of the training phase for the RL-TD3 agent in combination with the ESO observer and the corresponding DO observer, it can be seen that the evolution of these performances have a convergent evolution, which is an indication of the success of the training phase for the RL-TD3 algorithm. In fact, Figures 24, 25, and 26 (formerly Figures 25, 26, and 27) show the improved performance using RL-TD3 agent compared to the basic versions of the observers.
- We have corrected in the text by 100 iterations as shown in Figure 17 (formerly Figure 18 after the corrections in the first point).
- We have added LADRC controller with DO-type observer in order to compare the performance of PMSM control system based on PI controller and PMSM sensorless control system based on LADRC controller with ESO-type observer.
Therefore, Figure 18 shows the evolution of the PMSM rotor speed, electromagnetic torque Te, stator currents ia, ib, ic, and id and iq currents in the d-q reference frame. The PMSM rotor speed reference sequence is as follows: ωref = [1000 1200 1400 900]rpm for a load torque TL = 0.5Nm. Figure 18 (a) shows the evolution of the parameters of PMSM control system based on PI speed controller, Figure 18 (b) shows the evolution of the parameters of PMSM sensorless control system based on LADRC controller with ESO-type observer, Figure 18 (c) shows the evolution of the parameters of PMSM sensorless control system based on LADRC controller with DO-type observer, and Figure 18 (d) shows the comparison between the performance of PMSM control system based on classical type PI controller, PMSM sensorless control system based on LADRC controller with ESO-type observer, and PMSM sensorless control system based on LADRC controller with DO-type observer, respectively. It can be seen that the difference in performance between the three control systems is minimal.
Author Response File: Author Response.pdf
Reviewer 3 Report
Detail comments can be seen in the attachment.
Comments for author File: Comments.pdf
Author Response
Dear reviewer, thanks for your recommendations.
1) We added the statement “In the usual case: Ld = Lq and Rd = Rq = Rs.” before equation (1).
Indeed the index e (for example θe) is used for electrical quantities and without index for mechanical quantities (for example θ, ω). We respect these usual notations in all our papers. In Figures we changed theta with θe, like inputs in Clarke and Park transforms (θe = np·θ).
2), 3), 4) and 5) Indeed there is a mistake in Figure 4, namely instead of . In the presentation and synthesis of the ESO-type observer in Section 3, two linearized models of the PMSM are used, one of order 3 and one of order 2. In the first case the ESO-type observer has order 3 (θ, ω, and the total disturbance are estimated), and the LADRC-type controller has order 2. The equations involved are those between (9) and (25), including equations (9) and (11) to show the general transition from nonlinear to linear system. Indeed z1, z2, and z3 are the estimations value of the rotor mechanical angle θ, rotor mechanical angular speed ω, and the total disturbance. Between equations (26) and (34) (only as an example without being involved in the numerical simulations) the simplified one-order case of the PMSM description equations is presented which implies an ESO-type observer of order 2 and the LADRC-type controller of order 1. In this case z1 and z2 are the estimations value of the rotor mechanical angular speed ω and the total disturbance.
In the rest of the article in the Matlab/Simulink implementation and in the numerical simulations performed the ESO-type observer case has order 3.
6) This paper presents the global FOC control structure of a PMSM in which the outer rotor speed control loop is implemented using a LADRC controller. Typically, the operation of the LADRC controller requires rejection of the disturbance acting on the system, which is estimated using an ESO-type observer. Similar performance for a LADRC controller can be obtained by replacing the ESO-type observer with a DO-type observer. In addition, numerical simulations even show a slight improvement by using DO-type observer instead of ESO-type observer in the LADRC controller structure. In summary we can say (it can be seen from the structure of the LADRC controller given by equation (11)) that the LADRC controller is of medium complexity, but that it is based on an accurate estimation of the perturbations by using ESO-type observer or DO-type observer.
7) We have added LADRC controller with DO-type observer in order to compare the performance of PMSM control system based on PI controller and PMSM sensorless control system based on LADRC controller with ESO-type observer.
Therefore, Figure 18 shows the evolution of the PMSM rotor speed, electromagnetic torque Te, stator currents ia, ib, ic, and id and iq currents in the d-q reference frame. The PMSM rotor speed reference sequence is as follows: ωref = [1000 1200 1400 900]rpm for a load torque TL = 0.5Nm. Figure 18 (a) shows the evolution of the parameters of PMSM control system based on PI speed controller, Figure 18 (b) shows the evolution of the parameters of PMSM sensorless control system based on LADRC controller with ESO-type observer, Figure 18 (c) shows the evolution of the parameters of PMSM sensorless control system based on LADRC controller with DO-type observer, and Figure 18 (d) shows the comparison between the performance of PMSM control system based on classical type PI controller, PMSM sensorless control system based on LADRC controller with ESO-type observer, and PMSM sensorless control system based on LADRC controller with DO-type observer, respectively. It can be seen that the difference in performance between the three control systems is minimal.
8) The use of an observer for PMSM rotor speed estimation prints the sensorless control system i.e: PMSM sensorless control system based on different types of controllers and observers. We have made the modifications in the text of the manuscript.
Author Response File: Author Response.pdf
