Digital Simulation and Identification of Faults with Neural Network Reasoners in Brushed Actuators Employed in an E-Brake System
Round 1
Reviewer 1 Report
This work proposes a data-driven method based on LTSM networks for the fault detection of electromechanical aircraft brakes. The proposed approach is compared with alternative strategies - KNN and Time Series Forest.
In general, the paper is interesting, well written and organized, addressing a problem of high interest for the scientific community. However I suggest that the authors address a few points:
- Line 94: I suggest to clarify (maybe with some examples) what is considered as IoT faluts. It seems from the effect of faults that IoT components do not include only telemetry equipment, but also feedback sensors and electronics for real-time control.
- Figure 1: The readability of the figure could be improved by using a larger font and/or higher resolution.
It is not clear what is the function of each block of the diagram. I suggest adding a brief description of the diagrams to explain how the model works.
The overall aircraft speed calculation layer is not represented, I suggest including it. - From Figure 1 it looks like the braking torque on the wheel is proportional to the brake pressure. As the coefficient of friction usually has a large uncertainty, are the model and fault detection algorithm robust to such uncertainty?
- Lines 145-150: How were the failure modes selected for the analysis?
- Lines 200-208: How are the failure scenarios sampled to collect the 120 examples and train/validate the network?
Is any variation in environmental condition (e.g. landing speed, tyre-ground friction coefficient, landing mass, ect) taken into account to test the robustness of the method? - Section 3.2: I suggest adding a figure to show the topology of the LSTM network
- The authors may expand their literature review by including some works regarding the use of data-driven approaches for fault detection, for example:
- Quattrocchi, G.; Berri, P.C.; Dalla Vedova, M.D.L.; Maggiore, P. Innovative Actuator Fault Identification Based on Back Electromotive Force Reconstruction. Actuators 2020, 9, 50. https://doi.org/10.3390/act9030050
- M. Li, G. Li and M. Zhong, "A data driven fault detection and isolation scheme for UAV flight control system," 2016 35th Chinese Control Conference (CCC), 2016, pp. 6778-6783, doi: 10.1109/ChiCC.2016.7554425.
Author Response
Dear Sir,
Thank you for your feedback. Please find below the amendments made and comments against the remarks.
- Line 94: I suggest to clarify (maybe with some examples) what is considered as IoT faluts. It seems from the effect of faults that IoT components do not include only telemetry equipment, but also feedback sensors and electronics for real-time control.
Line 150 (Previously 94) has been modified as follows to address the comment: “IoT faults, more specifically to sensor/actuator and interface components faults, are the only non-catastrophic events listed, as they could potentially reduce the braking performance, but their role in feedback loops can cause the braking force to deviate from expected values.”
- Figure 1: The readability of the figure could be improved by using a larger font and/or higher resolution. It is not clear what is the function of each block of the diagram. I suggest adding a brief description of the diagrams to explain how the model works. The overall aircraft speed calculation layer is not represented, I suggest including it.
Figures have been replaced. Due to the size and complexity of the model the clarity of the figures seems to be limited. The authors have done their best to provide figures of the best clarity possible.
- From Figure 1 it looks like the braking torque on the wheel is proportional to the brake pressure. As the coefficient of friction usually has a large uncertainty, are the model and fault detection algorithm robust to such uncertainty?
The model was developed for a case study on ideal conditions to represent a normal braking, so change in the parameters (friction) will certainly modify the results.
- Lines 145-150: How were the failure modes selected for the analysis?
The failure modes were selected according to how critical they could be and the diversity of effects they could show on the simulations.
Line 210 (previously 145) has been modified as: …normal performance model data, fault modes are selected according to their criticality and the variety of effects they could provide for data analysis purposes. Electrical and mechanical faults …
- Lines 200-208: How are the failure scenarios sampled to collect the 120 examples and train/validate the network? Is any variation in environmental condition (e.g. landing speed, tyre-ground friction coefficient, landing mass, ect) taken into account to test the robustness of the method?
The following was added at Line 262 in revised manuscript to address the comment: “Data was extracted such that the faults occurred at various instance of time in the process of breaking. This means that the velocity of the aircraft at the time of occurrence of fault varies throughout the dataset.”
- Section 3.2: I suggest adding a figure to show the topology of the LSTM network
The suggestion has been incorporated into the manuscript in Section 4.2.
- The authors may expand their literature review by including some works regarding the use of data-driven approaches for fault detection, for example:
Quattrocchi, G.; Berri, P.C.; Dalla Vedova, M.D.L.; Maggiore, P. Innovative Actuator Fault Identification Based on Back Electromotive Force Reconstruction. Actuators 2020, 9, 50. https://doi.org/10.3390/act9030050
- Li, G. Li and M. Zhong, "A data driven fault detection and isolation scheme for UAV flight control system," 2016 35th Chinese Control Conference (CCC), 2016, pp. 6778-6783, doi: 10.1109/ChiCC.2016.7554425
A section dedicated to literature review has been incorporated in the paper and the suggested literature has also been incorporated.
We hope all your concerns have been addressed. We kindly request you to review the changes and accept our submission.
Kind Regards,
Pablo, Dr Suresh, and Gouri
Author Response File: 
Author Response.docx
Reviewer 2 Report
The paper deals with the definition of a new reasoner to correctly isolate and identify the occurrence of a selected number of faults within an E-Brake for aeronautic applications. The subject is of interest, but i feel that the paper need some significant improvements to be accepted.
Hereby enclosed are my comments.
1) Literature review is non-existant. It is true that works on PHM systems for electric-brakes are still scarce, but authors can draw several talking points from papers on the health monitoring of electro-mechanical actuators, flight control systems and so forth.
2) The case study is not described, and this poses a few critical questions on the proposed word. In example, how many EMAs are providing the breaking force (typically 3 to 4 due to safety issues), what is their configuration (are they direct-drive or geared?), which type of electric motor do they employ (BLDC, BLAC?). This information is extremely important. The presence of multiple actuators can (and will) mitigate the effect at the E-Brake level of any faults you are going to inject on one EMA.
3) The model adopted to generate the data set is not described and is only adressed through a reference. I suggest authors to better detail the definition of the simulation environment, to better clarify the results obtained within the work. In particular, which kind of model has been used to describe the electric motor (is it a complete three-phase model with PWM, assuming a BLDC or BLAC motor, or is it a simple mono-phase representation?).
4) It is not clear how the authors have described the uncertainty affecting the definition of the "features". Please provide some information regarding which operating conditions have been considered while generating the data-set. In example, have you considered different runway conditions (wet, dry and so forth)? Have you considered the effects of temperature variation, or mass fluctuation of the aircraft (due to the variable number of passengers, fuel in the tank and so forth)? This is critical in understanding whether the reasoners proposed in the paper have been trained and verified on realistic data-sets.
5) In the conclusions, authors write that the data used to build the reasoners are "clean". Please notice that it is possible to add white noise to simulink signals.
6) Figures are in low resolution and are difficult to read. Please improve their quality.
7) Please provide more details on how the faults were modelled within the simulation environment.
8) I'm not sure about the oscillatory behavior of the motor current (which current? Unless it is a brushed motor did you mean the quadrature current?).
Author Response
Dear Sir,
Thank you for your feedback. Please find below the amendments made and comments against the remarks.
- Literature review is non-existant. It is true that works on PHM systems for electric-brakes are still scarce, but authors can draw several talking points from papers on the health monitoring of electro-mechanical actuators, flight control systems and so forth.
A section dedicated to literature review has been added to the manuscript.
- The case study is not described, and this poses a few critical questions on the proposed word. In example, how many EMAs are providing the breaking force (typically 3 to 4 due to safety issues), what is their configuration (are they direct-drive or geared?), which type of electric motor do they employ (BLDC, BLAC?). This information is extremely important. The presence of multiple actuators can (and will) mitigate the effect at the E-Brake level of any faults you are going to inject on one EMA.
- The model is a simplified version with a 1 actuator brake. It is certain that with a 4 actuator redundant braking system the effect of one of the failing is considerably reduced or barely noticeable, but the purpose of having a 1 actuator brake is to obtain a more visual and impactful data for data analysis.
- The electric motor used is a DC brushed motor. The reason this motor is used is because of the simplicity that the Simscape (MATLAB/Simulink) block gives to simulate the actuator and the faults.
- The same has been addressed by adding a paragraph at line 174 :
“A DC brushed motor block from the Simscape environment provides electric and torque parameters useful for the model. Usually, a 4-actuator brake per wheel would be used in real applications increasing the system redundancy and robustness. The failure of 1 actuator on a 4 EMA system will certainly reduce the effects on an aircraft braking, but the purpose of having this 1 EMA model is to obtain more visual and impactful information for data analysis.”
- The model adopted to generate the data set is not described and is only addressed through a reference. I suggest authors to better detail the definition of the simulation environment, to better clarify the results obtained within the work. In particular, which kind of model has been used to describe the electric motor (is it a complete three-phase model with PWM, assuming a BLDC or BLAC motor, or is it a simple mono-phase representation?).
The manuscript had previously mentioned that the simulation was in an ideal situation with no external condition affecting the braking. To provide further clarity lines 164 and 166 (previously 109 and 110) were modified as follows:
“The environment on which the simulation is working is on ideal conditions, this is, no external or environmental conditions are affecting the braking”
“The model is a simple representation of a single electromechanical actuator providing the necessary braking force to an aircraft. An ABS system is included in the model to give the simulation a more realistic approach.”
- It is not clear how the authors have described the uncertainty affecting the definition of the "features". Please provide some information regarding which operating conditions have been considered while generating the data-set. In example, have you considered different runway conditions (wet, dry and so forth)? Have you considered the effects of temperature variation, or mass fluctuation of the aircraft (due to the variable number of passengers, fuel in the tank and so forth)? This is critical in understanding whether the reasoners proposed in the paper have been trained and verified on realistic data-sets.
“The conditions for simulation are ideal and no external conditions affecting breaking” (refer line 165”.
The following lines were added:
“Data was extracted such that the faults occurred at various instance of time in the process of breaking. This means that the velocity of the aircraft at the time of occurrence of fault varies throughout the dataset. (line 262 of revised manuscript).”
- In the conclusions, authors write that the data used to build the reasoners are "clean". Please notice that it is possible to add white noise to simulink signals.
The authors have suggested this be undertaken as part of future work – line 549-552.
- Figures are in low resolution and are difficult to read. Please improve their quality.
Graphs and Pictures have been replaced. Due to the size and complexity of the model the clarity of the figures seems to be limited. The authors have done their best to provide figures of the best clarity possible.
- Please provide more details on how the faults were modelled within the simulation environment.
It is mentioned in the manuscript how the faults were modelled into the simulation. Information about this can be found in lines 217-221 (jamming) and 229 to 234 (OC and IOC).
- I'm not sure about the oscillatory behavior of the motor current (which current? Unless it is a brushed motor did you mean the quadrature current?).
The motor used in the simulation is indeed a brushed DC motor (as stated in Line 174 in the amended manuscript). The current shown represents the current fed to the motor and not the quadrature current. The current shown is a single phase current, which oscillates as more or less torque is required for the braking action.
We hope all your concerns have been addressed. We kindly request you to review the changes and accept our submission.
Kind Regards,
Pablo, Dr Suresh, and Gouri
Author Response File: Author Response.docx
Round 2
Reviewer 1 Report
The authors addressed most of the points highlighted in my first review, so I suggest to accept the work in the present form.
Author Response
Dear Sir,
Thank you for accepting the work. We have made some edits to the manuscript as suggested by the second reviewer.
The title of the paper has now been changed to "Digital simulation and identification of faults with NN reasoners in brushed actuators employed in E-Brake systems" as suggested to provide better clarity of the work undertaken. The material has been re-worded to focus the emphasis on brushed-DC motor actuators instead of the entire landing gear braking system. A section has also been created to discuss the future work that can be carried to further improve the results of the study mentioned in the paper.
We kindly request to review the paper and recommend the paper for publication.
Kind Regards,
Dr Suresh, Mr Garza, and Ms Ramesh
Reviewer 2 Report
I find the paper improved from a presentation stand point, but the concerns on the study proposed in this paper remain.
Authors presents and compare a few Machine Learning techniques to detect and isolate a few faults in an E-Brake system. The analytical work is commendable and the level of detail of the presentation is highly appreciated. There are however a number of inconsitencies that prevents me to reccomend the paper for publication in the present form.
1) The architecture of the E-Brake system is not realistic, since only one EMA is considered. Moreover, the Brushed DC motors are rarely used in aviation systems due to concerns over the brushes wear. A typical configuration sees the use of three or four EMA driven by Brushless DC or Brushless AC motors. The effects of the presence of a fault in the actuator on the brake's behavior is different wether one EMA or reduntant EMAs are considered. As such, it is likely that the results presented in this paper are not applicable to a real case scenario.
2) Authors performed the feature selection process and the training of the reasoners without considering the effects that external disturbances and variations in the operating conditions can have on the features themselves. This is a major oversight. The behavior of data driven routines is heavily dependent on the data distribution, as most of the misclassification errors typically occurs in the proximity of the classification boundaries. In absence of a rigorous description of the uncertainty associated with each selected feature the presented performance marks are not meaningful as they are not evaluated on a realistic data set.
3) On the subject of the evaluation of the reasoners performances, the confusion matrixes presented in figures 11 - 13 show that the performance scores were obtained on a very low number of sets. In principle, such operation should be performed on large data set representative of all of the possible operating conditions that the E-Brake system could face during its operating life. In abscence of this characterization and considering that the paper is not supported by experimental results, there are serious questions on the credibility of the results provided.
I would suggest the authors to reconsider the work described in this paper and submit a new version of their work in the future. The theme is of significant interest within the aeronautic industry and I feel that applying the methods presented in this paper to more realistic models/data set would be of great interest for the journal and for the PHM community.
In particular, I would suggest to:
- expand the data set employed to train and evaluate the reasoners performances
- if possible, consider a more realistic configuration of the E-Brake system. If this is not possible or too time consuming, a possible solution would be to shift the attention from the E-Brake system towards the actuator. In this sense, I would advice the authors to change the title of the paper to "Digital simulation and identification of faults with NN reasoners in brushed-actuators employed in E-Brake systems", to better highlight that the study is not performed considering a complete braking system and that brushed-DC motors are employed. In a similar way, I would suggest to revise the abstract and the introduction avoiding to mention that the paper has been prepared with a "generic E-Brake" system configuration. I.e., in line 12 instead of writing "a generic electric braking system for landing gear" authors could write "a brushed EMA employed in an electric braking system" and so forth.
If these changes are applied I would recomend the paper for publication.
Author Response
Dear Sir,
Thank you for your valuable suggestions for the improvement of our work, we deeply appreciate it. We have taken into consideration your work and made the suggested amendments.
The title of the paper has now been changed to "Digital simulation and identification of faults with NN reasoners in brushed actuators employed in E-Brake systems" as suggested to provide better clarity of the work undertaken. The material has been re-worded to focus the emphasis on brushed-DC motor actuators instead of the entire landing gear braking system. A section has also been created to discuss the future work that can be carried to further improve the results of the study mentioned in the paper.
We kindly request to review the paper and recommend the paper for publication.
Kind Regards,
Dr Suresh, Mr Garza, and Ms Ramesh
