Application of Wavelet Scattering and Machine Learning on Structural Health Diagnosis for Quadcopter

Abstract

The aim of this study was to examine the health diagnosis classification method of quadcopter structures with different mixed faults. The loosening of the motor mount, damage to the propeller, and the loosening of the arm mount were the main fault conditions investigated. Data were first acquired under non-fault conditions and the conditions of the three types of fault. Then, the features of the vibration and pulse width modulation signals were extracted by root mean square, standard deviation, and sample entropy. Moreover, the features of the audio signal were extracted by wavelet scattering, which contains time-frequency domain information that provides significant power for classification. In this paper, we propose a simple machine learning method, based on the k-Nearest Neighbor (kNN), not only for classification but also demonstrating the efficacy of the features. To test the limits of accuracy, different configurations of kNN parameters are deployed, in addition to the features. In summary, as a result of the highly efficacious features, despite mixed fault conditions, the accuracy reached 90.73%. This method improves the accuracy of mixed faults’ classification and maintains a certain level of classification effectiveness.

1. Introduction

The development of unmanned aircraft systems is rapidly expanding. Using these systems, operators can perform and complete tasks, thus enhancing people’s lives. Drones provide convenience and benefits, but also increase the risks associated with flight safety. Therefore, ensuring flight safety is a topic that is crucial to the use of drones.

This study was dedicated to a health diagnosis classification system of quadcopter structures. In general, quadcopters are composed of motors, propellers, arms, landing gear, a fuselage, flight control computers, electronic transmissions, batteries, and signal receivers. Damage suffered by the quadcopter structure during the operation of flight missions may result in accidents and serious damage to property, and risks to personal safety.

Many types of fault and failure may affect quadcopters. In the evaluation of faults, Kandaswamy et al. (2017) arranged them into three main types. Following the collection of real flight data from multiple onboard sensors, statistical methods were applied to derive a feature-set of signals that can help in post-analysis [1]. Regarding the power device, Yong Keong Yap et al. (2014) mounted an inertia measurement unit (IMU) on the arm of a drone, where it located the proximity of the vibration source. Then, the signal was transformed by fast Fourier transform (FFT) to identify the patterns of different faults [2]. Misra et al. (2018) used an experimental arm to simulate the flying situation. The audio, thrust, power, and vibration signals were used to build a random forest model which can not only detect faults, but also classify the faults. Padraig Cunningham et al. (2007) proposed a classifier based on the k-Nearest Neighbor (kNN), which is a supervised machine learning method. It assessed the k-Nearest Neighbor to classify the input data [3].

For quadcopter diagnosis, De-Li Cheng et al. (2019) used the signal collected from the flight controller to diagnose the health of drones. In addition to the non-fault condition, a loose motor screw and damaged propeller were used to develop a diagnosis system. A self-organizing map [4] and kNN were applied to the classification and the diagnosis of the fault level [5]. Huimin et al. (2018) integrated a drone, flight controller, Raspberry Pi, and temperature sensors to detect whether the temperatures of motors were too high, and, if so, the system landed automatically. In addition, the temperature assessment system integrated reinforcement learning to ensure the reliability of the decisions [6].

In summary of the above review, we found that it is almost mandatory to add additional sensors to elevate the rate of classification because the recording of intrinsic sensors requires computation power, and the level of sensors may not be as good as required. In addition to sensors, melded faults have rarely been considered by other researchers. Therefore, the purpose of this research was to improve the quadcopter health diagnosis system, especially regarding combined faults and the use of different sensors. Flight and noise data were collected for analysis and to extract features. After processing the data, we established a machine learning algorithm that was used for classification, for which the classification accuracy was expected to be more than 80–90% for combined faults. The aim of this system was not only to shorten the time required for people to check quadcopters, but also to detect potential and invisible damage to the quadcopters’ components.

The remainder of the article is outlined as follows. In Section 2, several statistics methods used to extract the features are introduced. The equipment and experiment are outlined in Section 3, which also explains flight test in detail. In Section 4, the processes of signal and feature extraction are explained, and the discrepancies between different states are demonstrated. The results are presented and discussed in Section 5, and conclusions are presented in Section 6.

2. Materials and Methods

2.1. Feature Engineering

Feature engineering [7] is used to process current, thrust, vibration, and audio signals generated during in-flight experiments. In turn, this processing generates feature data. Because the time-domain features used in this article are relatively common, their introduction provided in this paper is short. Conversely, more detailed introductions are provided for the sample and wavelet theories because they are rarely seen in the existing research.

2.1.1. Time-Domain Signal Feature Engineering

The standard deviation (SD) can display the degree of dispersion between the data and its data group. The degree of dispersion of the vibration signal within the interval can be measured.

$$SD=\sqrt{\frac{1}{N-1}{{\displaystyle \sum }}_{i=1}^N{\left(x_i-\mu \right)}^2} , i=1, 2, 3,\dots ,N$$

(1)

where N is the total number of data points; x_i is a series of data and μ is its mean value.

The root mean square (RMS) is used to calculate the mean difference between data and a dataset. It is a statistical method that is often used in the processing of the direct current, sine waves, and normally distributed multi-frequency signals. It can be used as a method to evaluate the energy level of the current and voltage in the dataset interval.

$$RMS=\sqrt{\frac{{{\displaystyle \sum }}_{i=1}^n{x}_i^2}{n}},i=1, 2,3,\dots$$

(2)

Sample entropy (SE) [8] is used to measure the complexity of a time-series signal and has been applied in many fields. SE is a modification of approximate entropy (AE) and a powerful method used to analyze time-series signals. SE can measure the degree of chaos of the physical signal when the results are not relative to the size of the sample. In practice, a piece of the data signal is divided into multiple samples; then, the similarity between each sample and the other samples is assessed to determine the confusion degree of the signal in a certain time interval.

A brief introduction of sample entropy is introduced as follows. Let the signal $X$ for N data points be: $x_1, x_2,x_3,\dots ,x_N$. Then, dividing the data into every m data points yields the $N-m+1$ subset. After segmentation, this can be arranged into a template space: $X_{\left(N-m+1\right)\times \left(m\right)}$.

$$X_{\left(N-m+1\right)\times \left(m\right)}=\left[\begin{array}{cccc}{x}_1& x_2& \dots & x_m\\ x_2& x_3& \dots & x_{m+1}\\ ⋮& ⋮& \ddots & ⋮\\ x_{N-m+1}& x_{N-m+2}& \dots & x_N\end{array}\right]$$

(3)

First, calculating the largest distance between each template $d_{ij}$:

$$d_{ij}=max\left(\left|X\left(i,1\right)-X\left(j,1\right)\right|,\left|X\left(i,2\right)-X\left(j,2\right)\right|,\dots ,\left|X\left(i,i+m-1\right)-X\left(j,j+m-1\right)\right|\right)$$

(4)

From Equation (4), the largest distance can be determined using the Chebyshev Distance. Then, the Heaviside nonlinear function is used to calculate self-similarity $D_{ij}$:

$$D_{ij}=H\left(d_{ij}\right)=\left\{\begin{array}{c}1, d_{ij<r}\\ 0, d_{ij \ge r}\end{array}\right.$$

(5)

where r is the threshold, which is usually the proportion of the standard deviation of the signal. Using the standard deviation ensures that the result is not affected by the different scales of fluctuation. Moreover, the sum of all $D_{ij}$ s divided by $N-m$ (it is not divided by $N-m+1$ because SE does not compare the distance itself).

$${\delta }_m\left(r,N\right)=\frac{{{\displaystyle \sum }}_{i=1}^{N-m}Dij}{N-m}$$

(6)

Finally, by adopting m + 1 data points to build a new template space, and processing the data from Equations (3)–(6) to obtain ${\delta }_{m+1}\left(r,N\right)$, the final value of sample entropy can be determined:

$$SE\left(m,r,N\right)=-log\frac{{\delta }_m\left(r,N\right)}{{\delta }_{m+1}\left(r,N\right)}$$

(7)

where m is the mth sample, r is the threshold, δ is the sample value.

2.1.2. Frequency-Domain Signal Feature Engineering

Wavelet analysis is a time-frequency analysis method [9] that can not only be used to obtain the power spectrum of a signal, but also to analyze and control the time resolution and frequency resolution [10,11]. In this study, data features were extracted from the audio signal using wavelet scattering theory, which can be used to make a further classification assessment.

The Morlet wavelet is based on the use of wavelet transformation for a series of signals:

$$\psi \left(t\right)=e^{i{\omega }_{0}t}{e}^{\frac{-t^2}{2}}$$

(8)

where ${\omega }_0$ is the wave number. The Morlet wave function can be further dilated and translated:

$${\psi }_{\left(a,b\right)}\left(t\right)=\frac{1}{\sqrt{a}}\psi \left(\frac{t-b}{a}\right)$$

(9)

where a is the extension (dilation) and b is the translation. As a result of this property, regardless of whether the frequency is high or low, different time and frequency resolutions can be used for effective analysis. The signal analysis in the time and frequency domains can be obtained by wavelet transform:

$$W_x={{\displaystyle \int }}_{-\infty }^{\infty }x\left(t\right)\frac{1}{\sqrt{a}}\psi \left(\frac{t-b}{a}\right)dt$$

(10)

Application of the short-time Fourier transform (STFT) to the divided signal can also increase the time resolution of the signal. However, the adjustment of the bandwidth and central frequency of the filter via scaling results in better time and frequency resolutions for the wavelet transform.

Wavelet scattering transformation was proposed by Joakim Anden and Stephane Mallat in 2014 [12]. The theory is that a wavelet-transformed time-series signal is analyzed using the Convolution Neural Network (CNN) framework, and a wavelet function (high-pass filter) is convolved with the scaling function of a Gabor function (low-pass filter). Equation (11a) is the Gabor function and Equation (11b) is the scaling function of the Gabor function.

$$g\left(t\right)=\frac{1}{\sqrt{2\mathsf{\pi }}\sigma }{e}^{\frac{-t^2}{2{\mathsf{\sigma }}^2}}{e}^{i\theta x}$$

(11a)

$${\varnothing }_J\left(t\right)=2^{-2JQ}g\left(2^{-JQ}t\right)$$

(11b)

where θ is the center frequency; when its value is zero, it is a Gaussian function. σ is the standard deviation of the data signal. J is the scale and Q is a number of wavelet functions per octave; for example, there are 12 notes in an octave for music. The values of J have several octaves for high-pass decomposition.

Using different scales, convolutional signals, and wavelet functions, from low- to high-frequency information a locally deformed and scale-invariant wavelet conversion coefficient can be obtained:

$$W_x={(x\left(t\right)\star \varnothing \left(t\right),x\left(t\right)\star {\psi }_{\lambda }\left(t\right))}_{t\in R,\lambda \in \Lambda }$$

(12)

where λ is the center frequency of the wavelet function, and the mean value of the wavelet coefficients is zero. $x\left(t\right)$ represents the original signal, $\varnothing \left(t\right)$ is the scaling function, and ${\psi }_{\lambda }\left(t\right)$ is the wavelet function. In order to obtain the characteristic information of the signal, the absolute value is added, which is the modulus. According to [12], the modulus provides the stability and translation stability features, i.e.,

$$U_{\lambda }x\left(t\right)=\left|x\left(t\right)\star {\psi }_{\lambda }\left(t\right)\right|$$

(13)

The modulus is used to compute a lower frequency envelope. In order to obtain stable wavelet coefficients, the scaling function (low-pass filter) is convolved, and we obtain the wavelet scattering characteristic coefficient of the signal:

$$S_{1}x\left(t,{\lambda }_1\right)=\left|x\left(t\right)\star {\psi }_{{\lambda }_1}\left(t\right)\right|\star {\varnothing }_J\left(t\right)$$

(14)

In the expression of the wavelet scattering framework, a series of wavelet scattering coefficients are analyzed using the deep convolution network architecture.

In order to obtain a low variance and stable features, an averaging function and modulus are applied after the wavelet convolution. There is a sequence of steps to achieve wavelet scattering coefficients as shown in Figure 1. First, the signal is convolved with the scaling function to obtain the zero-order scattering coefficient of the signal, $S_{0x}$. For the second layer, the signal is convolved with the wavelet function, $\psi$ (first-order filter), and the modulus is determined. This is then convolved with the scaling function to obtain the first-order wavelet scattering coefficient. When determining the next-order wavelet scattering coefficient, the original sequence information must be restored because averaging loses the high frequency information. Therefore, the next wavelet function (second-order filter) needs to be convolved to the original sequence information to ensure that information is not lost. For high-frequency information, in order to obtain a locally deformed and fixed-scale wavelet scattering coefficient, it is necessary to convolve the scaling function to obtain a stable wavelet scattering coefficient.

$$S_{2}x\left[{\lambda }_{1,}{\lambda }_2\right]=\left|\left|x\left(t\right)\star {\psi }_{{\lambda }_1}\left(t\right)\right|\star {\psi }_{{\lambda }_2}\left(t\right)\right|\star {\varnothing }_J\left(t\right)$$

(15)

If the multiple wavelet functions (filters) are set, the network architecture can continue to iterate continuously on the next wavelet function:

$$S_{n}x\left[{\lambda }_{1,}{\lambda }_2,\dots ,{\lambda }_n\right]=\left|\left|\left|x\left(t\right)\star {\psi }_{{\lambda }_1}\left(t\right)\right|\star {\psi }_{{\lambda }_2}\left(t\right)\right|\right|\star \dots \left|\star *{\psi }_{{\lambda }_n}\left(t\right)\right|\star {\varnothing }_J\left(t\right)$$

(16)

A series of wavelet scattering coefficient characteristics can then be obtained:

$$\{S_{0}x\left(t\right),S_{1}x\left(t,{\lambda }_1\right),S_{2}x\left(t,{\lambda }_1,{\lambda }_2\right),\dots ,S_{n}x\left(t,{\lambda }_1,{\lambda }_2,\dots ,{\lambda }_n\right)\}$$

(17)

The wavelet filter banks in multiple wavelet filter libraries are used to analyze the low- and high-frequencies of the complete signal information to obtain frequency domain information. In this study, we used a scaling function (low-pass filter) and two wavelet functions (high-pass filter) to obtain the wavelet scattering coefficients of the signal. Note that, normally, second-order wavelet scattering transform is sufficient because the energy attenuates at each level.

3. Experiment Setup

3.1. Drone System

In this experiment, the reference drone system comprised commercially available equipment used in drones shown in Figure 2a. The details of the equipment and the flight test setup are shown in the

Table 1. The introduction of the components of the drones.

Motor	Commercially available T-motor Air Gear 450 multi-rotor power set, which contains DC motor and propeller. The brushless DC motors are AIR2216 with a KV value of 880.
Propeller	A self-tightening propeller of the T1045 model was selected with the specification of a 10-inch blade and pitch of 4.5°; constructed from nylon and glass fiber.
Electrical speed controller	The electronic transmission was a T-motor AIR 20A with a continuous current of 20 A.
Flight controller	The flight control module mounted on the drone was a Pixhawk 2 produced by HEX, which was equipped with Ardupilot V3.6.7 [13] open-source firmware, as shown in Figure 2b. The processor used a 32-bit ARM Cortex M4 core, and memory (RAM) and flash memory were 256 KB and 2 MB, respectively. Pixhawk 2 includes 3 inertial measurement units (IMUs), an accelerometer, and a magnetometer, barometer, and gyroscope. The Mission Planner is used as ground station [14].
GPS	A GPS receiver was chosen from the GPS positioning and navigation module produced by HEX, as shown in Figure 2c. This uses a Ublox NEO-M8P chip module, which can receive satellite systems such as GPS and GLONASS.

.

3.1.1. Flight Data and Vibration Signal Acquisition

In this research, we initially attempted to use an accelerator to collect the vibration signal. However, due to the limitation at a high frequency, a Sony PCM-A10 audio sensor was used instead, which is expressed specifically in Table 2.

Table 2. The introduction of the audio sensor and the computer used.

Sony PCM-A10	The audio recording sensor is shown in Figure 3. The recording format was LPCM 44.1 kHz, 16 bit/MP3 128 KBPS, and the frequency response was 40–16 kHz with the sampling frequency of 44.1 kHz. The output power of the audio sensor is 200 mW, the playback format is MP3, and its weight is about 82 g.
Computer	Windows 10 was the operating software using a system processor of Intel(R)Core(TM)i7-7700 CPU, memory (RAM) of 16 GB, 64 bit operating system, and x64 type processor.

Table 2. The introduction of the audio sensor and the computer used.

Sony PCM-A10	The audio recording sensor is shown in Figure 3. The recording format was LPCM 44.1 kHz, 16 bit/MP3 128 KBPS, and the frequency response was 40–16 kHz with the sampling frequency of 44.1 kHz. The output power of the audio sensor is 200 mW, the playback format is MP3, and its weight is about 82 g.
Computer	Windows 10 was the operating software using a system processor of Intel(R)Core(TM)i7-7700 CPU, memory (RAM) of 16 GB, 64 bit operating system, and x64 type processor.

3.2. Experiment Process

3.2.1. Flight Plan

The experimental environment and flight conditions of this study were formulated so that each damage flight test dataset was completed under the same environment and restrictions, thereby ensuring the consistency of the environment for capturing flight data from the quadcopter.

First, the flight tests were carried out in an open outdoor area with wind speed below 2 m/s and a flight time of 10–11 minutes. Second, to ensure that the flight test was not affected by insufficient power, the voltage of each battery in each flight test was not less than 3.7 volts. Third, the flight experiment used in this study was set at “Loiter Mode” to maintain the current position, heading, and altitude of the experimental drone. This mode must have a GPS with a 3D-position fix and Horizontal Dilution of Precision (HDOP) below 2.0. Fourth, an additional audio sensor was installed on the quadcopter, and the microphone was positioned at the proximity of motor.

The flight path of the quadcopter was planned using Mission Planner software and by loading path files. Seven commands for path planning were used in this study. Four waypoint positions were designed and each waypoint had a radius of two meters with height of five meters.

3.2.2. Faults Definition and Test

According to Cheng et al. [5], the following two conditions are defined as a “non-fault state”. First, it should be ensured that the faulty parts of the drone are in good condition; second, the drone must be able to fly stably for several minutes. A quadcopter that passes the above two conditions is defined as being in a “non-fault state”. In this study, follow-up research and testing of the UAV fault state were based on this definition of the “non-fault state”.

According to Misra et al. [15], where there is no damage or fault in the power unit of the quadrotor UAV, the fault states of the power unit loading components can be roughly divided into three types: the damaged propeller, the loosening of the motor base, and the loosened propeller. In addition, Cheng et al. [5] believe that there is also a risk of loosening the screws of the arm base connecting the power distribution board and the upper cover, causing the arm to loosen, which is also a fault state.

The three fault states tested in this study were considered to be critical but the drone was still flyable. The fault states were all occurred at the unique arm of the drone, namely, the two sides of the blades were broken shown in Figure 4a, the two diagonal screws of the motor base were loosened (red circle) shown in Figure 4b, and the two connecting screws between the arm base and the upper plate were loosened (red circle) shown in Figure 4c. In addition, we examined a mixture of the three fault states. Due to the combination of these three critical fault states, the quad is unable to execute normal flight, due to the poor mechanisms, but it can still hover stably. By collecting critical flight data, the ML model can classify the states that relate to the fault, while ignoring damage states that are too minor. The combination of the three fault states resulted in a table of eight fault states. The flight data was classified to determine if, in different kinds of damage situations, the types of damage can be effectively distinguished so that effective maintenance can be carried out and the damage immediately reduced.

4. Feature Extraction

4.1. Signal Preprocessing

The pre-processing of the signal is crucial because, if the redundant values are not eliminated, the results of feature extraction will be affected. The signals of take-off and landing were removed, and the acceleration signals below 5 Hz were also removed because we only need vibration signals rather than the those relating to the maneuver. The high-pass filter was used to filter the low signals, and only the frame and mechanism vibration signal were retained. The vibration signal was taken completely using the built-in high-pass filter function of MATLAB^®. Furthermore, after feature extraction, datapoints that lay outside of two standard deviations were removed. This constituted the extreme value removal method applied to eliminate extreme values, which enabled more stable classification by the model.

4.2. Time Domain Features

4.2.1. RCOU Signal

Remote control output (RCOU) is a parameter calculated by the flight controller and is a PWM (pulse-width modulation) signal to differentiate the speed of motors. The calculation source of RCOU is the input of the pilot, and RCOU also depends on the instant attitude of a drone. Then, the sources are input into the control loop and the control algorithm calculates the degree of attitude of the next step. In the final part of the control loop, four PWM signals are computed to provide the drone with the desired degree of attitude. Therefore, RCOU is a meaningful parameter for the analysis the behavior of drones, and is related to the health of mechanical components. Furthermore, the PWM signal normally has a strong relationship with the current; therefore, RCOU can also be an indicator of current.

4.2.2. Comparison of Features

For accelerometer and RCOU signals, the root mean square (RMS), standard deviation (STD), average (AVG), and sample entropy were applied to abstract the feature from the original signal. Additionally, the sample entropy was applied to the accelerometer because it is suitable for analysis of complex signals and identifying obscure patterns. In Figure 5, a simple demonstration of different kinds of states shows the discrepancy in magnitude between the features. It can be seen that the magnitudes of AVG and STD of the damaged propeller were larger than the others. However, it is difficult to distinguish non-fault and motor screw loosening using AVG and RMS, in addition to the STD features of both the PWM signal and accelerometer, because of the smaller difference. Nonetheless, the sample entropy is a powerful characteristic in this circumstance, and enables classification of slight differences. In Figure 5b, the difference between the three kinds of states can be clearly distinguished.

4.3. Time-Frequency Domain Features

In the characterization part of the audio sensor data, we used wavelet scattering data characterization and labelled the training data. The purpose was to evaluate the subsequent machine learning algorithm classification model and divide the experimental data into training data and testing data. In this research, we used the MATLAB^® function library and examples for testing.

4.3.1. Comparison of Features

The upper of Figure 6a shows the original signal of the non-fault state, where the horizontal axis is the frequency and the vertical axis is magnitude. The lower image of Figure 6a is the zero-order wavelet scattering coefficients, and shows that the numerical values are most frequently between −0.05 and 0.05. Figure 6b shows the visualization of Equation (14), and represents the first-order scattering coefficients. The values most frequently occur between 0.1 and 0.7, and the figure shows the analyzing frequency interval of wavelet scattering, which is 5–22,050 Hz the half of maximum sampling rate. Compared to the other first-order scattering coefficient scattergrams (Figure 7b,e), no obvious feature is seen.

On the contrary, the visualization second-order scattering coefficient (Equation (15)) in Figure 6c provides sufficient characteristics. Compared to the damaged propeller state (Figure 7c) and loosening of the motor screw (Figure 7f), the discrepancy of the scattergrams can be clearly detected by the human eye, in addition to being classified by the machine learning method. Apparently, the second-order scattering coefficient has an excellent ability to find obscure information; this why we used this novel method.

5. Results and Discussion

5.1. Machine Learning Model

5.1.1. Establishment of Models

As mentioned above, two data sources were used to build the two ML models: (1) vibration and RCOU signals obtained from the flight data files, as shown in Figure 5d–f; and (2) data based on the audio sensor signal, as shown in Figure 6 and Figure 7. These features reveal information inside the data, and we built two vectors that contain the time-domain and time-frequency domain features. Then, using the two training datasets, two classification models were built to analyze their accuracy. Both of these models used the nearest neighbor (kNN) classification model, but their parameters were different. Through repeated experiments, the model parameters that are most suitable for the flight data parameters of the quadcopter were found. In this research, we used the MATLAB^® nearest neighbor classification model function library.

The kNN machine learning algorithm was used because we aimed study the efficacy of the features. Thus, by using the simple ML algorithm, the results could be as clear as possible. In the process of building the kNN model, several parameters were tuned to identify the best classification model.

Additionally, we used confusion matrixes to evaluate the model. In the field of machine learning, a confusion matrix is a comparison table for evaluating classification models, which shows the accuracy of the model and the individual discrimination rate. The confusion matrix is mainly evaluated through the proportional relationship between actual and predicted values. The accuracy formula of the confusion matrix is shown below:

$$\mathrm{Accuracy}=\frac{\mathrm{True} \mathrm{Positive}}{\mathrm{True} \mathrm{Positive}+\mathrm{False} \mathrm{Negative}+\mathrm{True} \mathrm{Negative}+\mathrm{False} \mathrm{Positive}}$$

(18)

5.1.2. Deployment of Models

The main comparison between the models was based on two sources, namely, the data from the flight controller and the data from the audio sensors. This deployment was used to assess the ability of the intrinsic information of additional sensors. First, ML models were built only with non-fault and fault states. This step determines whether the fault has occurred. Second, models of single faults, double faults, and triple faults were trained to evaluate the classification performance. Based on the deployment of two sensors, we analyzed the prediction results and used the achieved accuracy as an indicator to assess the performance of models.

In the following paragraphs, the model built using the data from the flight controller is also called the time-domain model, because the features it used could only be extracted by time-domain features. Alternatively, the model built using the data from the audio sensors is called the time-frequency domain model.

5.2. Analysis of the Fault Classification Model of a Single Component

5.2.1. Analysis of the Time-Domain Model

This section analyzes the fault state classification rate by comparing the non-fault state with other single fault states. Additionally, it also illustrates the effective arrangement of parameters.

Note that average, root mean square, standard deviation, and sample entropy are the features of the training data. Among the three kNN machine learning models’ parameter settings, the three main tuning parameters were the distance metric, distance weighted, and the number of nearest neighbors. For the model built for the non-fault state and the damaged propeller, the second type of parameter setting had the highest accuracy for the damaged propeller state. The accuracy was 91 percent and the running time was 1.32 s (

Table 3. Comparison of parameters of models trained using the time-domain feature when the propeller is damaged.

	1	2	3
Tie-Breaking Algorithm	Nearest Point Search	Nearest Point Search	Nearest Point Search
Distance Metric	Euclidean distance	Cosine similarity	Cosine similarity
Distance Weighted	Same weight on each point	Same weight on each point	Inverse square on each point
Nearest Neighbor Search	Brute-force search	Brute-force search	Brute-force search
Number Of Nearest Neighbor	3	3	7
Standardize	Yes	Yes	Yes
Running Time	1.18 s	1.32 s	0.63 s
Accuracy	87%	91%	90%

).

The same steps were used to find the kNN classification model of the other fault states. The best fault kNN classification model could then be identified based on time-domain (flight controller) data analysis. The same process was also able to be used in the analysis of double and triple faults.

5.2.2. Analysis of Time-Frequency Domain Model

As mentioned in Section 4.3.1., the second-order scattering coefficient shows strong classification capability. The high values compared to the non-fault state in the second-order scattering scattergram (Figure 7c) indicate that the analytical scattering coefficient values can be used as the characteristics of the fault state of the propeller fault.

The labels were set on the zeroth-order scattering coefficients graph, the first-order scattering coefficients graph, and the second-order scattering coefficients graph of the propeller fault signals. Then, the kNN model was built using the labeled features which contain non-fault and fault states. Among the three kNN machine learning model parameter settings, the first type of parameter setting achieved the highest accuracy for the damaged propeller state. The accuracy was 98.77 percent and the running time was 112 s (

Table 4. Comparison table of model parameters used by the time-frequency domain for the propeller fault.

	1	2	3
Tie-Breaking Algorithm	Nearest Point Search	Nearest Point Search	Nearest Point Search
Distance Metric	Euclidean distance	Cosine similarity	Cosine similarity
Distance Weighted	Same weight on each point	Same weight on each point	Inverse square on each point
Nearest Neighbor Search	Brute-force search	Brute-force search	Brute-force search
Number Of Nearest Neighbor	3	3	7
Standardize	Yes	Yes	Yes
Running Time	112 s	108 s	108 s
Accuracy	98.77%	97.77%	98.04%

).

The single fault kNN models of the time-frequency domain data have better classification effects, but the results of the two models both achieved an accuracy rate of over 88.5 percent as shown in

Table 5. The best single fault kNN classification model based on time-domain data analysis.

	Damaged Propeller	Loosening of Motor Screw	Loosening of Arm Screw
Running Time	1.32 s	0.67 s	0.61 s
Accuracy	91%	88.5%	92.75%

and

Table 6. The best single fault kNN classification model based on time-frequency domain data analysis.

	Damaged Propeller	Loosening of Motor Screw	Loosening of Arm Screw
Running Time	112 s	107 s	111 s
Accuracy	98.77%	91.70%	98.78%

. It can also be found that the loosening of the motor screw had a lower accuracy rate because the fault is relatively slight and does not produce a visible change in the data. Regarding the running time, the model built using time-domain data had a faster solution speed due to less computational load. Other models used a large amount of data, resulting in a longer running time, and the wavelet scattering transform also needed considerable computation time.

5.3. Analysis of the Combined Fault Model

5.3.1. Double Faults

In the comparison of the double fault model, the double fault kNN models for time-frequency domain data analysis all yielded better classification results than the time-domain model. Compared to the single fault kNN model, the accuracy rate maintained a high accuracy rate of over 93.5% as shown in

Table 7. The best double fault kNN classification model based on time-frequency domain data analysis.

	Damaged Propeller and Loosening of Arm Screw	Loosening of Motor Screw and Loosening of Arm Screw	Loosening of Motor Screw and Damaged Propeller
Running Time	216 s	216 s	199 s
Accuracy	97.72%	98.77%	93.52%

The accuracy rate of the double fault kNN model for the time-domain data analysis was around 70–75%, which indicates a reduction in the accuracy rate of nearly 20% as shown in

Table 8. The best double fault kNN classification model based on time-domain data analysis.

	Damaged Propeller and Loosening of Arm Screw	Loosening of Motor Screw and Loosening of Arm Screw	Loosening of Motor Screw and Damaged Propeller
Running Time	0.65 s	0.57 s	0.77 s
Accuracy	75.12%	76.75%	70.75%

. It can also be found that the accuracy of the loosening of the motor screw and the propeller fault still had low accuracy, whereas regarding the running time, the double fault kNN model of the time-domain data analysis still had a faster solution speed.

5.3.2. Triple Faults

Compared to the triple fault kNN model, the double fault kNN models had a better classification effect and accuracy due to the increase in complexity. However, as shown in

Table 9. The best triple fault kNN classification model based on time-frequency domain data analysis.

	Damaged Propeller and Loosening of Motor Screw and Loosening of Arm Screw
Running Time	446 s
Accuracy	90.73%

, the time-frequency domain model still maintained an accuracy rate of over 90%, and the accuracy rate of the triple fault kNN model for time-domain data analysis was close to about 60% (

Table 10. The best triple fault kNN classification model based on time-domain data analysis.

	Damaged Propeller and Loosening of Motor Screw and Loosening of Arm Screw
Running Time	0.65 s
Accuracy	58.6875%

). It can be found that the more types needing to be classified, the greater the reduction in the accuracy rate of the kNN classification models. The accuracy of the time-domain data model was much reduced, whereas in terms of running time, the triple fault kNN model of time-domain data analysis had a much faster solution speed, However, the running time of the other models was significantly longer than previously.

5.4. Model Comparison

The final models are built by eight states which are non-fault, individual faults, and melded faults. Due to multiple combination, the simple representation is expressed in

Table 11. Simple expressions used to represent states.

State	Mark
Non-fault	A
Damaged propeller	B
Loosening of motor screw	C
Loosening of arm screw	D
No specific classification	E

.

Table 12. Confusion matrix of the model based on time-domain data.

	Prediction	A	B	C	D	B + C	B + D	C + D	B + C + C
Actual
A		167	3	10	4	4	3	4	5
B		6	106	13	24	6	16	14	15
C		6	6	111	2	3	5	29	38
D		16	20	9	111	12	11	14	7
B + C		15	15	18	14	89	14	23	12
B + D		12	19	13	15	12	91	15	23
C + D		1	3	22	0	1	3	137	33
B + C + D		0	3	28	3	3	10	26	127

and

Table 13. Confusion matrix of model based on time-frequency domain data.

	Prediction	A	B	C	D	B + C	B + D	C + D	B + C + C	E
Actual
A		181	2	0	26	0	0	1	0	0
B		1	196	0	0	0	0	0	0	0
C		13	1	185	14	2	0	16	4	0
D		5	1	0	154	0	0	1	0	0
B + C		0	0	1	1	195	1	1	2	0
B + D		1	0	0	0	1	203	1	0	0
C + D		0	2	16	3	1	0	177	0	0
B + C + D		0	4	2	1	7	0	2	197	0
E		2	0	3	6	2	0	4	1	0

show the result between two models. Table 12 shows that the percentage of classification errors is related to the fault of the motor screw loosening. The main reason for this is that the features of loosening of the motor screw are relatively inconspicuous, and are thus easily misclassified.

In addition, Table 13 shows another confusion matrix for non-specific classifications, especially in any non-fault or other fault states that have no match in the significant features with the real data. There are two main reasons for this. The first relates to the pre-processing of data: the dataset with unobvious features was not completely eliminated. The second is that the parameter setting of the kNN model does not force the classification of specific non-fault or fault types, thus generating predictions as a non-specific classification.

6. Conclusions

6.1. Contribution

The main contributions of this research are summarized as followings:

• In this study, we successfully used the flight data of a quadcopter to classify mul-tiple fault types in the quadcopter structure, including the classification of the non-fault state, single faults, double faults, and triple faults. Even in the cases of the non-fault state and a mixture of three faults, the classification system achieved an accuracy of 90.73%.
• In this research, we developed a health diagnosis classification system for the structure of the quadcopter. This system is composed of two kNN models. The first classification model using flight controller sensor data can be used for the classification of single faults in equipment. The recovery rate is high and it less time is required for analysis. The second time-frequency domain data classification model can be used for the classification of double fault and triple fault states. It takes a long time to analyze but has high accuracy.
• In this study, the wavelet scattering theory was successfully used to analyze the characteristics of the audio signal of the quadcopter. The method is a highly practical signal analysis tool, which can effectively identify the characteristics of various fault signals from the quadcopter. This application is also the main contribution of this research.

6.2. Summary

• The results of the comparison indicate the data from intrinsic sensors have inherent disadvantages. Compared to the audio sensor, the sampling rate is too low, which means that this data contains less information. However, for flight safety, it is dangerous to add too much load to the computer. There is still an apparent limit to the use of intrinsic sensors.
• The classification ability and recovery rate decrease with the increase in the number of types of faults. In addition, this study found that there is a trade-off between the accuracy and running time. Real-time classification requires additional sensors or light and powerful algorithms for effective classification.
• Vibration analysis requires a significant amount of data, and the frequency-domain analysis is required for advanced classification. The above demands require a large amount of computational power; therefore, complex vibration analysis may not be suitable for real-time diagnosis. Nevertheless, a health monitoring system can still be used on the server side, and the health states can be tracked using its powerful classification ability.