Classification of Grain-Oriented Electrical Steel Sheets by Magnetic Barkhausen Noise Using Time-Frequency Analysis and Selected Machine Learning Algorithms

Abstract

In this paper, a combination of Magnetic Barkhausen Noise (MBN) and several classical machine learning (ML) methods were used to evaluate both the grade and the magnetic directions of conventional and high grain oriented electrical sheets subjected to selected surface engineering methods. The presented analysis was conducted to compare the performance of two machine learning approaches, classical ML and deep learning (DL), in reference to the same MBN examination problem and based on the same database. Thus, during the experiment, 26 classical ML algorithms were used including decision trees, discriminant analysis, support vector machines, naïve Bayes, nearest neighbor, artificial neural networks and ensemble classifiers. The experiments were carried out considering a different number of recognized magnetic directions and hence the number of determined classes as well. The results of classification accuracy of the applied ML methods were compared with those obtained for the DL model presented in a previous paper. The highest accuracy was obtained for ML models based on artificial neural networks and ensemble bagged trees. However, the accuracy did not reach 89% in the best case—for the smallest number of determined classes. Nevertheless, the achieved results generally indicated an approx. 10 percent advantage of the deep learning model over the classical ones in terms of accuracy in each of the considered cases.

1. Introduction

Grain-Oriented (GO) electrical steels are frequently used for industrial electrical engineering applications. They are particularly important in shaping electromagnetic field distributions during the production of machines, such as transformers, in order to optimise these devices’ operational properties [1,2,3]. The magnetic parameters of these materials have been significantly improved due to technological progress in their manufacturing process [4]. Various surface engineering methods applied to GO electrical sheets, such as laser or mechanical scribing, have made it possible to reduce losses. Consequently, these techniques have an impact on the increase in efficiency of the electrical machines. GO steels are characterized by preferable magnetic performance depending on the direction of magnetization and this effect is known as magnetic anisotropy. Thus, the material contains axes of easy and hard magnetization. When an external magnetic field is applied to a ferromagnetic material in the direction parallel to the axis of easy magnetization rather than the axis of hard magnetization, the reorganization of the domain structure occurs with greater activity and dynamics. Thus, to obtain the desired properties of the designed electrical machines during production, the ability to carry out quick non-destructive verification tests of the angular magnetic properties of the steel sheet becomes crucial.

To determine the directions of the axes of easy and hard magnetization, various methods [5,6,7,8,9] have been used for example: induced magnetic field [7], torque [5] or magnetization curves [5] measurement methods. A promising method for measuring axes of easy and hard magnetization is the magnetic Barkhausen noise method. It is related to the Barkhausen effect occurring during the process of magnetic domain structure reorganization which takes place when the AC magnetic field is applied to the ferromagnetic material. The Barkhausen effect has a stochastic, non-stationary nature and its course strictly depends on the microstructure and mechanical stress of the tested material [10]. The rearrangement of the domain structure causes local abrupt changes of the magnetization in the sample. When a coil is applied to such a material, a corresponding voltage will be induced in the coil, called the MBN signal [11]. Considering the close relationship between the mechanical and magnetic properties of ferromagnetic materials, it becomes possible both to influence the magnetic properties of steel using engineering methods and to test those steels subsequently as well [10]. Due to these properties, the MBN method is also applicable to study the anisotropy of the magnetic properties of steel [12,13,14,15,16,17,18,19]. The subjects of the research are often steel elements used in pipelines services. Strongly directional magnetic properties in these elements, e.g., can have a large impact on the effectiveness of the non-destructive method used to test these components, such as the magnetic flux leakage [10,12,14,15]. In [12], the authors used the MBN method to investigate the origin of the magnetic easy axis and characterize the anisotropy of pipeline API X70 steel. In paper [12], the authors proposed a method for determining the magnetic easy axis affected by the roll magnetic anisotropy in API-5L steel, while in [14,15], the authors presented the results of a study on the identification of various processes distinguished in magnetization dynamics based on the analysis of an MBN course. In [17], the authors presented the results of research on the utilization of MBN for the monitoring of variation in magnetic anisotropy affected by crystallographic texture in cold rolled low carbon steel sheets. Anisotropy was also tested in grain-oriented steels. Exemplary, in [13] the authors demonstrated the possibility of conducting quick tests and showing the relationship between the initial permeability and the root mean square of the MBN signal.

Nevertheless, given the complexity of Barkhausen noise signals, the application of data processing techniques becomes a critical step in obtaining often hidden and complex components of knowledge about the state of the material under study. The majority of MBN analyses are performed in the time domain T, however the frequency domain F is also considered [20]. However, it should be noted that the MBN signal changes both in time and frequency, and the nature and dynamics of the Barkhausen effect are dependent on the type of material and the homogeneity of its structure. As a result, it becomes crucial to track changes in the dynamics of the MBN course over time. Thus, a variety of analytic approaches based on time-frequency (TF) transformations of the MBN signal have recently been developed [21,22,23,24,25]. The article [21] presents an analysis of the MBN TF representations obtained for the determination of the influence of case hardening on MBN characteristics. Several different transformation algorithms were applied and qualitatively compared. Studies [22] and [23] analyze MBN spectrograms to investigate the processes taking place in electrical steel sheets during magnetization and provide a quantitative description of these phenomena as a function of the measurement angle. In [23], a resultant anisotropy of the steel was analyzed based on the TF spectrograms of the entire MBN burst, while in [22] the possibility of determining the influence of various factors on the magnetization dynamics was considered by extraction of bursts subsequences. In [23] and [24] the authors discussed the effect of signal processing parameters and the influence of measurement conditions. For that reason, various magnetization currents, frequencies and shapes of the excitation signal on the obtained TF spectrograms were considered to achieve the greatest distinguishability between the states of the examined steel. In article [25], the time-frequency analysis along with the J-A model was used to study the effect of stresses. The discussed works showed that the analysis of combined time and frequency (TF) domain representations can provide information that is additional to knowledge gathered through traditional approaches. It should be noted, however, that no single feature of the signal expressed in any domain adequately explains the influence of all physical factors affecting a ferromagnetic material. As a result, typical signal analysis approaches may not be enough. Therefore, there is an increasing necessity to apply techniques of searching for complicated rules and relationships within measurement data to build knowledge about the tested material based on multiple features analyses. Two issues are important in this area. The first concerns the search for human-definable features in the measured signals. The second relates to the use of methods based on automatic search for features that do not have straight definition. Currently, methods based on artificial intelligence are very popular in research on finding patterns from data. Various methods, such as machine learning algorithms, shallow and deep artificial neural networks, are used. In the context of MBN, several works [26,27,28,29] using artificial intelligence methods to extract information from signals can be found in the literature. In [26], a Deep Convolutional Neural Network (DCNN) was used to classify various CGO and HGO electrical sheets subjected to a different method of surface engineering. In addition, the proposed deep network model allowed for the evaluation of the angular deviation of a magnetic easy axis from the rolling direction. In [27], the MBN signals expressed in the time domain and autoencoders were used to estimate the magnetocrystalline energy of the examined steel, whereas in [28] a convolutional neural network for the quantitative description of plastic deformation was proposed. Last but not least, paper [29] presented an exploratory data analysis and the application of machine learning methods to describe the surface quality of a material.

Additionally, considering the degree of the measured signal processing, the algorithms can be fed with data representing different abstraction levels. In case of the first representation type, the original raw signals are used (data of low abstraction level). The second representation refers to the features vector (data of medium abstraction level) extracted from those measured raw signals (i.e., characteristic or statistical value such as median, mean, maximum or minimum). Under the first representation type, the full information gathered within the measured signal is transferred to the processing algorithms. In the case of the second representation type, the computational effort is much lower, however, it carries a possible loss of information. In the previous paper [26], the authors presented the first type of approach, where the DCNN was utilized. Classification was performed on time-frequency spectrograms of measured MBN signals. This allowed the authors to obtain good classification results. Deep learning makes it possible to search for complex relationships between data, and especially in the case of a very large data set, it shows very good performance. In contrast to the direct feature engineering used in classical ML, the DL procedure is far more complicated, and the results are more challenging to interpret. Therefore, in this paper, to verify the performance and assess the possible advantage of the first type approach to MBN analysis, the second type of measured data representation was used. For this need, first, the TF features were extracted from the TF spectrograms obtained in [26]. Then, considering feature vectors for each measured case, a further classification approach was performed using several machine learning methods. For the broad spectrum of analysis, the classification task was realised using several frequently applied standard (classical) machine learning methods, i.e., decision trees, discriminant analysis, support vector machines, naïve Bayes, nearest neighbor, artificial neural networks and ensemble classifiers. The work was carried out on the same TF representation of measured data, thanks to which it is possible to directly compare the performance of both approaches, classical ML and DL. This type of research has not been conducted so far for the purpose of analyzing MBN signals; thus, such a comparison may be beneficial in the future development of expert systems based on this method.

2. Experimental Setup

The measurements were carried out for samples made of several grades of 3% SiFe grain-oriented electrical steel sheets. Figure 1 shows the rotation machine operating in four axes (X, Y, Z and A) with the attached transducer placed over the sample. The transducer combines two sections: an excitation section and a measurement section. The transducer is made up of a C-shaped core with an excitation coil and a coil for measuring the magnetic flux density (B-coil). The MBN signal measurement section consists of two coils wound on a ferromagnetic core (MBN-coil). The details of the transducer and the measuring system were given in [22,23,30].

Conventional (CGO) and high grain-oriented (HGO) steel grades were utilized for sample preparation. The steel sheets had different thicknesses of 0.23 mm, 0.27 mm and 0.30 mm, and were subjected to various surface engineering methods, including coating with and decoating of oxide layer, laser scribing of the surface or nitriding. A detailed description of the tested samples, along with the metallographic assessment of their properties, can be found in the previous paper [26]. The set of the inspected samples with their basic description is given in Figure 2.

For each sample, the measurements were made for different angular positions of the transducer between 0° and 180° using an angular step of 1.8° (for 0° position the main axis of the transducer is along the axis transverse (TD) to the rolling direction of steel; for 90° position the main axis of the transducer is along the rolling direction of steel—RD). The excitation signal was sinusoidal with a frequency of 10 Hz. The field strength obtained during the measurements was about 1.6 kA/m. MBN signals were collected via a data acquisition (DAQ) card (NI-PCI-6251) using a sampling frequency of 250 kS/s. Prior to the acquisition, first, the signals were bandpass filtered in the frequency range of 2 kHz to 100 kHz. Next, the gathered MBN signals were transformed into TF representation using short-time Fourier transform (STFT). It allows for the determination of the content of sinusoidal components of different frequency and phase in local sections of the base signal as it changes over time [22,23,30]. In the transformation process, the Kaiser function with a size of 512 points was used, which resulted into a resolution in time of 512 µs and a frequency of 488 Hz.

3. MBN Signals Time-Frequency Transformation and Feature Extraction

Further, the MBN TF representations S_BN(t, f) were expressed in the form of spectrograms |S_BN(t, f)|². Figure 3 shows the exemplary spectrograms, obtained for MBN signals acquired for four different samples and the orientation of the transducer along the RD (90°): the CGO steel sample with thickness of 0.23 mm (C_23#1), the CGO steel sample with thickness of 0.27 mm after decoating of the oxide layer (C_27P#2), the CGO steel sample with thickness of 0.27 mm after nitriding process (C_27N#2) and the HGO steel sample after laser scribing (H_23LS#1).

The results were presented in common range and colour map was expressed in linear scale. In the case of the CGO C_23#1 sample (Figure 3a), there are two clearly visible areas of increased activity. The first is in the vicinity of 0.02 s, and the next is in the subperiod from about 0.03 to about 0.04 s. Figure 3b shows a spectrogram achieved for the laser scribed HGO H_23LS#1. In this case, on the obtained TF characteristics, three clearly distinguished areas of activity can be noted, which also cover a much wider time segment of the magnetic field half-period than in the previous cases. Figure 3c shows the spectrogram obtained for CGO sample C_27P#2 subjected to decoating of the oxide layer. As it can be seen, the presented characteristic of activity in the TF space is similar to that presented in Figure 3a in the sense that it also shows two distinguished areas of activity. Nevertheless, in this case, the areas of this activity cover a smaller space of the TF distribution, which is especially noticeable in the case of the second activity area. In addition, the amplitudes over the spectrogram, in general, take lower values (it should be emphasized that both sheets come from different manufacturers thus are characterized by a different value of magnetic permeability and have different thickness as well, which affects the differences in received power density levels). In general, at least the three key phases of the magnetization process can be distinguished for GO steels (visible especially when magnetization direction is close to the easy magnetization axis): the first is related to the reversed domain nucleation process bonded with the magnetocrystalline anisotropy, while the second is related to the movement of the 180° domains occurring close to the main MBN peak, and the third is related to the movement of the 90° domains [22]. While in the case of HGO three areas that may refer to the described magnetization phases are clearly visible, in the other two cases (Figure 3a,c) they may be hidden. In the case of C_23#1, the first visible activity subperiod is relatively compact in time while the second one is more stretched. This may indicate the overlapping of the subperiods of the second and third magnetization phases in that situation. Similarly, in the case of C_27P#2, around 0.04 s, one more area with a much lower level of value can be noticed. Figure 3d shows the spectrogram of nitrided CGO steel which was first subjected to the oxide layer decoating process. This spectrogram is characterized by only a single, but wide, area of activity, which proves the significant influence of nitriding on the original structure of the material. A broader presentation of the spectrograms obtained during the research and the discussion is presented in the previous paper. Nevertheless, based on the presented exemplary distributions of |S_BN(t, f)|², changes in the course of the instantaneous dynamics and the degree of activity of the MBN can be clearly noticed depending on the steel grade and machining process.

Next, further processing is required. Thus, the feature extraction procedures were performed to quantify information expressed by the TF spectrograms and track the changes in the analysed distributions. These features [30] refer to the description of instantaneous, as well as general, changes in the time-frequency representation. They allow for the definition of 66 features which can be divided into three major groups: describing statistical properties (such as minimum, maximum, mean, median, centroid, variance, standard deviation, skewness or kurtosis, etc.); describing the dynamics of variance, uniformity of distribution or degree of disorder (i.e., the TF distribution’s shape, the energy or entropy); describing the characteristics’ values referring to the symmetry, centre shift in the TF plane, flatness, homogeneity or monotonicity, etc. The definitions and the properties of the used TF features were discussed in previous paper by the authors [30]. Considering only the features calculated from the entire spectrogram, the vector dimension was reduced to 13 selected features.

Feature extraction was performed for all spectrograms. Thus, a feature vector was obtained for each examined steel grade and each measured angular position of the transducer. Pictorially, the features can be presented using a polar chart. Such a presentation makes it possible to observe how the value of a given feature changes with the angular position of the transducer, which allows for the evaluation of the angular distribution of the magnetic properties of a given steel. Exemplary polar charts are presented in Figure 4.

The figure shows the obtained angular characteristics of the four selected features for the single selected sample (the CGO steel after laser scribing C_27LS#1). Each chart was normalized to the (0–1) range. As it can be seen, these distributions have a different shape depending on the given TF feature. Figure 4a presents the distribution of the concentration measure feature BN_CM. As it can be seen, from 0° to about 75° there is a monotonic increase in the value, and then after reaching maximum value, a slight decrease in the value of the feature is noticeable until an angle of 90° is reached. The BN_CM feature assumes higher values for spectrograms with an even energy distribution across the entire TF plane, but at the same time it is not sensitive to small values. Therefore, as the level of MBN activity increases throughout the spectral band while approaching the vicinity of the easy magnetization axis, this feature also takes higher values. Figure 4b presents the distribution of the spectral entropy feature BN_SE. The value of this feature in the angular interval between 0° and 60° maintains almost a constant close to 1, while for angle between 60° and 90°, there is a sharp decrease in the value to 0. Because BN_SE can assess the degree of disorder in a distribution, when the feature reaches values close to 0 for the direction close to the axis of easy magnetization, it can be interpreted as leading the distribution to attain a higher degree of order and to accommodate the existing energy states. Figure 4c shows the coefficient of variation BN_CoV feature. The value increases from 0° to about a 90° angle, where at about 90° it obtains the highest value. The BN_CoV expresses the ratio of the mean and the standard deviation values and the observed monotonic increase of its value from TD to RD direction confirms previous observations. The last presented feature distribution (Figure 4d) relates to the statistical quantity of the spectrograms—the median value BN_Median. The BN_Median characteristic is well corelated with the concentration measure BN_CM feature. This supports prior findings that at angles congruent with or near the RD direction, spectrogram’s value essentially grows over the whole TF space. The resulting distributions allow for the same conclusions to be made. A general indication of the directions of high and low MBN activity can be seen; these directions are also consistent with the directions of easy and hard magnetization, respectively.

Figure 5 presents the angular characteristics of a single selected feature, BN_CM, for three selected samples of different grades: 0.30 mm thick CGO steel C_30#1, 0.30 mm thick laser scribed CGO C_30LS#1 and 0.27 mm thick nitrided CGO steel C_27N#2. All distributions were normalized to the greatest value from all distributions. At first, one can notice the significant difference in the BN_CM value levels between samples. Furthermore, for each sample, a different course of characteristic is obtained. In the case of C_30#1 (a red-dotted chart), one can observe a monotonic, generally constant, growth of the feature’s value, starting from the TD direction and heading into the RD direction, with the maximum close to the easy magnetization axis. The characteristic of the BN_CM for the C_30LS#1 laser scribed CGO (yellow-dotted chart) takes much smaller values compared to the results obtained for other presented in the figure; however, as in the first case, the distribution presents a relatively narrow angular range where the greatest feature’s value occurred. In case of the nitrided CGO C_27N#2 (purple-dotted line), a great increase in value of the feature in the angular range from 0° to about 60° can be noticed, while after reaching the greatest value, a gentle decrease in the value is further observed while approaching the 90° angle.

Due to the various characteristics of the features, it is necessary to apply an appropriate decision-making method. It is possible to use an expert to make this decision but then there is a human factor that can cause erroneous predictions (emotions, not-well physical condition, random happenstance). As a substitute for this, an algorithmized decision-making mechanism in the form of a machine learning model can be used.

The next part of the paper will present the methodology for conducting research using machine learning algorithms, which will be described, and a summary of the results obtained will be presented.

4. Machine Learning (ML) Algorithms and Results

Artificial intelligence algorithms are becoming increasingly powerful nowadays through the increasing computing capabilities of computers. One of the most accurate machine learning algorithms is artificial neural networks (ANN). Nowadays, considering the last decade of advancement, one can split ANN into shallow and deep networks. The difference between the two groups refers to the number of layers between the input and output, called the hidden layers. The shallow ANN are made up of much fewer layers completing a specific task, while the deep networks can have a much more complex structure, consisting of layers that can perform different functions. In this paper, the shallow ANN were used. The ANN can be categorized considering the information transferring direction, into: feed forward and backpropagation [31]. The first type is characterized by the fact that the input data goes forward after each neuron and returns the result. In the second case, the error (the difference between the expected output and the received one) is calculated and then the weights are updated to minimize the error. Over the years, the ANN models have been refined to produce good classification and regression results. The inspiration for such an algorithm originates from the connections of biological neurons. They are used in various issues, such as in ocean engineering [32], to forecast performance of solar still [33], as well as in medical applications [34]. Artificial neural networks have also been used in assessing the degradation state of a material in contact with hydrogen, which can change the mechanical properties of the material [35]. The ANN network, along with the application of an algorithm to optimize its layers, has been shown to be an effective tool for predicting product decomposition in solid waste gasification [36]. ANNs have also found use in the analysis of MBNs, for example, to assess the stress values [37,38] or hardness and case hardening depth [39]. It should be mentioned that various classical machine learning techniques, such as support vector machines (SVM), decision trees, Bayesian approaches or regression models also have been used recently as a result of improved computational capabilities. Each algorithm is based on different methods for extracting patterns from data. For example, SVM can be used to implement linear or non-linear classification. The main part of which is to set an appropriate margin between the separating hyperspace and learning examples. Meanwhile, in the case of decision trees, the decision-making process is carried out by adding more nodes and branches to the tree. This process proceeds from the perspective of information increment towards the selected feature. It is repeated until each side reaches the leaf nodes. The algorithm evolved into the random forest algorithm which is a blackbox type algorithm [40,41]. One of the simplest classifiers that uses a measure of the distance between objects in multidimensional space is k-nearest neighbor (kNN). Since it is assumed that items near to one another are similar, objects are classified in the database according to the class of their nearest neighbors. The naïve Bayesian classifier assumes that features in a class are not dependent to any other feature. New data are classified based on the highest probability of belonging to a specific class. Discriminant analysis classifies data by finding linear combinations of features. Machine learning is used nowadays in many applications, e.g., rock type recognition using self-organizing maps [42], in logistics [43] or medical applications [44]. These methods have been refined over the years. When the use of a single model becomes insufficient, the use of ensemble learning methods is applied, which refer to combining several weaker models into a stronger ensemble. In the case of this article, e.g., a bagged decision tree algorithm is used. It allows one to select individual trees and then choose the class that received the best result [40], but the algorithm is complex and difficult to interpret. These methods are supported by appropriate modifications, for example, bagging (which is short for Bootstrap AGGregatING), boosting or stacking. In the case of MBN, the machine learning algorithms have been used to assess surface quality [29], predict residual stress [20,45], detect Barkhausen large jumps [46] and predict hardness and case depth [45,47]. All these applications show the great importance of the applied ML approach on the performance of the MBN method, which confirms the need to consider the route in further applications.

4.1. Discription of Considered Set of Classical ML Algorithms

In this paper, a wide variety of classifiers were employed in the task, and several versions were evaluated in each case to provide the possibility of broad comparisons of performance between classical ML and DL approaches. Considering the properties discussed in the main section of the paper, frequently applied types of classifiers have been used to conduct experiments, that is, decision trees, discriminant analysis, SVM, naïve Bayes, kNNs, ensemble classifiers and ANNs. The decision trees and discriminant analysis are relatively fast and are not characterized by high complexity of interpretation. kNN has similar properties. On the other hand, they may not provide satisfactory efficiency of the prediction process. The naïve Bayes classifier, although it is relatively simple, often outperforms more sophisticated classifiers. Models based on SVM and ANN allow for high accuracy, but they are much more complex. Ensemble learning combining multiple base classifiers (e.g., decisions tress or nearest neighbor) enables classification rules to be strengthened to achieve higher predictor accuracy. Thus, the wide experiment was introduced to search for a promising classifier type and its configuration. The following factors were taken into consideration while choosing the settings for each variant: training speed, memory utilization, prediction accuracy and response complexity.

Table 1. Classifiers with selected hyperparameters.

Tree	Fine	Medium	Coarse
Maximum number of splits	100	20	4
Split criterion	Gini’s diversity index	Gini’s diversity index	Gini’s diversity index
Discriminant	Linear	Quadratic
Preset	Linear	Quadratic
SVM	Linear	Quadratic	Cubic	Gaussian(with scaling)
Kernel function	Linear	Quadratic	Cubic	3xGaussian(with scaling) Kernel scale (1, 3.6, 14)
Multi-class method	One vs. One	One vs. One	One vs. One	One vs. One
KNN	Fine	Medium	Coarse	Cosine	Cubic	Weighted
Number of neighbours	1	10	100	10	10	10
Distance metric	Euclidean	Euclidean	Euclidean	Cosine	Minkowski (cubic)	Euclidean
Distance weight	Equal	Equal	Equal	Equal	Equal	Squared inverse
Naïve Bayes	Gaussian Naïve Bayes
Preset	Gaussian Naïve Bayes
Neural Network	Narrow	Medium	Wide	Bilayered	Trilayered
Preset	Narrow neural network	Medium neural network	Wide neural network	Bilayered Neural Network	Trilayered Neural Network
Number of fully connected layer	1	1	1	2	3
First layer size	10	25	100	10	10
Second Layer size	X	X	X	10	10
Third Layer size	X	X	X	X	10
Activation	ReLU	ReLU	ReLU	ReLU	ReLU
Iteration limit	1000	1000	1000	1000	1000
Regularization strength	0	0	0	0	0
Standardize data	Yes	Yes	Yes	Yes	Yes
Ensemble	Boosted Trees	Bagged Trees	Subspace Discriminant	Subspace KNN	RUSBoostedTrees
Preset	Boosted Trees	Bagged Trees	Subspace Discriminant	Subspace KNN	RUSBoostedTrees
Ensemble method	AdaBoost	Bag	Subspace	Subspace	RUSBoost
Learner Type	Decision Tree	Decision Tree	Discriminant	Nearest neighbours	Decision Tree
Maximum number of splits	20	25,999	X	X	20
Number of learners	30	30	30	30	30
Learning rate	0.1	X	X	X	0.1
Subspace dimension	X	X	7	7	X

Note: The red frames depict the algorithms and their base configuration for which the highest accuracy was obtained during the computational experiment.

lists seven classifier types considered during the experiment together with the most crucial variables for each of their particular versions. In connection with the theory about the lack of free lunches in machine learning [48] and the impact of multiple elements related to the dataset, multiple classifiers were analyzed in the context of the set of features created from the spectrograms.

The experiments were realized using an application built into MATLAB software called Classification Learner [49]. The experiment was carried out for the set of configurations listed in the Table 1. The data transformation scheme is presented in Figure 6. As it was described in the previous section, first the spectrograms were parameterized, then, the feature vectors were used as the input to the ML algorithms.

4.2. Presentation of the Results Obtained for Classical ML Models

From all calculated classifiers, only those that delivered the best outcome for the test data were presented. During the learning process, the data were subjected to 10-fold cross-validation and 20% of the feature vector cases (records) in the database were used to obtain the final verification. During the experiments, the performance of the successively analysed ML algorithms was investigated. The effect of varied transducer angular step size on the accuracy obtained for given the ML algorithm was assessed in these studies. Although the basic transducer rotation step used for the measurement was 1.8°, it is possible to reduce the angular measurement resolution and increase the rotation step by only considering a portion of the measurements (portion of basic steps) during the learning process. Thus, for this reason, the analysed rotation steps that the transducer took between successive measurements were equal to 1.8°, 3.6°, 9° and 21.6°. The angular step size influences both the number of unique instances in the training set and the ML algorithm’s resolution of differentiating unique cases. Using this method, the accuracy %Acc of proper prediction for different angular resolutions of measurements was calculated according to:

$$\%Acc=\frac{PR}{NoR}\times 100$$

(1)

where: PR is the number of properly predicted records and NoR is the number of all records in the database.

Table 2. The highest accuracy score obtained for all trained ML models for four angular resolutions.

Angular Resolution	1.8°	3.6°	9°	21.6°
Method	Ensemble Bagged Tree	Neural Network—Medium	Neural Network—Narrow	Neural network—Narrow
%Acc [%]	26.3	50.9	76.3	88.8

presents the best result obtained for all trained ML models for four angular resolutions: 1.8° (there were 51 angles, thus 51 classes as well considered between TD and RD), 3.6° (26 classes), 9° (11 classes), 21.6° (5 classes). Finally, to determine the total number of classes for all samples, the above values of the number of angle classes should be multiplied by ten, that is by the total number of the inspected steel grades.

In the Table 2, in the row named “Method” by successive columns referring to the results achieved for different angular resolutions, the algorithm for which the best result was obtained was indicated. In the case of angular resolution 1.8°, the greatest accuracy was reached for the ensemble bagged tree algorithm, while for other resolutions, it was artificial neural network (with different setup) that was the most accurate. Generally, in case of the highest number of classes (referring to the angular resolution of 1.8°—the highest angular resolution case) the Ensembles Bagged Trees models showed an advantage of 5–10% over the naïve Bayes and kNNs methods. The ANN and SVM structures did not reach enough flexibility and failed to obtain the solution in this case. Along with the reduction of the class numbers (by reducing the angular resolution to 3.6), ANNs (especially narrow and medium neural network structures, for which the accuracies differed from each other by approx. 1% maximum) obtained 6–8% advantage over the Ensembled Subspace Discriminant and Bagged Trees, leaving kNN models far behind (which reached close to 20% of difference in accuracy in reference to the ANNs’ result). Further reduction of the class number, referring to the reduction of the angular resolution to 9 degrees, sustained the percentage advantage of the ANN models (narrow, medium and bilayered, reaching similar accuracies within 1% of each other) over the ensembled ones (Subspace Discriminant and Bagged Trees). However, in this case, the SVM models (especially the linear and quadratic ones) were up to just 3 percent behind the ANN models. Finally, in the last case (of the smallest number of classes and the angular resolution of 21.6 degrees), the order of the best models did not change, but the difference of the accuracy between them decreased and did not exceed 3% this time. Moreover, as it can be seen, the best accuracy result was obtained for an angular resolution of 21.6° and the worst for an angular resolution of 1.8°, which is as expected. Utilizing greater angular steps increases the proportion of correctly identified steel grades and magnetic directions (the angles of the transducer alignment with respect to TD and RD). This is because the classification process has been obviously simplified and the number of potential classes has been decreased (there were 510 classes for 1.8° step size, but only 50 for 21.6°). However, it is crucial to achieve both the highest possible classification accuracy and the greatest number of detected directions, which, in this case, define the resolution of the classification operation. An accuracy of 50.9% was obtained for an angular resolution of 3.6°, which was stated in the previous study [26] as the most optimal for identifying grades of GO sheets and their angles (according to the average misalignment angle between the RD and easy magnetic axis of the GO steels).

4.3. Performance Comparison of the Obtained Classical ML Models and the DL Model along with a Discussion

Next, the performance of the obtained ML models was validated by comparison to that of the DCNN previously attained [26]. The detailed configuration of utilized DCNNs have been shown in

Table 3. Structure of the DCNN [26].

Lp.	Layers	Size	Stride	Number of Filters
1	Image Input	145 × 94 × 1	N/A	N/A
2	Convolution	60 × 60	1 × 1	32
3	Batch Normalization	N/A	N/A	N/A
4	ReLU	N/A	N/A	N/A
5	AveragePooling	8 × 8	5 × 5	N/A
6	Convolutuon	4 × 4	3 × 3	128
7	Batch Normalization	N/A	N/A	N/A
8	ReLU	N/A	N/A	N/A
9	AveragePooling	1 × 1	2 × 2	N/A
10	Fully Connected	N/A	N/A	N/A
11	Softmax	N/A	N/A	N/A
12	Classification Output	N/A	N/A	N/A

, while

Table 4. Training algorithm and its parameters [26].

Parameter	Value/Name
Training algorithm	RMSProp
Initial learning rate	0.005
Mini Batch Size	32
Number of epoch	75

presents the chosen training algorithm along with its parameters, which were selected through optimization methods. The details of the process of adjustment of the DCNN structure and training algorithm parameters were presented in [26].

Table 5. Accuracy for DCNN and accuracy difference between two approaches.

Angle Res.	%Acc DCNN [%] [26]	(%Acc DCNN—%Acc) [%]
1.8°	39.5	10.2
3.6°	62.3	11.4
9°	90.2	13.9
21.6°	98.9	10.1

presents the accuracies (%Acc DCNN) of the DCNN model obtained in [26] for the same datasets (of TF representations of MBN measured signals) as those used in this paper. The table also includes the differences in the accuracies obtained for the DCNN and the best ML models (%Acc DCNN—%Acc) presented in Table 2. As it can be seen, the greatest difference was obtained for an angle resolution of 9° and the smallest for 21.6°. Nevertheless, in each of the analyzed cases, there is a clear advantage (oscillating within 10%) of using a solution based on deep networks, i.e., based on a full representation of (raw) measurement data.

When analyzing the obtained results of approaches of both types in the context of their efficiency for the analysis of MBN signals, several aspects should be discussed. The deep learning approach requires a much bigger representation of data. Thus, it is time-consuming but allows one to capture all information, to automate the process of search and definition of features and to obtain subtle sensitivity to changes in MBN signals. On the other hand, the feature extraction process leads to a higher abstraction level of the analyzed TF data. Further, a representation with a higher level of abstraction leads to generalization of data but also to the loss of some information, which may be reflected in the loss of accuracy of the applied ML algorithms. In the case of a stochastic-type signal, such as the MBN one, this may be of much importance for the final capability of proper material assessment, especially if observed differences in signals between successive considered material states are not obvious. However, simultaneously, this approach reduces the complexity of the problem (due to generalization of data via representation of features) and shortens the analysis time as well. The performance of the ML models can be improved by combining the ML model with the feature selection procedure allowing for definition of the set of relevant features for the analysis. One should also consider optimizing the hyperparameters of the selected ML algorithms through the possible use of parameter grid search algorithms or with the help of swarm algorithms to obtain better results. Thanks to this, it is possible to reduce the distance between the two discussed approaches. Additionally, it is possible to further increase accuracy by implementing an update algorithm supporting the predictions of models (such as the one presented in the previous publication [26]).

On the other hand, the advantage of the deep machine learning approach in terms of accuracy is clearly visible, especially in the context of the growing number of analyzed base cases and labeled classes. Classic ML models reach a state of accuracy saturation faster (along with the decreasing number of considered classes). It can be seen on the basis of the growing difference between the results achieved in favor of the DL model for the angular resolution of 1.8, 3.6 and 9 degrees, respectively (see Table 5). The ML models slightly reduce this difference only after the DL model approaches the limit of 100 percent accuracy (for the resolution of 21.6° because of the closing to maximum limit).

5. Conclusions

This article presents the results of work on the application of machine learning to the analysis of the time-frequency representation of Barkhausen’s noise signals for the purpose of evaluating the angular properties of selected grain-oriented electrical steel sheets. The paper refers to two possible approaches based on a different form of representation of input signals, relating to the degree of their postprocessing. For the requirements of the first approach, in the previous work [26], the authors presented the results of a procedure for analyzing the basic representation of MBN signals (raw signals transformed into time-frequency domain) based on deep learning. This paper presents a second type: a classic approach based on the extraction of unambiguously definable features of signals, and the search for intricate rules and relationships between them. For this purpose, a group of classic machine learning algorithms with varying degrees of model complexity and computational requirements was considered. Ensembled classifiers (Bagged Trees along with the Subspace Discriminant) showed relatively good behavior in reference to the others in each analyzed case. This showed a potential in this classifier type. However, it must be underlined that the ANN and SVM models performed better in the case of a smaller number of labelled classes, but the difference was not significant. Finally, both approaches, referring to the classical ML models and the DL one, were compared. The obtained results make it possible to compare the accuracy of both approaches to MBN analysis by carrying out research on a common database. Such a comparison was not the subject of any previous work and thus is a value added to future work on this topic.

Presented in this article, the comparison of machine learning methods (based on the parameter vector) with deep learning (based on the full representation of measurement signals) shows the superiority of the latter approach in the context of the achieved accuracy. In the case of machine learning, the results in accuracy were 10.1% to 13.9% worse than those obtained from deep learning methods. This leads to the need to pay special attention to the deep learning methods that were presented in the previous paper [26]. Nevertheless, it seems that considering the high availability of computer units with relatively high computing capabilities, the possibility of building systems based on deep learning seems to have significant potential for use in measurement data analysis systems, especially of a stochastic nature, such as magnetic Barkhausen noise.