A Hybrid Cracked Tiers Detection System Based on Adaptive Correlation Features Selection and Deep Belief Neural Networks

Abstract

Tire defects are crucial for safe driving. Specialized experts or expensive tools such as stereo depth cameras and depth gages are usually used to investigate these defects. In image processing, feature extraction, reduction, and classification are presented as three challenging and symmetric ways to affect the performance of machine learning models. This paper proposes a hybrid system for cracked tire detection based on the adaptive selection of correlation features and deep belief neural networks. The proposed system has three steps: feature extraction, selection, and classification. First, the oriented gradient histogram extracts features from the tire images. Second, the proposed adaptive correlation feature selection selects important features with a threshold value adapted to the nature of the images. The last step of the system is to predict the image category based on the deep belief neural networks technique. The proposed model is tested and evaluated using real images of cracked and normal tires. The experimental results show that the proposed solution performs better than the current studies in effectively classifying tire defect images. The proposed hybrid cracked tire detection system based on adaptive correlation feature selection and Deep Belief Neural Networks’ performance provided better classification accuracy (88.90%) than that of Belief Neural Networks (81.6%) and Convolution Neural Networks (85.59%).

1. Introduction

Recently, the increase of electronic devices and evolving digital technologies have increased the complexity of data [1]. For that, the framework of artificial intelligence (AI) for handling data has been improved. Image classification is considered one of the biggest challenges for machine learning and is one of the complex approaches of AI. Several intelligent applications deal directly with images and the necessity for life management, such as healthy applications [2,3,4,5], security management systems [6,7,8], and safe driving [9,10,11]. Intelligent applications that deal with safety management systems are essential for researchers, companies, and humans. Detecting wear and cracks in tires is an important application for safe driving [12]. According to the National Highway Traffic Safety Administration (AHTSA), more than 9500 people died from destroyed car tires in 2021 [12]. Despite the importance of tires, few drivers keep an eye on them. Although vehicle owners know the dangers of underinflated tires, they continue to use them. With lesser-known problems such as tire aging, oxidation, and cracking, drivers are unaware that they need to replace them, so they avoid inspection altogether.

Deep learning is increasingly used for visual condition monitoring to gain insights from complex transformations and even to transition from prognostics to diagnosis [13,14,15,16]. These developments have been used with infrared imaging to detect wear and other problems in mechanical systems. The systems check whether a vehicle’s tires are broken or normal and have to analyze the snapshot of the tires. Therefore, high domination of features is attracted from images during the analysis process [17]. Generally, feature extraction algorithms describe an image with many features [3]. Not all of these features need to be insignificant to the subject of the image, so feature selection is necessary to optimize the components extracted from the image and improve machine learning performance. The direct application of machine learning on these features resulted in a poor performance [18,19]. Feature selection can be essential in enhancing the correct detection rate of machine learning algorithms.

This paper proposes a hybrid cracked tiers detection system based on adaptive correlation features selection and deep belief neural networks (H-DBN). The proposed framework consists of feature extraction using histogram of oriented gradient (HOG), selecting optimal features using hybrid gradient correlation-based feature selection (HCS), and DBN as a predictor. The problem with detecting cracks in tires is overlapping textures. The HOG extracts the features of tires without analyzing the overlap in the features of the images. The proposed feature selection approach selects the features that achieve the necessary discrimination or distinguish the different classes as well as possible. Figure 1 illustrates the proposed scenario for implementing intelligent eyes to detect a vehicle tire defect.

The dataset in [10] was used as a benchmark to test the validity of the H-DBN system. The metrics accuracy, precision, recall, and F1 score were used to evaluate the performance of H-DBN. We compare the proposed (H-DBN) system with three machine learning algorithms: naïve Bayes (BN), random forest (RF), and decision tree (DT). In addition, the performance of H-DBN compared with recent tire crack detection models in terms of accuracy.

Several studies have enforced different classification models for text messages to develop image classification. Most methods used to classify images suffer from low recognition rates when dealing with images that are very similar in texture. The overlap of texture features leads to a high error rate in classification, so machine learning and deep learning may not achieve a high accuracy rate.

Siegel et al. [10] used convolution neural network (CNN) to determine the cracks in vehicle tires. The authors explore the need for monitoring tire health and consider the evolution of a densely connected convolutional neural network. This model achieves 78.5% accuracy on cropped sample images, outperforming human performance of 55%.

In [20], a new model for classifying tire wear was proposed. The tread feature is highlighted by an attention mechanism proposed by the authors. The overlap of natural and cracked textures makes it difficult for deep learning algorithms to distinguish between them, so without early processing of the images, the algorithm certainly achieves low accuracy in recognizing the tire category.

Ali et al. [3] used personal component analysis (PCA) and histogram-based gradient (HOG) as descriptors of X-ray images for machine learning algorithms. The features selected in the proposed model are based on the wrapper model. The limitation of the proposed model is the inconsistency of the results because the wrapper model is based on randomness in selecting features. Too et al. [21] proposed two CNN models, one for feature extraction and the second for the image prediction category. The limitation of the proposed framework is the ability of convolution to extract visual patterns from different spatial positions.

Kumar [22] proposed a new framework for a feature selection model based on deep learning and the Pearson correlation coefficient. The features extracted from the traditional deep learning architectures ResNet152 and Google Net are optimized in the feature space of the proposed model using the concepts of Pearson correlation coefficient (PCC) and variance threshold. The restriction of the proposed model to static thresholds and static methods does not fit the dynamics associated with most of the problems.

2. Materials and Methods

This section discusses the Materials and Methods used in this paper.

2.1. Deep Belief Networks (DBN)

A deep belief network (DBN) is a type of neural network that includes three techniques: a deep neural network (DNN), cascading restricted Boltzmann machines (RBMs), and a back-propagation neural network (BPNN) [23]. RBMs are typically used for dimensionality reduction, extraction, and collaborative feature filtering before feeding into BPNN. In the deep belief network algorithm, the DNN is quickly trained by DBN due to using the greedy layer-wise unsupervised training method as a training technique [24]. Like many other neural networks, the DBN is based on the fundamental principle of initializing feed-forward neural networks (FFNNs) with unsupervised pre-training on unlabeled data before fine-tuning with labeled data [25]. The first RBM is trained to use contrastive divergence (CD). The weights of all RBMs are similarly trained layer by layer until the last RBM. The RBMs in the lower layers of the RBM stack learn lower-level features, and the RBMs in the higher layers learn higher-level features of the training data. Figure 2 illustrates the architecture of a DBN.

The main constrictor of RBM is visible (n nodes), Bernoulli random value hidden units (m nodes), and the weight matrix connects between them with the size ($m\times n$) [23]. Figure 3 illustrates the graphical representation of the RBM.

According to the contrastive divergence algorithm (CD) strategy, the RBM trains the sample value of (v) to compute the probability of the hidden unit (h). Equation (1) calculates the energy function E(v, h) of the joint configuration {v, h} [26]:

$$E\left(v,h\right)=-{\displaystyle \sum }_{i=1}^m{b}_i{v}_i-{\displaystyle \sum }_{j=1}^n{c}_j{h}_j-{\displaystyle \sum }_{j=1}^n{\displaystyle \sum }_{i=1}^m{v}_i{w}_{ij}{h}_j$$

(1)

where $b$ and $c$ are the biases value of visible and hidden units, and $w$ is the weight matrix that connects the visible units to the hidden ones. The joint probability of pair $\left(v,h\right)$ is computed by Equation (2):

$$p\left(v,h\right)=\frac{{\mathrm{e}}^{-E\left(v,h\right)}}{{{\displaystyle \sum }}_{v,h}{\mathrm{e}}^{-E\left(v,h\right)}}$$

(2)

Since the units within the visible and hidden layers are not connected, the activation probabilities of ith visible unit and jth hidden unit is given as:

$$p\left(v_i=1\mid h\right)=\sigma \left({\displaystyle \sum }_{j=1}^m{h}_j{w}_{ij}+a_i\right)$$

(3)

$$p\left(h_j=1\mid v\right)=\sigma \left({{\displaystyle \sum }}_{i=1}^n{v}_i{w}_{ij}+b_i\right)$$

(4)

where $\sigma (·)$ is the sigmoid logistics function.

The weight between the visible and hidden units is updated based on the positive gradient minus the negative gradient ($v^{\prime }{h}^{\prime \top }$) times some learning rate ($\alpha$). Equation (5) calculates the update in the weight.

$$\Delta W=\alpha v{h}^{\top }-v^{\prime }{h}^{\prime \top }$$

(5)

2.2. Histogram of Oriented Gradients (HOG)

Histogram of oriented gradients (HOG) is a feature extraction technique from 2D images. It is an efficient image descriptor of the texture image [27]. The HOG extracts feature based on the oriented gradient of colors in localized portions in each window (sub-region or small regions of the image). The number of features extracted from the image by HOG depends on the cell size, number of blocks, and number of orientations [28]. Based on gradient magnitude $m \left(x,y\right)$ and orientation ${\theta }_{x, y}$, the cell’s characteristics $\left(x,y\right)$ are calculated based on the spectral value of a pixel in position $\left(x,y\right)$. Equations (6) and (7) use the x- and y-directional gradients $dx \left(x,y\right)$ and $dy\left(x,y\right)$ to derive the magnitude and orientation of pixel $\left(x,y\right)$ [3].

$$m_{x,y}=\sqrt[2]{dx {\left(x,y\right)}^2-dy{\left(x,y\right)}^2}$$

(6)

$${\theta }_{x, y}=\{\begin{array}{c}ta{n}^{-1}\left(\frac{dy\left(x,y\right)}{dx\left(x,y\right)}\right)-\pi if dx\left(x,y\right) and dy\left(x,y\right)>0 \\ ta{n}^{-1}\left(\frac{dy\left(x,y\right)}{dx\left(x,y\right)}\right)+\pi if dx\left(x,y\right) and dy\left(x,y\right)<0\\ ta{n}^{-1}\left(\frac{dy\left(x,y\right)}{dx\left(x,y\right)}\right) otherwise \end{array}$$

(7)

2.3. Correlation-Based Feature Selection (CFS)

The feature selection technique is one of the essential processes for optimizing the performance of machine learning algorithms [29,30]. The filter model is distinguished from the other feature selection models by the stability of the selected features and the number of features [31].

The CFS technique is a filter method unrelated to the chosen classification model. As the name implies, correlations are the only intrinsic qualities used to evaluate feature subsets. It determines the strength of the correlation between variables. Equation (8) calculates the correlation $t_{\left(x,y\right)}$ of features $x,y$, The feature is selected if $t_{\left(x,y\right)}$ greater than the critical value at the 0.05 significance level [32]. Equation (9) calculates the correlation between features x and y.

$$r_{\left(x,y\right)}=\frac{{{\displaystyle \sum }}_i\left(x_i-{\overline{x}}_i\right)\left(y_i-{\overline{y}}_i\right)}{\sqrt{{{\displaystyle \sum }}_i{\left(x_i-{\overline{x}}_i\right)}^2{{\displaystyle \sum }}_j{\left(y_j-{\overline{y}}_j\right)}^2}}$$

(8)

$$t_{\left(x,y\right)}=r\sqrt{\frac{n-1}{1-r^2}}$$

(9)

where n is the dimension of the feature.

2.4. Proposed Hybrid Adaptive–CFS and DBN (H-DBN):

The proposed hybrid adaptive–correlation-based feature selection and deep belief neural networks (H-DBN) system trains the model via four steps before: preprocessing, extracting features, selecting optimal and high correlation features, and finally, predicting the data (image) category. Figure 4 illustrates the main steps of the proposed H-DBN model.

3. Preprocessing

At this stage, the dimensions of the images are the same so that the character extraction step can extract features of the same dimension from all images. The nearest neighbor interpolation technology is used to normalize the size of the image to ($700\times 800$) pixels. The method used to resize the image by replication is the nearest neighbor interpolation. Accordingly, resampling and interpolation are used to resize the images. Equation (10) generates the image $g\left(m,n\right)$ by factorizing ($c$) in the ($m$) direction and ($d$) in the ($n$) direction from the image $f\left(m,n\right).$

$$g\left(m, n\right)=\{\begin{array}{c}\frac{1}{cd},-\frac{c}{2} \le m<\frac{c}{2},-\frac{d}{2} \le n<\frac{d}{2}\\ 0 otherwise \end{array}$$

(10)

4. Extraction Features

In the proposed H-DBN, the HOG is used as a feature extractor. This step extracts many features, some of which may not be helpful. Therefore, selecting features with high significance that can be used as input data for the DBN algorithm is necessary. Figure 5 illustrates the overlap of features extracted from HOG.

5. Adaptive Correlation-Based and Variance Feature Selection (ACFS)

The selected features based on only the correlation coefficient do not give definitive evidence to impotent features [33,34,35]. The selected features found on only the correlation coefficient do not give definitive proof these features are essential. This paper proposes a new approach to feature selection: adaptive correlation-based and variance feature selection (ACFS). It selects features based on the correlation coefficient and the feature’s variance with each class in the dataset. The correlation coefficient shows the extent of the relationship of the trait with the rest of the traits. To support this relationship, the proposed model should show the significance of the change in the specific characteristic data in both classes. In the proposed ACFS, if the evidence for the adjective changes significantly in a particular category, then it is impossible to rely on the tagging of the adjective to distinguish the adjective category. Equation (12) calculates the correlation coefficient $c$ of feature $x$.

$$V={\displaystyle \sum }_{j=1}^k(\frac{{\displaystyle \sum }{\left(x_{i,j}-\overline{x_j}\right)}^2}{n-1})$$

(11)

$$C_x=\frac{{{\displaystyle \sum }}_{i=1}^{n}t\left(x,y_i\right)+V^{-1}}{N.K}$$

(12)

where N is the number of features, K: the number of classes, $x_{i,j}$ each point in ith location includes into class j$, j=1,2,3,\dots ,n$ and $i=1,2,3,\dots ,k$, and V is the variance between feature x and another feature of the same data class.

The selection of essential features dramatically improves the performance of the machine training algorithm; therefore, the set of a threshold defining important features must be carefully prepared. If the threshold value is in the interval [−∞, +∞], it is not easy to choose the optimal value of the threshold. This paper proposes adding ±β to the gradient search to find the optimal threshold value. The initial value (${\alpha }_o$) of the threshold is zero (zero is the minor possible threshold). After each iteration (t), the model either adds β to the current threshold (${\alpha }_{t+1}$) or subtracts β from the threshold (${\alpha }_{t+1 }$), depending on the accuracy of the selected features. The model continues by adding $\mathsf{\beta }$ until reflecting the accuracy of machine learning. Equation (13) illustrates the modification of the threshold during the search progress.

$${\alpha }_{t+1 }=\{\begin{array}{c}{\alpha }_t+\beta , A_{t+1}<A_t\\ {\alpha }_t-\beta , otherwise\end{array}$$

(13)

where A is the accuracy, $\mathsf{\beta }$ random value between [0, 1].

It was limiting the search value to the best threshold and adding weight gradually decreasing after each search step.

$$\omega =0.2-\left(\frac{t}{t_{max}}\right)\ast 0.1$$

(14)

Equation (15) is a new form of Equation (13):

$${\alpha }_{t+1 }=\{\begin{array}{c}{\alpha }_t+\beta \omega , A_{t+1}<A_t\\ {\alpha }_t-\beta \omega , otherwise\end{array}$$

(15)

Figure 6 shows the bin’s interval of the value α with the frequency of each value.

Figure 7 shows the flow of the proposed ACFS.

Algorithm 1 illustrates the flow of the proposed ACFS.

Algorithm 1: ACFS.

X← Data //vector of features

N← max number of gradient searches

αt←0

initialization;

while i ≤ N do

//select features according to Eq(14)

F ← select features(αt, X) // t current iteration

F1 ← select features((αt + β), X)

F2 ← select features((αt − β),X)

// Evaluated features F, F1, and F2 by DBN or any machine learning algorithms

Acc ← Evaluate features(F)

Acc1 ← Evaluate features (F1)

Acc2 ← Evaluate features (F2)

If Acc < Acc1 then

i ← 0

F ← F1

Else if Acc < Acc2 then

i ← 0

F ← F2

Else

i ← i+1

Return F // Optimal Features

6. Experiment Results

This section discusses the dataset and empirical results.

6.1. Dataset

A dataset with a total of 1028 images of the tire (cracked/normal) was used in this study [9]. The images were taken at various angles, from various distances, in different lighting situations, and with multiple quantities of dirt on them. Suppose the classifier is made available to the general public as a diagnostic service. This strategy ensures that the training set is typical of the kinds of photographs an end-user would take with their cell phone. Figure 8 shows an example from the tire dataset

6.2. Empirical Results

In this section, the validity of the proposed H-DBN model is tested in two experiments. The first experiment compares the proposed H-DBN with machine learning algorithms (naïve Bayes (NB), random forest (RF), and decision tree (DT)). The second experiment compares the proposed H-DBN with current models for crack detection in tires. This experiment checked the validity of the proposed feature selection on the performance of machine learning algorithms.

The algorithm NB did not perform well compared to the other algorithms because it uses a probability factor in estimating the data type. It also does not cope with the complexity of evidence from overlapping features. The accuracy of NB improved from 64% to 68% when ACFS used for feature selection before entering the data in NB. DT and RF depend on the entropy of the features used in the training algorithms, so the high accuracy depends on the correlations of the features. The results of RF and DT without feature selection (stand-alone) reach 78.6% and 63.9% of RF and DT, respectively. With feature selection from CFS, the archive of RF achieves 78.8% accuracy and DT 63.9%. The algorithms achieved high results in selecting features from the data using the proposed ACFS method compared to previous results. The proposed feature selection model improves the accuracy, precision, recognition, and f1 score of DT and RF. RF achieves 66.1%, 64.9%, 64.8%, and 64.8% in accuracy, precision, recall, and f1 score, respectively. A large number of features entered into the algorithm ANN caused the algorithm to fail to achieve positive results, as it reached a consistent diagnostic accuracy of up to (67.10%) without using feature selection. They improve the accuracy of ANN by 1% with the new feature selection. This strongly indicates that the feature extraction algorithms improved the discrimination between the different categories. The new method improved the rate of class prediction by DBN to 9.8%.

Table 1. Comparison between naïve Bayes (NB), random forest (RF), decision tree (DT), and deep belief neural network (DBN).

Predictor	Feature Selection Model	Accuracy	Precision	Recall	F1-Score
NB	Stand alone	0.649	0.824	0.504	0.402
	CFS	0.662	0.621	0.553	0.528
	ACFS	0.686	0.686	0.578	0.558
RF	Stand alone	0.786	0.704	0.734	0.715
	CFS	0.798	0.726	0.740	0.732
	ACFS	0.809	0.740	0.747	0.743
DT	Stand alone	0.615	0.602	0.610	0.560
	CFS	0.639	0.624	0.628	0.625
	ACFS	0.661	0.649	0.648	0.648
ANN	Stand-alone	0.671	0.726	0.541	0.483
	CFS	0.651	0.688	0.556	0.508
	ACFS	0.682	0.769	0.593	0.547
DBN	Stand alone	0.816	0.798	0.832	0.805
	CFS	0.859	0.828	0.899	0.842
	ACFS	0.890	0.872	0.883	0.877

shows the comparison of the results according to the accuracy, precision, recognition, and f1 score criteria between the algorithms (NB, RF, DT, and DBN) of Standalone, CFS, and ACFS.

Figure 9 illustrates the enhancement in the true negative rate (TNR) of machine learning when using the ACFS.

DBN achieves the best accuracy, so the analysis performance of DBN when using ACFS is necessary for constructing the new framework. The proposed framework significantly improved the performance of DBN in terms of the rate of true positives rate (TPR) and the rate of true negatives (TNR). Figure 10 shows the comparison between the stand-alone DBN and the proposed H-DBN.

The proposed model shows a significant effect by establishing a high correlation between precision and recall. See receiver operating characteristic (ROC) for a visualization of the evaluation and comparison of the performance of DBN and the proposed H-DBN. Figure 11 shows the performance evaluation of DBN and H-DBN concerning ROC.

Table 2. Compare the proposed hybrid adaptive–correlation-based feature selection—deep belief neural network (H-DBN).

Reference	Algorithm Name	Accuracy
[10]	A smartphone-operable densely connected convolutional neural network for tire condition assessment	78%
[20]	Efficient tire wear and defect detection algorithm based on deep learning	85%
[3]	Efficient intelligent system for diagnosis of pneumonia in X-ray images empowered with an initial clustering	82%
Proposed H-DBN		88%

compares the proposed H-DBN and three works in image classification over cracked tire prediction. These works were chosen as benchmarks because they are close in principle to the proposed method. The systems proposed in [10,20] for detecting cracks in tires can be compared with the proposed H-DBN model to verify the validity of the proposed system. In [3], feature selection is proposed after feature extraction by HOG, so that an indication can be given of the extent to which the selected features match the machine learning algorithm.

The proposed system outperformed standard machine learning and deep learning in terms of correct classification accuracy rate. In other words, the proposed feature selection repeatedly improved the accuracy of classifications made by machine learning algorithms and demonstrated its value as a possible way to achieve this goal.

7. Conclusions and Future Works

One of the biggest challenges for machine learning is image classification. One important application for safe driving is detecting cracks in tires. The correct detection rate of machine learning algorithms can be increased by better feature selection. The three components of the proposed framework are DBN as a predictor, hybrid gradient correlation-based feature selection to select the best features and histogram of oriented gradient (HOG) for feature extraction. It is not enough to use feature extraction directly for machine learning to extract many features common to image classes, such as the sky region and patches of color such as blue, black, etc. Therefore, feature selection and feature optimization are necessary to achieve good discrimination between image categories. The RBM in the DBN algorithm plays a vital role in efficiently optimizing features. For future work, we propose to use the RBM before the apartment layer of the deep learning architecture.