Multi-Feature Extraction and Selection Method to Diagnose Burn Depth from Burn Images

Abstract

Burn wound depth is a significant determinant of patient treatment. Typically, the evaluation of burn depth relies heavily on the clinical experience of doctors. Even experienced surgeons may not achieve high accuracy and speed in diagnosing burn depth. Thus, intelligent burn depth classification is useful and valuable. Here, an intelligent classification method for burn depth based on machine learning techniques is proposed. In particular, this method involves extracting color, texture, and depth features from images, and sequentially cascading these features. Then, an iterative selection method based on random forest feature importance measure is applied. The selected features are input into the random forest classifier to evaluate this proposed method using the standard burn dataset. This method classifies burn images, achieving an accuracy of 91.76% when classified into two categories and 80.74% when classified into three categories. The comprehensive experimental results indicate that this proposed method is capable of learning effective features from limited data samples and identifying burn depth effectively.

1. Introduction

Burns are a global public health issue, with an estimated 11 million people requiring medical care for burns each year, according to the World Health Organization (WHO). Additionally, 180,000 people die from burns annually, with the vast majority occurring in low- and middle-income countries [1].

Damage from burn affects all cell proteins, causing denaturation. Based on clinical observations of skin anatomy, burns are commonly classified into four categories based on depth: superficial, superficial dermal, deep dermal, and full-thickness burns [2,3].

• Superficial burn (first-degree burn [2])

Superficial burn only affects the skin epidermis, resulting in redness without blisters, dry surface, and pain upon touch, often accompany by edema.

2.

Superficial dermal burn

Superficial dermal burn penetrates into the papillary or superficial layer of the dermis, showing redness and moistness. Applying pressure to the red area may cause it to turn white and then back to red upon release, often accompanied by moderate edema and blister formation within minutes of injury.

3.

Deep dermal burn

Deep dermal burn extends down to the reticular layer of the dermis or deeper, with colors appearing red or waxy white. When pressure is applied, the red area continues to turn white, but upon pressure release, capillary refill may disappear or be slow. Blisters are usually not present, and the exposed surface of the wound is moist. Edema in the burn area is typically pronounced.

4.

Full-thickness burn

Full-thickness burn penetrates can affect the subcutaneous tissue layer. The skin affected by the burn is dry and tough, with colors possibly ranging from brown to red. A notable feature of a full-thickness burn is the absence of pain and pronounced edema in the burn area [4].

For superficial burn, the wounds can heal on their own without leaving scars. Thus, the analysis of it is not performed in this work. Superficial dermal burns can also heal on their own, but improper treatment may cause skin discoloration, which can gradually recover over time. Surgical treatment, grafting, or reconstructive therapy are necessary for deep dermal burn and full-thickness burn. Otherwise, the wound could worsen and become infected [5]. Deep dermal burn and full-thickness burn can be classified together because they both require surgical intervention and are unable to self-heal. Superficial burn and superficial dermal burn do not require surgery and can heal on their own [6].

Therefore, burns are categorized into superficial dermal burn, deep dermal burn, and full-thickness burn in this paper. Figure 1 illustrates these three types of burn injuries from left to right: superficial dermal burn, deep dermal burn, and full-thickness burn. In addition, deep dermal burns are combined with full-thickness burns into one category designated non-self-healing burns, while superficial skin burns are referred to as self-healing burns.

Currently, the initial clinical assessment of burn areas is often challenging because medical doctors must consider various factors during diagnosis, including the cause of the injury, the extent of the lesions, and tissue penetration [4]. The diagnostic accuracy of clinical assessment through visual and tactile examinations ranges between 50% and 80% [7]. In the worst-case scenario, some medical experts may have a difference of nearly 30% in diagnostic accuracy [7]. The high cost and workload of experienced doctors pose challenges during treatment, particularly in regions with lower economic levels, where ensuring the quality of burn patient care is difficult [8]. Considering these issues, a tool that automatically assesses the depth of burns is crucial for assisting clinicians in determining the severity of burns.

The schematic diagram of the method proposed in this work is shown in Figure 2. Firstly, the color histograms of the RGB, HSV, and L*A*B* color models are extracted from burn images as their color features. Texture features are then extracted by applying a Gabor filter to the images. Deep features are extracted in a modified ResNet-50 network named Burn-ResNet51. These three features are concatenated and arranged, and a feature selection method based on random forest (RF) is applied to reduce the dimensionality of the features in the training set, filtering out less important features. Subsequently, the corresponding features from the test set are selected. Finally, classification is performed using six classifiers, such as random forest. In summary, this study contributes in the following ways:

• This proposed method can extract multiple types of features from burn images, which can represent the characteristics of images from multiple perspectives. Thus, the differences between different types of burn images can be captured more accurately and comprehensively.
• Simple modifications are made to the ResNet-50 network, creating a new network named Burn-ResNet51, which reduces the dimensionality of features extracted in the max pooling layer to an optimal value, enabling better expression of deep features.
• A random-forest-based feature extraction method is used before classification, which mitigates the negative impact of redundant features.
• To ensure that each feature contributes to the model’s performance enhancement, multiple ablation experiments are conducted, thereby verifying the rationality of this proposed model and helping guide subsequent improvements to the model.
• This proposed method is validated on the Burns BIP_US dataset and achieves the best performance compared to other common burn depth classification methods [9].

The rest of this paper is organized as follows. Section 2 introduces the related literature reviews or surveys of the methods involved in this study. In Section 3, four main approaches used in burn depth diagnosis are presented. Then, some experiments are conducted in Section 4 to prove the effectiveness of this proposed method. Section 5 illustrates possible future research directions. Finally, Section 6 offers the conclusion.

2. Related Work

Imaging techniques were used to assess burn depth in the past. Laser Doppler imaging (LDI) is one of the more effective traditional assessment methods. But the limitation of this approach lies in its accuracy of less than 80% within 24–48 h after the patient’s injury. Moreover, LDI also suffers from high costs and difficult calibration [10].

With the advancement of technology, machine learning models are increasingly being used for the classification of burn depth. For example, Badea et al. [11] used color as the primary feature to distinguish between different burn depths. However, the performance of this method was only comparable to the average performance reported in the literature by specialized doctors. Similarly, Ref. [12] leveraged color features to employ multidimensional scaling (MDS) to determine certain physical features, which were then transformed into mathematical features. Through classifiers and feature selection methods, the accuracy in binary classification reached 79.73%.

Since a single-color feature provides limited information, Ref. [13] also considered texture features in addition to color features. Feature extraction was performed using discrete wavelet transform (DWT) followed by principal component analysis (PCA) to reduce feature dimensionality. Texture features were then extracted from the decomposed images using gray-level co-occurrence matrix (GLCM). Similarly, the researchers in Ref. [14] also studied color and texture features. Their method transfers the images from the RGB space to the Cr space (one of YCbCr transformation) to separate the skin region from others, while contrast and uniformity were selected as texture features. This method performed well when classifying sub-images of burn pictures into two categories using SVM. Yadav et al. [9] used a binary classification method to classify a burn image dataset. The experiment combined traditional feature extraction methods, including the HOG feature extraction method, with an SVM classifier, achieving a classification accuracy of 82.43%.

In addition to color and texture features, depth features can also assist clinical doctors in identifying the depth of burns. For example, Rostami et al. [15] applied a deep learning method using the pre-trained deep convolutional neural network AlexNet on the dataset of burn images for fine-tuning and classification. The accuracy was 77.8% when classified into three categories and 82.43% when classified into two categories. Furthermore, the authors of [16] used a dataset of multiple images of body parts; their proposed method used three pre-trained and fine-tuned CNN models, ResNet-50, VGG-16, and VGG-19, for burn image classification. Ultimately, ResNet-50 attained an accuracy of 83.52% in the classification of burn depth into two categories, and surpassed the performance of other models. When the features extracted from the fully connected layer of ResNet-50 were combined with an SVM classifier, the accuracy increased to 84.85%. Moreover, Ref. [17] also used a Simultaneously Evolved SVM (SE-SVM) to classify deep features extracted from four different deep convolutional networks and applied them to diagnose COVID-19 from X-ray images. These studies demonstrate that pre-trained deep convolutional neural networks can extract effective features and the features can be used in the traditional classifier, thereby improving classification accuracy to a certain extent.

ResNet-50 not only demonstrates excellent performance in diagnosing burn depth but also shows promising results in the field of brain tumor classification. Kumar et al. [18] proposed a method that utilizes transfer learning with residual networks and initializes the ResNet-50 model with image weights using the RNGAP (global average pooling) model. This approach enables the model to perform three-level brain tumor image classification tasks. A similar approach can be seen in this study, Abubakar et al. [19] extracted deep features in fully connected layers of ResNet50 and VGG16. Then, combining these features with SVM also leads to good classification of burn depth. These studies [15,16,18,19] have shown that utilizing the ResNet-50 model can alleviate the gradient vanishing problem. Ref. [18] demonstrated that the global average pooling layer can effectively reduce overfitting, thereby improving classification performance. However, the network models used in these studies were either classified after the fully connected softmax layer [15,18], or only features extracted from the final fully connected layer were used for classification [16,19,20,21]. This suggests that features can also be extracted from the convolutional layers, pooling layers, or fully connected layers of the network and classified using other classifiers to achieve transfer learning. Although the classification part can utilize artificial neural networks composed of fully connected layers, softmax layers, and output layers, its efficiency still cannot surpass certain specific classifiers in some cases. Accordingly, studies [22,23,24] indicated that features extracted from layers other than the fully connected layer exhibit good generalization capabilities. Ref. [25] also mentioned that features extracted from the fully connected layer have inferior portability. Therefore, features are considered to be extracted from the global average pooling layer of the modified ResNet-50 network to obtain depth features, which may enhance the generalization capabilities of the features and partially alleviate overfitting.

Compared to the methods mentioned above, the authors in [26,27,28] attempted to enhance intelligent diagnostic performance by utilizing multiple features. The authors of [28] selected color features, texture features, and deep features extracted by stacked sparse autoencoders, combined the three extracted features, and input them into multiple classifiers. Ultimately, the accuracy of dividing burn injury images into two categories and three categories was 85.86% and 76.87%. Although, this method is effective, there may be redundant information among the extracted features. However, in Ref. [26], the authors reduced the dimension of the features by PCA after extracting the deep features. Then, they used the SVM classifier for face classification; the classification accuracy on face dataset reached 97.19%. Similarly, in Ref. [27], the authors used the univariate measurement and Poisson distribution feature selection approach after extracting and fusing deep features. Finally, the selected features were input into the Multi-class Support Vector Machine (MC-SVM) for classification of skin diseases. The experimental results show that this method achieved higher performance visually and in terms of enhanced quantitative evaluation with enhanced accuracy.

Therefore, directly combining multiple features is not the desired choice. The studies mentioned above proved that proper feature selection methods can improve classification performance. Additionally, there may still be room for improvement in the extracted deep features. Taking these points into consideration, the aim is to design a novel method for extracting depth features. A range of diverse features are then integrated and feature selection techniques are employed to identify useful ones, thereby enhancing the performance of the method. In

Table 1. Summary table of research related to burn depth diagnosis.

Studies	Advantages	Disadvantages	Published Year
Hoeksema et al. [10]	The accuracy of LDI is higher in burn depth assessment after the fifth day of burn injury.	The assessment accuracy in the early stage of burn is lower. High cost. Difficult to calibrate.	2009
Serrano et al. [12]	The accuracy of distinguishing burns that can heal themselves or not is higher.	Vulnerable to factors affecting the color of images.	2016
Kuan et al. [13]	Focus on both color and texture Low cost and velocity.	Low accuracy.	2017
Wantanajittikul et al. [14]	Focus on both color and texture.	The accuracy is low when burns are classified into three categories.	2012
Yadav et al. [9]	The accuracy of distinguishing burns that can heal themselves or not is higher. Accurately classify burns with blisters.	The accuracy is low when burns are classified into three categories. Focus only on color.	2019
Rostami et al. [15]	High accuracy.	Require more computational resources. Focus only on deep features.	2021
Zhang et al. [28]	Focus on multiple features of burn images. High accuracy.	Require more computational resources.	2021
Abubakar et al. [19]	High accuracy.	Focus only on deep features. Require more computational resources. Program runs slower.	2020

, a summary table of the work related to burn depth diagnosis covered in this section is presented to facilitate comparison with this proposed approach. It should be noted that the advantages and disadvantages listed in the table are compared to those of other research in Table 1.

3. Method

In this section, the methods for extracting color features, texture features, and depth features from burn images specifically are discussed, as well as the method of feature selection after feature extraction to reduce dimensionality for burn depth classification.

3.1. Color Feature

Typically, medical experts often analyze the depth of burns based on color characteristics such as pink-white, beige, or dark brown in the burn area, or its texture features [29]. Different depths of burns exhibit different color appearances, with superficial dermal burn appearing as red and possibly moist, deep burns showing a mixture of red or waxy white, and full-thickness burns presenting skin colors of brown or red [3]. Therefore, theoretically, color can serve as an important basis for classifying burn depth in burn images.

The color histogram method is utilized to extract color features from images. Color histogram is widely used in various image retrieval tasks [30]. Essentially, this method calculates the number of occurrences of color intensities in the entire image or specific regions of interest. A histogram is generated for each color channel of the color model. Then, the histogram is quantized, as each entry in the histogram can be represented by a range of intensity values, and this process is known as binning. Further details on the color histogram can be explored in Refs. [31,32].

RGB (red, green, blue), HSV (hue, saturation, value), and L*A*B* (lightness, a* chromaticity layer, b* chromaticity layer) are three commonly used color spaces [33]. Generally, the RGB color space is easy to understand and use but may not be suitable for describing color attributes such as brightness and saturation. The HSV color space is used for tasks such as color recognition and segmentation in image processing [33]. It provides a more intuitive and natural way to describe color attributes, which makes it suitable for color-related operations in image processing. The L*A*B* color space is closer to the human visual system, offering more precise color description and control and performing well in image segmentation tasks [34]. Thus, different color spaces can express varying image information.

Here, images are considered to be converted into these three color spaces. Next, color histograms are used to count the occurrences of each color channel in each space and then merge them into color features. Figure 3a shows a superficial dermal burn image, while Figure 3b represents the RGB color space histogram as shown below. The histogram in Figure 3b is generated using the Color Thresholder tool with the bin of the histograms for each channel set to 256 [35].

In the RGB color space of a color image, the color histogram is represented by the k-level intensity values for the R, G, and B channels [28]. Specifically, the color histogram function for the R channel is defined as follows:

$$H_R=\left[H_{R1}, H_{R2}, ... H_{Rk}\right]$$

(1)

$H_R$ represents all pixel values of the R channel’s k-level intensity value [31]. Similarly, in the HSV color space, the color histogram is represented by the k-level values for the H, S, and V channels. In the L*A*B* color space, the color histogram is represented by k-level values for the L*, a*, and b* channels. These color histograms can be concatenated to form a vector-representing image. Ultimately, the color features of each image can be defined as follows:

$$\begin{gathered} {Color}_i=[ H_{R1}, ... H_{Rk}, H_{G1}, ... H_{Gk}, H_{B1}, ... H_{Bk}, \\ \hspace{1em}\hspace{1em}\hspace{1em}{H}_{H1}, ... H_{Hk}, H_{S1}, ... H_{Sk}, H_{V1}, ... H_{Vk}, \\ \hspace{1em}\hspace{1em}\hspace{1em}{H}_{L1}, ... H_{Lk}, H_{a1}, ... H_{ak}, H_{b1}, ... H_{bk} ] \end{gathered}$$

(2)

${Color}_i$ represents the color feature of the ith sample. $H_{Rk}$, $H_{Gk}$, $H_{Bk}$ are the k-level vectors of the R, G and B channels, respectively, $H_{Hk}$, $H_{Sk}$, $H_{Vk}$, $H_{Lk}$, $H_{ak}$ and $H_{bk}$ are produced based on a similar approach.

3.2. Texture Feature

Skin with varying depths of burns exhibits different texture characteristics. In addition to assessing the color of the burned area, medical experts also consider texture features when making diagnoses, such as determining whether the affected area is moist [12].

The Gabor filter has garnered significant attention and has become a prominent method for extracting texture features due to its ability to optimally localize in the spatial domain and the frequency domain. Gabor filter feature extractor comprises a filter bank with filters of varying frequencies and orientations. These feature extractors can transform images from their original space into a feature space. Therefore, the texture features of each image are represented by unique features in the feature space [36].

A key advantage of the Gabor filter is its capacity to capture image details at various frequencies (scales) and orientations [37], offering the best spatial and frequency resolution combination [38]. Currently, the Gabor filter is extensively utilized in diverse fields, including diabetes detection [39], Alzheimer’s disease monitoring [37], and face recognition [40]. Therefore, the Gabor filter is chosen for extracting texture from burn images. For each pixel I(i,j) in each image i, the kernel of the Gabor filter can be defined using the following formula:

$$Gabor(i,j)=exp\left\{-{\displaystyle \frac{{(icos\theta +jsin\theta )}^2+{\eta }^2{(-isin\theta +jcos\theta )}^2}{{2\sigma }^2}}\right\}·cos(2\pi {\displaystyle \frac{icos\theta +jsin\theta }{\lambda }}+\xi )$$

(3)

The wavelength λ and direction θ serve as the primary parameters of Gabor filter. λ denotes the filter’s periodicity in the spatial domain, while θ dictates the filter’s orientation for detecting edges and textures in the image. ξ signifies the offset, σ denotes the standard deviation, and η represents the ratio of dimension to diameter. Drawing from insights glean from prior studies [28], eight directions and five wavelengths are employed in the used Gabor filter. The eight directions include the following:

$$\theta =\{ 0°, 22.5°, ..., 157.5° \}$$

(4)

These five wavelengths can be expressed as follows:

$$\lambda =\{ 2^{{\displaystyle \frac{k}{2}}} |k=1, 2, \dots , 5 \}$$

(5)

A kernel size of 39 × 39 is utilized in this work. Subsequently, the Gabor kernel filters each burn image to obtain texture features.

3.3. Depth Feature

Inspired by the success of deep learning applications, deep learning is used to extract features and classify burn depth. Unlike traditional manual methods, deep learning can automatically extract useful information, rendering it more adaptable to diverse scenarios. Typically, the shallower layers of a deep network capture basic features like texture and edge information, while deeper layers construct intricate feature hierarchies that exceed human visual perception [41].

Deep Convolutional Neural Networks (CNNs) possess the ability to autonomously learn hierarchical features from low to high levels. Theoretically, networks with more layers can tackle increasingly complex tasks, albeit accompanied by challenges, such as a decrease in saturation or accuracy, or vanishing and exploding gradients. ResNet-50 alleviates both problems by using deep residual pretrained architecture [42]. The ResNet-50 architecture is trained on the ImageNet dataset, which comprises approximately 1.5 million natural scene images. It reduces the computing cost by using a smaller dataset for training and achieves a better accuracy [43].

The ResNet-50 network can effectively extract a large number of complex features from images, and its advantages have been verified in numerous studies [15,16,42]. Furthermore, some studies have demonstrated that features extracted from layers other than the fully connected layer possess good generalization ability [25,44]. Therefore, inspired by these results, an attempt is made to extract features from the global average pooling layer.

Here, burn images are subjected to simple transformations (simple transformation includes random rotation of the images by 0°–180°, random scaling by 0 to 10 times, and the size of the images being adjusted to 224 × 224 × 3). Then, these images are validated ten times by five basic CNN models.

Burn images are categorized into three classes: superficial dermal burn, deep dermal burn, and full-thickness burn. Additionally, these burn images can also be presented as two classes, i.e., self-healing burn (superficial dermal burn) and non-self-healing burn (deep dermal burn and full-thickness burn). The mean classification accuracies are then presented for fairly comparing.

Table 2. The results of using different CNN models for classifying two or three burn classes.

CNN Model	Accuracy (Two Categories) (%)	Accuracy (Three Categories) (%)
ResNet-18	82.84	68.47
ResNet-50	85.60	73.38
ResNet-101	82.77	70.95
VGG-16	80.41	71.37
DenseNet-19	86.49	71.62

shows the accuracies of using different CNN models for classifying two or three burn classes.

Considering both the classification performance and the model type, as seen in Table 2, DenseNet-19 achieves the highest accuracy for burns with 2 classes, with ResNet-50 closely following by a margin of about 1%. However, ResNet-50 outperforms the others, showcasing a superiority of around 1.7% over DenseNet-19 when the burn depth consisting of 3 classes. Interestingly, despite having the most extensive network layers, ResNet-101 did not exhibit the best performance in this scenario. These basic CNN networks contain several layers for classification with the same function (including the full connection layer, SoftMax layer, and classification layer). Therefore, the reason for the varying accuracy rates may be the different features of the burn images that can be extracted. To improve the network, two perspectives are considered. One is to change the internal structure of the network in order to extract more comprehensive feature information from images or using alternative classification methods. In the following part, this proposed method for extracting deep features is introduced from the perspective of changing the internal network structure. As for the other perspective, detailed explanations are provided in the experimental section.

To ensure a more comprehensive extraction of depth features, a novel CNN model, named Burn-ResNet51, is introduced to classify burn images based on burn depth. Burn-ResNet51 employs transfer learning and adjustment of parameters of the original network to extract profound features essential for accurate classification. In particular, ResNet-50, as the backbone of the proposed network, comprises 49 convolutional layers. It uses the residual structure to effectively mitigates issues such as gradient vanishing and explosion [45]. However, extracting features directly from the final fully connected layer may lead to excessively large feature dimensions, resulting in reduced classification performance and an increased risk of overfitting.

To enhance performance, Burn-ResNet51 has modified the ResNet-50 architecture. When dividing burn depth into two or three categories, the structure of Burn-ResNet51 remains unchanged, while the parameters of some layers are adjusted. The following section elaborates on how the structure of Burn-ResNet51 is derived.

Initially, the last three layers of the pre-trained ResNet-50 architecture (the fully connected layer, softmax layer, and classification layer) are adapted to suit this task. Specifically, the original fully connected layer in the pre-trained network is replaced with a new fully connected layer that categorizes burn images into three or two classes. Subsequently, three additional layers (‘Conv’, ‘Batch_Normaliz’, and ‘Activation_Relu’) are inserted between the ‘activation_49_relu’ and ‘avg_pool’ layers of the ResNet-50 architecture, as shown in Figure 4b. The parameters of ‘Conv’ and ‘Batch_Normaliz’ are fine-tuned to automatically extract image features. After these adjustments, the ‘avg_pool’ layer is followed by the last three newly added layers (‘full_connected’, ‘softmax’, and ‘ClassificationLayer’), as previously mentioned [44]. Figure 4 provides a visual representation of both the basic network structure and the modified network; the red box and red arrow in Figure 4a need to be modified, and the blue boxes in Figure 4b were modified.

Figure 4a illustrates the original ResNet-50 architecture, while Figure 4b showcases the modified Burn-ResNet51 architecture after incorporating new layers. In this section, regarding the three newly introduced layers, the number of filters in the convolutional layer and the ε in the normalization layer are primarily fine-tuned. The rationale behind adjusting these parameters is elaborated on below.

Studies [46,47] have demonstrated that the kernel size and the number of filters in the convolutional layer significantly impact classification accuracy. Therefore, the kernel size of the new added convolutional layers in Burn-ResNet51 is set to 3 × 3, which is the same as the kernel size of the new convolutional layer in Ref. [44] with a similar network structure to the one in this work. Meanwhile, for the number of filters, [48] is taken into account, which suggests that the feature maps extracted from the final convolutional layer of a CNN can be transformed into feature vectors. This statement can be interpreted as follows: each convolutional filter contributes to a specific component of the feature vectors, with the dimensionality of the extracted feature vectors being directly linked to the number of convolutional filters utilized, and the number of feature maps dictating the total count of extracted feature vectors.

Especially in this work, where the dataset is relatively small and the feature dimensions of depth features are not very large, the dimensionality of features and the corresponding number of filters appear particularly important. As a result, the influence of four different filter quantities (200, 300, 400) on classification accuracy is explored.

Moreover, according to the definition of batch normalization in Ref. [49], a mini-batch Ɓ of size m is taken into account, which signifies the presence of m activation values within the mini-batch. Ɓ is represented as follows:

$$\mathit{Ɓ}=\left\{{ x}_1, x_2, ... x_m \right\}$$

(6)

If i is any value between 1 and m, the $\widehat{x}$_i is obtained by the normalization of x_i, which can be expressed as follows:

$${\widehat{x}}_i={\displaystyle \frac{x_i-{\mu }_{Ɓ}}{\sqrt{{\sigma }_{Ɓ}^2+\epsilon }}}$$

(7)

where ${\mu }_{Ɓ}$ and ${\sigma }_{Ɓ}$ represent the mean and variance of ${ x}_1$ to $x_m$ in each mini-batch, respectively. ε is a constant added to the mini-batch variance for numerical stability.

To make sure that the transformation inserted in the network can represent the identity transform, $y_i$ is used to represent the linear change in ${\widehat{x}}_i$. The Batch Normalizing Transform is expressed as in the following formula:

$$y_i=B{N}_{\gamma ,\beta }\left(x_i\right)=\gamma {\widehat{x}}_i+\beta$$

(8)

To sum up, within this Batch Normalizing Transform algorithm, ε serves as a constant added to the small batch variance to uphold numerical stability [49]. Consequently, the focus is on exploring the impact of various ε values (1 × 10⁻³, 1 × 10⁻⁴, 1 × 10⁻⁵) on classification accuracy during parameter selection.

With the above parameters set, the burn images are simply transformed (simple transformation includes random rotation of the images by 0°–180°, random scaling by 0 to 10 times, and the size of the images being adjusted to 224 × 224 × 3); then, the new images are inputted into the burn-Resnet51 network. Subsequently, features are extracted from the ‘avg_pool’ layer as the depth features of the burn image.

3.4. Feature Selection

The initial feature extraction process generates a large number of features. To enhance model performance through the selection of more effective features, a random-forest-based approach is employed to assess feature importance and facilitate selection. This method is commonly utilized in variable selection scenarios involving dense features [50,51,52,53]. Random forest stands out as an efficient technique requiring minimal parameter configuration. It excels in handling multi-class classification and multivariate regression tasks and has found successful applications across various computer vision domains [52], including face recognition [53], action recognition [50], and facial expression recognition [51].

Random forest comprises an ensemble of decision trees built on multiple bootstrap samples. Bootstrapping is a random sampling method with replacement from the training dataset. Additionally, a subset of features is randomly chosen from the entire sample to construct each node of the tree [54].

A key aspect of the random forest feature selection method is the utilization of Out-Of-Bag (OOB) error estimates. The Out-Of-Bag set represents a sample not utilized during tree training. Initially, random forest estimates the OOB error ($err\left(X^j\right)$) for each feature, followed by randomly substituting the feature values with those from the OOB sample set. Subsequently, the OOB error is recalculated for the altered feature value ($err\left(X_{oob}^j\right)$) [52]. The permutation importance measure, a widely adopted feature importance metric, is leveraged. Feature importance ($VI$) is determined by the increase in average error when a feature ($X^j$) randomly alters one of its values from the OOB set ($X_{oob}^j$). The number of trees in the forest is expressed as nb_tree, and feature importance ($VI$) is as follows [54]:

$$VI(X^j)={\displaystyle \frac{1}{nb\_tree}}\sum _j\left(\right|err\left(X^j\right)-err\left(X_{oob}^j\right)\left|\right)$$

(9)

The random forest method is employed for feature selection, and the specific operational process is illustrated in Figure 5. Initially, a random forest model with m decision trees is constructed with a specified training set. Subsequently, the feature importance measure ($VI$) for the selected feature set is computed in each iteration using the random forest model prepared based on the Out-Of-Bag (OOB) set selected during training. The features are then arranged in descending order of $VI$, and the top n features are selected. Then, the corresponding n features are selected out from the test set. Ultimately, various classifiers, e.g., decision tree, KNN, etc., are utilized for classification, and the classification outcome is determined. Further details regarding the specific parameter configurations involved in the feature selection process will be discussed in subsequent experiments.

From this section, a 900-dimensional color feature vector is extracted for the three color spaces using color histograms. Subsequently, a 40-dimensional texture feature vector is extracted using Gabor filters, along with a 300-dimensional depth feature vector obtained from the Burn-ResNet51 model. Therefore, a total of 1240-dimensional feature vector is entered into the feature selector. The next section presents the fine-tuning and selection of some of the remaining parameters.

4. Experiment

This section covers the experimental aspects. First, the source and content of the dataset used in this experiment are introduced. Next, the process of selecting the optimal number of filters in the convolutional layer and the ε value in the normalization layer during deep feature extraction are discussed. The method of selecting the optimal number of selected features and the number of trees in nb_tree during feature selection are elaborated based on experiments. Finally, different combinations of three features are classified by random forest classifier. At the same time, some other common classifiers are also used for comparison and evaluation.

To evaluate the performance of the proposed method, accuracy, precision, recall, and F1 score are used as metrics to evaluate the classification performance, and are defined as follows. TP, FP, TN and FN represent true positive, false positive, true negative and false negative errors, respectively [28].

$$Accuracy={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}$$

(10)

$$Precision={\displaystyle \frac{TP}{TP+FP}}$$

(11)

$$Recall={\displaystyle \frac{TP}{TP+FN}}$$

(12)

$$F1-score={\displaystyle \frac{2TP}{2TP+FP+FN}}$$

(13)

During experimentation with diverse classifiers, fine-tuning is conducted to achieve optimal outcomes. Furthermore, each group of experiments is randomly repeated 20 times with the various aforementioned settings; then, their mean classification accuracies are reported for fair comparison. The experiments are performed on a PC with Intel(R) i7-4770 (2.90 GHz) CPU, NVIDIA GeForce RTX 2060 and 16GB RAM. The depth feature extraction, feature selection, and classification parts of the program are available at https://github.com/windgoddy/zxz/tree/main (accessed on 22 August 2024).

4.1. Dataset Setting

In this experiment, the Burns BIP_US dataset from [9] is chosen, which comprises 94 standard burn images. Following the dataset configuration in previous studies [28], twenty burn images are used for training, including 9 superficial dermal burns, 5 deep dermal burns, and 6 full-thickness burns. The remaining 74 images are designated the test set, consisting of 33 superficial dermal burns, 20 deep dermal burns, and 21 full-thickness burns.

Table 3. A statistical table of the sample quantities for different types of burns in the training and test sets.

Burn Depth Type		Training Set		Testing Set
Three Categories	Two Categories	Three Categories	Two Categories	Three Categories	Two Categories
Superficial dermal burn	Capable of self-healing	9	9	33	33
Deep dermal burn	Incapable of self-healing	5	11	20	41
Full-thickness burn		6		21
Total		20		74

provides a visual representation of the dataset’s configuration.

4.2. Depth Feature Parameter Setting

In order to select the optimal number of filters (NumFilters) and the ε value, a total of 12 combinations of NumFilters and ε are considered. Then, different combinations of these parameters are used to implement the proposed method.

In this section, the following strategy is used to find the optimal depth feature parameters for this proposed method. First, each parameter combination is used to extract deep features, which are then combined with pre-prepared color and texture features. Subsequently, feature selection is performed on the training dataset using parameters that will be discussed in the next section as optimal, and the corresponding features for the test dataset are selected. Finally, classification is carried out using a random forest classifier, with the classification results evaluated based on accuracy. Note that, in addition to the two parameters, other parts of the proposed Burn-ResNet51 model for extracting depth features have been adjusted and parameterized according to the image category.

Table 4. Classification accuracy with different combinations of NumFilters and ɛ.

ɛ	NumFilters	Two Categories	Three Categories
		Accuracy (%)	Accuracy (%)
0.001	200	87.77	78.42
0.0001		89.67	76.49
0.00001		87.77	74.26
0.001	300	88.38	80.74
0.0001		91.76	75.68
0.00001		90.68	80.55
0.001	400	90.82	78.85
0.0001		89.32	77.10
0.00001		85.14	65.43

shows the 12 combinations of NumFilters and ε, along with the corresponding classification results.

From this table, it can be observed that when ε is 0.0001 and NumFilters is 300, the highest accuracy of 91.76% is achieved when classifying burn images into two categories. Similarly, when ε is 0.001 and NumFilters is also 300, the highest accuracy of 80.74% is obtained when classifying burn images into three categories. This indicates that combining 300 extracted features with different values of ε can lead to optimal classification results.

Furthermore, it is evident from these combinations that these changes in the number of filters or ε value have a varying degree of impact on the accuracy of the classification. It follows that selecting appropriate values for NumFilters and ε is crucial for the extraction of deep features. Therefore, 0.0001 is used for ε and 300 for NumFilters with two classes of burn depth. Thus, 0.001 is used for ε and 300 for NumFilters with three classes of burn depth.

4.3. Feature Selection Parameter Settings

The random forest feature selection method is sensitive to the number of selected features (NumFeatures) and the number of trees (NumTrees) in the forest model [54]. These two parameters are used to adjust the performance of the model and improve the learning ability. Therefore, the following strategy is employed to determine the optimal parameters in this section. First, the depth features are extracted using the parameter combinations selected in the previous section, and then they are combined with color and texture features.

Subsequently, the NumTrees in the forest is fixed at 50 when the burn image is divided into two classes and at 600 when divided into three classes. At the same time, NumFeatures for candidate selection is defined as (200, 300… 1000, 1248 (no feature selection)). Different combinations of NumTrees and NumFeatures selected from the candidate range are used to implement this proposed method. Finally, the random forest classifier is used for classification, and the results for different NumFeatures are shown in

Table 5. Classification accuracy under different number of features.

NumFeatures	Two Categories	Three Categories
	Accuracy (%)	Accuracy (%)
200	91.21	78.24
300	90.27	79.73
400	90.27	80.14
500	90.88	80.74
600	91.15	80.54
700	91.76	77.03
800	91.01	76.49
900	90.74	76.95
1000	90.81	77.03
1248 (no feature selection)	90.88	78.39

.

Similarly, according to the NumFeatures determined by the above experiment, when the burn image is divided into two classes, NumFeatures is fixed to 700, and when the burn image is divided into three classes, NumFeatures is fixed to 500. The number of candidate trees is also defined as (10, 50, 100, 200, 300… 700). The proposed method is performed by using different combinations of NumFeatures and the parameter NumTrees selected from the candidate range. Finally, the random forest classifier is used for classification, and the classification results for different NumTrees are shown in

Table 6. Classification accuracy under different number of trees.

NumTrees	Two Categories	Three Categories
	Accuracy (%)	Accuracy (%)
10	90.35	64.87
50	91.76	74.32
100	91.15	74.32
200	90.58	78.38
300	90.55	79.73
400	90.88	79.73
500	91.28	79.73
600	91.35	80.74
700	90.88	79.19

.

From the two tables above, it can be observed that for binary image classification, the random forest feature selection method achieves optimal performance when selecting 700 features and using 50 trees. The optimal performance is achieved with 500 features and 600 trees when classifying burn images into three categories. Therefore, 700 is used for NumFeatures and 50 for NumTrees with two classes of burn depth. Furthermore, 500 is used for NumFeatures and 600 for NumTrees with three classes of burn depth.

These results show that the selected NumFeatures and the NumTrees in the forest have a certain impact on classification performance. Especially in the case of multiple classification, where parameter variations affect performance, whereas the impact is less significant for binary classification. It is observed that, in the case of binary classification, changes in the NumFeatures and NumTrees parameters do not have a significant impact on the experimental results. However, these parameters are still used. This is not only due to its improvement in accuracy by about 1%, but also its reduction in feature dimension and computational cost.

It should be noted that the two features selected in this section will only be used in the experiment where all three features are employed (the experiment in Section 4.4 using a random forest as a classifier). For other classification experiments, the parameters described in this section are not used. Instead, a similar approach is used to determine the specific parameters applicable to different scenarios.

4.4. Experiments after Combining the Three Features

After determining the necessary parameters, in order to more accurately evaluate the performance of the proposed method and further explore the relationship between these features and classification performance, several experiments are designed to investigate these issues. Following the classification process, Accuracy, Precision, Recall, and F1-score are applied as evaluation metrics for classification performance. The subsequent part provides detailed descriptions of these experiments and analyzes the experimental results.

4.4.1. Experimental Result

First, three different sets of features are extracted and concatenated. Then, feature selection is performed on the training dataset, and the corresponding features for the test dataset are selected. A random forest classifier was used to classify the depth of burn images. In addition, the classification results are compared with those of other common classifiers: support vector machine optimized with particle swarm optimization (PSO-SVM) [55], decision tree, k-nearest neighbors (KNN) [2], long short-term memory (LSTM), and 1-dimension convolutional neural network (1D-CNN).

In order to strike a balance between model complexity, generalization capability, and computational efficiency, the key parameter values of each classifier were tuned within the specified range according to experience in such a way that the highest average of recognition accuracy was obtained. For the random forest classifier, the number of trees is chosen as the main parameter to be adjusted. The process begins with 50 trees, gradually increasing the number. The number of trees is then adjusted based on classification performance for different parameters until a satisfactory result is achieved. Ultimately, the number of trees in the random forest classifier is fixed at 300. Similar adjustments are made for other classifiers to attain better outcomes.

The final classification results for dividing burn depth into two categories using three features are presented in

Table 7. The classification results of different classifiers when three features are selected and for categorizing burn images into two burn classes.

Feature Combination	Accuracy (%)	Precision/Recall/F1-Score (%)	Category I/II (%)
Random Forest	91.76	91.62/91.98/91.70	94.09/89.88
PSO-SVM	90.95	90.78/91.03/90.87	91.81/90.24
Decision Tree	83.45	82.73/81.32/80.97	78.89/83.76
KNN	85.88	86.76/86.56/85.73	90.26/82.87
LSTM	87.30	88.18/87.90/87.21	93.48/82.32
1D-CNN	90.54	90.74/91.17/90.53	96.97/85.37

, while the results for dividing burn depth into three categories are shown in Table 7. To visually compare the relationship between the results of each classifier, the contents of Table 7 and

Table 8. The classification results of different classifiers when three features are selected and for categorizing burn images into three burn classes.

Feature Combination	Accuracy (%)	Precision/Recall/F1-Score (%)	Category I/II/III (%)
Random Forest	80.74	77.77/77.99/77.64	96.21/67.04/70.71
PSO-SVM	76.62	76.70/75.86/73.37	98.70/45.74/84.14
Decision Tree	67.77	63.21/64.85/65.29	82.91/59.04/50.57
KNN	69.89	68.28/67.07/66.43	81.57/56.58/64.05
LSTM	70.27	69.05/70.46/65.32	92.70/37.11/80.57
1D-CNN	74.51	71.09/72.82/70.30	95.3049.44/73.71

are plotted with bar graphs (as shown in Figure 6 and Figure 7).

From Table 7, it can be seen that compared to other classifiers, the random forest classifier performs better in classifying the concatenation of three features into two categories (Accuracy 91.76%, Precision 91.62%, Recall 91.98%, F1-score 91.70%). Specifically, the detection accuracy of self-healing burns using random forest is 94.09%, while the detection accuracy of non-self-healing burns is 89.88%. From Table 8, the random forest classifier outperforms other classifiers in classifying the concatenation of three features into three categories (Accuracy 80.74%, Precision 77.77%, Recall 77.99%, F1-score 77.64%). In particular, the accuracy of superficial dermal burn using random forest is 96.21%, that of deep dermal burn is 67.04%, and that of full-thickness burn is 70.71%.

To further evaluate the performance of this proposed method, the images of the original dataset are rearranged evenly, and 20, 30, 40, 50, and 60 images are set for training, while the remaining images are reserved for testing. The accuracy rates obtained by random forest classification after dividing the data into three classes and two classes are shown in Figure 8. It can be seen from the chart that by changing the size of the training set, the performance of the model gradually improves as the amount of data increases, which indicates that this proposed model has good generalization ability.

4.4.2. Analysis of Experimental Results

Three effective burn depth classification methods selected from the relevant literature are compared with this proposed method, as shown in

Table 9. When selecting three features, a comparison is carried out between the results of classifying burn images into different categories and the results of Refs. [9,15,28].

	Studies	Accuracy (%)	Precision/Recall/F1-Score (%)	Category I/II/III (%)
Three Categories	The proposed method	80.74	77.77/77.99/77.64	96.21/67.04/70.71
	Rostami et al. [15]	77.8	\	\
	Zhang et al. [28]	76.87	75.28/76.83/75.21	93.35/55.92/78.23
Two Categories	The proposed method	91.76	91.62/91.98/91.70	94.09/89.88
	Rostami et al. [15]	90.5	90.6/87.9/89.22	\
	Zhang et al. [28]	85.86	86.79/87.72/86.14	93.48/84.74
	Yadav et al. [9]	82.43	82/88/85	\

. Meanwhile, to provide a more intuitive comparison of the advantages of this proposed method, a bar chart (Figure 9) is plotted based on the results of dividing the burn images into two categories, as shown in Table 9.

Table 9 shows a detailed comparison of the best performance of these methods and that of this proposed method, and it can be seen that the results have different degrees of improvement in various aspects. Especially when classifying burn images into three categories, the accuracy of this proposed method is obviously higher than that of other methods.

To assess whether the proposed method has statistical significance, a paired t-test with two tails [56] is conducted on the accuracy obtained with different classifiers. As shown in

Table 10. The p-values calculated for the accuracies obtained by the different classifiers.

	PSO-SVM	Decision Tree	KNN	LSTM	1D-CNN
Three categories	9.58 × 10−5	2.45 × 10−12	5.92 × 10−14	3.89 × 10−16	3.89 × 10−10
Two categories	0.022257	0.000132	1.87 × 10−14	4.44 × 10−18	7.33 × 10−26

, the p-values of all these classifiers are all lower than 0.05, indicating that the proposed method has statistically significant differences in terms of accuracy compared to the other approaches mentioned. Although the accuracy of using PSO-SVM to classify images into two classes is 90.95%, which is close to the best classification result, its p-value is approximately 0.02. Statistically, the performance of the random forest classifier is better than that of PSO-SVM. Therefore, it can be concluded that the proposed method outperforms the other methods presented in terms of performance.

Following many experiments, 12 representative burn images are selected from the classification results of each classifier as a group, resulting in five groups for the five classifiers, as shown in Figure 10. The images framed in red boxes are images that cannot be correctly classified in either the two-class or three-class cases. The images framed in blue boxes can only be correctly divided into two categories. The images framed in green boxes can be correctly classified in both cases. To make it easier to explain, there are circular markers in the bottom right corner of some images, and circles of the same color indicate the same image.

It can be observed from Figure 10 that the red circles mark images which belong to deep dermal burn; these cannot be correctly classified by any classifier. The yellow circles mark images that can be correctly classified by the PSO-SVM classifier and decision tree classifier. The green circles mark images that can be correctly classified by the random forest classifier, LSTM classifier, and 1D-CNN classifier. The image marked with the blue circle can only be correctly classified by the KNN classifier. Furthermore, it is found that the random forest classifier and 1D-CNN classifier may achieve better classification results for images with blisters.

4.5. Ablation Study

Combining the findings from Table 7 and Table 8, it is evident that the random forest classifier demonstrates the most favorable performance, followed by PSO-SVM. These results indicate that the proposed approach is effective in classifying burn depth in burn images. However, it is not conclusively established that all three types of features contribute positively to classification. Therefore, a series of ablation experiments are conducted, involving the isolation of each feature for individual classification using six classifiers. Subsequently, pairwise combinations of these features are tested with the random forest classifier to verify that each feature contributes to the classification.

Table 11. The classification results of different classifiers when depth features are selected and for categorizing burn images into two burn classes.

Feature Combination	Accuracy (%)	Precision/Recall/F1-Score (%)	Category I/II (%)
Random Forest	88.92	88.04/88.18/88.09	88.09/88.27
PSO-SVM	90.54	90.37/90.58/90.46	90.91/90.24
Decision Tree	83.45	84.62/83.60/83.17	84.60/82.61
KNN	85.54	86.05/85.58/85.27	84.61/86.55
LSTM	86.08	86.95/86.18/85.87	87.12/85.24
1D-CNN	89.48	89.94/89.73/89.16	93.17/86.29

and

Table 12. The classification results of different classifiers when depth features are selected and for categorizing burn images into three burn classes.

Feature Combination	Accuracy (%)	Precision/Recall/F1-Score (%)	Category I/II/III (%)
Random Forest	75.95	73.20/75.46/73.60	88.18/59.63/78.57
PSO-SVM	76.08	73.11/75.11/73.47	90/58.89/76.43
Decision Tree	66.28	63.54/66.98/63.91	80.75/45.88/72.50
KNN	68.92	69.47/65.91/62.16	85.33/59.31/51.08
LSTM	75.87	74.25/75.23/72.04	87.27/55.56/82.86
1D-CNN	75.14	73.40/74.42/72.29	89.09/57.04/77.14

present the results of categorizing depth features into two classes and three classes using classifiers after extracting them individually. It is evident that PSO-SVM achieves the best performance, with an accuracy of 90.54% for two classes and 76.08% for three classes. Random forest, 1D-CNN, and LSTM show comparable performance, indicating that the depth features extracted using Burn-ResNet51 network possess good discriminative ability to aid classifiers in distinguishing between different categories. The satisfactory performance of these classifiers may also suggest a certain level of universality in the depth features. Thus, the extraction of depth features using Burn-ResNet51 network can be considered effective and provides a solid foundation for further research. However, from another perspective, when categorized into two classes, the individual depth features already achieve an accuracy of 89.19% when classified with the random forest, indicating that the influence of color and texture features on performance is limited.

When categorizing color features into two classes and three classes using classifiers after extracting them individually, it can be clearly seen from

Table 13. The classification results of different classifiers when color features are selected and for categorizing burn images into two burn classes.

Feature Combination	Accuracy (%)	Precision/Recall/F1-Score (%)	Category I/II (%)
Random Forest	69.12	70.04/69.98/69.11	77.88/62.07
PSO-SVM	71.69	72.94/71.66/70.54	68.79/74.54
Decision Tree	68.38	69.18/68.75/67.80	70.74/66.77
KNN	68.18	70.80/69.80/67.75	79.47/60.11
LSTM	72.84	73.92/71.98/71.41	64.09/79.88
1D-CNN	71.35	71.63/71.31/71.08	72.33/70.29

and

Table 14. The classification results of different classifiers when color features are selected and for categorizing burn images into three burn classes.

Feature Combination	Accuracy (%)	Precision/Recall/F1-Score (%)	Category I/II/III (%)
Random Forest	62.03	58.22/57.62/56.78	86.97/39.81/46.07
PSO-SVM	57.16	54.87/51.76/50.79	83.63/35.93/35.71
Decision Tree	63.24	62.67/61.36/60.38	71.48/58.28/54.32
KNN	55.68	55.74/54.3/53.45	78.30/38.48/45.81
LSTM	56.03	53.74/54.89/50.70	83.03/25.93/55.71
1D-CNN	52.65	50.18/51.34/47.43	73.93/26.48/52.43

that individual color features are less effective than depth features in conveying comprehensive information about burn images. This may be due to the fact that color features are more susceptible to noise, illumination changes, and other issues. When the images are categorized into three classes, random forest and decision tree classifiers exhibit superior classification performance compared to other classifiers. However, PSO-SVM still performs better in the two-class scenario.

Table 15. The classification results of different classifiers when texture features are selected and for categorizing burn images into two burn classes.

Feature Combination	Accuracy (%)	Precision/Recall/F1-Score (%)	Category I/II (%)
Random Forest	63.72	70.21/59.97/56.29	25.30/94.63
PSO-SVM	61.28	65.02/57.54/53.51	22.88/92.20
Decision Tree	61.68	62.12/61.27/60.07	56.66/65.88
KNN	60.14	61.19/60.20/59.15	56.06/64.34
LSTM	48.58	46.03/46.06/44.00	22.73/69.39
1D-CNN	54.87	52.68/51.94/49.32	24.85/79.02

and

Table 16. The classification results of different classifiers when texture features are selected and for categorizing burn images into three burn classes.

Feature Combination	Accuracy (%)	Precision/Recall/F1-Score (%)	Category I/II/III (%)
Random Forest	36.62	48.42/42.08/36.82	22.88/33.07/66.29
PSO-SVM	40.54	46.56/42.92/39.01	29.79/51.96/45.71
Decision Tree	43.24	41.71/40.57/43.71	55.14/45.04/19.14
KNN	47.1	46.90/44.26/43.17	58.31/46.22/28.27
LSTM	33.84	36.59/34.01/31.17	20.97/44.48/39.57
1D-CNN	33.03	36.79/37.19/33.11	32.21/25.37/53.57

present the results of categorizing texture features into two classes and three classes using classifiers after extracting them individually. It is evident that the classification performance of texture features alone is not ideal. However, valuable information can still be derived. For instance, in the two-class scenario, random forest, PSO-SVM, LSTM, and 1D-CNN classifiers are more sensitive to the second class of images, which may partially compensate for the lower sensitivity of color features to the second class. This observation may also indicate that texture features can extract image information from a different perspective than color features. However, the classification results of texture features are not as good in the three-class scenario.

After conducting experiments separately on each feature, it can be concluded that depth features can more accurately represent burn image information compared to color and texture features. However, it is still not very effective to prove that all three features have a positive effect on classification. Therefore, the three types of features are combined pairwise for analysis and verification.

Table 17. The classification results of random forest classifier when features are combined in pairs and for categorizing burn images into two burn classes.

Feature Combination	Accuracy (%)	Precision/Recall/F1-Score (%)	Category I/II (%)
Color + Texture	69.32	69.29/69.51/69.22	71.21/67.80
Color + Depth	90.41	90.26/90.54/90.33	91.81/89.27
Texture + Depth	89.19	88.76/88.91/88.81	88.79/89.02

and

Table 18. The classification results of random forest classifier when features are combined in pairs and for categorizing burn images into three burn classes.

Feature Combination	Accuracy (%)	Precision/Recall/F1-Score (%)	Category I/II/III (%)
Color + Texture	63.04	58.68/57.83/57.14	86.67/41.11/45.71
Color + Depth	80.27	77.11/77.66/77.04	96.36/65.19/71.43
Texture + Depth	78.58	76.05/77.73/75.37	96.97/54.44/81.79

display the results of combining color features with texture features, color features with depth features, and texture features with depth features, and classifying them using the random forest classifier.

To provide a more intuitive comparison of the contributions made by different features in classification, all the classification results obtained using random forest in this section are compiled into a bar chart (as shown in Figure 11). It can be observed that despite the inferior classification performance when classifying texture features individually, they can still play a role when combined with other features, especially in the three-class scenario, as demonstrated in the previous experiments.

From all the experiments above, it can be seen that the proposed method is effective and feasible for classifying burn images of different depths, demonstrating that each feature contributes to the classification.

5. Discussion

A method is proposed to extract multiple features from burn images and perform feature selection. In particular, Burn-ResNet51 network is used for depth feature extraction and the random forest feature selection method is used for feature selection, enhancing the classification performance. Finally, utilizing random forest to classify burn images, the results show that the accuracy of binary classification is 91.76%, and the accuracy of three-class classification is 80.74%.

When extracting color features, color histograms are extracted from the RGB, HSV, and L*A*B* color spaces of the images as their respective color features. The advantage of this approach lies in that each color space has its unique color representation method, which can provide more color information. At the same time, color histograms can display the distribution of various colors in the image, which is helpful for image identification and classification. Currently, most studies consider the commonly used color models, namely RGB, HSV, and L*A*B* [20,25,31]. However, in some research, other color models also perform well. For example, HSI performs well in image enhancement [57], and YCbCr performs well in skin color segmentation [58]. Since the color of burned skin also contains some skin color characteristics, the YCbCr color space may also perform well. Therefore, it may be possible to extract more accurate color features by comparing various color spaces in different color extraction methods (color histograms, color correlograms, color co-occurrence matrices) [59].

Similarly, the Gabor filter used for extracting texture features possesses the drawback of non-orthogonality, resulting in redundant features at different scales. Most of the time, a single-filter resolution may not effectively capture the spatial structure in natural textures [60]. In order to compare with the Gabor filter, another geometric feature descriptor [61,62] is selected, namely the local binary pattern (LBP) [28], as a method for extracting features. However, after replacing the Gabor filter with LBP, three types of features were selected, achieving an accuracy rate of 90.28% for dividing burn depth into two categories and 79.84% for dividing it into three categories. Both of these experimental results are approximately 1% lower than those achieved by this proposed method. When only features extracted by LBP were selected, the accuracy rate for dividing burn depth into two categories was 48.92%, while the accuracy rate for dividing it into three categories was 35.37%. The experimental results are all lower than those of the original method; therefore, the Gabor filter is still chosen to extract features.

With the continuous development of deep learning, network structures are becoming increasingly intricate. Different tasks correspond to their own unique network structures and parameters, greatly enhancing the upper limit of deep learning. Therefore, future research should comprehensively consider features from fully connected layers, convolutional layers and various 1D-CNN architectures, and fine-tune parameters to optimize image classification.

In order to further evaluate the performance of the method, the total computing time is recorded (as shown in

Table 19. Summary table of several kinds of computing time of the proposed method.

	Two Categories	Three Categories
Training time (s)	73.94	70.28
Testing time (s)	2.97	7.54
Computing time (s)	76.91	77.82
Computing time without feature selection (s)	76.99	73.25

), which is the time taken to train the model on the burn injury images in the training set and then classify the images once. The total computing time includes training time and testing time, with records also kept for the computing time without feature selection. When the burn depth is divided into two classes, the computing time is 76.91 s. For three classes, the total computing time is 77.82 s. It was observed that when images are categorized into three categories, the computing time for the feature selection process increases significantly due to the excessive number of trees. However, at the same time, the corresponding accuracy also improves significantly (as shown in Table 6). Moreover, a random forest is used as a feature selection method, which is based on random sampling and is sensitive to noise or outliers, potentially affecting the accuracy of feature selection. Therefore, selecting an efficient and stable feature selection method is also an important direction for future research.

From the dataset selected in this work, the number of burn images is limited, and there is a lack of complex burn skin images (e.g., burn skin with skin diseases) for comparison analysis. This insufficiency could undermine the performance of the proposed approach when dealing with different depth regions of complex burn images or burn images with background interference. Therefore, it is necessary to collect more diverse burn samples to further validate the effectiveness of the proposed method [28].

In addition, this method is based on the analysis of burn images, which makes it difficult to completely avoid the influence of factors such as brightness, illumination angle, sharpness, background information. While the classification performance of this proposed method has been greatly improved compared with other relatively simple methods, it is still difficult to distinguish the correct category of the images similar to those marked by blue and red circles in Figure 10, which have a large color difference from other images in the same class. Therefore, perhaps combining the method proposed in this study with the incomplete multi-view clustering (IMC) method that restores views with missing information [63] or the method from Ref. [64] that solves the double incomplete learning problem of multi-view multi-label classification with missing views and missing labels would be very helpful for the classification of burn depth in burn images.

6. Conclusions

A novel method is introduced to assist medical experts in determining the depth of burn injuries through automated evaluation of burn images. This method utilizes color histograms extracted from three different color models as color features, and employs Gabor filters to extract texture features. Subsequently, features are extracted from the global average pooling layer of an enhanced Burn-ResNet51 network (a derived network of the ResNet-50) as depth features. These three types of features are then combined and selected using a feature selection method based on random forest. Finally, a classifier is used to classify the selected features.

A comprehensive series of experiments is conducted to highlight the critical role of deep learning in feature extraction and to validate the effectiveness of this proposed method. Compared with other existing burn depth classification techniques, this proposed method exhibits superior performance. In the future, more burn injury samples can be collected for testing, and it is also an interesting research direction to choose appropriate feature extraction methods and feature selection methods based on the characteristics of burn injury images.