Discrimination of Steel Coatings with Different Degradation Levels by Near-Infrared (NIR) Spectroscopy and Deep Learning

Abstract

Assessing the current condition of protective organic coatings on steel structures is an important but challenging task, particularly when it comes to complex structures located in harsh environments. Near-infrared (NIR) spectroscopy is a rapid, low-cost, and nondestructive analytical technique with applications ranging from agriculture, food, and remote sensing to pharmaceuticals. In this study, an objective and reliable NIR-based technique is proposed for the accurate distinction between different coating conditions during their degradation process. In addition, a state-of-the-art deep learning method using a one-dimensional convolutional neural network (1-D CNN) is explored to automatically extract features from the spectrum. The characteristics of the spectrum show a downward trend over the entire wavenumber period, and two major absorption peaks were observed around 5250 and 4400 cm⁻¹. The experimental results indicate that the proposed deep network structure can powerfully extract the complex characteristics inside the spectrum, and the classification accuracy of the training and testing data was 99.84% and 95.23%, respectively, which suggests that NIR spectroscopy coupled with a deep learning algorithm could be used for the rapid and accurate inspection of steel coatings.

1. Introduction

Assessing the current condition of protective organic coatings on steel structures is an important but challenging task. It is crucial for asset owners to understand when the coating is no longer effective, so the repair can be applied before damage is done to structures due to the failure of protective coatings, and the improvement in maintenance planning can have a significant financial benefit. Common coating assessment practices involve trained inspectors performing close-up visual inspections and using a rating system. However, these practices are not only labor-intensive but also time-consuming and pose significant safety and logistical challenges when structures are located in harsh places [1]. Therefore, an efficient and intelligent evaluation approach is highly required for making informed decisions about protective coating maintenance.

Near-infrared (NIR) spectroscopy is a rapid, nondestructive and relatively inexpensive analytical technique. It characterizes materials based on their absorption intensity in the NIR region of the electromagnetic spectrum, which ranges from 700 to 2500 nm. These optical responses in the NIR region reflect vibrations of molecular functional groups containing atoms like C, N, O, and S or chemical bonds between atoms, such as C=O, C=H and C-O-C, which allows researchers to analyze samples of organic composition [2]. Therefore, the technique has found broad application in agriculture, food, pharmaceuticals, remote sensing, and several other fields [3,4,5]. For example, Piehl et al. [6] presented the first quantitative and qualitative analysis of plastic contamination in agricultural soil based on the NIR technique. Via Fourier transform infrared (FTIR), they were able to identify and quantify macro and microplastic pieces within the investigated area, which therefore provided important data to determine the extent of contamination concerning agroecosystems. Valand et al. [7] provided an extensive review of the application of NIR spectroscopy for food adulteration and authenticity. The literature has shown that, over the last decades, NIR has proven itself to be a trustworthy technology for examining foods for adulteration and authenticity. It is not only a fast, easy, and generally cost-effective method, but it also can combine with other analytical chemistry techniques to develop validated or standardized methods.

Meanwhile, methods used to extract and process the NIR analytical information to produce quantitative and qualitative models has evolved during the last decade. Sohn et al. [8] proposed a combination model of Savitzky–Golay and a support vector machine (SVM) to classify six different Amaranthus species in Korea, and the result shows that Vis-NIR spectroscopy with an SVM model has the capability to discriminate Amaranthus species with a notable accuracy up to 99.7%. Nawar et al. [9] established a random forest (RF) modeling approach for the quantitative analysis of soil organic carbon (OC). The results suggest that RF regression with spiking provides an accurate prediction of OC under both indoor laboratory and on-site field scanning conditions. Sampaio et al. [10] compared the performance of partial least squares (PLS), interval-PLS, synergy interval-PLS, and moving windows-PLS models, and developed an optimal regression model with high accuracy for rice amylose determination. Recently, the deep learning algorithm has attracted increasing attention for NIR researchers; it has shown great capabilities in creating powerful analytic models based on multilayer abstraction to represent concepts or features [11,12,13]. Chen et al. [14] proposed a framework for a backpropagation, neural deep learning 1D-CNN to predict the nutrition components in soil samples; Rong et al. [15] applied a simple CNN architecture with a single convolutional layer to distinguish peach varieties; Gholizadehhis et al. [16] examined the capability of vis–NIR spectra coupled with traditional machine learning techniques and a fully connected neural network (FNN) to assess potentially toxic elements in forest soils. The results show that FNN provides better results in the availability of a large sample size. These research works indicate that deep learning can be successfully applied to NIR sensor data analysis.

A lot of research works have been conducted with respect to the application of NIR technology in the field of monitoring or the inspection of steel structure coatings [17,18,19]. Kishigami et al. [20] recently demonstrated the use of NIR in the estimation of the degree of abrasion of coating thickness. It was found that the observed infrared intensity could be used to estimate the top coating thickness based on the calibration relationship they discovered. Omar et al. [21] developed a novel integrated device based on FTIR and micro-electromechanics to make structural analyses of the epoxy coating of steel pipelines. It shows that their instrument was useful for on-site material analysis, especially in the investigation of the mechanical properties and detection of the distribution of particles inside the material. Raeissi et al. [22] explored the use of a k-means clustering algorithm on the NIR portion of hyperspectral images (NIR-HIS) to provide diagnostic information about the spatial inhomogeneities of the chemical structures of an applied steel coating.

The aim of the study was to present an innovative approach based on NIR spectroscopy as a novel solution to objectively assess the condition of the protective coatings applied to steel structures. Moreover, this method of nondestructive evaluation could provide precise and automatic grading in the assessment of coating degradation due to age or environmental factors. The potential of the approach was shown using a real NIR dataset acquired from prepared coating samples with an accelerated aging process. In addition, a modern convolutional neural network (CNN) was developed to classify the different grades of corrosion based on their NIR data. CNNs have achieved promising performances in such classification tasks due to their large flexibility regarding the dimensionality of the operational layers, their depth and breadth, and their ability to extract strong features about the input data [23]. With the excellent learning ability of CNNs, a spatial distribution of the intensity variations at particular absorption lines in the electromagnetic spectrum of NIR data can be reconstructed as essential features, which can be used for the identification and for the evaluation of chemical changes that occur as a result of coating degradation.

2. Materials and Data Collection

2.1. Sample Preparation

The protective coating used in this study was based on three layers of a composite coating system that was applied to the world-famous Hongkong-Zhuhai-Macro Bridge (HZMB). The coating system was formulated with zinc-rich epoxy primer, MIO epoxy intermediate paint, and a fluorocarbon topcoat, and each layer has a thickness of 100, 200, and 80 μm, respectively. Moreover, the composite coating was applied to steel plates using a high-pressure airless spraying method, following the specifications of painting and coating for steel structures in China. Chinese steel of grade Q235 was used for the steel plates, of which the dimensions were 100 mm × 50 mm × 10 mm. Figure 1 shows a real image of a steel plate with the composite coating.

2.2. Data Acquisition and Labelling

In order to simulate the behavior of coating degradation, 50 steel plates with a protective composite coating were exposed to a salt spray test device to accelerate corrosion growth. The salt spray test was performed following the instruction of the ISO 9227 standard, and each coated plate was grouped and subjected to a different exposure time: 120, 240, 360, 480, 600, 720, 840, 960, and 1080 h. Each sample was rinsed with water and air-dried before NIR data collection. The data collection was based on FTIR measurements performed using a BRUKER Lumos FTIR microscope. Reflectance spectra were recorded in the range of 8500 to 4000 cm⁻¹. The resolution of the obtained spectra was 3.5 cm⁻¹. For each coated sheet, six points located in the corner and center were measured individually. A total of 300 individual point spectra were recorded per one measurement, and a total of 10 measurements were made based on the different exposure times.

The labelling task was performed shortly after the NIR data collection of each steel sheet, and a human expert was present at the collection site to make an observation of the actual sample surface and assigned a label corresponding to their condition ratings. A four-level rating system is applied by inspectors and, in our study, is described in

Table 1. The four-level coating condition rating system.

Coating Condition	Number of Training Set	Number of Testing Set	Description
Level 1	375	99	The coating remains intact
Level 2	566	146	The coating is slightly degraded, with speckled rusting in areas that are less than 1% of the total surface area.
Level 3	356	90	The coating is moderately degraded, with speckled rusting in areas greater than 1% and less than 40% of the total surface area.
Level 4	286	73	The coating is no longer effective, with speckled rusting in areas larger than 40% of the total surface area.
Total	1583	408	/

. The NIR dataset of a steel coating is then generated and referred to as a matrix of 1991 rows and 1201 columns. Each row represents the spectral information of the individual point from the coated steel sheet; each column (descriptive features) represents the reflectance magnitude of the NIR spectral band with a specific wavelength, which ranges from 8500 to 4000 cm⁻¹; the last column has the condition class labels that are associated with each point. As shown in Table 1, the dataset is programmed to randomly divide into a training set and a testing set with a proportion of 80:20. The training set is used to train the model parameters and the testing set is used to check the classification performance of the trained model.

3. Methodology

3.1. Pre-Processing of NIR Data

Preprocessing of spectral data is the most important step before the subsequent modeling and analyzing. The objective of preprocessing is to remove physical phenomena in the spectra, such as baseline drift, high-frequency noise, and mutual interference between the components [9]. In this study, multiple preprocessing techniques were adopted based on four categories: smoothing, scatter correction, spectral derivatives, and wavelet denoising.

Table 2. Comparison of preprocessing method denoising effects.

Categories	Pre-Processing Methods	RMSE	SNR
Smoothing	Mean average (MA) smoothing	0.0545	20.18
	Savitzkygolay (SG) smoothing	0.0112	29.94
Scatter Correction	Multiplicative scatter correction (MSC)	0.0322	28.50
	Standard normal variate (SNV)	0.0322	28.50
Spectral Derivatives	1st Derivatives	0.2208	8.05
	2nd Derivatives	0.2253	7.88
Wavelet Denoising	Haar wavelet	0.2813	5.94
	Daubechies wavelet	0.0178	29.89
	Coiflets wavelet	0.0109	34.16
	Symmlets wavelet	0.0127	32.86

presented general information about these methods and a comparison of their denoising effect using root mean square error ($RMSE$) and signal-to-noise ratio ($SNR$). The two metrics are calculated by

$$\begin{array}{c}SNR=10\times \log \left(\frac{{{\displaystyle \sum }}_{n=1}^{N}f{(n)}^2}{{{\displaystyle \sum }}_{n=1}^N{[f\left(n\right)-\widehat{f}\left(n\right)]}^2}\right)\end{array}$$

(1)

$$\begin{array}{c}RMSE=\sqrt{\frac{1}{n}{\displaystyle {\displaystyle \sum }_n}{[f\left(n\right)-\widehat{f}\left(n\right)]}^2}\end{array}$$

(2)

where $f\left(n\right)$ is the original spectrum, $\widehat{f}\left(n\right)$ is the denoised spectrum, and $n$ is the sampling point. It can be seen that SG smoothing, MSC, 1st derivatives, and wavelet denoising with coiflets base have the best denoising effect in their category, respectively. However, the optimal preprocessing method will be determined according to the performance and robustness of the classification model, which is explained in detail in the next section.

3.2. Architecture of the Proposed CNN Based Model

The CNN network can be variously arranged depending on the designed parameters and depths of the structure as well as the training method of the network. The basic architecture of the 1-D CNN in this paper consists of an input layer, multiple convolutional and pooling layers stacked together, a fully connected layer, and an output layer. A schematic diagram of this process is shown in Figure 2. As the obtained NIR input data represent a one-dimensional vector, the convolutional layer filters the input data with a one-dimensional kernel to obtain subtle feature information. A convolution kernel size of 1 × 9 is adopted in the first two convolutional layers to quickly obtain rough feature information, and then the convolution kernel with a small size of 1 × 3 is selected to extract more subtle features. In order to control the shrinkage of the dimensions, full zero padding and upward rounding are adopted in the process of convolution. The output of every convolutional layer is then passed to the pooling layer to reduce the size of the feature map, and the maximum operation is used. The size of the pooling layer is set as 1 × 2 with a step of 2 in this study. The final feature maps from the pooling layer are then flattened and passed to the fully connected layers at the end of the network. The Rectified Linear Unit (ReLU), the most commonly used activation function, is adapted in this model to implement nonlinear transformations [24]. The model applied Batch Normalization over the output of the convolutional layer to carry out the standardization process and the dropout mechanism is used to alleviate overfitting. The function of cross entropy is selected as the loss function in this study to quantify the difference between two probability distributions [25]. Lastly, the mathematical function of Softmax is implemented for the neural network of multiple classifications. In order to train the proposed CNN model, the learning rate is set to 0.001, the batch size is set to 64, and the epoch of training is set to 100.

3.3. Performance Evaluation

In order to evaluate the overall performance of the proposed model, the following four criteria were used: classification accuracy, precision, recall, and F1 score. These metrics are calculated by

$$\mathrm{Aaccuracy}=\frac{{\displaystyle \sum _{i=1}^n}\left(T{P}_i+F{N}_i\right)}{{\displaystyle \sum _{i=1}^n}\left(T{P}_i+T{N}_i+F{P}_i+F{N}_i\right)}$$

(3)

$$\mathrm{Precision}=\frac{1}{n}\sum _{i=1}^n\frac{T{P}_i}{T{P}_i+F{P}_i}$$

(4)

$$\mathrm{Recall}=\frac{1}{n}\sum _{i=1}^n\frac{T{P}_i}{T{P}_i+F{N}_i}$$

(5)

$$F1=\frac{1}{n}\sum _{i=1}^n\frac{2T{P}_i}{2T{P}_i+F{P}_i+F{N}_i}$$

(6)

where $TP$, $TN$, $FP$, and $FN$ are true positive, true negative, false positive, and false negative, respectively. A confusion matrix is provided to demonstrate insight information for the predictions and to comprehend other classification metrics.

3.4. Software Tools

The Python 3.6 program with PyCharm IDE was used to perform spectral extraction, preprocessing, and other analysis models. The 1D-CNN model was programmed using the Pytorch framework, running on the graphics processing unit.

4. Results and Discussion

4.1. Spectral Characteristic

The original and mean NIR spectra of the steel coatings with different grades of corrosion are shown in Figure 3a,b, respectively. Overall, the original spectrum of the four kinds of coating conditions showed a downward trend over the entire wavenumber period. Two major absorption peaks were observed around 5250 and 4400 cm⁻¹. A peak around 5250 cm⁻¹ is typically attributed to the hydroxyl ring, which represents a combination of asymmetric stretching and bending of O-H, and the peak around 4400 cm⁻¹ represents the combination band of the second overtone of the epoxy ring [22,26,27]. Both characteristics are highly correlated to coating degradation. Another clear trend from Figure 3b was a decrease in the coating degradation in the reflectance of the spectra features. This phenomenon reflected the thickness and total surface coverage of the corrosion compounds within the analysis area [28].

4.2. Preprocessing Method

NIR spectral data are often interfered with by stray light, noise, baseline drift, and other factors, thus affecting the final qualitative and quantitative analysis results. In this paper, multiple pretreatment methods were used for the comparative analysis, and the following accuracy results of the proposed CNN model are shown in

Table 3. The accuracy results of the 1D-CNN model with different preprocessing methods.

Categories	Pre-Processing Methods	Accuracy (%)
Raw Data	/	91.8
Smoothing	Mean average (MA) smoothing	92.8
	Savitzkygolay (SG) smoothing	95.8
Scatter Correction	Multiplicative scatter correction (MSC)	94.7
	Standard normal variate (SNV)	95.0
Spectral Derivatives	1st Derivatives	80.3
	2nd Derivatives	61.5
Wavelet Denoising	Haar wavelet	55.7
	Daubechies wavelet	85.1
	Coiflets wavelet	91.0
	Symmlets wavelet	84.8

. Compared with other methods, SG smoothing is more competitive, with a 95.8% accuracy, but the MSC and SNV methods reached close accuracy results of around 95%. Obviously, spectral derivative methods are not suitable for the proposed model. It is worth mentioning that wavelet methods (coiflets and symmlets family) showed an inferior performance of 85–90% accuracy while having better denoising results based on the RMSE and SNR results in Table 2. In summary, SG smoothing was selected as the preprocessing method for the proposed discriminant model.

4.3. CNN-Based Steel Coating Condition Assessment Model

The results for loss value and prediction accuracy on the training set and test set are shown in Figure 4 and Figure 5. As can be seen in Figure 4, the loss value shows a trend that drops down rapidly during the first 20 epochs, then decreases slowly until it is steady. In the end, the loss values of the training set and the testing set were 0.759 and 0.829, respectively. In Figure 5, it is observed that the classification performance of the 1D-CNN model shows a trend of rapid rise, gradual and slow increase, and then tends to be stable. After 40 epochs, the accuracy rate of the testing and training sets reach 99.84% and 95.23%, respectively.

In order to better explain the prediction results of the 1D-CNN model for each category in the test set, a confusion matrix was introduced and is shown in Figure 6. As can be seen from the confusion matrix, the predictions of a coating with a level 1 condition are all correct, and only one sample with a level 2 coating condition is falsely predicted as being level 3. The coating samples with a level 3 condition show the lowest prediction accuracy, with a total of 14 samples being misclassified; most of them are falsely predicted as a level 4 condition. Similarly, seven coating samples with a level 4 condition are misclassified to level 3. The result of the confusion matrix indicates that most false predictions occur in condition levels 3 and 4.

The overall performance results are shown in

Table 4. The performance results of the proposed 1D-CNN model.

Performance	Train (%)	Test (%)
Accuracy	99.84	95.23
Precision score	99.83	94.90
Recall score	99.73	94.74
F1 score	99.81	94.48

, which confirms that, for limited high-dimensional data, the proposed 1D-CNN model has excellent classification ability and can discriminate the condition levels of steel coatings. This reveals the superiority of the deep learning model with a high ability for feature extraction and learning over traditional processing.

5. Conclusions

In this paper, a 1D-CNN-based model was developed for the assessment of steel coating conditions using NIR data. Four levels of coating degradation were constructed by artificially-accelerated aging, and the data were collected based on different exposure times. With the current popular research method of deep learning, NIR data can be directly input into the model, extracting feature information from the spectrum and conducting automatic learning. Therefore, an accurate assessment of the coating condition can be achieved. The major findings of this paper can be summarized as follows:

• The characteristics of the spectrum showed a downward trend over the entire wavenumber period, and two major absorption peaks were observed around 5250 and 4400 cm⁻¹;
• A decrease in the reflectance of the spectrum features was observed along with the coating degradation process;
• A comparison of the different preprocessing methods indicated that the SG smoothing method was the most suitable method for the proposed model to effectively improve classification performance;
• Based on the above data and pretreatment, the experimental results of the proposed model achieved an overall prediction accuracy of 95.8% and very minimal error measures.

All the findings suggest that the proposed 1D-CNN framework coupled with NIR data has great potential for steel coating assessment and can provide rapid and accurate predictions of coating degradation levels. The next steps in this research should be to collect more coating data under a complex environment to build a comprehensive database; thus, the robustness of the model can be improved.