Online Pyrometry Calibration for Industrial Combustion Process Monitoring

Abstract

Temperature and its distribution are crucial for combustion monitoring and control. For this application, digital camera-based pyrometers become increasingly popular, due to its relatively low cost. However, these pyrometers are not universally applicable due to the dependence of calibration. Compared with pyrometers, monitoring cameras exist in all most every combustion chamber. Although these cameras, theologically, have the ability to measure temperature, due to lack of calibration they are only used for visualization to support the decisions of operators. Almost all existing calibration methods are laboratory-based, and hence cannot calibrate a camera in operation. This paper proposes an online calibration method. It uses a pre-calibrated camera as a standard pyrometer to calibrate another camera in operation. The calibration is based on a photo taken by the pyrometry-camera at a position close to the camera in operation. Since the calibration does not affect the use of the camera in operation, it sharply reduces the cost and difficulty of pyrometer calibration. In this paper, a procedure of online calibration is proposed, and the advice about how to set camera parameters is given. Besides, the radio pyrometry is revised for a wider temperature range. The online calibration algorithm is developed based on two assumptions for images of the same flame taken in proximity: (1) there are common regions between the two images taken at close position; (2) there are some constant characteristic temperatures between the two-dimensional temperature distributions of the same flame taken from different angles. And those two assumptions are verified in a real industrial plants. Based on these two verified features, a temperature distribution matching algorithm is developed to calibrate pyrometers online. This method was tested and validated in an industrial-scale municipal solid waste incinerator. The accuracy of the calibrated pyrometer is sufficient for flame monitoring and control.

1. Introduction

Combustion may be found in a wide range of industrial processes, such as thermal power generation, and metallurgy processes [1], where combustion is closely associated with pollution emission and energy consumption. For environmental protection and energy saving, efficient and precise combustion control and monitoring become increasingly important [2].

In current practice, most combustion control and monitoring technologies are based on the analysis of flue gases. Nevertheless, measurement provided by a flue gas analyser might be neither sufficient nor in time to indicate an abnormal state of a combustion process [2,3], because the flue gas analyser and the combustion chamber are distantly separated in most industrial processes. As a result, it is obviously insufficient to take action on burner settings to optimize the combustion processes only based on the information supplied by flue gas analysers.

This situation could be improved if the flame temperature and distribution could be used for combustion control and monitoring [4].

Digital flame imaging has been historically an invaluable technology in combustion monitoring since it provides spatial status of the flames. During past decades, many advanced flame imaging devices have been developed for temperature distribution measurement. These are mainly divided into two categories: active imaging technologies (also called laser-based technologies) [5,6] and passive imaging technologies [7,8,9]. Due to the high cost and difficulty to set up, the former are almost only applied to laboratorial systems [2]. By contrast, the latter receive more attention in industry, because they could achieve high temporal and spatial resolution at a reasonable cost by employing conventional digital cameras [2,3].

Nowadays, video cameras could be found in almost all industrial combustion facilities. They are, however, only utilized for online display on control room monitors. The operator subjectively analyses the flame image based on previous experience to identify the combustion state [10]. Theoretically, it is possible to convert a video camera to a pyrometer after a proper calibration. However, such a pyrometer is still not widely adopted in industry. Currently, with the development of machine learning and image processing technologies, more and more research attentions are attracted on how to monitor the combustion state with the deep information of the flame image [11,12,13,14]. The features extracted by machine learning methods can help to better understand the state of the current operating process and they can be used in close-loop control [15]. However, those methods treated the combustion process as a black box resulting in that the performance of the monitoring model mainly depends on the scale and quality of the labelled data [16], and have no clear physical meaning. If the domain knowledge and the physical principles can be integrated in machine learning, it would improve the model performance and generalization [17,18]. Compared with raw image data, temperature and its distribution are easier to link with domain knowledge and physical principles. Therefore, if the flame temperature distribution could be measured, performance of combustion monitoring and control would be significantly improved.

Calibration is one of the biggest challenges to equip digital cameras with pyrometry [19], because the parameters of digital camera-based temperature measurement model are closely related to the inherent attributes of cameras. There are two classes of commonly used calibration methods at present:

1.

Indirect methods: camera spectral response calibration

2.

Direct methods: estimating model parameters with reference temperature

The indirect methods focus on estimating camera spectral response curve. The most common approach in this category is the radiometric self-calibration based on High Dynamic Range (HDR) imaging [20]. In this method, the exposure setting of the camera to be calibrated is gradually increased resulting in a spectrum of images from underexposed to saturated. However, the video cameras used in industries might not be set exposure freely.

For the direct methods, Equation (1) represents the general direct calibration process.

$$\theta =arg\underset{\theta }{min}\parallel {\mathbf{T}}_g-\mathbf{f}(\mathbf{y},\theta )\parallel$$

(1)

where ${\mathbf{T}}_g$ is the reference temperature and $\mathbf{f}(.)$ represents the temperature measurement model. $\theta$ is the model parameter and $\mathbf{y}$ is the sensor output signal which is used to measure temperature. As shown in Equation (1), the calibration is a process of estimating unknown parameters in the temperature measurement model by comparing estimated temperature with reference temperature. The reference temperature is usually supplied by the blackbody furnace [21]. That means the calibration experiment must be done in advance in a laboratory.

In summary, employing those existing methods, a specific series of photos have to be taken for calibration, which are either the same picture with different camera parameters, or a series of blackbody furnace radiation pictures for the target temperature range. That means that the camera could not be used for online monitoring during the calibration process. Moreover, if the existing video camera is replaced with a calibrated camera, although the temperature distribution could be measured, the operational knowledge based on the original camera image may become invalid, because the flame images in colours, shapes and positions captured by different cameras may not be the same. For not fully automated combustion processes, such operational knowledges are essential for daily operation. Using a new camera may significantly affect normal operation. Besides, it is difficult to add one optical access to combustion chamber for the camera in a practical device. Indeed, windows and window mounts have to withstand flow’s pressure and temperature in the chamber. Film cooling using purge gas may also be required for in-situ measurements to cool the windows and prevent soot and particles from striking and depositing on the surface. Although future combustors may be designed to have optical access components, retrofitting existing systems still need clever engineering [3]. As a result, it will be desirable if the calibration could be accomplished online. Consistently, it will improve the performance of combustion control and monitoring in a very convenient and inexpensive way.

In this paper, a procedure for on-line calibration of a video camera in operation with a pre-calibrated pyrometer is proposed. Because the pre-calibrated pyrometer and the video camera take flame images at different locations, angles and times, whilst industrial flames are always flickering, it is difficult to find a common region in flame images taken by the two cameras. To solve this problem, in this paper, an on-line calibration algorithm based on flame temperature distribution matching is developed. The calibration procedure and the algorithm are tested in an industrial-scale municipal solid waste incinerator (MSWI).

This paper focuses on how to calibrate the digital camera in use for pyrometry. The main contributions are as follows:

1.

An experimental procedure for online pyrometry calibration of the camera in use is proposed.

2.

Experimentally, it is found that there are two peaks in the flame temperature distribution, and the temperature corresponding to the peak remains unchanged in a short period. Based on this phenomenon, a digital camera-based pyrometry calibration algorithm is developed.

3.

The procedure and algorithm are tested in an industrial-scale MSWI. The calibration was successfully completed without affecting normal operation of the plant. The calibrated pyrometer meets the accuracy requirements of combustion control and monitoring.

The remainder of this paper is organized as follows. Section 2 reviews the principle of radiation thermometry and pyrometry based on digital camera. The procedure of the online calibration is then proposed in Section 3. In Section 4, the details of the calibration algorithm are given. A real calibration case study is reported in Section 5. Finally, this work is summarised in Section 6.

2. Digital Camera-Based Pyrometry Model

2.1. Principle of Thermal Radiation Temperature Measurement

According to Planck’s Law [22], the radiation intensity $I_{\lambda }$ of a grey body object at wavelength $\lambda$ per unit area per unit wavelength is:

$$I_{\lambda }(\lambda ,T)=\epsilon \left(\lambda \right)\frac{2\pi h{c}^2}{{\lambda }^5}\frac{1}{\exp \left[\frac{hc}{\lambda kT}\right]-1}$$

(2)

where h is the Planck’s constant, c is the speed of light, k is the Boltzmann constant, T is the grey body object temperature and $\epsilon \left(\lambda \right)\in [0,1]$ is the object emissivity.

If the object wavelength $\lambda$ is within the range of 300 nm to 1000 nm and the object temperature T is between 800 k to 2000 k, $hc/\left(\lambda kT\right)\gg 1$. Then, Planck’s Law can be simplified to Wien’s Law [22]:

$$I_{\lambda }(\lambda ,T)=\epsilon \left(\lambda \right)\frac{2\pi h{c}^2}{{\lambda }^5}\exp [-\frac{hc}{\lambda kT}]$$

(3)

2.2. Digital Camera-Based Pyrometry

In order to capture colourful image, the colour filters are configured in the front of image sensors. The digital cameras use red, green and blue filters at the same time. The camera image sensor model can be expressed as [20]:

$$I_i=K_i{\int }_{380}^{780}{I}_{\lambda }(\lambda ,T)·{\eta }_i\left(\lambda \right)d\lambda ,i=R,G,B$$

(4)

where $R,G$ and B stand for the Red, Green and Blue channels, $I_i$ is the intensity of sensor response, $K_i$ is the amplification factor of sensor and ${\eta }_i\left(\lambda \right)$ spectral response factor. Applying mean value theorems for definite integrals to Equation (4), it can be simplified to:

$$I_i=400K_i{\eta }_i\left({\lambda }_i\right)I_{\lambda }\left({\lambda }_i,T\right),i=R,G,B$$

(5)

where ${\lambda }_i$ is a constant between 380 nm to 780 nm.

And substituting Equation (3) into Equation (5) and taking the logarithm yields:

$$\begin{array}{cc}\hfill & \frac{1}{T}=\frac{b\left({\lambda }_i\right)}{a\left({\lambda }_i\right)}-\frac{1}{a\left({\lambda }_i\right)}\log \left(I_i\right)=\tilde{b}+\tilde{a}\log \left(I_i\right)\hfill \\ \hfill & a\left({\lambda }_i\right)\triangleq \frac{hc}{k{\lambda }_i}\hfill \\ \hfill & b\left({\lambda }_i\right)\triangleq -5{\lambda }_i+\log \left(400K_i{\eta }_i\epsilon \left({\lambda }_i\right)2\pi h{c}^2\right)\hfill \\ \hfill & \tilde{b}\triangleq \frac{b\left({\lambda }_i\right)}{a\left({\lambda }_i\right)}\hfill \\ \hfill & \tilde{a}\triangleq -\frac{1}{a\left({\lambda }_i\right)},i=R,G,B\hfill \end{array}$$

(6)

where let $\tilde{a}=-1/a\left({\lambda }_i\right)$ and $\tilde{b}=b\left({\lambda }_i\right)/a\left({\lambda }_i\right)$ to simplify the expression. $\tilde{a}$ and $\tilde{b}$ are the pyrometry model parameters. According to [21], if the emissivity $\epsilon$ is wavelength $\lambda$ independent at the known temperature, the radio pyrometry can yield better result. It is reasonable to set $\epsilon \left({\lambda }_R\right)/\epsilon \left({\lambda }_G\right)\approx 1$ [23]. Besides, two-colour pyrometry is less sensitive to noise than other higher radio methods [21]. Applying the radio pyrometry, Equation (5) is changed into:

$$\begin{array}{c}\hfill r\triangleq {\displaystyle \frac{I_R}{I_G}}={\displaystyle \frac{K_R{\eta }_R{\lambda }_G^5}{K_G{\eta }_G{\lambda }_R^5}}{e}^{{\displaystyle \frac{-hc}{kT}}({\displaystyle \frac{1}{{\lambda }_R}}-{\displaystyle \frac{1}{{\lambda }_G}})}\end{array}$$

(7)

where r is defined as the ratio of R channel radiation intensity to G channel radiation intensity, and taking the logarithm on both sides of Equation (7) yields:

$$\begin{array}{cc}\hfill & \frac{1}{T}=\frac{\overline{b}}{\overline{a}}-\frac{1}{\overline{a}}\log \left(r\right)={\tilde{b}}_r+{\tilde{a}}_r\log \left(I_i\right)\hfill \\ \hfill & \overline{a}\triangleq \frac{hc}{k}({\displaystyle \frac{1}{{\lambda }_R}}-{\displaystyle \frac{1}{{\lambda }_G}})\hfill \\ \hfill & \overline{b}\triangleq \log ({\displaystyle \frac{K_R{\eta }_R{\lambda }_G^5}{K_G{\eta }_G{\lambda }_R^5}})\hfill \\ \hfill & {\tilde{b}}_r\triangleq \frac{\overline{b}}{\overline{a}}\hfill \\ \hfill & {\tilde{a}}_r\triangleq -\frac{1}{\overline{a}}\hfill \end{array}$$

(8)

${\tilde{a}}_r$ and ${\tilde{b}}_r$ are the radio pyrometry model’s parameters, and express Equation (8) in matrix form:

$${\displaystyle \frac{1}{{\mathbf{T}}_g}}=\mathbf{R}\theta$$

(9)

where $1/{\mathbf{T}}_g\in {\mathbb{R}}^{N\times 1}$ is a vector of the ground-true temperatures of the samples. $\theta ={\left[{\tilde{b}}_r{\tilde{a}}_r\right]}^{\top }\in {\mathbb{R}}^{2\times 1}$ is the parameter of this pyrometry model, and ${\theta }^{*}$ is the optimal parameter. Each element in $\mathbf{R}\in {\mathbb{R}}^{N\times 2}$ is

$$\begin{array}{cc}\hfill R_j& =\left[1\log \left(r_j\right)\right],j=1,2,\dots ,N\hfill \\ \hfill \mathbf{R}& =\left[\begin{array}{c}{R}_1\\ R_2\\ \dots \\ R_j\\ \dots \\ R_N\end{array}\right]\hfill \end{array}$$

(10)

where the subscript j denotes the j-th sample.

If the temperature of N pixels is known, the calibration process can be formulated as

$${\theta }^{*}=arg\underset{\theta }{min}\parallel {\displaystyle \frac{1}{{\mathbf{T}}_g}}-\mathbf{R}\theta \parallel .$$

(11)

And the closed-form solution of Equation (11) is

$${\theta }^{*}={\left({\mathbf{R}}^{\top }\mathbf{R}\right)}^{-1}{\mathbf{R}}^{\top }{\displaystyle \frac{1}{{\mathbf{T}}_g}}.$$

(12)

2.3. Revision of Radio Pyrometry

Combining Equations (8) and (5), it can be seen that the parameters of the radio pyrometry model are affected by temperature, so the traditional radio pyrometry is only effective within a certain temperature range.

In the light of Equation (5), under the same radiation intensity, the intensity of sensor response ${\lambda }_i$ is determined as long as the temperature is determined. In another word, ${\lambda }_i$ is a function of T or $1/T$. And the traditional radio pyrometry ignores the change of $\lambda$ caused by temperature change, resulting from which the radio pyrometry is only satisfied in a narrow temperature range. Even after calibration, it is difficult to ensure high accuracy of temperature measurement at a large temperature range. However, the temperature range of flames is always wide in some large combustion processes, such as the municipal solid waste incineration process.

According to Equation (8), $1/T$ is a function of $\log \left(r\right)$, and ${\lambda }_i$ can be considered as a function of $\log \left(r\right)$. So ${\tilde{b}}_r$ and ${\tilde{a}}_r$ can be treated as functions of $\log \left(r\right)$. Based on this, a Taylor expansion of Equation (8) around $\log \left(r\right)=\log \left(r_0\right)$ gives:

$$\begin{array}{cc}\hfill \frac{1}{T}& \approx {\tilde{b}}_r\left(\log \left(r_0\right)\right)+{\tilde{b}}_r^{\prime }\left(\log \left(r_0\right)\right)(\log \left(r\right)-\log \left(r_0\right))\hfill \\ \hfill & +({\tilde{a}}_r\left(\log \left(r_0\right)\right)+{\tilde{a}}_r^{\prime }\left(\log \left(r_0\right)\right)(\log \left(r\right)-\log \left(r_0\right)))\log \left(r\right)\hfill \\ \hfill & ={\overline{b}}_r+{\overline{a}}_{r1}\log \left(r\right)+{\overline{a}}_{r2}\log {\left(r\right)}^2\hfill \end{array}$$

(13)

where $r_0$ is a nominal point of r. ${\tilde{b}}_r^{\prime }$ and ${\tilde{a}}_r^{\prime }$ stand for the first derivative of ${\tilde{b}}_r$ and ${\tilde{a}}_r$ with respect to $\log \left(r\right)$. So $1/T$ can be approximately expressed as a quadratic function about the logarithm of image sensor response intensity $\log \left(r\right)$. $\overline{b}$, ${\overline{a}}_1$ and ${\overline{a}}_2$ are the parameters of this quadratic function. Here, the matrix form of the revised radio pyrometry is

$$\begin{array}{cc}\hfill {\displaystyle \frac{1}{{\mathbf{T}}_g}}& =\mathbf{R}\theta \hfill \\ \hfill R_j& =\left[1\log \left(r_j\right)\log {\left(r_j\right)}^2\right],j=1,2,\dots ,N\hfill \\ \hfill \mathbf{R}& =\left[\begin{array}{c}{R}_1\\ R_2\\ \dots \\ R_j\\ \dots \\ R_N\end{array}\right]\in {\mathbb{R}}^{N\times 3}\hfill \\ \hfill \theta & ={\left[{\overline{b}}_r{\overline{a}}_{r1}{\overline{a}}_{r2}\right]}^{\top }\in {\mathbb{R}}^{3\times 1}.\hfill \end{array}$$

(14)

and Equations (13) and (14) are the revisions of radio pyrometry, they can be used in a wide range of temperature. Their calibration process and their closed-form solution can still be expressed by Equations (11) and (12).

3. Procedure of Online Pyrometer Calibration

The procedure of online pyrometer calibration is shown in Figure 1. The calibration process is generally divided into two parts: offline preparation and online experiment. Offline preparation is to calibrate a digital camera for pyrometry in labs using traditional approaches. The online experiment is to take photo at the viewing port around the video camera using the pre-calibrated camera, and then the online calibration algorithm is executed to estimate the parameters of the pyrometry model. Indeed, it is necessary to select camera parameters during the offline calibration, and appropriate camera parameters would improve temperature measurement accuracy. So in this section, firstly, advice is given on setting camera parameters via reviewing the physical model of the camera sensor, and the procedure of online pyrometer calibration will be described in detail next.

3.1. Physical Model of Camera Sensor

Figure 2 depicts four cells of a conventional camera sensor array. The sensor converts an instant, time-varying light field $I(x,y,t)$ to electric charge $c(n,m)$, which is stored in cell $(n,m)$. Formulate this process:

$$c(n,m)={\int }_0^{T_{ap}}{\int }_{C_{nm}}I(x,y,t)dxdydt$$

(15)

where $T_{ap}$ is the aperture time of the camera and $C_{nm}$ denotes the region of cell $(n,m)$. It is worth noting that each sensor of the camera has only one filter, either R or G or B. RGB colour filters are arranged particularly in digital cameras. Figure 2 is one of the filter patterns called RGGB. In order to get the colour of each pixel, demosaicing needs to be done, which interpolates the colour value of the pixels of the same colour in the neighbourhood. Take pixel $(n,m)$ as an example:

$$\begin{array}{cc}\hfill & c{(n,m)}_R=c(n,m)\hfill \\ \hfill & c{(n,m)}_G=\frac{1}{4}(c(n-1,m)+c(n+1,m)+c(n,m-1)+c(n,m+1))\hfill \\ \hfill & c{(n,m)}_B=\frac{1}{4}(c(n-1,m-1)+c(n+1,m-1)+c(n-1,m+1)+c(n+1,m+1))\hfill \end{array}.$$

(16)

For flame imaging, the light field can be expressed as Equation (3). And the traditional radio pyrometry considers that the light field $I(x,y,t)$ remains unchanged during 0 to $T_{ap}$, and because the single sensor of the digital camera is very small, the spatial variation of light field $I(x,y,t)$ could be also ignored. Thus, Equation (15) is simplified into:

$$c(n,m)=T_{ap}{A}_{nm}I(n,m)$$

(17)

where $A_{nm}$ denotes the area of a single camera sensor and $I(m,n)$ is the light field at $(m,n)$. Because $T_{ap}$ and $A_{nm}$ are constant,

$${\displaystyle \frac{c{(n,m)}_R}{c{(n,m)}_G}}={\displaystyle \frac{I{(n,m)}_R}{I{(n,m)}_G}}.$$

(18)

Because of this simplification, the shorter $T_{ap}$ is and the smaller $A_{nm}$ is, the more accurate the pyrometer will be. Therefore, setting the maximum shutter speed and choosing a high-resolution camera can obtain relatively higher accuracy for temperature measurement.

Furthermore, high ISO (camera’s sensitivity to light) usually introduce noise into the image, so setting a smaller ISO is better.

3.2. Offline Calibration of Digital Camera for Pyrometry

As shown in Equation (1), the calibration is a process of estimating unknown parameters in the temperature measurement model by comparing the estimated temperature with a certain ground-true temperature. The blackbody furnace can provide the reference temperature [21]. The photos should be shot in manual mode. The high shutter speed and small ISO should be setted according to the advice proposed in Section 3.1. Finally, the appropriate aperture size is setted to get sharp images without exposure. The reference temperature is then provided by the blackbody furnace. After that we need to set the blackbody furnace at different temperatures, and use the camera to take pictures of the blackbody at each temperature. The temperature range of the blackbody furnace should cover the potential application range. The calibration can be completed by Equation (12).

3.3. Online Calibration Setup of Digital Camera for Pyrometry

Currently, video cameras have already been installed in all most every large combustion equipments, and meanwhile, they usually have some viewing ports for human to watch combustion chamber directly. And some of those ports are physically close to the position of video cameras as a backup in case the cameras fail. So it makes sense that there are common regions between the visual fields of the video camera and those of viewing ports. That means the images taken at the viewing port have the common region with the frame captured by the video camera. Figure 3a illustrates the sketch of this scenario. So the flame temperature distribution measured via taking images at these viewing ports using a pre-calibrated pyrometer can be used to calibrate the video camera as long as the measured temperature can correspond to the video frame.

Figure 3a illustrates the scenario of the online calibration experiment. For the sake of convenience, the combustion monitoring video camera in use is referred to as the Source Camera, and the image captured by this camera is referred to as the Source Image. The image-based pyrometer (pre-calibrated camera for pyrometry) is called Reference Camera, and the image taken by this pyrometer is called Reference Image.

Usually, there will be a viewing port near the video camera for the operator to directly observe the flame. Therefore, as shown in Figure 3b, if flames are shot from the viewing port with the reference camera, it makes sense to assume that the source image has a certain common region with the reference image and the common region represents the same part of flame. Next section introduces the online calibration algorithm to obtain temperature measurement model parameters.

4. Online Pyrometer Calibration Algorithm Based on Flame Temperature Distribution Matching

As Figure 3b shows, there is a common region between the reference image and the source image, and the common region represents the same part of flame.

However, the reference image and the source image are taken from different positions, different angles and by using different cameras. As depicted in Figure 4, those two images need to be transformed into a same coordinate to align those images. The image registration technology is used to accomplish this task. If the images have enough salient structures–features [24], such as the corner point, this task will be solved more easily and efficiently.

However, the source image is full of flame in a real industrial plant. What’s worse, burning gases or vaporized fuel are always in turbulent flows, which makes the flame never be stable. Thus, even for flames captured at same times and same place, the difference between the two images could be very large. Besides, the resolutions of source camera and reference camera may differ by 1–2 orders of magnitude and the source camera may not be given a tune-up with colour balance adjustment. For example, the same object appears red in the source image but yellow in the reference image. All in all, the image registration technology may not work well in this scenario. Despite this, there are still some special zones of flame with characteristic temperature in a short time. If those areas can be detected in the source and reference images separately, the calibration can be also accomplished.

4.1. Flame Temperature Distribution and Flame Zones

As shown in Figure 5, according to flame colour, the flame can usually be divided into several parts. There exist two interesting characteristics between flame temperature distribution and flame zones:

1.

The innermost zone of flame is the unburnt vapours, this zone is the coldest. The temperature of this zone is relatively consistent at similar combustion condition. According to flame reaction kinetics [25,26], the unburnt vapours temperature range is narrow.

2.

The edge of the flame is the hottest zone of the flame, and the highest temperature of a flame is always similar at the same combustion condition too [25].

Those two characteristics are also confirmed by the measurement of the temperature field in an industrial-grade MSWI.

Figure 6a–d show images and thermal images taken from the same MSW incinerator by a camera at difficult angles and close times. As shown in Figure 6e, in MSWI, the flame temperature distribution has two modes, and the peaks of two mode are around 1400 K and 1506 K respectively. Besides, multiple temperature fields captured at different angles show the same phenomenon.

Figure 7 shows the region with temperature between 1398 K and 1402 K is the innermost flame, and the region with temperature between 1504 K and 1508 K is the edge of flame. In summary, the flame temperature distribution has two main modes, which correspond to innermost zone of flame and the edge of flame. The experiments show that the temperature distribution peaks are constant for two modes even if they are captured at different angles and times. Therefore, if the innermost flame and the edge of the flame can be accurately identified, these two characteristics can be used for calibration.

However, edge area has been changing due to the impact of turbulence. Because of the measuring noise, the non-flame area may also be regarded as the flames. Therefore, it is difficult to accurately determine the location of the edge of the flame. Compared with flame edge, the innermost zone of flame is much easier to be segmented. The innermost flame segmentation algorithm will be described in detail in the next subsection.

4.2. Innermost Flame Segmentation Algorithm

The innermost zone of flame usually has the highest grey value. According to this, Otsu’s method [27] is used to segment the innermost zone easily. It is worth noting that since there are multiple zones of flame, Otsu’s method is required to be performed multiple times to get multilevel threshold. The number of times is adjusted according to the actual division effect, until the area corresponding to the maximum threshold matches the innermost flame. The number usually is 5–8 in practice. At the same time, some morphological operations can be taken to reduce the influence of measurement errors so that few pixels in the non-innermost zone are misclassified into this zone.

The procedure of innermost flame segmentation is as follows:

1:

Transform the original colour image I into the greyscale image $I_{gray}$;

2:

The Otsu algorithm [27,28] is used multiple times to calculate the multilevel thresholds for the segmented areas.

3:

Get the binary image $I_{BW}$ using the maximum threshold calculated in step 2;

4:

Morphologically open the binary image $I_{BW}$ in order to reduce misclassification.

4.3. Temperature Distribution Matching Based Calibration Algorithm

Because the area or the number of pixels of the innermost zone in source and reference images are different, one-to-one correspondence between pixels can not be done directly. Based on the discussion in Section 4.1, the temperature distributions in two images are similar. The temperature distributions consist of 2 main modes, whose peaks are almost identical on different images. According to this, the calibration of source camera for pyrometry can be formulated as

$$\begin{array}{cc}\hfill {\theta }^{*}=& \underset{\theta }{\mathrm{argmin}}\{{[\max \left(p\left({\mathbf{T}}_{r,in}\right)\right)-\max \left(p\left({\mathbf{T}}_{s,in}\left(\theta \right)\right)\right)]}^2\hfill \\ & +{[\max \left(p\left({\mathbf{T}}_{r,non\text{-}in}\right)\right)-\max \left(p\left({\mathbf{T}}_{s,non\text{-}in}\left(\theta \right)\right)\right)]}^2\}\hfill \end{array}$$

(19)

where $p(.)$ stands for the probability density function. ${\mathbf{T}}_r$ is the temperature in reference image, ${\mathbf{T}}_s$ is the temperature in source image and $\theta$ is the pyrometry model parameter. ${\mathbf{T}}_s$ is a function of $\theta$. Subscripts $in$ and non-in are denoted as innermost zone and non-innermost zone respectively. The objective is to minimize the difference between the two characteristic temperatures measured from the source image and the reference image. Substituting the pyrometry model (Equation (9)) into Equation (19):

$$\begin{array}{cc}\hfill {\theta }^{*}=& \underset{\theta }{\mathrm{argmin}}\{[\max \left(p\left({\mathbf{T}}_{r,in}\right)\right)-\max (p(\frac{1}{\theta {\mathbf{R}}_{in}})){]}^2\hfill \\ & +[\max (p({\mathbf{T}}_{r,non\text{-}in}))-\max (p(\frac{1}{\theta {\mathbf{R}}_{non\text{-}in}})){]}^2\}.\hfill \end{array}$$

(20)

However, there is no prior information to know what distribution ${\mathbf{T}}_s$ and ${\mathbf{T}}_r$ follow. In order to compute $p\left(\mathbf{T}\right)$, one solution is to estimate the probability density function (PDF) directly. In this work, the Kernel Density Estimation (KDE) approach [29] is used to estimate the PDF $p(.)$. The KDE approach is suitable for estimating the PDF for univariate random processes [30]. Because ${\mathbf{T}}_s$ and ${\mathbf{T}}_r$ are univariate, the KDE approach is particularly appropriate to estimate $p\left(\mathbf{T}\right)$.

Assume $\mathbf{T}$ is a random variable, its probability density function is denoted by $p\left(\mathbf{T}\right)$, so the estimation of the probability density function $\widehat{p}\left(\mathbf{T}\right)$ at the point $\mathbf{T}$ through kernel function $K(.)$ is as follows:

$$\widehat{p}\left(\mathbf{T}\right)={\displaystyle \frac{1}{Mh}}\sum _{k=1}^{M}K(\mathbf{T}-T_k)$$

(21)

where $T_k,k=1,2,\dots ,M$ stands for the k-th sample of $\mathbf{T}$, M denotes the number of samples, and h is the bandwidth. The selection of bandwidth h controls the smoothness of the resulting probability density curve. There are several ways to determine the bandwidth h. Equation (22) could be used to generate a reasonable estimation of the optimal bandwidth while minimizing the approximation of the mean integrated square error [30,31].

$$h_{opt}=1.06\sigma M^{-1/5}$$

(22)

where $\sigma$ denotes the standard deviation of $\mathbf{T}$.

5. Experiment and Validations

5.1. Offline Preparation—Convert a Digital Camera into a Pyrometer in Laboratory

A pyrometry-calibrated camera should be ready prior to the online calibration tests. The Cannon Mark III is used as the reference temperature camera in this paper. It is critical to prevent underexposure or saturation through setting proper camera parameters while doing two-colour pyrometry on flame images. The camera parameters will also affect the accuracy of temperature measurements as described in Section 3.1. Besides, due to the wide range of temperature distribution, it is necessary to use the revised radio pyrometry model (Equation (13)).

Table 1. Comparison of the Calculated Temperature with the Blackbody Furnace.

ISO	Aperture	MSE	MSE
	Time (s)	(Linear)	(Quadratic)
200	1/1000	20.89	8.41
200	1/800	78.95	5.61
200	1/600	87.52	13.85
250	1/1000	81.11	8.43
250	1/800	91.10	17.39
250	1/600	44.71	18.95
320	1/1000	95.33	16.55
320	1/800	44.62	20.89
320	1/600	44.65	11.60
500	1/1000	45.64	12.88
500	1/800	35.52	27.48
500	1/600	41.87	17.55
800	1/1000	42.54	19.23
800	1/800	40.51	24.39
800	1/600	32.92	33.82

shows the temperature measurement accuracy of different camera parameters and models results.

It can be seen that the revised model (Quadric) is better than the ordinary one (Linear). Higher ISO would cause larger errors. In general, the smaller aperture time is, the better accuracy can be achieved. However, too small ISO and small aperture time might make that the photoelectric response is not obvious, which is called underexposure and would also reduce accuracy. Thus, the ISO and aperture time should be considered together to prevent underexposure or saturation.

In this paper, the camera is configured to record images at 1/800 s with an ISO of 200 and a maximum aperture of f/2.0. The pyrometry model is shown in Equation (13) with the parameters $r=I_R/I_G$, $[{\overline{b}}_r,{\overline{a}}_{r,1},{\overline{a}}_{r,2}]=[7.14\times {10}^{-4},-4.44\times {10}^{-4},9.70\times {10}^{-4}]$.

Table 2. Comparison of the Calculated Temperature with the Blackbody Furnace Using the calibrated pyrometer.

Blackbody Temp. (K)	Temp. Calculate (K)	Errors (%)
1123.15	1141.67	1.62
1173.15	1164.65	0.70
1223.15	1222.64	0.04
1273.15	1275.54	0.19
1323.15	1328.40	0.40
1373.15	1375.05	0.14
1423.15	1415.90	0.51
1473.15	1473.70	0.04

gives a comparison between the setting values of blackbody furnace. The maximum error for the temperature is 1.62%. This result has been approved by the local official instrument-testing organization.

5.2. Online Calibration Experiment—A Real Industrial Experiment of MSWI Flame Temperature Measurement

Experiments were conducted on a real industrial MSW incinerator. This incinerator can process more than 600 tons of MSW per day. As shown in Figure 8, there are two video cameras installed in the MSW incinerator. The frame they take does not overlap, and there are three viewing windows in the same height with the cameras.

The neighbouring camera and the viewing window are close enough to ensure that a common area exists between the reference image captured at the viewing window and the source image captured by the video camera. Thus, this scenario well represents the scene in Figure 3.

The pyrometry-calibrated camera introduced in the last subsection is used as the reference camera, and the left video camera in Figure 8 is used as the source camera. The image taken by the pyrometry-calibrated camera is considered as reference image. The video frame is the source image. Figure 9 shows the source image and the reference image in this case. Although, they are taken at the close position and time, the two images look so different. However, it is easy to see that the grey level of the flame closest to the waste is larger than that of other areas. Thus, the innermost flame segmentation algorithm proposed before still works.

The innermost zones of flame in the two pictures are extracted using the algorithm proposed in Section 4.2, as shown in Figure 10. The dark red part of the images is the innermost zone of flame. And the flame temperature in the reference image is estimated using Equation (13) with the parameters chosen in Section 5.1. Next, Equation (21) is employed to estimate the distributions of flame temperature of innermost zone and non-innermost zone. The $\max \left(p\left({\mathbf{T}}_{r,in}\right)\right)$ is 1400.0 K and $\max \left(p\left({\mathbf{T}}_{r,non\text{-}in}\right)\right)$ is 1506.3 K in this case. After computing the ratio $r=I_R/I_G$ of innermost zone and non-innermost zone in source image, interior point method [32] is used to solve Equation (20), and therefore the calibration is completed.

The temperature distribution of the source image and reference image is shown in Figure 11.

It can be seen that the flame temperature range in the source image and reference image is close, and it can be considered that the calibration is completed. After the calibration, the temperature distributions measured by video camera maintain two characteristic temperatures. Figure 12 shows the two characteristic temperature variations over time during the image acquisition period. It can be seen that the characteristic temperatures are remained unchanged. This also shows that the previous assumption is true. In summary, due to the temperature distributions and the two characteristic temperatures measured by reference and source camera are similar in the image acquisition period, it could be considered that the online calibration is successful. The temperature measured by this pyrometry-calibrated video camera can be used for the monitoring of combustion status in the waste incinerator. The pyrometry calibrated video camera is in operation since it is deployed in a real MSW incinerator.

6. Conclusions

Flame temperature and its distribution are important for combustion monitoring and control. Although a lot of technologies have been developed to measure the flame temperature distribution, due to the high cost and lack of robustness of instruments, there are still few plants using the advance instruments. On the other hand, the digital camera has been already equipped in almost every plant. For now, it is only used for observing flame morphology and structure. A lot of researches have reported that the digital cameras can be used to measure temperature distribution after being calibrated properly. The traditional calibration methods need to be carried out in the laboratory, so it requires professional laboratory personnel and re-installation of the camera, which is infeasible for practice.

In this paper, a convenient, low-cost, online pyrometer calibration method is proposed. This method takes advantage of the characteristics of the flame temperature distribution and make online calibration possible. The novelty includes:

1.

Only taking one photo by a calibrated pyrometer at a close position to the video camera being used, the calibration can be accomplished.

2.

It significantly reduces the cost and difficulty of pyrometer calibration.

3.

This method can enhance the ability of combustion state diagnosis in a very convenient and cheap way.

Theoretically, the upper limit of the measurement accuracy of this method is the same as the accuracy of the reference pyrometer used. Although the actual measurement accuracy is not strictly verified, in this paper, it is verified that both characteristic temperatures measured with an online calibrated camera over a period of time are close to those measured by a pre-calibrated camera. From this result, the measurement accuracy is sufficient in the combustion monitoring.

Generally speaking, the calibration framework proposed in this paper may also be used on other types of sensors, i.e., calibrating the un-calibrated sensor with the calibrated sensor.