Study on the Image Processing Methods for a Flame Exposed to an Incense Smoke Environment

Abstract

Identification of flames to detect fires is hindered by the smoke generated from Chinese incense in traditional temples. Especially during holiday periods, smoke presents a large influence on the effectiveness of image-based flame identification. To have a deep understanding of the incense smoke impacting the flame outline, a series of tests were conducted to study the flame, varying incense smoke concentration and test time, respectively. It is found that when the flame is exposed to a thin incense smoke environment, nearly all the methods used for flame identification are effective. When the flame is surrounded by thick smoke, the flame image after treating by the self-adaptive image histogram equalization method is blurry. When the retinex algorithm is used for image treatment, the blue color near the flame is detected, which enlarges the flame area detection. The retinex algorithm can be used to obtain a clear flame outline even when the flame is exposed to a cloud of thick smoke. This is important for flame identification in the traditional Chinese temples where the thick smoke surrounds them, especially during national holiday periods. This work attempts to provide a potential method for flame identification and improve the safety level of historic buildings.

1. Introduction

Now, several historical buildings have disappeared because of serious fires [1,2]. For example, the wooden Shuri Castle which is a 500-year-old world heritage site in Japan was destroyed by a fire in 2019. The Notre-Dame cathedral in Paris was damaged by a fire in 2019. In addition, a wooden bridge named Wanan, which lasted over 900 years, was destroyed by a sudden fire in the Fujian province of China in the August of 2022. On the one hand, advanced technology is highly developed to prevent buildings from big fires. However, the cultural heritage sites in the world are dangerous due to fires, especially in the area of Asia [3,4]. Its traditional cultural heritage is characterized by wooden structures. As wooden structures are weathered for long periods, their reaction-to-fire performance becomes serious [5,6]. It is known to all that initial fire inhibition is a key step to stopping a fire disaster. Fire warning through real-time images has been widely used in the field of firefighting, and fire detection models do not require particularly clear flame profiles to determine the presence of fire. However, its application in Chinese historical buildings needs to be reconsidered. Because of the continuous burning of incense, incense smoke accumulates more and more densely, resulting in a rapid decrease in flame visibility. At the same time, because the decorative lighting of temples in ancient architecture is dim, the interior decoration is dominated by red and yellow tones, which has a great impact on the recognition of flame contours. If the flame is rashly determined when visibility is low, it is likely to cause misjudgment and waste rescue efforts. The influence of acoustic energy on the suppression of soot from an acetylene diffusion flame was reported recently [7], and the fundamental kinetic mechanisms and emission mitigation in ammonia combustion is presented [8]. Different from the above fuels, the multiple elements and analysis of heavy metals have been found by online in situ detection [9,10]. It is a fact that thick incense smoke usually surrounds historical buildings because the thick smoke is believed to be lucky for the future for Chinese people [11], as shown in Figure 1. From the aspect of fire early warning, the effect of smoke concentration on the image was reported to be significant and large [12]. The understanding of smoke concentration affecting flame outline could provide insight into the development of a fast flame identification technology concerning wooden historic buildings.

There have been several attempts in the literature to develop approaches that address the smoke concentration effects on the flame image. It is reported that a haze removal approach based on external and internal clues has been developed for sky/river-alike scenes by an optimization solution [13]. A deep-learning-based architecture (denoted by MSRL-DehazeNet) for single-image haze removal relying on multi-scale residual learning (MSRL) and image decomposition is now available [12]. In addition, several other methods have been used for smoke or haze removal, such as deep fusion network [14], patch-map-based hybrid learning DehazeNet [15], non-local dehazing network [16], an end-to-end network [17], improved color attenuation [18], multi-scale residual convolutional neural network [19], and so on. Although the theory and calculation method presented are available, the comparison of each method regarding the same condition is absent. In addition, the code used for each image processing method is always limited, and during the image-based fire early warning process, the high reliability mainly depends on the image processing methods and the surrounding environment. The smoke concentration impacting the flame image is inferred to be related to many factors [20,21], including distance [22], flame size [23], smoke concentration [24], and so on. However, in the small chamber, fixed smoke concentrations and visibility are difficult to control quantitatively. Secondly, there is also no universal and effective indicator for the measurement of smoke concentration density, which indirectly leads to the lack of a corresponding unified indicator measure for the representation of image processing results. Because the factors affecting the shape of the flame in the smoke environment are very complex, it is impossible to carry out exactly the same repeated experiments, and a large number of experiments are needed to obtain a unified measure about the image processing, which is well worth the subsequent in-depth study. In general, the main emphasis of available research is focused. However, a comprehensive understanding of smoke concentration and image processing methods’ influence on the flame outline had not yet been achieved until now.

In the current contribution, to have a deep understanding of the image processing methods’ influence on the flame outline, the theory of several typical image processing methods is presented, and then a series of tests are conducted by differing the flame size and varying the smoke concentration. The discussion of the corresponding results on the profile of flame size follows this. This study hopes to provide a potential understanding of the effects of smoke concentration on flame identification.

2. Materials and Methods

2.1. Description of Image Treatment Methods

2.1.1. Introduction of Haze Removal Using Dark Channel Prior

The model of haze image formation used in the field of computer vision and computer graphics is usually described as follows [25,26,27,28]:

$$I\left(x\right)=J\left(x\right)t\left(x\right)+A\left(1-t\left(x\right)\right)$$

(1)

where $J$, $A,$ and $t$ are the observed intensity, the scene radiance, the global atmospheric light, and the medium transmission describing the portion of the light, respectively. Observation of haze-free outdoor images is the basis of dark channel prior. For image J, it is defined as the following equation [29]:

$$J^{dark}\left(x\right)=\underset{c\in \left\{r,g,b\right\}}{\min }\left(\underset{y\in \Omega \left(x\right)}{\min }\left(J^c\left(y\right)\right)\right)$$

(2)

where $J^c$, $\Omega \left(x\right)$, and $c\in \left\{r,g,b\right\}$ represent the color channel of $J$, the local patch centered at $x$, and the color channel index, respectively. The $J^{dark}$ is called the dark channel of J. The dark channel prior is defined as the above statistical observation or knowledge. The final scene radiance $J\left(x\right)$ is recovered by Equation (3) [29]:

$$J\left(x\right)=\frac{I\left(x\right)-A}{max\left(t\left(x\right),t_0\right)}+A$$

(3)

A typical value of $t_0$ is set as 0.1.

2.1.2. Introduction of Variational Image Dehazing Using a Fuzzy Membership Function

A depth-like map is composed of the degree of membership, which is obtained using the fuzzy membership function. The transmission map can be expressed as the following equation [30]:

$$T=1-\omega ·\frac{{\mu }_{g^R\times mean{A}^R+}{\mu }_{g^G\times mean{A}^G+}{\mu }_{g^B\times mean{A}^B}}{mean{A}^R+mean{A}^G+mean{A}^B}$$

(4)

where $g\left(x\right)$ is the pixels of the hazy image, and $g$ is the hazy image (universal set); ${\mu }_A\left(x\right)$ is the membership value for pixels of the hazy image for A. The membership value of the hazy image pixels for A is defined by the trapezoid-type membership function where $T$ is a value that is between 0 and 1. The $mean{A}^c$ is the average value of $min{A}^c$ and $max{A}^c$. In addition, a constant parameter to control the amount of fog removal is represented by $\omega$.

2.1.3. Introduction of Real-Time Polarimetric Dehazing

The dehazed image developed by Schechner et al. is described as follows [31]:

$$\tilde{r}=\frac{\tilde{ı}-\tilde{\alpha }}{e^{-\tilde{\beta }\tilde{z}}}$$

(5)

where $\tilde{ı}$, $\tilde{\alpha }=\left({\tilde{\iota }}^{\perp }-{\tilde{\iota }}^{\parallel }\right)∕\tilde{p}$, $\tilde{p}$, $\beta$, and $\tilde{z}$ represent the hazed intensity image, the air light, the degree of polarization of the air light, the scattering coefficient, and the distance to the object, respectively. The image is attenuated by the transmittance $t\left(\tilde{z}\right)=e^{-\tilde{\beta }\tilde{z}}$ which is the ratio of the light arriving at the sensor from the object to the light emanating from the object. By using some optimized mathematics, a transmittance is obtained [31]:

$$t\left(\tilde{z}\right)=1-\frac{\tilde{\alpha }}{{\tilde{\alpha }}_{\infty }}$$

(6)

where ${\tilde{\alpha }}_{\mathrm{\infty }}$ is the airtight from an object at an infinite distance without the transmittance. The optimum form for the imaging polarimeter is obtained by substituting in the airlight and transmittance values into Equation (6) [31]:

$$\tilde{r}=\frac{\tilde{ı}-\left({\tilde{\iota }}^{\perp }-{\tilde{\iota }}^{\parallel }\right)∕\tilde{p}}{1-\left({\tilde{\iota }}^{\perp }-{\tilde{\iota }}^{\parallel }\right)∕\tilde{p}{\tilde{\alpha }}_{\infty }}$$

(7)

2.1.4. Introduction of Multi-Scale Retinex for Color Image Enhancement

The multi-scale retinex can be compactly written as the following [32]:

$$F_i\left(x,y\right)=\sum _{n=1}^N{W}_n·\left\{\mathit{log}\left[S_i\left(x,y\right)\right]-\mathit{log}\left[S_i\left(x,y\right)\ast M_n\left(x,y\right)\right]\right\}$$

(8)

where the subscript $i\in R,G,B$, $N$, and $W_n$ represent the three color bands, the number of scales being used, and the weighting factors for the scales, respectively. The $M_n\left(x, \mathrm{a}\mathrm{n}\mathrm{d} y\right)$ are the surround functions, which are provided by the following [32]:

$$M_n\left(x,y\right)=K_{n}exp\left[-\left(x^2+y^2\right)∕{\sigma }_n^2\right]$$

(9)

In this equation, the ${\sigma }_n$ is the standard deviation of the Gaussian distribution. The $K_n$ are selected so that $\iint F\left(x,y\right)d_x{d}_y=1$.

2.1.5. Introduction of Improved Self-Adaptive Image Histogram Equalization Algorithm

When the image is treated by this method, the traditional histogram of the image and the light adjustment are applied. The following equation is used to conduct the light adjustment [33]:

$$\begin{gathered} T_k=\frac{\left(L-1\right)-a{S}_{min}}{S_{max}-S_{min}}\left(S_k-S_{min}\right)+a{S}_{min} \\ k=1,2,\cdots ,L-1,0<a<1 \end{gathered}$$

(10)

where $T_k$, $S_{max}$, $S_{min}$, and $S_k$ represent the gray level value of the resulting image, the biggest gray level value of the pixels, the smallest gray level value of the pixels, and the gray level value of a pixel, respectively. $a$ is the brightness compensation coefficient whose value is between 0 and 1. The $k$ is the gray level.

2.1.6. Introduction of the Automatic Color Equalization Algorithm

Equation (11) provides the main calculation of this method. Each pixel $p$ of the output image R is obtained by calculating the channel $c$ using the following equation [34]:

$$R_c\left(p\right)=\frac{\sum _{j\in Subset,j\ne p}\frac{r\left(I\left(p\right)-I\left(j\right)\right)}{d\left(p,j\right)}}{\sum _{j\in Subset,j\ne p}\frac{r_{max}}{d\left(p,j\right)}}$$

(11)

In this equation, $I\left(p\right)-I\left(j\right)$ is the lateral inhibition mechanism, $d\left(p,j\right)$ is a distance function that weights the amount of local or global contribution, $r\left(·\right)$ is the function that accounts for the relative lightness appearance of the pixel. The range of values in $R_c$ on each channel is independently linearly scaled to the range [0, 255] by the following equation [34]:

$$O_c\left(p\right)=round\left[127.5+s_c{R}_c\left(p\right)\right]$$

(12)

where I is the input image, R is an intermediate result, and O is the output image. The subscript c denotes the different channels. $s_c$ is the slope of the segment $\left[\left(m_c,0\right),\left(M_c,255\right)\right]$ which are obtained by the followings [34]:

$$M_c=\underset{p}{\mathit{max}}\left[R_c\left(p\right)\right]$$

(13)

$$m_c=\underset{p}{\mathit{min}}\left[R_c\left(p\right)\right]$$

(14)

2.2. Description of the Flame Observation Test

To obtain the flame observation when it is exposed to incense smoke with a varying concentration, a series of tests were conducted. The experimental description is illustrated in Figure 2. The cube was made by an acrylic panel of the size of L × W × H = 1.0 m × 1.0 m × 0.5 m. A fixed incense was used to produce the same smoke from historic buildings. Before capturing each flame, the incense smoke box was kept for nearly 3 min until the smoke was stable inside the box. A camera (Canon Rebel T3i 1080P, Tokyo, Japan) was used to record the base position images at 30 frames per second with a spatial resolution of 0.47 mm/pixel. The hood was without ventilation, and the ambient temperature and moisture were 30–37 °C and 66–90%, respectively. The fuel to produce a flame was solid alcohol. This kind of fuel was used to keep the same test condition.

Based on previous studies by our experimental team, the research method in this paper serves as a continuation study of the previous research method [35], and the error in experimental reproducibility is within an acceptable range and does not affect the results due to different measurements from multiple experiments.

3. Results

3.1. Flame Observation Change with Varying Smoke Concentration

The fact that cigarettes have a huge influence on flame identification is a fact that cannot be ignored. In the traditional Chinese concept, it is believed that the heavy incense smoke of traditional temples signifies good luck. However, because traditional Chinese temples are usually built of wood, the fire hazard is exacerbated by the large number of people who converge in the temples during traditional holidays to pray for peace. Under this condition, if the fire of a wooden structure is not identified timely in the early stage, it could result in a serious disaster [36]. The fire shows a big threat to the safety of visitors and cultural heritage [37]. As shown in Figure 3, in our calibration test, the flame becomes blurry when the smoke concentration increases. Finally, the flame could not be identified due to the high concentration of incense smoke. During the test, incense smoke concentration is linear with the accumulation time because of its steady combustion [38,39,40]. The flame shape is captured by a high-definition camera. Compared with the original figure of flame at t = 0 min, the flame shape becomes blurry from t = 0 min to t = 15 min. When the test time approaches at t = 24 min, most of the flame is invisible. After 6 min, the flame nearly disappears from the window. Therefore, a good image processing method is necessary for flame identification in the early stage of fire. The timely flame identification could keep the historical wooden buildings from being damaged because the flame spread rate over weathered wood is fast [41,42,43]. In the small chamber, the fixed smoke concentration and visibility are difficult to be controlled quantitatively [44]. Just as shown in Figure 3l, the RGB value increases from the beginning and reaches the peak at t = 3 min, where R = 949, B = 818, and G = 752. As the smoke concentration increases, the RGB value decreases and approaches the minimum value at t = 27 min, where R = 583, B = 565, and G = 549. This means that the image becomes blurred. From this point of view, the smoke has a significant impact on the visibility of the flame.

3.2. The Smoke Processing Methods’ Influence on Flame Identification

As discussed above, several figure treatment methods have been used for image dehazing and deraining [45]. This is because the smoke concentration is inferred to have a big effect on the flame outline [46], just as the two different conditions are shown in Figure 3 and Figure 4. Figure 3 shows the flame observation change when it is exposed to a low concentration of incense smoke. Figure 4 describes the flame change when it is exposed to a high concentration of smoke. From Figure 3, it is observed that after processing, the flame sharpness is increased since it becomes clear. The results of the seven treatments show that for low concentrations of smoke, the smoke treatment method has little effect on the flame profile. It is found that all the methods could be used for image processing when the smoke is not thick. The summary of RGB peak values is described in Figure 4h. It shows that the steady figures are found in Figure 4c,d since the RGB deviation is small. However, Figure 4d is much more illustrious than Figure 4e.

As the test time increases, the smoke concentration accumulates, resulting in a blurry flame outline due to the heavy smoke, which is shown in Figure 5a. In it, the RGB value is 741, 734, and 741, respectively. The influence of smoke treatment methods on the flame outline is obvious, compared with the original Figure 5a. From Figure 5, it is believed that, after image processing, Figure 5b,d,e could be used for flame identification. This means three methods are suitable for flame outline description when it is exposed to a heavy incense smoke condition. Regarding Figure 5f,g, the color of the surrounding environment is approaching the same color of the flame, resulting in bad results of flame identification. The R value of Figure 5f has a peak R = 1155. Figure 5h indicates the summary result of all figures. Therefore, the smoke concentration impacts flame identification. It is necessary to discuss the relationship between smoke concentration, flame shape, and the figure treatment method.

4. Discussion

It is a fact that image-based flame identification has been widely used in the building fire alarm system [47]. Once the initial fire is detected, the suppression action can be activated to protect the surrounding persons and goods from being damaged. Typically, fire alarms require a high level of reliability in flame detection. Therefore, timely and accurate capture of flames is a key technology for fire protection in historic buildings.

When the flame is exposed to a thin incense smoke, such as test No. 3 to test No. 9, it is found that nearly all six image processing methods could be used for the flame image processing. As the concentration of smoke increases, the flame becomes blurry. When the flame is exposed to a thick smoke, although some methods are effective for flame detection, the flame outline is still not easy to timely produce.

Regarding the treatment method using dark channels prior, it is useful to dehaze smoke when the smoke is not high. After processing the figure by it, the flame is clear. As the smoke changes from thin to thick, from test No. 6 to test No. 24, most cases are good enough. Regarding test No. 27, the flame is hard to observe because of thick smoke. It shows that the dark channel prior method finds it hard to reproduce the flame outline. In a real case, the smoke concentration changes correspondingly. Therefore, this method is not well suited to thick smoke cases.

When the fuzzy membership function method and real-time polarimetric method are used for image treatment, both of the two methods were found to be effective in reproducing a clear flame with respect to a thin smoke environment. However, as the smoke concentration increases from a low to a high value, the sharpness of the image processed by the real-time polarimetric method is believed to be clearer than the one treated by the fuzzy method. Neither method can be used for images where it is difficult to observe the flame. In general, the real-time polarization method is appropriate.

The retinex algorithm method for enhancing a single image has been focused on by many researchers under different conditions [48]. Currently, the retinex algorithm for enhancing the flame image is evaluated by a series of image capturing. In terms of flame area or flame profile, it shows the best results among all the treatments, as it is valid under all test conditions. The blue color near the flame is found when this method is used for the case with thin smoke. It may have a small impact on flame area detection. As the incense smoke increases, the blue color reduces little by little. Especially, when the smoke is thick, the traditional methods are found to be unsuitable for processing images. In this condition, only the retinex algorithm method could reproduce a clear flame, which is important for the fire alarm system.

In addition to the above methods, two equalization algorithms method are used in this work. One is the self-adaptive image histogram equalization algorithm method and the other one is the automatic color equalization algorithm method. It is found that the automatic color equation method could reproduce a clearer flame than the one treated by a self-adaptive method. When the smoke concentration increases to a very high level, the outline of the flame becomes clear. In contrast to the automatic color equation method, when the flame image is processed with the adaptive image histogram equalization method, the entire image becomes blurred. In this case, the outline of the flame is not easily detected. In conclusion, flame detection depends, to a large extent, on the image processing method.

5. Conclusions

To find suitable image processing methods for flame detection in traditional Chinese temples, several typical methods which are used in other fields are evaluated and detailed. A series of tests are also conducted to support the results. During the experiment, flame images resulting from changes in cigarette concentration were obtained by the camera. The relationship between the smoke concentration and the observed flame is also discussed in detail and illustrated. The following are obtained:

(1)

The sharpness of flame image varies with the incense smoke concentration. Dense smoke has a significant impact on the flame observation of image-based fire detection systems. In traditional historic buildings, the flame image treatment should be well considered because of the unique Chinese culture involving the combustion of incense. Without using the image processing method, the flame is not easily detected accurately and timely. The suitable image processing methods should be selected and compared regarding the fixed conditions.

(2)

When the flame is exposed to a thin incense smoke, nearly all the methods are effective for flame identification. Although it is found that the flame image treated by the self-adaptive image histogram equalization method makes the entire image blurry, it still works well under the usual condition. When the retinex algorithm method is used for image treatment, the blue color appearing around the flame may have a small impact on the flame area detection.

(3)

During traditional Chinese holidays, flame recognition of ancient buildings such as historical temples is the focus of research, because they are surrounded by thick smoke, which will cause huge casualties and property damage in case of fire. When the flame is surrounded by thick smoke, using retinex algorithm to process the image can obtain a clear flame outline, which is a good solution to this problem.