On the Potential of Flaming Hotspot Detection at Night via Multiband Visible/Near-Infrared Imaging

Abstract

The severity of wildfires is increasing and has driven increases in nighttime fire activity. Enhanced capability to detect the active burning regions of wildfires at night could significantly improve the effectiveness of wildfire management operations. Potassium line emission in the NIR near 770 nm is a signature of active burning. We test the use of multi-band imaging from an aircraft at night to distinguish a wood-burning fire from artificial light sources. We find that a simple ratio of the signals in two broad bands, one including 770 nm, effectively discriminates the fire from artificial light sources. This offers the possibility of nighttime fire detection with high spatial resolution using silicon sensors sensitive in the NIR.

1. Introduction

Wildfires are increasing in severity and socioeconomic impact. In 2020, more than 50,000 wildfires were recorded within the United States that burned a total of more than 10 million acres [1]. Nighttime fire activity is increasing even more rapidly than daytime activity, as a result of hotter and drier nights [2]. In 2003–2020, nighttime fire activity increased by 40–50% [2]. This subsequently decreases the usual nighttime barriers to fire propagation and promotes fire spread leading to larger fires. Large fires have the strongest nighttime activity [3]. The increase in nighttime fire activity has impacted wildfire suppression. Nighttime mitigation operations are increasingly common, enabled by the availability of night flying aircraft [4]. Innovations in capabilities to detect wildfires at night, particularly the most actively burning regions, could significantly improve the effectiveness of wildfire management operations. This could have a major impact on the lives of people living in fire-prone areas.

At typical wildfire temperatures, the fire radiative power (FRP) is dominated by emission in the mid-wave infrared (IR) band (MWIR) of 3.5–5 μm. This band is typically used for monitoring of fires from space [5] (for a recent review see [6]). These products have typical ground resolution from hundreds of meters (e.g., 375 m from the Visible Infrared Imaging Radiometer Suite or VIIRS [7]). Such resolutions are much larger than the typical (∼2 m) transverse dimension of fire fronts [8,9]. This hinders accurate derivation of fire properties for the active/flaming regions of a fire; the FRP from a pixel containing a small flaming region can equal that of a pixel with smoldering over a large area [10].

In contrast, radiation in the visible/NIR is robustly associated with flaming and the most active regions of wildfire. This can be understood because generation of visible/NIR photons requires more energy than IR photons. Visible/NIR light is produced when the fire combustion products (soot and complete-oxidized gases) absorb enough energy during combustion to excite the component atoms to energy levels that lead to emission of visible light [11,12]. Combined use of visible band or Day-And-Night Band (DNB covering 400–900 nm) and NIR from VIIRS can improve the detection of fires and characterize the fire phase and combustion efficiency at night [10].

With broad-band visible light alone, it can be difficult to distinguish burning from non-burning material. Vodacek et al. [13] proposed use of line emission from flaming regions for wildfire detection, specifically a doublet of lines from potassium. Figure 1 shows a spectrum of flaming wood. The flame spectrum is dominated by emission from a doublet from potassium (K) at wavelengths of 766.5 nm and 768.9 nm in the near-infrared (NIR). Potassium constitutes up to 7% of vegetation by mass, therefore is present at significant concentrations in wildfire fuel. The line is excited in active flaming regions and not in smoldering regions or by reflection of sunlight. The 768.9 nm line lies near an Oxygen (O₂) absorption feature which reduces the signal by up to 60% for high-altitude or space-based observations. The 766.5 nm line is attenuated only by the O₂ continuum and not significantly affected.

Use of the VIS-NIR is advantageous because silicon-based sensors can be used. The tremendous investment by industry in silicon detectors for VIS-NIR imaging has led to the availability of sensors with high pixel count (up to tens of megapixels) and very low readout noise (of a few electrons or even a fraction of an electron). The high pixel count enables coverage of wide fields of view with good angular resolution; this translates to coverage of wide ground swaths with good ground resolution. The low readout noise leads to high sensitivity in low-light conditions. Modern complementary metal–oxide–semiconductor (CMOS) sensors are designed to operate well at room temperature. Hence, they do not require cooling, unlike most SWIR and MWIR sensors. Recent sensor development targeted for night-time surveillance has led to improvements in the sensor quantum efficiency in the NIR band containing the potassium line.

High contrast versus background is key to fire identification. The low level of solar radiation in the MWIR band is a important factor enabling detection of fire signatures in that band [14]. Solar radiation is strong in the NIR band near the potassium line. Moreover, vegetation has high reflectance in the NIR, longward of the red edge at ∼700 nm [15] increasing the background during daylight. Previous efforts on use of the potassium line have focused on the daytime, when fires are most active [13,14,16,17]. The metrics used to identify fire are the “K emission index” which is the ratio of radiance at the potassium line (770 nm) to that in a proximal band (780 nm) [13] and the “advanced K band difference” which is a normalized difference between a narrow band containing the potassium line and nearby background bands [14]. The strong background of reflected light from vegetation near the potassium line may confound these metrics by inducing strong correlations between the radiances in the NIR bands near the potassium line [14].

Here, we propose to use observations of the potassium line to detect the flaming regions of fires at night, motivated by the recent increase in nighttime fire activity. This reduces the background due to solar radiation reflected by vegetation by a factor of more than ${10}^5$ even with a full moon. The predominant background is then artificial lighting. While wildfires do often burn in remote areas, the vast majority (84%) of wildfires in the US are started by humans [18]. These ignitions occur mainly at road networks, the wildland–urban interface, and boundaries between agricultural and forested areas. While these areas can have low population density, there is still infrastructure present, and it is important to discriminate fire from artificial lights to eliminate false positives in fire detection. Furthermore, fires at the wildland–urban interface cause the greatest loss of life and property destruction and are also the most difficult to fight [19].

Fortunately, the predominant artificial lighting sources do not produce strong emission near the potassium line, including sources that produce light via burning such as propane flames [20]. This may enable the use of relatively broad spectral bands for fire identification. Broad band two-dimensional imaging can be accomplished with much simpler instrumentation than the push-broom spectrometers used for most previous results on potassium line fire detection [13,14,16,17]. Imaging with narrow band filters has also been suggested [13,21] and used [22]. However, only interference filters achieve the very narrow bandpasses desired. Interference depends on the path length traveled by light through the filter material. The path length depends on the angle of incidence, therefore a constant bandpass can be maintained for interference filters only over a limited field of view [23]. Colored glass filters are an alternative that can produce passbands that remain constant over wide fields of view. However, the passbands are generally much wider than can be achieved with interference filters.

To achieve potassium line detections at night, we propose the use of multi-band, wide-field, two-dimensional imaging. The key question is whether relatively broad filter bands, on the order of 90 nm full width at half maximum, are effective in distinguishing fire from artificial light sources present at night. To address this question, we built a four-camera system providing quasi-simultaneous images in four filter bands and flew it on an aircraft over an urban environment including a small, wood-burning fire. In the following, we describe the camera system with an emphasis on the filter selection, acquisition and processing of the images from the flight, and the effectiveness of the system for fire detection. We conclude that the system effective in discriminating fire from artificial light sources. We discuss potential avenues for improvements and applications of multi-band imaging for fire detection at night.

2. Methods

2.1. Camera System and Filter Selection

We built a camera system containing four cameras with different filters selected to enable discrimination between the light generated by fire versus by artificial sources. We selected short focal-length lenses providing wide fields of view to maximize the ground area coverage. DMK 33UX290 Cameras from The Imaging Source, LLC, were used. These cameras employ the Sony IMX290LLR monochrome sensor which is a back-illuminated CMOS sensor sensitive to visible and NIR light. The pixel technology offers high light collection efficiency and low noise making the sensor sensitive in low light environments. Each IMX290 contains a 1920 × 1080 array of 2.9 μm square pixels. The cameras use a USB interface. We used the Imaging Source software development kit and wrote software to run the four cameras. The image acquisitions were not simultaneous, with a delay of up to 4 s between capture of the first and last images in a set of four. This will be improved in a future iteration of the instrument. Tamron 13FM04IR lenses were used. The 13FM04IR provides good light collection with an f-number of 1.2 and good transmission in the visible and NIR. The lenses have a focal length of 4 mm which when used with the IMX290 sensor provides a field of view of ${70}^{\circ }\times {43}^{\circ }$.

It is important that the camera system filters provide constant bandpass across the full field of view. We used StablEDGE^®filters from Midwest Optical Systems, Inc (Palatine, IL, USA), that incorporate absorptive filter glass to provide a stable bandpass even for large incidence angles. This minimizes the wavelength shift across the camera field of view. The filter bands were chosen to enable discrimination between the spectra of fire and artificial light sources as shown in Figure 1. The band passes of our selected filters are shown in Figure 2. The MidOpt BP735 filter was chosen to provide high transmission at the potassium lines while minimizing the transmission at the 809 nm sodium line produced by high-pressure sodium lamps that are used extensively in street and other outdoor lighting. The BP635, BP540, and BP470 filters were selected to capture the visible-band emission of fluorescent, high-pressure sodium, and LED lamps. The BP540 and BP635 are paired with SP700 filters to block non-zero response of those filters in the NIR. Hereafter, we refer to those as the BP540 and BP635 images for brevity.

2.2. Image Acquisition and Processing

The camera system was mounted in a Bonanza aircraft. The cameras view the ground through a 5-inch diameter hole in the passenger section of the aircraft. The aircraft was flown by Schnell and the camera system operated by Tammes. The aircraft was flown over Iowa City, Iowa, USA, which provided a variety of artificial lights. A wood-burning fire was set in a 0.9 m diameter fire pit as shown in Figure 1. The wood was mainly Ash. The fire pit was located in a residential neighborhood (“Lucas Farms”) within Iowa City.

The data analysed here consist of one set of four images obtained while the aircraft was close to directly overhead of the fire pit. An exposure time of 20 milliseconds was used for all four cameras to limit blurring due to aircraft motion. Images were taken in 12-bit mode and all image pixel values were well below saturation. The aircraft was an altitude of 776 m in nearly level flight at a speed of 54.5 m s⁻¹. The pixel size on the ground was 0.56 m and the aircraft motion during each exposure was 1.09 m.

The camera were rotated at different multiples of 90° within the assembly to allow for cable access and there was off-axis vignetting, particularly for the two cameras with two filters. This reduced the useful extent of the overlapping regions. We analyzed a field of view of 900 × 600 pixels where all cameras had good response. To align the images, we manually selected and estimated the centroids of compact, stationary light sources common to the BP635 image and at least one other image. We then used the ProjectiveTransform and warp routines in the Sci-Kit image transform module to perform a projective transformation on the image coordinates to align the other images to the BP635 image while maintaining the original range of pixel values. This corrected for the lack of simultaneity in image capture and differences in the camera pointing direction, rotation, and lens focal length. Figure 3 shows a false color image created using the BP735, BP635, and BP540 images. The image alignment appears to be accurate.

2.3. Geolocation and Ground Validation

After alignment, we geolocated the combined image. To do this, we identified street lights in the images that are marked with yellow crosses in panel (b) of Figure 3. We measured the GPS coordinates of the street lamps to use as georeferencing fixed points [24]. We then overlaid the night image with a daytime image as shown in the figure. The accuracy of the alignment is demonstrated by the good registration between features visible in both the night and day images. The accuracy is limited by the size of the light pools under the street lights.

Geolocation enabled ground validation based on a subsequent in-person survey of the neighborhood covered in the images. The position of the fire marked with a red circle in Figure 3 matches the measured coordinates of the fire. The other light sources were identified as artificial light sources of various types used in residential settings. We note that there are a few cases where the blue/green/red components appear to be shifted. These occur along the streets near the top and bottom of the image. There are no stationary sources at these locations; they are likely moving automobiles with the spatial shifts arising because the different images were not obtained simultaneously.

3. Results

We manually selected a variety of different types of light sources as marked on Figure 3a with dotted circles. For the compact light sources, we defined circular regions that fully capture the source. For extended sources, mainly the upwards reflection of street lights, we used circles that capture the brightest part. The regions have radii of at least 5.4 pixels.

We extracted the pixel values within each region for each filter image. The signal for each source for each filter was calculated by finding the average pixel value and then subtracting the background level. We used nine background regions, each with a radius of 18.3 pixels, placed in dark areas spread across the image to define the background level for each filter. The variations in the background levels have a standard deviation of 1.2% of the median signal level for the BP540 image and less than 0.6% for the other filters.

To analyze the spectral variations between the sources, we use ratios of the signal in different filter bands. Specifically, we use the ratio (BP735/BP635) of the signal in the BP735 image to that in the BP635 image and the ratio (BP470/BP540) of signal in the BP470 image to that in the BP540 image. We calculate errors on the signal levels and hence the ratios from the signal Poisson statistics and including the uncertainty in the background level.

Figure 4 panel a shows the distribution of band ratios for the various light sources. The wood-burning fire is marked with a red circle. The fire has the highest BP735/BP635 ratio which lies significantly above all of the other light sources. This is due to the strong emission at the potassium line.

The light sources appear to form into groups in the band ratio diagram. The symbol colors in Figure 4 indicate our preliminary manually assigned groupings. To enable some initial classification of the light sources, the circles used to mark the sources in the RGB image in Figure 3 use the same colors. Some light sources are not included in groups; these are black in Figure 4 and with white circles in Figure 3. The street lights present all fall into the cyan group. They are LED street lights, in the ATB0 series from American Electric Lighting. One street light (at 229, 865) lies out of the grouping and appears to contaminated by lights from a nearby automobile.

For comparison, we calculated band ratios for artificial light sources using the spectra in [20] and including both the filter transmission and sensor quantum efficiency. These are shown in panel b of Figure 4. The fire is well separated from all of the artificial light sources. The BP735 has 8% transmission at 809 nm, but this appears to be low enough to adequately suppress the sodium line produced by high-pressure sodium lamps based on the clear separation in panel b of Figure 4. The value of the BP735/BP635 ratio for the fire is very similar to that in the flight data.

The artificial light sources are colored using the same scheme as for the flight data. The orange cluster contains incandescent, quart halogen, mercury vapor, liquid oil, and pressurized gas lamps. The cyan cluster contains fluorescent, high-pressure sodium, white LED, and metal halide lamps. One fluorescent lamp spectrum lies above the cyan cluster. Low-pressure sodium is the lime-colored hexagon. The BP470/BP540 ratio for many of the sources, including the fire, is lower in the flight data. There are also small shifts in the BP735/BP635 ratios. The spectra are measured using direct light, while the aircraft camera sees mainly reflected light. This may account for the shifts in the band ratios between the two plots.

4. Discussion

We conclude that multiband VIS/NIR imaging can be effective for the detection of fire in the flaming phase. The fire spectrum is sufficiently different from that of artificial light sources that it can be robustly identified even with the very crude spectral information provided by a few relatively broad bands. Indeed, use of only two bands, the BP735 and BP635 appears to provide good discrimination.

The low mass, volume, and power required for the VIS/NIR multi-band imaging system makes it feasible to fly on an unmanned aerial system (UAS). One potential application would be to employ a VIS/NIR imager on a High-Altitude Long-Endurance (HALE) UAS to provide information on active burning in wildfires to assist in fire management. For example, the Swift HALE can fly 5 kg payloads at an altitude of 20 km for 30 days or more at a speed of 25 m s⁻¹ [25]. The high altitude, in the “upper Class E” airspace, avoids interference with fire suppression aircraft enabling continuous monitoring. The VIS/NIR imager can be accommodated on a HALE-UAS due to its low mass (2 kg), compact size (5000 cm³), and low power consumption (22 W). The HALE-UAS payload constraints would preclude flight of an IR imaging system with significantly larger mass, volume, and power requirements.

The VIS/NIR imager also offers significantly higher pixel counts than typical in IR imaging systems. The ground pixel size should be matched to the fire depth, the dimension of the flaming regions transverse to the fire line, which is typically about 2–3 m [8,9]. We adopt a ground resolution of 2 m. Use of a high-pixel-count sensor would enable coverage of a wide ground swath. E.g., the Sony IMX264 has 5.1 megapixels arranged as 2464 × 2056 that would allow coverage of a 4.9 km wide swath with 2 m pixels. At a speed of 25 m/s, an area of 440 km² could be covered in one hour of flight. Hence, even large fires, such as the 2022 Hermit’s Peak fire in New Mexico, USA, could be covered in a single night. Smaller fires or selected areas of larger fires could be covered in the one or two hours before dawn, providing up-to-date maps of the flaming regions.

To be useful in guiding fire suppression activities, the VIS/NIR imager data must be accessed and processed in near real time. This would require real-time telemetry from the UAS. Combined with an on-board GPS and attitude and heading reference system, it would then be possible to provide georeferenced maps of active burning fronts. This would enable activities such as hot spotting to suppress the most active regions of the fire and could help maintain the safety of fire camps during the night.

Further study is required on the effect of clouds and smoke on detection of the potassium line from fires. Clouds can prevent fire detection in nearly all VIS/IR bands. The first step in MWIR fire detection algorithms is usually removal of pixels marked as cloudy. Addition of visible light to IR fire detection may actually offer some advantage as fire light can transmit through thin clouds. Fire can be detected in thin cloud conditions at night by combining the VIIRS DNB and IR data [26]. Smoke particles can more strongly attenuate radiation at 770 nm than at 4 μm, which needs to be further studied in the context of fire detection. However, our experience with VIIRS DNB at night has clearly shown that the smoke impact on fire light transmission is minimal for detecting flaming fires [10]. The reason is that the flaming fires at the fire front typically emit far fewer smoke particles than the smoldering fires behind the fire front. Flaming often occurs above the surface, on the top of the canopy, where oxygen (O₂) supply is more sufficient making it easier for flaming to occur. As a result, flaming often occurs above the surface smoke layer. Once covered by smoke, flaming is quickly suppressed and changes to smoldering. Therefore, while smoke particles may affect the light radiative transfer, the impact is not as great as one would think if the fire light were under the thick smoke plume; in such conditions, the supply of O₂ is reduced and smoldering occurs. We also note that [17] presents a clear detection of the potassium line from a completely smoke-covered pixel.

Beyond our main focus on fire light detection, we also wish to use such instruments to explore the possible use of low light optical/NIR remote sensing for environmental studies. Potential applications including mapping the growth of urban areas [27], monitoring the impact of natural disasters and conflicts on human activity, and measuring particulate matter air quality [28]. Classification of different types of artificial lights is key to such studies. Wang et al. [28] showed the modern artificial light spectrum has avoided the water vapor absorption lines (for the sake of energy efficiency), and hence, measuring light intensity over cities can be used to study surface particulate matter air quality at night, as long as the fire pixel detection and classification of bulb types can be made, which shows promise in this concept paper. In the future, we plan to further develop our methodology for light source classification. We will obtain ground spectra of selected light sources and identify their type for comparison with the band ratios measured in flight. We will test more sophisticated classification techniques such as discriminant analysis [20].