Underwater Spectral Imaging System Based on Liquid Crystal Tunable Filter

Abstract

In the past decade, underwater spectral imaging (USI) has shown great potential in underwater exploration for its high spectral and spatial resolution. This proposal presents a stare-type USI system combined with the liquid crystal tunable filter (LCTF) spectral splitting device. Considering the working features of LCTF and the theoretical model of USI, the core structure containing “imaging lens-LCTF-imaging sensor” is designed and developed. The system is compact, and the optical geometry is constructed minimally. The spectral calibration test analysis proved that the spectral response range of the system covers a full band of 400 nm to 700 nm with the highest spectral resolution between 6.7 nm and 18.5 nm. The experiments show that the system can quickly collect high-quality spectral image data by switching between different spectral bands arbitrarily. The designed prototype provides a feasible and reliable spectral imaging solution for in situ underwater targets observation with high spectrum collecting efficiency.

1. Introduction

The fundamental of spectral imaging is to fuse multi-dimensional spatial geometric information of the image with spectral information radiating from the target surface to obtain spectral image cube data [1]. It is widely used for military, remote sensing detection [2], marine observation purposes [3], etc. In the 21st century, marine observation became an important debate and research topic [4,5], which presently relies on different techniques, such as satellite remote sensing based on spectral imaging, and underwater target detection based on acoustics and optical imaging. Although the methods mentioned above are well developed and can carry out continuous observation on a large scale, there are difficulties associated with underwater observation. It is highly vulnerable to various interferences, which causes lower detection accuracy. Underwater monitoring based on acoustic detection and optical imaging have a relatively low resolution in detecting the targeted object. Similarly, observation based on remote sensing is limited to surface or shallow water observation due to higher detection distance and interference caused by the air–water interface.

To overcome the hybrid effect of the air–water interface, the idea of USI is introduced to achieve relatively higher target detection accuracy in underwater monitoring. The ability to collect spontaneous and clear insight into diverse targets’ image and spectral information gives USI a distinct advantage in underwater detection. In the marine observation system, it can be utilized to detect corals’ health status, which can efficiently maintain the coral reefs and other organisms in the surrounding habitat [6]. It can also monitor the health and growth status of aquatic products in aquaculture, which can highly improve the quality of production. Being an advanced addition to underwater imaging, USI is also one of the most powerful technologies for surveying marine oil and gas, archeology [7], benthic habitat mapping, and seabed minerals exploration.

Spectral imaging is still a new and emerging technology for underwater imaging; however, researchers have designed some equipment for underwater surveillance systems. Since 2009, the Norwegian University of Science and Technology (NTNU), in cooperation with the Ecotone company, has started to study the applications of push-broom underwater hyperspectral imaging (UHI) based on prism splitting methods for underwater target detection [8]. Several versions of UHI with different specifications have been designed and developed, which have been installed to perform continuous underwater monitoring. In 2012, NTNU developed a fourth-generation digital UHI [9] and deployed this on a small, remotely operated vehicle to perform submarine hyperspectral imaging of cold-water corals [10]. The developed system has also been used to detect submarine pipelines and sunken ships [11]. Since then, UHI has been installed to carry out several underwater operations for algae [12], corals [13], and different seabed minerals exploration [14,15].

In 2016, Zhejiang University China developed a multispectral imaging system based on a narrowband rotating filter to switch between different spectral channels [16]. The rotatory filter system consists of 30 filters divided into two rotating wheels to reduce the system’s volume. The team performed several underwater spectral imaging and system calibration operations [17] and later researched spectral image reconstruction algorithm [18,19].

A German institute, Max Plank Institute of Marine Microbiology, in 2017 developed a handheld push-broom UHI system called the HyperDriver to monitor the shallow marine ecosystem [20]. The system was equipped with a spectral imager, color imager, auxiliary lighting, and sensors to simultaneously capture hyperspectral and color images, bottom altitude, depth, irradiance, and chemical parameters. The UHI can provide hyperspectral information with centimeter spatial resolution.

In 2018, researchers from China collaborated with Helmholtz Institute of Oceanography in Germany to develop an underwater multispectral imaging system based on a narrowband light-emitting diode (LED) light source [21]. The method utilized eight narrowband led light sources with adjustable light intensity to achieve spectral splitting, using a black and white industrial camera, imaging lens, and single-chip microcomputer to control LED to switch flashing in turns and to capture images synchronously.

Recently, in 2021, another UHI based on push-broom scan imaging mode was designed and developed by the Ocean University of China [22]. In the model, prism-grating-prism structure is adopted as the spectral unit to reduce spectral line bending. After calibration and underwater experimentation, the compact structured UHI system resulted in high image quality and spectral resolution. The system achieved high stability and good detection, realizing in large-scale target detection.

Several spectral imaging techniques are being adopted to perform a wide range of underwater detection, including starring type, snapshot, and push-broom imaging methods [23]. To maximize the performance and minimize the mechanical complexity of spectral imaging for underwater application, this paper presents the design and development of a stare-type USI system based on LCTF. The system is designed in a compact and optically simplified manner. The critical parameters of spectral imaging are calibrated accordingly for better performance in the underwater environment. The developed prototype is tested to meet the needs of high-quality spectral imaging with higher spectral resolution and high signal-to-noise ratio (SNR) to overcome the attenuation in image quality caused by the water body.

The rest of the paper is arranged as follows: Section 2 describes the design and development of the USI system, explaining its core components and specifications. Prototype testing and calibration are included in Section 3. The performance of the designed system in underwater conditions is described in Section 4. Finally, in Section 5, the proposed work is experimentally concluded, and future research directions and discussion are noted.

2. USI System Design and Development

Figure 1 presents the compact structure of the designed prototype USI system. The design of the stare-type USI system adopts the core optical formation, composed of an imaging lens-LCTF-imaging sensor. To adjust precise focusing, the large-diameter lens is controlled by an electric stepper motor. The transmission wavelength of the LCTF is controlled to achieve narrowband scanning in the visible spectral range. A scientific-grade charged coupled device (CCD) camera is used as an imaging sensor in the system to realize the imaging sensitivity of the underwater environment. The detailed specification of parameters in the optical module is presented in

Table 1. The detailed specification of parameters in the optical module of the USI system.

Component	Type	Made	Parameter	Specification
Lens	Autofocus	IS STM, Canon, Japan	Drive	Stepper motor
			Focus range	18–135 mm
			Max aperture	3.5–5.6
			Min aperture	22–38
			Length/Diameter	10 cm/7.6 cm
			Lens composition	12 groups of 16 lenses
LCTF	Lyot	Vis-10 varispec filter, CRI, USA	Spectral range	400–720 nm
			Modulation step	1 nm
			Spectral resolution	3.9–18.1 nm
			Aperture	20 mm
			Angular FOV	15°
			Max flux	500 MW/cm2
			Response time	50 ms
			Size	9 cm × 5 cm × 5 cm
Imaging sensor	CCD	Lm165M, Lumenera, Canada	Core	Monochromatic 2/3″
			Max resolution	1390 × 1040
			Pixel depth	8-bit or 12-bit
			Frame rate	15 fps
			Dynamic range	66 dB
			Power	2.5 W
			Communication	USB 2.0

.

In the control module, the microcomputer (NUC6i5SYK, Intel, Santa Clara, CA, USA) is responsible for sending control instructions, data transmission, storage, and processing. The camera and LCTF are connected with the microcomputer through USB protocol. Spectral imaging parameters and LCTF central wavelength switching can be set directly through system software. The microcomputer sends a switching command to the stepper motor drive circuit through the remote I/O module (IPAM 1808, GuanHangda, China) to accurately drive the stepper motor rotation inside the lens.

To achieve high accuracy of the USI model, the functional characteristics of each component and corresponding strategies are analyzed, and a control algorithm and software are designed on Visual Studio 2015. The algorithm controls various system functions such as; CCD camera function, adjustment of the lens by controlling stepper motor drive, and LCTF tuning for scanning step size, range, and interval. The control algorithm of the system also holds several sub-interfaces, such as “image acquisition”, “LED control”, and “multi-spectrum image data analysis”. Microsoft Foundation Classes (MFC) is utilized for software interface, and OpenCV is integrated for image visualization and processing operations. Modules and sub-functions of the control system are displayed in Appendix A Figure A1. Lastly, all the modules are installed in a waterproof mechanical encapsulation for underwater operations; the structure parameters are described in

Table 2. Specification of parameters in the waterproof mechanical structure of the designed USI system.

Component	Material	Parameter	Specification
Outer cylindrical case	6061-T6 aluminum alloy	Length	400 mm
		Inner diameter	160 mm
		Wall thickness	5 mm
		Working depth	100 m at 1 Mpa
		Max pressure resistance	1.80 Mpa
		Surface treatment	Anode oxidation

.

3. Spectral Testing and Calibration

After the optical system installation and the mechanical design pressure test analysis, the USI system prototype is sealed to maintain waterproof design requirements. Later, the imaging parameters of the prototype, such as spectral resolution, spectral response sensitivity, and underwater detection sensitivity limit, are tested and calibrated using a standard monochromator and spectrometer.

3.1. Spectral Resolution

Based on the definitions of spectral resolution, the designed prototype is calibrated for full-width at half-maximum (FWHM) of the response spectrum of the system on different wavelengths, and the least distinguishable central wavelength interval between two narrowband monochromatic beams. The experimental setup is shown in Figure 2; after execution, the relative spectral response and FWHM are calculated, as shown in Figure 3 and Figure 4. The FWHM of the response spectrum of the system is 6.7 nm to 18.5 nm. The highest spectral resolution can reach 10 nm, and the modulation offset of central wavelength is less than ±1.45 nm.

Further testing, as shown in Figure 5, was conducted using a mercury argon lamp to verify the spectral resolution of the designed prototype. The lamp emits a high-intensity spectrum at a specific wavelength ranging from 400 nm to 700 nm. The relative spectral intensity distribution curve justifies that the system can sufficiently distinguish the highly narrowband characteristic peaks emitted by the lamp.

3.2. Spectral Radiometric Calibration

The pixel response of the actual signal is directly proportional to the exposure time and the spectral radiance of the corresponding target. Therefore, the one-to-one correspondence (calibration relation) between the spectral response measured by the USI system and the known spectral radiance can be established at a certain distance. In this section, the calibration relationship between system response and absolute radiance is measured in the air to calculate the total spectral radiance corresponding to unit pixel response.

Figure 6 explains the experimental plan and standard instruments, including halogen lamp (F970), regulated power supply, and white board surface (made of barium sulfate or polytetrafluoroethylene, regarded as an ideal Lambert diffuse surface). The filament center of the lamp and the center of the whiteboard are 50 cm apart on the same optical axis. For system calibration, darkroom imaging at full-band from 400 nm to 700 nm is performed at the step of 1 nm. The calibration performance of the system at different operating wavelengths and unit exposure time (1 ms) is shown in Figure 7. Compared with the standard spectral radiance, the USI system yielded a residual sum of squares of 2.275 × 10⁻⁵ for all bands, with an average relative error of 2.36%.

3.3. Spectral Detection Sensitivity

A low illumination environment easily restricts the practical application of the USI system due to the influence of non-uniform spectral absorption and scattering caused by water. Therefore, it is necessary to measure the detection threshold of the lowest incident spectral radiance in different working bands, which is called spectral detection sensitivity (SDS) in the paper. SDS shows the ability of the system to detect the weakest signal during imaging under zero supplementary illumination.

The designed prototype is calibrated at the systems’ working temperature of 37 °C and electron gain of 1. One hundred darkroom images are captured during experiments at the maximum exposure time. The average standard deviation is calculated as 133.7, which is applied as the amplitude of random noise. Theoretically, when the SNR (that is, effective signal to random noise) is 1, the response of effective signal at maximum exposure time is 133.7, and normalized response at unit exposure is 0.002674. According to unit exposure, time and absolute spectral radiance corresponding to the unit pixel response value, the SDS limit of the system at 37 °C for 400 nm to 700 nm is calculated as shown in Figure 8.

4. Underwater Testing

To explore the potential of designed USI system for practical applications, the prototype is tested for its performance in an underwater environment after the calibration of key spectral imaging parameters. Figure A2 and Figure A3 present the USI setup combined with a supplementary light source used for a series of underwater trials. The detection range of the USI system is explored in pool testing under simultaneous illumination of six high-power (150 W) LED light sources. As in Figure 9, several spectral image data of a whiteboard (50 cm × 40 cm, with 80% spectral reflectance) at an 8 m distance are collected under varying wavelengths.

As shown in Figure 10, the brightness distribution of the whiteboard surface at different imaging distances is relatively uniform. The standard deviation of brightness fluctuation to the average brightness of the whiteboard spectral image of 410 nm to 700 nm is 0.017 to 0.070. The brightness fluctuation of the whiteboard area in the spectral image of the 400 nm band is relatively large, which is mainly due to the lower system response and low SNR of the working wavelength. The interference of stray light in the water body can be highly observed. It is also noticed that, at a farther imaging distance, the ratio of brightness of target whiteboard and background is closer to 1 near 400 nm and 700 nm. It is concluded that the accuracy of spectral image data is highly disrupted with weaker target signals due to higher interference caused by water backscattering.

Furthermore, due to the non-uniform attenuation of spectral energy by water, the spectral energy distribution received by the USI system is distorted. Therefore, the associated problem requires the necessary compensation of spectral energy attenuation for spectral image data collected at different underwater imaging distances. The attenuation degree of underwater spectral energy is related to the imaging distance, which causes distortion in the original spectral features. To minimize the influence of the water body on spectral characteristics of the same target, the spectral response of the target at different imaging ranges is normalized to the spectral response at the same distance.

Additional testing is conducted on a color plate as the target, and a series of spectral images are collected at different imaging distances, as shown in Figure 11. Following the previous results, the SNR and spectral information reliability of spectral images at 400 nm and 700 nm, the data processing and analysis of 410 nm to 690 nm for color plate is carried out, as shown in Figure 12.

As shown in Figure 13, through the water attenuation compensation, the results of color plate spectral images in the 550 nm band and 650 nm band at different imaging distances are uniformly compensated to 7 m. From the relative compensation errors of varying color blocks, it can also be seen that the accuracy of compensation is high, which is very close to the actual spectral response under the target compensation distance. Therefore, through the water attenuation compensation algorithm, the spectral response at different lengths can be accurately corrected to the target value at the same distance so that the spectral characteristics at different distances are consistent and provide a more realistic spectral image.

5. Conclusions

Based on the in-depth investigation of the development status of spectral imaging technology and research of USI theory, in this paper, a stare-type spectral imaging system was designed for underwater applications. An LCTF was used as a principal spectral splitting device in the system. A series of calibration tests was performed in order to analyze the accuracy of the data collected by the system. The proposed USI model highly simplifies the mechanical complexity and enhances the spectral imaging performance. The working wavelength of the system was maintained in order to be easily modulated and switched along with the full-band spectral range of 400 nm to 700 nm with a spectral resolution better than 10 nm. The FWHM of the response spectrum at different central wavelengths was 6.7 nm to 18.5 nm.

Following the excellent performance of the prototype USI system, it can be concluded that the proposed USI system has the advantage of compact mechanical structure and simplified optical geometry, high efficiency, high SNR, and enhanced resolution. Compared with push-broom spectral imaging models, it has the most straightforward optical geometric correction. To sum up, the designed system can be considered highly reliable for in situ non-contact monitoring of seabed, coral reef observation, etc. Future studies on the system will include modeling and correcting noises caused by water bodies, which will improve spectral imaging quality. The system will be employed for detailed in situ monitoring of different underwater targets.