Next Article in Journal
Nano-Bismuth-Sulfide for Advanced Optoelectronics
Previous Article in Journal
Double-Clad Bismuth-Doped Fiber with a Rectangular Inner Cladding for Laser Application
Previous Article in Special Issue
ReDDLE-Net: Reflectance Decomposition for Directional Light Estimation
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Electro-Optical Sensors for Atmospheric Turbulence Strength Characterization with Embedded Edge AI Processing of Scintillation Patterns

by
Ernst Polnau
1,
Don L. N. Hettiarachchi
1 and
Mikhail A. Vorontsov
1,2,*
1
Intelligent Optics Laboratory, School of Engineering, University of Dayton, 300 College Park, Dayton, OH 45469, USA
2
Optonica LLC, Spring Valley, OH 45370, USA
*
Author to whom correspondence should be addressed.
Photonics 2022, 9(11), 789; https://doi.org/10.3390/photonics9110789
Submission received: 31 August 2022 / Revised: 20 October 2022 / Accepted: 20 October 2022 / Published: 24 October 2022
(This article belongs to the Special Issue The Interplay between Photonics and Machine Learning)

Abstract

:
This study introduces electro-optical (EO) sensors (TurbNet sensors) that utilize a remote laser beacon (either coherent or incoherent) and an optical receiver with CCD camera and embedded edge AI computer (Jetson Xavier Nx) for in situ evaluation of the path-averaged atmospheric turbulence refractive index structure parameter C n 2 at a high temporal rate. Evaluation of C n 2 values was performed using deep neural network (DNN)-based real-time processing of short-exposure laser-beacon light intensity scintillation patterns (images) captured by a TurbNet sensor optical receiver. Several pre-trained DNN models were loaded onto the AI computer and used for TurbNet sensor performance evaluation in a set of atmospheric propagation inference trials under diverse turbulence and meteorological conditions. DNN model training, validation, and testing were performed using datasets comprised of a large number of instances of scintillation frames and corresponding reference (“true”) C n 2 values that were measured side-by-side with a commercial scintillometer (BLS 2000). Generation of datasets and inference trials was performed at the University of Dayton’s (UD) 7-km atmospheric propagation test range. The results demonstrated a 70–90% correlation between C n 2 values obtained with the TurbNet sensors and those measured side-by-side with the scintillometer.

1. Introduction

Performance of atmospheric electro-optical (EO) systems, such as free-space laser communication, remote sensing, active imaging, directed energy, and optical surveillance can be significantly degraded by atmospheric effects (e.g., optical turbulence, refractivity and absorption) [1,2,3,4,5]. Atmospheric turbulence causes the most detrimental impact on laser-beam and image characteristics, especially in the deep turbulence conditions typical for slant and/or extended-range propagation scenarios [6]. In contrast with refractivity and absorption, atmospheric turbulence strength, as characterized by the refractive index structure parameter C n 2 , can strongly fluctuate during only a few seconds for a stationary target [7,8] and by an order of magnitude for high-velocity targets when the line-of-site rapidly sweeps across a large volume of turbulence.
To evaluate and mitigate the negative impact of atmospheric effects on the performance of EO systems, it is necessary for these effects to be accurately characterized and potentially forecast along the line-of-site to the target (including moving targets) at a temporal resolution that is significantly higher (in situ) than in today’s available atmospheric turbulence characterization EO sensors. In situ turbulence strength characterization can be applied for real-time parameter adjustment in wavefront sensing, beam control and adaptive optics systems [5,9,10], for turbulence effects mitigation in atmospheric imaging [11], and to reduce the bit error rate in laser communication systems [12,13,14]. Conventional electro-optics (EO) sensors currently used for C n 2 measurements are based on estimation of the statistical characteristics of the received optical field, e.g., the laser beam scintillation index, focal spot wander and/or widening, image sharpness, etc. [6,15,16]. Time-averaging is used to calculate these statistical characteristics based on sensing data collected over a relatively long time (typically a few minutes), resulting in low temporal resolution for C n 2 evaluation.
A novel approach for path-integrated C n 2 parameter estimation at high temporal resolution was introduced in [8], where deep neural network (DNN)-based signal processing models (Cn^2Net DNN models) were developed for C n 2 value prediction based on the processing of short-exposure laser beam intensity scintillation patterns. Preliminary results that describe the hardware implementation of this approach using EO sensor configurations (TurbNet sensors) with coherent and incoherent laser beacons and deep-machine-learning-based signal processing have been reported in conference presentations [17,18]. In this study we further extend these early results.
The application of deep-machine-learning approaches to turbulence strength characterization is a rapidly growing research area. The major emphasis in recent publications in this field has been on the development of DNN architectures for C n 2 prediction by utilizing pre-recorded or simulated input data obtained using meteorological sensors [19], numerical weather prediction simulations [12], or wave-optics numerical simulations of turbulence-degraded imagery [11,20]. The goal of the research presented here was to experimentally demonstrate and compare the performance of DNN-based TurbNet sensors over a 7 km distance for path-integrated C n 2 parameter evaluation at a high temporal rate (approximately 1.5 s. per C n 2 measurement) under a wide range of turbulence, environmental and meteorological conditions.
This paper provides an in-depth description of TurbNet sensors, including the EO hardware (Section 2), the pre-processing of scintillation images (Section 3), the generation of datasets during the set of atmospheric measurement trials (Section 4), and DNN model training, validation, and testing (Section 5). The performance evaluation of TurbNet sensors in atmospheric inference experiments for C n 2 value prediction using pre-trained DNN models under diverse atmospheric turbulence and weather conditions is described in Section 6. The results of TurbNet sensor development and evaluation are summarized in Section 7.

2. TurbNet Sensor: Hardware Implementation and Experimental Setting

A TurbNet sensor is comprised of optical receiver modules, a laser beacon located at opposite ends of an atmospheric propagation path, and DNN signal-processing hardware (Figure 1). In the experiments described here, we used two laser-beacon types: coherent, based on a single-mode laser source, and incoherent, utilizing a laser-emitting diode (LED). The corresponding TurbNet sensor configurations are referred to here as TurbNet-LB and TurbNet-LED for convenience.
The laser-beacon module of the TurbNet-LB sensor in Figure 2 (top right) is based on a single-mode fiber-collimator with an aperture diameter of 50 mm emitting a collimated Gaussian beam of 30 mm width at 1064 nm wavelength (about 5 mW power). The fiber collimator was mounted on a gimbal platform used for angular alignment of the beacon beam, directing it towards the optical receiver module located at the opposite side of the 7 km propagation path at the UD atmospheric test range, as illustrated in Figure 1. The characteristic laser-beam footprint at the receiver plane was of the order of 60 cm.
Due to the impact of environmental factors (e.g., weather conditions, position of the sun and clouds) and atmospheric refractivity, the laser-beacon footprint slowly drifted with respect to the optical receiver during the experimental measurement trials. The characteristic range of these drifts was of the order of the beam footprint size over a 60 to 90 min timescale. To exclude the influence of laser-beam footprint centroid drift on measurements, the laser beacon was frequently (approximately every 10–15 min) re-aligned. This circumstance precluded uninterrupted multi-hour data collection, which is desirable for the generation of large datasets of data instances (scintillation images and C n 2 values) that are representative of a wide range of turbulence and weather conditions. These datasets were used for DNN model training, validation, and testing, as described below (see Section 5 and Section 6).
Note that laser-beam divergence of the TurbNet-LB beacon can, in principle, be intentionally increased to enlarge the laser-beam footprint at the optical receiver plane, thus mitigating the impact of laser-beacon misalignment on measurements. Nevertheless, to maintain high contrast in the scintillation images captured by the CCD camera, increasing the divergence would require a corresponding increase in laser power, which was undesirable for eye-safety reasons.
The laser-beam footprint drift issue was addressed in the TurbNet-LED sensor by utilizing an LED (940 nm center wavelength, 1.0 W output power) as a light source. To reduce the emitted light divergence, the LED was placed in the focal plane of a collimating Fresnel lens with a 30 cm aperture and 30 cm focal length, as shown in Figure 2 (bottom right). The laser-beam footprint diameter at the optical receiver plane for the LED-based beacon was about 20 m. Significantly enlarging the beam footprint dramatically reduced the impact of environmental factors, thus allowing continuous measurements over 24 h without the need for laser-beacon re-alignment.
The optical receivers of both TurbNet sensors were comprised of a telescope lens (L1) and imaging lens (L2) that were utilized for re-imaging the telescope pupil into a CCD camera, as illustrated in Figure 1. A diaphragm (D1) in the focal plane of the telescope was used to reduce the impact of ambient light.
The TurbNet-LB receiver module in Figure 2 (top left) was implemented based on a refractive telescope of aperture diameter D = 11 cm and focal distance F = 77 cm. The light-gathering power of the telescope was not sufficient to obtain high-contrast scintillation images with the LED beacon. For this reason, in the TurbNet-LED sensor receiver module in Figure 2 (bottom left), the refractive telescope was replaced by a reflective Schmidt–Cassegrain type telescope having an approximately 2.8-times larger aperture diameter (D = 30.48 cm, F = 3.048 m). Note that the LED-based sensor necessitated a more expensive and bulkier receiver telescope in comparison with the receiver telescope of the TurbNet-LB sensor.
Both TurbNet sensors utilized an identical CCD camera (12-bit Allied Vision sensor with 808 × 608 pixel resolution) to capture short-exposure laser-light-intensity distributions (scintillation frames). The camera integration time (exposure time) τCCD was adjusted based on the received laser-light-intensity level: shorter (τCCD = 0.1 ms) for the TurbNet-LB sensor with the coherent laser beacon, and significantly longer (τCCD = 3 ms) for the TurbNet-LED sensor due to the relatively low received laser-light intensity.
A characteristic example of scintillation frames captured by the TurbNet sensors under different atmospheric turbulence conditions is illustrated in Figure 3. From visual assessment, the spatial structures of the scintillation images are noticeably different for various turbulence strengths.

3. Pre-Processing of Scintillation Images and Dataset Generation

Pre-processing of the input scintillation images for C n 2 assestment by the TurbNet sensors was performed using an embedded-edge AI computer (Jetson Xavier Nx) that was synchronized with the scintillometer. The C n 2 values sequentially measured by the scintillometer were considered as “ground truth” for DNN model optimization, training, testing, and performance evaluation in the inference experiments. The scintillometer was pre-set to provide a single C n 2 measurement per minute—the shortest measurement rate available with the BLS 2000 instrument. During each Δ t s c i n = 60 s time interval between sequential C n 2 measurements, the CCD camera captured 300 frames, corresponding to a frame rate of 5 frames/s.
The GStreamer multimedia framework was used to create a pipeline from the CCD camera to the OpenCV application of the AI computer, as illustrated in Figure 4. The pipeline was used to pre-set the camera frame rate and exposure time, convert the camera raw video stream into AVI format, and downscale from a 10-bit to 8-bit grayscale video stream. Image pre-processing steps were carried out for each frame before saving it to the dataset or inputting to the DNN models and included resizing frames to 128 × 128 pixel resolution, masking the region of interest (ROI), excluding low-contrast frames, and performing intensity normalization on the maximum pixel value within each frame. Corresponding to the receiver telescope aperture, the ROI was defined as an inscribed circle for the TurbNet-LB sensor and as two inscribed concentric circles for the TurbNet-LED (see Figure 3). Pre-processed frames were tagged with the corresponding “true” C n 2 values to create a dataset composed of instances (normalized C n 2 values and the corresponding 300 scintillation frames) obtained during the data-collection trials. The same media pipeline and preprocessing steps were applied to real-time inference experiments. A description of software and driver versions used for implementation of the GStreamer framework and AI processor can be found in [18].

4. Data Collection Experimental Trials

The experimental setup described in Section 2 (see Figure 1) was utilized for synchronous recording of short-exposure scintillation images and corresponding “true” C n 2 values under diverse turbulence and environmental conditions using both the TurbNet-LB and TurbNet-LED sensors. Summaries of the data collection atmospheric trials are presented in Table 1 and Table 2, which include information about date and local time (EDT) of the measurement trials, the number of recorded scintillation frames, and weather conditions. During these trials a total of approximately 0.25·106 (TurbNet-LB dataset) and 1.3·106 (TurbNet-LED dataset) data instances were acquired with the TurbNet-LB and TurbNet-LED sensors, respectively. Note that the 300 scintillation frames captured between sequential scintillometer measurements were associated with a single C n 2 value (see Section 3).
Data collection with the TurbNet-LB sensor (Table 1) was performed during 12 relatively short (lasting less than six hours) measurement trials conducted between 15 July 2021 and 2 September 2021. During each trial the laser beacon required frequent re-alignment due to laser-beacon footprint drift (see Section 2), which made it difficult to pursue data collection representing day and night turbulence variability. Enduring data-collection trials were conducted using the TurbNet-LED sensor, which did not require LED beacon realignment and, thus, allowed continuous measurements over durations exceeding 24 h. Correspondingly, with the TurbNet-LED sensor, a large dataset (5.2-times larger than with the TurbNet-LB sensor) was obtained from only five measurement trials (lasting many hours), as summarized in Table 2.
The distributions of scintillation frames obtained under different turbulence conditions in the TurbNet datasets are illustrated in Figure 5 by histograms showing the number of scintillation frames within the interval Δ C n 2 = C n , 0 2 = 1.0 10 15 m 2 / 3 (histogram bin width). Note that the first bin acounts for all scintillation frames aquired under weak turbulence conditions corresponding to C n 2 C n , 0 2 .
It is apparent that the distribution of scintillation frames in the TurbNet-LB dataset in Figure 5 (left) is more balanced (more uniform across C n 2 values) than the corresponding TurbNet-LED dataset (Figure 5 (right)), which shows that a predominantly large number of frames were recorded under medium-strength turbulence ( C n , 0 2 C n 2 2 C n , 0 2 ) . This was not a surprise, as data collection with the TurbNet-LED sensor was performed continuously for multiple days under wide-ranging turbulence conditions. This non-uniformity in the distribution of instances in the TurbNet-LED dataset is not desirable for DNN model training, as it leads to higher error in C n 2 prediction for both low- and strong-turbulence conditions due to the obvious bias in training data volume towards medium-strength turbulence. In the case of the TurbNet-LED dataset containing a large number of instances, the desired uniformity in data distribution (dataset balancing) was achieved by selective partial removal of data instances belonging to the medium-strength turbulence range. The modified (balanced) TurbNet-LED dataset, comprised of approximately an equal number of instances (0.25·106) as in the TurbNet-LB dataset, was used for DNN model training as described in Section 5. In general, removal of instances from a dataset is not the optimal method to balance it. Nevertheless, from a practical viewpoint, it was convenient to conduct a single “unsupervised” atmospheric data collection trial for 24 h and to further “balance” this dataset by selective and random removal of data instances recorded under medium-strength turbulence. An alternative was to conduct many additional “supervised” atmospheric data-collection trials by selecting days and times when C n 2 was either low or high.
For data collection using the TurbNet-LB sensor, uniformity of data distribution across the C n 2 span was achieved without data removal by intentionally increasing the number of “supervised” measurement trials performed under both low- and high-turbulence conditions.

5. DNN Model Training, Validation, and Testing

Processing of scintillation images in the TurbNet sensors was performed using the DNN (Cn^2Net) model architecture illustrated in Figure 6 [8]. A set of 16 pre-processed consecutive scintillation frames from the CCD camera were used as the input for MFEB = 16 feature-extraction blocks (FEBs) with identical topology and trainable weights. The input frames entered the DNN models with a sliding window shift of mshift = 8 frames. The parameter mshift (1≤ mshift ≤16) controls the frequency of each frame reprocessing within the 16-input frame sequence. This parameter (DNN hyper-parameter) was used for C n 2 prediction accuracy optimization and stabilization of the DNN training process. Input frames were simultaneously processed by the FEBs using three convolutional and max-pooling layers, followed by perceptron and fully connected layers. Additional details of the Cn^2Net DNN model can be found in [8]. The Cn^2Net software code was used to generate several (about 5–15) pretrained DNN models obtained using different realizations of the initial random weights.
Both the TurbNet-LP and modified (balanced) TurbNet-LED datasets were subdivided into three subsets: training (70%), validation (20%), and testing (10%). The DNN models were trained with identical training and validation subsets at a learning rate of 0.0001 using 100 epochs and applying an early stop to the validation phase using 10% of the number of epochs as the validation patience (number of epochs allowed to improve performance without error decrease).
The mean squared error (MSE) in C n 2 prediction was selected as the cost function for performance evaluation of the DNN model-training process. The DNN models that predicted C n 2 with less than 0.1 MSE in the training process were selected to evaluate the “never seen” data from the corresponding validation subsets. The three DNN models that demonstrated the best performance for each TurbNet-LB and TurbNet-LED data subsets were saved and loaded to the AI processor for real-time C n 2 prediction in the inference experiments. The outputs of these models were averaged to increase prediction accuracy. A characteristic example of Cn^2Net model optimization is presented in Figure 7 by scatter plots that compare C n 2 values predicted by a DNN model to the “ground truth” measurements obtained for the training, validation, and testing subsets of the TurbNet-LED dataset. As shown in Figure 7, the highest prediction errors were observed for data instances corresponding to strong-turbulence conditions. Note that most of the prediction errors were in the ± 20% range for both TurbNet sensors.

6. Performance Evaluation of TurbNet Sensors in Real-time Inference Experiments

Performance evaluation (inference) trials of the TurbNet sensors were conducted using the experimental setup shown in Figure 1. The sensors’ CCD cameras and AI computers were synchronized with the scintillometer that provided “ground-truth” C n 2 values at a Δ t s c i n = 60 second time rate. The AI computer received scintillation frames at a time interval of Δ t f r a m e = 200   ms . Each scintillation frame was pre-processed and added to the image buffer to create an input sequence containing 16 sequentially captured and pre-processed scintillation frames. Each of the Nmodel pre-trained DNN models (Nmodel varied from one to five) loaded into the AI computers received identical input sequences of 16 pre-processed scintillation frames in sequential order. For a given scintillation frame input sequence, prediction of a C n 2 value by a single DNN model required about Δ t m o d e l = 100 ms. The overall processing time Δ t D N N depended on the number of DNN models M m o d e l used—a parameter that is contigent on such factors as the characteristics of the training dataset (e.g., volume, distribution of data instances, etc.), the DNN training approach, the selection of DNN models for inference experiments, and a compromise between the processing time Δ t D N N and C n 2 prediction accuracy. In the atmospheric inference trials conducted, the C n 2 prediction rate was varied from approximately Δ t D N N = 1.4 s (Nmodel = 1) to Δ t D N N 1.8 s (Nmodel = 5), which was from 33 to 43 times faster than the corresponding C n 2 sensing rate of the commercial scintillometer used to obtain “ground-truth” C n 2 values.
The output data provided by the AI computer consisted of C n , j 2 ( t m ) (j = 1,…, Nmodel) predictions independently computed by the DNN models, their average (model-average) value C ^ n 2 ( t m ) , and the moving-average (also referred to as the rolling-average) prediction C ¯ n 2 ( t m ) , which was obtained based on ten sequential model average C ^ n 2 ( t m ) outputs. Here t m is the timestamp with time interval Δ t D N N .
TurbNet sensor performance was evaluated in a set of atmospheric interference trials repeated several weeks (from one to 24) after the last day of the training dataset collection trials to ensure that performance was not affected by environmental factors, including diurnal and seasonal changes in the weather. Exemplary results of the atmospheric inference trials are presented in Figure 8 and Figure 9, where the “true” C n 2 values measured by the scintillometer are compared to both model-average C ^ n 2 ( t m ) and moving-average C ¯ n 2 ( t m ) TurbNet sensor outputs for the entire range of turbulence conditions observed during the inference trials, which lasted approximately two hours (Figure 8 top and Figure 9) and 24 h (Figure 8 bottom). Note that the last inference trial (see Figure 9) was performed in the middle of a sunny, warm day in early May 2022, six months after recording the corresponding DNN training dataset during several days of relatively cold weather in November 2021 (see Table 2).
The plots in Figure 8 and Figure 9 show a close match between the sequence of datapoints [ C ^ n 2 ( t m ) and C ¯ n 2 ( t m ) ] obtained with the TurbNet sensors and the corresponding measurements [ C n 2 ( t k ) ] obtained with the scintillometer. The corresponding values of the correlation coefficient γ computed for the moving-average DNN output and scintillometer datapoints were γ = 0.7 for the inference trial in Figure 8 (top), γ = 0.77 for the trials in Figure 8 (bottom), and γ = 0.9 for the inference trial in Figure 9. Note that a similar span of the correlation coefficient γ was observed in several more recent field experiments.
Because of the high C n 2 sensing rate, the TurbNet sensors can be used to monitor rapid changes in atmospheric turbulence dynamics. This unique capability is illustrated in Figure 10. The measured vs. DNN-predicted C n 2 time evolution plots in this figure represent exemplary segments of the corresponding dependences in Figure 8. The span of model-average C ^ n 2 ( t m ) data provided by the TurbNet-LB and TurbNet-LED sensors clearly indicates the significant changes that occurred in turbulence strength during the 60 s time interval between sequential C n 2 measurements by the scintillometer. The DNN-predicted C ^ n 2 values show about a two-fold change in Figure 10 (left) and nearly a five-fold change in Figure 10 (right).
The selected scintillation images shown in Figure 10 (bottom) correspond to the highest and lowest C ^ n 2 values (points A through D in Figure 10 (top)). The scintillation pattern spatial features in these images corroborate the TurbNet sensor measurements in Figure 10 (top).
These results show that the TurbNet sensors enabled detection of changes in turbulence strength at high temporal resolution (in real-time), which cannot be achieved with conventional EO instruments based on collection and time-averaging of sensing data.

7. Summary and Concluding Remarks

In this paper, we discuss the implementation and field evaluation of novel EO sensors (TurbNet sensors) that enable atmospheric turbulence strength characterization at high temporal resolution via DNN-based processing of scintillation patterns originating from a remote laser or LED beacon, using an embedded-edge AI computer (Jetson Xavier Nx). The DNN models were developed and trained with datasets composed of a large number of instances (scintillation images and their corresponding C n 2 values) collected during several atmospheric measurement trials over a 7 km propagation path. TurbNet sensor performance was validated in a set of atmospheric inference trials performed side-by-side with a conventional scintillometer. The results of the inference trials demonstrated good correspondence (correlation coefficient γ ranging from 0.7 to 0.9) between the scintillometer measurements and turbulence strength assessment with the TurbNet sensors. It was shown that, due to the high temporal rate (between 1.4 and 1.8 s per measurement, depending on the number of DNN models used), the TurbNet sensors enabled real-time evaluation of rapid changes in atmospheric turbulence dynamics.
For practical applications of EO sensors utilizing DNN-based signal processing for C n 2 evaluation, one can potentially avoid atmospheric field trials used for the collection of DNN training datasets, which require side-by-side installation of both a DNN-based EO sensor (TurbNet) and a reference scintillometer, as described in Section 3. The training dataset can be generated using wave-optics numerical simulations that provide accurate modeling of laser-beam propagation over any selected turbulence characterization site and, thus, can be utilized for the generation of sufficiently large datasets (SIM datasets) for DNN training. A DNN that is trained with this dataset can then be further used for real-time processing of scintillation images and C n 2 prediction at the selected site, as described in [8]. In the case of DNN training using a simulated dataset, a scintillometer for collection of the training data in the field is not required. Furthermore, the EO sensor with a DNN model trained using a SIM dataset could be utilized at other propagation sites simply by computing a new SIM dataset specific for that site, with corresponding DNN retraining [8].
In conclusion, it should be noted that since DNN models of TurbNet sensors are “trained” using a dataset (training dataset) obtained for specific system parameters and propagation lengths, deviation from these pre-set conditions results in a corresponding decline in C n 2 prediction accuracy. As recently shown, this problem can be addressed via scaling of the TurbNet sensor output C n 2 values [23]. The required C n 2 scaling factors can be obtained separately for each system parameter or propagation length using either an analytical expression derived from the classical Kolmogorov turbulence theory (theory-based scaling), or through wave-optics numerical modeling and simulations (M&S-based scaling) mimicking sensor operation. For the TurbNet sensor described in this paper, the theory-based scaling factor α(L) that minimizes C n 2 prediction error for the propagation distance L is given by the simple expression α(L) = L0L, where L0 is the propagation length (L0 = 7 km) used for generation of the training dataset. A similar scaling (adjustment) of the EO sensor output signal to address changes in path length or wavelength is used in conventional C n 2 sensing systems. Typically, these adjustments are also based on either theory (available analytical expressions), simulations, or experimental measurements performed during sensor calibration [15,21].
Furthermore, conventional EO atmospheric turbulence sensing systems (e.g., scintillometers [15], differential image motion monitor (DIMM) sensors [16], etc.) are based on analytical expressions derived from Kolmogorov turbulence theory, and, for this reason, are restrained by the theoretical assumptions and simplifications made in the derivations of these formulas. This significantly limits the utilization of these sensing systems outside the provisions prescribed by the theory, which are specific for each sensor type, e.g., weak scintillations, quasi-monochromatic, spatially coherent laser-beacon beam, point-like incoherent light source, etc. Violation of these prescribed requirements can lead to low accuracy or even incorrect C n 2 sensing data. This limitation of conventional C n 2 sensors can be overcome in the DNN-based sensing approach described here, which can be exploited for various EO sensing modalities (e.g., turbulence strength characterization using a partially coherent optical wave generated by the LED beacon of the TurbNet-LED sensor). The only requirement is that instantaneously measured output sensing data in such systems distinctly change in response to variations in atmospheric turbulence conditions.

Author Contributions

Problem statement and TurbNet sensing concept: M.A.V.; experimental setup and sensor implementation: E.P. and D.L.N.H.; experimental data collection: E.P. and D.L.N.H.; signal processing and DNN model development: D.L.N.H.; writing paper: M.A.V.; analysis and discussion of obtained results: M.A.V., E.P. and D.L.N.H.; preparation of the manuscript for submission: M.A.V., D.L.N.H. and E.P. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the US Office of Naval Research through the contract N00014-17-1-2535 and the US Air Force Phase I STTR through the contract FA864922P0048.

Data Availability Statement

The data that support the presented results are available from the corresponding author upon reasonable request.

Acknowledgments

The authors would like to thank Artem Vorontsov for providing valuable insights into the development of the DNN models utilized in this paper.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

References

  1. Majumdar, A.K.; Ricklin, J.C. Free-Space Laser Communications; Springer: New York, NY, USA, 2008. [Google Scholar]
  2. Miller, N.; Widiker, J.; McManamon, P.; Haus, J. Active Multi-Aperture Imaging through Turbulence. In Proceedings of the SPIE 8395, Acquisition, Tracking, Pointing, and Laser Systems Technologies XXVI, Baltimore, MD, USA, 15 May 2012; p. 839504. [Google Scholar] [CrossRef]
  3. Perram, G.P.; Cusumano, S.J.; Hengehold, R.L.; Fiorino, S.T. An Introduction to Laser Weapon Systems; Directed Energy Professional Society: Albuquerque, NM, USA, 2010. [Google Scholar]
  4. Weiner, M.M. Atmospheric Turbulence in Optical Surveillance Systems. Appl. Opt. 1967, 6, 1984–1991. [Google Scholar] [CrossRef] [PubMed]
  5. Hirose, M.; Miyamura, N.; Sato, S. Deviation-based wavefront correction using the SPGD algorithm for high-resolution optical remote sensing. Appl. Opt. 2022, 61, 6722–6728. [Google Scholar] [CrossRef] [PubMed]
  6. Andrews, L.C.; Phillips, R.L. Laser Beam Propagation through Random Media; SPIE Publications: Bellingham, WA, USA, 1998. [Google Scholar] [CrossRef]
  7. Tunick, A. Optical turbulence parameters characterized via optical measurements over a 2.33 km free-space laser path. Opt. Exp. 2008, 16, 14646. [Google Scholar] [CrossRef] [PubMed]
  8. Vorontsov, A.M.; Vorontsov, M.A.; Filimonov, G.A.; Polnau, E. Atmospheric Turbulence Study with Deep Machine Learning of Intensity Scintillation Patterns. Appl. Sci. 2020, 10, 8136. [Google Scholar] [CrossRef]
  9. Lionis, A.; Tsigopoulos, A.; Cohn, K. An Application of Artificial Neural Networks to Estimate the Performance of High-Energy Laser Weapons in Maritime Environments. Technologies 2022, 10, 71. [Google Scholar] [CrossRef]
  10. Bolbasova, L.A.; Lukin, V.P. Atmospheric Research for Adaptive Optics. Atmos. Ocean Opt. 2022, 35, 288–302. [Google Scholar] [CrossRef]
  11. LeMaster, D.A.; Leung, S.; Mendoza-Schrock, O.L. Joint object classification and turbulence strength estimation using convolutional neural networks. Appl. Opt. 2021, 60, G40–G48. [Google Scholar] [CrossRef] [PubMed]
  12. Ma, S.; Hao, S.; Zhao, Q.; Xu, C.; Xiao, J. Prediction of Atmospheric Turbulence Refractive Index Structure Constant based on Deep Learning. In Proceedings of the SPIE 11717 24th National Laser Conference & Fifteenth National Conference on Laser Technology and Opto-Electronics, Shanghai, China, 2 December 2020; p. 1171709. [Google Scholar] [CrossRef]
  13. Pasupathi, T.; Selvi, A.V.J. Real time experimental investigation of adaptive optics compensation technique for free space optical communication. Photon. Netw. Commun. 2022. [Google Scholar] [CrossRef]
  14. Bart, M.P.; Savino, N.J.; Regmi, P.; Cohen, L.; Safavi, H.; Shaw, H.C.; Lohani, S.; Searles, T.A.; Kirby, B.T.; Lee, H.; et al. Deep learning for enhanced free-space optical communications. arXiv 2022, arXiv:2208.07712. [Google Scholar]
  15. Wang, T.; Ochs, G.R.; Clifford, S.F. A saturation-resistant optical scintillometer to measure Cn2. J. Opt. Soc. Am. 1978, 68, 334–338. [Google Scholar] [CrossRef]
  16. Belen’kii, M.S.; Roberts, D.W.; Stewart, J.M.; Gimmestad, G.G.; Dagle, W.R. Experimental validation of the differential image motion lidar concept. Opt. Lett. 2000, 25, 518–520. [Google Scholar] [CrossRef] [PubMed]
  17. Hettiarachchi, D.L.N.; Polnau, E.; Vorontsov, M.A. High Temporal Resolution Sensing of Atmospheric Turbulence Refractive Index Structure Parameter (Cn^2) based on Embedded Edge AI-Processing of Scintillation Images. In Proceedings of the SPIE 12102, Real-Time Image Processing and Deep Learning, Orlando, FL, USA, 27 May 2022; p. 121020B. [Google Scholar] [CrossRef]
  18. Hettiarachchi, D.L.N.; Polnau, E.; Vorontsov, M.A. Electro-Optics Sensor with Embedded Edge AI-Computing for Atmospheric Turbulence Refractive Index Structure Parameter (Cn2) Sensing at High Temporal Resolution. In Proceedings of the SPIE 11993, Free-Space Laser Communications XXXIV, San Francisco, CA, USA, 4 March 2022; p. 119930X. [Google Scholar] [CrossRef]
  19. Wang, Y.; Basu, S. Using an artificial neural network approach to estimate surface-layer optical turbulence at Mauna Loa, Hawaii. Opt. Lett. 2016, 41, 2334–2337. [Google Scholar] [CrossRef] [PubMed]
  20. Nair, G.N.; Mei, K.; Patel, V.M. AT-DDPM: Restoring Faces degraded by Atmospheric Turbulence using Denoising Diffusion Probabilistic Models. arXiv 2022, arXiv:2208.11284. [Google Scholar]
  21. Large Aperture Scintillometer BLS2000 User’s Manual. Available online: https://www.scintec.com (accessed on 27 August 2020).
  22. Tyson, R.K.; Frazier, B.W. Field Guide to Adaptive Optics; SPIE Press: Bellingham, WA, USA, 2012. [Google Scholar] [CrossRef] [Green Version]
  23. Filimonov, G.; Vorontsov, M. Towards Artificial-Intelligence-Based Optical Scintillometer: Scaling Issue. In Proceedings of the IEEE International Symposium on Antennas and Propagation and USNC-URSI Radio Science, Denver, CO, USA, 10–15 July 2022. [Google Scholar]
Figure 1. Schematic of the experimental setup used for TurbNet sensor development and evaluation. The laser/LED beacon modules and scintillometer transmitter were located inside an instrumental installation on the 40-m-high roof of the VA Medical Center (VAMC) in Dayton, Ohio. Optical receiver modules of both the TurbNet sensor and the scintillometer were installed directly behind the window inside the UD Intelligent Optics Laboratory (UD/IOL). Locations of EO modules are indicated by arrows. Insert shows the propagation path altitude profile. This experimental setting is described in more detail in [8]. Both data collection and inference experiments were performed using side-by-side C n 2 measurements with a commercial (Scintec BLS 2000 [21]) scintillometer serving as “ground truth” reference. The Jetson Xavier Nx AI computer was used for DNN-based C n 2 value prediction.
Figure 1. Schematic of the experimental setup used for TurbNet sensor development and evaluation. The laser/LED beacon modules and scintillometer transmitter were located inside an instrumental installation on the 40-m-high roof of the VA Medical Center (VAMC) in Dayton, Ohio. Optical receiver modules of both the TurbNet sensor and the scintillometer were installed directly behind the window inside the UD Intelligent Optics Laboratory (UD/IOL). Locations of EO modules are indicated by arrows. Insert shows the propagation path altitude profile. This experimental setting is described in more detail in [8]. Both data collection and inference experiments were performed using side-by-side C n 2 measurements with a commercial (Scintec BLS 2000 [21]) scintillometer serving as “ground truth” reference. The Jetson Xavier Nx AI computer was used for DNN-based C n 2 value prediction.
Photonics 09 00789 g001
Figure 2. Receiver and laser-beacon modules of the TurbNet-LB (top row) and TurbNet-LED (bottom row) sensors.
Figure 2. Receiver and laser-beacon modules of the TurbNet-LB (top row) and TurbNet-LED (bottom row) sensors.
Photonics 09 00789 g002
Figure 3. Example of scintillation patterns acquired from the TurbNet-LB sensor (top row) and TurbNet-LED sensor (bottom row) for weak-to-strong turbulence conditions, as characterized by C n 2 values measured by the BLS 2000 scintillometer. Scintillation images obtained with the TurbNet-LED sensor have a characteristic black circular area due to laser-light obscuration by the Schmidt–Cassegrain telescope secondary mirror.
Figure 3. Example of scintillation patterns acquired from the TurbNet-LB sensor (top row) and TurbNet-LED sensor (bottom row) for weak-to-strong turbulence conditions, as characterized by C n 2 values measured by the BLS 2000 scintillometer. Scintillation images obtained with the TurbNet-LED sensor have a characteristic black circular area due to laser-light obscuration by the Schmidt–Cassegrain telescope secondary mirror.
Photonics 09 00789 g003
Figure 4. Flow-diagram visualizing the sequence of steps performed in scintillation frames pre-processing in the TurbNet sensors.
Figure 4. Flow-diagram visualizing the sequence of steps performed in scintillation frames pre-processing in the TurbNet sensors.
Photonics 09 00789 g004
Figure 5. Distribution of scintillation frames in the TurbNet-LB (left) and TurbNet-LED (right) datasets obtained under different turbulence conditions ( C n , 0 2 = 1.0 10 15 m 2 / 3 ).
Figure 5. Distribution of scintillation frames in the TurbNet-LB (left) and TurbNet-LED (right) datasets obtained under different turbulence conditions ( C n , 0 2 = 1.0 10 15 m 2 / 3 ).
Photonics 09 00789 g005
Figure 6. Cn^2Net DNN model architecture used for C n 2 prediction via processing of scintillation images (see [8] for detailed description).
Figure 6. Cn^2Net DNN model architecture used for C n 2 prediction via processing of scintillation images (see [8] for detailed description).
Photonics 09 00789 g006
Figure 7. Exemplary scatterplots illustrating predicted C n 2 values for DNN model training (left), validation (middle), and testing (right) vs. DNN input sequence number MB in the corresponding subset of the TurbNet-LED dataset (MB = 1.0K corresponds to 103 input sequences and C n , 0 2 = 1.0 10 15 m 2 / 3 ). The corresponding results for DNN training, validation, and testing on the TurbNet-LB dataset are similar and, therefore, are not presented.
Figure 7. Exemplary scatterplots illustrating predicted C n 2 values for DNN model training (left), validation (middle), and testing (right) vs. DNN input sequence number MB in the corresponding subset of the TurbNet-LED dataset (MB = 1.0K corresponds to 103 input sequences and C n , 0 2 = 1.0 10 15 m 2 / 3 ). The corresponding results for DNN training, validation, and testing on the TurbNet-LB dataset are similar and, therefore, are not presented.
Photonics 09 00789 g007
Figure 8. Comparison of C n 2 ( t k ) values measured using the BLS2000 scintillometer (red curve) with the DNN model-average C ^ n 2 ( t m ) (green curve) and the moving-average C ¯ n 2 ( t m ) (blue curve) obtained with the TurbNet-LB (top) and TurbNet-LED (bottom) sensors during atmospheric inference trials performed on 9 September 2021 and 9–10 December 2021 ( C n , 0 2 = 1.0 10 15 m 2 / 3 ) . The moving average, computed using ten sequential model-average outputs, corresponding to 18 seconds for the TurbNet-LB and 16 seconds for the TurbNet-LED sensor.
Figure 8. Comparison of C n 2 ( t k ) values measured using the BLS2000 scintillometer (red curve) with the DNN model-average C ^ n 2 ( t m ) (green curve) and the moving-average C ¯ n 2 ( t m ) (blue curve) obtained with the TurbNet-LB (top) and TurbNet-LED (bottom) sensors during atmospheric inference trials performed on 9 September 2021 and 9–10 December 2021 ( C n , 0 2 = 1.0 10 15 m 2 / 3 ) . The moving average, computed using ten sequential model-average outputs, corresponding to 18 seconds for the TurbNet-LB and 16 seconds for the TurbNet-LED sensor.
Photonics 09 00789 g008
Figure 9. “Measurement-vs.-prediction” dataplots (see Figure 8 caption) obtained during the atmospheric inference trial with the TurbNet-LED sensor performed six months after collection of the dataset used for DNN model training.
Figure 9. “Measurement-vs.-prediction” dataplots (see Figure 8 caption) obtained during the atmospheric inference trial with the TurbNet-LED sensor performed six months after collection of the dataset used for DNN model training.
Photonics 09 00789 g009
Figure 10. Deviation in predicted C n 2 values obtained with TurbNet-LB (left) and TurbNet-LED (right) sensors during an exemplary Δ t s c i n = 60 second interval corresponding to sequential C n 2 measurements by the scintillometer (horizontal red line), where DNN model-average C ^ n 2 ( t m ) and moving-average C ¯ n 2 ( t m ) predictions are shown by green and blue curves, respectively. The characteristic grey-scale scintillation images below correspond to the highest (A), (C) and lowest (B), (D) points of the model-averaged curves. Here, C n , 0 2 = 1.0 10 15 m 2 / 3 and tm and tk are timestamps of data received from the scintillometer [ C n 2 ( t k ) ] and TurbNet sensors, respectively. The characteristic Fried parameter values r 0 = ( 0.423 k 2 C n 2 L ) 3 / 5 [22] corresponding to points A-D are: r0 = 5 cm and 8.2 cm for A and B, and r0 = 1.6 cm and 4.2 cm for C and D. Here k = 2π/λ, λ = 940 nm (TurbNet-LED), λ = 1064 nm (TurbNet-LB), L = 7 km.
Figure 10. Deviation in predicted C n 2 values obtained with TurbNet-LB (left) and TurbNet-LED (right) sensors during an exemplary Δ t s c i n = 60 second interval corresponding to sequential C n 2 measurements by the scintillometer (horizontal red line), where DNN model-average C ^ n 2 ( t m ) and moving-average C ¯ n 2 ( t m ) predictions are shown by green and blue curves, respectively. The characteristic grey-scale scintillation images below correspond to the highest (A), (C) and lowest (B), (D) points of the model-averaged curves. Here, C n , 0 2 = 1.0 10 15 m 2 / 3 and tm and tk are timestamps of data received from the scintillometer [ C n 2 ( t k ) ] and TurbNet sensors, respectively. The characteristic Fried parameter values r 0 = ( 0.423 k 2 C n 2 L ) 3 / 5 [22] corresponding to points A-D are: r0 = 5 cm and 8.2 cm for A and B, and r0 = 1.6 cm and 4.2 cm for C and D. Here k = 2π/λ, λ = 940 nm (TurbNet-LED), λ = 1064 nm (TurbNet-LB), L = 7 km.
Photonics 09 00789 g010
Table 1. Log of experimental trials conducted with the TurbNet-LB sensor.
Table 1. Log of experimental trials conducted with the TurbNet-LB sensor.
DateStart TimeEnd Time# FramesWeather Condition
2021-07-1416:0320:1937,800Partly sunny
2021-07-1512:4813:359900Sunny and passing clouds
2021-07-1913:0513:4912,300Sunny
2021-07-2113:4917:2949,800Sunny and broken clouds
2021-07-2214:0814:4812,000Sunny
2021-08-1015:1417:0219,800Partly sunny
2021-08-1113:5214:4515,300Sunny and broken clouds
2021-08-1811:5614:3316,500Partly sunny
2021-08-1916:0716:336900Partly sunny
2021-08-2310:2715:4048,600Partly sunny
2021-08-2413:5816:1844,700Sunny and scattered clouds
2021-09-0218:0118:5915,300Sunny
Table 2. Log of experimental trials conducted with the TurbNet-LED sensor.
Table 2. Log of experimental trials conducted with the TurbNet-LED sensor.
DateStart TimeEnd Time# FramesWeather Condition
2021-11-0914:2823:59171,600Partly sunny, cloudy, and passing clouds
2021-11-1000:0023:59432,000Passing clouds, clear, fog, and scattered clouds
2021-11-1100:0013:59252,000Passing clouds, clear, and partly sunny
2021-11-1812:5623:59199,200Partly sunny and passing clouds
2021-11-1900:0017:33316,200Clear, overcast, and sunny
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Polnau, E.; Hettiarachchi, D.L.N.; Vorontsov, M.A. Electro-Optical Sensors for Atmospheric Turbulence Strength Characterization with Embedded Edge AI Processing of Scintillation Patterns. Photonics 2022, 9, 789. https://doi.org/10.3390/photonics9110789

AMA Style

Polnau E, Hettiarachchi DLN, Vorontsov MA. Electro-Optical Sensors for Atmospheric Turbulence Strength Characterization with Embedded Edge AI Processing of Scintillation Patterns. Photonics. 2022; 9(11):789. https://doi.org/10.3390/photonics9110789

Chicago/Turabian Style

Polnau, Ernst, Don L. N. Hettiarachchi, and Mikhail A. Vorontsov. 2022. "Electro-Optical Sensors for Atmospheric Turbulence Strength Characterization with Embedded Edge AI Processing of Scintillation Patterns" Photonics 9, no. 11: 789. https://doi.org/10.3390/photonics9110789

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop