Leveraging Gaussian Processes in Remote Sensing

Abstract

Power grid reliability is crucial to supporting critical infrastructure, but monitoring and maintenance activities are expensive and sometimes dangerous. Monitoring the power grid involves diverse sources of data, including those inherent to the power operation (inertia, damping, etc.) and ambient atmospheric weather data. TheAutonomous Intelligence Measurements and Sensor Systems (AIMS) project at the Oak Ridge National Laboratory is a project to develop a machine-controlled response team capable of autonomous inspection and reporting with the explicit goal of improved grid reliability. Gaussian processes (GPs) are a well-established Bayesian method for analyzing data. GPs have been successful in satellite sensing for physical parameter estimation, and the use of drones for remote sensing is becoming increasingly common. However, the computational complexity of GPs limits their scalability. This is a challenge when dealing with remote sensing datasets, where acquiring large amounts of data is common. Alternatively, traditional machine learning methods perform quickly and accurately but lack the generalizability innate to GPs. The main objective of this review is to gather burgeoning research that leverages Gaussian processes and machine learning in remote sensing applications to assess the current state of the art. The contributions of these works show that GP methods achieve superior model performance in satellite and drone applications. However, more research using drone technology is necessary. Furthermore, there is not a clear consensus on which methods are the best for reducing computational complexity. This review paves several routes for further research as part of the AIMS project.

1. Introduction

Power grid reliability is crucial to supporting critical infrastructure. Service interruptions can have devastating consequences for communication, emergency, and transportation services, just to name a few. In addition to infrastructure, nearly every element of modern-day living relies on a stable and functioning power grid, including heating and cooling systems, lighting, refrigeration, telecommunications, and internet access. Interruptions in the power supply can range from minor inconveniences, such as the temporary loss of Internet or television services, to more significant problems, like the inability to access medical equipment or emergency services during critical situations [1]. As climate conditions worsen, the reliability of the power grid becomes even more significant. Extreme weather events, such as hurricanes, wildfires, or severe storms can cause serious damage to power infrastructure, leading to prolonged power outages. Rising temperatures and changing weather patterns can strain the grid [2], resulting in increased demand for electricity, which, in turn, stresses transmission and distribution systems.

Managing the power grid is no easy task under ideal conditions, to say nothing of the threat of physical and cyber attacks [3]. With this in mind, the Grid Communications and Security group at Oak Ridge National Laboratory (ORNL) researches integrated sensing and communications platforms to ensure efficient and reliable grid communications. The proposed project titled Autonomous Intelligence Measurements and Sensor Systems (AIMS) is currently underway. The goal of this research is to develop a machine-controlled response team with sensors, mobile platforms, communications, and data storage and analysis components. These elements will make up a system capable of autonomous inspection and reporting with the explicit goal of improved grid reliability. The system will comprise advanced sensors, generative design AI, communications, autonomic systems, swarming technologies, energy harvesting, collaborative operation between drones and the command center, and training of grid state measurements. Figure 1 presents a graphical overview of AIMS. AIMS has significant future implications for infrastructure management.

Burgeoning research that leverages Gaussian processes and machine learning in combination is the subject of this review. Remote sensing plays a major role in AIMS, as its main goal will be achieved using sensor-equipped drones. Each drone may have one or more sensors collecting different types of pertinent data, not to mention its own data, i.e., battery life, state, LiDAR, etc. It is not the intent of this project to have multiple independently operating drones but to have a swarm that works together; therefore, the system will need to share data between drones and the command center. The data collected by drones will also need to be integrated with the data inherent to the power grid, like power flow, damping, and inertia, as well as ambient atmospheric data, like wind speed and temperature. Integrating all of these data reflects the complex processes of the grid, highlighting the need for interpretable analysis.

1.1. Remote Sensing

Remote sensing applications have gained significant traction in autonomous surveillance [4,5,6], enabling the monitoring and collection of data related to environmental and industrial processes. Various devices, such as satellites, unmanned aerial vehicles (UAVs) or drones, and ocean crawlers, offer valuable tools for data acquisition in diverse contexts. When it comes to monitoring the power grid, leveraging drone and sensing technology provides a new opportunity. Drones equipped with sensors and cameras can be deployed to capture high-resolution imagery and collect data related power infrastructure, such as transmission lines, substations, and power distribution equipment. These data can include visual imagery, thermal imagery, LiDAR (Light Detection and Ranging) data, and other relevant sensor readings. Algorithms combining GPs and ML can identify patterns, detect anomalies, and predict potential failures or maintenance needs based on the data. This enables more efficient and targeted maintenance and inspection activities, leading to improved grid reliability and reduced downtime. The benefits of using autonomous drones for power grid monitoring are numerous and include the following:

• Efficiency: Drones can quickly and autonomously cover large areas, making it feasible to monitor extensive power infrastructure efficiently. They can capture data from challenging or inaccessible locations, such as remote areas or areas with difficult terrain, more easily and cost-effectively than traditional manual inspections.
• Safety: By using drones, human inspectors can avoid hazardous or risky situations, such as working at heights or navigating dangerous environments. Drones can reach areas that may be unsafe for personnel, minimizing the potential risks associated with manual inspections.
• Data accuracy and detail: Drones equipped with advanced sensors can capture high-resolution imagery and collect precise measurements, providing detailed data on the condition of power infrastructure. This allows for better assessment and understanding of the grid’s health and can aid in detecting early signs of potential issues.
• Real-time monitoring: Drones can be deployed for regular or periodic monitoring, providing real-time or near-real-time data on the grid’s status. This enables proactive maintenance and prompt response to changing conditions.
• Cost-effectiveness: Adopting drone technology for power grid monitoring can lead to cost savings compared to traditional inspection methods. Drones can cover larger areas in less time, reduce the need for manual inspections, and improve the efficiency of maintenance operations.

In terms of the power grid, there are a variety of useful sensors, including cameras (infrared, ultra-violet, and visible light), as well as acoustic, radio frequency, temperature, and methane sensors, to name a few. The AIMS projects uses multiple drones in order to accommodate specialty sensors. The output of this monitoring consists of multiple types of data in multiple formats, so robust methods are necessary for analysis. The Gaussian process is a well-established stochastic method that can model complex relationships while maintaining interpretability. This method continues to be successful in remote sensing applications using satellites and physics-based modeling. However, few applications of Gaussian processes have been reported using drones in a power grid setting. Drones provide higher-resolution images and, in turn, more data, and many of the data relevant to the power grid are rooted in physics. The AIMS project provides practical opportunities to apply Gaussian processes in a novel scenario.

1.2. Gaussian Processes

1.2.1. Definition

A Gaussian process (GP) is a generalization of the Gaussian distribution. It is a stochastic process that can be used for regression (GPR) and classification (GPC) [7,8]. Using standard notation, if X is a vector of random variables that follow a Gaussian distribution, then $\mathbf{X}={\left[\begin{array}{cccc}{X}_1& X_2& \dots & X_n\end{array}\right]}^T$ and $\mathbf{X}\sim \mathcal{N}(\mu ,\Sigma )$, where $\mu$ is the mean and $\Sigma$ is the covariance matrix. The collection of corresponding function values ($f\left(x\right)={\left[\begin{array}{cccc}f\left(x_1\right)& f\left(x_2\right)& \dots & f\left(x_n\right)\end{array}\right]}^T$) also follows a Gaussian distribution [9], so $f\left(\mathbf{x}\right)$ is a Gaussian process. Assume independent and identically distributed Gaussian noise ($ϵ\sim \mathcal{N}(0,{\sigma }^2)$, then $f\left(\mathbf{x}\right)$ is defined as follows:

$$f\left(\mathbf{x}\right)\sim \mathcal{GP}(m\left(\mathbf{x}\right),k(\mathbf{x},{\mathbf{x}}^{\prime }))$$

(1)

$$m\left(\mathbf{x}\right)=\mathbb{E}\left[f\right(\mathbf{x}\left)\right]$$

(2)

$$k(\mathbf{x},{\mathbf{x}}^{\prime })=\mathbb{E}\left[(f\left(\mathbf{x}\right)-m\left(\mathbf{x}\right))(f\left({\mathbf{x}}^{\prime }\right)-m\left({\mathbf{x}}^{\prime }\right))\right]+{\delta }_{ij}{\sigma }^2$$

(3)

where $m\left(\mathbf{x}\right)$ is the mean function and $k(\mathbf{x},{\mathbf{x}}^{\prime })$ is the covariance function, also called a kernel.

1.2.2. Advantages

Since GPs are rooted in probabilistic methods, they provide reliable measures of their own uncertainty and are fairly interpretable [10], making them ideal for “physics-based” modeling related to the grid. In real-world grid scenarios, precision is very important, but the data can be very noisy. The biggest advantage of using GPs is the ability to model noise and their inherent interpretability, which can lead to the discovery of factors that influence physical processes. This is in opposition to the so-called “black box” of some machine learning (ML) algorithms, which are less interpretable by nature. However, GPs have high computational costs, especially when dealing with large datasets or complex models.

GPs have exciting implications for power grid control. Parameter estimation, load forecasting, and anomaly detection are all capabilities of GPs, just to name a few. Many aspects of power grid operations are amenable to physics-based modeling, given their well-established scientific properties (voltage, current, acoustics, temperature, etc.). The power grid is a complex system that provides a crucial service, making the interpretable results of GPs a significant benefit as opposed to “black box” methods.

1.2.3. Computational Complexity

As mentioned in the prior section, the main disadvantage of GPs is the associated computational cost. The computational complexity of GPs typically grows cubically with the number of data points, limiting their scalability [11]. Solving a GP involves several matrix operations; therefore, large datasets can become prohibitive. This is a challenge when dealing with remote sensing datasets, where acquiring a significant amount of data is common. Several sources reviewed in this paper directly address this problem in their methodologies. Listed below are strategies to reduce computational complexity.

• Hierarchical low-rank approximation: This method uses low-rank approximations in hierarchies. Breaking the approximation up in this way transforms the multi-dimensional calculation into a linear solution [12].
• Nearest-neighbor GP: This method uses a simple hierarchical nearest-neighbor algorithm to create subsets that approximate the covariance matrix [13].
• Spatial meta-kriging: This method uses the geometric median to pool subsets that are representative of the full data distribution, as opposed to other methods, where each subset may have a starkly different distribution [14].
• Vecchia approximation: This method leverages the statistical property according to which the joint density can be written as the product of conditional densities and uses various variable ordering and conditioning schemes to reduce complexity [7].
• Permutation and grouping: This method also uses ordering to reduce computational cost but focuses on a permutation algorithm that automatically chooses the most efficient grouping [15].
• Fourier feature transform:

  (a)

  Using Fourier feature transforms in place of covariance calculations reduces computational cost [16].

  (b)

  Circulant embedding uses Fourier feature transforms to diagonalize the covariance matrix, saving computational costs [17].

• Sparse approximation: Inducing sparsity into datasets is another way to create manageable subsets to reduce computational cost [18].
• Online processes: Online processes can reduce computational cost because data are processed sequentially in a constant stream rather than training on the entire dataset [19].

I reviewed references directly related to remote sensing; however, remote sensing at power substations using multiple drones and sensors is a novel task. For that reason, this paper highlights different types of remote sensing and power grid applications. Some applications employ the use of drones, while others involve the use of ocean crawlers and satellites. The common thread among them is the use of Gaussian processes and other stochastic methods in remote sensing to obtain usable information. This review includes over forty sources from a variety of journals and conference proceedings, most from 2017 to 2023, since this field is quickly developing. The content of this paper is organized in the following sections: Power Flow and Load Forecasting, Incorporating Weather, Physics-based Simulations, Additional Gaussian Processes, and Drone Functionality. These sections represent the interconnected nature of the AIMS project. Successful future implementation relies on the combination of grid, weather, and drone data.

2. Power Flow and Load Forecasting

Efficient operation of the power grid relies on forecasting. Forecasting in power grid operations is difficult because of its complexity—not only in its technical operation but also because of climate factors associated with outdoor operation. Wind forces, for example, have serious consequences for power grid operation but are very difficult to model. The AIMS project provides opportunities to collect diverse data in real time that can be used in forecasting methods.

A physics-informed Gaussian process regression (PI-GPR) was implemented, taking advantage of established physical processes to predict the phase angle, angular speed, and wind mechanical power of a three-generator power grid system [20]. The proposed method is similar to a traditional GPR but uses stochastic differential equations to estimate prior statistics instead of the likelihood. The authors compared the PI-GPR to a standard GPR model, the ensemble Kauman filter (EnKF) method, and autoregressive integrated moving average (ARIMA). They used log predictive probability to measure and compare accuracy. PI-GPR is slightly less accurate but comparable to GPR and has a smaller standard deviation overall. EnKF is an online method, while PI-GPR, as applied, is an offline process. Therefore, EnKF is less computationally complex. However, because PI-GPR does all of the prior estimating offline, the actual forecast is less computationally complex than that of EnKF. In terms of performance, PI-GPR is comparable to EnKF. PI-GPR outperformed ARIMA and was found to be more robust to noisy data. Although PI-GPR does not significantly outperform other methods overall, the first two seconds of forecasting are extremely precise. Figure 2 shows a comparison of PI-GPR and data-driven forecasting.

Another successful method for short-term prediction of electric loads is multi-output Gaussian process regression (MOGP) [21]. Future (24-h) predictions are based on previous load, temperature, and dewpoint values. The authors evaluated MOGP using mean absolute percentage error as an accuracy measure in all cases. Simulation cases for prediction included using previous load; previous load and dewpoint data; previous load and temperature data; and previous load, dewpoint, and temperature data. The following six different kernels were tested for MOGP performance: linear, Matérn, polynomial, rational quadratic (RQ), piecewise (P.) polynomial, and radial basis function (RBF) kernels. Prediction using previous (7 day) load performed best with a linear kernel. In all other cases, the best performance was achieved using an RQ kernel with 7 days of previous load and 2 days of previous dewpoint and temperature data. The results showed that superior prediction was achieved using previous load and temperature data versus using previous load and dewpoint data. The prediction made using previous load, dewpoint, and temperature data outperformed all other cases. To reach its goals for grid state measurements and collaborative operation, AIMS incorporates ambient data, like temperature and dew point. Opportunities to understand how climate affects the grid and how it can be used to influence grid operations is an interesting area for future work.

Chance constrained optimal power flow (CC-OPF) is a stochastic method used for modeling demand in power grid operations under conditions that allow for “violation probability” [22]. This method is meant to account for the uncertainties that accompany a complex system like a power grid, for example, significant and unexpected weather events. A major drawback of CC-OPF is the associated computational complexity. A novel framework inserting GP into this process using Python [22] overcomes this issue while maintaining accuracy. Under this framework, the data-driven GP replaces the alternating current (AC) power flow equations, eliminating the need for knowledge of grid configuration and parameters. This framework was put into practice using synthetic data from multiple test cases from the Institute of Electrical and Electronics Engineers (IEEE) [23]. The authors used the IEEE 9-bus Test System with three generators and three loads and the IEEE 39-bus Test System with ten generators and twenty-one loads with the same voltage level and power ratio. In all cases, loads and renewable sources were uncertain. Root Mean Square Error (RMSE) measures the accuracy of GP prediction, and the computation time, generation cost, and empirical spread of output variables also evaluate the overall performance. In general, RMSE is low, but computational complexity remains an issue in this application and an area for future work.

Initial choices for kernel and basis functions in GPR have different effects on performance. Evaluating load forecasts using multiple kernel and basis functions [24] provides insight into this effect. The authors used mean average percentage error (MAPE) to consider three basis functions (constant, zero, and linear) and six kernel functions (exponential, squared exponential, Matérn 3/2, Matérn 5/2, rational quadratic, and Ard exponential). Performance varied by location, basis-kernel combination, and month of year. This does make sense in that weather plays a large role in this model, but it is not particularly revealing of which combinations should be used. That said, in comparison to artificial neural networks (ANN) and regression tree applications, GPR outperformed both in terms of MAPE.

As mentioned previously, the ACPF solution is non-linear and difficult to solve. A general solution framework using voltage and bus injection using GPR was proposed in [25] in a simulated voltage-limit violation scenario. The authors found a strong relationship between mean absolute error (MAE) and voltage variance. Thus, higher predicted standard deviation leads to higher approximation error. The proposed method outperformed standard methods, including data-based methods, and is not computationally burdensome. In this case, GPR has shown promise in evaluating non-linear processes while maintaining interpretability.

Load forecasting during anomalous events is difficult, given that most applications use historic data that may not reflect the uncertainty of these rare events. Deep Gaussian process (DGP) analysis accounts for limited data in this scenario by mimicking the layering idea of a neural network (NN) [26]. In this study, DGP outperformed back-propagation NN, sparse GP (SGP), and variational autoencoder DGP (VAE-DGP) in terms of MAPE. In general, the proposed DGP is better at capturing the uncertainty of the limited data during an uncertain event. It is clear in Figure 3 that DGP captures all of the data throughout the entire distribution.

A method using stochastic distributed energy resources [27] to prompt pickup during blackouts also addresses the maintenance of power supply during anomalous events. The goal is to improve the system response between interruption and restoration of power supply. The authors applied this method as an optimization problem using ad hoc methods without needing to consider the power flow equations, form the power balance constraints through chance constraints, and enact connectivity constraints. A simulated case study revealed that understanding the stochastic nature of distributed energy resources is valuable for supply adequacy probabilities, but there is still a tradeoff between supply adequacy and load served. This work also did not consider energy balance, which would be necessary in a real-world scenario.

Similarly, maintaining power balance during extreme weather events can be difficult, since, by definition, they do not occur very often. A stochastic, multi-scenario method to model electrical power and energy balance successfully accounted for extreme weather effects [28] in a hydropower and renewable energy application. The proposed model minimizes the total cost function based on certain constraints, as in a traditional OPF problem. In this case, constraints include unit maintenance constraints, unit operation constraints, inter-regional power trade constraints, and system operation constraints. The authors applied this method in a simulation under normal circumstances and under extreme weather for comparison. Under the normal conditions, operation was more economical, but the renewable energy spill probability was higher. Under the extreme conditions, both operations exhibited load losses, but less loss was experienced under the extreme condition. These results show that the extreme weather model can promote renewable energy consumption under the conditions of extreme weather events and that the model is better at adapting to weather change, both of which maintain power balance. However, this approach should be further explored to determine the tradeoffs between cost and flexibility.

GPs are not always the most suitable methods for a problem, specifically if the data or noise are non-Gaussian. A distributionally robust stochastic method for optimal power flow accounts for non-Gaussian forecast error while maintaining interpretability [29]. Other stochastic methods (like Gaussian methods) must make assumptions about the data that can only lead to overestimation or underestimation of risk. The authors contend that a data-driven approach that does not require these assumptions allows for more correct control of the tradeoffs between economic efficiency and risk constraint violation, providing an optimal solution to the power flow problem. This method is successful in demonstrating control over parameters in applications to overvoltages related to photovoltaic power and N − 1 security line flow constraints related to wind generators [30]. The strength of this approach is its modeling of forecast error, which leads to improved performance. That said, this approach is theoretically demanding and was not directly compared to other methods.

This section shows that power flow and load forecasting predictions do not simply rely on technical grid operations. Weather does play a role in how the grid will perform. For that reason, incorporating weather data is crucial to a well-rounded power flow forecast.

3. Incorporating Weather

Climate change presents new challenges for power grid operation. Strong winds can damage infrastructure by causing galloping power lines, flying branches and debris and by downing power poles. Extreme high or low temperatures create significant demand on power grids. They can cause power lines to expand or contract due to their material composition. All of these factors make it clear that incorporating weather data into grid operations is a necessity. Wind speed and temperature, in particular, are the focus of this section.

3.1. Wind Speed Forecasting

Wind speed is difficult to forecast because it is random in nature and, therefore, there is no probability distribution that totally describes it. Pobočíkováa et al. [31] compared four different distributions—two-parameter Weibull, three-parameter Weibull, Gamma, and lognormal distributions—to ascertain the most suitable among them. The three-parameter Weibull distribution outperformed the other distributions, but none were significantly fit. Statistically, these distributions are interpretable, but a more flexible distribution may be useful here.

Statistical models are usually limited to short-term prediction, but hybrid models seek to overcome this [32]. GP regression combined with other meteorological data and atmospheric stability ratings on public data from UK weather stations outperformed the current numerical method. However, it was somewhat location-dependent. Furthermore, this method is interested in wind prediction to improve the accuracy of wind power output prediction, not specifically in forecasting power grid loads. Another study on a hybrid model combing GPs with recurrent neural networks (RNN) similarly sought to increase prediction accuracy while maintaining the probabilistic properties of GPs [33]. This method uses a GP to provide necessary information to enhance the RNN model. The authors applied this method to data from the National Wind Energy Technology Center to assess its accuracy. The method outperformed other state-of-the-art methods in terms of RMSE, but accuracy decreased as the forecast step increased, and computational time was not reported.

A sparse GP with a moving window strategy was proposed for prediction of wind gusts in [18]. Historical data from numerical weather prediction (NWP) and the European Center for Medium-Range Weather forecasts (ECMWF), as well as observation data from the 2022 Winter Olympics, were used to train and validate the proposed method. The proposed method outperformed the ECMWF, GP regression, random forest, ECMWF-Sparse GP regression, and ECMWF-multiple linear regression models in both short-term and long-term forecasting. The authors noted that the spatial correlation of the input variables may have affected the accuracy of this method. In addition to the sparse method, the moving window method was meant to reduce computational complexity, although computational times were not reported.

An application using a Warped Gaussian Process (WGP) was presented in [19]. The authors used sparse online WGP in order to account for the time-based nature of the data and the high computational cost associated with WGP. Once again, the purpose of this analysis was to better predict wind power production. On data from an actual wind farm, the proposed method was competitive with GP and splines quantile regression. However, future work could improve hyperparameter estimation, including heteroscedasticity, changes in wind direction, and terrain. Figure 4 shows wind power production compared to the observed data with confidence intervals. The results show the flexibility of WGP in action.

3.2. Temperature Detection from Remotely Sensed Images

The AIMS projects allows for real-time monitoring of grid assets using drones equipped with cameras. These cameras can be used for object and change detection but also provide opportunities to estimate temperature. As mentioned, extreme temperatures strain the grid. Overheated lines or transformers are hazardous to operation, and high temperatures are associated with arcing. In addition to temperature sensors and ambient temperature data, using remotely sensed images for temperature estimation offers redundancy and real-time assessment. This is an area with very little current research in relation to the power grid.

Temperature sensing for identification of hot spots on power lines and transformers is valuable to power grid operations. Hot spots can be indicative of failure or potential failure and affect efficiency. Infrared (IR) imaging is a convenient way to estimate temperature, and IR cameras can be easily mounted on drones for inspection, as in the AIMS project. A new method using LiDAR and long-distance IR cameras was proposed in [34], using temperature–radiation and radiation–digital distance mapping to approximate temperature more accurately. The process uses an offline method for data collection and interpolation (calculating distance through digital count and radiation via Planck’s law) and an online method for analysis. In the online process, LiDAR provides the distance measure, and the IR camera provides the radiation measure. In a black-body application, the proposed method outperformed forward-looking IR and a single mapping method at 3.5 and 7 m according to MSE. That said, under non-controlled conditions, atmospheric elements like humidity may have negative effects on results.

An application to stream management considered the problems atmospheric conditions can add to temperature estimation from high- and low-resolution IR images [35]. Spectral mixing analysis (SMA) was the method of choice, since it accounts for the spectral signature, topography, shadowing effects, and spatial resolution. This method was not compared to others and was limited by the instrumentation of the time (2001). Although this analysis was focused on stream management, it is relevant to power grid applications in that the stream is an asset that must be distinguished from the air temperature surrounding it, much like power lines.

A direct application of IR imaging for hotspot identification [36] using drones leverages information from both IR images and visible-light images using image registration methodology. This process involves first identifying the power lines in the image using a Gaussian filter, edge detection, and Hough transform. After identifying power lines, a histogram threshold is used for hotspot identification. Once again, this method was intended to test functionality and was not compared to other methods, nor did it estimate temperature directly.

In addition to real-time monitoring, using collected data to train and improve models can benefit grid operations. Simulations are often used to model complex processes and may take more serious computational time and energy. The next section goes into detail on the use of physics-based models that take computational complexity into account.

4. Physics-Based Modeling

Physics-based simulation is a process to model a dynamic system using the laws of physics. The appeal of physics-based models is their innate reflection of the physical world (time and space) and their generalizability. Artificial intelligence methods like machine learning perform quickly and with high accuracy but, at times, are intractable and may not utilize statistical frameworks to their advantage. The drawback of statistical methods in big data applications is the large computational costs associated with using them. However, new methods have been developed that reduce computational time while using the power of statistics in data analysis. This section is divided into subsections that reflect the underlying structure exploited by authors in their analyses, including Spatial Statistics, Low Rank and Sparse Approximation, Spectral Applications, and Gaussian Process Emulators.

4.1. Spatial Statistics

A common problem in large datasets in remote sensing applications is incomplete lattice or missing data. A multi-output GP regression combines data from multiple sensors with time series to understand their statistical relationship [37]. This application is meant to fill the gaps in the data left by some remote sensing operations—in this case, the gaps created by cloud cover in satellite images. Incorporating a time series allows for GPR prediction to estimate the data in time based on multiple different sensors while accounting for the relationships among the sensors. The authors applied this method to various vegetation land-cover areas of interest. Since crop growth differs depending on the season, the authors further explored their method based on pixel information for spatial analysis in addition to the time series. In the short term, the proposed multi-output method outperformed standard GP, but computational complexity remains an issue for this method. In general, long-term prediction using the proposed method is not highly accurate, but the interpretability of this method still stands out.

A new approach using Bayesian and Maximum Likelihood Estimation (MLE) was proposed in [17] to address this problem. A Markov-chain Monte Carlo (MCMC) approach was adopted for Bayesian inference, and a Monte Carlo expectation-maximization algorithm was adopted for MLE in a GP. The approach was evaluated using simulated data (different lattice sizes and missingness values) and real-world satellite data on sea surface temperatures in the Pacific Ocean (see Figure 5). Bayesian estimation incorporates prior knowledge and quantifies uncertainty but is computationally expensive and statistically complex. MLE offers simplicity and computational efficiency at the expense of explicit uncertainty quantification. Further research should explore hybrid approaches that combine the strengths of both methods for parameter estimation in GPs.

A divide-and-conquer strategy within the Bayesian paradigm addresses scalability in spatial process modeling of geostatistical data [15]. The authors introduced the concept of “spatial meta-kriging” (SMK), which involves partitioning the data into subsets, analyzing each subset using a Bayesian spatial process model, and then obtaining approximate posterior inference for the entire dataset by combining the individual posterior distributions from each subset. The approach offers full posterior predictive inference at arbitrary locations for the outcome, as well as the residual spatial surface, after accounting for spatially oriented predictors. The authors provided theoretical results and empirical illustrations using different simulation experiments and a geostatistical analysis of Pacific Ocean sea surface temperature data. This article highlights the importance of addressing scalability issues in spatial process modeling and provides a practical and intuitive approach for analyzing massive spatially and temporally indexed databases. Areas of future work include scalable inference in two different situations, namely when spatiotemporal interpolation is sought at discrete time points (e.g., monthly or yearly data) and when spatiotemporal interpolation is sought at arbitrary locations and time points. Multivariate spatial data analysis is another unexplored avenue.

New methods proposed in [16] highlight the limitations of using stationary kernels and the need for nonstationary kernels to model complex spatio-temporal processes. The authors introduce a new class of non-stationary kernels that can be learned from the data using a spectral representation of the covariance function and a GP regression model. Non-stationary random Fourier features are a type of kernel method that can be used to model non-stationary covariance functions in spatial statistics. Non-stationary kernels depend on the inputs themselves, unlike stationary kernels, which only depend on the lag vector between two points. The proposed method for learning non-stationary kernels from the data is to use a spectral representation of the covariance function. The frequencies used in the feature maps are treated as kernel parameters that are optimized by maximizing the log-marginal likelihood. Other examples of non-stationary kernels for this type of modeling are polynomial kernels and neural network kernels. The authors applied this method to daily high stock prices and temperature data. Overall, the results show that non-stationary kernels of arbitrary complexity were as easy to implement as stationary kernels and gain in computational efficiency.

In the past, image processing was largely focused on aerial and satellite images with low spatial resolution. As very high-resolution imaging becomes more accessible, complex scenes require different methods to classify images. A novel framework using semantic segmentation and higher-order conditional random fields (CRF) exploits the range of contextual information available with higher-order CRF [5]. The authors’ approach uses harmonic label co-existence (which is typical), in addition to introducing local object co-occurrence. This method ensures more practical labeling by penalizing unexpected label combinations and labels that rarely occur together (i.e., a tree segment should not appear to be surrounded by car segments). The authors combat high computational costs with move-making graph cuts. Compared to other methods, HCRF achieved the highest accuracy, cleared noisy labeling, and resulted in the most practical labels.

4.2. Low-Rank and Sparse Approximation

A hierarchical low-rank approximation scheme for maximum likelihood estimation that is computationally efficient for large datasets was proposed in [12]. The generalized hierarchical low-rank approximation method (HLR) diverges from other methods by completing a low-rank approximation for each hierarchy, meaning the approximated covariance is not low-rank. Computational efficiency is gained by performing many more small calculations of eigenvalues and eigenvectors but avoiding the entire eigendecompositon. A numerical study, simulation study, and real-world study were conducted, comparing HLR to several other methods. The numerical study showed that HLR outperformed other methods in terms of dependence, smoothness, and noise level and improved approximation. The simulation study revealed that HLR outperformed other methods in terms of mean square error, and the real-world study showed better performance of HLR based on likelihood. However, in the real-world study, a neural network performed faster than HLR.

Alternate formulations of the proposed hierarchical nearest-neighbor Gaussian process (NNGP) models introduced in [13] expand on the previous work. In this case, the focus was exploiting the sparsity of large spatial datasets. The NNGP model takes multiple forms, namely the collapsed NNGP, the response NNGP, and the conjugate NNGP. The collapsed NNGP uses a Markov-chain Monte Carlo (MCMC) method to sample from the posterior and create an NNGP model capable of posterior predictive inference. This model operates most closely to a full GP model but remains computationally expensive, since it recovers random spatial information and makes predictions. The response model similarly samples from the posterior to create a posterior predictive inference response NNGP; however, it does not fully recover latent information. To its advantage, this reduces computational cost while allowing for full Bayesian analysis and prediction at arbitrary locations. The conjugate NNGP is the most well-rounded because it retains the computational efficiency of the response and achieves exact Bayesian inference. The computational advantage is the result of sampling from the marginal posterior distributions instead of the posterior. However, the conjugate NNGP introduces a hyperparameter that requires tuning. The authors’ work is the first to successfully use full Bayesian analysis on a spatial dataset at a scale of five million locations. Future work in this area should include analysis of spatial datasets from non-stationary processes, an appropriate application for AIMS, which will use drones equipped with LiDAR for the purposes of monitoring a power substation.

Fast evaluation of the likelihood of parameter inference in large datasets was also explored with Vecchia approximations [7]. Vecchia approximations are a class of approximations that reduce the computational complexity of GPs by exploiting the spatial structure of the data. The authors discussed the following two different variable orderings for the Vecchia approximation: response-first ordering and latent-first ordering. In response-first ordering, the response variables are ordered first, followed by the latent variables. This ordering is useful when the response variables are of primary interest and the latent variables are used to model the spatial dependence (the opposite for latent-first ordering). There are three different schemes for response-first ordering (full conditioning, partial conditioning, and no conditioning) and two different schemes for latent-first ordering (full conditioning and no conditioning). A simulation revealed that the low-rank and sparse approximations did well in terms of both accuracy and computational efficiency. Response-first ordering scales linearly, leading to computational efficiency gains in two dimensions. The choice of variable ordering and conditioning also has a significant impact on accuracy and efficiency. The first-order methods were also applied to data from a real-world Orbiting Carbon Observatory 2 satellite, with similar results (see Figure 6) for response-first full conditioning (RF-full), response-first standard conditioning (RF-stand), and response-first ordering with independent conditioning (RF-ind).

4.3. Spectral Applications

Class-specific random forest (CSRF) successfully combines machine learning and stochastic processes [38]. CSRF, created by the authors, combines cross-correlation measures into random forest applications for classification of hyperspectral images. Spectral–spatial cross-correlation preserves the spectral–spatial components of the image, then uses class-specific trees to classify the image. Applying CSRF to an Indian Pines Dataset and a Washington, D.C., mall dataset revealed that CSRF outperformed support vector machine (SVM) methods and achieved the highest accuracy when combined with a composite kernel. That said, CSRF could benefit from future efforts in parameter tuning and computational cost analysis.

Change detection is the process of identifying and quantifying changes in remotely sensed images. Detecting changes in these images is challenging due to variations in lighting conditions, sensor characteristics, and geometric distortions. It is only made more complicated by multimodal sensor systems. A Bayesian approach to change detection was explored with the proposal of an unsupervised pixel pairwise-based Markov Random Field (MRF) model for multimodal change detection in remote sensing images [4]. MRF captures spatial relationships between pixels in remotely sensed images by incorporating contextual information and dependencies between neighboring pixels. The MRF framework effectively models the spatial interactions by defining an energy function that incorporates the pairwise potentials and assigns labels to each pixel. This approach includes local and global contextual cues, leading to improved accuracy. MRF outperformed other models in terms of both accuracy and computational efficiency. Future work should focus on scalability and the potential for real-time change detection applications.

A GPR method that addresses the added complexity of panoramic images was proposed in [39] in an object detection application. The proposed method accounts for geometric distortions and object pose mismatches while employing a sparseness constraint to address the high-dimensionality problems that GPR faces. The model learned hyperparameters and trained offline; then, the authors used an online method for detection. An application to regions of interest from panoramic images for car detection demonstrated its success in terms of the Receiving Operator Characteristic (ROC). The proposed GPR method was more successful than direct classification with no preprocessing and classification after warping the images into perspective patches, with a high positive identification rate and a low false identification rate. This method clearly accounts for geometric distortions and pose mismatches but should be compared to methods like convolutional neural networks (CNN). Furthermore, how the region of interest is determined was not reported in the article.

Real-time anomaly detection is an important factor of power grid monitoring and key to the goals of AIMS. Current methods are limited by the assumption that signals are stationary, which is not practical in a real-world scenario. A novel method using the spectral correlation function (SCF) allows for improved characterization of grid-signal distortions because it evaluates non-stationary signals [40]. This method uses the correlation between spectral components of a signal to extract meaningful features. The authors applied SCF to real-world data from the Grid Event Signature Library and compared it to the amplitude-phase (AP) method, Fast Fourier Transform (FFT), and power spectral density (PSD) using t-distributed Stochastic Neighbor Embedding (t-SNE). SCF outperformed the other methods in terms of distinctive grouping and distinguished between anomaly events. Future work should investigate the classification of multiple types of grid signatures and the use of non-linear models.

4.4. Gaussian Process Emulators

One particular type of physics-based GP is called an emulator. An emulator, also called a surrogate or proxy model, builds a statistical model meant to mimic a particular process [6,41]. The advantage of emulators over full GPs is the increased computational efficiency, resulting in tractable models that can produce confidence intervals. Emulators focus on the use of physics-aware GPs in remote sensing and their implications for accurate and interpretable predictions. Figure 7 illustrates the general process. A joint GP model combining actual data with simulated data, a latent force model (LFM) that encodes ordinary differential equations, and an Automatic Gaussian Process Emulator (AGAPE) for forward and inverse modeling with automatic emulation were proposed in [41] to explore emulators as generators. Simulated data from a same-site scenario versus a cross-site scenario were evaluated to determine if the inclusion improved prediction accuracy. Although the joint GP model did not outperform a full GP method, it was proven to be a safe way to combine simulated data with real data. The GP-LFM model combines data-driven Bayesian approaches with purely mechanistic models and can be used as a generative model, it maintains interpretability, and shows good extrapolation capabilities. However, there is significant computational cost associated with this method, and it is only applicable to differentiable processes. In a toy example with multi-output and scalar inputs, AGAPE outperformed other methods in approximating the underlying function it was emulating. However, this method’s success is very dependent on the choice of good starting points.

Emulators were further studied by using active learning to mimic complex and costly computational codes in a multi-output setting [6]. The authors referred to a generic active emulation (AE) function and active multi-output Gaussian process emulator (AMOGAPE). They focused on a sequential method that uses an acquisition function optimized via gradient-based methods. In this case, the authors applied AMOGAPE as a GP interpolator and regression formulation to synthetic and real-world data. They compared AMOGAPE to other sequential and non-sequential methods using synthetic data. In both cases, AMOGAPE outperformed other available methods as the number of nodes increased. Finally, the authors applied AMOGAPE to the leaf canopy PROSAIL radiative transfer model (RTM) in two-dimensional and three-dimensional space. Once again, AMOGAPE outperformed random sampling in both cases. Future work should involve the use of alternative acquisition and kernel functions, as well as performance measures other than root mean square error.

5. Additional Gaussian Applications

The aforementioned problems with traditional GPs are addressed by utilizing Deep Gaussian Processes (DGPs) that account for complex kernel processes, scale to large datasets, and improve prediction accuracy [8]. A DGP creates hierarchies of GP models in a similar structure to that of deep neural networks. This setup improves computational efficiency, since it can model multiple simple kernels, as opposed to a complex kernel, as required by some GP models. The authors applied this method to superspectral sounding data with multiple models, namely a full GP model, a sparse GP model, and DGPs with one to four layers. DGPs with two or more layers outperform the full GP model and are more computationally efficient.

Learning control-oriented models involving the development of optimal experiment design, receding horizon optimal control, and continuous model improvement are also available for GP applications [10]. Data-driven control strategies are possible with the integration of GPs in control frameworks because they can model system uncertainties/anomalies and predict behavior. Over time, the incorporation of prior knowledge allows for optimized control actions and more adaptability within system dynamics. A case study of a simulated electricity grid with real-world data verified this method in a “demand response” problem. This research provides a valuable framework for bridging machine learning and controls for physical systems by using GPs. Further research should focus on addressing scalability and real-time implementation.

A truly autonomous system requires no user intervention. Practically speaking, in some applications, this is not possible. In practice, data do not always follow a Gaussian distribution, so they are transformed (usually with a logarithmic or exponential function). A Warped Gaussian Process (WGP) addresses this by using a model that automatically learns the optimal transformation by “warping” the predictions of a standard GP model and using continuous optimization to infer the parameters [42]. Another advantage of a WGP is that the learned warping function is accessible. The authors compared the performance of the novel WGP to a standard GP and variational heteroscedastic GP (VHGP) through three different experimental applications, namely the estimation of ocean chlorophyll concentration, vegetation parameter retrieval, and causal inference in a set of 28 geoscience problems. The chlorophyll experiment revealed that WGP outperformed standard GP and VHGP in all comparisons when using raw data. Vegetation parameter retrieval also showed that WGP had lower uncertainty and more homogenous results than the compared methods. Finally, WGP performed better than the other methods in causal inference. These combined results show that WGP performs well and has explanatory capabilities. Future work would benefit from applications of WGPs to nonparametric functions.

The sources cited thus far are relevant to the analytical component of the AIMS project or, essentially, the work done on data that have been collected via drone. However, the drone itself has innate data (LiDAR, battery life, etc.) that should be optimized for grid control. The best practices will be the result of well-planned path design and efficient use of battery life.

6. Drone Functionality

6.1. Exploration

One element of drone navigation is exploration. Programming a specific path for a drone can be efficient in scenarios where the operation is routine; however, in cases in which an event is not routine, an autonomous drone needs to be able to make path decisions. In such events, a drone may need to be deployed to gather information and would ideally know when it has collected enough or must continue searching. GPs can determine the best trajectory based on available data while measuring the uncertainty of the unexplored area, i.e., what is the probability that an unexplored area would have new, valuable information?

Modern remote sensing provides a novel way to characterize important features of unexplored phenomena of interest [43]. The authors treated the combination of available measurements and the probable uncertainty as an optimization problem. Parameters that estimate the cost of maneuvering were also incorporated. The authors evaluated this process in a simulated space exploration with test cases that emphasized and de-emphasized exploration. Case 1 favored exploration over data refinement, Case 2 placed less emphasis on exploring the whole field of interest, Case 3 decreased emphasis on exploration over time as data were collected, and Case 4 emphasized exploration dependent on the cost of maneuvering. The best performance was achieved under conditions that favored exploration. Figure 8 shows a comparison of all four cases.

The autonomous exploration of unknown phenomena was further investigated via a process termed “informative planning” [44]. In this case, the focus was on long-term planning, where computational expense remains a continuous problem as data accumulate. Therefore, the authors used Sparse Online Gaussian Processes (SOGPs) to reduce high computational costs. Online learning is pertinent here, as the SOGP needs to incorporate the temporal variation of the environment, re-estimating the hyperparameters when needed. This process needs spatiotemporal cues to justify the computational cost of any updates. The authors applied this method to a simulated ocean environment using real-world salinity data. When hyperparameters were manually set, performance was weak, but data-driven selection improved performance and reflected the ground truth.

6.2. Energy Harvesting

Energy harvesting (EH) is crucial in remote sensing operations. EH refers to the process of harnessing energy from external sources (i.e., wind, solar, etc.) and storing it [45]. A critical component of the AIMS project is the use of drones and sensors, which have a finite battery life. Charging stations, drones, sensors, etc., need to record the battery level so that the system can make decisions accordingly. Drones need to retain enough battery to return to their charging stations, and sensors need to indicate low battery life to the main control system to ensure their timely replacement. Adding stochastic processes to this system can contribute to more efficient drone operation and improved sensor battery life. Furthermore, electric-field energy harvesting is of interest to power grid operations in general [46].

A novel method using an Age of Information (AOI)-based status update system for a two-state stochastic process extends recent AOI research by considering the state of the stochastic process (normal or alarm) when the status has changed but the monitoring application has not been informed [47]. The system consists of an EH sensor that sends status updates to a destination node under the condition that that the stochastic process is in one of two states, namely normal or alarm, for a certain amount of time. At every time slot, the sensor generates a status update, then determines if the monitoring application is advised. If there is a state change, the system should leverage that with the AOI at the destination node to determine if an update is desirable, given available energy resources. Consideration of all these variables is a cost minimization problem to determine the best EH policy, achieved using a Markov decision process. The authors concluded that reserving energy is necessary to prepare for unexpected, long periods of an alarm state. In practical terms, status updates are less frequent during a normal state and much more frequent during an alarm state. Figure 9 depicts the cost of different combinations of state transitions.

7. Conclusions

This review highlights notable progress in leveraging Gaussian processes in remote sensing applications. The integration of the computational capabilities provided by machine learning and the analytical tractability of Gaussian analysis presents an advantageous combination for grid control. In summary, GPs are a well-established statistical method with initial success in remote sensing and grid applications like forecasting, parameter retrieval, physics-based simulation, and drone functionality. Since much of the existing work in this field has primarily focused on simulated or theoretical applications, future work should focus on opportunities to translate these efforts into practical, real-time evaluation. The AIMS project can provide opportunities to (1) test drone exploration; (2) collect real-world grid data in the form of images, acoustics, ambient weather data, etc.; and (3) integrate diverse data from different sources, including drone telemetry, grid-related data, and weather data. Further research in this area should continue to investigate methods of improving computational efficiency. Implementing these techniques using an interpretable method like GP will enhance the ability to effectively remotely manage the power grid in a dynamic environment using the physics-based data associated with its processes. Future research also has the potential for significant advancements in grid monitoring, strengthening the ability to detect and respond to emerging threats such as physical attacks, cyber attacks, and the adverse impacts of climate change.