Earth Observation Satellite Imagery Information Based Decision Support Using Machine Learning

Abstract

This paper presented a review on the capabilities of machine learning algorithms toward Earth observation data modelling and information extraction. The main purpose was to identify new trends in the application of or research on machine learning and Earth observation—as well as to help researchers positioning new development in these domains, considering the latest peer-reviewed articles. A review of Earth observation concepts was presented, as well as current approaches and available data, followed by different machine learning applications and algorithms. Special attention was given to the contribution, potential and capabilities of Earth observation-machine learning approaches. The findings suggested that the combination of Earth observation and machine learning was successfully applied in several different fields across the world. Additionally, it was observed that all machine learning categories could be used to analyse Earth observation data or to improve acquisition processes and that RF, SVM, K-Means, NN (CNN and GAN) and A2C were among the most-used techniques. In conclusion, the combination of these technologies could prove to be crucial in a wide range of fields (e.g., agriculture, climate and biology) and should be further explored for each specific domain.

1. Introduction

Humanity has been pushing the Earth and its inherent subsystems (e.g., climate, biosphere) to their limit. Therefore, there is a real risk of moving our planet outside a safe operating space, which could lead to severe problems in a near future [1,2,3]. According to Calvin et al. [4], the majority of the research on the Human-Earth system has occurred individually. Hence, it became crucial to seek new ways or optimize the existing ones to build an enduring future for humanity [5].

In order to understand and assess the dynamics of the Human-Earth system [6], the planet has been continuously monitored through Earth observation (EO) satellites [7]. Since 1972, several hundred satellites have been launched, and as of this writing, there are more than 2000 active EO satellites [8,9], and their data already exceed the petabyte scale [7]. According to Murthy et al. [10], Xie et al. [11] and Ferreira et al. [12], the main reason behind this is the fact that EO is possibly the only cost-effective technology able to continuously provide global data at different scales. Additionally, the diversity of EO technology allows for the acquisition of data through different sensors (e.g., optical, infrared, thermal, radar and LiDAR) at different spatial, spectral and temporal resolutions. Although its benefits are myriad, the volume of data produced by EO sources (commonly represented by means of multispectral imagery) can trigger a number of problems [7]; therefore, appropriate methods and tools (e.g., Machine Learning (ML) [9,13]) are required to analyze enormous quantities of data and provide important insights regarding embedded information [12].

The term machine learning (ML) was coined in 1959 and defined as “the field of study that provides to computers the ability to learn without being explicitly programmed for that” [14]. Hence, ML has focused on the development of algorithms that allow researchers to optimize a performance criterion using historical data and experience. Over the years, the ML field has been growing, mainly due to its ability to handle huge amounts of data and extract relevant information from it, its unique characteristics pertaining to classification and forecasting, and the growth of computational power. Therefore, it has been creating new opportunities for several fields and applications (e.g., climate change [15], biodiversity deterioration [16], land-use change (e.g., deforestation), natural disasters [17], emerging diseases and other health risks [18] and management of natural resources [19]), including areas that use EO data [12]. In fact, according to Andries et al. [9] and Landry et al. [13], the majority of methods used for processing EO data are based on ML techniques.

The main research question of this work corresponds to “How does EO data and ML techniques contribute and influence the monitorization and analysis of human-earth interaction across different fields and domains?”. Therefore, this work was intended to explore the EO and ML fields to better understand their importance in today’s world. To accomplish this, a systematic review of the topics and interactions of Earth observation and machine learning was conducted. Thus, as depicted in Figure 1, this review analyzed the relationship between these areas, highlighting major approaches and applications of different types of ML techniques applied, successfully, to EO data.

The paper’s structure has been divided as follows: in Section 2, we described how the research was conducted and presented the history, concepts, description, and contributions of EO. In addition, the necessity of using ML techniques to analyze data, was detailed as well as its workflow and types. Afterward, in Section 3, a review on the importance of ML and EO was presented, highlighted by case studies of different ML categories applied to EO data in benefit of different knowledge areas. In addition, in Section 4 and Section 5, further considerations were addressed and discussed concerning EO, ML, their relation and new paths and approaches to increase the applicability and knowledge concerning these subjects.

2. Literature Review

To lay the foundations and the theoretical basis for this paper, a systematic review was conducted using Google Scholar [20] and ScienceDirect [21], considering published articles in peer-reviewed journals. In this research, the topics “Earth observation” and “machine learning” were the core search criteria. To ensure the identification of relevant case studies for each ML category and type, expressions such as “classification”, “regression”, “clustering” and “dimension reduction techniques/models”, as well as “supervised”, “unsupervised”, “semi-supervised” and “reinforcement learning” were used in conjunction with “Earth observation” and/or a specific area name (e.g., “agriculture”, “mining” and “renewable energy”). The search was refined to retain state of the art results by taking into consideration the latest studies while retaining historical reports and agreements.

2.1. Earth Observation

Earth observation (EO) is a technology that provides a unique perspective on our world [22] by covering different approaches; it can include the use of drones, aircrafts and satellites. The era of satellite EO began in 1959 with the launch of Explorer 7 [23]. Currently, there are more than 2000 active EO satellites operated by different institutions (i.e., space agencies, governments and commercial operators) [8,9], creating increased availability of information and, consequently, relevant researches and applications involving EO data [24].

EO can be seen as the gathering of information concerning the planet’s physical, chemical and biological systems [18]. Therefore, EO data is considered an example of a Big Data source—in fact, the term Big Earth Data was coined by Guo et al. [25] to describe massive EO datasets. EO data can be acquired at low cost, over long periods of time, and used to monitor, assess and comprehend the entire Earth system, while addressing negative impacts inflicted by the human civilization. The United Nations report [26] confirmed the viability of using EO data to produce official SDGs statistics for several applications, such as in agriculture [27], urban and land planning [28] and food security [29].

2.1.1. Earth Observation Data

EO satellite imagery can be classified based on the sensors used. They can be either passive, receiving emitted or reflected natural radiation by the Earth’s atmosphere and surface, or active, receiving echoes from signals transmitted by the sensor, reflected or refracted by the Earth’s atmosphere and surface [9]. Aside from differences related to the type of EO sensors, the data provided by satellites can also be distinguished by their orbits. A geostationary orbit (GEO) means that the satellite tracks the same area by following the Earth’s rotation at an altitude of 35,786 km. Low Earth orbit (LEO) means that the satellite orbit is relatively close to Earth’s surface (usually at an altitude below 1000 km), and medium Earth orbit comprises a range of orbits between LEO and GEO [30].

EO sensors provide data in four resolutions: (i) spectral: the ability to define/distinguish wavelengths and ranges of radiation (hence, different spectral bands provide a spectral signature for specific land cover types [9], such as: soil [31], water [32] or buildings) [33]; (ii) spatial: refers to the area that each pixel represents on the surface; (iii) radiometric: indicates the degree of light intensities the sensor is able to distinguish [34]; and (iv) temporal: the revisit time, namely the frequency with which sensors cross a specific area on Earth. EO images can be used to identify characteristics of interest, based on how images are presented and their inherent properties, in several fields (as presented in Figure 2) such as agriculture [35], forestry [36], water [37] and urban areas [36].

Figure 2 presents the resolution requirements (i.e., spatial, spectral, radiometric and temporal) for several fields such as agriculture, land cover and water quality. The climate field is the least demanding regarding spatial and temporal resolutions, yet it has a higher demand for spectral resolution. On the opposite side is the emergency response field, which requires the finest spatial resolution possible and a higher revisit time; however, it has the lowest spectral resolution requirement.

2.1.2. Earth Observation Limitations

Despite the advantages and applications of EO, there are also limitations associated with it, such as:

• the impossibility of having very high spectral and spatial resolution in the same data [39];
• the higher the spatial resolution, the longer the time between images of a specific area [40];
• the more advanced the sensor, the less likely one is to find historical data [40];
• the higher the spatial resolution, the higher the cost of that data [40];
• image obstruction may occur (i.e., clouds and vegetation) [26];
• the impossibility of direct observation of the bottom of bodies of water such as oceans, rivers and lakes.

2.1.3. Earth Observation Applications

Despite the disadvantages above, EO has been widely used in several fields, mainly due to its cost-effective contributions and continuous data acquisition over large areas and periods of time. According to COPERNICUS [41], areas such as: agriculture [35], which plays a crucial role in our society [42]; climate [43], where the effects of its changes (i.e., storms, droughts, fires and flooding) are becoming more recurrent and stronger, having a huge impact in essential global ecosystems such as agriculture and natural resources [44]; forests [36], in which resources are continuously depleting [45]; marine habitats [46], which represent more than 70% of Earth’s surface and generate 50% of the oxygen, 97% of the water and 1/6 of the animal protein that our society needs [47]; mining [48], where metals and minerals that are essential for modern applications (e.g., high-tech products, renewable energy, buildings) are obtained [49]; and renewable energy [50], where there is a global agreement to increase the use and support of renewable sources (e.g., solar energy, wind power, hydropower, biomass energy, waves, tides, and geothermal energy) [51] are among those with higher relevance. Moreover, EO can also be used to exploit new opportunities (i.e., management of natural resources) [19], as well as to produce relevant statistics and indicators to analyze and quantify the SD [9,26,52]. However, given the amount of data generated through EO sources, it is essential to employ methodologies focused on intelligent techniques (e.g., machine learning) to preprocess the data, extract hidden insights and predict or classify future events or samples.

2.2. Machine Learning

Machine learning (ML) is a subdomain of artificial intelligence (AI), which, according to Samuel [14], aims to provide machines with the ability to learn from data without being explicitly programmed [53]. The study and development of algorithms has played a major role in ML, aiming to train the machines to learn and make decisions based on the data and algorithms provided [54,55]. The popularity of ML is vast and increasing, which, over time, has originated many subdomains [56], including statistical learning methods [57], data mining [58], image recognition [59], natural language processing [60] and deep learning [61]. Moreover, ML applications cover a wide range of fields, including agriculture [62], renewable energies [63], disasters [64], climate [65], construction [66], human living conditions [67] and health systems [68]. This is a result of ML’s reliability, cost-effectiveness, ability to address uncertainty and speed and ability to solve problems and reach decisions.

2.2.1. Machine Learning Workflow

The process of using ML begins with raw data (e.g., EO data). These data are critically important for the success of an ML solution, and thus, should be highly representative of the intended process or case study. The more quality, quantity and diversity of data, the better the results will likely be. Additionally, raw data can be parsed, cleaned, transformed, and preprocessed [69]. Next, a review and analysis of similar case studies or applications may be performed in order to preselect a number of ML algorithms that could be suitable. Then, those algorithms are applied to the data, where it is possible to use a combination of algorithms (e.g., using a feature selection algorithm followed by a classification one [70]), compare and evaluate performances or optimize the algorithms. As a result of this learning procedure, the ML algorithm will be able to categorize or catalogue, predict actions or outcomes, detect anomalous or unexpected behaviors and identify unknown patterns and relationships [71]. Figure 3 represents a simple ML workflow approach.

Apart from the generic ML approach and all its inherent steps, it is also important to select the most appropriate ML algorithm based on the desired outcome [74].

2.2.2. Machine Learning Algorithms Categorization

Machine learning algorithms (MLAs), as mentioned, play a major role in the ML workflow. According to Gurevich [75], Dourish [76] and Yanofsky [77], there is some debate about the definition of an algorithm and even if it can be defined. Despite that, several entities and researchers have their definitions; for example, Merriam-Webster [78] defined an algorithm as “a step-by-step procedure for solving a problem or accomplishing some end”, while Hill [79] defined it as “a finite, abstract, effective, compound control structure, imperatively given, accomplishing a given purpose under given provisions” and Kaartinen [80] proposed an informal definition reading “any well-defined computational procedure that takes some value, or set of values, as input and produces some value, or set of values, as output”. Therefore, an MLA should be represented by a pseudo-code in which each step of the MLA implementation is well defined. As an example, Figure 4 presents the pseudo-code of a Decision Tree (i.e., ID3).

There are different types of MLAs, categorized according to their learning strategy and capable of performing different tasks. Figure 5 presents the types of ML, following studies presented by Dey [53], Dutta et al. [81] and Moubayed et al. [82]. These are: supervised, unsupervised, semi-supervised and reinforcement, followed by the tasks of each ML type (i.e., classification, regression, clustering and dimensionality reduction), as well as examples of use cases and ML algorithms for each task.

2.2.3. Machine Learning Algorithm Selection

Choosing the right MLA depends on several factors, including data size, quality and diversity, as well as the application purpose. Additional considerations include accuracy, training time, output interpretability, and much more. Therefore, the choice of the right MLA is a combination of business needs, specification, experimentation and time available [71,85]. It is important to notice that each technique has its strengths and limitations, and in any application, the use of more than one technique can provide a greater perception compared to the isolated use. This was further explored in the subsequent section.

3. Machine Learning Algorithms Applied to Earth Observation Data

As mentioned above, ML algorithms are extremely useful and offer numerous benefits. A substantial number of ML algorithms have been used and described in the literature, performing a wide range of tasks in a variety of domains (e.g., agriculture [62], renewable energies [63], disasters [64], climate [65], construction [66], human living conditions [67] and health systems [68]).

In the following subsections, the authors detailed the different learning types and tasks presented in Section 2.2.2 and included examples of their application in conjunction with EO data.

3.1. Supervised Learning

Supervised learning techniques learn from a labeled training data set, where the output (variable to be classified or predicted) is known. Categorized as classification (Section 3.1.1) or regression algorithms (Section 3.1.2), they use patterns found in the training dataset to classify or predict the output on unlabeled data/observations [86,87].

3.1.1. Classification Algorithms

A classification algorithm is applicable when the overall aim is to accurately assign an observation to a class [70,88,89]. There are a broad range of classification approaches, as presented in

Table 1. Examples of classification algorithms using EO data.

Algorithm	Field	Main Finding	Reference
CA-Markov	Land Use	The proposed approach confirmed its suitability for urban planning, having a superior performance compared to the global one.	[90]
Canonical Correlation Forest (CCF)	Climate	The model, based on decision trees, was used to classify local climate zones, achieving a good performance.	[65]
Convolutional Neural Network (CNN)	Land Use	The approach based on CNN achieved an accuracy of ≅98% for land use and land cover analysis.	[91]
	Living Conditions	Deep learning demonstrated a high potential in mapping areas of deprived living conditions.	[92]
Conv1D	Agriculture	Developed an efficient framework for multi-temporal crops classification.	[93]
Faster R-CNN	Slavery	Used to help liberate slaves by mapping brick kilns.	[67]
Feature Filtering and Enhancement (FFE)	Agriculture	The proposed method performed similarly to support vector machine (SVM) and random forest (RF) in the classification of crops with similar phenology.	[94]
Logical Analysis of Data (LAD)	Land Cover	The approach allowed the authors to differentiate hyperspectral subclasses from classes.	[95]
Random Forest (RF)	Agriculture	Multitemporal crop classification reduced the unfavorable effects of using single-date acquisition.	[96]
	Forest	Sentinel-2 was considered a powerful source of data for forest monitoring and mapping.	[97]
		RF was the best method to predict and map the area and volume of eucalyptus.	[98]
	Wetland	The developed framework for coastal plain wetlands classification had high accuracy.	[99]
Support Vector Machine (SVM)	Marine Habitat	SVM and K-Nearest Neighbor classifiers achieved an accuracy higher than 90% on mapping coastal marine habitat.	[46]
Word ExtrAction for time SEries cLassification plus Multivariate Unsupervised Symbols and dErivatives (WEASEL+MUSE)	Land Cover	The multivariate time series algorithm showed high accuracy for rare land cover classes.	[100]

, which also demonstrates the impact and potential use of these techniques in conjunction with EO data.

3.1.2. Regression Algorithms

A regression algorithm is applicable in cases where the main goal is to predict/estimate a continuous output variable of a given observation [88,101].

Table 2. Examples of regression algorithms using EO data.

Algorithm	Field	Main Finding	Reference
Boosted Regression Tree (BRT)	Agriculture	The results obtained from the comparison of methods showed that BRT was the best to predict maize yield.	[102]
Bootstrap Aggregation Tree (BAGGTREE)	Terrestrial Ecosystem	The best performance to obtain the latent heat evaporation using a large dataset was achieved by BAGTREE.	[103]
Gaussian Processes Regression (GPR)	Pollution	The improved GPR had a high accuracy compared to the original GPR and other methods predicting the CO2 emissions.	[104]
Geographically Weighted Regression (GWR)	Freshwater Habitat	GWR technique was accurate in the estimation of stream bathymetry of water with a depth less than 1 m.	[105]
Gradient Boosting Regression Tree (GBRT)	Agriculture	Results increased the potential of using Sentinel-2 to obtain cotton Leaf Area Index and comparison of methods showed that the GBRT was the best.	[106]
Multiple Linear Regression (MLR)	Grassland	Vegetation indices acquired from Sentinel 2 have high potential concerning grasslands productivity, management, monitoring and conservation.	[107]
	Water Quality	Landsat 7 images were a solid option for assessing water quality characteristics.	[108]
Random Forest Regression (RFR)	Drought	The use of ML to acquire the normalized microwave reflection index was an effective way to monitor the variation of vegetation water content to predict droughts.	[109]
	Land Cover	RF Regression was very accurate (96%) in delineating house-attached, semipublic and public green spaces.	[110]
	Landslide	Catchment map units and model selection are crucial for the performance of landslide susceptibility maps.	[111]
	Renewable Energy Sources	During spring and autumn, it was harder to predict the hourly solar irradiation, compared to winter and summer.	[50]
	Spread of Diseases	By mapping the relationship between EO variables and vector population, the proposed RF regression methodology was able to predict the temporal distribution of yellow fever mosquito populations.	[112]
Regression Tree (RT)	Pollution	RT effectively estimated carbon dynamics and allowed the analysis of its impacts on meteorology and vegetation.	[113]
Support Vector Regression (SVR)	Agriculture	Estimated the crop yield at a pixel level using ML proved to be an accurate approach.	[62]

synthetizes the findings within the scope of regression algorithms used in combination with EO data.

3.2. Unsupervised Learning

Unsupervised learning techniques use an unlabeled training dataset, where the output variable is unknown, to learn. Categorized as clustering or dimensional reduction algorithms, these learn by exploring and finding patterns or relationships in the training data set [86,87,114].

3.2.1. Clustering Algorithms

A clustering algorithm is appropriate when the purpose is to associate or divide observations into clusters [88,115]. There are a broad range of clustering approaches, as presented in

Table 3. Examples of clustering algorithms using EO data.

Algorithm	Field	Main Finding	Reference
Balanced Iterative Reducing and Clustering using Hierarchies (BIRCH)	Climate	The techniques used, such as K-Means and BIRCH, demonstrated their suitability when applied to the climate domain.	[43]
Burned Area (BA)	Wildfires	The presented algorithm for global burned area mapping was able to adapt to different ecosystems and spatial resolution data.	[64]
Density-based Spatial Clustering of Applications with Noise (DBSCAN)	Construction	The proposed method used to segment individual buildings had a good performance, with datasets acquired from densely built-up areas.	[66]
	Geomorphology	The proposed DBSCAN methodology for geomorphological analysis facilitated the detection of movements of a rock glacier.	[116]
Fuzzy C-Means (FCM)	Mining	The results showed that FCM was superior to K-Means and SOM for mineral favorability mapping.	[117]
Fuzzy K-Means	Soil Degradation	Assessment of spatial variability and mapping of soil properties provide an important link in identifying soil degradation spots.	[118]
Hierarchical Cluster Analysis (HCA)	Sustainability Level	The results obtained using HCA showed that Sweden had the highest level of sustainability among European countries, compared to Greece, Bulgaria and Romania.	[119]
K-Means	Agriculture	The proposed methodology, based on K-Means, and crop images, had a good performance estimating rice yield.	[120]
	Land Change	The proposed approach, based on K-Means, demonstrated better detection accuracies and visual performance for land cover and land change detection, compared to several methods.	[121]
Nearest Neighbour	Seismic	The method analyzed was reliable and effective in the identification of sequences of earthquakes.	[122]
Optimized kernel-based Fuzzy C-Means (FCM)	Agriculture	Optimized kernel-based FCM gave more accurate agriculture crop maps when compared with the classical FCM and K-Means.	[123]
Optimum-Path Forest (OPF)	Land Cover	The proposed clustering method outperformed the original approach for remote sensing segmentation in land cover classification.	[124]
Sandbars Extraction (SE)	Sandbars	The proposed algorithm demonstrated a high potential to be used for the extraction of sandbars positions.	[125]
Subtractive Clustering (SC)	Renewable Energy Sources	The choice of the clustering technique played a crucial function in the forecasting of the gross wind power output.	[63]

, which also clearly shows the impact and potential use of these techniques in conjunction with EO data.

3.2.2. Dimensionality Reduction

Dimension reductions typically follow two main approaches: feature selection (FS), applicable when there is the necessity to select fewer characteristics [126,127], and feature extraction, when the information needs to be synthesized through transformation. The aim is to create a small set of features covering much of the detail of the initial dataset [70,128,129]. Then, these features/characteristics can be fed into other techniques or otherwise used as an end result [88].

Table 4. Examples of application of dimensionality reduction approaches using EO data.

Algorithm	Field	Main Finding	Reference
Bimodal Unsupervised Dimension Reduction Algorithm (BOUNDER)	Disease Identification: Cancer	The proposed model was able to identify cancer types and distinguish cancer cells from healthy ones.	[68]
Denoising Autoencoder (DA)	Disorder Identification: Autism	High redundancy of features had implications for the replacement of social behaviors that were focused on behavioral diagnoses and interventions.	[130]
Deep Variational Autoencoder (DVA)	RNA Sequencing	The DVA achieved a greater performance compared to methods like PCA and t-SNE, providing a better representation of rare cell populations.	[131]
Principal Component Analysis (PCA)	Water Resources	The proposed approach proved to be effective and accurate at assessing water resources at catchment scale.	[132]
Stepwise Discriminant Analysis (SDA) & PCA	Water Sources	SDA and PCA improved the accuracy of water source recognition.	[133]
Autoencoder	Electricity	The proposed method improved the forecasting of electricity prices and was more accurate than the Independent Electricity System Operator prediction	[134]
Isomap	Manufacturing	Dimension reduction techniques improved the performance of methods from other categories and Isomap had the best performance for manufacturing quality prediction.	[135]
Bivariate Dimension Reduction (BDR)	Structural Reliability	The BDR method proved to be effective for structural reliability analysis.	[136]
Locally Linear Embedding (LLE)	Software	LLE and LSTM had a better performance for software system fault prediction compared to other algorithms.	[137]
Greedy Stepwise Search based Feature Selection (GSSFS)	Land Cover	The results demonstrated that FS improved the classification accuracy of land cover classification.	[138]
Whale Optimization Algorithm (WOA)	Land Cover	The proposed method demonstrated better results compared to other methods for land cover classification in almost all tests.	[139]
Recursive Feature Elimination (RFE)	Land Use	FS using the Classification Optimization Score metric reduced the processing time and produced higher classification accuracy for land use and land cover classification using very-high resolution data.	[140]
Randomized-Singular Value Decomposition (RSVD)	Pollution	The new method demonstrated itself to be a powerful approach to optimize knowledge emerging from atmospheric observations of N2O.	[141]
ReliefF	Terrestrial Ecosystem	FS methods allow the extraction of valuable information to create accurate maps of areas infested by invasive plant species.	[142]

synthetizes the findings within the scope of dimensionality reduction approaches used in combination with EO data.

3.3. Semi-Supervised Learning

Semi-supervised learning techniques fall in between supervised and unsupervised ones since they use labeled and unlabeled data for training. Their major goal is to overcome the limitations of supervised (i.e., cost associated with labeling) and unsupervised learning (i.e., difficulty to accurately cluster unknown data) [87,143,144]. As shown in

Table 5. Examples of application of semi-supervised learning techniques using EO data.

Algorithm	Field	Main Finding	Reference
Aggregation	Climate	Proposed a method to aggregate aerosol optical depth estimations from multiple satellite instruments into a more accurate estimation.	[145]
Bidirectional Long Short-Term Memory (BiLSTM)	Text Classification	Proposed a text classification framework that enabled the efficient evaluation of bibliographic records derived from bibliographic databases and accurately selected articles relevant to the research objective.	[146]
Convolutional Neural Network (CNN)	Land Cover	The results demonstrated that the proposed approach, which used hyperspectral image data, provided competitive results to state-of-the-art methods.	[147]
		Proposed a method based on CNN for hyperspectral image classification. The results suggested that it had similar results when compared with traditional CNN.	[148]
		The results demonstrated the superiority of the proposed CNN in comparison with the general CNN-based classifiers.	[149]
Constrained Agglomerative Hierarchical Clustering (CAHC)	Land Cover	The proposed strategy demonstrated its potential to better reconcile human perception of landscape pattern type mapping with unsupervised clustering results.	[150]
Generative Adversarial Network (GAN)	Change Detection	Experimental results carried out on very high-resolution image data sets demonstrated the effectiveness of the proposed method.	[151]
	Object Recognition	The authors’ model achieved a better and more stable recognition performance, in comparison to the supervised CNN, DCGAN, DRAGAN, WGAN-GP, as well as other traditional semi-supervised models.	[149]
		The proposed method achieved state-of-the-art results in recognition tasks. Additionally, it was proven that using the generated images in the training dataset increased accuracy.	[152]
Neural Network (NN)	Forest	The proposed semi-supervised pipeline for the design of tree crowns based on RGB data yielded accurate forecasts in natural landscapes.	[153]
Random Forest (RF)	Land Cover	The authors proposed an active semi-supervised random forest classifier for hyperspectral image classification which achieved better classification performance when compared with state-of-the-art results.	[154]
	Mining	The semi-supervised learning scheme provided a promising way for mineral prospection in under-explored areas.	[48]
Self-Organizing Map (SOM)	Land cover	The results showed that the proposed method outperformed existing methods for satellite image classification.	[155]
Root Distance based Boundary Sampling (RDBS)	Biology	An active semi-supervised learning framework was implemented jointly with a new active learning method (RDBS). From the results, it was observed that the semi-supervised classifiers achieved similar results to the supervised ones, with less annotated samples. Moreover, the RDBS presented better results in almost every scenario.	[156]

, semi-supervised techniques have a wide range of applications related to the use of EO data.

3.4. Reinforcement Learning

Reinforcement learning techniques learn by interacting with an environment. They have no knowledge concerning which actions they should take until given a situation, and then they discover through trial and error which actions yield the greatest rewards. Their main objective is to choose actions that maximize the expected reward and minimize punishments over a given amount of time [86,87,114]. Due to the nature of these techniques, the case studies found in the literature that combined these techniques with EO data were not directly applied to a specific field. Instead, it was found that they played a key role in EO data acquisition (e.g., scheduling, optimization, image retrieval and autonomous navigation).

Table 6. Examples of application of reinforcement learning techniques using EO data.

Algorithm	Field	Main Finding	Reference
Actor-Critic Algorithm (A2C)	Scheduling	Compared with the general heuristic rules, experiments prove that it is more effective and robust.	[157]
		Demonstrated how reinforcement learning can be used in EO satellite scheduling to reduce the time-to-completion of large-area requests. The proposed method challenged the state-of-the-art heuristics.	[158]
Asynchronous Advantage Actor-Critic (A3C)	Management & Efficiency	The A3C algorithm was applied to model and simulate the multi-dimensional resource allocation problem of the Space Information Network (SIN). The results suggested that it could improve the expected benefits and an efficient utilization of the SIN resources.	[159]
Cooperative Neuro-Evolution of Augmenting Topologies (C-NEAT)	Scheduling	The main contribution was the data-driven parallel scheduling approach for large-scale optimization, where the prediction model, based on C-NEAT, and the task assignment strategy outperformed other models with traditional training algorithms and inflexible assignment strategies, respectively.	[160]
Deep Graph Embedding and Learning		A deep reinforcement learning solution that could automatically learn a policy for multi-satellite scheduling was adapted. This solution failed to outperform state-of-the-art methods. However, it was determined that it might be fast enough to potentially generate decisions in near real-time.	[161]
Fast-Recurrent Deterministic Policy Gradient (Fast-RDPG)	Autonomous Navigation	Experimental results demonstrated that the method could enable unmanned aerial vehicles to autonomously perform navigation in a virtual large-scale complex environment.	[162]
Long-Short Term-memory (LSTM)	Task Planning	The results obtained demonstrated that the proposed method could effectively solve the satellite onboard observation task planning problem with high accuracy and low profit gap.	[163]
Pursuit Reinforcement Guided Competitive Learning (PRCL)	Image Retrieval	The proposed method was relatively fast at retrieval tasks in comparison to existing conventional online clustering, and was much faster than others for the multi-stage retrieval of images and scale estimation.	[164]
Q-Learning	Image Analysis	A deep reinforcement learning model for unsupervised hyperspectral band selection was proposed, and the results demonstrated its effectiveness.	[165]
	Scheduling	The equations, approach, and method developed could be effectively employed for various satellite operations (i.e., scheduling) that are becoming increasingly more complex.	[166]
Radial Basis Function Neural Network (RBF NN)	Control	An intelligent autonomous thermal control strategy based on reinforcement; earning for proportional-integral-derivative (PID) parameter adaptive self-tuning was proposed, and it proved to be better than the traditional PID control and switch control.	[167]
Scheduling Network	Scheduling	The results showed that the proposed real-time scheduling method for image satellites could achieve a good performance with real-time speed and immediate respond style.	[168]
Two-Phase Neural Combinatorial Optimization Method (TPNCO)		The TPNCO-RL method was more effective than a multi-objective genetic algorithm in the scheduling phase.	[169]
Variational Autoencoder and Reinforcement Learning based Two-stage Multi-task Learning Model (VRTMM)	Image Captioning	The experiment result indicated that the proposed model was effective on remote sensing image captioning and achieved the new state-of-the-art result.	[170]

synthetizes the findings within the scope of reinforcement techniques used to optimized and improve the acquisition of EO data.

4. Discussion

Monitoring the human-Earth system has huge relevance to raising awareness and understanding the impact of human actions. Thus, combining Earth observation and machine learning technologies could leverage the processes of monitoring, analyzing and comprehending Earth’s subsystems.

The EO data can be used in different fields and for different purposes. However to select the most appropriate EO data source, resolution parameters (i.e., spatial, spectral, temporal and radiometric) must be taken into account. Given the amount and richness of generated EO data, it is crucial to use effective and efficient techniques (e.g., ML algorithms) to analyze this data. The ML workflow approach requires raw data (e.g., EO data) to be, for example, cleaned, transformed and preprocessed. Furthermore, the selected ML algorithms (tasks that depend on several factors, such as data size, quality and diversity, as well as the application purpose) are trained with a portion (e.g., 80%) of the data available (training dataset) and then validated with the remaining data (test dataset). After that, the applied algorithms should be compared and/or optimized in order to select the most suitable one with the best hyperparameters. As a result of this learning procedure, the ML algorithm will be able to produce the final results.

This work comprised a collection of studies, concerning the application of different techniques of EO and ML on different fields, which helped to increase the knowledge and comprehension on those domains. Despite the accomplishment of the proposed objective, several potential improvements were identified for further work that could take place (e.g., the exploration of different EO data sources, development and implementation of other ML techniques, understanding of the relationship between different data types and the choice of ML techniques and exploration of the application of EO-ML on other domains and fields).

5. Conclusions

EO is a technology able to provide data at a global scale over long periods of time and at a low cost, providing a natural ground for data mining techniques such as machine learning. This paper highlighted the role and application of Earth observation (EO)- and machine learning (ML)-related technologies. This comprehensive review considered a wide range of research studies that employed several categories of ML (i.e., supervised, unsupervised, semi-supervised and reinforcement learning) to handle EO data and address a variety of challenges across different fields. Overall, given the advantages of EO technology (i.e., cost-effective and continuous data acquisition over large areas and periods of time) and ML techniques (i.e., ability to analyze huge volumes of data, infer new knowledge and find hidden patterns), as well as the recent advances in these domains, such technologies have been creating new opportunities to analyze, monitor and solve challenges in a wide range of fields.

From the studies reviewed, it was possible to infer that the combination of EO-ML was successfully applied in several different fields and domains across the world. All the ML categories could effectively be used to analyze EO data; however, reinforcement learning is most often used to optimize and improve EO data collection/acquisition processes. Among all the categories, the most-used techniques were RF, SVM, K-Means, NN (CNN and GAN) and A2C. Findings confirmed the importance of EO and ML in today’s challenges in efforts to solve problems, optimize processes and even acquire new knowledge on specific matters. Additionally, the development and application of different and new ML approaches should be explored further, along with extensive studies concerning the comparison and analysis of different techniques to help researchers positioning new developments and studies in these domains—as well as their fields of interest.