Spatiotemporal Prediction of Conflict Fatality Risk Using Convolutional Neural Networks and Satellite Imagery

Abstract

As both satellite imagery and image-based machine learning methods continue to improve and become more accessible, they are being utilized in an increasing number of sectors and applications. Recent applications using convolutional neural networks (CNNs) and satellite imagery include estimating socioeconomic and development indicators such as poverty, road quality, and conflict. This article builds on existing work leveraging satellite imagery and machine learning for estimation or prediction, to explore the potential to extend these methods temporally. Using Landsat 8 imagery and data from the Armed Conflict Location & Event Data Project (ACLED) we produce subnational predictions of the risk of conflict fatalities in Nigeria during 2015, 2017, and 2019 using distinct models trained on both yearly and six-month windows of data from the preceding year. We find that predictions at conflict sites leveraging imagery from the preceding year for training can predict conflict fatalities in the following year with an area under the receiver operating characteristic curve (AUC) of over 75% on average. While models consistently outperform a baseline comparison, and performance in individual periods can be strong (AUC > 80%), changes based on ground conditions such as the geographic scope of conflict can degrade performance in subsequent periods. In addition, we find that training models using an entire year of data slightly outperform models using only six months of data. Overall, the findings suggest CNN-based methods are moderately effective at detecting features in Landsat satellite imagery associated with the risk of fatalities from conflict events across time periods.

1. Introduction

Conflict in Sub-Saharan nations such as Nigeria can have many drivers such as ethnic and religious strife (including terrorist groups), climate change and drought, violence against civilians, and general corruption [1]. The resulting conflicts can range in form from violent clashes, explosions, or attacks on civilians to one-sided protests, riots, or strategic developments [2,3]. Methods to forecast conflict and the risk of violence (i.e., fatalities) can provide utility to humanitarian organizations whose efforts are needed to support people in conflict-impacted areas, yet are also impeded by ongoing conflict (As an example, see https://www.usaid.gov/humanitarian-assistance/nigeria for an overview of USAID’s work in Nigeria. Accessed on 1 July 2024). Advances in the use of satellite imagery and deep learning provide opportunities to remotely develop forecasts of future deadly conflict that can support humanitarian efforts in conflict-affected areas.

Human and environmental systems can be observed with satellite imagery in many ways that may be directly relevant for conflict, such as detecting drought conditions or agricultural encroachment into pastoral lands [4]. The severity of conflict events in particular, in terms of whether the event results in fatalities, can be linked to observable conditions. Data around conflict event sites in Nigeria reveals that the median population of non-fatal event locations is nearly triple that of fatal event locations, and non-fatal event areas are on average 40% urban while fatal event areas are on average only 20% urban (Based on population [5], land cover [6], and precipitation [7] data within a 3 km buffer around conflict event sites in Nigeria from 2015, 2017, and 2019 from the Armed Conflict Location & Event Data Project (ACLED) [3]. Population, land cover, and precipitation data prepared following methods in [8]). In addition, the median coverage of rainfed cropland in fatal event locations is nearly 40% yet only 7.5% in non-fatal locations. Irrigated cropland however is only present in around 1% of all conflict event areas. Given the increased occurrence of fatal events in areas with rainfed cropland, it is notable that average rainfall (See World Bank Climate Change Knowledge Portal https://climateknowledgeportal.worldbank.org/country/nigeria/climate-data-historical. Accessed on 1 July 2024) at non-fatal event locations is in line with national rainfall averages, while fatal event locations are approximately 15% lower (based on fatal and non-fatal event locations where at least 25% of the nearby area was rainfed cropland). These geospatial characteristics provide meaningful insight into potential information (e.g., populous urban areas, dry cropland) that can be detected from even moderate-resolution satellite imagery over time in order to forecast fatal conflict events.

Convolutional neural networks (ConvNets or CNNs) have been a focus of recent work that is connecting advancements in geospatial AI with research focused on human development [9]. CNNs are a class of neural networks well-suited for a variety of computer vision tasks that require detecting spatial patterns or features within images [10,11,12,13]. Numerous implementations of CNNs have been developed, dealing with image classification, object detection, and more [14].

Geospatial applications of CNNs include using spectral signatures and phenological trends captured by satellite imagery to perform pixel and image classification [15,16], land use classification [17], and the detection of objects such as vehicles or buildings in satellite imagery [18,19]. In recent years, researchers focused on development economics have leveraged these methods to improve upon existing methods to fill the gaps in sparse datasets such as household surveys [20]. By using abundant remotely sensed data sources (satellite imagery) as inputs, CNNs are trained using existing sparse data (e.g., poverty metrics from household surveys) as labels. The resulting trained network can be used to predict the sparse data in additional geographical areas based on satellite imagery alone.

Building on work within the geospatial community [21], Jean et al. showed that CNN-based estimates of poverty offered improvements on previous methods of using nighttime lights to estimate poverty [20]. Over the past several years there have been a number of additional contributions to the literature utilizing CNNs and satellite imagery to predict human development indicators [9]. Much of this work has focused on exploring effectiveness in new geographic contexts, along with advancements based on incorporating new data or improvements to machine learning algorithms (e.g., [22]). Poverty estimates have been produced using a variety of available satellite imagery across most of Africa [23], as well as Mexico [24], Bangladesh & India [25], and Sri Lanka [26]. In addition, literature addressing the usage of CNN and satellite image-based methods to predict other development indicators has explored infrastructure quality assessments [27,28,29], crop yield [30], education & health-related metrics [31,32], population mapping [33,34], and conflict [35]. The potential for application, improvement, and operationalization of these methods across varying sectors remains a topic of discussion [9,36].

While much of this work has shown promising results, the literature has highlighted some limitations [32] as well as the abundance of opportunities for future research to advance the practical application of these methods [9,35,36]. One important area yet to be fully addressed when predicting development indicators using CNNs and satellite imagery is the capacity for accurate predictions across varying and discrete time periods. Producing time series estimates is necessary for many applications [9] such as geospatial impact evaluations that aim to leverage geospatial data over time to study the impact of development projects and other interventions [37,38,39,40,41,42]. To date, the literature has largely focused on using imagery that is a combination of a broad temporal range, or imagery tied to a single, specific period based on available training data, with limited evaluation of temporal transferability of models [9]. Recent work has indicated that existing approaches with strong cross-sectional performance may not perform as well across multiple points in time [43].

More broadly, a range of existing literature has highlighted the capability of deep learning methods such as CNNs to leverage time series data [44,45]. The Time Series Land Cover Challenge (TiSeLaC) [46] is one effort that engages with these problems and has led to the advancement of time series CNNs for classifying land cover types using satellite imagery [47]. Notably, [48], the winners of the 2017 TiSeLaC, were able to accurately classify multispectral time series Landsat 8 imagery over 99% of the time, using a combination of MLPs (multilayer perceptrons) and CNNs [48]. Related time series crop classification efforts have also led to the development of novel neural architectures [49]. Numerous applications in recent literature provide opportunities to explore whether existing time series approaches for deep learning can be extended to using CNNs to estimate development indicators [20,23,24,26,36]. In particular, recent work demonstrating that for a discrete temporal window, CNNs can detect features from moderate resolution imagery associated with the risk of conflict fatalities is well suited to be expanded as a time series application [35].

A large body of existing work in the conflict prediction space has explored a range of methods, including machine learning, to forecast conflict in varying geographical contexts [50]. Applications of existing methods have often been limited in either their temporal or spatial precision, scope, or reliability [51,52]. While some approaches incorporate explicitly spatial modeling and/or data [53], many use non-spatial inputs such as text analysis from media content and other sources [54]. Several recent efforts using geospatial data have been a part of the Violence Early Warning System’s (ViEWS) prediction competition. The models created have leveraged an array of geospatial data with modeling techniques including logistic regressions, random forests, a variety of neural networks, and other ML algorithms [55,56]. A defining feature of the ViEWS-related efforts is the use of either the country level or coarse (approximately 50 × 50 km) grid cells as the sampling/prediction unit along with predefined features (e.g., demographics, GDP, past conflict data) as input to train models [57,58,59]. Recent work in the conflict prediction space has sought to improve the resolution of predictions, using 25 × 25 km grid cells, along with predefined features from a range of remote sensing data sources such as land cover, population, and crop data [60]. A critical distinction between these approaches and existing imagery-based deep learning efforts is the use of predefined input features and the resolution of the sampling/prediction units [35].

Building on existing applications of CNNs and satellite imagery in order to produce multi temporal predictions of future fatal conflict has the potential to benefit a range of development, security, and other applications that are currently limited by available data. Estimates of the spatiotemporal risk of conflict fatalities—as opposed to only the spatial location of any future conflict events—are particularly valuable for practicable applications including humanitarian and aid efforts, which require determining whether it is safe for personnel to operate in potentially dangerous regions. Models that estimate the risk of conflict fatalities should a conflict event occur in the future could be used by organizations to assess the risk of deploying personnel to those locations. Finer resolution predictions, as well as depending on only multispectral imagery for model inputs, can provide additional operational value over existing approaches which depend on many inputs and use coarse-resolution spatial grids for predictions.

In this piece, we explicitly focus on the degree to which the fatality of potential future conflict events can be forecast using satellite imagery and deep learning. The research builds on previous work to estimate conflict fatality in order to further explore the effectiveness and challenges of detecting precursors to fatal conflict from satellite imagery at multiple discrete points in time. We train CNN models using satellite imagery and conflict event data from both yearly and six-month windows from 2014 to 2019 to address two research questions: (RQ1) Can CNNs detect features from moderate resolution satellite imagery associated with conflict fatality at multiple discrete points in time? And if so, (RQ2) how does model performance vary based on the temporal window size used for training? In doing so, we also aim to contribute to the broader literature by exploring the potential of CNN-based approaches to estimate development indicators using satellite imagery at multiple points in time in order to support practical time series applications.

The paper is structured as follows. Section 2 presents the methods used to prepare Landsat 8 imagery and ACLED data samples for training, as well as the approach for training and validating CNN models. The results of the training and validation are reviewed in Section 3. Finally, Section 4 discusses potential paths forward based on this work and a conclusion.

2. Materials and Methods

To assess whether CNNs can detect features from satellite imagery associated with conflict fatality at multiple discrete points in time, and how model performance varies based on the temporal window size used for training, we train a series of ResNet (residual network) [14] CNNs using imagery from Landsat 8 around ACLED conflict event locations which are labeled based on whether a conflict fatality occurred during the associated event. ResNets are a well-established type of CNN that have been used in similar applications leveraging satellite imagery to estimate socioeconomic factors [26]. Landsat 8 imagery provides sufficient resolution to detail a wide range of features including environmental conditions and land use with broad spatial and temporal coverage including Nigeria during the study window. Finally, ACLED offers extensive coverage of conflict events and related fatality counts that can be used to label imagery [35].

Throughout this paper, predicting conflict fatality refers specifically to determining the risk of a fatality if conflict were to occur at a given location. Individual CNNs are trained for three distinct time periods of conflict event data (2015, 2017, 2019) using: (1) twelve months of imagery from one full calendar year prior to the conflict event year (January–December), (2) six months of imagery starting twelve months prior to the event year (January–June), or (3) six months of imagery starting six months prior to the event year (July–December) (see Figure 1). Testing both yearly and six-month windows will provide insight into both the general effectiveness of temporal models, as well as whether smaller windows (A) provide sufficient data to train models, and (B) can improve models by enabling the detection of seasonal trends.

The resulting models are assessed based on their ability to accurately classify validation data left out from the original training. Finally, the performance of models from each discrete time period using both yearly and six-month windows of data are compared with one another, as well as with a naive baseline conflict prediction approach based only on the location of fatal conflict from the previous year. Each of these steps is detailed further in this section.

2.1. Data Preparation

NASA and USGS have made Landsat 8 imagery free and publicly available since its launch in 2013. Like its predecessor Landsat 7, Landsat 8 imagery captures data for the entire planet every 16 days at 30 m resolution [61]. It consists of eleven spectral bands of imagery, captured using its operational land imager (OLI) and thermal infrared sensor (TIRS) [62] (

Table 1. Landsat 8 Operational Land Imager (OLI) and Thermal Infrared Sensor (TIRS) Bands [35].

Bands	Wavelength (Micrometers)	Resolution (Meters)
Band 1—Coastal aerosol	0.433–0.453	30
Band 2—Blue	0.450–0.515	30
Band 3—Green	0.525–0.600	30
Band 4—Red	0.630–0.680	30
Band 5—Near Infrared (NIR)	0.845–0.885	30
Band 6—Shortwave Infrared (SWIR) 1	1.560–1.660	30
Band 7—Shortwave Infrared (SWIR) 2	2.100–2.300	30
Band 8—Panchromatic	0.500–0.680	15
Band 9—Cirrus	1.360–1.390	30
Band 10—Thermal Infrared (TIRS) 1	10.60–11.20	30 *
Band 11—Thermal Infrared (TIRS) 2	11.50–12.50	30 *

* Collected at 100 m but resampled to 30 m to match OLI bands.

).

Individual scenes of Landsat 8 imagery covering Nigeria are acquired using the EarthExplorer platform and bulk download tools [63,64] for 2014, 2016, and 2018 to be paired with conflict data from 2015, 2017, and 2019. These scenes, acquired at sixteen-day intervals, are subsequently mosaiced and aggregated to six-month and yearly windows for each year by taking the average pixel values for each band. Cloud pixels within individual scenes are masked based on the Landsat quality assessment band, ignoring any pixels marked as medium or high confidence of clouds [65].

In this analysis, we seek to construct three different temporal aggregations of Landsat data: two six-month periods and the twelve-month period associated with the year prior to the detected conflict event. The six-month window aggregations are performed based on the first and second half of each calendar year (i.e., January–June and July–December). The twelve-month window aggregated uses all data from the year preceding the year in which the event took place (i.e., January–December). Of the eleven bands available from Landsat 8, all individual and non overlapping bands are utilized (i.e., the panchromatic band is dropped) to include any potentially relevant spatial features across the detectable spectrum. The result of this imagery preprocessing is an aggregate image of the entirety of Nigeria for each of the ten bands for each year at yearly and six-month windows.

Conflict locations and fatality counts are downloaded using ACLED’s data export tool [3]. The dataset consists of a date, coordinates, and fatality count for each recorded conflict event. This data is aggregated into yearly and six-month windows for 2015, 2017, and 2019 to be paired with Landsat 8 imagery from the previous year. For each event, the conflict fatality count is converted into a binary value indicating whether or not a fatality occurred (1 or 0, respectively). This binary classification is used to label imagery associated with the conflict location during the training and validation of CNN models.

Imagery from 2014, 2016, and 2018 is paired with conflict fatality data from 2015, 2017, and 2019, respectively, at yearly and six-month windows as listed in

Table 2. Temporal windows and associated conflict event counts used for training models.

Imagery Window	Conflict Window	Conflict Event Count (Training/Validation Splits)
2014 all	2015 all	1673 (1422/251)
2014 January–June	2015 January–June	997 (847/150)
2014 July–December	2015 July–December	676 (575/101)
2016 all	2017 all	1644 (1397/247)
2016 January–June	2017 January–June	817 (694/123)
2016 July–December	2017 July–December	827 (703/124)
2018 all	2019 all	2218 (1885/333)
2018 January–June	2019 January–June	1214 (1032/182)
2018 July–December	2019 July–December	1004 (853/151)

. For each time period and temporal window, samples are prepared for training and validation of the CNNs. Individual samples consist of a location (longitude and latitude) based on a conflict event which can be used to load georeferenced imagery at that location, and a label of whether the associated conflict event had a fatality or not. Each conflict event location is treated as a unique sample as conflict event locations are not consistent across time periods, and therefore do not allow time-series sampling of locations.

Imagery for each sample consists of a 244 × 244 × 10 image stack around the sample location (approximately a 7 × 7 km area), representing each of the ten bands of Landsat 8 imagery detailed in Table 1. In the event that the classes within the sample sets are imbalanced (fatal and non-fatal events), the larger class is trimmed to the size of the smaller class (observations from the larger class are randomly dropped until the classes are balanced; on average, fatal events represent approximately 44% of the data). The balanced class samples are then split into separate datasets to be used during the training and validation stages (85% training, 15% validation), and the samples are duplicated nine times to increase model exposure during training (considerations regarding sampling schemes are included in Section 4). Finally, the process of producing the training and validation dataset is repeated five times to ensure the outcomes are reproducible regardless of randomization during sample generation.

2.2. Training

To train CNNs using the sampling data frames generated from Landsat imagery and ACLED conflict data, a ResNet CNN architecture is implemented using PyTorch (v1). This work employs ResNets pre-trained on ImageNet [66] to benefit from transfer learning, which has been shown to be effective in a range of applications within the broader machine learning community including satellite imagery-based application [20,35,67,68,69,70,71,72,73]. To utilize transfer learning, a CNN first learns generalizable features—such as basic lines, shapes, and other patterns from a much larger (although potentially unrelated) set of images such as ImageNet. The pre-trained CNN retains that knowledge and can then be fine-tuned using application-specific data (e.g., satellite imagery with conflict fatality labels). This process allows leveraging CNNs for applications that would not have sufficient data to properly train a newly initialized CNN.

The default ResNet architecture is modified to work with ten bands of imagery (compared to the three bands of red, green, and blue found in common images, including those in ImageNet). Following examples in the literature, the additional bands are initialized using an average of the weights for each of the red, green, and blue bands [20,35]. The final layer of the CNN responsible for classification is then replaced in order to utilize the binary classes identifying whether a conflict event was fatal. Five sample data frames for training and validation are created for each time period and temporal window listed in Table 2 to mitigate the chance that results are influenced by the random generation of an individual sample data frame. For each sample data frame generated, 16 CNNs are trained during a grid search using combinations of the specific ResNet architecture, the learning rate, gamma, and step size as described in

Table 3. Network parameters adjusted during testing, and associated values.

Parameter	Values
Network	Resnet18, Resnet50
Learning Rate	0.0001, 0.00001
Gamma	0.25, 0.5
Step Size	10, 15

.

The values tested for each of the hyperparameters, seen in Table 3, are based on results from earlier work which showed minimal variation in performance across hyperparameters Goodman et al. [35]. Despite the lack of hyperparameter impact in previous work, we include a range of hyperparameter values to assess potential variation in optimal hyperparameters across time periods and temporal windows tested in this paper. For all tests, stochastic gradient descent (SGD) optimization is used over 60 epochs with a batch size of 64. Across all time periods, temporal windows, and hyperparameter combinations a total of 720 CNNs are trained.

2.3. Validation

For each CNN, a set of validation data (15% of total data for each period, see Table 2) is withheld from the training data, and subsequently used to produce predictions. The result of each prediction is the probability of a fatality at the location of a conflict event.

A confusion matrix defining the relationship of all possible outcomes for a binary classifier is generated to assess the performance of the models. In the confusion matrix, actual and predicted values are either positive (a fatality occurred from a conflict event) or negative (no fatality occurred from a conflict event) as seen in

Table 4. Confusion matrix definitions.

Actual	Predicted	Classification
Positive	Positive	true positive (tp)
Positive	Negative	false negative (fp)
Negative	Positive	false positive (fn)
Negative	Negative	true negative (tn)

.

The results from the confusion matrix can be used to produce Receiver Operating Characteristic (ROC) curves, which are a useful visualization to understand model performance [74]. An ROC curve plots the true positive rate on the y-axis and the false positive rate on the x-axis as the threshold for detecting positive cases based on probability is varied from 0 to 1. Using the ROC curve, we then generate the area under the ROC curve (AUC). In an ideal classifier, which has a true positive rate of 1.0 regardless of the false positive rate, the AUC would be 1.0. For a random or “no skill” classifier, the AUC would be 0.5. The value of the AUC for a model can be used to assess overall performance, as well as to identify a threshold value suitable for a specific application, and has been used in other applications of satellite imagery and CNNs within the development community [27]. Identifying suitable threshold values is of particular importance in conflict and security applications given the potential implications of false negative predictions.

To establish a baseline for comparing model accuracy, we generate a naive predictor of conflict fatality risk based only on proximity to the locations of fatal conflict events from the prior year. For each conflict window listed in Table 2 we evaluate the percentage of both fatal and non-fatal events correctly classified using this naive predictor. Four versions of the naive predictor are tested using proximity thresholds of 1 km, 5 km, 7 km, and 15 km. For example, when using the 5 km naive predictor for an event location in 2019, a fatal conflict event would be predicted if a fatal conflict event occurred within 5 km during 2018. Although the threshold distances used are not directly comparable to the imagery window size used in the CNN (approximately 7.3 × 7.3 km), the threshold distances were selected to assess naive performance across a broad range. Smaller proximity distances are more likely to result in better true negative rates (and worse true positive rates) while larger distances will likely result in better true positive rates (and worse true negative rates).

3. Results

Using the methods described in Section 2, 720 CNNs were trained to predict the risk of conflict fatality across three distinct time periods using yearly and six-month windows of imagery and conflict data (Table 2). The models also explored the impact of 16 different CNN hyperparameter combinations (Table 3). To ensure model performance is not dependent on a particular random sample generation, testing included repeating the sample generation process five times. A baseline accuracy comparison was produced using the maximum accuracy of a naive predictor described in Section 2 employed across all time periods and temporal windows. The accuracy results from all CNN models, along with the baseline, can be seen in Figure 2.

Average accuracy across time periods and temporal windows was approximately 70%, with models using yearly windows of data slightly outperforming models using six-month windows overall. Model performance in the 2018–2019 time period was lower as contrasted to the 2014–2015 and 2016–2017 periods. Despite the significant decline in performance of the 2019 models, the minimum accuracy of all CNN models exceeded the 56% baseline accuracy of the naive predictor.

The accuracy of the naive predictor when only classifying fatal events (true positive rate) (

Table 5. Accuracy of naive predictor based on proximity to previous fatal conflict locations when using varying proximity thresholds.

Category		Proximity Threshold
	Year	1 km	5 km	7 km	15 km
Mean % Fatal Correct	2015	0.62	0.67	0.71	0.81
	2017	0.67	0.72	0.74	0.83
	2019	0.52	0.62	0.67	0.82
Mean % Non-Fatal Correct	2015	0.48	0.40	0.38	0.29
	2017	0.36	0.29	0.27	0.21
	2019	0.40	0.34	0.30	0.16

) is only slightly below that of the CNN models’ average accuracy. However, the naive predictor’s accuracy when classifying non-fatal events (true negative rate) falls significantly compared to the CNN (indicating a high false positive rate) and results in worse overall performance. The baseline comparison with the naive predictor provides some confirmation that the CNN is detecting meaningful features from imagery that extend beyond solely the geographic locations of past events or exact features from training data, which allows the models to identify conflict in areas outside of those on which it was trained. The impact of a significant geographic shift in the location of training data on performance is discussed further in Section 4.

Across all tests, no noticeable changes based on hyperparameters were found and no notable changes in hyperparameter performance were associated with specific time periods or data windows. These results build upon findings from earlier work that found fine-tuning of hyperparameters within a generally acceptable range does not have a significant impact on model performance [35]. A notable additional finding is that a broader range between minimum and maximum values was seen across tests when using the six-month windows of data compared to yearly windows.

ROC curves were produced for the best-performing hyperparameter combination across all tests. The optimal hyperparameter combination consisted of ResNet50 CNN architecture, a learning rate of 0.0001, a gamma of 0.25, and a step size of 15. These hyperparameters produced a mean accuracy of 72% across all tests, with a maximum accuracy of 82% using six-month windows of data, and a maximum of 78% using yearly windows. ROC curves and an accompanying AUC metric were produced to gauge the ability of the models to effectively predict true positives without generating excessive false positives. An example of the ROC curves generated is seen in Figure 3 and shows the results based on using 2014 imagery from January–December to predict the 2015 (January–December) probability of a fatality from conflict. In this example, five curves are shown, showing the results from our five randomized permutations of test and training data. All ROC curves are included in Appendix A.3. The average AUC for each temporal pair is shown in

Table 6. Average ROC AUC from models for each time period and temporal window.

Imagery Temporal	Conflict Temporal	AUC ROC
2014 January–December	2015 January–December	0.815
2014 January–June	2015 January–June	0.790
2014 July–December	2015 July–December	0.796
2016 January–December	2017 January–December	0.807
2016 January–June	2017 January–June	0.782
2016 July–December	2017 July–December	0.797
2018 January–December	2019 January–December	0.722
2018 January–June	2019 January–June	0.681
2018 July–December	2019 July–December	0.678

.

AUC values when using the yearly data windows were consistently higher than when using the six-month windows. For both the 2015 and 2017 models, the AUC dropped by 2.5% when moving from yearly to six-month windows. For the 2019 model, the AUC when using a six-month data window dropped by nearly 5%. There was a significant drop in performance based on AUC in 2019 in line with earlier findings when compared to the 2015 and 2017 models. The 2019 model AUC values were approximately 10% less than the corresponding 2015 and 2017 models’ AUC.

Overall, these results indicate that the CNNs are able to leverage sufficient information from satellite imagery to consistently outperform a baseline naive prediction at multiple time periods and temporal windows. Using six-month windows of data can achieve similar performance to using yearly windows of data, but both may be susceptible to degraded performance due to changes in the underlying data based on ground conditions. The shifts in performance seen across the time periods explored during tests—primarily poorer performance in 2019—along with other limitations and potential future research directions are discussed in the following section.

4. Discussion

This paper presented evidence that CNN-based methods are moderately effective at identifying features in satellite imagery associated with the risk of fatalities from conflict events across time periods. These findings contribute to a broader set of literature using CNNs and satellite imagery to predict development indicators at a single year or coarser time periods [20,24,26,32,35].

As a byproduct of testing performance over multiple time periods, this work was able to observe the model’s behavior when the geographic distribution of samples shifted over time. The decrease in performance seen in the models using 2018/19 data—compared to models using data from 2014/15 or 2016/17—was one of the most pronounced changes found throughout testing. As shown in Figure 4, this change in performance was associated with a notable shift in the geographic distribution of conflict locations used for training (an increase in conflict events in the northwest of the country). This reinforces the importance of testing out-of-sample predictions, across both space and time, presented in literature [26], and suggests a need for additional training and validation data to improve the robustness of the models.

Addressing the amount and spatial coverage of samples used for training and validation has been handled in multiple ways in literature exploring CNN-based approaches to predict development indicators. Previous work to produce estimates of poverty using CNNs and satellite imagery incorporated data across multiple geographic extents (countries), and shown varying performance based on which areas were used to train models, as well as which country was being evaluated [20]. Other work has advised caution when extending predictions across geographic areas and also suggests that temporal variation in imagery may impact performance [26]. Given the potential impact of shifting the geographic distribution of locations over time on model performance, applications of these methods should consider the geographic distribution of training and validation data as well as the distribution of locations where the model will be used to generate predictions. The lack of considerable variation in performance across hyperparameters seen during optimization may be one indicator of the generalizability of the model and could facilitate training models on broader geographic areas.

The amount of data available also impacts the approaches that are used to train models. Transfer learning and pre-trained CNNs can be leveraged to reduce the dependence on large training data sets, yet there is still a need for enough data to sufficiently fine-tune pre-trained networks and represent all geographic regions where the model may be used for predictions. For example, insufficient training data may be one of the reasons for a decrease in performance when moving from using a full-year window of data to a six-month window. Across tests, there was a slightly larger range between minimum and maximum accuracy when using six-month windows which may be indicative of the instability of model performance when relying on smaller training sets. The potential ability and value of narrower temporal windows to reflect seasonal trends (by aggregating less underlying data and therefore containing less noise from seasonal variation) may have either been negated by the associated drop in samples, not realized due to seasonal misalignment, or simply not been meaningful when predicting the fatality of conflict events.

One possible solution to help address limitations around sample size is to train a model using a combination of samples across time periods. In addition to increasing geographic coverage and sample size, combining data across time periods would enable exploring the model’s ability to predict conflict fatality beyond the temporal range on which they were trained. Related to this, the specific seasonality of sub-annual time periods used for training could impact model performance and would require further research to evaluate. The extensibility of models to time periods beyond those on which they were trained would be necessary to support many real-world applications (i.e., using historical data to predict future conflict fatality risk). While the current approach could be used to gain general insight into extensibility, the lack of repeat observations across time periods does limit our ability to gauge the performance of a time series of estimates (i.e., using true panel data). Similarly, the current work provides little explanatory power as to the drivers of conflict/fatalities, yet may be able to contribute to broader literature exploring the nature of conflict through further research into the underlying features in satellite imagery associated with conflict fatalities.

Another aspect of real-world applications not explored in this paper is the spatial scale of predictions. This paper focuses on predictions and validation in terms of known conflict events at specific locations. Future estimates would likely be made in the form of either a continuous surface or a grid across an entire region. In these scenarios, the resolution of the surface or distance between grid points used would play a significant role in the accuracy of predictions. Given possible concerns with the accuracy of the underlying georeferencing of conflict events datasets [75], and that a single coordinate is often a simplification of the extent of a conflict event, it may also be practical to aggregate predictions to relevant local units such as administrative zones.

As methods for predicting conflict and fatalities progress, applications may aim to generate predictions for narrower windows of time (e.g., three-month windows) or further examine the existence of seasonal trends. Using the presented methodology may result in issues of (a) producing cloud-free aggregates of satellite imagery without missing data and (b) suffering reduced performance due to limitations regarding the temporal precision of recorded conflict events. Despite the sixteen-day revisit time of Landsat 8 imagery, not all images result in usable scenes due to cloud cover or other quality issues. As a result, it is necessary to have a sufficient temporal window over which to aggregate scenes and achieve full coverage. A possible solution to this problem would be to aggregate a longer period of imagery than the prediction interval. For example, a model could be trained to use a six-month window of imagery to produce predictions for a three-month window of conflict fatality. Limitations based on the temporal precision of the underlying conflict data used for training would be more difficult to overcome. Potential solutions include identifying new sources of data with improved temporal precision or working with data providers to improve data collection. In addition, while the arbitrary six-month windows did not benefit from seasonal trends in imagery, they also only showed minimal decreases in performance and it is possible more specific windows (e.g., harvest seasons, dry season) could produce better results.

In addition to the areas of future research discussed above, multiple opportunities exist for improving the existing methodology based on progress within the broader machine learning community. Examples of pathways towards improving model performance include the use of ensemble models [76,77,78], 3D convolutional layers [79,80], and other CNN architectures such as ResNeXt [81]. Applying transformations such as rotating or flipping the input satellite image are other viable means of increasing exposure to data variations when using limited training data [82,83]. These methods have been used widely and may offer an improvement of the basic sampling scheme presented in this paper [84].

Similarly, opportunities may exist to leverage alternative sources of data for imagery and conflict records. Finer resolution imagery products such as Sentinel 2 (European Space Agency) or PlanetScope (Planet Labs) may support detecting features associated with conflict that cannot be detected from Landsat 8 imagery. In addition, improvements to conflict datasets to produce more precise location records (or utilizing only subsets of datasets such as ACLED that were recorded at precise levels) may enable more effective training of deep learning models.

5. Conclusions

The work presented in this paper sought to address two research questions: (RQ1) Can CNNs detect features from moderate resolution satellite imagery associated with conflict fatality at multiple discrete points in time? And if so, (RQ2) how does model performance vary based on the temporal window size used for training? Using Landsat 8 imagery and ACLED conflict data, this paper showed that models trained using six-month and yearly windows of data were able to achieve comparable performance at multiple points in time. While predictive ability varied, overall metrics of both accuracy as well as ROC area under the curve both reached approximately 75% on average and outperformed a naive baseline predictor. These results advance existing work in the field by providing evidence that features in moderate-resolution satellite imagery that are associated with conflict fatalities can be detected using CNNs at multiple points in time, as well as when using varying windows of data. Combined, these findings provide a pathway forward for producing future estimates of the risk of conflict fatalities that may be useful in a variety of development and security applications.