Machine Learning Methods for Quantifying Uncertainty in Prospectivity Mapping of Magmatic-Hydrothermal Gold Deposits: A Case Study from Juruena Mineral Province, Northern Mato Grosso, Brazil

Abstract

Mineral prospectivity mapping (MPM), like other geoscience fields, is subject to a variety of uncertainties. When data about unfavorable sites to find deposits (i.e., drill intersections to barren rocks) is lacking in MPM using machine learning (ML) methods, the synthetic generation of negative datasets is required. As a result, techniques for selecting point locations to represent negative examples must be employed. Several approaches have been proposed in the past; however, one can never be certain that the points chosen are true negatives or, at the very least, optimal for training. As a consequence, methodologies that account for the uncertainty of the generation of negative datasets in MPM are needed. In this paper, we compare two criteria for selecting negative examples and quantify the uncertainty associated with this process by generating 400 potential maps for each of the three ML methods utilized (200 maps for each criterion), which include random forest (RF), support vector machine (SVM), and k-nearest neighbors (KNC). The results showed that applying a geological constraint to the creation of negative datasets reduced prediction uncertainty and improved overall model performance but produced larger areas of very high probability (i.e., >0.9) when compared to using only the spatial distribution of known deposits and occurrences as a constraint. SHAP values were used to find approximations for the importance of features in nonlinear methods, and kernel density estimations were used to examine how they varied depending on the negative dataset used to train the ML models. Prospectivity models for magmatic-hydrothermal gold deposits were generated using data from the shuttle radar terrain mission, gamma-ray, magnetic lineaments, and proximity to dykes. The Juruena Mineral Province, situated in Northern Mato Grosso, Brazil, represented the case study for this work.

1. Introduction

The goal of mineral prospectivity mapping (MPM) is to define favorable areas for the discovery of new deposits. It is a multi-criteria process that can be guided by geological knowledge, i.e., understanding of the responsible mechanisms for ore-formation (knowledge-driven), geoscientific data that can serve as a signature or represent an ore-forming setting, capable of driving the search for new potential deposit areas (data-driven), or a hybrid approach (knowledge- and data-driven) [1]. Traditionally, the delineation of mineral exploration targets is conducted, for example, through the analysis of evidential layers in the Geographic Information System (GIS). Nevertheless, manual data processing is affected by the limited capacity of the human brain to process multiple variables at the same time. Consequently, this leads to a lack of reproducibility, depending only on the assumption made by experts. Recently, Machine Learning (ML) algorithms have recently been used to overcome these limitations by automatically mapping relationships in n-dimensional datasets. When used to find the underlying models by learning relationships from evidential layers to classify probable mineralized locations, supervised learning methods may lead to more accurate prospective zone delimitation [2,3]. In MPM, supervised approaches consist of dichotomous problems in which models must be provided with two classes in order to classify whether a location is prospective or not: (i) a class of interest (e.g., known deposit locations); (ii) a class of contrast (e.g., non-deposit locations). Nonetheless, deposits are the result of rare events and non-random ore-forming processes, whereas non-deposits are the result of common and random geological processes, i.e., an undefined class with unknown locations [4].

Many authors have studied the distribution and behavior of spatial phenomena using techniques such as Fry analysis [5], fractal analysis [6], and point pattern analysis [7]. Deposits are point entities at regional or district scales, and their spatial distribution can be studied using such techniques, which contribute to a better understanding of the geological features that control their occurrence and geometry [4]. Following that, the spatial analysis of a class of interest (i.e., deposit locations or positive examples) may support the selection of contrasting examples (i.e., non-deposit locations or negative examples). However, there is still some uncertainty in this process because even when studying spatial patterns, it is difficult to map all scales at which mineralization controls operate. As a result, one can never guarantee that negative training datasets generated using random point locations constrained by geological or spatial distribution criteria represent true negatives (i.e., barren sites).

Previous work introduced a few methods for performing negative training sampling in MPM. One could, for example, use the spatial distribution of deposits to define areas where new deposits are unlikely to be discovered [8] and sample negative examples at ‘unfavorable’ lithologic units or drilling sites that appear barren [9]. Following that, ref. [10] investigated the effects of random negative sampling by comparing the potential maps generated by 50 negative training datasets produced from random point locations.

In this paper, we developed a machine learning-based framework for quantifying uncertainty in gold prospectivity mapping. Hundreds of potential maps were generated during the process using three widely used ML methods, including random forest (RF), support vector machine (SVM), and k-nearest neighbors classifier (KNC), to delimit prospective zones and quantify the uncertainty associated with negative random sampling. For each ML method, we generated 400 negative training datasets using random locations constrained by spatial, as well as spatial and geological, criteria. This methodology was applied to a real case study in Juruena Mineral Province, northern Mato Grosso State, Brazil.

2. Geological Setting of the Juruena Mineral Province

2.1. Regional Geology and Tectonics

The Juruena Mineral Province (JMP) is a gold-(polymetallic) region located in the south-central portion of the Amazonian Craton (AC). The AC is a tectonically stable area of approximately 4.5 million km${}^2$ composed of two Precambrian shields (Guiana and Central Brazil), which are separated in its central area by the Phanerozoic covers of the Solimões and Amazon basins [11,12]. Early studies divided the region into six major geochronological provinces: Central Amazonian (>2.3 Ga), Maroni-Itacaiúnas (1.95–1.80 Ga), Ventuari-Tapajós (1.95–1.80 Ga), Rio Negro-Juruena (1.8–1.55 Ga), Rondonian-San Ignácio (1.55–1.3 Ga), and Sunsás (1.3–1.0 Ga) [13]. These provinces evolved due to a sequence of continental accretion events characterized by a succession of magmatic arcs. Lately, Refs. [13,14] associated the origin of Sunsás and some parts of Rondonian-San Ignácio and Maroni-Itacaiúnas provinces with continental collisional events.

Santos et al. [15,16], propose seven tectonic provinces, including Carajás (3.0–2.5 Ga), Amazônia Central (Archean), Transamazonas (2.26–2.01 Ga), Tapajós-Parima (2.03–1.88 Ga), Rio Negro-Juruena (1.82–1.52 Ga), Rondônia-Juruena (1.82–1.4 Ga), and Sunsás-k’Mudku (1.45–1.10 Ga). Juliani et al. [17] divide the south portion of the AC into two Paleoproterozoic magmatic arcs along the E-W direction.

According to [18], the JMP has been tectonically juxtaposed to the Tapajós Mineral Province (1.69 to 1.63 Ga) through a roughly E-W-trending shear zone. The combination of new results and observations gathered by [19], together with pre-existing data from the literature, on the plutonic and volcanic rocks of the eastern portion of the JMP indicate long-lived magmatism in the southern part of the AC between 2037 and 1757 Ma. The region formed through tectonic switching is represented by alternating periods of crustal thickening during flat subduction and crustal extension during slab-rollback at a single active convergent margin of an accretionary orogen [19].

2.2. The Juruena Mineral Province and Gold Mineralization

The JMP comprises a belt approximately 500 km long by 100 km wide constituted of Peixoto de Azevedo Domain (2.8–1.86 Ga), São Marcelo-Cabeça Group (1.85 Ga), Juruena Supersuite (1.81–1.75 Ga), Nova Monte Verde Complex (1.81–1.76 Ga), Colíder Group (1.81–1.76 Ga), Teles Pires suite (1.79–1.75 Ga), Roosevelt Group (1.76–1.74 Ga), Caiabis Group (1.98–1.37 Ga), and Alto Tapajós basin (0.30 Ga). The Peixoto de Azevedo Domain represents a remaining cratonic block of the southern part of the Ventuari-Tapajós Province (VTP), a continuation of the VTP to the south of the Alto Tapajós basin [20].

The geological framework of the eastern JMP is divided into five major lithostratigraphic units and has been simplified from [21] according to new proposals by [20], into (Figure 1): (i) 2.86–1.78 Ga Peixoto de Azevedo Domain; (ii) 1.82–1.78 Ga Colíder Group; (iii) 1.78–1.75 Ga Teles Pires suite; (iv) unconstrained rocks; and (v) 1.98–1.37 Ga Caibis Group, and Phanerozoic sedimentary cover sequences. The rocks in the province are mostly represented by Paleoproterozoic oxidized calc-alkaline, medium-to high-K, peraluminous to metaluminous I-type granites, volcanic and volcano-sedimentary sequences, with the presence of subordinate later A-type granites and volcanic rocks. Spatial analysis indicates a close relationship between the primary gold occurrences and I-type granites (calc-alkaline and oxidized), volcanic rocks, and volcano-sedimentary sequences that originated in continental magmatic arcs. Studies investigating hydrothermal alteration, styles, ore paragenesis, fluid inclusion, and isotopic data suggest the formation of the deposits in Paleoproterozoic magmatic-hydrothermal systems distributed in different crustal levels and positions [22].

Gold deposits in the JMP are mostly concentrated along with a set of WNW-ESE shear zones (so-called Peru-Trairão) and correspond to Cenozoic placers, eluvial deposits, and hydrothermal occurrences [23,24]. From 1980 to 1999, the region produced over 160 t of gold, mostly related to secondary ore sources. The region is currently focusing on primary types as part of a new exploration cycle. The primary gold mineralization was summarized based on the mineralization style and paragenesis of ore bodies as [25]: (i) veins, stockworks, and disseminated in shallow-emplaced granites; (ii) veins hosted in faults that crosscut granitoids; and (iii) epithermal gold-(polymetallic) veins in porphyries, with a more limited occurrence in volcanic and volcaniclastic rocks. The majority of gold occurrences in the JMP are associated with aplite and intermediate to mafic volcanic dykes [22].

3. Materials and Methods

3.1. Feature Selection and Feature Engineering

We used airborne and geological data, freely available in the database of the Geological Survey of Brazil (GeoSGB-CPRM), and shuttle radar terrain mission (SRTM) data from the Earth Explorer platform of United States Geological Survey (USGS). The geological understanding of the magmatic-hydrothermal gold systems present in the JMP guided feature selection and engineering. The data were used to derive representations of source, transport, and alteration, which are key factors for ore-forming processes in the province.

The airborne data, i.e., gamma-ray, belong to Project 1121—Norte do Mato Grosso, coordinated by the Geological Survey of Brazil (CPRM) from 2012 to 2014. The survey consists of north-south lines spaced at 500 m, tie-lines separated by 10,000 m, terrain clearance of 100 m, and data collected each 8 (magnetic field) and 80 m (gamma-ray spectrometry), approximately. The gamma-ray data were used to compute the ratios between eTh/K and eU/K. This procedure allows highlighting possible potassic alteration or sericitized sites, commonly associated with the gold mineralization in the JMP [26,27]. Thus, the ML algorithms are expected to search for low ratio values due to the differential mobility between these three elements (K > Th and K > U) [28]. We created three maps to represent potential transport paths and sources for hydrothermal fluids in the region by buffering magnetic lineaments and dykes. Since the deposits are primarily found in lowland zones within the study area, the digital elevation model was left unchanged.

All of the features were re-interpolated to an identical regular grid with cell sizes of 125 m (Figure 2). To oversample and undersample the gamma-ray and elevation data, the linear and nearest neighbors methods were used, respectively. All the analyses were run using QGIS and Python Language.

3.2. Negative Sampling Methodology

In geoscientific problems, model validation presents issues due to spatial autocorrelation. That happens because the data violate the assumption of independence of most standard statistical procedures once natural phenomena create spatial continuity of physical/chemical properties [29,30]. Given this, to avoid over-optimistic results, it is preferable to split the data into training and test sets according to geographic regions. In this paper, the training and test sets are composed only of samples within the north and south portions of the study area, respectively.

Two sampling constraints were used to ensure that negative datasets were created using random point locations known as ‘unfavorables’ for deposits of the type sought: (i) point pattern analysis (PPA) was used to determine how close the deposits are to one another in space. This allows one to determine how far away from known deposits the likelihood of discovering a new one is sufficiently low and thus set the minimum distance between positive and negative examples (MDPN). In our case, an MDPN of 9.92 km was set according to the major breaking point before the sill present in the graph in Figure 3, which corresponds to a likelihood of $3.45\%$ of finding a new deposit; (ii) the lithologic unit favorability is a metric that represents the strength of the spatial association (SA) between rock units and known deposits. The SA is calculated as the proportion of a unit’s area covered by known deposits divided by the proportion of the study area occupied by the same unit [1] (Figure 4). It represents how likely a lithologic unit is to host magmatic-hydrothermal gold deposits in this context and could be used as another evidential feature. For this study, however, we used this metric as the second constraint for creating negative datasets. As a result, negative examples were chosen only in locations where the SA is zero.

An ML-based framework was used to evaluate how these criteria influence the uncertainty of the predictions. The following steps comprise this framework: (1) create the negative dataset using the constraint(s); (2) merge it with the positive dataset created using the positive examples available in the study area (i.e., deposits and/or occurrences); (3) divide the new dataset into training and testing sets; (4) use the training set for training the model(s); (5) generate the potential map of the test area; (6) repeat the process n-times, storing the predictions and metrics for each realization; and (7) calculate the mean and standard deviation of the potential maps. This procedure was used to generate the negative datasets (Figure 5). Each model type resulted in the generation of 400 potential maps.

3.3. Model Training

The models were trained using the known magmatic-hydrothermal gold deposits and occurrences found in the records of the GeoSGB-CPRM. In the study area, there are 11 deposits, of which 6 belong to the training area and 5 to the test area. Of all mineral occurrences, 21 are in the training area, and 27 are in the test area. The negative datasets contain the same number of samples as the sum of positive examples in the correspondent areas, i.e., the ML models were trained with 32 positive and 32 negative examples.

A random search algorithm was used to define the optimal hyperparameters [31]. This algorithm performs a random search for the best hyperparameters in a parameter space defined by the user. It also performs cross-validation on batches of the training set. We used 100 iterations and 5 folds to determine the best hyperparameters for the three model types (

Table 1. Optimal hyperparameters defined for the two adopted criteria using random search.

Model		Hyperparameters
	n-estimators	bootstrap	criterion	max-depth
RF	300	False	gini	5
	kernel	gamma	C
SVM	RBF	0.5	0.25
	n-neighbors	weights	algorithm	p
KNC	7	distance	auto	2

).

3.4. Algorithms

3.4.1. Random Forest

Random forest (RF) consists of a combination of tree predictors that depend on values of a random vector sampled independently and with the same distribution for all trees in the forest [32]. It is an ensemble machine learning method that combines multiple decision trees, realizing repeated predictions of the same phenomenon represented by the training dataset [1].

In ensemble methods, for the kth tree, a random vector ${\mathsf{\Theta }}_k$ is generated, independently of the past random vectors ${\mathsf{\Theta }}_1$, …, ${\mathsf{\Theta }}_{k-1}$, but presenting the same distribution. A tree grows using the training set and ${\mathsf{\Theta }}_k$, resulting in a classifier $h(x,{\mathsf{\Theta }}_k)$ where x is an input vector. In dichotomous problems, as the classifications in MPM, the model iteratively splits the dependent variable (root mode) into binary pieces (leaf nodes) in every decision tree. The trees then search along all the splits in order to find the best one that maximizes the purity of the resulting trees [33].

There are many approximations for measuring impurity. The most frequently used ones are entropy, gain-ratio, Gini index, and chi-square. The entropy, for example, attempts to maximize the mutual information, constructing an equal probability node in the decision (Equation (1)):

$$Entropy=-\sum _j{p}_j{log}_2{p}_j;$$

(1)

While the classification error can be measured as (Equation (2)):

$$ClassificationError=1-max{p}_j;$$

(2)

where $p_j$ is the probability of class j.

3.4.2. Support Vector Machine

Support vector machines (SVMs) are supervised learning methods used to model datasets in classification and regression problems. The algorithm initially seeks to construct a linear classifier to separate different classes with the widest decision boundaries. In more complex cases, in which the origin feature spaces are not linearly separable, the input data have to be transformed into a higher n-dimensional feature space through various kernel functions, where a classification problem can be performed by an (n − 1) dimensional plane referred to as a hyperplane. A soft margin kind is utilized to reduce the misclassification [34].

For linear problems, in a hyperplane decision function $f\left(x\right)=sgn\left(\right(w·x)+b)$, the Vapnik–Chervonenkis (VC) dimension is controlled by the norm of the weight vector w, and given a training set $(x_1,y_1),\dots ,(x_l,y_l)$, with $x_i\in {\mathbb{R}}^n$, and $y_i\in \pm 1$, a border/margin that better separates classes can be found by minimizing ${\displaystyle \frac{1}{2}}{\parallel w\parallel }^2\mathrm{subject}\mathrm{to}{y}_i·((x_i·w)+b)\ge 1\mathrm{for}i=1,\dots ,l,$ [35].

In nonlinear problems, for a training dataset defined as ${\left\{x_i\right\}}_{i=1}^n$ with n feature vectors, a labeled target dataset ${\left\{y_i\right\}}_{i=1}^n$ is associated with each vector $x_i$. The labels in this study, are $y=1$, and $y=0$, indicating, respectively, deposits and non-deposits. As the input data cannot be linearly separated in the original feature space, they are mapped into a higher-dimensional space H by a mapping function $\mathsf{\Phi }:{\mathbb{R}}^n\to H$ [36].

There are four popular kernel functions used in SVM algorithms, which include linear, radial basis, polynomial, and sigmoid functions. For geoscientific problems, the radial basis function (RBF) is considered the most suitable due to its simplicity and adaptability [37]. The RBF can be described as (Equation (3)):

$$K(x_i,x_j)=exp(-\gamma \parallel x_i-x_j{\parallel }^2),\gamma >0,$$

(3)

in which $\gamma$ determines the width of the RBF.

3.4.3. K-Nearest Neighbors

K-nearest neighbors (KNN) is one of the most fundamental and simple classification methods. It aims to perform discriminant analysis when reliable parametric estimates of probability densities are unknown or difficult to determine. The KNN classifier is commonly based on the Euclidean distance, seeking to segment and find patterns by learning from a training set $(x_{i1},x_{i2},\dots ,x_{ip})$, with n being the total number of input samples $(i=1,2,\dots ,n)$, and p the total number of features $(j=1,2,\dots ,p)$ [38]. The Euclidean distance between sample $x_i$ and $x_l(l=1,2,\dots ,n)$ is defined as in Equation (4).

$$d(x_i,x_l)=\sqrt{{(x_{i1}-x_{l1})}^2+\cdots +{(x_{ip}+x_{lp})}^2}.$$

(4)

Classification typically involves partitioning samples into training and testing categories. Consider $x_i$ and x as the training and test sample, respectively, and let $\omega$ be the true class of a training set and $\widehat{\omega }$ be the predicted class for a test set $(\omega ,\widehat{\omega }=1,2,\dots ,\mathsf{\Omega })$, where $\mathsf{\Omega }$ is the total number of classes. During the training process, only the true class $\omega$ is used. When testing, the model predicts $\widehat{\omega }$ of each sample. For $K=1$, the predicted class of test sample x is set equal to the class $\omega$ of its nearest neighbor, where $m_i$ is a nearest neighbor to x if Equation (5) is true.

$$d(m_i,x)=mi{n}_j\left\{d(m_j,x)\right\}.$$

(5)

Generalizing $\forall K\in {\mathbb{Z}}_{+}^{\ast }$, the predicted class of test sample x is set equal to the most frequent true class among K nearest training samples. This forms the decision rule $D:x\to \widehat{\omega }$ [38].

4. Results and Discussion

Two hundred potential maps were generated for each of the two aforementioned negative dataset criteria. Following that, we created six maps that represented the average predicted prospectivities for each model type and criterion (Figure 6). Additionally, the standard deviation was calculated for each of these scenarios in order to create maps that represented the inconsistency of the predictions in the test area locations (Figure 7).

The maps generated using the second criterion show optimistic results compared to those obtained using the first criterion. It is demonstrated by the increase in the area associated with very high probabilities (i.e., >0.9): when adopting the first criterion, the RF, SVM, and KNC models predicted very high probabilities in $15.2\%$, $8.2\%$, and $21.4\%$ of the test area; whereas adopting the second criterion the RF, SVM, and KNC models predicted very high probabilities in $17.7\%$, $16.6\%$, and $26.4\%$ of the test area, respectively. On the other hand, there was a reduction in the predictions’ average standard deviation associated with deposit and occurrence locations (Figure 8).

The AUC scores in the three ML models were stored at each realization in order to analyze their performances in all scenarios. This way, graphs exhibiting the models’ reliability could be generated (Figure 9). These graphs display the cumulative mean and standard deviation of the AUC scores along with the realizations. They also show the effect of the criteria used to generate the negative training datasets on the general model’s performance. The standard deviations of the AUCs, for example, converge from 125 realizations in both cases; however, for models in which both the spatial distribution of deposits and favorability of lithologic units were used to constrain the selection of negative examples, the overall performances tend to be more stable, with standard deviations ranging between 0.037 and 0.052 as opposed to 0.044 and 0.063 in the first case. Furthermore, it appears that using geological knowledge to limit the space of possibilities for negative examples leads to a significant increase in performance. The cumulative AUCs in the first case ranged from 0.63 (for SVM models) to 0.77 (for KNC models), whereas in the second case, we achieved scores ranging from 0.77 (for RF models) to 0.86. (for KNC models). This is reflected in a reduction of positive example misclassification. The mode of the confusion matrices shows that when both criteria were used, most RF and KNC models correctly predicted all deposit locations (Figure 10).

Effective prospectivity maps require models with reasonable input, i.e., proxies representing some or all of the mechanisms responsible for ore formation. Obtaining reliable and interpretable outputs, however, is just as important as providing appropriate input information to models. Although, due to the ‘black-box’ nature of machine learning modeling processes, it can be a difficult task. The feature importance is used to interpret the results of ML models. However, when using nonlinear methods, a simpler approximation is required in order to explain individual predictions from ML models. For this reason, we calculated SHAP values to approximate the feature importance of the SVM and KNC models [39]. The kernel density estimations (KDE) of the ‘feature importance’ of all models—trained using all negative training datasets generated—could thus be computed (Figure 11). The KDEs demonstrate that the three ML methods approximate linear solutions that are similar. Besides that, it is interesting to see how much the individual importance of the features varies depending on the negative dataset provided. This means that the models can map to the same solution in various ways. Thus, by running multiple models, one can obtain the average (sometimes converging) outcome and its associated uncertainty. In this case, we can see that the elevation and proximity to dykes are the two most important features, regardless of the criterion used. It demonstrates that, while the overall performance and stability of the models varied, the geological interpretation for the found solutions is the same for both negative example generation criteria.

5. Conclusions

This study examined the effects of negative dataset generation in MPM when two different criteria were used to limit the selection of negative examples: (1) negative examples chosen outside the buffering of positive examples; (2) negative examples chosen in locations both outside the buffering of positive examples and without any spatial association to gold deposits. During the process, 200 potential maps for three ML methods were generated, including random forest, support vector machine, and k-nearest neighbors classifier. We were able to quantify the models’ uncertainty by calculating the mean and standard deviation of the potential maps, which revealed where the models’ predictions are more stable. Furthermore, we were able to see that, while using the second criterion produced more optimistic results, it reduced the standard deviation of predictions in deposit and occurrence locations. Besides that, when we added the favorability of lithologic units to the constraints, the number of false negatives in the RF and KNC predictions decreased to zero. The features had similar importance from one criterion to another, implying that the outcomes have similar interpretations or physical meaning. The elevation and proximity to dykes were the two most important features in this study. The Juruena Mineral Province in Northern Mato Grosso, Brazil, was used as a real case study for this research.