AKL-ABC: An Automatic Approximate Bayesian Computation Approach Based on Kernel Learning

Abstract

Bayesian statistical inference under unknown or hard to asses likelihood functions is a very challenging task. Currently, approximate Bayesian computation (ABC) techniques have emerged as a widely used set of likelihood-free methods. A vast number of ABC-based approaches have appeared in the literature; however, they all share a hard dependence on free parameters selection, demanding expensive tuning procedures. In this paper, we introduce an automatic kernel learning-based ABC approach, termed AKL-ABC, to automatically compute posterior estimations from a weighting-based inference. To reach this goal, we propose a kernel learning stage to code similarities between simulation and parameter spaces using a centered kernel alignment (CKA) that is automated via an Information theoretic learning approach. Besides, a local neighborhood selection (LNS) algorithm is used to highlight local dependencies over simulations relying on graph theory. Attained results on synthetic and real-world datasets show our approach is a quite competitive method compared to other non-automatic state-of-the-art ABC techniques.

1. Introduction

Statistical inference aims to infer a set of model parameters using measured data that comes from a system under a particular scenario [1]. Usually, this kind of task is defiance due to noise present in data after the measurement stage [2]. In this sense, two main general approaches are commonly used in the state-of-the-art to proceed with the inference [3]: (i) The frequentist approach, where inference should give right answers in repeated use under an unconditional perspective that tends to focus more on analysis than on methods (e.g., convergence rates, consistency) [4]; in a frequentist approach, procedures (rule decisions) can come from anywhere, so they do not have to be explicitly derived from a probability model. (ii) The Bayesian approach, where inferences should be made conditioned on the current data under an expert-based perspective that, from prior information about the studied phenomenon, determines a probability density function for the model parameters via the Bayes’ theorem [5]; such a density, known as posterior, not only allows model checking and validation, predictive inference, and decision making but also it can tackle point and interval estimation [6].

Bayesian statistical inference tasks require to compute the likelihood function, which states how likely particular values of some statistical parameters are for a given set of observed data [1]. These approaches leverage the inclusion of a priori knowledge about the studied phenomenon into the posterior distribution. Indeed, straightforward models gather an analytic expression for the likelihood function facilitating the evidence assessment; then, the posterior can be precisely computed. Notwithstanding, for complex models, such as those that assemble high non-linearities or stochastic behavior, the model complexity means that there is no analytical formula for the likelihood function or that it is computationally intractable and can not be evaluated in any practical amount of time, standing for a really challenging scenario to perform statistical inference using Bayesian techniques [7].

Approximate Bayesian computation (ABC) emerged like a free-likelihood method to deal with the intractability mentioned above. It was originally introduced as a solution for performing statistical inference in the field of molecular biology. The first ABC algorithm was proposed to study the demographic history of the Y chromosome [8]. However, the use of ABC techniques has influenced several research areas like systems biology [9], climate analysis [10], ecological modeling [11], nuclear imaging [12], and astronomy [13], just to mention some of them. Fundamentally, an ABC method assesses an auxiliary model with different parameter values drawn from a prior distribution to calculate simulations that are compared to the observed data [14,15]. Mainly, in the face of a large number of features and observations, different authors use statistical parameters to summarize and characterize the data [16,17,18,19]. However, the selection of proper and sufficient summary statistics could be difficult for complex models. This fact has led to the need to explore alternative approaches that rely on kernel functions to embed and compare distributions into a reproducing kernel hilbert space (RKHS) [20,21]. Nonetheless, the techniques mentioned above require the estimation of different parameters related to the similarity computation among simulations to approximate the posterior. Then, expensive tuning procedures such as grid search and cross-validation are carried out. Moreover, the user requires a vast knowledge concerning the ABC algorithm and the studied data to properly tune the free parameters, yielding to a high influence in the posterior approximation quality.

In this paper, we introduce an automatic ABC algorithm, termed automatic kernel learning ABC (AKL-ABC), which comprises a kernel learning stage based on a centered kernel alignment (CKA) technique to assess the matching between similarities defined over parameter and simulation spaces in ABC [22]. This paper is an extension of our proposal presented in [23]. Namely, here we provide a series of improvements and contributions:

• We propose a novel automatic ABC approach for computing posterior estimates avoiding any tuning procedure of free parameters. In detail, a Mahalanobis distance is optimized through a CKA-based algorithm to code the simulation and parameter space matching and an information theoretic learning (ITL)-based method to learn the kernel bandwidth. Furthermore, a graph representation is carried out to highlight local dependencies utilizing a local neighborhood selection (LNS).
• The mathematical models regarding AKL-ABC are described and enhanced (including the CKA and LNS coupling with ABC through kernel machines and graph theory).
• The experiments are expanded and explained in detail considering well-known challenging databases.
• A free parameter analysis is provided to show the performance of our AKL-ABC as an automatic approximate inference method.

Achieved results on synthetic and real-world inference problems demonstrate that our AKL-ABC is robust to substantial changes in data dynamics. Namely, the experimental results show that our automatic extension of ABC is competitive with other state-of-the-art works, and has a significant advantage concerning the automatic selection of free parameters. Additionally, a MATLAB implementation of our approach is publicly released.

The remainder of this paper is organized as follows: Section 2 presents the related work. Section 3 introduces the materials and methods and provides the mathematical foundations behind our proposal. Section 4 describes the experimental setup, and the obtained results are discussed in  Section 5. Finally, the conclusions are outlined in Section 6.

2. Related Work

Recent progress in ABC-based inference has incorporated different research areas into the principal ABC framework to accomplish more accurate posterior estimations. While several ABC methods can be found across the literature, three main groups can be highlighted according to their main features [24]: Summary statistics-based approaches, weighting-based techniques, and regression-adjustment methods.

2.1. Summary Statistics-Based Approaches

The selection of proper and sufficient summary statistics is a crucial issue in ABC [25]. Wood [18] introduced a synthetic likelihood modeled as a multivariate normal whose mean and covariance are determined from summary statistics. The Gaussian likelihood assumption allows Markov chain Monte Carlo (MCMC) sampling through a rejection kernel. However, the unbiased estimates of the mean and covariance lead to unbiased posterior estimates; moreover, the MCMC sampler requires the tuning of free parameters. Fearnhead and Prangle [16] proved that the optimal summary statistic for inferring the model parameters, under the assumption of the quadratic loss function, is the true posterior mean given the observed data. They used this fact to model such a posterior mean as a linear model that learns summary statistics directly from data, which is further used in standard ABC. Nonetheless, their semi-automatic approach still has the problem of free parameter selection. Indirect inference, traditionally used in the context of maximum likelihood estimation (MLE), has also been introduced in the context of ABC. Gleim and Pigorsch [26] proposed to use a score vector of an auxiliary model fitted via MLE as summary statistics. Then, comparing the score of the model equipped with the observed data against scores fitted with simulations provides an idea of candidates that follows the posterior distribution. Again, this approach focuses on determining summary statistics rather than an overall automatic ABC.

2.2. Weighting-Based Techniques

Comparing summary statistics, to accept or reject candidates depends on thresholds that must be adjusted. An alternative approach introduced by Nakagome et al. [27] stands for a weighting-based inference where all posterior candidates are weighted rather than accepted or rejected. In particular, a conditional mean embedding operator sets up a mapping from summary statistics to model parameters. Such a notion of similarity (assessed in an RKHS) allows setting a probability for the input candidates. While the ABC procedure works in a completely different way, the selection of summary statistics and kernel-free parameters remains. Attempting to avoid the need for summary statistics, Park et al. [20] introduced the maximum mean discrepancy (MMD) criterion to compare probability measures. Using the kernel embedding of distributions into an RKHS, the MMD provides a measure between the distribution of simulated and observed data. It allows assigning a weight to each candidate, using a similarity kernel. The absence of summary statistics comes at the expense of a high computational load. While the authors provided faster approximations for the MMD computation, the need for free parameter selection is not eliminated.

2.3. Regression-Adjustment Methods

Different ABC techniques have been proposed for understanding regression as a fundamental concept in machine learning. Mitrovic et al. [28] modeled the functional relationship between simulations and the optimal choice of summary statistics to encode the structure of a generative model. While their flexible framework regulates the kind and amount of information extracted from data, the optimal construction of summary statistics requires multiple sets of free parameters. So, expensive and problem-dependent tuning procedures are yielded. Regarding the number of particles in ABC, Meeds and Welling [29] developed a surrogate model that works as an artificial likelihood to define an adequate amount of simulations in ABC via Gaussian process-based regression. Though the number of required simulations to produce posterior estimations is reduced, the significant difficulty resides in the hyper-parameter tuning. Neural networks and deep learning also have been used to model the relationship between parameter values and summary statistics. Jiang et al. [30] interpreted the posterior mean as a summary statistic by connecting the full dataset to the input layer of a deep neural network (DNN). They attempted to use regularization methods for training the neural network but did not obtain significant improvement. Creel [31] also used DNN to find the posterior mean based on a subset of predefined summary statistics rather than using the full dataset. However, the tuning of a large number of free parameters is still an issue in DNN-based ABC.

3. Materials and Methods

In this section, we provide a brief introduction to the ABC fundamentals. First, we introduce the straightforward ABC rejection algorithm and illustrate the usage of kernel methods and Hilbert embedding in the context of ABC. Afterward, we present our automatic ABC approach based on kernel learning and graph theory.

3.1. ABC Fundamentals

The central aim of Bayesian statistical inference concerns the calculation of the posterior distribution $p\left(\theta \right|y)$ for a set of model parameters $\theta =\mathsf{\Theta }$ given the observed data $y=\mathcal{X}$. In particular, the likelihood function $p\left(y\right|\theta )$ leverages the previous knowledge, as expressed in the prior distribution $\zeta \left(\theta \right)$, into the posterior via Bayes’ theorem. However, when the complexity of the analyzed system leads to an intractable likelihood, neither exact nor sampled posterior $p\left(\theta \right|y)\propto p(y\left|\theta \right)\zeta \left(\theta \right)$ can be computed. ABC approaches emerged to facilitate such an inference via simulation of the likelihood through a generative model of the system $\mathcal{M}:\mathsf{\Theta }\to \mathcal{X}$ that is statistically related to a conditional probability $p\left(x\right|\theta ),$ where $x=\mathcal{X}$ is a random variable standing for the simulated data [14]. Fundamentally, an ABC-based framework relies on the acceptance and rejection of candidates $\theta$ using their corresponding simulated samples x based on a distance function ${\mathrm{d}}_{\mathcal{X}}:\mathcal{X}\times \mathcal{X}\to {\mathbb{R}}^{+}$. The rejection in ABC is conducted by sampling multiple model parameters from $\zeta \left(\theta \right)$. The auxiliary model $\mathcal{M}$ generates simulations x that follow the conditional distribution $p\left(x\right|\theta )$; then, the subset $\{{\theta :\mathrm{d}}_{\mathcal{X}}(x,y)<\epsilon \}$ (where $\epsilon ={\mathbb{R}}^{+}$ is a threshold) is accepted to follow the posterior distribution. In turn, an approximate posterior can be estimated such that:

$$\widehat{p}\left(\theta \right|y;\epsilon )\propto \widehat{p}\left(y\right|\theta ;\epsilon )\zeta \left(\theta \right),$$

(1)

where:

$$\widehat{p}\left(y\right|\theta ;\epsilon )={\int }_{\mathcal{B}\left(y;\epsilon \right)}p\left(x\right|\theta )dx,\mathcal{B}\left(y;\epsilon \right)=\{x:{\mathrm{d}}_{\mathcal{X}}(x,y)<\epsilon \}.$$

(2)

Notice in Equation (2) how an accurate posterior relies upon an appropriate distance ${\mathrm{d}}_{\mathcal{X}}$ and a suitable $\epsilon$-value. Still, it is often challenging to apply a distance directly on $\mathcal{X}$ when dealing with real data since it is commonly formed by a large number of observations and features. In such a case, some alternatives use a mapping $s=\vartheta \left(x\right)$ before calculating the distance, where $s=\mathcal{S}$ is a feature space and $\vartheta :\mathcal{X}\to \mathcal{S}$ [17]. The previous setting is widely known as the straightforward ABC rejection algorithm.

Nonetheless, the use of $\vartheta \left(x\right)$ can lead to a non-sufficient feature space leaking information for complex models. As a consequence, some ABC-based inference approaches approximate the posterior $\widehat{p}\left(y\right|\theta ;ϵ)$ as the convolution of the true likelihood $p\left(y\right|\theta )$ and a kernel function $\kappa :\mathcal{X}\times \mathcal{X}\to \mathbb{R}$, which imposes a constraint to the rejection of samples as the inner product $\kappa (x,y)={\langle \varphi \left(x\right),\varphi \left(y\right)\rangle }_{\mathcal{H}}$ in a reproducing kernel Hilbert space (RKHS) $\mathcal{H},$ where $\varphi :\mathcal{X}\to \mathcal{H}$ [20]. In practice, given N samples ${\{x_n\sim P_{X_n}\}}_{n=1}^N$ drawn from $p\left(x\right|{\theta }_n),$ with ${\theta }_n\sim \zeta \left(\theta \right),$ and observed data $y\sim P_Y,$ a weighted sample set $W={\left\{{\theta }_n,w_n\right\}}_{n=1}^N$ is calculated by fixing:

$$w_n=\frac{{\kappa }_G\left({\mathrm{d}}_{\mathcal{H}}(P_{X_n},P_Y);ϵ\right)}{{\displaystyle \sum _{n=1}^N{\kappa }_G\left({\mathrm{d}}_{\mathcal{H}}(P_{X_n},P_Y);ϵ\right)}},$$

(3)

where ${\kappa }_G({\mathrm{d}}_{\mathcal{H}}(·,·);ϵ)$ is a Gaussian kernel defined as:

$${\kappa }_G\left({\mathrm{d}}_{\mathcal{H}}(P_{X_n},P_Y);ϵ\right)=exp\left(\frac{-{\mathrm{d}}_{\mathcal{H}}^2(P_{X_n},P_Y)}{2{ϵ}^2}\right),$$

(4)

where $ϵ={\mathbb{R}}^{+}$ is the kernel bandwidth and ${\mathrm{d}}_{\mathcal{H}}:\mathcal{H}\times \mathcal{H}\to {\mathbb{R}}^{+}$ represents a distance over the Hilbert embedding-based mappings between the distributions $P_{X_n}$ and $P_Y$. Figure 1 displays the main pipeline of the ABC rejection and Hilbert embedding-based ABC approaches, respectively. Finally, the set W is found as in Equation (3) to approximate $p\left(\theta \right|y)$ via the weighting-based posterior expectation as:

$$\widehat{p}\left(\theta \right|y)=\sum _{n=1}^N{w}_n{\kappa }_G({\mathrm{d}}_e(\theta ,{\theta }_n);{\sigma }_{\theta }),$$

(5)

where ${\mathrm{d}}_e(·,·)$ stands for the Euclidean distance and ${\sigma }_{\theta }={\mathbb{R}}^{+}$ is the kernel width.

3.2. Automatic ABC Based on Kernel Learning

Traditionally, the ABC methods do not code the information of the parameter space $\mathsf{\Theta }$ towards the inference. Namely, the deduction of the model parameters is performed using only explicit information from simulations and observations (see how in Figure 1 there is no direct connection between ${\left\{{\theta }_n\right\}}_{n=1}^N$ and the inference stage). Consequently, we introduce a novel kernel learning-based ABC approach, termed automatic kernel learning ABC (AKL-ABC), that codes the similarities between candidates in $\mathsf{\Theta }$ into the inference stage to obtain an automatic version of ABC. In particular, AKL-ABC comprises a kernel learning stage based on a statistical alignment to assess the matching between similarities defined over parameters and simulations. Moreover, a local neighborhood selection (LNS) algorithm is utilized to highlight local dependencies over candidates in $\mathsf{\Theta }$ based on graph theory to enhance the kernel similarities through a pruning scheme. The main steps of our AKL-ABC are summarized in the diagram shown in Figure 2 and described below.

3.2.1. Kernel Learning in the Context of ABC

The estimation of the ABC-based posterior as in Equations (3) and (5) requires tuning of the $ϵ$ value. To avoid its tuning, we introduce a statistical alignment approach for enhancing the inference task. The purpose behind this procedure is to include the information contained in the candidates ${\left\{{\theta }_n\right\}}_{n=1}^N$ to improve the comparison stage carried out over simulations and observations. Let $\Psi ={\{{\theta }_n,x_n\}}_{n=1}^N$ be the set of N candidates ${\theta }_n={\mathbb{R}}^P\sim \zeta \left(\theta \right)$ and their corresponding simulations $x_n={\mathbb{R}}^Q\sim p\left(x\right|\theta )$. Further, let ${\kappa }_{\theta }:\mathsf{\Theta }\times \mathsf{\Theta }\to {\mathbb{R}}^{+}$ be a similarity measure between candidates in $\mathsf{\Theta }$ that defines the kernel matrix ${\mathbf{K}}_{\mathbf{\theta }}={\mathbb{R}}^{N\times N}$ holding elements:

$$k_{\theta }({\theta }_n,{\theta }_{n^{\prime }})=\left\{\begin{array}{cc}exp(-{\mathrm{d}}_{\mathsf{\Theta }}^2({\theta }_n,{\theta }_{n^{\prime }}))\hfill & {\theta }_n={\mathsf{\Omega }}_{n^{\prime }}\hfill \\ 0\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

(6)

where ${\mathsf{\Omega }}_{n^{\prime }}$ is a set holding the M-nearest neighbors of ${\theta }_{n^{\prime }}$ in the sense of the distance ${\mathrm{d}}_{\mathsf{\Theta }}:\mathsf{\Theta }\times \mathsf{\Theta }\to {\mathbb{R}}^{+}$. In this paper, to avoid large variations among components of ${\theta }_n$ we rely on the Mahalanobis distance as follows:

$${\mathrm{d}}_{\mathsf{\Theta }}^2({\theta }_n,{\theta }_{n^{\prime }})={({\theta }_n-{\theta }_{n^{\prime }})}^{\top }{\mathbf{\Sigma }}_{\mathbf{\Theta }}^{-1}({\theta }_n-{\theta }_{n^{\prime }})$$

(7)

where ${\mathbf{\Sigma }}_{\mathsf{\Theta }}={\mathbb{R}}^{P\times P}$ is the sample covariance matrix. Concerning the feature space $\mathcal{S}$, we assess the similarity via the Gaussian kernel function $k_s:\mathcal{S}\times \mathcal{S}\to {\mathbb{R}}^{+}$, $k_s(\vartheta \left(x_n\right),\vartheta \left(x_{n^{\prime }}\right))=exp(-{\mathrm{d}}_{\mathcal{S}}^2(\vartheta \left(x_n\right),\vartheta \left(x_{n^{\prime }}\right))/2{\gamma }^2)$, to build the matrix ${\mathbf{K}}_{\mathit{s}}={\mathbb{R}}^{N\times N},$ where ${\mathrm{d}}_{\mathcal{S}}^2:\mathcal{S}\times \mathcal{S}\to {\mathbb{R}}^{+}$ and $\vartheta :\mathcal{X}\to \mathcal{S}$ is a feature mapping. Here, to perform the pairwise comparison between simulations in $\mathcal{S}$ we use also the Mahalanobis distance of the form [32]:

$${\mathrm{d}}_{\mathcal{S}}^2(\vartheta \left(x_n\right),\vartheta \left(x_{n^{\prime }}\right))={(\vartheta \left(x_n\right)-\vartheta \left(x_{n^{\prime }}\right))}^{\top }\mathit{A}{\mathit{A}}^{\top }(\vartheta \left(x_n\right)-\vartheta \left(x_{n^{\prime }}\right)),$$

(8)

where ${\mathbf{\Sigma }}_{\mathcal{S}}^{-1}=\mathit{A}{\mathit{A}}^{\top }$ stands for the inverse covariance matrix of $\vartheta \left(x_n\right)={\mathbb{R}}^D$ and $\mathit{A}={\mathbb{R}}^{D\times d}$ ($d\le D$). In this sense, we use the information respecting the similarities among candidates in $\mathsf{\Theta }$, represented via ${\mathbf{K}}_{\mathbf{\theta }}$, to state a notion of similarity between the simulations and a target observation in $\mathcal{S}$, represented via ${\mathbf{K}}_{\mathit{s}}(\mathit{A},\gamma )$. Therefore, we use a CKA-based measure between the kernel matrices as follows [22]:

$$\widehat{\rho }({\mathbf{K}}_{\mathbf{\theta }},{\mathbf{K}}_{\mathit{s}}(\mathit{A},\gamma ))=\frac{{\langle {\overline{\mathbf{K}}}_{\mathbf{\theta }},{\overline{\mathbf{K}}}_{\mathit{s}}\rangle }_{\mathrm{F}}}{\sqrt{{\langle {\overline{\mathbf{K}}}_{\mathbf{\theta }},{\overline{\mathbf{K}}}_{\mathbf{\theta }}\rangle }_{\mathrm{F}}{\langle {\overline{\mathbf{K}}}_{\mathit{s}},{\overline{\mathbf{K}}}_{\mathit{s}}\rangle }_{\mathrm{F}}}},$$

(9)

where $\overline{\mathbf{K}}$ stands for the centered kernel as $\overline{\mathbf{K}}=\tilde{\mathit{I}}\mathbf{K}\tilde{\mathit{I}}$, being $\tilde{\mathit{I}}=\mathit{I}-{\mathbf{1}}^{\top }\mathbf{1}/N$ the empirical centering matrix, $\mathit{I}={\mathbb{R}}^{N\times N}$ is the identity matrix, and $\mathbf{1}={\mathbb{R}}^N$ is the all-ones vector. Moreover, The notation ${\langle ·,·\rangle }_{\mathrm{F}}$ represents the matrix-based Frobenius norm. In Equation (9), $\widehat{\rho }(·,·)$ is a data-driven estimator that aims to quantify the similarity between the parameter space and the feature space.

To find the projection matrix $\mathit{A}$, we consider the following optimization problem:

$$\widehat{\mathit{A}}=\arg \underset{\mathit{A},\gamma }{max}log\left(\widehat{\rho }({\mathbf{K}}_{\mathbf{\theta }},{\mathbf{K}}_{\mathit{s}}(\mathit{A},\gamma ))\right),$$

(10)

where the logarithm function is used for mathematical convenience. Estimation of $\widehat{\rho }$ in Equation (10) relies on the explicit objective [33]:

$$\widehat{\rho }({\mathbf{K}}_{\mathbf{\theta }},{\mathbf{K}}_{\mathit{s}}(\mathit{A},\gamma ))=log\left(\mathrm{tr}\left({\mathbf{K}}_{\mathit{s}}\tilde{\mathit{I}}{\mathbf{K}}_{\mathbf{\theta }}\tilde{\mathit{I}}\right)\right)-0.5log\left(\mathrm{tr}\left({\mathbf{K}}_{\mathit{s}}\tilde{\mathit{I}}{\mathbf{K}}_{\mathit{s}}\tilde{\mathit{I}}\right)\right)+{\rho }_0,$$

(11)

where ${\rho }_0=\mathbb{R}$ is a constant in $\mathit{A}$. In this regard, given an initial guess ${\mathit{A}}^0$ (calculated, for instance, using the well-known principal component analysis (PCA) algorithm), the projection matrix $\mathit{A}$ is updated according to the following gradient-descent rule:

$${\mathit{A}}^{t+1}={\mathit{A}}^t-{\mu }_{\mathit{A}}^t{\nabla }_{\mathit{A}}\left[\widehat{\rho }({\mathbf{K}}_{\mathbf{\theta }},{\mathbf{K}}_{\mathit{s}}({\mathit{A}}^t,{\gamma }^t))\right],$$

(12)

where ${\mu }_{\mathit{A}}^t={\mathbb{R}}^{+}$ is the step size of the learning rule and ${\nabla }_{\mathit{A}}\left[\widehat{\rho }(·,·)\right]$ stands for the gradient with respect to $\mathit{A}$ of the objective function (11), defined as follows:

$${\nabla }_{\mathit{A}}\left[\widehat{\rho }({\mathbf{K}}_{\mathbf{\theta }},{\mathbf{K}}_{\mathit{s}}({\mathit{A}}^t,{\gamma }^t))\right]=-4{\mathit{V}}^{\top }(\Delta \circ {\mathbf{K}}_{\mathit{s}})-\mathrm{diag}\left({\mathbf{1}}^{\top }(\Delta \circ {\mathbf{K}}_{\mathit{s}})\right)\mathit{V}{\mathit{A}}^t,$$

(13)

where $\mathit{V}={\mathbb{R}}^{N\times D}$ is a matrix whose n-th row holds the mapped simulation $\vartheta \left(x_n\right)$, and the notations $\mathrm{diag}(·)$ and ∘ denote the diagonal operator and the Hadamard product, respectively. Moreover, $\Delta ={\mathbb{R}}^{N\times N}$ is the gradient of the objective function with respect to ${\mathbf{K}}_{\mathit{s}}$:

$$\Delta =\frac{\tilde{\mathit{I}}{\mathbf{K}}_{\mathbf{\theta }}\tilde{\mathit{I}}}{\mathrm{tr}\left({\mathbf{K}}_{\mathit{s}}\tilde{\mathit{I}}{\mathbf{K}}_{\mathbf{\theta }}\tilde{\mathit{I}}\right)}-\frac{\tilde{\mathit{I}}{\mathbf{K}}_{\mathit{s}}\tilde{\mathit{I}}}{\mathrm{tr}\left({\mathbf{K}}_{\mathit{s}}\tilde{\mathit{I}}{\mathbf{K}}_{\mathit{s}}\tilde{\mathit{I}}\right)}.$$

(14)

In addition to the optimal learning of the matrix $\mathit{A}$, the optimization problem in Equation (11) also requires the tuning of the Gaussian kernel bandwidth $\gamma$. This joint parameter estimation can be carried out optimizing one variable recursively at a time while the other variable remains unchanged. Namely, the calculation of the rule in Equation (12) is achieved under a constant ${\gamma }^t-$ value. In turn, this kernel bandwidth is estimated (under constant $\widehat{\mathit{A}}$) using information theoretic learning (ITL) criteria intending to maximize the overall variability of the information potential ($H_{\alpha }$) computed for all ${\{z_n=\vartheta {\left(x_n\right)}^{\top }\widehat{\mathit{A}}\}}_{n=1}^N$, so that all the information force magnitudes spread more widely [34]:

$${\gamma }^t=\arg \underset{\gamma }{max}\mathrm{var}\{H_{\alpha }\left(z_n\right|{\gamma }^t):n=1,2,\dots ,N\};\alpha ={\mathbb{R}}^{+}.$$

(15)

In particular, we use the Renyi’s quadratic entropy, $H_2(·|{\gamma }^t)$, to apply the following gradient-descent update rule over the Gaussian kernel bandwidth:

$${\gamma }^{t+1}={\gamma }^t-{\mu }_{\gamma }^t{\nabla }_{\gamma }\left[\mathrm{var}\left\{H_2\left(z_n^t\right|{\gamma }^t)\right\}\right],$$

(16)

where ${\mu }_{\gamma }^t={\mathbb{R}}^{+}$ is the step size of the learning rule. See [34] for more details on the derivation of ${\nabla }_{\gamma }$.

3.2.2. $ϵ$ Tuning Through Nearest Neighbors Based on Graph Theory

Tuning the $ϵ$-value to approximate the posterior weights as in Equation (3) is a critical step. Depending on the distance output values, a particular choice of $ϵ$ would produce a peaked posterior when just a few numbers of weights have larger values or lead to a posterior similar to the prior distribution in the limit condition when $w_n\to 1/N,\forall n=\{1,2,\dots ,N\}$. Bearing this in mind, we use the truncated representation of Equation (6) as an alternative to avoid the influence of $ϵ$ via the concept of neighborhood. Namely, an optimal selection of the number of nearest neighbors $M=\mathbb{N}$ reveals the prior representative samples in the posterior distribution.

The M-value can be fixed manually after an exhaustive search based on cross-validation; however, that would hinder the automatic philosophy of this work. To avoid this issue, we use an automatic technique based on locally linear embedding (LLE) and graph theory, the local neighborhood selection (LNS) algorithm, to facilitate the selection of the optimal number of nearest neighbors [35]. LNS aims to identify a suitable number of neighbors for each sample taking into account the structure of the dataset. Specifically, this algorithm is rooted in the idea that when a region around a point is linear and dense, the Euclidean and Geodesic distances obtain a similar set of nearest neighbors for each sample; otherwise, the Euclidean distance will detect short connections while the geodesic distance will identify the right neighbors of each sample. For a better illustration, Figure 3 shows the nearest neighbors for a particular sample in the well-known Swiss-Roll manifold (filled bullet) using both the Euclidean and the geodesic distances. Notice how the Euclidean distance selects neighbors that do not follow the structure of the manifold (Figure 3a) while the geodesic distance understands the actual structure leading to a proper selection of the nearest neighbors (Figure 3b). Figure 3c shows the completed connected graph for all points in the dataset. The LNS algorithm devoted to the estimation of the ABC posterior is described in detail in Appendix A.

Lastly, the ABC-based inference stage is automated via the weighted sample set ${\left\{({\theta }_n,w_n)\right\}}_{n=1}^N$, where each $w_n$ is calculated as follows:

$$w_n=\frac{{\kappa }_E(z,z_n)}{{\displaystyle \sum _{n=1}^N{\kappa }_E(z,z_n)}},$$

(17)

where ${\kappa }_E:{\mathbb{R}}^d\times {\mathbb{R}}^d\to \mathbb{R}$ is a similarity kernel defined as:

$${\kappa }_E(z,z_n)=\left\{\begin{array}{cc}exp(-||z-z_n|{|}_2^2)\hfill & z_n=\mathcal{Z}\hfill \\ 0,\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

(18)

where $\mathcal{Z}$ is a set holding the M-nearest neighbors of $z=\vartheta {\left(y\right)}^{\top }\widehat{\mathit{A}}$ in the sense of the Euclidean distance.

In short, Algorithm 1 shows the proposed AKL-ABC approach. Furthermore, a MATLAB implementation of the AKL-ABC is publicly available (https://github.com/WilsonGV/AKL-ABC.git).

Algorithm 1 AKL-ABC algorithm
Input: Observed data: y, prior: $\zeta \left(\theta \right)$, mapping: $\vartheta$.
Output: Posterior estimation: $\widehat{p}\left(\theta \right|y).$
Kernel learning stage:
1:  ${\Psi }^{{}^{\prime }}={\left\{({\theta }_n^{\prime },x_n^{\prime })\right\}}_{n=1}^N;$${\theta }_n^{\prime }\sim \zeta \left(\theta \right),$$x_n^{\prime }\sim p\left(x\right|{\theta }_n^{\prime })$	▹ Draw training data.
2:  $M=\mathrm{median}\left\{\mathbb{M}\right\}$	▹ Highlight local dependencies in $\mathsf{\Theta }$ using LNS.
3:  $\widehat{\mathit{A}}=\arg {max}_{\mathit{A}}log\left(\widehat{\rho }({\mathbf{K}}_{\mathit{s}}\left(\mathit{A}\right),{\mathbf{K}}_{\mathbf{\theta }})\right)$	▹ Compute the CKA based on $\vartheta ,$ $M,$ and ${\Psi }^{\prime }$
Inference stage:
4:  $\Psi ={\left\{({\theta }_n,x_n)\right\}}_{n=1}^N;$ ${\theta }_n\sim \zeta \left(\theta \right),$ $x_n\sim p\left(x\right|{\theta }_n)$	▹ Draw simulated data.
5:  $z=\vartheta {\left(y\right)}^{\mathrm{T}}\widehat{\mathit{A}}$	▹ Project features of observed data
6:  for  $n=1,\dots ,N$  do
7:   $z_n=\vartheta {\left(x_n\right)}^{\mathrm{T}}\widehat{\mathit{A}}$	▹ ▹ Project features of simulated data
8:   ${\tilde{w}}_n={\kappa }_E(z,z_n)$	▹ Compute the n-th weight value.
9:  end for
10:  $w_n={\tilde{w}}_n/{\sum }_{n=1}^N{\tilde{w}}_n$	▹ Normalize the weights
10:  $\widehat{p}\left(\theta \right|y)={\sum }_{n=1}^N{w}_n{\kappa }_G({\mathrm{d}}_e(\theta ,{\theta }_n);{\sigma }_{\theta })$	▹ Approximate the posterior.

3.3. Theoretical Aspects of AKL-ABC

One of the main contributions of AKL-ABC is the computation of automatic posterior estimates with no need for expensive procedures to select free parameters. We now address two major theoretical aspects of AKL-ABC: Learning performance and computational complexity.

To investigate the learning performance of AKL-ABC, we must analyze the CKA-based measure over matrices ${\mathbf{K}}_{\mathit{s}}={\mathbb{R}}^{N\times N}$ and ${\mathbf{K}}_{\mathbf{\theta }}={\mathbb{R}}^{N\times N}$. The empirical alignment $\widehat{\rho }({\mathbf{K}}_{\mathit{s}},{\mathbf{K}}_{\mathbf{\theta }})$, as defined in Equation (9), is a statistical approximation of the true alignment $\rho (k_s,k_{\theta })$ defined as [36]:

$$\rho (k_s,k_{\theta })=\frac{{\langle k_s,k_{\theta }\rangle }_{\mathcal{P}}}{{\langle k_s,k_s\rangle }_{\mathcal{P}}{\langle k_{\theta },k_{\theta }\rangle }_{\mathcal{P}}},$$

(19)

where ${\langle f,g\rangle }_{\mathcal{P}}=\int f(a,b)g(a,b)d\mathcal{P}\left(a\right)d\mathcal{P}\left(b\right)$ based on the input distribution $\mathcal{P}$.

Theorem 1. 

(Cortes et al. [22]) Assume $k_s(·,·)\le \varpi$ and $k_{\theta }(·,·)\le {\varpi }^{\prime }$. For any $\varsigma >0$, with probability at least $1-\varsigma$, the following inequality holds:

$$|\rho (k_s,k_{\theta })-\widehat{\rho }({\mathbf{K}}_{\mathit{s}},{\mathbf{K}}_{\mathbf{\theta }})|\le 18\mathsf{\Lambda }\left[\frac{3}{N}+8\sqrt{\frac{log\frac{6}{\varsigma }}{2N}}\right],$$

where $\mathsf{\Lambda }=max(\varpi {\varpi }^{\prime }/\mathbb{E}\left[k_s^2\right],\varpi {\varpi }^{\prime }/\mathbb{E}\left[k_{\theta }^2\right])$.

Proposition 1.

Given the sample set ${\left\{({\theta }_n,w_n)\right\}}_{n=1}^N$, where the weights $w_n$ are fixed according to the AKL-ABC algorithm, the expectation $\mathbb{E}\left[\widehat{p}\left(\theta \right|y)\right]={\sum }_{n=1}^N{w}_n{\theta }_n$ tends to the expected value of the posterior distribution as $N\to \infty$.

Proof. 

According to Theorem 1, the empirical CKA-based alignment used in AKL-ABC ($\widehat{\rho }({\mathbf{K}}_{\mathit{s}},{\mathbf{K}}_{\mathbf{\theta }})$) tends asymptotically to the true statistical alignment ($\rho (k_s,k_{\theta })$) as the number of input points tend to infinity. As a consequence, ${\mathbf{K}}_{\mathit{s}}-{\mathbf{K}}_{\mathbf{\theta }}$ tends to the null matrix as $N\to \infty$ meaning that the notion of similarity defined over simulations and parameters spaces is the same. Then, the neighborhood around projected features of observed data $z=\vartheta {\left(y\right)}^{\top }\widehat{\mathit{A}}$, according to Equations (17) and (18), leads to a neighborhood around the true value of the model parameters in the sense of the statistical input distribution $\mathcal{P}$. □

Remark 1.

While the LNS algorithm operates over the input set ${\left\{{\theta }_n\right\}}_{n=1}^N$ (see Appendix A), the concept of neighborhood is directly transferred from the parameter space to the simulations space. Namely, the similarity assessment is exchangeable from one space to another due to the asymptotic behavior of the statistical alignment with the number of input points.

Finally, taking into account the main steps of AKL-ABC, we can find its computational complexity: The LNS algorithm requires $O(P{N}^2+\varrho N^2)$ operations, where $\varrho \le \lceil \sqrt{N}\rceil$ takes a dynamic integer value since the LNS method determines the number of neighbors by adapting the dataset distribution (see Appendix A) [35]; furthermore, a number of $O(N^2+G{N}^2)$ operations is required to find the projection matrix $\widehat{\mathit{A}}$ based on ${\mathbf{K}}_{\mathit{s}}$ and ${\mathbf{K}}_{\mathbf{\theta }}$, where G is the number of desired gradient-descend steps. Then, the total complexity of our AKL-ABC is $O\left(N^2(P+\varrho +G+1)\right)$.

4. Experimental Setup

To evaluate the performance of our AKL-ABC approach, we analyze three different attributes: (i) Posterior quality approximation in applications that comprise both synthetic and real datasets, (ii) suitable convergence concerning the number of ABC samples (N), and (iii) adequate selection of the number of nearest neighbors (M).

4.1. Datasets and Quality Assessment

We examine the next experiments following [20]: A toy problem concerning synthetic data from a mixture model and a Bayesian inference problem for a real-world ecological dynamic system. Each experiment is described below.

Toy problem: We inspect a mixture of uniform distributions of the form:

$$p\left(x\right|\mathbf{\pi })=\sum _{c=1}^C{\pi }_c\mathcal{U}(c-1,c),$$

(20)

where $\mathbf{\pi }={\{{\pi }_c={\mathbb{R}}^{+}\}}_{c=1}^C$ stores the mixing coefficients holding ${\sum }_{c=1}^C{\pi }_c=1$, and $C=\mathbb{N}$ is the number of components. Here, the aim is to estimate the posterior $p\left(\mathbf{\pi }\right|y)$ for $C=5$, given synthetic observations y drawn from the mixture with true parameters (target): ${\mathbf{\pi }}^{*}=\{0.25,0.04,0.33,0.04,0.34\}$. Besides, as quality assessment for the toy problem we utilize the Euclidean distance as follows [20]:

$$\mathcal{E}=\left|\right|{\mathbf{\pi }}^{*}-\widehat{\mathbf{\pi }}{\left|\right|}_2,$$

(21)

where $\widehat{\mathbf{\pi }}$ is the expected value of the posterior found using the weights ${\left\{w_n\right\}}_{n=1}^N$.

Real-world problem: Inference tasks over dynamic ecological systems representing chaotic and near-chaotic domains is quite a challenge [7,9]. We considered the problem of inferring the dynamics of an adult blowfly population. Particularly, Wood [18] introduced a model for describing population dynamics using a differential equation as follows:

$$N_{t+1}=P{N}_{t-\tau }exp\left(-\frac{N_{t-\tau }}{N_0}\right)e_t+N_{t}exp(-\delta {ϵ}_t),$$

(22)

where $N_{t+1}$ denotes the observation time at $t+1$, which is determined by the time-lagged observations $N_t$ and $N_{t-\tau }$. Moreover, $e_t$ and ${ϵ}_t$ stand for the Gamma distributed noise as $e_t\sim \mathcal{G}(1/{\sigma }_p^2,{\sigma }_p^2)$ and ${ϵ}_t\sim \mathcal{G}(1/{\sigma }_d^2,{\sigma }_d^2)$, respectively. Here, our aim is to estimate the posterior of the parameters $\mathbf{\theta }=\{P,N_0,{\sigma }_d,{\sigma }_p,\tau ,\delta \}$ given observed data concerning a time series of 180 observations (available in the Supplementary Materials of [18]). For concrete testing, we adopt log-normal distributions for setting priors over $\mathbf{\theta }$[29]: $log\left(P\right)\sim \mathcal{N}(2,2^2),$$log\left(N_0\right)\sim \mathcal{N}(6,1),$$log\left({\sigma }_d\right)\sim \mathcal{N}(-0.5,1),$$log\left({\sigma }_p\right)\sim \mathcal{N}(-0.5,1),$$log\left(\tau \right)\sim \mathcal{N}(2.7,1),$$log\left(\delta \right)\sim \mathcal{N}(-1,{0.4}^2)$.

On the other hand, because in a real-world inference task there is no target value for the model parameters, the quantitative assessment must rely on the quality of predictions. Thus, the Euclidean distance is computed to measure the performance in the feature space as follows [20]:

$$\mathcal{E}=\left|\right|\vartheta \left(y\right)-\vartheta \left(x_n\right|\widehat{\mathbf{\theta }}){\left|\right|}_2,$$

(23)

where $x_n|\widehat{\mathbf{\theta }}$ is a simulation from the model given the expected value of the posterior.

4.2. AKL-ABC Training and Method Comparison

For comparison purposes, we considered the K2-ABC approach proposed by Park et al. [20] due to its excellent performance over other methods such as: Indirect score ABC (IS-ABC) [19], semi-automatic ABC (SA-ABC) [16], kernel ABC (K-ABC) [27], and synthetic likelihood ABC (SL-ABC) [18]. We selected the previous benchmark since they belong to the two main groups of ABC algorithms considered in the scope of this paper, the ones that compute posterior estimations based on summary statistics (IS-ABC, SA-ABC, SL-ABC), and the ones that produce posterior estimates via weighting-based inferences (K-ABC, K2-ABC). In particular, K2-ABC uses a mapping of the observed and simulated data from the simulation space to an RKHS and employs a maximum mean discrepancy (MMD) criterion to construct a dissimilarity measure between distributions of observed and simulated data. While this method has outstanding results, it requires the tuning of free parameters. On the other hand, we can find the best possible performance of our AKL-ABC by running the inference stage in Algorithm 1 with ${\tilde{w}}_n={\kappa }_E({\mathbf{\pi }}^{*},{\mathbf{\pi }}_n)$; we refer to this approach as “best”.

Regarding the toy problem, we draw $N=1000$ samples from a symmetric Dirichlet prior, $\mathbf{\pi }\sim \mathrm{Dirichlet}\left(\mathbf{1}\right)$, and then used the mixture model to form the simulated data by drawing 400 observations for each prior candidate. Moreover, we employ a histogram with ten bins as feature mapping ($\vartheta$) in AKL-ABC and fix the Gaussian kernel bandwidths as 0.1 and 0.001 for the characteristic and similarity kernels, respectively, in the K2-ABC method. These parameters were tuned according to the cross-validation procedure planted by Park et al. [20].

For the real-world problem, we generate $N=5000$ samples from the prior and then assess the model to form the simulated data by drawing 180 observations for each prior candidate. Besides, as feature mapping ($\vartheta$), we selected the ten statistics used by Park et al. [20]. Furthermore, due to fluctuations produced by ${ϵ}_t$ and $e_t$, we draw 100 simulations from the model given the expected value of the posterior and compute the boxplots of $\mathcal{E}$ for each method.

As suggested by authors in [32], we fixed the free parameters of the CKA algorithm regarding the gradient-descend optimization as follows: The adaptive step size of the learning rules are adjusted such that ${\mu }_{\gamma }^t$ and ${\mu }_{\mathit{A}}^t$ decrease gradually from $1\mathit{e}-4$ to $1\mathit{e}-5$ through a maximum number of iterations empirically limited up to 100. Moreover, the initial guess for the rotation matrix ${\mathit{A}}^0$ is computed using the well-known PCA method retaining the 95% of the variance explained (which defines the number of columns of $\mathit{A}$, $d\le D$), while ${\gamma }^0$ is calculated as the median of the input data Euclidean distances.

5. Results and Discussion

5.1. Toy Problem Results

Since this is a full controlled experiment with known parameters ${\mathbf{\pi }}^{*},$ we can find the best possible performance of our AKL-ABC by running the inference stage with ${\tilde{w}}_n={\kappa }_E({\mathbf{\pi }}^{*},{\mathbf{\pi }}_n)$. The previous setting is equivalent to think that the statistical alignment found via CKA between ${\mathbf{K}}_{\mathbf{\theta }}$ and ${\mathbf{K}}_{\mathit{s}}$ yields to ${\mathbf{K}}_{\mathbf{\theta }}$ = ${\mathbf{K}}_{\mathit{s}}$. Figure 4 shows the “best” performance along with K2-ABC and AKL-ABC results over the uniform mixture problem. In Figure 4a, the expected value of the posterior computed for all methods is close to the target. In particular, we obtained ${\mathcal{E}}_{Best}=0.030\pm 0.039$, ${\mathcal{E}}_{\mathbf{K}2-\mathrm{ABC}}=0.063\pm 0.042$, and ${\mathcal{E}}_{\mathrm{AKL}-\mathrm{ABC}}=0.064\pm 0.041$. Notice how the “best” approach achieves the lowest possible error providing a lower bound that the AKL-ABC could reach in the ideal case of perfect statistical alignment. These results show that our AKL-ABC is a competitive estimator to K2-ABC with a significant advantage concerning the automatic selection of free parameters without requiring any tuning procedure. Besides, to provide a better understanding of the AKL-ABC effectiveness, in Figure 4b we show the weights for the five nearest neighbors (according to the LNS algorithm) used to compute the posteriors. As noted, the majority of the chosen simulations for AKL-ABC match the selected candidates using the “best”, although our approach never observes the target.

To observe the stability of the proposed AKL-ABC concerning the number of simulations, we select $y^{*}$ to be a set of 400 synthetic observations drawn from the model (see Equation (20)) using the target ${\mathbf{\pi }}^{*}$. Then, we solve repeatedly the problem of inferring an approximation for $p\left(\mathbf{\theta }\right|y^{*})$ by using each method, increasing N with a step-size of 100. Figure 5a shows the resulting $\mathcal{E}$ vs. N curve where the larger the number of simulations the lower and more confident the approximation error. While our AKL-ABC approach obtains a similar performance in comparison with the K2-ABC method in terms of stability and approximation error, our methodology overcomes the benchmark regarding its automatic philosophy that avoids any tuning procedure.

In turn, we compare the performance of the LNS algorithm concerning the choice of the M-nearest neighbors in AKL-ABC against a conventional grid search using $y^{*}$ as observed data and N = 400 simulations. The $\mathcal{E}$ vs. M curve in Figure 5b proves that either small or large neighborhoods result in significant errors because, in the first case, the data structure is weakly encoded and, in the second case, the estimation is biased towards the population average. Besides, the larger the number of neighbors, the more confident the posterior estimate, since the set of weights ${\left\{w_n\right\}}_{n=1}^N$ contains a larger number of non-zeros values. Therefore, introducing the LNS algorithm into the AKL-ABC results in an automatically computed number of neighbors (M = 5) lying on the minimum error region.

5.2. Real Dataset Results

Inferring the model parameters in this blowfly dataset is a very challenging task since the system dynamics can easily move from stable to chaotic regimes. The auxiliary model would produce completely different simulations in face of minimal fluctuations of the parameters [18,29]. This states an interesting scenario to test the performance and robustness of our AKL-ABC. In Figure 6, we provide the prior and the posterior approximations for each parameter, fixing ${\sigma }_{\theta }$ according to [37]. Notice how our proposal updates the beliefs about the model parameters leading to more concentrated posteriors. In the case of $log\left({\sigma }_p\right)$, two modes reflect different intervals with probable values for driving the noise realization associated with egg production in the blowfly population. However, there is a predominant mode that states higher probabilities for this parameter. Moreover, Figure 6g shows the closest and farthest predictions to the observed data selected from 100 realizations used to compute $\mathcal{E}$, showing that the inferred posterior lays on a stable regime. Finally, Figure 7 shows the performance of AKL-ABC compared against different ABC-based methods tested on the blowfly dataset by Park et al. [20]. As seen, our proposed method is a quite competitive approach to the benchmark. In particular, the confidence intervals of $\mathcal{E}$ are $(1.0620,1.1232)$, $(0.9923,1.1543)$, and $(1.8401,2.0464)$ for the AKL-ABC, K2-ABC, and SL-ABC methods, respectively. The smaller the confidence interval, the more stable the mean posterior prediction since the model dynamic straightforwardly falls into chaotic regimes, even with minimal changes of the model parameters. Thus, the narrowest confidence interval obtained in AKL-ABC proves its capability to deal with complex dynamic data. Furthermore, AKL-ABC holds a significant advantage concerning the automatic selection of free parameters.

5.3. Computational Tractability

One of the primary considerations in ABC is computational tractability. Because there are no tuning procedures required in our AKL-ABC, the time complexity for a given dataset ${\left\{{\theta }_n={\mathbb{R}}^P,\vartheta \left(x_n\right)={\mathbb{R}}^D\right\}}_{n=1}^N$ is $O\left(N^2(P+\varrho +G+1)\right)$. Nonetheless, in the case of other non-automatic ABC methods, the number of free parameters and the grid they define notably augment the computational burden. Namely, if the number of required operations for a given ABC method is denoted by $O\left(\mathcal{N}\right)$, the final time complexity for tuning F free parameters is $O\left(\mathcal{N}\left({\prod }_{i=1}^F{\lambda }_i\right)\right)$, where ${\lambda }_i=\mathbb{N}$ is the number of all possible values to be explored for the i-th free parameter. Notice that the thinner the exploration grid, the more prohibited the number of overall operations. For instance, the K2-ABC approach requires $O\left(N^2\right)$ operations [20]. However, the grid search needed for tuning the free parameters increases the computational complexity exponentially to $O\left(N^2\left({\lambda }_1{\lambda }_2\right)\right)$, where ${\lambda }_1,{\lambda }_2$ are related to grids defined over characteristic and similarity kernel widths. See that the performance of AKL-ABC depends on $\varrho$ and G; however, in practice, the CKA and LNS algorithm have a fast convergence [32,35].

6. Conclusions

In this paper, we focus on the problem of automatically performing Bayesian statistical inference under the intractability of the likelihood function. In particular, we propose an automatic enhancement of the well-known ABC algorithm devoted to approximate Bayesian inference called AKL-ABC. In particular, we include a metric learning approach based on a CKA framework to quantify the statistical alignment between parameter and simulation spaces in ABC. Then, a Mahalanobis distance is learned through CKA, and a graph representation based on a local neighborhood selection algorithm is employed to reveal local relationships among parameter and simulation samples. Notably, AKL-ABC has an advantage over other ABC approaches: The statistical alignment over parameter and simulations spaces and the concept of neighborhood introduce additional information in the inference procedure such that the overall ABC framework does not require the tuning of any free parameters. Attained results on a synthetic dataset and a real-world ecological system show the introduced AKL-ABC is robust to substantial changes in data dynamics and produces quite competitive posterior approximation compared to other non-automatic state-of-the-art ABC methods.

Future work includes the extension of AKL-ABC for high-dimensional problems where a large number of observations could be prohibited, via a possibly global neighborhood selection approach that supports a faster computation of the number of neighbors (M) required in AKL-ABC, taking into account comprehensive features rather than local properties of the input data. Moreover, the inclusion of other dissimilarity measures besides the Mahalanobis distance, coupled with the neighborhood-based philosophy of AKL-ABC, is also a potential line of research to deal with applications that gather complex and noisy data. Lastly, the computational burden would be enhanced based on stochastic gradient approaches [38].