Big Data Blind Separation

Abstract

Data or signal separation is one of the critical areas of data analysis. In this work, the problem of non-negative data separation is considered. The problem can be briefly described as follows: given $\mathbf{X}\in {\mathbb{R}}^{m\times N}$, find $\mathbf{A}\in {\mathbb{R}}^{m\times n}$ and $\mathbf{S}\in {\mathbb{R}}_{+}^{n\times N}$ such that $\mathbf{X}=\mathbf{A}\mathbf{S}$. Specifically, the problem with sparse locally dominant sources is addressed in this work. Although the problem is well studied in the literature, a test to validate the locally dominant assumption is not yet available. In addition to that, the typical approaches available in the literature sequentially extract the elements of the mixing matrix. In this work, a mathematical modeling-based approach is presented that can simultaneously validate the assumption, and separate the given mixture data. In addition to that, a correntropy-based measure is proposed to reduce the model size. The approach presented in this paper is suitable for big data separation. Numerical experiments are conducted to illustrate the performance and validity of the proposed approach.

1. Introduction

Transforming data into information is a key research direction of the current scientific age. During the data collection phase, it is often the case that the data cannot be collected from the actual data generating locations (sources). Typically, a nearby physically connected location (station) that is accessible can be used for the data collection. If the station is influenced by more than one source, then the data collected at the station provides mixed information from the multiple sources (mixture data). This creates a challenging problem of identifying the source data from the given mixture data. Such a problem is typically known as a data (or signal) separation problem.

In this paper, a linear mixing type data separation problem is considered. The generative model of the problem in its standard form can be written as:

$$\mathbf{X}=\mathbf{A}\mathbf{S},$$

(1)

where $\mathbf{X}\in {\mathbb{R}}^{m\times N}$ denotes the given mixture matrix, $\mathbf{A}\in {\mathbb{R}}^{m\times n}$ is the unknown mixing matrix, and $\mathbf{S}\in {\mathbb{R}}^{n\times N}$ denotes the unknown source matrix. The problem can be further classified into overdetermined (m > n), undetermined (m < n), and square or determined (m = n) cases. The overdetermined case can be transformed into the square case by using the Principal Component Analysis (PCA) method [1,2]. The undetermined cases often result in loss of information or redundancy in the representation. Usually, approximate recovery is done based on the prior probability assumptions. Typically, Gaussian or Laplacian priors are used to estimate the mixing matrix. The idea is to identify the directions of maximum density via clustering. These directions correspond to the identification of the mixing matrix [3,4]. Once the mixing matrix is estimated, the source matrix is obtained by solving series of least square problems. In this paper, the data separation problem for $m\ge n$ cases will be considered.

In addition to that, the above problem given in Equation (1) can be solved under numerous assumptions based on the level of available information. In this work, both $\mathbf{A}$ and $\mathbf{S}$ are assumed to be unknown. In such a scenario, the above signal separation problem is known as the “Blind” Signal Separation (BSS) problem. Seminal ideas of the BSS problem can be found in [5], where the authors illustrated the idea of BSS via an example of two source signals (n = 2) and two mixture signals (m = 2). Their objective was to recover source signals from the mixture signals, without any further information. However, it has been known since then that the recovery of sources from a linear mixture is imperfect, and pragmatic recovery needs various identifiability conditions. In [6], various approaches and the identifiable conditions of the BSS problem are summarized.

One of the well known traditional solution approaches to the BSS problem is the Independent Component Analysis (ICA) [7]. The ICA-based approaches are built upon the key assumption of the statistical independence among the rows of $\mathbf{S}$. The basic objective of ICA is to separate the input data into statically independent components. During the last three decades, there have been a large amount of papers on signal analysis that are devoted to the usage of ICA. These approaches are designed not only for the linear but also for the nonlinear signal mixing scenarios. They have also been designed for the overdetermined, undetermined, as well as square cases of the BSS problem. Some of the well known approaches in ICA are based on mutual information [8]; negentropy [9]; projection pursuit [10]; infomax [11]. Recent applications of ICA include bio-medical data analysis [12,13,14,15,16,17]; power system analysis [18], and audio and speech [19]. It is out of the scope of this paper to summarize all the typical approaches of ICA. Therefore, interested readers are referred to [7,20] (and the references therein). One of the major critiques pertaining to ICA is the existence of independent sources. Some of the signal analysis areas (such as biomedical and hyperspectral image analysis) may not satisfy the independence criterion (see [21]). Thus, alternative approaches to ICA for BSS have also been a critical research direction in the area of signal analysis.

Other prominent approaches of BSS, apart from ICA, assume some sparsity or geometric structure in $\mathbf{S}$ [22]. In this work, the focus is on the sparsity-based approaches of extracting non-negative sources. This area is widely known as Sparse Component Analysis (SCA) [23]. Specifically, following are the basic assumptions that are considered while solving the SCA problem.

Basic SCA assumptions:

• Every column of the source matrix is non-negative.
• Source matrix has a full row rank.
• Mixing matrix has a full column rank, and $m\ge n$.
• The rows of the source matrix, and columns of the mixing matrix have unit norm.
• Source matrix is sparse.

The first assumption provides a mathematical advantage in designing the solution algorithms. Basically, the non-negativity assumption transforms the BSS problem into a convex programming problem [24,25,26]. In addition to that, non-negative source signals are very common in sound and image analysis. The next two assumptions ensure that the problem is recoverable (solvable). The fourth assumption is perhaps the limit of all BSS approaches, which is related to scalability and uniqueness (see [27]). The fifth assumption is the key sparsity assumption of the SCA-related approaches. Different scenarios of the SCA problem arise with different structures of the sparsity. The typical structures of sparsity discussed in the SCA literature can be classified as:

• Locally Dominant Case: In addition to the basic assumptions, for a given row r of $\mathbf{S}$, there exists at least one unique column c such that:

  $$s_{i,c}\left\{\begin{array}{cc}>0\hfill & \mathrm{if}i=r\hfill \\ =0\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

  (2)
• Locally Latent Case: In addition to the basic assumptions, for a given row r of $\mathbf{S}$, there exists at least $(n-1)$ linearly independent and unique columns $C_r$ such that:

  $$s_{i,c}\left\{\begin{array}{cc}=0\hfill & \mathrm{if}i=r\&c\in C_r\hfill \\ >0\hfill & \mathrm{if}i\ne r\&c\in C_r\hfill \end{array}\right.$$

  (3)
• General Sparse Case: This is the default case.

The first case is one of the widely known cases in the SCA literature (see [28,29] for recent literature reviews). The second case is new to the of SCA literature, and few recent papers address this case [30]. The first two cases have identifiability conditions, which assure identification and recovery of the source signals. If the conditions of the first two cases do not apply to the given data, then the SCA problem belongs to the last general case. Typically, the general case may not have perfect identification (apart from the scalability and uniqueness issues). For the general case, minimum volume-based approaches [26,31,32], and extreme direction-based approaches [33] have been proposed to approximately recover the source matrix. Some special cases of sparse structure, apart from the above cases, that can recover original source data have also been studied in the literature, for example see [34].

In addition to the above, the conditions on $\mathbf{X}$ that improve the separability of sources are studied in the literature [35,36]. Methods that exploit spectral variability can be seen in [37]. Time series (frequency and transformation analysis) based methods to identify sparse sources have been presented in [4,38]. Further methods developed on the assumption of SCA can be found in [39,40]. The prominent application areas of SCA include, but are not limited to, the following: Blind Hyperspectral Unmixing (BHU) [41], chemical analysis [42], Nuclear Magnetic Resonance (NMR) spectroscopy [43], etc. Figure 1 portrays various linear BSS methods available in the literature. The grayed areas in the figure represent the research areas that will not be considered further in this paper. Thus, the branches unrelated to the paper that emerge from the grayed areas are ignored in the figure.

One of the critical gaps found in the SCA literature is the unavailability of a method to test the existence of a locally dominant assumption from the mixture data ($\mathbf{X}$). In this paper, a novel mathematical programming and correntropy-based approach is presented for testing the locally dominant source assumption. The proposed approach also provides a solution for the locally dominant SCA problem.

Throughout this paper, the following notation styles are used. A capital letter bold face character, like $\mathbf{B}$, indicates a matrix. A small letter bold face character with or without a subscript, like ${\mathbf{b}}_r$, indicates the rth column vector of matrix $\mathbf{B}$. A small letter bold face character with a special subscript, like ${\mathbf{b}}_{p•}$, indicates the transpose of the pth row vector of matrix $\mathbf{B}$. A non-bold small letter character, like $b_{p,r}$, represents the pth row rth column element of matrix $\mathbf{B}$.

The rest of the paper is organized as follows: Section 2 introduces the locally dominant case. Specifically, it displays the existing formulations from the literature, and presents the proposed novel formulation. A correntropy-based ranking method to eliminate the non-extreme data points is developed in Section 3. By incorporating the proposed model and the proposed ranking method, a tailored solution approach for the big data separation problem is developed in Section 4. A numerical study to assert the performance of the proposed approach is illustrated in Section 5. Finally, the paper is concluded with discussions in Section 6.

2. Locally Dominant Case

Consider the following determined or square version of the SCA model:

$$\mathbf{X}=\mathbf{A}\mathbf{S}.$$

(4)

Each column ${\mathbf{x}}_i$ for $i=1,\dots ,N$ of the mixture matrix ($\mathbf{X}$) can be represented as follows:

$${\mathbf{x}}_i=\sum _{j=1}^n{s}_{j,i}{\mathbf{a}}_j\forall i=1,\dots ,N,$$

(5)

where ${\mathbf{a}}_j$ is the jth column of the mixing matrix ($\mathbf{A}$), and $s_{j,i}$ is the jth row ith column element of the source matrix ($\mathbf{S}$). Equation (5) highlights that every column vector of $\mathbf{X}$ is a linear combination of the column vectors of $\mathbf{A}$. Since the source matrix ($\mathbf{S}$) is non-negative (i.e., $s_{j,i}\ge 0$ for all i & j), the combination is a conic combination. Thus, the columns of $\mathbf{X}$ are spanned by the columns of $\mathbf{A}$. In other words, the extreme column vectors of $\mathbf{X}$ are the columns of $\mathbf{A}$. Therefore, the locally dominant case boils down to the identification of the extreme vectors of $\mathbf{X}$ [24].

If all the columns of $\mathbf{X}$ are non-negative, then normalizing every column of $\mathbf{X}$ with respect to Norm-1 makes the columns of $\mathbf{X}$ coplanar. That is, all the columns are contained in the following lower dimensional plane:

$$\sum _{j=1}^n{x}_{j,i}=1\forall i=1,\dots ,N.$$

(6)

Now, the extreme points of $\mathbf{X}$ on the lower dimensional plane correspond to the columns of $\mathbf{A}$. In addition to that, if some of the elements of $\mathbf{X}$ are negative, then the columns of $\mathbf{X}$ are projected onto a suitable lower dimensional plane. There are many approaches in the literature that are designed to work on this lower dimensional plane (affine hull) [25,44]. The advantage of working on this plane is that the extreme vector columns of $\mathbf{X}$ will form the vertices of a lower dimensional simplex. Thus, identifying the extreme points will result in the identification of the mixing matrix. Next, a few well known mathematical formulations and solution approaches for SCA from the literature are presented.

2.1. Conventional Formulations

One of the earliest mathematical formulations that identifies the extreme vectors of $\mathbf{X}$ is proposed in [24]. The idea is to pick one column of $\mathbf{X}$, say ${\mathbf{x}}_c$, and check the possibility of it being an extreme vector. The formulation corresponding to ${\mathbf{x}}_c$ is given as follows:

$$\begin{array}{cc}\hfill min.:& \hfill \\ & {∥{\displaystyle \sum _{\begin{array}{c}i=1\\ i\ne c\end{array}}^N{\alpha }_i{\mathbf{x}}_i-{\mathbf{x}}_c}∥}_2\hfill \\ \hfill s.t.:& \hfill \end{array}$$

(7)

$$\begin{array}{c}{\alpha }_i\ge 0\forall i=1,\dots ,N,\&i\ne c,\hfill \end{array}$$

(8)

where ${\alpha }_i\ge 0$, $\in \mathbb{R}$ is the variable that corresponds to the weight of ${\mathbf{x}}_i\in \mathbf{X}$ for $i=1,\dots ,N$. The key idea that is exploited in the formulation is that the extreme vectors cannot be represented by a non-negative weighted combination of the other data vectors. The above formulation is a least square minimization problem. In the worse case, the formulation has to be executed N times for each ${\mathbf{x}}_c$, i.e., for $c=1,\dots ,N$. In the best case, the formulation has to be executed n times. A nonzero value of the objective function indicates that the vector ${\mathbf{x}}_c$ is an extreme vector of $\mathbf{X}$.

Another approach, called Convex Analysis of Mixtures of Non-negative Sources (CAMNS), that works on the affine hull is presented in [25]. The solution approach of CAMNS involves two major steps. In the first step, the parameters $\mathbf{C}\in {\mathbb{R}}^{N\times (n-1)}$ and $\mathbf{d}\in {\mathbb{R}}^N$ of the affine hull are estimated as follows:

$$\begin{array}{c}\hfill \mathbf{d}=\frac{1}{N}\sum _{j=1}^N{\mathbf{x}}_{j•}\end{array}$$

(9)

$$\begin{array}{c}\hfill \mathbf{U}={\mathbf{X}}^T-\mathbf{d}\end{array}$$

(10)

$$\begin{array}{c}\hfill \mathbf{C}={\left[eigv\left(\mathbf{U}{\mathbf{U}}^T\right)\right]}_{1,\dots ,(n-1)}\end{array}$$

(11)

where ${\mathbf{x}}_{j•}$ is the jth row of $\mathbf{X}$. In Equation (10), vector $\mathbf{d}$ is subtracted from all the rows of $\mathbf{X}$, and in Equation (11), matrix $\mathbf{C}$ contains the columns corresponding to the eigenvectors associated with the largest $(n-1)$ eigenvalues of $\mathbf{U}{\mathbf{U}}^T$. Basically, this first step is similar to the dimensionality reduction process executed in the PCA. In the second step, the following mathematical model is repeatedly solved:

$$\begin{array}{cc}\hfill max/min:& \hfill \\ & {\mathbf{r}}^T(\mathbf{C}\mathit{\alpha }+\mathbf{d})\hfill \\ \hfill s.t.:& \hfill \end{array}$$

(12)

$$\begin{array}{c}{\mathbf{c}}_{i•}^T\mathit{\alpha }+d_i\ge 0\forall i=1,\dots ,N\hfill \end{array}$$

(13)

where $\mathbf{r}\in {\mathbb{R}}^N$ is a generated vector, ${\mathbf{c}}_{i•}$ is an ith row of $\mathbf{C}$, and $\mathit{\alpha }\in {\mathbb{R}}^{(n-1)}$ is the unknown variable. The above formulation exploits the notion that the optimal solution of a linear program exists at the extreme points. For a given $\mathbf{r}$, the formulation is solved twice; one time as the maximization problem and the other time as the minimization problem. This is done in order to get one (or maybe two) new extreme point(s) in every iteration. The $(n-1)$ extreme points are identified by resolving the above formulation repeatedly with respect to different $\mathbf{r}$ vectors. The crux of this method is hidden in the generation of $\mathbf{r}$ and convergence of the approach, which are the main concepts presented in [25].

Notice that the above two ideologies extract the columns of $\mathbf{A}$ sequentially. In addition to the above approaches, typical approaches in the literature identify the columns of $\mathbf{A}$ sequentially [28,29,45]. Therefore, there is no mechanism to validate the locally dominant assumption from $\mathbf{X}$. One of the main objectives of this paper is to build such a mechanism using a mathematical formulation-based approach. In the following subsection, a novel formulation is presented that can provide the above mechanism.

2.2. Envelope Formulation

In this section, a mathematical model that can simultaneously identify all the extreme vectors of $\mathbf{X}$ under the locally dominant assumption is proposed. To the best of our knowledge, an exact method that identifies all the extreme vectors in a single shot is unavailable.

Let ${\mathbf{X}}_2$ be the data obtained after normalizing each column of $\mathbf{X}$ with respect to Norm-2. Let ${\mathbf{y}}_i\in {\mathbf{X}}_2$ be the ith column of ${\mathbf{X}}_2$. Let ${\mathbf{a}}^{\mathbf{T}}\mathbf{y}=b$ be a plane that is inclined in such a way that all the columns of ${\mathbf{X}}_2$ are contained in one halfspace of the hyperplane, and the origin is in the other halfspace. Such a hyperplane is referred to as linear envelope in this work. The envelope can be written as:

$${\mathbf{a}}^{\mathbf{T}}{\mathbf{y}}_{\mathbf{i}}\ge b\forall i=1,\dots ,N,$$

(14)

where $\mathbf{a}\in {\mathbb{R}}^m$ corresponds to the normal vector of the envelope, and $b\ge 0$ is a constant. Out of infinite possible representations of the above envelope, an envelope with $b=1$ is selected for further analysis. Now, the distance between any vector ${\mathbf{y}}_i$ and the envelope can be written as:

$$\begin{array}{cc}\hfill d\left({\mathbf{y}}_i\right)& =\frac{|{\mathbf{a}}^{\mathbf{T}}{\mathbf{y}}_{\mathbf{i}}-1|}{{\left|\right|\mathbf{a}\left|\right|}_2}\hfill \end{array}$$

(15)

$$\begin{array}{c}=\frac{{\mathbf{a}}^{\mathbf{T}}{\mathbf{y}}_{\mathbf{i}}-1}{{\left|\right|\mathbf{a}\left|\right|}_2}\hfill \end{array}$$

(16)

$$\begin{array}{c}\propto {\mathbf{a}}^{\mathbf{T}}{\mathbf{y}}_{\mathbf{i}}-1=p\left({\mathbf{y}}_i\right)\forall i=1,\dots ,N.\hfill \end{array}$$

(17)

Equation (16) follows from Equation (14). Ignoring the denominator (${\left|\right|\mathbf{a}\left|\right|}_2$) in Equation (16) results in a proportional or scaled distance ($p\left({\mathbf{y}}_i\right)$). Among infinite possible linear envelopes, the aim is to find the tightest or supporting envelope. A formulation that can identify the tightest envelope is given as follows:

$$\begin{array}{cc}\hfill min.:& \hfill \\ & \sum _{i=1}^N({\mathbf{a}}^T{\mathbf{y}}_i-1)\hfill \\ \hfill s.t.:& \hfill \end{array}$$

(18)

$$\begin{array}{c}{\mathbf{a}}^T{\mathbf{y}}_i\ge 1\forall i=1,\dots ,N,\hfill \end{array}$$

(19)

where $\mathbf{a}\in {\mathbb{R}}^m$ is the unknown variable. The above formulation can be equivalently written as:

$$\underset{-}{\mathrm{Formulation}}:$$

(20)

$$\begin{array}{cc}\hfill min.:& \hfill \\ & {\mathit{\mu }}^T\mathbf{a}\hfill \\ \hfill s.t.:& \hfill \end{array}$$

(21)

$$\begin{array}{c}{\mathbf{y}}_i^T\mathbf{a}\ge 1\forall i=1,\dots ,N,\hfill \end{array}$$

(22)

where $\mathit{\mu }\in {\mathbb{R}}^m$ is defined as $\mathit{\mu }={\sum }_{i=1}^N{\mathbf{y}}_i$. It is assumed that the duplicate and/or all zero columns of ${\mathbf{X}}_2$ are removed before the execution of the above model. The aim of the above model is to find the envelope that has the minimum distance with respect to all the columns of ${\mathbf{X}}_2$. The above formulation is linear, and needs to be executed only once to identify all the extreme vectors. That is, at the optimal solution, the data points (${\mathbf{y}}_i$’s) corresponding to the active constraints in Equation (19) will correspond to the extreme vectors of ${\mathbf{X}}_2$. It can be seen that the above formulation is always feasible. Furthermore, due to the design of the constraint given in Equation (19), the problem is linear. That is, the design allowed the usage of proportional distance (Equation (17)) instead of the nonlinear actual distance (Equation (15)).

In addition to that, from the LP theory, only m constraints will be active at the optimal solution. Let $\Theta$ be the matrix containing the columns of $\mathbf{X}$ corresponding to the m active constraints. Identifying the columns of $\mathbf{A}$ requires the following additional steps: Calculate ${\mathbf{q}}_i={\Theta }^{-1}{\mathbf{x}}_i$ for $i=1,\dots ,N$. If ${\mathbf{q}}_i\ge \mathbf{0}$ for $i=1,\dots ,N$, then $\Theta$ corresponds to $\mathbf{A}$. However, if any element of ${\mathbf{q}}_i$ is strictly less than 0 for any $i=1,\dots ,N$, then it indicates that the locally dominant assumption is invalid for the given $\mathbf{X}$. Thus, this serves as a test for the existence of the locally dominant assumption.

The above test works for typical data mixing without noise. However, the image data is usually integer data. There is always rounding, taking place at the source or mixture level. Therefore, the rounding effect needs to be incorporated into the above condition. It is proposed in this paper to follow the heuristic method to incorporate the rounding effect. Let $\mathit{\nu }=0.5|{\Theta }^{-1}\mathbf{e}|$, where $\mathbf{e}\in {\mathbb{R}}^n$ is a vector of all ones. The notion is that the rounding will create a maximum error of $\pm 0.5$ in each pixel element. Thus, the maximum error in any pixel element will be strictly less than $0.5$. Therefore, the check for image data will be as follows: If ${\mathbf{q}}_i+\mathit{\nu }\ge \mathbf{0}$ for $i=1,\dots ,N$, then $\Theta$ corresponds to $\mathbf{A}$.

Furthermore, the above idea can be extended for mixing scenarios containing noise. For instance, a level of tolerance can be used to analyze the noisy mixture data. Let $\mathit{\psi }\ge \mathbf{0}$, $\in {\mathbb{R}}^m$ be a tolerance parameter selected by the user. Then, based on the earlier discussion, the check for noisy data will be as follows: If ${\mathbf{q}}_i+\mathit{\psi }\ge \mathbf{0}$ for $i=1,\dots ,N$, then $\Theta$ corresponds to $\mathbf{A}$. The precise value of $\mathit{\psi }$ may not be available for a given scenario. Hence, the parameter will be empirically selected based on trial experiments. In Section 5, an experiment that highlights the usage of parameter $\mathit{\psi }$ for noisy data is illustrated.

3. Point Correntropy

Correntropy is a generalized correlation measure based on the concept of entropy. It is typically used in detecting local similarity between two random variables. Roughly speaking, it is a kernelized version of the conventional correlation measure. The measure first appeared in [46,47], and its usage as a cost function was illustrated in [48,49,50,51,52]. The optimization properties of the cost function are presented in [53]. The correntropy cost function (or the correntropic loss) for N errors is defined as:

$$\mathcal{F}\left(\mathit{\epsilon }\right)=\beta (1-\frac{1}{N}\sum _{i=1}^{N}k({\epsilon }_i,\sigma )),$$

(23)

where $\beta ={\left[1-e^{\left(\frac{-1}{2{\sigma }^2}\right)}\right]}^{-1}$ is a scaling parameter, $\mathit{\epsilon }\in {\mathbb{R}}^N$ is an array of errors, and $k\left(\right)$ is the transformation kernel function with parameter $\sigma$. In this work, a Gaussian kernel is selected, i.e., $k(\epsilon ,\sigma )=e^{\left(\frac{-{\epsilon }^2}{2{\sigma }^2}\right)}$. Equation (23) is readily separable with respect to sample errors, and can be rewritten as:

$$\mathcal{F}\left(\mathit{\epsilon }\right)=\sum _{i=1}^{N}f\left({\epsilon }_i\right)$$

(24)

where $f\left({\epsilon }_i\right)=\frac{\beta }{N}(1-k({\epsilon }_i,\sigma ))$, and it will be referred to as point estimate of the correntropic loss.

Let ${\epsilon }_i$ be an error corresponding to the ith column vector of $\mathbf{X}$. For a given kernel parameter $\sigma$, the point estimate provides similarity information of the ith vector with respect to the other data vectors of $\mathbf{X}$. Based on the geometry of the vectors, the extreme vectors’ similarity with respect to the central vectors should be typically less than the other non-extreme vectors of $\mathbf{X}$. Thus, the point estimate of the correntropic loss function can be used as a measure to differentiate extreme and non-extreme vectors of $\mathbf{X}$.

4. Solution Methodology

Our goal is to develop a geometric separation method for the non-negative data mixing problem, that can be applied to the “Big Data” scenarios. The concepts developed in Section 2 and Section 3 are tailored with respect to big data, and the following solution approach is proposed. The summary of the proposed approach is illustrated in Figure 2 and Algorithm 1.

Algorithm 1: The Proposed Algorithm.

Data: Given $\mathbf{X}\in {\mathbb{R}}^{m\times N}$  Result: Find $\mathbf{A}\in {\mathbb{R}}^{m\times n}$ and $\mathbf{S}\in {\mathbb{R}}_{+}^{n\times N}$ such that $\mathbf{X}=\mathbf{A}\mathbf{S}$  ${\mathbf{X}}_2$ = normalize($\mathbf{X}$);  Remove all zero columns and duplicate columns from ${\mathbf{X}}_2$, and say ${\mathbf{X}}_2\in {\mathbb{R}}^{m\times \widehat{N}}$;  Estimate $\sigma$ from $\mathbb{S}$;  Obtain ${\mathbf{X}}_R$ by removing all columns with the 50 percentile point correntropy criterion from ${\mathbf{X}}_2$;  Let ${\mathbf{X}}_E={\mathbf{X}}_2\backslash {\mathbf{X}}_R$;  Let ${\mathbf{y}}_i$ be the ith column of ${\mathbf{X}}_2$;  $\mathbf{a}$ = Solution of LP Formulation (20) with respect to data ${\mathbf{X}}_R$;  while ${\mathbf{a}}^T{\mathbf{y}}_i<1fori=1,\dots ,N$ do    end  Let $\Theta$ be the matrix containing the columns of $\mathbf{X}$ corresponding to the active constraints at the optimal solution of Formulation (20);  Calculate ${\mathbf{q}}_i={\Theta }^{-1}{\mathbf{x}}_i$ for $i=1,\dots ,N$;  Set $\mathit{\psi }$ equal to 0 for non-noisy non-image data mixing, equal to $\mathit{\nu }$ for non-noisy image data mixing, or equal to the user-specified value for noisy mixing.;  if ${\mathbf{q}}_i+\mathit{\psi }\ge \mathbf{0}\forall i=1,\dots ,N$ then    else    end

Data Ranking: As seen earlier, the extreme vectors of ${\mathbf{X}}_2$ contain all the relevant information that is needed for separation (i.e., identifying $\mathbf{A}$ and $\mathbf{S}$). Other data vectors are redundant in identifying the mixing matrix. Thus, the point estimate of the correntropic loss can be evaluated at all the data points with respect to the central columns of ${\mathbf{X}}_2$. Those data points that have low value of the correntropic loss can be removed from the data set ${\mathbf{X}}_2$ without losing any information. The major issues in implementing the above idea are as follows: how to select the right value for $\sigma$, and how to define ${\epsilon }_i$ corresponding to ${\mathbf{y}}_i$ for $i=1,\dots ,N$.

The value of $\sigma$ represents the kernel width, and should be large enough to contain the central vectors. However, it should be small enough to exclude the extreme vectors. In the following, we propose a practical method to estimate the value of $\sigma$. Let $\mathbb{S}$ be a sample of columns randomly selected from ${\mathbf{X}}_2$. Let ${\delta }_j={\displaystyle \underset{\begin{array}{c}\forall i1,i2\in \mathbb{S}\\ i1<i2\end{array}}{max}}\{y_{j,i1}-y_{j,i2}\}$, where $y_{j,i1}$ is the jth element of ${\mathbf{y}}_{i1}$. Based on the trial experiments, we found that $\sigma =\frac{{\sum }_{j=1}^n{\delta }_j}{\sqrt{2n}}$ is a good choice for the kernel width. Furthermore, the value of $\mathbf{c}$ should correspond to the center of the columns of ${\mathbf{X}}_2$. An approximate estimate for $\mathbf{c}$ can be $\mathbf{c}=\frac{\mathbf{g}+\mathbf{h}}{2}$, where $g_j={\displaystyle \underset{j}{max}}\left\{y_{j,i}\right\}$, $h_j={\displaystyle \underset{j}{min}}\left\{y_{j,i}\right\}$ for $i\in \mathbb{S}$. Thus, simple (and practical for big data) estimate of error for ${\mathbf{y}}_i$ will be ${\epsilon }_i={\mathbf{y}}_i-\mathbf{c}$ for $i=1,\dots ,N$. Based on the trial experiments, it can be concluded that the larger the size of $\mathbb{S}$, the better the estimation. Furthermore, the strategy to eliminate the columns from the trial experiments is as follows. All the columns of ${\mathbf{X}}_2$ that have the point estimate value lower than 50 percentile are removed from further consideration.

Handling a Large Number of Constraints: From Formulation (20), it can be seen that the big data corresponds to a large number of constraints. However, only m constraints are active, and the rest of the constraints are redundant (i.e., the rest of the constraints will never be active). The proposed data ranking method eliminates a good amount of the redundant constraints, depending upon the distribution of columns in the data set.

Let ${\mathbf{X}}_R\subseteq {\mathbf{X}}_2$ be the data matrix obtained after eliminating possible central vectors, and let ${\mathbf{X}}_E\subset {\mathbf{X}}_2$ be the eliminated central vectors. If the columns of ${\mathbf{X}}_2$ are reorganized, then there exists a partition such that ${\mathbf{X}}_2=\left[{\mathbf{X}}_R\right|{\mathbf{X}}_E]$. Once the tightest envelope is obtained for ${\mathbf{X}}_R$ by solving Formulation (20), the envelope is validated with respect to ${\mathbf{X}}_E$. If all the columns of ${\mathbf{X}}_E$ fall in the same half space (i.e., Equation (14) is feasible with respect to ${\mathbf{X}}_2$), then it can be guaranteed that none of the extreme vectors of ${\mathbf{X}}_2$ were eliminated. However, if there is any infeasibility detected, then the column of ${\mathbf{X}}_E$ with maximum infeasibility is added to the LP, and the LP is resolved.

Performance Index: In order to evaluate the performance of the proposed approach, distance-based metrics will be used. Specifically, two metrics (one for the mixing matrix, and the other for the source matrix) will be used in this work. The following error measure is for the mixing matrix:

$$e_A=\frac{1}{n^2}\sum _{j=1}^n{∥\frac{{\mathbf{a}}_j}{\left|\right|{\mathbf{a}}_j\left|\right|}-\frac{{\widehat{\mathbf{a}}}_{\left[j\right]}}{\left|\right|{\widehat{\mathbf{a}}}_{\left[j\right]}\left|\right|}∥}_2$$

(25)

where ${\mathbf{a}}_j$ is the jth column of the original mixing matrix $\mathbf{A}$, and ${\widehat{\mathbf{a}}}_{\left[j\right]}$ is the corresponding column to ${\mathbf{a}}_j$, obtained from the recovered mixing matrix $\widehat{\mathbf{A}}$. The corresponding columns are identified by the Hungarian algorithm [54]. The source matrix is obtained as follows:

$$\widehat{\mathbf{S}}=\underset{\mathbf{S}\in {\mathbb{R}}_{+}^{n\times N}}{argmin}\{\parallel \mathbf{X}-\widehat{\mathbf{A}}\mathbf{S}{\parallel }_2\}$$

(26)

The above approach can be replaced by $\widehat{\mathbf{S}}={\widehat{\mathbf{A}}}^{-1}\mathbf{X}$, whenever ${\widehat{\mathbf{A}}}^{-1}$ exists. Similar to $e_A$, the following error measure is for the source matrix:

$$e_S=\frac{1}{nN}\sum _{j=1}^n{∥\frac{{\mathbf{s}}_{j•}}{\left|\right|{\mathbf{s}}_{j•}\left|\right|}-\frac{{\widehat{\mathbf{s}}}_{[j•]}}{\left|\right|{\widehat{\mathbf{s}}}_{[j•]}\left|\right|}∥}_2$$

(27)

where ${\mathbf{s}}_{j•}$ is the jth row of the original source matrix $\mathbf{S}$, and ${\widehat{\mathbf{s}}}_{[j•]}$ is the corresponding row to ${\mathbf{s}}_{j•}$, obtained from the recovered source matrix $\widehat{\mathbf{S}}$.

5. Numerical Experiments

In order to illustrate the performance of the proposed approach, numerical experiments are presented. The experiments are divided into four groups. In the first group of experiments, simulated non-noisy data is used to test the performance and sensitivity of the proposed approach. In the second group of experiments, image data mixtures are used to test the applicability of the proposed approach on real image data. The third and fourth group of experiments compare the proposed approach with the well known SCA methods in the literature. In all the instances of this section, the following specifications were used: $\mathbb{S}=\{1,\dots ,N\}$. All the random mixing matrices contain columns with unit norm. The LP resolving step is skipped in order to identify the number of instances in which the extreme vectors were eliminated. The LP was solved via the dual simplex method, using the state of the art Cplex 12.0 solver [55]. All the instances were solved on an Intel Xeon 2.4 GHz workstation, with 16 logical processors and 32 GB of RAM.

5.1. Simulated Data Separation

Setup: Given n and N, a random source matrix $\mathbf{S}$ that satisfies the locally dominant assumption is generated. For the generated $\mathbf{S}$ matrix, 100 random mixing matrices $\mathbf{A}$ are generated. Then, using the $\mathbf{X}=\mathbf{A}\mathbf{S}$ equation, 100 random mixture matrices are generated. On each mixture matrix, the proposed approach is implemented. This study is executed for all the following combinations of n $=5,7,9$ and 11, and N = 1000, 5000, 10,000, 50,000, 100,000, 500,000 and 1,000,000. In addition to that, this study is also executed for n $=10,20,40,60,80$ and 100 when N = 1,000,000. The value of $\mathit{\psi }$ is set to zero in this experiment.

Results: Using one mixture matrix as input, and using the proposed approach, matrices $\widehat{\mathbf{A}}$ and $\widehat{\mathbf{S}}$ are recovered. This experiment is repeated 100 times for a given combination of n and N. The performance of the proposed approach is displayed in

Table 1. Performance of the proposed approach on the simulated data.

n × N	mErrA	vErrA	mErrS	vErrS	mTime	vTime	mRed	vRed	nMiss
5 × 1000	1.04 × 10${}^{-17}$	9.78 × 10${}^{-36}$	1.17 × 10${}^{-20}$	4.76 × 10${}^{-42}$	0.0647	0.07095	50	0	0
5 × 5000	1.08 × 10${}^{-17}$	1.16 × 10${}^{-35}$	2.03 × 10${}^{-21}$	5.24 × 10${}^{-43}$	0.0756	0.04114	50	0	0
5 × 10,000	1.07 × 10${}^{-17}$	1.26 × 10${}^{-35}$	9.50 × 10${}^{-22}$	1.88 × 10${}^{-43}$	0.08	0.16436	50	0	0
5 × 50,000	1.10 × 10${}^{-17}$	1.13 × 10${}^{-35}$	5.10 × 10${}^{-22}$	5.39 × 10${}^{-44}$	0.1427	0.09452	50	0	0
5 × 100,000	1.02 × 10${}^{-17}$	8.22 × 10${}^{-36}$	4.85 × 10${}^{-22}$	3.21 × 10${}^{-44}$	0.2254	0.12459	50	0	0
5 × 500,000	1.01 × 10${}^{-17}$	9.28 × 10${}^{-36}$	1.81 × 10${}^{-22}$	6.50 × 10${}^{-45}$	1.0166	2.2	50	0	0
5 × 1,000,000	1.03 × 10${}^{-17}$	1.11 × 10${}^{-35}$	1.19 × 10${}^{-24}$	8.55 × 10${}^{-50}$	2.0526	1.9	50	0	0
7 × 1000	7.28 × 10${}^{-18}$	3.47 × 10${}^{-36}$	1.30 × 10${}^{-20}$	3.41 × 10${}^{-42}$	0.0641	0.05835	50	0	0
7 × 5000	6.85 × 10${}^{-18}$	2.72 × 10${}^{-36}$	1.71 × 10${}^{-21}$	1.08 × 10${}^{-43}$	0.0816	0.06362	50	0	0
7 × 10,000	6.86 × 10${}^{-18}$	2.28 × 10${}^{-36}$	1.01 × 10${}^{-21}$	7.70 × 10${}^{-44}$	0.0891	0.11015	50	0	0
7 × 50,000	7.08 × 10${}^{-18}$	1.89 × 10${}^{-36}$	7.05 × 10${}^{-22}$	4.27 × 10${}^{-44}$	0.1689	0.14786	50	0	0
7 × 100,000	6.78 × 10${}^{-18}$	2.46 × 10${}^{-36}$	1.14 × 10${}^{-22}$	5.67 × 10${}^{-45}$	0.2675	0.14104	50	0	0
7 × 500,000	7.26 × 10${}^{-18}$	2.61 × 10${}^{-36}$	6.11 × 10${}^{-23}$	1.80 × 10${}^{-45}$	1.2584	1.3	50	0	0
7 × 1,000,000	6.85 × 10${}^{-18}$	2.62 × 10${}^{-36}$	9.93 × 10${}^{-23}$	1.77 × 10${}^{-45}$	2.5154	3.3	50	0	0
9 × 1000	5.50 × 10${}^{-18}$	1.05 × 10${}^{-36}$	1.59 × 10${}^{-20}$	2.94 × 10${}^{-42}$	0.067	0.10896	50	0	0
9 × 5000	5.82 × 10${}^{-18}$	1.42 × 10${}^{-36}$	2.44 × 10${}^{-21}$	1.96 × 10${}^{-43}$	0.0812	0.05954	50	0	0
9 × 10,000	5.62 × 10${}^{-18}$	1.30 × 10${}^{-36}$	8.13 × 10${}^{-22}$	4.63 × 10${}^{-44}$	0.0835	0.08635	50	0	0
9 × 50,000	5.52 × 10${}^{-18}$	1.38 × 10${}^{-36}$	2.62 × 10${}^{-22}$	5.69 × 10${}^{-45}$	0.1821	0.12421	50	0	0
9 × 100,000	5.57 × 10${}^{-18}$	1.31 × 10${}^{-36}$	3.79 × 10${}^{-22}$	1.68 × 10${}^{-44}$	0.3066	0.1855	50	0	0
9 × 500,000	5.40 × 10${}^{-18}$	1.36 × 10${}^{-36}$	9.78 × 10${}^{-23}$	1.84 × 10${}^{-45}$	1.5029	1	50	0	0
9 × 1,000,000	5.32 × 10${}^{-18}$	1.21 × 10${}^{-36}$	1.49 × 10${}^{-22}$	2.10 × 10${}^{-45}$	3.0258	3.1	50	0	0
11 × 1000	4.05 × 10${}^{-18}$	4.77 × 10${}^{-37}$	1.75 × 10${}^{-20}$	3.13 × 10${}^{-42}$	0.0672	0.02826	50	0	0
11 × 5000	4.16 × 10${}^{-18}$	5.04 × 10${}^{-37}$	2.03 × 10${}^{-21}$	9.47 × 10${}^{-44}$	0.096	0.10778	50	0	0
11 × 10,000	4.21 × 10${}^{-18}$	4.87 × 10${}^{-37}$	1.31 × 10${}^{-21}$	7.33 × 10${}^{-44}$	0.0916	0.00928	50	0	0
11 × 50,000	4.09 × 10${}^{-18}$	5.02 × 10${}^{-37}$	6.61 × 10${}^{-22}$	2.97 × 10${}^{-44}$	0.2137	0.05729	50	0	0
11 × 100,000	4.12 × 10${}^{-18}$	3.79 × 10${}^{-37}$	1.90 × 10${}^{-22}$	6.92 × 10${}^{-45}$	0.355	0.25758	50	0	0
11 × 500,000	4.14 × 10${}^{-18}$	3.96 × 10${}^{-37}$	1.18 × 10${}^{-22}$	1.66 × 10${}^{-45}$	1.8358	0.69888	50	0	0
11 × 1,000,000	4.21 × 10${}^{-18}$	4.37 × 10${}^{-37}$	8.91 × 10${}^{-23}$	8.11 × 10${}^{-46}$	3.667	5.4	50	0	0
10 × 1,000,000	4.71 × 10${}^{-18}$	7.54 × 10${}^{-37}$	8.46 × 10${}^{-23}$	8.89 × 10${}^{-46}$	3.4245	2.5	50	0	0
20 × 1,000,000	1.83 × 10${}^{-18}$	5.10 × 10${}^{-38}$	1.12 × 10${}^{-22}$	6.32 × 10${}^{-46}$	6.6053	9.3	50	0	0
40 × 1,000,000	7.84 × 10${}^{-19}$	5.33 × 10${}^{-39}$	4.22 × 10${}^{-23}$	9.61 × 10${}^{-47}$	14.8988	64.8	50	0	0
60 × 1,000,000	7.32 × 10${}^{-19}$	4.62 × 10${}^{-38}$	1.05 × 10${}^{-22}$	2.18 × 10${}^{-46}$	20.2509	628.9	50	0	0
80 × 1,000,000	6.51 × 10${}^{-19}$	6.39 × 10${}^{-39}$	7.51 × 10${}^{-23}$	1.40 × 10${}^{-46}$	27.0654	101.7	50	0	0
100 × 1,000,000	5.13 × 10${}^{-19}$	2.83 × 10${}^{-39}$	4.65 × 10${}^{-23}$	5.57 × 10${}^{-47}$	33.7978	136.6	50	0	0

. The column corresponding to mErrA (vErrA) indicates the mean (variance) of error $e_A$ over the 100 instances. Similarly, columns mErrS and vErrS correspond to the mean and variance of error $e_S$ respectively. The column corresponding to mTime (vTime) indicates the mean (variance) of the solution time per instance in seconds (milliseconds) over the 100 instances. In addition to that, the column corresponding to mRed (vRed) indicates the mean (variance) of the percentage of columns eliminated over the 100 instances. Finally, the column corresponding to nMiss indicates the number of times the extreme vectors were eliminated based on the 50 percentile criterion. Since the mixtures are clean (i.e., no noise is added to the mixture data), the recovery is perfect. This can be seen from the very low average error (mErrA and mErrS) over the 100 iterations. Furthermore, the method is consistent in the recovery of the matrices, and it can be justified from the low variance in the error (vErrA and vErrS). The suitability and applicability of the proposed approach to big data can be seen from the solution time. For instance, Figure 3 and Figure 4 illustrate the average time in seconds required to solve one instance of the proposed approach for N data points and n data sources. The behavior of the solution time with respect to ${\log }_{10}\left(N\right)$ is exponential. In other words, the solution time increases linearly with respect to N. Furthermore, from Figure 4, it can be observed that the solution time is linear with respect to n. Thus, the algorithm is suitable for big data scenarios. Moreover, the 50 percentile criterion removes exactly 50% of the data points in all the cases, with zero variance. This is due to the fact that the source matrices are uniformly randomly generated. Due to the uniform generation of the source matrices, none of the extreme vectors were eliminated.

5.2. Image Mixture Separation

Setup: In the following experiments, image data available from the literature and online repositories are considered (see

Table 2. Image Mixture Separation Data.

Image Set	n	N
Chest X-rays	2	26,896
Scenery	3	65,536
CT Scans	5	16,384
Zip Codes	7	12,672
Finger Print	9	90,000

). Each source image of an image set is reshaped into one row vector. Then, the reshaped images are row-wise stacked together to generate the $\mathbf{S}$ matrix. The source matrices are pre-processed in order to satisfy the locally dominant assumption. Next, for each source matrix $\mathbf{S}$, 100 random $\mathbf{A}$ matrices are generated, and correspondingly 100 random $\mathbf{X}$ matrices are analyzed using the proposed approach. Table 2 summarizes the details of the image sets that are considered in this subsection. The first column conveys the name of the image set that is being considered. The column corresponding to n indicates the total number of sources, and the column corresponding to N indicates the total data points (or column vectors) in $\mathbf{X}$. The value of $\mathit{\psi }$ is set to $\mathit{\nu }$ in this experiment.

Results:

Table 3. Image mixture separation results.

Image Set	mErrA	vErrA	mErrS	vErrS	mTime	vTime	mRed	vRed	nMiss
Chest X-rays	2.45 × 10${}^{-17}$	2.81 × 10${}^{-34}$	1.33 × 10${}^{-22}$	2.02 × 10${}^{-44}$	0.0755	4.89 × 10${}^{-5}$	81.632	0.0101	0
Scenery	2.06 × 10${}^{-17}$	5.30 × 10${}^{-35}$	4.10 × 10${}^{-22}$	4.12 × 10${}^{-44}$	0.109	9.81 × 10${}^{-5}$	73.2417	0.0033	7
CT Scan	1.19 × 10${}^{-17}$	9.09 × 10${}^{-36}$	4.80 × 10${}^{-22}$	1.20 × 10${}^{-44}$	0.0679	1.34 × 10${}^{-4}$	89.7026	0.0012	4
Zip Codes	7.36 × 10${}^{-18}$	2.61 × 10${}^{-36}$	4.96 × 10${}^{-22}$	8.17 × 10${}^{-45}$	0.0787	3.74 × 10${}^{-5}$	74.0513	0.0047	6
Finger Print	5.72 × 10${}^{-18}$	1.19 × 10${}^{-36}$	1.67 × 10${}^{-22}$	2.23 × 10${}^{-45}$	0.2716	1.57 × 10${}^{-4}$	55.952	6.49 × 10${}^{-6}$	0

displays the results after executing the proposed approach on the 100 mixture instances of each image set. The columns have the notation similar to the earlier experiment, except that the vTime column units are in seconds. Moreover, Figure 5, Figure 6, Figure 7, Figure 8 and Figure 9 depict the results. Low mean errors ($e_A$ and $e_S$) over the 100 runs are obtained for all the image sets. This shows that the method precisely recovers $\mathbf{A}$ and $\mathbf{S}$ matrices. A low value in the corresponding variance column indicates the high level of consistency of the proposed approach. The solution time, specifically for the finger print data set, indicates the applicability of the proposed approach for big data with complex image mixing scenarios. Based on the results, it can be seen that the 50 percentile criterion eliminates a good amount (more than 50%) of the redundant columns. However, in some instances (at most 7 percentage in one instance), the criterion eliminated some of the extreme vectors.

5.3. Comparative Experiment-I

Setup: Given n and N, a random source matrix $\mathbf{S}$ that does not satisfy the locally dominant assumption is generated in this experiment. For the generated $\mathbf{S}$ matrix, 100 random mixing matrices $\mathbf{A}$ are generated. Then, using the $\mathbf{X}=\mathbf{A}\mathbf{S}$ equation, 100 random mixture matrices are generated. This study is executed for n $=5,7,9,11,13$ and 15, and for N = 10,000. The value of $\mathit{\psi }$ is set to zero in this experiment. Well known methods from the SCA literature are compared with the proposed approach. The methods that are used for the comparison are N-FINDR ([44]), VCA ([45]), MVSA ([56]). The objective of this experiment is to highlight that the above three (like the other typical algorithms in the literature) do not have the capability to test the locally dominant assumption from the knowledge of $\mathbf{X}$. To the best of my knowledge, only an exhaustive search similar to the one presented in [24] can do such a test. However, the proposed approach does not require such an exhaustive search.

Results:

Table 4. Comparative experiment-I separation results.

n	N	VCA		MVSA		N-FINDR		Proposed
		mErrA	vErrA	mErrA	vErrA	mErrA	vErrA	ErrA	TnMiss
5	10,000	0.0755	7.08 × 10${}^{-5}$	0.0813	6.65 × 10${}^{-5}$	0.0905	4.14 × 10${}^{-5}$	—	100
7	10,000	0.056	9.63 × 10${}^{-6}$	0.0567	2.6 × 10${}^{-5}$	0.0604	7.23 × 10${}^{-6}$	—	100
9	10,000	0.0422	2.96 × 10${}^{-6}$	0.0402	1.13 × 10${}^{-5}$	0.0441	1.34 × 10${}^{-6}$	—	100
11	10,000	0.0333	8.56 × 10${}^{-7}$	0.0314	4.16 × 10${}^{-6}$	0.0342	7.16 × 10${}^{-7}$	—	100
13	10,000	0.0269	3.28 × 10${}^{-7}$	0.0252	1.56 × 10${}^{-6}$	0.0274	3.6 × 10${}^{-7}$	—	100
15	10,000	0.0223	1.52 × 10${}^{-7}$	0.0212	7.53 × 10${}^{-7}$	0.0226	1.8 × 10${}^{-7}$	—	100

displays the results after executing the proposed and selected approaches on the 100 randomly generated mixture instances. The column corresponding to mErrA (vErrA) indicates the mean (variance) of error $e_A$ over the 100 instances for the three methods used from the literature. The lines in the column corresponding to ErrA indicate that the proposed approach was unable to identify any mixing matrix. The reason for the lines is the non-existence of the locally dominant assumption in the source data. This information is captured in the column corresponding to TnMiss. The numbers in the TnMiss column indicate the total number of times the proposed approach exited with the “no locally dominant sources” token. Based on the results, it can be seen that the other algorithms try to find the best match for the columns of $\mathbf{A}$. However, they are unable to validate the locally dominant assumption. This is due to the fact that no such test is available in the literature. However, in all the scenarios, the proposed approach was able to conclude that the input data is not a mixture of sources that contain the locally dominant signals.

5.4. Comparative Experiment-II

Setup: In this experiment, a random source matrix $\mathbf{S}$ that satisfies the locally dominant assumption, is generated. For the generated $\mathbf{S}$ matrix, 100 random mixing matrices $\mathbf{A}$ are generated. Then, using the $\mathbf{X}=\mathbf{A}\mathbf{S}$ equation, 100 random mixture matrices are generated. In each mixture matrix, $5\%$ of the columns are randomly selected, and a uniform noise between 0 and $0.01$ is added to all the elements of the selected columns. This study is executed for n $=5,7,9,11,13$ and 15, and for N = 10,000. Well known methods from the SCA literature are compared with the proposed approach. The methods that are used for the comparison are N-FINDR ([44]), VCA ([45]), MVSA ([56]). The objective of this experiment is to comparatively assess the performance of the proposed and selected methods in noisy data. In this experiment, the value of $\mathit{\psi }$ is defined as follows: $\mathit{\psi }=\rho |{\Theta }^{-1}\mathbf{e}|$, where $\mathbf{e}\in {\mathbb{R}}^n$ is a vector of all ones, and $\rho$ takes the following values: $0,0.2,0.4,\dots ,1$.

Results:

Table 5. Comparative experiment-II separation results.

n	N	VCA	MVSA	N-FINDR	Proposed
5	10,000	0.0798	0.0872	0.0954	6.43 × ${10}^{-10}$
7	10,000	0.0573	0.045	0.0658	4.73 × ${10}^{-10}$
9	10,000	0.0407	0.0309	0.0467	3.77 × ${10}^{-10}$
11	10,000	0.0321	0.0265	0.0357	0.0012
13	10,000	0.0255	0.0231	0.0284	0.0007
15	10,000	0.0212	0.0197	0.0235	0.0021

displays the results after executing the proposed and selected approaches on the 100 randomly generated noisy mixture instances. The columns corresponding to VCA, MVSA and N-FINDR present the average error $e_A$ over the 100 instances. The proposed approach is executed 100 times for each value of $\rho$, and the average error $e_A$ for each value of $\rho$ is archived. The column corresponding to the proposed approach presents the best of the average errors $e_A$ over the values of $\rho$.

Table 6. Effect of parameter $\rho$ on the total number of failures.

n	N	$\mathit{\rho }$ = 0	$\mathit{\rho }$ = 0.2	$\mathit{\rho }$ = 0.4	$\mathit{\rho }$ = 0.6	$\mathit{\rho }$ = 0.8	$\mathit{\rho }$ = 1
5	10,000	0	0	0	0	0	0
7	10,000	92	24	13	7	5	4
9	10,000	99	32	16	12	7	6
11	10,000	100	47	27	19	15	10
13	10,000	100	53	26	17	15	13
15	10,000	100	69	45	29	22	18

(Figure 10) indicates the total number of times the method fails (succeeds) to identify the mixing matrix, for various values of $\rho$. Based on the low value of error in the proposed column, and the trends depicted in Figure 10, it can be seen that the proposed approach recovers $\mathbf{A}$ and $\mathbf{S}$ in the majority of the noisy instances for higher values of $\rho$. Moreover, as n increases, the complexity of mixing increases, and thus the proposed approach requires a higher value of $\rho$ for the recovery of $\mathbf{A}$ and $\mathbf{S}$.

6. Discussion and Conclusions

The SCA approaches are relatively new to the BSS problem when compared to the ICA approaches. The main critique that often appears with respect to locally dominant SCA approaches is the validity of the locally dominant criterion. In this paper, a mathematical modeling-based approach is proposed that can validate the existence of the locally dominant criterion from the given mixture matrix. That is, the formulation can be used not only to identify the mixing matrix, but also to validate the assumption presented in Equation (2). Although the approach is proposed for the determined case, it can also be applied to the overdetermined cases. The columns of the matrix $\mathbf{X}$ are proportional to the number of constraints in the proposed LP. Thus, big data often leads to LP with many redundant constraints. We propose the usage of interior point methods, when the total number of constraints is very high [57]. Moreover, LP decomposition-based approaches for SCA can also be developed to improve the solution time [58]. In addition to that, the LP presolve theory [59] can be designed to eliminate the redundant constraints in the proposed SCA approach. Roughly speaking, the point correntropic ranking method may be seen as a novel probabilistic approach for removing LP redundant constraints. The proposed method of estimating the point correntropy is computationally cheap, and can be applied to the big data scenarios. From the simulated data study, it can be seen that if the input data is uniformly distributed, then the 50 percentile criterion can be raised to a higher value. However, from the image data set, it is clear that the real world data is rarely uniformly distributed. Thus, the 50 percentile criterion is a good estimate to avoid the LP resolving. From the comparative experiments, it can be concluded that the proposed approach validates the locally dominant assumption in non-noisy and noisy mixing scenarios. To summarize, the proposed approach provides new insights into the BSS problem.