**1. Introduction**

Hyperspectral data consists of hundreds of contiguous narrow spectral bands and has been widely used in many fields [1]. Due to the limitation of the sensor's spatial resolution, there exist mixed pixels consisting of several material signatures. To address this problem, hyperspectral unmixing (HU) has been adopted to decompose mixed pixels into endmember signatures and their corresponding proportions. According to the availablity of the prior knowledge, HU methods can be divided into three categories: unsupervised [2–5], semisupervised [6], and supervised [7] methods. We can also categorize HU methods into geometric methods and statistical methods. The pixel purity index (PPI) [8], N-FINDR [9], vertex component analysis (VCA) [10] and the simplex growing algorithm (SGA) [11] are the most famous geometric methods. The relationships among these methods are explored in [12–14]. There are also many statistical methods for hyperspectral unmixing [15–17]. Nonnegative matrix factorization (NMF) [18] is a typical statistical method [19]. It has been shown to be promising in extracting sparse and interpretable representations from a data matrix. The NMF decomposes a data matrix into two low-rank matrices with nonnegative constraint [20]. The decomposition results of NMF consist of a basis matrix and a coefficient matrix, which provide an intuitive and interpretable representation of data. As an unsupervised method, NMF is applied to hyperspectral unmixing and shows its advantages in many situations. To reduce the solution space, constraints on endmembers [21–24] and abundances [25,26] have been exploited and used in NMF. Recently, a sparseness constraint has been added to NMF to generate unique solutions and leads to better results [25,27]. The *L*1 constraint is a widely-used sparseness constraint. However, *L*1 regularization has the limitation that it cannot enforce further sparseness when the abundance sum-to-one constraint is used. The *L*1/2 constraint is representative of *Lp*(0 < *p* < <sup>1</sup>). The solution of the *L*1/2 regularizer is sparser compared with that of the *L*1 regularizer. However, the *L*1/2

regularizer also brings nonconvexity to the optimization problem. The nonconvex optimization problem with the *L*1/2 regularizer can be solved by transforming the *L*1/2 regularizer into a series of convex weighted *L*1 regularizers [28]. *L*1/2-NMF is a popular NMF regularization method [29]. The authors have shown that the *L*1/2 regularizer can overcome the limitation of the *L*1 regularizer and enforce a sufficiently sparse solution. On the contrary, *L*2-NMF generates smooth results rather than sparse results [30]. In [31], piecewise smooth nonsmooth (PSnsNMF) and piecewise smooth NMF with sparseness constraints (PSNMFSC) are proposed by incorporating the piecewise smoothness of spectral data and sparseness of endmember abundances. The authors of [32] propose NMF with sparseness and smoothness constraints (NMFSSC). However, NMFSSC does not consider the sparsity level of data and just imposes sparseness and smoothness constraints simultaneously. In data-guided sparsity-regularized nonnegative matrix factorization (DgS-NMF) [33], the sudden change areas are assumed to be highly mixed and a heuristic method is proposed to employ the spatial similarity to learn the mixed level in the hyperspectral images. The pixel with a higher sparsity level corresponds to a sparser constraint (from the *L*1 norm to the *L*0 norm). In [34], a learning-based sparsity method is proposed to learn a guidance map from the unmixing results and impose an adaptive *lp*(0 < *p* ≤ 1)-constraint.

In this paper, we propose a nonnegative matrix factorization with data-guided constraints (DGC-NMF). Unlike traditional constrained NMF methods that impose the same constraint over entire data, DGC-NMF firstly evaluates the sparsity level of each pixels' abundances and then decides which kind of constraint should be imposed on the abundances of a pixel adaptively. In real hyperspectral images, the sparsity levels of the pixels' abundances are varied and the pixels do not necessarily possess spatial dependence with their neighboring pixels. To preserve the distinctive sparsity information of each pixel's abundances, the sparsity levels of pixels can be learnt via an NMF unmixing process without any sparseness constraint imposed. Therefore, each pixel's abundances could enjoy a individual constraint according to its sparsity level in our method. In evenly mixed areas, the sparseness constraint may not contribute to achieving a smooth abundance vector of a pixel. Therefore, we also introduce the *L*2 constraint to reduce extreme abundance values and promote the evenness of pixels' abundance vector. Whether an *L*1/2 constraint or an *L*2 constraint is imposed on a pixel is learnt from its abundances' sparsity level. In this way, our method could adaptively enforce sparse or smooth abundance results in regions with different mixed levels. The experimental results of synthetic and real data demonstrate the effectiveness of DGC-NMF.

The main contributions of this paper include two aspects. Firstly, we provide a method to evaluate the sparsity level of data in different areas and obtain the sparseness map of data. The effectiveness of this method has been verified using data with various sparsity levels. Secondly, we propose a novel NMF method which makes use of the sparsity information from data and adaptively imposes constraints according to the mixed levels of pixels. We analyze the sparsity behaviors of NMF with different regularizations and indicate that NMF with fixed constraints may be not applicable for a hyperspectral image with various sparsity levels, while it has been proven that the proposed DGC-NMF is capable.

The remainder of this paper is organized as follows. Section 2 gives a brief introduction of the NMF algorithm and the NMF with the *<sup>L</sup>*1/2 or the *L*2 constraint. Section 3 presents the proposed DGC-NMF and provides the proof that the objective decreases along the iterates of the algorithm. Section 4 validates the effectiveness of the proposed method on synthetic data and real hyperspectral images. Finally, Section 5 concludes this paper.
