A Characterization of the Domain of Beta-Divergence and Its Connection to Bregman Variational Model

Abstract

In image and signal processing, the beta-divergence is well known as a similarity measure between two positive objects. However, it is unclear whether or not the distance-like structure of beta-divergence is preserved, if we extend the domain of the beta-divergence to the negative region. In this article, we study the domain of the beta-divergence and its connection to the Bregman-divergence associated with the convex function of Legendre type. In fact, we show that the domain of beta-divergence (and the corresponding Bregman-divergence) include negative region under the mild condition on the beta value. Additionally, through the relation between the beta-divergence and the Bregman-divergence, we can reformulate various variational models appearing in image processing problems into a unified framework, namely the Bregman variational model. This model has a strong advantage compared to the beta-divergence-based model due to the dual structure of the Bregman-divergence. As an example, we demonstrate how we can build up a convex reformulated variational model with a negative domain for the classic nonconvex problem, which usually appears in synthetic aperture radar image processing problems.

1. Introduction

In general, the domain of a divergence [1,2] is that confined not by the positiveness of variables but by the positiveness of a divergence (i.e., $D\left(b\right|u)\ge 0$). Therefore, the domain of a divergence could be defined to include negative region while keeping positiveness of the divergence. To the best of our knowledge, it is unclear when the domain of the $\beta$-divergence (and the corresponding Bregman-divergence) include the negative region. In this article, we systematically explore the domains of the $\beta$-divergence [2] and the corresponding Bregman-divergence associated with the convex function of Legendre type [3].

The $\beta$-divergence [2,4,5,6,7] is a general framework of similarity measures induced from various statistical models, such as Poisson, Gamma, Gaussian, Inverse Gaussian, compound Poisson, and Tweedie distribution. For the connection between the $\beta$-divergence and the various statistical distributions, see [8]. Among the diverse statistical distributions, the Tweedie distribution has a unique feature, i.e., the unit deviance of the Tweedie distribution [8] corresponds to the $\beta$-divergence with $\beta \in \mathbb{R}\setminus (1,2)$. It is interesting that $(1,2)$ is a vital range of $\beta$ while defining a convex right Bregman proximity operator [9,10]. We will address this issue in more details in Section 4. In addition, the $\beta$-divergence is also used as a distance-like measure in diverse areas, for instance, synthetic aperture radar (SAR) image processing [11,12], audio spectrogram comparison [6,13], and brain EEG signal processing [7].

We note that authors in [7] show the usefulness of the $\beta$-divergence with $\beta >1$ as a robust similarity measure against outliers between two probability distributions. Here, outliers (rare events) are that have extremely low probability and thus they exist near zero probability. However, the (generalized) Kullback–Leibler-divergence (i.e., $\beta$-divergence $D_{\beta =1}\left(b\right|u)$), which is a commonly used similarity measure for probability distributions, is undefined at zero ($u=0$). See Figure 1a. Therefore, it is not easy to obtain robustness against outliers through the (generalized) Kullback–Leibler-divergence. On the contrary, the $\beta$-divergence with $\beta >1$ (i.e., $D_{\beta >1}\left(b\right|u)$) is well defined at zero ($u=0$) and thus it is more robust to outliers than the Kullback–Leibler-divergence. For more details, see [4,5,7]. We also note that if the variables of $\beta$-divergence are not probability distributions (i.e., unnormalized) then outliers correspond to the variables that have extremely large values (≫1) [14]. To detect such kind of outliers under the Gamma distribution assumption, the $\beta$-divergence with $\beta \in [-1,0]$ is used as a distance-like measure in [11]. See also Figure 1c.

In the case of SAR image data processing, speckle noise is modeled with the Gamma distribution and thus the negative log-likelihood function, which appears in speckle reduction problem, corresponds to the $\beta$-divergence with $\beta =0$, i.e., the Itakura–Saito-divergence. Actually, this model is highly nonconvex [15]. Therefore, various transforms are introduced to relax nonconvexity of the Gamma distribution related speckle reduction model [16,17,18,19,20,21]. Recently, we have shown that the $\beta$-divergence with $\beta \in (0,1)$ can be used as a transform-less convex relaxation model for SAR speckle reduction problem [12]. Generally, the data captured via a SAR system has extremely high dynamic range [22,23]. Under this harsh environment, $\beta$-divergence with $\beta \in (-1,0)$ is successfully used as a similarity measure for separation of the strong scatterers in SAR data [11]. In addition, the $\beta$-divergence is also used for the decomposition of magnitude data of audio spectrograms [6]. In these applications, the domains of data are generally assumed to be positive. However, the domain of the $\beta$-divergence can be extended to the negative region. In fact, if $\beta =2$, then the $\beta$-divergence is exactly the square of ${\ell }_2$-distance, the domain of which naturally includes a negative region. Surprisingly, in this article, we show that, under the mild condition on $\beta$, there are infinitely many $\beta$-divergences that have a negative domain.

It is known that the $\beta$-divergence can be reformulated with the Bregman-divergence [2,6]. However, if we restrict the base function of the Bregman-divergence as the convex function of Legendre type, then some part of the $\beta$-divergence cannot be expressed through the Bregman-divergence (see

Table 1. We compare the domain of the $\beta$-divergence and the domain of the Bregman-divergence associated with the convex function of Legendre type in (19). We note that the domain ${\mathbb{R}}_{+}^n\times {\mathbb{R}}_{+}^n$ ($\beta >1$) and the domain ${\mathbb{R}}_{--}^n\times {\mathbb{R}}_{--}^n$ ($\beta =0$) do not exist in the Bregman-divergence. ${}^{\ast }$If we relax the Legendre condition of Φ as a convex and smooth function, then the Bregman-divergence $D_{\Phi }$ also exists in the region $\beta >1$ with $dom\Phi ={\mathbb{R}}_{+}^n$.

Region	$\mathit{\beta }$-Divergence		Bregman-Divergence
entire	$\beta >1,\beta \in {\mathbb{R}}_e$	${\Omega }_L={\Omega }_R={\mathbb{R}}^n$	$\beta >1,\beta \in {\mathbb{R}}_e$	$dom\Phi ={\mathbb{R}}^n$
	$\mathbf{\beta }>\mathbf{1}$	${\Omega }_L={\Omega }_R={\mathbb{R}}_{+}^n$	-${}^{\ast }$	-
positive	$0<\beta \le 1$	${\Omega }_L={\mathbb{R}}_{+}^n$ and ${\Omega }_R={\mathbb{R}}_{++}^n$	$0<\beta \le 1$	$dom\Phi ={\mathbb{R}}_{+}^n$
	$\beta \le 0$	${\Omega }_L={\Omega }_R={\mathbb{R}}_{++}^n$	$\beta \le 0$	$dom\Phi ={\mathbb{R}}_{++}^n$
negative	$0<\beta <1,\beta \in {\mathbb{R}}_e$	${\Omega }_L={\mathbb{R}}_{-}^n$ and ${\Omega }_R={\mathbb{R}}_{--}^n$	$0<\beta <1,\beta \in {\mathbb{R}}_e$	$dom\Phi ={\mathbb{R}}_{-}^n$
	$\mathbf{\beta }\le \mathbf{0},\beta \in {\mathbb{R}}_e$	${\Omega }_L={\Omega }_R={\mathbb{R}}_{--}^n$	$\mathbf{\beta }<\mathbf{0},\beta \in {\mathbb{R}}_e$	$dom\Phi ={\mathbb{R}}_{--}^n$

). Although the Bregman-divergence associated with a convex function of Legendre type does not exactly match with the $\beta$-divergence, due to the fruitful mathematical structure of the convex function of Legendre type, the associated Bregman-divergence has many useful properties. For instance, the dual formulation of the Bregman-divergence associated with the convex function of Legendre type can be used as a convex reformulation of some nonconvex problems under the certain condition on its domain [24]. In this article, we demonstrate that, by using the dual Bregman-divergence with the negative convex domain, we can make a convex reformulated Bregman variational model for the classic nonconvex problem that appears in the SAR image noise reduction problem [15]. We also show that we can unify the various variational models appearing in image processing problems as the Bregman variational model having sparsity constraints, e.g., total variation [25,26] (we called it Bregman-TV). Actually, the Bregman variational model corresponds to the right Bregman proximity operator [9,10]. See also [9,10,24,27,28,29,30] for theoretical analysis of the Bregman-divergence and related interesting properties of it.

1.1. Background

In this section, we review typical examples of the $\beta$-divergence, i.e., Itakura–Saito-divergence, Generalized Kullback–Leibler-divergence (I-divergence), and ${norm}^2$-distance. In addition, we introduce the Bregman-divergence and the corresponding Bregman variational model with sparsity constraints.

Let us start with the $\beta$-divergence $D_{\beta }:{\Omega }_L\times {\Omega }_R\to {\mathbb{R}}_{+}$ given by

$$D_{\beta }\left(b\right|u)=\langle {\int }_u^b\left(\frac{b-x}{x^{2-\beta }}\right)dx,\mathbf{1}\rangle ,$$

(1)

where ${\Omega }_L\times {\Omega }_R=\{(b,u)\in {\mathbb{R}}^n\times {\mathbb{R}}^n|0\le D_{\beta }\left(b\right|u)<\infty \}$ is the domain of the $\beta$-divergence. Actually, the domain ${\Omega }_L\times {\Omega }_R$ corresponds to the effective domain in optimization [3,31]. We call ${\Omega }_L$ and ${\Omega }_R$ as the left and right domains of the $\beta$-divergence. In addition, we assume that the left and right domains, ${\Omega }_L$ and ${\Omega }_R$, are convex sets, respectively. That is, if $a,b\in {\Omega }_L$ (or ${\Omega }_R$), then the line segment between two points also satisfies $\overline{ab}\in {\Omega }_L$ (or ${\Omega }_R$). Note that $\langle a,d\rangle ={\sum }_{i=1}^n{a}_i{d}_i$, $a=(a_1,\dots ,a_n)\in {\mathbb{R}}^n$, $d=(d_1,\dots ,d_n)\in {\mathbb{R}}^n$, $\mathbf{1}$ is the all one vector in ${\mathbb{R}}^n$, ${\mathbb{R}}_{+}=\{x\in \mathbb{R}|x\ge 0\}$, ${\mathbb{R}}_{++}=\{x\in \mathbb{R}|x>0\}$, ${\mathbb{R}}_{-}=\{x\in \mathbb{R}|x\le 0\}$, and ${\mathbb{R}}_{--}=\{x\in \mathbb{R}|x<0\}$. In addition, integration, multiplication, and division are performed component-wise. Based on a selection of $\beta$, we can recover the famous representatives of the $\beta$-divergence, i.e., Itakura–Saito-divergence [4,5,13], I-divergence (or generalized Kullback–Leibler-divergence) [20,32], and ${norm}^2$-distance [25,26]. These three divergences are important examples of the $\beta$-divergence, since they show three different types of domains of the $\beta$-divergence. We summarize them in the following.

• Itakura–Saito-divergence ($\beta =0$): ${\Omega }_L={\mathbb{R}}_{++}^n$ and ${\Omega }_R={\mathbb{R}}_{++}^n:$

  $$D_{\beta }\left(b\right|u)=\langle \left(\frac{b}{u}\right)-ln\left(\frac{b}{u}\right)-\mathbf{1},\mathbf{1}\rangle .$$

  Usually, the left and right domain, i.e., ${\Omega }_L$ and ${\Omega }_R$ of Itakura–Saito-divergence, are defined as positive and ${\Omega }_L={\Omega }_R$ [12,13]. However, due to the scale invariance property of it, the variables b and u can be negative at the same time, even within the logarithmic function, i.e., ${\Omega }_L={\Omega }_R={\mathbb{R}}_{--}^n$. Based on this keen observation, in this article, we develop a new methodology that systematically detects a domain having the negative region. The Itakura–Saito-divergence is a typical example that can be expressed by the $\beta$-divergence and the Bregman-divergence at the same time. However, it has the negative domain in the $\beta$-divergence framework, but not in the Bregman-divergence framework (see Table 1).
• Generalized Kullback–Leibler-divergence (I-divergence) ($\beta =1$): ${\Omega }_L={\mathbb{R}}_{+}^n$ and ${\Omega }_R={\mathbb{R}}_{++}^n:$

  $$D_{\beta }\left(b\right|u)=\langle bln\left(\frac{b}{u}\right)-(b-u),\mathbf{1}\rangle ,$$

  where we naturally assume that $0ln0=0.$ Interestingly, it has different left and right domains, i.e., ${\Omega }_L\ne {\Omega }_R$. Due to the asymmetric structure of the domain of I-divergence, we need to carefully handle the $\beta$-divergence at the boundary of each domain. We categorize the class of the $\beta$-divergence that has the asymmetric domain structure in Section 2.
• $nor{m}^2$-distance ($\beta =2$): ${\Omega }_L={\mathbb{R}}^n$ and ${\Omega }_R={\mathbb{R}}^n:$

  $$D_{\beta }\left(b\right|u)=\frac{1}{2}{\parallel b-u\parallel }_2^2.$$

  This divergence is preferable to other divergences, since it has ${\mathbb{R}}^n$ as its domain for each variable. Unlike the previous two divergences, the domain of it naturally includes a negative region ${\mathbb{R}}_{-}^n$. Surprisingly, there are infinitely many $\beta$-divergences having ${\mathbb{R}}^n$ as its domain. We will show it in Section 2.

Additionally, we introduce the Bregman-divergence associated with the convex function of Legendre type [3]. The Bregman-divergence $D_{\Phi }:\Omega \times int(\Omega )\to {\mathbb{R}}_{+}$ is formulated as

$$D_{\Phi }\left(b\right|u)=\Phi \left(b\right)-\Phi \left(u\right)-\langle b-u,\nabla \Phi \left(u\right)\rangle ,$$

(2)

where the base function $\Phi$ is the convex function of Legendre type [3], $\Omega =dom(\Phi )=$ $\{x\in {\mathbb{R}}^n|\Phi \left(x\right)\in \mathbb{R}\}$, and $int(\Omega )$ is the interior of $\Omega$. In fact, it is relatively interior of $\Omega$, i.e., $ri(\Omega )$. Note that $ri(\Omega )$ is the interior of $\Omega$ relative to its affine hull, which is the smallest affine set including $\Omega$. Therefore, the relative interior $ri(\Omega )$ coincides with the interior $int(\Omega )$ when the affine hull of $\Omega$ is ${\mathbb{R}}^n$. For more details, see Chapter 2.H in [31]. In this article, since the $\beta$-divergence (1) is separable in terms of dimension, the affine hull of $\Omega$ is always ${\mathbb{R}}^n$ and thus we simply use $int(\Omega )$ instead of $ri(\Omega )$. Note that the typical examples of the $\beta$-divergence in the above (Itakura–Saito-divergence, I-divergence, and ${norm}^2$-distance) can be reformulated with the Bregman-divergence (2) by using the convex function of Legendre type $\Phi$ and the associated domain $\Omega$:

• Itakura–Saito-divergence: $\Phi \left(x\right)=\langle -lnx,\mathbf{1}\rangle$ with $\Omega ={\mathbb{R}}_{++}^n,$
• I-divergence: $\Phi \left(x\right)=\langle xlnx,\mathbf{1}\rangle$ with $\Omega ={\mathbb{R}}_{+}^n,$
• $nor{m}^2$-distance: $\Phi \left(x\right)=\langle \frac{1}{2}{x}^2,\mathbf{1}\rangle$ with $\Omega ={\mathbb{R}}^n.$

The domain of the second variable of the Bregman-divergence (2) is always open set $int(dom\Phi )$. However, the right domain ${\Omega }_R$ of the second variable of the $\beta$-divergence (1) could be a closed set. In the coming section, we thoroughly analyze the relation between the Bregman-divergence and the $\beta$-divergence with regard to its domain. Based on the Bregman-divergence (2), we introduce the Bregman variational model that unifies the various minimization problems appearing in image processing:

$$\underset{u}{\min }\{D_{\Phi }\left(b\right|u)+\lambda R\left(u\right)|u\in int(\Omega )\},$$

(3)

where b is the observed data and $R\left(u\right)$ is the sparsity enforcing regularization term, such as total variation [26]. In image processing, (3) corresponds to the denoising problem under the various noise distributions: Poisson, Speckle, Gaussian noise, etc. However, in optimization, it is known as (nonconvex) right Bregman proximity operator under mild conditions. See [9,10,24,30] for more details on the Bregman operator.

1.2. Overview

The article is organized as follows. In Section 2, we analyze the structure of the domain of the $\beta$-divergence. In Section 3, we study various mathematical structures of the $\beta$-divergence through the Bregman-divergence associated with the convex function of Legendre type. In Section 4, we introduce the Bregman variational model and its dual formulation for convex reformulation of the classic nonconvex problem that appears in the SAR speckle reduction problem. In addition, we introduce the right and left Bregman proximal operator. We give our conclusions in Section 5.

2. A Characterization of the Domain ${\Omega }_L\times {\Omega }_R$ of the $\beta$-Divergence

In this section, we analyze the structure of the $\beta$-divergence and the associated domain ${\Omega }_L\times {\Omega }_R$ based on the so-called extended logarithmic function.

Let us start with a definition of the extended logarithmic function that is essential in characterizing the domain of the $\beta$-divergence. We note that it corresponds to an equivalence class of Tsallis’s generalized logarithmic function [1,33] with an extention to the negative domain.

Definition 1.

Let $\alpha \in \mathbb{R},$ $u=(u_1,\dots ,u_n)\in dom\left({ln}_{\alpha }\right),$ and

$${ln}_{\alpha ,c}\left(u\right)=\left({ln}_{\alpha ,c}\left(u_1\right),{ln}_{\alpha ,c}\left(u_2\right),\dots ,{ln}_{\alpha ,c}\left(u_n\right)\right),$$

where ${ln}_{\alpha ,c}\left(u_i\right)={\int }_c^{u_i}{x}^{-\alpha }dx$ and $c\in {\mathbb{R}}_{c,u}=\{c\in \mathbb{R}|{ln}_{\alpha ,c}\left(u\right)\in {\mathbb{R}}^n\mathrm{and}c\ne u_i,i=1,\dots ,n\}.$ Then, the extended logarithmic function is defined as an equivalence class

$${\left[{ln}_{\alpha }\left(u\right)\right]}_c=\{x\in {\mathbb{R}}^n|x={ln}_{\alpha ,c}\left(u\right),c\in {\mathbb{R}}_{c,u}\}.$$

For simplicity, we leave out all constants after integration and then we attain

$${ln}_{\alpha }\left(u\right)=\left\{\begin{array}{c}lnu,if\alpha =1,\hfill \\ \frac{1}{1-\alpha }{u}^{1-\alpha },\mathit{otherwise},\hfill \end{array}\right.$$

(4)

where $dom\left({ln}_{\alpha }\right)=\{x\in {\mathbb{R}}^n|{ln}_{\alpha }\left(x\right)\in {\mathbb{R}}^n\}$. We call (4) as the extended logarithmic function instead of the equivalence class ${\left[{ln}_{\alpha }\left(u\right)\right]}_c$, unless otherwise specified.

Note that the domain and range of ${ln}_{\alpha }\left(u\right)$ (4) are given in

Table 2. The domain and range of the extended logarithmic function ${ln}_{\alpha }\left(x\right)$ defined in (4).

	$\mathit{\alpha }=1$		$\mathit{\alpha }<1$			$\mathit{\alpha }>1$
		$\mathit{\alpha }\in {\mathbb{R}}_{\mathit{o}}$	$\mathit{\alpha }\in {\mathbb{R}}_{\mathit{e}}$	$\mathit{\alpha }\in {\mathbb{R}}_{\mathit{x}}$	$\mathit{\alpha }\in {\mathbb{R}}_{\mathit{o}}$	$\mathit{\alpha }\in {\mathbb{R}}_{\mathit{e}}$	$\mathit{\alpha }\in {\mathbb{R}}_{\mathit{x}}$
$dom\left({ln}_{\alpha }\right)$	${\mathbb{R}}_{++}$	$\mathbb{R}$	$\mathbb{R}$	${\mathbb{R}}_{+}$	${\mathbb{R}}_{++}$ or ${\mathbb{R}}_{--}$	${\mathbb{R}}_{++}$ or ${\mathbb{R}}_{--}$	${\mathbb{R}}_{++}$
$range\left({ln}_{\alpha }\right)$	$\mathbb{R}$	${\mathbb{R}}_{+}$	$\mathbb{R}$	${\mathbb{R}}_{+}$	${\mathbb{R}}_{--}$	${\mathbb{R}}_{--}$ or ${\mathbb{R}}_{++}$	${\mathbb{R}}_{--}$

. In addition, we illustrate the structure of the extended logarithmic function in Figure 2. As noticed in Definition 1, the extended logarithmic function is defined as an equivalence class ${\left[{ln}_{\alpha }\left(u\right)\right]}_c$ with respect to c. If we set $c=1,$ then we can recover Tsallis’ generalized logarithmic function [1,33] on its positive domain $dom\left({ln}_{\alpha }\right)\subseteq {\mathbb{R}}_{+}$. See Figure 2a. However, we cannot use the generalized logarithm (i.e., ${ln}_{\alpha ,c=1}\left(u\right)$) in the negative domain. In fact, if $\alpha >1$ and ${\mathbb{R}}_{-}\subset dom\left({ln}_{\alpha }\right)$ then Tsallis’ generalized logarithmic function is undefined, e.g., ${ln}_{4,c=1}(-1)={\int }_1^{-1}{x}^{-4}dx\notin \mathbb{R}$. On the other hand, the proposed extended logarithmic function (4) is well defined on $\mathbb{R}$ for all $\alpha$, since we can choose an appropriate c having the same sign with u even if $\alpha >1$, e.g., ${ln}_{4,c=-2}(-1)={\int }_{-2}^{-1}{x}^{-4}dx\in \mathbb{R}$. See Figure 2d and Table 2. Indeed, the extended logarithmic function is useful when we simplify the complicated structure of the $\beta$-divergence. As described in the following Definition 2, the $\beta$-divergence is defined based on the difference of two extended logarithmic functions. In other words, the $\beta$-divergence is invariance with respect to a constant function in the extended logarithmic function (4). It is interesting that the Bregman-divergence (2) also has a similar invariance property with respect to an affine function in the base function $\Phi$ (see Proposition 1).

Definition 2.

Let $(b,u)\in {\Omega }_L\times {\Omega }_R$$=\{(b,u)\in {\mathbb{R}}^n\times {\mathbb{R}}^n|D_{\beta }\left(b\right|u)\in {\mathbb{R}}_{+}\}$ and $\beta \in \mathbb{R}$. Then, the β-divergence is defined by

$$\begin{array}{ccc}\hfill D_{\beta }\left(b\right|u)& =& \langle b[{ln}_{2-\beta }\left(b\right)-{ln}_{2-\beta }\left(u\right)]-[{ln}_{1-\beta }\left(b\right)-{ln}_{1-\beta }\left(u\right)],\mathbf{1}\rangle \hfill \\ \hfill & =& \langle b{\int }_u^b{u}^{\beta -2}du-{\int }_u^b{u}^{\beta -1}du,\mathbf{1}\rangle .\hfill \end{array}$$

(5)

After integration, we get the well-known formula of β-divergence:

$$\begin{array}{c}\hfill D_{\beta }\left(b\right|u)=\left\{\begin{array}{c}\langle \left(\frac{b}{u}\right)-ln\left(\frac{b}{u}\right)-\mathbf{1},\mathbf{1}\rangle ,\mathit{if}\beta =0,\hfill \\ \langle bln\left(\frac{b}{u}\right)-(b-u),\mathbf{1}\rangle ,\mathit{if}\beta =1,\hfill \\ \langle \frac{b}{\beta -1}(b^{\beta -1}-u^{\beta -1})-\frac{1}{\beta }(b^{\beta }-u^{\beta }),\mathbf{1}\rangle ,\mathit{if}\beta \ne 0,1.\hfill \end{array}\right.\end{array}$$

(6)

Although the $\beta$-divergence has a unified formula (5) via the extended logarithm (4), unfortunately, the determination of the domain ${\Omega }_L\times {\Omega }_R$ of the $\beta$-divergence heavily depends on $\beta$. Before we go any further, let us introduce the most important equivalence classes in this article. It will simplify complicated notations appearing in the $\beta$-divergence and the Bregman-divergence.

$$\begin{array}{c}{\mathbb{R}}_e=\{2k/(2l+1)|k,l\in \mathbb{Z},\}\hfill \\ {\mathbb{R}}_o=\{(2k+1)/(2l+1)|k,l\in \mathbb{Z},\}\hfill \\ {\mathbb{R}}_x=\mathbb{R}\setminus {\mathbb{R}}_o\cup {\mathbb{R}}_e.\hfill \end{array}$$

(7)

Note that ${\mathbb{R}}_e$ and ${\mathbb{R}}_o$ are subsets of the rational number and satisfy ${\mathbb{R}}_e\cap {\mathbb{R}}_o=\varnothing$, while ${\mathbb{R}}_x$ is composed of all irrational numbers with a subset of the rational number that are not in ${\mathbb{R}}_e\cup {\mathbb{R}}_o$. For instance, ${\mathbb{R}}_e=\{0,\pm \frac{2}{3},\pm \frac{4}{3},\dots \}$, ${\mathbb{R}}_o=\{\pm 1,\pm \frac{1}{3},\pm \frac{5}{3},\dots \}$ and ${\mathbb{R}}_x=\{\pm \frac{1}{4},\pm \frac{1}{2},\pm \sqrt{2},\dots \}$.

Since the $\beta$-divergence (6) is developed based on the extended logarithmic function (i.e., power functions), inherently, we have to quantify the domain of a power function $p\left(x\right)=x^{\alpha }$ and its inverse function $p^{-1}\left(y\right)=y^{1/\alpha }.$ Actually, if x is positive, then a power function $p\left(x\right)$ and the corresponding inverse function is well defined, irrespective of the choice of an exponent $\alpha \in \mathbb{R}\setminus \left\{0\right\}$. On the other hand, in the case of negative domain, e.g., $x<0$, the domain of a power function $p\left(x\right)$ severely depends on the choice of an exponent $\alpha$. With newly introduced equivalence classes in (7), we can easily categorize the domain of a power function $p\left(x\right)=x^{\alpha }$ and its inverse function $p^{-1}\left(y\right)=y^{1/\alpha },\alpha \ne 0$. We summarize it in the following Lemma.

Lemma 1.

Let $\alpha \in \mathbb{R}$ and $do{m}_{-}\left(p\right)=\{x\in {\mathbb{R}}_{--}|p\left(x\right)\in \mathbb{R}\}=domp\cap {\mathbb{R}}_{--}$ be the negative domain of a power function $p\left(x\right)=x^{\alpha }.$ Then, $p:do{m}_{-}\left(p\right)\to rang{e}_{-}\left(p\right)$ has its negative domain

$$do{m}_{-}\left(p\right)=\left\{\begin{array}{c}{\mathbb{R}}_{--},\mathit{if}\alpha \in {\mathbb{R}}_e\cup {\mathbb{R}}_o,\hfill \\ \varnothing ,\mathit{if}\alpha \in {\mathbb{R}}_x,\hfill \end{array}\right.$$

and the corresponding range

$$\begin{array}{c}\hfill {range}_{-}\left(p\right)=\left\{\begin{array}{c}{\mathbb{R}}_{--},\mathit{if}\alpha \in {\mathbb{R}}_o,\hfill \\ {\mathbb{R}}_{++},\mathit{if}\alpha \in {\mathbb{R}}_e.\hfill \end{array}\right.\end{array}$$

In addition, if $\alpha \in {\mathbb{R}}_o,$ then the inverse function of p is well defined and transparent on $do{m}_{-}\left(p\right)$, i.e., $p^{-1}\circ p\left(x\right)=x$ for all $x\in do{m}_{-}\left(p\right)$. However, if $\alpha \in {\mathbb{R}}_e\setminus \left\{0\right\},$ then $p^{-1}\circ p\left(x\right)\ne x$ for all $x\in do{m}_{-}\left(p\right)$.

Proof. 

For any $x\in {\mathbb{R}}_{--}$, let $x=(-1)\left|x\right|,$ then the power function p is expressed as $p\left(x\right)={(-1)}^{\alpha }{\left|x\right|}^{\alpha },\forall \alpha \in \mathbb{R}.$ We note that the negative real domain of $p\left(x\right)$ is well-defined only if ${(-1)}^{\alpha }\in \{-1,+1\}$. To clarify the evaluation of ${(-1)}^{\alpha }$, let us express it in a polar form:

$${(-1)}^{\alpha }=e^{i\alpha (2l+1)\pi },\forall l\in \mathbb{Z}.$$

(8)

Then, we get

$${(-1)}^{\alpha }=\left\{\begin{array}{c}1,\mathrm{if}\alpha \in {\mathbb{R}}_e,\hfill \\ -1,\mathrm{if}\alpha \in {\mathbb{R}}_o,\hfill \\ \delta ,\mathrm{if}\alpha \in {\mathbb{R}}_x,\hfill \end{array}\right.$$

where $\delta \in \mathbb{C}\setminus \mathbb{R}.$ Regarding the inverse function $p^{-1}\left(y\right)=y^{1/\alpha },$ we have ${(-1)}^{1/a}\in \mathbb{C}\setminus \mathbb{R}$ for all $\alpha \in {\mathbb{R}}_e\setminus \left\{0\right\}.$ However, if $\alpha \in {\mathbb{R}}_o,$ then ${(-1)}^{1/\alpha }=-1.$ That is, we have $p^{-1}\circ p\left(x\right)=x$, for all $x\in {\mathbb{R}}_{--}.$ ☐

Now, with the equivalence classes (7) and Lemma 1, we classify domains of the $\beta$-divergence. The details are given in the following Theorem and

Table 3. A classification of the domain ${\Omega }_L\times {\Omega }_R=\{(b,u)\in {\mathbb{R}}^n\times {\mathbb{R}}^n|D_{\beta }\left(b\right|u)\in {\mathbb{R}}_{+}\}$ of the $\beta$-divergence in terms of $\beta$.

$\mathit{\beta }$	${\mathbf{\Omega }}_{\mathit{L}}\times {\mathbf{\Omega }}_{\mathit{R}}$
$\beta >1$ and $\beta \notin {\mathbb{R}}_e$	${\mathbb{R}}_{+}^n\times {\mathbb{R}}_{+}^n$
$\beta >1$ and $\beta \in {\mathbb{R}}_e$	${\mathbb{R}}^n\times {\mathbb{R}}^n$
$0<\beta \le 1$ and $\beta \notin {\mathbb{R}}_e$	${\mathbb{R}}_{+}^n\times {\mathbb{R}}_{++}^n$
$0<\beta \le 1$ and $\beta \in {\mathbb{R}}_e$	${\mathbb{R}}_{+}^n\times {\mathbb{R}}_{++}^n$ or ${\mathbb{R}}_{-}^n\times {\mathbb{R}}_{--}^n$
$\beta \le 0$ and $\beta \notin {\mathbb{R}}_e$	${\mathbb{R}}_{++}^n\times {\mathbb{R}}_{++}^n$
$\beta \le 0$ and $\beta \in {\mathbb{R}}_e$	${\mathbb{R}}_{++}^n\times {\mathbb{R}}_{++}^n$ or ${\mathbb{R}}_{--}^n\cup {\mathbb{R}}_{--}^n$

. See also Figure 1 for the overall structure of the $\beta$-divergence on its domain.

Theorem 1.

Let us consider the domain of the β-divergence

$${\Omega }_L\times {\Omega }_R=\{(b,u)\in {\mathbb{R}}^n\times {\mathbb{R}}^n|D_{\beta }\left(b\right|u)\in {\mathbb{R}}_{+}\},$$

where

$$D_{\beta }\left(b\right|u)=\langle {\int }_u^b(b-x)G\left(x\right)dx,\mathbf{1}\rangle$$

and $G\left(x\right)=x^{\beta -2}=(x_1^{\beta -2},x_2^{\beta -2},\dots ,x_n^{\beta -2}).$ In addition, let us assume that the minimum value of $G\left(x\right)$ on its domain is nonnegative, i.e.,

$$0\le M_G=mi{n}_x\left\{G\left(x\right)\right|x_i\in [\min \{u_i,b_i\},max\{u_i,b_i\}],i=1,\dots ,n\},$$

(9)

where $(b,u)\in {\Omega }_L\times {\Omega }_R.$ Then, the domain of the β-divergence ${\Omega }_L\times {\Omega }_R$ is classified as in Table 3. Note that ${\Omega }_R\subseteq {\Omega }_L$ for all $\beta \in \mathbb{R}$. In particular, if $\beta \in (0,1],$ then ${\Omega }_R⊊{\Omega }_L.$

Proof. 

Due to the assumption $M_G\ge 0$, we can easily obtain the positiveness of the $\beta$-divergence by the following inequality

$$D_{\beta }\left(b\right|u)=\langle {\int }_u^b(b-x)x^{\beta -2}dx,\mathbf{1}\rangle =\langle {\int }_u^b(b-x)G\left(x\right)dx,\mathbf{1}\rangle \ge \frac{1}{2}{\parallel b-u\parallel }_2^2{M}_G\ge 0.$$

(10)

Consequently, we only need to fulfill the following two conditions: (1) Is the domain of the $\beta$-divergence determined to satisfy (9)? (2) Is the $\beta$-divergence well-defined on its domain?

• Case 1: $M_G\ge 0,$ for all $(b,u)\in {\Omega }_L\times {\Omega }_R$.
  If $(b,u)\in {\mathbb{R}}_{++}^n\times {\mathbb{R}}_{++}^n$ then it is trivial to show $M_G\ge 0$ and thus it is always true that ${\mathbb{R}}_{++}^n\times {\mathbb{R}}_{++}^n\subseteq {\Omega }_L\times {\Omega }_R$. Now, we will find $\beta$ such that ${\mathbb{R}}_{--}\times {\mathbb{R}}_{--}\subset {\Omega }_L\times {\Omega }_R$ is satisfied. From Lemma 1, if $x_i<0$ then, we get the following

  $$G\left(x_i\right)=\left\{\begin{array}{c}|x_i{|}^{\beta -2},\mathrm{if}\beta \in {\mathbb{R}}_e,\\ -|x_i{|}^{\beta -2},\mathrm{if}\beta \in {\mathbb{R}}_o,\\ \varnothing ,\mathrm{if}\beta \in {\mathbb{R}}_x,\end{array}\right.$$

  Based on $\beta \in \mathbb{R}$, we have two different cases regarding the domain ${\Omega }_L\times {\Omega }_R.$

  -

  If $\beta \notin {\mathbb{R}}_e$, then, due to (9), the negative region cannot be included into the domain of the $\beta$-divergence. Therefore, we have

  $${\Omega }_L\times {\Omega }_R\subseteq {\mathbb{R}}_{+}^n\times {\mathbb{R}}_{+}^n.$$

  -

  If $\beta \in {\mathbb{R}}_e$, then due to (9), the domain of the $\beta$-divergence can be defined to include negative region. That is, we have

  $${\mathbb{R}}_{--}^n\times {\mathbb{R}}_{--}^n\subseteq {\Omega }_L\times {\Omega }_R.$$

• Case 2: $D_{\beta }\left(b\right|u)<+\infty ,$ for all $(b,u)\in {\Omega }_L\times {\Omega }_R$.
  Basically, the $\beta$-divergence can be expressed as

  $$D_{\beta }\left(b\right|u)=\langle b{\int }_u^b{x}^{\beta -2}dx-{\int }_u^b{x}^{\beta -1}dx,\mathbf{1}\rangle .$$

  That is, it is based on integrations of power functions of real variables u and b in ${\mathbb{R}}^n$. Therefore, we only need to see whether or not the integration in $D_{\beta }\left(b\right|u)$ is well defined at $\left\{\mathbf{0}\right\}.$ We note that, after integration, the exponents of $b\in {\Omega }_L$ and $u\in {\Omega }_R$ are different and thus the corresponding domains ${\Omega }_R$ and ${\Omega }_L$ could be different as well. Hence, we should consider the following three different cases:

  -

  $\mathbf{\beta }>\mathbf{1}$: We do not have any singularity at $\left\{\mathbf{0}\right\}$ with respect to $b\in {\Omega }_L$ and $u\in {\Omega }_R$. Therefore, we have $\left\{\mathbf{0}\right\}\subset {\Omega }_L={\Omega }_R.$

  -

  $\mathbf{0}<\mathbf{\beta }\le \mathbf{1}$: After integration, $b\in {\Omega }_L$ does not have any singularity at $\left\{\mathbf{0}\right\}$. However, $u\in {\Omega }_R$ has a singularity at $\left\{\mathbf{0}\right\}$. Therefore, we have $\left\{\mathbf{0}\right\}\subset {\Omega }_L$ but $\left\{\mathbf{0}\right\}\not\subset {\Omega }_R$ and thus ${\Omega }_L\ne {\Omega }_R$ in this region.

  -

  $\mathbf{\beta }\le \mathbf{0}$: In this case, both $b\in {\Omega }_L$ and $u\in {\Omega }_R$ have singularity at $\left\{\mathbf{0}\right\}$. Thus, $\left\{\mathbf{0}\right\}\not\subset {\Omega }_L={\Omega }_R.$

Based upon the analysis in Cases 1 and 2, we have six different choices of domain ${\Omega }_L\times {\Omega }_R$ for the $\beta$-divergence. It is summarized in Table 3 and illustrated in Figure 1. Since we only consider a convex domain, ${\Omega }_L$ and ${\Omega }_R$ should be selected as a convex set for each variable. In addition, due to the inherent integral formulation of $\beta$-divergence between b and u, the domain of both variables should be determined to have the same sign. ☐

As observed in Table 3, if $\beta \in {\mathbb{R}}_e\cap {\mathbb{R}}_{-}$, then there is a symmetry in the selection of the domain of the $\beta$-divergence. That is, ${\Omega }_L\times {\Omega }_R={\mathbb{R}}_{++}^n\times {\mathbb{R}}_{++}^n$ or ${\mathbb{R}}_{--}^n\times {\mathbb{R}}_{--}^n.$ Especially, if $\beta =0$, then the $\beta$-divergence corresponds to the Itakura–Saito-divergence $D_{\beta }\left(b\right|u)=\langle \left(\frac{b}{u}\right)-log\left(\frac{b}{u}\right)-\mathbf{1},\mathbf{1}\rangle ,$ where the domain of it can be ${\mathbb{R}}_{++}^n\times {\mathbb{R}}_{++}^n$ or ${\mathbb{R}}_{--}^n\times {\mathbb{R}}_{--}^n.$ The positive domain is generally preferable, since it is related to real applications, e.g., intensity data type in the SAR system [11,12]. However, if we reformulate the $\beta$-divergence with the Bregman-divergence, then the negative domain commonly appears in the dual Bregman-divergence. In addition, we note that, due to Theorem 1, the $\beta$-divergence with the domain defined in Table 3 satisfies the following distance-like properties of the generic divergence [1,2]:

$$\begin{array}{cc}\hfill & D_{\beta }\left(b\right|u)\in {\mathbb{R}}_{+},\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill & D_{\beta }\left(b\right|u)=0\mathrm{if}\mathrm{and}\mathrm{only}\mathrm{if}b=u,\hfill \end{array}$$

(12)

where (11) is followed from the Definition of the domain of the $\beta$-divergence and (10). Note that (12) is satisfied, if we restrict the domain of the $\beta$-divergence as ${\Omega }_R\times {\Omega }_R.$ In fact, let us assume that $(b,u)\in {\Omega }_R\times {\Omega }_R$. It is trivial to show that $b=u\Rightarrow D_{\beta }\left(b\right|u)$. Therefore, we only need to show $D_{\beta }\left(b\right|u)=0\Rightarrow b=u$. Letting $b\ne u$, we then get $D_{\beta }\left(b\right|u)\ge \frac{1}{2}{\parallel b-u\parallel }_2^2{M}_G>0$ from (10).

Since the $\beta$-divergence (5) with its domain defined in Table 3 has the distance-like properties, (11) and (12), we can make a variational model with the $\beta$-divergence and a regularization term for smoothness constraint of the given data b. The following is an example of a variational model based on the $\beta$-divergence [12]

$$u_{\beta }^{\ast }=\underset{u\in B}{\arg \min }{F}_{\beta }\left(u\right)=D_{\beta }\left(b\right|u)+\lambda R\left(u\right),$$

(13)

where $\lambda >0$ and $B\subseteq {\Omega }_R$ is a domain of $F_{\beta }$ for a given $b\in B$. Note that B is an open convex set induced from the physical constraints of the observed data b. In the case of a prior $R\left(u\right)$, it can be a sparsity-enforcing function such as total variation (TV) $TV\left(u\right)={\parallel \nabla u\parallel }_1$ [25,26] and frame [34]. We call (13) the β-sparse model or β-TV [12], if TV is used as a prior. Under the domain restriction in Table 3, actually, we have lots of freedom in choosing $\beta \in \mathbb{R}$ of the $\beta$-sparse model (13). However, if we add additional constraints, such as convexity onto the $\beta$-divergence, then interestingly, the possible choice of $\beta$ is dramatically reduced to a small set. For example, $D_{\beta }\left(b\right|u)$ with respect to u is convex on its domain only if $\beta \in [1,2]$ [6]. Outside of this region, i.e., $\beta \in \mathbb{R}\setminus [1,2]$, the convexity of it depends on the given data b [12]. In Section 4.1, we analyze the convexity of the $\beta$-divergence via the right Bregman proximity operator [28].

Although the proposed $\beta$-sparse model $F_{\beta }\left(u\right)$ in (13) is not convex in general, $F_{\beta }\left(u\right)$ has an interesting global optimum property [11,12], in case $\lambda =0$. See also [35,36]. For completeness of the article, we add it below.

Theorem 2 ([11,36]).

For a given observed data $\{b^1,..,b^{\left|N\right|}\}$, let B be an open convex set in ${\Omega }_R$, $\beta \in \mathbb{R},$ $b^j\in B,$ and $\mu =\frac{1}{\left|N\right|}{\sum }_{j=1}^{\left|N\right|}{b}^j.$ Then, we obtain that

$$\sum _{j=1}^{\left|N\right|}{D}_{\beta }\left(b^j\right|u)\ge \sum _{j=1}^{\left|N\right|}{D}_{\beta }\left(b^j\right|\mu )$$

(14)

is always satisfied, regardless of the choice of $u\in B.$

Note that $\mu$ in (14) corresponds to the $\beta$-centroid in segmentation problem and is related to the Bregman centroid, which is extensively studied in [37]. In SAR image processing, if $\beta =0$, then (14) corresponds to the multi-looking process, which is commonly used to reduce speckle noise in SAR data [12,22,23].

3. The Bregman-Divergence Associated with the Convex Function of Legendre Type for the $\beta$-Divergence

In this section, we study the Bregman-divergence associated with the convex function of Legendre type and its connection to the $\beta$-divergence. Although there is partial equivalence between two divergences, the Bregman-divergence has an important mathematical dual formulation. With a negative domain, the dual Bregman-divergence is unambiguously useful for convex reformulation of the nonconvex $\beta$-sparse model (13) (see Section 4).

Before we proceed further, let us first review the convex function of Legendre type (see Section 26 [3]). See also [27,29].

Definition 3.

Let $\Phi :\Omega \to \mathbb{R}$ be lower semicontinuous, convex, and proper function defined on $\Omega =dom\Phi \subseteq {\mathbb{R}}^n$. Then, Φ is the convex function of Legendre type (or Legendre), if the following conditions are satisfied (i.e., Φ is essentially strictly convex and essentially smooth):

1. 

$int(\Omega )\ne \varnothing .$

2. 

Φ is differentiable on $int(\Omega ).$

3. 

$\forall x\in bd(\Omega )$ and $\forall y\in int(\Omega ),$

$$\underset{t\downarrow 0}{lim}\langle \nabla \Phi (x+t(y-x)),y-x\rangle =-\infty .$$

4. 

Φ is strictly convex on $int(\Omega ).$

Here, $\Omega =dom\Phi =\{x\in {\mathbb{R}}^n\mid \Phi \left(u\right)\in \mathbb{R}\}$ is a convex set and $bd(\Omega )$ is the boundary of the domain Ω.

The main advantage of the convex function of Legendre type is that the inverse function of the gradient of it has an isomorphism with the gradient of its conjugate function as described below. This is a useful property when we characterize the dual structure of the Bregman-divergence associated with the convex function of Legendre type.

Theorem 3 ([3,27]).

Let $\Omega =dom\Phi$ and ${\Omega }^{\ast }=dom{\Phi }^{\ast }=\{x^{\ast }\in {\mathbb{R}}^n|{\Phi }^{\ast }\left(x^{\ast }\right)\in \mathbb{R}\}.$ Then, the function Φ is the convex function of Legendre type if and only if its conjugate

$${\Phi }^{\ast }\left(x\right)=\underset{\xi \in \Omega }{sup}\langle x,\xi \rangle -\Phi \left(\xi \right)$$

is the convex function of Legendre type. In this case, the gradient mapping

$$\nabla \Phi :int(\Omega )\to int\left({\Omega }^{\ast }\right)$$

(15)

is an isomorphism with its inverse mapping ${(\nabla \Phi )}^{-1}=\nabla {\Phi }^{\ast }$.

For more details on Theorem 3, see Theorem 26.5 in [3] and Fact 2.9 in [27]. Let us assume that $\Phi$ be the convex function of Legendre type. Then, we can define the Bregman-divergence $D_{\Phi }:\Omega \times int(\Omega )\to {\mathbb{R}}_{+}$ associated with Legendre $\Phi$:

$$D_{\Phi }\left(b\right|u)=\Phi \left(b\right)-\Phi \left(u\right)-\langle b-u,\nabla \Phi \left(u\right)\rangle ,$$

(16)

where $b\in \Omega$ and $u\in int(\Omega )$. Several functions we are interested in are in a category of the convex function of Legendre type. For instance, Shannon entropy function $\Phi \left(x\right)=\langle xlogx,\mathbf{1}\rangle$ is a typical example of Legendre. the Bregman-divergence associated with it corresponds to the $\beta$-divergence with $\beta =1,$ i.e., Generalized Kullback–Leibler-divergence. We note that there is the convex function of Legendre type which does not have a corresponding $\beta$-divergence. For instance, Fermi–Dirac entropy function $\Phi \left(x\right)=\langle xlnx+(1-x)ln(1-x),\mathbf{1}\rangle$ is Legendre and the associated Bregman divergence is the Logistic loss function $D_{\Phi }\left(b\right|u)=\langle bln\left(\frac{b}{u}\right)+(1-b)ln\left(\frac{1-b}{1-u}\right),\mathbf{1}\rangle .$ See [27,36] for more details on the Bregman-divergence. In the following, we summarize various useful features of the Bregman-divergence associated with Legendre $\Phi$.

Theorem 4.

Let $\Omega =dom\Phi$ and ${\Omega }^{\ast }=dom{\Phi }^{\ast }.$ Then, the Bregman-divergence associated with the convex function of Legendre type Φ satisfies the following.

1. 

$D_{\Phi }\left(b\right|u)$ is strictly convex with respect to b on $int(\Omega )$.

2. 

For any $u\in int(\Omega )$, $D_{\Phi }\left(b\right|u)$ is coercive with respect to b, i.e., ${lim}_{\parallel b\parallel \to \infty }{D}_{\Phi }\left(b\right|u)\to \infty .$

3. 

For some $b\in int(\Omega )$, $D_{\Phi }\left(b\right|u)$ is coercive with respect to u if and only if ${\Omega }^{\ast }=int\left({\Omega }^{\ast }\right)$.

4. 

$D_{\Phi }\left(b\right|u)=0$ if and only if $b=u,$ where $b\in int(\Omega )$

5. 

For any $b,u\in int(\Omega ),$

$$D_{\Phi }\left(b\right|u)=D_{{\Phi }^{\ast }}(\nabla \Phi \left(u\right),\nabla \Phi \left(b\right)),$$

(17)

where ${\Phi }^{\ast }$ is the conjugate function of Φ.

6. 

For any $b^{\ast },u^{\ast }\in int\left({\Omega }^{\ast }\right),$

$$D_{{\Phi }^{\ast }}\left(b^{\ast }\right|u^{\ast })=D_{\Phi }(\nabla {\Phi }^{\ast }\left(u^{\ast }\right),\nabla {\Phi }^{\ast }\left(b^{\ast }\right)).$$

Proof. 

Since $\Phi$ is the convex function of Legendre type, $\Phi$ is strictly convex on $int(\Omega )$. Hence, $\left(1\right)$ is trivial. The proofs of $\left(2\right)$–$\left(6\right)$ are in Theorem 3.7, Theorem 3.9 and Corollary 3.11 in [27]. ☐

In the above Theorem, the dual formulation (17) is a unique feature of the Bregman-divergence with Legendre. Unfortunately, the $\beta$-divergence does not have a corresponding dual concept. Later, we will show how to use the dual Bregman-divergence (17) to make a convex reformulated model of the nonconvex $\beta$-sparse model (13). In addition, we note that the $\beta$-divergence (5) is established based on the extended logarithmic function (4), which is an equivalence class in terms of a constant function. Therefore, we can say that the $\beta$-divergence is invariant with respect to a constant function in the extended logarithmic function. Interestingly, the base function $\Phi$ of the Bregman-divergence also has such kind of invariance property with respect to an affine function. For this, $\Phi$ does not need to be Legendre. However, for simplicity, we assume that $\Phi$ is Legendre. The details are following.

Proposition 1.

Let us define an equivalence class of the convex function of Legendre type Φ in terms of affine function as follows:

$${[\Phi ]}_{\mathcal{A}}=\{{\Phi }^{\prime }\left(x\right)\in \mathbb{R}|{\Phi }^{\prime }\left(x\right)=\Phi \left(x\right)+\langle c,x\rangle +\langle d,\mathbf{1}\rangle ,c,d\in {\mathbb{R}}^n\},$$

(18)

where $\mathcal{A}=\{\langle c,x\rangle +\langle d,\mathbf{1}\rangle |c,d\in {\mathbb{R}}^n\}.$ Then, the Bregman-divergence $D_{{[\Phi ]}_{\mathcal{A}}}$ associated with an equivalence class ${[\Phi ]}_{\mathcal{A}}$ is equal to the Bregman-divergence $D_{\Phi }$ associated with Φ, irrespective of the choice of an affine function $\mathcal{A}$.

Proof. 

We have the following equivalence with respect to an arbitrary affine function $\mathcal{A}$:

$$\begin{array}{ccc}\hfill D_{{[\Phi ]}_{\mathcal{A}}}\left(b\right|u)& =& {[\Phi \left(b\right)]}_{\mathcal{A}}-{[\Phi \left(u\right)]}_{\mathcal{A}}-\langle b-u,\nabla {[\Phi \left(u\right)]}_{\mathcal{A}}\rangle \hfill \\ & =& (\Phi (b)+\langle c,b\rangle +\langle d,\mathbf{1}\rangle )-(\Phi (u)+\langle c,u\rangle +\langle d,\mathbf{1}\rangle )-\langle b-u,\nabla \Phi \left(u\right)+c\rangle \hfill \\ & =& \Phi \left(b\right)-\Phi \left(u\right)-\langle b-u,\nabla \Phi \left(u\right)\rangle \hfill \\ & =& D_{\Phi }\left(b\right|u),\hfill \end{array}$$

where $(b,u)\in \Omega \times int(\Omega )$ and $\Omega =dom\Phi .$ Therefore, $\Phi$ with an arbitrary affine function $\mathcal{A}$ (i.e., an equivalence class ${[\Phi ]}_{\mathcal{A}}$ in terms of affine function $\mathcal{A}$) does not change the structure of the Bregman-divergence $D_{\Phi }\left(b\right|u)$ at all. ☐

To connect the $\beta$-divergence and the Bregman-divergence associated with Legendre, we need to find a specialized convex function of Legendre type. Based on the comments in [2], we use an integral formula of the extended logarithmic function for the special convex function of Legendre type. Through this connection, we can reformulate the $\beta$-divergence into the Bregman-divergence associated with the convex function of Legendre type. The details are following.

Theorem 5.

Let $x\in \Omega \subseteq {\mathbb{R}}^n$ and

$$\begin{array}{ccc}\hfill \Phi \left(x\right)& =& \langle {\int }_{\mathbf{d}}^x{ln}_{2-\beta }\left(t\right)dt,\mathbf{1}\rangle =\left\{\begin{array}{c}\langle -lnx,\mathbf{1}\rangle ,\mathit{if}\beta =0,\hfill \\ \langle xlnx,\mathbf{1}\rangle ,\mathit{if}\beta =1,\hfill \\ \langle \frac{1}{\beta (\beta -1)}{x}^{\beta },\mathbf{1}\rangle ,\mathit{if}\beta \ne 0,1,\hfill \end{array}\right.\hfill \end{array}$$

(19)

where $\Omega =dom\Phi =\{x\in {\mathbb{R}}^n|\Phi \left(x\right)\in \mathbb{R}\}$, ${ln}_{\alpha }\left(t\right)$ is the extended logarithmic function in (4), $\mathbf{d}$ is an arbitrary constant vector in ${\mathbb{R}}^n$ and be selected to satisfy the condition $\Phi \left(x\right)\in \mathbb{R}$. For simplicity, by Proposition 1, we dropped all affine function in $\Phi \left(x\right).$ Then, Φ in (19) is the convex function of Legendre type with the domain Ω given below:

$$\left\{\begin{array}{c}I.\mathit{entire}\mathit{region}:\hfill \\ \beta >1,\beta \in {\mathbb{R}}_e\mathit{and}\Omega ={\mathbb{R}}^n,\hfill \\ \mathit{II}.\mathit{positive}\mathit{region}:\hfill \\ 0<\beta \le 1\mathit{and}\Omega ={\mathbb{R}}_{+}^n,\hfill \\ \beta \le 0\mathit{and}\Omega ={\mathbb{R}}_{++}^n,\hfill \\ \mathit{III}.\mathit{negative}\mathit{region}:\hfill \\ 0<\beta <1,\beta \in {\mathbb{R}}_e\mathit{and}\Omega ={\mathbb{R}}_{-}^n,\hfill \\ \beta <0,\beta \in {\mathbb{R}}_e\mathit{and}\Omega ={\mathbb{R}}_{--}^n.\hfill \end{array}\right.$$

(20)

Proof. 

For simplicity, all affine functions are left out based on Proposition 1. In addition, it is trivial to show that $\Phi \left(x\right)$ is the Burg entropy ($\langle -lnx,\mathbf{1}\rangle$ with $\Omega ={\mathbb{R}}_{++}^n$) if $\beta =0$ and the Shannon entropy ($\langle xlnx,\mathbf{1}\rangle$ with $\Omega ={\mathbb{R}}_{+}^n$) if $\beta =1$. They are well-known examples of Legendre. As noticed in (2), the corresponding Bregman-divergences are Itakura–Saito-divergence and Generalized Kullback–Leibler-divergence.

Now, we only need to check whether $\Phi \left(x\right)=\langle \frac{1}{\beta (\beta -1)}{x}^{\beta },\mathbf{1}\rangle$ ($\beta \ne 0,1$) is Legendre or not. Among four conditions in Definition 3, it is trivial to show that $\Phi \left(x\right)$ satisfies the conditions 1 and 2. In the end, $\Phi \left(x\right)$ with $\beta \ne 0,1$ and two Legendre conditions 3 and 4 are left.

I.

Condition 3 in Definition 3:

Since $\Phi \left(x\right)=\langle \frac{1}{\beta (\beta -1)}{x}^{\beta },\mathbf{1}\rangle$ ($\beta \ne 0,1$) is a power function, the only possible boundary of $\Phi$ is $\left\{\mathbf{0}\right\}$. Therefore, we search $\beta \in \mathbb{R}\setminus \{0,1\}$ to find where condition 3 is satisfied. Since $\nabla \Phi \left(x\right)=\frac{1}{\beta -1}{x}^{\beta -1}$, we get the following at the potential boundary $\left\{\mathbf{0}\right\}:$

-

$y\in {\mathbb{R}}_{++}^n$:

$$\begin{array}{ccc}\hfill \underset{t\downarrow 0}{lim}\langle \nabla \Phi (0+ty),y\rangle & =& \left\{\begin{array}{c}-\infty ,\mathrm{if}\beta <1\mathrm{and}\beta \ne 0,\hfill \\ 0,\mathrm{if}\beta >1.\hfill \end{array}\right.\hfill \end{array}$$

-

$y\in {\mathbb{R}}_{--}^n$:

$$\begin{array}{ccc}\hfill \underset{t\downarrow 0}{lim}\langle \nabla \Phi (0+ty),y\rangle & =& \left\{\begin{array}{c}-\infty ,\mathrm{if}\beta <1\mathrm{and}\beta \in {\mathbb{R}}_e\setminus \left\{0\right\},\hfill \\ +\infty ,\mathrm{if}\beta <1\mathrm{and}\beta \in {\mathbb{R}}_o,\hfill \\ 0,\mathrm{if}\beta >1.\hfill \end{array}\right.\hfill \end{array}$$

In summary, the following are the possible domains and the corresponding $\beta$ for condition 3. We note that, if $\beta >1$, then $\mathbf{0}\notin bd(dom\Phi )$:

-

${\mathbb{R}}_{++}^n\subseteq \Omega$ and $\beta \in (-\infty ,1)\setminus \left\{0\right\},$

-

${\mathbb{R}}_{--}^n\subseteq \Omega$ and $\beta \in ((-\infty ,1)\setminus \left\{0\right\})\cap {\mathbb{R}}_e.$

II.

Condition 4 in Definition 3:

It can be easily checked by the fact that $\Phi$ is strictly convex on $int(\Omega )$ if and only if $\nabla \Phi$ is strictly monotone, that is, the following is satisfied [38] :

$$\langle \nabla \Phi \left(x\right)-\nabla \Phi \left(x^{\prime }\right),x-x^{\prime }\rangle >0\mathrm{whenever}x\ne x^{\prime }.$$

Since $\nabla \Phi$ is separable in terms of dimension, we only need to show that $\nabla \Phi \left(x\right)$ is a strictly increasing function on $int(\Omega )\cap \mathbb{R}.$ Note that if ${\nabla }^2\Phi \left(x\right)=x^{\beta -2}>0$ on an open region, then $\Phi$ is strictly convex (i.e., $\nabla \Phi$ is strictly increasing) in that region:

-

${\nabla }^2\Phi \left(x\right)>0$ if $\beta \in \mathbb{R}\setminus \{0,1\}$ and $x\in {\mathbb{R}}_{++}^n,$

-

${\nabla }^2\Phi \left(x\right)>0$ if $\beta \in (\mathbb{R}\setminus \{0,1\})\cap {\mathbb{R}}_e$ and $x\in {\mathbb{R}}_{--}^n.$

Note that at $\left\{\mathbf{0}\right\}$, we need to directly show that $\nabla \Phi$ is strictly increasing.

Now, we integrate the information in the above Legendre conditions 3 and 4 for the decision of the domain $\Omega =dom\Phi$ based on $\beta$. The details are the following:

• $\beta >1$

  -

  $\beta \in {\mathbb{R}}_e:$ $dom\Phi ={\mathbb{R}}^n$ and $\forall x\in {\mathbb{R}}_{-}^n,x^{\beta -1}=-{\left|x\right|}^{\beta -1}.$ Therefore, $\nabla \Phi \left(x\right)$ is strictly increasing function on $int(dom\Phi )$ since $\nabla \Phi \left(x\right)=-\nabla \Phi (-x).$

  -

  $\beta \in {\mathbb{R}}_o:$ $dom\Phi ={\mathbb{R}}^n$ but $\Phi$ is an odd function with respect to zero and thus it is not a convex function.

  -

  $\beta \in {\mathbb{R}}_x:$ $dom\Phi ={\mathbb{R}}_{+}^n$ but it does not satisfy condition 3 at $\mathbf{0}$.

• $0<\beta <1$

  -

  $\beta \in {\mathbb{R}}_e:$ $dom\Phi ={\mathbb{R}}^n$ but $dom\nabla \Phi ={\mathbb{R}}^n\setminus \left\{\mathbf{0}\right\}$ is not a convex set. Since $int(dom\Phi )\subset dom\nabla \Phi$ [3], to keep convexity of the domain $int(dom\Phi ),$ we need to select $int(dom\Phi )={\mathbb{R}}_{--}^n$ or ${\mathbb{R}}_{++}^n.$ That is, we have $dom\Phi ={\mathbb{R}}_{-}^n$ or $dom\Phi ={\mathbb{R}}_{+}^n.$ In both cases, we know that $\nabla \Phi$ is strictly increasing on $int(dom\Phi )$, since ${\nabla }^2\Phi =x^{\beta -2}>0,\forall x\in int(dom\Phi )$.

  -

  $\beta \in {\mathbb{R}}_o:$ Following the case $\beta \in {\mathbb{R}}_e$, we have $dom\Phi ={\mathbb{R}}_{+}^n$ or ${\mathbb{R}}_{-}^n.$ If $dom\Phi ={\mathbb{R}}_{+}^n$, then $\nabla \Phi$ is strictly increasing on $int(dom\Phi )$ (${\nabla }^2\Phi >0$). However, if $x<0$, then ${\nabla }^2\Phi \left(x\right)<0$ and thus it is not convex on its negative domain.

  -

  $\beta \in {\mathbb{R}}_x:$ $dom\Phi ={\mathbb{R}}_{+}^n$ and $\nabla \Phi$ is strictly increasing function on $int(dom\Phi )$ since ${\nabla }^2\Phi =x^{\beta -2}>0$ for all $x\in int(dom\Phi ).$

  -

  Condition 3 is satisfied on the above selected domain.

• $\beta <0$

  -

  $\beta \in {\mathbb{R}}_e:$ $dom\Phi ={\mathbb{R}}^n\setminus \left\{\mathbf{0}\right\}$ is not a convex set. Therefore, we need to restrict $dom\Phi$ as a convex set, i.e., $dom\Phi ={\mathbb{R}}_{--}^n$ or $dom\Phi ={\mathbb{R}}_{++}^n$. On both domains, $\nabla \Phi$ is strictly increasing (${\nabla }^2\Phi >0,\forall x\in dom\Phi$).

  -

  $\beta \in {\mathbb{R}}_o:$ Following the case $\beta \in {\mathbb{R}}_e,$ we have $dom\Phi ={\mathbb{R}}_{++}^n$ or ${\mathbb{R}}_{--}^n.$ If $dom\Phi ={\mathbb{R}}_{++}^n$ then $\nabla \Phi$ is strictly increasing on $int(dom\Phi ).$

  -

  $\beta \in {\mathbb{R}}_x:$ $dom\Phi ={\mathbb{R}}_{++}^n$ and $\nabla \Phi$ is strictly increasing function on $int(dom\Phi )$ since ${\nabla }^2\Phi =x^{\beta -2}>0$ for all $x\in int(dom\Phi ).$

  -

  Condition 3 is satisfied on the above selected domain.

 ☐

Remark 1.

Note that Φ in (19) should be an equivalence class

$${[\Phi \left(x\right)]}_{\mathcal{A}}=\langle {\int }_{\mathbf{d}}^x{\left[{ln}_{2-\beta }\left(t\right)\right]}_{\mathbf{c}}dt,\mathbf{1}\rangle$$

with an affine function $\mathcal{A}=\{\langle \alpha ,x\rangle +\langle \gamma ,\mathbf{1}\rangle |\alpha =f\left(\mathbf{c}\right),\gamma =g(\mathbf{c},\mathbf{d})\}.$ Here, ${\left[{ln}_{2-\beta }\left(t\right)\right]}_{\mathbf{c}}$ is an equivalence class of an extended logarithmic function in (4). As observed in (18), we have $D_{{[\Phi ]}_{\mathcal{A}}}\left(b\right|u)=D_{\Phi }\left(b\right|u)$ for any affine function $\mathcal{A}$. Therefore, we can drop all affine function in ${[\Phi ]}_{\mathcal{A}}.$

Since $\Phi$ in (19) is Legendre under the domain condition (20), we can establish a new Bregman-divergence associated with $\Phi$ in (19). Interestingly, it corresponds to the $\beta$-divergence (1) [2]. However, there is a mismatch between the domain of the Bregman-divergence in (20) and the domain of the $\beta$-divergence in Table 3. We summarize it in Table 1. As a matter of fact, the positive domain with $\beta >1$ is not defined in $\Phi$ (19) due to the Legendre condition. In addition, in the case of $\beta =0$, the negative domain ${\mathbb{R}}_{--}^n\times {\mathbb{R}}_{--}^n$ is not defined in the Bregman-divergence with $\Phi$ (19). In the following Theorem, we show that under the restriction of the domain of the $\beta$-divergence to the domain of the Bregman-divergence, we can get an equivalence between the $\beta$-divergence and the Bregman-divergence associated with Legendre $\Phi$ (19).

Theorem 6.

Let us consider the β-divergence (1) and the Bregman-divergence (16) associated with Legendre Φ (19). If we restrict the domain of the β-divergence with the domain of the Bregman-divergence associated with Φ (19), which is ${\Omega }_L\times {\Omega }_R=dom\Phi \times int(dom\Phi )$ (see Table 1), then the β-divergence is equal to the Bregman-divergence associated with Legendre Φ(19).

Proof. 

Since the domain of the $\beta$-divergence ${\Omega }_L\times {\Omega }_R$ is set up with $dom\Phi \times int(dom\Phi )$, the $\beta$-divergence is well defined with the restricted domain. In the following, we show an equivalence between the $\beta$-divergence and the Bregman-divergence under the domain condition of the Bregman-divergence:

$$\begin{array}{ccc}\hfill D_{\beta }\left(b\right|u)& =& \langle {\int }_u^b\frac{b-t}{t^{2-\beta }}dt,\mathbf{1}\rangle =\langle -{\int }_u^b{t}^{\beta -1}dt,\mathbf{1}\rangle +\langle b,{\int }_u^b{t}^{\beta -2}dt\rangle \hfill \\ & =& \langle (1-\beta ){\int }_u^b\nabla \Phi \left(t\right)dt,\mathbf{1}\rangle +\langle b,\frac{1}{\beta -1}(b^{\beta -1}-u^{\beta -1})\rangle \hfill \\ & =& \Phi \left(b\right)-\Phi \left(u\right)-\frac{1}{\beta -1}{u}^{\beta -1}(b-u)\hfill \\ & =& \Phi \left(b\right)-\Phi \left(u\right)-\langle b-u,\nabla \Phi \left(u\right)\rangle .\hfill \end{array}$$

Note that we do not use any $\nabla \Phi \left(b\right)$ information in the above derivation and thus the above equivalence is well satisfied within the domain of the Bregman-divergence associated with Legendre $\Phi$ (19). ☐

In the following Theorem, we calculate the conjugate function ${\Phi }^{\ast }$ and the corresponding domain $dom{\Phi }^{\ast }$ of the convex function of Legendre type $\Phi$ defined in (19). The computation of the domain of ${\Phi }^{\ast }$ is useful in determining the structure of $D_{\Phi }\left(b\right|u)$. For instance, as noticed in Theorem 4 (3), if $dom{\Phi }^{\ast }$ is open, then the corresponding Bregman-divergence $D_{\Phi }\left(b\right|u)$ is coercive with respect to $u\in int(dom\Phi )$. Surprisingly, when $\beta \in [0,1)$, $D_{\Phi }\left(b\right|u)$ is not convex but coercive with respect to u. This fact is importantly used in SAR speckle reduction problems [12,15,21].

Theorem 7.

Let Φ (19) be the convex function of Legendre type with $dom\Phi$ (20). Then, ${\Phi }^{\ast }$, the conjugate of Φ, and the corresponding domain $dom{\Phi }^{\ast }$ is calculated as follows:

• $\beta =0$ and $dom\Phi ={\mathbb{R}}_{++}^n$: $dom{\Phi }^{\ast }={\mathbb{R}}_{--}^n$

  $${\Phi }^{\ast }\left(x\right)=-\langle \mathbf{1}+ln(-x),\mathbf{1}\rangle .$$
• $\beta =1$ and $dom\Phi ={\mathbb{R}}_{+}^n$: $dom{\Phi }^{\ast }={\mathbb{R}}^n$

  $${\Phi }^{\ast }\left(x\right)=\langle \exp (x-\mathbf{1}),\mathbf{1}\rangle .$$
• $\beta \notin \{0,1\}:$

  $${\Phi }^{\ast }\left(x\right)=\langle \frac{1}{\beta }{\left((\beta -1)x\right)}^{\frac{\beta }{\beta -1}},\mathbf{1}\rangle .$$
  In this case, $dom{\Phi }^{\ast }$ depends on β:

  $$\left\{\begin{array}{c}\beta >1,\beta \in {\mathbb{R}}_e,\mathit{and}dom\Phi ={\mathbb{R}}^n:dom{\Phi }^{\ast }={\mathbb{R}}^n,\hfill \\ 0<\beta <1,\mathit{and}dom\Phi ={\mathbb{R}}_{+}^n:dom{\Phi }^{\ast }={\mathbb{R}}_{--}^n,\hfill \\ 0<\beta <1,\beta \in {\mathbb{R}}_e,\mathit{and}dom\Phi ={\mathbb{R}}_{-}^n:dom{\Phi }^{\ast }={\mathbb{R}}_{++}^n,\hfill \\ \beta <0,\mathit{and}dom\Phi ={\mathbb{R}}_{++}^n:dom{\Phi }^{\ast }={\mathbb{R}}_{-}^n,\hfill \\ \beta <0,\beta \in {\mathbb{R}}_e,\mathit{and}dom\Phi ={\mathbb{R}}_{--}^n:dom{\Phi }^{\ast }={\mathbb{R}}_{+}^n.\hfill \end{array}\right.$$

  (21)

Proof. 

Since $\Phi$ is Legendre, from Theorem 3, if $x\in int\left(dom{\Phi }^{\ast }\right),$ then we have

$${\Phi }^{\ast }\left(x\right)=\langle x,\nabla {\Phi }^{-1}\left(x\right)\rangle -\Phi (\nabla {\Phi }^{-1}\left(x\right)).$$

(22)

As noticed in (20), the domain of $\Phi$ (19) depends on $\beta$ and thus the domain of its conjugate function ${\Phi }^{\ast }$ also depends on $\beta$. We categorize $dom{\Phi }^{\ast }$ below, by using (22):

• $\beta =0:$ $\Phi \left(x\right)=\langle -lnx,\mathbf{1}\rangle$ and $dom\Phi ={\mathbb{R}}_{++}^n.$ From (22), the conjugate function ${\Phi }^{\ast }$ is calculated as

  $$\begin{array}{c}\hfill {\Phi }^{\ast }\left(x\right)=\langle x\left(-\frac{1}{x}\right)+ln\left(-\frac{1}{x}\right),\mathbf{1}\rangle =\langle -\mathbf{1}-ln(-x),\mathbf{1}\rangle .\end{array}$$

  Therefore, the domain of ${\Phi }^{\ast }$ becomes $dom{\Phi }^{\ast }={\mathbb{R}}_{--}^n.$
• $\beta =1:$ $\Phi \left(x\right)=\langle xlnx,\mathbf{1}\rangle$ and $dom\Phi ={\mathbb{R}}_{+}^n.$ From (22), the conjugate function ${\Phi }^{\ast }$ is calculated as

  $$\begin{array}{ccc}\hfill {\Phi }^{\ast }\left(x\right)& =& \langle x\exp (x-1)-\exp (x-1)ln\left(\exp \right(x-1\left)\right),\mathbf{1}\rangle \hfill \\ & =& \langle x\exp (x-1)-(x-1)\exp (x-1),\mathbf{1}\rangle =\langle \exp (x-1),\mathbf{1}\rangle .\hfill \end{array}$$

  It is trivial to show that $dom{\Phi }^{\ast }={\mathbb{R}}^n.$
• $\beta \ne 0,1:$ $\Phi \left(x\right)=\langle \frac{1}{\beta (\beta -1)}{x}^{\beta },\mathbf{1}\rangle$ and $dom\Phi$ is given in (20). By simple calculation, we get

  $$\nabla {\Phi }^{-1}\left(x\right)={\left((\beta -1)x\right)}^{\frac{1}{(\beta -1)}},x\in int\left(dom{\Phi }^{\ast }\right).$$

  (23)

  and from (22), the conjugate function ${\Phi }^{\ast }$ is derived as follows:

  $$\begin{array}{cc}\hfill {\Phi }^{\ast }\left(x\right)& =\langle x{\left((\beta -1)x\right)}^{1/(\beta -1)}-\frac{1}{\beta (\beta -1)}{\left((\beta -1)x\right)}^{\beta /(\beta -1)},\mathbf{1}\rangle \hfill \\ & =\langle \frac{1}{\beta }{\left((\beta -1)x\right)}^{\beta /(\beta -1)},\mathbf{1}\rangle .\hfill \end{array}$$

  (24)
  Now, we need to decide the domain

  $$dom{\Phi }^{\ast }=\{x\in {\mathbb{R}}^n|{\Phi }^{\ast }\left(x\right)\in \mathbb{R}\}.$$

  While we identify $dom{\Phi }^{\ast }$, it should be selected based on the following isomorphism (in Theorem 3)

  $$\nabla \Phi :int(dom\Phi )\to int\left(dom{\Phi }^{\ast }\right),$$

  where $\nabla \Phi \left(x\right)=\frac{1}{\beta -1}{x}^{\beta -1}={(\nabla {\Phi }^{\ast })}^{-1}$ and the following estimation

  $$\beta /(\beta -1)\in {\mathbb{R}}_e,\mathit{if}\beta \in {\mathbb{R}}_e.$$
  With the above information and the classification of $dom\Phi$ in (20), we are going to decide $dom{\Phi }^{\ast }$ based on $\beta$.

  -

  $\beta >1$ and $\beta \in {\mathbb{R}}_e$ and $dom\Phi ={\mathbb{R}}^n$:

  We have $\beta /(\beta -1)\in {\mathbb{R}}_e\cap {\mathbb{R}}_{++}$ and thus $dom{\Phi }^{\ast }={\mathbb{R}}^n.$ In addition, for all $x\in {\mathbb{R}}^n$, $\nabla \Phi \left(x\right)=\frac{1}{\beta -1}{x}^{\beta -1}\in {\mathbb{R}}^n$ is well defined.

  -

  $0<\beta <1$ and $dom\Phi ={\mathbb{R}}_{+}^n$:

  In this case, $\beta /(\beta -1)<0$ and $\beta -1<0$. Therefore, for all $\beta \in (0,1)$, we have $dom{\Phi }^{\ast }={\mathbb{R}}_{--}^n.$ In addition, for all $x\in {\mathbb{R}}_{++}^n,$ we have $\nabla \Phi \left(x\right)\in {\mathbb{R}}_{--}^n.$ That is, the isomorphism between $int(dom\Phi )$ and $int\left(dom{\Phi }^{\ast }\right)$ is well defined.

  -

  $0<\beta <1$ and $\beta \in {\mathbb{R}}_e$ and $dom\Phi ={\mathbb{R}}_{-}^n$:

  In this case, $\beta /(\beta -1)<0$ and $\beta /(\beta -1)\in {\mathbb{R}}_e$ and $\beta -1<0$. Therefore, the possible $dom{\Phi }^{\ast }$ is ${\mathbb{R}}_{--}^n$ or ${\mathbb{R}}_{++}^n$. However, we need to choose $dom{\Phi }^{\ast }={\mathbb{R}}_{++}^n$ from the isomorphic mapping $\nabla \Phi :int(dom\Phi )\to int\left(dom{\Phi }^{\ast }\right)$.

  -

  $\beta <0$ and $dom\Phi ={\mathbb{R}}_{++}^n$:

  In this case, $0<\beta /(\beta -1)<1$. Therefore, we have $dom{\Phi }^{\ast }={\mathbb{R}}_{-}^n.$ Actually, this domain is well matched with the bijective mapping $\nabla \Phi \left(x\right)=\frac{1}{\beta -1}{x}^{\beta -1}<\mathbf{0}$ for all $x>\mathbf{0}.$

  -

  $\beta <0$ and $\beta \in {\mathbb{R}}_e$ and $dom\Phi ={\mathbb{R}}_{--}^n$:

  In this case, $\beta /(\beta -1)\in (0,1)\cap {\mathbb{R}}_e$. The possible domain is $dom{\Phi }^{\ast }={\mathbb{R}}^n.$ However, $dom\nabla {\Phi }^{\ast }={\mathbb{R}}_{--}^n\cup {\mathbb{R}}_{++}^n.$ From Theorem 23.4 in [3] , $int\left(dom{\Phi }^{\ast }\right)\subset dom\nabla {\Phi }^{\ast }$. Due to the convexity constraint of the domain, $dom{\Phi }^{\ast }={\mathbb{R}}_{+}^n$ or $dom{\Phi }^{\ast }={\mathbb{R}}_{-}^n$. Through the isomorphic mapping $\nabla \Phi \left(x\right)=\frac{1}{\beta -1}{x}^{\beta -1}>\mathbf{0},\forall x<\mathbf{0},$ we select $dom{\Phi }^{\ast }={\mathbb{R}}_{+}^n.$

 ☐

In the following, the global minimization property of the $\beta$-divergence in Theorem 2 is reformulated with the Bregman-divergence. For more details, see [35,36].

Theorem 8 ([36]).

For all $b^i\in int(dom\Phi )$ with an index set $i\in N$, the following inequality is always satisfied, irrespective of the choice of $u\in int(dom\Phi )$:

$$\sum _{i\in N}{D}_{\Phi }\left(b^i\right|u)\ge \sum _{i\in N}{D}_{\Phi }\left(b^i\right|\mu ),$$

(25)

where $\mu =\frac{1}{\left|N\right|}{\sum }_{i\in N}{b}^i$ and $\left|N\right|$ is the cardinality of the set N.

Proof. 

Let us start with the generalized Pythagoras Theorem [35] of the Bregman-divergence:

$$D_{\Phi }\left(b\right|u)-D_{\Phi }\left(b\right|\mu )=D_{\Phi }\left(\mu \right|u)+\langle \nabla \Phi \left(\mu \right)-\nabla \Phi \left(u\right),b-\mu \rangle .$$

(26)

For all $b_i\in int(dom\Phi )$ with the index set N, let $\mu =\frac{1}{\left|N\right|}{\sum }_{i\in N}{b}^i$. Then, from (26), we get

$$\frac{1}{\left|N\right|}\sum _{i\in N}\left[D_{\Phi }\left(b^i\right|u)-D_{\Phi }\left(b^i\right|\mu )=D_{\Phi }\left(\mu \right|u)+\langle \nabla \Phi \left(\mu \right)-\nabla \Phi \left(u\right),b^i-\mu \rangle \right].$$

Since $D_{\Phi }\left(\mu \right|u)\ge 0,$ we have

$$\sum _{i\in N}{D}_{\Phi }\left(b^i\right|u)\ge \sum _{i\in N}{D}_{\Phi }\left(b^i\right|\mu ),$$

irrespective of the choice of $u\in int(dom\Phi ).$ ☐

Note that $\mu$ in (25) corresponds to the Bregman centroid, which is extensively studied in [37]. Now, we are ready to jump into the variational model having the Bregman-divergence as its fitting term. Many various important variational models induced from the statistical distribution are in this category.

4. Bregman Variational Model—Bregman-TV

In this section, we study the $\beta$-sparse model (13) with TV regularization via Bregman divergence (16) associated with Legendre (19) under the domain condition in (20). First, we introduce Bregman proximity operators in Section 4.1 and then we demonstrate how to use dual Bregman-divergence with the negative domain for convex reformulation of the nonconvex $\beta$-TV model [12] in Section 4.2.

The image data in general is observed in $2D$ array and have a limited dynamic range, due to physical constraints of the image capturing system. Therefore, let us assume that the observed image data is bounded and also column-wise stacked. That is, $b\in B\subset {\mathbb{R}}_{+}^n$, where B is an open and bounded convex set. Now, we start with the following Bregman variational model with total variation, i.e., Bregman-TV.

$$\underset{u\in B}{\min }{D}_{\Phi }\left(b\right|Lu)+\lambda TV\left(u\right),$$

(27)

where $TV\left(u\right)={\sum }_{i=1}^n\parallel \nabla u_i\parallel$ is a typical sparsity constraint in image processing, $L:{\mathbb{R}}_{+}^n\to {\mathbb{R}}_{+}^n$ is a linear mapping, $\lambda >0$, and $B\subseteq int(dom\Phi )\subset {\mathbb{R}}_{+}^n$. Although the domain B is nonnegative in real applications, through the dual formulation of the Bregman-divergence (17), the nonpositive domain is very common and sometimes is useful for convex reformulation of nonconvex variational models appearing in SAR image enhancement problems. See Theorem 7 for the negative domain of the conjugate function ${\Phi }^{\ast }$.

We note that L is a matrix with nonnegative entries and it is designed based on various applications of image processing, e.g., for the image deblurring problem, L is a blur or a convolution matrix; for an image inpainting problem, L is a binary mask matrix; for an image denoising problem, L is an identity matrix. See [39,40] for more details on image denoising, deblurring, and inpainting problems with total variation or other sparsity constraints such as wavelet frames. The following are typical examples of the Bregman-TV induced from the various physical noise sources, e.g., Gaussian, Poisson, and Speckle noise:

• $\beta =2$: Image restoration problems (e.g., denoising, deblurring, inpainting) under Gaussian noise [25,26,41]

  $$\underset{u\in B}{\min }\frac{1}{2}{\parallel b-Lu\parallel }_2^2+\lambda TV\left(u\right).$$

  (28)
• $\beta =1$: Image restoration problems (e.g., denoising, deblurring, inpainting) under Poisson noise [32]

  $$\underset{u\in B}{\min }\langle Lu-bln\left(Lu\right),\mathbf{1}\rangle +\lambda TV\left(u\right).$$

  (29)
• $\beta =0$: Image restoration problems (e.g., denoising, deblurring, inpainting) under Gamma multiplicative noise (or speckle noise) [15]

  $$\underset{u\in B}{\min }\langle \left(\frac{b}{Lu}\right)+ln\left(Lu\right),\mathbf{1}\rangle +\lambda TV\left(u\right).$$

  (30)
• $\beta \in (0,1):$ A convex relaxed model [12] for the above SAR image restoration model (30). Additionally, this region is related to the compound Poisson distribution [8]

  $$\underset{u\in V}{\min }\langle \frac{b{\left(Lu\right)}^{\beta -1}}{1-\beta }+\frac{{\left(Lu\right)}^{\beta }}{\beta },\mathbf{1}\rangle +\lambda TV\left(u\right).$$

  (31)

For the remainder of this article, we only consider the image denoising problem ($L=I$). It is also known as the (nonconvex) right Bregman proximity operator [10].

4.1. Bregman Proximity Operators

In this section, we introduce the right and left Bregman proximity operator [9,10,30] based on the Bregman-divergence associated with $\Phi$ (19). In this section, let us assume that $\Phi$ (19) is convex, smooth, and (dimensionally) separable function (not necessarily Legendre). That is, the Bregman-divergence $D_{\Phi }\left(b\right|u)$ associated with $\Phi$ (19) also exists in the positive domain $dom\Phi ={\mathbb{R}}_{+}^n$ with $\beta >1$. See Table 1.

We note that the Bregman-divergence $D_{\Phi }\left(b\right|u)$ associated with $\Phi$ (19) is strictly convex with respect to b (see Theorem 4). On the other hand, convexity of $D_{\Phi }\left(b\right|u)$ with respect to u strongly depends on the observed data b and $\beta$ in $\Phi$ (19). Based on [9,12,28], we present three different convexities of the Bregman-divergence associated with $\Phi .$ Let $\Omega =dom\Phi$. Then, we have the following:

• The Bregman-divergence $D_{\Phi }$ (16) is jointly convex if $D_{\Phi }\left(b\right|u)$ is convex with respect to $(b,u)$ on $int(\Omega )\times int(\Omega ).$
• The Bregman-divergence $D_{\Phi }$ (16) is separately convex if $D_{\Phi }\left(b\right|u)$ is convex with respect to $u\in int(\Omega )$ for all $b\in int(\Omega ).$
• The Bregman-divergence $D_{\Phi }$ (16) is conditionally convex if $D_{\Phi }\left(b\right|u)$ is convex with respect to $u\in B$ for all $b\in B$, where $B\subseteq int(\Omega )$ is an open convex set and depends on b.

We note that the conditional convexity is first introduced in this article based on the previous analysis of the $\beta$-TV model [12]. The reason we are interested in conditional convexity is that, in real applications, the dynamic range of the observed data is very limited. For instance, the observed image data via an optical camera have 8-bit resolution (i.e., $b\in [0,2^8]$) [41] and the intensity level of the backscattered radar signal in SAR system is 32-bit resolution (i.e., $b\in [0,2^{32}]$) at most [21]. Therefore, it is natural to consider convexity depending on the given data b.

The following Theorem, mostly based on Theorem 3.3 in [28], is useful in characterizing convexity of the Bregman-divergence $D_{\Phi }\left(b\right|u)$ associated with $\Phi$ (19). We note that $Diag\left(A\right)$ is a vector with diagonal entries of a matrix (or tensor) A. Also, a function f is concave if and only if $f(\alpha u+(1-\alpha \left)v\right)\ge \alpha f\left(u\right)+(1-\alpha )f\left(v\right),$ for all $\alpha \in [0,1].$

Theorem 9.

Let $D_{\Phi }\left(b\right|u)$ be the Bregman-divergence associated with convex, smooth, and (dimensionally) separable function Φ. In addition, we assume that $h=\mathrm{Diag}({\nabla }^2\Phi )>\mathbf{0}$, then we have the following useful criterion for the convexity of $D_{\Phi }\left(b\right|u).$ Here, $\Omega =dom\Phi .$

(i) 

$D_{\Phi }\left(b\right|u)$ is jointly convex if and only if $1/h$ is concave. Note that, since $h=(h_1,\dots ,h_n)$ is (dimensionally) separable, $1/h$ is defined as $1/h\left(u\right)=(1/h_1\left(u\right),\dots ,1/h_n\left(u\right))$, it is concave if and only if h satisfy the following inequality:

$$h\left(u\right)+\nabla h\left(u\right)(u-b)\ge \frac{h^2\left(u\right)}{h\left(b\right)},\forall b,u\in int(\Omega ).$$

(32)

Moreover, if ${\nabla }^{2}h$ exists, then $D_{\Phi }\left(b\right|u)$ is jointly convex if and only if

$$h\left(u\right)\mathrm{Diag}\left({\nabla }^{2}h\left(u\right)\right)\ge 2{\left(\mathrm{Diag}(\nabla h)\right)}^2.$$

(33)

(ii) 

$D_{\Phi }\left(b\right|u)$ is separately convex if and only if

$$h\left(u\right)+\nabla h\left(u\right)(u-b)\ge \mathbf{0},\forall b,u\in int(\Omega ).$$

(34)

(iii) 

$D_{\Phi }\left(b\right|u)$ is conditionally convex if and only if

$$h\left(u\right)+\nabla h\left(u\right)(u-b)\ge \mathbf{0},\forall b,u\in B,$$

(35)

where B is an open convex set in $int(\Omega )$ and depends on b [12,42].

Proof. 

The proof of the first two convexities are given in Theorem 3.3 in [28]. For the conditional convexity, let us take second derivatives of $D_{\Phi }\left(b\right|u)$ with respect to u. Then, we get

$$h\left(u\right)+\nabla h\left(u\right)(u-b)\ge \mathbf{0}.$$

For each $b\in B_b\subset int(dom\Phi )$, we can find the domain of $u\in B_u\subset int(dom\Phi )$ satisfying the above condition. Let B be a convex and open set satisfying $B\subseteq B_b\cap B_u$. Then, we have the conditional convexity condition in (35). ☐

The following Theorem shows an interesting result that $D_{\Phi }\left(b\right|u)$ associated with $\Phi$ (19) is convex on its whole domain with respect to u in a very limited region $\beta \in [1,2].$ From a statistical point of view, this region is a little bit curious. In fact, if $\beta \in (1,2)$ then the Bregman-divergence $D_{\Phi }\left(b\right|u)$ does not have the corresponding statistical Tweedie distribution [8].

Theorem 10.

Let Φ be a convex and smooth function in (19) (not necessarily Legendre). Then, $D_{\Phi }\left(b\right|u)$ is separately convex (and also jointly convex) with the following domain conditions:

$$int(\Omega )=\left\{\begin{array}{c}{\mathbb{R}}_{++}^n,\mathit{if}\beta \in [1,2),\hfill \\ {\mathbb{R}}_{--}^n,\mathit{if}\beta \in [1,2)\cap {\mathbb{R}}_e,\hfill \\ {\mathbb{R}}^n,\mathit{if}\beta =2.\hfill \end{array}\right.$$

(36)

Due to the physical constraints of the observed data b, if we further restrict the domain of $b,$ then we have conditional convexity of $D_{\Phi }\left(b\right|u)$. Let $\alpha =(\beta -2)/(\beta -1),$ if $\beta \ne 1$ and $B_{m/M}^{\pm }$ be a constant vector in ${\mathbb{R}}^n$ representing $B_{m/M}^{\pm }\mathbf{1}$. Then, we have the following:

• Case I: Let us assume that the given data b be positive and have the following limitation in measurement

  $$b\in int\left({\Omega }_b^{+}\right)=\{b\in {\mathbb{R}}_{++}^n|b\in (B_m^{+},B_M^{+})\}.$$

  (37)
  Then, $D_{\Phi }\left(b\right|u)$ is conditionally convex on B, which is given below:

  -

  $\beta \in (2,+\infty ):$$B=\{u\in {\mathbb{R}}_{++}^n|u\in (max(B_m^{+},\alpha B_M^{+}),B_M^{+}),\}$ where $\alpha \in (0,1).$

  -

  $\beta \in (-\infty ,1):$ $B=\{u\in {\mathbb{R}}_{++}^n|u\in (B_m^{+},min(\alpha B_m^{+},B_M^{+})),\}$ where $\alpha \in (1,+\infty ).$

• Case II: Let us assume that the given data b is negative and has the following limitation in measurement

  $$b\in int\left({\Omega }_b^{-}\right)=\{b\in {\mathbb{R}}_{--}^n|b\in (B_m^{-},B_M^{-})\}.$$

  (38)
  Then, $D_{\Phi }\left(b\right|u)$ is conditionally convex on B, which is given below:

  -

  $\beta \in (2,+\infty )\cap {\mathbb{R}}_e:$$B=\{u\in {\mathbb{R}}_{--}^n|u\in (B_m^{-},min(\alpha B_m^{-},B_M^{-})),\}$ where $\alpha \in (0,1).$

  -

  $\beta \in (-\infty ,1)\cap {\mathbb{R}}_e\setminus \left\{0\right\}:$ $B=\{u\in {\mathbb{R}}_{--}^n|u\in (max(B_m^{-},\alpha B_M^{-}),B_M^{-}),\}$ where $\alpha \in (1,2)\cup (2,+\infty ).$

Proof. 

Since $\Phi$ in (19) is sufficiently smooth, we use (34) with $h\left(u\right)=u^{\beta -2}$. To find separately convex region, we need to find $\beta$ satisfying

$$h\left(u\right)+\nabla h\left(u\right)(u-b)=u^{\beta -3}[(\beta -1)u-(\beta -2)b]\ge \mathbf{0},\forall b,u\in int(\Omega ),$$

(39)

where we need to decide the corresponding domain $\Omega$ based on Table 1. In the case of conditional convexity, let us assume that the domain of b is limited as (37) or (38). In the following, we summarize the separate and conditional convexity of $D_{\Phi }$.

• $int(\Omega )={\mathbb{R}}^n$ with $\beta >1$ and $\beta \in {\mathbb{R}}_e$:
  We simplify (39) as

  $$u\ge 0\mathrm{and}u\ge \alpha boru\le 0\mathrm{and}u\le \alpha b.$$
  Then, we have

  -

  $\beta =2:$ $\alpha =0$ and thus u does not depend on b in (39). In fact, for any $b\in int(\Omega )$, we can select arbitrary $u\in int(\Omega )$.

  -

  $\beta \ne 2:$ In this case, the domain of u depends on b. For instance, if $\beta =3$ and $b>0$, then the domain of u bounded below, i.e., $u\ge 0.5b>0$. Therefore, the domain of u cannot be the whole region $int(\Omega )={\mathbb{R}}^n.$ This restriction is related to conditional convexity of $D_{\Phi }$ and is given in the following cases; $int(\Omega )={\mathbb{R}}_{++}^n(\beta \in \mathbb{R})$ and ${\mathbb{R}}_{--}^n(\beta \in {\mathbb{R}}_e\setminus \left\{0\right\}).$

• $int(\Omega )={\mathbb{R}}_{++}^n$ and $\beta \in \mathbb{R}$:
  We simplify (39) as

  $$(\beta -1)u\ge (\beta -2)b.$$

  (40)

  -

  $\beta >1$: (40) is simplified as $u\ge \alpha b$ and we get

  ∗

  $\beta \in (1,2)$: Since $\alpha \in (-\infty ,0)$, $u\ge \alpha b$ is satisfied for all $u,b\in int(\Omega )={\mathbb{R}}_{++}^n$.

  ∗

  $\beta >2:$ Let us assume that the given data b is bounded, i.e., (37) is satisfied. From $u\ge \alpha b$ and $\alpha \in (0,1)$, we have to satisfy the condition $u\ge \alpha B_M^{+}$ and thus the restricted domain B corresponding to (37) is given as

  $$B=\{u\in {\mathbb{R}}_{++}^n|u\in (max(B_m^{+},\alpha B_M^{+}),B_M^{+})\}.$$

  In fact, for all $b\in B$, $D_{\Phi }\left(b\right|u)$ is convex in terms of $u\in B$.

  -

  $\beta =1:$ (40) is satisfied for all $u,b\in int(\Omega )={\mathbb{R}}_{++}^n.$

  -

  $\beta <1$: Let us assume that b satisfies (37). Then, from $u\le \alpha b$ with $\alpha \in (1,+\infty )$, we get $u\le \alpha B_m^{+}$ and thus the restricted domain B corresponding to (37) is given as

  $$B=\{u\in {\mathbb{R}}_{++}^n|u\in (B_m^{+},min(\alpha B_m^{+},B_M^{+}))\}.$$

• $int(\Omega )={\mathbb{R}}_{--}^n$ and $\beta \in {\mathbb{R}}_e\setminus \left\{0\right\}$:
  We simplify (39) as

  $$(\beta -1)u\le (\beta -2)b.$$

  (41)

  -

  $\beta >1:$ (41) is simplified as $u\le \alpha b$ and we get

  ∗

  $\beta \in (1,2):$ $\alpha \in (-\infty ,0)$ and $u\le \alpha b$ is satisfied for all $u,b\in int(\Omega )={\mathbb{R}}_{--}^n$

  ∗

  $\beta >2:$ Let us assume that b satisfies (38). Then, from $u\le \alpha b$ with $\alpha \in (0,1)$, we get $u\le \alpha B_m^{-}$ and thus the restricted domain B corresponding to (38) is given as

  $$B=\{u\in {\mathbb{R}}_{--}^n|u\in (B_m^{-},min(\alpha B_m^{-},B_M^{-}))\}.$$

  -

  $\beta <1$: Let us assume that b satisfy (38). Then, from $u\ge \alpha b$ with $\alpha \in (1,2)\cup (2,+\infty )$, we get $u\ge \alpha B_M^{-}$ and thus the restricted domain B corresponding to (38) is given as

  $$B=\{u\in {\mathbb{R}}_{--}^n|u\in (max(B_m^{-},\alpha B_M^{-}),B_M^{-})\}.$$

In addition, we note that it is not difficult to see that $D_{\Phi }\left(b\right|u)$ is jointly convex on its domain $int(\Omega )\times int(\Omega ),$ where $int(\Omega )$ is given in (36). From the joint convexity condition in (33), we get the following condition

$$(\beta -2)(\beta -1)u^{2(\beta -3)}\le 0.$$

Since the exponent of u is even, i.e., $u^{2(\beta -3)}\ge 0$ for all $u\in int(\Omega )$ (36), if $\beta \in [1,2],$ then $(\beta -2)(\beta -1)\le 0$ and thus $D_{\Phi }\left(b\right|u)$ is jointly convex under the domain constraints in (36). ☐

Based on joint (and separate) convexity of $D_{\Phi }$ associated with $\Phi$ (19) on the domain (36), we can define the right Bregman proximity operator ${\overrightarrow{P}}_{\lambda TV}:int(\Omega )\to int(\Omega )$ associated with $\Phi$ (19) as follows:

$${\overrightarrow{P}}_{\lambda TV}\left(b\right)=\underset{u\in int(\Omega )}{\arg \min }{D}_{\Phi }\left(b\right|u)+\lambda TV\left(u\right),$$

(42)

where $D_{\Phi }\left(b\right|u)$ associated with $\Phi$ (19) is strictly convex, smooth, and coercive with respect to u and total variation is also convex and a coercive function [40]. We note that $D_{\Phi }\left(b\right|u)$ with the domain $int(\Omega )$(36) is coercive with respect to u, due to the joint convexity condition in (32) and $\frac{h^2\left(u\right)}{h\left(b\right)}>0$. For more details on the right Bregman proximal operator, see [10,24]. We should be cautious that, although the right Bregman proximal operator ${\overrightarrow{P}}_{\lambda TV}\left(b\right)$ (42) is well defined for the given data $b\in int(\Omega )$, its usefulness in real applications is limited, due to the separately convex condition $\beta \in [1,2]$. Actually, in the case of $\beta =2$, it is just an ordinary proximal operator [9]. Note that, in real applications such as SAR [12], $D_{\Phi }\left(b\right|u)$ with $\beta \in [0,1)$ and the positive domain constraints (37) (i.e., $D_{\Phi }\left(b\right|u)$ is conditionally convex) is used. In this case, it corresponds to not convex but coercive right Bregman proximity operator. See Theorem 4 (3) for the coercivity of the operator on its domain $int(\Omega )$.

Now, let us consider $\Phi$ (19) with the domain condition in Table 1. Then, $D_{\Phi }\left(b\right|u)$ is coercive and strictly convex in terms of b (see Theorem 4). Hence, we can also define the left Bregman proximity operator ${\overleftarrow{P}}_{\lambda TV}:int(\Omega )\to int(\Omega )$ as follows:

$${\overleftarrow{P}}_{\lambda TV}\left(u\right)=\underset{b\in int(\Omega )}{\arg \min }{D}_{\Phi }\left(b\right|u)+\lambda TV\left(b\right).$$

(43)

Unlike the right Bregman proximity operator, the left Bregman proximity operator (43) associated with the base function $\Phi$ in (19) is strictly convex and coercive for all $\beta \in \mathbb{R}$ on its domain $int(\Omega )$, where $\Omega =dom\Phi$. We note that the left Bregman proximity operator can be characterized in a more simple way as

$${\overleftarrow{P}}_{\lambda TV}={(\nabla \Phi +\lambda \partial TV)}^{-1}\circ \nabla \Phi ,$$

where $\partial TV$ is a subgradient of $TV.$ Actually, ${\overleftarrow{P}}_{\lambda TV}$ is a maximal monotone operator. See [30,43] for more details on Bregman proximity operator and the corresponding Bregman–Moreau envelopes.

Remark 2.

We could use β-divergence to define proximity operators. For instance, the right β-divergence proximity operator can be defined as

$${\overrightarrow{P}}_{\lambda TV}^{\beta }\left(b\right)=\underset{u\in {\Omega }_R}{\arg \min }{D}_{\beta }\left(b\right|u)+\lambda TV\left(u\right).$$

(44)

Instead of TV in (44), if we use an indicator function

$${\iota }_S\left(u\right)=\left\{\begin{array}{c}0,\mathit{if}u\in S,\hfill \\ +\infty ,\mathit{if}u\notin S,\hfill \end{array}\right.$$

for a convex set S, then we get the right β-divergence projection operator for S as follows:

$${\overrightarrow{P}}_{{\iota }_S}^{\beta }\left(b\right)=\underset{u\in {\Omega }_R}{\arg \min }{D}_{\beta }\left(b\right|u)+{\iota }_S\left(u\right).$$

(45)

It is interesting that the robustness of β-divergence [7] can be explained through the right β-divergence projection operator (45). Let us assume that $b,u$ in (45) are probability distributions (i.e., ${\sum }_i{b}_i=1$ and ${\sum }_i{u}_i=1$) and S is a set of Gaussian distributions. Here, the notation is slightly abused, since the Gaussian distribution is a continuous probability distribution and it is not a convex set. We note that the (generalized) Kullback–Leibler (KL) divergence (i.e., $D_{\beta }\left(b\right|u)$ with $\beta =1$), which is a commonly used similarity measure between two probability distributions, is undefined at zero probability ($u=0$). See Figure 1a and Table 1. However, outliers (i.e., rare events) have extremely low probability and thus they exist near zero probability. In this case, KL-divergence amplify the value near zero, i.e., ${lim}_{u\to 0}{D}_{\beta =1}\left(b\right|u)=+\infty$. However, when $\beta >1$, as noticed in Figure 1a and Table 1, ${lim}_{u\to 0}{D}_{\beta >1}\left(b\right|u)<+\infty$. Thus, outliers which exist near zero are not weighted too much. Hence, the right β-divergence projection operator (45) with $\beta >1$ is more robust to outliers than the KL-divergence-based operator. For more details, see [4,5,7]. Note that we can also define the left β-divergence proximity operator as

$${\overleftarrow{P}}_{\lambda TV}^{\beta }\left(u\right)=\underset{b\in {\Omega }_L}{\arg \min }{D}_{\beta }\left(b\right|u)+\lambda TV\left(b\right).$$

(46)

4.2. Dual Bregman-Divergence-Based Left Bregman Operator for a Convex Reformulation of the Bregman-TV with $\beta <1$

In this section, we introduce a convex reformulation of the nonconvex Bregman-TV (27) ($\beta <1$ and $L=I$) associated with $\Phi$ (19), which is the convex function of Legendre type and its domain is given in Table 1. Note that the problems we study in this section are related to the speckle reduction problem [12,15].

Due to the Theorem 4, we have the following reformulated Bregman-TV:

$$\underset{u\in B}{\min }{D}_{{\Phi }^{\ast }}(\nabla \Phi \left(u\right)|\nabla \Phi \left(b\right))+\lambda TV\left(u\right),$$

(47)

where $B=\{x\in {\mathbb{R}}_{++}^n|x\in (B_m^{+},B_M^{+})\}$ with $B_m^{+}\ge ϵ\mathbf{1}$ for some $ϵ>0.$ See [12,42] for real SAR data processing applications where the box constraint B is a critical element of the performance. Now, let $w=\nabla \Phi \left(u\right)$ and the corresponding domain $\nabla \Phi \left(B\right)$, then we have the left Bregman proximity operator associated with the dual Bregman-divergence:

$${\overleftarrow{P}}_{\lambda TV\circ \nabla {\Phi }^{\ast }}(\nabla \Phi \left(b\right))=\underset{w\in \nabla \Phi \left(B\right)}{\arg \min }F\left(w\right)=D_{{\Phi }^{\ast }}\left(w\right|\nabla \Phi \left(b\right))+\lambda TV(\nabla {\Phi }^{\ast }\left(w\right)),$$

(48)

where $\nabla \Phi \left(B\right)\subseteq int\left(dom{\Phi }^{\ast }\right)$ and we use $\nabla \Phi \circ \nabla {\Phi }^{\ast }\left(w\right)=w$. Since $\beta <1$, $\nabla \Phi =\frac{1}{\beta -1}{x}^{\beta -1}$ is strictly increasing on its domain $int(dom\Phi )={\mathbb{R}}_{++}^n$. Therefore, we have the transformed domain defined on ${\mathbb{R}}_{--}^n$ as

$$\nabla \Phi \left(B\right)=\{x\in {\mathbb{R}}_{--}^n|x\in (\nabla \Phi \left(B_m^{+}\right),\nabla \Phi \left(B_M^{+}\right))\},$$

(49)

where $-\infty <\nabla \Phi \left(ϵ\right)\le \nabla \Phi \left(B_m^{+}\right)<\nabla \Phi \left(B_M^{+}\right)<0$ and it is also a convex set. Moreover, $\nabla {\Phi }^{\ast }$ is also a strictly convex function by the following Lemma.

Lemma 2.

Let us assume that $\beta <1$ and $int(dom\Phi )={\mathbb{R}}_{++}^n.$ Then, we have

$$\nabla {\Phi }^{\ast }\left(w\right)={\left[(\beta -1)w\right]}^{1/(\beta -1)},$$

where $w\in int\left(dom{\Phi }^{\ast }\right)={\mathbb{R}}_{--}^n$. Note that $\nabla {\Phi }^{\ast }$ is strictly increasing and strictly convex on its domain $int\left(dom{\Phi }^{\ast }\right).$ However, it is not coercive but bounded below. In fact, we have $\nabla {\Phi }^{\ast }\left(w\right)\ge 0,\forall w\in int\left(dom{\Phi }^{\ast }\right)$.

Proof. 

Let $f\left(w\right)=\nabla {\Phi }^{\ast }\left(w\right)$. Then, since $\beta <1$ and $w\in {\mathbb{R}}_{--}^n$, we have

$${\nabla }^{2}f\left(w\right)=(2-\beta ){\left[(\beta -1)w\right]}^{\frac{3-2\beta }{\beta -1}}>0.$$

Since ${\Phi }^{\ast }$ is Legendre, $\nabla {\Phi }^{\ast }$ is strictly increasing on its domain. We note that, although $\nabla {\Phi }^{\ast }$ is strictly convex and strictly increasing, it is not coercive but bounded below, i.e.,

$$\underset{w\to -\infty }{lim}\nabla {\Phi }^{\ast }\left(w\right)=0.$$

(50)

 ☐

Finally, by using strict convexity and strictly increasing property of $\nabla {\Phi }^{\ast }$, we have a unique solution of the minimization problem in (48).

Theorem 11.

Let Φ (19) be the convex function of Legendre type and its domain is given in Table 1. In addition, we assume that $\beta <1$ and $B=\{x\in {\mathbb{R}}_{++}^n|x\in (B_m^{+},B_M^{+})\}.$ Then, for the given data $b\in B$, the left Bregman proximity operator (48) associated with the dual Bregman-divergence is well-defined. That is, there is a unique solution $w^{\ast }={\arg \min }_{w\in \nabla \Phi \left(B\right)}F\left(w\right).$

Proof. 

Since $\Phi$ (19) is Legendre, ${\Phi }^{\ast }$ is also Legendre on the domain ${\mathbb{R}}_{--}^n(=int\left(dom{\Phi }^{\ast }\right))$. Therefore, $D_{{\Phi }^{\ast }}\left(w\right|x)$ is strictly convex and coercive in terms of w by Theorem 4 (2). In addition, $TV$ is a composition of ${\parallel ·\parallel }_1\circ \mathcal{D}$, where $\mathcal{D}$ is a linear matrix (i.e, first order difference matrix) [44] and ${\parallel a\parallel }_1={\sum }_i\left|a_i\right|.$ Hence, we have

$$TV(\nabla {\Phi }^{\ast }\left(w\right))={\parallel ·\parallel }_1\circ \mathcal{D}\circ \nabla {\Phi }^{\ast }\left(w\right),$$

where $\nabla {\Phi }^{\ast }\left(w\right)$ is strictly convex (Lemma 2) and $\mathcal{D}$ is a linear operator and ${\parallel ·\parallel }_1$ is a simple metric (convex and increasing). Therefore, $TV(\nabla {\Phi }^{\ast }\left(w\right))$ is also a convex function (see Section IV.2.1 in [38]). In addition, since $\nabla {\Phi }^{\ast }\left(w\right)$ is lower bounded (Lemma 2), $TV(\nabla {\Phi }^{\ast }\left(w\right))$ is also lower bound. Then, the objective function $F\left(w\right)$ in (48) is coercive (see Lemma 2.12 in [10]) and strictly convex. In the end, the left Bregman proximity operator associated with dual Bregman-divergence has an unique solution (see Proposition 3.5 in [10]) as

$$w^{\ast }={\overleftarrow{P}}_{TV\circ \nabla {\Phi }^{\ast }}(\nabla \Phi \left(b\right)),$$

where $b\in B$ and $w^{\ast }={\arg \min }_{w\in \nabla \Phi \left(B\right)}F\left(w\right)$ in (48). Regarding the domain $\nabla \Phi \left(B\right)$, since ${\Phi }^{\ast }$ is Legendre, $|\nabla {\Phi }^{\ast }\left(x\right)|\to \infty$ as $\left|x\right|\to 0.$ Therefore, we need to keep a distance from $\left\{\mathbf{0}\right\}$ to assure $|\nabla {\Phi }^{\ast }\left(x\right)|<+\infty$. In fact, as noticed in (49), the transformed domain $\nabla \Phi \left(B\right)$ is a convex set and away from $\left\{\mathbf{0}\right\}$. ☐

The above Theorem is quite surprising. We can get a unique solution of the nonconvex Bregman-TV (27) ($\beta <1$ and $L=I$) through the left Bregman proximity operator (48) with an additional isomorphic transformation mapping $\nabla {\Phi }^{\ast }\circ \nabla \Phi =I$ as

$$u^{\ast }\in \nabla {\Phi }^{\ast }\circ {\overleftarrow{P}}_{(\lambda TV\circ \nabla {\Phi }^{\ast })}\circ \nabla \Phi \left(b\right).$$

(51)

However, in general, due to the severe nonlinearity of $\nabla {\Phi }^{\ast }$ within the non-smooth regularizer, i.e., $TV(\nabla {\Phi }^{\ast }\left(w\right))$, it is not easy to design a stable numerical algorithm to find a solution $u^{\ast }$ in (51). To overcome this drawback, we can directly modify (47) with a constraint $w=\nabla \Phi \left(u\right)$ as

$$\underset{w,u}{\min }{D}_{{\Phi }^{\ast }}\left(w\right|\nabla \Phi \left(b\right))+\lambda TV\left(u\right),$$

(52)

subject to the following constraints:

$$\left\{\begin{array}{c}u\in B,\hfill \\ w\in \nabla \Phi \left(B\right),\hfill \\ w=\nabla \Phi \left(u\right)(\mathrm{or}u=\nabla {\Phi }^{\ast }\left(w\right)).\hfill \end{array}\right.$$

(53)

Since $w=\nabla \Phi \left(u\right)$ is a nonlinear constraint and thus we cannot directly apply highly sophisticated augmented Lagrangian-based optimization algorithm. As a heuristic, to remedy these nonlinear constraints, we may consider the following penalty method [45]:

$$(w^{(k+1)},u^{(k+1)})=\underset{w\in \nabla \Phi \left(B\right),u\in B}{\arg \min }{D}_{{\Phi }^{\ast }}\left(w\right|\nabla \Phi \left(b\right))+\frac{\rho }{2}\parallel w-\nabla \Phi \left(u^{\left(k\right)}\right){\parallel }^2+\frac{\tau }{2}{\parallel u-\nabla {\Phi }^{\ast }(w^{\left(k\right)})\parallel }^2+\lambda TV\left(u\right).$$

(54)

This model is convex in terms of w and u, respectively. However, it is not convex with respect to $(u,w)$. In case of speckle reduction problems (30), nonlinearity of $\nabla \Phi$ could be reduced by using a shifting technique in [42].

In the following example, we show how (51) can be applied to relax nonconvex speckle reduction problems (30) with $L=I$.

Example 1.

Let us consider the following nonconvex minimization problems. For a given $b\in B$,

$$\underset{u\in B}{\min }{D}_{\Phi }\left(b\right|u)+\lambda TV\left(u\right),$$

(55)

where $B=\{x\in {\mathbb{R}}_{++}^n|x\in (B_m^{+},B_M^{+})\}$ is an open convex set with $B_m^{+}\ge ϵ\mathbf{1}$ for some $ϵ>0.$ Note that $\Phi \left(u\right)=\langle -lnu,\mathbf{1}\rangle$ is the convex function of Legendre type ($\beta =0$) and

$$D_{\Phi }\left(b\right|u)=\langle \left(\frac{b}{u}\right)-ln\left(\frac{b}{u}\right)-\mathbf{1},\mathbf{1}\rangle$$

is the Bregman-divergence associated with Φ (Burg entropy) function. This model is known as AA-model [15]. It is well known that it is not easy to find a global minimizer of (55), due to the severe nonconvexity of $D_{\Phi }\left(b\right|u)$ in terms of u [15,21]. Therefore, various transform-based convex relaxation approaches are introduced [12,16,17,18,19,20,21,42]. In this example, we are going to use dual Bregman-divergence to find a solution of (55). We note that $\Phi \left(u\right)=\langle -lnu,\mathbf{1}\rangle$ is the convex function of Legendre type on its domain $int(dom\Phi )={\mathbb{R}}_{++}^n$. Hence, by Theorem 7, we get the following corresponding conjugate function:

$${\Phi }^{\ast }\left(w\right)=\langle -\mathbf{1}-ln(-w),\mathbf{1}\rangle .$$

This function is also the convex function of Legendre type on its domain $int\left(dom{\Phi }^{\ast }\right)={\mathbb{R}}_{--}^n.$ Now, by using the dual Bregman-divergence, we get a convex reformulated version of (55) as

$$\left\{\begin{array}{c}{w}^{\ast }=\underset{w\in B_w}{\arg \min }F\left(w\right),\hfill \\ u^{\ast }=-\frac{\mathbf{1}}{w^{\ast }},\hfill \end{array}\right.$$

(56)

where

$$F\left(w\right)=-\langle ln(-w),\mathbf{1}\rangle -\langle w,b\rangle +TV\left(-\frac{\mathbf{1}}{w}\right)$$

and

$$B_w=\left\{x\in {\mathbb{R}}_{--}^n|x\in \left(-\frac{\mathbf{1}}{B_m^{+}},-\frac{\mathbf{1}}{B_M^{+}}\right)\right\}.$$

We note that $B_w=\nabla \Phi \left(B\right)$ is a convex set and $TV(-\frac{\mathbf{1}}{w})$ is also convex for all $w\in {\mathbb{R}}_{--}^n.$ Therefore, the objective function $F\left(w\right)$ in (56) is strictly convex on its domain $B_w$. In addition, due to the Theorem 4 (2), $F\left(w\right)$ is coercive in the domain ${\mathbb{R}}_{--}^n$. Therefore, we have a unique solution $u^{\ast }$ of (55). A similar inverse transformation on the positive domain ${\mathbb{R}}_{++}^n$ itself is introduced in [45,46].

5. Conclusions

In this article, we introduced the extended logarithmic function and, based on that, we could redefine the domain of the $\beta$-divergence. In fact, we have found that if $\beta$ is in the class ${\mathbb{R}}_e=\{x\in \mathbb{R}|x=2k/(2l+1),k,l\in \mathbb{Z}\}$, then the negative region ${\mathbb{R}}_{--}^n$ should be included into the domain of the $\beta$-divergence. In addition, if we use the integral of the extended logarithmic function as a base function of the Bregman-divergence, then we have a partial equivalence between the $\beta$-divergence and the Bregman-divergence associated with the Legendre base function. Last but not least, by using dual formulation of the Bregman-divergence associated with convex function of Legendre type and the negative domain of it, we have shown that we could make a convex reformulated model of the nonconvex variational model that appears in the SAR speckle reduction problem. The approaches in this article could be extended to other divergences, such as $\alpha$- and $\gamma$-divergences [2]. In addition, we could plug the presented model into various segmentation problems [11,47,48].