λ-Deformation: A Canonical Framework for Statistical Manifolds of Constant Curvature

Abstract

This paper systematically presents the $\lambda$-deformation as the canonical framework of deformation to the dually flat (Hessian) geometry, which has been well established in information geometry. We show that, based on deforming the Legendre duality, all objects in the Hessian case have their correspondence in the $\lambda$-deformed case: $\lambda$-convexity, $\lambda$-conjugation, $\lambda$-biorthogonality, $\lambda$-logarithmic divergence, $\lambda$-exponential and $\lambda$-mixture families, etc. In particular, $\lambda$-deformation unifies Tsallis and Rényi deformations by relating them to two manifestations of an identical $\lambda$-exponential family, under subtractive or divisive probability normalization, respectively. Unlike the different Hessian geometries of the exponential and mixture families, the $\lambda$-exponential family, in turn, coincides with the $\lambda$-mixture family after a change of random variables. The resulting statistical manifolds, while still carrying a dualistic structure, replace the Hessian metric and a pair of dually flat conjugate affine connections with a conformal Hessian metric and a pair of projectively flat connections carrying constant (nonzero) curvature. Thus, $\lambda$-deformation is a canonical framework in generalizing the well-known dually flat Hessian structure of information geometry.

1. Introduction

Information geometry is a differential-geometric framework for studying finite-dimensional statistical models that coherently integrates the following notions:

(i)

A differentiable manifold $\mathcal{M}$ consisting of probability density functions or finite measures on a common sample space;

(ii)

A divergence function $D\left[p\right|\left|p^{\prime }\right]$ that defines an asymmetric proximity between points p, $p^{\prime }$ in $\mathcal{M}$;

(iii)

A Riemannian metric $\mathtt{g}$ plus a pair of torsion-free dual (conjugate) affine connections $\nabla ,{\nabla }^{*}$ on $\mathcal{M}$.

For completeness, we recall that a pair of affine connections ∇, ${\nabla }^{*}$ on $\mathcal{M}$ are said to be dual (or conjugate) with respect to a Riemannian metric $\mathtt{g}$ if for any vector fields X, Y, and Z on $\mathcal{M}$, one has:

$$Z\mathtt{g}(X,Y)=\mathtt{g}({\nabla }_{Z}X,Y)+\mathtt{g}(X,{\nabla }_Z^{*}Y).$$

(1)

Here, $(\mathcal{M},\mathtt{g},\nabla ,{\nabla }^{*})$ is called a dualistic structure. When D is the Kullback–Leibler divergence (or more generally, f-divergence), the induced Riemannian metric $\mathtt{g}$ is the Fisher–Rao metric, and the induced cubic form $C={\nabla }^{*}-\nabla$ is the Amari–Chentsov tensor [1]. It can be shown that the Fisher–Rao metric and the Amari–Chentsov tensor are unique invariants, of respectively second and third orders, under sufficient statistics on the manifold $\mathcal{M}$ [2].

Geometrically, the standard model (denoted the $\mathcal{S}$-model in this paper) uses a pair of affine connections that are torsion-free, though in general, they are not curvature-free. An alternative, “partially flat” model (denoted the $\mathcal{P}$-model in this paper) was recently investigated in [3], leading to the notion of “statistical mirror symmetry” [4]. Under the $\mathcal{P}$-model, the affine connections ∇ and ${\nabla }^{*}$ are allowed to carry torsion, but are both curvature-free. See [4] for the geometric properties of the $\mathcal{P}$-model leading to a symplectic-to-complex correspondence characteristic of mirror Calabi–Yau manifolds studied in string theory and mathematical physics.

Within the usual $\mathcal{S}$-model, a special case is the dually flat geometry where the Riemannian metric can be expressed under special coordinate systems as a Hessian metric. Two prominent examples are the exponential family and the mixture family, where the Hessian metric coincides with the Fisher–Rao metric. The Hessian geometry is said to be dually flat because the Riemann curvature tensors of both the primal and the dual connections vanish; the corresponding primal and dual affine coordinate systems are linked via Legendre transformations by a pair of convex potentials. For an exponential family, these coordinates are precisely the natural (canonical) and mixture (expectation) coordinate systems, respectively. Note that the Hessian metric itself is not flat as its Levi-Civita connection contains curvature in general.

Between the well-understood dually flat Hessian geometry and the full-blown $\mathcal{S}$-model, there is a wide range of geometries capturing various probability models. Of special interest are generalizations of the exponential family, namely deformed exponential families. The $\varphi$-exponential family was introduced in the context of statistical physics [5]; it was later shown [6] to be equivalent to the U-model [7] motivated by applications in machine learning—[6] revealed that both the $\varphi$- and U-models can be generated from the $(\rho ,\tau )$-model [8] through the mechanism of “gauge selection”. The $(\rho ,\tau )$-metric generalizes the Fisher–Rao metric and may lead to a conformal Hessian metric for a $\varphi$-exponential family. However, the connections are typically not curvature-free unless a special type of gauge is selected; this underlies the geometric characterization of the q-exponential model of Tsallis by [9,10,11].

In recent years, the second author [12], motivated by previous works with Pal on mathematical finance and optimal transport [13,14,15,16], studied a class of deformed exponential families generating constant curvatures through the use of a new divergence function called logarithmic divergence. By constant (information geometric) curvature, we mean that both the primal and dual Riemann curvature tensors have (the same) constant sectional curvature with respect to $\mathtt{g}$. In [17], the present authors developed a unified framework, based on the notions of λ-duality and the λ-exponential family, which appears to provide a canonical extension of the dually flat geometry to the constant curvature case. Previously, statistical manifolds with constant curvature were studied using the abstract tools of affine differential geometry; see, e.g., [1,18] (also see [19]). Our framework provides a concrete approach and an explicit construction that elucidates how the properties of the exponential family and the dually flat geometry may be extended to the constant curvature case. In this paper, a careful exposition of the $\lambda$-deformation framework is provided from the perspective of $\lambda$-duality, namely the $\lambda$-deformation of Legendre duality.

The rest of the paper is organized as follows. In Section 2, we review the standard $\mathcal{S}$-model of information geometry with a focus on the dually flat geometry, based on convex duality and Bregman divergence, of the exponential and mixture families. The section closes with a preview of $\lambda$-deformation by introducing a suite of four deformation functions, as two pairs of mutually inverse functions: ${log}_{\lambda }$ versus ${exp}_{\lambda }$ and ${\kappa }_{\lambda }$ versus ${\gamma }_{\lambda }$, with the first pair deforming log and exp and the second pair deforming the identity function. In Section 3, we describe the $\lambda$-duality, which deforms the standard convex duality. In particular, we compare $\lambda$-duality and standard Legendre duality and show their relations to each other upon a change of parameterization. In Section 4, we define the $\lambda$-gradient and then the $\lambda$-logarithmic divergence and study the constant curvature information geometry the latter induces. In Section 5, we relate $\lambda$-divergence to Rényi entropy by introducing the $\lambda$-exponential and $\lambda$-mixture family. The two expressions of the $\lambda$-exponential family under divisive and subtractive normalization correspond to, respectively, Rényi deformation and Tsallis deformation. Section 6 concludes with a comparison of $\lambda$-deformation with the standard dually flat (Hessian) framework.

2. The Standard Model of Information Geometry

2.1. The Standard Model

We begin by recalling the standard framework (referred to as the $\mathcal{S}$-model) of parametric information geometry [1,20]. Let $\mathcal{M}$ be a finite-dimensional differentiable manifold with dimension d and $\theta =({\theta }^1,\dots ,{\theta }^d)$ be a local coordinate system. The most important case is where $\mathcal{M}$ is a manifold of parametric probability density functions. However, the idea of deforming Legendre duality to $\lambda$-duality and hence dually flat (Hessian) manifolds to manifolds of constant curvature discussed in Section 3 and Section 4 is entirely general and does not rely on $\mathcal{M}$ being a manifold of probability density functions.

Let $(\mathcal{X},\mu )$ be a measure space, where $\mu$ is called the reference (or dominating) measure. Let $\Theta \subset {\mathbb{R}}^d$ be an open domain. A parametric family of density functions is a mapping $\theta \in \Theta \mapsto p(·|\theta )$, where each $p(·|\theta )$ is a probability density function with respect to $\mu$, i.e., ${\int }_{\mathcal{X}}p\left(\zeta \right|\theta )d\mu \left(\zeta \right)=1$. We assume that the family is sufficient regular such that all analytical operations (such as differentiation under the integral sign) can be performed as needed.

While a dualistic structure $(\mathcal{M},\mathtt{g},\nabla ,{\nabla }^{*})$ can be defined abstractly, in practice, it is often constructed by a divergence, namely a smooth, non-negative function $D[·||·]$ on $\mathcal{M}\times \mathcal{M}$ such that $D\left[p\right|\left|p^{\prime }\right]=0$ only if $p=p^{\prime }$ and the (0,2)-tensor $\mathtt{g}$ it induces on $\mathcal{M}$ (see (2) below) is positive definite. Intuitively, $D\left[p\right|\left|p^{\prime }\right]$ defines a notion of “asymmetric distance” between points p and $p^{\prime }$ of $\mathcal{M}$. When $\mathcal{M}$ is a manifold of density functions, a prominent example is the Kullback–Leibler (KL) divergence (relative entropy) given by:

$$H\left[p\right||p^{\prime }]=\int plog\frac{p}{p^{\prime }}d\mu .$$

When dealing with parametric probability families, p and $p^{\prime }$ are replaced by $p(·|\theta )$ and $p(·|{\theta }^{\prime })$, then we denote $D\left[p\right|\left|p^{\prime }\right]$ as $D(\theta ,{\theta }^{\prime })$ with an abuse of notation, that is:

$$D\left[p(·|\theta )\right||p(·|{\theta }^{\prime })]\equiv D(\theta ,{\theta }^{\prime }),$$

and similarly for H as well—the notation of $\left[p\right|\left|p^{\prime }\right]$ in the divergence for probability density functions emphasizes the non-symmetricity in p, $p^{\prime }$; see [1].

Eguchi [21] showed that any divergence function (called a “contrast function” there) induces a dualistic structure $(\mathcal{M},\mathtt{g},\nabla ,{\nabla }^{*})$. In local coordinates, given $D(\theta ,{\theta }^{\prime })$, the components $g_{ij}$ of the metric $\mathtt{g}$ are given by:

$$g_{ij}\left(\theta \right)=-{\left.\frac{{\partial }^2}{\partial {\theta }^i\partial {\theta }^{\prime j}}D(\theta ,{\theta }^{\prime })\right|}_{\theta ={\theta }^{\prime }}={\left.\frac{{\partial }^2}{\partial {\theta }^i\partial {\theta }^j}D(\theta ,{\theta }^{\prime })\right|}_{\theta ={\theta }^{\prime }},$$

(2)

and the Christoffel symbols of the conjugate connections ∇ and ${\nabla }^{*}$ are given respectively by:

$${\Gamma }_{ij,k}\left(\theta \right)=-{\left.\frac{{\partial }^3}{\partial {\theta }^i\partial {\theta }^j\partial {\theta }^{\prime k}}D(\theta ,{\theta }^{\prime })\right|}_{{\theta }^{\prime }=\theta },{\Gamma }_{ij,k}^{*}\left(\theta \right)=-{\left.\frac{{\partial }^3}{\partial {\theta }^{\prime i}\partial {\theta }^{\prime j}\partial {\theta }^k}D(\theta ,{\theta }^{\prime })\right|}_{\theta ={\theta }^{\prime }}.$$

(3)

Conversely, given any dualistic structure $(\mathcal{M},\mathtt{g},\nabla ,{\nabla }^{*})$, there exists a divergence D that induces it, but this D is not unique in general [22]. Thus, the standard model $\mathcal{S}$ is completely encoded by the choice of a divergence function.

2.2. Dually Flat Geometry

The most important example of a dualistic structure is the dually flat geometry, which is induced by a Bregman divergence [23]. Let $\mathcal{M}$ be prescribed with an affine coordinate chart $\theta \in \Theta$ on an open convex set $\Theta \subset {\mathbb{R}}^d$. Let $\varphi :\Theta \to \mathbb{R}$ be a differentiable convex function; specifically, we assumed that $\varphi$ is $C^2$ and its Hessian ${\mathsf{D}}^2\varphi$ is strictly positive definite. The Bregman divergence of $\varphi$ is defined by:

$$B_{\varphi }(\theta ,{\theta }^{\prime })=\varphi \left(\theta \right)-\varphi \left({\theta }^{\prime }\right)-\mathsf{D}\varphi \left({\theta }^{\prime }\right)·(\theta -{\theta }^{\prime }),\theta ,{\theta }^{\prime }\in \Theta ,$$

where $\mathsf{D}\varphi \left(\theta \right)={(\frac{\partial }{\partial {\theta }^1}\varphi \left(\theta \right),\dots ,\frac{\partial }{\partial {\theta }^d}\varphi \left(\theta \right))}^{\top }$ is the Euclidean gradient and $a·b$ denotes the standard dot product. We call $\theta \in \Theta$ the primal coordinates, and $\eta =\mathsf{D}\varphi \left(\theta \right)$ the dual coordinates, where the inverse of $\mathsf{D}\varphi$ is given by $\theta =\mathsf{D}{\varphi }^{*}\left(\eta \right)$. Here, the Legendre conjugate ${\varphi }^{*}$ (or convex conjugate) of $\varphi$ is defined by:

$${\varphi }^{*}\left(\eta \right)=\underset{\theta }{sup}\left(\theta ·\eta -\varphi \left(\theta \right)\right).$$

(4)

Then, the components $g_{ij}$ of the Riemannian metric $\mathtt{g}$, under the respective local coordinate system, are given by:

$$g_{ij}\left(\theta \right)=\frac{{\partial }^2}{\partial {\theta }^i\partial {\theta }^j}\varphi \left(\theta \right),g_{ij}\left(\eta \right)=\frac{{\partial }^2}{\partial {\eta }^i\partial {\eta }^j}{\varphi }^{*}\left(\eta \right).$$

(5)

In particular, $\mathtt{g}$ is a Hessian metric with potential $\varphi$ (resp. ${\varphi }^{*}$) under $\theta$ (resp. $\eta$). Furthermore, the Christoffel symbols of ∇ and ${\nabla }^{*}$ are given respectively by:

$${\Gamma }_{ij,k}\left(\theta \right)=0,{\Gamma }_{ij,k}^{*}\left(\eta \right)=0.$$

(6)

From (6), we see that the Riemann curvature tensors of both ∇ and ${\nabla }^{*}$ vanish. Thus, we call this a dually flat geometry. Furthermore, a ∇-geodesic (resp. ${\nabla }^{*}$-geodesic) is a constant velocity straight line under the $\theta$ (resp. $\eta$) coordinate system.

Moreover, the $\theta$ and $\eta$ coordinates are biorthogonal in the sense that:

$$\mathtt{g}\left(\frac{\partial }{\partial {\theta }^i},\frac{\partial }{\partial {\eta }^j}\right)={\delta }_{ij},$$

(7)

and the Bregman divergence takes the forms of:

$$B_{\varphi }(\theta ,{\theta }^{\prime }\left({\eta }^{\prime }\right))=A_{\varphi }(\theta ,{\eta }^{\prime })=A_{{\varphi }^{*}}({\eta }^{\prime },\theta )=B_{{\varphi }^{*}}({\eta }^{\prime },\eta \left(\theta \right))$$

with $\eta =\mathsf{D}\varphi \left(\theta \right)$ and ${\theta }^{\prime }=\mathsf{D}{\varphi }^{*}\left({\eta }^{\prime }\right)$, where A is called the canonical divergence:

$$A_{\varphi }(\theta ,{\eta }^{\prime })=\varphi \left(\theta \right)+{\varphi }^{*}\left({\eta }^{\prime }\right)-\theta ·{\eta }^{\prime }=A_{{\varphi }^{*}}({\eta }^{\prime },\theta ).$$

(8)

Following [24,25], we call the equality between two expressions of B and the equality between two expressions of A in (8) reference–representation biduality. In [26], the identity (8) was used to motivate a family of Fenchel–Young losses in the context of regularized prediction in machine learning. Last but not least, the Bregman divergence satisfies the generalized Pythagorean theorem: given points P, Q, and R, we have the equality:

$$B_{\varphi }({\theta }_Q,{\theta }_P)+B_{\varphi }({\theta }_R,{\theta }_Q)=B_{\varphi }({\theta }_R,{\theta }_P)$$

if and only if the ∇-geodesic between Q and R and the ${\nabla }^{*}$-geodesic between Q and P meet $\mathtt{g}$-orthogonally at Q. As we will see in Section 4, all the properties above have natural generalizations under our $\lambda$-framework. We stress that the dually flat geometry depends crucially on classical convex (or Legendre) duality, as seen from (4) and (8).

2.3. Exponential and Mixture Families

The dually flat Hessian geometry arises naturally in the exponential and mixture families of probability densities. Given a reference measure $\mu$ on a state space $\mathcal{X}$, an exponential family is a parameterized probability density $p^{\left(e\right)}(·|\theta )$ of the form:

$$p^{\left(e\right)}\left(\zeta \right|\theta )=e^{\theta ·F\left(\zeta \right)-\varphi \left(\theta \right)},$$

(9)

where $\theta =({\theta }^1,\dots ,{\theta }^d)\in \Theta \subseteq {\mathbb{R}}^d$ and $F\left(\zeta \right)=(F_1\left(\zeta \right),\dots ,F_d\left(\zeta \right))$ is a vector of sufficient statistics. In (9), the cumulant generating function $\varphi$, defined by:

$$\varphi \left(\theta \right)=log\int e^{\theta ·F}d\mu ,$$

enforces the normalization $\int p^{\left(e\right)}d\mu =1$. The exponential family generalizes the Boltzmann–Gibbs distribution in statistical physics, where $Z\left(\theta \right)=e^{\varphi \left(\theta \right)}$ is called the partition function.

The information geometry of the exponential family begins with the observation that $\varphi$ is convex. Then, $\varphi$ defines a Bregman divergence $B_{\varphi }$ giving rise to a dually flat structure. It can be shown that the Bregman divergence is a KL-divergence:

$$B_{\varphi }(\theta ,{\theta }^{\prime })=H[p^{\left(e\right)}(·|{\theta }^{\prime })\left|\right|p^{\left(e\right)}(·|\theta )].$$

The induced Riemannian metric $\mathtt{g}$, the Fisher–Rao metric given by (in matrix components $g_{ij}$), becomes a Hessian metric ${\mathsf{D}}^2\varphi$:

$$g_{ij}\left(\theta \right)=\int \left(\frac{\partial }{\partial {\theta }^i}log{p}^{\left(e\right)}\left(\zeta \right|\theta )\right)\left(\frac{\partial }{\partial {\theta }^j}log{p}^{\left(e\right)}\left(\zeta \right|\theta )\right)p^{\left(e\right)}\left(\zeta \right|\theta )d\mu =\frac{{\partial }^2}{\partial {\theta }^i\partial {\theta }^j}\varphi \left(\theta \right).$$

Equivalently, $\mathtt{g}\left(\theta \right)$ is the covariance matrix of the sufficient statistics F:

$$g_{ij}\left(\theta \right)=\int p^{\left(e\right)}\left(\zeta \right|\theta )\left(F_i\left(\zeta \right)-\int p^{\left(e\right)}\left(\zeta \right|\theta )F_i\left(\zeta \right)\right)\left(F_j\left(\zeta \right)-\int p^{\left(e\right)}\left(\zeta \right|\theta )F_j\left(\zeta \right)\right)d\mu .$$

Furthermore, the dual coordinate $\eta =\mathsf{D}\varphi \left(\theta \right)$ is the expectation coordinates given by:

$$\eta =\int p^{\left(e\right)}\left(\zeta \right|\theta )F\left(\zeta \right)d\mu ,$$

and the dual potential function ${\varphi }^{*}$ is, as a function of $\eta$, the negative Shannon entropy:

$${\varphi }^{*}\left(\eta \right)=-H\left[p^{\left(e\right)}(·|\theta )\right]=\int p^{\left(e\right)}\left(\zeta \right|\theta )log{p}^{\left(e\right)}\left(\zeta \right|\theta )d\mu .$$

A theoretical justification for the exponential family is that it maximizes the Shannon entropy under the constraints of its expected value of the vector of random functions $F(·)$.

The mixture family is another probability family that is very useful in both theory and applications. Let $P_0\left(\zeta \right),P_1\left(\zeta \right),\dots ,P_d\left(\zeta \right)$ be a set of affinely independent probability densities with respect to the same dominating measure $\mu$. Given mixture parameters ${\eta }_i>0$ for $i=0,\cdots ,d$ with ${\sum }_{i=0}^d{\eta }_i=1$, the mixture family $p^{\left(m\right)}(·|\eta )$ is defined by:

$$p^{\left(m\right)}\left(\zeta \right|\eta )=\sum _{i=0}^d{\eta }_i{P}_i\left(\zeta \right)=P_0\left(\zeta \right)+\sum _{i=1}^d{\eta }_i(P_i\left(\zeta \right)-P_0\left(\zeta \right)),$$

where $({\eta }_1,\dots ,{\eta }_d)$ may be taken as the independent parameters. It can be shown that the negative Shannon entropy:

$$\psi \left(\eta \right)=-H\left[p^{\left(m\right)}(·|\eta )\right]=\int p^{\left(m\right)}\left(\zeta \right|\eta )log{p}^{\left(m\right)}\left(\zeta \right|\eta )d\mu$$

of a mixture family is convex in $\eta$. Using $\psi$ as the potential function, we have:

$$B_{\psi }(\eta ,{\eta }^{\prime })=H[p^{\left(m\right)}(·|\eta )\left|\right|p^{\left(m\right)}(·||{\eta }^{\prime }\left)\right],$$

which is again a KL-divergence and induces a dually flat geometry. In summary, the exponential and mixture families are both dually flat when the geometry is induced by the KL-divergence.

For completeness, we note that the convex conjugate of $\psi \left(\eta \right)$ is:

$${\psi }^{*}\left(\theta \right)=-\int P_0\left(\zeta \right)log{p}^{\left(m\right)}\left(\zeta \right|\eta )d\mu ,$$

with conjugate parameters $\theta =\mathsf{D}\psi$ given by:

$${\theta }^i=\int (P_i\left(\zeta \right)-P_0\left(\zeta \right))log{p}^{\left(m\right)}\left(\zeta \right|\eta )d\mu .$$

2.4. Deforming exp and log

The exponential function used in the exponential family:

$$p^{\left(e\right)}\left(\zeta \right|\theta )=exp\{\theta ·F\left(\zeta \right)-\varphi \left(\theta \right)\}=\frac{e^{\theta ·F\left(\zeta \right)}}{Z\left(\theta \right)}$$

allows the cumulant generating function $\varphi \left(\theta \right)$ (also called the potential function) and the partition function $Z\left(\theta \right)$ to be linked by the simple relation $\varphi =logZ$. The equivalence of using $\varphi$ as subtractive normalization and Z as divisive normalization of the same exponential family $\int p^{\left(e\right)}\left(\zeta \right|\theta )d\mu =1$ is due to the elementary, but crucial property $exp(x+y)=exp\left(x\right)exp\left(y\right)$ of the exponential function. Using a functional form other than exp (exponential function) or log (logarithm function) is referred to as deformation in information geometric (statistical and information theoretic) contexts, and the resulting probability families are called “deformed” families. Typically, this is performed by regarding log, or equivalently exp, as special cases of some parametric class of functions that include them as special members.

More generally, the exponential/logarithmic function can be considered within a non-parametric function space that includes exp or log as a special member. Several approaches can be found in the literature, including the $\varphi$-deformed exponential approach by Naudts [5,27,28], the conjugate $(\rho ,\tau )$-embedding approach by the first author [8,25,29], and the U-model by Eguchi [7,30]. The $\varphi$-model and U-model are both one-function models, while the $(\rho ,\tau )$-model uses two free functions. It eventually became clear in the 2018 paper [6] by Naudts and the first author that (i) the $\varphi$- and U-model turned out to be equivalent; (ii) they are special cases of the $(\rho ,\tau )$-model upon a particular fixing of the “gauge freedom”; (iii) the corresponding $(\rho ,\tau )$-geometry of the manifold of the $\varphi$-exponential family can have different appearances (gauges freedom), such as a Hessian geometry (under one type of gauge selection) and a conformal Hessian geometry (under another type of gauge selection). The work [6] unified the intermediary results in [10,11,31] and provided a general deformation framework that preserves the rigid interlocking of: (i) the functional form of entropy, cross-entropy, and relative entropy (divergence); (ii) the functional form of the deformed probability family with the corresponding normalization and potential and the duality between the natural and expectation parameterizations; (iii) the expressions of the Riemannian metric (Fisher–Rao metric in general and Hessian metric in particular) and of the conjugate connections. Some of these concepts have their correspondence in nonparametric probability families as well [32,33,34].

Although the $(\rho ,\tau )$-model may admit a conformal Hessian metric (more rigorously stated: the $\varphi$-exponential family with the $(\rho ,\tau )$-metric under a certain gauge will lead to conformal Hessian geometry), the dual connections are not projectively flat (as the geometry studied by [12]). As a result, while the connections are not flat (torsion-free, but not curvature-free), they are not in general of the constant-curvature-type either. Therefore they are “too general” and do not generate the space of constant curvatures.

2.5. Highlights of $\lambda$-Deformation

Here enters $\lambda$-deformation as a middle ground [17]. The $\lambda$-deformation theory absorbs the q-deformation model of Tsallis and the $F^{(\pm \alpha )}$ model of Wong [12] in deforming the exponential family and unifies the subtractive and divisive normalization—this is an occasion where subtractive and divisive normalizations are still linked by a simple reparameterization of the probability family.

Let us introduce some notations. Consider the following deformed logarithm and exponential functions (note the slight difference to the ${log}_q$ notation used by Tsallis, in the way how the subscript indicates the deformation parameter):

$${log}_{\lambda }\left(t\right)=\frac{1}{\lambda }\left(t^{\lambda }-1\right),{exp}_{\lambda }\left(t\right)={(1+\lambda t)}^{1/\lambda }.$$

More precisely, we define ${exp}_{\lambda }:\mathbb{R}\to [0,\infty ]$ (where $\lambda \in \mathbb{R},\lambda \ne 0$) by:

$${exp}_{\lambda }\left(t\right)={\left[1+\lambda t\right]}_{+}^{1/\lambda },$$

where ${\left[a\right]}_{+}=max\{a,0\}$. In our analysis, we assumed implicitly that $1+\lambda t>0$, which is shown to hold for $\lambda$-duality, so the subscript + can be omitted. Furthermore, $\frac{d}{dt}{exp}_{\lambda }\left(t\right)={\left[{exp}_{\lambda }\left(t\right)\right]}^{1-\lambda }$, so ${exp}_{\lambda }(·)$ is convex if and only if $\lambda <1$. For this reason, we restricted $\lambda$ to this range as in [9,28]. Below, we also took $logt=-\infty$ whenever $t\le 0$. Note that our notation differs slightly from Tsallis’ indexing of the deformed logarithm and exponential functions; see Section 5.

Next, we construct another pair of inverse functions ${\kappa }_{\lambda },{\gamma }_{\lambda }$ by:

$${\kappa }_{\lambda }=log\circ {exp}_{\lambda },{\gamma }_{\lambda }={log}_{\lambda }\circ exp,$$

where ∘ denotes function composition. Explicitly, they are:

$${\kappa }_{\lambda }\left(t\right)=\frac{1}{\lambda }log(1+\lambda t),{\gamma }_{\lambda }\left(t\right)=\frac{1}{\lambda }\left(e^{\lambda t}-1\right).$$

(10)

This suite of four functions, namely ${exp}_{\lambda },{log}_{\lambda }$ as an inverse pair and ${\kappa }_{\lambda },{\gamma }_{\lambda }$ as another inverse pair, is called λ-deformation and used in the discussions of $\lambda$-convexity, $\lambda$-conjugation, and $\lambda$-duality. Regular exponential and logarithmic functions are recovered when $\lambda \to 0$, whence both ${\kappa }_{\lambda }$ and ${\gamma }_{\lambda }$ reduce to the identity function.

Using these four functions, Wong and Zhang [17] developed the $\lambda$-deformation framework to solve the problem of relating the exponential family under subtractive normalization:

$$p^{\left(\lambda \right)}\left(\zeta \right|\theta )={exp}_{\lambda }(\theta ·F\left(\zeta \right)-{\varphi }_{\lambda }\left(\theta \right))$$

to that under divisive normalization:

$$p^{\left(\lambda \right)}\left(\zeta \right|\vartheta )={exp}_{\lambda }(\vartheta ·F\left(\zeta \right))e^{-{\phi }_{\lambda }\left(\vartheta \right)}.$$

There, the same $\lambda$-deformed exponential family can be expressed by two parameterizations $\theta$ and $\vartheta$ linked through:

$$\theta =\vartheta e^{-\lambda {\phi }_{\lambda }\left(\vartheta \right)}⟺\vartheta =\frac{\theta }{1-\lambda {\varphi }_{\lambda }\left(\theta \right)},$$

while the normalization functions ${\varphi }_{\lambda }$ and ${\phi }_{\lambda }$ (with different domains) are linked through:

$${\varphi }_{\lambda }\left(\theta \right)={\gamma }_{-\lambda }\left({\phi }_{\lambda }\left(\vartheta \right)\right)⟺{\phi }_{\lambda }\left(\vartheta \right)={\kappa }_{-\lambda }\left({\varphi }_{\lambda }\left(\theta \right)\right).$$

The $\lambda$-deformation framework led to a unified way of looking at the Tsallis entropy (related to the subtractive denormalization) and Rényi entropy (related to the divisive normalization), as well as generating new insights into the distinction between the exponential and mixture families through the lens of deformation theory. To understand this deformation better, we describe the underlying mathematical framework of $\lambda$-deformation.

3. Deforming the Legendre Duality: $\mathbf{\lambda }$-Duality

In this section, we describe the $\lambda$-duality and a its link to the standard Legendre duality. We start by defining the notions of $\lambda$-conjugate and $\lambda$-convexity/$\lambda$-concavity, then draw a parallel to the regular Legendre duality. We proceed to establish a formal correspondence between the $\lambda$-duality and classical convex duality, including the associated notions of the $\lambda$-gradient, $\lambda$-logarithmic divergence, etc. Some of the derivations are illustrative, yet heuristic—a rigorous analysis in the spirit of Rockafellar [35] is yet to be performed in future research.

3.1. Legendre Duality and Bregman Divergence Reviewed

Recall from (4) that the convex conjugate of a function f on ${\mathbb{R}}^d$ is defined by:

$$f^{*}\left(u\right)=\underset{x}{sup}\left(x·u-f\left(x\right)\right),u\in {\mathbb{R}}^d.$$

(11)

It can be proven that:

(i)

$f^{*}$ is convex;

(ii)

${\left({\left(f^{*}\right)}^{*}\right)}^{*}=f^{*}$;

(iii)

${\left(f^{*}\right)}^{*}=f$ if f is convex and lower semicontinuous.

When f is further differentiable, then the Legendre transformation:

$$u=\mathsf{D}f\left(x\right),$$

which can be motivated by the first-order condition in (11), defines a “dual variable” u, satisfying the Fenchel identity:

$$f\left(x\right)+f^{*}\left(u\right)=x·u.$$

We have $x=\mathsf{D}{f}^{*}\left(u\right)$, provided the second derivative or ${\mathsf{D}}^{2}f$ is positive definite. The function f also defines a Bregman divergence $B_f$ given by:

$$B_f(x,x^{\prime })=f\left(x\right)-f\left(x^{\prime }\right)-\mathsf{D}f\left(x^{\prime }\right)·(x-x^{\prime })\ge 0.$$

(12)

The Bregman divergence satisfies the reference–representation biduality [24,25] in the sense that:

$$B_f(x,x^{\prime })=B_{f^{*}}(u^{\prime },u)$$

where $u=\mathsf{D}f\left(x\right),u^{\prime }=\mathsf{D}f\left(x^{\prime }\right).$ Note that when f is convex and differentiable, the non-negativity of the Bregman divergence encodes the fact that for any $x,x^{\prime }$:

$$f\left(x\right)-f\left(x^{\prime }\right)\ge \mathsf{D}f\left(x^{\prime }\right)·(x-x^{\prime }).$$

3.2. $\lambda$-Deformation of Legendre Duality

The main idea behind the $\lambda$-deformation of the Legendre duality (“$\lambda$-duality”) is to replace the term $x·u$ in (11) by a monotone transformation of $x·u$. Given a parameter $\lambda \in \mathbb{R}\backslash \left\{0\right\}$, later revealed to be the curvature parameter of the information geometric characterization, we replace the term $x·u$ by:

$${\kappa }_{\lambda }(x·u)=\frac{1}{\lambda }log(1+\lambda x·u),$$

(13)

where ${\kappa }_{\lambda }\left(t\right)$ and its inverse ${\gamma }_{\lambda }\left(t\right)$ are given by (10). With this in mind we give the following definition.

Definition 1

($\lambda$-conjugation). Let $\Omega ,{\Omega }^{\prime }\subset {\mathbb{R}}^d$. Given a function $f:\Omega \to \mathbb{R}$, we define its λ-conjugate $f^{\left(\lambda \right)}$ by:

$$f^{\left(\lambda \right)}\left(u\right)=\underset{x\in \Omega }{sup}\left({\kappa }_{\lambda }(x·u)-f\left(x\right)\right),u\in {\Omega }^{\prime }.$$

(14)

Generalized convex dualities have been heavily used in optimal transport theory [36,37] to characterize the optimal transport plans; in this context, it is called the c-duality where c is the cost function of the transport problem. A major novelty of our framework is that the functional form of ${\kappa }_{\lambda }$ (and of ${\gamma }_{\lambda }$) leads to explicit formulas, which are not available in the general case. We remark that this is closely related to the fact that the associated information geometry has constant curvature $\lambda$.

It turns out that the $\lambda$-conjugation defined by (14) corresponds to an appropriately generalized notion of convexity or concavity, through the aid of the function ${\gamma }_{\lambda }$ given by (10). Henceforth, we let $\lambda \in \mathbb{R}\backslash \left\{0\right\}$ be a fixed constant.

Definition 2

($\lambda$-exponential convexity and concavity). Let $\Omega \subset {\mathbb{R}}^d$ be an open convex set. A function $f:\Omega \to \mathbb{R}$ is said to be λ-exponentially convex (“λ-convex”), or λ-exponentially concave (“λ-concave”), if:

$$G_{\lambda ,f}^{}\left(x\right)=({\gamma }_{\lambda }\circ f)\left(x\right)=\frac{1}{\lambda }\left(e^{\lambda f\left(x\right)}-1\right)$$

is convex, or concave, on Ω. When f is $C^2$, we have equivalently that f is λ-convex, or λ-concave, if the Hessian of $G_{\lambda ,f}^{}\equiv {\gamma }_{\lambda }\circ f$ is positive definite, or negative definite.

Note that the additive term $-1/\lambda$ in the above definition of $G_{\lambda ,f}^{}\left(x\right)=\frac{1}{\lambda }(e^{\lambda f\left(x\right)}-1)$ is not necessary; it is included so that ${lim}_{\lambda \to 0}{G}_{\lambda ,f}^{}\left(x\right)=f\left(x\right)$, meaning that in the limiting case of zero-convexity is just ordinary convexity.

It is easily shown that, for $\lambda >0$ a fixed positive number,

(i)

f is $\lambda$-convex if and only if $-f$ is $(-\lambda )$-concave;

(ii)

f is $\lambda$-concave if and only if $-f$ is $(-\lambda )$-convex.

Proposition 1.

Given any $f:\Omega \to \mathbb{R}$, we define variable $\tilde{x}$, which has range $\tilde{\Omega }\subset {\mathbb{R}}^d$, and function $g:\tilde{\Omega }\to \mathbb{R}$ by:

$$\begin{array}{ccc}\hfill \tilde{x}& =& x{e}^{-\lambda f\left(x\right)}=x(1-\lambda G_{-\lambda ,f}\left(x\right)),\hfill \end{array}$$

(15)

$$\begin{array}{ccc}\hfill g\left(\tilde{x}\right)& =& \frac{1}{-\lambda }\left(e^{-\lambda f\left(x\right)}-1\right)={\gamma }_{-\lambda }\left(f\left(x\right)\right)=G_{-\lambda ,f}^{}\left(x\right).\hfill \end{array}$$

(16)

Then, the convex (Legendre) conjugate $g^{*}$ of the function g:

$$g^{*}\left(u\right)=\underset{\tilde{x}\in \tilde{\Omega }}{sup}\left(\tilde{x}·u-g\left(\tilde{x}\right)\right)$$

is related to the λ-conjugate $f^{\left(\lambda \right)}$ of the function f via:

$$g^{*}\left(u\right)=\frac{1}{\lambda }\left(e^{\lambda f^{\left(\lambda \right)}\left(u\right)}-1\right)={\gamma }_{\lambda }\left(f^{\left(\lambda \right)}\left(u\right)\right)=G_{\lambda ,f^{\left(\lambda \right)}}^{}\left(u\right).$$

(17)

Proof. 

We first prove the following identities:

$$\begin{array}{ccc}\hfill (1+\lambda x·u)e^{-\lambda f\left(x\right)}& =& e^{-\lambda f\left(x\right)}+\lambda e^{-\lambda f\left(x\right)}x·u\hfill \\ & =& (1-\lambda g\left(\tilde{x}\right))+\lambda \tilde{x}·u\hfill \\ & =& 1+\lambda (\tilde{x}·u-g\left(\tilde{x}\right))\hfill \end{array}$$

where, going from the first to the second line, we used (15) and the fact:

$$1-\lambda g\left(\tilde{x}\right)=e^{-\lambda f\left(x\right)},$$

which is a re-write of the definition of g given by (16).

With the above identity, we can proceed to prove this proposition. For $u\in {\Omega }^{\prime }$, we have:

$$\begin{array}{ccc}\hfill f^{\left(\lambda \right)}\left(u\right)& =& \underset{x\in \Omega }{sup}\left(\frac{1}{\lambda }log(1+\lambda (x·u))-f\left(x\right)\right)\hfill \\ & =& \underset{\tilde{x}\in \tilde{\Omega }}{sup}\frac{1}{\lambda }log\left(1+\lambda (\tilde{x}·u-g\left(\tilde{x}\right))\right)\hfill \\ & =& \frac{1}{\lambda }log\left(1+\lambda \underset{\tilde{x}\in \tilde{\Omega }}{sup}\left(\tilde{x}·u-g\left(\tilde{x}\right)\right)\right)\hfill \\ & =& \frac{1}{\lambda }log(1+\lambda g^{*}\left(u\right))\hfill \\ & =& {\kappa }_{\lambda }\left(g^{*}\left(u\right)\right).\hfill \end{array}$$

Recasting the above relation yields (17). □

Recall that from convex analysis, $g^{*}$ is always a convex function regardless of whether g is convex (by the property of Legendre conjugation). The expression of $g^{*}\left(u\right)={\gamma }_{\lambda }\left(f^{\left(\lambda \right)}\left(u\right)\right)$ in (17) therefore implies that $f^{\left(\lambda \right)}$ is $\lambda$-convex, by the definition of $\lambda$-convexity.

Corollary 1.

For any $f:\Omega \to \mathbb{R}$, its λ-conjugate $f^{\left(\lambda \right)}\left(u\right)$ as defined by (14) is a λ-convex function of u on ${\Omega }^{\prime }$ (note ${\Omega }^{\prime }$ may not necessarily be convex).

Proof. 

We can also give a direct proof (essentially reversing the steps of the proof of Proposition 1).

$$\begin{array}{ccc}\hfill g^{*}\left(u\right)& =& \underset{\tilde{x}\in \tilde{\Omega }}{sup}\left(\tilde{x}·u-g\left(\tilde{x}\right)\right)\hfill \\ & =& \underset{x\in \Omega }{sup}\left(e^{-\lambda f\left(x\right)}(x·u)-\frac{1}{-\lambda }\left(e^{-\lambda f\left(x\right)}-1\right)\right)\hfill \\ & =& \underset{x\in \Omega }{sup}\frac{1}{\lambda }\left((1+\lambda x·u)e^{-\lambda f\left(x\right)}\right)-\frac{1}{\lambda }\hfill \\ & =& \frac{1}{\lambda }\left(\underset{x\in \Omega }{sup}{e}^{log(1+\lambda x·u)-\lambda f\left(x\right)}-1\right)\hfill \\ & =& \frac{1}{\lambda }\left(e^{{sup}_{x\in \Omega }(log(1+\lambda x·u)-\lambda f\left(x\right))}-1\right)\hfill \\ & =& \frac{1}{\lambda }\left(e^{\lambda f^{\left(\lambda \right)}\left(u\right)}-1\right)\hfill \\ & =& {\gamma }_{\lambda }\left(f^{\left(\lambda \right)}\left(u\right)\right).\hfill \end{array}$$

□

Corollary 1 is the extension of the claim that for any f, the standard Legendre conjugate $f^{*}$ as given by (11) is always a convex function. Because of this, we can prove, in analogy to the standard Legendre conjugation ∗, the following relations:

(i)

${\left({\left(f^{\left(\lambda \right)}\right)}^{\left(\lambda \right)}\right)}^{\left(\lambda \right)}=f^{\left(\lambda \right)}$ for any f.

(ii)

${\left(f^{\left(\lambda \right)}\right)}^{\left(\lambda \right)}=f$ if f is $\lambda$-convex.

3.3. Relations between the $\lambda$-Duality and Legendre Duality

We proceed to establish a formal relationship between the $\lambda$-duality and the ordinary Legendre duality, by relating the $\lambda$-conjugation of a $\lambda$-convex function f, denoted by $f^{\left(\lambda \right)}$, to the standard Legendre conjugation of a function (denoted by ∗).

We continue the analysis performed in Proposition 1. Taking $\lambda$-conjugation for a second time,

$$\begin{array}{ccc}\hfill {\left(f^{\left(\lambda \right)}\right)}^{\left(\lambda \right)}\left(x\right)& =& \underset{u\in {\Omega }^{\prime }}{sup}\frac{1}{\lambda }\left(log(1+\lambda (x·u\left)\right)\right)-f^{\left(\lambda \right)}\left(u\right)\hfill \\ & =& \underset{\tilde{u}\in {\Omega }^{\prime }}{sup}\frac{1}{\lambda }\left(log(1+\lambda (x·\tilde{u}-\tilde{g}\left(\tilde{u}\right)))\right)\hfill \\ & =& \frac{1}{\lambda }log\left(1+\lambda \underset{\tilde{u}\in {\Omega }^{\prime }}{sup}(x·\tilde{u}-\tilde{g}\left(\tilde{u}\right))\right)\hfill \\ & =& \frac{1}{\lambda }log(1+\lambda {\tilde{g}}^{*}\left(x\right))\hfill \\ & =& {\kappa }_{\lambda }\left({\tilde{g}}^{*}\left(x\right)\right).\hfill \end{array}$$

Here, the variable $\tilde{u}$ is defined by:

$$\tilde{u}=u{e}^{-\lambda f^{\left(\lambda \right)}\left(u\right)},$$

and the function $\tilde{g}$ by:

$$\tilde{g}\left(\tilde{u}\right)={\gamma }_{-\lambda }\left(f^{\left(\lambda \right)}\left(u\right)\right)=G_{-\lambda ,f^{\left(\lambda \right)}}^{}\left(u\right).$$

(18)

In the event when f is $\lambda$-convex, then ${\left(f^{\left(\lambda \right)}\right)}^{\lambda }=f$. Therefore:

$${\tilde{g}}^{*}\left(x\right)={\gamma }_{\lambda }\left(f\left(x\right)\right)=G_{\lambda ,f}^{}\left(x\right).$$

Therefore, $\tilde{g}\left(\tilde{u}\right)={\left(G_{\lambda ,f}^{}\right)}^{*}\left(\tilde{u}\right)$. That is, the function $\tilde{g}$ is just the (regular) Legendre conjugation ${}^{*}$ of the function $G_{\lambda ,f}^{}\left(x\right)$. In $\tilde{u}$ parameterization, the $\tilde{g}$ function has the expression of (18) with $\tilde{u}$ and u related by (20). This parallels the fact that $g\left(\tilde{x}\right)={\left(G_{\lambda ,f^{\left(\lambda \right)}}^{}\right)}^{*}\left(\tilde{x}\right)$, and in $\tilde{x}$ parameterization, the g function has the expression of (16) with $\tilde{x}$ and x related by (19).

Summarizing the above, we have:

Theorem 1

(Connecting $\lambda$-duality to Legendre duality). Let f be a λ-convex function and $f^{\left(\lambda \right)}$ be its λ-conjugate. Denote two functions g and $\tilde{g}$:

$$\begin{array}{ccc}\hfill g\left(\tilde{x}\right)& =& G_{-\lambda ,f}^{}\left(x\right)={\gamma }_{-\lambda }\left(f\left(x\right)\right),\hfill \\ \hfill \tilde{g}\left(\tilde{u}\right)& =& G_{-\lambda ,f^{\left(\lambda \right)}}^{}\left(u\right)={\gamma }_{-\lambda }\left(f^{\left(\lambda \right)}\left(u\right)\right),\hfill \end{array}$$

where the two variables $\tilde{x}$ and $\tilde{u}$ are given by:

$$\begin{array}{ccc}\hfill & & \tilde{x}=x{e}^{-\lambda f\left(x\right)}⟺x=\frac{\tilde{x}}{1-\lambda g\left(\tilde{x}\right)},\hfill \end{array}$$

(19)

$$\begin{array}{ccc}\hfill & & \tilde{u}=u{e}^{-\lambda f^{\left(\lambda \right)}\left(u\right)}⟺u=\frac{\tilde{u}}{1-\lambda \tilde{g}\left(\tilde{u}\right)}.\hfill \end{array}$$

(20)

Then, the following statements are equivalent:

(i) 

The $(x,u)$ variables satisfy the λ-duality of a pair of λ-convex functions $(f,f^{\left(\lambda \right)})$:

$${\kappa }_{\lambda }(x·u)=f\left(x\right)+f^{\left(\lambda \right)}\left(u\right);$$

(21)

(ii) 

The $(\tilde{x},u)$ variables satisfy the Legendre duality of a pair of convex functions $(g,g^{*})$:

$$\tilde{x}·u=g\left(\tilde{x}\right)+g^{*}\left(u\right)$$

(22)

with:

$$g^{*}\left(u\right)=G_{\lambda ,f^{\left(\lambda \right)}}^{}\left(u\right)={\gamma }_{\lambda }\left(f^{\left(\lambda \right)}\left(u\right)\right);$$

(iii) 

The $(x,\tilde{u})$ variables satisfy the Legendre duality of a pair of convex functions $(\tilde{g},{\tilde{g}}^{*})$:

$$x·\tilde{u}={\tilde{g}}^{*}\left(x\right)+\tilde{g}\left(\tilde{u}\right)$$

(23)

with:

$${\tilde{g}}^{*}\left(x\right)=G_{\lambda ,f}^{}\left(x\right)={\gamma }_{\lambda }\left(f\left(x\right)\right).$$

Proof. 

To prove the equivalence of (21) and (22), we re-write the latter as:

$$e^{-\lambda f\left(x\right)}x·u={\gamma }_{-\lambda }\left(f\left(x\right)\right)+{\gamma }_{\lambda }\left(f^{\left(\lambda \right)}\left(u\right)\right)=\frac{1}{\lambda }\left(e^{\lambda f^{\left(\lambda \right)}\left(u\right)}-e^{-\lambda f\left(x\right)}\right),$$

where we inserted the following relations:

$$g\left(\tilde{x}\right)={\gamma }_{-\lambda }\left(f\left(x\right)\right),g^{*}\left(u\right)={\gamma }_{\lambda }\left(f^{\left(\lambda \right)}\left(u\right)\right)$$

and replaced $\tilde{x}$ by x using (19). Multiplying $e^{\lambda f\left(x\right)}$ on both sides, we obtain:

$$x·u=\frac{1}{\lambda }\left(e^{\lambda (f^{\left(\lambda \right)}\left(u\right)+f\left(x\right))}-1\right)={\gamma }_{\lambda }(f^{\left(\lambda \right)}\left(u\right)+f\left(x\right)).$$

(24)

Noting ${\left({\gamma }_{\lambda }\right)}^{-1}={\kappa }_{\lambda }$ verifies (21). To prove the equivalence of (21) and (23), we rely on an analogous identity:

$$(1+\lambda x·u)e^{-\lambda f^{\left(\lambda \right)}\left(u\right)}=1+\lambda (x·\tilde{u}-\tilde{g}\left(\tilde{u}\right)),$$

where:

$$\tilde{u}=u{e}^{-\lambda f^{\left(\lambda \right)}\left(u\right)},\tilde{g}\left(\tilde{u}\right)={\gamma }_{-\lambda }\left(f^{\left(\lambda \right)}\left(u\right)\right).$$

We have, after multiplying $e^{\lambda f^{\left(\lambda \right)}\left(u\right)}$ on both sides of (24),

$$x·u{e}^{-\lambda f^{\left(\lambda \right)}\left(u\right)}=\frac{1}{\lambda }\left(e^{\lambda f\left(x\right)}-e^{-\lambda f^{\left(\lambda \right)}\left(u\right)}\right)={\gamma }_{\lambda }\left(f\left(x\right)\right)+{\gamma }_{-\lambda }\left(f^{\left(\lambda \right)}\left(u\right)\right)={\tilde{g}}^{*}\left(x\right)+\tilde{g}\left(\tilde{u}\right),$$

where the last step used:

$${\tilde{g}}^{*}(x)={\gamma }_{\lambda }({(f^{(\lambda )})}^{(\lambda )}(x)),\tilde{g}\left(\tilde{u}\right)={\gamma }_{-\lambda }\left(f^{\left(\lambda \right)}\left(u\right)\right).$$

Noting ${\left(f^{\left(\lambda \right)}\right)}^{\left(\lambda \right)}=f$ due to f assumed to be $\lambda$-convex, then (23) follows. □

We see that the functions ${\gamma }_{\lambda }$ and ${\gamma }_{-\lambda }$ serve as link functions from the $(f,f^{\left(\lambda \right)})$-pair of the $\lambda$-deformed Legendre conjugation to the $(g,g^{*})$-pair and the $(\tilde{g},{\tilde{g}}^{*})$-pair of the regular Legendre conjugation.

4. $\mathbf{\lambda }$-Logarithmic Divergence and Its Dualistic Geometry

In this section, we study the $\lambda$-deformation of the Bregman (canonical) divergence function and the resulting dualistic geometry (Riemannian metric and dual connections), which correspond to the $\lambda$-duality. This involves first establishing the $\lambda$-deformation to the gradient operation (so-called $\lambda$-gradient), which then leads to the so-called $\lambda$-logarithmic divergence function as deformation to the Bregman divergence. Finally, we show that the resulting Riemannian metric is a conformal Hessian metric, while the resulting dual connections are projectively flat (with constant curvature). The conformal and projective factor is parameterized by $\lambda$, which gives the curvature of the constant curvature space.

4.1. $\lambda$-Gradient

Definition 3

($\lambda$-gradient). For $x\in \Omega$, define the λ-gradient ${\mathsf{D}}^{\left(\lambda \right)}f$ by:

$${\mathsf{D}}^{\left(\lambda \right)}f\left(x\right)=\frac{1}{1-\lambda \mathsf{D}f\left(x\right)·x}\mathsf{D}f\left(x\right).$$

(25)

The work of [17] (Theorem 2.2) showed the above formula for deforming the gradient of a function motivated by the $\lambda$-duality setting. For mathematical convenience, it is proven under some regularity conditions; a full generalization along the lines of [35] is a natural direction for further research.

Theorem 2

($\lambda$-gradient for $\lambda$-duality). Let $\lambda \ne 0$, and let f be a λ-exponentially convex function that is $C^2$ on some open convex set $\Omega \subset {\mathbb{R}}^d$, such that (a) ${\mathsf{D}}^2{G}_{\lambda ,f}^{}$ is strictly positive definite and (b) $1-\lambda \mathsf{D}f\left(x\right)·x>0$ on Ω. Then we have

(i) 

${\mathsf{D}}^{\left(\lambda \right)}f$ is a $C^1$-diffeomorphism from Ω to its range ${\Omega }^{\prime }$.

(ii) 

Denote $u={\mathsf{D}}^{\left(\lambda \right)}f\left(x\right)$. We have $1+\lambda x·u>0$, and the following identity holds:

$$f\left(x\right)+f^{\left(\lambda \right)}\left(u\right)=\frac{1}{\lambda }log(1+\lambda x·u)\equiv {\kappa }_{\lambda }(x·u).$$

(iii) 

Furthermore, $x={\mathsf{D}}^{\left(\lambda \right)}{f}^{\left(\lambda \right)}\left(u\right)$.

Note that the $\lambda$-gradient ${\mathsf{D}}^{\left(\lambda \right)}f$ differs from the regular gradient $\mathsf{D}f$ by a scalar multiplication. The duality between x and u under the $\lambda$-duality is mediated by a dual variable $u={\mathsf{D}}^{\left(\lambda \right)}f\left(x\right)$, which plays an important role in what follows.

Let:

(a)

$u={\mathsf{D}}^{\left(\lambda \right)}f\left(x\right)$ denote the $\lambda$-conjugate variable corresponding to x with respect to $f\left(x\right)$;

(b)

$\widehat{u}=\mathsf{D}g\left(\tilde{x}\right)$ be the Legendre conjugate variable corresponding to $\tilde{x}$ with respect to $g\left(\tilde{x}\right)$;

(c)

$x={\mathsf{D}}^{\left(\lambda \right)}{f}^{\left(\lambda \right)}\left(u\right)$ denote the $\lambda$-conjugate variable corresponding to u with respect to $f^{\left(\lambda \right)}\left(u\right)$;

(d)

$\widehat{x}=\mathsf{D}\tilde{g}\left(\tilde{u}\right)$ be the Legendre conjugate variable corresponding to $\tilde{u}$ with respect to $\tilde{g}\left(\tilde{u}\right)$.

Is there a simple relationship between them? The following proposition says $u\left(x\right)=\widehat{u}\left(\tilde{x}\right)$, where $\tilde{x}$ and x are linked by (19), and $x\left(u\right)=\widehat{x}\left(\tilde{u}\right)$, where $\tilde{u}$ and u are linked by (20).

Proposition 2.

We have:

$$u={\mathsf{D}}_x^{\left(\lambda \right)}f\left(x\right)={\mathsf{D}}_{\tilde{x}}^{}g\left(\tilde{x}\right),x={\mathsf{D}}_u^{\left(\lambda \right)}{f}^{\left(\lambda \right)}\left(u\right)={\mathsf{D}}_{\tilde{u}}^{}\tilde{g}\left(\tilde{u}\right)$$

Here, we add the subscript to $\mathsf{D}$ to emphasize the argument with respect to which the derivative is taken.

Proof. 

We use matrix notations where the gradient is regarded as a column vector. Applying the multivariate chain rule to (17), we have:

$${\left({\mathsf{D}}_{\tilde{x}}g\left(\tilde{x}\right)\right)}^{\top }=e^{-\lambda f\left(x\right)}{\left({\mathsf{D}}_{x}f\left(x\right)\right)}^{\top }\frac{\partial x}{\partial \tilde{x}}\left(\tilde{x}\right),$$

where $\frac{\partial x}{\partial \tilde{x}}\left(\tilde{x}\right)$ is the Jacobian of the transformation $\tilde{x}\mapsto x$ and ${(·)}^{\top }$ denotes the transpose, For two vectors x and y, their outer product is denoted by $x{y}^{\top }$, which is a rank-one square matrix with the $(i,j)$-entry $x^i{y}^j$.

From (15), we have:

$$\frac{\partial \tilde{x}}{\partial x}\left(x\right)=e^{-\lambda f\left(x\right)}\left(\mathrm{I}-\lambda x{\left({\mathsf{D}}_{x}f\left(x\right)\right)}^{\top }\right).$$

Since $1-\lambda {\mathsf{D}}_{x}f\left(x\right)·x>0$ by assumption, we can invert the Jacobian by the Sherman–Morrison formula (see [12], Proposition 4) to obtain:

$$\frac{\partial x}{\partial \tilde{x}}\left(\tilde{x}\right)=e^{\lambda f\left(x\right)}\left(\mathrm{I}+\frac{\lambda x{\left({\mathsf{D}}_{x}f\left(x\right)\right)}^{\top }}{1-\lambda {\mathsf{D}}_{x}f\left(x\right)·x}\right).$$

Plugging this into the above, we have:

$${\left({\mathsf{D}}_{\tilde{x}}g\left(\tilde{x}\right)\right)}^{\top }=\frac{{\left({\mathsf{D}}_{x}f\left(x\right)\right)}^{\top }}{1-\lambda {\mathsf{D}}_{x}f\left(x\right)·x}.$$

Using (25) to relate ${\mathsf{D}}_x^{\left(\lambda \right)}f\left(x\right)$ to ${\mathsf{D}}_{x}f\left(x\right)$, the first relation involving ${\mathsf{D}}_{\tilde{x}}g\left(\tilde{x}\right)$ is proven. The proof of the second relation in this proposition is analogous. □

Just as ordinary convexity leads to the notion of Bregman divergence (12), the notion of $\lambda$-exponential convexity leads to a generalization that we call the λ-logarithmic divergence. Henceforth, we let $f:\Omega \to \mathbb{R}$ be a $\lambda$-exponentially convex function on an open convex domain $\Omega \subset {\mathbb{R}}^d$, and we assumed that the regularity conditions in Theorem 2 hold.

4.2. $\lambda$-Logarithmic Divergence

By the definition of the $\lambda$-convexity, we have that $G_{\lambda ,f}^{}\left(x\right)={\gamma }_{\lambda }\left(f\left(x\right)\right)$ is convex on $\Omega$. By the ordinary convexity of $G_{\lambda ,f}^{}$, we have:

$$G_{\lambda ,f}^{}\left(x\right)-G_{\lambda ,f}^{}\left(x^{\prime }\right)\ge \mathsf{D}{G}_{\lambda ,f}^{}\left(x^{\prime }\right)·(x-x^{\prime }),x,x^{\prime }\in \Omega .$$

In terms of f, we have, after some manipulations,

$${\gamma }_{\lambda }(f\left(x\right)-f\left(x^{\prime }\right))\ge \mathsf{D}f\left(x^{\prime }\right)·(x-x^{\prime }).$$

Since ${\gamma }_{\lambda }$ is increasing, we have:

$$f\left(x\right)-f\left(x^{\prime }\right)\ge {\left({\gamma }_{\lambda }\right)}^{-1}\left(\mathsf{D}f\left(x^{\prime }\right)·(x-x^{\prime })\right)={\kappa }_{\lambda }\left(\mathsf{D}f\left(x^{\prime }\right)·(x-x^{\prime })\right).$$

This motivates the following definition.

Definition 4

($\lambda$-logarithmic divergence). We define the λ-logarithmic divergence of f by:

$$\begin{array}{cc}\hfill L_{\lambda ,f}(x,x^{\prime })& =f\left(x\right)-f\left(x^{\prime }\right)-{\kappa }_{\lambda }\left(\mathsf{D}f\left(x^{\prime }\right)·(x-x^{\prime })\right)\hfill \\ \hfill & =f\left(x\right)-f\left(x^{\prime }\right)-\frac{1}{\lambda }log(1+\lambda \mathsf{D}f\left(x^{\prime }\right)·(x-x^{\prime })),x,x^{\prime }\in \Omega .\hfill \end{array}$$

(26)

See Figure 1 for a graphical illustration. We note that the logarithmic correction in (26) corresponds to a logarithmic first-order approximation, based at $x^{\prime }$, which is possible due to the $\lambda$-exponential convexity of f. We also note that when $\lambda >0$, it is possible that $L_{\lambda ,f}(x,x^{\prime })=\infty$. Nevertheless, $L_{\lambda ,f}(x,x^{\prime })$ is finite when x and $x^{\prime }$ are sufficiently close. Formally, letting $\lambda \to 0$ in (26) recovers the Bregman divergence.

4.3. $\lambda$-Logarithmic Divergence in Different Forms

We now prove a lemma about the relationship of the variables $x,u$ and gradients or $\lambda$-gradients of f or $f^{\left(\lambda \right)}$. We assumed, for convenience, that $1+\lambda x·u>0$ for all $x\in \Omega ,u\in {\Omega }^{\prime }$.

Lemma 1.

Given $u={\mathsf{D}}_x^{\left(\lambda \right)}f\left(x\right)$ or equivalently $x={\mathsf{D}}_u^{\left(\lambda \right)}{f}^{\left(\lambda \right)}\left(u\right)$, for arbitrary $x^{\prime },u^{\prime }$ (such that the expressions are well defined), we have the following identities:

$$\begin{array}{ccc}\hfill {\kappa }_{\lambda }(u·x^{\prime })-{\kappa }_{\lambda }(u·x)& =& {\kappa }_{\lambda }(u·(x^{\prime }-x){\left({\Pi }_{\lambda }\right)}^{-1}),\hfill \end{array}$$

(27)

$$\begin{array}{ccc}\hfill {\kappa }_{\lambda }(u^{\prime }·x)-{\kappa }_{\lambda }(u·x)& =& {\kappa }_{\lambda }((u^{\prime }-u)·x{\left({\Pi }_{\lambda }\right)}^{-1}).\hfill \end{array}$$

(28)

where ${\Pi }_{\lambda }$ is a multiplicative factor (function of x or u) given by:

$${\Pi }_{\lambda }\equiv 1+\lambda x·u=\frac{1}{1-\lambda \mathsf{D}f\left(x\right)·x}=\frac{1}{1-\lambda \mathsf{D}{f}^{\left(\lambda \right)}\left(u\right)·u}.$$

Proof. 

Since $u={\mathsf{D}}_x^{\left(\lambda \right)}f\left(x\right)$, substituting (25), we have:

$$u·x=\frac{\mathsf{D}f\left(x\right)·x}{1-\lambda \mathsf{D}f\left(x\right)·x}$$

and:

$$1+\lambda u·x=\frac{1}{1-\lambda \mathsf{D}f\left(x\right)·x}$$

so:

$$1+\lambda u·x^{\prime }=\frac{1+\lambda \mathsf{D}f\left(x\right)·(x^{\prime }-x)}{1-\lambda \mathsf{D}f\left(x\right)·x}=(1+\lambda u·x)(1+\lambda \mathsf{D}f\left(x\right)·(x^{\prime }-x)).$$

Taking the logarithm and rearranging, we obtain (27).

On the other hand, because:

$$x={\mathsf{D}}_u^{\left(\lambda \right)}{f}^{\left(\lambda \right)}\left(u\right)=\frac{\mathsf{D}{f}^{\left(\lambda \right)}\left(u\right)}{1-\lambda \mathsf{D}{f}^{\left(\lambda \right)}\left(u\right)·u},$$

we also have:

$$1+\lambda x·u=\frac{1}{1-\lambda \mathsf{D}{f}^{\left(\lambda \right)}\left(u\right)·u}.$$

The proof of (28) is similar. □

In this above lemma, $x^{\prime }$ and $u^{\prime }$ are arbitrary; it is interesting that a modified form of “linearity” holds even though ${\kappa }_{\lambda }$ is itself nonlinear. As a consequence, we have an alternative expression for $L_{\lambda ,f}(x,x^{\prime })$.

Proposition 3.

$L_{\lambda ,f}(x,x^{\prime })$ defined by (26) can also be written as:

$$L_{\lambda ,f}(x,x^{\prime })=f\left(x\right)-f\left(x^{\prime }\right)-{\kappa }_{\lambda }(x·u^{\prime })+{\kappa }_{\lambda }(x^{\prime }·u^{\prime }).$$

where $u^{\prime }={\mathsf{D}}^{\left(\lambda \right)}f\left(x^{\prime }\right)$.

Of course, we may express the $\lambda$-logarithmic divergence using the conjugate variables $u,u^{\prime }$ as well. Indeed, we have the analogous reference–representation biduality (see [24,25]) that is characteristic of Bregman divergence and canonical divergence for dually flat spaces, that is (8). See [38] for the reference–representation biduality of a general c-divergence (which includes both the Bregman and logarithmic divergences) based on optimal transport.

Theorem 3.

The λ-logarithmic divergence satisfies the reference–representation biduality, namely:

$$L_{\lambda ,f^{\left(\lambda \right)}}(u^{\prime },u)=L_{\lambda ,f}(x,x^{\prime }),$$

where $u={\mathsf{D}}^{\left(\lambda \right)}f\left(x\right)$ and $u^{\prime }={\mathsf{D}}^{\left(\lambda \right)}f\left(x^{\prime }\right)$. Moreover, define the λ-deformed canonical divergence $A_{\lambda ,f}$ by:

$$A_{\lambda ,f}(x,u^{\prime })=f\left(x\right)+f^{\left(\lambda \right)}\left(u^{\prime }\right)-\frac{1}{\lambda }log(1+\lambda x·u^{\prime })=A_{\lambda ,f^{\left(\lambda \right)}}(u^{\prime },x).$$

We have:

$$L_{\lambda ,f}(x,x^{\prime })=A_{\lambda ,f}(x,u^{\prime })=A_{\lambda ,f^{\left(\lambda \right)}}(u^{\prime },x)=L_{\lambda ,f^{\left(\lambda \right)}}(u^{\prime },u).$$

Proposition 3 also allows us to derive our next theorem (Theorem 4) linking $\lambda$-logarithmic divergence and Bregman divergence (also see [19] for a discussion of conformal divergence in the affine immersion setting).

Theorem 4.

The canonical forms of the λ-logarithmic divergence $A_{\lambda ,f}$ and $A_{\lambda ,f^{\left(\lambda \right)}}$ are related to the canonical forms of the Bregman divergence $A_{g^{*}}$ and $A_{\tilde{g}}$ via a conformal transformation and the non-linear link function ${\kappa }_{-\lambda }$:

$$\begin{array}{ccc}& & A_{\lambda ,f^{\left(\lambda \right)}}(u^{\prime },x)={\kappa }_{-\lambda }\left(e^{-\lambda f^{\left(\lambda \right)}\left(u^{\prime }\right)}{A}_{g^{*}}(u^{\prime },\tilde{x})\right),\hfill \\ & =& A_{\lambda ,f}(x,u^{\prime })={\kappa }_{-\lambda }\left(e^{-\lambda f\left(x\right)}{A}_{{\tilde{g}}^{*}}(x,\widetilde{u^{\prime }})\right).\hfill \end{array}$$

Proof. 

$$\begin{array}{cc}{A}_{\lambda ,f^{\left(\lambda \right)}}(u^{\prime },x)& =f^{\left(\lambda \right)}\left(u^{\prime }\right)+f\left(x\right)-\frac{1}{\lambda }log(1+\lambda u^{\prime }·x)\hfill \\ & =f^{\left(\lambda \right)}\left(u^{\prime }\right)-\frac{1}{\lambda }log\left((1+\lambda u^{\prime }·x)e^{-\lambda f\left(x\right)}\right)\hfill \\ & =\frac{1}{\lambda }log(1+\lambda g^{*}\left(u^{\prime }\right))-\frac{1}{\lambda }log\left(e^{-\lambda f\left(x\right)}+\lambda u^{\prime }·\left(x{e}^{-\lambda f\left(x\right)}\right)\right)\hfill \\ & =\frac{1}{\lambda }log(1+\lambda g^{*}\left(u^{\prime }\right))-\frac{1}{\lambda }log(1-\lambda g\left(\tilde{x}\right)+\lambda u^{\prime }·\tilde{x})\hfill \\ & =-\frac{1}{\lambda }log\left(\frac{1+\lambda (u^{\prime }·\tilde{x}-g\left(\tilde{x}\right))}{1+\lambda g^{*}\left(u^{\prime }\right)}\right)\hfill \\ & =-\frac{1}{\lambda }log\left(1+\lambda \frac{u^{\prime }·\tilde{x}-g\left(\tilde{x}\right)-g^{*}\left(u^{\prime }\right)}{1+\lambda g^{*}\left(u^{\prime }\right)}\right)\hfill \\ & =-\frac{1}{\lambda }log\left(1-\lambda \frac{A_{g^{*}}(u^{\prime },\tilde{x})}{1+\lambda g^{*}\left(u^{\prime }\right)}\right)\hfill \\ & =-\frac{1}{\lambda }log\left(1-\lambda \left(e^{-\lambda f^{\left(\lambda \right)}\left(u^{\prime }\right)}{A}_{g^{*}}(u^{\prime },\tilde{x})\right)\right)\hfill \\ & ={\kappa }_{-\lambda }\left(e^{-\lambda f^{\left(\lambda \right)}\left(u^{\prime }\right)}{A}_{g^{*}}(u^{\prime },\tilde{x})\right).\hfill \end{array}$$

The proof of the second line of Theorem 4 is similar. We have $A_{\lambda ,f^{\left(\lambda \right)}}(u^{\prime },x)=A_{\lambda ,f}(x,u^{\prime })$ from Theorem 3. □

4.4. Dualistic Geometry of $\lambda$-Logarithmic Divergence

Regard $x\in \Omega$ as the primal (global) coordinate system of a manifold $\mathcal{M}$. As described in Section 2.1, we may use the $\lambda$-logarithmic divergence $L_{\lambda ,f}$ of f to construct a dualistic structure $(\mathcal{M},\mathtt{g},\nabla ,{\nabla }^{*})$. In this subsection, we provide explicit expressions of the corresponding coefficients and state some key geometric consequences.

We begin with the Riemannian metric.

Theorem 5.

The Riemannian metric $\mathtt{g}$ induced from $L_{\lambda ,f}(x,x^{\prime })$ is given in primal coordinate x by:

$$\mathtt{g}\left(x\right)={\mathsf{D}}^{2}f\left(x\right)+\lambda \left(\mathsf{D}f\left(x\right)\right){\left(\mathsf{D}f\left(x\right)\right)}^{\top }=e^{-\lambda f\left(x\right)}{\mathsf{D}}^2{G}_{\lambda ,f}^{}\left(x\right).$$

(29)

Proof. 

According to (2), we perform direct differentiation of (26):

$$g_{ij}\left(x\right)={\left.\frac{{\partial }^2}{\partial x^i\partial x^j}{L}_{\lambda ,f}(x,x^{\prime })\right|}_{x=x^{\prime }}$$

and obtain the expression of (29). □

By symmetry, under the dual coordinate system $u={\mathsf{D}}^{\left(\lambda \right)}f\left(x\right)$, we have:

$$\mathtt{g}\left(u\right)={\mathsf{D}}^2{f}^{\left(\lambda \right)}\left(u\right)+\lambda (\mathsf{D}{f}^{\left(\lambda \right)}\left(u\right)){(\mathsf{D}{f}^{\left(\lambda \right)}\left(u\right))}^{\top }.$$

From the first equality in (29), we see that $\mathtt{g}$ is a rank-one correction of the Hessian matrix ${\mathsf{D}}^{2}f\left(x\right)$. From the second equality, we see that $\mathtt{g}$ is in fact a conformal Hessian metric, i.e., it has the form $\mathtt{g}\left(x\right)=e^{-\lambda f\left(x\right)}{\mathtt{g}}_0\left(x\right)$, where ${\mathtt{g}}_0\left(x\right)={\mathsf{D}}^2{G}_{\lambda ,f}^{}\left(x\right)$ is the Hessian metric induced by the convex function $G_{\lambda ,f}^{}\left(x\right)=\frac{1}{\lambda }(e^{\lambda f\left(x\right)}-1)$. This conclusion is entirely anticipated from Theorem 4.

To compute the Christoffel symbols of the primal and dual connections, we need an expression of the inverse of the Riemannian metric $\mathtt{g}\left(x\right)$ as a matrix. This is provided by the following proposition.

Proposition 4.

The metric $\mathtt{g}$ can be expressed as:

$$\mathtt{g}\left(x\right)=\frac{1}{{\Pi }_{\lambda }\left(x\right)}\left(I_d-\frac{\lambda }{{\Pi }_{\lambda }\left(x\right)}u{x}^{\top }\right)\frac{\partial u}{\partial x}\left(x\right),$$

(30)

where $\frac{\partial u}{\partial x}$ is the Jacobian matrix of the coordinate transformation $x\mapsto u$ and $I_d$ is the $d\times d$ identity matrix with Kronecker ${\delta }_{ij}$ as its entries. Here:

$${\Pi }_{\lambda }\left(x\right)=1+\lambda x·u=\frac{1}{1-\lambda \mathsf{D}f\left(x\right)·x}$$

and ${\Pi }_{\lambda }\left(x\right)>0$ for $x\in \Omega$ and $u={\mathsf{D}}^{\left(\lambda \right)}f\left(x\right)$, due to Part (ii) of Theorem 2.

Moreover, the inverse of $\mathtt{g}\left(x\right)$ can be expressed as:

$${\left(\mathtt{g}\left(x\right)\right)}^{-1}={\Pi }_{\lambda }\left(x\right)\frac{\partial x}{\partial u}\left(u\right)(I_d+\lambda u{x}^{\top }).$$

(31)

Proof. 

Using the $\lambda$-logarithmic divergence represented as the generalized canonical divergence $A_{\lambda ,f}$ (26), we apply (2) to obtain:

$$\begin{array}{cc}\hfill g_{ij}\left(x\right)& =-{\left.\frac{{\partial }^2}{\partial x^i\partial x^{\prime j}}{L}_{\lambda ,f}(x,x^{\prime })\right|}_{x=x^{\prime }}\hfill \\ \hfill & =-{\left.\frac{{\partial }^2}{\partial x^i\partial x^{\prime j}}\left\{f\left(x\right)+f^{\left(\lambda \right)}\left(u^{\prime }\right)-\frac{1}{\lambda }log(1+\lambda x·u^{\prime })\right\}\right|}_{x=x^{\prime }}\hfill \\ \hfill & =\frac{1}{{\Pi }_{\lambda }\left(x\right)}\left\{\frac{\partial u^i}{\partial x^j}-\frac{\lambda }{{\Pi }_{\lambda }\left(x\right)}{u}^i\sum _{k=1}^d{x}^k\frac{\partial u^k}{\partial x^j}\right\}.\hfill \end{array}$$

Expressing the above expression using matrix notations gives (30). Formula (31) follows by inverting (30) using the Sherman–Morrison formula. □

Under the dualistic structure induced by a $\lambda$-logarithmic divergence, the primal and dual coordinate vector fields are no longer biorthogonal in the sense of (7). Nevertheless, we have the following generalization. Again, we write ${\Pi }_{\lambda }\left(x\right)=1+\lambda x·u$.

Corollary 2.

The inner product of the coordinate vector fields $\frac{\partial }{\partial x^i}$ and $\frac{\partial }{\partial u^j}$ is given by a λ-deformed “biorthogonality” relation:

$$\mathtt{g}\left(\frac{\partial }{\partial x^i},\frac{\partial }{\partial u^j}\right)=\frac{1}{{\Pi }_{\lambda }\left(x\right)}{\delta }_{ij}-\frac{\lambda }{{\Pi }_{\lambda }{\left(x\right)}^2}{x}^j{u}^i.$$

Proof. 

Write $\frac{\partial }{\partial u^j}={\sum }_{m=1}^d\frac{\partial x^m}{\partial u^j}\frac{\partial }{\partial x^m}$. Then:

$$\mathtt{g}\left(\frac{\partial }{\partial x^i},\frac{\partial }{\partial u^j}\right)=\sum _{m=1}^d\frac{\partial x^m}{\partial u^j}\mathtt{g}\left(\frac{\partial }{\partial x^i},\frac{\partial }{\partial x^m}\right).$$

Simplifying the expression using (30) gives the result. For details, see ([12], Proposition 8). □

Theorem 6.

The Christoffel symbols of the primal connection ∇ are given by:

$${\Gamma }_{ij,k}\left(x\right)=-\frac{\lambda }{{\Pi }_{\lambda }{\left(x\right)}^2}\left(u^j\frac{\partial u^i}{\partial x^k}+u^i\frac{\partial u^j}{\partial x^k}\right)+\frac{2{\lambda }^2}{{\Pi }_{\lambda }{\left(x\right)}^3}\sum _{\ell =1}^d{u}^i{u}^j{x}^{\ell }\frac{\partial u^{\ell }}{\partial x^k},$$

where ${\Pi }_{\lambda }\left(x\right)=1+\lambda x·u$ as in Proposition 4. Furthermore, let ${\Gamma }_{ij}^k={\sum }_{\ell =1}^d{\Gamma }_{ij,\ell }{g}^{\ell k}$ be the Christoffel symbol of the second kind, then:

$${\Gamma }_{ij}^k\left(x\right)=\frac{-\lambda }{{\Pi }_{\lambda }\left(x\right)}\left(u^i{\delta }_j^k+u^j{\delta }_i^k\right)=-\lambda \left(\frac{\partial f}{\partial x^i}\left(x\right){\delta }_j^k+\frac{\partial f}{\partial x^j}\left(x\right){\delta }_i^k\right),$$

(32)

where δ is the Kronecker delta.

Similarly, under the dual coordinate system u, the Christoffel symbol (of the second kind) of the dual connection ${\nabla }^{*}$ is given by:

$${\Gamma }_{ij}^{*k}\left(u\right)=-\lambda \left(\frac{\partial f^{\left(\lambda \right)}}{\partial u^i}\left(u\right){\delta }_j^k+\frac{\partial f^{\left(\lambda \right)}}{\partial u^j}\left(u\right){\delta }_i^k\right).$$

(33)

Proof. 

This is a straightforward computation using (3) and Proposition 4. The details, which are a minor modification of the proof of ([12], Proposition 5), are omitted. □

Although the curvatures of ∇ and ${\nabla }^{*}$ are nonzero, it can be shown that ∇ and ${\nabla }^{*}$ are both projectively flat, i.e., each of them is projectively equivalent to a flat connection. Specifically, any ∇-geodesic (resp. ${\nabla }^{*}$-geodesic) is a time-reparameterized straight line under the x (resp. u) coordinate system.

Theorem 7.

The sectional curvatures of ∇ and ${\nabla }^{*}$ with respect to $\mathtt{g}$ are both equal to λ.

Proof. 

See ([12], Theorem 15). □

Using the dual projective flatness and Corollary 2, Reference ([12], Theorem 16) showed that the $\lambda$-logarithmic divergence satisfies a generalized Pythagorean theorem, which generalizes the property of Bregman divergence outlined in Section 2.2.

Theorem 8

(Generalized Pythagorean theorem). Let $P,Q,R\in \mathcal{M}$. Then:

$$L_{\lambda ,f}(x_Q,x_P)+L_{\lambda ,f}(x_R,x_Q)=L_{\lambda ,f}(x_R,x_P)$$

if and only if the ∇-geodesic between Q and R and the ${\nabla }^{*}$-geodesic between Q and P meet $\mathtt{g}$-orthogonally at Q.

To summarize, the dually flat geometry becomes a dually projectively flat geometry with constant sectional curvature $\lambda$, and the Hessian metric becomes a conformal Hessian metric. Nevertheless, the primal and dual geodesics are still straight lines (up to time reparametrizations), and the generalized Pythagorean theorem holds.

We say that the above $\lambda$-deformation framework is “canonical” because the statistical manifold $(\mathcal{M},\mathtt{g},\nabla ,{\nabla }^{*})$, with a conformal Hessian metric $g_{ij}$ given by (29) and a pair of dual projectively flat affine connections ${\Gamma }_{ij}^k,{\Gamma }_{ij}^{*k}$ given by (32) and (33), is the only statistical structure with constant curvature ([12], Theorem 15). Moreover, given such a statistical manifold, one can construct locally a $\lambda$-logarithmic divergence, which induces the given geometry.

5. Linking $\mathbf{\lambda }$-Deformation to Rényi Entropy and Divergence

5.1. Relation between Tsallis’ and Rényi’s Deformation Expressions

Recall that Tsallis [39], in the context of statistical physics, introduced the generalized entropy:

$$H_{\lambda }^{\mathrm{Tsallis}}\left[p\right]=\int p{log}_{\lambda }\left(\frac{1}{p}\right)d\mu =\frac{1}{\lambda }\left(\int {\left(p\left(\zeta \right)\right)}^{\lambda }-1\right)d\mu ;$$

note that we use $\lambda$ here in place of $q=1-\lambda$ as in [40].

Tsallis entropy is related to Rényi entropy [41], defined as:

$$H_{\lambda }^{\mathrm{Rényi}}\left[p\right]:=\frac{1}{\lambda }log\left(\int p^{1-\lambda }\left(\zeta \right)d\mu \right),$$

through a monotonic transformation:

$$H_{\lambda }^{\mathrm{Tsallis}}\left[p\right]=\frac{1}{\lambda }\left(e^{\lambda H_{\lambda }^{\mathrm{Rényi}}\left[p\right]}-1\right).$$

In our current notation,

$$H_{\lambda }^{\mathrm{Tsallis}}\left[p\right]={\gamma }_{\lambda }\left(H_{\lambda }^{\mathrm{Rényi}}\left[p\right]\right)⟺H_{\lambda }^{\mathrm{Rényi}}\left[p\right]={\kappa }_{\lambda }\left(H_{\lambda }^{\mathrm{Tsallis}}\left[p\right]\right).$$

Rényi divergence (with Rényi index $1-\lambda$) is defined by:

$$H_{\lambda }^{\mathrm{Rényi}}\left[p\right||p^{\prime }]=\frac{-1}{\lambda }log\int {\left(p\left(\zeta \right)\right)}^{1-\lambda }{\left(p^{\prime }\left(\zeta \right)\right)}^{\lambda }d\mu .$$

Rényi divergence is additive: given two product measures $p_1\otimes p_2$ and $p_1^{\prime }\otimes p_2^{\prime }$, we have:

$$H_{\lambda }^{\mathrm{Rényi}}[p_1\otimes p_2\left|\right|p_1^{\prime }\otimes p_2^{\prime }]=H_{\lambda }^{\mathrm{Rényi}}[p_1\left|\right|p_1^{\prime }]+H_{\lambda }^{\mathrm{Rényi}}[p_2\left|\right|p_2^{\prime }].$$

Because Tsallis entropy is not additive, this has been used as an argument for favoring Rényi entropy as a physical concept over Tsallis entropy; see [28] (Section 9.3) and [42].

5.2. $\lambda$-Exponential Family

Under the $\lambda$-deformation, there is an intrinsic link between the subtractive and divisive normalizations of the $\lambda$-deformed exponential family. Starting with the observation:

$$e^{{\kappa }_{\lambda }\left(t\right)}={(1+\lambda t)}^{1/\lambda }={exp}_{\lambda }\left(t\right),$$

we investigate the identity:

$${(1+\lambda \vartheta ·F\left(\zeta \right))}^{1/\lambda }{e}^{-{\phi }_{\lambda }\left(\vartheta \right)}={(1+\lambda (\theta ·F\left(\zeta \right)-{\varphi }_{\lambda }\left(\theta \right)))}^{1/\lambda }.$$

Taking the $\lambda$-th power and equating both sides, we obtain the conditions for the above identity to hold:

$$\begin{array}{ccc}\hfill \theta =\vartheta e^{-\lambda {\phi }_{\lambda }\left(\vartheta \right)}& ⟺& \vartheta =\frac{\theta }{1-\lambda {\varphi }_{\lambda }\left(\theta \right)},\hfill \\ \hfill {\varphi }_{\lambda }\left(\theta \right)=\frac{1}{-\lambda }(e^{-\lambda {\phi }_{\lambda }\left(\vartheta \right)}-1)& ⟺& {\phi }_{\lambda }\left(\vartheta \right)=-\frac{1}{\lambda }log(1-\lambda {\varphi }_{\lambda }\left(\theta \right)).\hfill \end{array}$$

This fact led us to define a $\lambda$-exponential family that can be normalized both subtractively and divisively: the former denoted by $p\left(\zeta \right|\theta )$ and the latter denoted by $p\left(\zeta \right|\vartheta )$.

Proposition 5

(Reparameterization equivalence). Let $\lambda \ne 0$. With respect to a given reference measure μ and a fixed vector of random functions $F\left(\zeta \right)=(F_1\left(\zeta \right),\dots ,F_d\left(\zeta \right))$, the λ-exponential family is given by $p^{\left(\lambda \right)}\left(\zeta \right|\theta )$ under subtractive normalization and by $p^{\left(\lambda \right)}\left(\zeta \right|\vartheta )$ under divisive normalization; they are reparametrizations of each other:

$$p^{\left(\lambda \right)}\left(\zeta \right|\theta )={exp}_{\lambda }(\theta ·F\left(\zeta \right)-{\varphi }_{\lambda }\left(\theta \right))={exp}_{\lambda }(\vartheta ·F\left(\zeta \right))e^{-{\phi }_{\lambda }\left(\vartheta \right)}=p^{\left(\lambda \right)}\left(\zeta \right|\vartheta ).$$

(34)

Here, the function ${\varphi }_{\lambda }\left(\theta \right)$ is called subtractive $\lambda$-potential and used for subtractive normalization, while ${\phi }_{\lambda }\left(\vartheta \right)$ is called divisive $\lambda$-potential and used for divisive normalization. Note that ${\varphi }_{\lambda }$ and ${\phi }_{\lambda }$ may not have same domains. They satisfy:

$${\varphi }_{\lambda }\left(\theta \right)={\gamma }_{-\lambda }\left({\phi }_{\lambda }\left(\vartheta \right)\right)⟺{\phi }_{\lambda }\left(\vartheta \right)={\kappa }_{-\lambda }\left({\varphi }_{\lambda }\left(\theta \right)\right),$$

where:

$${\kappa }_{-\lambda }\left(t\right)=-\frac{1}{\lambda }log(1-\lambda t),{\gamma }_{-\lambda }\left(t\right)=\frac{1}{\lambda }\left(1-e^{-\lambda t}\right).$$

Note again that we use $\vartheta$ for the divisive normalization setting and distinguish it from $\theta$ for the subtractive normalization setting. For later convenience, we also note:

$$e^{-\lambda {\phi }_{\lambda }\left(\vartheta \right)}=1-\lambda {\varphi }_{\lambda }\left(\theta \right).$$

5.2.1. Under Subtractive Normalization

The deformed exponential family takes the form:

$$p\left(\zeta \right|\theta )={exp}_{\lambda }\left(\theta ·F\left(\zeta \right)-{\varphi }_{\lambda }\left(\theta \right)\right)$$

(35)

where $\theta ·F\left(\zeta \right)={\sum }_{i=1}^d{\theta }^i{F}_i\left(\zeta \right)$, and the subtractive $\lambda$-potential ${\varphi }_{\lambda }\left(\theta \right)$ is specified by the normalization:

$$1=\int p\left(\zeta \right|\theta )d\mu =\int {exp}_{\lambda }\left(\theta ·F\left(\zeta \right)-{\varphi }_{\lambda }\left(\theta \right)\right)d\mu .$$

This leads to:

$$\frac{\partial {\varphi }_{\lambda }}{\partial {\theta }^i}=\int \tilde{p}\left(\zeta \right|\theta )F_i\left(\zeta \right)d\mu$$

with the escort transformation given by:

$$\tilde{p}\left(\zeta \right|\theta )=\frac{{\left(p\left(\zeta \right|\theta )\right)}^{1-\lambda }}{\int {\left(p\left(\zeta \right|\theta )\right)}^{1-\lambda }d\mu }.$$

(36)

Clearly, when $\lambda \to 0$, we recover the regular exponential family (9). It was Tsallis who introduced the q-exponential family, where $q=1-\lambda$.

5.2.2. Under Divisive Normalization

To deform the exponential family through divisive normalization, we use a smooth monotone function ${\kappa }_{\lambda }(·)$ and define a parametric probability family, which takes the form:

$$logp\left(\zeta \right|\vartheta )={\kappa }_{\lambda }(\vartheta ·F\left(\zeta \right))-{\phi }_{\lambda }\left(\vartheta \right).$$

Note that we use the symbol $\vartheta$ to distinguish it from the parameter $\theta$ in the subtractive case. Here:

$${\phi }_{\lambda }\left(\vartheta \right)=log\int e^{{\kappa }_{\lambda }(\vartheta ·F\left(\zeta \right))}d\mu$$

is the divisive normalization function, and it was assumed that:

$$\int e^{{\kappa }_{\lambda }(\vartheta ·F\left(\zeta \right))}d\mu <\infty$$

in the domain of $\vartheta$ (the natural parameter set). It is possible that the support of the density depends on the parameter $\vartheta$, as in the case of the q-exponential family; see [17]. To avoid technicalities, we assumed that the support of $p\left(\zeta \right|\vartheta )$ is independent of $\vartheta$.

Writing out ${\kappa }_{\lambda }\left(\right)$, the resulting family is:

$$p\left(\zeta \right|\vartheta )={(1+\lambda \vartheta ·F\left(\zeta \right))}^{1/\lambda }{e}^{-{\phi }_{\lambda }\left(\vartheta \right)},$$

(37)

where the divisive $\lambda$-potential ${\phi }_{\lambda }\left(\vartheta \right)$ is given by:

$${\phi }_{\lambda }\left(\vartheta \right)=log\int {(1+\lambda \vartheta ·F\left(\zeta \right))}^{1/\lambda }d\mu$$

(38)

is finite on the parameter set. This family unifies the ${\mathcal{F}}^{(\pm \alpha )}$-families introduced in [12].

5.3. $\lambda$-Mixture Family

We next define a mixture-type family dual to the $\lambda$-exponential family, in an analogous way that an exponential family is dual to the mixture family. The form of the family is justified by its compatibility with the $\lambda$-duality.

Definition 5

($\lambda$-mixture family). Let $\lambda \ne 0,1$ be given. The λ-mixture family with respect to a fixed set of densities $P_0\left(\zeta \right),P_1\left(\zeta \right),\dots ,P_d\left(\zeta \right)$ is defined by:

$$p^{\left(\lambda \right)}\left(\zeta \right|\eta )=\frac{1}{Z_{\lambda }\left(\eta \right)}{\left(\sum _{i=0}^d{\eta }_i{\tilde{P}}_i\left(\zeta \right)\right)}^{1/(1-\lambda )},$$

(39)

where $\eta =({\eta }_1,\cdots ,{\eta }_d)$ is the mixture parameter satisfying $0\le {\eta }_i\le 1$ and ${\eta }_0=1-{\sum }_{i=1}^d{\eta }_i>0$. Here, ${\tilde{P}}_i,i=0,1,\cdots ,d$ denotes the escort transformation, as given by (36), of the given $P_i$’s:

$${\tilde{P}}_i\left(\zeta \right)=\frac{{\left(P_i\left(\zeta \right)\right)}^{1-\lambda }}{\int {\left(P_i\left(\zeta \right)\right)}^{1-\lambda }d\mu },$$

where the denominator is assumed to exist, and $Z_{\lambda }\left(\eta \right)$ represents the integral:

$$\begin{array}{cc}\hfill Z_{\lambda }\left(\eta \right)& =\int {\left(\sum _{i=0}^d{\eta }_i{\tilde{P}}_i\left(\zeta \right)\right)}^{1/(1-\lambda )}d\mu ,\hfill \end{array}$$

which is assumed to converge for all η and to be differentiable under the integral sign.

Denote:

$$C_i=\int {\left(P_i\left(\zeta \right)\right)}^{1-\lambda }d\mu$$

and:

$${\tilde{\eta }}_i=\frac{1}{h_{\lambda }\left(\eta \right)}\frac{{\eta }_i}{C_i}$$

where:

$$h_{\lambda }\left(\eta \right)=\sum _{i=0}^d\frac{{\eta }_i}{C_i}.$$

Then, $0\le {\tilde{\eta }}_i\le 1$ and ${\sum }_{i=0}^d{\tilde{\eta }}_i=1.$ We can express $p^{\left(\lambda \right)}$ now in $\tilde{\eta }$:

$$\begin{array}{cc}\hfill & p^{\left(\lambda \right)}=\frac{1}{Z_{\lambda }\left(\eta \right)}{\left(\sum _{i=0}^d{\eta }_i{\tilde{P}}_i\left(\zeta \right)\right)}^{1/(1-\lambda )}=\frac{h_{\lambda }\left(\eta \right)}{Z_{\lambda }\left(\eta \right)}{\left(\sum _{i=0}^d{\tilde{\eta }}_i{\left(P_i\left(\zeta \right)\right)}^{1-\lambda }\right)}^{1/(1-\lambda )}\hfill \\ \hfill & =e^{log{h}_{\lambda }\left(\eta \right)-log{Z}_{\lambda }\left(\eta \right)}{\left(\left(1-\sum _{i=1}^d{\tilde{\eta }}_i\right){\left(P_0\left(\zeta \right)\right)}^{1-\lambda }+\sum _{i=1}^d{\tilde{\eta }}_i{\left(P_i\left(\zeta \right)\right)}^{1-\lambda }\right)}^{1/(1-\lambda )}\hfill \\ \hfill & =e^{log{h}_{\lambda }\left(\eta \right)-log{Z}_{\lambda }\left(\eta \right)}{\left(1+\sum _{i=1}^d{\tilde{\eta }}_i\frac{{\left(P_i\left(\zeta \right)\right)}^{1-\lambda }-{\left(P_0\left(\zeta \right)\right)}^{1-\lambda }}{{\left(P_0\left(\zeta \right)\right)}^{1-\lambda }}\right)}^{1/(1-\lambda )}{P}_0\left(\zeta \right).\hfill \end{array}$$

Setting:

$$F_i\left(\zeta \right)=\frac{1}{1-\lambda }\left({\left(\frac{P_i\left(\zeta \right)}{P_0\left(\zeta \right)}\right)}^{1-\lambda }-1\right),$$

with $d\nu =P_0\left(\zeta \right)d\mu$, the density of the $\lambda$-mixture family $p^{\left(\lambda \right)}$ with respect to the new measure $\nu$ now has the form:

$$p^{\left(\lambda \right)}\left(\zeta \right|\tilde{\eta })={\left(1+(1-\lambda )\tilde{\eta }·F\left(\zeta \right)\right)}^{1/(1-\lambda )}{e}^{-{\psi }_{1-\lambda }\left(\tilde{\eta }\right)},$$

where ${\psi }_{1-\lambda }\left(\tilde{\eta }\right)=log{Z}_{\lambda }\left(\eta \right)-log{h}_{\lambda }\left(\eta \right)$. Thus, we showed the following:

Proposition 6

(Relation between $\lambda$-exponential and $\lambda$-mixture families). Suppose $\lambda \ne 0,1$. A λ-mixture family with pure densities:

$$P\left(\zeta \right)=\{P_0\left(\zeta \right),P_1\left(\zeta \right),\cdots ,P_d\left(\zeta \right)\}$$

becomes a λ-exponential family with the vector of random functions:

$$F\left(\zeta \right)=\{F_1\left(\zeta \right),\cdots ,F_d\left(\zeta \right)\}$$

after a transformation of the dominating measure $d\mu \to d\nu =P_0\left(\zeta \right)d\mu$ and the random variables $P\left(\zeta \right)\to F\left(\zeta \right)$:

$$F_i\left(\zeta \right)=\frac{1}{1-\lambda }\left({\left(\frac{P_i\left(\zeta \right)}{P_0\left(\zeta \right)}\right)}^{1-\lambda }-1\right)={log}_{1-\lambda }\left(\frac{P_i\left(\zeta \right)}{P_0\left(\zeta \right)}\right),$$

and a reparameterization $\eta \to \tilde{\eta }$:

$${\tilde{\eta }}_i=\frac{1}{h_{\lambda }{C}_i}{\eta }_i⟺{\eta }_i={\tilde{\eta }}_i\left(h_{\lambda }{C}_i\right)$$

with:

$$h_{\lambda }=\sum _{i=0}^d\frac{{\eta }_i}{C_i}=\sum _{i=0}^d{\tilde{\eta }}_i{C}_i.$$

5.4. Potential Functions as Rényi Entropies

We now show that our $\lambda$-duality framework is naturally compatible with the $\lambda$-exponential and $\lambda$-mixture families, with Rényi entropy and Rényi divergence replacing Shannon entropy and Kullback–Leibler divergence. In what follows, we assume $\lambda <1$.

Proposition 7

(For the $\lambda$-exponential family). With respect to the λ-exponential family defined by (37) with divisive potential function ${\phi }_{\lambda }$ given by (38), we have:

(i) 

${\phi }_{\lambda }\left(\vartheta \right)$ is λ-convex. Moreover, $1-\lambda \mathsf{D}{\phi }_{\lambda }\left(\vartheta \right)·\theta >0$.

(ii) 

The λ-conjugate variable $\eta ={\mathsf{D}}^{\left(\lambda \right)}{\phi }_{\lambda }\left(\vartheta \right)=\mathsf{D}{\varphi }_{\lambda }\left(\theta \right)$ is the the escort expectation:

$$\eta =\frac{\int {\left(p\left(\zeta \right|\theta )\right)}^{1-\lambda }F\left(\zeta \right)d\mu }{\int {\left(p\left(\zeta \right|\theta )\right)}^{1-\lambda }d\mu }=\int \tilde{p}\left(\zeta \right|\theta )F\left(\zeta \right)d\mu .$$

(iii) 

The λ-conjugate function ${\psi }_{\lambda }\left(\eta \right)$ with respect to ${\phi }_{\lambda }\left(\vartheta \right)$ is given by:

$${\psi }_{\lambda }\left(\eta \right)=-H_{\lambda }^{\mathrm{Rényi}}\left[p(·|\vartheta )\right].$$

(iv) 

The λ-logarithmic divergence is the Rényi divergence:

$$L_{\lambda ,{\phi }_{\lambda }}(\vartheta ,{\vartheta }^{\prime })=H_{\lambda }^{\mathrm{Rényi}}[p(·|{\vartheta }^{\prime })\left|\right|p(·|\vartheta )].$$

Proposition 8

(For the $\lambda$-mixture family). With respect to the λ-mixture family given by (39) with its potential function ${\psi }_{\lambda }\left(\eta \right)$ given by:

$${\psi }_{\lambda }\left(\eta \right)=\frac{1-\lambda }{\lambda }log\int {\left(\sum _{i=0}^d{\eta }_i{\tilde{P}}_i\right)}^{1/(1-\lambda )}d\mu =\frac{1-\lambda }{\lambda }log{Z}_{\lambda }\left(\eta \right),$$

we have:

(i) 

The potential function ${\psi }_{\lambda }\left(\eta \right)$ is a λ-convex function of η.

(ii) 

The potential function ${\psi }_{\lambda }\left(\eta \right)$ is given by:

$${\psi }_{\lambda }\left(\eta \right)=-H_{\lambda }^{\mathrm{Rényi}}\left[p(·|\eta )\right].$$

(iii) 

The λ-logarithmic divergence is the Rényi divergence:

$$L_{\lambda ,{\psi }_{\lambda }}(\eta ,{\eta }^{\prime })=H_{\lambda }^{\mathrm{Rényi}}[p(·|\eta )\left|\right|p(·|{\eta }^{\prime })].$$

The proofs of the above Proposition 7 (about the $\lambda$-exponential family) and Proposition 8 (about the $\lambda$-mixture family) can be found in [17].

6. Summary and Conclusions

Our paper summarizes a canonical approach to deforming exponential and mixture families and the associated dually flat Hessian geometry. The $\lambda$-exponential family we introduced has two parameterizations (35) and (37):

$$p^{\left(\lambda \right)}\left(\zeta \right|·)={exp}_{\lambda }(\vartheta ·F\left(\zeta \right))e^{-{\phi }_{\lambda }\left(\vartheta \right)}={exp}_{\lambda }(\theta ·F\left(\zeta \right)-{\varphi }_{\lambda }\left(\theta \right)).$$

The two expressions reflect subtractive and divisive normalizations—a typical example of the former is the q-exponential family with associated Tsallis entropy, whereas an example of the latter is the ${\mathcal{F}}^{(\pm \alpha )}$-family and the associated Rényi entropy. These two versions of deformation to an exponential family are two faces of the same coin; furthermore, the $\lambda$-exponential family is also linked to the $\lambda$-mixture family, when $\lambda \ne 0,1$, via a reparameterization of the random functions $F\left(\zeta \right)$ above.

The coincidence of these two parameterizations of the deformed family is associated with the $\lambda$-duality, which is the main focus of our exposition. The $\lambda$-duality is a “deformation” (see

Table 1. Generalization of objects from the Hessian (dually flat) geometry to the $\lambda$-deformed (dually projectively flat) geometry.

Objects	Conventional ($\mathit{\lambda }=0$)	$\mathit{\lambda }$-Deformed
transformation	Legendre	$\lambda$-Legendre
conjugation	${sup}_x(x·u-f\left(x\right))$	${sup}_x({\kappa }_{\lambda }(x·u)-f\left(x\right))$
potentials	convex	$\lambda$-convex
associated divergence	Bregman	$\lambda$-logarithmic
Riemannian metric	Hessian	conformal Hessian
affine connections	dually flat	dually projectively flat
curvature of connections	0	constant $\lambda \ne 0$
biorthogonal coordinates	$(x,u)$	$(\tilde{x},u)$ or $(x,\tilde{u})$
probability family	exponential	$\lambda$-exponential
probability family	mixture	$\lambda$-mixture
associated divergence	Kullback–Leibler	Rényi
associated entropy	Shannon	Rényi/Tsallis

) of the usual Legendre duality reviewed in Section 3.1. In a nutshell, instead of convex functions, we worked with $\lambda$-convex functions f such that $\frac{1}{\lambda }(e^{\lambda f}-1)$ is convex, for a fixed $\lambda \ne 0$. Furthermore, instead of the convex conjugate, we used the $\lambda$-conjugate given by:

$$f^{\left(\lambda \right)}\left(u\right)=\underset{x}{sup}\left(\frac{1}{\lambda }log(1+\lambda x·u)-f\left(x\right)\right).$$

The expression of the $\lambda$-duality:

$${\kappa }_{\lambda }(x·u)=f\left(x\right)+f^{\left(\lambda \right)}\left(u\right),$$

turns out to be a re-write of the Legendre duality between $\tilde{x}$ and u:

$$\tilde{x}·u=g\left(\tilde{x}\right)+g^{*}\left(u\right),with\tilde{x}=x{e}^{-\lambda f\left(x\right)};$$

and a re-write of the Legendre duality between x and $\tilde{u}$:

$$x·\tilde{u}={\tilde{g}}^{*}\left(x\right)+\tilde{g}\left(\tilde{u}\right),with\tilde{u}=u{e}^{-\lambda f^{\left(\lambda \right)}\left(u\right)}.$$

Therefore, $\lambda$-duality is in essence the Legendre duality with a $\lambda$-dependent rescaling of the variables:

$$\tilde{x}=x{e}^{-\lambda f\left(x\right)}⟺x=\frac{\tilde{x}}{1-\lambda g\left(\tilde{x}\right)}$$

and:

$$\tilde{u}=u{e}^{-\lambda f^{\left(\lambda \right)}\left(u\right)}⟺u=\frac{\tilde{u}}{1-\lambda \tilde{g}\left(\tilde{u}\right)}.$$

The two pairs of convex functions $g,g^{*}$ and $\tilde{g},{\tilde{g}}^{*}$ are linked with the pair of $\lambda$-convex functions $f,f^{\left(\lambda \right)}$ via:

$$\begin{array}{ccc}\hfill g\left(\tilde{x}\right)& =& G_{-\lambda ,f}^{}\left(x\right)={\gamma }_{-\lambda }\left(f\left(x\right)\right)=({\gamma }_{-\lambda }\circ f)\left(x\right);\hfill \\ \hfill g^{*}\left(u\right)& =& G_{\lambda ,f^{\left(\lambda \right)}}^{}\left(u\right)={\gamma }_{\lambda }\left(f^{\left(\lambda \right)}\left(u\right)\right)=({\gamma }_{\lambda }\circ f^{\left(\lambda \right)})\left(u\right);\hfill \\ \hfill \tilde{g}\left(\tilde{u}\right)& =& G_{-\lambda ,f^{\left(\lambda \right)}}^{}\left(u\right)={\gamma }_{-\lambda }\left(f^{\left(\lambda \right)}\left(u\right)\right)=({\gamma }_{-\lambda }\circ f^{\left(\lambda \right)})\left(u\right);\hfill \\ \hfill {\tilde{g}}^{*}\left(x\right)& =& G_{\lambda ,f}^{}\left(x\right)={\gamma }_{\lambda }\left(f\left(x\right)\right)=({\gamma }_{\lambda }\circ f)\left(x\right).\hfill \end{array}$$

The $\lambda$-duality leads to nontrivial mathematical questions, e.g., a differential calculus in the spirit of Rockafellar and analogous to functions of the Legendre type. Some of the derivations in the current paper were heuristic, and a complete and rigorous development is left for future research.

Coming back to the probability families, we first verified that the subtractive potential ${\varphi }_{\lambda }\left(\theta \right)$ is convex in $\theta$ and the divisive potential ${\phi }_{\lambda }\left(\vartheta \right)$ is $\lambda$-convex in $\vartheta$. Subtractive normalization using ${\varphi }_{\lambda }\left(\theta \right)$ is associated with the regular Legendre duality, whereas divisive normalization using ${\phi }_{\lambda }\left(\vartheta \right)$ is associated with the $\lambda$-duality. This gives an interpretation of the distinctiveness of Rényi entropy (used in the latter) from Tsallis entropy (used in the former) based on their intimate connection to the $\lambda$-duality (for $\lambda \ne 0$) or to the Legendre duality. As $\lambda$ is the parameter that controls the curvature in the Riemannian geometry of these probability families (see [12]), our framework provides a simple parametric deformation from the dually flat geometry (of the exponential model) to the dually projectively flat geometry (of the $\lambda$-exponential model). We expect that this framework will generate new insights in the applications of the q-exponential family and related concepts in statistical physics and information science.