On Monotone Embedding in Information Geometry

Abstract

A paper was published (Harsha and Subrahamanian Moosath, 2014) in which the authors claimed to have discovered an extension to Amari’s α-geometry through a general monotone embedding function. It will be pointed out here that this so-called (F, G)-geometry (which includes F-geometry as a special case) is identical to Zhang’s (2004) extension to the α-geometry, where the name of the pair of monotone embedding functions ρ and τ were used instead of F and H used in Harsha and Subrahamanian Moosath (2014). Their weighting function G for the Riemannian metric appears cosmetically due to a rewrite of the score function in log-representation as opposed to (ρ, τ)-representation in Zhang (2004). It is further shown here that the resulting metric and α-connections obtained by Zhang (2004) through arbitrary monotone embeddings is a unique extension of the α-geometric structure. As a special case, Naudts’ (2004) ϕ-logarithm embedding (using the so-called log_ϕ function) is recovered with the identification ρ = ϕ, τ = log_ϕ, with ϕ-exponential exp_ϕ given by the associated convex function linking the two representations.

In a recent paper that appeared in Entropy (Harsha and Subrahamanian Moosath, 2014) [1], the authors proposed an extension to Amari’s α-geometry, which they call F - or (F, G)-geometry, where F is a monotone embedding function and G is the weighting function for taking the expectation of random variables in calculating the Riemannian metric (G = 1 reduces to F -geometry, with the standard Fisher–Rao metric). This paper serves the purpose of pointing out that (F, G)-geometry as proposed is the same as what Zhang (2004) [2] has obtained for extending the α-geometry and captured in his subsequent work [4–8]. The metric and affine connections proposed by [1] are identical to [2] apart from the notations: the embedding functions F and H in [1] were denoted as ρ and τ in [2], and weighting function G in [1] is a trivial rewriting of the convex function f used by [2].

This paper will start in Section 1 with a review of Amari’s α-geometry and α-embedding, a review of Zhang’s (2004) [2] extension to ρ-embedding with an arbitrary monotone function and a summary of Harsha and Subrahamanian Moosath (2014) [1]. Then, the equivalence of [1] to [2] is shown. In Section 2, after analyzing the group of monotone embedding functions, a stronger statement is made: the construction of [2] is a unique dualistic extension of Amari’s α-geometry through arbitrary monotone embedding in place of α-embedding. As an important special case, we illustrate how the deformed logarithm log_ϕ associated with an arbitrary strictly increasing function ϕ as investigated by Naudts (2004) [3] arises naturally from identifying ϕ with ρ and with a proper choice of the auxiliary function f as a part of Zhang’s theory.

1. Equivalence of (F, G)-Geometry to Zhang’s (2004) [2] (ρ, τ)-Geometry

1.1. Amari’s α-Geometry and α-Embedding

The now standard differential geometric characterization of the manifold $ℳ$_Θ = {p(·|θ), θ∈Θ ⊆ ℝⁿ} of parametric probability functions p (probability density or probability distributions) is through the Fisher–Rao metric g_ij as its Riemannian metric:

$$g_{ij}(\theta )={\mathrm{E}}_{\mu }\left\{p(\zeta |\theta )\frac{\partial \log p(\zeta |\theta )}{\partial {\theta }^i}\frac{\partial \log p(\zeta |\theta )}{\partial {\theta }^j}\right\}$$

(1)

and a family of α-connections (given by Amari [9,10]) with coefficients Γ^(α) (α ∈ ℝ):

$${\mathrm{\Gamma }}_{ij,k}^{(\alpha )}(\theta )={\mathrm{E}}_{\mu }\left\{\left(\frac{1-\alpha }{2}\frac{\partial \log p(\zeta |\theta )}{\partial {\theta }^i}\frac{\partial \log p(\zeta |\theta )}{\partial {\theta }^j}+\frac{{\partial }^2\log p(\zeta |\theta )}{\partial {\theta }^i\partial {\theta }^j}\right)\frac{\partial p(\zeta |\theta )}{\partial {\theta }^k}\right\}.$$

(2)

Here, E_µ denotes the expectation with respect to a background measure µ of the random variable denoted by ζ:

$${\mathrm{E}}_{\mu }\{\cdot \}={\displaystyle \int (\cdot )d\mu (\zeta )}.$$

(3)

The α-connection is constructed as a convex combination of a pair of conjugate connections Γ, Γ^*

$${\mathrm{\Gamma }}_{ij,k}^{(\alpha )}(\theta )=\frac{1+\alpha }{2}{\mathrm{\Gamma }}_{ij,k}(\theta )+\frac{1-\alpha }{2}{\mathrm{\Gamma }}_{ij,k}^{*}(\theta ),$$

(4)

where Γ ≡ Γ⁽¹⁾ is frequently called e-connection (α = 1) and Γ^* ≡ Γ⁽⁻¹⁾ called m-connection (α = −1). A Riemannian manifold M_µ with its metric g and the family of α-connections Γ^(α) in the form of (1) and (2) has been called α-geometry. Amari’s α-geometry can be specified in terms of a symmetric (0, 2)-tensor g_ij (the Fisher–Rao metric) and a totally symmetric (0, 3)-tensor T_ijk (sometimes called the Amari–Chentsov tensor), which is linked to the α-connections via:

$${\mathrm{\Gamma }}_{ij,k}^{(\alpha )}={\mathrm{\Gamma }}_{ij,k}^{LC}(\theta )-\frac{\alpha }{2}{T}_{ijk}(\theta ),$$

(5)

where ${\mathrm{\Gamma }}_{ij,k}^{LC}$ is the Levi–Civita connection corresponding to the Riemannian metric g.

As an extension of the logarithmic embedding l(p) = log p of probability density function p, an α-embedding function [10] is defined through l^(α) : ℝ⁺ → ℝ:

$$l^{(\alpha )}(t)=\{\begin{array}{cc}\log t& \alpha =1\\ \frac{2}{1-\alpha }{t}^{(1-\alpha )/2}& \alpha \ne 1\end{array}.$$

(6)

It is an interesting observation (e.g., p. 46 in [11]) that the α-geometry can be recovered under such α-representation (scaling) of the probability function, that is the Fisher–Rao metric turns out to be α-independent (i.e., embedding independent) and the ±1-connections precisely the α-connections:

$$g_{ij}(\theta )={\mathrm{E}}_{\mu }\left\{\frac{\partial l^{(\alpha )}(p(\zeta |\theta ))}{\partial {\theta }^i}\frac{\partial l^{(-\alpha )}(p(\zeta |\theta ))}{\partial {\theta }^j}\right\},$$

(7)

$${\mathrm{\Gamma }}_{ij,k}^{(\alpha )}(\theta )={\mathrm{E}}_{\mu }\left\{\frac{{\partial }^2{l}^{(\alpha )}(p(\zeta |\theta ))}{\partial {\theta }^i\partial {\theta }^j}\frac{\partial l^{(-\alpha )}(p(\zeta |\theta ))}{\partial {\theta }^k}\right\}.$$

(8)

A variance of α-embedding of a probability function plays an important role in Tsallis statistics; see [12–14]. On the geometric side, [15,16] illuminated that the α-scaling of the probability functions leads to a conformal transformation.

1.2. Zhang (2004) [2] Extension: ρ-Embedding and (ρ, τ)-Geometry

Zhang [2,4,6] obtained generalizations of the α-geometry for a pair of monotone embeddings, called ρ- and τ-embeddings generalizing α-embedding. Given any smooth strictly convex function f : ℝ → ℝ, with convex conjugate f^* given by:

$$f^{*}\left(t\right)=t{\left(f\prime \right)}^{-1}\left(t\right)-f\left({\left(f\prime \right)}^{-1}\left(t\right)\right),$$

(9)

Zhang (2004) defines a pair of conjugate representations [2] (Section 3.2) using two strictly increasing functions ρ, τ from ℝ → ℝ:

• we call ρ-representation of a probability function p the mapping p ↦ ρ(p);
• we say τ-representation of the probability function p ↦τ(p) is conjugate to ρ-representation with respect to a smooth and strictly convex function f, or simply τ is f-conjugate to ρ, if:

  $$\tau \left(p\right)=f\prime \left(\rho \left(p\right)\right)={\left(\left(f^{*}\right)\prime \right)}^{-1}\left(\rho \left(p\right)\right),$$

  (10)

  which can be equivalently written as:

  $$\rho \left(p\right)={\left(f\prime \right)}^{-1}\left(\tau \left(p\right)\right)={\left(f^{*}\right)}^{\prime }\left(\tau \left(p\right)\right).$$

  (11)

These equalities in (10) and (11) hold, and they are equivalent, because f′ and (f^*)′ are both strictly increasing (due to their strict convexity) and that (f^*)^* = f, (f^*)′ = (f′)⁻¹. Sometimes, we write f′ = σ, (f^*)⁻¹ = σ⁻¹ for convenience, so σ(ρ) = τ, σ⁻¹(τ) = ρ, for a strictly increasing function τ.

As a first example, we may set ρ(t) = t, τ(t) = log t. Then, we can derive that f^*(t) = exp(t) and f(t) = t log t − t + 1. That ρ(p) and τ(p) are just the p and log p representation reflects the conventional dual embeddings that have later been extended to ϕ- and log_ϕ-embedding in ([3]). In Section 2.2, it will be shown that Naudts’ ϕ-logarithm formulation is recovered as a special case of the (ρ, τ)-embedding.

As another example, we may set ρ(p) = l^(β)(p) to be the β-representation given by Equation (6); this would have been traditionally called “alpha-embedding”, except we use the symbol β, so that the α-parameter will be reserved for indexing α-connections. In this case, the conjugate representation is the (−β)-representation τ(p) = l^(−β)(p):

$$\rho \left(p\right)=l^{(\beta )}\left(p\right)\leftrightarrow \tau \left(p\right)=l^{(-\beta )}\left(p\right).$$

(12)

In this case, ρ and τ are conjugate with respect to f, where f is given by:

$$f(t)=\frac{2}{1+\beta }{\left(\left(\frac{1-\beta }{2}\right)t\right)}^{\frac{2}{1-\beta }},f^{*}(t)=\frac{2}{1-\beta }{\left(\left(\frac{1+\beta }{2}\right)t\right)}^{\frac{2}{1+\beta }}.$$

(13)

Based on divergence functions constructed under monotone embedding, Zhang ([2]) showed:

Proposition 1. ([2], Proposition 7) Using an arbitrary monotone embedding function ρ and an arbitrary smooth strictly convex function f, a generalization of α-geometry is obtained, with metric and α-connections taking the form:

$$g_{ij}(\theta )=E_{\mu }\left\{f^{″}(\rho (p(\zeta |\theta ))\frac{\partial \rho (p(\zeta |\theta ))}{\partial {\theta }^i}\frac{\partial \rho (p(\zeta |\theta ))}{\partial {\theta }^j}\right\}$$

(14)

$${\mathrm{\Gamma }}_{ij,k}^{(\alpha )}(\theta )=E_{\mu }\left\{\frac{1-\alpha }{2}{f}^{‴}(\rho (p(\zeta |\theta )))A_{ijk}+f^{″}(\rho (p(\zeta |\theta )))B_{ijk}\right\},$$

(15)

where:

$$A_{ijk}(\zeta ,\theta )=\frac{\partial \rho (p(\zeta |\theta ))}{\partial {\theta }^i}\frac{\partial \rho (p(\zeta |\theta ))}{\partial {\theta }^j}\frac{\partial \rho (p(\zeta |\theta ))}{\partial {\theta }^k},B_{ijk}(\zeta ,\theta )=\frac{{\partial }^2\rho (p(\zeta |\theta ))}{\partial {\theta }^i\partial {\theta }^j}\frac{\partial \rho (p(\zeta |\theta ))}{\partial {\theta }^k}.$$

(16)

As special cases,

$${\mathrm{\Gamma }}_{ij,k}(\theta )={\mathrm{E}}_{\mu }\left\{f^{″}(\rho (p))\frac{{\partial }^2\rho (p)}{\partial {\theta }^i\partial {\theta }^j}\frac{\partial \rho (p)}{\partial {\theta }^k}\right\},$$

(17)

$${\mathrm{\Gamma }}_{ij,k}^{*}(\theta )={\mathrm{E}}_{\mu }\left\{\frac{\partial \rho (p)}{\partial {\theta }^k}\left(f^{‴}(\rho (p))\frac{\partial \rho (p)}{\partial {\theta }^i}\frac{\partial \rho (p)}{\partial {\theta }^j}+f^{″}(\rho (p))\frac{{\partial }^2\rho (p)}{\partial {\theta }^i\partial {\theta }^j}\right)\right\}.$$

(18)

Furthermore, taking a pair of monotone representations, the metric tensor and affine connections stated in Proposition 1 have dualistic expressions:

Corollary 1. ([2], Proposition 8) Using two arbitrary monotone embedding functions ρ and τ, the metric and α-connections of (14)–(16) are:

$$g_{ij}(\theta )=E_{\mu }\left\{\frac{\partial \rho (p(\zeta |\theta ))}{\partial {\theta }^i}\frac{\partial \tau (p(\zeta |\theta ))}{\partial {\theta }^j}\right\},$$

(19)

$${\mathrm{\Gamma }}_{ij,k}^{(\alpha )}(\theta )=E_{\mu }\left\{\frac{1-\alpha }{2}\frac{{\partial }^2\tau (p(\zeta ,\theta ))}{\partial {\theta }^i\partial {\theta }^j}\frac{\partial \rho (p(\zeta |\theta ))}{\partial {\theta }^k}+\frac{1+\alpha }{2}\frac{{\partial }^2\rho (p(\zeta |\theta ))}{\partial {\theta }^i\partial {\theta }^j}\frac{\partial \tau (p(\zeta |\theta ))}{\partial {\theta }^k}\right\}.$$

(20)

As a special case, when ρ, τ take the familiar alpha-embeddings (12) (using β as the parameter), the α-connections becomes (αβ)-connections:

$${\mathrm{\Gamma }}_{ij,k}^{(\alpha )}(\theta )={\mathrm{E}}_{\mu }\left\{\left(\frac{1-\alpha \beta }{2}\frac{\partial \log p(\zeta |\theta )}{\partial {\theta }^i}\frac{\partial \log p(\zeta |\theta )}{\partial {\theta }^j}+\frac{{\partial }^2\log p(\zeta |\theta ))}{\partial {\theta }^i\partial {\theta }^j}\right)\frac{\partial p(\zeta |\theta )}{\partial {\theta }^k}\right\},$$

(21)

with the product α · β playing the role of the alpha-parameter indexing the family of connections.

1.3. Harsha and Subrahamanian Moosath’s (2014) Work [1]

Using a monotone embedding function denoted as F and a weighting function denoted as G (G = 1 is a special case to reduce to what they called F -geometry), these authors [1] proposed (F, G)-metric as (their Equation (33) in [1]):

$$g_{ij}^{F,G}={\mathrm{E}}_{\mu }\left\{pG(p)\frac{\partial \log p}{\partial {\theta }^i}\frac{\partial \log p}{\partial {\theta }^j}\right\}$$

(22)

with affine connection given as (their Equation (34)):

$${\mathrm{\Gamma }}_{ijk}^{F,G}={\mathrm{E}}_{\mu }\left\{pG(p)\frac{\partial \log p}{\partial {\theta }^k}\left(\frac{{\partial }^2\log p}{\partial {\theta }^i\partial {\theta }^j}+\left(1+\frac{p{F}^{″}(p)}{F^{\prime }(p)}\right)\frac{\partial \log p}{\partial {\theta }^i}\frac{\partial \log p}{\partial {\theta }^j}\right)\right\}.$$

(23)

Note E_p{(·)} = E_µ{(·)p}. (23) is the expression for the e-connection (α = 1), ${\mathrm{\Gamma }}_{ijk}^{F,G}$. To express the conjugate connection (m-connection, α = −1), ${\mathrm{\Gamma }}_{ijk}^{H,G}$, a dual embedding function H is introduced, which is shown ([1], Theorem 3.2) to be related to F and G via (their Equation (36)):

$$H^{\prime }(p)=\frac{G(p)}{p{F}^{\prime }(p)}.$$

(24)

In such a case, the conjugate connection ${\mathrm{\Gamma }}_{ijk}^{H,G}$ (sic, more accurately ${\left({\mathrm{\Gamma }}^{F,G}\right)}_{ijk}^{*}$) is expressed as (their Equation (37)):

$${\mathrm{\Gamma }}_{ijk}^{H,G}={\mathrm{E}}_{\mu }\left\{pG(p)\frac{\partial \log p}{\partial {\theta }^k}\left(\frac{{\partial }^2\log p}{\partial {\theta }^i\partial {\theta }^j}+\left(\frac{p{G}^{\prime }(p)}{G(p)}-\frac{p{F}^{″}(p)}{F^{\prime }(p)}\right)\frac{\partial \log p}{\partial {\theta }^i}\frac{\partial \log p}{\partial {\theta }^j}\right)\right\}.$$

(25)

We now show the equivalence of the three expressions (14), (17), (18) from the work [2] with the three corresponding expressions (22), (23), (25) from the work [1].

Statement 1. Equations (14) and (22) give the same Riemannian metric; Equations (17) and (23) give the same affine connection; and Equations (18) and (25) give the same conjugate connection, as long as:

$$F\left(p\right)=\rho \left(p\right),G\left(p\right)={\left(\rho \prime \right)}^{2}pf″\left(\rho \left(p\right)\right).$$

(26)

Proof. Re-writing (14), and keeping in mind:

$$\frac{\partial \rho (p)}{\partial {\theta }^i}={\rho }^{\prime }(p)\frac{\partial p}{\partial {\theta }^i}=p{\rho }^{\prime }(p)\frac{\partial \log p}{\partial {\theta }^i},$$

(27)

so:

$$g_{ij}(\theta )={\mathrm{E}}_{\mu }\left\{f^{″}{\left(\rho (p)(p{\rho }^{\prime }(p)\right)}^2\frac{\partial \log p}{\partial {\theta }^i}\frac{\partial \log p}{\partial {\theta }^j}\right\}.$$

(28)

Comparing the above with (22), obviously, F is just ρ, and G is linked to f and ρ:

$$G\left(p\right)={\left(\rho \prime \right)}^{2}pf″\left(\rho \left(p\right)\right)=p\rho \prime \left(p\right)\tau \prime \left(p\right)$$

(29)

where we have used (10).

Next, differentiate (27); we obtain:

$$\frac{{\partial }^2\rho (p)}{\partial {\theta }^i\partial {\theta }^j}=\frac{\partial p}{\partial {\theta }^j}{\rho }^{\prime }(p)\frac{\partial \log p}{\partial {\theta }^i}+p{\rho }^{″}(p)\frac{\partial p}{\partial {\theta }^j}\frac{\partial \log p}{\partial {\theta }^i}+p{\rho }^{\prime }(p)\frac{{\partial }^2\log p}{\partial {\theta }^i\partial {\theta }^j}$$

(30)

$$=p{\rho }^{\prime }(p)\left(\frac{\partial \log p}{\partial {\theta }^i}\frac{\partial \log p}{\partial {\theta }^j}+\frac{{\partial }^2\log p}{\partial {\theta }^i\partial {\theta }^j}+\frac{p{\rho }^{″}(p)}{{\rho }^{\prime }(p)}\frac{\partial \log p}{\partial {\theta }^i}\frac{\partial \log p}{\partial {\theta }^j}\right)$$

(31)

$$=p{\rho }^{\prime }(p)\left(\frac{{\partial }^2\log p}{\partial {\theta }^i\partial {\theta }^j}+\left(1+\frac{p{\rho }^{″}(p)}{{\rho }^{\prime }(p)}\right)\frac{\partial \log p}{\partial {\theta }^i}\frac{\partial \log p}{\partial {\theta }^j}\right).$$

(32)

Identifying F = ρ and making use of (29), we see that (17) is precisely (23).

Finally, differentiate (29),

$$G\prime \left(p\right)={\left(\rho \prime \right)}^{2}f″\left(\rho \left(p\right)\right)+{\left(\rho \prime \right)}^{3}p{f}^{‴}\left(\rho \left(p\right)\right)+2\rho \prime \left(p\right)\rho ″\left(p\right)pf″\left(\rho \left(p\right)\right).$$

(33)

Therefore,

$$\frac{p{G}^{\prime }(p)}{G(p)}-\frac{p{F}^{″}(p)}{F^{\prime }(p)}=1+\frac{p{\rho }^{\prime }(p)f^{‴}(\rho (p))}{f^{″}(\rho (p))}+\frac{p{\rho }^{″}(p)}{{\rho }^{\prime }(p)}.$$

(34)

After substituting (34) and (29) into (25) and making use of (31), the expression (18) results.

Statement 2. The conjugate embedding function H is the same as τ. The conjugate connection (25), when expressed using H, has the same form as (23) for${\mathrm{\Gamma }}_{ij,k}^{G,F}$ using F.

Proof. Applying Definition (24) immediately yields H′ = τ′. Therefore, (apart from constant) H(p) = τ(p). Next, we will express (25) explicitly using the conjugate embedding function H (rather than F) and the weighting function G. That is to say, we will simplify the terms in the middle parenthesis of (25):

$$\frac{p{G}^{\prime }(p)}{G(p)}-\frac{p{F}^{″}(p)}{F^{\prime }(p)}=p{\left(\log \frac{G(p)}{F^{\prime }(p)}\right)}^{\prime }=p(\log \left(pH\prime \left(p\right)\right)\prime =p\left(\log p+\log H\prime \left(p\right)\right)\prime$$

(35)

$$=p\left(\frac{1}{p}+\frac{H^{″}(p)}{H^{\prime }}\right)=1+\frac{p{H}^{″}}{H^{\prime }(p)}.$$

(36)

Hence, (25) has the same expression as (23) showing the duality between the embedding function H and the embedding function F.

By Statement 1, starting from F (that is, ρ) and G and imposing conjugacy requirement on the pair of affine connections, one is guaranteed to derive H (that is, τ) as the conjugate embedding function.

From Statements 1 and 2, we conclude that, Harsha and Moosath’s F -embedding [1] replicates the ρ-embedding of Zhang (2004) [2]; the conjugate H-embedding turns out to be identical to τ-embedding of [2]. Contrary to the authors’ claim (Remark 3.7 of [1], p. 2480), (F, G)-geometry is identical to Zhang’s (ρ, τ) geometry [2]. In particular, their F -geometry is recovered by simply choosing f to satisfy f″(t) = 1/(ρ⁻¹(t) (ρ′(ρ⁻¹(t)))²), for a given ρ. The subsequent development in their paper [1], e.g., the definition of the F -affine manifold (their Equation (50)), replicates the definition of ρ-affine manifold in [2] (Section 3.4).

During the review of their manuscript [1] and in subsequent personal communications, these authors argued that they used a different approach: (F, G)-geometry is derived by embedding the manifold into the space of random variables and suitably defining the inner product through using the F -expectation (their Equation (15)) and (F, G)-expectation (their Equation (32)) as a general weighted expectation of a random variable, while Zhang (2004) [2] derived the geometry through constructing a divergence function. This difference, however, is entirely superficial, because the relationship between divergence functions and geometric structure (metric and affine connection) is well-established by Eguchi’s work [17,18] and known to information geometers. Therefore, neither the approach nor the results of Harsha and Moosath’s proposed (F, H, G) extension to Amari’s α-geometry differs from Zhang’s proposed (ρ, τ, f) extension, with the following correspondence in different symbols by the two papers:

$$F\iff \rho ,H\iff \tau ,$$

(37)

$$G\left(t\right)\iff t\rho \prime \left(t\right)\tau \prime \left(t\right)=tf″\left(\rho \left(t\right)\right){\left(\rho \prime \left(t\right)\right)}^2=t(f^{*})″\left(\tau \left(t\right)\right){\left(\tau \prime \left(t\right)\right)}^2;$$

(38)

the difference in the representation of score function as log-representation in [1] or under ρ or τ-representation in [2] is cosmetic.

2. Uniqueness of (ρ, τ)-Geometry and Representation Duality

2.1. Monotone Embedding as a Transformation Group

Monotone representations of any given probability function form a transformation group, with functional composition as group composition operation and the functional inverse as the group inverse operation. This was pointed out by Zhang [6] (Section 2.2.2). We state it as a lemma here.

Lemma 1. Denote Ω as the set of strictly increasing functions from ℝ → ℝ. Then, (Ω, ○) forms a group, with ○ denoting functional composition.

Proof. We easily verify that:

• closure for ○: for any ρ₁, ρ₂ ∈ Ω, ρ₂ ○ ρ₁, defined as ρ₂(ρ₁(·)), is strictly increasing, and hence, ρ₂ ○ ρ₁ ∈ Ω;
• existence of unique identity element: the identity function ι, which satisfies ρ ○ ι = ι ○ ρ = ρ, is strictly increasing, and hence, ι ∈ Ω and is unique;
• existence of inverse: for any ρ ∈ Ω, its functional inverse ρ⁻¹, which satisfies ρ⁻¹○ρ = ρ⁻¹○ρ = ι, is also strictly increasing, and hence, ρ⁻¹ ∈ Ω;
• associativity of ○: for any three ρ₁, ρ₂, ρ₃ ∈ Ω, then (ρ₁ ○ ρ₂) ○ ρ₃ = ρ₁ ○ (ρ₂ ○ ρ₃).

Recall that the derivative of smooth strictly convex functions are strictly increasing functions. From this perspective, f′ = τ ○ ρ⁻¹ = τ(ρ⁻¹(·)), (f*)′ = ρ ○ τ⁻¹ = ρ(τ⁻¹(·)), encountered above, are themselves two mutually inverse strictly increasing functions. This is the rationale behind Zhang’s ([2]) choice of f (and f^*) as the auxiliary function to capture conjugate embedding, rather than using G as in [1]. The following identities are useful; they are obtained by differentiating (10) and (11):

$$f″\left(\rho \left(t\right)\right)\rho \prime \left(t\right)=\tau \prime \left(t\right),(f^{*})″\left(\tau \left(t\right)\right)\tau \prime \left(t\right)=\rho \prime \left(t\right);$$

(39)

therefore:

$$f″\left(\rho \left(t\right)\right){\left(\rho \prime \left(t\right)\right)}^2=(f^{*})″\left(\tau \left(t\right)\right){\left(\tau \prime \left(t\right)\right)}^2,$$

(40)

and:

$$f″\left(\rho \left(t\right)\right)(f^{*})″\left(\tau \left(t\right)\right)=1.$$

(41)

With respect to (41), taking log on both sides yields:

$$\log f″\left(\rho \left(t\right)\right)+\log (f^{*})″\left(\tau \left(t\right)\right)=0.$$

(42)

Move and differentiate:

$$\frac{f^{‴}(\rho (t)){\rho }^{\prime }(t)}{f^{″}(\rho (t))}=-\frac{{(f*)}^{‴}(\tau (t)){\tau }^{\prime }(t)}{{\left(f*\right)}^{\prime \prime }(\tau (t))}.$$

(43)

Making use of (40) yields:

$$f^{‴}\left(\rho \left(t\right)\right){\left(\rho \prime \left(t\right)\right)}^3=-{(f^{*})}^{‴}\left(\tau \left(t\right)\right){\left(\tau \prime \left(t\right)\right)}^3.$$

(44)

Note the coupling between f and ρ, τ given by (10), (11), (40) and (44). They allow us to cast (14) and (15) in terms of f^* and τ.

Among the triple (f, ρ, τ), given any two, the third is specified. In particular, if we arbitrary choose two strictly increasing functions ρ and τ as embedding functions and require them to be conjugate embeddings, then f is specified by f′(t) = τ(ρ⁻¹(t)). In terms of conjugate function f^*, the relation is (f^*)′(t) = ρ(τ⁻¹(t)). The function f (or f^*) is important in constructing the general class of divergence function.

2.2. Naudts’ ϕ-Logarithm as a Special Case

In his 2004 publication [3], Naudts considered the “deformed” logarithm function as an extension to the exponential family of densities that is log-linear. Given a strictly increasing and strictly positive function ϕ : ℝ₊ → ℝ₊, the ϕ-logarithm is defined as:

$${\log }_{\varphi }(t)={\displaystyle {\int }_1^t\frac{1}{\varphi (s)}}ds,(t>0).$$

(45)

The deformed exponential denoted exp_ψ, is defined by:

$${\exp }_{\psi }(t)=1+{\displaystyle {\int }_0^t\psi (s)ds.}$$

(46)

(Naudts (2004) used the notation exp_ϕ, so our current rendition has a subtle difference shown as (48) and (49) below.) It can be shown that the deformed functions log_ϕ and exp_ψ are in fact inverse functions of each other if:

$$\psi \left({\log }_{\varphi }\left(t\right)\right)=\varphi \left(t\right),\psi \left(t\right)=\varphi \left({\exp }_{\psi }\left(t\right)\right).$$

(47)

Stated alternatively, the deformed logarithmic function h(t) = log_ϕ(t) can be viewed as the solution to the following integral and its equivalent differential equation:

$$h(t)={\displaystyle {\int }_1^t\frac{1}{\psi (h(s))}}ds\iff \frac{dh}{dt}=\frac{1}{\psi (h(t))},$$

(48)

whereas the deformed exponential function h(t) = exp_ψ(t) can be viewed as the solution to the following integral and its equivalent differential equation:

$$h(t)=1+{\displaystyle {\int }_0^t\varphi (h(s))ds\iff \frac{dh}{dt}=\varphi (h(t))}.$$

(49)

We now show that the above formulation can be re-written as (ρ, τ)-embeddings with a particular choice of f (or equivalently, f^*) function. Set ϕ(t) = ρ(t) and f^*(t) = exp_ψ(t), so that (f^*)′(t) = ψ(t) from (46). Therefore, we derive:

$${\log }_{\varphi }\left(t\right)={\psi }^{-1}\left(\varphi \left(t\right)\right)={\left({(f^{*})}^{\prime }\right)}^{-1}\left(\rho \left(t\right)\right)=f\prime \left(\rho \left(t\right)\right)=\tau \left(t\right).$$

That is, when ϕ is chosen as ρ-representation, the deformed logarithm log_ϕ turns out to be the τ-representation, while the deformed exponential is nothing but f^*. The relationship (47) is identical to (10) and (11).

In the ϕ-logarithm approach, once ϕ (that is, ρ) is specified, then log_ϕ (that is, τ) is specified, through the integral relation (45). Viewing τ(·) = f′(ρ(·)), the relation (45) essentially specifies a strictly convex function f, through its derivative f′, which operates on ρ.

Proposition 2. Denote ρ ≡ ϕ. The deformed logarithmic transformation ϕ → log_ϕ given by (45) can be viewed as the function composition f′ : ρ → f′(ρ), where f is given by:

$$f\left(\rho \left(t\right)\right)=\rho \left(t\right)f\prime \left(\rho \left(t\right)\right)-t.$$

(50)

Equivalently, using conjugate function f^* given by (9),

$$\rho =(f^{*})\prime \circ {(f^{*})}^{-1},$$

(51)

or

$$\rho =\frac{1}{{\left({\left(f\ast \right)}^{-1}\right)}^{\prime }}.$$

(52)

Proof. From (45), we write:

$$f^{\prime }(\rho (t))={\displaystyle {\int }_1^t\frac{1}{\rho (s)}}ds,$$

(53)

with unknown f. Multiply both sides by ρ′(t) and then integrate from one to x; the left-hand side of (53) is:

$${\displaystyle {\int }_1^x{f}^{\prime }(\rho (t)){\rho }^{\prime }(t)dt={\displaystyle {\int }_1^x{f}^{\prime }(\rho (t))d(\rho (t))}=f(\rho (x))-f(\rho (1))}.$$

The right-hand side of (53), after the same operation, is:

$$\begin{array}{l}{\displaystyle {\int }_1^x{\rho }^{\prime }(t)dt}{\displaystyle {\int }_1^t\frac{1}{\rho (s)}ds={\displaystyle {\int }_1^x\frac{1}{\rho (s)}ds}}{\displaystyle {\int }_s^x{\rho }^{\prime }(t)dt}={\displaystyle {\int }_1^x\frac{\rho (x)-\rho (s)}{\rho (s)}}ds\\ ={\displaystyle {\int }_1^x\left(\frac{\rho (x)}{\rho (s)}-1\right)ds=\rho (x)\left({\displaystyle {\int }_1^x\frac{1}{\rho (s)}ds}\right)-{\displaystyle {\int }_1^{x}ds}=\rho (x)f^{\prime }(\rho (x))-(x-1)}.\end{array}$$

Clearly, f′(ρ(1)) = 0 by (53). We set f(ρ(1)) = −1. Comparing expressions from the left- and right-hand side, we obtain (50).

Applying (9), we obtain the equivalent expression:

$$f^{*}\left(f\prime \left(\rho \left(t\right)\right)\right)=t.$$

That is, f is chosen, such that f^* ○ f′ is the inverse function of ρ, or:

$$\rho ={\left(f^{*}\circ f\prime \right)}^{-1}={\left(f\prime \right)}^{-1}\circ {\left(f^{*}\right)}^{-1}={\left(f^{*}\right)}^{\prime }\circ {\left(f^{*}\right)}^{-1}.$$

Hence, (51) holds.

Finally, differentiate the identity:

$$f^{*}\left({\left(f^{*}\right)}^{-1}\left(t\right)\right)=t,$$

we obtain:

$$1={\left(f^{*}\right)}^{\prime }\left({\left(f^{*}\right)}^{-1}\left(t\right)\right)·{\left(f^{*}\right)}^{-1}\left(t\right)=\rho \left(t\right)·{\left(f^{*}\right)}^{-1}\left(t\right)$$

upon substituting (51). Hence, (52) holds.

The expression (51) in Proposition 2 shows that for any ρ, if one can find a decomposition: ρ = g′○g⁻¹ in terms of g, then g would be the ρ-exponential, g⁻¹ the ρ-logarithm and g′ the linking function. In the case of ϕ ↦ log_ϕ transformation, g = f^*(t).

Naudts’ ([3]) deformed logarithm/exponential embedding approach and Zhang’s ([2]) (ρ, τ)-embedding approach can be seen as playing complementary roles in information geometry: the former makes it easy to generalize the exponentiation and logarithm as inverse operations obeying desired differential/integral equations, while the latter makes it apparent how conjugate (ρ, τ)-embeddings lead to bidualistic expressions for the underlying geometric structures (metric and conjugate connections).

2.3. Uniqueness of (ρ, τ)-Geometry

It is known [19,20] that the Fisher–Rao metric and α-connections (equivalently, Amari–Chentsov tensor T) are the only invariants of sufficient statistics under the Markov morphism of a random variable. In [22,23], the Fisher–Rao metric has been extended to allow a weighting function. In [2,6], general weighting functions for affine connections were made compatible with the generalized (i.e., weighted) Fisher–Rao metric, since they result from divergence functions that are allowed to have the freedom of monotone embedding. The recent reinvention [1] constructed weighted connections that turned out to be identical to the expressions given by [2]. A natural question is, then, whether Zhang’s (ρ, τ) geometry is the unique construction given the freedom of arbitrary monotone embedding. Below, arguments will be provided, along with a proof, for a positive answer to this question.

First, when a probability function p(ζ|θ) (as a function of a random variable indexed by ζ and a background measure of µ) is embedded into the parametric manifold $ℳ$_Θ, there are several traditional choices for tangent vectors: ∂_ip, ∂_i log p, ${\partial }_i\sqrt{p}$, etc. Each of these are linked with a weighting function (expectation operator), so that the tangent vectors are zero-mean random variables:

$$0={\mathrm{E}}_{\mu }\left\{{\partial }_{i}p\right\}={\mathrm{E}}_{\mu }\{(p){\partial }_i\log p\}={\mathrm{E}}_{\mu }\left\{\left(\sqrt{p}\right){\partial }_i\left(\sqrt{p}\right)\right\}=\cdots$$

(54)

where the weighting functions are, respectively, one, $p,\sqrt{p}$:

$$0={\mathrm{E}}_{\mu }\left\{{\partial }_{i}p\right\}={\mathrm{E}}_p\left\{{\partial }_i\log p\right\}={\mathrm{E}}_{\sqrt{p}}\left\{{\partial }_i\left(\sqrt{p}\right)\right\}=\cdots$$

For these various choices, the direction of the tangent vectors are all the same. We can consider the above as special cases of ρ-embedding, with ρ(t) = t, log t, $\sqrt{t}$, respectively. Because ∂_i(ρ(p)) = ρ′(p)∂_ip, so a tangent vector retains its direction with any choice of monotone embedding function.

To investigate the weighting function for general monotone ρ-embedding, let us consider the f-normalization (foliation) condition, cf. [21],

$${\mathrm{E}}_{\mu }\{f(\rho (p)\}=1,$$

(55)

where f is a given convex function. Differentiate the above; we get:

$$0={\mathrm{E}}_{\mu }\left\{f^{\prime }(\rho (p))\frac{\partial {\rho }^{\prime }(p)}{\partial {\theta }^i}\right\}={\mathrm{E}}_{\mu }\{\tau (p){\partial }_i\rho \}.$$

(56)

Therefore, we can see that τ(p) = f′(ρ(p)), what we have called the f-conjugate of ρ, is precisely the weighting function to make ∂_iρ a zero-mean random function at any point of $ℳ$_Θ (i.e., for any value of θ ∈ Θ).

Next, consider the Fisher–Rao metric (1), which can be written as E_µ{∂_ip ∂_j log p} = E_µ{∂_i log p ∂_jp}, the pairing of a random function with a random functional under two embeddings p and log p. A natural generalization (see [6]) is to use two (independently chosen) monotone embeddings ρ, τ:

$$g_{ij}\left(\theta \right)={\mathrm{E}}_{\mu }\left\{{\partial }_i\rho {\partial }_j\tau \right\}={\mathrm{E}}_{\mu }\left\{{\partial }_j\rho {\partial }_i\tau \right\}={\mathrm{E}}_{\mu }\{\rho \prime \left(p\right)\tau \prime \left(p\right){\partial }_{i}p{\partial }_{j}p\}.$$

(57)

This is precisely (14), with the weighting function for the Riemannian metric as f″(ρ(p))(ρ′(p))² = τ′(p)ρ′(p), when tangent vectors are expressed as ∂_ip (identity representation). When ρ-representation or τ-representation is adopted, the weighting function is simply f″(ρ(p)) or (f^*)″(τ(p)), respectively.

Third, given ρ, τ embedding, we can construct two affine connections on the manifold as follows. Differentiate (57),

$$\frac{\partial g_{ij}(\theta )}{\partial {\theta }^k}={\mathrm{E}}_{\mu }\left\{\frac{{\partial }^2\rho (p)}{\partial {\theta }^k\partial {\theta }^i}\frac{\partial \tau }{\partial {\theta }^j}+\frac{{\partial }^2\tau (p)}{\partial {\theta }^k\partial {\theta }^j}\frac{\partial \rho (p)}{\partial {\theta }^i}\right\},$$

(58)

and compare with the relation that defines conjugate connections:

$$\frac{\partial g_{ij}(\theta )}{\partial {\theta }^k}={\mathrm{\Gamma }}_{ki,j}(\theta )+{\mathrm{\Gamma }}_{kj,i}^{*}(\theta );$$

(59)

we can identify:

$${\mathrm{E}}_{\mu }\left\{\frac{{\partial }^2\rho (p)}{\partial {\theta }^k\partial {\theta }^i}\frac{\partial \tau (p)}{\partial {\theta }^j}\right\}$$

(60)

with Γ_ki,j and:

$${\mathrm{E}}_{\mu }\left\{\frac{{\partial }^2\tau (p)}{\partial {\theta }^k\partial {\theta }^i}\frac{\partial \rho (p)}{\partial {\theta }^j}\right\}$$

(61)

with ${\mathrm{\Gamma }}_{kj,i}^{*}$, respectively. Their difference is, by definition, the Amari–Chentsov (0,3)-tensor T:

$$T_{ijk}(\theta )\equiv {\mathrm{E}}_{\mu }\left\{\frac{{\partial }^2\tau (p)}{\partial {\theta }^i\partial {\theta }^j}\frac{\partial \rho (p)}{\partial {\theta }^k}-\frac{{\partial }^2\rho (p)}{\partial {\theta }^i\partial {\theta }^j}\frac{\partial \tau (p)}{\partial {\theta }^k}\right\}.$$

(62)

Proposition 3. T as given by (62) is a totally symmetric (0,3)-tensor.

Proof. First, we prove that T (θ) is totally symmetric:

$$T_{ijk}=T_{jik}=T_{ikj}=T_{jki}=T_{kij}=T_{kji}.$$

(63)

Since (62) clearly implies T_ijk = T_jik, we only need to establish T_ijk = T_ikj. Applying the chain-rule of differentiation,

$$\frac{\partial }{\partial {\theta }^i}\left(\frac{\partial \tau (p)}{\partial {\theta }^j}\frac{\partial \rho (p)}{\partial {\theta }^k}\right)=\frac{{\partial }^2\tau (p)}{\partial {\theta }^i\partial {\theta }^j}\frac{\partial \rho (p)}{\partial {\theta }^k}+\frac{{\partial }^2\rho (p)}{\partial {\theta }^i\partial {\theta }^k}\frac{\partial \tau (p)}{\partial {\theta }^j},$$

(64)

$$\frac{\partial }{\partial {\theta }^i}\left(\frac{\partial \rho (p)}{\partial {\theta }^j}\frac{\partial \tau (p)}{\partial {\theta }^k}\right)=\frac{{\partial }^2\rho (p)}{\partial {\theta }^i\partial {\theta }^j}\frac{\partial \tau (p)}{\partial {\theta }^k}+\frac{{\partial }^2\tau (p)}{\partial {\theta }^i\partial {\theta }^k}\frac{\partial \rho (p)}{\partial {\theta }^j},$$

(65)

and taking into account:

$$\frac{\partial \tau (p)}{\partial {\theta }^j}\frac{\partial \rho (p)}{\partial {\theta }^k}=\frac{\partial \tau (p)}{\partial {\theta }^k}\frac{\partial \rho (p)}{\partial {\theta }^j}={\tau }^{\prime }(p){\rho }^{\prime }(p)\frac{\partial p}{\partial {\theta }^j}\frac{\partial p}{\partial {\theta }^k},$$

(66)

(62) becomes:

$$T_{ijk}(\theta )={\mathrm{E}}_{\mu }\left\{-\left(\frac{{\partial }^2\rho (p)}{\partial {\theta }^i\partial {\theta }^k}\frac{\partial \tau (p)}{\partial {\theta }^j}-\frac{{\partial }^2\tau (p)}{\partial {\theta }^i\partial {\theta }^k}\frac{\partial \rho (p)}{\partial {\theta }^j}\right)\right\}=T_{ikj}(\theta ).$$

(67)

Next, we prove that T_ijk is indeed a (0,3)-tensor. This is done through examining the behavior of T under a coordinate transform $\theta \mapsto \overline{\theta }$, with the (inverse) Jacobian matrix $\frac{\partial {\theta }^k}{\partial {\overline{\theta }}^l}$, which affects:

$$\frac{\partial \rho (p)}{\partial {\overline{\theta }}^i}={\displaystyle \sum _l\frac{\partial \rho (p)}{\partial {\theta }^l}}\frac{\partial {\theta }^l}{\partial {\overline{\theta }}^i},\frac{\partial \tau (p)}{\partial {\overline{\theta }}^i}={\displaystyle \sum _l\frac{\partial \tau (p)}{\partial {\theta }^l}}\frac{\partial {\theta }^l}{\partial {\overline{\theta }}^i},$$

(68)

and:

$$\frac{{\partial }^2\rho (p)}{\partial {\overline{\theta }}^i\partial {\overline{\theta }}^j}={\displaystyle \sum _{l,m}\frac{{\partial }^2\rho (p)}{\partial {\theta }^l\partial {\theta }^m}}\frac{\partial {\theta }^l}{\partial {\overline{\theta }}^i}\frac{\partial {\theta }^m}{\partial {\overline{\theta }}^j}+{\displaystyle \sum _l\frac{\partial \rho (p)}{\partial {\overline{\theta }}^l}}\frac{{\partial }^2{\theta }^l}{\partial {\overline{\theta }}^i\partial {\overline{\theta }}^j},$$

(69)

$$\frac{{\partial }^2\tau (p)}{\partial {\overline{\theta }}^i\partial {\overline{\theta }}^j}={\displaystyle \sum _{l,m}\frac{{\partial }^2\tau (p)}{\partial {\theta }^l\partial {\theta }^m}}\frac{\partial {\theta }^l}{\partial {\overline{\theta }}^i}\frac{\partial {\theta }^m}{\partial {\overline{\theta }}^j}+{\displaystyle \sum _l\frac{\partial \tau (p)}{\partial {\overline{\theta }}^l}}\frac{{\partial }^2{\theta }^l}{\partial {\overline{\theta }}^i\partial {\overline{\theta }}^j}.$$

(70)

Therefore:

$${\overline{T}}_{ijk}(\overline{\theta })\equiv {\mathrm{E}}_{\mu }\left\{\frac{{\partial }^2\tau (p)}{\partial {\overline{\theta }}^i\partial {\overline{\theta }}^j}\frac{\partial \rho (p)}{\partial {\overline{\theta }}^k}-\frac{{\partial }^2\rho (p)}{\partial {\overline{\theta }}^i\partial {\overline{\theta }}^j}\frac{\partial \tau (p)}{\partial {\overline{\theta }}^k}\right\}={\displaystyle \sum _{lmn}\frac{\partial {\theta }^i}{\partial {\overline{\theta }}^l}}\frac{\partial {\theta }^j}{\partial {\overline{\theta }}^m}\frac{\partial {\theta }^k}{\partial {\overline{\theta }}^n}{T}_{lmn}(\theta ).$$

(71)

after substituting (69), (70) and (62). T indeed transforms to $\overline{T}$ in a manner that defines a (0, 3)-tensor. Therefore, the proposition is proven.

We now cast the Amari–Chentsov tensor T in an alternative form that gives an explicit form of weighting function. Given ρ, τ, because of Lemma 1, there exists another monotone embedding σ, such that σ(ρ) = τ. Differentiating,

$$\frac{\partial \sigma (\rho (p))}{\partial {\theta }^i}={\sigma }^{\prime }(\rho (p))\frac{\partial \rho (p)}{\partial {\theta }^i}.$$

(72)

Differentiate again, we obtain:

$$\frac{{\partial }^2\sigma (\rho (p))}{\partial {\theta }^i\partial {\theta }^j}={\sigma }^{″}(\rho (p))\frac{\partial \rho (p)}{\partial {\theta }^j}\frac{\partial \rho (p)}{\partial {\theta }^j}+{\sigma }^{\prime }(\rho (p))\frac{{\partial }^2\rho (p)}{\partial {\theta }^i\partial {\theta }^j}.$$

(73)

Substituting the above into (62), we obtain an expression of T in terms of ρ (which plays the role of embedding function) and σ (which plays the role of weighting function):

$$T_{ijk}(\theta )={\mathrm{E}}_{\mu }\left\{{\sigma }^{″}(\rho (p))\frac{\partial \rho (p)}{\partial {\theta }^i}\frac{\partial \rho (p)}{\partial {\theta }^j}\frac{\partial \rho (p)}{\partial {\theta }^k}\right\}.$$

(74)

Similarly, we can obtain:

$$T_{ijk}(\theta )=-{\mathrm{E}}_{\mu }\left\{{\left({\sigma }^{-1}\right)}^{″}(\tau (p))\frac{\partial \tau (p)}{\partial {\theta }^i}\frac{\partial \tau (p)}{\partial {\theta }^j}\frac{\partial \tau (p)}{\partial {\theta }^k}\right\}.$$

(75)

Therefore, under τ-representation, σ⁻¹ (the inverse function of σ) serves as the weighting function. Note that σ = f′, σ⁻¹ = (f^*)′ when ρ and τ are said to be conjugate. Furthermore, note the negative sign in (75) compared with (74); this precisely reflects “representation duality” with a ρ ↔ τ exchange.

To summarize, because α-geometry {$ℳ$, g, T} is uniquely specified given a Riemannian metric g and the Amari–Chentsov tensor T, the above derivations show that they both enjoy the freedom of two monotone/convex functions, with the freedom in specifying g coupled to the freedom in specifying T in the same way that the metric and connections are coupled via Codazzi relation for statistical manifolds. That the weighting functions used to construct linear, symmetric bilinear and totally symmetric trilinear functionals (on random functions) turns out to be f′(ρ(·)), f″(ρ(·)), f″′(ρ(·)), respectively, is noteworthy. See [6] for more discussions.

2.4. Representation Duality versus Reference Duality

Going beyond extending α-embedding to dual monotonic embeddings, Reference [2] illuminated two different senses of duality in the α-geometry. Prior to [2], there have been several different usages of α-parameter in Amari’s theory of information geometry [10,11]:

• parameterizing the divergence functions (α-divergences);
• parameterizing monotone embedding of probability functions (α-embedding);
• parameterizing the convex mixture of connections (α-connections).

Zhang (2004) [2] showed that (1) and (2) reflect two different types of duality in information geometry, with (1) concerning the reference/comparison status of a pair of points (functions) expressed in the divergence function (“reference duality”) and (2) concerning their representation under arbitrary monotone scaling (“representation duality”). Both can lead to (3), the family of α-connections. Therefore, care has to be taken in carefully delineating these two kinds of duality; for instance, the αβ-connection we derived in (21) reflects how reference duality and representation duality interacts in the alpha-connections.

The present analysis elaborated representation duality in information geometry by working out the freedom in allowing two (independently chosen) embedding functions ρ, τ or, equivalently, one embedding function ρ along with a weighting function f, while the (ρ, f) pair can be dually chosen to be the (τ, f^*) pair. Naudts’ (2004) [3] ϕ-logarithm is but a special case of the (ρ, τ) duality, in which f′ plays the role of the “integral-of-the-reciprocal” operation, that is taking the log of a function. This linkage then leads to f^* and τ as inverse functions. The phenomena of biduality emerges when exchanging ρ ↔ τ or (ρ, f) ↔ (τ, f^*) leads to invariance of the Riemannian metric, but switches the two connections (the latter half of the statement is equivalent to changing signs of the Amari–Chentsov tensor). Therefore, the present paper, while elaborating the theory developed in [2], re-asserts the distinction between two distinct kinds of duality that was originally confounded in Amari’s theory of α-geometry, one through the freedom of selecting monotone embedding functions (“representation duality”) and the other through the freedom of assigning referential status to points for pair comparison (“reference duality”).

Finally, it is noted that the (bi)dualistic structure of the (ρ, τ)-geometry (generalizing α-geometry) is preserved in the non-parametric (infinite-dimensional) setting, as well [4,6], with the α-connection structure cast in a more general way. Theorem 1 of [4] gives non-parametric expressions of the metric and connections under monotone embedding, mirroring the forms (14) and (15) in the parametric case.

3. Conclusion

The Riemannian metric with the pair of conjugate connections derived by Harsha and Moosath [1] are identical to the (ρ, τ)-geometry obtained by Zhang in [2]. The (ρ, τ)-embedding also recovers Naudts’ deformed logarithm/exponential formulation. It is further shown in this paper that such (ρ, τ)-geometry obtained is, when α-embedding is relaxed to arbitrary monotone embeddings, the unique extension of Amari’s α-geometry in terms of its representational freedom.