4.1.3. Starting on the Constraint Surface

When searching for the isotropic boundary condition, interesting paths occur if we start with an initial isotropic distribution, but require the mean vector to change. That is, if the initial distribution already resides on the terminal constraint surface, it would seem counter-intuitive if the geodesic is compelled to leave this constraint. However, as seen in Figure 3, this is exactly what happens.

Employing the same modeling equations as the previous example, we define the initial distribution as

$$
\mu\_0 = [-3, 3], \quad \Sigma\_0 = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} \tag{31}
$$

which is already isotropic. We search for the closest final distribution that is isotropic but with a mean vector *μ*<sup>1</sup> = [3, −4]. If we track evolution of just the mean vector, it starts in Quadrant II and moves to a distribution in Quadrant IV. Qualitatively, the initial and final distributions are geometrically symmetric. However, the isotropic uncertainty is considerably different.

The insights gleaned from this example shed new light into information evolution. Naively, one would think the shortest path would be one that maintains its current shape and just moves along the constraint surface to reach the desired *μ*. However, as shown in Figure 3, this is not the case. In fact, intermediate distributions obtain covariance matrices with *σ*<sup>12</sup> < 0, as demonstrated in Figure 3c. Instead of just staying on the constraint surface, the distributions stretch in the direction of the desired mean, which explains negative values of the covariance between the variables. This elongation of the covariance in the direction of final mean vector suggests that the information metric prefers uncertainty reduction in regions with few plausible solutions. Instead, the distributions along the geodesic "reach" for their destination, i.e., the initial mean vector in Quadrant II and final in Quadrant IV. This behavior is illustrated by the intermediate (red) ellipse in Figure 4.

**Figure 3.** Above are evolutions of the geodesic from an initial isotropic distribution to a final isotropic distribution with a different mean vector. In (**a**), the values of all five parameters are shown at each iteration. Figure (**b**) highlights the values of *σ*<sup>12</sup> showing that it leaves the constraint surface and acquires negative values. The individual variances of the variables also temporarily abandon their required isotropicity as seen in (**c**). In (**c**), the dotted line shows the path of the *σ*<sup>2</sup> <sup>1</sup> and *<sup>σ</sup>*<sup>2</sup> <sup>2</sup> and the solid line shows the isotropic constraint surface.

**Figure 4.** The ellipses above illustrate uncertainty contour for three different density functions along the geodesic. The initial distribution shown in the top left and final distribution in the bottom right are isotropic. The uncertainty evolution is clearly visible in the intermediate distribution which acquires negative covariance values as it "reaches" towards the final distribution.

To reiterate this insightful behavior of information flow, a second example was conducted with a mean vector that starts in Quadrant III, *μ*<sup>0</sup> = [−3, −3], and seeks out a final mean vector in Quadrant I, *μ*<sup>1</sup> = [3, 4], as shown in Figure 5. The initial distribution is still isotropic and the requirement to end isotropic remains. However, as seen in Figure 6, the values of *σ*<sup>12</sup> acquire positive values along the geodesic. The path of the main diagonal variances remains unchanged.

**Figure 5.** Similar to Figure 4, uncertainty level curves of three densities along the geodesic are shown. This time, the intermediate distribution acquires *σ*<sup>12</sup> > 0 along the geodesic as the distributions move from Quadrant III to Quadrant I along the dotted path.

**Figure 6.** *Cont*.

**Figure 6.** Illustration of geodesic paths from an initial isotropic distribution with a mean vector in Quadrant III and a final isotropic distribution with a mean vector Quadrant I. In (**a**), the evolutions of all five parameters are shown. Figure (**b**) highlights the values of *σ*<sup>12</sup> showing that it leaves the constraint surface and acquires positive values in contrast to the previous example. The individual variances of the variables also temporarily abandon their required isotropicity as seen in Figure 3c. In (**c**), the dotted line shows the path of the *σ*<sup>2</sup> <sup>1</sup> and *<sup>σ</sup>*<sup>2</sup> <sup>2</sup> , with the solid red line representing the isotropic constraint surface.

## *4.2. Initial and Terminal Variable-Endpoint Conditions*

It is possible to place transversality conditions on both the initial and final boundaries, with each being entirely independent of the other. Essentially, we are searching for a geodesic between two almost unknown distributions, with only minimal knowledge about the constraint set characterizing the initial and final hypersurfaces.

We consider the example where the final distribution is prescribed to be isotropic as before, but now the initial distribution must have a mean vector with equal components. This enforcement has the practical benefit of reducing the parameter dimensionality of allowable distributions. The problem is formulated as

min <sup>F</sup>[*θ*] = <sup>1</sup> 2 *x*<sup>1</sup> *x*0 ˙ *θTg*(*θ*) ˙ *θdx θ*<sup>0</sup> = *φ*0(*μ*0, Σ0), *θ*<sup>1</sup> = *φ*1(*μ*1, Σ1) (32)

where *φ*<sup>0</sup> and *φ*<sup>1</sup> represent the initial and final transversality constraint surfaces, such that

$$
\phi\_0(\mu\_{01}, \mu\_{02}) = \mu\_{01} - \mu\_{02} = 0 \quad \text{and} \quad \phi\_1(\sigma\_1^2, \sigma\_2^2) = \sigma\_1^2 - \sigma\_2^2 = 0. \tag{33}
$$

To demonstrate this concretely, the initial distribution with unknown mean vector is prescribed with the following covariance matrix

$$
\Sigma\_0 = \begin{bmatrix} 10 & 0 \\ 0 & 2 \end{bmatrix}. \tag{34}
$$

Similarly, the unknown isotropic terminal distribution is given the mean vector *μ*<sup>1</sup> = [−3, 13].

Using Equation (17), the constraint requiring the initial distribution to reside on *φ*<sup>0</sup> imposes that the geodesic satisfy

$$(\sigma\_2^2 - \sigma\_{12})\dot{\mu}\_1 + (\sigma\_1^2 - \sigma\_{12})\dot{\mu}\_2 = 0. \tag{35}$$

As shown in Figure 7, the unknown initial mean vector satisfying the *φ*<sup>1</sup> is *μ*<sup>0</sup> = [7.4, 7.4] and the final isotropic distribution has *σ*<sup>2</sup> <sup>1</sup> = *<sup>σ</sup>*<sup>2</sup> <sup>2</sup> = 26.4. The behavior of the geodesic under these variable-endpoint conditions is shown in Figure 8.

**Figure 7.** Alternate visualization of the resulting geodesic when both endpoints are allowed to vary. The uncertainty contour ellipses of the initial, intermediate and final distribution are also shown. The initial distribution represented by the bottom right ellipse, has components of the mean vector that are equal. The final distribution, at the top left of the path, isotropic.

**Figure 8.** Illustration of geodesics resulting from initial and terminal variable-endpoint boundary conditions. Figure (**a**) shows the behavior of all parameters along the geodesic. Figure (**b**) shows how the geodesic (dashed) evolves the *initial* distribution to reach the constraint (solid) surface defined by the means. Similarly, Figure (**c**) shows the evolution of *final* distribution to the isotropic covariance constraint.

#### **5. Conclusions**

In this work, we have explored new formulations for working on information geometric manifolds. Previous and contemporary work using the Riemannian geometry of statistical manifolds has focused on establishing geodesics between two fixed-endpoint distributions. Here, by employing techniques from the calculus of variations, we have developed constructions that allow variable endpoints that are prescribed by a constraint set rather than fixed points.

These transversality conditions on initial and final distributions, enable new insights into how information evolves under different constraint use cases. Though this present effort focused on just a small variety of constraints on Gaussian manifolds, this approach can be readily extended to other statistical families. This novel approach of relaxing fixed endpoints and moving constraint sets has the potential to impact several application domains that employ information geometric models. In future research, we plan to recast the problem of optimal distribution discovery, in areas such as model selection and domain adaptation, using the presented framework, which allows for greater expressive power. We also plan to investigate observational data scenarios where parameters must be estimated and impacts uncertainty modeling of the manifold parameters.

**Author Contributions:** Conceptualization: T.H. and A.M.P.; Investigation: T.H. and A.M.P.; Formal Analysis: T.H. and A.M.P.; Methodology: T.H. and A.M.P.; Validation and Software: T.H. and A.M.P.; Original Draft Preparation: T.H.; Writing (Review and Editing): T.H. and A.M.P. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest.
