Manifold Calculus in System Theory and Control—Fundamentals and First-Order Systems

Abstract

The aim of the present tutorial paper is to recall notions from manifold calculus and to illustrate how these tools prove useful in describing system-theoretic properties. Special emphasis is put on embedded manifold calculus (which is coordinate-free and relies on the embedding of a manifold into a larger ambient space). In addition, we also consider the control of non-linear systems whose states belong to curved manifolds. As a case study, synchronization of non-linear systems by feedback control on smooth manifolds (including Lie groups) is surveyed. Special emphasis is also put on numerical methods to simulate non-linear control systems on curved manifolds. The present tutorial is meant to cover a portion of the mentioned topics, such as first-order systems, but it does not cover topics such as covariant derivation and second-order dynamical systems, which will be covered in a subsequent tutorial paper.

1. Introduction

The theory of dynamical systems whose state spaces possess the structures of curved manifolds has been applied primarily in physics (especially to mathematically describe the theory of general relativity at the beginning of the 20th century). More recently, dynamical systems on manifolds have proven their relevance in a number of subjects in engineering and applied sciences such as in robotics and in biomedical engineering [1,2,3,4,5] to name a few. The observation at the core of such applications is that those dynamical systems whose descriptive variables are bound to one another by non-linear holonomic constraints may be studied by means of the rich variety of mathematical tools provided by manifold calculus (a shortened terminology to denote ’calculus on manifold’) and may be framed in the class of dynamical systems on manifold.

The structures of the state manifolds of such dynamical systems depend on the application at hand. As a concrete example, whenever the dynamics of a rigid body is concerned, the state manifold of choice is the special orthogonal group $\mathbb{SO}\left(3\right)$, because it encodes the attitude of a flying drone or a submarine robot. Most applications of interest (such as the computation of the dynamics of flying bodies) concern well-studied and well-understood curved manifolds, such as the special orthogonal group, the unit hypersphere and the group of symmetric, positive-definite matrices.

From the perspective of performing numerical simulations of dynamical systems on a computing platform, it is necessary to design adequate numerical methods to compute approximate solutions which still meet the structures of the state manifolds. Classical numerical methods, such as those in the Euler class or in the Runge–Kutta class, will fail when applied directly to such dynamical systems because they were designed to work on flat spaces and cannot cope with non-flat manifolds [6].

As a specific applied field, the time synchronization of first-order dynamical systems on curved state manifolds by non-linear control will be surveyed. The time synchronization of non-autonomous dynamical systems has been applied in physiology [7,8], ecology [9], atmosphere physics [10], neurology [11] and many more applied fields [12,13,14]. Synchronization theory is a topic that appears to be very interesting from an abstract point of view and, at the same time, very useful in a number of applications. Indeed, synchronization theory appears to be an exciting and interdisciplinary research topic that combines key ideas from system theory, control theory and manifold calculus. In particular, non-linear control theory on manifolds may supply tools to design control fields to force a pair of dynamical systems into synchronizing their dynamics over time [15,16].

The present tutorial paper is devoted to recalling fundamental notions from manifold calculus and to explaining how these concepts apply to system theory and to non-linear control theory, taking time synchronization as a representative case of study in non-linear control. The present contribution is oriented to readers who either possess a good command of calculus but not of manifold calculus, or to readers who claim an understanding of theoretical manifold calculus, but do not have enough insight in the application/computation aspects of this field. In addition, a basic understanding of system theory and control theory will be assumed. The content of the present tutorial paper may be summarized as follows:

• It provides a clear and well-motivated introduction to manifold calculus, the basis of system and control theories on manifolds, with special emphasis on computational and applicational aspects. The present contribution provides practical formulas to deal with those real-valued manifolds that, in the author’s experience, are the most accessed in engineering and applied science problems. As a matter of fact, complex-valued manifolds are not treated at all.
• It clearly states and illustrates the idea that, when one wishes to perform a simulation, by a computing platform, of dynamical systems on manifolds described in terms of differential equations, it is necessary to time-discretize such differential equations in a suitable way. In order to achieve such a discretization, it is not safe to invoke standard discretization methods (such as the ones based on Euler forward–backward discretization), which do not work as they stand on curved manifolds. One should therefore resort to more sophisticated numerical integration techniques.
• By the author’s choice, the present tutorial paper does not carry any graphical illustrations nor any numerical simulation results. Readers who are interested in deepening their understanding of this topic are invited to to sketch graphs autonomously and to code examples in their favorite programming language.

The present paper is organized as follows. Section 2 lays out some introductory material, such as the motivation behind manifold calculus and a list of manifolds mostly accessed in applications. Section 3 introduces the notion of curves and tangent bundles and allied topics, such as normal spaces. Section 4 provides a first introduction, with examples, to first-order dynamical systems on manifolds. Section 5 presents a special kind of derivative of a function having a manifold as a domain and a manifold as a co-domain, termed a pushforward map. Section 6 introduces a class of manifolds that have peculiar features: the Lie groups. Section 7, in detail, treats the fundamental concept of metrization of a curved space through the more familiar notion of metrization of vector spaces. Section 8 introduces notions such as geodesic lines, Riemannian distance and exponential maps. Section 9 surveys the notion of Riemannian gradient and illustrates such a concept via examples. Section 10 introduces and exemplifies the notion of parallel transport along a curve, which is of paramount importance in manifold calculus and its computer-based implementations. Section 11 outlines the concept of manifold retraction and vector transport, which are computationally convenient approximations of exponential maps and parallel transport, respectively. Section 12 illustrates a feedback control theory suitable for application to first-order systems insisting on state manifolds. Section 13 introduces the notion of Riemannian Hessian, which stems from a second-order approximation of a manifold-to-scalar function, and recalls optimization algorithms that extend the Newton method to look for a zero of a vector field. Section 14 concludes this tutorial paper.

As a distinctive aspect of the present tutorial paper, the main flow of discussion is based on coordinate-free (or component-free) expressions, which facilitates the implementation of the main equations by a matrix-friendly computational language (such as MATLAB${}^{\copyright }$). Coordinate-free manifold calculus is introduced by the technical tool of embedded calculus, which stems from embedding a manifold into a larger ambient space where the calculation rules are simpler and more familiar to readers. The starred subsections, namely subsections marked by an asterisk (*), address some specific arguments, related to coordinate-prone manifold calculus, which may be skipped by the uninterested readers without detriment to the comprehension of the main flow of this presentation.

The present tutorial paper does not cover a number of subjects, such as the covariant derivation of vector fields, continuous-time second-order dynamical systems arising from a Lagrangian framework nor higher-order discrete-time dynamical systems, nor the key topics related to manifold curvature. These topics will be the subject of a forthcoming tutorial paper.

2. Coordinate-Free Embedded Manifold Calculus

In this section, we recall the relevant notation and fundamental properties of matrix calculus, as well as several examples of manifolds of interest in engineering and applied sciences.

In manifold calculus, both embedded (or extrinsic) and intrinsic coordinates may be accessed. With the aim of elucidating the difference between extrinsic and intrinsic coordinates for manifold elements, let us mention the case of the space $\mathbb{SO}\left(2\right)$ of planar rotations. The space $\mathbb{SO}\left(2\right)$ is a mono-dimensional manifold; therefore, any element $X\in \mathbb{SO}\left(2\right)$ may be pointed to by one intrinsic coordinate: a matrix $X\in \mathbb{SO}\left(2\right)$ may be represented as

$$X\left(\theta \right)=\left[\begin{array}{cc}cos\theta & sin\theta \\ -sin\theta & cos\theta \end{array}\right],$$

(1)

with $\theta \in [0,2\pi )$. On the other hand, by embedding the space $\mathbb{SO}\left(2\right)$ into the space ${\mathbb{R}}^{2\times 2}$ of $2\times 2$ real-valued matrices, it turns out that any element of $\mathbb{SO}\left(2\right)$ may be regarded as a two-by-two real-valued matrix

$$X=\left[\begin{array}{cc}{X}_{11}& X_{12}\\ X_{21}& X_{22}\end{array}\right],$$

(2)

whose entries must satisfy the four non-linear constrains:

$$\left\{\begin{array}{c}{X}_{11}^2+X_{21}^2=1,\hfill \\ X_{22}^2+X_{12}^2=1,\hfill \\ X_{11}{X}_{12}+X_{21}{X}_{22}=0,\hfill \\ X_{11}{X}_{22}-X_{12}{X}_{21}=1.\hfill \end{array}\right.$$

(3)

The four parameters $X_{11}$, $X_{12}$, $X_{21}$, $X_{22}$ denote embedded or extrinsic coordinates. The orthogonality and normality constraints of the columns of the matrix X may be represented through two coordinate-free constraints in a compact way as

$$X^{\top }X=I_2 \mathrm{and} det\left(X\right)=1,$$

(4)

where a superscript ${}^{\top }$ denotes matrix transposition and $I_2$ denotes a $2\times 2$ identity matrix.

Another illustrative example is the ordinary sphere ${\mathbb{S}}^2$ that may be parametrized as follows:

$$\left\{\begin{array}{c}x=cos\theta sin\phi \hfill \\ y=sin\theta sin\phi \hfill \\ z=cos\phi \hfill \end{array}\right.$$

(5)

with parameters $0\le \theta <2\pi$ and $0\le \phi \le \pi$ (in cartography, the coordinate $\theta$ is termed ‘latitude’, while the coordinate $\phi$ is termed ‘longitude’.) In general, the dimension of a manifold denotes the minimal number of parameters that are required to individuate a point on a manifold. While in common speech the ordinary sphere is termed ‘three-dimensional’, it is indeed a bi-dimensional manifold embedded in a three-dimensional ambient space. The dimension of the manifold is 2; the dimension of the embedding space, instead, is 3.

In general, in engineering and applied mathematics, using intrinsic coordinates is deprecated. As we have just seen, representing an ordinary sphere through intrinsic coordinates is fairly easy, but what about, for example, a hypersphere ${\mathbb{S}}^9$? Any intrinsic representation would require nine angles and a series of complicated trigonometric expressions. Assuming that a manifold of interest may be embedded into a larger ambient space $\mathbb{A}$, it is way easier to represent one such manifold as a subset of points of an ambient space that meet certain constraints. For example, the sphere ${\mathbb{S}}^9$ may be embedded in ${\mathbb{R}}^{10}$ and every point of such a sphere may be represented by an array $x\in {\mathbb{R}}^{10}$ such that

$$\sum _{i=1}^{10}{x}_i^2=x^{\top }x=1.$$

(6)

Clearly, the ten coordinates $x_i$, termed embedded coordinates, turn out to be unnecessary (nine coordinates would be enough). The existence of an embedding may be evaluated through classical results of manifold calculus, which also state the best dimension of the ambient space $\mathbb{A}$ on the basis of the structural properties of the embedding. Two such results are the strong Whitney embedding theorem and the Nash isometric embedding theorem for Riemannian manifolds. In the present tutorial, we are concerned exclusively with manifolds that may be embedded into a linear ambient space.

The number of extrinsic coordinates is, in general, far larger than the number of intrinsic coordinates; hence, a coordinate-free embedded representation is, in general, redundant. However, as long as computer implementation is concerned, a coordinate-free representation is advantageous because computing languages, such as MATLAB${}^{\copyright }$ and Python, generally deal seamlessly with bi-dimensional arrays. With reference to the sphere, let us examine a simple example that tells us why manifolds need their own calculus.

Example 1.

Take a point $x\in {\mathbb{S}}^2$ and a direction $v\in {\mathbb{R}}^3-\left\{0\right\}$, such that $x^{\top }v=0$, and let us write the parametric equation of a line departing from x in the direction v as $\ell \left(t\right):=x+tv$, with $0\le t\le 1$. It is important to point out that none of the points over such a line belong to the sphere, except for the point x. In fact, we have

$${\ell }^{\top }\left(t\right)\ell \left(t\right)={(x+tv)}^{\top }(x+tv)=1+t^2{\parallel v\parallel }^2,$$

(7)

since v is orthogonal to x. It turns out that ${\ell }^{\top }\left(t\right)\ell \left(t\right)\ne 1$ if $t\ne 0$, hence the assertion. ■

This simple example tells us that standard constructs, such as straight lines, do not work on curved manifolds, hence the need of a specific calculus to be consistently developed.

2.1. General Notation and Properties

In general, column arrays are denoted by a lower-case letter, while matrices are denoted by an upper-case letter. Manifold elements, for a generic manifold, are denoted as lower-case letters.

A number of matrix functions and factorizations will be invoked in this paper, namely:

• Matrix trace: The trace of a square matrix $M\in {\mathbb{R}}^{p\times p}$ (namely the sum of its principal-diagonal entries) is denoted by $\mathrm{tr}\left(M\right)$. Matrix trace has a cyclic permutation invariance property. For example, given three conformable (i.e., mutually multipliable) matrices $A,B,C$, it holds that:

  $$\mathrm{tr}\left(ABC\right)=\mathrm{tr}\left(BCA\right)=\mathrm{tr}\left(CAB\right).$$

  (8)
• Matrix square root: Given a matrix $M\in {\mathbb{R}}^{p\times p}$, its square root R is the unique matrix such that $R^2=P$. Not every matrix admits a square root. Special square roots (such as a symmetric square root) will be defined later.
• Spectral factorization: Given a matrix $M\in {\mathbb{R}}^{p\times p}$, let us assume that there exists an orthogonal matrix X (i.e., one such that $X^{\top }X=I_p$) and a real diagonal matrix D such that $M=XD{X}^{\top }$. The expression on the right-hand side denotes the spectral factorization of the matrix M. Such a factorization turns out to be very useful in evaluating matrix polynomials. For example, it holds that

  $$\begin{array}{cc}\hfill & M^4={\left(XD{X}^{\top }\right)}^4=\left(XD{X}^{\top }\right)\left(XD{X}^{\top }\right)\left(XD{X}^{\top }\right)\left(XD{X}^{\top }\right)=\hfill \\ \hfill & XD\left(X^{\top }X\right)D\left(X^{\top }X\right)D\left(X^{\top }X\right)D{X}^{\top }=XDDDD{X}^{\top }=X{D}^4{X}^{\top }.\hfill \end{array}$$

  (9)
  Exponentiating full matrices is cumbersome, while exponentiating diagonal matrices is amiable.
• Thin QR factorization: Any matrix $M\in {\mathbb{R}}^{p\times q}$, with $p\ge q$, may be factored as the product of a $p\times p$ orthogonal matrix Q and a $p\times q$ upper triangular matrix R, namely $M=QR$. In general, a QR factorization is not unique [17]. To remove such kind of indeterminacy, the R-factor may be chosen with strictly positive entries on its main diagonal, so that the factorization is unique.
• Compact singular value factorization (SVD): Compact SVD of a matrix $M\in {\mathbb{R}}^{p\times q}$ is a matrix factorization of the type $M=AD{B}^{\top }$ in which D is square diagonal of size $r\times r$, where $r\le min\{p,q\}$ is the rank of M, and has only nonzero singular values. In this variant, A denotes a $p\times r$ matrix and B denotes a $q\times r$ matrix, such that $A^{\top }A=B^{\top }B=I_r$ [18].
• Polar factorization: Given a real-valued $p\times n$ matrix M, its polar factorization is written as $M=XS$, where X denotes a $p\times n$ matrix such that $X^{\top }X=I_n$, termed polar factor, and S denotes a symmetric positive semidefinite $n\times n$ matrix [19]. The polar factorization of a matrix always exists and, if the matrix is full rank, its polar factor is unique.
• Matrix exponential: Given a matrix $M\in {\mathbb{R}}^{p\times p}$, its matrix exponential is denoted as $\mathrm{Exp}\left(M\right)$. Matrix exponential is defined via a series as

  $$\mathrm{Exp}\left(M\right):=\sum _{k=0}^{\infty }\frac{M^k}{k!}.$$

  (10)
  There exist special formulas to compute the matrix exponential via a finite number of operations for special matrices (see, for example, [20] and references therein). For example, for a symmetric matrix $M\in {\mathbb{R}}^{p\times p}$, that admits a spectral factorization $M=XD{X}^{\top }$, it holds that

  $$\mathrm{Exp}\left(M\right)=X\mathrm{Exp}\left(D\right)X^{\top },$$

  (11)

  where $\mathrm{Exp}\left(\mathrm{diag}({\lambda }_1,{\lambda }_2,{\lambda }_3,\dots ,{\lambda }_p)\right)=\mathrm{diag}({\mathrm{e}}^{{\lambda }_1},{\mathrm{e}}^{{\lambda }_2},{\mathrm{e}}^{{\lambda }_3},\dots ,{\mathrm{e}}^{{\lambda }_p})$.
• Principal matrix logarithm: Given a matrix $M\in {\mathbb{R}}^{p\times p}$, its principal matrix logarithm is denoted as $\mathrm{Log}\left(M\right)$. Matrix logarithm is defined via a series as

  $$\mathrm{Log}\left(M\right):=-\sum _{k=1}^{\infty }\frac{{(I_p-M)}^k}{k},\mathrm{defined} \mathrm{for}\parallel I_p-M\parallel <1.$$

  (12)
  In the specialized literaure it is possible to find special formulas to compute the principal matrix logarithm via a finite number of operations for special matrices. For example, for a positive-definite matrix $M\in {\mathbb{R}}^{p\times p}$, which admits a spectral factorization $M=XD{X}^{\top }$, it holds that

  $$\mathrm{Log}\left(M\right)=X\mathrm{Log}\left(D\right)X^{\top },$$

  (13)

  where $\mathrm{Log}\left(\mathrm{diag}({\lambda }_1,{\lambda }_2,{\lambda }_3,\dots ,{\lambda }_p)\right)=\mathrm{diag}(log{\lambda }_1,log{\lambda }_2,log{\lambda }_3,\dots ,log{\lambda }_p)$. Recall that any symmetric, positive-definite matrix has only positive eigenvalues; hence, $\mathrm{Log}\left(D\right)$ is a real-valued matrix.

The other notation used within this paper is defined at its earliest occurrence. We just recall that the symbol $:=$ denotes a definition and that the derivative of a function $F\left(t\right)$ with respect to its scalar parameter t is denoted as $\dot{F}$.

2.2. Manifolds and Embedded Manifolds (or Submanifolds)

The formal definition of a smooth manifold is quite convoluted, as it requires notions from mathematical topology [21]. More practically, a manifold may be essentially regarded as a generalization of surfaces in higher dimensions that is endowed with the noticeable property of being locally similar to a flat space. In addition, a manifold may generally be regarded as an abstract mathematical object, not necessarily ready for computation, whereas, in practice, manifolds of interest in applied science and engineering are essentially matrix manifolds.

Let us consider a smooth manifold $\mathbb{M}$ and a point x on it. From an abstract point of view, x is an element of a set $\mathbb{M}$ and does not necessarily carry any intrinsic numerical features. In order to be able to develop calculus on manifolds, to, for instance, compute the directional derivative of a function $f:\mathbb{M}\to \mathbb{R}$, it is convenient to ‘coordinatize’ a manifold. To this aim, let us take a neighborhood (open set) $U\subset \mathbb{M}$ that contains the point x and a coordinate map $\psi :U\to {\mathbb{R}}^p$. The size p denotes the minimal number of coordinates that is necessary to specify the location of the point x unequivocally and is taken as the dimension of the manifold. The map $\psi$ needs to be a one-to-one map. In this way, we attach a set of coordinates $\psi \left(x\right)$ to the point x. Such a construction establishes a smooth one-to-one correspondence between a point on a manifold and its coordinate point; therefore, the two concepts may be confused and we may safely speak of a point $x\in \mathbb{M}$ when actually speaking of its coordinates.

The above theoretical construction carries a number of practical drawbacks. For instance, notice that in general a manifold cannot be covered by a single coordinate map. Indeed, in general a manifold needs to be covered by a number of neighborhoods $U_k$, each of which is equipped with a coordinate map ${\psi }_k:U_k\to {\mathbb{R}}^p$. The set $\left\{U_k\right\}$ thus forms a basis for the manifold. Such a basis need not be finite, although it needs to be countable and is termed ‘atlas’. In general, the basis neighborhoods $U_k$ may happen to be overlapping one another; hence, the coordinate maps ${\psi }_k$ need to satisfy the compatibility conditions. Such conditions formalize the natural requirement that there need to be a one-to-one smooth correspondence between any two different coordinate systems insisting on regions of the manifolds belonging to more than one neighborhood. In formal terms, if $U_k\cap U_h\ne \varnothing$, then the ‘transition functions’ ${\psi }_k^{-1}\circ {\psi }_h$ and ${\psi }_h^{-1}\circ {\psi }_k$ should possess the structure of diffeomorphisms, namely $C^{\infty }$ functions endowed with $C^{\infty }$ inverses.

As mentioned in the introduction, in the present tutorial we are neglecting coordinates in favor of embeddings, except in starred sections that cover coordinate-prone calculations and that may be skipped by uninterested readers. A smooth manifold is by nature a continuous object. Manifolds of interest in applications, described in embedded terms, may be summarized as follows:

• Hypercube: The simplest manifold of interest is perhaps the hypercube ${\mathbb{R}}^p$, which is essentially the set spanned by p real-valued variables (or p-tuples).
• Hypersphere, oblique manifold, hyperellipsoid: A hypersphere is represented as ${\mathbb{S}}^{p-1}:=\{x\in {\mathbb{R}}^p\mid x^{\top }x=1\}$ and is the subset of points of the hypercube with unit Euclidean distance from point 0. This is a smooth manifold of dimension $p-1$ embedded in the hypercube ${\mathbb{R}}^p$; in fact, with only $p-1$ coordinates, we can identify unequivocally any point on a sphere. In [21], it is shown how to ‘coordinatize’ such a manifold through, e.g., the stereographic projection, which requires two coordinate maps applied to two convenient neighborhoods (each including only one ‘pole’) on the sphere. The special cases are ${\mathbb{S}}^1$, the unit circle, and ${\mathbb{S}}^2$, the ordinary sphere. There exist a number of applications insisting on the hyperspheres ${\mathbb{S}}^{p-1}$ such as, for instance, blind deconvolution [22,23], data classification [24], adaptive pattern recognition [25] and motion planning, optimization, and verification in robotics and in computational biology [26]. A smooth manifold closely related to the unit hypersphere is the oblique manifold [27], defined as:

  $$\mathrm{OB}\left(p\right):=\{X\in {\mathbb{R}}^{p\times p}\mid \mathrm{diag}\left(X^{\top }X\right)=I_p\},$$

  (14)

  where the operator $\mathrm{diag}(·)$ returns the zero matrix except for the main diagonal that is copied from the main diagonal of its argument. The structure of the oblique manifold may be easily studied on the basis of the unit hypersphere; in fact, the following identification holds true:

  $$\mathrm{OB}\left(p\right)\cong \underset{p\mathrm{times}}{\underbrace{{\mathbb{S}}^{p-1}\times {\mathbb{S}}^{p-1}\times \cdots \times {\mathbb{S}}^{p-1}}},$$

  (15)

  hence, each column of a $\mathrm{OB}\left(p\right)$-matrix may be treated as a ${\mathbb{S}}^{p-1}$-column array. The hyperellipsoid ${\mathrm{L}}^{p-1}$ is defined as:

  $${\mathrm{L}}^{p-1}:=\{x\in {\mathbb{R}}^p\mid {\parallel DRx\parallel }^2=1\},$$

  (16)

  with $D\in {\mathbb{R}}^{p\times p}$ being diagonal and positive-definite and $R\in {\mathbb{R}}^{p\times p}$ being a hyper-rotation such that $R^{\top }R=I_p$ and $det\left(R\right)=1$. The mathematical structure of the hyperellipsoid ${\mathrm{L}}^{p-1}$ may be studied on the basis of the manifold structure of the hypersphere. The hyperellipsoid ${\mathrm{L}}^{p-1}$ is used, e.g., in the calibration of magnetometers [28].
• General linear group and special linear group: The general linear group is defined as $\mathbb{GL}\left(p\right):=\{X\in {\mathbb{R}}^{p\times p}\mid det\left(X\right)\ne 0$}. This is the subset of the space of $p\times p$ matrices ${\mathbb{R}}^{p\times p}$ which are invertible. The special linear group is defined as $\mathrm{Sl}\left(p\right):=\{X\in {\mathbb{R}}^{p\times p}\mid det\left(X\right)=1$}. This is the subset of the general linear group made by all matrices with a unitary determinant.
• Orthogonal group, special orthogonal group, special Euclidean group: An orthogonal group of size p is defined by $\mathbb{O}\left(p\right):=\{X\in {\mathbb{R}}^{p\times p}\mid X^{\top }X=I_p\}$. The manifold $\mathbb{O}\left(p\right)$ has dimension $2^{-1}p(p-1)$. In fact, every matrix in $\mathbb{O}\left(p\right)$ possesses $p^2$ entries which are constrained by $2^{-1}p(p+1)$ orthogonality/normality restrictions. The manifold of special orthogonal matrices is defined as $\mathbb{SO}\left(p\right):=\left\{X\in {\mathbb{R}}^{p\times p}\mid X^{\top }X=I_p,det\left(X\right)=1\right\}$. A smooth manifold closely related to the special orthogonal group is the special Euclidean group, denoted as $\mathbb{SE}\left(p\right)$, that finds applications in robotics (see, e.g., [29]). The special Euclidean group is a set of $(p+1)\times (p+1)$ matrices defined as:

  $$\mathbb{SE}\left(p\right):=\left\{\left.\left[\begin{array}{cc}X& \delta \\ 0& 1\end{array}\right]\right|X\in \mathbb{SO}\left(p\right),\delta \in {\mathbb{R}}^p\right\}.$$

  (17)
• Stiefel manifold: The (compact) Stiefel manifold is defined as:

  $$\mathrm{St}(n,p):=\{X\in {\mathbb{R}}^{n\times p}\mid X^{\top }X=I_p\},$$

  (18)

  where $p\le n$. Avery Stiefel matrix has $np$ entries, but its elements are constrained by $2^{-1}p(p-1)$ non-linear constraints, and hence the dimension of a Stiefel manifold $\mathrm{St}(n,p)$ is $np-2^{-1}p(p-1)$. Exemplary applications are blind source separation [30], non-negative matrix factorization [31], best basis search/selection [32], electronic structures computation [33] and factor analysis in psychometrics [34]. A generalization of the Stiefel manifold, to be applied to principal subspace tracking, was studied in the contribution [35]. Such a generalized Stiefel manifold is defined as:

  $${\mathrm{St}}_B(n,p)=\{X\in {\mathbb{R}}^{n\times p}\mid X^{\top }BX=I_p\},$$

  (19)

  where B denotes any symmetric, positive-definite $n\times n$ matrix. The contribution [35] studied the structure of the tangent bundle of the generalized Stiefel manifold and suggested a computationally convenient calculus for this manifold.
• Real symplectic group: The real symplectic group is defined as

  $$\mathrm{Sp}\left(2n\right):=\{Q\in {\mathbb{R}}^{2n\times 2n}|Q^{\top }JQ=J\},J:=\left[\begin{array}{cc}{0}_n& I_n\\ -I_n& 0_n\end{array}\right],$$

  (20)

  where the symbol $I_n$ denotes again a $n\times n$ identity matrix and the symbol $0_n$ denotes a whole-zero $n\times n$ matrix. The skew-symmetric matrix J enjoys the following properties: $J^2=-I_{2n}$, $J^{-1}=J^{\top }=-J$.
• Manifold of symmetric, positive-definite (SPD) matrices: The main features of the space

  $${\mathbb{S}}^{+}\left(p\right):=\{P\in {\mathbb{R}}^{p\times p}|P=P^{\top },P>0\}$$

  (21)

  of symmetric, positive-definite matrices was surveyed, e.g., in [36,37]. We recall that a matrix $P\in {\mathbb{R}}^{p\times p}$ is termed positive definite if for every column array $z\in {\mathbb{R}}^p-\left\{0\right\}$ it holds that $z^{\top }Pz>0$. A useful property is that the (real) eigenvalues of any SPD matrix are all strictly positive. A related manifold is the space ${\mathbb{S}}^{+}(n,p)$ of symmetric fixed-rank positive-semidefinite matrices, that may be defined as:

  $${\mathbb{S}}^{+}(n,p):=\{X{X}^{\top }\mid X\in {\mathbb{R}}^{n\times p},\mathrm{rank}\left(X\right)=p\}.$$

  (22)
  The growing use of low-rank matrix approximations to retain tractability in large-scale applications boosted extensions of the calculus of positive-definite matrices to their low-rank counterparts [38].
• Grassmann manifold: A Grassmann manifold $\mathrm{Gr}(n,p)$ is a set of subspaces of ${\mathbb{R}}^n$ spanned by p-independent vectors, namely

  $$\mathrm{Gr}(n,p)=\left\{\mathrm{span}(w_1,w_2,w_3,\dots ,w_p)\right\},$$

  (23)

  with $w_1,w_2,w_3,\dots ,w_p\in {\mathbb{R}}^n$ being a p-tuple of arbitrary and linearly independent n-dimensional arrays. Grassmann manifolds are compact, smooth manifolds and are special cases of more general objects termed flag manifolds [39]. A representation of any of such subspaces may be assumed as the equivalence class $\left[X\right]=\{XR\mid X\in \mathrm{St}(n,p),R\in \mathbb{O}(p\left)\right\}$ [33]. In practice, an element $\left[X\right]$ of the Grassmann manifold $\mathrm{Gr}(n,p)$ is represented by a matrix in $\mathrm{St}(n,p)$, whose columns span the subspace $\left[X\right]$.

It is easy to see that manifolds of interest in applications may exhibit large dimensions. Coordinatizing these manifolds may be inconvenient for practical purposes when their sizes are larger than some units. For this reason, matrix manifolds are treated as submanifolds of ${\mathbb{R}}^p$ or ${\mathbb{R}}^{p\times n}$ and their elements are represented by solid arrays.

In general, we shall denote as $\mathbb{A}$ an ambient space that a manifold is embedded into. We shall assume that any ambient space $\mathbb{A}$ of interest in the present tutorial paper is a Euclidean vector space, namely, a finite-dimensional vector space over the real numbers endowed with an inner product, that we shall denote as ${\langle ·,·\rangle }^{\mathbb{A}}$.

Example 2.

An elementary example of manifold calculus involving the circle ${\mathbb{S}}^1$ is the Borsuk–Ulam theorem.

Theorem 1.

Let $f:{\mathbb{S}}^1\to \mathbb{R}$ denote a continuous function. There exist two antipodal points $x,-x\in {\mathbb{S}}^1$ such that $f\left(x\right)=f(-x)$.

Proof. 

Let $g\left(y\right):=f\left(y\right)-f(-y)$ with $y\in {\mathbb{S}}^1$. Choose an arbitrary point $x_0\in {\mathbb{S}}^1$:

• If $g\left(x_0\right)=0$, then $x=x_0$;
• If $g\left(x_0\right)\ne 0$, notice that $g(-x_0)=f(-x_0)-f(-(-x_0))$ = $-(f\left(x_0\right)-f(-x_0))=-g\left(x_0\right)$. Since g is continuous and takes a different sign in two different points of its domain, there must exist at least a point $x\in {\mathbb{S}}^1$ such that $g\left(x\right)=0$.

The above cases prove the assertion by exhaustion. □

The Borsuk–Ulam theorem has an interesting consequence for the temperature of the Earth. In fact, let us identify the equator with ${\mathbb{S}}^1$ and the temperature over the equator with the function f, then the Borsuk–Ulam theorem implies that there exist two antipodal points over the equator where the temperature is the same. Indeed, the Borsuk–Ulam theorem holds in every dimension, which implies, for example, that if ${\mathbb{S}}^2$ represents the Earth’s surface and $f:{\mathbb{S}}^2\to {\mathbb{R}}^2$ returns the atmospheric temperature and the Earth’s pressure in a point of the Earth surface, then there exist two antipodal points on Earth that exhibit the same temperature-pressure value pair! ■

Matrix factorizations are characterized by a paramount importance in applied calculations, as illustrated in the following.

Example 3.

Let us recall that every symmetric, positive-definite matrix may be factorized as:

$$P=RD{R}^{\top },$$

(24)

where $P\in {\mathbb{S}}^{+}\left(n\right)$, $R\in \mathbb{SO}\left(n\right)$ and D denotes a diagonal $n\times n$ matrix. In fact, D is the matrix whose in-diagonal entries coincide with the eigenvalues and R is the matrix whose columns coincide with the eigenvectors of P. The factorization (24) simplifies some calculations, for example, it holds that:

$$\begin{array}{cc}\hfill & P^3=PPP=\left(RD{R}^{\top }\right)\left(RD{R}^{\top }\right)\left(RD{R}^{\top }\right)=RD\left(R^{\top }R\right)D\left(R^{\top }R\right)D{R}^{\top }\hfill \\ \hfill & \phantom{P^3}=RDDD{R}^{\top }=R{D}^3{R}^{\top },\hfill \\ \hfill & \sqrt{P}=R\sqrt{D}{R}^{\top },\hfill \\ \hfill & \mathrm{Exp}\left(P\right)=\sum _{i=0}^{\infty }\frac{P^i}{i!}=\sum _{i=0}^{\infty }\frac{{\left(RD{R}^{\top }\right)}^i}{i!}=\sum _{i=0}^{\infty }\frac{R{D}^i{R}^{\top }}{i!}=R\mathrm{Exp}\left(D\right)R^{\top },\hfill \\ \hfill & P^{-1}=R{D}^{-1}{R}^{\top }.\hfill \end{array}$$

(25)

The net result is that convoluted matrix operations may be turned into fairly simple computations that require a finite number of elementary operations. ■

Manifolds may be classified as compact or non-compact:

• The manifolds ${\mathbb{S}}^{n-1}\subset \mathbb{A}$, with $\mathbb{A}:={\mathbb{R}}^n$, and $\mathbb{SO}\left(n\right)\subset \mathbb{A}$, with $\mathbb{A}:={\mathbb{R}}^{n\times n}$, are compact since there exists a ball $\mathbb{B}(0,r)\subset \mathbb{A}$, with $r<\infty$, that contains them.
• The manifold ${\mathbb{S}}^{+}\left(n\right)$ is non-compact since no t finite-radius ball exists that contains it.

In practical terms, the elements of a compact manifold are necessarily limited in value, while the elements of a non-compact manifold may take arbitrarily large values.

From a computational point of view, dynamical systems on compact manifolds, even if they turn out to be unstable, do not pose serious implementation problems, while dynamical systems on non-compact manifolds may result in implementational difficulties (and easily cause runtime errors).

3. Smooth Curves, Tangent Vector Fields, Tangent Spaces and Bundle, Normal Spaces

An interesting object we may think of on a smooth manifold $\mathbb{M}$ is a smooth curve $\gamma :[-ϵ,ϵ]\to \mathbb{M}$, with $ϵ>0$. It is worth remarking that a curve may cross different coordinate charts $(U_k,{\psi }_k)$; therefore, it is generally necessary to split a curve in as many branches (or segments) as coordinate charts it crosses. The function $\gamma$ describes a curve on the manifold $\mathbb{M}$ delimited by the endpoints $\gamma (-ϵ)$ and $\gamma \left(ϵ\right)$.

3.1. Curves and Bundles for Embedded Manifolds

Let us assume that a manifold $\mathbb{M}$ is embedded into an ambient space $\mathbb{A}$ of suitable dimensions (for instance, the sphere ${\mathbb{S}}^{p-1}$ is embedded in the ambient space $\mathbb{A}:={\mathbb{R}}^p$). In the following, a number of examples, and a counterexample, are discussed to clarify the important notion of a smooth curve on a manifold.

Example 4.

On a hypersphere ${\mathbb{S}}^{n-1}$, consider the following function:

$$\gamma \left(t\right):=\frac{x+tv}{\sqrt{{(x+tv)}^{\top }(x+tv)}}$$

(26)

with $t\in [-ϵ,ϵ]$ and $x,v\in {\mathbb{R}}^n$ being arbitrary (but let us avoid the singularity $x+tv=0$). Notice that $\gamma \left(0\right)=x/\parallel x\parallel$. To verify that such a function γ traces indeed a curve on the hyersphere it suffices to show that, for every $t\in [-ϵ,ϵ]$ and $x,v\in {\mathbb{R}}^n$, it holds that $\gamma \left(t\right)\in {\mathbb{S}}^{n-1}$. Indeed,

$${\gamma }^{\top }\left(t\right)\gamma \left(t\right)=\frac{{(x+tv)}^{\top }(x+tv)}{{\left[\sqrt{{(x+tv)}^{\top }(x+tv)}\right]}^2}=1.$$

(27)

In addition, consider the following function

$$\gamma \left(t\right):=xcos\left(\sqrt{v^{\top }v}t\right)+\frac{v}{\sqrt{v^{\top }v}}sin\left(\sqrt{v^{\top }v}t\right),$$

(28)

with $t\in [-ϵ,ϵ]$ and with $x\in {\mathbb{S}}^{n-1}$ and $v\in {\mathbb{R}}^n-\left\{0\right\}$ such that $v^{\top }x=0$. (In this case, it turns out that $\gamma \left(0\right)=x$.) Similarly to the previous example, in order to prove that γ is a valid curve on the hypersphere, it suffices to show that $\gamma \left(t\right)\in {\mathbb{S}}^{n-1}$ for every $t\in [-ϵ,ϵ]$:

$$\begin{array}{cc}\hfill {\gamma }^{\top }\left(t\right)\gamma \left(t\right)=& [xcos(\parallel v\parallel t)+\frac{v}{\parallel v\parallel }{sin(\parallel v\parallel t)]}^{\top }[xcos(\parallel v\parallel t)+\frac{v}{\parallel v\parallel }sin(\parallel v\parallel t)]\hfill \\ \hfill =& \left(x^{\top }x\right){cos}^2(\parallel v\parallel t)+2\frac{x^{\top }v}{\parallel v\parallel }sin(\parallel v\parallel t)cos(\parallel v\parallel t)+\frac{v^{\top }v}{{\parallel v\parallel }^2}{sin}^2(\parallel v\parallel t)\hfill \\ \hfill =& \left(1\right){cos}^2(\parallel v\parallel t)+2\frac{\left(0\right)}{\parallel v\parallel }sin(\parallel v\parallel t)cos(\parallel v\parallel t)+\left(1\right){sin}^2(\parallel v\parallel t)=1.\hfill \end{array}$$

(29)

It is not difficult to evaluate ${\gamma }^{\prime }\left(t\right)$ and, in particular, to show that ${\gamma }^{\prime }\left(0\right)=v$.

Let us now consider an example of a curve over the manifold $\mathbb{SO}\left(n\right)$ of the hyper-rotations. In particular, let us examine the following curve on the manifold $\mathbb{SO}\left(2\right)$:

$$\gamma \left(t\right):=\left[\begin{array}{cc}cos\left(bt\right)& sin\left(bt\right)\\ -sin\left(bt\right)& cos\left(bt\right)\end{array}\right]$$

(30)

with $t\in [-ϵ,ϵ]$ and $b\in \mathbb{R}$ being arbitrarily chosen. (Notice that $\gamma \left(0\right)=I_2$.) In order to show that function γ represents a curve over the space of planar rotations, it suffices to compute the product ${\gamma }^{\top }\left(t\right)\gamma \left(t\right)$ and to show that it keeps equal to the identity $I_2$ and that $det\left(\gamma \right(t\left)\right)=1$ for every value of t in its range:

$$\begin{array}{cc}\hfill {\gamma }^{\top }\left(t\right)\gamma \left(t\right)=& {\left[\begin{array}{cc}cos\left(bt\right)& sin\left(bt\right)\\ -sin\left(bt\right)& cos\left(bt\right)\end{array}\right]}^{\top }\left[\begin{array}{cc}cos\left(bt\right)& sin\left(bt\right)\\ -sin\left(bt\right)& cos\left(bt\right)\end{array}\right]\hfill \\ \hfill =& \left[\begin{array}{cc}cos\left(bt\right)& -sin\left(bt\right)\\ sin\left(bt\right)& cos\left(bt\right)\end{array}\right]\left[\begin{array}{cc}cos\left(bt\right)& sin\left(bt\right)\\ -sin\left(bt\right)& cos\left(bt\right)\end{array}\right]\hfill \\ \hfill =& \left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right],\hfill \\ \hfill det\left(\gamma \right(t\left)\right)=& {cos}^2\left(bt\right)-(-{sin}^2\left(bt\right))=1.\hfill \end{array}$$

(31)

Further to this, let us prove that the following function represents a curve over the manifold $\mathbb{SO}\left(n\right)$:

$$\gamma \left(t\right):=X(I_n+tH){(I_n-tH)}^{-1}$$

(32)

with $t\in [-ϵ,ϵ]$, $X\in \mathbb{SO}\left(n\right)$ and $H^{\top }=-H\in {\mathbb{R}}^{n\times n}$. (Even in this case, it holds that $\gamma \left(0\right)=X$.) In order to prove such a statement, it is necessary to compute ${\gamma }^{\top }\left(t\right)\gamma \left(t\right)$ and to show that it equals $I_n$ and that $det\left(\gamma \right(t\left)\right)=1$ at any time:

$$\begin{array}{cc}\hfill {\gamma }^{\top }\left(t\right)\gamma \left(t\right)=& {\left[X(I_n+tH){(I_n-tH)}^{-1}\right]}^{\top }X(I_n+tH){(I_n-tH)}^{-1}\hfill \\ \hfill =& {(I_n-tH)}^{-\top }{(I_n+tH)}^{\top }\left(X^{\top }X\right)(I_n+tH){(I_n-tH)}^{-1}\hfill \\ \hfill =& {(I_n+tH)}^{-1}{(I_n+tH)}^{\top }(I_n+tH){(I_n-tH)}^{-1}\hfill \\ \hfill =& {(I_n+tH)}^{\top }\left[{(I_n+tH)}^{-1}(I_n+tH)\right]{(I_n-tH)}^{-1}\hfill \\ \hfill =& {(I_n+tH)}^{\top }{(I_n-tH)}^{-1}\hfill \\ \hfill =& (I_n-tH){(I_n-tH)}^{-1}=I_n,\hfill \\ \hfill det\left(\gamma \right(t\left)\right)=& det\left(X\right)det(I_n+tH){det}^{-1}(I_n-tH)\hfill \\ \hfill =& det\left(X\right)det(I_n+tH){det}^{-1}\left({(I_n+tH)}^{\top }\right)=1,\hfill \end{array}$$

(33)

where the superscript ${}^{-\top }$ denotes the inverse of the transposed matrix.

Notice that, in the proof, we have made use of some matrix identities, including the commutativity property $(I_n+tH){(I_n-tH)}^{-1}={(I_n-tH)}^{-1}(I_n+tH)$ and $det\left(A\right)=det\left(A^{\top }\right)$.

As a further example, let us consider the following function, which we shall prove to represent a curve over the manifold of the $2\times 2$ symmetric, positive-definite matrices:

$$\gamma \left(t\right):=\left[\begin{array}{cc}a{(t+1)}^2& b{(t+1)}^2\\ b{(t+1)}^2& c{(t+1)}^2\end{array}\right]$$

(34)

with $t\in [-ϵ,ϵ]$, $0<ϵ<1$, $a>0$ and $ac-b^2>0$. (Notice that $\gamma \left(0\right)=\left[\begin{array}{cc}a& b\\ b& c\end{array}\right]$.) To prove the assertion, it suffices to show that ${\gamma }^{\top }\left(t\right)=\gamma \left(t\right)$ and that the eigenvalues of $\gamma \left(t\right)$ are strictly positive for any t in its range. Symmetry is immediately verified. The eigenvalues are $a{(t+1)}^2>0$ and $(ac-b^2){(t+1)}^4>0$.

Let us now consider a counterexample. Define the following function:

$$\gamma \left(t\right):=P+tQ,$$

(35)

with $t\in [-ϵ,ϵ]$ and $P,Q\in {\mathbb{S}}^{+}\left(n\right)$ arbitrary. In general, the above function does not represent a curve in ${\mathbb{S}}^{+}\left(n\right)$. To prove such an assertion, it suffices to recall that, in general, a linear combination of two positive-definite matrices does not result in a positive-definite matrix. As a numerical example, consider the following: $P:=\left[\begin{array}{cc}4& 1\\ 1& 1\end{array}\right]$ and $Q:=\left[\begin{array}{cc}2& -1\\ -1& 1\end{array}\right]$. Then, $\gamma \left(t\right)=\left[\begin{array}{cc}4+2t& 1-t\\ 1-t& 1+t\end{array}\right]$. It is readily verified that if $t\in [-4,4]$, then $\gamma \left(t\right)$ is not necessarily positive-definite for every value of t in the range (for example, $\gamma (-3)$ is not positive-definite). From this counterexample, we can verify that the manifold ${\mathbb{S}}^{+}\left(n\right)$, as all manifolds just exemplified, is not a linear space with respect to standard matrix operations. ■

From a system-theoretic perspective, a curve may be thought of as a trajectory generated by a dynamical system whose state space is a smooth manifold. Take, for instance, the case of a little magnetized ball rolling over a metallic spherical surface: no matter how the little ball moves over the surface of the sphere, during its motion the contact point will describe a curve $\gamma \left(t\right)$ over the larger sphere from the initial time $t=-ϵ$ to $t=+ϵ$. The parameter t may be thought of as a time index, whose progression corresponds to the flowing of time. Thinking of physical objects moving over curved surfaces helps in understanding the main concepts in manifold calculus, even though a certain degree of abstraction is still necessary, not only because the dimensions of manifolds of interest may be far larger than 3, but also because most manifolds are difficult to visualize in practice.

A noteworthy property of curves is that they are equivalent up to a regular reparameterization.

Comparing the position over a smooth curve in two close-by instants provides the notion of speed, namely, how quickly an object is moving along a trajectory and in which direction. In particular, denoting by ${\gamma }_x:[-ϵ,ϵ]\to \mathbb{M}$ a curve such that ${\gamma }_x\left(0\right)=x$, the quantity

$$v_x:=\underset{h\to 0}{lim}\frac{{\gamma }_x\left(h\right)-{\gamma }_x\left(0\right)}{h}={\left.\frac{\mathrm{d}{\gamma }_x\left(t\right)}{\mathrm{d}t}\right|}_{t=0}={\dot{\gamma }}_x\left(0\right)\in \mathbb{A},$$

(36)

denotes such a speed, which is represented as a tangent vector $v_x$ at the point x on the manifold. Clearly, the vector $v_x$ does not belong to the curved manifold $\mathbb{M}$, whereas it is tangent to it in the point x. (Notice that in this tutorial the word ‘vector’ is reserved to tangent vectors, while a column-type array—which may represent a location—is termed ‘array’.)

Consider every possible smooth curve on a manifold of dimension p passing through the point x and compute the tangent vectors to all these curves in the point x at once. The collection of these tangent vectors span a linear space of dimension p, which is referred to as tangent space (or ‘tangent plane’) to the manifold $\mathbb{M}$ at the point x, and is denoted with

$$T_x\mathbb{M}:=\{{\dot{\gamma }}_x\left(0\right)\mid \forall {\gamma }_x\}\subset \mathbb{A}.$$

(37)

Since there exist infinitely many curves passing through a given point with the same velocity, one should define a tangent vector as an equivalence class of curves passing through such a given point, while being tangent to each other at that point. A tangent plane is hence a collection of such vectors. In addition, we should underline that each tangent space is a subset of the ambient space due to the special nature of $\mathbb{A}$, being both a point set and a vector space.

Taking a point $x\in \mathbb{M}$ and a tangent vector $v_x\in \mathbb{M}$, any pair $(x,v_x)$ is thought of as belonging to an abstract set termed tangent bundle, defined as

$$T\mathbb{M}:=\{(x,v)\in {\mathbb{A}}^2\mid x\in \mathbb{M},v\in T_x\mathbb{M}\}.$$

(38)

In order to facilitate computation, it is useful to introduce the concept of the normal space of an embedded manifold in a given point under a chosen metric:

$$N_x\mathbb{M}:=\{z\in \mathbb{A}\mid {\langle z,v\rangle }^{\mathbb{A}}=0,\forall v\in T_x\mathbb{M}\}.$$

(39)

The normal space $N_x\mathbb{M}$ represents the orthogonal complement of the tangent space with respect to a Euclidean ambient space $\mathbb{A}$ that the manifold $\mathbb{M}$ is embedded into (in fact, some authors denote normal spaces as $T_x^{\perp }\mathbb{M}$).

Let us examine the structure of the tangent/normal spaces of a number of smooth manifolds:

• Hypercube: Since the space ${\mathbb{R}}^p$ is linear, when it is embedded into itself, each tangent space coincides to the whole space, namely, for every, $x\in {\mathbb{R}}^p$ it holds that $T_x{\mathbb{R}}^p\equiv {\mathbb{R}}^p$. Since a normal space is the orthogonal complement to a tangent space, we must conclude that $N_x{\mathbb{R}}^p=\left\{0\right\}$.
• Hypersphere: At every point $x\in {\mathbb{S}}^{p-1}$, the tangent space has the structure

  $$T_x{\mathbb{S}}^{p-1}:=\{v\in {\mathbb{R}}^p\mid v^{\top }x=0\}.$$

  (40)
  The normal space $N_x{\mathbb{S}}^{p-1}$ at every point of the hypersphere, which is the orthogonal complement of the tangent space with respect to the ambient space $\mathbb{A}:={\mathbb{R}}^p$ that the manifold ${\mathbb{S}}^{p-1}$ is embedded in, has the structure

  $$N_x{\mathbb{S}}^{p-1}:=\{\lambda x\mid \lambda \in \mathbb{R}\}$$

  (41)

  in the case that the ambient space is equipped with the Euclidean inner product ${\langle v,w\rangle }^{\mathbb{A}}:=v^{\top }w$.
• Special orthogonal group: The tangent space of the manifold $\mathbb{SO}\left(p\right)$ has the structure

  $$T_X\mathbb{SO}\left(p\right)=\{V\in {\mathbb{R}}^{p\times p}\mid V^{\top }X+X^{\top }V=0_p\}.$$

  (42)
  This may be proven by differentiating a generic curve $\gamma \left(t\right)\in \mathbb{SO}\left(p\right)$ passing through X at $t=0$. Every such curve satisfies the orthogonal-group characteristic equation ${\gamma }^{\top }\left(t\right)\gamma \left(t\right)=I_p$; therefore, after differentiation, one obtains

  $${\dot{\gamma }}^{\top }\left(0\right)\gamma \left(0\right)+{\gamma }^{\top }\left(0\right)\dot{\gamma }\left(0\right)=0_p.$$

  (43)
  By recalling that the tangent space is formed by velocity vectors $\dot{\gamma }\left(0\right)$, the above-mentioned result is readily achieved. Provided the ambient space $\mathbb{A}:={\mathbb{R}}^{p\times p}$ is endowed with the canonical Euclidean metric ${\langle V,W\rangle }^{\mathbb{A}}:=\mathrm{tr}\left(V^{\top }W\right)$, the normal space at a point X may be defined as

  $$N_X\mathbb{SO}\left(p\right)=\{N\in {\mathbb{R}}^{p\times p}\mid \mathrm{tr}\left(N^{\top }V\right)=0,\forall V\in T_X\mathbb{SO}\left(p\right)\}.$$

  (44)
  It is easy to convince oneself that every tangent vector $V\in T_X\mathbb{SO}\left(p\right)$ may be written as $V=XH$, with H skew-symmetric (i.e., such that $H^{\top }=-H$); then, any element $N\in N_X\mathbb{SO}\left(p\right)$ may be written as $N=XS$, with $S=S^{\top }$. In fact, the normality condition implies $0=\mathrm{tr}\left(V^{\top }\left(XS\right)\right)=\mathrm{tr}\left(S{V}^{\top }X\right)=\mathrm{tr}\left(X^{\top }V{S}^{\top }\right)$, which is equivalent to $-\mathrm{tr}\left(\left(V^{\top }X\right)S^{\top }\right)$; therefore, the normality condition may be recast as $\mathrm{tr}\left(\left(V^{\top }X\right)(S-S^{\top })\right)=0$. It is hence necessary and sufficient that $S=S^{\top }$; therefore,

  $$N_X\mathbb{SO}\left(p\right)=\{XS\mid S^{\top }=S\in {\mathbb{R}}^{p\times p}\}.$$

  (45)
• Stiefel manifold: Given a trajectory $[-ϵ,ϵ]\ni t\mapsto X\left(t\right)\in \mathrm{St}(n,p)$, derivation with respect to the parameter t yields ${\dot{X}}^{\top }X+X^{\top }\dot{X}=0$, which means that the tangent space to the manifold $\mathrm{St}(n,p)$ in a point $X\in \mathrm{St}(n,p)$ has the structure:

  $$T_X\mathrm{St}(n,p)=\{V\in {\mathbb{R}}^{n\times p}\mid V^{\top }X+X^{\top }V=0\}.$$

  (46)
  The normal space has the structure:

  $$N_X\mathrm{St}(n,p)=\{XS\mid S\in {\mathbb{R}}^{p\times p},S^{\top }-S=0\}.$$

  (47)
• Real symplectic group: The tangent space associated with the real symplectic group has the structure:

  $$T_Q\mathrm{Sp}\left(2n\right)=\{V\in {\mathbb{R}}^{2n\times 2n}\mid V^{\top }JQ+Q^{\top }JV=0_{2n}\}.$$

  (48)
  The tangent spaces and the normal space associated with the real symplectic group may be characterized as follows:

  $$\left\{\begin{array}{c}{T}_Q\mathrm{Sp}\left(2n\right)=\{V=QJS\mid S\in {\mathbb{R}}^{2n\times 2n},S^{\top }=S\},\hfill \\ N_Q\mathrm{Sp}\left(2n\right)=\{N=JQH\mid H\in {\mathbb{R}}^{2n\times 2n},H^{\top }=-H\}.\hfill \end{array}\right.$$

  (49)
• Space of symmetric, positive-definite matrices: Given a point $P\in {\mathbb{S}}^{+}\left(p\right)$, its tangent bundle may be characterized simply by observing that every curve $\gamma :[-ϵ,ϵ]\to {\mathbb{S}}^{+}\left(p\right)$ satisfies $\gamma \left(t\right)=\gamma {\left(t\right)}^{\top }$ and $\gamma \left(t\right)>0$. Only the equality constraint influences the structure of the tangent space; therefore,

  $$T_P{\mathbb{S}}^{+}\left(p\right)=\{S\in {\mathbb{R}}^{p\times p}\mid S^{\top }=S\}.$$

  (50)
  Notice that every tangent space is identical to each other as it does not depend on the base point P.
• Grassmann manifold: For every element $\left[X\right]\in \mathrm{Gr}(n,p)$, the tangent space may be represented as:

  $$T_{\left[X\right]}\mathrm{Gr}(n,p)=\{V\in {\mathbb{R}}^{n\times p}|X^{\top }V=0\}.$$

  (51)
  A tangent space $T_{\left[X\right]}\mathrm{Gr}(n,p)$ may be decomposed as the direct sum of a horizontal space and of a vertical space at $\left[X\right]\in \mathrm{Gr}(n,p)$[40]. Starting from a point $\left[X\right]\in \mathrm{Gr}(n,p)$, moving along a horizontal direction causes a change in subspace, while moving along a vertical direction does not change the subspace $\left[X\right]$.

3.2. Vector Fields

Let us now consider a map $v_x:\mathbb{M}\to T\mathbb{M}$ that associates a tangent vector $v_x$ with every point x of a manifold. One such map is termed tangent vector field (or simply vector field, for short). The set of all tangent vector fields of a manifold $\mathbb{M}$ is denoted as $\Gamma \left(\mathbb{M}\right)$. Vector fields are objects of prime importance in manifold calculus. In physical system dynamics, vector fields are associated with, e.g., speed and acceleration.

Example 5.

A vector field on a manifold $\mathbb{M}$ is a function $f:\mathbb{M}\to T\mathbb{M}$ that assigns to every point $x\in \mathbb{M}$ a tangent vector $f\left(x\right)\in T_x\mathbb{M}$. A vector field may even depend on the time parameter, in which case it will be denoted as $f(t,x)$ with $f:\mathbb{R}\times \mathbb{M}\to T\mathbb{M}$.

Let us now consider the hypersphere ${\mathbb{S}}^7$ embedded in $\mathbb{A}:={\mathbb{R}}^8$ and the function $f:{\mathbb{S}}^7\to {\mathbb{R}}^8$ defined as:

$$f\left(x\right):=(I_8-x{x}^{\top })Ax,$$

(52)

where x indicates a column array of 8 elements and $A\in {\mathbb{R}}^{8\times 8}$ is an arbitrary constant matrix. Such a function describes a vector field in $\Gamma \left({\mathbb{S}}^7\right)$ because it holds that $f\left(x\right)\in T_x{\mathbb{S}}^7$ for every x. In fact,

$$x^{\top }f\left(x\right)=x^{\top }(Ax-x{x}^{\top }Ax)=x^{\top }Ax-\left(x^{\top }x\right)\left(x^{\top }Ax\right)=0.$$

(53)

Notice that the tangent vector $f\left(x\right)$ is inextricably tied to the point x on the manifold. In fact, in general, taking two distinct points $x,y\in \mathbb{M}$, it turns out that $f\left(x\right)\notin T_y\mathbb{M}$. It is then sensible to describe a vector field as a set of pairs:

$$(x,f(x\left)\right)\in T\mathbb{M}.$$

(54)

As a further example, let us consider the manifold $\mathbb{SO}\left(8\right)$ embedded in $\mathbb{A}:={\mathbb{R}}^{8\times 8}$ and a function $f:\mathbb{R}\times \mathbb{SO}\left(8\right)\to {\mathbb{R}}^{8\times 8}$:

$$f(t,R):=RAt+R^{2}B{R}^{\top },$$

(55)

where R is a $8\times 8$ orthogonal matrix variable, while A and B are $8\times 8$ skew-symmetric constant matrices (namely, $A^{\top }=-A$ and $B^{\top }=-B$). Such a function represents a (time-varying) vector field on the manifold of hyper-rotations in that, for every $R\in \mathbb{SO}\left(8\right)$ and $t\in \mathbb{R}$, it holds that $f(t,R)\in T_R\mathbb{SO}\left(8\right)$. Such an assertion may be proven as follows

$$\begin{array}{cc}\hfill R^{\top }f(t,R)+f^{\top }(t,R)R=& R^{\top }(RAt+R^{2}B{R}^{\top })+{(RAt+R^{2}B{R}^{\top })}^{\top }R\hfill \\ \hfill =& \left(R^{\top }R\right)At+\left(R^{\top }R\right)RB{R}^{\top }+A^{\top }\left(R^{\top }R\right)t+R{B}^{\top }{R}^{\top }\left(R^{\top }R\right)\hfill \\ \hfill =& At+RB{R}^{\top }-At-RB{R}^{\top }\hfill \\ \hfill =& 0_n.\hfill \end{array}$$

(56)

As a last example, let us take the manifold ${\mathbb{S}}^{+}\left(5\right)$ embedded in $\mathbb{A}:={\mathbb{R}}^{5\times 5}$ and a function $f:{\mathbb{S}}^{+}\left(5\right)\to {\mathbb{R}}^{5\times 5}$ defined as

$$f\left(P\right):=P+P^2+3P^{-1},$$

(57)

where P is a $5\times 5$ symmetric, positive-definite matrix variable (hence, even $P^2=PP$ and $P^{-1}$ are symmetric, positive-definite). It turns out that the function f represents a vector field in $\Gamma \left({\mathbb{S}}^{+}\left(5\right)\right)$; in fact, for every $P\in {\mathbb{S}}^{+}\left(5\right)$, it holds that $f\left(P\right)\in T_P{\mathbb{S}}^{+}\left(5\right)$:

$$\begin{array}{cc}\hfill f^{\top }\left(P\right)-f\left(P\right)=& {(P+P^2+3P^{-1})}^{\top }-(P+P^2+3P^{-1})\hfill \\ \hfill =& P^{\top }+P^{2\top }+3P^{-\top }-P-P^2-3P^{-1}\hfill \\ \hfill =& P+P^2+3P^{-1}-P-P^2-3P^{-1}\hfill \\ \hfill =& 0_n.\hfill \end{array}$$

(58)

(Notice that, in the proof, only the symmetry of the matrix P and of its powers $P^2$ and $P^{-1}$ has been made use of.) ■

As a matter of fact, manifold calculus primarily deals with two kinds of objects:

• Points on a manifold, denoted as $x\in \mathbb{M}\subset \mathbb{A}$;
• Tangent vectors, denoted as $v\in T_x\mathbb{M}\subset \mathbb{A}$.

and occasionally on normal arrays, which are instrumental in calculations. In the case of matrix manifolds, which are of prime importance in applications, the ambient space $\mathbb{A}:={\mathbb{R}}^{p\times q}$; hence, points x and tangent vectors v are essentially arrays or matrices (either rectangular or square).

3.3. Canonical Curves, Canonical Basis of a Tangent Space*

In intrinsic manifold calculus, the way to regard, e.g., tangent spaces and vector fields is based on differential operators [21]. Formally, if $\mathcal{F}$ denotes a smooth function space, a tangent vector $v\in T_x\mathbb{M}$ is defined in such a way that $v:\mathcal{F}\to \mathbb{R}$; namely, $v\left(f\right)$ denotes the directional derivative of the function $f\in \mathcal{F}$ along the direction v. Recall that a differential operator in $T_x\mathbb{M}$ may be written as a linear combination of elementary differential operators, namely the derivatives along the basis axes, through some coefficients.

Let $\mathbb{M}$ denote a smooth manifold of dimension p and let ${\gamma }_x:[-ϵ,ϵ]\to \mathbb{M}$ denote any smooth curve such that ${\gamma }_x\left(0\right)=x$. Let us assume, for simplicity, that the image of ${\gamma }_x$ is entirely contained in a chart $(U,\psi )$. The map $\psi \circ {\gamma }_x$ traces out a smooth curve in the parameter space ${\mathbb{R}}^p$, which is differentiable in the usual multivariable calculus sense. Let us denote the intrinsic coordinates of the points over the curve as

$$({\theta }^1\left(t\right),{\theta }^2\left(t\right),{\theta }^3\left(t\right),\dots ,{\theta }^p\left(t\right))=\psi \left({\gamma }_x\left(t\right)\right).$$

(59)

The superscript notation ${\theta }^i$ to denote the $i$th coordinate is standard in manifold calculus. This notation helps when checking expressions written in intrinsic coordinates. Since the chart $\psi$ is invertible by definition, we may represent the curve $\gamma$ in local coordinates by

$${\gamma }_x\left(t\right)={\psi }^{-1}({\theta }^1\left(t\right),{\theta }^2\left(t\right),{\theta }^3\left(t\right),\dots ,{\theta }^p\left(t\right)).$$

(60)

In particular, it holds that $x={\psi }^{-1}({\theta }^1\left(0\right),{\theta }^2\left(0\right),{\theta }^3\left(0\right),\dots ,{\theta }^p\left(0\right))$. On the basis of the representation (60), we may define as many as pcanonical curves as ${\gamma }_x^i\left(t\right):={\psi }^{-1}({\theta }^1\left(0\right),{\theta }^2\left(0\right),\dots ,{\theta }^i\left(0\right)+t,\dots ,{\theta }^p\left(0\right))$ for $t\in [-ϵ,ϵ]$ and $i=1,2,3,\dots ,p$. Namely, a canonical curve around a point is traced on a manifold by letting only one of the p coordinates vary at a time. Let us now consider the tangent vectors

$${\partial }_i^x:={\dot{\gamma }}_x^i\left(0\right)={\left.\frac{\mathrm{d}{\gamma }_x^i\left(t\right)}{\mathrm{d}t}\right|}_{t=0}\in T_x\mathbb{M}.$$

(61)

The set $\{{\partial }_1^x,{\partial }_2^x,\dots ,{\partial }_p^x\}$ is termed canonical basis of the vector space $T_x\mathbb{M}$. Its property of being a basis is implied by the fact that such canonical vectors are linearly independent from one another. Therefore, any tangent vector may be written as a linear combination of canonical vectors, that is

$$v_x=v_x^1{\partial }_1^x+v_x^2{\partial }_2^x+\cdots +v_x^p{\partial }_p^x,$$

(62)

where the quantities $v_x^1$ represent the coefficients of the linear combination. To shorten the equations, the above expression may be written simply as $v=v^i{\partial }_i$, where the information about what happens at the point x has been suppressed and the summation is implied by the presence of a repeated index (i), one in upper position (${}^i$) and one in lower position (${}_i$), which is commonly referred to as Einstein summation convention.

Example 6.

Let us consider the case of a sphere ${\mathbb{S}}^2$ embedded into ${\mathbb{R}}^3$. A parametrization of ${\mathbb{S}}^2$ (excluding the South pole) is

$$x={\psi }^{-1}({\theta }^1,{\theta }^2):=\left[\begin{array}{c}sin{\theta }^{1}cos{\theta }^2\\ sin{\theta }^{1}sin{\theta }^2\\ cos{\theta }^1\end{array}\right]$$

(63)

for $0\le {\theta }^1\le \pi$ (where ${\theta }^1$ is commonly referred to as ‘latitude’) and $0\le {\theta }^2<2\pi$ (where ${\theta }^2$ is commonly referred to as ‘longitude’). For example, the point ${\psi }^{-1}(0,0)$ coincides with the North pole, while the point ${\psi }^{-1}(\frac{\pi }{2},0)$ lays on the equator of the sphere. Given a point $x={\psi }^{-1}({\theta }_0^1,{\theta }_0^2)$, the curve

$${\gamma }_x^1\left(t\right)=\left[\begin{array}{c}sin({\theta }_0^1+t)cos{\theta }_0^2\\ sin({\theta }_0^1+t)sin{\theta }_0^2\\ cos({\theta }_0^1+t)\end{array}\right]$$

(64)

traces a ‘meridian’ through x, while the curve

$${\gamma }_x^2\left(t\right)=\left[\begin{array}{c}sin{\theta }_0^{1}cos({\theta }_0^2+t)\\ sin{\theta }_0^{1}sin({\theta }_0^2+t)\\ cos{\theta }_0^1\end{array}\right]$$

(65)

traces a ‘parallel’ through x. Hence, by definition, the tangent space $T_x{\mathbb{S}}^2$ is spanned by the canonical basis vectors

$${\partial }_1^x={\dot{\gamma }}_x^1\left(0\right)={\left.\frac{\mathrm{d}{\gamma }_x^1\left(t\right)}{\mathrm{d}t}\right|}_{t=0}=\left[\begin{array}{c}cos{\theta }_0^{1}cos{\theta }_0^2\\ cos{\theta }_0^{1}sin{\theta }_0^2\\ -sin{\theta }_0^1\end{array}\right]$$

(66)

and

$${\partial }_2^x={\dot{\gamma }}_x^2\left(0\right)={\left.\frac{\mathrm{d}{\gamma }_x^2\left(t\right)}{\mathrm{d}t}\right|}_{t=0}=\left[\begin{array}{c}-sin{\theta }_0^{1}sin{\theta }_0^2\\ sin{\theta }_0^{1}cos{\theta }_0^2\\ 0\end{array}\right].$$

(67)

For example, it is readily seen that

$${\partial }_1^{(\frac{\pi }{2},0)}=\left[\begin{array}{c}0\\ 0\\ -1\end{array}\right],{\partial }_2^{(\frac{\pi }{2},0)}=\left[\begin{array}{c}0\\ 1\\ 0\end{array}\right],$$

(68)

represents tangent vectors at ${\left[100\right]}^{\top }$ (i.e., the North pole). ■

The dual space to a tangent space $T_x\mathbb{M}$, denoted as $T_x^{★}\mathbb{M}$, is termed cotangent space. Elements of a cotangent space are termed covectors$\omega \in T_x^{*}\mathbb{M}$. Covectors combine with vectors by ‘annihilation’ to a scalar. Given a canonical basis $\{{\partial }_1^x,{\partial }_2^x,\dots ,{\partial }_p^x\}$ of $T_x\mathbb{M}$, it is possible to define a canonical basis $\{\mathrm{d}{x}^1,\mathrm{d}{x}^2,\dots ,\mathrm{d}{x}^p\}$ of $T_x^{*}\mathbb{M}$ by the conditions $\mathrm{d}{x}^i\left({\partial }_j\right)={\delta }_j^i$. The union of all cotangent spaces form a cotangent bundle$T^{*}\mathbb{M}:=\{(x,\omega )\mid x\in \mathbb{M},\omega \in T_x^{*}\mathbb{M}\}$.

4. First-Order Dynamical Systems on Manifolds

A number of physical phenomena are described in system theory as systems of coupled differential equations in several variables. A fairly general representation of such systems is

$$\dot{x}\left(t\right)=f(t,x\left(t\right)),x\left(0\right)=x_0\in {\mathbb{R}}^p,t\in [0,t_{\mathrm{f}}],$$

(69)

where $x\in {\mathbb{R}}^p$ denotes a state variable array and represents a set of descriptive variables, and $f:\mathbb{R}\times {\mathbb{R}}^p\to {\mathbb{R}}^p$ represents a state-transition function. The multi-variable array-type function f denotes a (possibly time-varying) vector field that represents the set of velocities of change of the state of the system. The solutions of the differential Equation (69) represent integral curves of the vector field f. In this case, the space ${\mathbb{R}}^p$ represents the flat state space and the velocity space of the model (69) at once.

In order to take into account further constraints on the state variables, termed invariants, it is convenient to introduce the notion of curved state space in the form of a smooth manifold. This logical process leads to a type of first-order dynamical system described by

$$\dot{x}\left(t\right)=f(t,x\left(t\right)),x\left(0\right)=x_0\in \mathbb{M},t\in [0,t_{\mathrm{f}}],$$

(70)

where $f:\mathbb{R}\times \mathbb{M}\to T\mathbb{M}$ denotes the state-transition function of the mathematical model and $x\in \mathbb{M}$ denotes the state of the system at any given time t. Even in this case, the function f denotes the velocity field; however, in this case, the state space is the manifold $\mathbb{M}$, while the velocity space is the tangent bundle $T\mathbb{M}$.

Let us underline two aspects of the above equation:

• Meaning of $\dot{x}\left(t\right)$ in the expression (70): Since we assumed the manifold $\mathbb{M}$ to be embedded in an ambient space $\mathbb{A}$ of type ${\mathbb{R}}^{p\times q}$, the quantity $\dot{x}\left(t\right)$ is an array or a matrix made of the derivatives of the entries of $x\left(t\right)$ with respect to the time parameter t. Let us recall, however, that even if such a specification helps one understand the subject, we shall never write any relation involving single components of such matrix-type objects as we shall treat any state variable x as a whole, (except in a low-dimensional example). The solution $x\left(t\right)$ of the differential Equation (70) is the trajectory of the system and is represented by a curve on the manifold $\mathbb{M}$; hence, $\dot{x}\left(t\right)$ denotes the speed along the trajectory since, for every $t\in [0,t_{\mathrm{f}}]$, it holds that $(x\left(t\right),\dot{x}\left(t\right))\in T\mathbb{M}$, namely $\dot{x}\in \Gamma \left(\mathbb{M}\right)$.
• Separation (or non-equivalency) of first-order and second-order systems: When dealing with dynamical systems in ${\mathbb{R}}^{p\times q}$, the system (69) is actually fairly general since, upon introducing additional variables, it is possible to turn an n-th order system into a first-order system. We consider now how such a property stems from the legitimate ‘confusion’ between the state space and velocity space. In contrast to this, when dealing with dynamical systems on manifolds, such confusion is not legitimate, since $x\in \mathbb{M}$ while $\dot{x}\in T_x\mathbb{M}$; namely, the system state and the system velocity belong to very different spaces. It suffices to recall that $\mathbb{M}$ is generally a curved space (i.e., non linear) while each $T_x\mathbb{M}$ is a vector space (flat, linear). For this reason, second-order systems are not assimilable to first-order systems.

Example 7.

Let us reconsider the function (52). On the basis of such a vector field, we may define the following first-order system on the hypersphere ${\mathbb{S}}^{n-1}$:

$$\dot{x}\left(t\right)=(I_n-x\left(t\right)x^{\top }\left(t\right))Ax\left(t\right),x\left(0\right)=x_0\in {\mathbb{S}}^{n-1}.$$

(71)

Such a system may be rewritten equivalently as:

$$\dot{x}=Ax-\left(x^{\top }Ax\right)x.$$

(72)

Let us examine the equilibrium points of such dynamical system. Any equilibrium point $x_{★}$ must satisfy the equation

$$A{x}_{★}=\left(x_{★}^{\top }A{x}_{★}\right)x_{★}.$$

(73)

Whenever A is symmetric and positive-definite, Equation (73) establishes that $x_{★}$ is an eigenvector of matrix A. In effect, the state of the system (72) evolves towards an eigenvector of the matrix A. For this reason, the dynamical system (73) represents a prototypical example of a continuous-time calculation system.

The differential Equation (72) is commonly referred to as theOja equationand was studied by Prof. Erkki Oja from the Helsinki University of Technology (currently Aalto University). It was studied in several contexts, including automation and control [41], and constitutes an exemplification of the fact that even a fairly simple computing element might be able to perform a complex calculations, namely, extracting an eigenvector $x_{★}$ (and the corresponding eigenvalue ${\lambda }_{★}:=x_{★}^{\top }A{x}_{★}$) from an arbitrarily-sized matrix. ■

Let us now consider an example, drawn from circuit theory, that introduces the notion of ‘invariants’ for a dynamical system, which indeed motivates the study of dynamical systems on manifolds.

Example 8.

In order to emphasize the notion ofinvariantsin connection to dynamical systems, let us consider the simple model of an ideal DC-to-DC converter studied in [42]. The mathematical model of such converter reads:

$$\left\{\begin{array}{c}{C}_1\frac{\mathrm{d}{v}_1}{\mathrm{d}t}=(1-u)i_3,\hfill \\ C_2\frac{\mathrm{d}{v}_2}{\mathrm{d}t}=u{i}_3,\hfill \\ L_3\frac{\mathrm{d}{i}_3}{\mathrm{d}t}=-(1-u)v_1-u{v}_2,\hfill \end{array}\right.$$

(74)

where $v_1,v_2$ are state voltages (across capacitors), $i_3$ is a state current (across an inductor) and $u\in \{0,1\}$ denotes the control input (a switch). The following function is an invariant for such an electrical circuit

$$\mathcal{I}:=\frac{1}{2}{C}_1{v}_1^2+\frac{1}{2}{C}_2{v}_2^2+\frac{1}{2}{L}_3{i}_3^2.$$

(75)

Indeed, such a quantity represents the total energy across the electrical network. By invariant it is meant that $\dot{\mathcal{I}}=0$, namely that the time-function $\mathcal{I}\left(t\right)$ keeps the same value for every t. Notice, in fact, that

$$\begin{array}{cc}\hfill \frac{\mathrm{d}\mathcal{I}}{\mathrm{d}t}=& v_1{C}_1\frac{\mathrm{d}{v}_1}{\mathrm{d}t}+v_2{C}_2\frac{\mathrm{d}{v}_2}{\mathrm{d}t}+i_3{L}_3\frac{\mathrm{d}{i}_3}{\mathrm{d}t}\hfill \\ \hfill =& v_1(1-u)i_3+v_{2}u{i}_3+i_3(-(1-u)v_1-u{v}_2)\hfill \\ \hfill =& 0.\hfill \end{array}$$

(76)

In order to simplify the analysis of such a dynamical system, let us define the following abstract state variables:

$$x_1:=\sqrt{\frac{C_1}{2}}{v}_1,x_2:=\sqrt{\frac{C_2}{2}}{v}_2,x_3:=\sqrt{\frac{L_3}{2}}{i}_3.$$

(77)

Let us arrange the above state variables into the state array $x:=\left[\begin{array}{c}{x}_1\\ x_2\\ x_3\end{array}\right]$. The system (74) may thus be rewritten as $\dot{x}=f(t,x)$, with

$$f\left(t,\left[\begin{array}{c}{x}_1\\ x_2\\ x_3\end{array}\right]\right):=\underset{=:H\left(t\right)}{\underbrace{\left[\begin{array}{ccc}0& 0& \sqrt{\frac{1}{C_1{L}_3}}(1-u\left(t\right))\\ 0& 0& \sqrt{\frac{1}{C_2{L}_3}}u\left(t\right)\\ -\sqrt{\frac{1}{C_1{L}_3}}(1-u\left(t\right))& -\sqrt{\frac{1}{C_2{L}_3}}u\left(t\right)& 0\end{array}\right]}}\left[\begin{array}{c}{x}_1\\ x_2\\ x_3\end{array}\right].$$

(78)

In short, $f(t,x)=H\left(t\right)x$.

In terms of the new state variables, the invariant may be rewritten as:

$$\mathcal{I}\left(x\right):=x_1^2+x_2^2+x_3^2.$$

(79)

Assuming, for instance, that $\mathcal{I}\left(x_0\right)=1$, Equation (79) establishes that the state x belongs to the sphere ${\mathbb{S}}^2$. Hence, we may observe that the state space of the DC-to-DC converter is the unit sphere ${\mathbb{S}}^2$ embedded in ${\mathbb{R}}^3$.

As a last verification step, let us prove that $f(t,x)=H\left(t\right)x\in T_x{\mathbb{S}}^2$ for every t. It suffices to prove that ${\left(Hx\right)}^{\top }x=0$, namely that $x^{\top }{H}^{\top }x=0$. By definition of H, it is readily found to be a skew-symmetric matrix, namely that for every t, it holds that $H^{\top }\left(t\right)=-H\left(t\right)$. By transposing the product $x^{\top }{H}^{\top }x$, we find the sought result; in fact,

$${\left(x^{\top }{H}^{\top }x\right)}^{\top }=x^{\top }Hx=-\left(x^{\top }{H}^{\top }x\right),$$

(80)

and since $x^{\top }{H}^{\top }x$ is a scalar, it must hold that ${\left(x^{\top }{H}^{\top }x\right)}^{\top }=x^{\top }{H}^{\top }x$, therefore it must be equal to zero. ■

First-order dynamical systems are characterized by a (possibly varying) vector field that determines its dynamics, as illustrated by the following example.

Example 9.

Let us consider the following first-order dynamical system on the manifold $\mathbb{SO}\left(3\right)$ of three-dimensional rotations in space:

$$\dot{R}\left(t\right)=A-R\left(t\right)A^{\top }R\left(t\right),R\left(0\right)=I_3.$$

(81)

In this system, the matrix $R\in \mathbb{SO}\left(3\right)$ represents the orientation of a moving orthogonal frame attached to a rigid body with respect to an inertial reference frame, while the arbitrary constant matrix $A\in {\mathbb{R}}^{3\times 3}$ determines its rotational speed (namely, the orientation of the axis of rotation and the rotational velocity).

The orientation matrix is often termedattitudefor rigid bodies such as drones and satellites:

• The instance $R=I_3$ indicates that the object is horizontal with respect to the reference frame;
• The instance $R\ne I_3$ indicates that it is necessary to rotate the body-fixed axes to align them to the inertial axes.

The dynamical system (81) is of the type $\dot{R}=f\left(R\right)$. Let us verify that $f\left(R\right):=A-R{A}^{\top }R$ is a vector field of $T\mathbb{SO}\left(3\right)$. To show such a property, it suffices to prove that $f^{\top }\left(R\right)R+R^{\top }f\left(R\right)=0_3$. This is in fact true:

$$\begin{array}{cc}\hfill {(A-R{A}^{\top }R)}^{\top }R+R^{\top }(A-R{A}^{\top }R)=& A^{\top }R-R^{\top }A\left(R^{\top }R\right)+R^{\top }A-\left(R^{\top }R\right)A^{\top }R\hfill \\ \hfill =& A^{\top }R-R^{\top }A+R^{\top }A-A^{\top }R\hfill \\ \hfill =& 0_3.\hfill \end{array}$$

(82)

It is worth noticing that the property $f\in \Gamma \left(\mathbb{SO}\right(3\left)\right)$ holds true even in the case that A is a time-varying matrix field, namely $A=A\left(t\right)$. ■

5. Tangent Maps: Pushforward and Pullback

Let us consider two Riemannian manifolds $\mathbb{M}$ and $\mathbb{N}$. Any smooth function $f:\mathbb{M}\to \mathbb{N}$ transforms a curve in $\mathbb{M}$ into a curve in $\mathbb{N}$. Since both curves are associated with their velocity vector fields, which define their tangent spaces, one might wonder how the function f maps tangent spaces in $\mathbb{M}$ into tangent spaces in $\mathbb{N}$. The answer is materialized as a pushforward map.

Let $x\in \mathbb{M}$, then a pushforward map ${\mathrm{d}}_{x}f:T_x\mathbb{M}\to T_{f\left(x\right)}\mathbb{N}$ is defined such that for every smooth curve ${\gamma }_x:[-ϵ,ϵ]\to \mathbb{M}$, ${\gamma }_x\left(0\right)=x$ it holds that:

$${\left.\frac{\mathrm{d}}{\mathrm{d}t}f\left({\gamma }_x\left(t\right)\right)\right|}_{t=0}=:{\mathrm{d}}_{x}f\left({\dot{\gamma }}_x\left(0\right)\right).$$

(83)

In general, therefore, given a function $f:\mathbb{M}\to \mathbb{N}$ that maps a point x from the manifold $\mathbb{M}$ to the point $f\left(x\right)$ on the manifold $\mathbb{N}$, the map ${\mathrm{d}}_{x}f$ associates to any tangent vector v belonging to the tangent space $T_x\mathbb{M}$ a tangent vector ${\mathrm{d}}_{x}f\left(v\right)$ belonging to the tangent space $T_{f\left(x\right)}\mathbb{N}$ by ‘pushing’ such vector to such space.

A pushforward map is indeed a linear approximation of a smooth map on tangent spaces. Any tangent map ${\mathrm{d}}_{x}f\left(v\right)$ is linear in the argument v. The map ${\mathrm{d}}_{x}f$ at a point x represents, in practical terms, the best linear approximation of the function f near x. A pushforward map may be regarded as a generalization of the total derivative of ordinary calculus. (Some authors would denote a pushforward map by an asterisk as in $f_{★}$; however, this notation takes up the space of the reference point x hence hindering it.)

Let us consider two special cases of interest.

• Pushforward of a manifold-to-scalar function: The special case that $\mathbb{N}:=\mathbb{R}$, namely that f is a manifold-to-scalar function, is particularly important in applications. Such a special case will be covered later since it involves the notion of Riemannian gradient.
• Pushforward of a matrix-to-matrix function: This is the case that the smooth manifolds $\mathbb{M}$ and $\mathbb{N}$ are real matrix manifolds embedded in ${\mathbb{R}}^{p\times p}$. Any smooth function between any such pair of manifolds is of matrix-to-matrix type. Let us assume that the function f is analytic about a point $X_0\in \mathbb{M}$, namely, that it may be expressed as a polynomial series:

  $$f\left(X\right)=\sum _{k=0}^{\infty }{a}_k{(X-X_0)}^k,a_k\in \mathbb{R}.$$

  (84)
  Then, the pushforward map ${\mathrm{d}}_{X}f\left(V\right)$ in a point $X\in \mathbb{M}$ applied to the tangent direction $V\in T_X\mathbb{M}$ may be expressed as:

  $${\mathrm{d}}_{X}f\left(V\right)=\sum _{k=1}^{\infty }{a}_k\sum _{r=0}^{k-1}{(X-X_0)}^{r}V{(X-X_0)}^{k-r-1}.$$

  (85)
  It is easily recognized that the tangent map $\mathrm{d}{f}_X\left(V\right)$ is linear in the argument V. As a reference for the readers, we recall the analytic expansion of three matrix-to-matrix functions, that may be used to compute the corresponding pushforward maps:
  • Matrix exponential: For the map $f\left(X\right)=\mathrm{Exp}\left(X\right)$, it holds that $X_0=0$, $a_k={(k!)}^{-1}$ for $k\ge 0$;
  • Principal matrix logarithm: For the map $f\left(X\right)=\mathrm{Log}\left(X\right)$, it holds that $X_0=I_p$, $a_0=0$, $a_k={(-1)}^{k+1}{k}^{-1}$ for $k\ge 1$;
  • Matrix inversion: For the map $f\left(X\right)=X^{-1}$, it holds that $X_0=I_p$, $a_k={(-1)}^k$ for $k\ge 0$.

The inverse of a pushforward map ${\mathrm{d}}_{x}f$ is termed pullback map and is denoted as ${\left({\mathrm{d}}_{x}f\right)}^{-1}$. A map ${\left({\mathrm{d}}_{x}f\right)}^{-1}\left(w\right)$ ‘pulls’ any tangent vector w from the tangent space $T_{f\left(x\right)}\mathbb{N}$ back to the tangent space $T_x\mathbb{M}$.

6. Lie Groups, Lie Algebras, Lie Brackets

Lie groups are hybrid mathematical constructions sharing properties that characterize smooth manifolds and algebraic groups. Let us recall that an algebraic group is an algebraic structure $(\mathbb{G},m,i,e)$ made of a set $\mathbb{G}$, either discrete or continuous, endowed with an internal operation denoted as $m:\mathbb{G}\times \mathbb{G}\to \mathbb{G}$, usually referred to as group multiplication (not necessarily a multiplication is the standard sense, though), an inversion operation denoted as $i:\mathbb{G}\to \mathbb{G}$, and an identity element with respect to group multiplication denoted as e. The multiplication, inversion and identity are related in such a way that, for every triple $x,y,z\in \mathbb{G}$, it holds that:

$$\begin{array}{cc}\hfill & m(x,e)=m(e,x)=x\hfill \\ \hfill & m(x,i(x\left)\right)=m\left(i\right(x),x)=e,\hfill \\ \hfill & m(x,m(y,z\left)\right)=m\left(m\right(x,y),z).\hfill \end{array}$$

(86)

Note that, in general, group multiplication is not commutative, i.e. $m(x,y)\ne m(y,x)$.

Two instances of algebraic groups are $(\mathbb{Z},+,-,0)$, a discrete group, and $(\mathbb{GL}\left(p\right),·{,}^{-1},I_p$), a continuous group. The structure $(\mathbb{Z},+,-,0)$ represents the set of integer numbers (either positive or negative, also including 0) with the standard addition as a group multiplication, which implies that the inverse is the standard subtraction while the identity is the 0 element. The second example, namely the structure $(\mathbb{GL}\left(p\right),·{,}^{-1},I_p$), represents the subset of non-singular matrices of size $p\times p$, endowed with the standard matrix-to-matrix multiplication ‘·’ as group multiplication. In such a case, the inverse operation coincides with the standard matrix inversion while the group identity coincides with the identity matrix $I_p$. It is straightforward to show that such group operations/identities satisfy the recalled group axioms. A counterexample of a structure that is not an algebraic group is given by the set of non-negative integer numbers ${\mathbb{Z}}_0^{+}$, which does not form a group under standard addition/subtraction (in fact, the subtraction of two positive integers does not necessarily return a positive integer).

With the notions of algebraic groups and smooth manifolds, we may now define a well-known object of manifold calculus, namely a Lie group. A Lie group conjugates the properties of an algebraic group and of a smooth manifold, as it is a set endowed with group properties and a manifold structure. Paraphrasing Wirth:

Lie group := Manifold + Algebraic group.

Let us denote by $T_x\mathbb{G}$ the tangent space of a Lie group $\mathbb{G}$ at a point $x\in \mathbb{G}$. The tangent space at the identity, namely $T_e\mathbb{G}$, represents a special instance of tangent spaces. In fact, such a tangent space, upon being endowed with a binary operator termed Lie brackets, possesses the structure of a so-called Lie algebra and will be denoted as $\mathfrak{g}$.

Let us examine the structure of the Lie algebra of the following Lie groups:

• Hypercube: The hypercube, also known as a translation group, is a Lie group under standard matrix sum (matrix subtraction and zero matrix complete the group structure). The Lie algebra of ${\mathbb{R}}^p$ coincides with itself.
• General linear group and the special linear group: Both the general linear group $\mathbb{GL}\left(p\right)$ and the special linear group $\mathrm{Sl}\left(p\right)$ are Lie groups under standard matrix multiplication and inversion. The Lie algebra of the general linear group, namely $\mathfrak{gl}\left(p\right)$, coincides with ${\mathbb{R}}^{p\times p}$. The linear algebra associated with the special linear group is more interesting and its determination involves some clever matrix computations. Let us consider $\mathrm{Sl}\left(p\right)$ endowed with standard matrix multiplication, inversion and $I_p$ as the group identity and a curve $\gamma :[-ϵ,ϵ]\to \mathrm{Sl}\left(p\right)$ defined by $\gamma \left(t\right):=\mathrm{Exp}\left(tA\right)$, where $\mathrm{Exp}$ denotes the matrix exponential and $A\in {\mathbb{R}}^{p\times p}$. Clearly, $\gamma \left(0\right)=I_p$; hence, $\dot{\gamma }\left(0\right)$ represents any element of the Lie algebra $\mathfrak{sl}\left(p\right)$. It is not hard to prove that

  $$\dot{\gamma }\left(t\right)=A\mathrm{Exp}\left(tA\right),$$

  (87)

  hence, $\dot{\gamma }\left(0\right)=A\in \mathfrak{sl}\left(p\right)$. Now, let us recall a result from matrix calculus [43]:

  $${\left.\frac{\mathrm{d}}{\mathrm{d}t}det\left(\gamma \left(t\right)\right)\right|}_{t=0}=\mathrm{tr}\left(\dot{\gamma }\left(0\right)\right).$$

  (88)
  In the present case, since $det\left(\gamma \right(t\left)\right)\equiv 1$, it follows from the above considerations that $\mathrm{tr}\left(A\right)=0$. In conclusion, we found that

  $$\mathfrak{sl}\left(p\right)=\{A\in {\mathbb{R}}^{p\times p}\mid \mathrm{tr}\left(A\right)=0\},$$

  (89)

  the space of traceless matrices. The algebra $\mathfrak{sl}\left(p\right)$ has dimension $p^2-1$.
• Special orthogonal group: The $\mathbb{SO}\left(n\right)$ manifold is a Lie group under standard matrix multiplication. The Lie algebra associated with the special orthogonal group is the set of skew-symmetric matrices $\mathfrak{so}\left(p\right):=\{H\in {\mathbb{R}}^{p\times p}\mid H+H^{\top }=0_p\}$. In fact, at the identity it holds that $T_{I_p}\mathbb{SO}\left(p\right)=\mathfrak{so}\left(p\right)$. The Lie algebra $\mathfrak{so}\left(p\right)$ is a vector space of dimension $2^{-1}p(p-1)$.
• Real symplectic group: The Lie algebra associated with the real symplectic group may be characterized as follows:

  $$\mathfrak{sp}\left(2n\right)=\{H=JS\mid S\in {\mathbb{R}}^{2n\times 2n},S^{\top }=S\},$$

  (90)

  where the quantity J denotes again the fundamental skew-symmetric matrix.
• Manifold of symmetric, positive-definite matrices: The Lie algebra associated with the Lie group ${\mathbb{S}}^{+}\left(p\right)$ is the set of $p\times p$ symmetric matrices, namely ${\mathfrak{s}}^{+}\left(p\right):=\{S\in {\mathbb{R}}^{p\times p}\mid S^{\top }=S\}$. The space of symmetric, positive-definite matrices is not a group under standard matrix multiplication. We recall from [44] the following group structure $({\mathbb{S}}^{+}\left(p\right),m,i,e)$:

  -

  Multiplication: $m(P,Q):=\mathrm{Exp}(\mathrm{Log}P+\mathrm{Log}Q)$, (logarithmic multiplication), with $P,Q\in {\mathbb{S}}^{+}\left(p\right)$, where ‘$\mathrm{Log}$’ denotes the principal matrix logarithm;

  -

  Identity element: $e=I_p$ (notice that $\mathrm{Log}{I}_p=0_p$);

  -

  Inverse: $i\left(P\right)=\mathrm{Exp}(-\mathrm{Log}P)$ (matrix inversion), with $P\in {\mathbb{S}}^{+}\left(p\right)$ (any symmetric, positive-definite matrix is non-singular).

  It is easy to verify that the proposed instances of $m,i,e$ satisfy the algebraic-group axioms in ${\mathbb{S}}^{+}\left(p\right)$. Additionally, the logarithmic multiplication on ${\mathbb{S}}^{+}\left(p\right)$ is compatible with its smooth manifold structure, as the map $(P,Q)\mapsto m(P,i(Q\left)\right)=\mathrm{Exp}(\mathrm{Log}P-\mathrm{Log}Q)$ is smooth.

An essential peculiarity of any Lie groups $(\mathbb{G},m,i,e)$ is that the whole group may always be brought back to a convenient neighborhood of the identity e and the same holds true for every tangent space $T_x\mathbb{G}$, $\forall x\in \mathbb{G}$, which may be brought back to the algebra $\mathfrak{g}$. Let us consider, for instance, a curve ${\gamma }_x:[-ϵ,ϵ]\to \mathbb{G}$ such that ${\gamma }_x\left(0\right)=x$. We may define a new curve

$${\tilde{\gamma }}_x\left(t\right):=m({\gamma }_x\left(t\right),i\left(x\right))$$

(91)

that has the property ${\tilde{\gamma }}_x\left(0\right)=e$. Conversely, ${\gamma }_x\left(t\right)=m({\tilde{\gamma }}_x\left(t\right),x)$. This operation closely resembles a translation of a curve into a neighborhood of the group identity.

The above observation leads to defining two Lie-group functions:

• Right translation: Defined as a function $R_x:\mathbb{G}\to \mathbb{G}$ by $R_x\left(y\right):=m(y,x)$ for every pair $(x,y)\in \mathbb{G}$;
• Left-translation: Defined as a function $L_x:\mathbb{G}\to \mathbb{G}$ by $L_x\left(y\right):=m(i\left(x\right),y)$ for every pair $(x,y)\in \mathbb{G}$;

the distinction between the left translation and right translation is somewhat arbitrary: we chose a rather non-standard one.

Notice that $R_x$ and $L_x$ commute; in fact,

$$R_x\left(L_x\left(y\right)\right)=m(m(i\left(x\right),y),x)=m(i\left(x\right),m(y,x))=L_x\left(R_x\left(y\right)\right).$$

(92)

Let us consider a simple example to get acquainted with the notion of left/right translation.

Example 10.

Let us particularize the notion of left/right translation to the familiar hypercube. In ${\mathbb{R}}^p$, considered as a group under standard array addition, these functions may be defined as $R_x\left(y\right):=y+x$ and $L_x\left(y\right)=-x+y$. The composition $R_x\left(L_x\left(y\right)\right)$ returns $R_x\left(L_x\left(y\right)\right)=L_x\left(y\right)+x=-x+y+x$, namely $R_x\left(L_x\left(y\right)\right)=y$; hence, $R_x\circ L_x={\mathrm{id}}_x$. Since $R_x$ and $L_x$ commute, we may say that, in this specific case, the functions $R_x$ and $L_x$ are the inverse of one another. In general, this is not true, though, and the failure of the composition $R_x\circ L_x$ to equal an identity map is caused by the lack of commutativity of a group. ■

Now, let us consider the curve $L_x\left({\gamma }_x\right)$: it crosses the identity e at $t=0$. Consequently, the pushforward ${\mathrm{d}}_x{L}_x$ maps every vector in $T_x\mathbb{G}$ to a vector in $T_e\mathbb{G}$. Namely,

$${\mathrm{d}}_x{L}_x:T_x\mathbb{G}\to \mathfrak{g}.$$

(93)

Likewise, taking an arbitrary smooth curve ${\gamma }_e$ passing by the identity $e\in \mathbb{G}$ at $t=0$ and considering the curve $R_x\left({\gamma }_e\right)$, it is readily seen that the latter crosses the point x at $t=0$. Consequently, the pushforward ${\mathrm{d}}_e{R}_x$ maps every vector in $T_e\mathbb{G}$ to a vector in $T_x\mathbb{G}$. Namely,

$${\mathrm{d}}_e{R}_x:\mathfrak{g}\to T_x\mathbb{G}.$$

(94)

Recall that $T_x\mathbb{G}$ and $\mathfrak{g}$ are of equal dimension and that the pushforward map is linear; hence, the pushforward map ${\mathrm{d}}_x{L}_x$ allows us to translate a vector belonging to a tangent space of a group to a vector belonging to its algebra (and vice versa through its pullback). This is the reason for which the Lie algebra of a Lie group is sometimes termed the ‘generator’ of a group.

It is easy to see that, if the structure of $\mathfrak{g}$ is known for a group $\mathbb{G}$, it might be convenient to coordinatize a neighborhood of the identity of $\mathbb{G}$ through elements of the associated algebra with the help of a conveniently selected homeomorphism (namely, a continuous function between topological spaces that has a continuous inverse function). Such a homeomorphism is known in the literature as an exponential map and is denoted as $exp:\mathfrak{g}\to \mathbb{G}$. We shall discuss the notion of exponential map for general manifolds in a later section.

As mentioned, a Lie algebra is endowed with a binary operator termed Lie bracket, denoted as $[·,·]$. Let us survey its derivation. Let us define the function ${\Psi }_x:\mathbb{G}\to \mathbb{G}$ termed inner isomorphism as

$${\Psi }_x\left(y\right):=L_x\left(R_x\left(y\right)\right).$$

(95)

The inner isomorphism provides a measure of non-commutativity of a Lie group. It is interesting to notice that the function ${\Psi }_x$ preserves the inner product of $\mathbb{G}$; in fact, taken two elements $y,z\in \mathbb{G}$, it holds that

$$\begin{array}{cc}\hfill {\Psi }_x\left(m(y,z)\right)=& m\left(i\right(x),m(m(y,z),x\left)\right)\hfill \\ \hfill =& m\left(i\right(x),m(m\left(m\right(y,x),m(i\left(x\right),z\left)\right),x\left)\right)\hfill \\ \hfill =& m\left(m\right(i\left(x\right),m(y,x)),m(i\left(x\right),m(z,x)\left)\right)\hfill \\ \hfill =& m({\Psi }_x\left(y\right),{\Psi }_x\left(z\right));\hfill \end{array}$$

(96)

hence, it represents a Lie-group automorphism.

Now, let us take the differential of ${\Psi }_x\left(y\right)$ with respect to the variable y at $y=e$. To this aim, it suffices to consider a curve $y:[-ϵ,ϵ]\to \mathbb{G}$ such that $y\left(0\right)=e$ and an element $x\in \mathbb{G}$, and to compute the derivative of ${\Psi }_x\left(y\left(t\right)\right)$ with respect to t at $t=0$. The calculation of the derivative gives

$$\frac{\mathrm{d}}{\mathrm{d}t}{\Psi }_x\left(y\left(t\right)\right)=\left({\mathrm{d}}_{y\left(t\right)}{\Psi }_x\right)\left(\dot{y}\left(t\right)\right)=\left({\mathrm{d}}_{R_x\left(y\left(t\right)\right)}{L}_x\right)\circ \left({\mathrm{d}}_{y\left(t\right)}{R}_x\right)\left(\dot{y}\left(t\right)\right).$$

(97)

Setting $y=e$ and letting $v:=\dot{y}\left(0\right)\in \mathfrak{g}$ and ${\mathrm{Ad}}_x:={\mathrm{d}}_e{\Psi }_x$ yields

$${\mathrm{Ad}}_x\left(v\right):=\left({\mathrm{d}}_x{L}_x\right)\circ \left({\mathrm{d}}_e{R}_x\right)\left(v\right).$$

(98)

By virtue of the properties (93) and (94), ${\mathrm{Ad}}_x$ is a linear map from $\mathfrak{g}$ to itself, namely an endomorphism in $\mathrm{End}\left(\mathfrak{g}\right)$. The map ${\mathrm{Ad}}_x$ is termed adjoint representation of the Lie algebra $\mathfrak{g}$.

In addition, let us now consider a smooth curve $x:[-ϵ,ϵ]\to \mathbb{G}$ such that $x\left(0\right)=e$ and $\dot{x}\left(0\right)=u\in \mathfrak{g}$ and let us evaluate the map

$${\mathrm{ad}}_u:={\left.\frac{\mathrm{d}}{\mathrm{d}t}\left({\mathrm{d}}_x{L}_x\right)\circ \left({\mathrm{d}}_e{R}_x\right)\right|}_{t=0}.$$

(99)

By the chain rule, one obtains

$${\mathrm{ad}}_u:={\left.\frac{\mathrm{d}}{\mathrm{d}t}\left({\mathrm{d}}_{x\left(t\right)}{L}_{x\left(t\right)}\right)\right|}_{t=0}\circ \left({\mathrm{d}}_e{R}_e\right)+\left({\mathrm{d}}_e{L}_e\right)\circ {\left.\frac{\mathrm{d}}{\mathrm{d}t}\left({\mathrm{d}}_e{R}_{x\left(t\right)}\right)\right|}_{t=0}.$$

(100)

This is termed adjoint operator and coincides with the Lie bracket of two tangent vectors up to sign, namely

$$[u,v]:=-{\mathrm{ad}}_u\left(v\right).$$

(101)

The Lie bracket provides a measure of non-commutativity of the algebra $\mathfrak{g}$.

Example 11.

Let us consider the Lie group $\mathbb{GL}\left(n\right)$ endowed with standard matrix multiplication and let us write down explicitly the inner isomorphism, the adjoint representation and the Lie bracket.

Since $L_X\left(Y\right)=X^{-1}Y$ and $R_X\left(Y\right)=YX$, the inner isomorphism reads ${\Psi }_X\left(Y\right)=X^{-1}YX$.

Taking a curve $Y\left(t\right)\in \mathbb{GL}\left(n\right)$ such that $Y\left(0\right)=I_n$ and $\dot{Y}\left(0\right)=V\in {\mathbb{R}}^{n\times n}$ yields

$${\mathrm{Ad}}_X\left(V\right)={\left.\frac{\mathrm{d}}{\mathrm{d}t}\left(X^{-1}Y\left(t\right)X\right)\right|}_{t=0}=X^{-1}VX.$$

(102)

To end with, taking a curve $X\left(t\right)\in \mathbb{GL}\left(n\right)$ such that $X\left(0\right)=I_n$ and $\dot{X}\left(0\right)=U\in {\mathbb{R}}^{n\times n}$ yields

$$\begin{array}{c}\hfill {\mathrm{ad}}_U\left(V\right)={\left.\frac{\mathrm{d}}{\mathrm{d}t}\left(X^{-1}\left(t\right)VX\left(t\right)\right)\right|}_{t=0}=(-UV)+VU=-[U,V].\end{array}$$

(103)

In the above calculation, we have used a known matrix identity, namely

$$\frac{\mathrm{d}}{\mathrm{d}t}{X}^{-1}\left(t\right)=-X^{-1}\left(t\right)\dot{X}\left(t\right)X^{-1}\left(t\right).$$

(104)

In this case, $[A,B]:=AB-BA$ denotes the matrix commutator. ■

7. Metrization, Riemannian Manifolds

A Riemannian manifold is a smooth manifold $\mathbb{M}$ whose tangent bundle is equipped with a smooth family of positive-definite inner products $\mathbb{M}\ni x\mapsto {\langle ·,·\rangle }_x\in \mathbb{R}$. An inner product is locally defined at every point $x\in \mathbb{M}$ of a manifold as a bilinear function from $T_x\mathbb{M}\times T_x\mathbb{M}$ to $\mathbb{R}$. It is important to remark that the inner product acts on two elements of a tangent space to the manifold at a given point x; it therefore depends (smoothly) on the point x. Hence, such an inner product gives rise to a local metric. Whenever an inner product ${\langle ·,·\rangle }_x$ does not depend explicitly on the point x, it is termed uniform. Given two tangent vectors $v,w\in T_x\mathbb{M}$, their inner products are denoted as ${\langle v,w\rangle }_x$.

7.1. Coordinate-Free Metrization by Inner Products and Metric Kernels

Let us recall some of the general properties of any inner product. In general, any vector space $\mathbb{V}$ may be endowed with an inner product (also termed scalar product) denoted by $\langle ·,·\rangle :\mathbb{V}\times \mathbb{V}\to \mathbb{R}$. Any inner product has a set of properties:

• For every $u,v\in \mathbb{V}$, it holds that $\langle u,v\rangle =\langle v,u\rangle$,
• For every $u,v,w\in \mathbb{V}$, it holds that $\langle u+w,v\rangle =\langle u,v\rangle +\langle w,v\rangle$,
• For every $u,v\in \mathbb{V}$ and $c\in \mathbb{R}$, it holds that $\langle cu,v\rangle =c\langle u,v\rangle$,
• The norm of a vector $v\in \mathbb{V}$ is defined as $\parallel v\parallel :=\sqrt{\langle v,v\rangle }$,
• An inner product is non-degenerate if and only if $\langle u,v\rangle =0$ for every $v\in \mathbb{V}$ implies $u=0$.

Let us consider a simple example about the above notions.

Example 12.

Let us consider the special case $\mathbb{V}:={\mathbb{R}}^p$ and the inner product:

$$\langle u,v\rangle =u^{\top }Sv,$$

(105)

with S being a symmetric, $p\times p$ matrix. If S, in addition to being symmetric, is also positive-definite, then such an inner product is non-degenerate. ■

It pays to recall the notion of adjoint operator with respect to a metric on a vector space. Let us consider again the vector space $\mathbb{V}$ endowed with an inner product $\langle ·,·\rangle :\mathbb{V}\times \mathbb{V}\to \mathbb{R}$ and a linear operator $\omega :\mathbb{V}\to \mathbb{V}$. Ad adjoint operator ${\omega }^{†}:\mathbb{V}\to \mathbb{V}$ is one such that, for every $v,w\in \mathbb{V}$, it holds that

$$\langle {\omega }^{†}\left(v\right),w\rangle =\langle v,\omega \left(w\right)\rangle .$$

(106)

A self-adjoint operator is one such that ${\omega }^{†}=\omega$, while an antiadjoint operator is one such that ${\omega }^{†}=-\omega$. The following observation on self-adjoint operators in quadratic forms is in order.

Observation 1.

Every operator may be decomposed as the sum of a self-adjoint and of an antiadjoint operator; in fact, it holds that

$$\omega =\frac{1}{2}(\omega +{\omega }^{†})+\frac{1}{2}(\omega -{\omega }^{†}).$$

(107)

As far as quadratic forms $\langle \omega \left(v\right),v\rangle$ are concerned, only the self-adjoint component plays a role. In fact, it is not hard to show that

$$\langle \omega \left(v\right),v\rangle =\langle \frac{1}{2}(\omega +{\omega }^{†})\left(v\right),v\rangle .$$

(108)

This is indeed true for any arbitrary inner product of the type $\langle \omega \left(v\right),w\rangle$, in fact, some straightforward algebraic work reveals that

$$\langle \omega \left(v\right),w\rangle =\frac{1}{2}\left\{\langle \omega (w+v),w+v\rangle -\langle \omega (w-v),w-v\rangle \right\}.$$

(109)

Since the right-hand side is a combination of quadratic forms, the left-hand side depends only on the self-adjoint part of ω.

In the case of a manifold, at every point, $x\in \mathbb{M}$ corresponds a tangent space $T_x\mathbb{M}$ that may be endowed with an inner product. Therefore, to denote the inner product assigned to the vector space $T_x\mathbb{M}$, we use the notation ${\langle ·,·\rangle }_x$.

Example 13.

Given a Riemannian manifold $(\mathbb{M},\langle ·,·\rangle )$ and two smooth vector fields $f,g\in \Gamma \left(\mathbb{M}\right)$, we may define the function $h:\mathbb{M}\to \mathbb{R}$ as

$$h\left(x\right):={\langle f\left(x\right),g\left(x\right)\rangle }_x.$$

(110)

Notice that the inner product variessmoothlyfrom one tangent space to another; hence, the function h is regular in $\mathbb{M}$. The function $h:\mathbb{M}\to \mathbb{R}$ represents a scalar field on $\mathbb{M}$.

As a further example, let us consider an arbitrary curve $\gamma :[-ϵ,ϵ]\to \mathbb{M}$ over a manifold $\mathbb{M}$. On the basis of such curve, we can define a function $\phi :[-ϵ,ϵ]\to \mathbb{R}$ as

$$\phi \left(t\right):={\langle f\left(\gamma \left(t\right)\right),g\left(\gamma \left(t\right)\right)\rangle }_{\gamma \left(t\right)}.$$

(111)

Notice that the inner product ‘accompanies’ the point $x=\gamma \left(t\right)$ that travels along the curve and involves different tangent spaces. Even the function $\phi :[-ϵ,ϵ]\to \mathbb{R}$ represents a scalar fieldrestrictedto the curve γ. ■

On a Riemannian manifold $\mathbb{M}$, the norm of a vector $v\in T_x\mathbb{M}$ is defined as ${\parallel v\parallel }_x:=\sqrt{{\langle v,v\rangle }_x}$. A specific property of Riemannian manifolds is that the norm is positive-definite; namely ${\parallel v\parallel }_x^2\ge 0$, where ${\parallel v\parallel }_x=0$ if an only if $v=0$.

Choosing the ‘right’ metric for a given manifold in a given application is one of the most challenging aspects of metrization, especially because it is seldom unclear what ‘right’ practically means. When in doubt, it is possible to resort to canonical metrics. A number of canonical metrics for different manifolds of interest are summarized in the following:

• Hypercube: For the space ${\mathbb{R}}^p$ of (column) arrays, the canonical metric is the Euclidean metric ${\langle u,v\rangle }_x:=u^{\top }v$, for every $u,v\in {\mathbb{R}}^p$, while for the space ${\mathbb{R}}^{p\times q}$ of rectangular matrices, the canonical metric is the Euclidean metric ${\langle U,V\rangle }_X:=\mathrm{tr}\left(U^{\top }V\right)$, for every $U,V\in {\mathbb{R}}^{p\times q}$. Notice that both these metrics do not depend explicitly on the point that they are calculated at, and hence they are uniform.
• Hypersphere: The hypersphere ${\mathbb{S}}^{p-1}$ embedded into the ambient space ${\mathbb{R}}^p$ inherits its canonical metric; hence, we shall choose ${\langle u,v\rangle }_x:=u^{\top }v$ for every $u,v\in T_x{\mathbb{S}}^{p-1}$. Even this metric is uniform.
• General linear group and the special linear group: A metric for the general linear group $\mathbb{GL}\left(p\right)\ni A$ is:

  $${\langle U,V\rangle }_A:=\mathrm{tr}\left({\left(A^{-1}U\right)}^{\top }\left(A^{-1}V\right)\right),\forall U,V\in T_A\mathbb{GL}\left(p\right).$$

  (112)
  Such a metric was popularized, for instance, in [45], in the context of machine learning.
• Special orthogonal group: The canonical metric in $\mathbb{SO}\left(p\right)$ is defined as ${\langle U,V\rangle }_X:=\mathrm{tr}\left(U^{\top }V\right)$, for any $X\in \mathbb{SO}\left(p\right)$ and $U,V\in T_X\mathbb{SO}\left(p\right)$. Notice that the norm of a tangent vector $V\in T_X\mathbb{SO}\left(p\right)$ is ${\parallel V\parallel }_X=\sqrt{\langle V,V\rangle }=\sqrt{\mathrm{tr}\left(V^{\top }V\right)}=:{\parallel V\parallel }_{\mathrm{F}}$, known as the Frobenius norm [18].
• Stiefel manifold: There are two well-known metrics for the Stiefel manifold, namely, the Euclidean metric and the canonical metric.
  • Euclidean metric: A possible metric that the Stiefel manifold may be endowed with is the Euclidean metric, inherited from the embedding of $\mathrm{St}(n,p)$ in ${\mathbb{R}}^{n\times p}$:

    $${\langle U,V\rangle }_X:=\mathrm{tr}\left(U^{\top }V\right),U,V\in T_X\mathrm{St}(n,p).$$

    (113)
  • Canonical metric: The Stiefel manifold may be endowed with a second kind of metric, termed ‘canonical metric’. The associated inner product reads:

    $${\langle U,V\rangle }_X:=\mathrm{tr}\left(U^{\top }\left(I_n-\frac{1}{2}X{X}^{\top }\right)V\right),U,V\in T_X\mathrm{St}(n,p),$$

    (114)

    which, unlike the Euclidean metric (113), is not uniform over the Stiefel manifold.
• Real symplectic group: There exist two known metrics in the scientific literature that were applied to the real symplectic group.
  • Khvedelidze–Mladenov metric: A metric for the real symplectic group $\mathrm{Sp}\left(2n\right)$ is:

    $${\langle U,V\rangle }_Q:=\mathrm{tr}\left(Q^{-1}U{Q}^{-1}V\right),\forall U,V\in T_Q\mathrm{Sp}\left(2n\right).$$

    (115)
    It is referred to as Khvedelidze–Mladenov metric (or KM metric, for short, [46]). This is an indefinite metric; hence, a manifold endowed with this metric is not Riemannian (it is in fact referred to as a pseudo-Riemannian manifold).
  • Euclidean metric: A further metric for the real symplectic group $\mathrm{Sp}\left(2n\right)$ is:

    $${\langle U,V\rangle }_Q:=\mathrm{tr}\left({\left(Q^{-1}U\right)}^{\top }\left(Q^{-1}V\right)\right),\forall U,V\in T_Q\mathrm{Sp}\left(2n\right).$$

    (116)
    Such a metric is inherited from the embedding of a real symplectic group $\mathrm{Sp}\left(2n\right)$ into the space of real invertible matrices $\mathbb{GL}\left(2n\right)$ that, in turn, is embedded into the real hypercube ${\mathbb{R}}^{2n\times 2n}$ and hence inherits its canonical metric.
• Space of symmetric, positive-definite matrices: The canonical metric in ${\mathbb{S}}^{+}\left(p\right)$ is defined as ${\langle U,V\rangle }_P:=\mathrm{tr}\left(U{P}^{-1}V{P}^{-1}\right)$, for any $P\in {\mathbb{S}}^{+}\left(p\right)$ and $U,V\in T_P{\mathbb{S}}^{+}\left(p\right)$. Clearly, this is not a uniform metric.
• Grassmann manifold: The canonical metric on a Grassmann manifold is

  $${\langle U,V\rangle }_{\left[X\right]}=\mathrm{tr}\left(U^{\top }V\right),\forall U,V\in T_{\left[X\right]}\mathrm{Gr}(n,p),$$

  (117)

  which corresponds to the Euclidean metric in the Stiefel manifold that is used to represent elements of the Grassmann manifold.

Example 14.

The inner product (115) gives rise to a metric which is not positive-definite on the space $\mathrm{Sp}\left(2n\right)\subset \mathbb{GL}\left(2n\right)$. To verify this property, it suffices to evaluate the structure of the squared norm ${\parallel V\parallel }_Q^2=\mathrm{tr}\left({\left(Q^{-1}V\right)}^2\right)$ with $Q\in \mathrm{Sp}\left(2n\right)$ and $V\in T_Q\mathrm{Sp}\left(2n\right)$. By the structure of the tangent space $T_Q\mathrm{Sp}\left(2n\right)$, it is known that $Q^{-1}V=JS$ with $S\in {\mathbb{R}}^{2n\times 2n}$ symmetric. It holds that:

$$JS=\left[\begin{array}{cc}{0}_n& I_n\\ -I_n& 0_n\end{array}\right]\left[\begin{array}{cc}{S}_1& S_2\\ S_2^T& S_3\end{array}\right]=\left[\begin{array}{cc}{S}_2^T& S_3\\ -S_1& -S_2\end{array}\right],$$

with $S_1,S_3\in {\mathbb{R}}^{n\times n}$ being symmetric and $S_2\in {\mathbb{R}}^{n\times n}$ being arbitrary. Hence, $\mathrm{tr}\left({\left(JS\right)}^2\right)=2\mathrm{tr}\left(S_2^2\right)-2\mathrm{tr}\left(S_1{S}_3\right)$, which has an indefinite sign. ■

In general, there exists a canonical metric inherited from the ambient space $\mathbb{A}$ that a manifold $\mathbb{M}$ is embedded in. We know that whenever a manifold is embedded in an ambient space $\mathbb{A}$, the inner product between two tangent vectors may be written as

$${\langle u,w\rangle }_x={\langle u,G_x\left(w\right)\rangle }^{\mathbb{A}},$$

(118)

where ${\langle ·,·\rangle }^{\mathbb{A}}$ denotes an inner product in $\mathbb{A}$. The expression (118) is based on a metric kernel $G:T\mathbb{M}\to T\mathbb{M}$. Metric kernels play a prominent role in coordinate-free embedded manifold calculus since a kernel and its derivative determine most of the main functions and maps that we shall encounter in the next sections. We assume that the metric kernel has the following properties:

• Linearity: $G_x\left(v\right)$ is linear in v, namely $G_x(v+\alpha w)=G_x\left(v\right)+\alpha G_x\left(w\right)$, for every $\alpha \in \mathbb{R}$, $x\in \mathbb{M}$ and $v,w\in T_x\mathbb{M}$;
• Symmetry: $G_x$ is a self-adjoint operator, namely ${\langle u,G_x\left(w\right)\rangle }^{\mathbb{A}}={\langle G_x\left(u\right),w\rangle }^{\mathbb{A}}$;
• Closure with respect to $T\mathbb{M}$: $G_x$ is an endomorphism of $T_x\mathbb{M}$, namely $G_x\left(v\right)\in T_x\mathbb{M}$, for every $x\in \mathbb{M}$ and $v\in T_x\mathbb{M}$;
• Invertibility: $G_x$ is invertible, namely, its inverse $G_x^{-1}$ is well-defined for every $x\in \mathbb{M}$.

In general, a metric kernel is not well-defined in the whole ambient space $\mathbb{A}$. We shall invoke the fact that it is well-defined in $T\mathbb{M}$ and that it might be extended to $\mathbb{M}\times \mathbb{A}$, in fact in the expression $G_x\left(a\right)$ one must take $x\in \mathbb{M}$ but it is allowable to take $a\in \mathbb{A}$, since the metric kernel is linear in the argument a. Occasionally, we might need to extend the metric kernel G by an operator $\overline{G}:{\mathbb{A}}^2\to \mathbb{A}$ that is defined at least in a neighborhood of $\mathbb{M}\times \mathbb{A}$ and such that $\overline{G}\equiv G$ in $T\mathbb{M}$. (In principle, such an ‘extendability’ requirement is not necessary, although it facilitates some computations and certainly clarifies some computational developments.)

Example 15.

Let us consider a hypersphere ${\mathbb{S}}^{n-1}\subset \mathbb{A}$ where $\mathbb{A}:={\mathbb{R}}^n$ endowed with a metric ${\langle u,w\rangle }_x={\langle u,w\rangle }^{\mathbb{A}}:=u^{\top }w$. It is readily seen that, in this example, $G_x={\mathrm{id}}_x$; therefore, the metric kernel is linear, self-adjoint, invertible and an endomorphism of $T_x{\mathbb{S}}^{n-1}$.

Let us further consider the manifold ${\mathbb{S}}^{+}\left(n\right)\subset \mathbb{A}$ endowed with the metric ${\langle U,W\rangle }_P:=\mathrm{tr}\left(U{P}^{-1}W{P}^{-1}\right)$, where $\mathbb{A}:={\mathbb{R}}^{n\times n}$ is endowed with the metric ${\langle U,V\rangle }^{\mathbb{A}}:=\mathrm{tr}\left(U^{\top }V\right)$. In this case, the metric kernel does not coincide with the identity, rather:

$$G_P\left(W\right):=P^{-1}W{P}^{-1}.$$

(119)

Let us verify that such metric kernel has the four mentioned properties:

1. 

Linearity: $G_P\left(W\right)$ appears to be linear in W; in fact, it holds that $G_P(V+\alpha W)=P^{-1}(V+\alpha W)P^{-1}=P^{-1}V{P}^{-1}+\alpha P^{-1}W{P}^{-1}$, for every $\alpha \in \mathbb{R}$, $V,W\in T_P\mathbb{A}$,

2. 

Symmetry: $G_P$ is self-adjoint; in fact, it holds that ${\langle U,G_P\left(W\right)\rangle }^{\mathbb{A}}=\mathrm{tr}\left(U\left(P^{-1}W{P}^{-1}\right)\right)=\mathrm{tr}\left(\left(P^{-1}U{P}^{-1}\right)W\right)={\langle G_P\left(U\right),W\rangle }^{\mathbb{A}}$, by virtue of the cyclic permutation invariance of the trace operator,

3. 

Closure: $G_P$ turns out to be an endomorphism of $T_P{\mathbb{S}}^{+}\left(n\right)$; in fact, for every matrix $U\in T_P{\mathbb{S}}^{+}\left(n\right)$, it holds that $G_P\left(U\right)=P^{-1}U{P}^{-1}$ is a symmetric matrix. Hence, it belongs to $T_P{\mathbb{S}}^{+}\left(n\right)$ (notice that ${\left(P^{-1}U{P}^{-1}\right)}^{\top }=P^{-\top }{U}^{\top }{P}^{-\top }=P^{-1}U{P}^{-1}$ because both U and P are symmetric),

4. 

Invertibility: $G_P$ is invertible; in fact, if $W=G_P\left(U\right)=P^{-1}U{P}^{-1}$, then $U=PWP=:G_P^{-1}\left(W\right)$.

Notice that $G_P$ is not well-defined in ${\mathbb{R}}^{n\times n}$ (in fact, if P is taken in ${\mathbb{R}}^{n\times n}$, $P^{-1}$ does not necessarily exist).

As a last example, let us consider the Stiefel manifold $\mathrm{St}(n,p)\subset \mathbb{A}$ is endowed with the metric ${\langle U,W\rangle }_X:=\mathrm{tr}\left(U^{\top }(I_n-\frac{1}{2}X{X}^{\top })W\right)$, where $\mathbb{A}:={\mathbb{R}}^{n\times p}$ endowed with ${\langle U,V\rangle }^{\mathbb{A}}:=\mathrm{tr}\left(U^{\top }V\right)$. In this case, the metric kernel $G_X:{\mathbb{R}}^{n\times p}\to {\mathbb{R}}^{n\times p}$ reads

$$G_X\left(W\right):=(I_n-\frac{1}{2}X{X}^{\top })W.$$

(120)

Linearity and symmetry are readily proven. Closure may be proven as follows. Take $X\in \mathrm{St}(n,p)$ and $W\in T_X\mathrm{St}(n,p)$, namely $W^{\top }X+X^{\top }W=0_p$. Define $U:=(I_n-\frac{1}{2}X{X}^{\top })W$. It now suffices to show that $U\in T_X\mathrm{St}(n,p)$, which holds true; in fact,

$$\begin{array}{cc}\hfill U^{\top }X+X^{\top }U=& W^{\top }X-\frac{1}{2}{W}^{\top }X{X}^{\top }X+X^{\top }W-\frac{1}{2}{X}^{\top }X{X}^{\top }W\hfill \\ \hfill =& W^{\top }X+X^{\top }W-\frac{1}{2}\left(W^{\top }X+X^{\top }W\right)\hfill \\ \hfill =& 0_p.\hfill \end{array}$$

(121)

The invertibility of the matrix kernel follows from the invertibility of the matrix $I_n-\frac{1}{2}X{X}^{\top }$. From the Sherman–Morrison–Woodbury (matrix inversion) formula [47]

$${(A+UCV)}^{-1}=A^{-1}-A^{-1}U{(C^{-1}+V{A}^{-1}U)}^{-1}V{A}^{-1},$$

(122)

it follows, upon setting $A:=I_n$, $C:=-\frac{1}{2}{I}_p$, $U:=X$ and $V:=X^{\top }$, that

$${\left(I_n-\frac{1}{2}X{X}^{\top }\right)}^{-1}=I_n-X{(-2I_p+X^{\top }X)}^{-1}{X}^{\top }=I_n+X{X}^{\top },$$

(123)

that exists for every $X\in \mathrm{St}(n,p)$. ■

Further, let us examine separately and concisely the structure of the metric kernel for the symplectic group endowed with its canonical metric.

Example 16.

Let us consider the symplectic group $\mathrm{Sp}\left(2n\right)$ endowed with the canonical metric ${\langle V,W\rangle }_Q:=\mathrm{tr}\left({\left(Q^{-1}V\right)}^{\top }\left(Q^{-1}W\right)\right)$. Assume the symplectic group $\mathrm{Sp}\left(2n\right)$ to be embedded in the ambient space $\mathbb{A}:={\mathbb{R}}^{2n\times 2n}$ endowed with the Euclidean metric ${\langle U,V\rangle }^{\mathbb{A}}:=\mathrm{tr}\left(U^{\top }V\right)$. A simple expansion gives

$${\langle V,W\rangle }_Q:=\mathrm{tr}\left(V^{\top }{Q}^{-\top }{Q}^{-1}W\right),$$

(124)

hence the metric kernel reads $G_Q\left(W\right):=\left(Q^{-\top }{Q}^{-1}\right)W$.

7.2. Covariancy, Contravariancy, Tensors*

The convention on index used so far comes from the nature of different objects in relation to their behavior upon coordinate changes.

Let a smooth manifold $\mathbb{M}$ be a submanifold of dimension p of a Euclidean space and let us fix a curve $\gamma :[-\epsilon ,\epsilon ]\to \mathbb{M}$, described by:

$${\gamma }_x\left(t\right)=(x^1\left(t\right),x^2\left(t\right),\dots ,x^p\left(t\right)).$$

(125)

The subscript ${}_x$ tells that the components of the curve are expressed through coordinates x. The tangent vector $v:={\dot{\gamma }}_x\left(0\right)$ has the components:

$$v_x=({\dot{x}}^1\left(0\right),{\dot{x}}^2\left(0\right),\dots ,{\dot{x}}^p\left(0\right)).$$

(126)

Now, let us introduce a change of variable $x^i:={\kappa }^i({\xi }^1,{\xi }^2,\dots ,{\xi }^p)$, for $i=1,2,\dots ,p$, such that

$$\begin{array}{c}\hfill {\gamma }_{\xi }\left(t\right)=({\kappa }^1({\xi }^1\left(t\right),{\xi }^2\left(t\right),\dots ,{\xi }^p\left(t\right)),\dots ,{\kappa }^p({\xi }^1\left(t\right),{\xi }^2\left(t\right),\dots ,{\xi }^p\left(t\right))).\end{array}$$

(127)

In the new coordinates, the tangent vector at $t=0$ is expressed as

$$v_{\xi }=\left(\frac{\partial {\kappa }^1}{\partial {\xi }^i}{\dot{\xi }}^i\left(0\right),\frac{\partial {\kappa }^2}{\partial {\xi }^i}{\dot{\xi }}^i\left(0\right),\dots ,\frac{\partial {\kappa }^p}{\partial {\xi }^i}{\dot{\xi }}^i\left(0\right)\right)=\left(\frac{\partial x^1}{\partial {\xi }^i}{\dot{\xi }}^i\left(0\right),\frac{\partial x^2}{\partial {\xi }^i}{\dot{\xi }}^i\left(0\right),\dots ,\frac{\partial x^p}{\partial {\xi }^i}{\dot{\xi }}^i\left(0\right)\right).$$

(128)

At the heart of intrinsic manifold calculus is the requirement that one such tangent vector stays exactly the same, no matter which coordinates are used. Observe that the components $v_x^j:={\dot{x}}^j\left(0\right)$ and the components $v_{\xi }^j:={\dot{\xi }}^j\left(0\right)$ are related by

$$v_x^j=\frac{\partial x^j}{\partial {\xi }^i}{v}_{\xi }^i.$$

(129)

Such a set of equations prescribes how the components of a tangent vector vary upon a coordinate change. Every object that obeys the above law is termed contravariant. Contravariant objects are marked by an upper index.

The canonical basis vectors ${\partial }_i$ do not obey the law (129). In fact, from the requirement that $v_x=v_{\xi }$, it follows that

$$v_x^j{\partial }_j^x=\frac{\partial x^j}{\partial {\xi }^i}{v}_{\xi }^i{\partial }_j^x=v_{\xi }^i{\partial }_i^{\xi },$$

(130)

which implies that $\frac{\partial x^j}{\partial {\xi }^i}{\partial }_j^x={\partial }_i^{\xi }$. Multiplying both sides by $\frac{\partial {\xi }^i}{\partial x^k}$ gives

$$\frac{\partial {\xi }^i}{\partial x^k}\frac{\partial x^j}{\partial {\xi }^i}{\partial }_j^x=\frac{\partial {\xi }^i}{\partial x^k}{\partial }_i^{\xi }.$$

(131)

Since $\frac{\partial {\xi }^i}{\partial x^k}\frac{\partial x^j}{\partial {\xi }^i}=\frac{\partial x^j}{\partial x^k}={\delta }_k^j$, we obtain the transformation law

$${\partial }_k^x=\frac{\partial {\xi }^i}{\partial x^k}{\partial }_i^{\xi }.$$

(132)

Every object that obeys the above law is termed covariant. Covariant objects are marked by a lower index.

Concerning tangent vectors, we may summarize the above results saying that the components of a tangent vector $v\in T_x\mathbb{M}$ are contravariant, while the basis of a tangent space $T_x\mathbb{M}$ is covariant.

Given two tangent vectors $u,v\in T_x\mathbb{M}$, their inner products may be written as

$${\langle u,v\rangle }_x={\langle u^i{\partial }_i^x,v^j{\partial }_j^x\rangle }_x=u^i{v}^j{\langle {\partial }_i^x,{\partial }_j^x\rangle }_x,$$

(133)

due to the bilinearity of the inner product. Now, define $g_{ij}\left(x\right):={\langle {\partial }_i^x,{\partial }_j^x\rangle }_x$. Then, ${\langle u,v\rangle }_x=g_{ij}\left(x\right)u^i{v}^j$. The functions $g_{ij}$ are clearly symmetric in their indexes (hence, on a smooth manifold of dimension p, these functions are $p^2$, but only $2^{-1}p(p-1)$ are independent). Moreover, these functions are covariant in both indexes.

Given the canonical basis $\{\mathrm{d}{x}^1,\mathrm{d}{x}^2,\dots ,\mathrm{d}{x}^p\}$ of a cotangent space $T_x^{*}\mathbb{M}$, every covector may be written as $\omega ={\omega }_i\mathrm{d}{x}^i$. It is easy to show that the components of a covector are covariant, while the basis elements of a cotangent space are contravariant.

Vectors and covectors are special cases of more general objects termed tensors. A vector is a $(1,0)$-tensor, while a covector is a $(0,1)$-tensor. In general, it is possible to construct $(p,q)$-tensors by an operation termed tensor product denoted by ⊗. Each component of one such tensor has p upper indexes and q lower indexes. For example, one can construct the metric tensor $g=g_{ij}\mathrm{d}{x}^i\otimes \mathrm{d}{x}^j$, which is a $(0,2)$-tensor. Its inverse is $g^{-1}=g^{ij}{\partial }_i\otimes {\partial }_j$, which is a $(2,0)$-tensor. Additionally, ${\delta }_j^i$ represents the mixed components of the so-termed fundamental tensor$\delta ={\delta }_j^i{\partial }_i\otimes \mathrm{d}{x}^j$, which is a $(1,1)$-tensor.

Example 17.

Let us show in what sense functions $g_{ij}\left(x\right)$ are covariant components of a tensor. Given a variable change $x=\kappa \left(\xi \right)$, from property (132) it follows that

$$\begin{array}{cc}\hfill g_{ij}\left(\kappa \left(\xi \right)\right)=& {\langle {\partial }_i^x,{\partial }_j^x\rangle }_{\kappa \left(\xi \right)}\hfill \\ \hfill =& {〈\frac{\partial {\xi }^k}{\partial x^i}{\partial }_k^{\xi },\frac{\partial {\xi }^h}{\partial x^j}{\partial }_h^{\xi }〉}_{\kappa \left(\xi \right)}\hfill \\ \hfill =& \frac{\partial {\xi }^k}{\partial x^i}\frac{\partial {\xi }^h}{\partial x^j}{\langle {\partial }_k^{\xi },{\partial }_h^{\xi }\rangle }_{\kappa \left(\xi \right)}\hfill \\ \hfill =& \frac{\partial {\xi }^k}{\partial x^i}\frac{\partial {\xi }^h}{\partial x^j}{g}_{kh}\left(\xi \right).\hfill \end{array}$$

(134)

This is the transformation law of a $(0,2)$-tensor. The point is that the components $g_{ij}\left(x\right)$ transform into components $g_{kh}\left(\xi \right)$ through a linear expression, whose coefficients depend on the derivatives of one set of coordinates with respect to the other set. ■

8. Geodesic Arc, Riemannian Distance, Exponential and Logarithmic Map

A geodesic on a smooth manifold may be intuitively looked at in different ways:

• On a general manifold, the concept of geodesic extends the concept of a straight line from a flat space to a curved space. In fact, let us consider a curved manifold $\mathbb{M}$ embedded in an ambient space ${\mathbb{R}}^p$. Such ambient space contains straight lines in the usual meaning, but the manifold, being curved, hardly accommodates any straight lines. Geodesics are curves that resemble straight lines in that they copy some of their distinguishing features.
• On a metrizable manifold, a geodesic connecting two points is locally defined as the shortest curve on the manifold connecting these endpoints. Therefore, once a metric is specified, the equation of the geodesic arises from the minimization of a length functional. Such a definition comes from the observation that any straight line in ${\mathbb{R}}^p$ is indeed the shortest path connecting any two given points.
• A further distinguishing feature of straight lines is that they are self-parallel, namely, sliding a straight lines infinitesimally along itself returns the same exact lines. Such a concept gives rise to a definition of geodesic which requires to specify the mathematical meaning of ‘sliding a piece of line infinitesimally along itself’. The technical argument to access such a definition is covariant derivation, which is not covered in the present tutorial (while it will be covered in a forthcoming review paper).
• Another intuitive interpretation is based on the observation that a geodesic emanating from a point on a manifold coincides with the path followed by a particle sliding on the manifold at a constant speed. For a manifold embedded in a larger ambient space and in special circumstances, this is equivalent to the requirement that the naïve (or embedded) acceleration of the particle is either zero or perpendicular to the tangent space to the manifold at every point of its trajectory. (In the present tutorial paper, we use the term naïve acceleration to distinguish it from covariant acceleration, which will only be defined in a subsequent tutorial.)

Starting from the above informal description, we are going to examine in detail the notion of embedded geodesy. In particular, we shall treat the problem according to an energy-minimizing principle (which is in fact equivalent to a length-minimizing principle).

8.1. Coordinate-Free Embedded Geodesy

The length of a smooth curve $\gamma :[-ϵ,ϵ]\to \mathbb{M}$ may be evaluated though a rectification formula as:

$${\mathrm{L}}_{\gamma }:={\int }_{-ϵ}^{ϵ}\sqrt{{\langle \dot{\gamma }\left(t\right),\dot{\gamma }\left(t\right)\rangle }_{\gamma \left(t\right)}}\mathrm{d}t.$$

(135)

The net result of this argument is that, through the definition of an inner product on the tangent bundle to a Riemannian manifold, we are able to measure the lengths of paths in the manifold itself, which turns the manifold into a metric space.

Example 18.

Let us consider the manifold described in intrinsic coordinates:

$$\mathbb{M}:=\{({\theta }^1,{\theta }^2,sin\left({\theta }^1\right)+1)\in {\mathbb{R}}^3\mid ({\theta }^1,{\theta }^2)\in {\mathbb{R}}^2\}.$$

(136)

Let us consider the curve $({\theta }^1,{\theta }^2)=(t,t)$ obtained by mapping a segment of the plane ${\mathbb{R}}^2$ to the manifold $\mathbb{M}$, namely:

$$\gamma \left(t\right):=(t,t,sin(t)+1),t\in [a,b].$$

(137)

Let us assume the manifold $\mathbb{M}$ to be endowed with the metric ${\langle v,w\rangle }_x:=v^{\top }w$. In order to evaluate the length of such a curve, it is necessary to evaluate the velocity field $\dot{\gamma }$:

$$\dot{\gamma }\left(t\right)=(1,1,cos\left(t\right)),t\in [a,b].$$

(138)

The length of the curve is now obtained through the rectification formula:

$$\mathrm{L}\left(\gamma \right):={\int }_a^b\sqrt{1^2+1^2+{cos}^2\left(t\right)}\mathrm{d}t={\int }_a^b\sqrt{2+{cos}^2\left(t\right)}\mathrm{d}t.$$

(139)

Such an integral may not be expressed in terms of elementary functions. ■

On the basis of the concepts of ‘curve’ and of ‘length of a curve’, it is possible to define the notion of ‘distance between two points on a Riemannian manifold’ $d(·,·)$.

To define a notion of distance, let us consider what follows:

• Given a manifold $(\mathbb{M},\langle ·,·\rangle )$, take two arbitrary points $p,q\in \mathbb{M}$;
• Choose a curve $\gamma :[0,1]\to \mathbb{M}$ that has p and q as endpoints, namely, such that $\gamma \left(0\right)=p$ and $\gamma \left(1\right)=q$;
• Take as as distance $d(p,q)$ the length of such a curve, namely $\mathrm{L}\left(\gamma \right)$; one such definition seems a good starting point, but needs to be perfected since there exist infinitely many curves joining two given points.

It is important to underline that the notion of distance is not univocally defined since it depends on the inner product that a manifold is endowed with. Since there exist infinitely many curves joining two points on a manifold, a distance between two points p and q is actually defined as the length of the shortest curve connecting such two points, namely

$$d(p,q):=\underset{\gamma }{inf}{\int }_0^1\sqrt{{\langle \dot{\gamma }\left(t\right),\dot{\gamma }\left(t\right)\rangle }_{\gamma \left(t\right)}}\mathrm{d}t.$$

(140)

The problem of selecting the shortest path connecting two points is far from easy to solve. A possible method to look for it is the so-called variational method that we are going to introduce in the following.

The key point is to introduce an energy functional defined as

$$\mathcal{E}\left(\gamma \right):={\int }_0^1{\langle \dot{\gamma }\left(t\right),\dot{\gamma }\left(t\right)\rangle }_{\gamma \left(t\right)}\mathrm{d}t,$$

(141)

whose minimum argument coincides to a geodesic. The variational method consists of looking for a curve that makes the energy functional stationary, namely, for a curve such that $\delta \mathcal{E}\left(\gamma \right)=0$. A concept that facilitates the formulation of the variational method is that of normal space to a manifold in a point.

Clearly, the above definition introduces yet another degree of freedom, since the definition of normal space is based on a choice of inner product ${\langle ·,·\rangle }^{\mathbb{A}}$ to evaluate orthogonality. It is instructive to notice that, since a normal space is defined as the orthogonal complement to a tangent space with respect to the ambient space, then

$$T_x\mathbb{M}\oplus N_x\mathbb{M}=\mathbb{A}.$$

(142)

In other terms, every element in $\mathbb{A}$ may be decomposed exactly as the sum of a tangent and a normal vector.

Example 19.

Let us evaluate the structure of the normal spaces to the hypersphere ${\mathbb{S}}^{n-1}$ embedded in $\mathbb{A}:={\mathbb{R}}^n$. Let us recall that ${\mathbb{S}}^{n-1}:=\{x\in {\mathbb{R}}^n\mid x^{\top }x=1\}$ and that $T_x{\mathbb{S}}^{n-1}=\{v\in {\mathbb{R}}^n\mid v^{\top }x=0\}$. Let us select ${\langle y,z\rangle }^{\mathbb{A}}:=y^{\top }z$ for every $y,z\in \mathbb{A}$. Since, by definition,

$$N_x{\mathbb{S}}^{n-1}:=\{z\in {\mathbb{R}}^n\mid z^{\top }v=0,\forall v\in {\mathbb{R}}^{n}suchthat{v}^{\top }x=0\},$$

(143)

it is easily found that

$$N_x{\mathbb{S}}^{n-1}=\{\lambda x\mid \lambda \in \mathbb{R}\}.$$

(144)

Let us further determine the structure of the normal spaces for the manifold $\mathbb{SO}\left(n\right)$. Let us recall that $\mathbb{SO}\left(n\right):=\{R\in {\mathbb{R}}^{n\times n}\mid R^{\top }R=I_n,det\left(R\right)=1\}$ and that $T_R\mathbb{SO}\left(n\right)=\{V\in {\mathbb{R}}^{n\times n}\mid V^{\top }R+R^{\top }V=0\}$. Let us take $\mathbb{A}:={\mathbb{R}}^{n\times n}$ and ${\langle Y,Z\rangle }^{\mathbb{A}}:=\mathrm{tr}\left(Y^{\top }Z\right)$ for $Y,Z\in \mathbb{A}$. Let us also recall that

$$V^{\top }R+R^{\top }V={\left(R^{\top }V\right)}^{\top }+R^{\top }V=H^{\top }+H=0_n,$$

(145)

where $H:=R^{\top }V$. Hence, any tangent vector at R may be written as $V=RH$ with H being skew-symmetric. By definition,

$$N_R\mathbb{SO}\left(n\right):=\{Z\in {\mathbb{R}}^{n\times n}\mid \mathrm{tr}\left(Z^{\top }V\right)=0,\forall V=RH,H^{\top }+H=0_n\},$$

(146)

and hence it is readily found that

$$N_R\mathbb{SO}\left(n\right)=\{RS\mid S\in {\mathbb{R}}^{n\times n},S^{\top }=S\}.$$

(147)

As a verification step, let us show that, for every $n\times n$ special orthogonal matrix R, skew-symmetric matrix H and symmetric matrix S, it holds that ${\langle RH,RS\rangle }^{\mathbb{A}}=0$. To this aim, let us show that:

$${\langle RH,RS\rangle }^{\mathbb{A}}=\mathrm{tr}\left({\left(RH\right)}^{\top }RS\right)=\mathrm{tr}\left(H^{\top }\left(R^{\top }R\right)S\right)=-\mathrm{tr}\left(HS\right).$$

(148)

Now, it is not hard to prove that every skew-symmetric matrix H is ‘orthogonal’ to every symmetric matrix S; in fact:

$$\underset{Because\mathrm{tr}\left(X\right)=\mathrm{tr}\left(X^{\top }\right)}{\underbrace{\mathrm{tr}\left(HS\right)=\mathrm{tr}\left({\left(HS\right)}^{\top }\right)}}=\mathrm{tr}\left(S^{\top }{H}^{\top }\right)=\underset{Because\mathrm{tr}\left(XY\right)=\mathrm{tr}\left(YX\right)}{\underbrace{\mathrm{tr}\left(S\right(-H\left)\right)=\mathrm{tr}\left(\right(-H\left)S\right)}}=-\mathrm{tr}\left(HS\right),$$

(149)

and therefore $\mathrm{tr}\left(HS\right)=0$. From this result it follows that ${\langle RH,RS\rangle }^{\mathbb{A}}=0$.

As a last example, let us determine the structure of the normal space to the manifold ${\mathbb{S}}^{+}\left(n\right)$. Recall that ${\mathbb{S}}^{+}\left(n\right):=\{P\in {\mathbb{R}}^{n\times n}\mid P^{\top }=P,P>0\}$ and that $T_P{\mathbb{S}}^{+}\left(n\right)=\{V\in {\mathbb{R}}^{n\times n}\mid V^{\top }-V=0\}$. In this case, choose $\mathbb{A}:={\mathbb{R}}^{n\times n}$ and ${\langle Y,Z\rangle }^{\mathbb{A}}:=\mathrm{tr}\left(Y^{\top }Z\right)$ for every $Y,Z\in \mathbb{A}$. By definition, it holds that

$$N_P{\mathbb{S}}^{+}\left(n\right):=\{Z\in {\mathbb{R}}^{n\times n}\mid \mathrm{tr}\left(Z^{\top }V\right)=0,\forall Vsuchthat{V}^{\top }-V=0\},$$

(150)

from which it turns out that

$$N_P{\mathbb{S}}^{+}\left(n\right)=\{Z\in {\mathbb{R}}^{n\times n}\mid Z^{\top }+Z=0\}.$$

(151)

In fact, $N_P{\mathbb{S}}^{+}\left(n\right)$ represents the set of $n\times n$ matrices that are orthogonal to all $n\times n$ symmetric matrices, which coincides with the set of all $n\times n$ skew-symmetric matrices. Notice that $N_P{\mathbb{S}}^{+}\left(n\right)$ does not depend explicitly on the base point P. ■

The notion of normal space affords formulating the variational problem $\delta \mathcal{E}\left(\gamma \right)=0$ as a differential inclusion. For every specific manifold, it will then be possible to turn such a differential inclusion into a differential equation to be solved under appropriate initial conditions. Let us consider a curve $\gamma :[0,1]\to \mathbb{M}$ and let us define the fundamental form$\mathcal{F}:T\mathbb{M}\to \mathbb{R}$ as:

$$\mathcal{F}(x,v):={\langle v,v\rangle }_x.$$

(152)

The following result holds.

Theorem 2.

For a manifold $\mathbb{M}$, the variational equation $\delta \mathcal{E}\left(\gamma \right)=0$ is equivalent to the following differential inclusion:

$$\frac{\partial \mathcal{F}}{\partial \gamma }-\frac{\mathrm{d}}{\mathrm{d}t}\frac{\partial \mathcal{F}}{\partial \dot{\gamma }}\in N_{\gamma }\mathbb{M}.$$

(153)

Notice that such a relation appears as a generalized form of the well-known Euler–Lagrange equation of rational mechanics [48].

Proof. 

In order to prove such an important result, notice that the energy functional may be rewritten as:

$$\mathcal{E}\left(\gamma \right)={\int }_0^1\mathcal{F}(\gamma ,\dot{\gamma })\mathrm{d}t.$$

(154)

It is important to notice that, by virtue of the property (118), for every $(x,v)\in T\mathbb{M}$, there exists a neighborhood of $(x,v)$ in ${\mathbb{A}}^2$, where the partial derivatives $\frac{\partial \mathcal{F}(x,v)}{\partial x}$ and $\frac{\partial \mathcal{F}(x,v)}{\partial v}$ exist (technically, one should use an extension $\overline{\mathcal{F}}$ in place of $\mathcal{F}$). Such partial derivatives are intended in the Gateaux sense, namely

$$\begin{array}{cc}\hfill & \underset{ϵ\to 0}{lim}\frac{\mathcal{F}(x+ϵu,v)-\mathcal{F}(x,v)}{ϵ}=:{〈\frac{\partial \mathcal{F}(x,v)}{\partial x},u〉}^{\mathbb{A}},\mathrm{for}\mathrm{every}u\in \mathbb{A},\hfill \\ \hfill & \underset{ϵ\to 0}{lim}\frac{\mathcal{F}(x,v+ϵu)-\mathcal{F}(x,v)}{ϵ}=:{〈\frac{\partial \mathcal{F}(x,v)}{\partial v},u〉}^{\mathbb{A}},\mathrm{for}\mathrm{every}u\in \mathbb{A}.\hfill \end{array}$$

(155)

Let us define an arbitrary smooth variation $\eta :[0,1]\to T\mathbb{M}$, namely a deviation from the geodesic $\gamma$, such that $\eta \left(0\right)=\eta \left(1\right)=0$.

For a sufficiently small constant $ϵ>0$, it is possible to evaluate the quantity $\mathcal{F}(\gamma +ϵ\eta ,\dot{\gamma }+ϵ\dot{\eta })$. Applying the Taylor series expansion in ${\mathbb{A}}^2$ gives:

$$\mathcal{F}(\gamma +ϵ\eta ,\dot{\gamma }+ϵ\dot{\eta })=\mathcal{F}(\gamma ,\dot{\gamma })+ϵ{〈\frac{\partial \mathcal{F}}{\partial \gamma },\eta 〉}^{\mathbb{A}}+ϵ{〈\frac{\partial \mathcal{F}}{\partial \dot{\gamma }},\dot{\eta }〉}^{\mathbb{A}}+\mathcal{O}\left({ϵ}^2\right),$$

(156)

where $\mathcal{O}$ denotes a Landau symbol and it is used to compactly represent the remainder of the series (e.g., in Lagrange form). The variation of the energy of a curve may be concretely defined as:

$$\begin{array}{cc}\hfill \delta \mathcal{E}\left(\gamma \right):=& \underset{ϵ\to 0}{lim}\frac{\mathcal{E}(\gamma +ϵ\eta )-\mathcal{E}\left(\gamma \right)}{ϵ}=\underset{ϵ\to 0}{lim}\frac{1}{ϵ}{\int }_0^1[\mathcal{F}(\gamma +ϵ\eta ,\dot{\gamma }+ϵ\dot{\eta })-\mathcal{F}(\gamma ,\dot{\gamma })]\mathrm{d}t\hfill \\ \hfill =& {\int }_0^1{〈\frac{\partial \mathcal{F}}{\partial \gamma },\eta 〉}^{\mathbb{A}}\mathrm{d}t+{\int }_0^1{〈\frac{\partial \mathcal{F}}{\partial \dot{\gamma }},\dot{\eta }〉}^{\mathbb{A}}\mathrm{d}t+{\int }_0^1\underset{ϵ\to 0}{lim}\frac{\mathcal{O}\left({ϵ}^2\right)}{ϵ}\mathrm{d}t.\hfill \end{array}$$

(157)

The last integral is null. The second last integral may be rewritten, upon integration by parts, as:

$${\int }_0^1{〈\frac{\partial \mathcal{F}}{\partial \dot{\gamma }},\dot{\eta }〉}^{\mathbb{A}}\mathrm{d}t={\left.{〈\frac{\partial \mathcal{F}}{\partial \dot{\gamma }},\eta 〉}^{\mathbb{A}}\right|}_0^1-{\int }_0^1{〈\frac{\mathrm{d}}{\mathrm{d}t}\frac{\partial \mathcal{F}}{\partial \dot{\gamma }},\eta 〉}^{\mathbb{A}}\mathrm{d}t.$$

(158)

The first addendum on the right-hand side is null because the variation $\eta$ vanishes at the endpoints of the curve; therefore,

$$\begin{array}{c}\hfill \delta \mathcal{E}\left(\gamma \right)={\int }_0^1{〈\frac{\partial \mathcal{F}}{\partial \gamma }-\frac{\mathrm{d}}{\mathrm{d}t}\frac{\partial \mathcal{F}}{\partial \dot{\gamma }},\eta 〉}^{\mathbb{A}}\mathrm{d}t.\end{array}$$

(159)

By imposing that $\delta \mathcal{E}\left(\gamma \right)=0$ it is readily obtained that, since the variation $\eta$ is arbitrary, it must hold the relation (153). □

Let us apply the above theorem to a number of manifolds.

Example 20.

Let us determine the equation of geodesics on the hypersphere ${\mathbb{S}}^{n-1}$ through Equation (153). Let us recall that ${\mathbb{S}}^{n-1}:=\{x\in {\mathbb{R}}^n\mid x^{\top }x=1\}$; assume that ${\langle v,w\rangle }_x:=v^{\top }w$ (independently of x) and let us recall that $N_x{\mathbb{S}}^{n-1}=\{\lambda x\mid \lambda \in \mathbb{R}\}$. Now, the equation of the geodesic reads:

$$\frac{\partial \left({\dot{\gamma }}^{\top }\dot{\gamma }\right)}{\partial \gamma }-\frac{\mathrm{d}}{\mathrm{d}t}\frac{\partial \left({\dot{\gamma }}^{\top }\dot{\gamma }\right)}{\partial \dot{\gamma }}=0-2\frac{\mathrm{d}\dot{\gamma }}{\mathrm{d}t}=-2\ddot{\gamma }\in \{\lambda \gamma \mid \lambda \in \mathbb{R}\}$$

(160)

In summary, the equation for the geodesic is

$$\ddot{\gamma }\left(t\right)=\lambda \gamma \left(t\right).$$

(161)

(Since λ is arbitrary, it absorbed the factor $-2$.) The geodesic equation is then a second-order differential equation in γ that needs two conditions to be solved.

As a further example, let us derive the geodesic equation for the manifold $\mathbb{SO}\left(n\right)$. Let us recall that $\mathbb{SO}\left(n\right):=\{R\in {\mathbb{R}}^{n\times n}\mid R^{\top }R=I_n,det\left(R\right)=1\}$; the canonical inner product in this space is ${\langle V,W\rangle }_R:=\mathrm{tr}\left(V^{\top }W\right)$ (independently of R) and that $N_R\mathbb{SO}\left(n\right)=\{RS\mid S\in {\mathbb{R}}^{n\times n},S=S^{\top }\}$. According to the general principle, the geodesic equation reads as

$$\frac{\partial \left({\dot{\gamma }}^{\top }\dot{\gamma }\right)}{\partial \gamma }-\frac{\mathrm{d}}{\mathrm{d}t}\frac{\partial \left({\dot{\gamma }}^{\top }\dot{\gamma }\right)}{\partial \dot{\gamma }}=0-2\frac{\mathrm{d}\dot{\gamma }}{\mathrm{d}t}=-2\ddot{\gamma }\in \{\gamma S\mid S\in {\mathbb{R}}^{n\times n},S=S^{\top }\}$$

(162)

In summary, the geodesic equation on the space $\mathbb{SO}\left(n\right)$ reads as

$$\ddot{\gamma }=\gamma S,$$

(163)

with S denoting an arbitrary symmetric function. ■

Through the notion of geodesic arcs, one may define the notion of Riemannian (or geodesic) distance between two nearby points in a smooth manifold. In fact, let us denote by $x,y\in \mathbb{M}$ two nearby points and assume that there exists a unique geodesic arc ${\gamma }_x^y:[0,1]\to \mathbb{M}$ such that ${\gamma }_x^y\left(0\right)=x$ and ${\gamma }_x^y\left(1\right)=y$. By definition, the length of such geodesic arcs is taken as the distance between its two endpoints, namely:

$$d(x,y):={\int }_0^1\sqrt{{\langle {\dot{\gamma }}_x^y\left(t\right),{\dot{\gamma }}_x^y\left(t\right)\rangle }_{{\gamma }_x^y\left(t\right)}}\mathrm{d}t.$$

(164)

It is worth noticing, at this point, that the expression of the geodesic distance may be rewritten in terms of the fundamental form $\mathcal{F}$, namely

$$d(x,y)={\int }_0^1\sqrt{\mathcal{F}({\gamma }_x^y,{\dot{\gamma }}_x^y)}\mathrm{d}t.$$

(165)

A noteworthy (and extremely useful) result that we are going to prove is that, along a geodesic, the fundamental form keeps constant. We shall see after the proof that such a result noticeably simplifies the computation of the geodesic distance.

Theorem 3.

Along a geodesic arc $\gamma :[0,1]\to \mathbb{M}$, the function $\mathcal{F}(\gamma \left(t\right),\dot{\gamma }\left(t\right))$ stays constant with respect to t.

Proof. 

Let us mention that, for any given pair $(x,v)\in T\mathbb{M}$, it holds that

$${〈\frac{\partial \mathcal{F}(x,v)}{\partial v},v〉}^{\mathbb{A}}=2\mathcal{F}(x,v),$$

(166)

which clearly stems from the fact that $\mathcal{F}(x,v)$ is quadratic in v. Such a property may be shown by recalling that $\mathcal{F}(x,v)={\langle v,G_x\left(v\right)\rangle }^{\mathbb{A}}$, where G denotes again a metric kernel, and by noticing that

$$\frac{\partial \mathcal{F}(x,v)}{\partial v}=2G_x\left(v\right),$$

(167)

from which it follows that

$${〈\frac{\partial \mathcal{F}(x,v)}{\partial v},v〉}^{\mathbb{A}}={〈2G_x\left(v\right),v〉}^{\mathbb{A}},$$

(168)

hence the property (166). Let us now show that the differential inclusion (153) and the property (166) imply the constancy of the fundamental form $\mathcal{F}$ along a geodesic, namely that:

$$\mathcal{F}(\gamma \left(t\right),\dot{\gamma }\left(t\right))=\mathcal{F}(\gamma \left(0\right),\dot{\gamma }\left(0\right))foreveryvaluet\in [0,1]$$

(169)

on every geodesic. By multivariable calculus it is readily proven that:

$$\frac{\mathrm{d}}{\mathrm{d}t}\mathcal{F}(\gamma ,\dot{\gamma })={〈\frac{\partial \mathcal{F}}{\partial \gamma },\frac{\mathrm{d}\gamma }{\mathrm{d}t}〉}^{\mathbb{A}}+{〈\frac{\partial \mathcal{F}}{\partial \dot{\gamma }},\frac{\mathrm{d}\dot{\gamma }}{\mathrm{d}t}〉}^{\mathbb{A}}.$$

(170)

Integrating such an equation over the interval $[0,t]$ gives:

$${\int }_0^t\mathrm{d}\mathcal{F}={\int }_0^t{〈\frac{\partial \mathcal{F}}{\partial \gamma },\frac{\mathrm{d}\gamma }{\mathrm{d}t}〉}^{\mathbb{A}}\mathrm{d}t+{\int }_0^t{〈\frac{\partial \mathcal{F}}{\partial \dot{\gamma }},\frac{\mathrm{d}\dot{\gamma }}{\mathrm{d}t}〉}^{\mathbb{A}}\mathrm{d}t.$$

(171)

The integral on the left-hand side is equal to ${\left.\mathcal{F}\right|}_0^t$, while the last integral on the right-hand side may be evaluated through integration by parts and equals

$$\begin{array}{cc}\hfill {\int }_0^t{〈\frac{\partial \mathcal{F}}{\partial \dot{\gamma }},\frac{\mathrm{d}\dot{\gamma }}{\mathrm{d}t}〉}^{\mathbb{A}}\mathrm{d}t=& {\left.{〈\frac{\partial \mathcal{F}}{\partial \dot{\gamma }},\frac{\mathrm{d}\gamma }{\mathrm{d}t}〉}^{\mathbb{A}}\right|}_0^t-{\int }_0^t{〈\frac{\mathrm{d}}{\mathrm{d}t}\frac{\partial \mathcal{F}}{\partial \dot{\gamma }},\frac{\mathrm{d}\gamma }{\mathrm{d}t}〉}^{\mathbb{A}}\mathrm{d}t\hfill \\ \hfill =& {\left.2\mathcal{F}\right|}_0^t-{\int }_0^t{〈\frac{\mathrm{d}}{\mathrm{d}t}\frac{\partial \mathcal{F}}{\partial \dot{\gamma }},\frac{\mathrm{d}\gamma }{\mathrm{d}t}〉}^{\mathbb{A}}\mathrm{d}t,\hfill \end{array}$$

(172)

by Equation (166). From Equations (171) and (172), it follows that:

$${\left.\mathcal{F}\right|}_0^t={\int }_0^t{〈\frac{\partial \mathcal{F}}{\partial \gamma }-\frac{\mathrm{d}}{\mathrm{d}t}\frac{\partial \mathcal{F}}{\partial \dot{\gamma }},\frac{\mathrm{d}\gamma }{\mathrm{d}t}〉}^{\mathbb{A}}\mathrm{d}t+2{\left.\mathcal{F}\right|}_0^t,$$

(173)

hence

$$-{\left.\mathcal{F}\right|}_0^t={\int }_0^t{〈\frac{\partial \mathcal{F}}{\partial \gamma }-\frac{\mathrm{d}}{\mathrm{d}t}\frac{\partial \mathcal{F}}{\partial \dot{\gamma }},\frac{\mathrm{d}\gamma }{\mathrm{d}t}〉}^{\mathbb{A}}\mathrm{d}t.$$

(174)

From the differential inclusion (153), it follows that the integrand is null; therefore, ${\left.\mathcal{F}\right|}_0^t=0$ for every $t\in [0,1]$. □

The above result means that any geodesic trajectory is traveled at a constant speed. Since, along a geodesic arc, the speed $\sqrt{{\langle {\dot{\gamma }}_x^y\left(t\right),{\dot{\gamma }}_x^y\left(t\right)\rangle }_{{\gamma }_x^y\left(t\right)}}$ is constant for any t, it holds that

$$d(x,y)=\sqrt{{\langle {\dot{\gamma }}_x^y\left(0\right),{\dot{\gamma }}_x^y\left(0\right)\rangle }_x}.$$

(175)

It is worth underlining that, provided that one knows the functional expression of the geodesic connecting two points, their distance does not require integration, but just algebraic operations. (The difficulty is hidden by the fact that finding the exact expression of a geodesic connecting two points may be more troublesome than one might expect.)

We make an observation in order to justify an idea presented at the beginning of this subsection about normal naïve acceleration for a geodesic.

Observation 2.

Let us take a closer look at the differential inclusion (153), where $\mathcal{F}(x,v)=\langle v,G_x\left(v\right)\rangle$, namely

$$\frac{\partial {\langle \dot{\gamma },G_{\gamma }\left(\dot{\gamma }\right)\rangle }^{\mathbb{A}}}{\partial \gamma }-\frac{\mathrm{d}}{\mathrm{d}t}\frac{\partial {\langle \dot{\gamma },G_{\gamma }\left(\dot{\gamma }\right)\rangle }^{\mathbb{A}}}{\partial \dot{\gamma }}\in N_{\gamma }\mathbb{M}.$$

(176)

Under the hypothesis that the metric kernel $G_x$ does not depend explicitly on the point x, the first term on the left-hand side vanishes. In this case, let us plainly assume that $G_x\equiv {\mathrm{id}}_x$. In addition, we have already established that $\frac{\partial {\langle \dot{\gamma },G_{\gamma }\left(\dot{\gamma }\right)\rangle }^{\mathbb{A}}}{\partial \dot{\gamma }}=2G_{\gamma }\left(\dot{\gamma }\right)$. The above differential inclusion may hence be written as

$$(G_x\equiv {\mathrm{id}}_x\Rightarrow )\ddot{\gamma }\in N_{\gamma }\mathbb{M},$$

(177)

namely, the naïve acceleration is perpendicular to the velocity. This is perhaps the simplest form of a geodesic equation on a manifold. This is, in fact, the case for the hypersphere surveyed in an example.

A geodesic arc may be expressed in terms of two pieces of information, in addition to its endpoints, such as the initial values $\gamma \left(0\right)=x\in \mathbb{M}$ and $\dot{\gamma }\left(0\right)=v\in T_x\mathbb{M}$, which represent the point where a geodesic departs from and its initial speed, respectively. A geodesic arc determined by these two pieces of information will be denoted as ${\gamma }_{x,v}:[0,1]\to \mathbb{M}$.

Example 21.

Let us consider a geodesic ${\gamma }_{x,v}:[0,1]\to \mathbb{M}$, with $(x,v)\in T\mathbb{M}$, such that ${\dot{\gamma }}_{x,v}\left(0\right)=v$. In this case, it holds that

$$\mathrm{L}\left({\gamma }_{x,v}\right)=\sqrt{{\langle v,v\rangle }_x}=\sqrt{\mathcal{F}(x,v)}.$$

(178)

■

8.2. Exponential and Logarithmic Maps

To a geodesic arc ${\gamma }_{x,v}:[0,1]\to \mathbb{M}$ is associated a map ${exp}_x:T_x\mathbb{M}\to \mathbb{M}$ defined as:

$${exp}_x\left(v\right):={\gamma }_{x,v}\left(1\right).$$

(179)

The function (179) is termed exponential map with pole $x\in \mathbb{M}$ and is of paramount importance in manifold calculus. The exponential map $exp:T\mathbb{M}\to \mathbb{M}$ takes two arguments, the point $x\in \mathbb{M}$ over the manifold and a vector $v\in T_x\mathbb{M}$ tangent to the manifold at the pole x, and returns a point on the manifold itself. A practical reading of the exponential map is that it advances a point x toward a direction v, like vector addition moves a point along a straight line in flat spaces. In other words, the following two expressions are the counterparts of each other in a flat space ${\mathbb{R}}^n$ and in a manifold $\mathbb{M}$:

$$\left\{\begin{array}{c}{\gamma }_{x,v}\left(t\right)=x+tvin{\mathbb{R}}^n,t\in [0,1],\hfill \\ {\gamma }_{x,v}\left(t\right)={exp}_x\left(tv\right)in\mathbb{M},t\in [0,1].\hfill \end{array}\right.$$

(180)

Much like the first expression denotes the shortest/straightest line on a flat space (endowed with a Euclidean metric), the second expression denotes the shortest/straightest line over a curved manifold.

The exponential map depends on the pair $(x,v)\in T\mathbb{M}$ and is locally invertible around $v=0$. This follows from the fact that ${exp}_x\left(0\right)=x$ and by local invertibility results. In the flat manifold ${\mathbb{R}}^n$, the exponential map may be inverted easily; in fact

$$\mathrm{from}y=x+v\mathrm{it}\mathrm{follows}\mathrm{that}v=y-x.$$

(181)

In the flat manifold ${\mathbb{R}}^n$, we recover the classical and intuitive meaning of the exponential map and of its inverse:

• Exponential map: Given a point x and a vector v in ${\mathbb{R}}^n$, the exponential map ${exp}_x\left(v\right):=x+v$ moves the point from x to $x+v$. (Indeed, the term ‘vector’ comes from the homonymus Latin term that means ‘transporter’.)
• Inverse exponential map: Given two points x and y in ${\mathbb{R}}^n$, the inverse of the exponential map ${exp}_x^{-1}\left(y\right)$ returns a vector v such that ${exp}_x\left(v\right)={exp}_x\left({exp}_x^{-1}\left(y\right)\right)=x+(y-x)\equiv y$. In other words, the inverse exponential map applied to two points $x,y$ determines the vector v that ‘transports’ x to y.

Let us observe that the inverse exponential map is non-symmetric in its arguments, namely the notation ${exp}_x^{-1}\left(y\right)$ makes it immediately clear that it is not allowed to swap x with y. Indeed, even in the flat space ${\mathbb{R}}^n$, it holds that ${exp}_x^{-1}\left(y\right):=y-x=-(x-y)=-{exp}_y^{-1}\left(x\right)$. (In the case of a curved manifold, such a reciprocity relation is more convoluted.)

Generally, the inverse exponential map is termed logarithmic map. On a manifold $\mathbb{M}$ where it is defined an exponential map $exp:T\mathbb{M}\to \mathbb{M}$ a logarithmic map is denoted as $log:{\mathbb{M}}^2\to T\mathbb{M}$. A logarithmic map takes as arguments two points $x,y\in \mathbb{M}$ and returns a vector$v={log}_x\left(y\right)\in T_x\mathbb{M}$. It is important to underline that a logarithmic map is defined only locally, namely, only if $x,y\in \mathbb{M}$ lay sufficiently close to one another. The lack of a global logarithmic map may be understood as follows. A logarithmic map is defined by the following equation:

$${exp}_x\left({log}_{x}y\right)=y,$$

(182)

and therefore, the existence of ${log}_{x}y$ is tied to the chance of determining one and only one geodesic arc ${\gamma }_x^y$ that connects the given points $x,y\in \mathbb{M}$, and then, by the equality ${\gamma }_x^y\left(t\right)={\gamma }_{x,v}\left(t\right)$, it follows that ${log}_{x}y\equiv v$. However, not every pair of points may be connected by a unique geodesic, and hence a global logarithmic map, in general, fails to exist. A quick example of such an unavoidable problem is found on the sphere ${\mathbb{S}}^2$; taking x as the North pole and y as the South pole, there exist infinitely many geodesic lines that connect the two poles. Hence, ${log}_{x}y$ is undetermined.

Let us examine the expressions of geodesic arcs, geodesic distances and exponential maps for manifolds of interest in system theory and non-linear control.

• Hypercube: The space ${\mathbb{R}}^p$ endowed with the Euclidean metric admits straight lines as geodesics; in fact, since $T_x{\mathbb{R}}^p={\mathbb{R}}^p$ it follows that $N_x{\mathbb{R}}^p=\left\{0\right\}$, and hence the geodesic equation is simply ${\ddot{\gamma }}_{x,v}=0$ and its solution is ${\gamma }_{x,v}\left(t\right)=x+tv$ for every $x\in {\mathbb{R}}^p$, $v\in T_x{\mathbb{R}}^p$ and $t\in [0,1]$. Now, take two points $x,y\in {\mathbb{R}}^p$ and look for a geodesic arc connecting them with $t\in [0,1]$. It is necessary to find a vector $v\in T_x{\mathbb{R}}^p$ such that ${\gamma }_{x,v}\left(1\right)=y$. Such a vector is clearly $v=y-x$; hence, the unique geodesic arc connecting x to y is ${\gamma }_x^y\left(t\right)=x+t(y-x)$. Since $\sqrt{{\langle {\dot{\gamma }}_x^y\left(0\right),{\dot{\gamma }}_x^y\left(0\right)\rangle }_x}=\parallel v\parallel$, the Riemannian distance reads

  $$d(x,y)=\parallel y-x\parallel ,$$

  (183)

  which is a well-known result from calculus and geometry.
• Hypersphere: On the hypersphere ${\mathbb{S}}^{p-1}$ embedded in the Euclidean space ${\mathbb{R}}^p$, a geodesic line may be conceived as a curve on which a particle, departing from the point $x\in {\mathbb{S}}^{p-1}$ with velocity $v\in T_x{\mathbb{S}}^{p-1}$, slides with constant speed $\parallel v\parallel$, where $\parallel ·\parallel$ denotes the standard $L_2$ vector norm. On the hypersphere, we denote such a curve as ${\gamma }_{x,v}\left(t\right)$, where the variable $t\in [0,1]$ provides a parametrization of the curve. The differential equation characterizing geodesics on the hypersphere may be determined by observing that, with the given conditions, in this case the naïve acceleration of the particle must be either null or normal to the tangent space at any point of the hypersphere itself, namely ${\ddot{\gamma }}_{x,v}\left(t\right)\in N_{{\gamma }_{x,v}\left(t\right)}{\mathbb{S}}^{p-1}$. Since the normal space to a hypersphere at a point x is radial along x, the geodesic equation reads as ${\ddot{\gamma }}_{x,v}\left(t\right)=\lambda {\gamma }_{x,v}\left(t\right)$. In explicit form, the equation of the geodesic on the unit hypersphere may be written as [49]:

  $${\gamma }_{x,v}\left(t\right)=xcos(\parallel v\parallel t)+{vsin(\parallel v\parallel t)\parallel v\parallel }^{-1},t\in [0,1],$$

  (184)

  as it is easy to verify by substitution. Additionally, it is easy to verify that ${\gamma }_{x,v}\left(0\right)=x$, ${\dot{\gamma }}_{x,v}\left(0\right)=v$ and that ${\gamma }_{x,v}{\left(t\right)}^{\top }{\gamma }_{x,v}\left(t\right)=1$ for every t. The exponential map associated with the above geodesic is

  $${exp}_x\left(v\right):=xcos(\parallel v\parallel )+{vsin(\parallel v\parallel )\parallel v\parallel }^{-1}.$$

  (185)
  The relationship (184) for the geodesic represents a ‘great circle’ on the hypersphere. Now let us take two points $x,y\in {\mathbb{S}}^{p-1}$ (non-antipodal, such that $x^{\top }y\ne -1$) and let us look for a geodesic arc of the form (184) connecting them. It is clearly necessary to find a vector $v\in T_x{\mathbb{S}}^{p-1}$ such that ${\gamma }_{x,v}\left(1\right)=y$. Such an equation in the unknown v may be expressed explicitly as

  $$xcos(\parallel v\parallel )+v\mathrm{sinc}(\parallel v\parallel )=y,$$

  (186)

  where ‘$\mathrm{sinc}$’ denotes the cardinal sine function. Pre-multiplying the above equation by $x^{\top }$ gives $cos(\parallel v\parallel )=x^{\top }y$, namely $\parallel v\parallel =\mathrm{acos}\left(x^{\top }y\right)$; hence

  $$v=\frac{y-x\left(x^{\top }y\right)}{\mathrm{sinc}\left(\mathrm{acos}\left(x^{\top }y\right)\right)}.$$

  (187)
  This expression represents the inverse of the exponential map applied to points $x,y\in {\mathbb{S}}^{p-1}$, namely

  $${log}_{x}y:=\frac{(I_p-x{x}^{\top })y}{\mathrm{sinc}\left(\mathrm{acos}\left(x^{\top }y\right)\right)}.$$

  (188)
  Notice that such a logarithmic map is defined only when $x^{\top }y\ne -1$, namely when the two points are antipodal. The unique geodesic arc connecting x to y is given by

  $${\gamma }_x^y\left(t\right)=xcos\left(\mathrm{acos}\left(x^{\top }y\right)t\right)+\frac{y-x\left(x^{\top }y\right)}{\mathrm{sinc}\left(\mathrm{acos}\left(x^{\top }y\right)\right)}\mathrm{sinc}\left(\mathrm{acos}\left(x^{\top }y\right)t\right).$$

  (189)
  A noticeable consequence is that, since $\sqrt{{\langle {\dot{\gamma }}_x^y\left(0\right),{\dot{\gamma }}_x^y\left(0\right)\rangle }_x}=\parallel v\parallel$, the Riemannian distance between the points x and y reads

  $$d(x,y)=\mathrm{acos}\left(x^{\top }y\right),$$

  (190)

  where the inverse cosine function ‘$\mathrm{acos}$’ returns a value in $[0,\pi ]$.
• Special orthogonal group: In general, it is not easy to obtain the expression of a geodesic arc on a given manifold in closed form. In the present case, with the assumptions considered, the geodesic on $\mathbb{SO}\left(p\right)$ departing from the identity with velocity $H\in \mathfrak{so}\left(p\right)$ has expression $\tilde{\gamma }\left(t\right)=exp\left(tH\right)$. (It is important to verify that $\tilde{\gamma }\left(0\right)=I_p$ and ${\left.\frac{d\tilde{\gamma }\left(t\right)}{dt}\right|}_{t=0}=H$.) It might be useful to verify such an essential result by the help of the following arguments. A geodesic $\tilde{\gamma }\left(t\right)$ on the Riemannian manifold $\mathbb{SO}\left(p\right)$, embedded in the Euclidean ambient space $\mathbb{A}:={\mathbb{R}}^{p\times p}$ and endowed with its canonical metric, departing from the identity $I_p$, should satisfy $\ddot{\tilde{\gamma }}\left(t\right)\in N_{\tilde{\gamma }\left(t\right)}\mathbb{SO}\left(p\right)$, and therefore it should hold:

  $$\ddot{\tilde{\gamma }}\left(t\right)=\tilde{\gamma }\left(t\right)S\left(t\right),with{S}^{\top }\left(t\right)=S\left(t\right).$$

  (191)
  Additionally, we know that any geodesic arc belongs entirely to the base manifold; therefore ${\tilde{\gamma }}^{\top }\left(t\right)\tilde{\gamma }\left(t\right)=I_p$. By differentiating such an expression two times with respect to the parameter t, one obtains:

  $${\ddot{\tilde{\gamma }}}^{\top }\left(t\right)\tilde{\gamma }\left(t\right)+2{\dot{\tilde{\gamma }}}^{\top }\left(t\right)\dot{\tilde{\gamma }}\left(t\right)+{\tilde{\gamma }}^{\top }\left(t\right)\ddot{\tilde{\gamma }}\left(t\right)=0_p.$$

  (192)
  By plugging Equation (191) into Equation (192), we find that $S\left(t\right)=-{\dot{\tilde{\gamma }}}^{\top }\left(t\right)\dot{\tilde{\gamma }}\left(t\right)$, which leads to the second-order differential equation on the orthogonal group:

  $$\ddot{\tilde{\gamma }}\left(t\right)=-\tilde{\gamma }\left(t\right)\left({\dot{\tilde{\gamma }}}^{\top }\left(t\right)\dot{\tilde{\gamma }}\left(t\right)\right),$$

  (193)

  to be solved with the initial conditions $\tilde{\gamma }\left(0\right)=I_p$ and $\dot{\tilde{\gamma }}\left(0\right)=H$. It is a straightforward task to verify that the solution to this second-order differential equation is given by the one-parameter curve $\tilde{\gamma }\left(t\right)=\mathrm{Exp}\left(tH\right)$, where ‘Exp’ denotes a matrix exponential.
  The expression of the geodesic arc in the position of interest may be made explicit by taking advantage of the Lie-group structure of the orthogonal group endowed with the canonical metric. In fact, let us consider the pair $X\in \mathbb{SO}\left(p\right)$ and $V\in T_X\mathbb{SO}\left(p\right)$ as well as the geodesic $\gamma \left(t\right)$ that emanates from X, namely $\gamma \left(0\right)=X$, with velocity V. We claim that the geodesic departing from $X\in \mathbb{SO}\left(p\right)$ in the direction $V\in T_X\mathbb{SO}\left(p\right)$ is:

  $${\gamma }_{X,V}\left(t\right)=X\mathrm{Exp}\left(t{X}^{\top }V\right).$$

  (194)
  In fact, let us consider the left-translated curve $\tilde{\gamma }\left(t\right)=X^{\top }{\gamma }_{X,V}\left(t\right)$. It has the following properties:

  The curve ${\gamma }_{X,V}\left(t\right)$ belongs to the orthogonal group at any time. This may be proven by computing the quantity ${\gamma }_{X,V}^{\top }\left(t\right){\gamma }_{X,V}\left(t\right)$ and taking into account that the identity ${\mathrm{Exp}}^{\top }\left(tH\right)=\mathrm{Exp}(-tH)$ holds true. Therefore,

  $$\begin{array}{cc}\hfill {\gamma }_{X,V}^{\top }\left(t\right){\gamma }_{X,V}\left(t\right)=& {\mathrm{Exp}}^{\top }\left(t{X}^{\top }V\right)X^{\top }X\mathrm{Exp}\left(t{X}^{\top }V\right)\hfill \\ \hfill =& \mathrm{Exp}(-t{X}^{\top }V)\mathrm{Exp}\left(t{X}^{\top }V\right)=\hfill \\ \hfill =& I_p.\hfill \end{array}$$

  (2)

  It satisfies Equation (193); hence, it is a geodesic. In fact, notice that $\tilde{\gamma }\left(t\right)=X^{\top }X\mathrm{Exp}\left(t{X}^{\top }V\right)=\mathrm{Exp}\left(t{X}^{\top }V\right)$; hence, $\dot{\tilde{\gamma }}\left(t\right)=X^{\top }V\mathrm{Exp}\left(t{X}^{\top }V\right)$ and $\ddot{\tilde{\gamma }}\left(t\right)={\left(X^{\top }V\right)}^2\mathrm{Exp}\left(t{X}^{\top }V\right)$. Now, the right-hand side of Equation (193) has the expression $-\mathrm{Exp}\left(t{X}^{\top }V\right){\left(X^{\top }V\mathrm{Exp}\left(t{X}^{\top }V\right)\right)}^{\top }{X}^{\top }V\mathrm{Exp}\left(t{X}^{\top }V\right)=-\mathrm{Exp}\left(t{X}^{\top }V\right)$$\mathrm{Exp}(-t{X}^{\top }V){\left(X^{\top }V\right)}^{\top }{X}^{\top }V\mathrm{Exp}\left(t{X}^{\top }V\right)={\left(X^{\top }V\right)}^2\mathrm{Exp}\left(t{X}^{\top }V\right)$ because $X^{\top }V\in \mathfrak{so}\left(p\right)$, hence the claim.

  (3)

  It satisfies $\tilde{\gamma }\left(0\right)=X$ and $\dot{\tilde{\gamma }}\left(0\right)=X\left(X^{\top }V\right)=V$; hence, it has the correct base point and direction.

  Therefore, the exponential map associated with the above geodesic expression reads

  $${exp}_X\left(V\right):=X\mathrm{Exp}\left(X^{\top }V\right).$$

  (195)
  Now, fixed two points $X,Y\in \mathbb{SO}\left(p\right)$, let us compute the geodesic arc ${\gamma }_X^Y:[0,1]\to \mathbb{SO}\left(p\right)$ that joins them. It all boils down in finding a tangent vector $V\in T_X\mathbb{SO}\left(p\right)$ such that $Y=X\mathrm{Exp}\left(X^{\top }V\right)$. Pre-multiplying this expression by $X^{\top }$ gives $X^{\top }Y=\mathrm{Exp}\left(X^{\top }V\right)$; hence, $\mathrm{Log}\left(X^{\top }Y\right)=X^{\top }V$. Pre-multiplying the last equality by X gives the sought tangent vector as $V=X\mathrm{Log}\left(X^{\top }Y\right)$. The logarithmic map associated with the above exponential map therefore reads

  $${log}_{X}Y:=X\mathrm{Log}\left(X^{\top }Y\right).$$

  (196)
  The Riemannian distance between X and Y then takes the expression

  $$d(X,Y)=\parallel X\mathrm{Log}\left(X^{\top }Y\right){\parallel }_{\mathrm{F}}={\parallel \mathrm{Log}\left(X^{\top }Y\right)\parallel }_{\mathrm{F}},$$

  (197)

  where ‘Log’ denotes, as usual, the matrix logarithm.
• Stiefel manifold: Let us consider the expression of geodesics corresponding to two metrics.
  • Euclidean metric: The solution of the geodesic equation, with the initial conditions ${\gamma }_{X,V}\left(0\right)=X\in \mathrm{St}(n,p)$ and ${\dot{\gamma }}_{X,V}\left(0\right)=V\in T_X\mathrm{St}(n,p)$, reads [33]:

    $${\gamma }_{X,V}\left(t\right)=\left[XV\right]\mathrm{Exp}\left(t\left[\begin{array}{cc}{X}^{\top }V& -V^{\top }V\\ I_p& X^{\top }V\end{array}\right]\right)I_{2p,p}\mathrm{Exp}(-t{X}^{\top }V),$$

    (198)

    for $t\in [0,1]$. The expression of the exponential map corresponding to the Euclidean metric reads, therefore,

    $${exp}_X\left(V\right):=\left[XV\right]\mathrm{Exp}\left(\left[\begin{array}{cc}{X}^{\top }V& -V^{\top }V\\ I_p& X^{\top }V\end{array}\right]\right)I_{2p,p}\mathrm{Exp}(-X^{\top }V),$$

    (199)

    while the expression of the logarithmic map and of the Riemannian distance between two given points, in closed form, are unknown at present (to the best of the author’s knowledge). The pseudo-identity matrix $I_{2p,p}$, with $2p$ rows and p columns, is just an identity matrix topping a zero matrix.
  • Canonical metric: The geodesic arc ${\gamma }_{X,V}$ may be computed as follows. Let Q and R denote the factors of the thin QR factorization of the matrix V, then:

    $${\gamma }_{X,V}\left(t\right)=\left[XQ\right]\mathrm{Exp}\left(t\left[\begin{array}{cc}{X}^{\top }V& -R^{\top }\\ R& 0_p\end{array}\right]\right)\left[\begin{array}{c}{I}_p\\ 0_p\end{array}\right],$$

    (200)

    for $t\in [0,1]$. The expression of the exponential map corresponding to the canonical metric is, therefore,

    $${exp}_X\left(V\right):=\left[XQ\right]\mathrm{Exp}\left(\left[\begin{array}{cc}{X}^{\top }V& -R^{\top }\\ R& 0_p\end{array}\right]\right)\left[\begin{array}{c}{I}_p\\ 0_p\end{array}\right],$$

    (201)

    while the expressions of the logarithmic map and of the Riemannian distance between two points are still unknown.
    In fact, neither the logarithmic map nor the geodesic distance are known in closed form for a Stiefel manifold.
• Real symplectic group: According to the two considered metrics, we have:
  • KM metric: Under the pseudo-Riemannian metric (115), it is indeed possible to solve the geodesic equation in closed form. The geodesic curve ${\gamma }_{Q,V}:[0,1]\to \mathrm{Sp}\left(2n\right)$ with $Q\in \mathrm{Sp}\left(2n\right)$ and $V\in T_Q\mathrm{Sp}\left(2n\right)$ corresponding to the indefinite Khvedelidze–Mladenov metric (115) has the expression:

    $${\gamma }_{Q,V}\left(t\right)=Q\mathrm{Exp}\left(t{Q}^{-1}V\right).$$

    (202)
    In fact, the geodesic equation in variational form is:

    $$\delta {\int }_0^1\mathrm{tr}\left({\gamma }^{-1}\dot{\gamma }{\gamma }^{-1}\dot{\gamma }\right)\mathrm{d}t=0.$$

    (203)
    The calculation of this variation is facilitated by the following rules of the calculus of variations:

    $$\begin{array}{ccc}& & \delta \left(Q^{-1}\right)=-Q^{-1}\left(\delta Q\right)Q^{-1},\hfill \end{array}$$

    (204)

    $$\begin{array}{ccc}& & \delta \left(\frac{\mathrm{d}Q}{\mathrm{d}t}\right)=\frac{\mathrm{d}}{\mathrm{d}t}\left(\delta Q\right),\hfill \end{array}$$

    (205)

    $$\begin{array}{ccc}& & \delta \left(QZ\right)=\left(\delta Q\right)Z+Q\left(\delta Z\right),\hfill \end{array}$$

    (206)

    for curves $Q,Z:[0,1]\to \mathrm{Sp}\left(2n\right)$. By computing the variations, integrating by parts and recalling that the variations vanish at endpoints, it is found that the geodesic equation in variational form reads:

    $${\int }_0^1\mathrm{tr}\left(\delta \gamma ({\gamma }^{-1}\ddot{\gamma }{\gamma }^{-1}-{\gamma }^{-1}\dot{\gamma }{\gamma }^{-1}\dot{\gamma }{\gamma }^{-1})\right)\mathrm{d}t=0.$$

    (207)
    The variation $\delta \gamma \in T_{\gamma }\mathrm{Sp}\left(2n\right)$ is arbitrary. By the structure of the normal space $N_Q\mathrm{Sp}\left(2n\right)$, the equation $\mathrm{tr}\left(P^{\top }\delta \gamma \right)=0$, with $\delta \gamma \in T_{\gamma }\mathrm{Sp}\left(2n\right)$, implies that $P^{\top }=HJ{\gamma }^{-1}$ with $H\in \mathfrak{so}\left(2n\right)$. Therefore, Equation (207) is satisfied if and only if:

    $${\gamma }^{-1}\ddot{\gamma }{\gamma }^{-1}-{\gamma }^{-1}\dot{\gamma }{\gamma }^{-1}\dot{\gamma }{\gamma }^{-1}=HJ{\gamma }^{-1},H\in \mathfrak{so}\left(2n\right),$$

    (208)

    or, equivalently,

    $$\ddot{\gamma }-\dot{\gamma }{\gamma }^{-1}\dot{\gamma }=\gamma HJ,$$

    (209)

    for some $H\in \mathfrak{so}\left(2n\right)$. In order to determine the value of matrix H, note that:

    $${\gamma }^{\top }J\gamma -J=0\Rightarrow {\ddot{\gamma }}^{\top }J\gamma +2{\dot{\gamma }}^{\top }J\dot{\gamma }+{\gamma }^{\top }\ddot{\gamma }=0.$$
    Substituting the expression $\ddot{\gamma }=\dot{\gamma }{\gamma }^{-1}\dot{\gamma }+\gamma HJ$ into the above equation yields the condition $JHJ=0$. Hence, $H=0$ and the geodesic equation reads:

    $$\ddot{\gamma }-\dot{\gamma }{\gamma }^{-1}\dot{\gamma }=0.$$

    (210)
    Its solution, with the initial conditions $\gamma \left(0\right)=Q\in \mathrm{Sp}\left(2n\right)$ and $\dot{\gamma }\left(0\right)=V\in T_X\mathrm{Sp}\left(2n\right)$, is found to be of the form (202). By definition of matrix exponential, it follows that ${\dot{\gamma }}_{Q,V}\left(t\right)=V\mathrm{Exp}\left(t{Q}^{-1}V\right)$.
  • Euclidean metric: The expression of the geodesic corresponding to the Euclidean metric was derived in [50]. Let $\gamma :[0,1]\to \mathrm{Sp}\left(2n\right)$ be a geodesic arc connecting the points $X,Y\in \mathrm{Sp}\left(2n\right)$. Let us define $h\left(t\right):={\gamma }^{-1}\left(t\right)\dot{\gamma }\left(t\right)$. The geodesic that minimizes the following energy functional

    $${\int }_0^1\mathrm{tr}\left(h^{\top }\left(t\right)h\left(t\right)\right)\mathrm{d}t,$$

    (211)

    is the solution of the differential (Lax) equation:

    $$\dot{h}\left(t\right)=h^{\top }\left(t\right)h\left(t\right)-h\left(t\right)h^{\top }\left(t\right).$$

    (212)
    Furthermore, for the initial conditions $\gamma \left(0\right)=X\in \mathrm{Sp}\left(2n\right)$ and $\dot{\gamma }\left(0\right)=V\in T_X\mathrm{Sp}\left(2n\right)$, the geodesic on the real symplectic group is given by

    $${\gamma }_{X,V}\left(t\right)=X\mathrm{Exp}\left(t{\left(X^{-1}V\right)}^{\top }\right)\mathrm{Exp}\left(t[\left(X^{-1}V\right)-{\left(X^{-1}V\right)}^{\top }]\right),$$

    (213)

    in the case that the real symplectic group is equipped by a Euclidean metric.
• Space of symmetric, positive-definite matrices: The geodesic arc, corresponding to the canonical metric, emanating from a point $P\in {\mathbb{S}}^{+}\left(p\right)$ in the direction $V\in T_P{\mathbb{S}}^{+}\left(p\right)$ has the expression:

  $${\gamma }_{P,V}\left(t\right)=\sqrt[\mathbb{S}]{P}\mathrm{Exp}\left(t\sqrt[\mathbb{S}]{P^{-1}}V\sqrt[\mathbb{S}]{P^{-1}}\right)\sqrt[\mathbb{S}]{P},$$

  (214)

  where $\sqrt[\mathbb{S}]{·}$ denotes a symmetric matrix square root. The exponential and the logarithmic maps for $P,Q\in {\mathbb{S}}^{+}\left(p\right)$ and $V\in T_P{\mathbb{S}}^{+}\left(p\right)$ thus read:

  $$\begin{array}{ccc}\hfill & & {exp}_P\left(V\right)=\sqrt[\mathbb{S}]{P}\mathrm{Exp}\left(\sqrt[\mathbb{S}]{P^{-1}}V\sqrt[\mathbb{S}]{P^{-1}}\right)\sqrt[\mathbb{S}]{P},\hfill \end{array}$$

  (215)

  $$\begin{array}{ccc}& & {log}_P\left(Q\right)=\sqrt[\mathbb{S}]{P}\mathrm{Log}\left(\sqrt[\mathbb{S}]{P^{-1}}Q\sqrt[\mathbb{S}]{P^{-1}}\right)\sqrt[\mathbb{S}]{P}.\hfill \end{array}$$

  (216)
  The symmetric matrix square root of a ${\mathbb{S}}^{+}\left(p\right)$ matrix P may be computed by means of its eigenvalue factorization. In fact, if the matrix P is factored as $X\mathrm{diag}({\lambda }_1,{\lambda }_2,{\lambda }_3,\dots ,{\lambda }_p)X^{\top }$, with $X\in \mathbb{O}\left(p\right)$ and every ${\lambda }_k>0$, then it holds that $\sqrt[\mathbb{S}]{P}=X\mathrm{diag}(\sqrt{{\lambda }_1},\sqrt{{\lambda }_2},\sqrt{{\lambda }_3},\dots ,$$\sqrt{{\lambda }_p})X^{\top }$. The squared Riemannian distance between two points $P,Q\in {\mathbb{S}}^{+}\left(p\right)$ is given by

  $$\begin{array}{cc}\hfill d^2(P,Q)& ={\langle {log}_{P}Q,{log}_{P}Q\rangle }_P=\mathrm{tr}\left(\left({log}_{P}Q\right)P^{-1}\left({log}_{P}Q\right)P^{-1}\right)\hfill \\ & =\mathrm{tr}\left(\sqrt[\mathbb{S}]{P}\mathrm{Log}\left(\sqrt[\mathbb{S}]{P^{-1}}Q\sqrt[\mathbb{S}]{P^{-1}}\right)\sqrt[\mathbb{S}]{P}{P}^{-1}\sqrt[\mathbb{S}]{P}\mathrm{Log}\left(\sqrt[\mathbb{S}]{P^{-1}}Q\sqrt[\mathbb{S}]{P^{-1}}\right)\sqrt[\mathbb{S}]{P}{P}^{-1}\right)\hfill \\ & =\mathrm{tr}\left({\mathrm{Log}}^2\left(\sqrt[\mathbb{S}]{P^{-1}}Q\sqrt[\mathbb{S}]{P^{-1}}\right)\sqrt[\mathbb{S}]{P}{P}^{-1}\sqrt[\mathbb{S}]{P}\right)\hfill \\ & =\mathrm{tr}\left({\mathrm{Log}}^2\left(\sqrt[\mathbb{S}]{P^{-1}}Q\sqrt[\mathbb{S}]{P^{-1}}\right)\right).\hfill \end{array}$$
  Notice that the following identity holds true:

  $$\mathrm{tr}{(I_p-\sqrt[\mathbb{S}]{P^{-1}}Q\sqrt[\mathbb{S}]{P^{-1}})}^k=\mathrm{tr}{(I_p-Q{P}^{-1})}^k,$$

  (217)

  and hence the Riemannian distance between two SPD matrices may be written equivalently as

  $$d^2(P,Q)=\mathrm{tr}\left({\mathrm{Log}}^2\left(Q{P}^{-1}\right)\right),$$

  (218)

  as obtained by the series expansion of the matrix logarithm function (cf. Section 2.1).
• Grassmann manifold: A geodesic arc on a Grassmann manifold emanating from $\left[X\right]\in \mathrm{Gr}(n,p)$ with the velocity $V\in T_{\left[X\right]}\mathrm{Gr}(n,p)$ may be written as:

  $$\begin{array}{cc}\hfill {\gamma }_{\left[X\right],V}\left(t\right)=& \left[XBA\right]\left[\begin{array}{c}cos\left(Dt\right)\\ sin\left(Dt\right)\end{array}\right]B^{\top },\hfill \\ \hfill =& XBcos\left(Dt\right)B^{\top }+Asin\left(Dt\right)B^{\top }.\hfill \end{array}$$

  (219)

  where $X\in \mathrm{St}(n,p)$ denotes a Stiefel matrix whose columns form an orthonormal basis of the subspace $\left[X\right]$ and $AD{B}^{\top }$ denotes the compact singular value factorization of the matrix V. The sin/cos functions applied to a diagonal matrix simply act, in terms of components, on the entries of the main diagonal. As a compact, smooth Riemannian manifold, a Grassmann manifold is geodesically complete, which implies that any two points on $\mathrm{Gr}(n,p)$ may be connected by a geodesic arc.
  The exponential map associated with the canonical metric reads:

  $${exp}_{\left[X\right]}\left(V\right)=XBcos\left(D\right)B^{\top }+Asin\left(D\right)B^{\top }.$$

  (220)
  The logarithmic map ${log}_{\left[X\right]}\left[Y\right]$ of two subspaces $\left[X\right],\left[Y\right]\in \mathrm{Gr}(n,p)$ is not easy to compute in general. In [40], it is shown that if their Stiefel representatives $X,Y\in \mathrm{St}(n,p)$ are such that the product $X^{\top }Y$ is symmetric, then ${log}_{\left[X\right]}\left[Y\right]=AD{B}^{\top }$, where $B(cosD)B^{\top }$ denotes the spectral factorization of the matrix $X^{\top }Y$ and $A:=(I-X{X}^{\top })YB{(sinD)}^{-1}$.

Example 22.

Let us compute the length of a geodesic curve ${\gamma }_x^y:[0,1]\to {\mathbb{S}}^{n-1}$ that, by definition, represents the distance between points $x,y\in {\mathbb{S}}^{n-1}$:

$$d(x,y):=\mathrm{L}\left({\gamma }_x^y\right).$$

(221)

Let us start by showing that:

$${\dot{\gamma }}_x^y\left(t\right)=\frac{1}{sin\theta }\left[\theta cos\left(\theta t\right)y-\theta cos\left(\theta \right(1-t\left)\right)x\right]$$

(222)

with $cos\theta :=x^{\top }y$; hence,

$$\begin{array}{cc}\hfill \parallel {\dot{\gamma }}_x^y{\left(t\right)\parallel }^2=& \frac{{\theta }^2}{{sin}^2\theta }\left[{cos}^2\left(\theta t\right)+{cos}^2\left(\theta (1-t)\right)-2cos\left(\theta t\right)cos\left(\theta (1-t)\right)x^{\top }y\right]\hfill \\ \hfill =& {\theta }^2={\mathrm{acos}}^2\left(x^{\top }y\right).\hfill \end{array}$$

(223)

It is instructive to verify that $\parallel {\dot{\gamma }}_x^y{\left(t\right)\parallel }^2$ does not actually depend on t, which facilitates the computation of the length. In fact, the length of ${\gamma }_x^y:[0,1]\to {\mathbb{S}}^{n-1}$ is given by:

$$\mathrm{L}\left({\gamma }_x^y\right)={\int }_0^1\parallel {\dot{\gamma }}_x^y\left(t\right)\parallel \mathrm{d}t=\mathrm{acos}\left(x^{\top }y\right){\int }_0^1\mathrm{d}t.$$

(224)

Therefore, the distance between $x,y\in {\mathbb{S}}^{n-1}$ equals:

$$d(x,y)=\mathrm{acos}\left(x^{\top }y\right).$$

(225)

As a further example, let us determine the length of the geodesic curve ${\gamma }_{R,V}:[0,1]\to \mathbb{SO}\left(n\right)$ given by the relationship (194). By definition, it holds that

$$\mathrm{L}\left({\gamma }_{R,V}\right)={\int }_0^1\parallel {\dot{\gamma }}_{R,V}\left(t\right)\parallel \mathrm{d}t.$$

(226)

Let us show that

$${\dot{\gamma }}_{R,V}\left(t\right)=\left(R{R}^{\top }\right)V\mathrm{Exp}\left(R^{\top }Vt\right)=V\mathrm{Exp}\left(R^{\top }Vt\right),$$

(227)

therefore

$$\begin{array}{cc}\hfill \mathrm{L}\left({\gamma }_{R,V}\right)=& {\int }_0^1\sqrt{\mathrm{tr}\left[{\left(V\mathrm{Exp}\left(R^{\top }Vt\right)\right)}^{\top }V\mathrm{Exp}\left(R^{\top }Vt\right)\right]}\mathrm{d}t\hfill \\ \hfill =& {\int }_0^1\sqrt{\mathrm{tr}\left[\mathrm{Exp}(-R^{\top }Vt)V^{\top }V\mathrm{Exp}\left(R^{\top }Vt\right)\right]}\mathrm{d}t\hfill \\ \hfill =& {\int }_0^1\parallel V\parallel \mathrm{d}t=\parallel V\parallel ,\hfill \end{array}$$

(228)

as proven in the general theoretical developments. ■

The following observation clarifies a technical argument.

Observation 3.

The parameter t in the expressions of geodesic lines, which we may identify astime, indicates which point of a geodesic one is referring to. Normally, the value $t=0$ corresponds to the initial point x, namely ${\gamma }_{x,v}\left(0\right)\equiv x$. It is instructive to observe that, in all examined cases, the velocity v and the time parameter t appear to be multiplied to one another. For example, the equation of the geodesic ${\gamma }_{x,v}$ on a hypersphere may be rewritten as

$${\gamma }_{x,v}\left(t\right)=cos(\parallel v\parallel t)x+sin(\parallel v\parallel t)\frac{vt}{\parallel v\parallel t},t>0.$$

(229)

This means that it is always possible to re-scale the parameter t as $\alpha t$ as long as the velocity v is re-scaled as $v/\alpha$, where $\alpha >0$. This is the reason why the time parameter t normally ranges in the interval $[0,1]$.

8.3. Geodesic Interpolation

The notions of metric, distance, geodesics, are related to the notion of interpolation. In fact, the interpolation between two points $x,y\in \mathbb{M}$ may be defined through the optimization problem [51]:

$${\iota }_x^y\left(t\right):=arg\underset{z\in \mathbb{M}}{min}[(1-t)d^2(x,z)+t{d}^2(z,y)],$$

(230)

with parameter $t\in [0,1]$ providing the degree of interpolation between the two points. The one-parameter curve ${\iota }_x^y:[0,1]\to \mathbb{M}$ describes a trajectory over the manifold $\mathbb{M}$, having x and y as endpoints, which may be regarded as an interpolation curve. For example, the value ${\iota }_x^y\left(\frac{1}{2}\right)$ is regarded as a midpoint between points x and y.

Example 23.

Let us consider two close points $X,Y\in \mathbb{SO}\left(p\right)$ and let us look for an interpolation of such points in $\mathbb{SO}\left(p\right)$ endowed with its canonical metric. We are looking for a minimizer of the criterion:

$$Z\mapsto (1-t)\parallel \mathrm{Log}\left(X^{\top }Z\right){\parallel }_{\mathrm{F}}^2+t{\parallel \mathrm{Log}\left(Y^{\top }Z\right)\parallel }_{\mathrm{F}}^2,$$

(231)

with $t\in [0,1]$ assigned. We may look for an explicit solution of the above minimization problem by reasoning as follows. The points X and Y may be connected by a geodesic arc ${\gamma }_{X,V}\left(\tau \right)=X\mathrm{Exp}\left(\tau X^{\top }V\right)$ with $\tau \in [0,1]$ and $V\in T_X\mathbb{SO}\left(p\right)$ such that ${\gamma }_{X,V}\left(1\right)=Y$, namely, with V satisfying condition $\mathrm{Exp}\left(X^{\top }V\right)=X^{\top }Y$. As the interpolating point ${\iota }_X^Y\left(t\right)$ must be as close as possible to both X and Y, it should belong to the geodesic ${\gamma }_{X,V}\left(\tau \right)$; therefore, we are left with the problem of minimizing the criterion function:

$$\begin{array}{ccc}\hfill \tau & \mapsto & (1-t)\parallel \mathrm{Log}\left(X^{\top }X\mathrm{Exp}\left(\tau X^{\top }V\right)\right){\parallel }_{\mathrm{F}}^2+t{\parallel \mathrm{Log}\left(Y^{\top }X\mathrm{Exp}\left(\tau X^{\top }V\right)\right)\parallel }_{\mathrm{F}}^2\hfill \\ \hfill & & =(1-t)\parallel \tau X^{\top }{V\parallel }_{\mathrm{F}}^2+t{\parallel (\tau -1)X^{\top }V\parallel }_{\mathrm{F}}^2\hfill \\ & & =((1-t){\tau }^2+t{(\tau -1)}^2){\parallel \mathrm{Log}\left(X^{\top }Y\right)\parallel }_{\mathrm{F}}^2,\hfill \end{array}$$

with respect to the variable τ. In the second line, we have used the identity $Y^{\top }X=\mathrm{Exp}(-X^{\top }V)$. The minimum of the above criterion function in $[0,1]$ is readily seen to be achieved in $\tau =t$; thus, the interpolating curve reads:

$${\iota }_X^Y\left(t\right)=X\mathrm{Exp}\left(t\mathrm{Log}\left(X^{\top }Y\right)\right)=X{\left(X^{\top }Y\right)}^t$$

(232)

for $0\le t\le 1$. Notice that a matrix power $A^{\alpha }$, with $A\in {\mathbb{R}}^{n\times n}$ and $\alpha \in \mathbb{R}$, is defined (inasmuch as it exists) in terms of matrix exponential/logarithm, hence the last identity. ■

Generalizing the notion of midpoint, one comes up with the notion of mean value (and of dispersion of a sample set around its mean value) in a metrizable manifold $\mathbb{M}$. We may consider what follows:

• The notion of ‘mean value’ of objects in a metrizable space should reflect the intuitive understanding that the mean value is an element of the space that locates ‘amidst’ the available objects. Therefore, a fundamental tool in the definition of mean value is a measure of ‘how far apart’ elements in the sample space lie to one another.
• The notion of ‘metric variance’ of objects in a metrizable space should be defined in a way that accounts for the dispersion of these objects about their mean values and also depends on how the dissimilarity of such objects is measured.

A way of defining the mean value of a set of objects $x_k\in \mathbb{M}$, $k=1,\dots ,K$, is provided by the notion of Fréchet sample mean. The Fréchet sample mean and associated sample metric variance [52] may be defined as:

$$\begin{array}{ccc}& & \mu :=arg\underset{x\in \mathbb{M}}{min}\frac{1}{K}\sum _{k=1}^K{d}^2(x,x_k),\hfill \end{array}$$

(233)

$$\begin{array}{ccc}& & {\sigma }^2:=\underset{x\in \mathbb{M}}{min}\frac{1}{K}\sum _{k=1}^K{d}^2(x,x_k),\hfill \end{array}$$

(234)

where d denotes a distance function in the Riemannian manifold $\mathbb{M}$. It is worth noting that there is no guarantee, in general, that the optimization problem (233) admits a unique solution. For example, consider the space ${\mathbb{S}}^{p-1}$ and a sample set consisting of two antipodal points; then, every point over the equator is a mean value. (Notice that a mean is Fréchet only if it is a global minimizer, otherwise it is a Karcher mean.) If the sample points are close enough to each other, it is known that the mean value is unique [53].

The notion of mean value of a set of points belonging to a curved manifold is utterly important in system theory and non-liner control. In fact, the mean value $\mu$ is, by definition, close to all points in a collection of points. Therefore, the tangent space $T_{\mu }\mathbb{M}$ associated with a cloud of data points may serve as a reference tangent space (see, for example, [54]).

8.4. Coordinate-Prone Geodesy, Christoffel Symbols*

In terms of the variation of an energy functional, a geodesic on a manifold $\mathbb{M}$ is defined as the unique curve $\gamma \left(t\right)\in \mathbb{M}$, $t\in [0,1]$ that meets the following requirement

$$\delta {\int }_0^1{\langle \dot{\gamma },\dot{\gamma }\rangle }_{\gamma }\mathrm{d}t=0,$$

(235)

where the symbol $\delta$ denotes the variation of the integral functional. The integral functional in (235) represents the total action associated with the parametrized smooth curve $\gamma :[0,1]\to \mathbb{M}$ of local coordinates $x^k={\gamma }^k\left(t\right)$ and may be written explicitly as:

$${\int }_0^1{\langle \dot{\gamma },\dot{\gamma }\rangle }_{\gamma }\mathrm{d}t={\int }_0^1{g}_{ij}\left(\gamma \left(t\right)\right){\dot{\gamma }}^i\left(t\right){\dot{\gamma }}^j\left(t\right)\mathrm{d}t.$$

(236)

The variation $\delta$ in the expression (235) corresponds to a perturbation of the total action (236). Let $\eta :[0,1]\to \mathbb{M}$ denote an arbitrary parametrized smooth curve of local coordinates ${\eta }^k\left(t\right)$ such that ${\eta }^k\left(0\right)={\eta }^k\left(1\right)=0$. Define the perturbed action as

$${\mathcal{A}}_{\eta }^{*}\left(h\right):={\int }_0^1{g}_{ij}(\gamma +h\eta )({\dot{\gamma }}^i+h{\dot{\eta }}^i)({\dot{\gamma }}^j+h{\dot{\eta }}^j)\mathrm{d}t,$$

(237)

with $h\in [-ϵ,ϵ]$, $ϵ>0$. The condition (235) may be expressed explicitly in terms of the perturbed action as:

$$\underset{h\to 0}{lim}\frac{{\mathcal{A}}_{\eta }^{*}\left(h\right)-{\mathcal{A}}_{\eta }^{*}\left(0\right)}{h}=0,\forall \mathrm{perturbation}\eta .$$

(238)

The perturbed total action may be expanded around $h=0$ as follows:

$$\begin{array}{cc}\hfill {\mathcal{A}}_{\eta }^{*}\left(h\right)=& {\int }_0^1{\left.\left[g_{ij}\left(x\right)+h\frac{\partial g_{ij}}{\partial x^k}{\eta }^k+\mathcal{O}\left(h^2\right)\right]\right|}_{x=\gamma \left(t\right)}[{\dot{\gamma }}^i{\dot{\gamma }}^j+h({\dot{\gamma }}^i{\dot{\eta }}^j+{\dot{\gamma }}^j{\dot{\eta }}^i)+\mathcal{O}\left(h^2\right)]\mathrm{d}t\hfill \\ \hfill =& {\int }_0^1{g}_{ij}\left(\gamma \right){\dot{\gamma }}^i{\dot{\gamma }}^j\mathrm{d}t+h{\int }_0^1{\left.\left[g_{ij}\left(x\right)({\dot{\gamma }}^i{\dot{\eta }}^j+{\dot{\gamma }}^j{\dot{\eta }}^i)+{\eta }^k\frac{\partial g_{ij}}{\partial x^k}{\dot{\gamma }}^i{\dot{\gamma }}^j\right]\right|}_{x=\gamma \left(t\right)}\mathrm{d}t+\mathcal{O}\left(h^2\right)\hfill \\ \hfill =& {\mathcal{A}}_{\eta }^{*}\left(0\right)+h{\int }_0^1{\left.\left[g_{ik}\left(x\right){\dot{\gamma }}^i{\dot{\eta }}^k+g_{kj}\left(x\right){\dot{\gamma }}^j{\dot{\eta }}^k+\frac{\partial g_{ij}}{\partial x^k}{\dot{\gamma }}^i{\dot{\gamma }}^j{\eta }^k\right]\right|}_{x=\gamma \left(t\right)}\mathrm{d}t+\mathcal{O}\left(h^2\right),\hfill \end{array}$$

where $\mathcal{O}\left(h^2\right)$ is such that ${lim}_{h\to 0}{h}^{-1}\mathcal{O}\left(h^2\right)=0$. The first term in the integral on the right-hand side of the last line may be integrated by parts, namely:

$${\int }_0^1{g}_{ik}\left(\gamma \right){\dot{\gamma }}^i{\dot{\eta }}^k\mathrm{d}t={\left.g_{ik}\left(\gamma \right){\dot{\gamma }}^i{\eta }^k\right|}_0^1-{\int }_0^1\frac{d\left(g_{ik}\left(\gamma \right){\dot{\gamma }}^i\right)}{dt}{\eta }^k\mathrm{d}t,$$

(239)

whose first term on the right-hand side vanishes to zero because ${\eta }^k\left(0\right)={\eta }^k\left(1\right)=0$, hence:

$${\int }_0^1{g}_{ik}\left(\gamma \right){\dot{\gamma }}^i{\dot{\eta }}^k\mathrm{d}t=-{\int }_0^1{\left.\left(\frac{\partial g_{ik}}{\partial x^j}{\dot{\gamma }}^i{\dot{\gamma }}^j+g_{ik}\left(x\right){\ddot{\gamma }}^i\right)\right|}_{x=\gamma \left(t\right)}{\eta }^k\mathrm{d}t.$$

(240)

A similar result holds for the second term within the integral on the right-hand side. Therefore, it holds that:

$$\frac{{\mathcal{A}}_{\eta }^{*}\left(h\right)-{\mathcal{A}}_{\eta }^{*}\left(0\right)}{h}={\int }_0^1{\left.\left[\left(\frac{\partial g_{ij}}{\partial x^k}-\frac{\partial g_{ik}}{\partial x^j}-\frac{\partial g_{kj}}{\partial x^i}\right){\dot{\gamma }}^i{\dot{\gamma }}^j-2g_{ik}\left(x\right){\ddot{\gamma }}^i\right]\right|}_{x=\gamma \left(t\right)}{\eta }^k\mathrm{d}t+\frac{\mathcal{O}\left(h^2\right)}{h}.$$

The condition (238), therefore, implies that:

$$g_{ik}{\ddot{\gamma }}^i+\frac{1}{2}\left(-\frac{\partial g_{ij}}{\partial x^k}+\frac{\partial g_{ik}}{\partial x^j}+\frac{\partial g_{kj}}{\partial x^i}\right){\dot{\gamma }}^i{\dot{\gamma }}^j=0.$$

(241)

Let us now recall the notion of Christoffel symbols of the first kind (named after Elwin Bruno Christoffel) associated with a metric tensor of components $g_{ij}$. These coefficients are defined as:

$${\Gamma }_{ijk}:=\frac{1}{2}\left(\frac{\partial g_{jk}}{\partial x^i}+\frac{\partial g_{ik}}{\partial x^j}-\frac{\partial g_{ij}}{\partial x^k}\right).$$

(242)

On top of it, the Christoffel symbols of the second kind are defined as ${\Gamma }_{ij}^h:=g^{hk}{\Gamma }_{ijk}$. The Christoffel symbols of the second kind are symmetric in the covariant indices, namely, ${\Gamma }_{ij}^h={\Gamma }_{ji}^h$.

On the basis of the Christoffel symbols, Equation (241) may be rewritten as

$$g_{ik}{\ddot{\gamma }}^i+{\Gamma }_{ijk}{\dot{\gamma }}^i{\dot{\gamma }}^j=0.$$

(243)

Multiplying by $g^{hk}$ and introducing the Christoffel symbols of the second kind, Equation (243) may be written as:

$${\ddot{\gamma }}^i+{\Gamma }_{jk}^i{\dot{\gamma }}^j{\dot{\gamma }}^k=0.$$

(244)

Such a system of second-order differential equations needs two initial conditions to be solved. Typical initial conditions are ${\gamma }^i\left(0\right)=x^i$ and ${\dot{\gamma }}^i\left(0\right)=v^i$.

Example 24.

One might wonder if the Christoffel symbols defined in (242) are the components of a tensor. The answer is negative, because none of its lower index denote covariancy. ■

9. Riemannian Gradient of a Manifold-to-Scalar Function

The gradient of a scalar function in a point of its manifold-type domain represents a degree of variability of the function in a vicinity of such a point. In the present section, we are going to survey the notion of ‘Riemannian gradient’ starting from more familiar cases to obtain a full definition for manifold-to-scalar functions.

9.1. Riemannian Gradient: Motivation and Definition

In the simplest case of a scalar function of a scalar variable $f:\mathbb{R}\to \mathbb{R}$, its degree of variability is quantified by its slope $\frac{\mathrm{d}f}{\mathrm{d}x}$. In fact, let us recall that, given a point $x\in \mathbb{R}$, moving slightly away from it of a little amount $h\in \mathbb{R}$, the value of the function $f(x+h)$ may be related to the value of $f\left(x\right)$ through a Taylor series expansion. Such an expansion, truncated to a first-order term by a Lagrange-type remainder, reads

$$f(x+h)=f\left(x\right)+\frac{\mathrm{d}f\left(x\right)}{\mathrm{d}x}h+\mathcal{O}\left(h^2\right),$$

(245)

and hence the first derivative of the function (that is, its gradient) truly quantifies the variability of the function around the point x. In the above expression, $\mathcal{O}$ again denotes a Landau symbol.

In the more involved case of a scalar function of several variables, $f:{\mathbb{R}}^p\to \mathbb{R}$, the gradient $\frac{\partial f}{\partial x}$ quantifies again the degree of variability of the function around a point, although such information is spread along p axes. In fact, in this case, there exist multiple directions along which the variability of a function may be explored. After choosing a direction, one may move away along a straight line. Starting from a point $x\in {\mathbb{R}}^p$ and moving away along a direction $v\in {\mathbb{R}}^p$ of a small fraction t, the value of $f(x+tv)$ may be related to the value of $f\left(x\right)$ through a multivariable Taylor series expansion which, in the first order, reads

$$f(x+tv)=f\left(x\right)+{\left(\frac{\partial f\left(x\right)}{\partial x}\right)}^{\top }\left(tv\right)+\mathcal{O}\left(t^2\right).$$

(246)

Therefore, the gradient of the function f (which, in this case, may be termed ‘Euclidean gradient’ and coincides with the ‘Gateaux derivative’ and with the ‘Jacobian’ of the function) encodes the variability of the function and the quantity ${\left(\frac{\partial f\left(x\right)}{\partial x}\right)}^{\top }v$ represents the directional derivative of the function f at x along the direction v. In the following, we shall utilize the shorter notation ${\partial }_x:=\frac{\partial }{\partial x}$.

In the case of a scalar function whose domain is a curved manifold $f:\mathbb{M}\to \mathbb{R}$, its gradient, which we shall denote as ${\mathrm{grad}}_{x}f$, serves again to quantify the degree of variability of a function in the neighborhood of point $x\in \mathbb{M}$ along a given tangent direction. On the same line of reasoning as before, starting from a point $x\in \mathbb{M}$ one may think of moving away slightly along a given tangent direction $v\in T_x\mathbb{M}$. After choosing a direction, the simplest way to move away from a point along a given direction is to travel along a geodesic line for a short time. Thus, taking a geodesic ${\gamma }_{x,v}:[-ϵ,ϵ]\to \mathbb{M}$, the value of $f\left({\gamma }_{x,v}\right)$ may be related to the value $f\left(x\right)$ through a Taylor-type expansion, which is written as

$$f\left({\gamma }_{x,v}\left(t\right)\right)=f\left(x\right)+{\langle {\mathrm{grad}}_{x}f,tv\rangle }_x+\mathcal{O}\left(t^2\right).$$

(247)

Therefore, the gradient of the function f quantifies the variability of a function around a point, while ${\langle {\mathrm{grad}}_{x}f,v\rangle }_x$ may be interpreted as a directional derivative of the function f at x along a direction v. From the relationship (247), we see that

$$\frac{f\left({\gamma }_{x,v}\left(t\right)\right)-f\left(x\right)}{t}={\langle {\mathrm{grad}}_{x}f,v\rangle }_x+\frac{\mathcal{O}\left(t^2\right)}{t}.$$

(248)

Taking the limit as t approaches zero, the second addendum on the right-hand side vanishes to zero; hence, we obtain

$${\langle {\mathrm{grad}}_{x}f,v\rangle }_x=\underset{t\to 0}{lim}\frac{f\left({\gamma }_{x,v}\left(t\right)\right)-f\left(x\right)}{t}={\left.\frac{\mathrm{d}f\left({\gamma }_{x,v}\left(t\right)\right)}{\mathrm{d}t}\right|}_{t=0}.$$

(249)

The right-hand side is resemblant of a Gateaux derivative of the function f in the direction v. The relationship (249) may be indeed taken as a definition of the gradient of the function with the proviso that it should hold for every $v\in T_x\mathbb{M}$.

By taking a closer look at the expression (249), it is readily seen that it holds even if the geodesic line is replaced by any smooth curve $\gamma :[-ϵ,ϵ]\to \mathbb{M}$ as long as $\gamma \left(0\right)=x$ and $\dot{\gamma }\left(0\right)=v$, in which case we can write

$${\langle {\mathrm{grad}}_{x}f,v\rangle }_x=\underset{t\to 0}{lim}\frac{f\left(\gamma \right(t\left)\right)-f\left(x\right)}{t}.$$

(250)

Now let us underline an illusory contradiction in the above expression.

Observation 4.

The curve γ takes part only in closed proximity of the point x; therefore, the right-hand side of the above relation only depends on the point x and the tangent v, as well as the function f, and, in particular, it does not depend on the metric. In other words, the result of the expression ${lim}_{t\to 0}\frac{f\left(\gamma \right(t\left)\right)-f\left(x\right)}{t}$ quantifies how much the function f varies upon moving away from the point x toward the direction v. As opposed to this, the left-hand side of the relation (250) seems to depend on the metric $\langle ·,·\rangle$. Since the metric may be chosen arbitrarily and independently of the function f and of the quantities x and v, the above relation looks like a conceptual clash.

The only possible way to fix such an illusory contradiction is to recognize that the gradient itself must depend on the metric in such a way that the directional derivative ${\langle {\mathrm{grad}}_{x}f,v\rangle }_x$ does not. In other words, if we denote by ${\langle ·,·\rangle }^{\left(1\right)}$ and ${\langle ·,·\rangle }^{\left(2\right)}$ two metrics for the manifold $\mathbb{M}$, then it must hold that

$${\langle {\mathrm{grad}}_x^{\left(1\right)}f,v\rangle }_x^{\left(1\right)}={\langle {\mathrm{grad}}_x^{\left(2\right)}f,v\rangle }_x^{\left(2\right)}$$

(251)

for every $v\in T_x\mathbb{M}$, where ${\mathrm{grad}}_x^{\left(1\right)}f$ denotes the gradient of the function f at x corresponding to the metric ${\langle ·,·\rangle }^{\left(1\right)}$ and ${\mathrm{grad}}_x^{\left(2\right)}f$ denotes the gradient of the function f at x corresponding to the metric ${\langle ·,·\rangle }^{\left(2\right)}$.

Let us formalize the above discussion with the aim of formalizing the notion of Riemannian gradient on a Riemannian manifold. Let us consider a Riemannian manifold $\mathbb{M}$ embedded in an ambient space $\mathbb{A}$ and, for every point x, the tangent space $T_x\mathbb{M}$. Let us now consider a smooth function $f:\mathbb{M}\to \mathbb{R}$ and an inner product ${\langle ·,·\rangle }^{\mathbb{A}}$ in $\mathbb{A}$. We shall require that the function f is extendable to a neighborhood of x in $\mathbb{A}$ so as to be able to compute the Gateaux derivative ${\partial }_x$ of the function f with respect to x. (As usual, we are supposed to use an extension $\overline{f}$ in the following expressions.) Let us recall that

$$\underset{h\to 0}{lim}\frac{f(x+hv)-f\left(x\right)}{h}=:{\langle {\partial }_{x}f,v\rangle }^{\mathbb{A}}.$$

(252)

We denote the inner product that the tangent bundle $T\mathbb{M}$ is endowed with at $x\in \mathbb{M}$ as ${\langle ·,·\rangle }_x$. The Riemannian gradient ${\mathrm{grad}}_x^{\mathbb{M}}f$ of the function f over the manifold $\mathbb{M}$ at the point x is uniquely defined by the following two conditions:

• Tangency condition: For every $x\in \mathbb{M}$, ${\mathrm{grad}}_x^{\mathbb{M}}f\in T_x\mathbb{M}$.
• Metric compatibility condition: For every $x\in \mathbb{M}$ and every $v\in T_x\mathbb{M}\subset \mathbb{A}$, it holds that ${\langle {\mathrm{grad}}_x^{\mathbb{M}}f,v\rangle }_x={\langle {\partial }_{x}f,v\rangle }^{\mathbb{A}}$.

The tangency condition codifies the requirement that a Riemannian gradient must always be a tangent vector to the manifold, namely $(x,{\mathrm{grad}}_{x}f)\in T\mathbb{M}$, while the metric compatibility condition codifies the requirement that the inner product between a gradient vector and any other tangent vector belonging to the same tangent space takes a value that is invariant with respect to the chosen metric. The latter condition is better understood by considering that the linear component of the variation of a function when moving away from a point x of a quantity v is given by

$${\mathrm{d}}_{x}f\left(v\right)={\langle {\mathrm{grad}}_x^{\mathbb{M}}f,v\rangle }_x.$$

(253)

Clearly, the amount ${\mathrm{d}}_{x}f\left(v\right)$ depends only on x, f and the displacement v, certainly not on the metric. As a consequence, the gradient needs to depend on the metric. The ‘reference’ inner product is taken as the inner product that the ambient space is endowed with. In fact, the Riemannian gradient of a manifold-to-scalar function represents the rate of change of such function at a given point toward a given direction. The superscript ${}^{\mathbb{M}}$ may be removed for simplicity.

Let us survey the calculation of the Riemannian gradient in a number of spaces of interest in applications.

• Hypercube: In the space ${\mathbb{R}}^p$ endowed with the Euclidean metric, the Gateaux derivative of a regular function $f:{\mathbb{R}}^p\to \mathbb{R}$ is simply the column array of partial derivatives of function f with respect to the entries of the column array $x={[x^{\left(1\right)}{x}^{\left(2\right)}{x}^{\left(3\right)}\cdots x^{\left(p\right)}]}^{\top }$, namely

  $${\mathrm{grad}}_x^{{\mathbb{R}}^p}f={\partial }_{x}f:=\left[\begin{array}{c}\frac{\partial f}{\partial x^{\left(1\right)}}\\ \frac{\partial f}{\partial x^{\left(2\right)}}\\ ⋮\\ \frac{\partial f}{\partial x^{\left(p\right)}}\end{array}\right].$$

  (254)
  Likewise, the Gateaux derivative of a regular function $f:{\mathbb{R}}^{p\times q}\to \mathbb{R}$ is the Jacobian matrix of partial derivatives of function f with respect to the entries of the matrix X, denoted by $X^{(i,j)}$, namely

  $${\mathrm{grad}}_X^{{\mathbb{R}}^{p\times q}}f={\partial }_{X}f:=\left[\begin{array}{cccc}\frac{\partial f}{\partial X^{(1,1)}}& \frac{\partial f}{\partial X^{(1,2)}}& \cdots & \frac{\partial f}{\partial X^{(1,q)}}\\ \frac{\partial f}{\partial X^{(2,1)}}& \frac{\partial f}{\partial X^{(2,2)}}& \cdots & \frac{\partial f}{\partial X^{(2,q)}}\\ ⋮& ⋮& \ddots & ⋮\\ \frac{\partial f}{\partial X^{(p,1)}}& \frac{\partial f}{\partial X^{(p,2)}}& \cdots & \frac{\partial f}{\partial X^{(p,q)}}\end{array}\right].$$

  (255)
• Hypersphere: Given a regular function $f:{\mathbb{S}}^{p-1}\to \mathbb{R}$, its Riemannian gradient at $x\in {\mathbb{S}}^{p-1}$ is denoted as ${\mathrm{grad}}_x^{{\mathbb{S}}^{p-1}}f\in T_x{\mathbb{S}}^{p-1}$. The Riemannian gradient associated with the canonical metric has the expression:

  $${\mathrm{grad}}_x^{{\mathbb{S}}^{p-1}}f=(I_p-x{x}^{\top }){\partial }_{x}f.$$

  (256)
  In fact, the metric compatibility condition prescribes that $\langle {\mathrm{grad}}_x^{{\mathbb{S}}^{p-1}}f,v\rangle =\langle {\partial }_{x}f,v\rangle$, for every $v\in T_x{\mathbb{S}}^{p-1}$; hence, $\langle {\mathrm{grad}}_x^{{\mathbb{S}}^{p-1}}f-{\partial }_{x}f,v\rangle =0$ for every $v\in T_x{\mathbb{S}}^{p-1}$, and therefore ${\mathrm{grad}}_x^{{\mathbb{S}}^{p-1}}f-{\partial }_{x}f\in N_x{\mathbb{S}}^{p-1}$. It follows that

  $${\mathrm{grad}}_x^{{\mathbb{S}}^{p-1}}f={\partial }_{x}f+\lambda x,$$

  (257)

  for some $\lambda \in \mathbb{R}$. The tangency condition then entails ${\left({\mathrm{grad}}_x^{{\mathbb{S}}^{p-1}}f\right)}^{\top }x=0$, hence that ${({\partial }_{x}f+\lambda x)}^{\top }x=0$, namely $\lambda =-x^{\top }{\partial }_{x}f$, which gives the stated result.
• Special orthogonal group: Let us compute the Riemannian gradient of a regular function $f:\mathbb{SO}\left(p\right)\to \mathbb{R}$. Let the manifold $\mathbb{SO}\left(p\right)$ be equipped with its canonical metric. Let ${\mathrm{grad}}_X^{\mathbb{SO}\left(p\right)}f$ denote the gradient vector of a function f at $R\in \mathbb{SO}\left(p\right)$ derived from the canonical metric. According to the compatibility condition for the Riemannian gradient it must hold that:

  $$\langle V,{\mathrm{grad}}_R^{\mathbb{SO}\left(p\right)}f\rangle =\langle V,{\partial }_{R}f\rangle ,\forall V\in T_R\mathbb{SO}\left(p\right)$$

  and therefore:

  $$\langle V,{\mathrm{grad}}_R^{\mathbb{SO}\left(p\right)}f-{\partial }_{R}f\rangle =0,\forall V\in T_R\mathbb{SO}\left(p\right).$$
  This implies that the quantity ${\mathrm{grad}}_R^{\mathbb{SO}\left(p\right)}f-{\partial }_{R}f$ belongs to the normal space $N_R\mathbb{SO}\left(p\right)$, namely:

  $${\mathrm{grad}}_R^{\mathbb{SO}\left(p\right)}f={\partial }_{R}f+RS,withSbeingp\times psymmetric.$$

  (258)
  In order to determine the unknown matrix S, we may exploit the tangency condition, namely ${\left({\mathrm{grad}}_R^{\mathbb{SO}\left(p\right)}f\right)}^{\top }R+R^{\top }\left({\mathrm{grad}}_R^{\mathbb{SO}\left(p\right)}f\right)=0_p$. Let us first pre-multiply both sides of Equation (258) by $R^{\top }$, which gives:

  $$R^{\top }{\mathrm{grad}}_R^{\mathbb{SO}\left(p\right)}f=R^{\top }{\partial }_{R}f+S.$$
  The above equation, transposed hand by hand, gives:

  $${\left({\mathrm{grad}}_R^{\mathbb{SO}\left(p\right)}f\right)}^{\top }R={\partial }_R^{\top }fR+S.$$
  Hand-by-hand summation of the last two equations gives:

  $$R^{\top }{\partial }_{R}f+{\partial }_R^{\top }fR=-2S,$$

  that is:

  $$S=-\frac{1}{2}\left(R^{\top }{\partial }_{R}f+{\partial }_R^{\top }fR\right).$$

  (259)
  By plugging the expression (259) into the expression (258), we obtain the Riemannian gradient in the orthogonal group, corresponding to its canonical metric:

  $${\mathrm{grad}}_R^{\mathbb{SO}\left(p\right)}f=\frac{1}{2}\left({\partial }_{R}f-R{\partial }_R^{\top }fR\right).$$

  (260)
• Stiefel manifold: Let us compute the expression of the Riemannian gradient of a regular function $f:\mathrm{St}(n,p)\to \mathbb{R}$ at a point $X\in \mathrm{St}(n,p)$ corresponding to the Euclidean and the canonical metrics. Recall that the Riemannian gradient ${\mathrm{grad}}_X^{\mathrm{St}(n,p)}f$ in a Stiefel manifold embedded in the Euclidean space ${\mathbb{R}}^{n\times p}$ is the unique matrix in $T_X\mathrm{St}(n,p)$ such that:

  $$\mathrm{tr}\left(U^{\top }{\partial }_{X}f\right)={\langle U,{\mathrm{grad}}_{X}f\rangle }_X,\forall U\in T_X\mathrm{St}(n,p).$$

  (261)
  • Euclidean metric: The metric compatibility condition becomes $\mathrm{tr}(U^{\top }({\partial }_{X}f-$${\mathrm{grad}}_X^{\mathrm{St}(n,p)}f\left)\right)=0$, which implies that ${\mathrm{grad}}_X^{\mathrm{St}(n,p)}f={\partial }_{X}f+XS$, with S being $p\times p$ symmetric. Pre-multiplying both sides of this equation by the matrix $X^{\top }$ yields $S+X^{\top }{\partial }_{X}f=X^{\top }{\mathrm{grad}}_X^{\mathrm{St}(n,p)}f$. Transposing both hands of the above equation and summing hand by hand yields:

    $$S=-\frac{1}{2}\left({\partial }_X^{\top }fX+X^{\top }{\partial }_{X}f\right)+\frac{1}{2}\left({\left({\mathrm{grad}}_X^{\mathrm{St}(n,p)}f\right)}^{\top }X+X^{\top }{\mathrm{grad}}_X^{\mathrm{St}(n,p)}f\right).$$

    (262)
    From the condition ${\mathrm{grad}}_X^{\mathrm{St}(n,p)}f\in T_X\mathrm{St}(n,p)$, according to Equation (46), it follows that ${\left({\mathrm{grad}}_X^{\mathrm{St}(n,p)}f\right)}^{\top }X+X^{\top }{\mathrm{grad}}_X^{\mathrm{St}(n,p)}f=0$. In conclusion, the sought Riemannian gradient reads:

    $${\mathrm{grad}}_X^{\mathrm{St}(n,p)}f={\partial }_{X}f-\frac{1}{2}X\left({\partial }_X^{\top }fX+X^{\top }{\partial }_{X}f\right).$$

    (263)
  • Canonical metric: The metric compatibility condition prescribes that:

    $$\mathrm{tr}\left(U^{\top }\left({\partial }_{X}f-\left(I_n-\frac{1}{2}X{X}^{\top }\right){\mathrm{grad}}_X^{\mathrm{St}(n,p)}f\right)\right)=0,\forall U\in T_X\mathrm{St}(n,p),$$

    (264)

    and therefore, invoking the tangency condition as well, it turns out that:

    $$\left\{\begin{array}{c}{\partial }_{X}f-\left(I_n-\frac{1}{2}X{X}^{\top }\right){\mathrm{grad}}_X^{\mathrm{St}(n,p)}f=XS,\hfill \\ S=\frac{1}{2}\left(X^{\top }{\partial }_{X}f+{\partial }_X^{\top }fX\right).\hfill \end{array}\right.$$

    (265)
    Solving for the Riemannian gradient yields the final expression:

    $${\mathrm{grad}}_X^{\mathrm{St}(n,p)}f={\partial }_{x}f-X{\partial }_X^{\top }fX.$$

    (266)
• Real symplectic group: According to the two considered metrics, the expression of the gradient may be computed as outlined below.
  • KM metric: The structure of the pseudo-Riemannian gradient of a regular function $f:\mathrm{Sp}\left(2n\right)\to \mathbb{R}$ associated with the Khvedelidze–Mladenov metric (115) is given by

    $${\nabla }_{Q}f=\frac{1}{2}QJ\left(Q^{\top }{\partial }_{Q}fJ-J{\partial }_Q^{\top }fQ\right).$$

    (267)
    In fact, the pseudo-Riemannian gradient of a regular function $f:\mathrm{Sp}\left(2n\right)\to \mathbb{R}$ associated with the metric (115) is computed as the solution of the following system of equations:

    $$\left\{\begin{array}{c}\mathrm{tr}\left(Q^{-1}{\mathrm{grad}}_{Q}f{Q}^{-1}V\right)=\mathrm{tr}\left({\partial }_Q^{\top }fV\right),\forall V\in T_Q\mathrm{Sp}\left(2n\right),\hfill \\ {\mathrm{grad}}_Q^{\top }fJQ+Q^{\top }J{\mathrm{grad}}_{Q}f=0.\hfill \end{array}\right.$$

    (268)
    The first constraint ensures the compatibility of the pseudo-Riemannian gradient with the chosen metric, while the second constraint enforces the requirement for the pseudo-Riemannian gradient to lay on a specific tangent space. In particular, the metric compatibility condition may be recast as:

    $$\mathrm{tr}\left(V^{\top }({\partial }_{Q}f-Q^{-\top }{\mathrm{grad}}_Q^{\top }f{Q}^{-\top })\right)=0.$$

    (269)
    The above condition implies that ${\partial }_{Q}f-Q^{-\top }{\mathrm{grad}}_Q^{\top }f{Q}^{-\top }\in N_Q\mathrm{Sp}\left(2n\right)$, and hence that ${\partial }_{Q}f-Q^{-\top }{\mathrm{grad}}_Q^{\top }f{Q}^{-\top }=JQH$ with $H\in \mathfrak{so}\left(2n\right)$. Therefore, the pseudo-Riemannian gradient of the criterion function f has the expression:

    $${\nabla }_{Q}f=Q{\partial }_Q^{\top }fQ-QHJ.$$

    (270)
    In order to determine the value of the unknown skew-symmetric matrix H, it is sufficient to plug the expression (270) of the gradient within the tangency condition, which becomes:

    $${\left(Q{\partial }_Q^{\top }fQ-QHJ\right)}^{\top }JQ+Q^{\top }J\left(Q{\partial }_Q^{\top }fQ-QHJ\right)=0.$$

    (271)
    Solving for H gives:

    $$H=-\frac{1}{2}\left(J{Q}^{\top }{\partial }_{Q}f+{\partial }_Q^{\top }fQJ\right).$$

    (272)
    Plugging the above expression into (270) gives the result (267).
  • Euclidean metric: The structure of the Riemannian gradient ${\nabla }_{Q}f$ of a regular function $f:\mathrm{Sp}\left(2n\right)\to \mathbb{R}$ corresponding to the metric (116) reads:

    $${\nabla }_{Q}f=\frac{1}{2}QJ\left({\partial }_Q^{\top }fQJ-J{Q}^{\top }{\partial }_{Q}f\right).$$

    (273)
    In fact, the Riemannian gradient ${\nabla }_{Q}f$ must satisfy the conditions:

    $$\left\{\begin{array}{c}{\nabla }_{Q}f=QJ(H-Q^{-1}J{\partial }_{Q}f),\hfill \\ H=\frac{1}{2}(Q^{-1}J{\partial }_{Q}f+{\partial }_Q^{\top }fQJ),\hfill \end{array}\right.$$

    (274)

    from which the expression of the Riemannian gradient associated with the metric (116) follows.
• Manifold of symmetric, positive-definite matrices: The Riemannian gradient ${\mathrm{grad}}_{P}f$ of the function $f:{\mathbb{S}}^{+}\left(p\right)\to \mathbb{R}$ may be calculated as the unique vector in $T_P{\mathbb{S}}^{+}\left(p\right)$ that satisfies the following equation:

  $$\mathrm{tr}\left(V{\partial }_{P}f\right)=\mathrm{tr}\left(V{P}^{-1}\left({\mathrm{grad}}_P^{{\mathbb{S}}^{+}\left(p\right)}f\right)P^{-1}\right),\forall V\in T_P{\mathbb{S}}^{+}\left(n\right).$$

  (275)
  The solution of the above equation satisfies:

  $${\partial }_{P}f-P^{-1}\left({\mathrm{grad}}_P^{{\mathbb{S}}^{+}\left(p\right)}f\right)P^{-1}=\frac{1}{2}\left({\partial }_{P}f-{\partial }_P^{\top }f\right),$$

  (276)

  and hence the expression of the sought Riemannian gradient follows:

  $${\mathrm{grad}}_P^{{\mathbb{S}}^{+}\left(p\right)}f=\frac{1}{2}P\left({\partial }_{P}f+{\partial }_P^{\top }f\right)P.$$

  (277)
• Grassmann manifold: The Riemannian gradient ${\mathrm{grad}}_{\left[X\right]}f$ may be calculated by its definition and reads as:

  $${\nabla }_{\left[X\right]}^{\mathrm{Gr}(n,p)}f=(I_n-X{X}^{\top }){\partial }_{X}f.$$

  (278)

The above calculations may be conveniently unified by recalling the notion of metric kernel, which affords turning the metric compatibility condition into a projection. Let us recall that the inner product ${\langle v,u\rangle }_x$ may be written as ${\langle v,G_x\left(u\right)\rangle }^{\mathbb{A}}$. On the basis of such an equality, the metric compatibility condition may be rewritten as

$${\langle G_x\left({\mathrm{grad}}_x^{\mathbb{M}}f\right),v\rangle }^{\mathbb{A}}={\langle {\partial }_{x}f,v\rangle }^{\mathbb{A}}foreveryv\in T_x\mathbb{M}.$$

(279)

Equivalently, ${\langle G_x\left({\mathrm{grad}}_x^{\mathbb{M}}f\right)-{\partial }_{x}f,v\rangle }^{\mathbb{A}}=0$ for every $v\in T_x\mathbb{M}$. By the definition of a normal space, such a condition may be rewritten as

$$G_x\left({\mathrm{grad}}_x^{\mathbb{M}}f\right)-{\partial }_{x}f\in N_x\mathbb{M}.$$

(280)

In practical terms, the Gateaux derivative ${\partial }_{x}f\in \mathbb{A}$ may be uniquely decomposed into the sum of two terms, a tangent one (that it, the gradient) and a normal one. Upon discarding the normal component, the remainder is the sought gradient.

The latter observation may be expressed using the notion of orthogonal projection. Let us define an orthogonal projector ${\Pi }_x:\mathbb{A}\to T_x\mathbb{M}$ as an operator that maps an element of the ambient space to a tangent vector in a specific tangent space. Let us observe that

$${\Pi }_x\left(a\right)\in T_x\mathbb{M}\iff a-{\Pi }_x\left(a\right)\in N_x\mathbb{M}.$$

(281)

The projection operator ${\Pi }_x:\mathbb{A}\to T_x\mathbb{M}$ is defined by the two conditions:

• Tangency: ${\Pi }_x\left(a\right)\in T_x\mathbb{M}$ for $x\in \mathbb{M}$ and $a\in \mathbb{A}$.
• Complementarity: ${\langle v,a-{\Pi }_x\left(a\right)\rangle }^{\mathbb{A}}=0$ for all $v\in T_x\mathbb{M}$.

Notice that if $\nu \in N_x\mathbb{M}$ then ${\Pi }_x\left(\nu \right)=0$, while if $v\in T_x\mathbb{M}$ then ${\Pi }_x\left(v\right)=v$.

Observation 5.

Orthogonal projection stems from a minimization problem, namely, given a point $a\in \mathbb{A}$ and a vector space $\mathbb{V}$, the orthogonal projection of a on $\mathbb{V}$ is the shortest vector in $\mathbb{V}$ that approximates a, namely

$$\underset{v\in \mathbb{V}}{min}{\langle a-v,a-v\rangle }^{\mathbb{A}}.$$

(282)

Applying orthogonal projection to both sides of the relationship (280) leads to

$${\Pi }_x\left(G_x\left({\mathrm{grad}}_x^{\mathbb{M}}f\right)\right)-{\Pi }_x\left({\partial }_{x}f\right)=0,$$

(283)

but since ${\Pi }_x\left(G_x\left({\mathrm{grad}}_x^{\mathbb{M}}f\right)\right)=G_x\left({\mathrm{grad}}_x^{\mathbb{M}}f\right)$, we may write the explicit formula

$${\mathrm{grad}}_x^{\mathbb{M}}f=G_x^{-1}\left({\Pi }_x\left({\partial }_{x}f\right)\right).$$

(284)

The latter expression is quite suggestive, as it states that the Riemannian gradient of a function may be computed as the projection of its Gateaux derivative over a tangent space, to make it a tangent vector, further compensated by the metric kernel to take into account the effect of the metric (namely, to make it sure that the inner product of gradient with any tangent vector is independent of the metric). Notice that ${\langle {\mathrm{grad}}_x^{\mathbb{M}}f,v\rangle }_x={\langle G_x^{-1}\left({\Pi }_x\left({\partial }_{x}f\right)\right),G_x\left(v\right)\rangle }^{\mathbb{A}}$. Since $G_x$ is self-adjoint, such expression simplifies into ${\langle {\Pi }_x\left({\partial }_{x}f\right),v\rangle }^{\mathbb{A}}$ which is, in fact, independent of the metric kernel.

Let us summarize a few expressions of interest of orthogonal projection.

• Hypersphere: An expression of an orthogonal projector ${\Pi }_x:{\mathbb{R}}^n\to T_x{\mathbb{S}}^{p-1}$, for $x\in {\mathbb{S}}^{p-1}$, where the ambient space ${\mathbb{R}}^p$ is endowed with a Euclidean metric, is:

  $${\Pi }_x\left(a\right):=(I_p-x{x}^{\top })a.$$

  (285)
  Notice that $a-{\Pi }_x\left(a\right)=\left(x^{\top }a\right)x$ is radial (that is, directed along x) and hence normal.
• Stiefel manifold: It might be useful to define an orthogonal projection operator ${\Pi }_X:{\mathbb{R}}^{n\times p}\to T_X\mathrm{St}(n,p)$, for $X\in \mathrm{St}(n,p)$. Let us assume the ambient space $\mathbb{A}:={\mathbb{R}}^{n\times p}$ be endowed with a Euclidean metric ${\langle U,V\rangle }^{\mathbb{A}}:=\mathrm{tr}\left(U^{\top }V\right)$. In this case, the orthogonal projection takes the expression

  $${\Pi }_X\left(U\right):=U-\frac{1}{2}X(X^{\top }U+U^{\top }X),$$

  (286)
• Grassmann manifold: An expression of orthogonal projection ${\Pi }_{\left[X\right]}:{\mathbb{R}}^{n\times p}\to T_{\left[X\right]}\mathrm{Gr}$$(n,p)$, for $\left[X\right]\in \mathrm{Gr}(n,p)$ is

  $${\Pi }_{\left[X\right]}\left(V\right):=(I_n-X{X}^{\top })V.$$

  (287)

Let us apply the above considerations to a quadratic function on the hypersphere.

Example 25.

Let us consider the function $f:{\mathbb{S}}^{n-1}\to \mathbb{R}$ defined as:

$$f\left(x\right):=\frac{1}{2}{x}^{\top }{P}_{0}x,$$

(288)

with $P_0\in {\mathbb{S}}^{+}\left(n\right)$ being constant. We assume the hypersphere to be embedded in the ambient space ${\mathbb{R}}^n$ endowed with a Euclidean metric. The Euclidean gradient (and also the Gateaux derivative) of such a function reads as

$$\frac{\partial f}{\partial x}=P_{0}x.$$

(289)

Since, on the hypersphere, ${\Pi }_x\left(a\right)=(I-x{x}^{\top })a$, the Riemannian gradient of the function f takes the expression

$${\mathrm{grad}}_{x}f=(I_n-x{x}^{\top })P_{0}x=P_{0}x-\left(x^{\top }{P}_{0}x\right)x.$$

(290)

■

Let us verify that the expression (286) is indeed an orthogonal projection over the tangent bundle $T\mathrm{St}(n,p)$.

Example 26.

The expression ${\Pi }_X\left(U\right):=U-\frac{1}{2}X(X^{\top }U+U^{\top }X)$ realizes an orthogonal projection over the tangent space $T_X\mathrm{St}(n,p)$. To prove such a statement, it suffices to show that ${\Pi }_X\left(U\right)\in T_X\mathrm{St}(n,p)$ and that ${\langle U-{\Pi }_X\left(U\right),V\rangle }^{\mathbb{A}}=0$ for every $V\in T_X\mathrm{St}(n,p)$, which represent the conditions of tangency and of orthogonality, respectively. To what concerns the first property:

$$\begin{array}{cc}\hfill & X^{\top }{\Pi }_X\left(U\right)+{\Pi }_X^{\top }\left(U\right)X\hfill \\ \hfill & =X^{\top }U+U^{\top }X-\frac{1}{2}\left(X^{\top }X\right)(X^{\top }U+U^{\top }X)-\frac{1}{2}(X^{\top }U+U^{\top }X)\left(X^{\top }X\right)\hfill \\ \hfill & =0.\hfill \end{array}$$

(291)

To what concerns the second property, notice that

$$\begin{array}{cc}\hfill & {\langle U-{\Pi }_X\left(U\right),V\rangle }^{\mathbb{A}}\hfill \\ \hfill & =\mathrm{tr}(V^{\top }(U-U+\frac{1}{2}X(X^{\top }U+U^{\top }X))\hfill \\ \hfill & =\frac{1}{2}\mathrm{tr}\left(V^{\top }X(X^{\top }U+U^{\top }X)\right).\hfill \end{array}$$

(292)

Now observe that $H:=V^{\top }X$ is skew-symmetric, while $S:=X^{\top }U+U^{\top }X$ is symmetric, for every $X\in \mathrm{St}(n,p)$, $U\in \mathbb{A}$ and $V\in T_X\mathrm{St}(n,p)$. Since $\mathrm{tr}\left(HS\right)\equiv 0$, the statement holds true. ■

A further interesting counterexample clarifies the notion of orthogonality in orthogonal projection.

Example 27.

Let us consider the function ${\Pi }_X\left(U\right):=U-X{U}^{\top }X$ which realizes a projection over the tangent bundle $T\mathrm{St}(n,p)$ but not an orthogonal one as long as the Euclidean metric is concerned. Showing that it indeed realizes a projection is fairly easy; in fact,

$$\begin{array}{cc}\hfill & X^{\top }{\Pi }_X\left(U\right)+{\Pi }_X^{\top }\left(U\right)X\hfill \\ \hfill & =X^{\top }U-U^{\top }X+U^{\top }X-X^{\top }U\hfill \\ \hfill & =0.\hfill \end{array}$$

(293)

Likewise, we can show that such a projection isnotorthogonal in the Euclidean metric; in fact,

$$\begin{array}{cc}\hfill & {\langle U-{\Pi }_X\left(U\right),V\rangle }^{\mathbb{A}}\hfill \\ \hfill & =\mathrm{tr}\left(V^{\top }(U-U+X{U}^{\top }X)\right)\hfill \\ \hfill & =\mathrm{tr}\left(\left(V^{\top }X\right)\left(U^{\top }X\right)\right).\hfill \end{array}$$

(294)

Since the matrix product $U^{\top }X$ is arbitrary, the latter expression is not guaranteed to be null. ■

As a last example, let us consider the case of the symplectic group endowed with its canonical metric. The orthogonal projection is not easy to express in closed form.

Example 28.

Let us consider the symplectic group $\mathrm{Sp}\left(2n\right)$ embedded into the ambient space $\mathbb{A}:={\mathbb{R}}^{2n\times 2n}$ endowed with a Euclidean metric ${\langle A,B\rangle }^{\mathbb{A}}:=\mathrm{tr}\left(A^{\top }B\right)$. By definition of orthogonal projection, ${\Pi }_Q\left(A\right)$ must be an element of the tangent space $T_Q\mathrm{Sp}\left(2n\right)$, while $A-{\Pi }_Q\left(A\right)$ must be an element of the normal space $N_Q\mathrm{Sp}\left(2n\right)$. According to the characterization (49), it should then hold

$$A-{\Pi }_Q\left(A\right)=JQH,$$

(295)

namely ${\Pi }_Q\left(A\right)=A-JQH$, with H skew-symmetric to be determined. From the tangency condition, it follows that

$${(A-JQH)}^{\top }JQ+Q^{\top }J(A-JQH)=0.$$

(296)

Through some manipulations, the above equation may be rewritten as

$$H{Q}^{\top }Q+Q^{\top }QH=-A^{\top }JQ+Q^{\top }{J}^{\top }A.$$

(297)

All quantities in the above equation are known except for the unknown skew-symmetric matrix H. The above equation is an instance of the more general Sylvester equation (for a review, see, e.g., [55]).

According to [55], there exist a number of expressions to write the solution to such an equation. One solution is based on the Kronecker operations. In particular, in this case we can use the Kronecker sum $I_{2n}\oplus Q^{\top }Q$ and the vectorization operator ‘$\mathrm{vec}$’ to rewrite Equation (297) as

$$(I_{2n}\oplus Q^{\top }Q)\mathrm{vec}\left(H\right)=\mathrm{vec}(-A^{\top }JQ+Q^{\top }{J}^{\top }A).$$

(298)

The orthogonal projection operation may therefore be expressed as

$${\Pi }_Q\left(A\right)=A-JQ{\mathrm{vec}}^{-1}\left\{{(I_{2n}\oplus Q^{\top }Q)}^{-1}\mathrm{vec}(-A^{\top }JQ+Q^{\top }{J}^{\top }A)\right\}.$$

(299)

A more elegant expression stems from an integral representation of the solution of the Sylvester Equation (297), namely

$${\Pi }_Q\left(A\right)=A+JQ{\int }_0^{+\infty }\mathrm{Exp}\left(Q^{\top }Qt\right)(-A^{\top }JQ+Q^{\top }{J}^{\top }A)\mathrm{Exp}\left(Q^{\top }Qt\right)\mathrm{d}t.$$

(300)

For the integral representation of the solution of a Sylvester equation, we refer the readers to the aforementioned review paper [55]. We mention that none of these solutions are efficient from a computational point of view.

9.2. Application of Riemannian Gradient to Optimization on Manifold

An important application of the Riemannian gradient, which is of specific interest in system theory and non-linear control, is gradient-based mathematical optimization, which we are going to briefly survey in the remainder of the present section.

Let us preliminarily observe that the relationship (250) that defines the notion of Riemannian gradient possess a further interpretation. Let us take a regular function $f:\mathbb{M}\to \mathbb{R}$, a regular curve $\gamma :[-ϵ,ϵ]\to \mathbb{M}$ and a metric $\langle ·,·\rangle$ in $T\mathbb{M}$. The functions f and $\gamma$ may be composed as $f\circ \gamma :[-ϵ,ϵ]\to \mathbb{R}$. Therefore, from the relation (250), it follows that:

$$\frac{\mathrm{d}}{\mathrm{d}t}f\left(\gamma \left(t\right)\right)={\langle {\mathrm{grad}}_{\gamma \left(t\right)}f,\dot{\gamma }\left(t\right)\rangle }_{\gamma \left(t\right)}$$

(301)

along the curve $\gamma$. As a matter of fact, such a relation is a counterpart of the familiar rule of derivation for composite functions

$$\frac{\mathrm{d}}{\mathrm{d}t}f\left(g\left(t\right)\right)=\frac{\mathrm{d}f}{\mathrm{d}g}\left(g\left(t\right)\right)\frac{\mathrm{d}g}{\mathrm{d}t}\left(t\right).$$

(302)

To determine the maximum of a function on a manifold it is possible to make use of a gradient method. Such a method is based on the following important property: given a regular function $f:\mathbb{M}\to \mathbb{R}$, the solution $x\left(t\right)$ of the following initial value problem

$$\dot{x}\left(t\right)=\pm {\mathrm{grad}}_{x\left(t\right)}f,x\left(0\right)=x_0\in \mathbb{M},$$

(303)

is such that

$$\underset{t\to +\infty }{lim}x\left(t\right)=x_{★}$$

(304)

where $x_{★}$ denotes a point of local minimum of the function f near $x_0$ if one chooses the sign ‘−’ in front of the gradient; otherwise, $x_{★}$ denotes a local maximum of the function f near $x_0$ if one chooses the sign ‘+’ in front of the gradient.

For a function with domain ${\mathbb{R}}^n$, the extremal points are among those that make its partial derivatives vanish to zero. A similar result holds for functions whose domain is a smooth manifold. Given a differentiable function $f:\mathbb{M}\to \mathbb{R}$, the extremal points are those $x\in \mathbb{M}$ for which:

$${\mathrm{grad}}_{x}f=0.$$

(305)

This observation may be exploited

• To determine those points $x_{★}$ to which the solutions of the differential Equation (303) will tend toward;
• Considering that Equation (305) is often non-linear and difficult to solve in closed form, while the differential Equation (303) may be solved (albeit approximately) by numerical recipes, the differential Equation (303) may be considered as a way to solve an equation such as (305).

On the basis of the relation (301), it is quite straightforward to show the following fundamental result.

Theorem 4.

The differential equation

$$\dot{x}\left(t\right)=-{\mathrm{grad}}_{x\left(t\right)}f,x\left(0\right)=x_0\in \mathbb{M},$$

(306)

generates trajectories toward points of the domain corresponding to ever-decreasing values of the function f.

Proof. 

Let us observe that

$$\dot{f}={\langle {\mathrm{grad}}_{x}f,\dot{x}\rangle }_x={\langle {\mathrm{grad}}_{x}f,-{\mathrm{grad}}_{x}f\rangle }_x=-{\parallel {\mathrm{grad}}_{x}f\parallel }_x^2\le 0,$$

(307)

where equality holds if and only if ${\mathrm{grad}}_{x}f=0$. □

In a fully analogous way, upon choosing the sign + in front of the gradient, the corresponding differential equation generates trajectories tending toward ever-increasing values of the function f. Notice that the above facts are completely independent of the initial value $x_0$ and from the chosen metric.

In practical terms, it is safe to assume that the function f admits maxima and minima. For example, in the case of a continuous function f defined on a compact manifold, such as the hypersphere ${\mathbb{S}}^{n-1}$, a theorem by Weierstrass ensures that f admits at least a maximum and a minimum. (In fact, the Weierstrass extreme value theorem holds for topological spaces which any manifold is an instance of.)

To summarize, the differential Equation (303) may be exploited to look for the extremal points of a function:

• The dynamical system $\dot{x}\left(t\right)=+{\mathrm{grad}}_{x\left(t\right)}f,x\left(0\right)=x_0\in \mathbb{M}$ generates a trajectory in the state space $\mathbb{M}$ that tends toward a point of maximum of the function f located near the initial state $x_0$,
• Conversely, the dynamical system $\dot{x}\left(t\right)=-{\mathrm{grad}}_{x\left(t\right)}f,x\left(0\right)=x_0\in \mathbb{M}$ generates a trajectory in the space $\mathbb{M}$ that tends toward a point of minimum of the function f located near the initial state $x_0$.

The locality property of such a gradient-based extremum search is due to the monotonicity property (307), which essentially shows that the trajectory of such systems fall in the basin of attraction of the extremal point that the initial condition also belongs to. In practice, the choice of an initial condition (also termed, in this context, initial guess), is a sensitive step that determines the success or the failure of extremal-point searching. In fact, a function f might possess several extremal points (namely, local maxima and minima). It pays to keep in mind that the extremal-point searching method based on the first-order dynamical systems (303) is able to determine only an extremal point at a time and, in particular, only the one nearest to the initial guess $x_0$. It is therefore important to put into effect some pre-estimation technique to initialize the search procedure correctly.

From an algorithmic point of view, as opposed to what is required in numerical simulation where precision is a common demand, in the case of optimization by a gradient method, only the last point of the trajectory, namely $x_{★}$, needs to be approximated with good precision, and hence the employed numerical method does not need to be of a high order.

Example 29.

Let us consider Oja’s equation [56,57]

$$\dot{x}\left(t\right)=(I_n-x\left(t\right)x^{\top }\left(t\right))P_{0}x\left(t\right),x\left(0\right)=x_0\in {\mathbb{S}}^{n-1},$$

(308)

with $P_0\in {\mathbb{S}}^{+}\left(n\right)$. As we have seen in the previous example, the right-hand side of Oja’s equation coincides with the Riemannian gradient of the function (often termed ‘Rayleigh quotient’) $f\left(x\right):=\frac{1}{2}{x}^{\top }{P}_{0}x$ over the unit hypersphere endowed with its canonical metric. Such a function is quadratic, continuous and differentiable and is defined over a compact space; hence, it admits a maximum over the space ${\mathbb{S}}^{n-1}$. Indeed, it admits n local maxima. The differential Equation (308) hence affords determining one of its local maxima depending on the initial condition.

The extremal points are those that make the gradient ${\mathrm{grad}}_{x}f=P_{0}x-\left(x^{\top }{P}_{0}x\right)x$ vanish to zero, namely the solutions $x\in {\mathbb{S}}^{n-1}$ to the equation

$$P_{0}x=\left(x^{\top }{P}_{0}x\right)x.$$

(309)

Clearly, the solutions $x_{★}$ coincide with the eigenvectors of the symmetric, positive-definite matrix $P_0$. Let us observe that the eigenvalue associated with the eigenvector $x_{★}$ is ${\lambda }_{★}:=x_{★}^{\top }{P}_0{x}_{★}$; hence, $f\left(x_{★}\right)=\frac{1}{2}{\lambda }_{★}$. Since the dynamical system (308) looks for the maximal value of the function f, Oja’s equation generates a trajectory that tends toward the eigenvector of $P_0$ corresponding to its maximal eigenvalue. ■

The differential equations of the kind

$$\dot{x}\left(t\right)={\mathrm{grad}}_{x\left(t\right)}f,$$

(310)

represent an instance of dynamical systems of the first order, on a manifold, of the gradient type. (Notice that to switch the sign in front of the gradient, it suffices to take the function $-f$ instead of f. Likewise, to rescale the gradient of a factor $\eta >0$, it suffices to take $\eta f$ instead of f.)

9.3. A Golden Gradient Rule: Gradient of Squared Distance

A golden calculation rule involving the notion of Riemannian gradient and that of Riemannian distance is as follows:

$${\mathrm{grad}}_x{d}^2(x,y)=-2{log}_{x}y.$$

(311)

The formal proof of this result which, of course, holds under technical conditions, traces back to [53]. It is instructive to verify this property in a couple of cases.

Example 30.

Let us verify the calculation rule (311) for the familiar case that $\mathbb{M}={\mathbb{R}}^p$. We have already shown that $d^2(x,y)={\parallel {log}_{x}y\parallel }^2$. In the case of the hypercube ${\mathbb{R}}^p$, endowed with a Euclidean metric, it holds that ${log}_{x}y=y-x$; therefore, $d^2(x,y)={\parallel x-y\parallel }^2$, the familiar Euclidean distance in ${\mathbb{R}}^p$. Now, let us observe that

$$\frac{{\partial \parallel x-y\parallel }^2}{\partial x}=2(x-y),$$

(312)

and henceforth

$$\frac{\partial d^2(x,y)}{\partial x}=-2(y-x)=-2{log}_{x}y.$$

(313)

In addition, we can verify such a property for the hypersphere ${\mathbb{S}}^{p-1}$. Let us recall that $d^2(x,y)={\arccos }^2\left(x^{\top }y\right)$; therefore, it holds that

$$\frac{\partial d^2(x,y)}{\partial x}=2d(x,y)\left(-\frac{1}{\sqrt{1-{\left(x^{\top }y\right)}^2}}\right)y.$$

(314)

By the calculation rule for the gradient on the unit hypersphere, we obtain

$${\mathrm{grad}}_x{d}^2(x,y)=-\frac{2d(x,y)}{sind(x,y)}(I_p-x{x}^{\top })y=-2\frac{(I_p-x{x}^{\top })y}{\mathrm{sinc}d(x,y)}$$

(315)

which coincides precisely with $-2{log}_{x}y$ on the unit hypersphere. ■

For those who are mathematically minded, it is interesting to read a sketch of proof of the golden rule (311) based on properties that we have already surveyed.

Theorem 5.

Let us consider a Riemannian manifold $(\mathbb{M},\langle ·,·\rangle$) embedded in an ambient space $(\mathbb{A},{\langle ·,·\rangle }^{\mathbb{A}})$ and a pair of points $x,y\in \mathbb{M}$ such that $d(x,y)$ and ${log}_{x}y$ are well defined. The Riemannian gradient of the squared distance $d^2(x,y)$ with respect to the variable x equals $-2{log}_{x}y$.

Proof. 

Let us define the following functions:

• An arbitrary sufficiently smooth curve ${\alpha }_x\left(t\right)$ such that ${\alpha }_x\left(0\right)=x$, while ${\dot{\alpha }}_x\left(t\right)$ is arbitrary;
• The fundamental form $\mathcal{F}(x,v):={\langle v,v\rangle }_x$;
• A geodesic curve $c(t,s)$ connecting the point y to the point ${\alpha }_x\left(t\right)$, emanating from the former, with parameter $s\in [0,1]$;
• The partial derivatives $\dot{c}(t,s):=\frac{\partial c}{\partial t}(t,s)$ and $c^{\prime }(t,s):=\frac{\partial c}{\partial s}(t,s)$.

In addition, notice that the following properties hold:

(P₁)

$c(t,0)=y$, therefore $\dot{c}(t,0)=0$;

(P₂)

$c(t,1)={\alpha }_x\left(t\right)$, therefore $\dot{c}(t,1)={\dot{\alpha }}_x\left(t\right)$;

(P₃)

$c^{\prime }(t,1)=-{log}_{{\alpha }_x\left(t\right)}y$. (In fact, notice that if ${\gamma }_y^x:[0,1]\to \mathbb{M}$ denotes a geodesic from x to y then it holds that ${\dot{\gamma }}_x^y\left(0\right)={log}_{y}x$ and ${\gamma }_y^x\left(s\right)={\gamma }_x^y(1-s)$; therefore, ${\dot{\gamma }}_y^x\left(1\right)=-{\dot{\gamma }}_x^y\left(0\right)=-{log}_{y}x$.)

Let us recall that the squared Riemannian distance between two points may be written in terms of the fundamental form, namely

$$d^2({\alpha }_x\left(t\right),y)=\mathcal{F}(c(t,s),c^{\prime }(t,s)),$$

(316)

where the right-hand side of the above equation is independent of s, as we have shown in Section 8. Let us integrate both sides of the above equation with respect to s in the interval $[0,1]$. This gives:

$$d^2({\alpha }_x\left(t\right),y)={\int }_0^1\mathcal{F}(c(t,s),c^{\prime }(t,s))\mathrm{d}s.$$

(317)

Now, let us compute the derivative of both sides with respect to the parameter t:

$$\begin{array}{cc}\hfill \frac{\mathrm{d}}{\mathrm{d}t}{d}^2({\alpha }_x\left(t\right),y)=& {\int }_0^1\frac{\mathrm{d}}{\mathrm{d}t}\mathcal{F}(c(t,s),c^{\prime }(t,s))\mathrm{d}s\hfill \\ \hfill =& {\int }_0^1{〈\frac{\partial \mathcal{F}}{\partial c}(t,s),\frac{\partial c}{\partial t}(t,s)〉}^{\mathbb{A}}\mathrm{d}s+{\int }_0^1{〈\frac{\partial \mathcal{F}}{\partial c^{\prime }}(t,s),\frac{\partial c^{\prime }}{\partial t}(t,s)〉}^{\mathbb{A}}\mathrm{d}s.\hfill \end{array}$$

(318)

The second integral may be rewritten through the rule of integration by parts upon observing that $\frac{\partial c^{\prime }}{\partial t}=\frac{\partial \dot{c}}{\partial s}$ (Schwarz) and recalling that $\frac{\partial \mathcal{F}(x,v)}{\partial v}=2G_x\left(v\right)$, where G denotes the metric kernel associated with the metrics in $\mathbb{M}$ and $\mathbb{A}$. The application of such rule gives:

$$\begin{array}{cc}\hfill {\int }_0^1{〈\frac{\partial \mathcal{F}}{\partial c^{\prime }}(t,s),\frac{\partial c^{\prime }}{\partial t}(t,s)〉}^{\mathbb{A}}\mathrm{d}s=& {\left.{\langle 2G_x\left(c^{\prime }(t,s)\right),\dot{c}(t,s)\rangle }^{\mathbb{A}}\right|}_{s=0}^1\hfill \\ & -{\int }_0^1{〈\frac{\mathrm{d}}{\mathrm{d}t}\frac{\partial \mathcal{F}}{\partial c^{\prime }}(t,s),\dot{c}(t,s)〉}^{\mathbb{A}}\mathrm{d}s.\hfill \end{array}$$

(319)

Putting the pieces together gives:

$$\begin{array}{cc}\hfill \frac{\mathrm{d}}{\mathrm{d}t}{d}^2({\alpha }_x\left(t\right),y)=& {\int }_0^1{〈\frac{\partial \mathcal{F}}{\partial c}(t,s)-\frac{\mathrm{d}}{\mathrm{d}t}\frac{\partial \mathcal{F}}{\partial c^{\prime }}(t,s),\dot{c}(t,s)〉}^{\mathbb{A}}\mathrm{d}s\hfill \\ \hfill & +{\left.{\langle 2c^{\prime }(t,s),\dot{c}(t,s)\rangle }_{c(t,s)}\right|}_{s=0}^1\hfill \end{array}$$

(320)

Since $c(·,s)$ is a geodesic, the first integral is null due to the differential inclusion (153); therefore, we rest with

$$\frac{\mathrm{d}}{\mathrm{d}t}{d}^2({\alpha }_x\left(t\right),y)={\langle 2c^{\prime }(t,1),\dot{c}(t,1)\rangle }_{c(t,1)}-{\langle 2c^{\prime }(t,0),\dot{c}(t,0)\rangle }_{c(t,0)}.$$

(321)

Now, due to property ${\mathrm{P}}_1$, the second term on the right-hand side of the above equality is null; hence, thanks to properties ${\mathrm{P}}_2$–${\mathrm{P}}_3$, the above equality simplifies to

$$\frac{\mathrm{d}}{\mathrm{d}t}{d}^2({\alpha }_x\left(t\right),y)=2{\langle -{log}_{{\alpha }_x\left(t\right)}y,{\dot{\alpha }}_x\left(t\right)\rangle }_{{\alpha }_x\left(t\right)}.$$

(322)

Setting $t=0$ and recalling the definition of gradient (249) proves the golden rule. □

Let us emphasize how useful such a result is, since in non-linear control a number of functions to differentiate are based on distances. Let us apply the golden calculation rule to a non-linear control problem known as consensus optimization.

Example 31.

A category of dynamical systems of gradient type on manifolds are those associated withconsensus optimizationamong a set of agents [58,59]. Let us consider, as an example, a dynamical system that affords reaching attitudinal consensus in a fleet of flying objects (such as drones).

The attitude of a flying agent (namely, its orientation in space) may be represented by a rotation matrix $R\in \mathbb{SO}\left(3\right)$ that summarizes the three attitudinal coordinatesroll,pitchandyaw. A fleet of N flying agents may be described as a set of N rotation matrices $R_i\in \mathbb{SO}\left(3\right)$ with $i=1,2,3,\dots ,N$.

The attitude of each agent in the fleet is then described by a time-function $R_i\left(t\right)$. Consensus optimization may be formulated as follows:

• Given a metric for the manifold $\mathbb{SO}\left(3\right)$, it is possible to define a distance function $d:\mathbb{SO}\left(3\right)\times \mathbb{SO}\left(3\right)\to {\mathbb{R}}_0^{+}$ between any pair of attitude matrices in $\mathbb{SO}\left(3\right)$;
• A hierarchy is established among the agents in a fleet through a set of weights $w_{ij}\ge 0$;
• Then, consensus optimization consists of determining an evolution law for each agent to minimize the distance between any pair of attitude matrices weighted according to the assigned hierarchy, namely, to minimize the function

  $$f_i:=\frac{1}{2}\sum _{j=1}^N{w}_{ij}{d}^2(R_i,R_j),withi=1,2,3,\dots ,N.$$

  (323)
  Notice that $d(R_i,R_i)=0$; hence, the values assigned to the diagonal coefficients $w_{ii}$ are unimportant.

On the basis of the calculation rule (311), it turns out that

$${\mathrm{grad}}_{R_i}{f}_i=-\sum _{j=1}^N{w}_{ij}{log}_{R_i}{R}_j.$$

(324)

The set of gradient-type control systems to achieve consensus optimization within the fleet reads:

$${\dot{R}}_i\left(t\right)=\sum _{j=1}^N{w}_{ij}{log}_{R_i\left(t\right)}{R}_j\left(t\right),R_i\left(0\right)=R_i^0\in \mathbb{SO}\left(3\right),$$

(325)

with $i=1,2,3,\dots ,N$ and $t\ge 0$. (Notice that ${log}_{R}R=0$.) ■

9.4. Riemannian Gradient in Coordinates*

Let $f:\mathbb{M}\to \mathbb{R}$ denote a smooth manifold-to-real function, whose gradient at a point $x\in \mathbb{M}$ is sought. Let us fix a coordinate system on a local chart that includes the point x whose local coordinates are $x^1,x^2,\dots x^p$.

In order to determine the gradient ${\mathrm{grad}}_{x}f$, let us make use of the metric compatibility condition, recalling that

• The Euclidean inner product reads $\langle {\partial }_{x}f,v\rangle =\frac{\partial f}{\partial x^k}{v}^k$.
• The inner product reads ${\langle {\mathrm{grad}}_{x}f,v\rangle }_x=g_{ik}{\left({\mathrm{grad}}_{x}f\right)}^i{v}^k$.

The metric compatibility condition requires that

$$\frac{\partial f_x}{\partial x^k}{v}^k=g_{ik}{\left({\mathrm{grad}}_{x}f\right)}^i{v}^k\mathrm{for}\mathrm{every}\mathrm{choice}\mathrm{of}v,$$

(326)

hence, $g_{ik}{\left({\mathrm{grad}}_{x}f\right)}^i=\frac{\partial f_x}{\partial x^k}$. Multiplying both sides by $g^{hk}$ one obtains $g^{hk}{g}_{ik}{\left({\mathrm{grad}}_{x}f\right)}^i=g^{hk}\frac{\partial f_x}{\partial x^k}$. Since $g^{hk}{g}_{ik}={\delta }_i^h$, we obtain

$${\left({\mathrm{grad}}_{x}f\right)}^h=g^{hk}\frac{\partial f_x}{\partial x^k}.$$

(327)

The Riemannian gradient in intrinsic coordinates is therefore expressed as

$${\mathrm{grad}}_{x}f=g^{hk}\frac{\partial f_x}{\partial x^k}{\partial }_h^x.$$

(328)

In manifold calculus, the Riemannian gradient has yet another interpretation in relation to the differential of a manifold-to-real function and to so-called musical isomorphisms. In fact, there exist canonical operators to convert contravariant components to covariant (also referred to as lowering an index) or vice versa (raising an index).

Given a regular function $f:\mathbb{M}\to \mathbb{R}$, its differential $\mathrm{d}f:T\mathbb{M}\to T^{*}\mathbb{M}$ is expressed by:

$$\mathrm{d}{f}_x\left(v\right)={\langle {\mathrm{grad}}_{x}f,v\rangle }_x,v\in T_x\mathbb{M},$$

(329)

where ${\mathrm{grad}}_{x}f:\mathbb{M}\to T_x\mathbb{M}$ denotes its Riemannian gradient. Namely, the differential represents the linear part of the change of a function when moving away from a point x in the direction v. In local coordinates:

$$\mathrm{d}{f}_x\left(v\right)=g_{ij}\left(x\right){\left({\nabla }_{x}f\right)}^i{v}^j.$$

(330)

Since the differential does not depend on the choice of coordinates, it must hold that $\mathrm{d}{f}_x\left(v\right)={\left({\partial }_{i}f\right)}_x{v}^i$. It follows the metric-compatibility condition $g_{ij}\left(x\right){\left({\nabla }_{x}f\right)}^j={\left({\partial }_{i}f\right)}_x$. Such expressions may be interpreted through ‘musical’ isomorphisms:

• Sharp isomorphism (${}^{♯}$): The ‘sharp’ isomorphism ${}^{♯}:T^{*}\mathbb{M}\to TM$ takes a cotangent vector and returns a tangent vector. Namely, given a cotangent vector $\omega ={\omega }_i\mathrm{d}{x}^i$, the sharp isomorphism acts like $v:={\omega }^{♯}=g^{ij}{\omega }_i{\partial }_j$. For example, in the case of gradient, one may say that ${\mathrm{grad}}_{x}f={\left(d{f}_x\right)}^{♯}$.
• Flat isomorphism (${}^{♭}$): The ‘flat’ isomorphism ${}^{♭}:T\mathbb{M}\to T^{*}M$ takes a tangent vector and returns a cotangent vector. Namely, given a tangent vector $v=v^i{\partial }_i^x$, the flat isomorphism acts like $\omega :=v^{♭}=g_{ij}{v}^i\mathrm{d}{x}^j$. Then, the differential is the dual of gradient via a flat isomorphism.

10. Parallelism and Parallel Transport along a Curve

Parallel transport along a curve on a manifold is a fundamental concept that emerges from the curved nature of manifolds and is deeply interwoven in the fabric of manifold calculus.

10.1. Properties and Definition of Parallel Transport

Given a curve on a Riemannian manifold $\mathbb{M}$ embedded into a Euclidean space $\mathbb{A}$, a point $x\in \mathbb{M}$ on the curve and a tangent vector $v\in T_x\mathbb{M}$ to the curve, if such a vector is moved to another point on the curve by ordinary rigid translation (which is available in $\mathbb{A}$ and may be referred to as ‘parallel translation’), in general it will not be tangent to the curve anymore. This observation suggests that it is necessary to define a notion of transport that is compatible with the structure of the manifold.

Another reason to motivate the notion of parallel transport is the lack of existence of ‘uniform’ vector fields on curved manifolds. On a flat space, such as ${\mathbb{R}}^p$, a uniform vector field $u:{\mathbb{R}}^p\to {\mathbb{R}}^p$ is a field $x\mapsto u_x$ such that $u_x$ is constant (same direction, length and orientation) for every $x\in {\mathbb{R}}^p$. On a curved manifold embedded in an ambient space $\mathbb{A}$, however, it is unlikely that a tangent vector field may take the same value in two points, because tangency at one point would most likely imply lack of tangency in another point. The closest notion to uniformity would be parallelism: Informally speaking, a vector field is said to be parallel along a curve if it keeps the same orientation with respect to such a curve’s own velocity field. Notice that this property is expressed in quite vague terms: such vagueness is indeed intended and implies that there exist infinitely many ways to interpret the notion of parallel transport.

Example 32.

Let $\mathbb{M}$ denote a Riemannian manifold and $x\mapsto w_x$ a tangent vector field on the tangent bundle $T\mathbb{M}$; namely, the function w assigns a tangent vector to each point x on the manifold. In the familiar case that $\mathbb{M}={\mathbb{R}}^3$, there exists a notion ofuniformvector field; namely, a vector field may take the same amplitude and direction in any point of the base-space $\mathbb{M}$. This is the case, for example, of an electric field within an ideal capacitor or an induction field within an ideal electrical coil. A distinguishing feature of a uniform vector field in ${\mathbb{R}}^3$ is that its directional derivative is zero everywhere; namely, given any point $x\in {\mathbb{R}}^3$ and any direction $v\in T_x{\mathbb{R}}^3$, it holds that

$$\underset{h\to 0}{lim}\frac{w_{x+hv}-w_x}{h}=0,$$

(331)

where the left-hand side represents the directional derivative of the vector field $w\in \Gamma \left(\mathbb{M}\right)$ along a direction v at the point x.

On a curved manifold $\mathbb{M}$, uniform vector fields hardly exist, as the rigid translation of a tangent vector to a different point of a manifold will most likely result in a non-tangent vector. The closest notion to uniformity that may be recovered on a curved manifold is that of parallelism. A parallel vector field needs to satisfy a generalized version of the condition (331) (which we shall not see in this survey since it requires covariant derivation, which will be covered in a separate survey). ■

The notion of parallelism of a vector field gives rise to an important canonical operator in manifold calculus, namely, parallel transport along a curve. Parallel transport allows one to, e.g., compare vectors belonging to different tangent spaces. On a smooth Riemannian manifold $\mathbb{M}$, fix a smooth curve $\gamma :[0,1]\to \mathbb{M}$. The parallel transport map associated with the curve $\gamma$ is denoted as ${\mathbb{P}}_{\gamma }^{s\to t}:T_{\gamma \left(s\right)}\mathbb{M}\to T_{\gamma \left(t\right)}\mathbb{M}$, which is a linear map for every $s,t\in [0,1]$, namely

$${\mathbb{P}}_{\gamma }^{s\to t}(\alpha w+\beta v)=\alpha {\mathbb{P}}_{\gamma }^{s\to t}\left(w\right)+\beta {\mathbb{P}}_{\gamma }^{s\to t}\left(v\right)$$

(332)

for every $\alpha ,\beta \in \mathbb{R}$ and $u,v\in T_{\gamma \left(s\right)}\mathbb{M}$. The parallel transport map depends smoothly on its arguments and is such that ${\mathbb{P}}_{\gamma }^{s\to s}$ coincides with the identity map and ${\mathbb{P}}_{\gamma }^{u\to t}\circ {\mathbb{P}}_{\gamma }^{s\to u}={\mathbb{P}}_{\gamma }^{s\to t}$ for every $s,u,t\in [0,1]$. (Notice that the last property holds on the same curve only, while it does not hold for different arcs.)

In most cases of importance in system theory and control, the main interest lies in the parallel transport of a tangent vector along a geodesic curve on a Riemannian manifold, rather than along a generic smooth curve. Parallel transport along a geodesic arc may be conjugated in at least two ways:

• Parallel transport along a geodesic arc joining two given points: Given two (sufficiently close) points $x,y\in \mathbb{M}$, and a tangent vector $w\in T_x\mathbb{M}$, the parallel transport of w from x to y along the geodesic arc connecting x to y is denoted by ${\mathbb{P}}^{x\to y}\left(w\right)$, since this notation shows all relevant information. In fact, the notation ${\mathbb{P}}^{x\to y}$ for the parallel transport operator is a shortened version of ${\mathbb{P}}_{{\gamma }_x^y}^{0\to 1}:T_x\mathbb{M}\to T_y\mathbb{M}$, where ${\gamma }_x^y:[0,1]\to \mathbb{M}$ would denote the geodesic arc such that ${\gamma }_x^y\left(0\right)=x$ and ${\gamma }_x^y\left(1\right)=y$.
• Parallel transport along a geodesic specified by a point and a tangent direction: Given a point $x\in \mathbb{M}$, and a tangent direction $v\in T_x\mathbb{M}$, the parallel transport of a vector $w\in T_x\mathbb{M}$ along the geodesic arc departing from the point x toward the direction v is denoted by ${\mathbb{P}}_{{\gamma }_{x,v}}^{0\to t}\left(w\right)$, where ${\gamma }_{x,v}:[-ϵ,ϵ]\to \mathbb{M}$, with $ϵ>0$ and $t\in [-ϵ,ϵ]$, would denote the geodesic curve such that ${\gamma }_{x,v}\left(0\right)=x$ and ${\dot{\gamma }}_{x,v}\left(0\right)=v$.

There exist other special cases of interest in control and automation. One such special case supplies a partial answer to the problem of effecting parallel transport on a manifold for which the structure of the parallel transport operator is unknown. Another special case is parallel transport along a closed loop, which may arise when dealing with periodic trajectories or self-intersecting trajectories.

• Self parallel transport along a geodesic arc: A distinguishing feature of a geodesic arc is that it parallel-transports its own initial slope. On a setting where the parallel transport operator is unknown in closed form, there exists no known way to transport a given vector w along a geodesic curve ${\gamma }_{x,v}$. However, it is possible to parallel-transport a given vector v along a geodesic ${\gamma }_{x,v}$, namely to compute ${\mathbb{P}}_{{\gamma }_{x,v}}^{0\to t}\left(v\right)$: it coincides exactly with ${\dot{\gamma }}_{x,v}\left(t\right)$. Such a numerical trick was invoked, for instance, in [39], Subsection III.A.
• Parallel transport along a closed loop: Parallel transport of a vector along a (piece-wise continuous) loopℓ is denoted as ${\mathbb{P}}_{\ell }:T_x\mathbb{M}\to T_x\mathbb{M}$, where x is termed base of the loop. In general, ${\mathbb{P}}_{\ell }\left(v\right)\ne v$, for $v\in T_x\mathbb{M}$. This phenomenon is referred to as anholonomy. Whenever ${\mathbb{P}}_{\ell }$ realizes an isometry, since $\parallel {\mathbb{P}}_{\ell }{\left(v\right)\parallel }_x={\parallel v\parallel }_x$, the operator ${\mathbb{P}}_{\ell }$ changes only the orientation of v. For example, if $\mathbb{M}\subset {\mathbb{R}}^3$, then we may say that ${\mathbb{P}}_{\ell }\left(v\right)$ is a rotated version of a tangent vector v laying in the same tangent space, namely, ${\mathbb{P}}_{\ell }$ may be represented as an element of the orthogonal group $\mathbb{O}\left(3\right)$. Intuitively, holonomy is specifically related to curved spaces, and therefore holonomy must be related to curvature. This conjecture is made precise by the Ambrose–Singer theorem [60].

In the following, we shall go through a detailed derivation of the notion of parallel transport along a curve on the tangent bundle of a manifold embedded in an ambient space. In general, the result of parallel transport depends on the curve; moreover, parallel transport may be conceived in a way that preserves the angle between transported vectors but not their lengths, or to preserve both.

10.2. Coordinate-Free Derivation of Parallel Transport

Let us develop a notion of parallel transport $\mathbb{P}$ such that, in every point of the curve $\gamma$, it holds that ${\mathbb{P}}_{\gamma }^{0\to t}\left(w\right)\in T_{\gamma }\mathbb{M}$. In addition, a fundamental requirement that parallel transport should fulfill is that, given two tangent vectors $u,w\in T_x\mathbb{M}$ that form an angle $\alpha$, parallel transport should preserve the value of such angle along the whole curve, namely the angle between ${\mathbb{P}}_{\gamma }^{0\to t}\left(u\right)$ and ${\mathbb{P}}_{\gamma }^{0\to t}\left(w\right)$. Let us recall that the angle $\alpha$ between two vectors $u,w$ in $T_x\mathbb{M}$ is defined by

$$cos\left(\alpha \right):=\frac{{\langle u,w\rangle }_x}{{\parallel u\parallel }_x{\parallel w\parallel }_x}.$$

(333)

We may thus require parallel transport to fulfill:

• Tangency condition: for every $t\in [0,1]$ it should hold that ${\mathbb{P}}_{\gamma }^{0\to t}\left(w\right)\in T_{\gamma \left(t\right)}\mathbb{M}$,
• Conformal isometry condition: for every value of the parameter $t\in [0,1]$ it should hold that ${\langle {\mathbb{P}}_{\gamma }^{0\to t}\left(u\right),{\mathbb{P}}_{\gamma }^{0\to t}\left(w\right)\rangle }_{\gamma \left(t\right)}$ keeps constant to ${\langle u,w\rangle }_x$ along the curve.

Let us verify that the second condition implies both conformality and isometry. To what concerns isometry, let us show that taking $w=u$ gives:

$${\langle {\mathbb{P}}_{\gamma }^{0\to t}\left(u\right),{\mathbb{P}}_{\gamma }^{0\to t}\left(u\right)\rangle }_{\gamma \left(t\right)}={\parallel {\mathbb{P}}_{\gamma }^{0\to t}\left(u\right)\parallel }_{\gamma \left(t\right)}^2,$$

(334)

from which it follows that $\parallel {\mathbb{P}}_{\gamma }^{0\to t}{\left(u\right)\parallel }_{\gamma \left(t\right)}$ keeps constant along the curve $\gamma$ to ${\parallel u\parallel }_x$. It is straightforward to prove conformality; in fact,

$$\frac{{\langle {\mathbb{P}}_{\gamma }^{0\to t}\left(u\right),{\mathbb{P}}_{\gamma }^{0\to t}\left(w\right)\rangle }_{\gamma \left(t\right)}}{\parallel {\mathbb{P}}_{\gamma }^{0\to t}{\left(u\right)\parallel }_{\gamma \left(t\right)}{\parallel {\mathbb{P}}_{\gamma }^{0\to t}\left(w\right)\parallel }_{\gamma \left(t\right)}}=\frac{{\langle u,w\rangle }_x}{{\parallel u\parallel }_x{\parallel w\parallel }_x},\forall t\in [0,1].$$

(335)

(Let us recall that a linear transformation that preserves angles is termed conformal.) It is important to underline that the above conditions do not define parallel transport univocally, but they represent minimal requirements for an operator to be qualified as parallel transport. In other words, parallel transport may be defined in several ways as long as it meets the above conditions, which turn out to be of great importance in non-linear control.

To ease notation, let us define the following fields:

• $\overline{u}\left(t\right):={\mathbb{P}}_{\gamma }^{0\to t}\left(u\right)$: a vector field $(\gamma \left(t\right),\overline{u}\left(t\right))$ that represents the evolution, over the curve $\gamma$, of the vector u that is tangent to the manifold at $\gamma \left(0\right)$ and after transport it will be tangent at $\gamma \left(1\right)$,
• $\overline{w}\left(t\right):={\mathbb{P}}_{\gamma \left(t\right)}^{0\to t}\left(w\right)$: analogous to $\overline{u}\left(t\right)$, both are transport fields,
• $\mathcal{Q}\left(t\right):={\langle \overline{u}\left(t\right),\overline{w}\left(t\right)\rangle }_{\gamma \left(t\right)}$: a scalar field that represents the inner product between the above two transport fields along the curve $\gamma$.

For a manifold $\mathbb{M}$ embedded in an ambient space $\mathbb{A}$ endowed with a metric ${\langle ·,·\rangle }^{\mathbb{A}}$, the condition of conformality and isometry may be written in a more detailed way. Let us recall from Section 7 that the inner product of two tangent vectors may be written as ${\langle u,w\rangle }_x={\langle u,G_x\left(w\right)\rangle }^{\mathbb{A}}$, where the metric kernel has key properties, namely $G_x\left(v\right)$ is linear in v, $G_x$ is self-adjoint (namely, ${\langle u,G_x\left(w\right)\rangle }^{\mathbb{A}}={\langle G_x\left(u\right),w\rangle }^{\mathbb{A}}$), $G_x$ is invertible and its inverse is denoted as $G_x^{-1}$, and $G_x$ maps $T_x\mathbb{M}$ to itself.

In order to perform the following calculations seamlessly, it pays to highlight a few calculation rules about the metric kernel G and the fundamental form $\mathcal{F}$:

• Taylor series expansion of the kernel G: Given $G_x\left(v\right)$ with $x\in \mathbb{M}$, $v\in T_x\mathbb{M}$, let us write

  $$G_{{\gamma }_{x,w}\left(t\right)}\left(v\right)=G_x\left(v\right)+G_x^{•}(v,tw)+\mathcal{O}\left(t^2\right),$$

  (336)

  where ${\gamma }_{x,w}:[-ϵ,ϵ]\to \mathbb{M}$ denotes any smooth curve such that ${\gamma }_{x,w}\left(0\right)=x$ and ${\dot{\gamma }}_{x,w}\left(0\right)=w$, with $w\in T_x\mathbb{M}$ denoting a direction along which a variation of the metric kernel is sought. The quantity $G_x^{•}(v,w)$ represents a first-order derivative of the kernel $G_x$ calculated with respect to the variable x. (Notation-wise, the ‘bullet’ derivative is a convenient way to denote a derivative that cannot be expressed in more specific terms. For example, if $f,g$ are real functions of a real variable, one might denote $\frac{\mathrm{d}}{\mathrm{d}x}f\left(g\left(x\right)\right)=\dot{f}\left(g\left(x\right)\right)\dot{g}\left(x\right)$ as $f^{•}(g\left(x\right),\dot{g}\left(x\right))$.) The formal definition goes like

  $$G_x^{•}(v,w):={\left.\frac{\mathrm{d}}{\mathrm{d}t}{G}_{{\gamma }_{x,w}\left(t\right)}\left(v\right)\right|}_{t=0},$$

  (337)

  where again ${\gamma }_{x,w}:[-ϵ,ϵ]\to \mathbb{M}$ denotes any smooth curve such that ${\gamma }_{x,w}\left(0\right)=x$ and ${\dot{\gamma }}_{x,w}\left(0\right)=w$. The function $G_x^{•}(v,w)$ is linear in the arguments v and w and, in general, non-linear in x. Notice that the extendability property of $G_x$ from $T_x\mathbb{M}$ to $\mathbb{A}$ holds also for the first argument of the derivative $G_x^{•}$.
• Commutativity with the inner product in the ambient space: In the expression ${\langle u,G_x^{•}(v,w)\rangle }^{\mathbb{A}}$ it is allowed to swap the arguments u and v. In fact, notice that

  $$\begin{array}{cc}\hfill {\langle u,G_x^{•}(v,w)\rangle }^{\mathbb{A}}=& {\langle u,{\left.\frac{\mathrm{d}}{\mathrm{d}t}{G}_{{\gamma }_{x,w}\left(t\right)}\left(v\right)\right|}_{t=0}\rangle }^{\mathbb{A}}={\left.\frac{\mathrm{d}}{\mathrm{d}t}{\langle u,G_{{\gamma }_{x,w}\left(t\right)}\left(v\right)\rangle }^{\mathbb{A}}\right|}_{t=0}\hfill \\ \hfill =& {\left.\frac{\mathrm{d}}{\mathrm{d}t}{\langle G_{{\gamma }_{x,w}\left(t\right)}\left(u\right),v\rangle }^{\mathbb{A}}\right|}_{t=0}={\langle {\left.\frac{\mathrm{d}}{\mathrm{d}t}{G}_{{\gamma }_{x,w}\left(t\right)}\left(u\right)\right|}_{t=0},v\rangle }^{\mathbb{A}}\hfill \\ \hfill =& {\langle v,G_x^{•}(u,w)\rangle }^{\mathbb{A}}.\hfill \end{array}$$

  (338)
  This property of $G_x^{•}$ holds even if the first argument of the derivative belongs to $\mathbb{A}$.
• Partial derivatives of the fundamental form: Let us recall the definition of the fundamental form $\mathcal{F}(x,v):={\langle v,G_x\left(v\right)\rangle }^{\mathbb{A}}$. Its partial derivatives read

  $$\frac{\partial \mathcal{F}}{\partial x}(x,v)=G_x^{•}(v,v),\frac{\partial \mathcal{F}}{\partial v}(x,v)=2G_x\left(v\right).$$

  (339)

Let us see a few examples to clarify the above definitions and properties.

Example 33.

Let us start considering the case of the manifold ${\mathbb{S}}^{+}\left(n\right)$ endowed with its canonical metric, embedded in the ambient space $\mathbb{A}:={\mathbb{R}}^{n\times n}$ endowed with a Euclidean metric. In this setting, we have seen that $G_P\left(V\right)=P^{-1}V{P}^{-1}$. By definition, its ‘bullet derivative’ reads

$$G_P^{•}(V,W)={\left.\frac{\mathrm{d}}{\mathrm{d}t}\left({\gamma }^{-1}V{\gamma }^{-1}\right)\right|}_{t=0},$$

(340)

where $\gamma :[-ϵ,ϵ]\to {\mathbb{S}}^{+}\left(n\right)$ denotes any smooth curve such that $\gamma \left(0\right)=P\in {\mathbb{S}}^{+}\left(n\right)$ and $\dot{\gamma }\left(0\right)=W\in T_P{\mathbb{S}}^{+}\left(n\right)$. By using the noticeable matrix-flow derivation rule (104), we obtain

$$\frac{\mathrm{d}}{\mathrm{d}t}\left({\gamma }^{-1}V{\gamma }^{-1}\right)=(-{\gamma }^{-1}\dot{\gamma }{\gamma }^{-1})V{\gamma }^{-1}+{\gamma }^{-1}V(-{\gamma }^{-1}\dot{\gamma }{\gamma }^{-1}).$$

(341)

Setting $t=0$ leads to the sought expression

$$G_P^{•}(V,W)=-P^{-1}W{P}^{-1}V{P}^{-1}-P^{-1}V{P}^{-1}W{P}^{-1}.$$

(342)

Let us verify the property of commutativity with the inner product. We may calculate that

$$\begin{array}{cc}\hfill {\langle G_P^{•}(V,W),U\rangle }^{\mathbb{A}}=& -\mathrm{tr}\left(P^{-1}W{P}^{-1}V{P}^{-1}U\right)-\mathrm{tr}\left(P^{-1}V{P}^{-1}W{P}^{-1}U\right),\hfill \\ \hfill {\langle G_P^{•}(U,W),V\rangle }^{\mathbb{A}}=& -\mathrm{tr}\left(P^{-1}U{P}^{-1}W{P}^{-1}V\right)-\mathrm{tr}\left(P^{-1}W{P}^{-1}U{P}^{-1}V\right).\hfill \end{array}$$

(343)

The two quantities may be proven equal by the cyclic permutation property of the trace.

Another example that we are surveying concerns the manifold $\mathrm{Sp}\left(2n\right)$ endowed with its canonical metric again embedded in the ambient space $\mathbb{A}:={\mathbb{R}}^{n\times n}$ endowed with a Euclidean metric. In this setting, we have seen that $G_Q\left(V\right)=Q^{-\top }{Q}^{-1}V$. Its ‘bullet derivative’ reads

$$G_Q^{•}(V,W)={\left.\frac{\mathrm{d}}{\mathrm{d}t}\left({\gamma }^{-\top }{\gamma }^{-1}V\right)\right|}_{t=0},$$

(344)

where $\gamma :[-ϵ,ϵ]\to \mathrm{Sp}\left(2n\right)$ denotes any smooth curve such that $\gamma \left(0\right)=Q\in \mathrm{Sp}\left(2n\right)$ and $\dot{\gamma }\left(0\right)=W\in T_Q\mathrm{Sp}\left(2n\right)$.

By invoking the identity (104) once again, we obtain

$$\frac{\mathrm{d}}{\mathrm{d}t}\left({\gamma }^{-\top }{\gamma }^{-1}\right)={(-{\gamma }^{-1}\dot{\gamma }{\gamma }^{-1})}^{\top }{\gamma }^{-1}+{\gamma }^{-\top }(-{\gamma }^{-1}\dot{\gamma }{\gamma }^{-1}).$$

(345)

Setting $t=0$ leads to the expression

$$G_Q^{•}(V,W)=-Q^{-\top }(W^{\top }{Q}^{-\top }+Q^{-1}W)Q^{-1}V.$$

(346)

Let us verify the commutativity property with the inner product. We may calculate that

$$\begin{array}{cc}\hfill {\langle U,G_P^{•}(V,W)\rangle }^{\mathbb{A}}=& -\mathrm{tr}\left(U^{\top }{Q}^{-\top }(W^{\top }{Q}^{-\top }+Q^{-1}W)Q^{-1}V\right),\hfill \\ \hfill {\langle V,G_P^{•}(U,W)\rangle }^{\mathbb{A}}=& -\mathrm{tr}\left(V^{\top }{Q}^{-\top }(W^{\top }{Q}^{-\top }+Q^{-1}W)Q^{-1}U\right).\hfill \end{array}$$

(347)

These two quantities may be proven equal by transposing the arguments of the traces. ■

The condition of conformality and isometry may be written very compactly as $\dot{\mathcal{Q}}=0$. By multivariable calculus and the above definition and properties, we obtain:

$$\dot{\mathcal{Q}}={\langle \dot{\overline{u}},G_{\gamma }\left(\overline{w}\right)\rangle }^{\mathbb{A}}+{\langle \overline{u},G_{\gamma }^{•}(\overline{w},\dot{\gamma })\rangle }^{\mathbb{A}}+{\langle \overline{u},G_{\gamma }\left(\dot{\overline{w}}\right)\rangle }^{\mathbb{A}}.$$

(348)

(This is one situation in which we need to invoke the ‘extendability’ property of the metric kernel; in fact, notice that $\dot{\overline{w}}$ does not necessarily belong to $T_{\gamma }\mathbb{M}$.) In the second term on the right-hand side, the vector fields $\overline{u}$ and $\overline{w}$ may be swapped, and hence we may write:

$$\dot{\mathcal{Q}}={\langle \dot{\overline{u}},G_{\gamma }\left(\overline{w}\right)\rangle }^{\mathbb{A}}+{\langle \overline{w},G_{\gamma }^{•}(\overline{u},\dot{\gamma })\rangle }^{\mathbb{A}}+{\langle \overline{u},G_{\gamma }\left(\dot{\overline{w}}\right)\rangle }^{\mathbb{A}}.$$

(349)

Since the above development is still too general to lead to a computable notion of parallel transport, let us refer parallelism to the velocity field associated with the curve $\gamma$, namely, let us set $\overline{w}:=\dot{\gamma }$. This means that the parallel transport operation will keep fixed the angle between the transported vector and the velocity vector associated with the curve. The expression of $\dot{\mathcal{Q}}$ becomes then:

$$\dot{\mathcal{Q}}={\langle \dot{\overline{u}},G_{\gamma }\left(\dot{\gamma }\right)\rangle }^{\mathbb{A}}+{\langle \dot{\gamma },G_{\gamma }^{•}(\overline{u},\dot{\gamma })\rangle }^{\mathbb{A}}+{\langle \overline{u},G_{\gamma }\left(\ddot{\gamma }\right)\rangle }^{\mathbb{A}}.$$

(350)

In the above relation, all quantities are known except for the transport field $\overline{u}$ of which we are seeking a temporal evolution law.

A special case, of sure interest in applications since it dispenses us from choosing a curve $\gamma$, is parallel transport along a geodesic line. Let thus $\gamma$ denote a geodesic and let us recall that, in this case, the expression $G_{\gamma }\left(\ddot{\gamma }\right)$ may be rewritten in terms of the velocity $\dot{\gamma }$ through the differential inclusion that characterizes a geodesic line. Let us then revamp the geodesic equation with the aim of writing it in a Christoffel normal form. Recall that, given the fundamental form for a line $\mathcal{F}(\gamma ,\dot{\gamma })={\langle \dot{\gamma },G_{\gamma }\left(\dot{\gamma }\right)\rangle }^{\mathbb{A}}$, whenever it is a geodesic line it holds that:

$$\frac{\partial \mathcal{F}}{\partial \gamma }-\frac{\mathrm{d}}{\mathrm{d}t}\frac{\partial \mathcal{F}}{\partial \dot{\gamma }}\in N_{\gamma }\mathbb{M}.$$

(351)

According to the relationships in (339)

$$\frac{\partial \mathcal{F}}{\partial \gamma }=G_{\gamma }^{•}(\dot{\gamma },\dot{\gamma }),\frac{\partial \mathcal{F}}{\partial \dot{\gamma }}=2G_{\gamma }\left(\dot{\gamma }\right),$$

(352)

and then

$$\frac{\mathrm{d}}{\mathrm{d}t}\frac{\partial \mathcal{F}}{\partial \dot{\gamma }}=2G_{\gamma }^{•}(\dot{\gamma },\dot{\gamma })+2G_{\gamma }\left(\ddot{\gamma }\right),$$

(353)

therefore the differential inclusion (351) that defines a geodesic line becomes

$$-G_{\gamma }^{•}(\dot{\gamma },\dot{\gamma })-2G_{\gamma }\left(\ddot{\gamma }\right)=B_{\gamma }(\dot{\gamma },\dot{\gamma }),$$

(354)

where $B_x:{\left(T_x\mathbb{M}\right)}^2\to N_x\mathbb{M}$ is a bilinear function of its vector arguments and returns an element of the normal space $N_x\mathbb{M}$. Ultimately, we obtained

$$G_{\gamma }\left(\ddot{\gamma }\right)=-\frac{1}{2}{G}_{\gamma }^{•}(\dot{\gamma },\dot{\gamma })-\frac{1}{2}{B}_{\gamma }(\dot{\gamma },\dot{\gamma }).$$

(355)

This same expression may be rewritten compactly in one of the following normal Christoffel forms:

$$\left\{\begin{array}{c}{G}_{\gamma }\left(\ddot{\gamma }\right)+{\overline{\Gamma }}_{\gamma }(\dot{\gamma },\dot{\gamma })=0,or\hfill \\ \ddot{\gamma }+{\overline{\overline{\Gamma }}}_{\gamma }(\dot{\gamma },\dot{\gamma })=0,\hfill \end{array}\right.$$

(356)

where the function ${\overline{\Gamma }}_x$ is termed Christoffel form of the $1\mathrm{st}$ kind, while the quantity ${\overline{\overline{\Gamma }}}_x:=G_x^{-1}\circ {\overline{\Gamma }}_x$ is termed Christoffel form of the $2\mathrm{nd}$ kind. Notice that only the restricted Christoffel form ${\overline{\Gamma }}_x(v,v)$ with $x\in \mathbb{M}$ and $v\in T_x\mathbb{M}$ has been defined so far, namely

$${\overline{\Gamma }}_x(v,v):=\frac{1}{2}{G}_x^{•}(v,v)+\frac{1}{2}{B}_x(v,v).$$

(357)

Since both terms on the right-hand side are bilinear in their vector arguments, the Christoffel form is quadratic. It is now necessary to define the full Christoffel form ${\Gamma }_x(u,w)$, with $x\in \mathbb{M}$ and $u,w\in T_x\mathbb{M}$ on the basis of its restricted version. Such a result may be achieved through the following polarization formula:

$${\overline{\Gamma }}_x(u,w):=\frac{1}{4}\left\{{\overline{\Gamma }}_x(u+w,u+w)-{\overline{\Gamma }}_x(u-w,u-w)\right\},$$

(358)

which stems from the requirements that the full Christoffel form is bilinear and symmetric in its vector argument, namely ${\overline{\Gamma }}_x(u,w)={\overline{\Gamma }}_x(w,u)$.

Employing the normal Christoffel form in the latest expression of $\dot{\mathcal{Q}}$ gives:

$$\dot{\mathcal{Q}}={\langle \dot{\overline{u}},G_{\gamma }\left(\dot{\gamma }\right)\rangle }^{\mathbb{A}}+{\langle \dot{\gamma },G_{\gamma }^{•}(\overline{u},\dot{\gamma })\rangle }^{\mathbb{A}}-{\langle \overline{u},{\overline{\Gamma }}_{\gamma }(\dot{\gamma },\dot{\gamma })\rangle }^{\mathbb{A}}.$$

(359)

Since $\overline{\Gamma }$ is derived by polarization from a restricted bilinear function, it is self-adjoint in both its arguments; hence, it commutes with the inner product ${\langle ·,·\rangle }^{\mathbb{A}}$. Therefore, in the third term on the right hand side it is possible to swap the transport field $\overline{u}$ with any of the two fields $\dot{\gamma }$, which leads to:

$$\begin{array}{cc}\hfill \dot{\mathcal{Q}}=& {\langle \dot{\overline{u}},G_{\gamma }\left(\dot{\gamma }\right)\rangle }^{\mathbb{A}}+{\langle \dot{\gamma },G_{\gamma }^{•}(\overline{u},\dot{\gamma })\rangle }^{\mathbb{A}}-{\langle \dot{\gamma },{\overline{\Gamma }}_{\gamma }(\overline{u},\dot{\gamma })\rangle }^{\mathbb{A}}\hfill \\ \hfill =& {\langle \dot{\overline{u}},G_{\gamma }\left(\dot{\gamma }\right)\rangle }^{\mathbb{A}}+{\langle \dot{\gamma },{\overline{\Gamma }}_{\gamma }(\overline{u},\dot{\gamma })\rangle }^{\mathbb{A}}\hfill \end{array}$$

(360)

where we have used the fact that $\overline{\Gamma }$ and $\frac{1}{2}{G}^{•}$ differ by a purely normal component.

In order to make the above addenda uniform to one another, let us introduce in the above expression twice the identity $G_{\gamma }\circ G_{\gamma }^{-1}(={\mathrm{id}}_{\gamma })$ obtaining:

$$\dot{\mathcal{Q}}={\langle G_{\gamma }\left(\dot{\gamma }\right),\dot{\overline{u}}\rangle }^{\mathbb{A}}+{\langle G_{\gamma }\left(\dot{\gamma }\right),G_{\gamma }^{-1}\circ {\overline{\Gamma }}_{\gamma }(\overline{u},\dot{\gamma })\rangle }^{\mathbb{A}}.$$

Now it is possible to gather the terms in $G_{\gamma }\left(\dot{\gamma }\right)$ and to write the above sum as a single inner product:

$$\dot{\mathcal{Q}}={\langle G_{\gamma }\left(\dot{\gamma }\right),\dot{\overline{u}}+G_{\gamma }^{-1}\circ {\overline{\Gamma }}_{\gamma }(\overline{u},\dot{\gamma })\rangle }^{\mathbb{A}}=0.$$

(361)

Ultimately, the last relationship still expresses the basic property of parallel transport to realize a conformal isometry, which still leaves much room open for a precise definition of parallel transport. The condition $\dot{\mathcal{Q}}=0$ alone would give rise to infinitely many equations to compute a transport field, and hence parallel transport. Among these infinitely many, one arises simply by requiring the right-hand side of the above inner product to vanish to zero. Namely:

$$\dot{\overline{u}}+{\overline{\overline{\Gamma }}}_{\gamma }(\overline{u},\dot{\gamma })=0.$$

(362)

The solution of such equation supplies an expression for the transport field which, in turn, supplies a notion of parallel transport along a geodesic line, namely

$${\mathbb{P}}^{x\to y}\left(u\right):=\overline{u}\left(1\right),$$

(363)

where $\overline{u}\left(t\right)$ denotes the solution of the differential Equation (363) with the initial condition $\overline{u}\left(0\right)=u$. Such a differential equation in $\mathbb{A}$ is first-order and linear in the unknown transport field. Since the set of solutions to a linear first-order differential equation is a linear space, parallel transport is a linear isomorphism of the tangent bundle $T\mathbb{M}$.

Example 34.

As anticipated, a noteworthy property of any geodesic line is that it self-transports its own velocity along. In other words, given a geodesic ${\gamma }_x^y:[0,1]\to \mathbb{M}$ that connects two points $x,y\in \mathbb{M}$, the following function turns out to realize parallel transport:

$${\mathbb{P}}_{{\gamma }_x^y}^{0\to t}\left({\dot{\gamma }}_x^y\left(0\right)\right):={\dot{\gamma }}_x^y\left(t\right).$$

(364)

Let us verify that it meets the fundamental properties any parallel transport should meet:

• Tangency: for every $t\in [0,1]$ it holds that ${\mathbb{P}}_{{\gamma }_x^y}^{0\to t}\left({\dot{\gamma }}_x^y\left(0\right)\right)\in T_{{\gamma }_x^y\left(t\right)}\mathbb{M}$; in fact, ${\mathbb{P}}_{{\gamma }_x^y}^{0\to t}\left({\dot{\gamma }}_x^y\left(0\right)\right)={\dot{\gamma }}_x^y\left(t\right)$ is tangent to the curve at every point ${\gamma }_x^y\left(t\right)$.
• Conformal isometry: for every $t\in [0,1]$ it holds that ${\langle {\mathbb{P}}_{{\gamma }_x^y}^{0\to t}\left({\dot{\gamma }}_x^y\left(0\right)\right),{\mathbb{P}}_{{\gamma }_x^y}^{0\to t}\left({\dot{\gamma }}_x^y\left(0\right)\right)\rangle }_{{\gamma }_x^y\left(t\right)}={\parallel {\dot{\gamma }}_x^y\left(t\right)\parallel }_{{\gamma }_x^y\left(t\right)}^2$ which is constant along a geodesic,

as claimed. ■

As a special case, it is worth considering a manifold $\mathbb{M}$ endowed with a uniform metric. In this instance, the metric kernel $G_x$ is independent of the point x, and therefore it holds that $G^{•}\equiv 0$ along every (geodesic) line. The transport field equation hence simplifies into

$$\dot{\overline{u}}=-\frac{1}{2}{G}^{-1}\circ B_{\gamma }(\overline{u},\dot{\gamma }).$$

(365)

In the ever simpler instance where the metric is Euclidean, the metric kernel coincides with the identity (namely $G_x\equiv {\mathrm{id}}_x$), and hence it turns out that

$$\dot{\overline{u}}=-\frac{1}{2}{B}_{\gamma }(\overline{u},\dot{\gamma }).$$

(366)

Recalling that the function B maps the double tangent bundle ${\left(T\mathbb{M}\right)}^2$ to the normal bundle $N\mathbb{M}$, it is quite straightforward to figure out a practical interpretation of the differential Equation (366). It basically prescribes that, when traveling from a point over the curve $\gamma$ to another infinitesimally close point, the transport field looses its normal component to stay tangent.

The following two examples aim at clarifying the above theoretical developments.

Example 35.

Let us apply the theoretical development just analyzed to determine a transport field equation on a hypersphere ${\mathbb{S}}^{n-1}\subset \mathbb{A}$ with $\mathbb{A}:={\mathbb{R}}^n$ endowed with the metric ${\langle u,w\rangle }_x:={\langle u,w\rangle }^{\mathbb{A}}=u^{\top }w$.

Let us start by determining the Christoffel form (of the first or second kind is unessential, in this case, because the metric kernel coincides with the identity). The easiest way to determine Christoffel forms is via a geodesic equation. The fundamental form, in this case, reads $\mathcal{F}:={\dot{\gamma }}^{\top }\dot{\gamma }$. From the fundamental form we find that

$$\frac{\partial \mathcal{F}}{\partial \gamma }=0and\frac{\partial \mathcal{F}}{\partial \dot{\gamma }}=2\dot{\gamma },$$

(367)

therefore

$$\frac{\mathrm{d}}{\mathrm{d}t}\frac{\partial \mathcal{F}}{\partial \dot{\gamma }}=2\ddot{\gamma }.$$

(368)

The geodesic equation then reads $\ddot{\gamma }=\lambda \gamma$; hence, $\lambda ={\ddot{\gamma }}^{\top }\gamma$. From the tangency of the velocity $\dot{\gamma }$, it follows that:

$${\dot{\gamma }}^{\top }\gamma =0\stackrel{\mathrm{d}/\mathrm{d}t}{\Rightarrow }{\ddot{\gamma }}^{\top }\gamma +{\dot{\gamma }}^{\top }\dot{\gamma }=0,$$

(369)

and henceforth ${\ddot{\gamma }}^{\top }\gamma =-{\parallel \dot{\gamma }\parallel }^2$. As a consequence, $\lambda =-\parallel \dot{\gamma }{\parallel }^2$ and:

$${\overline{\Gamma }}_x(u,w)={\overline{\overline{\Gamma }}}_x(u,w)=-\left(u^{\top }w\right)x.$$

(370)

We may now verify that ${\langle {\overline{\Gamma }}_x(u,w),v\rangle }^{\mathbb{A}}={\langle {\overline{\Gamma }}_x(v,w),u\rangle }^{\mathbb{A}}$. It holds that

$$\begin{array}{cc}\hfill {\langle {\overline{\Gamma }}_x(u,w),v\rangle }^{\mathbb{A}}=& -\left(u^{\top }w\right)\left(v^{\top }x\right)=0,\hfill \\ \hfill {\langle {\overline{\Gamma }}_x(v,w),u\rangle }^{\mathbb{A}}=& -\left(v^{\top }w\right)\left(u^{\top }x\right)=0,\hfill \end{array}$$

(371)

hence the assertion.

The equation of the transport field, in this case, reads as:

$$\dot{\overline{u}}+\left({\overline{u}}^{\top }\dot{\gamma }\right)\gamma =0.$$

(372)

Such differential equation may be solved and returns an expression for the transport field $\overline{u}$. The definition (363) returns then the expression for the parallel transport from the point $x\in {\mathbb{S}}^{n-1}$ of a vector $w\in T_x{\mathbb{S}}^{n-1}$ to the point $y\in {\mathbb{S}}^{n-1}$:

$${\mathbb{P}}^{x\to y}\left(w\right):=\left[I_n-\frac{(x+y)y^{\top }}{1+x^{\top }y}\right]w.$$

(373)

Let us verify some characteristic features of the above map:

• Linearity: The linearity of ${\mathbb{P}}^{x\to y}\left(w\right)$ with respect to w (but not with respect to x ad y) is quite apparent.
• Identity: Letting $y\equiv x$ leads to an identity map. In fact, it holds that ${\mathbb{P}}^{x\to x}\left(w\right)=\left[I_n-\frac{(x+x)x^{\top }}{1+x^{\top }x}\right]w=\left[I_n-\frac{2x{x}^{\top }}{2}\right]w=(I_n-x{x}^{\top })w=w-x\left(x^{\top }w\right)=w-0=w$.
• Tangency: It should hold that $y^{\top }{\mathbb{P}}^{x\to y}\left(w\right)=0$. In fact: $y^{\top }\left[I_n-\frac{(x+y)y^{\top }}{1+x^{\top }y}\right]w=y^{\top }w-\frac{y^{\top }(x+y)y^{\top }w}{1+x^{\top }y}=y^{\top }w-\frac{(y^{\top }x+1)y^{\top }w}{1+x^{\top }y}=y^{\top }w-y^{\top }w=0$.

Important properties of the map (373) have been hence verified. ■

Example 36.

Let us consider the case of the manifold ${\mathbb{S}}^{+}\left(n\right)\subset \mathbb{A}$ endowed with the metric ${\langle U,W\rangle }_P:=\mathrm{tr}\left(U{P}^{-1}W{P}^{-1}\right)$ embedded in the ambient space $\mathbb{A}:={\mathbb{R}}^{n\times n}$ endowed with the metric ${\langle U,V\rangle }^{\mathbb{A}}:=\mathrm{tr}\left(UV\right)$. Recall from Section 7 that the corresponding metric kernel takes the expression $G_P\left(W\right):=P^{-1}W{P}^{-1}$ (which, as a function of W, is well-defined in the whole ambient space $\mathbb{A}$).

Let us determine the Christoffel form of the second kind through calculations related to geodesy. Define the fundamental form on a curve as $\mathcal{F}(\gamma ,\dot{\gamma }):=\mathrm{tr}\left(\dot{\gamma }{\gamma }^{-1}\dot{\gamma }{\gamma }^{-1}\right)$ and observe that:

$$\begin{array}{ccc}\hfill \frac{\partial \mathcal{F}}{\partial \dot{\gamma }}& =& 2{\gamma }^{-1}\dot{\gamma }{\gamma }^{-1}[\equiv 2G_{\gamma }\left(\dot{\gamma }\right)],\hfill \end{array}$$

(374)

$$\begin{array}{ccc}\hfill \frac{\partial \mathcal{F}}{\partial \gamma }& =& -2{\gamma }^{-1}\dot{\gamma }{\gamma }^{-1}\dot{\gamma }{\gamma }^{-1}[\equiv G_{\gamma }^{•}(\dot{\gamma },\dot{\gamma })],\hfill \end{array}$$

(375)

from which it follows that:

$$\frac{\mathrm{d}}{\mathrm{d}t}\frac{\partial \mathcal{F}}{\partial \dot{\gamma }}=2\frac{\mathrm{d}}{\mathrm{d}t}\left({\gamma }^{-1}\right)\dot{\gamma }{\gamma }^{-1}+2{\gamma }^{-1}\ddot{\gamma }{\gamma }^{-1}+2{\gamma }^{-1}\dot{\gamma }\frac{\mathrm{d}}{\mathrm{d}t}\left({\gamma }^{-1}\right).$$

(376)

Recalling the matrix identity

$$\frac{\mathrm{d}}{\mathrm{d}t}\left({\gamma }^{-1}\right)=-{\gamma }^{-1}\dot{\gamma }{\gamma }^{-1},$$

leads to

$$\frac{\mathrm{d}}{\mathrm{d}t}\frac{\partial \mathcal{F}}{\partial \dot{\gamma }}=-4{\gamma }^{-1}\dot{\gamma }{\gamma }^{-1}\dot{\gamma }{\gamma }^{-1}+2{\gamma }^{-1}\ddot{\gamma }{\gamma }^{-1},$$

(377)

which, in turn, implies that:

$$\begin{array}{cc}\hfill \frac{\partial \mathcal{F}}{\partial \gamma }-\frac{\mathrm{d}}{\mathrm{d}t}\frac{\partial \mathcal{F}}{\partial \dot{\gamma }}=& -2{\gamma }^{-1}\dot{\gamma }{\gamma }^{-1}\dot{\gamma }{\gamma }^{-1}+4{\gamma }^{-1}\dot{\gamma }{\gamma }^{-1}\dot{\gamma }{\gamma }^{-1}-2{\gamma }^{-1}\ddot{\gamma }{\gamma }^{-1},\hfill \\ \hfill =& 2{\gamma }^{-1}\dot{\gamma }{\gamma }^{-1}\dot{\gamma }{\gamma }^{-1}-2{\gamma }^{-1}\ddot{\gamma }{\gamma }^{-1},\hfill \\ \hfill =& 2{\gamma }^{-1}(\dot{\gamma }{\gamma }^{-1}\dot{\gamma }-\ddot{\gamma }){\gamma }^{-1}.\hfill \end{array}$$

(378)

The transport equation, in normal Christoffel form, hence reads:

$$\ddot{\gamma }-\dot{\gamma }{\gamma }^{-1}\dot{\gamma }-\omega \gamma =0,$$

(379)

where ω denotes a skew-symmetric matrix function (namely ${\omega }^{\top }=-\omega$) such that the product $\omega \gamma$ belongs to the normal space at γ.

In order to determine a function ω, let us invoke the condition of tangency of the velocity field $\dot{\gamma }$ that, in this case, is expressed as ${\dot{\gamma }}^{\top }=\dot{\gamma }$. From such a condition, it follows that ${\ddot{\gamma }}^{\top }=\ddot{\gamma }$; hence:

$${(\ddot{\gamma }-\dot{\gamma }{\gamma }^{-1}\dot{\gamma }-\omega \gamma )}^{\top }=\ddot{\gamma }-\dot{\gamma }{\gamma }^{-1}\dot{\gamma }-\gamma {\omega }^{\top }=0,$$

(380)

from which it follows that $\omega =0$. The restricted Christoffel form of the second kind, in this case, hence reads as

$${\overline{\overline{\Gamma }}}_P(V,V):=-V{P}^{-1}V.$$

(381)

To determine the complete Christoffel form of the second kind, let us make use of the polarization formula (358):

$$\begin{array}{cc}\hfill {\overline{\overline{\Gamma }}}_P(V,W)& :=\frac{1}{4}\left\{{\overline{\overline{\Gamma }}}_P(V+W,V+W)+{\overline{\overline{\Gamma }}}_P(V-W,V-W)\right\}\hfill \\ \hfill & =\frac{1}{4}\left\{-(V+W)P^{-1}(V+W)+(V-W)P^{-1}(V-W)\right\}\hfill \\ \hfill & =-\frac{1}{2}V{P}^{-1}W-\frac{1}{2}W{P}^{-1}V.\hfill \end{array}$$

(382)

Let us now verify that the form $\overline{\Gamma }$ commutes with the ambient inner product. We have $\overline{\Gamma }=G\circ \overline{\overline{\Gamma }}$; therefore,

$${\overline{\Gamma }}_P(V,W)=-\frac{1}{2}{P}^{-1}(V{P}^{-1}W+W{P}^{-1}V)P^{-1}.$$

(383)

As a consequence, we have

$$\begin{array}{cc}\hfill {\langle {\overline{\Gamma }}_P(V,W),U\rangle }^{\mathbb{A}}=& -\frac{1}{2}\mathrm{tr}(U{P}^{-1}V{P}^{-1}W{P}^{-1}+U{P}^{-1}W{P}^{-1}V{P}^{-1}),\hfill \\ \hfill {\langle {\overline{\Gamma }}_P(U,W),V\rangle }^{\mathbb{A}}=& -\frac{1}{2}\mathrm{tr}(V{P}^{-1}U{P}^{-1}W{P}^{-1}+V{P}^{-1}W{P}^{-1}U{P}^{-1}),\hfill \end{array}$$

(384)

which may be easily proven equal by the cyclic commutation property of matrix trace.

As a consequence, and employing the relation (342), the parallel transport equation is found to be:

$$\dot{\overline{U}}+\gamma (-{\gamma }^{-1}\dot{\gamma }{\gamma }^{-1}\overline{U}{\gamma }^{-1}-{\gamma }^{-1}\overline{U}{\gamma }^{-1}\dot{\gamma }{\gamma }^{-1})\gamma -\left(-\frac{1}{2}\dot{\gamma }{\gamma }^{-1}\overline{U}-\frac{1}{2}\overline{U}{\gamma }^{-1}\dot{\gamma }\right)=0,$$

(385)

or, after a few straightforward simplifications,

$$\dot{\overline{U}}-\frac{1}{2}\left(\dot{\gamma }{\gamma }^{-1}\overline{U}+\overline{U}{\gamma }^{-1}\dot{\gamma }\right)=0.$$

(386)

Through the definition (363), we may obtain the expression of the parallel transport from a point $P\in {\mathbb{S}}^{+}\left(n\right)$ of a tangent vector $W\in T_P{\mathbb{S}}^{+}\left(n\right)$ to a point $Q\in {\mathbb{S}}^{+}\left(n\right)$:

$${\mathbb{P}}^{P\to Q}\left(W\right):=\sqrt{Q{P}^{-1}}W\sqrt{P^{-1}Q},$$

(387)

where the matrix square root $\sqrt{·}$ returns a symmetric matrix. Let us verify two properties of the found operator:

• Identity: Setting $Q\equiv P$ leads to parallel transport collapse to an identity map. Indeed, it holds that ${\mathbb{P}}^{P\to P}\left(W\right):=\sqrt{P{P}^{-1}}W\sqrt{P^{-1}P}=W$.
• Tangency: It holds that ${\left({\mathbb{P}}^{P\to Q}\left(W\right)\right)}^{\top }={\mathbb{P}}^{P\to Q}\left(W\right)$. In fact, we have ${\left({\mathbb{P}}^{P\to Q}\left(W\right)\right)}^{\top }={\left(P^{-1}Q\right)}^{\frac{\top }{2}}{W}^{\top }{\left(Q{P}^{-1}\right)}^{\frac{\top }{2}}={\left(Q^{\top }{P}^{-\top }\right)}^{\frac{1}{2}}W{\left(P^{-\top }{Q}^{\top }\right)}^{\frac{1}{2}}=\sqrt{Q{P}^{-1}}W\sqrt{P^{-1}Q}={\mathbb{P}}^{P\to Q}\left(W\right)$.

Linearity is apparent as well. ■

The calculations illustrated in the previous example requires the evaluation of the partial derivatives of the fundamental form $\mathcal{F}$, determined in (374) and (375). It is quite instructive to sketch such calculations.

Example 37.

Let us survey in detail the calculations that led to relations (374) and (375). Such calculations may be carried out through calculus (exact way) or by analytic approximations (which is more clear from a computational viewpoint).

To begin with, it is useful to analytically justify a matrix approximation, namely

$${(P+E)}^{-1}\approx P^{-1}-P^{-1}E{P}^{-1},$$

(388)

with $P\in {\mathbb{S}}^{+}\left(n\right)$ and $E\in {\mathbb{R}}^{n\times n}$ with $\parallel E\parallel$ small. Such approximation may be proven through the notion ofanalytic matrix function. By definition, a matrix-to-matrix function $f:{\mathbb{R}}^{n\times n}\to {\mathbb{R}}^{n\times n}$ is analytic in a point $X_0\in {\mathbb{R}}^{n\times n}$ if it admits, in a neighborhood of such a point, a polynomial series expansion, namely:

$$f\left(X\right)=\sum _{n=0}^{\infty }{a}_n{(X-X_0)}^n,for\parallel X-X_0\parallel <\rho$$

(389)

where $a_n\in \mathbb{R}$ are coefficients of the series and $\rho >0$ denotes the radius of convergence of the polynomial series.

Let us now consider the function $f\left(X\right):={(I_n+X)}^{-1}$. It is analytic in a neighborhood of the point $X=0_n$ and the associated polynomial series, truncated to the second term, reads:

$${(I_n+X)}^{-1}\approx I_n-X,$$

(390)

from which it readily follows that

$$\begin{array}{cc}\hfill {(P+E)}^{-1}=& {\left(P(I_n+P^{-1}E)\right)}^{-1}\hfill \\ \hfill =& {(I_n+P^{-1}E)}^{-1}{P}^{-1}\hfill \\ \hfill \approx & (I_n-P^{-1}E)P^{-1}\hfill \\ \hfill =& P^{-1}-P^{-1}E{P}^{-1}.\hfill \end{array}$$

(391)

To justify the partial derivatives (374) and (375), let us define $\mathcal{F}(P,V):=\mathrm{tr}\left(V{P}^{-1}V{P}^{-1}\right)$ and evaluate its partial derivatives with respect to P and V. The partial derivative of a matrix-to-scalar function may be thought of as the ‘coefficient’ of the linear term arising from an additive perturbation.

Applying such a practical idea to evaluate the partial derivative with respect to the first argument gives:

$$\mathcal{F}(P+E,V)=\mathcal{F}(P,V)+{〈\frac{\partial \mathcal{F}}{\partial P},E〉}^{\mathbb{A}}+\mathit{higher}\mathit{order}\mathit{terms},$$

(392)

where E denotes a matrix (termedperturbation) whose entries are small numbers (such that $\parallel E\parallel$ is very small). In the present case, we have:

$$\begin{array}{cc}\hfill \mathcal{F}(P+E,V)=& \mathrm{tr}\left(V{(P+E)}^{-1}V{(P+E)}^{-1}\right),\hfill \\ \hfill \approx & \mathrm{tr}\left(V(P^{-1}-P^{-1}E{P}^{-1})V(P^{-1}-P^{-1}E{P}^{-1})\right)\hfill \\ \hfill \approx & \mathrm{tr}\left(V{P}^{-1}V{P}^{-1}\right)-\mathrm{tr}\left(V{P}^{-1}E{P}^{-1}V{P}^{-1}\right)-\mathrm{tr}\left(V{P}^{-1}V{P}^{-1}E{P}^{-1}\right)\hfill \\ \hfill & +\mathit{higher}\mathit{order}\mathit{terms}\hfill \\ \hfill =& \mathcal{F}(P,V)-\mathrm{tr}\left(P^{-1}V{P}^{-1}V{P}^{-1}E\right)-\mathrm{tr}\left(P^{-1}V{P}^{-1}V{P}^{-1}E\right)\hfill \\ \hfill & +\mathit{higher}\mathit{order}\mathit{terms}\hfill \\ \hfill =& \mathcal{F}(P,V)-2\mathrm{tr}\left(P^{-1}V{P}^{-1}V{P}^{-1}E\right)+higherorderterms.\hfill \end{array}$$

(393)

Comparing this expression with (392), one obtains the sought result, namely:

$$\frac{\partial \mathcal{F}}{\partial P}=-2P^{-1}V{P}^{-1}V{P}^{-1}.$$

(394)

To what concerns the second argument, we may write that:

$$\mathcal{F}(P,V+E)=\mathcal{F}(P,V)+{〈\frac{\partial \mathcal{F}}{\partial V},E〉}^{\mathbb{A}}+\mathit{higher}\mathit{order}\mathit{terms},$$

(395)

where E again denotes a perturbation. In this case, we have:

$$\begin{array}{cc}\hfill \mathcal{F}(P,V+E)=& \mathrm{tr}\left((V+E)P^{-1}(V+E)P^{-1}\right),\hfill \\ \hfill =& \mathrm{tr}\left(V{P}^{-1}V{P}^{-1}\right)+\mathrm{tr}\left(E{P}^{-1}V{P}^{-1}\right)+\mathrm{tr}\left(V{P}^{-1}E{P}^{-1}\right)\hfill \\ \hfill & +\mathit{higher}\mathit{order}\mathit{terms}\hfill \\ \hfill =& \mathcal{F}(P,V)+\mathrm{tr}\left(P^{-1}V{P}^{-1}E\right)+\mathrm{tr}\left(P^{-1}V{P}^{-1}E\right)\hfill \\ \hfill & +\mathit{higher}\mathit{order}\mathit{terms}\hfill \\ \hfill =& \mathcal{F}(P,V)+2\mathrm{tr}\left(P^{-1}V{P}^{-1}E\right)+\mathit{higher}\mathit{order}\mathit{terms}.\hfill \end{array}$$

(396)

Comparing such an expression with (395) yields:

$$\frac{\partial \mathcal{F}}{\partial V}=2P^{-1}V{P}^{-1},$$

(397)

which is the sought result. ■

Let us summarizes a few known formulas about parallel transport. All the formulas given below hold under the provision that the manifolds are endowed with their canonical metrics.

• Hypercube: The hypercube endowed with the standard Euclidean metric is a flat space; hence, parallel transport may be realized as a rigid translation. Namely parallel transport is an identity map.
• Hypersphere: Parallel transport on the hypersphere ${\mathbb{S}}^{p-1}$ of the tangent vector $w\in T_x{\mathbb{S}}^{p-1}$ along the geodesic arc ${\gamma }_{x,v}$ of an extent t may be computed by [39]:

  $${\mathbb{P}}_{{\gamma }_{x,v}}^{0\to t}\left(w\right)=\left[I_p+\frac{cos(\parallel v\parallel t)-1}{{\parallel v\parallel }^2}v{v}^{\top }-sin(\parallel v\parallel t)x{v}^{\top }\right]w.$$

  (398)
  For a reference, readers might want to consult [61]. Starting from this, some mathematical work leads to the following result: given two points $x,y\in {\mathbb{S}}^{p-1}$ and a vector $w\in T_x{\mathbb{S}}^{n-1}$, the parallel transport operator ${\mathbb{P}}^{x\to y}:T_x{\mathbb{S}}^{p-1}\to T_y{\mathbb{S}}^{p-1}$ has the following structure:

  $${\mathbb{P}}^{x\to y}\left(w\right)=\left[I_p-\frac{(x+y)y^{\top }}{1+x^{\top }y}\right]w,$$

  (399)

  provided that $x^{\top }y\ne -1$ (namely, the points x and y are not antipodal one to another).
• Special orthogonal group: Parallel transport along a geodesic curve on the special orthogonal group $\mathbb{SO}\left(p\right)$ may be implemented through the following formula:

  $${\mathbb{P}}^{X\to Y}\left(W\right)=X\sqrt{X^{\top }Y}{X}^{\top }W\sqrt{X^{\top }Y}.$$

  (400)
• Stiefel manifold: To the best of this author’s knowledge, there appear to be no closed formulas about parallel transport on a Stiefel manifold $\mathrm{St}(n,p)$. This is one of those cases in which self-parallel transport might be invoked. A self-parallel transport expression, corresponding to the canonical metric, is

  $${\mathbb{P}}_{{\gamma }_{X,V}}^{0\to t}\left(V\right)=[V-X{R}^{\top }]\mathrm{Exp}\left(t\left[\begin{array}{cc}{X}^{\top }V& -R^{\top }\\ R& 0_p\end{array}\right]\right)\left[\begin{array}{c}{I}_p\\ 0_p\end{array}\right],$$

  (401)

  where R denotes the R-factor of the thin QR factorization of the matrix $(I_n-X{X}^{\top })V$.
• Manifold of symmetric, positive-definite (SPD) matrices: Given a geodesic arc ${\gamma }_{X,V}:[0,1]\to {\mathbb{S}}^{+}\left(p\right)$ and a tangent vector $W\in T_X{\mathbb{S}}^{+}\left(p\right)$, the expression of the parallel transport operator that shifts the tangent vector W along the geodesic arc ${\gamma }_{X,V}\left(t\right)$ toward the endpoint $t=1$ reads:

  $${\mathbb{P}}_{{\gamma }_{X,V}}^{0\to 1}\left(W\right)=\mathrm{Exp}\left(\frac{V{X}^{-1}}{2}\right)W\mathrm{Exp}\left(\frac{X^{-1}V}{2}\right).$$

  (402)
  Some mathematical work leads to the following result: given two points $X,Y\in {\mathbb{S}}^{+}\left(p\right)$ and a tangent vector $W\in T_X{\mathbb{S}}^{+}\left(p\right)$, the parallel transport operator ${\mathbb{P}}^{X\to Y}:T_X{\mathbb{S}}^{+}\left(p\right)\to T_Y{\mathbb{S}}^{+}\left(p\right)$ has the following structure:

  $${\mathbb{P}}^{X\to Y}\left(W\right)=\sqrt{Y{X}^{-1}}W\sqrt{X^{-1}Y}.$$

  (403)
• Grassmann manifold: Parallel transport of a vector $U\in T_{\left[X\right]}\mathrm{Gr}(n,p)$ along the geodesic ${\gamma }_{\left[X\right],V}$ with $V\in T_{\left[X\right]}\mathrm{Gr}(n,p)$ is given by [33]:

  $${\mathbb{P}}_{{\gamma }_{\left[X\right],V}}^{0\to t}\left(U\right)=\left(\left[XBA\right]\left[\begin{array}{c}-sin\left(Dt\right)\\ cos\left(Dt\right)\end{array}\right]A^{\top }+I_n-A{A}^{\top }\right)U,$$

  (404)

  where $AD{B}^{\top }$ denotes the compact singular value decomposition of the matrix V.

Let us derive the expression of the Christoffel form of the second kind for the special orthogonal group.

Example 38.

According to the canonical metric for the special orthogonal group $\mathbb{SO}\left(n\right)$, the fundamental form is $\mathcal{F}(R,V):=\mathrm{tr}\left(V^{\top }V\right)$; hence, the geodesic equation stemming from the differential inclusion reads as $\ddot{\gamma }=\gamma S$, where S is a symmetric function. Recalling that any geodesic line must satisfy the constraint ${\gamma }^{\top }\gamma =I_n$, deriving this twice with respect to time gives

$${\ddot{\gamma }}^{\top }\gamma +2{\dot{\gamma }}^{\top }\dot{\gamma }+{\gamma }^{\top }\ddot{\gamma }=0.$$

(405)

Plugging in the geodesic equation leads to $S=-{\dot{\gamma }}^{\top }\dot{\gamma }$, and hence the complete geodesic equation reads as $\ddot{\gamma }=-\gamma {\dot{\gamma }}^{\top }\dot{\gamma }$. Consequently, the restricted Christoffel form of the second kind associated with the special orthogonal group endowed with the canonical metric takes the expression

$${\overline{\overline{\Gamma }}}_R(V,V):=R\left(V^{\top }V\right).$$

(406)

The full Christoffel form descends from the polarization formula (358), which leads to

$${\overline{\overline{\Gamma }}}_R(V,W)=\frac{1}{2}R(V^{\top }W+W^{\top }V).$$

(407)

It is interesting to notice that ${\overline{\overline{\Gamma }}}_R(V,W)\in N_R\mathbb{SO}\left(n\right)$. ■

Let us derive the expression of the Christoffel form of the second kind for the Stiefel manifold.

Example 39.

As we have seen, the Stiefel manifold $\mathrm{St}(n,p)$ may be endowed both with a Euclidean metric and its canonical metric.

In the Euclidean metric, the fundamental form reads as $\mathcal{F}(X,V):=\mathrm{tr}\left(V^{\top }V\right)$; hence, the geodesic equation stemming from the differential inclusion reads as $\ddot{\gamma }=\gamma S$, where S is a symmetric function. Recalling that any geodesic must satisfy the constraint ${\gamma }^{\top }\gamma =I_p$, similarly to the calculations shown in the previous example, we find that

$${\overline{\overline{\Gamma }}}_X(V,V):=X\left(V^{\top }V\right).$$

(408)

The full Christoffel form descends from the polarization formula (358), which leads to

$${\overline{\overline{\Gamma }}}_X(V,W)=\frac{1}{2}X(V^{\top }W+W^{\top }V).$$

(409)

In the canonical metric, the fundamental form reads $\mathcal{F}(X,V):=\mathrm{tr}\left(V^{\top }\left(I_p-\frac{1}{2}X{X}^{\top }\right)V\right)$. Calculations of the geodesic equation show that

$$\begin{array}{cc}\hfill \frac{\partial \mathcal{F}}{\partial \dot{\gamma }}=& 2(I_p-\frac{1}{2}\gamma {\gamma }^{\top })\dot{\gamma },\hfill \\ \hfill \frac{\partial \mathcal{F}}{\partial \gamma }=& -\dot{\gamma }{\dot{\gamma }}^{\top }\gamma .\hfill \end{array}$$

(410)

From the first derivative, it follows that

$$\frac{\mathrm{d}}{\mathrm{d}t}\frac{\partial \mathcal{F}}{\partial \dot{\gamma }}=2(I_p-\frac{1}{2}\gamma {\gamma }^{\top })\ddot{\gamma }-(\dot{\gamma }{\gamma }^{\top }+\gamma {\dot{\gamma }}^{\top })\dot{\gamma }.$$

(411)

From the differential inclusion (153), it hence follows the equation

$$2(I_p-\frac{1}{2}\gamma {\gamma }^{\top })\ddot{\gamma }=-\dot{\gamma }{\dot{\gamma }}^{\top }\gamma +\dot{\gamma }{\gamma }^{\top }\dot{\gamma }+\gamma {\dot{\gamma }}^{\top }\dot{\gamma }-\gamma S,$$

(412)

where S is a symmetric matrix function to be determined. To solve for the naive derivative $\ddot{\gamma }$, let us recall that ${(I_p-\frac{1}{2}\gamma {\gamma }^{\top })}^{-1}=I_p+\gamma {\gamma }^{\top }$. Hence, the above equation may be rewritten as

$$\ddot{\gamma }=\dot{\gamma }{\gamma }^{\top }\dot{\gamma }+\gamma {\left({\gamma }^{\top }\dot{\gamma }\right)}^2-\gamma {\dot{\gamma }}^{\top }\dot{\gamma }-\gamma S,$$

(413)

where we have repeatedly used the fact that ${\dot{\gamma }}^{\top }\gamma =-{\gamma }^{\top }\dot{\gamma }$. Plugging the above relationship into the condition (405) leads to the expression $S=2{\left({\gamma }^{\top }\dot{\gamma }\right)}^2$. Plugging back such an expression into the relationship (413) leads to the geodesic equation

$$\ddot{\gamma }+\dot{\gamma }{\gamma }^{\top }\dot{\gamma }+\gamma ({\left({\gamma }^{\top }\dot{\gamma }\right)}^2+{\dot{\gamma }}^{\top }\dot{\gamma })=0,$$

(414)

from which stems the restricted Christoffel form of the second kind

$${\overline{\overline{\Gamma }}}_X(V,V)=V{V}^{\top }X+X{V}^{\top }(I_n-X{X}^{\top })V.$$

(415)

Calculations to obtain the full Christoffel form of the second kind lead to the expression

$${\overline{\overline{\Gamma }}}_X(V,W)=\frac{1}{2}\left\{(V{W}^{\top }+W{V}^{\top })X+X{W}^{\top }(I_n-X{X}^{\top })V+X{V}^{\top }(I_n-X{X}^{\top })W\right\}$$

(416)

through the polarization formula. ■

To conclude this subsection, let us prove an interesting result that concerns the parallel transport of a manifold logarithm, which will be invoked later in connection to non-linear control.

Theorem 6.

Given two points $x,y\in \mathbb{M}$, provided the manifold logarithms ${log}_{x}y$ and ${log}_{y}x$ exist, it holds that

$${\mathbb{P}}^{x\to y}\left({log}_{x}y\right)=-{log}_{y}x.$$

(417)

Proof. 

Let ${\gamma }_x^y:[0,1]\to \mathbb{M}$ denote the geodesic arc connecting x to y and let ${\tilde{\gamma }}_y^x\left(t\right):={\gamma }_x^y(1-t)$ denote the associated reversed geodesic. By definition of the logarithmic map, it holds that ${\dot{\gamma }}_x^y\left(0\right)={log}_{x}y$. Since ${\tilde{\gamma }}_y^x$ is a geodesic, it also holds that ${\dot{\tilde{\gamma }}}_y^x\left(0\right)={log}_{y}x$. In addition, by the definition of a reversed geodesic, it holds that ${\dot{\tilde{\gamma }}}_y^x\left(0\right)=-{\dot{\gamma }}_x^y\left(1\right)$. By the self-parallel transport property of any geodesic, it also follows that ${\dot{\gamma }}_x^y\left(1\right)={\mathbb{P}}^{x\to y}\left({\dot{\gamma }}_x^y\left(0\right)\right)$. The assertion is hence proven. □

10.3. Coordinate-Prone Derivation of Parallel Transport*

The basic requirement of parallel transport along a curve is that it ensures the transported vector to be tangent to the curve at any given point. Clearly, this condition by itself is too weak to give rise to a unique solution to the problem of parallel transport. In Riemannian manifold calculus, it is additionally required that parallel transport along a geodesic preserves the inner product between the transported tangent vector and the tangent to the geodesic curve.

Under such a proviso, a parallel-transport rule may be constructed as follows. Let $\gamma :[0,1]\to \mathbb{M}$ denote a geodesic curve on a p-dimensional Riemannian manifold $\mathbb{M}$ endowed with a metric tensor of components $g_{ij}$. Upon taking a vector $v\in T_{\gamma \left(0\right)}\mathbb{M}$, define a vector field $v^{\gamma }\in \Gamma \left(\mathbb{M}\right)$ (which might be well-defined only along the curve $\gamma$ and not necessarily outside of it) that denotes the transport of v along the curve $\gamma$. The tangency requirement is expressed by $v^{\gamma }\left(t\right)\in T_{\gamma \left(t\right)}\mathbb{M}$, for every $t\in [0,1]$. The inner-product preservation property of parallel transport is expressed by requiring that the sought vector field $v^{\gamma }$ meets the formal condition:

$${\langle v^{\gamma }\left(t\right),\dot{\gamma }\left(t\right)\rangle }_{\gamma \left(t\right)}=\mathrm{constant}\mathrm{over}\mathrm{the}\mathrm{curve}\gamma \left(t\right).$$

(418)

Using local coordinates $v^{\gamma }=(v^1,v^2,\dots ,v^p)$ and $\gamma =(x^1,x^2,\dots ,x^p)$ and differentiating the above equation with respect to the parameter t yields the condition:

$$\frac{\mathrm{d}}{\mathrm{d}t}\left(g_{ij}{v}^i{\dot{x}}^j\right)=0,$$

(419)

namely:

$$\frac{\partial g_{ij}}{\partial x^k}{\dot{x}}^k{\dot{x}}^j{v}^i+g_{ij}{\dot{x}}^j{\dot{v}}^i+g_{ih}{\ddot{x}}^h{v}^i=0.$$

(420)

Now, let us exploit the fact that the components $x^h$ describe a geodesic arc (namely, a self-parallel curve) on the Riemannian manifold $\mathbb{M}$. Accordingly, it holds that ${\ddot{x}}^h=-{\Gamma }_{kj}^h{\dot{x}}^k{\dot{x}}^j$. Making use of such identity in Equation (420) yields:

$$\left(\left(\frac{\partial g_{ij}}{\partial x^k}-g_{ih}{\Gamma }_{kj}^h\right){\dot{x}}^k{v}^i+g_{ij}{\dot{v}}^i\right){\dot{x}}^j=0.$$

(421)

Calculations with the Christoffel symbols show that the above equation may be rewritten as:

$$g_{hj}({\Gamma }_{ik}^h{v}^i{\dot{x}}^k+{\dot{v}}^h){\dot{x}}^j=0.$$

(422)

from such relation, one may derive the set of differential equations

$${\dot{v}}^h=-{\Gamma }_{ik}^h{v}^i{\dot{x}}^k.$$

(423)

which describe the evolution of the components of the transport field. Solving the above set of non-linear equations under appropriate initial conditions yields a rule to perform parallel transport.

11. Manifold Retraction and Vector Transport

Since the computation of an exponential map on a given manifold may be cumbersome, the notion of retraction is sometimes invoked. A manifold retraction map $R:T\mathbb{M}\to \mathbb{M}$ is a function that satisfies the following requirements [62] and that is easier to compute than an exponential map:

• Any restriction $R_x$ is defined in some open ball $B(0,{\rho }_x)$ of radius ${\rho }_x>0$ about $0\in T_x\mathbb{M}$ and is continuously differentiable;
• It holds that $R_x\left(v\right)=x$ if $v=0$;
• Let $v\left(t\right)\in B(0,{\rho }_x)\subset T_x\mathbb{M}$ denote any smooth function on the tangent space $T_x\mathbb{M}$, with $t\in [-ϵ,ϵ]$ and $v\left(0\right)=0$. The curve $y\left(t\right)=R_x\left(v\left(t\right)\right)$, for $t\in [-ϵ,ϵ]$, lies in a neighborhood of $x=y\left(0\right)$. It holds that $\dot{y}=\mathrm{d}{R}_x{|}_v\left(\dot{v}\right)$ where $\mathrm{d}{R}_x{|}_v:T_v{T}_x\mathbb{M}\to T_{R_x\left(v\right)}\mathbb{M}$ denotes a tangent map. Notice that $v\left(t\right)$ lies in $T_x\mathbb{M}$ for every t; hence, the derivative $\dot{v}$ is defined simply as

  $$\dot{v}\left(t\right):=\underset{h\to 0}{lim}\frac{v(t+h)-v\left(t\right)}{h}.$$

  (424)
  For $t=0$, it holds that $\mathrm{d}{R}_x{|}_0:T_0{T}_x\mathbb{M}\to T_{R_x\left(0\right)}\mathbb{M}$. Let us identify $T_0{T}_x\mathbb{M}\cong T_x\mathbb{M}$ and let us recall that $T_{R_x\left(0\right)}\mathbb{M}=T_x\mathbb{M}$. In order for $R_x$ to be a retraction, the map $\mathrm{d}{R}_x{|}_0$ must equate the identity map in $T_x\mathbb{M}$.

In practice, a retraction $R_x\left(v\right)$ sends a tangent vector $v\in T_{x}M$ to a manifold $\mathbb{M}\ni x$ into a neighbor of the point x. Any exponential map of a Riemannian manifold is a retraction. Another class of retractions was surveyed in [63].

Manifold retraction is a computationally convenient replacement of exponential map and, according to the definition given above, it behaves as a manifold exponential up to first order. As manifold logarithm is a local inverse of manifold exponential, one might wonder if a local inverse of retraction exists. Such problems have been studied in a number of papers and a possible solution has been codified under the name of a manifold lifting map [64].

Let us examine a few formulas of interest about manifold retraction.

• Hypercube: Not surprisingly, the simplest manifold retraction on the hypercube is realized through array addition, namely $R_x\left(v\right):=x+v$.
• Hypersphere: Let $x\in {\mathbb{S}}^{p-1}$ and $v\in T_x{\mathbb{S}}^{p-1}$. A simple retraction map on the unit hypersphere is

  $$R_x\left(v\right):=\frac{x+v}{\parallel x+v\parallel },$$

  (425)

  where $\parallel ·\parallel$ denotes a 2-norm. Let us verify that this map meets the three mentioned requirements to be a retraction:

  -

  Requirements 1 and 2: Easily verified by inspection.

  -

  Requirement 3: Let $y\left(t\right)=\frac{x+v\left(t\right)}{{\left[{(x+v\left(t\right))}^{\top }(x+v\left(t\right))\right]}^{1/2}}$. Deriving both sides with respect to t yields:

  $$\dot{y}=\frac{\dot{v}\parallel x+v\parallel +(x+v)\frac{1}{2}\frac{\dot{v}(x+v)+{(x+v)}^{\top }\dot{v}}{\parallel x+v\parallel }}{{\parallel x+v\parallel }^2}=\frac{\dot{v}{\parallel x+v\parallel }^2+(x+v){\dot{v}}^{\top }(x+v)}{{\parallel x+v\parallel }^3}.$$

  Setting $v=0$ leads to

  $$\dot{y}\left(0\right)=\frac{\dot{v}\left(0\right){\parallel x\parallel }^2+x{\dot{v}}^{\top }x}{{\parallel x\parallel }^3}.$$

  Since $\parallel x\parallel =1$ and $\dot{v}\in T_x{\mathbb{S}}^{p-1}$, it follows that ${\dot{v}}^{\top }x=0$, and hence the result is proven.

• Stiefel manifold: There exist a number of retractions on the Stiefel manifold, which are briefly outlined below.
  • Retraction based on QR factorization: In [64], it was shown that one of the retractions that map a tangent vector of $T_X\mathrm{St}(n,p)$ to $\mathrm{St}(n,p)$ is given by:

    $$R_X^{\mathrm{qr}}\left(V\right):=\mathrm{qf}(X+V),$$

    (426)

    where $\mathrm{qf}(·)$ denotes the Q-factor of the thin QR factorization of its ${\mathbb{R}}^{p\times n}$ matrix argument.
  • Retraction based on polar factorization: Given a point $X\in \mathrm{St}(n,p)$ and a vector $V\in T_X\mathrm{St}(n,p)$, the polar-factorization-based retraction may be written as [64]:

    $$R_X^{\mathrm{pol}}\left(V\right):=\mathrm{pf}(X+V),$$

    (427)

    where $\mathrm{pf}(·)$ denotes the polar factor of a given matrix. Such a retraction may be written in closed form. In fact, write $X+V=QS$. From the conditions $Q^{\top }Q=I_p$ and $S^{\top }=S$, it follows that:

    $$\begin{array}{c}\hfill {(X+V)}^{\top }(X+V)=S^{\top }{Q}^{\top }QS\Rightarrow X^{\top }X+X^{\top }V+V^{\top }X+V^{\top }V=S^2.\end{array}$$
    Since it holds that $X^{\top }X=I_p$ and $X^{\top }V+V^{\top }X=0$, and the matrix $I_p+V^{\top }V$ is positive-definite, one obtains $S={(I_p+V^{\top }V)}^{\frac{1}{2}}$. From the equality $X+V=QS$, the following closed-form expression for the polar factorization-based retraction is obtained:

    $$R_X^{\mathrm{pol}}\left(V\right)=(X+V){(I_p+V^{\top }V)}^{-\frac{1}{2}}.$$

    (428)
  • Orthographic retraction map: The paper [65] studies orthographic retractions on submanifolds of Euclidean spaces. In the paper [65], it is proven that, given a pair $(X,V)\in T\mathrm{St}(n,p)$, under the proviso that V is sufficiently close to $0\in T_X\mathrm{St}(n,p)$, there exists a normal array $Z\in N_X\mathrm{St}(n,p)$ such that the function

    $$R_X^{\mathrm{orth}}\left(V\right):=X+V+Z$$

    (429)

    is a retraction on $\mathrm{St}(n,p)$. By the structure of the normal spaces, the orthographic retraction map on the Stiefel manifold $\mathrm{St}(n,p)$ reads as:

    $$R_X^{\mathrm{orth}}\left(V\right)=X+V+XS,$$

    (430)

    provided that there exists a $p\times p$ symmetric matrix S such that $V+X(I_p+S)\in \mathrm{St}(n,p)$, namely, such that:

    $${(X+V+XS)}^T(X+V+XS)=I_p.$$

    (431)
    The above equation in the unknown matrix S may be written in plain form as:

    $$-S^2-S(X^{T}V+I_p)-(V^{T}X+I_p)S-V^{T}V=0.$$

    (432)
    The Equation (432) represents an instance of Continuous-time Algebraic Riccati Equation (CARE). The orthographic retraction map (430) may be computed numerically as shown in [64].
• Real symplectic group: Possible retractions on a real symplectic group $\mathrm{Sp}\left(2n\right)$ are

  $$R_Q\left(V\right):=Q\mathrm{Exp}\left(Q^{-1}V\right),$$

  (433)

  whose properties were studied in the contributions [66,67], and the one based on the Cayley map, as explained in [2].
• Grassmann manifold: There exist a number of retractions on the Grassmann manifold, which are briefly outlined below.
  • Retraction based on QR-factorization: In [40], one of the retractions that map a tangent vector $V\in T_{\left[X\right]}\mathrm{Gr}(n,p)$ onto $\mathrm{Gr}(n,p)$ is given by:

    $$R_X^{\mathrm{qr}}\left(V\right):=\mathrm{qf}(X+V),$$

    (434)

    where $\mathrm{qf}(·)$ denotes again the Q-factor of the thin QR factorization of its ${\mathbb{R}}^{p\times n}$ matrix argument and $X\in \mathrm{St}(n,p)$ is a Stiefel representative of the subspace $\left[X\right]$.
  • Retraction based on polar factorization: Given a subspace $\left[X\right]\in \mathrm{Gr}(n,p)$ and a tangent vector $V\in T_{\left[X\right]}\mathrm{Gr}(n,p)$, the polar-factorization-based retraction may be written as [40]:

    $$R_X^{\mathrm{pol}}\left(V\right):=(X+V){(I_p+V^{\top }V)}^{-\frac{1}{2}},$$

    (435)

    where $X\in \mathrm{St}(n,p)$ is a Stiefel representative of the subspace $\left[X\right]$.

The parallel transport Equation (362) is, in general, difficult to solve, and hence the parallel transport operator might not be available in closed form for manifolds of interest. Approximations of the exact parallel transport are available, as the ‘Schild’s ladder’ construction [68] and the vector transport method [62]. In order to define the notion of vector transport, a smooth manifold $\mathbb{M}$ is supposed to be embedded into a (possibly large) linear ambient space $\mathbb{A}$ of appropriate dimension. Upon endowing the linear space $\mathbb{A}$ with an inner product, it is possible to define an orthogonal projection operator ${\Pi }_x:\mathbb{A}\to T_x\mathbb{M}$, with $x\in \mathbb{M}$. Define the Whitney sum

$$T\mathbb{M}\oplus T\mathbb{M}:=\{(x,(v,w))\mid x\in \mathbb{M},v\in T_x\mathbb{M},w\in T_x\mathbb{M}\}.$$

(436)

Then, the vector transport operator

$$\mathrm{vt}:(x,(v,u))\in T\mathbb{M}\oplus T\mathbb{M}\mapsto w\in T_{{exp}_x\left(v\right)}\mathbb{M}$$

(437)

associated to an exponential map $exp:T\mathbb{M}\to \mathbb{M}$ is defined as:

$${\mathrm{vt}}_{x,v}\left(u\right):={\Pi }_{{exp}_x\left(v\right)}\left(u\right).$$

(438)

In practice, instead of parallel-translating the tangent vector $u\in T_x\mathbb{M}$ along a geodesic arc emanating from the point $x\in \mathbb{M}$ in the direction $v\in T_x\mathbb{M}$ as ${\mathbb{P}}_{{\gamma }_{x,v}}^{0\to 1}\left(u\right)$, vector transport moves rigidly the vector u within $\mathbb{A}$ up to the point y and then orthogonally projects the vector u over the tangent space $T_y\mathbb{M}$ where $y={exp}_x\left(v\right)$.

With a slight abuse of exact terminology, one may refer to vector transport of a vector $u\in T_x\mathbb{M}$ to a tangent space $T_y\mathbb{M}$ by ${\Pi }_y\left(u\right)$ for a fixed $y\in \mathbb{M}$. In practice, vector transport is based on the following procedure:

• Embed the manifold $\mathbb{M}$ into a metric ambient space $\mathbb{A}$.
• Translate rigidly the vector $u\in T_x\mathbb{M}$ across the ambient space $\mathbb{A}\supset \mathbb{M}$ to a point $y\in \mathbb{M}$.
• Project the vector u into the tangent space $T_y\mathbb{M}$ by means of a suitable orthogonal projector ${\Pi }_y:\mathbb{A}\to T_y\mathbb{M}$.

Vector transport associated with the above procedure is ${\Pi }_y\left(u\right)$, which moves the vector u from $T_x\mathbb{M}\subset A$ to $T_y\mathbb{M}$.

Depending on the manifold structure, vector transport might result to be much less expensive to compute than exact parallel transport. As a drawback, vector transport does not enjoy some of the fundamental properties of parallel transport; for instance, vector transport does not realize an isometry nor a conformal transformation. Isometry may be recovered through an appropriate normalization, though.

12. Control Systems on Manifolds and Numerical Implementation

This section presents an instance of error feedback control of first-order systems on manifold and their numerical implementation, with special emphasis to system synchronization. In the present research, we will regard synchronization as a goal to be achieved by non-linear proportional-type control. We shall see that the design of a synchronizing controller may be effected through the classical notion of Lyapunov function and we shall further introduce the notion of control effort to quantify the energy consumption associated with a control action.

12.1. Synchronization of First-Order Dynamical Systems via Feedback Control

Synchronization is meant primarily to make two identical (or twin) dynamical systems synchronize their dynamics overt time, provided that their initial states differ from one another and that one of them is able to access the state of the other. In order not to restrict how far such two initial states may lay apart, the state space $\mathbb{M}$ is assumed to be a geodesically complete path-connected manifold. In principle, the restriction that the two systems to synchronize must be identical is not necessary, as long as their mathematical models insist on the same state manifold.

In a system pair, the independent dynamical system will be referred to as the leader, while the controlled dynamical system will be referred to as the follower. In this section, we shall assume that the leader and the follower differ from one another, whereby the case of identical systems will follow as a special (although most meaningful) case.

We shall cover in this paper only the case that the leader is represented by a first order, non-autonomous dynamical system on manifold, described by the tangent-bundle differential equation

$${\dot{x}}^{\ell }\left(t\right)=f^{\ell }(t,x^{\ell }\left(t\right)),\mathrm{with}{x}^{\ell }\left(0\right)=x_0^{\ell }\in \mathbb{M},t\ge 0,$$

(439)

where $x^{\ell }\in \mathbb{M}$ denotes the leader’s state variable and $f^{\ell }:\mathbb{R}\times \mathbb{M}\to T\mathbb{M}$ denotes a possibly time-dependent state-transition operator. Likewise, the controlled follower is described by

$$\dot{x}\left(t\right)=f(t,x\left(t\right))+u\left(t\right),\mathrm{with}x\left(0\right)=x_0\in \mathbb{M},t\ge 0,$$

(440)

where $x\in \mathbb{M}$ denotes the follower’s state variable, $f:\mathbb{R}\times \mathbb{M}\to T\mathbb{M}$ denotes a state-transition operator of the follower, $u\in T\mathbb{M}$ denotes a tangent-bundle control field (in particular, $u\left(t\right)\in T_{x\left(t\right)}\mathbb{M}$) and, in general, the initial state $x_0\in \mathbb{M}$ differs from the initial state $x_0^{\ell }\in \mathbb{M}$.

The control field that will drive the follower to synchronize to the leader is defined on the basis of an instantaneous (non-delayed) distance-minimization design. Assume the state manifold $\mathbb{M}$ to be endowed with a specific metric and hence a distance function $d:\mathbb{M}\times \mathbb{M}\to \mathbb{R}$ and a logarithmic map $log:\mathbb{M}\times \mathbb{M}\to T\mathbb{M}$. Let us define the function

$$\mathcal{D}\left(t\right):=\frac{1}{2}{d}^2(x^{\ell }\left(t\right),x\left(t\right))=\frac{1}{2}{\langle {log}_{x^{\ell }\left(t\right)}\left(x\left(t\right)\right),{log}_{x^{\ell }\left(t\right)}\left(x\left(t\right)\right)\rangle }_{x^{\ell }\left(t\right)},$$

(441)

which is proportional to the squared Riemannian distance between the state of the leader and the state of the follower. The following result shows how to define a distance minimizing control action.

Theorem 7.

The control field

$$u:={\mathbb{P}}^{x^{\ell }\to x}\left(f^{\ell }(t,x^{\ell })\right)-f(t,x)+c{log}_x\left(x^{\ell }\right),$$

(442)

with $c>0$, minimizes the function $\mathcal{D}\left(t\right)$ asymptotically.

Proof. 

The derivative of the function $\mathcal{D}$ with respect to the time-parameter t reads

$$\begin{array}{cc}\hfill \dot{\mathcal{D}}=& {\langle -{log}_{x^{\ell }}\left(x\right),{\dot{x}}^{\ell }\rangle }_{x^{\ell }}+{\langle -{log}_x\left(x^{\ell }\right),\dot{x}\rangle }_x\hfill \\ \hfill =& -{\langle {log}_{x^{\ell }}\left(x\right),f^{\ell }(t,x^{\ell })\rangle }_{x^{\ell }}-{\langle {log}_x\left(x^{\ell }\right),f(t,x)+u\rangle }_x.\hfill \end{array}$$

(443)

The above expression appears as a sum of two terms referring to two different tangent spaces. It is convenient to move calculations to only one tangent space. We shall take, as the tangent space of reference, the one attached to the state of the follower. Assuming that parallel transport realizes a conformal isometry (see Section 10), it holds that

$${\langle {log}_{x^{\ell }}\left(x\right),f^{\ell }(t,x^{\ell })\rangle }_{x^{\ell }}={\langle {\mathbb{P}}^{x^{\ell }\to x}\left({log}_{x^{\ell }}\left(x\right)\right),{\mathbb{P}}^{x^{\ell }\to x}\left(f^{\ell }(t,x^{\ell })\right)\rangle }_x.$$

(444)

In addition, by Theorem 6, it holds that ${\mathbb{P}}^{x^{\ell }\to x}\left({log}_{x^{\ell }}\left(x\right)\right)=-{log}_x\left(x^{\ell }\right)$ because parallel transport and manifold logarithm are referred to the same geodesic line. Therefore, the expression (443) may be recast as

$$\begin{array}{cc}\hfill \dot{\mathcal{D}}=& {\langle {log}_x\left(x^{\ell }\right),{\mathbb{P}}^{x^{\ell }\to x}\left(f^{\ell }(t,x^{\ell })\right)\rangle }_x-{\langle {log}_x\left(x^{\ell }\right),f(t,x)+u\rangle }_x\hfill \\ \hfill =& {\langle {log}_x\left(x^{\ell }\right),{\mathbb{P}}^{x^{\ell }\to x}\left(f^{\ell }(t,x^{\ell })\right)-f(t,x)-u\rangle }_x,\hfill \end{array}$$

(445)

thanks to the linearity of the inner product. Let us set the control field u so that the sum ${\mathbb{P}}^{x^{\ell }\to x}\left(f^{\ell }(t,x^{\ell })\right)-f(t,x)-u$ in (445) is proportional to $-{log}_x\left(x^{\ell }\right)$, which leads to the expression (442). The choice that led to the control field (442) implies that the function $\mathcal{D}$ satisfies

$$\dot{\mathcal{D}}=-c{\langle {log}_x\left(x^{\ell }\right),{log}_x\left(x^{\ell }\right)\rangle }_x=-c{\parallel {log}_x\left(x^{\ell }\right)\parallel }_x^2=-2c\mathcal{D}.$$

(446)

Since the inner product defines a positive-definite local norm, the above equation entails the inequality $\dot{\mathcal{D}}\le 0$ at any time, which implies asymptotic synchronization. Equality may hold only when ${log}_x\left(x^{\ell }\right)=0$, which happens when the follower and the leader are perfectly synchronized. □

In the expression (442), the constant c takes the meaning of a communication strength between the leader and the follower. In addition, one might notice that the differential Equation (446) may be solved for $\mathcal{D}$ and gives $\mathcal{D}\left(t\right)\propto {\mathrm{e}}^{-2ct}$; hence, synchronization happens with exponential time speed (fast at the beginning, then slower). Such a result is completely independent of the metric that the state manifold $\mathbb{M}$ was endowed with and the speed of synchronization depends only on the constant c.

Let us consider, as an example, a simpler and more familiar case.

Example 40.

As a special case, when $\mathbb{M}={\mathbb{R}}^3$ endowed with the standard Euclidean metric, the parallel transport operator is simply the identity map, while ${log}_{x}y=y-x$. Therefore, the criterion function $\mathcal{D}$ takes the shape of a quadratic error, namely $\mathcal{D}=\frac{1}{2}{e}^{\top }e$, where $e:=x-x^{\ell }$, discussed in [69], and the corresponding control field takes the expression $u=f^{\ell }(t,x^{\ell })-f(t,x)+c(x^{\ell }-x)$, which coincides with the control field discussed in the paper [69].

The freedom in the choice of an inner product accounts for the generalization considered in [69] that consists of considering a Lyapunov function $\mathcal{D}:=\frac{1}{2}{e}^{\top }Pe$ with P being a symmetric, positive-definite weighting matrix. Such a weighting matrix seems to be absent from our formulation of a quadratic error (441), yet it is ‘hidden’ in such an expression. In fact, invoking once again an ambient space $\mathbb{A}$ that the state space is embedded within and a metric ${\langle ·,·\rangle }^{\mathbb{A}}$, we may rewrite the quadratic error (441) as

$$\mathcal{D}=\frac{1}{2}\langle {log}_x\left(x^{\ell }\right),G_x{({log}_x\left(x^{\ell }\right)\rangle }^{\mathbb{A}},$$

(447)

and hence, by comparison, we see that the synchronization error reads $e:={log}_x\left(x^{\ell }\right)$ and that the role of the weighting matrix P is played by the metric kernel G. ■

In control theory, it is customary to associate a scalar index, the control effort, to the control field [70]. In the context of systems on manifold, we define the control effort $\sigma :\mathbb{R}\to \mathbb{R}$ as

$$\sigma \left(t\right):={\langle u\left(t\right),u\left(t\right)\rangle }_{x\left(t\right)}={\parallel u\left(t\right)\parallel }_{x\left(t\right)}^2.$$

(448)

Whenever the follower system and the leader system are twins, namely their state-transition functions are identical $f^{\ell }\equiv f$, under mild continuity conditions on such state-transition functions the control effort vanishes asymptotically to zero. In fact, by the continuity of the state-transition function, it follows that ${\mathbb{P}}^{x^{\ell }\to x}\left(f(t,x^{\ell })\right)-f(t,x)$ approaches 0 as $x\left(t\right)$ approaches $x^{\ell }\left(t\right)$. Such a conclusion holds no longer true if the follower and leader systems’ state-transition functions differ from one another (namely, whenever $f^{\ell }\ne f$). It is worth underlining that the state-transition functions of the leader and of the follower systems, namely $f^{\ell }$ and f, may differ to one another even when these systems are identical since the information on the leader’s state as acquired by the follower may be affected by measurement noise, namely, it might hold that $f(t,·)=f^{\ell }(t,·)+\nu \left(t\right)$, where $\nu \left(t\right)$ denotes a measurement disturbance.

Another interesting observation concerns the speed of synchronization in connection to control effort saving. The constant c in the expression (442) influences the speed of synchronization, namely, the larger c, the speedier the synchronization. However, it is easy to see from the definition (448) that the constant c also affects the control effort, namely, the larger c, the more expensive the control action. Clearly, the proportional control constant c should be chosen as a tradeoff between speed and cost.

12.2. Numerical Methods to Simulate First-Order Systems

In the present subsection we shall recall from the specialized literature the notions of forward Euler method and of explicit $4\mathrm{th}$-order Runge–Kutta method to simulate numerically classical first-order, non-linear, non-autonomous systems. In addition, in the body of the present section we shall lay out extensions of such numerical methods to implement on a computing platform those mathematical dynamical systems insisting on curved manifolds. The main ideas to achieve such an extension may be summarized as follows:

Extension of straight stepping: Classical numerical methods tailored to ${\mathbb{R}}^n$ advance the solution from one step to the next by moving a system’s state along straight segments. On curved state manifolds, the notion of ‘straight segment’ needs to be replaced by the notion of ‘geodesic arc’, and henceforth additive stepping needs to be replaced by exponential-map-based stepping.

Extension of linear stages: High-order methods in ${\mathbb{R}}^n$ select a stepping direction as a linear combination of a number of estimations of a vector field (in the Runge–Kutta methods, these estimations are called ‘stages’). On curved spaces, moving directions are tangent vectors that cannot be combined together directly because they belong to different tangent spaces. Such tangent directions need to be ‘aligned’ together in a given tangent space by means of parallel transport and then combined together.

The forward Euler scheme (fEul) is perhaps the simplest numerical scheme known in the scientific literature to tackle an initial value problem. On a Euclidean space ${\mathbb{R}}^n$, the forward Euler method to simulate numerically a dynamical system $\dot{x}\left(t\right)=f(t,x\left(t\right))$ reads as

$$x_{k+1}=x_k+Tf(kT,x_k),k=0,1,2,3,\dots ,K,$$

(449)

where k denotes a step counter ranging from 0 to a given integer $K>0$, T denotes a time-discretization interval (generally, $T\ll 1$) and $x_k$ denotes an approximation to the true state $x\left(kT\right)$. The accuracy and numerical stability of this method turn out to be reasonable as long as the state-transition function meets certain conditions [71].

This Euler method moves forward the current state $x_k$ to the next state $x_{k+1}$ along a straight line directed toward $f(kT,x_k)$ of a fraction specified by T. Since curved manifolds admits no straight lines in the sense of Euclidean geometry, a plain forward Euler method is inherently unsuitable to cope with a tangent-bundle differential equation, as exemplified in the following.

Example 41.

To tackle the problem that arises about the numerical implementation of dynamical systems on manifolds, it is worth examining an explanatory example based on the low-dimensional manifold $\mathbb{SO}\left(2\right)$. As already outlined, by embedding the space $\mathbb{SO}\left(2\right)$ into the space $A:={\mathbb{R}}^{2\times 2}$, any element of $\mathbb{SO}\left(2\right)$ may be regarded as a 2-by-2 real-valued matrix $X=\left[\begin{array}{cc}{X}_{11}& X_{12}\\ X_{21}& X_{22}\end{array}\right]$ whose entries must satisfy the constrains: $X_{11}^2+X_{21}^2=1$, $X_{22}^2+X_{12}^2=1$, $X_{11}{X}_{12}+X_{21}{X}_{22}=0$ and $X_{11}{X}_{22}-X_{12}{X}_{21}=1$.

Let us consider the first-order dynamical system $\dot{X}=f\left(X\right)$ on the manifold $\mathbb{SO}\left(2\right)$, with $f:X\in \mathbb{SO}\left(2\right)\mapsto f\left(X\right)\in T_X\mathbb{SO}\left(2\right)$. Such a dynamical system may be written as a set of four differential equations of the type ${\dot{X}}_{ij}\left(t\right)=f_{ij}(X_{11}\left(t\right),X_{12}\left(t\right),X_{21}\left(t\right),X_{22}\left(t\right))$. In the present case, it holds that $f\left(X\right)=X\Omega$ with $\Omega =\left[\begin{array}{cc}0& \omega \\ -\omega & 0\end{array}\right]$, where $\omega =\omega \left(X\right(t\left)\right)\in \mathbb{R}$. The forward Euler stepping technique of numerical calculus to solve the above system of differential equations would read as $X_{k+1}=X_k+Tf\left(X_k\right)$, with $T>0$ denoting a step size and $k\ge 0$ a step counter. Such a numerical stepping method does not take into account the constraints on the entries of matrix X, namely, it generates a trajectory $k\mapsto X_k$ in the ambient space ${\mathbb{R}}^{2\times 2}$ rather than in the feasible manifold $\mathbb{SO}\left(2\right)$. Namely, starting from a point $X_k\in \mathbb{SO}\left(2\right)$, it would yield a new point $X_{k+1}\notin \mathbb{SO}\left(2\right)$. The reason of such behavior is that Euler techniques insist on flat spaces and do not cope automatically with curved manifolds.

It is instructive to investigate in detail the effect of an Euler stepping method in the solution of the differential equation $\dot{X}=X\Omega$. In such a context, the Euler stepping equation reads:

$$X_{k+1}=X_k(I_2+T{\Omega }_k),with {\Omega }_k=\left[\begin{array}{cc}0& {\omega }_k\\ -{\omega }_k& 0\end{array}\right],k\ge 0.$$

(450)

As the starting point, $X_0$ satisfies $X_0^{\top }{X}_0=I_2$, and it holds that:

$$X_1^{\top }{X}_1={(I_2+T{\Omega }_0)}^{\top }{X}_0^{\top }{X}_0(I_2+T{\Omega }_0)=I_2-T^2{\Omega }_0^2.$$

(451)

Notice that ${\Omega }_k^2=-{\omega }_k^2{I}_2$, and hence $X_1^{\top }{X}_1=(1+T^2{\omega }_0^2)I_2$. The result of the first step $X_1$ already lost normality of its columns of an additive amount $T^2{\omega }_0^2$ and changed its determinant from 1 to $\sqrt{1+T^2{\omega }_0^2}$. Since T is generally far smaller than 1, and since the deviation is proportional to $T^2$, such a deviation may not be apparent from the first steps, yet progressively detrimental. However, the first step keeps the orthogonality of the columns of the matrix X (such a result is peculiar of the case $\mathbb{SO}\left(2\right)$ only and does not copy to the general case $\mathbb{SO}\left(n\right)$ with $n>2$). For the next step, it holds that:

$$\begin{array}{cc}\hfill X_2^{\top }{X}_2=& {(I_2+T{\Omega }_1)}^{\top }{X}_1^{\top }{X}_1(I_2+T{\Omega }_1)\hfill \\ \hfill =& (1+T^2{\omega }_0^2)(I_2-T^2{\Omega }_1^2)\hfill \\ \hfill =& (1+T^2{\omega }_0^2)(1+T^2{\omega }_1^2)I_2.\hfill \end{array}$$

(452)

By induction, it is readily verified that the matrix $X_k$ keeps monotonically loosing the normality of its two columns of an identical amount. ■

The forward Euler method (449) can be extended to a smooth manifold by replacing the notion of straight line with the notion of geodesic line, to give

$$x_{k+1}={exp}_{x_k}\left(Tf(kT,x_k)\right),k=0,1,2,3,\dots ,K.$$

(453)

The generalization from (449) to (453) is conceived as follows. On a curved state manifold $\mathbb{M}$, each point $x_k$ belongs to $\mathbb{M}$ while each quantity $f(kT,x_k)$ is a tangent vector in $T_{x_k}\mathbb{M}$: these two quantities cannot be combined in an additive way, because $x_k+Tf(kT,x_k)\notin \mathbb{M}$, rather, such quantities are combined by the help of the exponential map, which describes a geodesic arc departing from the state $x_k$ in the tangent direction $f(kT,x_k)$. The above stepping method allows one to compute the first K discrete points of the trajectory generated by the dynamical system $\dot{x}=f(t,x)$, with $x\in \mathbb{M}$, given the initial state $x_0\in \mathbb{M}$. For a Euclidean state space $\mathbb{M}={\mathbb{R}}^n$, it holds that ${exp}_x\left(v\right)=x+v$; therefore, the scheme (453) is apparently a generalization of the well-known Euler scheme (449) from Euclidean to curved spaces.

The set of equations to simulate on a computing platform a leader system and a controlled follower system by the fEul method on a manifold are hence laid out as follows

$$\left\{\begin{array}{cc}{x}_{k+1}^{\ell }={exp}_{x_k^{\ell }}\left(T{f}^{\ell }(kT,x_k^{\ell })\right),\hfill & \mathbf{Free}\mathbf{leader}\hfill \\ u_k={\mathbb{P}}^{x_k^{\ell }\to x_k}\left(f^{\ell }(kT,x_k^{\ell })\right)-f(kT,x_k)+c{log}_{x_k}\left(x_k^{\ell }\right),\hfill & \mathbf{Controller}\hfill \\ x_{k+1}={exp}_{x_k}\left(T(f(kT,x_k)+u_k)\right),\hfill & \mathbf{Controlled}\mathbf{follower}\hfill \\ k=0,1,2,3,\dots ,K.\hfill \end{array}\right.$$

(454)

For the sake of comparison, we notice that, in the case of a leader-follower pair evolving on a Euclidean state space $\mathbb{M}={\mathbb{R}}^n$, the above Equation (454) would simplify into

$$\left\{\begin{array}{cc}{x}_{k+1}^{\ell }=x_k^{\ell }+T{f}^{\ell }(kT,x_k^{\ell }),\hfill & \mathbf{Free}\mathbf{leader}\hfill \\ u_k=f^{\ell }(kT,x_k^{\ell })-f(kT,x_k)+c(x_k^{\ell }-x_k),\hfill & \mathbf{Controller}\hfill \\ x_{k+1}=x_k+T(f(kT,x_k)+u_k),\hfill & \mathbf{Controlled}\mathbf{follower}\hfill \\ k=0,1,2,3,\dots ,K,\hfill \end{array}\right.$$

(455)

because $x_k^{\ell },x_k\in {\mathbb{R}}^n$ and $f,f^{\ell }:\mathbb{R}\times {\mathbb{R}}^n\to {\mathbb{R}}^n$.

A second class of methods that we would like to recall is the explicit, fourth-order Runge–Kutta algorithm. The family of Runge–Kutta numerical integration methods on Euclidean spaces ${\mathbb{R}}^n$ was conceived in order to increase the precision of lower-order methods, such as the Euler method [72]. The explicit 4th-order Runge–Kutta method (eRK4), is based on four partial increments (stages) that, combined together, lead to a complete step:

$$\left\{\begin{array}{c}{h}_{1,k}:=f(kT,x_k),\hfill \\ h_{2,k}:=f(kT+\frac{1}{2}T,x_k+\frac{1}{2}T{h}_{1,k}),\hfill \\ h_{3,k}:=f(kT+\frac{1}{2}T,x_k+\frac{1}{2}T{h}_{2,k}),\hfill \\ h_{4,k}:=f((k+1)T,x_k+T{h}_{3,k}),\hfill \\ x_{k+1}=x_k+T(\frac{h_{1,k}}{6}+\frac{h_{2,k}+h_{3,k}}{3}+\frac{h_{4,k}}{6}),\hfill \\ k=0,1,2,3,\dots ,K.\hfill \end{array}\right.$$

(456)

Similar to the Euler method, the eRK4 method moves forward the current state $x_k$ to the next state $x_{k+1}$ along a straight line toward a specific direction, except that such a direction is computed in a more convoluted way.

The eRK4 method may be extended to a curved manifold by appropriately converting each of the equations in (456). Such a conversion needs to take into account that the state space $\mathbb{M}$ is now a curved manifold:

$$\left\{\begin{array}{c}{h}_{1,k}:=f(kT,x_k),\hfill \\ x_{k+1/4}={exp}_{x_k}\left(\frac{1}{2}T{h}_{1,k}\right),\hfill \\ h_{2,k}:=f(kT+\frac{1}{2}T,x_{k+1/4}),{\tilde{h}}_{2,k}={\mathbb{P}}^{x_{k+1/4}\to x_k}\left(h_{2,k}\right),\hfill \\ x_{k+1/2}={exp}_{x_k}\left(\frac{1}{2}T{\tilde{h}}_{2,k}\right),\hfill \\ h_{3,k}:=f(kT+\frac{1}{2}T,x_{k+1/2}),{\tilde{h}}_{3,k}={\mathbb{P}}^{x_{k+1/2}\to x_k}\left(h_{3,k}\right),\hfill \\ x_{k+3/4}={exp}_{x_k}\left(T{\tilde{h}}_{3,k}\right),\hfill \\ h_{4,k}:=f((k+1)T,x_{k+3/4}),{\tilde{h}}_{4,k}={\mathbb{P}}^{x_{k+3/4}\to x_k}\left(h_{4,k}\right),\hfill \\ x_{k+1}={exp}_{x_k}\left(\frac{1}{6}T{h}_{1,k}+\frac{1}{3}T({\tilde{h}}_{2,k}+{\tilde{h}}_{3,k})+\frac{1}{6}T{\tilde{h}}_{4,k}\right),\hfill \\ k=0,1,2,3,\dots ,K.\hfill \end{array}\right.$$

(457)

(The notation $x_{k+1/4}$, $x_{k+1/2}$, …, used to indicate intermediate steps, is standard in numerical calculus.) The reasoning that led to this numerical scheme is outlined as follows: Once a direction $h_{1,k}$ is obtained in the first stage, it is used to determine the direction $h_{2,k}$ in the second stage; the formula to compute $h_{2,k}$ in (456) prescribes to evaluate the vector field $f(·,x)$ in a point $x_k+\frac{1}{2}T{h}_{1,k}$; this last expression, however, cannot be applied directly on a curved manifold and needs to be replaced by ${exp}_{x_k}\left(\frac{1}{2}T{h}_{1,k}\right)$, which gives the expression $h_{2,k}$ in (457); even so, the obtained result cannot be used directly, since the update rule in (456), which prescribes to advance the value $x_k$ by $x_{k+1}=x_k+T(\frac{1}{6}{h}_{1,k}+\frac{1}{3}{h}_{2,k}+\dots )$, needs to be translated into $x_{k+1}={exp}_{x_k}\left(\frac{1}{6}T{h}_{1,k}+\frac{1}{3}T{h}_{2,k}+\dots \right)$ which, in turn, requires the vector $h_{2,k}$ to belong to $T_{x_k}\mathbb{M}$, while it belongs to $T_{x_{k+1/4}}\mathbb{M}$; for this reason, it is necessary to make a further modification and to parallel transport the vector $h_{2,k}$ to $T_{x_k}\mathbb{M}$, namely, to compute ${\tilde{h}}_{2,k}={\mathbb{P}}^{x_{k+1/4}\to x_k}\left(h_{2,k}\right)$; the same holds for the subsequent stages.

The complete set of equations required to implement numerically a leader and a controlled follower by the eRK4 method on a manifold are

$$\left\{\begin{array}{cc}\left.\begin{array}{c}{h}_{1,k}^{\ell }:=f^{\ell }(kT,x_k^{\ell }),\hfill \\ x_{k+1/4}^{\ell }={exp}_{x_k^{\ell }}\left(\frac{1}{2}T{h}_{1,k}^{\ell }\right),\hfill \\ h_{2,k}^{\ell }:=f^{\ell }(kT+\frac{1}{2}T,x_{k+1/4}^{\ell }),{\tilde{h}}_{2,k}^{\ell }={\mathbb{P}}^{x_{k+1/4}^{\ell }\to x_k^{\ell }}\left(h_{2,k}^{\ell }\right),\hfill \\ x_{k+1/2}^{\ell }={exp}_{x_k^{\ell }}\left(\frac{1}{2}T{\tilde{h}}_{2,k}^{\ell }\right),\hfill \\ h_{3,k}^{\ell }:=f^{\ell }(kT+\frac{1}{2}T,x_{k+1/2}^{\ell }),{\tilde{h}}_{3,k}^{\ell }={\mathbb{P}}^{x_{k+1/2}^{\ell }\to x_k^{\ell }}\left(h_{3,k}^{\ell }\right),\hfill \\ x_{k+3/4}^{\ell }={exp}_{x_k^{\ell }}\left(T{\tilde{h}}_{3,k}^{\ell }\right),\hfill \\ h_{4,k}^{\ell }:=f^{\ell }((k+1)T,x_{k+3/4}^{\ell }),{\tilde{h}}_{4,k}^{\ell }={\mathbb{P}}^{x_{k+3/4}^{\ell }\to x_k^{\ell }}\left(h_{4,k}^{\ell }\right),\hfill \\ x_{k+1}^{\ell }={exp}_{x_k^{\ell }}\left(\frac{1}{6}T{h}_{1,k}^{\ell }+\frac{1}{3}T({\tilde{h}}_{2,k}^{\ell }+{\tilde{h}}_{3,k}^{\ell })+\frac{1}{6}T{\tilde{h}}_{4,k}^{\ell }\right)\hfill \end{array}\right\}\hfill & \mathbf{Leader}\hfill \\ u_k={\mathbb{P}}^{x_k^{\ell }\to x_k}\left(f^{\ell }(kT,x_k^{\ell })\right)-f(kT,x_k)+c{log}_{x_k}\left(x_k^{\ell }\right),\hfill & \mathbf{Controller}\hfill \\ \left.\begin{array}{l}{h}_{1,k}:=f(kT,x_k),\\ x_{k+1/4}={exp}_{x_k}\left(\frac{1}{2}T{h}_{1,k}\right),\hfill \\ h_{2,k}:=f(kT+\frac{1}{2}T,x_{k+1/4}),{\tilde{h}}_{2,k}={\mathbb{P}}^{x_{k+1/4}\to x_k}\left(h_{2,k}\right),\hfill \\ x_{k+1/2}={exp}_{x_k}\left(\frac{1}{2}T{\tilde{h}}_{2,k}\right),\hfill \\ h_{3,k}:=f(kT+\frac{1}{2}T,x_{k+1/2}),{\tilde{h}}_{3,k}={\mathbb{P}}^{x_{k+1/2}\to x_k}\left(h_{3,k}\right),\hfill \\ x_{k+3/4}={exp}_{x_k}\left(T{\tilde{h}}_{3,k}\right),\hfill \\ h_{4,k}:=f((k+1)T,x_{k+3/4}),{\tilde{h}}_{4,k}={\mathbb{P}}^{x_{k+3/4}\to x_k}\left(h_{4,k}\right),\hfill \\ x_{k+1}={exp}_{x_k}\left(\frac{1}{6}T{h}_{1,k}+\frac{1}{3}T({\tilde{h}}_{2,k}+{\tilde{h}}_{3,k})+\frac{1}{6}T{\tilde{h}}_{4,k}+T{u}_k\right),\hfill \end{array}\right\}\hfill & \mathbf{Follower}\hfill \\ k=0,1,2,3,\dots ,K.\hfill & \end{array}\right.$$

(458)

The implementation of the above equations on a computing platform does not pose serious concerns as long as the chosen development language possesses adequate commands to deal seamlessly with arrays and array-type functions.

13. Riemannian Hessian of a Manifold-to-Scalar Function

The next step in Taylor series approximation to a manifold-to-scalar function beyond the first-order term (through gradient) is the second-order termed based on the notion of ‘Hessian’.

13.1. Definition, Properties and Coordinate-Free Calculation of Riemannian Hessian

Given a differentiable function $f:\mathbb{M}\to \mathbb{R}$, its Riemannian Hessian at a point $x\in \mathbb{M}$ is a linear operator $H_{x}f:T_x\mathbb{M}\to T_x\mathbb{M}$ that appears in the quadratic term of the Taylor approximation

$$f\left({\gamma }_{x,v}\left(t\right)\right)=f\left(x\right)+{\langle {\mathrm{grad}}_{x}f,v\rangle }_{x}t+\frac{1}{2}{\langle H_{x}f\left(v\right),v\rangle }_x{t}^2+\mathcal{O}\left(t^3\right),$$

(459)

where ${\gamma }_{x,v}:[-ϵ,ϵ]\to \mathbb{M}$ denotes, in principle, any smooth curve such that ${\gamma }_{x,v}\left(0\right)=x$ and ${\dot{\gamma }}_{x,v}\left(0\right)=v$. A major problem with such liberality in the choice of a curve is that, since clearly

$${\langle H_x\left(v\right),v\rangle }_x={\left.\frac{{\mathrm{d}}^{2}f\left({\gamma }_{x,v}\left(t\right)\right)}{\mathrm{d}{t}^2}\right|}_{t=0},$$

(460)

the Hessian operator would not just depend on the pair $(x,v)$ but also on ${\ddot{\gamma }}_{x,v}\left(0\right)$, which is not prescribed. A Hessian that depends on the shape of the curve $\gamma$ which, after all, should be instrumental—not determinant—is hardly acceptable in practice, which is why the notion of a Hessian operator is generally defined with respect to a geodesic${\gamma }_{x,v}$. It is worth pointing out that, according to the ‘algebraic’ argument recalled in Observation 1, since the Hessian stems from a quadratic form, only its self-adjoint part plays a role. Indeed, since its very definition stems from a quadratic form, only its self-adjoint component may be determined.

Not surprisingly, the Riemannian Hessian on a manifold $\mathbb{M}$ embedded in an ambient space $\mathbb{A}$ is related to the ambient Hessian in $\mathbb{A}$. Still, there is a perhaps surprising outcome in its calculation, namely, it depends on the ambient gradient of f (while the standard Hessian does not). Let us recall what is meant by ambient Hessian ${\partial }_x^{2}f$ of a function $f:\mathbb{A}\to \mathbb{R}$, that is

$${\partial }_x^{2}f\left(v\right):=\underset{h\to 0}{lim}\frac{{\partial }_{{\rho }_{x,v}\left(h\right)}f-{\partial }_{x}f}{h}={\left.\frac{\mathrm{d}}{\mathrm{d}t}{\partial }_{{\rho }_{x,v}\left(t\right)}f\right|}_{t=0},$$

(461)

where ${\rho }_{x,v}:[-ϵ,ϵ]\to \mathbb{M}$ denotes any smooth curve such that ${\rho }_{x,v}\left(0\right)=x\in \mathbb{M}$ and ${\dot{\rho }}_{x,v}\left(0\right)=v\in T_x\mathbb{M}$—for instance, if $f:{\mathbb{R}}^p\to \mathbb{R}$, then ${\partial }_x^{2}f$ takes the form of a $p\times p$ matrix of partial derivatives and ${\partial }_x^{2}f\left(v\right)$ just denotes the matrix-to-column-array product $\left({\partial }_x^{2}f\right)v$.

To carry out the calculation of the Riemannian Hessian, let us recall the relationship (301) that may be written as

$$\frac{\mathrm{d}}{\mathrm{d}t}f\left({\gamma }_{x,v}\left(t\right)\right)={\langle {\mathrm{grad}}_{{\gamma }_{x,v}\left(t\right)}f,{\dot{\gamma }}_{x,v}\left(t\right)\rangle }_{{\gamma }_{x,v}\left(t\right)}.$$

(462)

Introducing the metric kernel and the inner product on the ambient space gives:

$$\frac{\mathrm{d}}{\mathrm{d}t}f\left({\gamma }_{x,v}\left(t\right)\right)={\langle G_{{\gamma }_{x,v}\left(t\right)}\left({\mathrm{grad}}_{{\gamma }_{x,v}\left(t\right)}f\right),{\dot{\gamma }}_{x,v}\left(t\right)\rangle }^{\mathbb{A}}.$$

(463)

Recalling the explicit expression (284) of the Riemannian gradient based on an orthogonal projector $\Pi$ gives

$$\frac{\mathrm{d}}{\mathrm{d}t}f\left({\gamma }_{x,v}\left(t\right)\right)={\langle {\Pi }_{{\gamma }_{x,v}\left(t\right)}\left({\partial }_{{\gamma }_{x,v}\left(t\right)}f\right),{\dot{\gamma }}_{x,v}\left(t\right)\rangle }^{\mathbb{A}}.$$

(464)

The right-hand side in the above expression has been written in a way that makes it easier to take a second derivative with respect to the parameter t, which reads as

$$\frac{{\mathrm{d}}^2}{\mathrm{d}{t}^2}f\left({\gamma }_{x,v}\left(t\right)\right)={\langle \frac{\mathrm{d}}{\mathrm{d}t}{\Pi }_{{\gamma }_{x,v}\left(t\right)}\left({\partial }_{{\gamma }_{x,v}\left(t\right)}f\right),{\dot{\gamma }}_{x,v}\left(t\right)\rangle }^{\mathbb{A}}+{\langle {\Pi }_{{\gamma }_{x,v}\left(t\right)}\left({\partial }_{{\gamma }_{x,v}\left(t\right)}f\right),{\ddot{\gamma }}_{x,v}\left(t\right)\rangle }^{\mathbb{A}}.$$

(465)

The first expression on the right-hand side may be written explicitly by introducing a linear operator ${\Pi }^{•}$ that represents a derivative of the orthogonal projector, while the second term on the right-hand side may be made more explicit by recalling, from the relationships in (356), the relation between the naïve acceleration over a geodesic and the Christoffel form of second kind. Let us define the linear operator

$${\Pi }_x^{•}(a,v):={\left.\frac{\mathrm{d}}{\mathrm{d}t}{\Pi }_{{\rho }_{x,v}\left(t\right)}\left(a\right)\right|}_{t=0},$$

(466)

where $x\in \mathbb{M}$ denotes a point where projection is carried out, $a\in \mathbb{A}$ is an element of the ambient space whose projection is sought, $v\in T_x\mathbb{M}$ is a tangent direction along which a variation of the projected vector ${\Pi }_x\left(a\right)$ is sought to be estimated and ${\rho }_{x,v}:[-ϵ,ϵ]\to \mathbb{M}$ is any smooth curve such that ${\rho }_{x,v}\left(0\right)=x$ and ${\dot{\rho }}_{x,v}\left(0\right)=v$.

To clarify the above definition, let us examine a useful computation rule.

Example 42.

Let us define $g:\mathbb{M}\to \mathbb{A}$ as an ambient-valued function of a manifold-valued argument and a smooth curve $\gamma :\mathbb{R}\to \mathbb{M}$. Let us now consider the composition ${\Pi }_{\gamma \left(t\right)}\left(g\left(\gamma \left(t\right)\right)\right)$. We aim at calculating its derivative with respect to the parameter t. It holds that

$$\frac{\mathrm{d}}{\mathrm{d}t}{\Pi }_{\gamma }\left(g\left(\gamma \right)\right)={\Pi }_{\gamma }^{•}(g\left(\gamma \right),\dot{\gamma })+{\Pi }_{\gamma }\left({\partial }_{\gamma }g\left(\dot{\gamma }\right)\right),$$

(467)

which is a linear function of $\dot{\gamma }$. ■

On the basis of the above calculation rule, of interest in the following developments, we can prove a collateral property of the map ${\Pi }^{•}$.

Example 43.

The map ${\Pi }^{•}$ applied totwotangent vectors returns a normal element. In fact, let us recall that ${\Pi }_x\left({\Pi }_x\left(a\right)\right)={\Pi }_x\left(a\right)$ for every $x\in \mathbb{M}$ and $a\in \mathbb{A}$. On any smooth curve $\gamma :[-ϵ,ϵ]\to \mathbb{M}$ such that $\gamma \left(0\right)=x$ and $\dot{\gamma }\left(0\right)=v$ it holds that

$${\Pi }_{\gamma }\left({\Pi }_{\gamma }\left(a\right)\right)={\Pi }_{\gamma }\left(a\right).$$

(468)

Deriving with respect to the parameter t and applying the computation rule (467), yields

$${\Pi }_{\gamma }^{•}({\Pi }_{\gamma }\left(a\right),\dot{\gamma })+{\Pi }_{\gamma }\left({\Pi }_{\gamma }^{•}(a,\dot{\gamma })\right)={\Pi }_{\gamma }^{•}(a,\dot{\gamma }).$$

(469)

Setting $t=0$ yields:

$${\Pi }_x^{•}({\Pi }_x\left(a\right),v)+{\Pi }_x\left({\Pi }_x^{•}(a,v)\right)={\Pi }_x^{•}(a,v).$$

(470)

Now we can set $w:={\Pi }_x\left(a\right)\in T_x\mathbb{M}$ and write

$${\Pi }_x^{•}(w,v)={\Pi }_x^{•}(a,v)-{\Pi }_x\left({\Pi }_x^{•}(a,v)\right).$$

(471)

The term on the right-hand side represents the normal component of ${\Pi }_x^{•}(a,v)$, and hence ${\Pi }_x\left({\Pi }_x^{•}(w,v)\right)=0$. ■

On the basis of the definition (337), the second derivative (465) may be rewritten as

$$\begin{array}{cc}\hfill \frac{{\mathrm{d}}^2}{\mathrm{d}{t}^2}f\left({\gamma }_{x,v}\left(t\right)\right)=& {\langle {\Pi }_{{\gamma }_{x,v}\left(t\right)}^{•}({\partial }_{{\gamma }_{x,v}\left(t\right)}f,{\dot{\gamma }}_{x,v}\left(t\right)),{\dot{\gamma }}_{x,v}\left(t\right)\rangle }^{\mathbb{A}}\hfill \\ \hfill & +{\langle {\Pi }_{{\gamma }_{x,v}\left(t\right)}\left(\frac{\mathrm{d}}{\mathrm{d}t}{\partial }_{{\gamma }_{x,v}\left(t\right)}f\right),{\dot{\gamma }}_{x,v}\left(t\right)\rangle }^{\mathbb{A}}\hfill \\ \hfill & -{\langle {\Pi }_{{\gamma }_{x,v}\left(t\right)}\left({\partial }_{{\gamma }_{x,v}\left(t\right)}f\right),{\overline{\overline{\Gamma }}}_{{\gamma }_{x,v}\left(t\right)}({\dot{\gamma }}_{x,v}\left(t\right),{\dot{\gamma }}_{x,v}\left(t\right))\rangle }^{\mathbb{A}}.\hfill \end{array}$$

(472)

Setting $t=0$ leads to

$${\langle H_{x}f\left(v\right),v\rangle }_x={\langle {\Pi }_x^{•}({\partial }_{x}f,v),v\rangle }^{\mathbb{A}}+{\langle {\Pi }_x\left({\partial }_x^{2}f\left(v\right)\right),v\rangle }^{\mathbb{A}}-{\langle {\Pi }_x\left({\partial }_{x}f\right),{\overline{\overline{\Gamma }}}_x(v,v)\rangle }^{\mathbb{A}}.$$

(473)

As a last step, recall that, in the rightmost term, it is possible to swap one instance of v with the term ${\Pi }_x\left({\partial }_{x}f\right)$; hence, we write

$${\langle G_x\left(H_{x}f\left(v\right)\right),v\rangle }^{\mathbb{A}}={\langle {\Pi }_x^{•}({\partial }_{x}f,v),v\rangle }^{\mathbb{A}}+{\langle {\Pi }_x\left({\partial }_x^{2}f\left(v\right)\right),v\rangle }^{\mathbb{A}}-{\langle {\overline{\overline{\Gamma }}}_x({\Pi }_x\left({\partial }_{x}f\right),v),v\rangle }^{\mathbb{A}},$$

(474)

which must hold for every $v\in T_x\mathbb{M}$. The last expression is equivalent to

$$G_x\left(H_{x}f\left(v\right)\right)-{\Pi }_x^{•}({\partial }_{x}f,v)-{\Pi }_x\left({\partial }_x^{2}f\left(v\right)\right)+{\overline{\overline{\Gamma }}}_x({\Pi }_x\left({\partial }_{x}f\right),v)\in N_x\mathbb{M}.$$

(475)

An explicit expression of the Riemannian Hessian may now be obtained by applying the orthogonal projector ${\Pi }_x$ to both sides and then the inverse metric kernel to the result, which ultimately gives

$$H_{x}f\left(v\right)=G_x^{-1}\left({\Pi }_x\left({\partial }_x^{2}f\left(v\right)\right)\right)+G_x^{-1}\left({\Pi }_x\left({\Pi }_x^{•}({\partial }_{x}f,v)\right)\right)-G_x^{-1}\left({\Pi }_x\left({\overline{\overline{\Gamma }}}_x({\Pi }_x\left({\partial }_{x}f\right),v)\right)\right).$$

(476)

Notice that, unless the difference between the last two terms is zero, the Riemannian Hessian depends on the ambient gradient ${\partial }_{x}f$.

It is worth examining separately a few special cases in which the expression of the Riemannian Hessian simplifies:

• Case that ${\Pi }_x$ is independent of x: This case occurs when tangent spaces coincide to one another. In this case, the projection operator may simply be denoted as $\Pi$ and it holds that ${\Pi }^{•}\equiv 0$; therefore,

  $$H_{x}f\left(v\right)=G_x^{-1}(\Pi \left({\partial }_x^{2}f\left(v\right)\right))-G_x^{-1}(\Pi \left({\overline{\overline{\Gamma }}}_x(\Pi \left({\partial }_{x}f\right),v)\right)).$$

  (477)
• Case that the metric kernel coincides with the identity: This is the case most generally covered in the literature that occurs when the ambient space $\mathbb{A}$ is a Euclidean space. In this case

  $$H_{x}f\left(v\right)={\Pi }_x\left({\Pi }_x^{•}({\partial }_{x}f,v)\right)+{\Pi }_x\left({\partial }_x^{2}f\left(v\right)\right)-{\Pi }_x\left({\overline{\overline{\Gamma }}}_x({\Pi }_x\left({\partial }_{x}f\right),v)\right).$$

  (478)
  This case was explicitly covered in [73], in which the expression of the Hessian was given in terms of Weingarten map. The importance of the Weingarten map in applied sciences was further highlighted in [74].
• Case that the gradient of the function f is null: In a point $x\in \mathbb{M}$ where ${\mathrm{grad}}_{x}f=0$ the expression of the Hessian simplifies noticeably. In fact the last two terms in (476) vanish to zero; therefore, it holds that

  $$H_{x}f\left(v\right)=(G_x^{-1}\circ {\Pi }_x)\left({\partial }_x^{2}f\left(v\right)\right).$$

  (479)
  This is in fact the only case in which the Hessian does not depend on the Christoffel form.
• Case of the manifold ${\mathbb{R}}^p$ endowed with a Euclidean metric: This is the reference case that we may look at for familiarity. In this case, $G_x={\mathrm{id}}_x$, ${\Pi }_x={\mathrm{id}}_x$, ${\overline{\Gamma }}_x=0$; hence,

  $$H_{x}f\left(v\right)={\partial }_x^{2}f\left(v\right),$$

  (480)

  which looks exactly as one expects.

To what concerns the operator ${\Pi }^{•}$, we are going to assume that the following commutation property with the ambient metric holds: For every $x\in \mathbb{M}$, $a\in \mathbb{A}$, $u,v\in T_x\mathbb{M}$ it is true that

$${\langle {\Pi }_x^{•}(a,v),w\rangle }^{\mathbb{A}}={\langle {\Pi }_x^{•}(a,w),v\rangle }^{\mathbb{A}}.$$

(481)

Let us go into some detail about the practical computation of the operator ${\Pi }^{•}$ by surveying an example related to the Stiefel manifold.

Example 44.

The Stiefel manifold embedded in its canonical ambient space endowed with the Euclidean metric has an orthogonal projector defined by the relation (286), namely [33]:

$${\Pi }_X\left(A\right):=A-\frac{1}{2}X(X^{\top }A+A^{\top }X).$$

(482)

In order to apply the definition (466), let us first take any smooth curve ${\rho }_{X,V}\left(t\right)$, with $X\in \mathrm{St}(n,p)$ and $V\in T_X\mathrm{St}(n,p)$, and write

$${\Pi }_{{\rho }_{X,V}\left(t\right)}\left(A\right):=A-\frac{1}{2}{\rho }_{X,V}\left(t\right)({\rho }_{X,V}^{\top }\left(t\right)A+A^{\top }{\rho }_{X,V}\left(t\right)).$$

(483)

Taking the first derivative, side by side, with respect to the parameter t gives:

$$\begin{array}{cc}\hfill \frac{\mathrm{d}}{\mathrm{d}t}{\Pi }_{{\rho }_{X,V}\left(t\right)}\left(A\right):=& -\frac{1}{2}{\dot{\rho }}_{X,V}\left(t\right)({\rho }_{X,V}^{\top }\left(t\right)A+A^{\top }{\rho }_{X,V}\left(t\right))\hfill \\ & -\frac{1}{2}{\rho }_{X,V}\left(t\right)({\dot{\rho }}_{X,V}^{\top }\left(t\right)A+A^{\top }{\dot{\rho }}_{X,V}\left(t\right)).\hfill \end{array}$$

(484)

Now, setting $t=0$ and recalling that ${\rho }_{X,V}\left(0\right)=X$ and ${\dot{\rho }}_{X,V}\left(0\right)=V$ yields

$${\Pi }_X^{•}(A,V)=-\frac{1}{2}X(V^{\top }A+A^{\top }V)-\frac{1}{2}V(X^{\top }A+A^{\top }X).$$

(485)

Let us verify the commutativity property (481). We have

$$\begin{array}{cc}& {\langle {\Pi }_X^{•}(A,V),W\rangle }^{\mathbb{A}}=-\frac{1}{2}\mathrm{tr}\left(W^{\top }X(V^{\top }A+A^{\top }V)\right)-\frac{1}{2}\mathrm{tr}\left(W^{\top }V(X^{\top }A+A^{\top }X)\right),\hfill \\ & {\langle {\Pi }_X^{•}(A,W),V\rangle }^{\mathbb{A}}=-\frac{1}{2}\mathrm{tr}\left(V^{\top }X(W^{\top }A+A^{\top }W)\right)-\frac{1}{2}\mathrm{tr}\left(V^{\top }W(X^{\top }A+A^{\top }X)\right).\hfill \end{array}$$

(486)

Notice that the first terms on the right-hand sides of both inner products are null, because $W^{\top }X$ and $V^{\top }X$ are skew-symmetric, while the remaining terms may be proven equal by the trace cyclic permutation property and by symmetry. ■

The case of the special orthogonal group is relevant and instructive as well.

Example 45.

The special orthogonal group $\mathbb{SO}\left(n\right)$ embedded into the ambient space $\mathbb{A}:={\mathbb{R}}^{n\times n}$, endowed with the Euclidean metric, admits the orthogonal projection

$${\Pi }_R\left(A\right):=\frac{1}{2}(A-R{A}^{\top }R).$$

(487)

Let us verify that ${\Pi }_R$ maps any matrix from $\mathbb{A}$ to the tangent space $T_R\mathbb{SO}\left(n\right)$:

$$R^{\top }{\Pi }_R\left(A\right)+{\Pi }_R^{\top }\left(A\right)R=\frac{1}{2}{R}^{\top }(A-R{A}^{\top }R)+\frac{1}{2}{(A-R{A}^{\top }R)}^{\top }R=0,$$

(488)

by easy matrix calculations. In addition, let us verify that $A-{\Pi }_R\left(A\right)\in N_R\mathbb{SO}\left(n\right)$:

$$\begin{array}{cc}\hfill {\langle A-{\Pi }_R\left(A\right),V\rangle }^{\mathbb{A}}=& \mathrm{tr}\left(V^{\top }\left(A-\frac{1}{2}(A-R{A}^{\top }R)\right)\right)\hfill \\ \hfill =& \frac{1}{2}\mathrm{tr}\left(V^{\top }A+\left(V^{\top }R\right)A^{\top }R\right)\hfill \\ \hfill =& \frac{1}{2}\mathrm{tr}\left(V^{\top }A+(-R^{\top }V)A^{\top }R\right)\hfill \\ \hfill =& \frac{1}{2}\mathrm{tr}\left(V^{\top }A-A^{\top }\left(R{R}^{\top }\right)V)\right)\hfill \\ \hfill =& \frac{1}{2}\mathrm{tr}(V^{\top }A-A^{\top }V)\hfill \\ \hfill =& 0.\hfill \end{array}$$

(489)

To apply the definition (466), let us first take a smooth curve ${\rho }_{R,V}\left(t\right)$, with $R\in \mathbb{SO}\left(n\right)$ and $V\in T_X\mathbb{SO}\left(n\right)$, and evaluate

$${\Pi }_{{\rho }_{R,V}\left(t\right)}\left(A\right):=\frac{1}{2}(A-{\rho }_{R,V}\left(t\right)A^{\top }{\rho }_{R,V}\left(t\right)).$$

(490)

Taking the first derivative, side by side, with respect to the parameter t gives:

$$\frac{\mathrm{d}}{\mathrm{d}t}{\Pi }_{{\rho }_{R,V}\left(t\right)}\left(A\right):=-\frac{1}{2}{\dot{\rho }}_{R,V}\left(t\right)A^{\top }{\rho }_{R,V}\left(t\right)-\frac{1}{2}{\rho }_{R,V}\left(t\right)A^{\top }{\dot{\rho }}_{R,V}\left(t\right).$$

(491)

Setting $t=0$ and recalling that ${\rho }_{R,V}\left(0\right)=R$ and ${\dot{\rho }}_{R,V}\left(0\right)=R$ yields

$${\Pi }_R^{•}(A,V)=-\frac{1}{2}\left(V{A}^{\top }R+R{A}^{\top }V\right).$$

(492)

To end this example, let us verify the commutativity property (481). Let us observe that

$$\begin{array}{cc}\hfill & {\langle {\Pi }_R^{•}(A,V),W\rangle }^{\mathbb{A}}=-\frac{1}{2}\mathrm{tr}\left(W^{\top }V{A}^{\top }R\right)-\frac{1}{2}\mathrm{tr}\left(W^{\top }R{A}^{\top }V\right),\hfill \\ \hfill & {\langle {\Pi }_R^{•}(A,W),V\rangle }^{\mathbb{A}}=-\frac{1}{2}\mathrm{tr}\left(V^{\top }W{A}^{\top }R\right)-\frac{1}{2}\mathrm{tr}\left(V^{\top }R{A}^{\top }W\right).\hfill \end{array}$$

(493)

The above expressions may be proven equal by some clever matrix manipulation. For example, by noticing that $W^{\top }=-R^{\top }W{R}^{\top }$ and $V=-R{V}^{\top }R$, one may show that

$$\begin{array}{cc}\hfill \mathrm{tr}\left(W^{\top }R{A}^{\top }V\right)=& \mathrm{tr}\left((-R^{\top }W{R}^{\top })R{A}^{\top }(-R{V}^{\top }R)\right)\hfill \\ \hfill =& \mathrm{tr}\left(W{A}^{\top }R{V}^{\top }\right)\hfill \\ \hfill =& \mathrm{tr}\left(V^{\top }W{A}^{\top }R\right),\hfill \end{array}$$

(494)

by repeated application of cyclic permutation invariance.

Let us complete the calculation of the Riemannian Hessian for the special orthogonal group. According to the Hessian formula (478), we preliminarily need to evaluate

$$\begin{array}{cc}\hfill {\Pi }_R\left({\Pi }_R^{•}(A,V)\right)=& \frac{1}{4}\left\{(A{V}^{\top }-V{A}^{\top })R-R(A^{\top }V-V^{\top }A)\right\},\hfill \\ \hfill {\Pi }_R\left({\overline{\overline{\Gamma }}}_R(V,W)\right)=& 0,\hfill \end{array}$$

(495)

therefore

$$\begin{array}{cc}\hfill H_{R}f\left(V\right)=& \frac{1}{2}({\partial }_R^{2}f\left(V\right)-R{\left({\partial }_R^{2}f\left(V\right)\right)}^{\top }R)\hfill \\ \hfill & -\frac{1}{4}\left\{({\partial }_{R}f{V}^{\top }-V{\partial }_R^{\top }f)R-R({\partial }_R^{\top }fV-V^{\top }{\partial }_{R}f)\right\}.\hfill \end{array}$$

(496)

It is easy to verify, by direct calculation, that $H_R\left(V\right)\in T_R\mathbb{SO}\left(n\right)$. ■

Let us survey the computation of the Riemannian Hessian on the hypersphere.

Example 46.

Let us compute the Riemannian Hessian associated with the hypersphere $\mathbb{M}:={\mathbb{S}}^{p-1}$ endowed with the canonical metric ${\langle u,v\rangle }_x:=u^{\top }v$, embedded in the ambient space $\mathbb{A}:={\mathbb{R}}^p$ endowed with the metric ${\langle u,v\rangle }^{\mathbb{A}}:=u^{\top }v$. Recall that it holds ${\Pi }_x\left(a\right):=(I_p-x{x}^{\top })a$ and ${\overline{\Gamma }}_x(u,v)=-x\left(u^{\top }v\right)$.

It is just necessary to compute the derivative ${\Pi }_x^{•}(a,v)$ and then to make use of the relationship (478). By definition, we have

$${\Pi }_x^{•}(a,v):={\left.\frac{\mathrm{d}}{\mathrm{d}t}{\Pi }_{{\rho }_{x,v}\left(t\right)}\left(a\right)\right|}_{t=0}=-(x{v}^{\top }+v{x}^{\top })a.$$

(497)

Ultimately, we obtain

$$H_{x}f\left(v\right)=(I_p-x{x}^{\top })\left({\partial }_x^{2}f\right)v-\left(x^{\top }{\partial }_{x}f\right)v,$$

(498)

which represents the Riemannian Hessian utilized in [61].

It is instrumental to notice that, for $a\in {\mathbb{R}}^p$ and $v\in T_x{\mathbb{S}}^{p-1}$, the quantity ${\Pi }_x^{•}(a,v)$ possesses a radial (normal) component $x\left(v^{\top }a\right)$ as well as a tangential component $v\left(x^{\top }a\right)$. Let us verify the property (481). We have

$$\begin{array}{cc}\hfill & {\langle {\Pi }_x^{•}(a,v),w\rangle }^{\mathbb{A}}=-\left(w^{\top }x\right)\left(v^{\top }a\right)+\left(w^{\top }v\right)\left(x^{\top }a\right),\hfill \\ \hfill & {\langle {\Pi }_x^{•}(a,w),v\rangle }^{\mathbb{A}}=-\left(v^{\top }x\right)\left(w^{\top }a\right)+\left(v^{\top }w\right)\left(x^{\top }a\right).\hfill \end{array}$$

(499)

The first terms on the right-hand sides are null because $v,w\in T_x{\mathbb{S}}^{p-1}$, while the remaining terms are to one another equal. ■

The space of symmetric, positive-definite matrices is interesting to what concerns Riemannian Hessian.

Example 47.

Let us recall that, for the space ${\mathbb{S}}^{+}\left(n\right)$, we have chosen $G_P\left(W\right):=P^{-1}W{P}^{-1}$, also it holds that ${\overline{\overline{\Gamma }}}_P(V,W):=-\frac{1}{2}V{P}^{-1}W-\frac{1}{2}W{P}^{-1}V$, ${\Pi }_P\left(A\right)=\frac{1}{2}(A+A^{\top })$ for any $A\in \mathbb{GL}\left(n\right)$. Since ${\Pi }_P$ does not explicitly depend on the point P, we may utilize Formula (477) to evaluate the Hessian. ■

As a last example, let us compute the Riemannian Hessian of a linear function. In the familiar ${\mathbb{R}}^n$ case, we should obtain a null Hessian, while in a manifold setting this is easily guessed not to be the case.

Example 48.

Let a smooth manifold $\mathbb{M}$ be embedded in an ambient space $\mathbb{A}$ and let $\ell :\mathbb{M}\to \mathbb{R}$ denote a linear function such that ${\partial }_x\ell =a\in \mathbb{A}$ constant and ${\partial }_x^2\ell =0$. Not having assumed anything else, the Hessian of the function ℓ should be calculated by means of the expression (476), namely

$$H_x\ell \left(v\right)=G_x^{-1}\left\{{\Pi }_x\left({\partial }_x^2\ell \left(v\right)+{\Pi }_x^{•}({\partial }_x\ell ,v)-{\overline{\overline{\Gamma }}}_x({\Pi }_x({\partial }_x\ell ),v)\right)\right\}.$$

(500)

In this case, the Riemannian Hessian reads as

$$H_x\ell \left(v\right)=G_x^{-1}\left\{{\Pi }_x\left({\Pi }_x^{•}(a,v)-{\overline{\overline{\Gamma }}}_x({\Pi }_x\left(a\right),v)\right)\right\},$$

(501)

which clearly varies from point to point on $\mathbb{M}$. ■

The property (481) implies that in an inner product such as ${\langle H_x\left(v\right),w\rangle }_x$ the tangent vector arguments may be swapped, which implies the Riemannian Hessian operator is self-adjoint, namely

$${\langle H_x\left(v\right),w\rangle }_x={\langle v,H_x\left(w\right)\rangle }_x.$$

(502)

This property may be proven directly by the relationship (476); in fact,

$$\begin{array}{cc}\hfill {\langle H_x\left(v\right),w\rangle }_x=& {\langle G_x^{-1}\left({\Pi }_x\left({\partial }_x^{2}f\left(v\right)\right)\right),G_x\left(w\right)\rangle }^{\mathbb{A}}\hfill \\ \hfill & +{\langle G_x^{-1}\left({\Pi }_x\left({\Pi }_x^{•}({\partial }_{x}f,v)\right)\right),G_x\left(w\right)\rangle }^{\mathbb{A}}\hfill \\ \hfill & -{\langle G_x^{-1}\left({\Pi }_x\left({\overline{\overline{\Gamma }}}_x({\Pi }_x\left({\partial }_{x}f\right),v)\right)\right),G_x\left(w\right)\rangle }^{\mathbb{A}}\hfill \\ \hfill =& {\langle {\Pi }_x\left({\partial }_x^{2}f\left(v\right)\right),w\rangle }^{\mathbb{A}}\hfill \\ \hfill & +{\langle {\Pi }_x\left({\Pi }_x^{•}({\partial }_{x}f,v)\right),w\rangle }^{\mathbb{A}}\hfill \\ \hfill & -{\langle {\Pi }_x\left({\overline{\overline{\Gamma }}}_x({\Pi }_x\left({\partial }_{x}f\right),v)\right),w\rangle }^{\mathbb{A}}\hfill \\ \hfill =& {\langle {\partial }_x^{2}f\left(v\right),w\rangle }^{\mathbb{A}}+{\langle {\Pi }_x^{•}({\partial }_{x}f,v),w\rangle }^{\mathbb{A}}-{\langle {\overline{\overline{\Gamma }}}_x({\Pi }_x\left({\partial }_{x}f\right),v),w\rangle }^{\mathbb{A}}.\hfill \end{array}$$

(503)

The first inner product is symmetric in v and w because the ambient Hessian is self-adjoint, the second term is self-adjoint by the property (481) and the last term is symmetric in v and w because the Christoffel form commutes with the inner product.

13.2. A Newton-like Optimization Algorithm

Given a function $f:\mathbb{M}\to \mathbb{R}$, we may define a quadratic approximation at a point $x\in \mathbb{M}$ in the direction $v\in T_x\mathbb{M}$ as

$$Q_f(x,v):=f\left(x\right)+{\langle {\mathrm{grad}}_{x}f,v\rangle }_x+\frac{1}{2}{\langle H_{x}f\left(v\right),v\rangle }_x.$$

(504)

As often recalled, the inner products may be evaluated in terms of ambient metrics and metric kernel. Therefore, the change in the value of the function f may be evaluated as

$$Q_f(x,v)-f\left(x\right)={\langle G_x\left({\mathrm{grad}}_{x}f\right),v\rangle }^{\mathbb{A}}+\frac{1}{2}{\langle G_x\left(H_{x}f\left(v\right)\right),v\rangle }^{\mathbb{A}}.$$

(505)

In optimization, it is fundamental to find a direction of maximal change from a given point, which may be determined by solving the following problem in v:

$$\frac{\partial }{\partial v}(Q_f(x,v)-f\left(x\right))=0.$$

(506)

A consequence of a property shown in the previous subsection, namely that the Riemannian Hessian is self-adjoint, is that

$${\partial }_v{\langle H_{x}f\left(v\right),v\rangle }_x=2H_{x}f\left(v\right).$$

(507)

Recalling that $G_x\circ H_{x}f$ is linear, the Equation (506) hence reads

$$G_x\left({\mathrm{grad}}_{x}f\right)+G_x\left(H_{x}f\left(v\right)\right)=0.$$

(508)

Recalling that $G_x$ is invertible, the above relationship simplifies to ${\mathrm{grad}}_{x}f+H_{x}f\left(v\right)=0$, which leads to the optimal direction

$$v=-{\left(H_{x}f\right)}^{-1}\left({\mathrm{grad}}_{x}f\right).$$

(509)

Notice that $v\in T_x\mathbb{M}$, and hence the result is consistent to what was expected. It is the case to notice that the inverse of the Hessian may be hard to express in closed form; hence, it is sometimes easier to just set up the (linear) equation $H_x\left(v\right)=-{\mathrm{grad}}_{x}f$ and solve it by any linear-algebra tool.

The optimal direction may be exploited in a Newton-like optimization algorithm as follows:

$$\left\{\begin{array}{c}{v}_k=-{\left(H_{x_k}f\right)}^{-1}\left({\mathrm{grad}}_{x_k}f\right),\hfill \\ x_{k+1}={exp}_{x_k}\left(h{v}_k\right),\hfill \end{array}\right.$$

(510)

where $k\ge 0$ is an integer step-counter, $h>0$ is a step-size and $x_0$ denotes an initial guess.

14. Conclusions

The present paper focused on manifold calculus to describe the system-theoretic properties of non-linear systems whose states belong to curved manifolds and was conceived as the first part of a longer tutorial in manifold calculus.

In particular, the present tutorial paper focuses mainly on mathematical definitions and calculation rules, expressed in the language of embedded manifold calculus, that form a knowledge base to develop further concepts and applications. A number of manifolds of interest in applications were covered and a number of examples clarified some collateral, yet interesting, aspects.

A section of the present paper focuses on the design of a manifold-type system synchronization algorithm based on feedback control and on developing numerical methods tailored to curved manifolds to implement such systems and algorithms.

Since the present contribution aimed to lay out the basic concepts in manifold calculus and Lie group theory, it only covers the basis of first-order dynamical systems and proportional type control. Second-order systems and their control require advanced notions, such as covariant derivation. Such advanced notions will be covered in a forthcoming contribution.