Probability Distributions Approximation via Fractional Moments and Maximum Entropy: Theoretical and Computational Aspects

Abstract

In the literature, the use of fractional moments to express the available information in the framework of maximum entropy (MaxEnt) approximation of a distribution F having finite or unbounded positive support, has been essentially considered as a computational tool to improve the performance of the analogous procedure based on integer moments. No attention has been paid to two formal aspects concerning fractional moments, such as conditions for the existence of the maximum entropy approximation based on them or convergence in entropy of this approximation to F. This paper aims to fill this gap by providing proofs of these two fundamental results. In fact, convergence in entropy can be involved in the optimal selection of the order of fractional moments for accelerating the convergence of the MaxEnt approximation to F, to clarify the entailment relationships of this type of convergence with other types of convergence useful in statistical applications, and to preserve some important prior features of the underlying F distribution.

1. Introduction

In statistical estimation, one often wants to guess an unknown probability distribution F, given certain observations based on it. There are generally infinitely many distributions consistent with the available data, and the question of which of these to select is an important one in many fields. The notion of entropy has been proposed as a remarkable tool for performing this choice. More precisely, the principle of maximum entropy was established by [1,2] as a tool for inference under uncertainty and consists of finding the most suitable probability distribution under the available information. As Jaynes [1] expressed it, the resulting MaxEnt distribution “… is the least biased estimate possible on the given information”. In summary, the MaxEnt method dictates what are the most “reasonable and objective” distribution subject to given constraints expressing the available information concerning the data generating mechanism: the analytical form of that constraints are chosen to look at the features of the distribution that we want to preserve in the MaxEnt approximation to guarantee its capability of modeling specific features of F.

It is a common choice to express the constraints in terms of expectations of some functions $g_j$ of X, i.e.,

$$\mathit{E}\left[g_j\left(X\right)\right]={\int }_{\mathcal{U}}{g}_j\left(x\right)f\left(x\right)dx=c_j,j=1,2,\dots ,n$$

(1)

and the resulting maximum entropy distribution (better, its density) emerges by maximizing the Shannon (differential) entropy

$$h_f=-{\int }_{\mathcal{U}}f\left(x\right)\ln f\left(x\right)dx$$

(2)

under a set of constraints (1) using calculus of variations and Lagrange’s multipliers method. The general solution, assuming an arbitrary set of $n+1$ constraints $g_j$, is given by [1]

$$f_n\left(x\right)=exp\left\{-{\lambda }_0-\sum _{j=1}^n{\lambda }_j{g}_j\left(x\right)\right\}$$

(3)

where ${\lambda }_1,\dots ,{\lambda }_n$, are the Lagrange multipliers linked to the set of adopted constraints (1) while the multiplier ${\lambda }_0$ guarantees the legitimacy of the distribution and $\mathit{E}\left(g_j\left(X\right)\right)$, $j=1,2,\dots ,n$ are the characterizing moments of the distribution. It becomes clear that the resulting maximum entropy distribution having density (3) is uniquely driven by the choice of the imposed constraints. This implies that this choice is the most important and determinative part of the MaxEnt method. In the end, the form of the MaxEnt approximation (3) of f is problem-dependent that is, its analytical form depends on the choice of the constraints $g_j$ describing the features of the distribution F that must be preserved in the approximation process.

As we said above, in constructing a density with the MaxEnt methodology, for practical purposes only partial information can be used. However, this does not preclude that we must have at our disposal some physical knowledge of the underlying problem that amounts to

• The solution to the problem is unique.
• The entire moment curve from which to pick up a finite number of arbitrary fractional moments is known.

The aim of the paper is focused on the opportunity offered by expressing the system of constraints (1) by fractional moments. This choice gives back a great flexibility to model a wide class of problems where the traditional integer moment constraints may be inefficient in recovering the available information. The theorem of entropy convergence stated for fractional moments setup and related other modes of convergence, offer a formal basis for the optimal choice of number and order of the fractional moments to be involved in the MaxEnt approximation procedures. In the paper we will stress the fact that theoretical and numerical aspects are inextricably linked to each other, justifying why both theoretical and computational aspects must be treated simultaneously in the paper. More precisely, in Section 2 some properties of fractional moments motivating their use in MaxEnt reconstruction of distribution will be discussed, in Section 3 some basics about Tchebycheff systems (T-systems, for brevity) will be recalled and using this tool, the existence and convergence in entropy of the MaxEnt distribution constrained by fractional moments will be proven in Section 4.1 and in Section 4.2, respectively. Finally, in Section 5 two crucial results concerning the optimal choice of the orders $\alpha$’s and the optimal number n of the fractional moments both based on the convergence in entropy of the MaxEnt distribution will be presented.

2. The Role of Fractional Moments in MaxEnt Setup

Constraints (1) expressed by integer moments play an important role in the inverse Hausdorff and Stieltjes classical moment problem that consists in determining an unknown probability mass or density function f corresponding to the distribution F from the knowledge of the sequence of its integer moments $m_j=\mathit{E}\left(X^j\right)={\int }_{\mathcal{U}}{x}^{j}f\left(x\right)dx$, $j\ge 1$, $m_0=1$, where $\mathcal{U}=[0,1]$ in Hausdorff case or $\mathcal{U}=[0,\infty )$ in Stieltjes case.

For practical purposes, if only n prefixed moments are taken into account to express the available information about the distribution, then many different (even an infinity!) probability distributions could be compatible with that information and non-uniqueness of the distribution recovered from them follows immediately. Hence, the question: What probability distribution is the best and with respect to what criterion? The answer follows naturally from Jaynes’ principle [1]: from the set of all probability distributions compatible with the n prefixed moments, choose the one that maximizes Shannon’s entropy that is, the so-called MaxEnt distribution.

The MaxEnt approximation of f constrained by first $n+1$ integer moments that is, $\mathit{E}\left[g_j\left(X\right)\right]=\mathit{E}\left(X^j\right)=m_j,j=1,\dots ,n$ where n is arbitrary large and $m_0=1$ being any density a normalized function, comes immediately from (3) that is,

$$f_n\left(x\right)=exp\left\{-{\lambda }_0-\sum _{j=1}^n{\lambda }_j{x}^j\right\}$$

(4)

where ${\lambda }_j,j=1,\dots ,n$, are the Lagrange multipliers linked to the set of adopted constraints while the multiplier ${\lambda }_0$ guarantees the legitimacy of the distribution. Widely known references are the books [3,4] and more recently [5]. These sources contain comprehensive details about a series of remarkable results paving the progress in moment problems for more than a century. Theoretical and computational aspects are inextricably linked to each other.

It is a well-known fact that in a determinate moment problem, the sequence of integer moments ${\left\{m_j\right\}}_{j=0}^{\infty }$ carries all the information concerning the distribution F; hence it may happen that the moments of high order also contain a considerable amount of it as in the case of asymmetric or heavy-tailed distributions. But it is also well known that the moment problem becomes ill-conditioned when the number (hence, the order) n of moments increases and to avoid numerical instability due to the ill-conditioning of Hankel matrices only the moments of small order n are involved. Neglecting higher-order moments implies losing the information carried by them with consequences on the quality of $f_n$ as an approximation of f. Furthermore, if the first few moments are not informative with respect to the distribution, the situation is even worse with $f_n$ being a definitely bad approximation of f.

For a practical example, Ref. [6] discuss the role played by the constraints choice in modeling probability distributions via the MaxEnt method for complex geophysical processes concluding that the usual choice based on integer moments in virtue of their physical meaning cannot be able, both for theoretical and empirical, i.e., data-driven, reasons to describe the relevant geophysical features that have to be preserved because characterizing the distribution (for more details, see the mentioned paper pp. 52–53).

At this point, a fundamental question is how to reformulate the MaxEnt solution of the moment problem in a suitable way to permit a reliable and efficient approximation through $f_n$ of the target density f. Or, equivalently, how to choose the (optimal) analytical form of the set of the MaxEnt constraints?

To try to find an answer to this crucial question, combining (2) and (3), the entropy of $f_n$ is given by

$$h_{f_n}=\sum _{j=0}^n{\lambda }_j{m}_j\ge h_f.$$

(5)

and point out that the quantity

$$h_{f_n}-h_f\ge 0$$

(6)

is a measure of residual uncertainty about the distribution of the random variable X associated with the MaxEnt approximation $f_n$ of f and corresponds to the maximum residual entropy-or equivalently the minimum information gain-associated to the knowledge of the first n moments of F not captured by (4).

When integer moments are used as constraints in a MaxEnt procedure, due to the mechanical choice of integer moments, the rate of reduction of the residual entropy (6) is very slow and becomes more and more negligible as the number n of moments increases. Consequently, it is urgent to look for a class of alternative approximants with faster convergence of $h_{f_n}$ to $h_f$: the class of MaxEnt distributions constrained by fractional moments

$$m_{{\alpha }_j}=\mathit{E}\left(X^{{\alpha }_j}\right)={\int }_{\mathcal{U}}{x}^{{\alpha }_j}f\left(x\right)dx,{\alpha }_j\in {IR}^{+}$$

(7)

represents a natural alternative to (4) where a proper choice of number n and $\alpha$ exponents built on the convergence in entropy theorem of Section 4.2, allows us to control the residual uncertainty reduction and at the end, to accelerate the rate of the convergence of $f_n$ to f, mitigating the effects of ill-conditioning due to the large value of n used. Indeed, since any fractional moment $m_{\alpha }$ can be obtained as a function of many (as computationally feasible) integer moments $m_j$ ([7,8] for details), the information available in the sequence of integer moments can be squeezed into a few fractional moments: for example, in case of heavy-tailed distributions where integer moments of high order are required, the use of a few fractional moments permit to avoid (or mitigate) ill-conditioning without losing the tail information which is crucial to model, for example, the risk of extreme events or their predictability in a computationally tractable environment. Refs. [9,10] give an application of the fractional moment method in the structural reliability analysis which is typically based on a model that describes the response, such as maximum deformation or stress, as a function of several random variables and they base the derivation of that model on the MaxEnt principle where constraints are specified in terms of the fractional moments, in place of commonly used integer moments to avoid the well known ill-conditioning problem, studying the numerical accuracy and efficiency of the proposed method. From the life-cycle perspective, Ref. [11] observes that probabilistic lifetime modeling of an engineering system provides important information for risk assessment of the system by evaluating the mean-time to failure, survival probability, dynamic hazard rate, and among others. This can be conducted again using fractional moments into the MaxEnt technique to approximate the distribution of interest. Further, in virtue of the flexibility due to the continuous nature of their order $\alpha$, fractional moments represent a valid alternative when physical principles of momentum are invalid and consequently, the use of integer moments to express the MaxEnt constraints reveals to be improper as it happens in many geophysical processes like daily rainfall distribution [6] or tree diameter distribution modeling [12].

Looking for more practical reasons motivating the use of fractional moments in addition to computational feasibility issues, it is interesting to note that sometimes the available information could be better exploited if the search for its optimal summaries took place on the entire moment curve rather than on a predetermined sequence of equispaced points (i.e., integer moments). For example, a large number of common families of probability distributions largely used in reliability and risk theory such as Gamma, Pareto, Rayleigh, and Lognormal to name a few, belong to the exponential family having logarithmic characterizing moments as shown in

Table 1. Some families of distributions and their characterizing moments.

Families of Distribution	Density	Characterizing Moments
$\mathrm{Gamma}(\gamma ,\beta )$	$\frac{x^{\gamma -1}exp\{-x/\beta \}}{\mathrm{\Gamma }\left(\gamma \right){\beta }^{\gamma }}{\mathit{1}I}_{{\mathbb{R}}^{+}}\left(x\right)$	$\mathit{E}\left[X\right],\mathit{E}\left[\ln \right(X\left)\right]$
$\mathrm{Pareto}(\gamma ,k)$	$\gamma k^{\gamma }{x}^{-(\gamma -1)}{\mathit{1}I}_{[k,+\infty )}\left(x\right)$	$\mathit{E}\left[\ln \right(X\left)\right]$
$\mathrm{Lognormal}(\mu ,\sigma )$	$\frac{exp\{-{[\ln \left(x\right)-\mu ]}^2/2{\sigma }^2\}}{\sqrt{2\pi }\sigma x}{\mathit{1}I}_{{\mathbb{R}}^{+}}\left(x\right)$	$\mathit{E}\left[\ln \left(X\right)\right],\mathit{E}\left[{\ln }^2\left(X\right)\right]$
$\mathrm{Rayleigh}\left({\sigma }^2\right)$	$\frac{xexp\{-(x^2/2{\sigma }^2)\}}{{\sigma }^2}{\mathit{1}I}_{{\mathbb{R}}^{+}}\left(x\right)$	$\mathit{E}\left[X^2\right],\mathit{E}\left[\ln \left(X\right)\right]$

(recalling that $x^{\alpha }=e^{\alpha \ln \left(x\right)})$.

It is possible to show that these families of distributions can be considered MaxEnt distributions with fractional moments as characterizing moments, in the sense that the analytic form of the latter is appropriate to capture the relevant (that is, characterizing) information and features of the corresponding distribution. For example, in the Lognormal case, the characterizing moments are $\left(\mathit{E}\left[\ln \left(X\right)\right],\mathit{E}\left[{\ln }^2\left(X\right)\right]\right)$, respectively. Then, by the known relationships

$$\underset{\alpha \to 0}{\lim }\frac{x^{\alpha }-1}{\alpha }=\ln \left(x\right)\mathrm{and}\underset{\alpha \to 0}{\lim }{\left(\frac{x^{\alpha }-1}{\alpha }\right)}^2=\underset{\alpha \to 0}{\lim }\frac{x^{2\alpha }-2x^{\alpha }+1}{{\alpha }^2}={\ln }^2\left(x\right)$$

it follows that the Lognormal density can be reconsidered as a MaxEnt one having $\{\mathit{E}\left(X^{\alpha }\right)$, $\mathit{E}\left(X^{2\alpha }\right)\}$, $\alpha \to 0$, as characterizing fractional moments.

The same line of reasoning and related results hold true if we consider a random sample $(X_1,X_2,\dots ,X_N)$ and the associated sample fractional moments

$${\widehat{m}}_{{\alpha }_j}=\frac{1}{N}\sum _{i=1}^N{X}_i^{{\alpha }_j},{\alpha }_j\in {IR}^{+}$$

(8)

to summarize the sample information needed for the MaxEnt estimation of the density f [7]. In this setup, the MaxEnt estimate $f_n$ represents a genuine non-parametric estimate of f where the constraints of appropriate number and order expressed by (8) represent the features of the distribution of X that must be preserved.

For assessing the feasibility of the moment problem solution based on fractional moments, three aspects must be now considered: the first one concerns the existence of the MaxEnt approximation based on fractional moments as a tool to express the constraints set, the second one consists in finding a formal proof of the convergence in entropy to f of the sequence of MaxEnt "fractional" approximation (Equation (10) below). The third aspect concerns the choice of the number n and the set of the orders ${\alpha }_j$ for $j=1,2,\dots ,n$ of the fractional moments $m_{{\alpha }_j}$. Convergence in entropy of the MaxEnt approximation (10) $f_n$ to f guarantees the residual uncertainty $h_{f_n}-h_f$ about the distribution of the random variable X associated with the approximation $f_n$ of f is minimal. And, for fixed n, it is natural to base the choice of the fractional order ${\alpha }_j,j=1,2,\dots ,n$, looking for the $\alpha$’s values that minimize the residual uncertainty (6) in the framework established by two important results due to [13].

More precisely, Lin’s Theorems 1 and 2 based on asserting that an analytic function on the right half complex plane is completely determined by its values on a sequence of points having an accumulation point there, guarantee that the fractional moments corresponding to the posed restrictions on the exponents to catch the aspects of the process that must be preserved, still characterize the underlying distribution. Specifically, these theorems are:

Theorem 1 (Lin (1992), Thm. 1).

A positive r.v. X is uniquely characterized by an infinite sequence of positive fractional moments ${\left\{m_{{\alpha }_j}\right\}}_{j=1}^{\infty }$ with distinct exponents ${\alpha }_j\in (0,{\alpha }^{*})$, $m_{{\alpha }^{*}}<\infty$, for some ${\alpha }^{*}>0$.

and

Theorem 2 (Lin (1992), Thm. 2).

If X is a r.v. assuming values from a bounded interval $[0,1]$ and ${\left\{{\alpha }_j\right\}}_{j=1}^{\infty }$ an infinite sequence of positive and distinct numbers satisfying

$$\underset{j\to \infty }{lim}{\alpha }_j=0\mathrm{and}\sum _{j=1}^{\infty }{\alpha }_j=+\infty$$

then the sequence of moments ${\left\{m_{{\alpha }_j}\right\}}_{j=1}^{\infty }$ characterizes X.

The following sections provide formal proofs and results related to each of the three aspects mentioned. But before proceeding, let us briefly recall an important technical result, which plays a pivotal role in performing the proofs.

3. A Reminder about T-Systems

T-systems represent a technical tool that plays a crucial role in proving both the existence and convergence in entropy of the MaxEnt approximation $f_n$ of f. For this reason, we will revisit briefly their main aspects, considering the two cases $X\in [0,\infty )$ and $X\in [0,1]$, separately. In the sequel, notations and results are borrowed from [14,15] where the T-systems are extensively investigated including general functions ${\left\{u_j\left(t\right)\right\}}_{j=0}^n$ on abstract set $\mathcal{E}$.

• $X\in \mathcal{U}=[0,\infty )$.

• The starting point is to consider that the set of continuous linearly independent real-valued functions ${\left\{u_j\left(t\right)\right\}}_{j=0}^n$, defined on the interval $\mathcal{U}=[0,\infty )$, constitutes a T-system of order n if any polynomial

  $$P\left(t\right)=\sum _{j=0}^n{a}_j{u}_j\left(t\right),\mathrm{with}\sum _{j=0}^n{a}_j^2>0$$

  has no more than n zeros on $[0,\infty )$. Equivalently, it is readily seen that ${\left\{u_j\left(t\right)\right\}}_{j=0}^n$ is a T-system if and only if the determinants of order $(n+1)$

  $$\det \Vert u_0\left(t\right),u_1\left(t\right),\dots ,u_n{\left(t\right)\Vert }_0^n=:\left|\begin{array}{cccc}{u}_0\left(t_0\right)& u_0\left(t_1\right)& \dots & u_0\left(t_n\right)\\ u_1\left(t_0\right)& u_1\left(t_1\right)& \dots & u_1\left(t_n\right)\\ ⋮& ⋮& & ⋮\\ u_n\left(t_0\right)& u_n\left(t_1\right)& \dots & u_n\left(t_n\right)\end{array}\right|$$

  are strictly positive for any choice of (distinct) pairs of elements $0\le t_0<t_1,\dots <t_n$ in $[0,\infty )$. According with the above definition the special set ${\{u_j\left(t\right)=t^{{\alpha }_j}\}}_{j=0}^n$ we are interested in, with distinct $0={\alpha }_0<{\alpha }_1,\dots <{\alpha }_n$, is a T-system having the properties

  (a)

  $u_j\left(t\right)=t^{{\alpha }_j}>0$ for each $0\le j\le n$

  (b)

  ${\lim }_{t\to \infty }\frac{t^{{\alpha }_j}}{t^{{\alpha }_n}}=0$ for each $j=0,\dots ,n-1$

  (c)

  if the set ${\{u_j\left(t\right)=t^{{\alpha }_j}\}}_{j=0}^n$ is a T-system, then ${\{u_j\left(t\right)=t^{{\alpha }_j}\}}_{j=0}^{n+1}$ is it too.

• The space ${\mathcal{M}}_{n+1}$ of moments, given by the convex hull generated by the points ${\left\{t^{{\alpha }_j}\right\}}_{j=0}^n$ has a nonempty interior. This set is convex but has a complex geometry. A good deal of the geometry of the classical moment spaces induced by the special T-system $\{1,t,t^2,...,t^n\}$ can be generalized to the case of the investigated T-system ${\{u_j\left(t\right)=t^{{\alpha }_j}\}}_{j=0}^n$. If the sequence of prescribed moments ${\left\{m_{{\alpha }_j}\right\}}_{j=0}^n$ is an inner point of ${\mathcal{M}}_{n+1}$ then there are uncountably many probability measure $d\sigma \left(t\right)$ having such prescribed moments, one of them being $d\sigma \left(t\right)=f_n\left(t\right)dt$. Elsewhere, if the sequence of prescribed moments ${\left\{m_{{\alpha }_j}\right\}}_{j=0}^n$ belongs to $\partial {\mathcal{M}}_{n+1}$, the boundary of ${\mathcal{M}}_{n+1}$, a unique measure supported on a finite set of points exists (the so-called lower principal representation) and the determinant of the below-defined Gram matrix $G_n$ becomes zero. For an arbitrary n, let $0={\alpha }_0<{\alpha }_1<\dots <{\alpha }_n$. For notational convenience we set

  $$m_{{\alpha }_i,{\alpha }_j}=\mathit{E}\left(X^{{\alpha }_i}{X}^{{\alpha }_j}\right)={\int }_{\mathcal{U}}{t}^{{\alpha }_i}{t}^{{\alpha }_j}f\left(t\right)dt={\int }_{\mathcal{U}}{t}^{{\alpha }_i+{\alpha }_j}f\left(t\right)dt.$$
• Let us now consider the probability measure $d\sigma \left(t\right)=f_n\left(t\right)dt$. Then $t^{{\alpha }_j}\in L_{d\sigma }^2\left(\mathcal{U}\right)$, where, as usual

  $$L_{d\sigma }^2\left(\mathcal{U}\right)=\{t^{{\alpha }_j}:{\int }_{\mathcal{U}}{t}^{2{\alpha }_j}d\sigma \left(t\right)={\int }_{\mathcal{U}}{t}^{2{\alpha }_j}{f}_n\left(t\right)dt<+\infty \}$$
• Thus the matrix $G_n={\left[m_{{\alpha }_i,{\alpha }_j}\right]}_{i,j=0}^n$ is the positive definite Gram matrix.
• The following Markov-Krein theorem ([14] Thm 5.1, p. 157; [15] Thm 1.1, p. 177) is fundamental to prove the convergence in entropy of the MaxEnt distribution: here we adapt it to fractional moments.

  Theorem 3 (Markov–Krein theorem).

  Given values of the first fractional moments ${\left\{m_{{\alpha }_j}\right\}}_{j=0}^n$$\in Int\left({\mathcal{M}}_{n+1}\right)$ so that the Gram matrix $G_n$ be positive definite, the integral ${\int }_{\mathcal{U}}{u}_{n+1}\left(t\right)d\sigma \left(t\right)$ over all the distributions $\sigma \left(t\right)$ having the assigned moments ${\left\{m_{{\alpha }_j}\right\}}_{j=0}^n$, has a minimum value $m_{{\alpha }_{n+1}}^{-}$ where

  $$m_{{\alpha }_{n+1}}^{-}={\int }_{\mathcal{U}}{u}_{n+1}\left(t\right)d\underline{\sigma }\left(t\right)$$

  (9)

  The corresponding measure $\underline{\sigma }$, under the form of a sum weighted Dirac delta function for which is uniquely determined, is the so-called lower principal representation. Furthermore the point $\{m_{{\alpha }_0},\dots ,m_{{\alpha }_n},m_{{\alpha }_{n+1}}^{-}\}$ belongs to $\partial {\mathcal{M}}_{n+2}$, the boundary of ${\mathcal{M}}_{n+2}$.

2.

$X\in \mathcal{U}=[0,1]$.

• In this case the procedure proposed for $X\in [0,\infty )$ runs more or less similarly since the involved functions ${\{u_j\left(t\right)=t^{{\alpha }_j}\}}_{j=0}^n$, $t\in [0,1]$ are T-systems too and an analogous Markov-Krein theorem ([14] Thm 1.1, p. 80; [15] Thm. 1.1, p. 109) is available. It should only be recalled that, in analogy with Theorem 3, given ${\left\{m_{{\alpha }_j}\right\}}_{j=0}^n\in Int\left({\mathcal{M}}_{n+1}\right)$, the moment $m_{{\alpha }_{n+1}}$ admits minimum and maximum value $m_{{\alpha }_{n+1}}^{-}$, $m_{{\alpha }_{n+1}}^{+}$, respectively, where

  $$\left(\genfrac{}{}{0pt}{}{m_{{\alpha }_{n+1}}^{+}}{m_{{\alpha }_{n+1}}^{-}}\right)={\int }_{\mathcal{U}}{u}_{n+1}\left(t\right)d\left(\genfrac{}{}{0pt}{}{\overline{\sigma }}{\underline{\sigma }}\right)$$

  Here, the corresponding measures $\underline{\sigma }$ and $\overline{\sigma }$ under the form of a sum-weighted Dirac delta function are uniquely determined and they are the so-called lower and upper principal representation, respectively, and the points $\{m_{{\alpha }_0},\dots ,m_{{\alpha }_n},m_{{\alpha }_{n+1}}^{\pm }\}\in \partial \left({\mathcal{M}}_{n+2}\right)$.

4. MaxEnt Solution of the Fractional Moment Problem

Once in both cases $X\in [0,1]$ and $X\in [0,\infty )$ the moment curve $m_X\left(\alpha \right)={\int }_{\mathcal{U}}{t}^{\alpha }f\left(t\right)dt$ has been obtained, the probability distribution constrained by fractional moments can be estimated (approximated) through the MaxEnt technique, which is essentially an extension of the commonly used integer moment-based MaxEnt procedure.

Note, that given a finite collection of (population or sample) fractional moments ${\left\{m_{{\alpha }_j}\right\}}_{j=0}^n$, with ${\alpha }_0=0$, the corresponding MaxEnt solution for f is

$$f_n\left(x\right)=exp\left\{-\sum _{j=0}^n{\lambda }_j{x}^{{\alpha }_j}\right\}$$

(10)

where the ${\lambda }_j$ are such that the (fractional) constraints

$${\int }_{\mathcal{U}}{x}^{{\alpha }_j}{f}_n\left(x\right)dx=m_{{\alpha }_j},j=0,\dots ,n$$

(11)

are satisfied, and $f_n$ depends on the $m_{{\alpha }_j}$ (thus on the ${\alpha }_j$) through the ${\lambda }_j$. Note, that in $[0,\infty )$ case ${\lambda }_n$ must take positive values in $[0,\infty )$ to guarantee $f_n$ integrability. The MaxEnt approximation $f_n$ of f has entropy

$$h_{f_n}=\sum _{j=0}^n{\lambda }_j{m}_{{\alpha }_j}$$

(12)

Here $({\lambda }_0,\dots ,{\lambda }_n)$ is the vector of Lagrange multipliers: if it is possible to determine Lagrange multipliers from the constraints ${\left\{m_{{\alpha }_j}\right\}}_{j=0}^n$, then the moment problem admits solution and $f_n$ is MaxEnt approximation of f which is unique in $\mathcal{U}$ due to strict concavity of (12). In this setup, two fundamental theoretical questions must be now addressed: the existence of the MaxEnt distribution $F_n$ and its convergence in entropy to F. The last two are crucial to exploit in real-world applications; the MaxEnt technique aims to recover the distribution F by the available information on X here summarized by a proper set of constraints expressed in terms of fractional moments, just to take into account what is discussed in Section 2.

4.1. Existence of MaxEnt Distribution

For the MaxEnt distribution existence, a close and evident analogy between the two cases integer and fractional moments there exist, being both the set of functions ${\{u_j\left(t\right)=t^{{\alpha }_j}\}}_{j=0}^n$ and ${\{u_j\left(t\right)=t^j\}}_{j=0}^n$ T-systems. The proof simply replaces Hankel matrices with Gram matrices above defined.

• Suppose that X has unbounded support, $\mathcal{U}={IR}^{+}$, and the first $n+1$ moments ${\left\{m_{{\alpha }_j}\right\}}_{j=0}^n\in Int\left({\mathcal{M}}_{n+1}\right)$ have been assigned, ${\lambda }_n\ge 0$ has to be to guarantee integrability of $f_n$. In analogy with the case of integer moments, being both integer and fractional moments T-systems, the above nonnegativity condition on ${\lambda }_n$ is crucial and renders the moment problem solvable only under certain restrictive assumptions on the prescribed moment vector ${\left\{m_{{\alpha }_j}\right\}}_{j=0}^n$. Consider (11) with n replaced by $n+1$, the first $n+1$ moments ${\left\{m_{{\alpha }_j}\right\}}_{j=0}^n$ held constant, whilst $m_{{\alpha }_{n+1}}$ varies continuously, so that the Lagrange multipliers ${\lambda }_j={\lambda }_j\left(m_{{\alpha }_{n+1}}\right)$, $j=0,\dots ,n+1$ are depending on $m_{{\alpha }_{n+1}}$. Differentiating both sides with respect to $m_{{\alpha }_{n+1}}$ one has

  $$G_{n+1}·{\left[\frac{d{\lambda }_0}{d{m}_{{\alpha }_{n+1}}},\dots ,\frac{d{\lambda }_{n+1}}{d{m}_{{\alpha }_{n+1}}}\right]}^{\prime }={[0,\dots ,0,-1]}^{\prime }$$

  (13)

  where ′ denotes the transpose. From $G_{n+1}$ symmetric and positive definite it follows

  $$\begin{array}{c}\hfill 0<\left[\frac{d{\lambda }_0}{d{m}_{{\alpha }_{n+1}}},\dots ,\frac{d{\lambda }_{n+1}}{d{m}_{{\alpha }_{n+1}}}\right]·G_{n+1}·{\left[\frac{d{\lambda }_0}{d{m}_{{\alpha }_{n+1}}},\dots ,\frac{d{\lambda }_{n+1}}{d{m}_{{\alpha }_{n+1}}}\right]}^{\prime }=\\ \hfill =\left[\frac{d{\lambda }_0}{d{m}_{{\alpha }_{n+1}}},\dots ,\frac{d{\lambda }_{n+1}}{d{m}_{{\alpha }_{n+1}}}\right]·{\left[0,\dots ,0,-1\right]}^{\prime }=-\frac{d{\lambda }_{n+1}}{d{m}_{{\alpha }_{n+1}}}\end{array}$$

  (14)

  Then $\frac{d{\lambda }_{n+1}}{d{m}_{{\alpha }_{n+1}}}<0$ and ${\lambda }_{n+1}$ monotonic decreasing function. MaxEnt machinery leads us to consider a further quantity

  $$m_{{\alpha }_{n+1}}^{+}={\int }_{\mathcal{U}}{t}^{{\alpha }_{n+1}}{f}_n\left(t\right)dt$$

  (15)

  with, in general, $m_{{\alpha }_{n+1}}^{+}\ne m_{{\alpha }_{n+1}}$.

• From now on, for the sake of brevity, in the arguments of $f_{n+1}$ and $h_{f_{n+1}}$ we will mention only those that take continuously varying values.

  (i)

  Assume $f_n$ exists. Once ${\left\{m_{{\alpha }_j}\right\}}_{j=0}^{n+1}$ are assigned and $m_{{\alpha }_{n+1}}$ varies continuously, combine together the following facts: ${\lambda }_{n+1}\left(m_{{\alpha }_{n+1}}\right)$ is a monotonic decreasing function, $f_{n+1}\left(m_{{\alpha }_{n+1}}^{+}\right)=f_n$ and take into account (9) and (15). One concludes that, if $f_n$ exists, the necessary and sufficient condition for the existence of $f_{n+1}$ is $m_{{\alpha }_{n+1}}^{-}<m_{{\alpha }_{n+1}}\le m_{{\alpha }_{n+1}}^{+}$, in analogy with the past investigated case concerning integer moments ([16], Appendix A).

  (ii)

  Assume $f_n$ does not exist. In such a case ${\lambda }_{n+1}>0$. Indeed, if it were ${\lambda }_{n+1}=0$ then we would have both $m_{{\alpha }_{n+1}}=m_{{\alpha }_{n+1}}^{+}$ and then $f_{n+1}=f_n$, contradicting the fact that $f_n$ does not exist. Consequently, $f_{n+1}$ exists for every set ${\left\{m_{{\alpha }_j}\right\}}_{j=0}^{n+1}\in Int\left({\mathcal{M}}_{n+2}\right)$. For practical purposes, $f_n$ doesn’t exist, both $f_{n-1}$ and $f_{n+1}$ exist. We can state that the problem of the non-existence of the MaxEnt density can be easily bypassed.

• Collecting together the items (i) and (ii) we conclude that the existence of $f_n$ is iteratively and numerically determined, starting from $f_1$ which exists.
• Proving the conditions of existence of the MaxEnt distribution we remarked the close analogy between the cases of fractional moments and integer moments. It is reasonable to expect similar analogies to arise also in the case in which an entropy value is to be attributed to the density in the case in which it does not exist so that the sequence of entropies ${\left\{h_{f_n}\right\}}_{n=1}^{\infty }$ is defined for every n. The issue was addressed in ([16], Thm. 1) and taking into account of the laboriousness of the proof, we limit ourselves to illustrating the tools involved and the results obtained.
• Some relevant facts need to be collected together. Since MaxEnt density $f_n$ does not exist, both $f_{n-1}$ and $f_{n+1}$ exist with entropies $h_{f_{n-1}}$ and $h_{f_{n+1}}$, respectively. Introduce now the following class of densities all having the same first moments ${\left\{m_{{\alpha }_j}\right\}}_{j=0}^n$

  $${\mathcal{C}}_n=:\left\{f\ge 0|{\int }_{\mathcal{U}}{x}^{j}f\left(x\right)dx=m_{{\alpha }_j},j=0,\dots ,n\right\}$$

  (16)
• In particular, we direct our attention to the density $f_{n+1}=f_{n+1}\left(m_{{\alpha }_{n+1}}\right)\in {\mathcal{C}}_n$, which thanks to Theorem 4 exists for any value $m_{{\alpha }_{n+1}}>m_{{\alpha }_{n+1}}^{-}$. As in integer moments case, $f_n$ may not exist so that $h_{f_n}$ is meaningless ([16], Thm. 1) proved the relationship ${\lim }_{\left(m_{n+1}\right)\to \infty }{h}_{f_{n+1}}\left(m_{n+1}\right)=h_{f_{n-1}}$, from which ${\sup }_{f\in {\mathcal{C}}_n}{h}_f=h_{f_{n-1}}$, although the current use of MaxEnt fails (here the last recalled ${\mathcal{C}}_n$ is the analog of (16) with $m_{{\alpha }_j}$ replaced by $m_j$ ). Since the entropy is non-increasing as n increases, the latter equality enables us to set $h_{f_n}=h_{f_{n-1}}$, filling the gap left by the nonexistence of the density $f_n$. We reformulate such a result in terms of fractional moments as ${\lim }_{\left(m_{{\alpha }_{n+1}}\right)\to \infty }{h}_{f_{n+1}}\left(m_{{\alpha }_{n+1}}\right)=h_{f_{n-1}}$, from which ${\sup }_{f\in {\mathcal{C}}_n}{h}_f=h_{f_{n-1}}$. That leads us to conclude, whenever $f_n$ does not exist the missing entropy $h_{f_n}$ is replaced with $h_{f_{n-1}}$, so that the sequence of entropies ${\left\{h_{f_n}\right\}}_{j=1}^{\infty }$ is defined for every n.
• We can thus reformulate the conditions of existence according to integer moments by means

  Theorem 4.

  Once the moment set ${\left\{m_{{\alpha }_j}\right\}}_{j=0}^{n-1}\in Int\left({\mathcal{M}}_n\right)$ is prescribed, suppose $f_{n-1}$ exists with its n-th moment $m_{{\alpha }_n}^{+}={\int }_{\mathcal{U}}{t}^{{\alpha }_n}{f}_{n-1}\left(t\right)dt$.

(i)

If $m_{{\alpha }_n}\le m_{{\alpha }_n}^{+}$, then $f_n$ exists; conversely if $m_{{\alpha }_n}>m_{{\alpha }_n}^{+}$$f_n$ does not exist. Thus the existence of $f_n$ is iteratively (and numerically only) determined from $f_{n-1}$ starting from $f_1$ which exists.

(ii)

If $f_n$ does not exist, both $f_{n-1}$ and $f_{n+1}$ exist for every $m_{{\alpha }_{n-1}}>m_{{\alpha }_{n-1}}^{-}$ and $m_{{\alpha }_{n+1}}>m_{{\alpha }_{n+1}}^{-}$, respectively. In addition, $h_{f_n}=h_{f_{n-1}}$ can be set.

2.

Suppose now that $X\in [0,1]$: the procedure employed in the unbounded support case 1. runs similarly since the involved functions ${\{u_j\left(t\right)=t^{{\alpha }_j}\}}_{j=0}^n$, $t\in [0,1]$ are T-systems too and an analogous Markov-Krein theorem ([15], Thm. 1.1, p. 109) is available. It should only be recalled that, in analogy with the above Theorem 4, given ${\left\{m_{{\alpha }_j}\right\}}_{j=0}^{n-1}\in Int\left({\mathcal{M}}_n\right)$, the moment $m_{{\alpha }_n}$ admits minimum and maximum value $m_{{\alpha }_n}^{-}$ and $m_{{\alpha }_n}^{+}$, respectively. The corresponding measures $\sigma =\underline{\sigma }$ and $\sigma =\overline{\sigma }$ under the form of a sum weighted Dirac delta function are uniquely determined and they are the so-called lower and upper principal representation, respectively, and the points $\{m_{{\alpha }_0},\dots ,m_{{\alpha }_{n-1}},m_{{\alpha }_n}^{\pm }\}\in \partial \left({\mathcal{M}}_{n+1}\right)$. Thanks to MaxEnt formalism Equation (13) continues to hold. Once the first moments ${\left\{m_{{\alpha }_j}\right\}}_{j=0}^n\in Int\left({\mathcal{M}}_{n+1}\right)$ have been assigned, the bounded support does not imply any restriction on the Lagrange multipliers, in particular, ${\lambda }_n$ can take on any real value. From (13), as $m_{{\alpha }_n}$ varies within the bounded range of its admissible values $(m_{{\alpha }_n}^{-},m_{{\alpha }_n}^{+})$, $det\left(G_{n-1}\right)>0$ is bounded. As $m_{{\alpha }_n}\to m_{{\alpha }_n}^{\pm }$, $f_n$ coincides with the measures $\underline{\sigma }$ and $\overline{\sigma }$. As a consequence $det\left(G_n\right)\to 0$, from which $\frac{d{\lambda }_{n+1}}{d{m}_{{\alpha }_{n+1}}}=-\frac{det\left(G_{n-1}\right)}{det\left(G_n\right)}\to -\infty$ and then ${\lambda }_{n+1}\to -\infty$ follows.

• Analog conclusions hold for the remaining Lagrange multipliers, pre and post-multiplying in $\frac{d{\lambda }_j}{d{m}_{{\alpha }_n}}$, with $j<n$, the matrix at the numerator by a suitable permutation matrix.
• In conclusion, given ${\left\{m_{{\alpha }_j}\right\}}_{j=0}^{n-1}\in Int\left({\mathcal{M}}_n\right)$ and assuming $f_{n-1}$ exists, $f_n$ exists if ${\left\{m_{{\alpha }_j}\right\}}_{j=0}^n\in Int\left({\mathcal{M}}_{n+1}\right)$. Equivalently, the existence of $f_n$ is iteratively determined, starting from $f_0$ (the uniform distribution) which exists. On the other hand, thanks again to the MaxEnt formalism the previous proof of existence continues to hold. The solvability of the problem under certain restrictive assumptions on the prescribed moment vector ceases to exist and consequently the following theorem holds:

  Theorem 5.

  If $X\in [0,1]$ a necessary and sufficient condition for the existence of the MaxEnt distribution $f_n$ is that the vector of moments is internal to the space of moments, that is ${\left\{m_{{\alpha }_j}\right\}}_{j=0}^n\in Int\left({\mathcal{M}}_{n+1}\right)$.

4.2. Entropy Convergence of MaxEnt Distribution

Convergence in entropy of $f_n$ to f in the case where the entropy (12) is finite or $-\infty$ and its implications, play a fundamental role in many applied problems where the focus is often put on the behavior of the tails of the distribution F that are crucial to study extreme events behavior and to evaluate the probability of their occurrence. In this direction, Ref. [17] stresses the fact that “…at the tails, the MaxEnt distribution oscillates because of the nonmonotonic nature of the polynomial embedded in the $f_n$. Thus, only the lower-order moments are typically considered, but in such cases, $f_n$ hardly models tails fatter than the Gaussian. Therefore, the tails of many distributions cannot be well fitted by the MaxEnt distribution with $n\le 4$” thus questioning the utility of the MaxEnt approach and consequent solution in this case.

We will prove the almost everywhere nature of the convergence in entropy to f of the MaxEnt approximation $f_n$ based on an optimal set of fractional moments and this will permit us to disprove the above claim “…MaxEnt distribution oscillates because of the nonmonotonic nature of the polynomial embedded in the $f_n$” and state that $f_n$ represents a reliable reconstruction of f and of the main features of the corresponding distribution F, including the tail behavior. Further, exploiting the convergence in entropy of $f_n$ to f, it is possible to formulate a criterion for choosing the optimal number n and the values ${\left\{{\alpha }_j\right\}}_{j=1}^n$ of the fractional exponents and then, the best set of fractional moments ${\left\{m_{{\alpha }_j}\right\}}_{j=1}^n$ (see Equation (29) below).

Finally, even if the tails of the distribution oscillate as stated by [17], if the focus is on evaluating (or estimating) appropriate numerical summaries of the distribution usually expressed in terms of expected values or quantiles, convergence in entropy ensures that the approximation error (Equations (25) and (26) below) can be controlled by a proper choice of the number and the orders of fractional moments and, consequently, the goodness and reliability of such summaries regardless of the oscillating nature of the tails of the MaxEnt distribution.

We are now in a position to prove the main result of this paper, which we enunciate below.

Theorem 6 (Main result).

If X is a positive random variable, having the moments sequence ${\left\{m_{{\alpha }_j}\right\}}_{j=0}^{\infty }$ characterizing a unique distribution, MaxEnt approximations converge in entropy to the underlying distribution, that is

$$\underset{n\to \infty }{\lim }{h}_{f_n}=h_f$$

(17)

with $h_f$ either finite or $-\infty$.

Proof. 

We begin giving the proof of Theorem 6 for $X\in [0,\infty )$. Then we just adjust the proof for $X\in [0,1]$.

1.

Suppose $X\in [0,\infty )$.

As $m_{{\alpha }_{n+1}}>m_{{\alpha }_{n+1}}^{-}$ varies, both $f_{n+1}=f_{n+1}\left(m_{{\alpha }_{n+1}}\right)$ (equivalently ${\lambda }_j={\lambda }_j\left(m_{{\alpha }_{n+1}}\right)$, $j=0,\dots ,n+1$) and then $h_{f_{n+1}}=h_{f_{n+1}}\left(m_{{\alpha }_{n+1}}\right)$ hold.

Consider $h_{f_{n+1}}\left(m_{{\alpha }_{n+1}}\right)$ and collect together (12) and the first equation of (13), we have

$$\frac{d{h}_{f_{n+1}}\left(m_{{\alpha }_{n+1}}\right)}{d{m}_{{\alpha }_{n+1}}}=\sum _{j=0}^n{m}_{{\alpha }_j}\frac{d{\lambda }_j\left(m_{{\alpha }_{n+1}}\right)}{d{m}_{{\alpha }_{n+1}}}+{\lambda }_{n+1}\left(m_{{\alpha }_{n+1}}\right)={\lambda }_{n+1}\left(m_{{\alpha }_{n+1}}\right)$$

from which, taking into account (14), $\frac{d^2{h}_{f_{n+1}}\left(m_{{\alpha }_{n+1}}\right)}{d{m}_{{\alpha }_{n+1}}^2}=\frac{d{\lambda }_{n+1}\left(m_{{\alpha }_{n+1}}\right)}{d{m}_{{\alpha }_{n+1}}}<0$. Thus $h_{f_{n+1}}\left(m_{{\alpha }_{n+1}}\right)$ is a differentiable concave function.

Enter Markov–Krein’s Theorem. From Theorem 4 and its consequences, as $m_{{\alpha }_{n+1}}\to m_{{\alpha }_{n+1}}^{-}$, $f_{n+1}\left(m_{{\alpha }_{n+1}}\right)$ can be assimilated to Dirac’s deltas set, equivalently to discrete distribution, the so-called lower principal representation$\underline{\sigma }$.

We recall for consistency between the differential entropy of a continuous random variable and the entropy of its discretization, the differential entropy of any discrete measure (being compared to the delta Dirac function) is assumed to be $-\infty$ ([18], pp. 247–249). As a consequence, $h_{f_{n+1}}\left(m_{{\alpha }_{n+1}}^{-}\right)=-\infty$ can be set. On the other hand, as $m_{{\alpha }_{n+1}}$ takes its own prescribed value, $h_{f_{n+1}}\ge h_f$ holds. Then, with $h_{f_{n+1}}\left(m_{{\alpha }_{n+1}}\right)$ being a continuous function, there exists a value, say ${\tilde{m}}_{{\alpha }_{n+1}}\in (m_{{\alpha }_{n+1}}^{-};m_{{\alpha }_{n+1}}]$, such that $h_{f_{n+1}}\left({\tilde{m}}_{{\alpha }_{n+1}}\right)=h_f$. Summarizing, we have seen that:

(i)

If ${\left\{{\alpha }_j\right\}}_0^{n+1}$ are assigned and $f_{n+1}$ is the corresponding MaxEnt density with entropy $h_{f_{n+1}}$, the sequence $\left\{h_{f_{n+1}}\right\}$ is monotonically decreasing and then convergent, with ${\lim }_{n\to \infty }{h}_{f_{n+1}}\ge h_f$;

(ii)

for each n, $h_{f_{n+1}}\left(m_{{\alpha }_{n+1}}\right)$ is concave function in $(m_{{\alpha }_{n+1}}^{-};m_{{\alpha }_{n+1}}]$; as $m_{{\alpha }_{n+1}}\to m_{{\alpha }_{n+1}}^{-}$, $h_{f_{n+1}}\left(m_{{\alpha }_{n+1}}^{-}\right)=-\infty$;

(iii)

there exists ${\tilde{m}}_{{\alpha }_{n+1}}\in (m_{{\alpha }_{n+1}}^{-};m_{{\alpha }_{n+1}}]$ such that $h_{f_{n+1}}\left({\tilde{m}}_{{\alpha }_{n+1}}\right)=h_f$.

Enter Lin’s Theorem. Consider the Theorem 1 and without loss of generality, it will be assumed the sequence ${\left\{m_{{\alpha }_j}\right\}}_0^{\infty }$ is asymptotically monotonic increasing. From Theorem 1, the sequence ${\left\{m_{{\alpha }_j}\right\}}_0^{\infty }$ is convergent and, under the above assumption, is asymptotically monotonic increasing. As $n\to \infty$, from both relationships $m_{{\alpha }_n}<m_{{\alpha }_{n+1}}^{-}<{\tilde{m}}_{{\alpha }_{n+1}}<m_{{\alpha }_{n+1}}$ and $(m_{{\alpha }_{n+1}}-m_{{\alpha }_n})\to 0$, it follows

(iv)

both $m_{{\alpha }_{n+1}}^{-}\to m_{{\alpha }_{n+1}}$ and ${\tilde{m}}_{{\alpha }_{n+1}}\to m_{{\alpha }_{n+1}}$.

Combining together just the above items (i)–(iv) drawn from Theorem 1 and Theorem 4, respectively, it follows

$$\underset{n\to \infty }{\lim }{h}_{f_{n+1}}=\underset{n\to \infty }{\lim }{h}_{f_{n+1}}\left({\tilde{m}}_{{\alpha }_{n+1}}\right)=h_f.$$

(18)

The employed methodology for the proof clearly suggests that the convergence in entropy holds true in both cases $h_f$ finite and $h_f=-\infty$. Indeed, assuming $h_f=-\infty$, as $n\to \infty$, $m_{{\tilde{\alpha }}_{n+1}}$ tends to $m_{{\alpha }_{n+1}}$, so that Equation (18) leads to ${\lim }_{n\to \infty }{h}_{f_{n+1}}=-\infty$ too. Previously we proved that whenever $f_n$ does not exist the missing entropy $h_{f_n}$ is replaced with $h_{f_{n-1}}$, so that the sequence of entropies ${\left\{h_{f_n}\right\}}_1^{\infty }$ is defined for every n. That fact gives full significance to (18).

2.

Suppose $X\in [0,1]$.

The procedure previously employed in $X\in [0,\infty )$ case, is likewise extended to $X\in [0,1]$ since the involved functions ${\{u_j\left(t\right)=t^{{\alpha }_j}\}}_0^{n+1}$, $t\in [0,1]$ are T-systems too and both an analogous Markov–Krein theorem ([15]—Thm. 1.1, p. 109) and Lin’s Theorem 6 are available (in the latter case, although the sequence ${\left\{m_{{\alpha }_j}\right\}}_{j=0}^{\infty }$ has to be monotonically decreasing, the proof is similar). Thanks to MaxEnt formalism, both Equation (13) and the used methodology to prove Theorem (6) hold true.

In conclusion, if $h_f$ is finite or $-\infty$, for $X\in [0,1]$ or $X\in [0,\infty )$, MaxEnt formalism enables us to prove the entropy convergence (Theorem (6)) by means of a unified procedure. □

We recall that the entropy convergence had been proved in [19]—Theorem 3.1, for the case with $X\in [0,1]$ and $h_f$ finite, by transforming the problem of Laplace transform inversion into a fractional moment one on $[0,1]$. Here, the author mentions Lin’s theorem, although the statements of such theorem are not actually used in the proof at all. Theorem 6 allows us to selecting $({\alpha }_1,\dots ,{\alpha }_n)$ in both cases $X\in [0,1]$ and $X\in [0,\infty )$. This choice is driven by the minimization of the residual $h_{f_n}-h_f$. For this purpose, limited to the case in which f has finite entropy $h_f$, a valid guide is given by the different modes of convergence stemming from just above proved entropy convergence and shortly recalled.

Thanks to MaxEnt formalism the below-described procedure holds true in both cases $X\in [0,1]$ and $X\in [0,\infty )$, so that with $\mathcal{U}$ we mean, without distinction, the support of $X\in [0,1]$ or $X\in [0,\infty )$.

4.3. Further Convergence Modes for Finite $h_f$

In the case in which $h_f$ is finite and then ${\inf }_n{h}_{f_n}$ is finite too, the following additional results may be drawn. These results configure in a chain of implications starting from the (almost everywhere) convergence in entropy and ending with the convergence in distribution: the aim is to justify the MaxEnt reconstruction of particular features of the distribution in which we are interested in governing their reliability by controlling their approximation error in terms of residual entropy $h_{f_n}-h_f$.

Let $m>n$ and $f_m$ and $f_n$ be the maxentropic solution of the truncated fractional moment problem, with m and n moments, respectively. Combining together the following two facts:

(a)

the monotonically non-increasing sequence $\left\{h_{f_n}\right\}$ converges to $h_f$ and then it is a Cauchy sequence

(b)

the Kullback–Leibler distance between $f_m$ and $f_n$ that share the same first n fractional moments given by

$$D(f_m,f_n)={\int }_{\mathcal{U}}{f}_m\ln \frac{f_m}{f_n}dx$$

(19)

implies

$$D(f_m,f_n)=h_{f_n}-h_{f_m}.$$

(20)

Hence, taking into account Pinsker’s inequality ([20], p. 390), it follows

$$\frac{1}{2}{\Vert f_m-f_n\Vert }_1^2\le D(f_m,f_n)=h_{f_n}-h_{f_m}$$

(21)

• By replacing $f_m$ with f, letting $n\to \infty$, recalling Theorem 6 and the completeness of the ${\mathcal{L}}^1$ space, it holds

  $$\frac{1}{2}{\Vert f_n-f\Vert }_1^2\le D(f,f_n)=h_{f_n}-h_f\to 0$$

  (22)

  and hence ${\left\{f_n\right\}}_{n=1}^{\infty }$ has limit f. Then ${\left\{f_n\right\}}_{n=1}^{\infty }$ has a subsequence pointwise convergent a.e. to f and the whole sequence ${\left\{f_n\right\}}_{n=1}^{\infty }$ is also convergent a.e. to the same limit, that is

  $$\underset{n\to \infty }{\lim }{f}_n=fa.e.$$

  (23)

  that explains the goodness of the approximation (or estimation, if in a sample setup) f through the MaxEnt $f_n$ based on fractional moments.

Since ${\left\{f_n\right\}}_{n=1}^{\infty }$ converges in ${\mathcal{L}}^1$-norm to f, then it converges to f also (in probability and) in distribution so that,

$$\underset{n\to \infty }{\lim }{F}_n\left(x\right)=F\left(x\right)$$

for all x at which $F\left(x\right)$ is continuous, where $F_n$ and F denote the cumulative distribution functions corresponding to $f_n$ and f, respectively. Then the approximation $f_n$ is particularly suitable for an accurate calculation of the expected values, since as $n\to \infty$ convergence in distribution is equivalent to

$$\underset{n\to \infty }{\lim }{\int }_{\mathcal{U}}g\left(x\right)f_n\left(x\right)dx={\int }_{\mathcal{U}}g\left(x\right)f\left(x\right)dx$$

(24)

for each bounded function g. Then from (21) and (24) it follows

$$\mid {\mathit{E}}_{f_n}\left(g\right)-{\mathit{E}}_f\left(g\right)\mid \le {\Vert g\Vert }_{\infty }\sqrt{2(h_{f_n}-h_f)}$$

(25)

The argument runs similarly whether quantiles have to be calculated. They may be configured as expected values of proper bounded functions: indeed, for fixed x, $F\left(x\right)=\mathit{E}\left[g\right(t\left)\right]$ with $g\left(t\right)=1$ if $t\in [0,x]$ and $g\left(t\right)=0$ if $t\in \mathcal{U}\setminus [0,x]$. Then we have

$$\begin{array}{c}\hfill \mid F_n\left(x\right)-F\left(x\right)\mid \le {\int }_0^x\mid f_n\left(t\right)-f\left(t\right)\mid dt\le {\int }_{\mathcal{U}}\mid f_n\left(t\right)-f\left(t\right)\mid dt\le \sqrt{2(h_{f_n}-h_f)}.\end{array}$$

(26)

The above convergence results suggest that a rapid convergence in entropy allows an accurate approximation of the desired density or its features that have to be preserved. So choosing an optimal set of ${\alpha }_j$’s indices in terms of numbers and values becomes the priority.

Remark 1.

For $X\in [0,1]$ or $X\in [0,\infty )$ with $h_f$ finite, combining Theorem 6 with (21), from ${\left\{h_{f_n}\right\}}_{n=1}^{\infty }$ Cauchy sequence, the continuous functions sequence ${\left\{f_n\right\}}_{n=1}^{\infty }$ is Cauchy sequence too and then uniformly convergent to f. Hence, f is a continuous function. Consequently, an accurate reconstruction of the distribution requires that the underlying density be continuous as well. From an engineering point of view, this request may seem obvious. This explains the reason why in some numerical tests appearing in the literature and concerning the reconstruction of discontinuous densities with entropic techniques using integer or fractional moments, the reconstruction obtained had proven to be somewhat inaccurate.

5. Optimal Choice and Optimal Number of $\mathbf{\alpha }$’s

As recalled in Section 1, in real-world problems the proposal of a stochastic model or a probabilistic law F for a phenomenon X must necessarily take into account the aspects of X that must be preserved and, in some sense, the proposal process is guided by them. As a consequence, although considering the same phenomenon, it could be necessary to compute several distributions, each referring to a specific aspect that has to be preserved and then identify the proposal according to it. Because of their flexible choice, fractional moments can be considered a valuable tool in this regard and for operational reasons, two main questions need now to be addressed: the choice of the number n and of the orders $\alpha$’s of the fractional moments involved in the MaxEnt approximation of F. Both questions have strong relationship with the notion of convergence in entropy of $F_n$ to F which plays a strategic role in finding appropriate answers to these questions.

5.1. The Choice of $({\alpha }_1,\dots ,{\alpha }_n)$

Once the theoretical problem of existence, and convergence in entropy from an arbitrary fractional moment sequence according to Lin’s theorems are solved, the approximation of the distribution becomes essentially a computational issue. For this reason, it is a matter of choosing a suitable set of exponents $({\alpha }_1,\dots ,{\alpha }_n)$.

Note, that Lin’s theorems provide a theoretical guarantee to the process of reconstructing. However, due to the underlying computational issues, there are infinitely many possibilities in the choice of a few fractional moments. Such a choice must rest on the characteristics of the quantity to be calculated and that are intended to be retained in the approximation process. In an equivalent way, unlike the integer moments, in the approximation of the distribution with fractional moments it is possible to incorporate further information that comes from the underlying physical problem. The idea that we follow here was originally proposed by [7] and further explored by [21]. The idea goes as follows.

• We shall denote by $f_n$ found in (10) to make explicit its dependence on n and implicitly on the $({\alpha }_1,\dots ,{\alpha }_n)$. These will be chosen as to minimize the Kullback–Leibler divergence (19) between the ”true” but unknown density f and the maxentropic solution $f_n$. From (12), $h_{f_n}={\sum }_{j=0}^n{\lambda }_j{m}_{{\alpha }_j}$ and this quantity equals $-{\int }_{\mathcal{U}}{f}_n\left(x\right)\ln f_n\left(x\right)dx$ because f and $f_n$ satisfy the same moments constraints. Therefore, minimizing (19) amounts to

  $$\arg \min \left\{{\int }_{\mathcal{U}}f\left(x\right)\ln \frac{f\left(x\right)}{f_n\left(x\right)}dx|{\alpha }_1,\dots ,{\alpha }_n\right\}=\arg \min \left\{h_{f_n}\right|{\alpha }_1,\dots ,{\alpha }_n\}.$$

  (27)
  In other words, $f_n$ is obtained through two consecutive minimization procedures with respect to $({\alpha }_1,\dots ,{\alpha }_n,{\lambda }_1,\dots ,{\lambda }_n)=(\mathit{\alpha },\mathit{\lambda })$, namely

  $$\underset{\mathit{\alpha }}{\min }\underset{\mathit{\lambda }}{\min }\left[h_{f_n}(\mathit{\lambda },\mathit{\alpha })\right]=\underset{\mathit{\alpha }}{\min }\underset{\mathit{\lambda }}{\min }\left[\ln \right({\int }_{\mathcal{U}}\exp (-\sum _{j=1}^n{\lambda }_j{x}^{{\alpha }_j})dx)+\sum _{j=1}^n{\lambda }_j{m}_{{\alpha }_j}]$$

  (28)

  for $n=1,2,\dots$. This method consists of an implementation of the nested minimization. That is for each fixed $\mathit{\alpha }$, first minimize $\mathit{\lambda }\to h_{f_n}(\mathit{\lambda },\mathit{\alpha })$ and then carry on the outer minimization with respect to $\alpha$. It is worth mentioning the choice criterion (27) stems from entropy-convergence Theorem 6. The inner minimization is easy because we are dealing with a convex function. But even though the function $\alpha \to m_{\alpha }=\mathit{E}\left[X^{\alpha }\right]$ is log-convex, the linear combination ${\sum }_{j=0}^n{\lambda }_j{m}_{{\alpha }_j}$ need not be so. However, the existence conditions of a unique solution for (28) remain a theoretical open issue.
  Being (28) multivariable and highly nonlinear unconstrained not convex optimization, the uniqueness of the MaxEnt solution may not be guaranteed, so the results greatly rely on the initial condition, i.e., different initial conditions may give different MaxEnt solutions. And even if the algorithm converges, there is no assurance that it will have converged to a global, rather than a local, optimum since conventional algorithms cannot distinguish between the two.
  For problems where finding an approximate global optimum is more important than finding a precise local optimum in a fixed amount of time, the Simulated Annealing Method may be preferable to exact algorithms. This explores the function’s entire surface and tries to optimize the function while moving both uphill and downhill. Thus, it is largely independent of the starting values, often a critical input in conventional algorithms. Further, it can escape from local optima and go on to find the global optimum.
  In conclusion, the crucial issue consists of solving the nested minimization which ranges over two distinct sets of variables ${\left\{{\alpha }_j\right\}}_1^n$ and ${\left\{{\lambda }_j\right\}}_1^n$. While each ${\alpha }_j$ takes its values into the interval $(0,{\alpha }_{max}]$, where ${\alpha }_{max}$ relies upon physical or numerical reasons, each ${\lambda }_j$ may assume any real value.
• Alternatively, taking into account for each fixed set $({\alpha }_1,\dots ,{\alpha }_n)$, the inner ${\min }_{{\lambda }_1,\dots ,{\lambda }_n}$ admits a unique solution being $h_{f_n}={\sum }_{j=0}^n{\lambda }_j{m}_{{\alpha }_j}$ convex function, the outer one could be calculated by Monte Carlo technique, replacing ${\min }_{{\alpha }_1,\dots ,{\alpha }_n}$ with ${\inf }_{{\alpha }_1,\dots ,{\alpha }_n}$, that is

  $$\underset{\mathit{\alpha }}{\inf }\underset{\mathit{\lambda }}{\min }\left[h_{f_n}(\mathit{\lambda },\mathit{\alpha })\right]=\underset{\mathit{\alpha }}{\inf }\underset{\mathit{\lambda }}{\min }\left[\ln \right({\int }_{\mathcal{U}}\exp (-\sum _{j=1}^n{\lambda }_j{x}^{{\alpha }_j})dx)+\sum _{j=1}^n{\lambda }_j{m}_{{\alpha }_j}]$$

  (29)

  with $n=1,2,\dots$. Indeed, Equation (29) is just a computational trick and replacing ${\min }_{{\alpha }_1,\dots ,{\alpha }_n}$ with ${\inf }_{{\alpha }_1,\dots ,{\alpha }_n}$ arises from the request for an estimator that guarantees faster convergence in entropy. This replacement does not conflict with the spirit of MaxEnt since, regardless of the estimation criterion (29) of $\alpha$’s, the resulting $f_n$ continues to be a MaxEnt distribution. Then, according to Theorem 6, Equations (25) and (26), expected values or quantiles can be accurately calculated, up to a predetermined tolerance by means of (29). Note, also that the estimation method (29) lends itself easily to taking into account the existence conditions of $f_n$ stated in Theorem 4: if $(f_n\mid {\alpha }_1,\dots ,{\alpha }_{n-1},{\alpha }_n)$ does not exist, $(f_{n-1}\mid {\alpha }_1,\dots ,{\alpha }_{n-1})$, with the same $({\alpha }_1,\dots ,{\alpha }_{n-1})$ as $f_n$, does exist. Consequently, $h_{f_{n-1}}$ is recalculated and the value $h_{f_{n-1}}$ is assumed.
  After having illustrated the $({\alpha }_1,\dots ,{\alpha }_n)$ selection criteria in the distribution calculation procedure we can reconnect again to the previously introduced problem of choice of constraints in the construction of the MaxEnt distribution. As a constraint, we can also include the choice of range $(0,{\alpha }_{max}]$ in which to place $({\alpha }_1,\dots ,{\alpha }_n)$ in the minimization procedure (29). As an example, if for physical reasons we know the underlying f has hazard rate function $h\left(x\right)=\frac{f\left(x\right)}{1-{\int }_0^{x}f\left(u\right)du}$ with prescribed properties (for instance, asymptotically decreasing to zero), the approximation $f_n$ would save such property. As a consequence we choose once more (29), but, as it is easy to verify taking into account (10) with exponents $({\alpha }_1,\dots ,{\alpha }_n)\in [0,1)$ (therefore, not optimal for the purposes of rapid convergence in entropy). Conversely, with the hazard rate asymptotically increasing to $+\infty$, $({\alpha }_1,\dots ,{\alpha }_n)$ with ${\alpha }_n>1$ accomplish that request. Consequently, in both cases, it is important that entropy convergence is ensured.
  In conclusion, the criterion (29) for the calculation of $f_n$ is elastic and lends itself to correctly describing multiple scenarios.

5.2. A Single-Loop Strategy for Approximating $f_n$ with $X\in [0,1]$

In past Section, the difficulties related to nested minimization (28) have been circumvented with the procedure Monte Carlo (29) which allows the computation of $\mathbf{\lambda }$ uniquely through the minimization of a convex function. In the case $X\in [0,1]$, exploiting the different modes of convergence previously proved, it is possible to further simplify the computation of $\mathit{\lambda }$ in (29) by replacing the inner ${\min }_{\lambda }$ with the solution of a suitable linear system of equations. Indeed, from (3) integrating by parts (see [21], for details), we have

$$exp(-\sum _{k=0}^n{\lambda }_k)+\sum _{k=1}^n{\alpha }_k{\lambda }_k{\mathit{E}}_{f_n}\left(X^{{\alpha }_j+{\alpha }_k}\right)=(1+{\alpha }_j){\mathit{E}}_{f_n}\left(X^{{\alpha }_j}\right),j=0,...,n$$

(30)

Subtracting from each equation of index $j=1,\dots ,n$ the one having index $j=0$, the following system of equations in the unknowns ${\lambda }_1,\dots ,{\lambda }_n$ is obtained

$$\sum _{k=1}^n{\alpha }_k{\lambda }_k\left[{\mathit{E}}_{f_n}\left(X^{{\alpha }_k+{\alpha }_{j-1}}\right)-{\mathit{E}}_{f_n}\left(X^{{\alpha }_k+{\alpha }_j}\right)\right]=(1+{\alpha }_{j-1}){\mathit{E}}_{f_n}\left(X^{{\alpha }_{j-1}}\right)-(1+{\alpha }_j){\mathit{E}}_{f_n}\left(X^{{\alpha }_j}\right)$$

(31)

for $j=1,\dots ,n$ where ${\mathit{E}}_{f_n}\left(X^{{\alpha }_j}\right)={\mathit{E}}_f\left(X^{{\alpha }_j}\right)$, $j=0,\dots ,n$ are known, whilst ${\mathit{E}}_{f_n}\left(X^{{\alpha }_k+{\alpha }_{j-1}}\right)$ and ${\mathit{E}}_{f_n}\left(X^{{\alpha }_k+{\alpha }_j}\right)$ are generally unknown. Now observe that taking (25) into account, the moment curves ${\mathit{E}}_{f_n}\left(X^{\alpha }\right)$ and ${\mathit{E}}_f\left(X^{\alpha }\right)$ corresponding to $f_n$ and f, respectively differ as follows

$$\mid {\mathit{E}}_{f_n}\left(X^{\alpha }\right)-{\mathit{E}}_f\left(X^{\alpha }\right)\mid \le \sqrt{2(h_{f_n}-h_f)}$$

(32)

Note, as well that with $\mathbf{\alpha }$ solution of (29), the two moment curves ${\mathit{E}}_f\left(X^{\alpha }\right)$ and ${\mathit{E}}_{f_n}\left(X^{\alpha }\right)$ interpolate in the Birkhoff–Hermite sense at the nodes (see [22]); that is, they are both interpolating and tangent at the nodes $\alpha$. This implies that

$$\begin{array}{cc}\hfill {\mathit{E}}_f\left(X^{{\alpha }_j}\right)& ={\mathit{E}}_{f_n}\left(X^{{\alpha }_j}\right),j=0,1,2,\dots ,n\hfill \\ \hfill {\mathit{E}}_f\left(X^{{\alpha }_j}\ln \left(X\right)\right)& ={\mathit{E}}_{f_n}\left[X^{{\alpha }_j}\ln \left(X\right)\right],j=1,2,\dots ,n.\hfill \end{array}$$

(33)

By adopting a guessed choice of $\mathit{\alpha }$ (and the Monte Carlo method may achieve this goal), from Theorem 6, $h_{f_n}=h_f$ follows, so that relying upon (32) and (33), both ${\mathit{E}}_{f_n}\left(X^{{\alpha }_k+{\alpha }_{j-1}}\right)={\mathit{E}}_f\left(X^{{\alpha }_k+{\alpha }_{j-1}}\right)$ and ${\mathit{E}}_{f_n}\left(X^{{\alpha }_k+{\alpha }_j}\right)={\mathit{E}}_f\left(X^{{\alpha }_k+{\alpha }_j}\right)$ can be set. Thus, with a guessed choice of $\mathit{\lambda }$, it is legitimate to assimilate (31) to a linear system with unknown $\mathit{\lambda }$ which admits a unique solution, being an identity relating $\mathit{\lambda }$ with a set of values optimally picked up from the moment curve ${\mathit{E}}_{f_n}\left(X^{\alpha }\right)\equiv {\mathit{E}}_f\left(X^{\alpha }\right)$.

We shall suppose that the solution to (31) coincides with that obtained by solving (29). This brings this section close to experimental mathematics. The necessary analysis to compute the error in the just above approximation is hard and the verification comes in a posteriori as the numerical results based on it make good sense. Indeed, the numerical evidence suggests that, with $n\simeq 6$ optimally chosen $\alpha$ according with (29), $h_{f_n}\simeq h_f$ is usually observed. From which, combining together (20) with (23) (after replacing $f_m$ with f) the relationship $f=f_n$ a.e. holds. Consequently, from (25) the last two densities have their respective moment curves coincident as well. Then in (31) ${\mathit{E}}_{f_n}\left(X^{{\alpha }_k+{\alpha }_j}\right)={\mathit{E}}_f\left(X^{{\alpha }_k+{\alpha }_j}\right)$ can be set which in turn enables us to state the solution of (31) coincides with the one obtained from (29). Which makes the ansatz plausible. The ill-conditioning of (31) remains to be investigated. In this regard, unlike integer moments, only a limited number $n\le 6$ of fractional moments are sufficient for an accurate estimate of $f_n$, so that ill-conditioning issues are avoided. Once n is fixed, a final consideration concerns the choice of ${\alpha }_{max}={\max }_{1\le j\le n}\left\{{\alpha }_j\right\}$. Since the simplified procedure just described essentially concerns the accurate calculation of expected values, the answer follows from (24) and (25): compatibly with numerical issues, ${\alpha }_{max}$ should be taken as large as possible so as not to raise further constraints on a rapid convergence in entropy. As a consequence, the approximate suggested procedure for replacing (29) is as follows:

• Once $\mathit{\alpha }$ is fixed, and ${\mathit{E}}_{f_n}\left(X^{{\alpha }_k+{\alpha }_{j-1}}\right)={\mathit{E}}_f\left(X^{{\alpha }_k+{\alpha }_{j-1}}\right)$, ${\mathit{E}}_{f_n}\left(X^{{\alpha }_k+{\alpha }_j}\right)={\mathit{E}}_f\left(X^{{\alpha }_k+{\alpha }_j}\right)$ are set, $\mathit{\lambda }$ are drawn solving the linear system (31);
• as $f_n$ integrates into one, ${\lambda }_0$ is given by

  $${\lambda }_0=\ln {\int }_{\mathcal{U}}exp\left(-\sum _{j=1}^n{\lambda }_j{x}^{{\alpha }_j}\right)dx$$

  (34)
• combining (31) with (34), $h_{f_n}={\lambda }_0+{\sum }_{j=1}^n{\lambda }_j{m}_{{\alpha }_j}$ is calculated and finally

  $$f_n^{\left(app\right)}:h_{f_n}^{\left(app\right)}=\underset{{\alpha }_1,\dots ,{\alpha }_n}{\inf }{h}_{f_n}={\lambda }_0+\sum _{j=1}^n{\lambda }_j{m}_{{\alpha }_j}$$

  (35)

In conclusion, the quick simplified approximate procedure permits to avoid the direct solution of (29) by solving the low order linear system (31) with unknown $\mathit{\lambda }$, doing the numerical integration (34) and performing the one-loop procedure (35) that runs on $\mathit{\alpha }$ by means of Monte Carlo technique, with a reduced number of unknowns. This procedure is computationally feasible and convenient and gives back an accurate approximation of $f_n$ (hence of f), as we will see in the Example 1.

Remark 2.

In principle, thanks to MaxEnt formalism, the above outlined procedure in $X\in [0,1]$ might be pairwise extended to $X\in [0,\infty )$. Indeed, in that case the recursive relationship relating Lagrange multipliers ${\left\{{\lambda }_j\right\}}_{j=1}^n$ with higher order moments is quite similar to (31). With a similar procedure to the case of rv $X\in [0,1]$, integrating by parts (11), the following linear system follows

$$\sum _{k=1}^n{\alpha }_k{\lambda }_k{\mathit{E}}_{f_n}\left(X^{{\alpha }_k+{\alpha }_j}\right)=(1+{\alpha }_j){\mathit{E}}_{f_n}\left(X^{{\alpha }_j}\right),k,j=1,\dots ,n$$

(36)

with ${\mathit{E}}_{f_n}\left(X^{{\alpha }_j}\right)={\mathit{E}}_f\left(X^{{\alpha }_j}\right)$, $j=1,\dots ,n$ known, whilst ${\mathit{E}}_{f_n}\left(X^{{\alpha }_k+{\alpha }_j}\right)$ are generally unknown and ${\lambda }_0$ given by (34). In analogy with $X\in [0,1]$, MaxEnt density $f_n$ converges in entropy (and then in distribution) to f according to Theorem 6. Then, starting from moderate values of n, ${\mathit{E}}_{f_n}\left(X^{{\alpha }_k+{\alpha }_j}\right)={\mathit{E}}_f\left(X^{{\alpha }_k+{\alpha }_j}\right)$, for each $j,k$, can be set, where ${IE}_f\left(X^{{\alpha }_k+{\alpha }_j}\right)$ are known quantities. As a consequence, (36) may considered a linear system admitting a unique solution $({\lambda }_1,\dots ,{\lambda }_n)$, being the involved matrix a nonsingular Gram matrix with distinct ${\left\{{\alpha }_j\right\}}_{j=1}^n$. However, the method lacks a theoretical ground, since two main issues remain open that is,

• how to guarantee ${\lambda }_n\ge 0$ in (36) to ensure integrability in (34)
• although the convergence in distribution is guaranteed, Equation (24) is not applicable being the function $g\left(x\right)=x^{\alpha }$ unbounded.

Nevertheless, the above theoretical drawbacks don’t preclude the possibility that, with a special given moment set ${\left\{m_{{\alpha }_j}\right\}}_{j=1}^n$, the method can guarantee accurate results.

Remark 3.

Suppose to consider a generic random sample $(X_1,X_2,\dots ,X_N)$ from a distribution having support $\mathcal{U}$ not necessarily $[0,1]$: for example, $\mathcal{U}={IR}^{+}$ or $\mathcal{U}=IR$. The simplified procedure for the MaxEnt estimation of the density f proposed in Section 4.2 for the case $X\in [0,1]$ can be easily applied to the case $X\in {IR}^{+}$ or the case $X\in IR$. Below we will briefly sketch some details.

• Case $X\in IR$: it is enough to transform the original sample data for instance through $Y=g\left(X\right)=\frac{1}{2}+\frac{1}{\pi }arctan\left(X\right)$, to obtain a transformed sample $Y_1,Y_2,\dots ,Y_N$ in $[0,1]$ and apply the simplified procedure of Section 4.2.
• Case $X\in {IR}^{+}$: in a similar way, the transformation $Y=g\left(X\right)=e^{-X}$ can be applied to the original data $(X_1,X_2,\dots ,X_N)$ obtaining a transformed sample $Y_1,Y_2,\dots ,Y_N$ once again in $[0,1]$ interval and the simplified procedure of Section 4.2 is immediately applicable. Note, that the empirical fractional moments of Y coincide with the empirical Laplace Transform of X, that is $\frac{1}{N}{\sum }_1^N{Y}_j^{\alpha }=\frac{1}{N}{\sum }_1^N{e}^{-\alpha X_j}$, which turns out to be the empirical version of the relationship ${\int }_0^1{y}^{\alpha }d{F}_Y\left(y\right)={\int }_0^{\infty }{e}^{-\alpha x}d{F}_X\left(x\right)$. This last relation leads us to conclude that also the numerical inversion of the Laplace transform can be reduced to a fractional moment problem in $[0,1]$.
• More recently [23] investigated the feature of an estimator relying upon the fractional moments for random variables supported on $IR$ by allowing the fractional powers to take complex numbers. Unlike other authors, they are dealing with the case that the negative values of a random variable are not negligible at all.

Once obtained the MaxEnt estimate $f_n^{\left(app\right)}$ of f in $[0,1]$, it is possible to come back to the original spaces $IR$ or ${IR}^{+}$ by the associated inverse transformations $f_n^{\left(app\right)}\left(x\right)=\mid g^{\prime }\left(x\right)\mid f_n^{\left(app\right)}\left(g\left(x\right)\right)$.

The case $X\in IR$ outlined in just above items 1. and 3. partially permits to disprove the criticism arising from [17] where it is asserted that the fractional moments technique is applicable in the case of positive r.v. X only.

5.3. The Choice of n

With just before considerations, as $h_f$ is finite in both (28) and (29), as well in the single loop strategy, arises the optimal choice criterion of n. Recall that by employing integer moments, the adding of a further moment could result in a negligible or even zero entropy decrease, followed by not negligible entropy decreasing with the subsequent moments. Consequently, a stopping criteria based solely on the difference in entropy when adding an additional integer moment could be misleading in choosing the optimal number of moments to use. On the contrary, with the choice of fractional moments according to (29), it is easy to deduce that the sequence ${\left\{h_{f_n}\right\}}_{n\in \mathit{N}}$ is strictly monotonic decreasing. Indeed, consider (29), fix n and calculate $({\alpha }_1,\dots ,{\alpha }_n)$ from which $h_{f_n}$. Next, put $n+1$ in (29). As a first step, take the special set $({\alpha }_1,\dots ,{\alpha }_{n+1})$ where the first entries $({\alpha }_1,\dots ,{\alpha }_n)$ coincide with the just above found and ${\alpha }_{n+1}>{\alpha }_n$ is kept arbitrarily (that is constrained minimization running on ${\alpha }_{n+1}$ only, whilst $({\alpha }_1,\dots ,{\alpha }_n)$ is held fixed). Calculate $h_{f_{n+1}}$ and call it $h_{f_{n+1}}^{*}$, with $h_{f_{n+1}}^{*}<h_{f_n}$. As a second step take $n+1$ in (29), where the minimum runs on $({\alpha }_1,\dots ,{\alpha }_{n+1})$, from which $h_{f_{n+1}}$ (that is unconstrained minimization). It follows $h_{f_{n+1}}<h_{f_{n+1}}^{*}<h_{f_n}$. The sequence ${\left\{h_{f_n}\right\}}_{n\in \mathit{N}}$ is strictly monotonic decreasing and converges to $h_f$. In the special case where the sequence ${\left\{h_{f_n}\right\}}_{n\in \mathit{N}}$ is bounded below, has a finite limit, so it is a Cauchy sequence. It leads us to conclude from (29) the rate of entropy decrease becomes smaller and smaller as n increases and the difference between successive entropies becomes reasonably small (which is up to the modeler to decide), one stops and accepts the density determined by the larger number of moments as the ‘true’ density.

We conclude the paper with a simple example just to see the fractional moment MaxEnt technique in action. It involves all the crucial theoretical results (fractional moments, entropy reduction, convergence in entropy, optimal choice of number and exponents) introduced in the previous sections of the paper.

Example 1.

Here our goal is to compare the performances of integer and fractional moments MaxEnt approximations of a given density function f in $[0,1]$. For the sake of comparison, we will also consider the approximation $f^{\left(app\right)}$ of f obtained by using fractional moments in the simplified MaxEnt procedure given in Section 4.2. Double arithmetic precision is used.

Consider

$$f\left(x\right)=\frac{\pi }{2}sin\left(\pi x\right){\mathbb{I}}_{[0,1]}\left(x\right)$$

with $h_f=-0.14472327456$. The integer moments (im) have the following recursive relationship

$$m_j=\mathit{E}\left(X^j\right)=\frac{1}{2}-\frac{j(j-1)}{{\pi }^2}{m}_{j-2},j\ge 2,m_0=1,m_1=\frac{1}{2}$$

whilst fractional moments (fm) $m_{{\alpha }_j}=\mathit{E}\left(X^{{\alpha }_j}\right)$ are explicitly obtained by numerical integration. The associated MaxEnt densities ${f_n}^{\left(im\right)}$ and ${f_n}^{\left(fm\right)}$ are given by (4) and (10), respectively.

Optimal fractional moments determine a fast entropy decreasing $h_{f_n}^{\left(fm\right)}-h_f$ and 4 or 5 of them capture f. But, a definitely higher number of integer moments is required to have a comparable reconstruction of f, incurring drastic numerical instability due to ill-conditioning for $n>12$: the first column of

Table 2. Residual entropy with integer (left), fractional (middle) moments and approximated method (right) for an increasing number n of moments.

n	${\mathit{h}}_{{\mathit{f}}_{\mathit{n}}}^{\left(\mathit{im}\right)}-{\mathit{h}}_{\mathit{f}}$	n	${\mathit{h}}_{{\mathit{f}}_{\mathit{n}}}^{\left(\mathit{fm}\right)}-{\mathit{h}}_{\mathit{f}}$	n	${\mathit{h}}_{{\mathit{f}}_{\mathit{n}}}^{\left(\mathit{app}\right)}-{\mathit{h}}_{\mathit{f}}$
2	$0.2577\times {10}^{-1}$	1	$0.8774\times {10}^{-1}$	1	$0.8893\times {10}^{-1}$
4	$0.5051\times {10}^{-2}$	2	$0.8077\times {10}^{-2}$	2	$0.2802\times {10}^{-2}$
6	$0.1626\times {10}^{-2}$	3	$0.4488\times {10}^{-3}$	3	$0.4851\times {10}^{-3}$
8	$0.1443\times {10}^{-2}$	4	$0.6043\times {10}^{-5}$	4	$0.4693\times {10}^{-4}$
10	$0.6698\times {10}^{-3}$	5	$0.4000\times {10}^{-6}$	5	$0.1196\times {10}^{-4}$
12	$0.5923\times {10}^{-3}$			6	$0.14152\times {10}^{-5}$

gives evidence of it and actually, the sequence ${\left\{h_{f_n}^{\left(im\right)}\right\}}_{n\in \mathit{N}}$ ceases to be a decreasing monotone sequence. The third column of Table 2 contains the residual entropy concerning the MaxEnt approximation $f_n^{\left(app\right)}$ of f given by the simplified MaxEnt procedure described in Section 4.2. A quick comparison allows us to conclude the closeness of the solution based on fractional moments and that based again on fractional moments but using the simplified procedure of Section 4.2. As a consequence of the chain of convergence implications descending from the convergence in entropy of $f_n^{\left(fm\right)}$ to f, the features of interest of the density f can be well approximated (or estimated, if in a sample setup) by the corresponding features of $f_n^{\left(fm\right)}$ or $f_n^{\left(app\right)}$ and governing the approximation error in terms of n.

6. Conclusions

The approximation of probability distributions with finite or unbounded positive support using fractional moments to express available information has been reconsidered. Compared to the classical MaxEnt, a novel feature of the proposed method is that the fractional exponent of the MaxEnt distribution is determined through the entropy maximization process, instead of being assigned a priori by an analyst. Theorems of the existence of the maximum entropy distribution and its convergence in entropy are provided. The latter allows us to reconsider the selection criteria of the fractional exponents in order to speed up the convergence in entropy and other related modes of convergence, as well as preserve some important prior features of the underlying distribution.

Since the release two decades ago of the first paper on the subject by the authors, numerous criticisms have been raised by several researchers who have used this methodology. The main criticism focused on the method of calculating the distribution consisting of nested minimization. There is no need to formalize ourselves on the existence or non-existence of the outer minimum in (28). With ${\min }_{{\alpha }_1,\dots ,{\alpha }_n}$ we would simply intend to carry down the entropy as fast as possible. Indeed, from the different ways of convergence that we have above listed, it can be deduced that the committed error in the calculation of expected values or quantiles is controlled precisely by residual entropy.

Relying on a solid proof of the entropy convergence theorem, different convergence modes are derived as well, we have tried to overcome that shortcoming. For this reason, a Monte Carlo method has been suggested which rests on solid foundations, being the only process of involved minimization performed on a function that is known to be convex.

Concerning the computational efforts of the suggested techniques, the algorithm associated with (29) requires a numerical integration subroutine while the simplified procedure of Section 4.2 needs a numerical routine for the solution of a linear system of equations, both tools available in any mathematical or statistical numerical package and seem to be definitely reasonable.

It is also remarkable to note that the use of a finite number of fractional moments to represent the available information properly tailored to the problem of interest is still possible when only a random sample from a given unknown distribution is available. The existence theorem and the convergence in entropy theorem give solid bases for the inferential procedures about the unknown f based on $f_n$.

After recalling that fractional moments can be included in the mathematical family of T-systems, they have helped to provide

• the conditions of existence of the density $f_n$;
• the convergence theorem in entropy from which other modes of convergence follow;
• an optimal choice and optimal number of the fractional exponents $\mathit{\alpha }$;
• assuming $X\in [0,1]$, a single-loop algorithm for approximating $f_n$.