Knowable Moments in Stochastics: Knowing Their Advantages

Abstract

Knowable moments, abbreviated as K-moments, are redefined as expectations of maxima or minima of a number of stochastic variables that are a sample of the variable of interest. The new definition enables applicability of the concept to any type of variable, continuous or discrete, and generalization for transformations thereof. While K-moments share some characteristics with classical and other moments, as well as with order statistics, they also have some unique features, which make them useful in relevant applications. These include the fact that they are knowable, i.e., reliably estimated from a sample for high orders. Moreover, unlike other moment types, K-moment values can be assigned values of distribution function by making optimal use of the entire dataset. In addition, K-moments offer the unique advantage of considering the estimation bias when the data are not an independent sample but a time series from a process with dependence. Both for samples and time series, the K-moment concept offers a strategy of model fitting, including its visualization, that is not shared with other methods. This enables utilization of the highest possible moment orders, which are particularly useful in modelling extremes that are closely associated with high-order moments.

1. Introduction

Knowable moments, abbreviated as K-moments, were introduced by Koutsoyiannis in 2019 [1] and further explored by Koutsoyiannis (2022) [2,3]. They were applied to research [4,5,6] and engineering studies [7,8,9]. However, all these studies are authored or coauthored by Koutsoyiannis, who introduced them. Possible explanations for the reluctance of other researchers to use them are (a) that they are not useful or (b) they are not a new tool, as for example asserted by an anonymous negative reviewer of an initial submission of the already mentioned study [7], who stated that “the proposed ‘K-moments’ methods […] is a renaming of the widely known and widely applied ‘Probability-Weighted Moments’ (PWM) and their combinations known as ‘L-moments’” [10,11]. Another explanation (c) is that their definition and their advantages are not paid attention to or are not well understood.

Conjecturing that the latter explanation (c) is correct, here we extend and clarify the meaning and features of K-moments, emphasizing their advantages, particularly those that are unique among all moment types. We also provide algorithmic details for the application of the framework. More specifically, we provide the following information.

• We redefine K-moments (Section 2.1) as expectations of maxima or minima of a number of stochastic variables that are independent, identically distributed copies of the stochastic variable of interest. The new definition is clearer, more rigorous and more intuitive. In most cases of interest (but not in all), the new definition does not imply changes in the computational framework (Section 2.3).
• We provide a more intuitive explanation of the statistical estimators of K-moments, which are adapted to the new definition, again without implying computational differences in most of the cases. Furthermore, we provide techniques to accelerate the calculations when the datasets are large, e.g., with million or billion values (Section 2.2).
• We extend the scope of K-moments from continuous stochastic variables to also include discrete variables (Section 2.4).
• We discuss cases of generalization where the new definition implies some differences, notational and computational, to the existing one (Section 2.5).
• We discuss the advantages of the K-moment framework, starting from the information it provides about what a classical moment estimator determines, which actually is not the true value of that classical moment (Section 3.1); this is the reason why classical moments are unknowable from samples, for orders beyond 3–4.
• We show that K-moments for moment orders up to 4 replace the information contained in classical moments and L-moments, such as summary statistics (Section 3.2).
• We show that the real power of the K-moment framework is its ability to estimate moments reliably and unbiasedly for very high orders up to the sample size, even if this is several million or more. We also show the ability of the framework to readily assign, in a simple manner, a value of the distribution function to each K-moment—a property not shared by any other type of moments (Section 3.3).
• Exploiting the above features, we show how K-moments provide a sound and flexible framework for model fitting, making optimal use of the entire dataset, rather than relying on a few moments (as in classical moments and L-moments) or assigning probabilities to single data values (order statistics). This enables utilization of the highest possible moment orders, which are particularly useful in modelling extreme highs or lows that are closely associated with high-order moments. The model fitting concept with K-moments is a new strategy not shared with any other types of methods of moments and includes visualization of the goodness of fit (Section 3.4).
• We illustrate that K-moment estimators offer the ability to estimate the probability density function from a sample—a unique feature of the K-moment methodology.
• We show that K-moments offer the unique advantage of taking into account the estimation bias when the data are not an independent sample but a time series from a process with dependence, even of long range (Section 3.6).
• We provide algorithmic details of the computational framework of K-moments (particularly in Section 2.2, Section 3.4 and Section 3.6)
• Finally, we discuss conceptual similarities and differences with other moment types (Section 4—Discussion) and summarize the overall conclusions (Section 5).

With respect to point 8, the K-moment framework offers the possibility of modelling extremes using merely the parent distribution, without reference to asymptotic Extreme Value distributions. This is a more reliable choice as it has been known that the convergence to the asymptotic Extreme Value distributions can be extraordinarily slow [2], while the non-asymptotic distributions of extremes can be quite difficult to determine, particularly when there is time dependence. Yet there is a theoretical connection of the K-moment framework with the asymptotic Extreme Value theory, related to assigning values of the distribution function to K-moments (point 7) at the distribution tails, as will be specified in Section 3.3.

Points 1–4, 6, and 12 are the original contributions of this work, while the remaining points provide a review of existing developments of the K-moment framework, with appropriate adaptations to the new definitions and with the aim to highlight its advantages and usefulness, which, as already mentioned, is the scope of the paper. In addition, all illustrations, including graphs and tables, and the appendix are original.

2. Definitions and Main Derivations

2.1. Definition and Meaning

Let $\underset{\_}{x}$ be a stochastic variable with a distribution function $F\left(x\right)$ and a tail function $\overline{F}\left(x\right):=1-F\left(x\right)$. Notice that we have adopted the Dutch (van Dantzig–Hemelrijk [12]) convention of underlining stochastic variables while common variables are not underlined. This convention is much clearer and more rigorous than its alternatives, which either (a) do not distinguish the stochastic from common variables or (b) use upper-case letters for stochastic variables and lower-case ones for common variables. Alternative (a), while being popular, particularly in Bayesian statistics texts, is ambiguous and can be misleading (see several examples in [2]). Alternative (b) is too restrictive (it prohibits using lower-case letters for stochastic variables), ambiguous (when the standard notation of a physical quantity is in upper case, it could be misinterpreted as a stochastic variable, while it is not), or both (for example, when using Greek letters, e.g., the Latin $x$ and the Greek $\chi$ have the same appearance in upper case, i.e., $X$; see additional remarks in [2]).

If $\underset{\_}{x}$ is of continuous type, we define its probability density function as $f\left(x\right):=\mathrm{d}F\left(x\right)/\mathrm{d}x$. If it is of discrete type, taking on the values $x_j,j=\mathrm{0,1},\dots ,J$, where possibly $J=\infty$, we define its probability mass function as $P_j:=P\left\{\underset{\_}{x}=x_j\right\}=F\left(x_j\right)-F\left(x_{j-1}\right)$.

We consider a sample of $\underset{\_}{x}$, i.e., a number $p$ of independent copies of the stochastic variable $\underset{\_}{x}$, i.e., ${\underset{\_}{x}}_1,{\underset{\_}{x}}_2,\dots ,{\underset{\_}{x}}_p$. If we arrange the variables in ascending order, the ith smallest, denoted as ${\underset{\_}{x}}_{\left(i:p\right)},i=1,\dots ,p$ is termed the ith order statistic. The largest (pth) order statistic is:

$${\underset{\_}{x}}_{\left(p\right)}:={\underset{\_}{x}}_{\left(p:p\right)}=\max \left({\underset{\_}{x}}_1,{\underset{\_}{x}}_2,\dots ,{\underset{\_}{x}}_p\right)$$

(1)

and the smallest (first) is

$${\underset{\_}{x}}_{\left(1:p\right)}=\min \left({\underset{\_}{x}}_1,{\underset{\_}{x}}_2,\dots ,{\underset{\_}{x}}_p\right)$$

(2)

Now we define the K-moments in terms of expectations of these variables, denoted as $\mathrm{E}\left[\right]$, in the following manner.

Definition 1.

The expectation of the largest of the p variables ${\underset{\_}{x}}_{\left(p\right)}$:

$$K_p^{\prime }:=\mathrm{E}\left[{\underset{\_}{x}}_{\left(p\right)}\right]=\mathrm{E}\left[\max \left({\underset{\_}{x}}_1,{\underset{\_}{x}}_2,\dots ,{\underset{\_}{x}}_p\right)\right]$$

(3)

is called the upper knowable moment (K-moment) of order p.

Definition 2.

The expectation of the smallest of the p variables ${\underset{\_}{x}}_{\left(1:p\right)}$:

$${\overline{K}}_p^{\prime }:=\mathrm{E}\left[{\underset{\_}{x}}_{\left(1:p\right)}\right]=\mathrm{E}\left[\min \left({\underset{\_}{x}}_1,{\underset{\_}{x}}_2,\dots ,{\underset{\_}{x}}_p\right)\right]$$

(4)

is called the lower knowable moment (K-moment) of order p.

Furthermore, we generalize the definition to transformations $g\left(\underset{\_}{x}\right)$ of the stochastic variable of interest $\underset{\_}{x}$. Thus, by setting $g\left(\underset{\_}{x}\right)={\underset{\_}{x}}^q$, where q is an integer, we obtain:

Definition 3.

The expectation:

$$K_{pq}^{\prime }:=\mathrm{E}\left[{\underset{\_}{x}}_{\left(p\right)}^q\right]=\mathrm{E}\left[\max \left({\underset{\_}{x}}_1^q,{\underset{\_}{x}}_2^q,\dots ,{\underset{\_}{x}}_p^q\right)\right]$$

(5)

is called the upper K-moment of orders p,q.

Definition 4.

The expectation:

$${\overline{K}}_{pq}^{\prime }:=\mathrm{E}\left[{\underset{\_}{x}}_{\left(1:p\right)}^q\right]=\mathrm{E}\left[\min \left({\underset{\_}{x}}_1^q,{\underset{\_}{x}}_2^q,\dots ,{\underset{\_}{x}}_p^q\right)\right]$$

(6)

is called the lower K-moment of orders p,q.

The above notation implies that we omit the subscript q when $q=1$. Likewise, we can define central K-moments by setting $g\left(\underset{\_}{x}\right)=({\underset{\_}{x}-\mu )}^q$.

Definition 5.

The expectation:

$$K_{pq}:=\mathrm{E}\left[{\left({\underset{\_}{x}}_{\left(p\right)}-\mu \right)}^q\right]=\mathrm{E}\left[\max \left({\left({\underset{\_}{x}}_1-\mu \right)}^q,{\left({\underset{\_}{x}}_2-\mu \right)}^q,\dots ,{\left({\underset{\_}{x}}_p-\mu \right)}^q\right)\right]$$

(7)

is called the upper central K-moment of orders p,q.

Definition 6.

The expectation:

$${\overline{K}}_{pq}:=\mathrm{E}\left[{\left({\underset{\_}{x}}_{\left(1:p\right)}-\mu \right)}^q\right]=\mathrm{E}\left[\min \left({\left({\underset{\_}{x}}_1-\mu \right)}^q,{\left({\underset{\_}{x}}_2-\mu \right)}^q,\dots ,{\left({\underset{\_}{x}}_p-\mu \right)}^q\right)\right]$$

(8)

is called the lower central K-moment of orders p,q.

As we will see below, K-moments can alternatively be written as expectations of products of ${\underset{\_}{x}}^q$ (or ${\left(\underset{\_}{x}-\mu \right)}^q$), the distribution function or tail function raised to the power $p-1$, and some adjustment factors ($A\left(\underset{\_}{x},p\right)$ or $\overline{A}\left(\underset{\_}{x},p\right)$). Specifically, they are expressed as

$$\begin{gathered} K_{pq}^{\prime }=p\mathrm{E}\left[A\left(\underset{\_}{x},p\right){F\left(\underset{\_}{x}\right)}^{p-1}{\underset{\_}{x}}^q\right],{\overline{K}}_{pq}^{\prime }=p\mathrm{E}\left[\overline{A}\left(\underset{\_}{x},p\right){\overline{F}\left(\underset{\_}{x}\right)}^{p-1}{\underset{\_}{x}}^q\right] \\ {K}_{pq}=p\mathrm{E}\left[A\left(\underset{\_}{x},p\right){F\left(\underset{\_}{x}\right)}^{p-1}{\left(\underset{\_}{x}-\mu \right)}^q\right],{\overline{K}}_{pq}=p\mathrm{E}\left[\overline{A}\left(\underset{\_}{x},p\right){\overline{F}\left(\underset{\_}{x}\right)}^{p-1}{\left(\underset{\_}{x}-\mu \right)}^q\right] \end{gathered}$$

(9)

where, in the most common cases, the adjustment factors $A\left(\underset{\_}{x},p\right)$ and $\overline{A}\left(\underset{\_}{x},p\right)$ can be omitted as $A\left(\underset{\_}{x},p\right)=\overline{A}\left(\underset{\_}{x},p\right)=1$. However, in some of the cases, they can take different values, as will be specified below.

Equation (9) justifies their name moments. As will be seen, for $p=1$, $A\left(\underset{\_}{x},1\right)=1$ in all cases. Therefore, by setting $p=1$, we recover the classical noncentral (raw) moments, i.e.,

$$K_{1q}^{\prime }={\overline{K}}_{1q}^{\prime }=\mathrm{E}\left[{\underset{\_}{x}}^q\right]=:{\mu }_q^{\prime },K_{1q}=\mathrm{E}\left[{\left(\underset{\_}{x}-\mu \right)}^q\right]=:{\mu }_q$$

(10)

However, whenever we have only an observed sample, and we do not know $F\left(x\right)$ from theoretical reasoning, the classical moments are unknowable, except if q is very low, as thoroughly justified [1,2]. On the other hand, if we choose a low q, namely $q=\mathrm{1,2}$ (and occasionally 3), then we can estimate in a reliable manner a moment of total order $p+q-1$, which can be made very high by choosing a high p. This enables knowing the high-order properties of a distribution from a sample, which justifies the name knowable moments. In the next section, we will see that the estimation of the K-moments can be easily made using order statistics.

While the K-moment framework allows for studying transformations of $\underset{\_}{x}$, the simplest case of the untransformed variable suffices for most statistical tasks. In this case, Equation (9) is simplified to

$$\begin{gathered} K_p^{\prime }=p\mathrm{E}\left[A\left(\underset{\_}{x},p\right){F\left(\underset{\_}{x}\right)}^{p-1}\underset{\_}{x}\right],{\overline{K}}_p^{\prime }=p\mathrm{E}\left[\overline{A}\left(\underset{\_}{x},p\right){\overline{F}\left(\underset{\_}{x}\right)}^{p-1}\underset{\_}{x}\right] \\ {K}_p=K_p^{\prime }-\mu ,{\overline{K}}_p={\overline{K}}_p^{\prime }-\mu \end{gathered}$$

(11)

where the last two relationships that give the central moments are a direct consequence of the fact that $\max \left({\underset{\_}{x}}_1-\mu ,{\underset{\_}{x}}_2-\mu ,\dots ,{\underset{\_}{x}}_p-\mu \right)$ = $\max \left({\underset{\_}{x}}_1,{\underset{\_}{x}}_2,\dots ,{\underset{\_}{x}}_p\right)-\mu$ (and likewise for the minimum).

2.2. Estimation

We assume that we have a sample of size n arranged in ascending order, ${\underset{\_}{x}}_{(1:n)}\le {\underset{\_}{x}}_{\left(2:n\right)}\le \dots \le {\underset{\_}{x}}_{(n:n)}$. We chose $p\le n$ of them at random and we wish to find the probability that a specific one, say ${\underset{\_}{x}}_{(i:n)}$, is the maximum of all p. Apparently, if $i<p$ then this probability is zero. If $i\ge p$, then the total number of combinations in which the ${\underset{\_}{x}}_{(i:n)}$ is the maximum among the p variables equals the number of ways of choosing the $p-1$ remaining variables among the possible $i-1$ of them, i.e., $\left(\begin{array}{c}i-1\\ p-1\end{array}\right)$. The total number of ways to choose any p variables out of n is $\left(\begin{array}{c}n\\ p\end{array}\right)$. Thus, the sought probability, call it $b_{inp}$, is

$$b_{inp}=\left(\genfrac{}{}{0pt}{}{i-1}{p-1}\right)/\left(\genfrac{}{}{0pt}{}{n}{p}\right)$$

(12)

Note that

$$\sum _{i=p}^n\left(\genfrac{}{}{0pt}{}{i-1}{p-1}\right)=\left(\genfrac{}{}{0pt}{}{n}{p}\right)$$

(13)

and hence

$$\sum _{i=p}^n{b}_{inp}=1$$

(14)

as expected.

Given that the probability of ${\underset{\_}{x}}_{(i:n)}$ being the maximum of the p out of n variables is $b_{inp}$, the estimator of the expectation of the K-moment ${\underset{\_}{K}}_p^{\prime }$ will be

$${\underset{\_}{\widehat{K}}}_p^{\prime }=\sum _{i=1}^n{b}_{inp}{\underset{\_}{x}}_{\left(i:n\right)}$$

(15)

To find the estimator of the lower K-moments, it suffices to reverse the order of the sample, i.e., to replace ${\underset{\_}{x}}_{\left(i:n\right)}$ with ${\underset{\_}{x}}_{\left(n-i+1:n\right)}$. Hence,

$${\widehat{\overline{\underset{\_}{K}}}}_p^{\prime }=\sum _{i=1}^n{b}_{inp}{\underset{\_}{x}}_{\left(n-i+1:n\right)}=\sum _{i=1}^n{b}_{n-i+1,n,p}{\underset{\_}{x}}_{\left(i:n\right)}$$

(16)

If we use all integer values of $p$ from 1 to $n$, from the ordered sample $x_{\left(i:n\right)}$ we can calculate $n$ upper moments ${\widehat{K}}_p^{\prime }$ and $n$ lower moments ${\widehat{\overline{K}}}_p^{\prime }$, a total of $2n$ quantities. However, the information contained in each of the sequences ${\widehat{K}}_p^{\prime }$ and ${\widehat{\overline{K}}}_p^{\prime }$ is equivalent to each other:

Proposition 1.

The equivalence relationships of upper and lower moment estimators are:

$${\widehat{\overline{K}}}_p^{\prime }=-\sum _{i=1}^p{\left(-1\right)}^i\left(\begin{array}{c}p\\ i\end{array}\right){\widehat{K}}_i^{\prime },{\widehat{K}}_p^{\prime }=-\sum _{i=1}^p{\left(-1\right)}^i\left(\begin{array}{c}p\\ i\end{array}\right){\widehat{\overline{K}}}_i^{\prime }$$

(17)

The proof is given in [2].

The above formulation is for integer moment order p. We can readily generalize for real p by replacing factorials with the Gamma function. After algebraic manipulations, we obtain the final expression:

$$b_{inp}=\left\{\begin{array}{ll}0,& i<p\\ p\frac{\mathsf{\Gamma }\left(n-p+1\right)}{\mathsf{\Gamma }\left(n+1\right)}\frac{\mathsf{\Gamma }\left(i\right)}{\mathsf{\Gamma }\left(i-p+1\right)},& i\ge p\ge 0\end{array}\right.$$

(18)

In the algorithmic evaluation of Equation (18), it is suggested to first calculate the logarithms of the gamma functions and add them (or subtract, as appropriate), and then exponentiate the sum.

For $p=1$, Equation (18) results in $b_{in1}=1/n$ and thus we recover the estimate of the mean, ${\widehat{K}}_1^{\prime }={\widehat{\overline{K}}}_1^{\prime }=\widehat{\mu }$. For $p=n$, Equation (18) results in $b_{nnn}=1$ with all other $b_{inn}=0$, and thus only the maximum (or the minimum) value of the sample is taken into account in the estimation. For $p>n$, estimation becomes impossible. Thus, the method permits the estimation of moment orders as high as the sample size, n. As p increases from 1 to n, fewer sample items are used in the estimation until it remains only one for $p=n$.

The following result is important to note:

Proposition 2.

The estimators (15)–(16) with $b_{inp}$ determined from (18), are unbiased.

The proof is given in [2]. By replacing $\underset{\_}{x}$ with a transformation $g\left(\underset{\_}{x}\right)$ we readily obtain an estimator of transformed K-moments but unbiasedness cannot be assured in all cases.

The calculations to estimate K-moments are easy and follow the steps listed below, which are also illustrated in a spreadsheet that accompanies the paper as Supplementary Information:

• We sort the sample $x_j$ into ascending order, i.e., we designate $x_{\left(i:n\right)},i=1,\dots ,n$.
• For a specific moment order $p$, we employ Equation (18) to find $b_{inp}$ for $i=1,\dots ,n$.
• From Equation (15), we calculate ${\widehat{K}}_p^{\prime }$.
• From Equation (16) we calculate ${\widehat{\overline{K}}}_p^{\prime }$.
• We repeat steps 2–4 for all required orders p.

Often the data include ties either because the stochastic variable of interest is of discrete type or because they are too many and are summarized in more manageable form. In this case, the calculations can be simplified by applying a single coefficient to each value appearing in the sample. Assuming that a certain value $x_j$ appears $j$ times in the sample, namely from positions $j_1$ to $j_2$ in the ordered sample, i.e., $x_{(j_1:n)}=\cdots =x_{(j_2:n)}=x_j$, with ${j=j}_2-j_1+1$, the value $x_j$ should be multiplied by the sum $\sum _{i=j_1}^{j_2}{b}_{inp}$. This sum is easy to calculate analytically, resulting in a concise expression:

$$b_{\left(j_1,j_2\right),n,p}:=\sum _{i=j_1}^{j_2}{b}_{inp}=\frac{j_2}{p}{b}_{j_{2}np}-\frac{j_1-p}{p}{b}_{j_{1}np}$$

(19)

It is easy to verify that for $j_2=j_1$, the result is $b_{\left(j_1,j_2\right),n,p}=b_{j_{1}np}$ as it should. Moreover, if $j_2=j_1-1$, which means that there is no appearance of a particular value ($j=0)$, then the result is $b_{\left(j_1,j_2\right),n,p}=0$, as required.

An illustration of the good performance of the estimation tactic provided by Equation (19) follows. A sample of 10,000 values was generated from the generalized Pareto distribution with mean $\mu =1$ and tail index $\xi =0.1$ (so that its scale parameter is $\lambda =0.9$). Its distribution function is shown in

Table 1. Equations of the moments of the generalized Pareto distribution (adapted from [2]) ¹.

Characteristic	Equation	Equation No.
Distribution function	$F\left(x\right)=1-{\left(1+\xi x/\lambda \right)}^{-1/\xi }$	(20)
Classical noncentral moment	${\mu }_p^{\prime }:=\mathrm{E}\left[x^p\right]=p{\left(\lambda /\xi \right)}^p\mathsf{{\rm B}}\left(p,1/\xi -p\right)$	(21)
Upper K-moment	$K_p^{\prime }:=\mathrm{E}\left[{\underset{\_}{x}}_{\left(p\right)}\right]=\left(\lambda /\xi \right)\left(p\mathrm{B}\left(p,1-\xi \right)-1\right)$	(22)
Lower K-moment	${\overline{K}}_p^{\prime }:=\mathrm{E}\left[{\underset{\_}{x}}_{\left(1:p\right)}\right]=\lambda /(p-\xi )$	(23)

¹ $\mathrm{B}(a,b)$ is the beta function.

, along with the expressions of the classical moments, and upper and lower K-moments, all of which are closed analytical expressions (see [2]). These expressions are evaluated for the above parameter values for order $p$ from 1 to 10,000, and the results are plotted in Figure 1. Note that for better legibility, the quantities ${{\mu }_p^{\prime }}^{1/p}$ were plotted for the classical moments, and that these diverge to infinity for $p\ge 1/\xi =10$. The classical moments were also estimated by the standard statistical estimators; naturally, these do not diverge for any p but rather for the maximum value p = 10,000 the quantity ${{\widehat{\mu }}_p^{\prime }}^{1/p}$ equals the maximum sample value, which happens to be 15.4. As a result, the estimates start to depart for the theoretical values even for low p (>4) and the difference is infinite for $p\ge 10$.

The upper and lower K-moments were estimated in two ways. In the detailed estimation, the entire synthetic sample of 10,000 values was used with Equations (15)–(18). In the summary estimation, the 10,000 values were grouped into 110 classes, and for each class $i=1,\dots ,110$ with x values ranging in $[a_i,a_{i+1})$ the x values were replaced by the midpoint $x_i={(a}_i+a_{i+1})/2$ and the estimation was made using Equation (19). It can be observed in Figure 1 that (a) the summary estimates are almost indistinguishable from the detailed estimates and (b) both estimates are almost indistinguishable from the theoretical values. These observations support the reliability of the K-moment concept. Furthermore, they suggest that when the sample size is large, a summary estimation is equally reliable and algorithmically much faster.

For the reader’s convenience, the calculations to produce Figure 1 as well as the figure per se are also provided in a spreadsheet that accompanies the paper as Supplementary Information.

Apparently, if the stochastic variable of interest is of discrete type, the estimation will always be made using Equation (19).

2.3. Theoretical Calculations for Continuous Stochastic Variables, q = 1

If $\underset{\_}{x}$ is a continuous stochastic variable, then the maximum of p variables, i.e., the variable ${\underset{\_}{x}}_{\left(p\right)}=\max \left({\underset{\_}{x}}_1,{\underset{\_}{x}}_2,\dots ,{\underset{\_}{x}}_p\right)$, will have distribution and density functions, respectively,

$$F^{\left(p\right)}\left(x\right)={F\left(x\right)}^p,f^{\left(p\right)}\left(x\right)=pf\left(x\right){F\left(x\right)}^{p-1}$$

(24)

where the former is the product of p instances of $F\left(x\right)$ (justified by the fact that the variables ${\underset{\_}{x}}_i$ are independent copies of $\underset{\_}{x}$, by definition of the sample concept), while the latter is the derivative of $F^{\left(p\right)}\left(x\right)$ with respect to $x$. It is then readily verified that the following proposition holds true.

Proposition 3.

For a continuous stochastic variable $\underset{\_}{x}$, the upper and lower K-moments of order p, are, respectively:

$$K_p^{\prime }=\mathrm{E}\left[{\underset{\_}{x}}_{\left(p\right)}\right]=\mathrm{E}\left[\max \left({\underset{\_}{x}}_1,{\underset{\_}{x}}_2,\dots ,{\underset{\_}{x}}_p\right)\right]=p\mathrm{E}\left[{\left(F\left(\underset{\_}{x}\right)\right)}^{p-1}\underset{\_}{x}\right]$$

(25)

$${\overline{K}}_p^{\prime }=\mathrm{E}\left[{\underset{\_}{x}}_{\left(1:p\right)}\right]=\mathrm{E}\left[\min \left({\underset{\_}{x}}_1,{\underset{\_}{x}}_2,\dots ,{\underset{\_}{x}}_p\right)\right]=p\mathrm{E}\left[{\left(\overline{F}\left(\underset{\_}{x}\right)\right)}^{p-1}\underset{\_}{x}\right]$$

(26)

Hence, with reference to Equation (11), the adjustment factors are $A\left(\underset{\_}{x},p\right)=\overline{A}\left(\underset{\_}{x},p\right)=1$ for continuous variables. It is easy to see that the sequence of $K_p^{\prime }$ is non-decreasing and that of ${\overline{K}}_p^{\prime }$ is non-increasing as p increases.

We note that the calculation of $K_p^{\prime }$ can be made by

$$K_p^{\prime }=p\mathrm{E}\left[{\left(F\left(\underset{\_}{x}\right)\right)}^{p-1}\underset{\_}{x}\right]=p\underset{-\infty }{\overset{\infty }{\int }}{\left(F\left(x\right)\right)}^{p-1}xf\left(x\right)\mathrm{d}x=p\underset{0}{\overset{1}{\int }}x\left(F\right)F^{p-1}\mathrm{d}F$$

(27)

where $x\left(F\right)$ is the inverse of function $F\left(x\right)$, known as the quantile function. Therefore, we can write

$$G\left(p\right)=\underset{0}{\overset{1}{\int }}x\left(F\right)F^{p-1}\mathrm{d}F,G\left(p\right):=\frac{K_p^{\prime }}{p}$$

(28)

In other words, the function $G\left(p\right)$ is the finite Mellin transform [13] of the quantile function $x\left(F\right)$. Consequently, if we know the K-moments $K_p^{\prime }$ for any $p$, and hence the function $G\left(p\right)$, by inverting the transform we can find the quantile function $x\left(F\right)$ and hence the distribution function $F\left(x\right)$. A similar result can be obtained for the lower K-moments ${\overline{K}}_p^{\prime }$.

2.4. Theoretical Calculations for Discrete Stochastic Variables, q = 1

If $\underset{\_}{x}$ is a discrete stochastic variable taking on the values $x_j,j=\mathrm{0,1},\dots ,J$, with probability mass function $P_j:=P\left\{\underset{\_}{x}=x_j\right\}=F\left(x_j\right)-F\left(x_{j-1}\right)$, then $P\left\{{\underset{\_}{x}}_{\left(p\right)}\le x_i\right\}=F{\left(x_i\right)}^p$ and, hence, the probability mass function of ${\underset{\_}{x}}_{\left(p\right)}$ will be

$$P_j^{\left(p\right)}:=P\left\{{\underset{\_}{x}}_{\left(p\right)}=x_j\right\}=P\left\{{\underset{\_}{x}}_{\left(p\right)}\le x_j\right\}-P\left\{{\underset{\_}{x}}_{\left(p\right)}<x_j\right\}=F{\left(x_j\right)}^p-F{\left(x_{j-1}\right)}^p$$

(29)

where we use the convention $F\left(x_{-1}\right)=0$. Consequently, the upper K-moment of order p is

$$K_p^{\prime }=\mathrm{E}\left[{\underset{\_}{x}}_{\left(p\right)}\right]=\sum _{j=0}^J\left(F{\left(x_j\right)}^p-F{\left(x_{j-1}\right)}^p\right)x_j$$

(30)

By expanding and making algebraic manipulations, we find

$$K_p^{\prime }=x_J-\sum _{j=0}^{J-1}F{\left(x_j\right)}^p\left(x_{j+1}-x_j\right)$$

(31)

In the most common case in which $x_j=j$ the following proposition holds true:

Proposition 4.

For a discrete stochastic variable $\underset{\_}{x}$, taking on values $j=0,\dots ,J$, where J is finite or infinite, the upper K-moment of order p, is, respectively:

$$K_p^{\prime }=J-\sum _{j=0}^{J-1}F{\left(j\right)}^p$$

(32)

$$K_p^{\prime }=\sum _{j=0}^{\infty }\left(1-F{\left(j\right)}^p\right)$$

(33)

For infinite $J$ and for large $j$, $F\left(j\right)\to 1$. Thus, the expression in parenthesis in Equation (33) tends to zero. Therefore, we can easily evaluate $K_p^{\prime }$ numerically from (32), and by choosing a large (but finite) upper limit $J$, the convergence is fast.

The above relationships do not use the general formula (9). If we want to use it, we should evaluate $A\left(x_i,p\right)$. From Equation (30) we obtain

$$K_p^{\prime }=\mathrm{E}\left[{\underset{\_}{x}}_{\left(p\right)}\right]=\sum _{j=0}^{J}F{\left(x_j\right)}^{p-1}\frac{\left(F\left(x_j\right)-F{\left(x_{j-1}\right)}^p/F{\left(x_j\right)}^{p-1}\right)}{p{P}_j}{P_{J}x}_j$$

(34)

Thus, by comparing this with Equation (9), we find

$$\begin{gathered} A\left(x_j,p\right)=\frac{F\left(x_j\right)-F{\left(x_{j-1}\right)}^p/F{\left(x_j\right)}^{p-1}}{p{P}_j} \\ =\frac{F\left(x_j\right)}{p\left(F\left(x_j\right)-F\left(x_{j-1}\right)\right)}\left(1-{\left(\frac{F\left(x_{i-1}\right)}{F\left(x_i\right)}\right)}^p\right) \end{gathered}$$

(35)

and finally

$$A\left(x_j,p\right)=\frac{1-{\left(F\left(x_{j-1}\right)/F\left(x_j\right)\right)}^p}{p\left(1-F\left(x_{j-1}\right)/F\left(x_j\right)\right)}$$

(36)

As j increases, $F\left(x_{j-1}\right)/F\left(x_j\right)\to 1$, and it is easy to see that

$$\underset{j\to \infty }{\lim }A\left(x_j,p\right)=1$$

(37)

thus, approaching the behaviour seen in continuous variables. On the other hand, for varying p

$$\underset{p\to 0}{\lim }A\left(x_j,p\right)=\frac{-\ln \left(F\left(x_{j-1}\right)/F\left(x_j\right)\right)}{1-F\left(x_{j-1}\right)/F\left(x_j\right)},\underset{p\to 1}{\lim }A\left(x_j,p\right)=1,\underset{p\to \infty }{\lim }A\left(x_j,p\right)=0$$

(38)

If one sets $A\left(x_j,p\right)=1$ to approximate $K_p^{\prime }$, the error is prohibitively large. This is illustrated in Figure 2 for two of the most common discrete-variable distributions, Poisson and geometric.

The Poisson distribution has probability mass, distribution, and characteristic K-moments, respectively,

$$P_j=e^{-\lambda }\frac{{\lambda }^j}{j!},F\left(x\right)=e^{-\lambda }\sum _{j=0}^{⌊x⌋}\frac{{\lambda }^j}{j!},{{\rm K}}_1^{\prime }={{\rm K}}_{12}=\lambda$$

(39)

and the geometric distribution

$$\begin{gathered} P_j=\lambda {\left(1-\lambda \right)}^j,F\left(x\right)=1-{\left(1-\lambda \right)}^{⌊x⌋+1}, \\ {{\rm K}}_1^{\prime }=\frac{1-\lambda }{\lambda },{{\rm K}}_2^{\prime }=\frac{\left(1-\lambda \right)\left(3-\lambda \right)}{\lambda \left(2-\lambda \right)},{\overline{{\rm K}}}_2^{\prime }=\frac{{\left(1-\lambda \right)}^2}{\lambda \left(2-\lambda \right)},{{\rm K}}_{12}=\frac{1-\lambda }{{\lambda }^2} \end{gathered}$$

(40)

Coming now to the lower K-moments, the probability mass function of ${\underset{\_}{x}}_{(1:p)}$ will be

$${\overline{P}}_j^{\left(p\right)}:=P\left\{{\underset{\_}{x}}_{\left(1:p\right)}=x_j\right\}=P\left\{{\underset{\_}{x}}_{\left(1:p\right)}\ge x_j\right\}-P\left\{{\underset{\_}{x}}_{\left(p\right)}>x_j\right\}=\overline{F}{\left(x_{j-1}\right)}^p-\overline{F}{\left(x_j\right)}^p$$

(41)

where we set $\overline{F}\left(x_{-1}\right)=1$. Consequently, the upper K-moment of order p is

$${\overline{K}}_p^{\prime }=\mathrm{E}\left[{\underset{\_}{x}}_{\left(1:p\right)}\right]=\sum _{j=0}^J\left(\overline{F}{\left(x_{j-1}\right)}^p-\overline{F}{\left(x_j\right)}^p\right)x_j$$

(42)

By expanding and making algebraic manipulations, we find

$${\overline{K}}_p^{\prime }=x_0+\sum _{j=0}^{J-1}\overline{F}{\left(x_j\right)}^p\left(x_{j+1}-x_j\right)$$

(43)

and in the most common case in which $x_j=j$, the following proposition holds true:

Proposition 5.

For a discrete stochastic variable $\underset{\_}{x}$, taking on values $j=0,\dots ,J$, where J is finite or infinite, the lower K-moment of order p, is, respectively:

$${\overline{K}}_p^{\prime }=\sum _{j=0}^{J-1}\overline{F}{\left(j\right)}^p$$

(44)

$${\overline{K}}_p^{\prime }=\sum _{j=0}^{\infty }\overline{F}{\left(j\right)}^p$$

(45)

Again, in the latter case, the convergence is fast and therefore we can easily evaluate ${\overline{K}}_p^{\prime }$ numerically by choosing a large J; notice that for large $J$, $\overline{F}\left(J\right)\to 0$, and the sum will not change if we choose an even larger $J$.

2.5. Theoretical Calculations for Continuous Stochastic Variables, q > 1

As already mentioned, it is possible to generalize the K-moments for transformations of the original variable $\underset{\_}{x}$. Here we consider the transformation $\underset{\_}{y}:=g\left(\underset{\_}{x}\right)=({\underset{\_}{x}-c)}^q$, where c is a real constant and $q\ge 1$ is an integer. This transformation will allow recovering the classical moments, raw and central, as special cases of the K-moments for $p=1,q>1$, which has some usefulness as will be seen in Section 3.1.

There are two distinct cases, depending on whether q is odd or even. We start with the former case, which is easier because the transformation is a monotonic (increasing) function. In this case, as demonstrated in Appendix A, we have

$$K_{p_y}^{\prime }=p\underset{-\mathrm{\infty }}{\overset{\mathrm{\infty }}{\int }}{F\left(x\right)}^{p-1}{\left(x-c\right)}^{q}f\left(x\right)\mathrm{d}x$$

(46)

For $c=0$ and for $c=\mu$, we obtain the following results.

Proposition 6.

For a continuous stochastic variable $\underset{\_}{x}$, the upper and lower K-moments of orders p,q where q is an odd integer, are, respectively:

$$K_{pq}^{\prime }=p\mathrm{E}\left[{F\left(\underset{\_}{x}\right)}^{p-1}{\underset{\_}{x}}^q\right],{\overline{K}}_{pq}^{\prime }=p\mathrm{E}\left[{\overline{F}\left(\underset{\_}{x}\right)}^{p-1}{\underset{\_}{x}}^q\right]$$

(47)

and the respective central K-moments are, respectively:

$$K_{pq}=p\mathrm{E}\left[{F\left(\underset{\_}{x}\right)}^{p-1}{\left(\underset{\_}{x}-\mu \right)}^q\right],{\overline{K}}_{pq}=p\mathrm{E}\left[{\overline{F}\left(\underset{\_}{x}\right)}^{p-1}{\left(\underset{\_}{x}-\mu \right)}^q\right]$$

(48)

This means that, again, $A\left(\underset{\_}{x},p\right)=\overline{A}\left(\underset{\_}{x},p\right)=1$.

Next, we consider the case where q is even, in which, as shown in Appendix A, we have

$$K_{p_y}^{\prime }=p\underset{-\mathrm{\infty }}{\overset{\mathrm{\infty }}{\int }}{\left(F\left(c+|x-c|\right)-F\left(c-|x-c|\right)\right)}^{p-1}{\left(x-c\right)}^{q}f\left(x\right)\mathrm{d}x$$

(49)

For $c=0$, we obtain the following result.

Proposition 7.

For a continuous stochastic variable $\underset{\_}{x}$, the upper K-moment of orders p,q where q is an even integer, is:

$$K_{pq}^{\prime }=p\mathrm{E}\left[{\left(F\left(\left|\underset{\_}{x}\right|\right)-F\left(-\left|\underset{\_}{x}\right|\right)\right)}^{p-1}{\underset{\_}{x}}^q\right]$$

(50)

and in the case that the variable is non-negative, it simplifies to

$$K_{pq}^{\prime }=p\mathrm{E}\left[{F\left(\underset{\_}{x}\right)}^{p-1}{\underset{\_}{x}}^q\right]$$

(51)

This means that for non-negative variables, again, $A\left(\underset{\_}{x},p\right)=1$.

For $c=\mu$, we find the upper central moments as follows:

Proposition 8.

For a continuous stochastic variable $\underset{\_}{x}$, the upper central K-moment of orders p,q where q is an even integer, is:

$$K_{pq}=p\mathrm{E}\left[{\left(F\left(\mu +|\underset{\_}{x}-\mu |\right)-F\left(\mu -|\underset{\_}{x}-\mu |\right)\right)}^{p-1}{\left(\underset{\_}{x}-\mu \right)}^q\right]$$

(52)

This means that, in this case, $A\left(\underset{\_}{x},p\right)\ne 1$. Specifically,

$${A\left(\underset{\_}{x},p\right)=\left(\frac{F\left(\mu +|\underset{\_}{x}-\mu |\right)-F\left(\mu -|\underset{\_}{x}-\mu |\right)}{F\left(\underset{\_}{x}\right)}\right)}^{p-1}$$

(53)

In the special case that the distribution is symmetric about $\mu$, as happens, e.g., in the normal distribution, it holds that $F\left(\mu -|\underset{\_}{x}-\mu |\right)=1-F\left(\mu +|\underset{\_}{x}-\mu |\right)$. Hence, we can write the following:

Proposition 9.

For a continuous stochastic variable $\underset{\_}{x}$ with a symmetric distribution, the upper central K-moment of orders p,q where q is an even integer, is:

$$K_{pq}=p\mathrm{E}\left[{\left|2F\left(\underset{\_}{x}\right)-1\right|}^{p-1}{\left(\underset{\_}{x}-\mu \right)}^q\right]=2p\underset{\mu }{\overset{\mathrm{\infty }}{\int }}{\left(2F\left(x\right)-1\right)}^{p-1}{\left(x-\mu \right)}^{q}f\left(x\right)\mathrm{d}x$$

(54)

Illustrations of the above results are given in Figure 3 for $q=2$, for the normal and exponential distributions, and for both noncentral and central upper K-moments. The exact values are compared to approximations obtained by the simplification $A\left(x,p\right)=1$. In the case of noncentral upper K-moments of the exponential distribution, $A\left(x,p\right)=1$ is exact, but this also gives good approximations in other cases, unless the distribution is symmetric (such as the normal distribution) and the moments central.

3. Applications and Advantages

3.1. Evaluation of Classical Moment Estimates

As we have seen (Equation (10)), the classical moments can be recovered as special cases of K-moments. Undoubtedly, the classical moments (and tools related to them such as cumulants) are very useful theoretical concepts. If the distribution function is specified, their true values can be derived easily. However, their estimation from samples is problematic for moment order beyond 3–4 (hence the term unknowable). In contrast, K-moments can be estimated reliably for high orders p (hence their name knowable), provided that the order q is low. Furthermore, K-moments can also predict the value of the estimates of the classical moments, which in fact is not the true value of the classical moment. This may sound paradoxical, given that the classical estimator

$${\widehat{\underset{\_}{\mu }}}_p^{\prime }:=\frac{1}{n}\sum _{i=1}^n{\underset{\_}{x}}_i^p$$

(55)

is unbiased for order p, however large. In practice, however, the convergence of ${\widehat{\mu }}_p^{\prime }$ to ${\mu }_p^{\prime }$ is extraordinarily slow. Thus, the value of ${\widehat{\mu }}_p^{\prime }$ deviates from ${\mu }_p^{\prime }$. This deviation has been termed [2] the slow convergence bias, because, theoretically speaking, based on the bias definition, there is no bias per se. The K-moments can give us an indication of what we can anticipate for the value of ${\widehat{\mu }}_p^{\prime }$. By examining the moment estimators, we establish relationships between K- and classical moments broader than Equation (10).

Specifically, as shown by Koutsoyiannis [2], the following relationship links the estimated with the true classical moment via the K-moments:

$${\widehat{\mu }}_p^{\prime }\approx \frac{K_{n1}^{{\prime }^p}}{K_{np}^{\prime }}{\mu }_p^{\prime }$$

(56)

If the classical moment ${\mu }_p^{\prime }$ is estimated as the average of m independent samples, each of size 𝑛, then the product $mn$ should be substituted for $n$ in Equation (56).

We illustrate the relationship of K- and classical moments using Monte Carlo simulation for the exponential distribution with a lower bound of zero and a scale parameter of 1 and the normal distribution with a mean of $\mu =1$ and a standard deviation of $\sigma =1$. The exponential distribution has simple expressions of its moments, i.e.,

$${\mu }_p^{\prime }=K_{1p}^{\prime }=p!,K_p^{\prime }=H_p,{\overline{K}}_p^{\prime }=1/p$$

(57)

where $H_p$ denotes the pth harmonic number. For q > 2, the K-moment $K_{pq}^{\prime }$ does not have a closed analytical expression but its calculation can be easily performed by numerical integration. For the normal distribution, the classical noncentral moments can be determined by a recursive relationship involving cumulants [2], which, in this particular case, takes the form

$${\mu }_p^{\prime }=\mu {\mu }_{p-1}^{\prime }+\left(p-1\right){\sigma }^2{\mu }_{p-2}^{\prime }$$

(58)

For the K-moments of the normal distribution and for $p\le 4$, the related expressions are contained in Table 4. For $p>4$, no analytical solution exists but numerical integration can readily give the K-moment values.

Figure 4 shows the simulation results for sample size $n=100$. The theoretical (true) moments, of orders 1 to 100, of the two distributions are compared to the empirical estimations from a single sample, as well as from an ensemble of m samples (by averaging m estimates where $m=1000$ and 10 for the exponential and normal distribution, respectively). For the exponential distribution, the single sample estimates are lower than the true moments for p ≥ 4 and the deviation increases with $p$, approaching one order of magnitude as p approaches n. For the normal distribution, the deviations are somewhat smaller.

From a practical point of view—and despite the theoretical guarantee that the classical moment estimates are unbiased—Figure 4 shows that these estimates depart from the true moments. Even the average estimates from 1000 different synthetic samples deviate substantially from the true moments for order p > 10. On the other hand, Equation (56), also plotted in Figure 4, very aptly represents the behaviour of the estimates in all cases.

In brief, the classical moment estimators do not estimate the classical moments but rather some hybrid quantities involving both classical and K-moments, as seen in Equation (56).

It should be noted that Equation (56) was extracted for non-negative stochastic variables. However, it can also be applied to variables that can also take negative values, provided that the mean is positive. This was the case for the illustrated normal distribution with $\mu =\sigma =1$ (Figure 4, right panel), which takes negative values with a probability of 16%.

3.2. Summary Statistics

Because the classical moments beyond order 3–4 are unknowable (from a sample), it has been a customary practice to use those for orders p from 1 to 4 only. These indicate the location, variability, skewness and kurtosis of a distribution, often called summary statistics. For $p=\mathrm{2,3},4$, the central classical moments are used after standardization by the variance raised in a proper power so that a nondimensional metric is obtained.

In using K-moments, as we have seen, we can estimate moments of very high orders, which are particularly useful for the study of extremes [2], as well as for simulation from non-normal distributions [14]. Yet we can use the K-moments of orders p from 1 to 4, to derive summary statistics, such as in the classical moments. To this aim, we enrol the notion of the discrete derivative of K-moments at zero. Earlier studies [1,2] have used the notion of hypercentral K-moments, which for parsimony we avoid defining and using here.

We set $k_j=K_{j-1}^{\prime }$ for $j\ge 0$ and $k_j={\overline{K}}_{-j+1}^{\prime }$ for $j\le 0$ and we determine the (centred) discrete derivatives of $k_j$ at $j=0$. The discrete derivatives are linear functions of $k_j$ terms, with coefficients given in [15] and reproduced in columns 2–8 of

Table 2. Calculation of discrete derivatives of K-moments.

Derivative	$\mathbf{Coefficients}\mathbf{of}{\mathit{k}}_{\mathit{j}}={\mathit{K}}_{\mathit{j}-1}^{\mathbf{\prime }}$$\mathbf{or}{\mathit{k}}_{\mathit{j}}={\overline{\mathit{K}}}_{\mathit{j}-1}^{\mathbf{\prime }}\mathbf{for}\mathbf{Lag}\mathit{j}$							Derivative Expression *	Result *
	−3	−2	−1	$0$	1	2	3
0				1				$K_1^{\prime }={\overline{K}}_1^{\prime }=\mu$	$K_1^{\prime }=\mu$
1			−1/2	0	½			$\frac{1}{2}\left(K_2^{\prime }-{\overline{K}}_2^{\prime }\right)$	$K_2$
2			1	−2	1			$\left[K_2^{\prime }+{\overline{K}}_2^{\prime }-2K_1^{\prime }\right]$	$\left[0\right]$
3		−1/2	1	0	−1	½		$\left[\frac{1}{2}\left(K_3^{\prime }-{\overline{K}}_3^{\prime }-2K_2^{\prime }+{2\overline{K}}_2^{\prime }\right)\right]$	$\left[-\frac{1}{2}{{\rm K}}_2\right]$
4		1	−4	6	−4	1		$K_3^{\prime }+{\overline{K}}_3^{\prime }-4K_2^{\prime }-4{\overline{K}}_2^{\prime }+6K_1^{\prime }$	$2K_3-3K_2$
5	−1/2	2	−5/2	0	5/2	−2	½	$\frac{1}{2}\left(K_4^{\prime }-{\overline{K}}_4^{\prime }-4K_3^{\prime }+4{\overline{K}}_3^{\prime }+5K_2^{\prime }-{5\overline{K}}_2^{\prime }\right)$	$K_4-2K_3+2{{\rm K}}_2$

* The expressions in square brackets are not used as they do not contain additional information.

. The calculations are presented in Table 2, where to find the final results we eliminated the lower K-moments as these can also be expressed in terms of upper K-moments. We notice in Table 2 that the second derivative is, by identity, zero and the third derivative is a multiple of the first. Therefore, these are skipped as they do not add any information. Our final result is this:

Proposition 10.

The mean and dispersion of a probability distribution are expressed by the zeroth, and first discrete derivatives of $k_j$ at $j=0$ and are equal to $K_1^{\prime }$ and $K_2$, respectively. The skewness and kurtosis of a probability distribution are expressed by the fourth and fifth discrete derivatives, respectively, of $k_j$ at $j=0$, standardized by $K_2$. These four summary statistics are expressed in terms of K-moments of first to fourth orders.

These summary statistics are shown in

Table 3. Standardized summary statistics derived by discrete derivatives of K-moments for $q=1$ or the zeroth derivative for $p=1$; the latter coincide with the classical summary statistics.

Characteristic	$\mathit{q}=1$	$\mathit{p}=1$
Location	$K_1^{\prime }={\overline{K}}_1^{\prime }=\mu$
Dispersion	$K_2$	$K_{12}$
Skewness	$2\frac{K_3}{K_2}-3$	$\frac{K_{13}}{{{\rm K}}_{12}^{3/2}}$
Kurtosis	$\frac{K_4}{K_2}-2\frac{K_3}{K_2}+2$	$\frac{K_{14}}{K_{12}^2}$

. We can also recover the classical summary statistics by increasing $q$ from 1 to 4, keeping $p=1$ and using the zeroth derivative alone (or else not taking derivatives at all). These are also shown in Table 3 and are identical to the classical summary statistics. Interestingly, the summary statistics for $q=1$ are related to those produced by L-moments [16], as it can be shown that

$$K_2={\lambda }_2,2\frac{K_3}{K_2}-3=\frac{{\lambda }_3}{{\lambda }_2},\frac{K_4}{K_2}-2\frac{K_3}{K_2}+2=\frac{1}{5}\frac{{\lambda }_4}{{\lambda }_2}+\frac{4}{5}$$

(59)

Apparently, as illustrated in Section 3.1, the sample estimates in the case $q=1$ are more reliable (closer to their theoretical values) than those for $p=1$. Some examples of theoretical values for customary distributions (normal, exponential, Pareto) are listed in

Table 4. Examples of summary statistics for customary distributions.

Case	Symbol	$\mathbf{Normal}(\mathit{\mu },\mathit{\sigma })$	$\mathbf{Exponential}\left(\mathit{\lambda }\right)$	$\mathbf{Generalized}\mathbf{Pareto}(\mathit{\lambda },\mathit{\xi })$
	$F\left(x\right)$	$\frac{1}{2}\mathrm{erfc}\left(\frac{\mu -x}{\sqrt{2}\sigma }\right)$	$1-e^{-x/\lambda }$	$1-{\left(1+\xi \frac{x}{\lambda }\right)}^{-1/\xi }$
$q=1$	$K_1^{\prime }$	$\mu$	$\lambda$	$\frac{\lambda }{1-\xi }$
	$K_2$	$\frac{\sigma }{\sqrt{\mathsf{\pi }}}=0.56419\sigma$	$\frac{\lambda }{2}=0.5\lambda$	$\frac{\lambda }{(1-\xi )(2-\xi )}$
	$K_3$	$\frac{3\sigma }{2\sqrt{\mathsf{\pi }}}=0.84628\sigma$	$\frac{5\lambda }{6}=0.83333\lambda$	$\frac{(5-\xi )\lambda }{(1-\xi )(2-\xi )(3-\xi )}$
	$K_4$	$\frac{6\mathrm{A}\mathrm{r}\mathrm{c}\mathrm{T}\mathrm{a}\mathrm{n}\left[\sqrt{2}\right]}{{\pi }^{3/2}}\sigma =1.02938\sigma$	$\frac{13\lambda }{12}=1.08333\lambda$	$\frac{(26-9\xi +{\xi }^2)\lambda }{(1-\xi )(2-\xi )(3-\xi )(4-\xi )}$
	$\mathrm{S}\mathrm{k}\mathrm{e}\mathrm{w}\mathrm{n}\mathrm{e}\mathrm{s}\mathrm{s},2\frac{K_3}{K_2}-3$	$0$	$\frac{1}{3}=0.33333$	$\frac{1+\xi }{3-\xi }$
	$\mathrm{K}\mathrm{u}\mathrm{r}\mathrm{t}\mathrm{o}\mathrm{s}\mathrm{i}\mathrm{s},\frac{K_4}{K_2}-2\frac{K_3}{K_2}+2$	$\frac{6}{\pi }\arctan \sqrt{2}-1=0.82452$	$\frac{5}{6}=0.83333$	$\frac{10-5\xi +{\xi }^2}{(3-\xi )(4-\xi )}$
$p=1$	$K_{12}={\mu }_2$	${\sigma }^2$	${\lambda }^2$	$\frac{{\lambda }^2}{{\left(1-\xi \right)}^2(1-2\xi )}$
	$K_{13}={\mu }_3$	0	${2\lambda }^3$	$\frac{2\left(1+\xi \right){\lambda }^3}{{\left(1-\xi \right)}^3(1-2\xi )(1-3\xi )}$
	$K_{14}={\mu }_4$	${3\sigma }^4$	${9\lambda }^4$	$\frac{3\left(3+\xi +{2\xi }^2\right){\lambda }^4}{{\left(1-\xi \right)}^4(1-2\xi )(1-3\xi )(1-4\xi )}$
	$\mathrm{S}\mathrm{k}\mathrm{e}\mathrm{w}\mathrm{n}\mathrm{e}\mathrm{s}\mathrm{s},\frac{K_{13}}{{{\rm K}}_{12}^{3/2}}$	0	2	$\frac{2\sqrt{1-2\xi }\left(1+\xi \right)}{1-3\xi }$
	$\mathrm{K}\mathrm{u}\mathrm{r}\mathrm{t}\mathrm{o}\mathrm{s}\mathrm{i}\mathrm{s},\frac{K_{14}}{K_{12}^2}$	3	9	$\frac{3(1-2\xi )\left(3+\xi +{2\xi }^2\right)}{(1-3\xi )(1-4\xi )}$

.

3.3. Estimation of Distribution Function

In an observed sample of size $n$, the ith smallest value is an estimate of the nth order statistic, ${\underset{\_}{x}}_{\left(i:n\right)}$. It is well known in statistics that the stochastic variable $\underset{\_}{u}:=F\left({\underset{\_}{x}}_{\left(i:n\right)}\right)$ has beta distribution with parameters $i$ and $n-i+1$ and hence its expected value is:

$$\mathrm{E}\left[\underset{\_}{u}\right]=\mathrm{E}\left[F\left({\underset{\_}{x}}_{\left(i:n\right)}\right)\right]=\frac{i}{n+1}$$

(60)

Equation (60) constitutes the most widely known and the most popular way of empirically assigning values of the distribution function (and return periods) to observed sample values. It is known as the Weibull plotting position (Weibull, 1939; [17]). Namely, to the observed value $x_{\left(i:n\right)}$ we assign a distribution function estimate $\widehat{F}\left(x_{\left(i:n\right)}\right)=i/(n+1)$, which is unbiased for $F\left(x_{\left(i:n\right)}\right)$ but not for transformations thereof or for $x_{\left(i:n\right)}$ per se. However, Equation (60) is not the earliest, as Hazen (1914; [18]) had proposed a different formula, $\widehat{F}\left(x_{\left(i:n\right)}\right)=(i-0.5)/n$, which looks similar but, as far as the maximum observation is concerned, the difference in the assigned exceedance probability (and return period) is dramatic, at a ratio of 2:1. Several other formulae have been proposed in the 20th century, which are listed in [2] (Table 5.4) and can be written in the general form

$$\widehat{F}\left(x_{\left(i:n\right)}\right)=\frac{i+A-1}{n+2A-1}$$

(61)

where $A$ is a constant $(0\le A\le 1)$. The formulae by Weibull and Hazen correspond to $A=1$ and $A=1/2$, respectively. Another common value is $A=5/8$, which was proposed by Blom (1958; [19]) for the normal distribution (see also [20]). The lowest possible value $A=0$ yields $\widehat{F}\left(x_{\left(i:n\right)}\right)=(i-1)/(n-1)$ and thus assigns the values 0 and 1 to the lowest and highest sample values, respectively.

The results for the highest and lowest values are

$$\widehat{F}\left(x_{\left(n\right)}\right)=\frac{n+A-1}{n+2A-1},\widehat{F}\left(x_{\left(1:n\right)}\right)=\frac{A}{n+2A-1}$$

(62)

These can also assign distribution function values to K-moments values, which are the expectations of the highest and lowest values:

$$\widehat{F}\left(K_p^{\prime }\right)=\frac{p+A-1}{p+2A-1},\widehat{F}\left({\overline{K}}_p^{\prime }\right)=\frac{A}{p+2A-1}$$

(63)

However, Koutsoyiannis [2] provided more accurate relationships for K-moment values, based on the following quantities:

Definition 7.

The quantities:

$${\Lambda }_p:=\frac{1}{p\overline{F}\left(K_p^{\prime }\right)},{\overline{\Lambda }}_p:=\frac{1}{pF\left({\overline{K}}_p^{\prime }\right)}$$

(64)

are termed the Λ-coefficients.

For given p and distribution function $F\left(x\right)$, the quantities $K_p^{\prime }$, ${\overline{K}}_p^{\prime }$, $\overline{F}\left(K_p^{\prime }\right)$, ${\Lambda }_p$ and ${\overline{\Lambda }}_p$ are analytically or numerically determined from their definitions, but this might be complicated. However, ${\Lambda }_p$ and ${\overline{\Lambda }}_p$ exhibit small variation with p, which makes a very good approximation possible if we first accurately determine (a) the values ${\Lambda }_1$ and ${\overline{\Lambda }}_1$ for $p=1$, and (b) the asymptotic values ${\Lambda }_{\infty }$ and ${\overline{\Lambda }}_{\mathrm{\infty }}$. The values ${\Lambda }_1$ and ${\overline{\Lambda }}_1$ are very easy to determine, as they refer to the probability of non-exceedance of the mean:

$${\Lambda }_1=\frac{1}{1-F\left(\mu \right)},{\overline{\Lambda }}_1=\frac{1}{F\left(\mu \right)}=\frac{{\Lambda }_1}{{\Lambda }_1-1}$$

(65)

and can also be reliably estimated from a sample by finding the proportion of sample values that are smaller than $\mu$. For symmetric distributions, $F\left(\mu \right)=1/2$ and thus ${\Lambda }_1={\overline{\Lambda }}_1=2$.

The asymptotic values can be found by utilizing results by Koutsoyiannis [2] and depend on the tail behaviour of the distribution, as shown in

Table 5. Asymptotic values of Λ-coefficients (based on [2]).

Characteristic	Definition of Tail Index 1	Asymptotic Λ	$\mathbf{Limit}\mathbf{for}\mathit{\zeta },{\mathit{\zeta }}^{\mathbf{\prime }},\mathit{\xi },{\mathit{\xi }}^{\mathbf{\prime }}=$				Equation No.
			0	1/2	1	$\mathit{\infty }$
Upper bounded by $c_{\mathrm{U}}$, tail index ${\zeta }^{\prime }$	$\underset{x\to c_{\mathrm{U}}}{\lim }(c_{\mathrm{U}}-{x)}^{-{\zeta }^{\prime }}\overline{F}(x)=l_1$	${\Lambda }_{\infty }=\mathsf{\Gamma }{\left(1+1/\zeta \prime \right)}^{-{\zeta }^{\prime }}$	0	$1/\sqrt{2}$	1	${\mathrm{e}}^{\mathsf{\gamma }}$	(66)
Upper unbounded, tail index $\xi$	$\underset{x\to \infty }{\lim }{x}^{1/\xi }\overline{F}\left(x\right)=l_2$	${\Lambda }_{\infty }=\mathsf{\Gamma }{\left(1-\xi \right)}^{1/\xi }$	${\mathrm{e}}^{\mathsf{\gamma }}$	$\mathsf{\pi }$	$\infty$	–	(67)
Lower bounded by $c_{\mathrm{L}}$, tail index $\zeta$	$\underset{x\to c_{\mathrm{L}}}{\lim }{(x-c_{\mathrm{L}})}^{-\zeta }F\left(x\right)=l_3$	${\overline{\Lambda }}_{\infty }=\mathsf{\Gamma }{\left(1+1/\zeta \right)}^{-\zeta }$	0	$1/\sqrt{2}$	1	${\mathrm{e}}^{\mathsf{\gamma }}$	(68)
Lower unbounded, tail index $\xi \prime$	$\underset{x\to -\infty }{\lim }{(-x)}^{1/{\xi }^{\prime }}F\left(x\right)=l_4$	${\overline{\Lambda }}_{\infty }=\mathsf{\Gamma }{\left(1-\xi \prime \right)}^{1/{\xi }^{\prime }}$	${\mathrm{e}}^{\mathsf{\gamma }}$	$\mathsf{\pi }$	$\infty$	–	(69)

¹ $l_i,i=1,\dots ,4$ are constants $<\infty$; $\mathsf{\Gamma }\left(a\right)$ is the gamma function; $\gamma =0.577216$ is the Euler constant and thus ${\mathrm{e}}^{\mathsf{\gamma }}=1.78107$.

. From the limits also listed in Table 5, we infer that when the distribution is upper or lower bounded, the upper or lower Λ-coefficient is in the range (0, ${\mathrm{e}}^{\mathsf{\gamma }}=1.78107$). Otherwise (when the distribution is unbounded), the Λ-coefficient is greater than ${\mathrm{e}}^{\mathsf{\gamma }}$. In unbounded variables that have finite variance (as happens with several natural processes), the tail index should be bounded by 1/2 ($\xi ,{\xi }^{\prime }<1/2$) and thus the upper or lower Λ-coefficient is in the range (${\mathrm{e}}^{\mathsf{\gamma }}=1.78107,\mathsf{\pi }=3.14159$).

For the most common case that the domain of the stochastic variable $\underset{\_}{x}$ is $[0,\infty )$ and the variance is finite, ${\Lambda }_{\infty }$ will be in the range $({\mathrm{e}}^{\mathsf{\gamma }},\mathsf{\pi }$), depending on the upper tail index $\xi$, and ${\overline{\Lambda }}_{\infty }$ will be in the range $(0,{\mathrm{e}}^{\mathsf{\gamma }}$), depending on the lower tail index $\zeta$. Both tail indices $\xi$ and $\zeta$ can be visualized in a double logarithmic plot of the ratio $F\left(x\right)/\overline{F}\left(x\right)$ vs. $x$. As $x\to 0$, given the definitions in Table 5, we will have $F\left(x\right)\propto x^{\zeta }$ and $F\left(x\right)/\overline{F}\left(x\right)\propto x^{\zeta }$. Therefore, $\zeta$ is the double logarithmic slope of $F\left(x\right)/\overline{F}\left(x\right)$ as $x\to 0$. On the other hand, as $x\to \infty$, we will have $1/\overline{F}\left(x\right)\propto x^{1/\xi }$ and $F\left(x\right)/\overline{F}\left(x\right)\propto x^{1/\xi }$. Therefore, $\xi$ is the inverse of the double logarithmic slope of $F\left(x\right)/\overline{F}\left(x\right)$ as $x\to \infty$.

Once the tail indices $\xi$ and $\zeta$ are known, the asymptotic Λ-coefficients can be determined from Table 5. Notably, most of the customary distributions, such as normal, lognormal, exponential, Gamma, Weibull, etc. (more specifically, those belonging to the domain of attraction of the Extreme Value Type I distribution; see [2], pp. 76–79 and 235–236), have an upper tail index $\xi =0$ and thus ${\Lambda }_{\infty }={\mathrm{e}}^{\mathsf{\gamma }}$.

Given the three quantities ${\Lambda }_1,$ ${\Lambda }_{\infty }$ and ${\overline{\Lambda }}_{\infty }$ (with ${\overline{\Lambda }}_1$ determined from Equation (65)), we can approximate ${\Lambda }_p$ for any order p by

$${\Lambda }_p\approx {\Lambda }_{\infty }+\frac{\left({\Lambda }_1-{\Lambda }_{\infty }\right)}{p},{\overline{\Lambda }}_p\approx {\overline{\Lambda }}_{\mathrm{\infty }}+\frac{{\overline{\Lambda }}_1-{\overline{\Lambda }}_{\mathrm{\infty }}}{p}$$

(70)

from which we find

$$\widehat{F}\left(K_p^{\prime }\right)\approx 1-\frac{1}{{\Lambda }_{\mathrm{\infty }}p+{\Lambda }_1-{\Lambda }_{\mathrm{\infty }}},\widehat{F}\left({\overline{K}}_p^{\prime }\right)\approx \frac{1}{{\overline{\Lambda }}_{\mathrm{\infty }}p+{\overline{\Lambda }}_1-{\overline{\Lambda }}_{\mathrm{\infty }}}$$

(71)

More accurate approximations are given in [2] (pp. 208–215), but these depend on the entire expression of distribution function—not only on its tail indices. Nonetheless, the approximation by Equation (71) suffices for most statistical tasks.

By comparing Equations (63) and (71), we deduce that the former is a special case of the latter, in which ${\Lambda }_1={\overline{\Lambda }}_1=2$ and ${\Lambda }_{\mathrm{\infty }}={\overline{\Lambda }}_{\mathrm{\infty }}=1/A$. We understand, thus, that the approximation by Equation (71), which is based on three independent parameters (${\Lambda }_1,$ ${\Lambda }_{\infty }$ and ${\overline{\Lambda }}_{\infty })$, is more accurate and flexible than that of plotting positions, i.e., Equations (61)–(63). There is an additional reason that makes the former even more accurate. Each $K_p^{\prime }$ or ${\overline{K}}_p^{\prime }$ (excepting the case $p=n$) is estimated by several sample values, while in Equations (61)–(63) each $x_{\left(i:n\right)}$ is a single data value. An additional advantage of using Equation (71) is that it can be applied for any desired p, integer or real (up to the sample size $n$) while the equations based on plotting positions work only for the individual values of the observed sample.

An illustration of the method for estimating the distribution function from samples is shown in Figure 5. This is for the generalized Pareto distribution and for a rather small sample size, $n=100$. Monte Carlo simulation was employed to generate 100 series from the same distribution and find the median at each p and the uncertainty band. The figure includes a Cartesian plot of the probability distribution function $F\left(x\right)$ (left), as well as a double logarithmic plot of the ratio $F\left(x\right)/\overline{F}\left(x\right)$. As discussed above, the latter enables visualizing the tails of the distribution. Note that this ratio is closely associated with the return period, a quantity widely used in engineering design [2], by

$$\frac{F\left(x\right)}{\stackrel{-}{F}\left(x\right)}=\frac{T-D}{D}=\frac{D}{\overline{T}-D}$$

(72)

where $T$ and $\overline{T}$ denote return periods of maxima and minima, respectively, and $D$ is the time step of observations.

The empirical estimates in Figure 5 are also compared with their theoretical values, also plotted in the figure. There is good agreement between the two, and the uncertainty band is fairly narrow.

When the stochastic variable of interest is discrete, the situation is more complicated [21,22]. In this case, the distribution function $F\left(x\right)$ is a discontinuous function with jumps at each possible realization of the stochastic variable, such as at the integers $j=\mathrm{0,1},\dots$. On the other hand, the K-moments, which are averages of maxima or minima, take on real values. We assume that there exists a distribution function, $F_{\mathrm{C}}\left(x\right)$, of a continuous variable, which yields the same K-moments as $F\left(x\right)$. Theoretically, this can be determined as follows. We first estimate the K-moments $K_p^{\prime }$ from the discrete $F\left(x\right)$ and we equate them with those of $F_{\mathrm{C}}\left(x\right)$. Knowing the latter, we use the inverse Mellin transform of (28) to estimate the quantile function $x_{\mathrm{C}}\left(F\right)$. By inverting the latter, we find $F_{\mathrm{C}}\left(x\right)$. However, this will be computationally cumbersome. Here we use the simplification:

$$F\left(x\right)=F_{\mathrm{C}}(⌊x⌋+c)$$

(73)

assuming a constant $c=0.25$, which was found optimal after numerical investigation. Based on this and assuming a broken-line form of $F_{\mathrm{C}}\left(x\right)$, we write

$$F_{\mathrm{C}}\left(x\right)=\left\{\begin{array}{ll}F\left(0\right)x/c& x\le c\\ F\left(⌊x-c⌋\right)+\left(F\left(⌊x-c⌋+1\right)-F\left(⌊x-c⌋\right)\right)\left(x-c-⌊x-c⌋\right),& x>c\end{array}\right.$$

(74)

This is illustrated for the Poisson distribution (Equation (39)) and geometric distribution (Equation (40)) in Figure 6. Both distributions belong to the domain of attraction of the Extreme Value I distribution and therefore ${\Lambda }_{\infty }={\mathrm{e}}^{\mathsf{\gamma }}$. From Equation (74), the lower tail is $\zeta =1$ and therefore ${\overline{\Lambda }}_{\mathrm{\infty }}=1$. Figure 6 is similar to Figure 5 and was constructed with the same method, except that the theoretical curve plotted is $F_{\mathrm{C}}\left(x\right)$ instead of $F\left(x\right)$. The reason is that it is the former that is consistent with the K-moment estimates, which indeed compare well with the theoretical expectation. The uncertainty bands are again fairly narrow.

3.4. Fitting a Distribution Function

The method of moments has been one of the standard techniques in fitting distribution functions to data. Another standard is the method of maximum likelihood, which is well reasoned and is based on an optimization logic. In contrast, the method of moments is based on solving equations and is not quite rigorously argued. Assuming that we fit a two-parameter model (say, a generalized Pareto distribution), the method of moments uses the first two classical (noncentral) moments and determines the two parameters by equating the sample moments to the theoretical moments of the distributions. A similar strategy is followed by the method of L-moments, which is regarded as more reliable than that of classical moments. However, Koutsoyiannis [2] raised the following two major questions on the logic of this strategy, particularly when the focus of the study is the occurrence of extremes:

• Why use the first and second moments and not, say, the second and third? One may easily justify the standard choice of using the lowest possible order of moments by the fact that higher moments are less accurately estimated. On the other hand, one may counter that, when we are interested in extremes, these are better reflected in higher-order moments. It is well known that a model can hardly be a perfect representation of reality. Thus, we cannot expect that a good model fitting on the first and second moment would be equally suitable for the distribution tail, i.e., the behaviour on extremes.
• Why use two moments and not more? The standard answer, that two equations suffice to find two unknowns, may be adequate from a theoretical mathematical point of view but it is not from an empirical and engineering one. (As the saying goes, to draw a straight line, a mathematician needs two points but an engineer needs three). Certainly, an optimization framework (as in maximizing likelihood or in minimizing fitting error) is much preferable and superior to an equation solving method.

The K-moments and their advantages, which are particularly strong for an extreme-oriented modelling, enable a different fitting strategy, which uses as many moments as required for the particular application. For example, assuming that we have a sample of size $n,$ we can estimate $n$ upper K-moments and $n$ lower K-moments for integer values of p. We can then use the former if we are interested in the body of the distribution and its upper tail, or the latter if we are interested in the body of the distribution and its lower tail, or we can use both sequences simultaneously. Alternatively, if the sample size is too large, we could choose a subset of them, e.g., $100$ values of p (not necessarily integer) arranged in a geometric progression from 1 to $n$. The objective of the fitting algorithm is to minimize the average mean square error (possibly weighted) between the empirical and theoretical K-F relationships.

Koutsoyiannis [2] discusses a possible criticism of using high-order K-moments (up to order $n$), which is the fact that the higher-order moments are more uncertain than the lower ones, and notes:

This criticism would be valid if the true distribution function was known to be the one chosen as a model for the real-world process studied. But this is hardly the case. Let us assume that in the time series of flow observations we have three very high values and that we have chosen a certain model, e.g., a Lognormal distribution. How can we be sure that the model is correct? If we are not sure (which actually is always the case), and if we are to design a certain engineering construction, would we prefer a fitting of the chosen model that is consistent with theoretical considerations, e.g., based on the maximum likelihood method, even if this yields a departure for the three high values? Or would we feel safer if our fitting represents well the three high values?

The steps of the fitting algorithm, adapted from [2], are outlined below. These steps are based on the upper K-moments, $K_p^{\prime }$, and are oriented toward a fitting that is good for the body of the distribution and the upper tail. If we are interested in extreme lows, we can substitute the lower K-moments ${\overline{K}}_p^{\prime }$ for the upper ones, and appropriately adapt the following steps. If we are equally interested in both tails, then we can use both upper and lower tails:

• We choose a number $m+1$ of moment orders, i.e., $p_i=n^{i/m},i=0,\dots ,m$, with $p_0=1,p_m=n$. The rationale for this is that when dealing with samples of size $n$ of the order of several thousands, the number m could be chosen much smaller, e.g., of the order of 100, to speed up calculations without compromising accuracy. The orders $p_i$ need not be natural numbers.
• We estimate the upper K-moments $K_{p_i}^{\prime }$ of orders $p_i,i=0,\dots ,m$ using Equations (15) and (18).
• Assuming default values of the distribution parameters, represented as a vector $\mathit{\lambda }$, we determine ${\Lambda }_1$ from Equation (65), the theoretical relationships between parameters of the specified distribution and the mean $\mu =K_1^{\prime }$ (such as those in Table 1 or Table 4) and the expression of the distribution function $F\left(x\right)$. Alternatively, but not preferably, we can estimate ${\Lambda }_1$ from the sample mean, along with Equation (65).
• As the vector $\mathit{\lambda }$ contains the tail index ξ, we determine ${\Lambda }_{\infty }$ from the relationships of Table 5.
• Given ${\Lambda }_1$ and ${\Lambda }_{\infty }$, for each $p_i$ we estimate the empirical distribution function value $\widehat{F}\left(K_{p_i}^{\prime }\right)$ from Equation (71).
• Given the parameter values in vector $\mathit{\lambda }$, for each K-moment $K_{p_i}^{\prime }$, we estimate the theoretical distribution function $F\left({\widehat{K}}_{p_i}^{\prime }\right)$ from the expression of the distribution function $F\left(x\right)$ setting $x={\widehat{K}}_{p_i}^{\prime }$.
• We form an expression for the total fitting error as the sum:

  $${\rm E}\left(\mathit{\lambda }\right):={\sum _{i=0}^m{w}_i\left(\ln \left(\frac{\widehat{F}\left(K_{p_i}^{\prime }\right)}{\widehat{\overline{F}}\left(K_{p_i}^{\prime }\right)}\right)-\ln \left(\frac{F\left({\widehat{K}}_{p_i}^{\prime }\right)}{\overline{F}\left({\widehat{K}}_{p_i}^{\prime }\right)}\right)\right)}^2$$

  (75)

  where $w_{\iota }$ denotes a weighting coefficient. The rationale of choosing the logarithmic deviations of the ratio $F\left(x\right)/\overline{F}\left(x\right)$, instead of $F\left(x\right)$, is that this better represents the distribution tails (see e.g., Figure 5, right). The error $E$ is a function of the chosen parameters $\mathit{\lambda },$ and we evaluate it for the chosen parameter set.
• We repeat the calculations of steps 3–7 for different sets of parameter vectors $\mathit{\lambda }$ until the fitting error becomes minimal. The repetitions are executed by a solver (available in every software environment) using as objective function (to be minimized) the error ${\rm E}\left(\mathit{\lambda }\right)$.

The procedure is typically very fast, almost instant, as the only step with computational burden, that in step 2, is executed only once. A default value of the weight is $w_i=1$. Two alternatives are:

$$w_i=\left\{\begin{array}{l}1,l\le K_{p_i}^{\prime }<u\\ 0,\mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\end{array}\right.,w_i={\left(\frac{\widehat{F}\left(K_{p_i}^{\prime }\right)}{\widehat{\overline{F}}\left(K_{p_i}^{\prime }\right)}\right)}^b$$

(76)

where the former focuses the fitting on a particular range of the variable, that between chosen lower and upper bounds $l$ and $u$, respectively, while the latter gives more emphasis on fitting to high values of the distribution function by choosing a constant $b>0$ (e.g., $b=0.5$).

Remarkably, the above procedure does not need evaluation of the theoretical K-moments per se (except for the mean $\mu =K_1^{\prime }$). While this evaluation is feasible for any distribution, it may be cumbersome as it may involve numerical integration. Notice in Equation (75) the quantity $\widehat{F}\left(K_{p_i}^{\prime }\right)$ is estimated from Equation (71) as a function of $p_i$ while $F\left({\widehat{K}}_{p_i}^{\prime }\right)$ is estimated from the data using Equations (15) and (18). This makes the procedure as simple as possible. In addition, it readily provides a visual comparison.

Several examples of distribution fitting using real world data are presented in [2]. Here we repeat, after adaptation, one of them for the sake of illustration. This is for hourly wind data of the MIT station in Boston, MA, USA (42.367° N, 71.033° W, 9.0 m), perhaps the longest worldwide. The data are available from the United States National Oceanic and Atmospheric Administration, but a great deal of effort was needed to convert them to hourly time series [23]. They cover, with minor gaps, the period of January 1945 to December 2014 (70 calendar years with a few missing values; a number of hourly values of $\mathrm{589,551}$, 1% of which are zero and were excluded from the analysis, which was made for the remaining $n=\mathrm{583,465}$ nonzero values). The model chosen is the ParetoBurr–Feller (PBF; [2]) distribution with four parameters (the tail indices $\zeta ,\xi$, the scale parameter $\lambda$ and the lower bound $x_{\mathrm{L}}$, whose characteristics that are needed for the fitting are:

$$\begin{gathered} F\left(x\right)=1-{\left(1+\zeta \xi {\left(\frac{x-x_{\mathrm{L}}}{\lambda }\right)}^{\zeta }\right)}^{-\frac{1}{\zeta \xi }}, \\ {\Lambda }_1={\left({1+\left(\frac{\mathrm{B}\left(\frac{1}{\zeta \xi }-\frac{1}{\zeta },\frac{1}{\zeta }\right)}{\zeta }\right)}^{\zeta }\right)}^{\frac{1}{\zeta \xi }},{\Lambda }_{\infty }={\mathsf{\Gamma }(1-\xi )}^{\frac{1}{\xi }},{\overline{\Lambda }}_{\infty }={\mathsf{\Gamma }\left(1+\frac{1}{\zeta }\right)}^{-\zeta } \end{gathered}$$

(77)

The resulting fit is shown in Figure 7 in terms of a plot of the ratio $F\left(x\right)/\overline{F}\left(x\right)$, which enables visualization of the tails. By comparing the theoretical curve with the empirical one based on K-moments, we see that the fitting is excellent for the entire range that can be observed from the sample, spanning 11 orders of magnitude of the ratio $F\left(x\right)/\overline{F}\left(x\right)$.

3.5. Estimation of Density Function

When assigning values of distribution function to the K-moments, the arrangement of point estimates thereof produces a smooth curve, which allows a direct estimate of the probability density. This is a strong advantage of the K-moments as it is the only method that can provide a detailed representation of the density function, replacing the rougher representation offered by the popular concept of the histogram [3].

Specifically, given the $2n-1$ sample estimates of K-moments ${\widehat{K}}_p^{\prime }$ and ${\widehat{\overline{K}}}_p^{\prime }$, namely the ordered values ${\widehat{\overline{K}}}_n^{\prime }\le {\widehat{\overline{K}}}_{n-1}^{\prime }\le \cdots \le {\widehat{\overline{K}}}_1^{\prime }\equiv {\widehat{K}}_1^{\prime }\le {\widehat{K}}_2^{\prime }\le \cdots {\widehat{K}}_{n-1}^{\prime }\le {\widehat{K}}_n^{\prime }$, and the estimates of the distribution function $\widehat{F}\left(K_p^{\prime }\right)$ and $\widehat{F}\left({\overline{K}}_p^{\prime }\right)$ from Equation (71), it is then straightforward to approximate the density function $f\left(x\right)=\mathrm{d}F\left(x\right)/\mathrm{d}x$ by the discrete derivative:

$$\widehat{f}\left(x\right)=\left\{\begin{array}{ll}\frac{\widehat{F}\left(K_{p+1}^{\prime }\right)-\widehat{F}\left(K_p^{\prime }\right)}{{\widehat{K}}_{p+1}^{\prime }-{\widehat{K}}_p^{\prime }},& {\widehat{K}}_p^{\prime }\le x<{\widehat{K}}_{p+1}^{\prime }\\ \frac{\widehat{F}\left({\overline{K}}_p^{\prime }\right)-\widehat{F}\left({\overline{K}}_{p+1}^{\prime }\right)}{{\widehat{\overline{K}}}_p^{\prime }-{\widehat{\overline{K}}}_{p+1}^{\prime }},& {\widehat{\overline{K}}}_{p+1}^{\prime }\le x<{\widehat{\overline{K}}}_p^{\prime }\end{array}\right.$$

(78)

This will result in $2n-2$ different values of the density $\widehat{f}\left(x\right)$, while it is possible to expand the number of estimation points by using non-integer orders p [3]. The method is illustrated in Figure 8 for the Generalized Pareto distribution, the same as in Figure 5, by means of Monte Carlo simulation with 100 realizations of samples of 100 items each. The estimated probability density, expressed in terms of the median of the simulations, harmonizes with the true shape of the probability density, and the uncertainty, depicted in terms of prediction limits, is low in the body of the distribution, but increases in the upper and lower tails.

3.6. Accounting for Estimation Bias in Time Series with Time Dependence

K-moments characterize the marginal (first order) distribution of a stochastic process and therefore are not affected by the dependence structure of the process. However, their estimators are affected. If we do not have a random sample of a stochastic variable, but rather a time series from a process with time dependence, then bias is induced to estimators of K-moments. Thus, unless the stochastic process is white noise, the estimator unbiasedness ceases to hold. It has been shown in [2] that the estimation bias is negligible in the case of processes with short-range dependence and those characterized by periodic oscillations.

However, in processes with long-range dependence (LRD), the bias can be substantial. LRD, also known as Hurst–Kolmogorov (HK) dynamics, is quantified by the Hurst parameter, H, which in turn is most conveniently defined through its climacogram as follows. Let us consider any stochastic process $\underset{\_}{x}\left(t\right)$ in continuous time t, and define the cumulative process as

$$\underset{\_}{X}\left(k\right):=\underset{0}{\overset{k}{\int }}\underset{\_}{x}\left(t\right)\mathrm{d}t$$

(79)

Then $\underset{\_}{X}\left(k\right)/k$ is the time averaged process. The climacogram $\gamma \left(k\right)$ of the process $\underset{\_}{x}\left(t\right)$ is the variance of the time averaged process at time scale $k$ [24], i.e.,

$$\gamma \left(k\right):=\mathrm{v}\mathrm{a}\mathrm{r}\left[\frac{\underset{\_}{X}\left(k\right)}{k}\right]$$

(80)

The climacogram of the HK process is given as a power function of the time scale k, i.e.,

$$\gamma \left(k\right)={\lambda }^2{\left(\frac{\alpha }{k}\right)}^{2-2H}$$

(81)

where $\alpha$ and $\lambda$ are scale parameters with units of time and $\left[x\right]$, respectively. In a purely random process, $H=1/2$, while in most natural processes, $1/2\le H\le 1$, as first observed by Hurst in 1951 [25]. A high value of $H$ (approaching 1) indicates an enhanced presence of patterns such as grouping of similar events (e.g., floods, droughts) in time, enhanced change and enhanced uncertainty (e.g., in future predictions). A low value of $H$ (<1/2) indicates enhanced fluctuation or antipersistence. Processes with more complex expressions of climacogram also exhibit LRD if

$$\underset{k\to \infty }{\lim }\gamma \left(k\right)k^{2-2H}=l$$

(82)

where $l$ is finite.

Koutsoyiannis [2] assumed that the estimate ${\widehat{{\rm K}}}_p^{\mathrm{d}}$ derived by the standard estimators (15)–(18), where the superscript ‘d’ stands for dependence, results in a bias that is proportional to $K_p$, so that the relative bias

$$\frac{{\widehat{{\rm K}}}_p^{\mathrm{d}}-K_p}{K_p}=\Theta \left(n,H\right)$$

(83)

does not depend on the moment order $p$ but rather only on the sample size $n$ and the Hurst parameter $H$. Based on this assumption, he derived the relative bias for an HK process as

$$\Theta \left(n,H\right)\approx \frac{2H\left(1-H\right)}{n-1}-\frac{1}{2{\left(n-1\right)}^{2-2H}}$$

(84)

Further, assuming that ${\widehat{{\rm K}}}_p^{\mathrm{d}}$ equals the true K-moment of an order $p^{\prime }<p$, the adjusted moment order $p^{\prime }$ was derived as

$$p^{\prime }\approx 2\Theta +\left(1-2\Theta \right)p^{\left({\left(1+\Theta \right)}^2\right)},{K_{p^{\prime }}=K}_p^{\mathrm{d}}=(1+\Theta )K_p$$

(85)

The above relationships enable the following simple algorithm to approximately estimate the true moments from a time series of length n, which is a realization of a process with LRD:

• We construct the climacogram and estimate the Hurst parameter $H$.
• From Equation (84), we estimate the relative bias $\Theta \left(n,H\right)$.
• We choose a moment order p and estimate by the standard estimators (15)–(18) the K-moment $K_p^{\prime }$ (or $K_p$) as if we had an independent sample.
• We adapt $p$ by Equation (85) and infer that the quantity estimated in step 3 is an estimate of $K_{p^{\prime }}^{\prime }$ (or $K_{p^{\prime }}$).

This is for moment order $q=1$. A similar approach for higher $q$ is discussed in [2].

An illustration is provided in Figure 9, which was made by stochastic simulation assuming an HK process with marginal distribution generalized Pareto. The generation of a simulated time series from the process is conducted using a simple method termed the multiple time-scale fluctuation approach [26]. This generates Gaussian stochastic variables $\underset{\_}{u}$ with climacogram that closely approximates that of Equation (81), by adding three independent linear Markov processes (autoregressive of order 1), each having a different lag-one autocorrelation coefficient, calculated as a function of H, as described in [26]. The Gaussian series are subsequently transformed to have any desired distribution $F$ (in our case, generalized Pareto) by $\underset{\_}{x}=F^{-1}\left(\Phi \left(\underset{\_}{u}\right)\right)$, where $\Phi$ is the normal distribution function.

The simulation results depicted in Figure 9 are for a high Hurst parameter, H = 0.9, to magnify the bias. Interestingly, what would be assigned as K-moment for p = 2000, without taking into account the effect of dependence, actually corresponds to the true K-moment of $p^{\prime }\approx 500$. This has dramatic consequences in the assignment of probabilities of exceedance (or return periods). Without adaptation, we would assign a probability of exceedance four times lower than the true one (or a four-fold return period). Another important observation from comparing the two panels of the figure, which are for the same marginal distribution, is the dramatic broadening of the prediction intervals in the case of LRD. This illustrates how dependence amplifies uncertainty.

The above technique has already been applied to the wind data example in Figure 7, as the estimated Hurst parameter is high, 𝐻 = 0.9, and its effect (bias) on distribution function estimation is substantial, with relative bias $\Theta =-0.06$. Removing the bias resulted in a reduction of the maximum order $p=n=\mathrm{583,465}$ to $p\prime =\mathrm{140,117}$ (i.e., $p^{\prime }<p/4)$.

4. Discussion

K-moments share some characteristics with the classical moments, L-moments [16] and probability-weighted moments (PWM; [27]), which are discussed in detail in [2] and in part in Section 3.1 and Section 3.2 here. On the other hand, the differences are huge. Thus, the K-moments are knowable with unbiased estimators (from samples) for high orders, up to the sample size $n$, while in classical moments, the convergence bias is substantial and the estimation uncertainty is orders of magnitude higher. Furthermore, as discussed in Section 3.1, the K-moments can reveal what the estimators of classical moments actually estimate, which is different from the true values thereof.

As per the L-moments and PWMs, the logic and strategies that is followed for these concepts do not differ from those of classical moments. Up to this point, the K-moments are similar to them and produce similar summary statistics, as discussed in Section 3.2. However, the real power of K-moments is that their values can directly be assigned probabilities, through Λ-coefficients. This is similar to what happens with order statistics, but with some advantages over the latter, as the K-moments make optimal use of the entire dataset, instead of using just one data value to assign probability. Thus, the sequence of K-moment values produces a smooth curve that can be used to empirically estimate the density function, a unique feature among all moment types.

The assignment of probabilities to numerous upper and lower K-moment values, for orders up to the sample size, enables a rich characterization of the distribution function, as well as a strategy for model fitting totally different from other methods of moments. This strategy involves the entire range of moment orders (rather than the first 1–4 moments), can focus on different parts of the distribution (body, upper tail, lower tail) and enables visualization of the fitting.

Another powerful characteristic of the K-moment framework, as presented above, is the simple and intuitive definition in terms of averages of maxima or minima of stochastic variables. This is different from other moment types and implies different behaviours (e.g., in discrete stochastic variables).

The fact that the K-moment framework assigns probabilities to moment values, which is a unique feature among all moment types, makes theoretical calculations thereof (i.e., by integration or numerical integration) unnecessary. Instead, the calculations are made in terms of the probabilities rather than values of the random variable. The simple approximation by Equation (71) is sufficient for most statistical tasks. When more accurate approximations are required, these can be found in [2].

Finally, the K-moment estimators, unlike other moment types, can explicitly (albeit approximately) take into account any existing dependence structure in case the data are not an independent sample but a time series (i.e., a realization of a stochastic process). This is also a unique feature among all types of moments.

5. Summary and Conclusions

Knowable moments or K-moments, defined as expectations of maxima or minima of a number of independent identically distributed stochastic variables, have some features that are not seen in other types of moments in stochastics and make them useful in relevant applications. Their name highlights the fact that, in contrast to classical moments, which are unknowable from samples for orders beyond 3–4, K-moments can be reliably and unbiasedly estimated from a sample for very high orders up to the sample size, even if this is several million or more. For such high orders, techniques to accelerate the calculations are proposed.

Not only can K-moments replace the information contained in classical moments and L-moments (e.g., summary statistics), but they can also provide information about what a classical moment estimator determines, which actually is not the true value of that classical moment.

An important feature of K-moment values is their ability to be assigned values of the distribution function. This characteristic is shared by order statistics and not by any other type of moments. However, there are substantial advantages over order statistics, because K-moment estimators make optimal use of the entire dataset, rather than relying on a single data value to be assigned a probability. This makes the estimation of the probability density function from a sample possible, a feature not shared by any other type of moments.

Finally, K-moments offer the unique advantage of taking into account the estimation bias when the data are not an independent sample but a time series from a process with dependence, even of long range. Both for samples and time series, the K-moment concept offers a strategy of model fitting, including its visualization, that is not shared with any other method. This enables utilization of the highest possible moment orders, which are particularly useful in modelling extreme highs or lows that are closely associated with high-order moments.