Moments of the Negative Multinomial Distribution

Abstract

The negative multinomial distribution appears in many areas of applications such as polarimetric image processing and the analysis of longitudinal count data. In previous studies, general formulas for the falling factorial moments and cumulants of the negative multinomial distribution were obtained. However, despite the availability of the moment generating function, no comprehensive formulas for the moments have been calculated thus far. This paper addresses this gap by presenting general formulas for both central and non-central moments of the negative multinomial distribution. These formulas are expressed in terms of binomial coefficients and Stirling numbers of the second kind. Utilizing these formulas, we provide explicit expressions for all central moments up to the fourth order and all non-central moments up to the eighth order.

1. Introduction

The negative multinomial distribution is a probability distribution that can be used to model count data, where the outcome of interest is the number of occurrences of $d\in \mathbb{N}$ different events when the number of failures (a failure means that, for a given trial, an object is not categorized in any of the d categories) is a fixed value $r\in \mathbb{N}$. It is a multivariate generalization of the well-known negative binomial distribution, for which $d=1$. For a general reference on the negative multinomial distribution and its properties, refer to Sibuya et al. [1] or Chapter 36 of Johnson et al. [2].

One of the main motivations for using the negative multinomial distribution is its ability to model overdispersion for count vectors, which happens when the variances of the count variables are larger than their mean (Fitzmaurice et al. [3]). The Poisson distribution for instance assumes that the mean and variance are equal but, in many real-world scenarios, this is often not the case. The negative multinomial distribution allows for modeling overdispersion by allowing for different variances for each event type (Cameron and Trivedi [4]).

Another motivation for using the negative multinomial distribution is its ability to handle excess zeros in count data, see, e.g., Haslett et al. [5]. Count data often exhibit zero inflation, where there are more zeros than would be expected under a Poisson distribution. The negative multinomial distribution provides a flexible framework for modeling zero inflation by allowing for different probabilities of zero occurrences for each event type.

A third motivation for using the negative multinomial distribution is its ability to model count data with multiple event types, see, e.g., [6,7,8,9,10,11,12,13]. In many applications, there is more than one type of event that can occur and the negative multinomial distribution allows for modeling the counts of each event type simultaneously. This is particularly useful in fields such as marketing, where the goal is to model the number of purchases of different products, or in ecology, where the goal is to model the counts of different species in a community [14].

Overall, the negative multinomial distribution provides a flexible and powerful tool for modeling count data in a variety of applications such as polarimetric image processing [15], the analysis of RNA-seq. data [16], pollen analysis [17], longitudinal data [6,7,12], etc. Its theoretical properties have been investigated in numerous papers, see, e.g., [18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33]. One can find extensions of the model in [16,34,35,36].

The ability of the negative multinomial distribution to handle overdispersion, zero inflation, and multiple event types makes it a valuable tool for data scientists and statisticians. Whether one is interested in modeling the number of purchases of different products, the counts of different species in a community, or any other count data, the negative multinomial distribution can provide valuable insights and inform decision-making.

In previous studies, Mosimann [17] derived general formulas for the falling factorial moments of the negative multinomial distribution, while Withers and Nadarajah [37] obtained expressions for the cumulants. Despite the availability of the moment generating function, no comprehensive formulas for the moments have been calculated thus far. Our goal in this paper is to address this gap by presenting general formulas for both central and non-central moments of the negative multinomial distribution.

Here is an outline of the paper. In Section 2, the necessary definitions and notations are introduced, along with a preliminary result on factorial moments of the negative multinomial distribution due to Mosimann [17]. The general formulas for the central and non-central moments of the negative multinomial distribution are stated and proved in Section 3. The numerical implementation of those formulas in Mathematica is provided in Section 4. Finally, in Section 5, our general formulas are applied to give explicit expressions for all central moments up to the fourth order and all non-central moments up to the eighth order. Open problems of interest are stated in Section 6.

2. The Negative Multinomial Distribution

For any $d\in \mathbb{N}$, let $\mathit{x}\in {[0,1]}^d$ be such that ${\parallel \mathit{x}\parallel }_1:={\sum }_{i=1}^d\left|x_i\right|<1$. The probability mass function $\mathit{k}\mapsto P_{r,\mathit{x}}\left(\mathit{k}\right)$ of the negative multinomial distribution is defined by

$$\begin{array}{cc}\hfill P_{r,\mathit{x}}\left(\mathit{k}\right):& =\frac{\Gamma (r+\parallel \mathit{k}{\parallel }_1)}{\Gamma \left(r\right){\prod }_{i=1}^d\Gamma (k_i+1)}{(1-\parallel \mathit{x}\parallel }_1{)}^r\prod _{i=1}^d{x}_i^{k_i}\hfill \\ \hfill & =\frac{\Gamma (r+\parallel \mathit{k}{\parallel }_1)}{\Gamma \left(r\right){\prod }_{i=1}^d\Gamma (k_i+1)}{(1-\parallel \mathit{x}\parallel }_1{)}^{r+{\parallel \mathit{k}\parallel }_1}\prod _{i=1}^d{y}_i^{k_i},\mathit{k}\in {\mathbb{N}}_0^d,\hfill \end{array}$$

(1)

where $r>0$ is a positive real number and $y_i:=x_i{/(1-\parallel \mathit{x}\parallel }_1)$ for all $i\in \{1,\dots ,d\}$. If a random vector $\mathsf{\eta }=({\eta }_1,\dots ,{\eta }_d)$ follows this distribution, we write for short $\mathsf{\eta }\sim \mathrm{Neg}\mathrm{Multinomial}(r,\mathit{x})$. In this paper, our main goal is to give general formulas for the non-central and central moments of (1), namely

$$\mathbb{E}\left[\prod _{i=1}^d{\eta }_i^{p_i}\right]\mathrm{and}\mathbb{E}\left[\prod _{i=1}^d{({\eta }_i-\mathbb{E}\left[{\eta }_i\right])}^{p_i}\right],p_1,\dots ,p_d\in {\mathbb{N}}_0.$$

(2)

We obtain the formulas using a combinatorial argument and the general expression for the falling factorial moments found by Mosimann [17], which we register in the lemma below.

Lemma 1

(Factorial moments). Let $\mathsf{\eta }\sim \mathrm{Neg}\mathrm{Multinomial}(r,\mathit{x})$. Then, for all $k_1,\dots ,k_d\in {\mathbb{N}}_0$,

$$\mathbb{E}\left[\prod _{i=1}^d{\eta }_i^{\left(k_i\right)}\right]={\left(r-1+{\parallel \mathit{k}\parallel }_1\right)}^{(\parallel \mathit{k}{\parallel }_1)}\prod _{i=1}^d{y}_i^{k_i},$$

where $m^{\left(k\right)}:=m(m-1)\dots (m-k+1)$ denotes the $kth$ order falling factorial of m.

The formulas we derive for the expectations in Equation (2) will be employed to calculate all the central moments up to the fourth order, as well as all the non-central moments up to the eighth order. For information about the moment generating function, the cumulant generating function, and expressions for the cumulants, refer to Withers and Nadarajah [37].

3. Results

First, we give a general formula for the non-central moments of the negative multinomial distribution in (1).

Theorem 1

(Non-central moments). Let $\mathsf{\eta }\sim \mathrm{Neg}\mathrm{Multinomial}(r,\mathit{x})$. Then, for all $p_1,\dots ,p_d\in {\mathbb{N}}_0$,

$$\mathbb{E}\left[\prod _{i=1}^d{\eta }_i^{p_i}\right]=\sum _{k_1=0}^{p_1}\dots \sum _{k_d=0}^{p_d}{\left(r-1+{\parallel \mathit{k}\parallel }_1\right)}^{(\parallel \mathit{k}{\parallel }_1)}\prod _{i=1}^d\left\{\genfrac{}{}{0.0pt}{}{p_i}{k_i}\right\}{y}_i^{k_i},$$

where $\left\{\genfrac{}{}{0.0pt}{}{p}{k}\right\}$ denotes a Stirling number of the second kind (i.e., the number of ways to partition a set of p objects into k non-empty subsets); recall that

$$y_i:=\frac{x_i}{1-{\parallel \mathit{x}\parallel }_1},\mathit{for}\mathit{all}i\in \{1,\dots ,d\}.$$

Proof. 

The following well-known relationship between the power $p\in {\mathbb{N}}_0$ of a number $x\in \mathbb{R}$ and the falling factorials of x is already established:

$$x^p=\sum _{k=0}^p\left\{\genfrac{}{}{0.0pt}{}{p}{k}\right\}{x}^{\left(k\right)}.$$

This relationship can be found in [38] (p. 262). By applying this formula to each ${\eta }_i^{p_i}$ and utilizing the linearity of expectation, we obtain the following:

$$\mathbb{E}\left[\prod _{i=1}^d{\eta }_i^{p_i}\right]=\sum _{k_1=0}^{p_1}\dots \sum _{k_d=0}^{p_d}\left\{\genfrac{}{}{0.0pt}{}{p_1}{k_1}\right\}\dots \left\{\genfrac{}{}{0.0pt}{}{p_d}{k_d}\right\}\mathbb{E}\left[\prod _{i=1}^d{\eta }_i^{\left(k_i\right)}\right],$$

Therefore, the conclusion is a direct consequence of Lemma 1.    □

We can now derive a comprehensive formula for the central moments of the negative multinomial distribution.

Theorem 2

(Central moments). Let $\mathsf{\eta }\sim \mathrm{Neg}\mathrm{Multinomial}(r,\mathit{x})$. Then, for all $p_1,\dots ,p_d\in {\mathbb{N}}_0$,

$$\begin{array}{cc}\hfill \mathbb{E}\left[\prod _{i=1}^d{({\eta }_i-\mathbb{E}\left[{\eta }_i\right])}^{p_i}\right]& =\sum _{{\ell }_1=0}^{p_1}\dots \sum _{{\ell }_d=0}^{p_d}\sum _{k_1=0}^{{\ell }_1}\dots \sum _{k_d=0}^{{\ell }_d}{\left(r-1+{\parallel \mathit{k}\parallel }_1\right)}^{(\parallel \mathit{k}{\parallel }_1)}\hfill \\ \hfill & ·{(-r)}^{{\sum }_{i=1}^d(p_i-{\ell }_i)}\prod _{i=1}^d\left(\genfrac{}{}{0pt}{}{p_i}{{\ell }_i}\right)\left\{\genfrac{}{}{0.0pt}{}{{\ell }_i}{k_i}\right\}{y}_i^{p_i-{\ell }_i+k_i},\hfill \end{array}$$

where $\left(\genfrac{}{}{0pt}{}{p}{\ell }\right)$ denotes the binomial coefficient $\frac{p!}{\ell !(p-\ell )!}$; recall that

$$y_i:=\frac{x_i}{1-{\parallel \mathit{x}\parallel }_1},\mathit{for}\mathit{all}i\in \{1,\dots ,d\}.$$

Proof. 

By applying the binomial formula to each factor ${({\eta }_i-\mathbb{E}\left[{\eta }_i\right])}^{p_i}$ and using the fact that $\mathbb{E}\left[{\eta }_i\right]=r{y}_i$ for all $i\in \{1,\dots ,d\}$, note that

$$\mathbb{E}\left[\prod _{i=1}^d{({\eta }_i-\mathbb{E}\left[{\eta }_i\right])}^{p_i}\right]=\sum _{{\ell }_1=0}^{p_1}\dots \sum _{{\ell }_d=0}^{p_d}\mathbb{E}\left[\prod _{i=1}^d{\eta }_i^{{\ell }_i}\right]·\prod _{i=1}^d\left(\genfrac{}{}{0pt}{}{p_i}{{\ell }_i}\right){(-r{y}_i)}^{p_i-{\ell }_i}.$$

Therefore, the conclusion is a direct consequence of Theorem 1.    □

4. Numerical Codes

The formulas in Theorems 1 and 2 can be put into practice in Mathematica through the following procedure:

    NonCentral[r_, x_, p_, d_] :=

      Sum[FactorialPower[r - 1 + Sum[k[i], {i, 1, d}],

      Sum[k[i], {i, 1, d}]] * Product[StirlingS2[p[[i]], k[i]] *

      (x[[i]] / (1 - Sum[x[[i]], {i, 1, d}])) ^ k[i], {i, 1, d}], ##] & @@

      ({k[#], 0, p[[#]]} & /@ Range[d]);

    Central[r_, x_, p_, d_] :=

      Sum[Sum[FactorialPower[r - 1 + Sum[k[i], {i, 1, d}],

      Sum[k[i], {i, 1, d}]] * (-r) ^ Sum[p[[i]] - ell[i], {i, 1, d}]

      * Product[Binomial[p[[i]], ell[i]] * StirlingS2[ell[i], k[i]]

      * (x[[i]] / (1 - Sum[x[[i]], {i, 1, d}])) ^

      (p[[i]] - ell[i] + k[i]), {i, 1, d}], ##] & @@

      ({k[#], 0, ell[#]} & /@ Range[d]), ##] & @@

      ({ell[#], 0, p[[#]]} & /@ Range[d]);

5. Explicit Formulas

In the two subsections below, we calculate (explicitly) all the non-central moments up to the eighth order and all the central moments up to the fourth order. Here is a table of the Stirling numbers of the second kind that we will use in our calculations:

$$\begin{array}{cc}\hfill & \left\{\genfrac{}{}{0.0pt}{}{0}{0}\right\}=1,\hfill \\ \hfill & \left\{\genfrac{}{}{0.0pt}{}{1}{0}\right\}=0,\left\{\genfrac{}{}{0.0pt}{}{1}{1}\right\}=1,\hfill \\ \hfill & \left\{\genfrac{}{}{0.0pt}{}{2}{0}\right\}=0,\left\{\genfrac{}{}{0.0pt}{}{2}{1}\right\}=1,\left\{\genfrac{}{}{0.0pt}{}{2}{2}\right\}=1,\hfill \\ \hfill & \left\{\genfrac{}{}{0.0pt}{}{3}{0}\right\}=0,\left\{\genfrac{}{}{0.0pt}{}{3}{1}\right\}=1,\left\{\genfrac{}{}{0.0pt}{}{3}{2}\right\}=3,\left\{\genfrac{}{}{0.0pt}{}{3}{3}\right\}=1,\hfill \\ \hfill & \left\{\genfrac{}{}{0.0pt}{}{4}{0}\right\}=0,\left\{\genfrac{}{}{0.0pt}{}{4}{1}\right\}=1,\left\{\genfrac{}{}{0.0pt}{}{4}{2}\right\}=7,\left\{\genfrac{}{}{0.0pt}{}{4}{3}\right\}=6,\left\{\genfrac{}{}{0.0pt}{}{4}{4}\right\}=1,\hfill \\ \hfill & \left\{\genfrac{}{}{0.0pt}{}{5}{0}\right\}=0,\left\{\genfrac{}{}{0.0pt}{}{5}{1}\right\}=1,\left\{\genfrac{}{}{0.0pt}{}{5}{2}\right\}=15,\left\{\genfrac{}{}{0.0pt}{}{5}{3}\right\}=25,\left\{\genfrac{}{}{0.0pt}{}{5}{4}\right\}=10,\left\{\genfrac{}{}{0.0pt}{}{5}{5}\right\}=1,\hfill \\ \hfill & \left\{\genfrac{}{}{0.0pt}{}{6}{0}\right\}=0,\left\{\genfrac{}{}{0.0pt}{}{6}{1}\right\}=1,\left\{\genfrac{}{}{0.0pt}{}{6}{2}\right\}=31,\left\{\genfrac{}{}{0.0pt}{}{6}{3}\right\}=90,\left\{\genfrac{}{}{0.0pt}{}{6}{4}\right\}=65,\left\{\genfrac{}{}{0.0pt}{}{6}{5}\right\}=15,\left\{\genfrac{}{}{0.0pt}{}{6}{6}\right\}=1,\hfill \\ \hfill & \left\{\genfrac{}{}{0.0pt}{}{7}{0}\right\}=0,\left\{\genfrac{}{}{0.0pt}{}{7}{1}\right\}=1,\left\{\genfrac{}{}{0.0pt}{}{7}{2}\right\}=63,\left\{\genfrac{}{}{0.0pt}{}{7}{3}\right\}=301,\left\{\genfrac{}{}{0.0pt}{}{7}{4}\right\}=350,\left\{\genfrac{}{}{0.0pt}{}{7}{5}\right\}=140,\left\{\genfrac{}{}{0.0pt}{}{7}{6}\right\}=21,\left\{\genfrac{}{}{0.0pt}{}{7}{7}\right\}=1,\hfill \\ \hfill & \left\{\genfrac{}{}{0.0pt}{}{8}{0}\right\}=0,\left\{\genfrac{}{}{0.0pt}{}{8}{1}\right\}=1,\left\{\genfrac{}{}{0.0pt}{}{8}{2}\right\}=127,\left\{\genfrac{}{}{0.0pt}{}{8}{3}\right\}=966,\left\{\genfrac{}{}{0.0pt}{}{8}{4}\right\}=1701,\left\{\genfrac{}{}{0.0pt}{}{8}{5}\right\}=1050,\left\{\genfrac{}{}{0.0pt}{}{8}{6}\right\}=266,\left\{\genfrac{}{}{0.0pt}{}{8}{7}\right\}=28,\left\{\genfrac{}{}{0.0pt}{}{8}{8}\right\}=1.\hfill \end{array}$$

5.1. Computation of the Non-Central Moments up to the Eighth Order

By utilizing the general expression outlined in Theorem 1 and eliminating the Stirling numbers $\left\{\genfrac{}{}{0.0pt}{}{p_i}{k_i}\right\}$ that are equal to zero, we obtain the following results effortlessly.

1st order: For $j_1\in \{1,\dots ,d\}$,

$$\begin{array}{cc}\hfill \mathbb{E}\left[{\eta }_{j_1}\right]& =y_{j_1}r.\hfill \end{array}$$

2nd order: For different $j_1,j_2\in \{1,\dots ,d\}$,

$$\begin{array}{cc}\hfill \mathbb{E}\left[{\eta }_{j_1}^2\right]& =y_{j_1}\left[r+{(r+1)}^{\left(2\right)}{y}_{j_1}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}{\eta }_{j_2}\right]& =y_{j_1}{y}_{j_2}{(r+1)}^{\left(2\right)}.\hfill \end{array}$$

3rd order: For different $j_1,j_2,j_3\in \{1,\dots ,d\}$,

$$\begin{array}{cc}\hfill \mathbb{E}\left[{\eta }_{j_1}^3\right]& =y_{j_1}\left[r+3{(r+1)}^{\left(2\right)}{y}_{j_1}+{(r+2)}^{\left(3\right)}{y}_{j_1}^2\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^2{\eta }_{j_2}\right]& =y_{j_1}{y}_{j_2}\left[{(r+1)}^{\left(2\right)}+{(r+2)}^{\left(3\right)}{y}_{j_1}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}{\eta }_{j_2}{\eta }_{j_3}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}{(r+2)}^{\left(3\right)}.\hfill \end{array}$$

4th order: For different $j_1,j_2,j_3,j_4\in \{1,\dots ,d\}$,

$$\begin{array}{cc}\hfill \mathbb{E}\left[{\eta }_{j_1}^4\right]& =y_{j_1}\left[r+7{(r+1)}^{\left(2\right)}{y}_{j_1}+6{(r+2)}^{\left(3\right)}{y}_{j_1}^2+{(r+3)}^{\left(4\right)}{y}_{j_1}^3\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^3{\eta }_{j_2}\right]& =y_{j_1}{y}_{j_2}\left[{(r+1)}^{\left(2\right)}+3{(r+2)}^{\left(3\right)}{y}_{j_1}+{(r+3)}^{\left(4\right)}{y}_{j_1}^2\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^2{\eta }_{j_2}^2\right]& =y_{j_1}{y}_{j_2}\left[{(r+1)}^{\left(2\right)}+{(r+2)}^{\left(3\right)}(y_{j_1}+y_{j_2})+{(r+3)}^{\left(4\right)}{y}_{j_1}{y}_{j_2}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^2{\eta }_{j_2}{\eta }_{j_3}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}\left[{(r+2)}^{\left(3\right)}+{(r+3)}^{\left(4\right)}{y}_{j_1}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}{\eta }_{j_2}{\eta }_{j_3}{\eta }_{j_4}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}{(r+3)}^{\left(4\right)}.\hfill \end{array}$$

5th order: For different $j_1,j_2,j_3,j_4,j_5\in \{1,\dots ,d\}$,

$$\begin{array}{cc}\hfill \mathbb{E}\left[{\eta }_{j_1}^5\right]& =y_{j_1}\left[\begin{array}{c}r+15{(r+1)}^{\left(2\right)}{y}_{j_1}+25{(r+2)}^{\left(3\right)}{y}_{j_1}^2\hfill \\ +10{(r+3)}^{\left(4\right)}{y}_{j_1}^3+{(r+4)}^{\left(5\right)}{y}_{j_1}^4\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^4{\eta }_{j_2}\right]& =y_{j_1}{y}_{j_2}\left[\begin{array}{c}{(r+1)}^{\left(2\right)}+7{(r+2)}^{\left(3\right)}{y}_{j_1}\hfill \\ +6{(r+3)}^{\left(4\right)}{y}_{j_1}^2+{(r+4)}^{\left(5\right)}{y}_{j_1}^3\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^3{\eta }_{j_2}^2\right]& =y_{j_1}{y}_{j_2}\left[\begin{array}{c}{(r+1)}^{\left(2\right)}+{(r+2)}^{\left(3\right)}(3y_{j_1}+y_{j_2})\hfill \\ +{(r+3)}^{\left(4\right)}(y_{j_1}^2+3y_{j_1}{y}_{j_2})+{(r+4)}^{\left(5\right)}{y}_{j_1}^2{y}_{j_2}\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^3{\eta }_{j_2}{\eta }_{j_3}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}\left[{(r+2)}^{\left(3\right)}+3{(r+3)}^{\left(4\right)}{y}_{j_1}+{(r+4)}^{\left(5\right)}{y}_{j_1}^2\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^2{\eta }_{j_2}^2{\eta }_{j_3}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}\left[{(r+2)}^{\left(3\right)}+{(r+3)}^{\left(4\right)}(y_{j_1}+y_{j_2})+{(r+4)}^{\left(5\right)}{y}_{j_1}{y}_{j_2}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^2{\eta }_{j_2}{\eta }_{j_3}{\eta }_{j_4}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}\left[{(r+3)}^{\left(4\right)}+{(r+4)}^{\left(5\right)}{y}_{j_1}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}{\eta }_{j_2}{\eta }_{j_3}{\eta }_{j_4}{\eta }_{j_5}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}{y}_{j_5}{(r+4)}^{\left(5\right)}.\hfill \end{array}$$

6th order: For different $j_1,j_2,j_3,j_4,j_5,j_6\in \{1,\dots ,d\}$,

$$\begin{array}{cc}\hfill \mathbb{E}\left[{\eta }_{j_1}^6\right]& =y_{j_1}\left[\begin{array}{c}r+31{(r+1)}^{\left(2\right)}{y}_{j_1}+90{(r+2)}^{\left(3\right)}{y}_{j_1}^2\hfill \\ +65{(r+3)}^{\left(4\right)}{y}_{j_1}^3+15{(r+4)}^{\left(5\right)}{y}_{j_1}^4+{(r+5)}^{\left(6\right)}{y}_{j_1}^5\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^5{\eta }_{j_2}\right]& =y_{j_1}{y}_{j_2}\left[\begin{array}{c}{(r+1)}^{\left(2\right)}+15{(r+2)}^{\left(3\right)}{y}_{j_1}+25{(r+3)}^{\left(4\right)}{y}_{j_1}^2\hfill \\ +10{(r+4)}^{\left(5\right)}{y}_{j_1}^3+{(r+5)}^{\left(6\right)}{y}_{j_1}^4\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^4{\eta }_{j_2}^2\right]& =y_{j_1}{y}_{j_2}\left[\begin{array}{c}{(r+1)}^{\left(2\right)}+{(r+2)}^{\left(3\right)}(7y_{j_1}+y_{j_2})\hfill \\ +{(r+3)}^{\left(4\right)}(6y_{j_1}^2+7y_{j_1}{y}_{j_2})\hfill \\ +{(r+4)}^{\left(5\right)}(y_{j_1}^3+6y_{j_1}^2{y}_{j_2})+{(r+5)}^{\left(6\right)}{y}_{j_1}^3{y}_{j_2}\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^4{\eta }_{j_2}{\eta }_{j_3}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}\left[\begin{array}{c}{(r+2)}^{\left(3\right)}+7{(r+3)}^{\left(4\right)}{y}_{j_1}\hfill \\ +6{(r+4)}^{\left(5\right)}{y}_{j_1}^2+{(r+5)}^{\left(6\right)}{y}_{j_1}^3\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^3{\eta }_{j_2}^3\right]& =y_{j_1}{y}_{j_2}\left[\begin{array}{c}{(r+1)}^{\left(2\right)}+{(r+2)}^{\left(3\right)}(3y_{j_1}+3y_{j_2})\hfill \\ +{(r+3)}^{\left(4\right)}(y_{j_1}^2+9y_{j_1}{y}_{j_2}+y_{j_2}^2)\hfill \\ +{(r+4)}^{\left(5\right)}(3y_{j_1}^2{y}_{j_2}+3y_{j_1}{y}_{j_2}^2)+{(r+5)}^{\left(6\right)}{y}_{j_1}^2{y}_{j_2}^2\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^3{\eta }_{j_2}^2{\eta }_{j_3}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}\left[\begin{array}{c}{(r+2)}^{\left(3\right)}+{(r+3)}^{\left(4\right)}(3y_{j_1}+y_{j_2})\hfill \\ +{(r+4)}^{\left(5\right)}(y_{j_1}^2+3y_{j_1}{y}_{j_2})+{(r+5)}^{\left(6\right)}{y}_{j_1}^2{y}_{j_2}\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^3{\eta }_{j_2}{\eta }_{j_3}{\eta }_{j_4}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}\left[{(r+3)}^{\left(4\right)}+3{(r+4)}^{\left(5\right)}{y}_{j_1}+{(r+5)}^{\left(6\right)}{y}_{j_1}^2\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^2{\eta }_{j_2}^2{\eta }_{j_3}^2\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}\left[\begin{array}{c}{(r+2)}^{\left(3\right)}+{(r+3)}^{\left(4\right)}(y_{j_1}+y_{j_2}+y_{j_3})\hfill \\ +{(r+4)}^{\left(5\right)}(y_{j_1}{y}_{j_2}+y_{j_1}{y}_{j_3}+y_{j_2}{y}_{j_3})+{(r+5)}^{\left(6\right)}{y}_{j_1}{y}_{j_2}{y}_{j_3}\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^2{\eta }_{j_2}^2{\eta }_{j_3}{\eta }_{j_4}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}\left[{(r+3)}^{\left(4\right)}+{(r+4)}^{\left(5\right)}(y_{j_1}+y_{j_2})+{(r+5)}^{\left(6\right)}{y}_{j_1}{y}_{j_2}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^2{\eta }_{j_2}{\eta }_{j_3}{\eta }_{j_4}{\eta }_{j_5}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}{y}_{j_5}\left[{(r+4)}^{\left(5\right)}+{(r+5)}^{\left(6\right)}{y}_{j_1}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}{\eta }_{j_2}{\eta }_{j_3}{\eta }_{j_4}{\eta }_{j_5}{\eta }_{j_6}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}{y}_{j_5}{y}_{j_6}{(r+5)}^{\left(6\right)}.\hfill \end{array}$$

7th order: For different $j_1,j_2,j_3,j_4,j_5,j_6,j_7\in \{1,\dots ,d\}$,

$$\begin{array}{cc}\hfill \mathbb{E}\left[{\eta }_{j_1}^7\right]& =y_{j_1}\left[\begin{array}{c}r+63{(r+1)}^{\left(2\right)}{y}_{j_1}+301{(r+2)}^{\left(3\right)}{y}_{j_1}^2\hfill \\ +350{(r+3)}^{\left(4\right)}{y}_{j_1}^3+140{(r+4)}^{\left(5\right)}{y}_{j_1}^4\hfill \\ +21{(r+5)}^{\left(6\right)}{y}_{j_1}^5+{(r+6)}^{\left(7\right)}{y}_{j_1}^6\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^6{\eta }_{j_2}\right]& =y_{j_1}{y}_{j_2}\left[\begin{array}{c}{(r+1)}^{\left(2\right)}+31{(r+2)}^{\left(3\right)}{y}_{j_1}+90{(r+3)}^{\left(4\right)}{y}_{j_1}^2\hfill \\ +65{(r+4)}^{\left(5\right)}{y}_{j_1}^3+15{(r+5)}^{\left(6\right)}{y}_{j_1}^4+{(r+6)}^{\left(7\right)}{y}_{j_1}^5\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^5{\eta }_{j_2}^2\right]& =y_{j_1}{y}_{j_2}\left[\begin{array}{c}{(r+1)}^{\left(2\right)}+{(r+2)}^{\left(3\right)}(15y_{j_1}+y_{j_2})\hfill \\ +{(r+3)}^{\left(4\right)}(25y_{j_1}^2+15y_{j_1}{y}_{j_2})\hfill \\ +{(r+4)}^{\left(5\right)}(10y_{j_1}^3+25y_{j_1}^2{y}_{j_2})\hfill \\ +{(r+5)}^{\left(6\right)}(y_{j_1}^4+10y_{j_1}^3{y}_{j_2})+{(r+6)}^{\left(7\right)}{y}_{j_1}^4{y}_{j_2}\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^5{\eta }_{j_2}{\eta }_{j_3}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}\left[\begin{array}{c}{(r+2)}^{\left(3\right)}+15{(r+3)}^{\left(4\right)}{y}_{j_1}+25{(r+4)}^{\left(5\right)}{y}_{j_1}^2\hfill \\ +10{(r+5)}^{\left(6\right)}{y}_{j_1}^3+{(r+6)}^{\left(7\right)}{y}_{j_1}^4\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^4{\eta }_{j_2}^3\right]& =y_{j_1}{y}_{j_2}\left[\begin{array}{c}{(r+1)}^{\left(2\right)}+{(r+2)}^{\left(3\right)}(7y_{j_1}+3y_{j_2})\hfill \\ +{(r+3)}^{\left(4\right)}(6y_{j_1}^2+21y_{j_1}{y}_{j_2}+y_{j_2}^2)\hfill \\ +{(r+4)}^{\left(5\right)}(y_{j_1}^3+18y_{j_1}^2{y}_{j_2}+7y_{j_1}{y}_{j_2}^2)\hfill \\ +{(r+5)}^{\left(6\right)}(3y_{j_1}^3{y}_{j_2}+6y_{j_1}^2{y}_{j_2}^2)+{(r+6)}^{\left(7\right)}{y}_{j_1}^3{y}_{j_2}^2\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^4{\eta }_{j_2}^2{\eta }_{j_3}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}\left[\begin{array}{c}{(r+2)}^{\left(3\right)}+{(r+3)}^{\left(4\right)}(7y_{j_1}+y_{j_2})\hfill \\ +{(r+4)}^{\left(5\right)}(6y_{j_1}^2+7y_{j_1}{y}_{j_2})\hfill \\ +{(r+5)}^{\left(6\right)}(y_{j_1}^3+6y_{j_1}^2{y}_{j_2})+{(r+6)}^{\left(7\right)}{y}_{j_1}^3{y}_{j_2}\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^4{\eta }_{j_2}{\eta }_{j_3}{\eta }_{j_4}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}\left[\begin{array}{c}{(r+3)}^{\left(4\right)}+7{(r+4)}^{\left(5\right)}{y}_{j_1}\hfill \\ +6{(r+5)}^{\left(6\right)}{y}_{j_1}^2+{(r+6)}^{\left(7\right)}{y}_{j_1}^3\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^3{\eta }_{j_2}^3{\eta }_{j_3}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}\left[\begin{array}{c}{(r+2)}^{\left(3\right)}+{(r+3)}^{\left(4\right)}(3y_{j_1}+3y_{j_2})\hfill \\ +{(r+4)}^{\left(5\right)}(y_{j_1}^2+9y_{j_1}{y}_{j_2}+y_{j_2}^2)\hfill \\ +{(r+5)}^{\left(6\right)}(3y_{j_1}^2{y}_{j_2}+3y_{j_1}{y}_{j_2}^2)+{(r+6)}^{\left(7\right)}{y}_{j_1}^2{y}_{j_2}^2\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^3{\eta }_{j_2}^2{\eta }_{j_3}^2\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}\left[\begin{array}{c}{(r+2)}^{\left(3\right)}+{(r+3)}^{\left(4\right)}(3y_{j_1}+y_{j_2}+y_{j_3})\hfill \\ +{(r+4)}^{\left(5\right)}(y_{j_1}^2+3y_{j_1}{y}_{j_2}+3y_{j_1}{y}_{j_3}+y_{j_2}{y}_{j_3})\hfill \\ +{(r+5)}^{\left(6\right)}(y_{j_1}^2{y}_{j_2}+y_{j_1}^2{y}_{j_3}+3y_{j_1}{y}_{j_2}{y}_{j_3})\hfill \\ +{(r+6)}^{\left(7\right)}{y}_{j_1}^2{y}_{j_2}{y}_{j_3}\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^3{\eta }_{j_2}^2{\eta }_{j_3}{\eta }_{j_4}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}\left[\begin{array}{c}{(r+3)}^{\left(4\right)}+{(r+4)}^{\left(5\right)}(3y_{j_1}+y_{j_2})\hfill \\ +{(r+5)}^{\left(6\right)}(y_{j_1}^2+3y_{j_1}{y}_{j_2})+{(r+6)}^{\left(7\right)}{y}_{j_1}^2{y}_{j_2}\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^3{\eta }_{j_2}{\eta }_{j_3}{\eta }_{j_4}{\eta }_{j_5}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}{y}_{j_5}\left[{(r+4)}^{\left(5\right)}+3{(r+5)}^{\left(6\right)}{y}_{j_1}+{(r+6)}^{\left(7\right)}{y}_{j_1}^2\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^2{\eta }_{j_2}^2{\eta }_{j_3}^2{\eta }_{j_4}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}\left[\begin{array}{c}{(r+3)}^{\left(4\right)}+{(r+4)}^{\left(5\right)}(y_{j_1}+y_{j_2}+y_{j_3})\hfill \\ +{(r+5)}^{\left(6\right)}(y_{j_1}{y}_{j_2}+y_{j_1}{y}_{j_3}+y_{j_2}{y}_{j_3})\hfill \\ +{(r+6)}^{\left(7\right)}{y}_{j_1}{y}_{j_2}{y}_{j_3}\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^2{\eta }_{j_2}^2{\eta }_{j_3}{\eta }_{j_4}{\eta }_{j_5}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}{y}_{j_5}\left[\begin{array}{c}{(r+4)}^{\left(5\right)}+{(r+5)}^{\left(6\right)}(y_{j_1}+y_{j_2})\hfill \\ +{(r+6)}^{\left(7\right)}{y}_{j_1}{y}_{j_2}\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^2{\eta }_{j_2}{\eta }_{j_3}{\eta }_{j_4}{\eta }_{j_5}{\eta }_{j_6}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}{y}_{j_5}{y}_{j_6}\left[{(r+5)}^{\left(6\right)}+{(r+6)}^{\left(7\right)}{y}_{j_1}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}{\eta }_{j_2}{\eta }_{j_3}{\eta }_{j_4}{\eta }_{j_5}{\eta }_{j_6}{\eta }_{j_7}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}{y}_{j_5}{y}_{j_6}{y}_{j_7}{(r+6)}^{\left(7\right)}.\hfill \end{array}$$

8th order: For different $j_1,j_2,j_3,j_4,j_5,j_6,j_7,j_8\in \{1,\dots ,d\}$,

$$\begin{array}{cc}\hfill \mathbb{E}\left[{\eta }_{j_1}^8\right]& =y_{j_1}\left[\begin{array}{c}r+127{(r+1)}^{\left(2\right)}{y}_{j_1}+966{(r+2)}^{\left(3\right)}{y}_{j_2}^2\hfill \\ +1701{(r+3)}^{\left(4\right)}{y}_{j_1}^3+1050{(r+4)}^{\left(5\right)}{y}_{j_1}^4\hfill \\ +266{(r+5)}^{\left(6\right)}{y}_{j_1}^5+28{(r+6)}^{\left(7\right)}{y}_{j_1}^6\hfill \\ +{(r+7)}^{\left(8\right)}{y}_{j_1}^7\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^7{\eta }_{j_2}\right]& =y_{j_1}{y}_{j_2}\left[\begin{array}{c}r+63{(r+2)}^{\left(3\right)}{y}_{j_1}+301{(r+3)}^{\left(4\right)}{y}_{j_1}^2\hfill \\ +350{(r+4)}^{\left(5\right)}{y}_{j_1}^3+140{(r+5)}^{\left(6\right)}{y}_{j_1}^4\hfill \\ +21{(r+6)}^{\left(7\right)}{y}_{j_1}^5+{(r+7)}^{\left(8\right)}{y}_{j_1}^6\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^6{\eta }_{j_2}^2\right]& =y_{j_1}{y}_{j_2}\left[\begin{array}{c}{(r+1)}^{\left(2\right)}+{(r+2)}^{\left(3\right)}(31y_{j_1}+y_{j_2})\hfill \\ +{(r+3)}^{\left(4\right)}(90y_{j_1}^2+31y_{j_1}{y}_{j_2})\hfill \\ +{(r+4)}^{\left(5\right)}(65y_{j_1}^3+90y_{j_1}^2{y}_{j_2})\hfill \\ +{(r+5)}^{\left(6\right)}(15y_{j_1}^4+65y_{j_1}^3{y}_{j_2})\hfill \\ +{(r+6)}^{\left(7\right)}(y_{j_1}^5+15y_{j_1}^4{y}_{j_2})+{(r+7)}^{\left(8\right)}{y}_{j_1}^5{y}_{j_2}\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^6{\eta }_{j_2}{\eta }_{j_3}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}\left[\begin{array}{c}{(r+2)}^{\left(3\right)}+31{(r+3)}^{\left(4\right)}{y}_{j_1}+90{(r+4)}^{\left(5\right)}{y}_{j_1}^2\hfill \\ +65{(r+5)}^{\left(6\right)}{y}_{j_1}^3+15{(r+6)}^{\left(7\right)}{y}_{j_1}^4+{(r+7)}^{\left(8\right)}{y}_{j_1}^5\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^5{\eta }_{j_2}^3\right]& =y_{j_1}{y}_{j_2}\left[\begin{array}{c}{(r+1)}^{\left(2\right)}+{(r+2)}^{\left(3\right)}(15y_{j_1}+3y_{j_2})\hfill \\ +{(r+3)}^{\left(4\right)}(25y_{j_1}^2+45y_{j_1}{y}_{j_2}+y_{j_2}^2)\hfill \\ +{(r+4)}^{\left(5\right)}(10y_{j_1}^3+75y_{j_1}^2{y}_{j_2}+15y_{j_1}{y}_{j_2}^2)\hfill \\ +{(r+5)}^{\left(6\right)}(y_{j_4}^4+30y_{j_1}^3{y}_{j_2}+25y_{j_1}^2{y}_{j_2}^2)\hfill \\ +{(r+6)}^{\left(7\right)}(3y_{j_1}^4{y}_{j_2}+10y_{j_1}^3{y}_{j_2}^2)+{(r+7)}^{\left(8\right)}{y}_{j_1}^4{y}_{j_2}^2\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^5{\eta }_{j_2}^2{\eta }_{j_3}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}\left[\begin{array}{c}{(r+2)}^{\left(3\right)}+{(r+3)}^{\left(4\right)}(15y_{j_1}+y_{j_2})\hfill \\ +{(r+4)}^{\left(5\right)}(25y_{j_1}^2+15y_{j_1}{y}_{j_2})\hfill \\ +{(r+5)}^{\left(6\right)}(10y_{j_1}^3+25y_{j_1}^2{y}_{j_2})\hfill \\ +{(r+6)}^{\left(7\right)}(y_{j_1}^4+10y_{j_1}^3{y}_{j_2})+{(r+7)}^{\left(8\right)}{y}_{j_1}^4{y}_{j_2}\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^5{\eta }_{j_2}{\eta }_{j_3}{\eta }_{j_4}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}\left[\begin{array}{c}{(r+3)}^{\left(4\right)}+15{(r+4)}^{\left(5\right)}{y}_{j_1}+25{(r+5)}^{\left(6\right)}{y}_{j_1}^2\hfill \\ +10{(r+6)}^{\left(7\right)}{y}_{j_1}^3+{(r+7)}^{\left(8\right)}{y}_{j_1}^4\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^4{\eta }_{j_2}^4\right]& =y_{j_1}{y}_{j_2}\left[\begin{array}{c}{(r+1)}^{\left(2\right)}+{(r+2)}^{\left(3\right)}(7y_{j_1}+7y_{j_2})\hfill \\ +{(r+3)}^{\left(4\right)}(6y_{j_1}^2+49y_{j_1}{y}_{j_2}+6y_{j_2}^2)\hfill \\ +{(r+4)}^{\left(5\right)}(y_{j_1}^3+42y_{j_1}^2{y}_{j_2}+42y_{j_1}{y}_{j_2}^2+y_{j_2}^3)\hfill \\ +{(r+5)}^{\left(6\right)}(7y_{j_1}^3{y}_{j_2}+36y_{j_1}^2{y}_{j_2}^2+7y_{j_1}{y}_{j_2}^3)\hfill \\ +{(r+6)}^{\left(7\right)}(6y_{j_1}^3{y}_{j_2}^2+6y_{j_1}^2{y}_{j_2}^3)+{(r+7)}^{\left(8\right)}{y}_{j_1}^3{y}_{j_2}^3\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^4{\eta }_{j_2}^3{\eta }_{j_3}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}\left[\begin{array}{c}{(r+2)}^{\left(3\right)}+{(r+3)}^{\left(4\right)}(7y_{j_1}+3y_{j_2})\hfill \\ +{(r+4)}^{\left(5\right)}(6y_{j_1}^2+21y_{j_1}{y}_{j_2}+y_{j_2}^2)\hfill \\ +{(r+5)}^{\left(6\right)}(y_{j_1}^3+18y_{j_1}^2{y}_{j_2}+7y_{j_1}{y}_{j_2}^2)\hfill \\ +{(r+6)}^{\left(7\right)}(3y_{j_1}^3{y}_{j_2}+6y_{j_1}^2{y}_{j_2}^2)+{(r+7)}^{\left(8\right)}{y}_{j_1}^3{y}_{j_2}^2\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^4{\eta }_{j_2}^2{\eta }_{j_3}^2\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}\left[\begin{array}{c}{(r+2)}^{\left(3\right)}+{(r+3)}^{\left(4\right)}(7y_{j_1}+y_{j_2}+y_{j_3})\hfill \\ +{(r+4)}^{\left(5\right)}(6y_{j_1}^2+7y_{j_1}{y}_{j_2}+7y_{j_1}{y}_{j_3}+y_{j_2}{y}_{j_3})\hfill \\ +{(r+5)}^{\left(6\right)}(y_{j_1}^3+6y_{j_1}^2{y}_{j_2}+6y_{j_1}^2{y}_{j_3}+7y_{j_1}{y}_{j_2}{y}_{j_3})\hfill \\ +{(r+6)}^{\left(7\right)}(y_{j_1}^3{y}_{j_2}+y_{j_1}^3{y}_{j_3}+6y_{j_1}^2{y}_{j_2}{y}_{j_3})\hfill \\ +{(r+7)}^{\left(8\right)}{y}_{j_1}^3{y}_{j_2}{y}_{j_3}\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^4{\eta }_{j_2}^2{\eta }_{j_3}{\eta }_{j_4}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}\left[\begin{array}{c}{(r+3)}^{\left(4\right)}+{(r+4)}^{\left(5\right)}(7y_{j_1}+y_{j_2})\hfill \\ +{(r+5)}^{\left(6\right)}(6y_{j_1}^2+7y_{j_1}{y}_{j_2})\hfill \\ +{(r+6)}^{\left(7\right)}(y_{j_1}^3+6y_{j_1}^2{y}_{j_2})+{(r+7)}^{\left(8\right)}{y}_{j_1}^3{y}_{j_2}\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^4{\eta }_{j_2}{\eta }_{j_3}{\eta }_{j_4}{\eta }_{j_5}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}{y}_{j_5}\left[\begin{array}{c}{(r+4)}^{\left(5\right)}+7{(r+5)}^{\left(6\right)}{y}_{j_1}+6{(r+6)}^{\left(7\right)}{y}_{j_1}^2\hfill \\ +{(r+7)}^{\left(8\right)}{y}_{j_1}^3\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^3{\eta }_{j_2}^3{\eta }_{j_3}^2\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}\left[\begin{array}{c}{(r+2)}^{\left(3\right)}+{(r+3)}^{\left(4\right)}(3y_{j_1}+3y_{j_2}+y_{j_3})\hfill \\ +{(r+4)}^{\left(5\right)}\left(\begin{array}{c}{y}_{j_1}^2+y_{j_2}^2+3y_{j_1}{y}_{j_3}\hfill \\ +3y_{j_2}{y}_{j_3}+9y_{j_1}{y}_{j_2}\hfill \end{array}\right)\hfill \\ +{(r+5)}^{\left(6\right)}\left(\begin{array}{c}{y}_{j_1}^2{y}_{j_3}+y_{j_2}^2{y}_{j_3}+3y_{j_1}^2{y}_{j_2}\hfill \\ +3y_{j_1}{y}_{j_2}^2+9y_{j_1}{y}_{j_2}{y}_{j_3}\hfill \end{array}\right)\hfill \\ +{(r+6)}^{\left(7\right)}(y_{j_1}^2{y}_{j_2}^2+3y_{j_1}^2{y}_{j_2}{y}_{j_3}+3y_{j_1}{y}_{j_2}^2{y}_{j_3})\hfill \\ +{(r+7)}^{\left(8\right)}{y}_{j_1}^2{y}_{j_2}^2{y}_{j_3}\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^3{\eta }_{j_2}^3{\eta }_{j_3}{\eta }_{j_4}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}\left[\begin{array}{c}{(r+3)}^{\left(4\right)}+{(r+4)}^{\left(5\right)}(3y_{j_1}+3y_{j_2})\hfill \\ +{(r+5)}^{\left(6\right)}(y_{j_1}^2+9y_{j_1}{y}_{j_2}+y_{j_2}^2)\hfill \\ +{(r+6)}^{\left(7\right)}(3y_{j_1}^2{y}_{j_2}+3y_{j_1}{y}_{j_2}^2)\hfill \\ +{(r+7)}^{\left(8\right)}{y}_{j_1}^2{y}_{j_2}^2\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^3{\eta }_{j_2}^2{\eta }_{j_3}^2{\eta }_{j_4}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}\left[\begin{array}{c}{(r+3)}^{\left(4\right)}+{(r+4)}^{\left(5\right)}(3y_{j_1}+y_{j_2}+y_{j_3})\hfill \\ +{(r+5)}^{\left(6\right)}(3y_{j_1}{y}_{j_2}+3y_{j_1}{y}_{j_3}+y_{j_2}{y}_{j_3})\hfill \\ +{(r+6)}^{\left(7\right)}(y_{j_1}^2{y}_{j_2}+y_{j_1}^2{y}_{j_3}+3y_{j_1}{y}_{j_2}{y}_{j_3})\hfill \\ +{(r+7)}^{\left(8\right)}{y}_{j_1}^2{y}_{j_2}{y}_{j_3}\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^3{\eta }_{j_2}^2{\eta }_{j_3}{\eta }_{j_4}{\eta }_{j_5}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}{y}_{j_5}\left[\begin{array}{c}{(r+4)}^{\left(5\right)}+{(r+5)}^{\left(6\right)}(3y_{j_1}+y_{j_2})\hfill \\ +{(r+6)}^{\left(7\right)}(y_{j_1}^2+3y_{j_1}{y}_{j_2})\hfill \\ +{(r+7)}^{\left(8\right)}{y}_{j_1}^2{y}_{j_2}\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^3{\eta }_{j_2}{\eta }_{j_3}{\eta }_{j_4}{\eta }_{j_5}{\eta }_{j_6}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}{y}_{j_5}{y}_{j_6}\left[{(r+5)}^{\left(6\right)}+3{(r+6)}^{\left(7\right)}{y}_{j_1}+{(r+7)}^{\left(8\right)}{y}_{j_1}^2\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^2{\eta }_{j_2}^2{\eta }_{j_3}^2{\eta }_{j_4}^2\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}\left[\begin{array}{c}{(r+3)}^{\left(4\right)}+{(r+4)}^{\left(5\right)}(y_{j_1}+y_{j_2}+y_{j_3}+y_{j_4})\hfill \\ +{(r+5)}^{\left(6\right)}\left(\begin{array}{c}{y}_{j_1}{y}_{j_2}+y_{j_1}{y}_{j_3}+y_{j_1}{y}_{j_4}\hfill \\ +y_{j_2}{y}_{j_3}+y_{j_2}{y}_{j_4}+y_{j_3}{y}_{j_4}\hfill \end{array}\right)\hfill \\ +{(r+6)}^{\left(7\right)}\left(\begin{array}{c}{y}_{j_1}{y}_{j_2}{y}_{j_3}+y_{j_1}{y}_{j_2}{y}_{j_4}\hfill \\ +y_{j_1}{y}_{j_3}{y}_{j_4}+y_{j_2}{y}_{j_3}{y}_{j_4}\hfill \end{array}\right)\hfill \\ +{(r+7)}^{\left(8\right)}{y}_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^2{\eta }_{j_2}^2{\eta }_{j_3}^2{\eta }_{j_4}{\eta }_{j_5}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}{y}_{j_5}\left[\begin{array}{c}{(r+4)}^{\left(5\right)}+{(r+5)}^{\left(6\right)}(y_{j_1}+y_{j_2}+y_{j_3})\hfill \\ +{(r+6)}^{\left(7\right)}(y_{j_1}{y}_{j_2}+y_{j_1}{y}_{j_3}+y_{j_2}{y}_{j_3})\hfill \\ +{(r+7)}^{\left(8\right)}{y}_{j_1}{y}_{j_2}{y}_{j_3}\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^2{\eta }_{j_2}^2{\eta }_{j_3}{\eta }_{j_4}{\eta }_{j_5}{\eta }_{j_6}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}{y}_{j_5}{y}_{j_6}\left[\begin{array}{c}{(r+5)}^{\left(6\right)}+{(r+6)}^{\left(7\right)}(y_{j_1}+y_{j_2})\hfill \\ +{(r+7)}^{\left(8\right)}{y}_{j_1}{y}_{j_2}\hfill \end{array}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}^2{\eta }_{j_2}{\eta }_{j_3}{\eta }_{j_4}{\eta }_{j_5}{\eta }_{j_6}{\eta }_{j_7}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}{y}_{j_5}{y}_{j_6}{y}_{j_7}\left[{(r+6)}^{\left(7\right)}+{(r+7)}^{\left(8\right)}{y}_{j_1}\right],\hfill \\ \hfill \mathbb{E}\left[{\eta }_{j_1}{\eta }_{j_2}{\eta }_{j_3}{\eta }_{j_4}{\eta }_{j_5}{\eta }_{j_6}{\eta }_{j_7}{\eta }_{j_8}\right]& =y_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}{y}_{j_5}{y}_{j_6}{y}_{j_7}{y}_{j_8}{(r+7)}^{\left(8\right)}.\hfill \end{array}$$

5.2. Computation of the Central Moments up to the Fourth Order

By combining the results of Section 5.1 with some algebraic manipulations, we are now able to calculate the central moments explicitly. The simplifications we apply to arrive at the final boxed expressions below were performed using Mathematica. We use a symbolic calculator like Mathematica to do the simplifications because many terms cancel each other out in every expression; it would be virtually impossible to do the simplifications by hand without making mistakes. While our methodology allows us to obtain simplified formulas for the central moments up to any order in principle (assuming we calculate explicit expressions for the appropriate higher order non-central moments in Section 5.1), it would be quite time-consuming for us to input the base formula for the central moments as a function of the non-central moments in Mathematica and let Mathematica do the simplifications beyond the fourth order. Therefore, for the sake of conciseness, we only present explicit simplified formulas for the central moments up to the fourth order below. It is worth noting that the numerical formulas we developed in Section 4 are fast for higher orders (i.e., beyond the fourth order) if the categorical probabilities $x_i$ are known; otherwise, Mathematica has trouble calculating for unknown values of $x_i$’s (i.e., Mathematica has trouble getting simplified general expressions by itself. This is why our approach below is necessary.

2nd order: For different $j_1,j_2\in \{1,\dots ,d\}$,

$$\begin{array}{cc}\hfill \mathbb{E}\left[{({\eta }_{j_1}-\mathbb{E}\left[{\eta }_{j_1}\right])}^2\right]& =\mathbb{E}\left[{\eta }_{j_1}^2\right]-{\left(\mathbb{E}\left[{\eta }_{j_1}\right]\right)}^2\hfill \\ \hfill & =y_{j_1}\left[r+{(r+1)}^{\left(2\right)}{y}_{j_1}\right]-r^2{y}_{j_1}^2\hfill \\ \hfill & =\begin{array}{|c|}\hline r{y}_{j_1}(1+y_{j_1})\\ \hline\end{array}\hfill \\ \hfill \mathbb{E}\left[({\eta }_{j_1}-\mathbb{E}\left[{\eta }_{j_1}\right])({\eta }_{j_2}-\mathbb{E}\left[{\eta }_{j_2}\right])\right]& =\mathbb{E}\left[{\eta }_{j_1}{\eta }_{j_2}\right]-\mathbb{E}\left[{\eta }_{j_1}\right]\mathbb{E}\left[{\eta }_{j_2}\right]\hfill \\ \hfill & ={(r+1)}^{\left(2\right)}{y}_{j_1}{y}_{j_2}-r{y}_{j_1}r{y}_{j_2}\hfill \\ \hfill & =\begin{array}{|c|}\hline r{y}_{j_1}{y}_{j_2}\\ \hline\end{array}.\hfill \end{array}$$

3rd order: For different $j_1,j_2,j_3\in \{1,\dots ,d\}$,

$$\begin{array}{cc}\hfill & \mathbb{E}\left[{({\eta }_{j_1}-\mathbb{E}\left[{\eta }_{j_1}\right])}^3\right]\hfill \\ \hfill & =\mathbb{E}\left[{\eta }_{j_1}^3\right]-3\mathbb{E}\left[{\eta }_{j_1}^2\right]\mathbb{E}\left[{\eta }_{j_1}\right]+2{\left(\mathbb{E}\left[{\eta }_{j_1}\right]\right)}^3\hfill \\ \hfill & =y_{j_1}\left[r+3{(r+1)}^{\left(2\right)}{y}_{j_1}+{(r+2)}^{\left(3\right)}{y}_{j_1}^2\right]-3y_{j_1}\left[r+{(r+1)}^{\left(2\right)}{y}_{j_1}\right]r{y}_{j_1}+2r^3{y}_{j_1}^3\hfill \\ \hfill & =\begin{array}{|c|}\hline r{y}_{j_1}(1+3y_{j_1}+2y_{j_1}^2)\\ \hline\end{array}\hfill \\ \hfill & \mathbb{E}\left[{({\eta }_{j_1}-\mathbb{E}\left[{\eta }_{j_1}\right])}^2({\eta }_{j_2}-\mathbb{E}\left[{\eta }_{j_2}\right])\right]\hfill \\ \hfill & =\mathbb{E}\left[{\eta }_{j_1}^2{\eta }_{j_2}\right]-\mathbb{E}\left[{\eta }_{j_1}^2\right]\mathbb{E}\left[{\eta }_{j_2}\right]-2\mathbb{E}\left[{\eta }_{j_1}{\eta }_{j_2}\right]\mathbb{E}\left[{\eta }_{j_1}\right]+2{\left(\mathbb{E}\left[{\eta }_{j_1}\right]\right)}^2\mathbb{E}\left[{\eta }_{j_2}\right]\hfill \\ \hfill & =y_{j_1}{y}_{j_2}\left[{(r+1)}^{\left(2\right)}+{(r+2)}^{\left(3\right)}{y}_{j_1}\right]-y_{j_1}\left[r+{(r+1)}^{\left(2\right)}{y}_{j_1}\right]r{y}_{j_2}\hfill \\ \hfill & -2{(r+1)}^{\left(2\right)}{y}_{j_1}{y}_{j_2}r{y}_{j_1}+2r^2{y}_{j_1}^{2}r{y}_{j_2}\hfill \\ \hfill & =\begin{array}{|c|}\hline r{y}_{j_1}(1+2y_{j_1})y_{j_2}\\ \hline\end{array}\hfill \\ \hfill & \mathbb{E}\left[({\eta }_{j_1}-\mathbb{E}\left[{\eta }_{j_1}\right])({\eta }_{j_2}-\mathbb{E}\left[{\eta }_{j_2}\right])({\eta }_{j_3}-\mathbb{E}\left[{\eta }_{j_3}\right])\right]\hfill \\ \hfill & =\mathbb{E}\left[{\eta }_{j_1}{\eta }_{j_2}{\eta }_{j_3}\right]-\mathbb{E}\left[{\eta }_{j_1}{\eta }_{j_2}\right]\mathbb{E}\left[{\eta }_{j_3}\right]-\mathbb{E}\left[{\eta }_{j_1}{\eta }_{j_3}\right]\mathbb{E}\left[{\eta }_{j_2}\right]\hfill \\ \hfill & -\mathbb{E}\left[{\eta }_{j_2}{\eta }_{j_3}\right]\mathbb{E}\left[{\eta }_{j_1}\right]+2\mathbb{E}\left[{\eta }_{j_1}\right]\mathbb{E}\left[{\eta }_{j_2}\right]\mathbb{E}\left[{\eta }_{j_3}\right]\hfill \\ \hfill & ={(r+2)}^{\left(3\right)}{y}_{j_1}{y}_{j_2}{y}_{j_3}-{(r+1)}^{\left(2\right)}{y}_{j_1}{y}_{j_2}r{y}_{j_3}-{(r+1)}^{\left(2\right)}{y}_{j_1}{y}_{j_3}r{y}_{j_2}\hfill \\ \hfill & -{(r+1)}^{\left(2\right)}{y}_{j_2}{y}_{j_3}r{y}_{j_1}+2r^3{y}_{j_1}{y}_{j_2}{y}_{j_3}\hfill \\ \hfill & =\begin{array}{|c|}\hline 2r{y}_{j_1}{y}_{j_2}{y}_{j_3}\\ \hline\end{array}.\hfill \end{array}$$

4th order: For different $j_1,j_2,j_3,j_4\in \{1,\dots ,d\}$,

$$\begin{array}{cc}\hfill & \mathbb{E}\left[{({\eta }_{j_1}-\mathbb{E}\left[{\eta }_{j_1}\right])}^4\right]\hfill \\ \hfill & =\mathbb{E}\left[{\eta }_{j_1}^4\right]-4\mathbb{E}\left[{\eta }_{j_1}^3\right]\mathbb{E}\left[{\eta }_{j_1}\right]+6\mathbb{E}\left[{\eta }_{j_1}^2\right]{\left(\mathbb{E}\left[{\eta }_{j_1}\right]\right)}^2-3{\left(\mathbb{E}\left[{\eta }_{j_1}\right]\right)}^4\hfill \\ \hfill & =y_{j_1}\left[r+7{(r+1)}^{\left(2\right)}{y}_{j_1}+6{(r+2)}^{\left(3\right)}{y}_{j_1}^2+{(r+3)}^{\left(4\right)}{y}_{j_1}^3\right]\hfill \\ \hfill & -4y_{j_1}\left[r+3{(r+1)}^{\left(2\right)}{y}_{j_1}+{(r+2)}^{\left(3\right)}{y}_{j_1}^2\right]r{y}_{j_1}\hfill \\ \hfill & +6y_{j_1}\left[r+{(r+1)}^{\left(2\right)}{y}_{j_1}\right]{\left(r{y}_{j_1}\right)}^2-3r^4{y}_{j_1}^4\hfill \\ \hfill & =\begin{array}{|c|}\hline r{y}_{j_1}(1+y_{j_1})(1+3(2+r)y_{j_1}+3(2+r)y_{j_1}^2)\\ \hline\end{array}\hfill \\ \hfill & \mathbb{E}\left[{({\eta }_{j_1}-\mathbb{E}\left[{\eta }_{j_1}\right])}^3({\eta }_{j_2}-\mathbb{E}\left[{\eta }_{j_2}\right])\right]\hfill \\ \hfill & =\mathbb{E}\left[{\eta }_{j_1}^3{\eta }_{j_2}\right]-\mathbb{E}\left[{\eta }_{j_1}^3\right]\mathbb{E}\left[{\eta }_{j_2}\right]-3\mathbb{E}\left[{\eta }_{j_1}^2{\eta }_{j_2}\right]\mathbb{E}\left[{\eta }_{j_1}\right]+3\mathbb{E}\left[{\eta }_{j_1}^2\right]\mathbb{E}\left[{\eta }_{j_1}\right]\mathbb{E}\left[{\eta }_{j_2}\right]\hfill \\ \hfill & +3\mathbb{E}\left[{\eta }_{j_1}{\eta }_{j_2}\right]{\left(\mathbb{E}\left[{\eta }_{j_1}\right]\right)}^2-3{\left(\mathbb{E}\left[{\eta }_{j_1}\right]\right)}^3\mathbb{E}\left[{\eta }_{j_2}\right]\hfill \\ \hfill & =y_{j_1}{y}_{j_2}\left[{(r+1)}^{\left(2\right)}+3{(r+2)}^{\left(3\right)}{y}_{j_1}+{(r+3)}^{\left(4\right)}{y}_{j_1}^2\right]\hfill \\ \hfill & -y_{j_1}\left[r+3{(r+1)}^{\left(2\right)}{y}_{j_1}+{(r+2)}^{\left(3\right)}{y}_{j_1}^2\right]r{y}_{j_2}\hfill \\ \hfill & -3y_{j_1}{y}_{j_2}\left[{(r+1)}^{\left(2\right)}+{(r+2)}^{\left(3\right)}{y}_{j_1}\right]r{y}_{j_1}+3y_{j_1}\left[r+{(r+1)}^{\left(2\right)}{y}_{j_1}\right]r{y}_{j_1}r{y}_{j_2}\hfill \\ \hfill & +3{(r+1)}^{\left(2\right)}{y}_{j_1}{y}_{j_2}{r}^2{y}_{j_1}^2-3r^3{y}_{j_1}^{3}r{y}_{j_2}\hfill \\ \hfill & =\begin{array}{|c|}\hline r{y}_{j_1}(1+3(2+r)y_{j_1}+3(2+r)y_{j_1}^2)y_{j_2}\\ \hline\end{array}\hfill \\ \hfill & \mathbb{E}\left[{({\eta }_{j_1}-\mathbb{E}\left[{\eta }_{j_1}\right])}^2{({\eta }_{j_2}-\mathbb{E}\left[{\eta }_{j_2}\right])}^2\right]\hfill \\ \hfill & =\mathbb{E}\left[{\eta }_{j_1}^2{\eta }_{j_2}^2\right]-2\mathbb{E}\left[{\eta }_{j_1}^2{\eta }_{j_2}\right]\mathbb{E}\left[{\eta }_{j_2}\right]-2\mathbb{E}\left[{\eta }_{j_1}{\eta }_{j_2}^2\right]\mathbb{E}\left[{\eta }_{j_1}\right]+\mathbb{E}\left[{\eta }_{j_1}^2\right]{\left(\mathbb{E}\left[{\eta }_{j_2}\right]\right)}^2+\mathbb{E}\left[{\eta }_{j_2}^2\right]{\left(\mathbb{E}\left[{\eta }_{j_1}\right]\right)}^2\hfill \\ \hfill & +4\mathbb{E}\left[{\eta }_{j_1}{\eta }_{j_2}\right]\mathbb{E}\left[{\eta }_{j_1}\right]\mathbb{E}\left[{\eta }_{j_2}\right]-3{\left(\mathbb{E}\left[{\eta }_{j_1}\right]\right)}^2{\left(\mathbb{E}\left[{\eta }_{j_2}\right]\right)}^2\hfill \\ \hfill & =y_{j_1}{y}_{j_2}\left[{(r+1)}^{\left(2\right)}+{(r+2)}^{\left(3\right)}(y_{j_1}+y_{j_2})+{(r+3)}^{\left(4\right)}{y}_{j_1}{y}_{j_2}\right]\hfill \\ \hfill & -2y_{j_1}{y}_{j_2}\left[{(r+1)}^{\left(2\right)}+{(r+2)}^{\left(3\right)}{y}_{j_1}\right]r{y}_{j_2}-2y_{j_1}{y}_{j_2}\left[{(r+1)}^{\left(2\right)}+{(r+2)}^{\left(3\right)}{y}_{j_2}\right]r{y}_{j_1}\hfill \\ \hfill & +y_{j_1}\left[r+{(r+1)}^{\left(2\right)}{y}_{j_1}\right]r^2{y}_{j_2}^2+y_{j_2}\left[r+{(r+1)}^{\left(2\right)}{y}_{j_2}\right]r^2{y}_{j_1}^2\hfill \\ \hfill & +4{(r+1)}^{\left(2\right)}{y}_{j_1}{y}_{j_2}r{y}_{j_1}r{y}_{j_2}-3r^2{y}_{j_1}^2{r}^2{y}_{j_2}^2\hfill \\ \hfill & =\begin{array}{|c|}\hline r{y}_{j_1}{y}_{j_2}(1+2y_{j_2}+y_{j_1}(2+6y_{j_2})+r(1+y_{j_1}+y_{j_2}+3y_{j_1}{y}_{j_2}))\\ \hline\end{array}\hfill \\ \hfill & \mathbb{E}\left[{({\eta }_{j_1}-\mathbb{E}\left[{\eta }_{j_1}\right])}^2({\eta }_{j_2}-\mathbb{E}\left[{\eta }_{j_2}\right])({\eta }_{j_3}-\mathbb{E}\left[{\eta }_{j_3}\right])\right]\hfill \\ \hfill & =\mathbb{E}\left[{\eta }_{j_1}^2{\eta }_{j_2}{\eta }_{j_3}\right]-\mathbb{E}\left[{\eta }_{j_1}^2{\eta }_{j_2}\right]\mathbb{E}\left[{\eta }_{j_3}\right]-\mathbb{E}\left[{\eta }_{j_1}^2{\eta }_{j_3}\right]\mathbb{E}\left[{\eta }_{j_2}\right]-2\mathbb{E}\left[{\eta }_{j_1}{\eta }_{j_2}{\eta }_{j_3}\right]\mathbb{E}\left[{\eta }_{j_1}\right]\hfill \\ \hfill & +\mathbb{E}\left[{\eta }_{j_1}^2\right]\mathbb{E}\left[{\eta }_{j_2}\right]\mathbb{E}\left[{\eta }_{j_3}\right]+2\mathbb{E}\left[{\eta }_{j_1}{\eta }_{j_2}\right]\mathbb{E}\left[{\eta }_{j_1}\right]\mathbb{E}\left[{\eta }_{j_3}\right]+2\mathbb{E}\left[{\eta }_{j_1}{\eta }_{j_3}\right]\mathbb{E}\left[{\eta }_{j_1}\right]\mathbb{E}\left[{\eta }_{j_2}\right]\hfill \\ \hfill & +\mathbb{E}\left[{\eta }_{j_2}{\eta }_{j_3}\right]{\left(\mathbb{E}\left[{\eta }_{j_1}\right]\right)}^2-3{\left(\mathbb{E}\left[{\eta }_{j_1}\right]\right)}^2\mathbb{E}\left[{\eta }_{j_2}\right]\mathbb{E}\left[{\eta }_{j_3}\right]\hfill \\ \hfill & =y_{j_1}{y}_{j_2}{y}_{j_3}\left[{(r+2)}^{\left(3\right)}+{(r+3)}^{\left(4\right)}{y}_{j_1}\right]-y_{j_1}{y}_{j_2}[{(r+1)}^{\left(2\right)}+{(r+2)}^{\left(3\right)}{y}_{j_1}]r{y}_{j_3}\hfill \\ \hfill & -y_{j_1}{y}_{j_3}\left[{(r+1)}^{\left(2\right)}+{(r+2)}^{\left(3\right)}{y}_{j_1}\right]r{y}_{j_2}-2{(r+2)}^{\left(3\right)}{y}_{j_1}{y}_{j_2}{y}_{j_3}r{y}_{j_1}\hfill \\ \hfill & +y_{j_1}\left[r+{(r+1)}^{\left(2\right)}{y}_{j_1}\right]r{y}_{j_2}r{y}_{j_3}+2{(r+1)}^{\left(2\right)}{y}_{j_1}{y}_{j_2}r{y}_{j_1}r{y}_{j_3}\hfill \\ \hfill & +2{(r+1)}^{\left(2\right)}{y}_{j_1}{y}_{j_3}r{y}_{j_1}r{y}_{j_2}+{(r+1)}^{\left(2\right)}{y}_{j_2}{y}_{j_3}{r}^2{y}_{j_1}^2-3r^2{y}_{j_1}^{2}r{y}_{j_2}r{y}_{j_3}\hfill \\ \hfill & =\begin{array}{|c|}\hline r(2+r)y_{j_1}(1+3y_{j_1})y_{j_2}{y}_{j_3}\\ \hline\end{array}\hfill \\ \hfill & \mathbb{E}\left[({\eta }_{j_1}-\mathbb{E}\left[{\eta }_{j_1}\right])({\eta }_{j_2}-\mathbb{E}\left[{\eta }_{j_2}\right])({\eta }_{j_3}-\mathbb{E}\left[{\eta }_{j_3}\right])({\eta }_{j_4}-\mathbb{E}\left[{\eta }_{j_4}\right])\right]\hfill \\ \hfill & =\mathbb{E}\left[{\eta }_{j_1}{\eta }_{j_2}{\eta }_{j_3}{\eta }_{j_4}\right]-\mathbb{E}\left[{\eta }_{j_1}{\eta }_{j_2}{\eta }_{j_3}\right]\mathbb{E}\left[{\eta }_{j_4}\right]-\mathbb{E}\left[{\eta }_{j_1}{\eta }_{j_2}{\eta }_{j_4}\right]\mathbb{E}\left[{\eta }_{j_3}\right]-\mathbb{E}\left[{\eta }_{j_1}{\eta }_{j_3}{\eta }_{j_4}\right]\mathbb{E}\left[{\eta }_{j_2}\right]\hfill \\ \hfill & -\mathbb{E}\left[{\eta }_{j_2}{\eta }_{j_3}{\eta }_{j_4}\right]\mathbb{E}\left[{\eta }_{j_1}\right]+\mathbb{E}\left[{\eta }_{j_1}{\eta }_{j_2}\right]\mathbb{E}\left[{\eta }_{j_3}\right]\mathbb{E}\left[{\eta }_{j_4}\right]+\mathbb{E}\left[{\eta }_{j_1}{\eta }_{j_3}\right]\mathbb{E}\left[{\eta }_{j_2}\right]\mathbb{E}\left[{\eta }_{j_4}\right]\hfill \\ \hfill & +\mathbb{E}\left[{\eta }_{j_1}{\eta }_{j_4}\right]\mathbb{E}\left[{\eta }_{j_2}\right]\mathbb{E}\left[{\eta }_{j_3}\right]+\mathbb{E}\left[{\eta }_{j_2}{\eta }_{j_3}\right]\mathbb{E}\left[{\eta }_{j_1}\right]\mathbb{E}\left[{\eta }_{j_4}\right]+\mathbb{E}\left[{\eta }_{j_2}{\eta }_{j_4}\right]\mathbb{E}\left[{\eta }_{j_1}\right]\mathbb{E}\left[{\eta }_{j_3}\right]\hfill \\ \hfill & +\mathbb{E}\left[{\eta }_{j_3}{\eta }_{j_4}\right]\mathbb{E}\left[{\eta }_{j_1}\right]\mathbb{E}\left[{\eta }_{j_2}\right]-3\mathbb{E}\left[{\eta }_{j_1}\right]\mathbb{E}\left[{\eta }_{j_2}\right]\mathbb{E}\left[{\eta }_{j_3}\right]\mathbb{E}\left[{\eta }_{j_4}\right]\hfill \\ \hfill & ={(r+3)}^{\left(4\right)}{y}_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}-{(r+2)}^{\left(3\right)}{y}_{j_1}{y}_{j_2}{y}_{j_3}r{y}_{j_4}-{(r+2)}^{\left(3\right)}{y}_{j_1}{y}_{j_2}{y}_{j_4}r{y}_{j_3}\hfill \\ \hfill & -{(r+2)}^{\left(3\right)}{y}_{j_1}{y}_{j_3}{y}_{j_4}r{y}_{j_2}-{(r+2)}^{\left(3\right)}{y}_{j_2}{y}_{j_3}{y}_{j_4}r{y}_{j_1}+{(r+1)}^{\left(2\right)}{y}_{j_1}{y}_{j_2}r{y}_{j_3}r{y}_{j_4}\hfill \\ \hfill & +{(r+1)}^{\left(2\right)}{y}_{j_1}{y}_{j_3}r{y}_{j_2}r{y}_{j_4}+{(r+1)}^{\left(2\right)}{y}_{j_1}{y}_{j_4}r{y}_{j_2}r{y}_{j_3}+{(r+1)}^{\left(2\right)}{y}_{j_2}{y}_{j_3}r{y}_{j_1}r{y}_{j_4}\hfill \\ \hfill & +{(r+1)}^{\left(2\right)}{y}_{j_2}{y}_{j_4}r{y}_{j_1}r{y}_{j_3}+{(r+1)}^{\left(2\right)}{y}_{j_3}{y}_{j_4}r{y}_{j_1}r{y}_{j_2}-3r^4{y}_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}\hfill \\ \hfill & =\begin{array}{|c|}\hline 3r(2+r)y_{j_1}{y}_{j_2}{y}_{j_3}{y}_{j_4}\\ \hline\end{array}.\hfill \end{array}$$

6. Open Problems

Here are some research questions for the reader that are of interest:

• Using the moment formulas in the present paper, extend to the negative multinomial distribution the local limit theorem, total variation bound and Le Cam distance bound found in Lemma 1, Theorems 3 and 4 of Ouimet [39] for the negative binomial distribution ($d=1$).
• Using the moment formulas in the present paper, study the asymptotic properties of the Bernstein estimator with a negative multinomial kernel, as was carried out for the Bernstein estimator with a multinomial kernel on the simplex in Ouimet [40].
• Investigate whether the negative multinomial distribution is a completely monotonic function of its parameters. This question was answered positively by Ouimet [29] and Qi et al. [41] for the multinomial distribution, who showed that it is in fact even logarithmically completely monotonic. The same result was extended to a matrix-parametrized generalization by Ouimet and Qi [42].