*5.2. Parameter Estimation under Additive and Multiplicative Observation Noises*

We consider the observations **Y***<sup>T</sup>* col(*Y*1,...,*YT*):

$$Y\_t = aX\_t + V\_{t\prime} \qquad t = \overline{1, T}.\tag{44}$$

Here:


We assume that the parameter *a* is random with unknown distribution, whose support set lies within the known set <sup>C</sup> [*c*1, *<sup>c</sup>*2]. The loss function has the form:

$$J(\overline{a}\_\prime F | \mathbf{Y}\_T) = \mathbb{E}\_F \left\{ ||a - \overline{a}(\mathbf{Y}\_T)||^2 | \mathbf{Y}\_T \right\}.$$

In this example our goal is to compare the minimax estimates of the parameter *a* under conformity constraint based either on the likelihood function or on the EDF.

The minimax estimations are calculated for the following parameter values: *a* = 2, *T* = 20, C = [2, 3], *σ* = 0.1. We use the proposed numerical procedure under a uniform mesh C*<sup>h</sup>* of the set C with the step *h* = 0.005. The example has some features. First, the observation model contains both the additive (**V***T*) and multiplicative (**X***T*) heterogeneous noises. Second, the available observed sample is not too long to provide the high quality for the consistent estimates. Third, the exact value of *a* is equal 2; meanwhile under the constraint absence there exists a discrete variant of the LFD with the finite support set {2, 3}. This means that the LFD is realized only in the considered observation model.

The likelihood conformity constraint looks similar to the one from the previous subsection:

$$\frac{\mathcal{L}(\mathbf{Y}\_{T}, F) - \min\_{q \in \mathcal{L}\_{h}} \mathcal{L}(\mathbf{Y}\_{T}|q)}{\max\_{q \in \mathcal{L}\_{h}} \mathcal{L}(\mathbf{Y}\_{T}|q) - \min\_{q \in \mathcal{L}\_{h}} \mathcal{L}(\mathbf{Y}\_{T}|q)} \geqslant r,\tag{45}$$

where *r* ∈ (0, 1) is a confidence ratio.

Figure <sup>3</sup> contains comparison of the minimax estimate *a*(*r*) with its actual value *<sup>a</sup>*, the (consistent asymptotically Gaussian) M-estimate *asub* <sup>2</sup> *<sup>T</sup>* <sup>∑</sup>*<sup>T</sup> <sup>t</sup>*=<sup>1</sup> *Yt*, obtained by the moment/substitution method [12] and the MLE *aMLE*.

Next, we investigate minimax posterior estimates under the conformity constraint based on the EDF. The constraint is of the form:

$$\frac{\max\_{F \in \mathbb{F}\_{\mathcal{E}\_h}} \mathfrak{M}(\mathbf{Y}\_{T}, F) - \mathfrak{M}(\mathbf{Y}\_{T}, F)}{\max\_{F \in \mathbb{F}\_{\mathcal{E}\_h}} \mathfrak{M}(\mathbf{Y}\_{T}, F) - \min\_{F \in \mathbb{F}\_{\mathcal{E}\_h}} \mathfrak{M}(\mathbf{Y}\_{T}, F)} \gtrsim r,\tag{46}$$

where *<sup>r</sup>* <sup>∈</sup> (0, 1) is some fixed confidence ratio, and <sup>F</sup>C*<sup>h</sup>* is a "mesh" approximation of the set <sup>F</sup><sup>C</sup> corresponding to the uniform "mesh" <sup>C</sup>*h*. The form (46) of the conformity constraint provides that it is active in the minimax optimization problem for any *r* ∈ (0, 1).

**Figure 3.** Estimation of the coefficient *a* under conformity constraint based on the likelihood function.

Figure 4 contains:


Note that *F*<sup>2</sup> *<sup>Y</sup>*(*y*) is a cdf of *Y* corresponding to the actual value of *a*.

**Figure 4.** The EDF of *Y* and different cdf's of *Y* under various choices of *a*.

Figure <sup>5</sup> contains a comparison of the minimax estimate *a*(*r*) under the conformity constraint, based on the EDF, with its actual value *a*, the moment/substitution estimate *asub* and the MLE *aMLE*.

The results of this experiment allow us to make the following conclusions.


**Figure 5.** Estimation of the coefficient *a* under conformity constraint based on the EDF.

#### **6. Conclusions**

The paper contains the statement and solution to a new minimax estimation problem for the uncertain stochastic regression parameters. The optimality criterion is the conditional mean square of the estimation error given the realized observations. The class of admissible estimators contains all (linear and nonlinear) statistics with finite variance. The a priori information concerning the estimated vector is incomplete: the vector is random and the part of its components lies in the known compact. The key feature of the considered problem is the presence of the additional constraints for the statistical uncertainty, restricting from below the correspondence degree between the uncertainty and realized observations. The paper presents various indices, characterizing this conformity via the likelihood function, the EDF and the sample mean.

We propose a reduction of the initial optimization problem in the abstract infinitedimensional spaces to the standard finite-dimensional QP problem with convex constraints along with an algorithm of its numerical realization and precision analysis.

The minimax estimation problem is solved in terms of the saddle points, i.e., besides the estimators with the guaranteed quality, we have a description of the LFDs. First, the investigation of the LFDs' domains allowed us the detection of the uncertain parameter values, which are the worst for the estimation. Second, the consideration of the performance index pair "conformity index–guaranteed estimation quality" uncovered rather a new conception of the parameter estimation under a vector optimality criterion. The paper contains an assertion, which states that the LFDs are Pareto-optimal for the vector-valued criterion above.

The paper focuses mostly on the conformity indices related to the likelihood function; thus, it is obvious that the performance of the minimax estimate is compared with the one of the MLE. In general, the MLE has several remarkable properties, in particular the asymptotic minimaxity under some additional restrictions [12]. However, the estimate is non-robust to the prior statistical uncertainty. The proposed minimax estimate can be considered as a robustified version of the MLE, which is ready for application in the cases of the short non-asymptotic samples or the violation of the conditions for the MLE asymptotic minimaxity.

The conformity constraints are not exhausted by the likelihood function. In the paper, we present other conformity indices based on the EDF and sample mean. We demonstrate that the minimax estimates with the EDF conformity constraint are better than the MLE. One of the points of the paper is that the flexible choice of the conformity indices and design of the additional conformity constraints for each individual applied estimation problem allows obtaining a tradeoff between the prior uncertainty and available observations.

The reason to choose one or another conformity index depends not only on the conditions of the specific practical estimation problem solved under the minimax settings. One of the essential conditions is the possibility of its quick computation for the subsequent verification of the conformity constraint. For example, calculation of the likelihood conformity constraint (33) with the guess value L(*y*|*q*) = max*<sup>q</sup>* L(*y*|*q*) tends to necessarily solve the auxiliary maximization problem for the likelihood function, which is nontrivial itself. Thus, the conformity indices based on the EDF or sample moments look more prospective from the computational point of view.

The applicability of the proposed minimax estimate also depends on the presence of the analytical formula of the estimates *w*(*y*|*q*), or the fast numerical algorithms of its calculation. In turn, this possibility is a base for the subsequent effective solution to the QP problem and specification of the LFD.

Finally, the key indicator affecting the estimate calculation process and its precision is the number of the mesh nodes in the approximation C(*n*) of the uncertainty set C. It is a function of "the size of C / the mesh step *n*" ratio and dimensionality *m* of C.

All of the factors above characterize the limits of possible applicability of the proposed minimax estimation method for the solution to one or another practical problem.

**Funding:** The research was supported by the Ministry of Science and Higher Education of the Russian Federation, project No. 075-15-2020-799.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Data sharing is not applicable to this article.

**Conflicts of Interest:** The author declares no conflict of interest.
