1. Introduction
The Laplace integrals find applications in numerous problems of mathematics and applied science, and the literature on these integrals is abundant. For example, let us mention the applications in statistical physics, see e.g., [
1] or Lecture 5 in [
2], in the pattern analysis [
3], in the large deviation theory [
4,
5,
6], where it is sometimes referred to as the Laplace–Varadhan method, in the analysis of Weibullian chaos [
7], in the asymptotic methods for large excursion probabilities [
8], in the asymptotic analysis of stochastic processes [
9], and in the calculation of the tunneling effects in quantum mechanics and quantum fields, see [
10,
11]. It can be used to essentially simplify Maslov’s type derivation of the Gibbs, Bose–Einstein and Pareto distribution [
12]. An infinite-dimensional version and a non-commutative versions of the Laplace approximations were developed recently in [
13,
14], respectively.
The majority of research on this topic is devoted to the asymptotic expansions, or even, following the general approach to large deviation of Varadhan, just to the logarithmic asymptotics, see also [
15]. In the present paper, following the recent trend for the searching of the best constants for the error term in the central-limit-type results, see [
16] and references therein, we are interested in exact estimates for the main error term of the Laplace approximation. This approach to Laplace integrals was initiated by the author in book [
9] (Appendix B), where the stress was on the integrals with complex phase. Here we aimed at making these asymptotic more precise for real phase including the most general case of both exponent and the pre-exponential term in the integral depending on the parameter (which is crucial for the applications to the classical conditional large numbers (LLN) that we have in mind here), and stressing two new applications, to the sums instead of integrals (Laplace–Varadhan asymptotics) and to the conditional law of large numbers (LLN) and central limit theorems (CLT) of large deviations.
The content of the paper is as follows. In
Section 2 we obtained the estimates for the error term in Laplace approximation with minimum of the phase in the interior of the domain of integration improving slightly on estimates from [
9], and in
Section 3 we derived the resulting LLN and CLT results. In
Section 4 and
Section 5 the same program was carried out for the case of phase minima occurring in the border of the domain. In
Section 6 we derived the analogous results for the case of sums, rather than integrals. In
Section 7 we show how our results can be applied to the conditional LLN and CLT of large deviations.
2. Phase Minimum Inside the Domain of Integration
Here we present the estimates of the remainder in the asymptotic formula for the Laplace integrals with the critical point of the phase lying in the interior of the domain of integration, adapting and streamlining the arguments of [
9].
Consider the integral
where
is an open bounded subset of the Euclidean space
, equipped with the Euclidean norm
, with Euclidean volume
, the amplitude
f and the phase
S are continuous real functions of
.
Remark 1. The assumption that Ω is bounded is not essential, but simplifies explicit estimates for the error terms. One should think of Ω as a bounded subset of the full domain of integration containing all minimum points of . If f is integrable outside Ω, the integral of over will be exponentially small as compared with Equation (
1).
Recall that the
kth order derivative
of a real function
on
can be viewed as the multi-linear map
The second derivative will be written as usual in the matrix form
We shall denote by
the corresponding norm defined as the lowest constant for which the estimate
holds for all
v.
Remark 2. It is a standard way to define norms of multi-linear mappings, see e.g., [17]. However, as all norms on finite-dimensional spaces are equivalent, the choice of a norm is not very essential here. Let us make now the following assumptions on the functions f and S:
(C1)
is a Lipshitz continuous function of
x with
(C2)
is a thrice continuously differentiable function in
x such that
and
for all
,
,
, with positive constants
; the latter condition can be concisely written as
where the usual ordering on symmetric matrices is used;
(C3) For any
there exists a unique point
of global minimum of
in
, and the ball
is contained in
. Let us denote by
the matrix of the second derivatives of
S at
, that is
Notice that from convexity of
S in
and Assumption (C3) it follows that
Our approach to the study of the Laplace integral
is based on its decomposition
with
Remark 3. In the proof below one can use instead of Equations (
2)
with , the lower bounds coming from the estimate of below, and the upper bound from the estimate of below. Proposition 1. Under Assumptions (C1)–(C3),where is a bounded function depending on , and is exponentially small, compared to the main term. Explicitly Proof. From the Taylor formula for functions on
it follows that
Consequently, for
we have by Assumption (C2) that
It follows then from Equation (
4) that
so that
To go further we shall need the Taylor expansion of
S up to the third order. Namely, from Equation (
9) we deduce the expansion
where, due to the equation
,
Turning to
we further decompose it into the four integrals
with
It follows from Equation (
14) that, for
,
. Using Equation (
14) again and the trivial estimate
, we conclude that, for
,
From the standard integral
we deduce that
Next,
or, using Equation (
17) with
,
Next,
where
is the area of the unit sphere in
. Changing
r to
z so that
and thus
, the last integral rewrites as
so that, using the inequality
,
Remark 4. For we get simplyand for the same with instead of 2.
Finally
is calculated explicitly giving the main term of asymptotics:
Summarizing the estimates for all integrals involved and performing elementary simplifications, in particular using
and
, yields estimate Equation (
7). □
Proposition 2. Under (C1)–(C3) assume additionally that S is four times differentiable and f has a Lipschitz continuous first derivatives with respect to x with Thenwhere the exponentially small term has exactly the same estimate as in the previous Proposition and is a bounded function depending on . Explicitly, Remark 5. The key difference in the error term here is the denominator N instead of in Equation (
6).
Proof. We again decompose
in the sum
with
given by Equation (
5) and estimate
by Equation (
12). Estimation of
needs more careful analysis using further terms of the Taylor expansion of
S and
f. Namely we decompose it first as
with
From Equation (
14) we get
From Equation (
17) with
we deduce that
To evaluate
we use the Taylor expansion of
S to the fourth order yielding
with
Consequently,
can be represented as
with
Using the estimate for
we obtain
To evaluate
we expand
f in Taylor series writing
Substituting this in
and using the fact that the integral of an odd function over a ball centered at the origin vanishes, we get
with
The first two integrals are estimated as above, that is
and
Finally,
was estimated in Proposition 1 by representing it as the difference between the integral over the whole space
and the integral over
, the first term yielding the main term of the asymptotics and the second one being exponentially small. Exponentially small terms are exactly the same as in the previous Proposition. Summarizing the estimates obtained and slightly simplifying, yields Equation (
22). □
3. LLN and CLT for Internal Minima of the Phase
Theorem 1. Let Ω be a bounded open subset of and , be continuous functions on satisfying conditions of Proposition 1. Assume that is strictly positive and the sequence of global minima converges, as , to a point belonging to the interior of Ω.
Let denote a Ω-valued random variable having density that is proportional to , that is (i) Then weakly converge to . More explicitly, for a smooth g, one haswith a constant depending on and , which can be explicitly derived from Equations (
7)
and (
8).
(ii) If additionally S satisfies the conditions of Proposition 2, thenwith a constant depending on and . Proof. From Propositions 1 and 2 we conclude that
and
in cases (i) and (ii) respectively. The estimates of Equations (
25) and (
26) are then obtained from the triangle inequality. □
Next we were interested in the convergence of the normalized fluctuations of
around
, namely, of the random variables
To simplify the formulas below we shall assume that , but everything remains valid under general f satisfying the assumptions above,
To analyze the fluctuations, we use their moment generating functions
for
.
The numerator in Equation (
30) can be written in the form of Equation (
1) as
where the new phase is
To shorten the notations, we shall denote by primes the derivatives of
S or
with respect to the variable
x.
is also convex, as
S is, and has the same derivatives of order 2 and higher as
S. To apply the Laplace method we need to find its point of global minimum, which coincides with its (unique) critical point, that we denote by
and that solves the equation
As a preliminary step to proving our CLT let us perform some elementary analysis of this equation proving its well posedness and finding its dependence on N in the first approximation. We shall need the following elementary result.
Lemma 1. Let be a smooth convex function in s.t. everywhere and . Then for any K the mapping is a diffeomorphism of the ball on its image and this image contains the ball .
Proof. Injectivity is straightforward from convexity. Let us prove the last statement, that is, that for any
there exists
such that
. For any
, this claim is equivalent to the existence of a fixed point for a mapping
in
. By the famous fixed point principle, to show the existence of a fixed point, it is sufficient to show that
maps
to itself, that is,
whenever
. Let
and take
. Then the symmetric matrix
is such that
for all
. Hence, if
we have
Hence, the inequality is fulfilled whenever , as was claimed. □
Thus the image of the set contains the ball of radius , so that for every there exists a unique such that .
On the other hand, for any
K we can take
, which is such that
for all
and
. Consequently, by Lemma 1, for such
p and
N, there exists a unique solution
of Equation (
31) in
, and
, i.e.,
Next, expanding
in the Taylor series around
(where
), we find from Equation (
31) that
and thus
(recall that we denote
).
This allows us to improve the preliminary estimate of Equation (
32) and to obtain
Hence from Equation (
33) we get
Finally we conclude that
with
We can now prove a convergence result that can be called the CLT for Laplace integrals.
Theorem 2. Under the assumption of Theorem 1 (i), assume additionally that converges to quickly enough, that iswith positive constants . Then the fluctuations converge weakly to a centered Gaussian random variable with the moment generating function Proof. We show that the moment generating functions of the fluctuations
given by Equation (
30) converge, as
, to the function
, the convergence being uniform on bounded subsets of
p. By the well known characterization of weak convergence this will apply the weak convergence of the random fluctuations
.
Applying Proposition 1 to the numerator and denominator of the r.h.s. of Equation (
30) we get, for
,
where
is a bounded function, with a bound, depending on
, that can be found explicitly from Equation (
7).
We have
with
and consequently
Using Equation (63) we conclude that
where the constant
c depends on
.
Next, from Equation (
35) we get
so that
with another constant
c depending on
. Consequently, we deduce from Equation (
41) that
with some functions
, which are bounded on bounded subsets of
p, implying the required convergence of the functions
. □
4. Phase Minimum on the Border of the Domain of Integration
Here we present the estimates of the remainder in the asymptotic formula for the Laplace integrals with the critical point of the phase lying on the boundary of the domain of integration.
Let us start with a simple one-dimensional result, which is version of the well known Watson lemma. The proof can be performed as above by decomposing the domain of integration into the two intervals: and . We omit the detail of the proof.
Lemma 2. Let and be two continuous functions on the domain with . Let f be continuously differentiable and S be twice continuously differentiable with respect to x, with the uniform boundsand the lower boundwith some strictly positive constants , where by primes we denote derivatives with respect to x. Then, for the Laplace integralwe have the asymptotic expressionwhere Remark 6. One can obtain similar result by decomposing for any , in which case the exponentially small term will get the estimate This also shows that Lemma 2 remains essentially valid for small a of order , , which is used in the proof of the next result.
Let us turn to the general case. Namely, assume
is a bounded open set in
. The coordinates in
will be denoted
with
. Let
with some smooth function
. It will be convenient to introduce the sections of
as the sets
We are interested in the asymptotics of the Laplace integral
with continuous functions
f and
S referred to as the amplitude and phase respectively.
Let us first discuss the case of
with a plane boundary, that is with
, or equivalently with
We shall assume the following:
(C1’)
is a continuously differentiable function on
(up to the border) with
(C2’)
is thrice continuously differentiable function of
x and
y such that
(where ≥ is the usual order on symmetric matrices) and
with positive constants
, and
Remark 7. As was noted above, the norms of higher derivatives in the estimates that we are using are their norms as multi-linear operators. For instance, is the minimum of constants α such that (C3’) For any
, there exists a unique point of global minimum of
S in
, this point lies on the boundary
, i.e., it has coordinates
with some
, and the box
is contained in
. We shall also use the sections
Let us denote by
the matrix of the second derivatives of
S as a function of
y at
, and by
the gradient of
S as a function of
x at
, that is
The approach of our analysis is to decompose the integral
into the sum of two integrals
over the sets
and
, to represent the first integral as the double integral, so that
and to use Proposition 1 for the estimation of
,
, and finally Lemma 2 to estimate
.
Theorem 3. Under the assumptions (C1’)–(C3’), the formulaholds for from Equation (47) and , where is an exponentially small term and Proof. Integral
from Equation (
50) yields clearly an exponentially small contribution, similar to the integral
in Proposition 1, so we omit the details here.
To calculate
we have to know critical points of the phase
as a function of
y, that is the solutions
of the equation
As
S is convex in
y, the solution is unique, if it exists. Proceeding as in Lemma 1, that is, searching for a fixed point of the mapping
we find that there exists a unique solution
of Equation (
54) whenever
such that
Next, using the Taylor expansion of
around the point
we get that
with
This implies
so that
which is an essential improvement as compared with the initial estimate of Equation (
56). It ensures that the distance from
to the border of
is of order
, so that Proposition 1 can in fact be applied to the integral
leading to
where
is exponentially small compared to the main term and
In order to apply Lemma 2 we need to get lower and upper bounds to the quantities
respectively.
But the second term vanishes. Hence
Next, differentiating Equation (
54) with respect to
y we obtain
implying the estimate
Consequently, using the formula for the differentiation of the determinant of invertible symmetric matrices,
we can estimate
Hence Lemma 2 can be applied to the calculation of
given by Equations (
51) and (
59) yielding Equation (
52). □
Remark 8. Arguing as in Proposition 2, one can improve the estimate of the remainder term in Equation (
52)
to be of order , by assuming more regularity on S and f. The general case of Equation (
45) can be directly reduced to the case of
from Equation (
47). In fact, changing coordinates
to
with
we get that
turns to
. Making this change of the variable of integration in
yields
with
,
. Assuming that these functions satisfy the conditions of Theorem 3 we obtain
where
and with similar change in the constants appearing in
and
.
5. LLN and CLT for Minima on the Boundary
The results on weak convergence of random variables with exponential densities given above for the case of the phase having minimum in the interior of the domain can be now recast for the case of the phase having minimum on the boundary of the domain of integration. The following statements are proved by literally the same argument as Theorems 1 and 2. We omit details.
Theorem 4. Let Ω be a bounded open set in with coordinates , , and let Let the functions , be a continuous functions on satisfying condition (C1’)- (C3’) from Theorem 3. Assume moreover that f is bounded below by a positive constants and that the sequence of global minima converges, as , to a point belonging to the interior of Ω.
Let denote a -valued random variable having density that is proportional to , that is Then weakly converge to a constant . More explicitly, for a smooth g, one haswith a constant c depending only on S (actually on the bounds for the derivatives of S up to the third order). Theorem 5. Under the assumptions of Theorem 4 assume additionally that Then the fluctuations converge weakly to a -dimensional random vector such that its last coordinates form a centered Gaussian random vector with the moment generating functionand the first coordinate is independent and represents a - exponential random variable. The rates of convergence with all explicit constants are obtained directly from Theorem 3. 6. Laplace Sums with Error Estimates
It is more or less straightforward to modify the above results to the of sums rather than integrals. Namely, instead of the integral
from Equation (
1) let us consider the sum
where
is an open polyhedron of the Euclidean space
, with Euclidean volume
, the amplitude
f and the phase
S are continuous real functions.
Theorem 6. Under the assumptions of Proposition 1,whereand where and are the same as in Proposition 1 and is yet another constant depending on . Proof. We use the well known (and easy to prove) fact (a simplified version of the Euler–Maclorin formula) that
Consequently,
where
is from Equation (
1). The first integral on the r.h.s. of Equation (
68) is clearly of order
, as compared with the main term of
given in Proposition 1. The pre-exponential term in the second integral vanishes at the critical point
of
. Hence the required estimate for the second integral is obtained directly from Proposition 1. □
Now all LLN and CLT results obtained above for continuous distributions can be reformulated and proved straightforwardly for the case of discrete random variables taking values in the lattice with probabilities proportional to .
7. Application to LLN and CLT of Large Deviations
Conditional LLN (conditioned on the sums of the corresponding random variables to stay in a certain prescribed domain, usually some linear subspace or a convex set) are well developed in probability (see e.g., [
2,
18] for two different contexts). The results above can be used to supply exact estimates for the error terms in these approximations. To illustrate this statement in the most transparent way let us start with the classical multidimensional local theorem of large deviations as given in [
4] (that extends earlier results of [
6]). Namely, let
be a sequence of independent identically distributed
-valued random vectors. Assume that the set
O of vectors
such that the moment generating function
is well defined has a nonempty interior
. It is well known (and easy to see) that the functions
v and
are convex and the sets
and its closure
are convex. The function
is called the entropy and it is concave. Moreover, the infimum in its definition is attained, so that there exists
such that
and the function
is a diffeomorphism of
onto some open domain
in
. Assume that the random variable
has a bounded probability density
, and define the family of distributions
with the densities
Let be the density of the averaged sum .
Theorem 1 of [
4] states (though we formulate it equivalently in terms of the density of
, rather than
as is done in [
4]) that if
is any compact set in
, then
where
s is arbitrary, the estimate is uniform for
,
is the matrix of the second moments of the distributions
, the coefficients
depend only on
moments of
and are uniformly bounded in
.
The densities of Equation (
69) are exactly of the type dealt with in our Theorems 1, 2, and 4, and Equation (
5). Thus, these theorems are applied directly for finding the rates of convergence for LLN and CLT for the sums of independent variables when
is reduced to some convex bounded set with smooth boundary or a linear subspace. These conditional versions of LLN may be applied even if
is not defined, so that the usual LLN does not hold.
When the random variable has values in a lattice, a version with sums, that is Theorem 6, should be applied to get the rates of convergence in the corresponding laws of large numbers.