**4.** L*p***-convergence to a Random Complete Linear Differential Equation When the Delay Tends to** 0

Given a discrete delay *τ* > 0, we denote the L*p*-solution (2) to (1) by *xτ*(*t*). We denote the L*p*-solutions (5) and (6) to (3) and (4) by *yτ*(*t*) and *zτ*(*t*), respectively, so that *xτ*(*t*) = *yτ*(*t*) + *zτ*(*t*). Thus, we are making the dependence on the delay *τ* explicit. If we put *τ* = 0 into (1), (3) and (4), we obtain random linear differential equations with no delay:

$$\begin{cases} \mathbf{x}\_0'(t,\omega) = (a(\omega) + b(\omega))\mathbf{x}\_0(t,\omega) + f(t,\omega), \; t \ge 0, \\ \mathbf{x}\_0(0,\omega) = \mathbf{g}(0,\omega), \end{cases} \tag{11}$$

$$\begin{cases} y\_0'(t,\omega) = (a(\omega) + b(\omega))y\_0(t,\omega), \; t \ge 0, \\ y\_0(0,\omega) = g(0,\omega), \end{cases} \tag{12}$$

$$\begin{cases} z\_0'(t,\omega) = (a(\omega) + b(\omega))z\_0(t,\omega) + f(t,\omega), \; t \ge 0, \\ z\_0(0,\omega) = 0, \end{cases} \tag{13}$$

respectively. The following results establish conditions under which (11), (12) and (13) have L*p*-solutions.

**Theorem 8** ([17] Corollary 4.1)**.** *Fix* <sup>1</sup> <sup>≤</sup> *<sup>p</sup>* <sup>&</sup>lt; <sup>∞</sup>*. If <sup>φ</sup>a*(*ζ*) <sup>&</sup>lt; <sup>∞</sup> *and <sup>φ</sup>b*(*ζ*) <sup>&</sup>lt; <sup>∞</sup> *for all <sup>ζ</sup>* <sup>∈</sup> <sup>R</sup>*, and <sup>g</sup>*(0) <sup>∈</sup> <sup>L</sup>*p*+*<sup>η</sup> for certain <sup>η</sup>* <sup>&</sup>gt; <sup>0</sup>*, then the stochastic process <sup>y</sup>*0(*t*) = *<sup>g</sup>*(0)e(*a*+*b*)*<sup>t</sup> is the unique* <sup>L</sup>*p-solution to (12).*

*On the other hand, if <sup>a</sup> and <sup>b</sup> are bounded random variables and <sup>g</sup>*(0) <sup>∈</sup> <sup>L</sup>*p, then the stochastic process y*0(*t*) = *g*(0)e(*a*+*b*)*<sup>t</sup> is the unique* L*p-solution to (12).*

**Theorem 9.** *Fix* <sup>1</sup> <sup>≤</sup> *<sup>p</sup>* <sup>&</sup>lt; <sup>∞</sup>*. If <sup>φ</sup>a*(*ζ*) <sup>&</sup>lt; <sup>∞</sup> *and <sup>φ</sup>b*(*ζ*) <sup>&</sup>lt; <sup>∞</sup> *for all <sup>ζ</sup>* <sup>∈</sup> <sup>R</sup>*, and <sup>f</sup> is continuous on* [0, <sup>∞</sup>) *in the* L*p*+*η-sense for certain η* > 0*, then the stochastic process z*0(*t*) = *t* <sup>0</sup> <sup>e</sup>(*a*+*b*)(*t*−*s*) *<sup>f</sup>*(*s*) <sup>d</sup>*<sup>s</sup> is the unique* L*p-solution to (13).*

*On the other hand, if a and b are bounded random variables and f is continuous on* [0, ∞) *in the* L*p-sense, then the stochastic process z*0(*t*) = *t* <sup>0</sup> <sup>e</sup>(*a*+*b*)(*t*−*s*) *<sup>f</sup>*(*s*) <sup>d</sup>*s is the unique* <sup>L</sup>*p-solution to (13).*

**Proof.** Take the first set of assumptions. Let *F*(*t*,*s*) = e(*a*+*b*)(*t*−*s*) *f*(*s*) be the process inside the integral sign. Since *φ<sup>a</sup>* < ∞ and *φ<sup>b</sup>* < ∞, the chain rule theorem (Proposition 1) allows differentiating e(*a*+*b*)*<sup>t</sup>* in L*q*, for each 1 <sup>≤</sup> *<sup>q</sup>* <sup>&</sup>lt; <sup>∞</sup>. In particular, e(*a*+*b*)(*t*−*s*) is L*q*-continuous at (*t*,*s*), for 1 <sup>≤</sup> *<sup>q</sup>* <sup>&</sup>lt; <sup>∞</sup>. As *<sup>f</sup>* is continuous on [0, ∞) in the L*p*+*η*-sense, we derive that *F* is L*p*-continuous at (*t*,*s*). It also exists *∂F <sup>∂</sup><sup>t</sup>* (*t*,*s*)=(*<sup>a</sup>* <sup>+</sup> *<sup>b</sup>*)e(*a*+*b*)(*t*−*s*) *<sup>f</sup>*(*s*) in L*p*. Since *<sup>a</sup>* <sup>+</sup> *<sup>b</sup>* has absolute moments of any order, (*<sup>a</sup>* <sup>+</sup> *<sup>b</sup>*)e(*a*+*b*)(*t*−*s*) is L*q*-continuous at (*t*,*s*), for 1 <sup>≤</sup> *<sup>q</sup>* <sup>&</sup>lt; <sup>∞</sup>. Then *<sup>∂</sup><sup>F</sup> <sup>∂</sup><sup>t</sup>* is L*p*-continuous at (*t*,*s*). By Proposition 3, *<sup>z</sup>*<sup>0</sup> is L*p*-differentiable and *z* <sup>0</sup>(*t*) = *<sup>F</sup>*(*t*, *<sup>t</sup>*) + *t* 0 *∂F <sup>∂</sup><sup>t</sup>* (*t*,*s*) d*s* = *f*(*t*)+(*a* + *b*)*z*0(*t*), and we are done.

Suppose that *a* and *b* are bounded random variables and *f* is continuous on [0, ∞) in the L*p*-sense. If *a* and *b* are bounded, then e(*a*+*b*)*<sup>t</sup>* is L∞-differentiable (this is because of an application of the deterministic mean value theorem; see ([17] Theorem 3.4)). Then an analogous proof to the previous paragraph works in this case, by only assuming that *f* is continuous on [0, ∞) in the L*p*-sense.

**Theorem 10.** *Fix* <sup>1</sup> <sup>≤</sup> *<sup>p</sup>* <sup>&</sup>lt; <sup>∞</sup>*. If <sup>φ</sup>a*(*ζ*) <sup>&</sup>lt; <sup>∞</sup> *and <sup>φ</sup>b*(*ζ*) <sup>&</sup>lt; <sup>∞</sup> *for all <sup>ζ</sup>* <sup>∈</sup> <sup>R</sup>*, <sup>g</sup>*(0) <sup>∈</sup> <sup>L</sup>*p*+*η, and <sup>f</sup> is continuous on* [0, ∞) *in the* L*p*+*η-sense for certain η* > 0*, then the stochastic process x*0(*t*) = *g*(0)e(*a*+*b*)*<sup>t</sup>* + *t* <sup>0</sup> <sup>e</sup>(*a*+*b*)(*t*−*s*) *<sup>f</sup>*(*s*) <sup>d</sup>*s is the unique* <sup>L</sup>*p-solution to (11).*

*On the other hand, if <sup>a</sup> and <sup>b</sup> are bounded random variables, <sup>g</sup>*(0) <sup>∈</sup> <sup>L</sup>*p, and <sup>f</sup> is continuous on* [0, <sup>∞</sup>) *in the* <sup>L</sup>*p-sense, then the stochastic process x*0(*t*) = *g*(0)e(*a*+*b*)*<sup>t</sup>* + *t* <sup>0</sup> <sup>e</sup>(*a*+*b*)(*t*−*s*) *<sup>f</sup>*(*s*) <sup>d</sup>*<sup>s</sup> is the unique* <sup>L</sup>*p-solution to (11).*

**Proof.** It is a consequence of Theorem 8 and Theorem 9 with *x*0(*t*) = *y*0(*t*) + *z*0(*t*).

Our goal is to establish conditions under which lim*τ*→<sup>0</sup> *<sup>x</sup>τ*(*t*) = *<sup>x</sup>*0(*t*) in L*p*, for each *<sup>t</sup>* <sup>≥</sup> 0. To do so, we will utilize lim*τ*→<sup>0</sup> *yτ*(*t*) = *y*0(*t*) and lim*τ*→<sup>0</sup> *zτ*(*t*) = *z*0(*t*).

The first limit, lim*τ*→<sup>0</sup> *yτ*(*t*) = *y*0(*t*), was demonstrated in ([17] Theorem 4.5), by using inequalities for the deterministic and random delayed exponential function ([36] Theorem A.3), ([17] Lemma 4.2, Lemma 4.3, Lemma 4.4).

**Theorem 11** ([17] Theorem 4.5)**.** *Fix* 1 ≤ *p* < ∞*. Let a and b be bounded random variables and let g be a stochastic process that belongs to <sup>C</sup>*1([−*τ*, 0]) *in the* <sup>L</sup>*p-sense. Then,* lim*τ*→<sup>0</sup> *<sup>y</sup>τ*(*t*) = *<sup>y</sup>*0(*t*) *in* <sup>L</sup>*p, uniformly on* [0, *T*]*, for each T* > 0*.*

Next we prove the convergence lim*τ*→<sup>0</sup> *zτ*(*t*) = *z*0(*t*). As a corollary, we will be able to derive lim*τ*→<sup>0</sup> *xτ*(*t*) = *x*0(*t*).

**Theorem 12.** *Fix* 1 ≤ *p* < ∞*. Let a and b be bounded random variables and let f be a continuous stochastic process on* [0, <sup>∞</sup>) *in the* <sup>L</sup>*p-sense. Then,* lim*τ*→<sup>0</sup> *<sup>z</sup>τ*(*t*) = *<sup>z</sup>*0(*t*) *in* <sup>L</sup>*p, uniformly on* [0, *<sup>T</sup>*]*, for each T* <sup>&</sup>gt; <sup>0</sup>*.*

**Proof.** Notice that *zτ*(*t*) defined by (6) (see the first paragraph of this section) exists by Theorem 5, which used the boundedness of *a* and *b* and the L*p*-continuity of *f* on [0, ∞). Analogously, *z*0(*t*) exists by Theorem 9.

Fix *t* ∈ [0, *T*]. We bound

$$\begin{aligned} \left\||z\_{\mathsf{T}}(t)-z\_{0}(t)|\right\|\_{\mathscr{P}} &\leq \int\_{0}^{t} \left\| \mathbf{e}^{a(t-s)}f(s) \left( \mathbf{e}^{b\_{1}t-\mathsf{T}-s}\_{\mathsf{T}} - \mathbf{e}^{b(t-s)} \right) \right\|\_{\mathscr{P}} \, \mathrm{d}s \\ &\leq \int\_{0}^{t} \left\| \mathbf{e}^{a(t-s)} \right\|\_{\infty} \left\| f(s) \right\|\_{\mathscr{P}} \left\| \mathbf{e}^{b\_{1}t-\mathsf{T}-s}\_{\mathsf{T}} - \mathbf{e}^{b(t-s)} \right\|\_{\mathscr{P}} \, \mathrm{d}s. \end{aligned}$$

We have *ea*(*t*−*s*)<sup>∞</sup> <sup>≤</sup> <sup>e</sup>*a*∞*<sup>T</sup>* and *<sup>f</sup>*(*s*)*<sup>p</sup>* <sup>≤</sup> *Cf* <sup>=</sup> max*s*∈[0,*T*] *<sup>f</sup>*(*s*)*p*. These bounds yield

$$\|\|z\_{\mathsf{T}}(t) - z\_0(t)\|\|\_{\mathcal{V}} \le \mathsf{C}\_f \mathsf{e}^{\|a\|\_{\infty}T} \int\_0^t \left\|\mathsf{e}^{b\_1, t - \mathsf{T} - s}\_{\mathsf{T}} - \mathsf{e}^{b(t - s)}\right\|\Big|\_{\infty} \,\mathrm{d}s.\tag{14}$$

Let *<sup>k</sup>* be a number such that *<sup>k</sup>* ≥ *b*1<sup>∞</sup> <sup>=</sup> e*aτb*∞, for all *<sup>τ</sup>* <sup>∈</sup> (0, 1]. By ([17] Lemma 4.3),

$$\left\| \mathbf{e}\_{\mathsf{T}}^{b\_1, t - \mathsf{T} - \mathsf{s}} - \mathbf{e}^{b\_1(t - \mathsf{s})} \right\|\_{\infty} \leq C\_{T, k} \cdot \mathsf{T}\_{\mathsf{V}} \tag{15}$$

for *t* ∈ [0, *T*], 0 ≤ *s* ≤ *t* and *τ* ∈ (0, 1]. On the other hand, by the deterministic mean value theorem (applied for each outcome *ω*),

$$\begin{aligned} \mathbf{e}^{b\_1(t-s)} - \mathbf{e}^{b(t-s)} &= \mathbf{e}^{\mathbf{e}^{-a\tau}b(t-s)} - \mathbf{e}^{b(t-s)} \\ &= b(t-s)\mathbf{e}^{\mathbf{f}\_{\mathbf{r},\omega}b(t-s)}(\mathbf{e}^{-a\tau} - 1), \end{aligned}$$

where *ξτ*,*<sup>ω</sup>* <sup>∈</sup> (1, e−*aτ*) <sup>∪</sup> (e−*aτ*, 1). In particular, <sup>|</sup>*ξτ*,*ω*| ≤ <sup>1</sup> <sup>+</sup> <sup>e</sup>*a*<sup>∞</sup> . We apply again the deterministic mean value theorem to e−*a<sup>τ</sup>* <sup>−</sup> 1:

$$\mathbf{e}^{-a\tau} - 1 = \mathbf{e}^{\frac{\mathbf{y}}{\sigma\_{\tau\omega}}}(-a\tau),$$

where *ξτ*,*<sup>ω</sup>* ∈ (−*aτ*, 0) ∪ (0, −*aτ*). In particular,

$$||\mathbf{e}^{-a\tau} - 1||\_{\infty} \le \mathbf{e}^{||a||\_{\infty}}||a||\_{\infty}\tau.$$

As a consequence,

$$\|\mathbf{e}^{b\_1(t-s)} - \mathbf{e}^{b(t-s)}\|\_{\infty} \le \underbrace{\|b\|\|\_{\infty} T \mathbf{e}^{\left(1 + \mathbf{e}^{\|\boldsymbol{a}\|\infty}\right) \|b\|\_{\infty} T} \mathbf{e}^{\|\boldsymbol{a}\|\_{\infty}} \|\boldsymbol{a}\|\_{\infty}}\_{\mathsf{T}\_{\boldsymbol{r}, \|\boldsymbol{a}\|\infty, \|b\|\_{\infty}}} \Upsilon. \tag{16}$$

By combining (15) and (16) and by the triangular inequality,

$$\begin{aligned} \left\| \left| \mathbf{e}\_{\mathbf{r}}^{b\_1 t - \tau - s} - \mathbf{e}^{b(t - s)} \right| \right\|\_{\infty} &\leq \left\| \mathbf{e}\_{\mathbf{r}}^{b\_1 t - \tau - s} - \mathbf{e}^{b\_1(t - s)} \right\|\_{\infty} \\ &+ \left\| \mathbf{e}^{b\_1(t - s)} - \mathbf{e}^{b(t - s)} \right\|\_{\infty} \leq \left( \mathbf{C}\_{T, k} + \overline{\mathbf{C}}\_{T, \|\mathbf{z}\|\_{\infty}, \|b\|\_{\infty}} \right) \tau. \end{aligned}$$

Substituting this inequality into (14),

$$||z\_{\mathsf{T}}(t) - z\_0(t)||\_{\mathcal{P}} \le \mathsf{C}\_f \mathbf{e}^{||a||\_{\infty}T} \left(\mathsf{C}\_{T,k} + \overline{\mathsf{C}}\_{T, \|a\|\_{\infty}, \|b\|\_{\infty}}\right) \tag{7} \stackrel{\mathsf{r} \to \mathsf{O}}{\longrightarrow} \mathsf{O}\_{\mathsf{r}}$$

uniformly on [0, *T*].

**Theorem 13.** *Fix* 1 ≤ *p* < ∞*. Let a and b be bounded random variables, let g be a stochastic process that belongs to <sup>C</sup>*1([−*τ*, 0]) *in the* <sup>L</sup>*p-sense, and let <sup>f</sup> be a continuous stochastic process on* [0, <sup>∞</sup>) *in the* <sup>L</sup>*p-sense. Then,* lim*τ*→<sup>0</sup> *<sup>x</sup>τ*(*t*) = *<sup>x</sup>*0(*t*) *in* <sup>L</sup>*p, uniformly on* [0, *<sup>T</sup>*]*, for each T* <sup>&</sup>gt; <sup>0</sup>*.*

**Proof.** This is a consequence of Theorem 11 and Theorem 12, with *xτ*(*t*) = *yτ*(*t*) + *zτ*(*t*) and *x*0(*t*) = *y*0(*t*) + *z*0(*t*).

**Example 1.** This is a test example, with arbitrary distributions, to show how (9) and (10) may be employed to compute the expectation and the variance of the stochastic solution. Theoretical results are also illustrated. Let *a* ∼ Beta(2, 3) and *b* ∼ Uniform(0.2, 1). Define *g*(*t*, *ω*) = sin(sin(*d*(*ω*)*t* <sup>2</sup>)) and *<sup>f</sup>*(*t*, *<sup>ω</sup>*) = cos(*te*(*ω*)2), where *<sup>d</sup>* and *<sup>e</sup>* are random variables with *<sup>d</sup>* <sup>∼</sup> Triangular(1, 1.15, 1.3) and *e* ∼ Uniform(0.1, 0.2). By using the chain rule theorem, Proposition 1, it is easy to prove that both *<sup>g</sup>* and *<sup>f</sup>* are *<sup>C</sup>*<sup>∞</sup> in the L*p*-sense, 1 <sup>≤</sup> *<sup>p</sup>* <sup>&</sup>lt; <sup>∞</sup>. The random variables *<sup>a</sup>*, *<sup>b</sup>*, *<sup>d</sup>* and *<sup>e</sup>* are assumed to be independent. Consider the solution stochastic process *xτ*(*t*) defined by (2). It is an L*p*-solution for all <sup>1</sup> <sup>≤</sup> *<sup>p</sup>* <sup>&</sup>lt; <sup>∞</sup>, by Theorem 7. With expressions (9) and (10), we can compute <sup>E</sup>[*xτ*(*t*)] and <sup>V</sup>[*xτ*(*t*)]; see Figure 1. The results agree with Monte Carlo simulation on (1). Observe that, as *τ* approaches 0, the solution stochastic process tends to the solution with no delay defined in Theorem 10, as predicted by Theorem 13.

**Figure 1.** Expectation (up) and variance (down) of *xτ*(*t*), Example 1.

**Example 2.** In this example, we specify new probability distributions for the input coefficients. Let *a* ∼ Uniform(0.2, 1), *b* ∼ Uniform(−1, 0), *d* ∼ Beta(1, 1.3) and *e* ∼ Uniform(−0.2, −0.1), all of them independent. The stochastic process *<sup>x</sup>τ*(*t*) given by (2) is an L*p*-solution for all 1 <sup>≤</sup> *<sup>p</sup>* <sup>&</sup>lt; <sup>∞</sup>, by Theorem 7. We compute E[*xτ*(*t*)] and V[*xτ*(*t*)] with (9) and (10), see Figure 2. Observe that the convergence when *τ* → 0 agrees with Theorem 13.

**Figure 2.** Expectation (up) and variance (down) of *xτ*(*t*), Example 2.

We now comment on some computational aspects. We have used the software Mathematica®, version 11.2 [37]. The integrals and expectations from (9) and (10) have been computed as multidimensional integrals with the built-in function NIntegrate (recall that the expectation is an integral with respect to the corresponding probability density function). Expression (9) does not pose serious numerical challenges, and one can use a standard NIntegrate routine with no specified options. However, for expression (10), we have set the option quasi-Monte Carlo with 10<sup>5</sup> sampling points (otherwise the computational time would increase dramatically). We have checked numerically that the following factors increase the computational time: large ratio *t*/*τ*; probability distributions with unbounded support for the input data; and moderate or large dimensions of the random space.
