Implementation of a Parallel Algorithm to Simulate the Type I Error Probability

Abstract

Simulating the probability of type I error is a powerful statistical tool that allows confirming if the statistical test achieves the established nominal level. However, its computational implementation has the drawback of significantly long execution times. Therefore, this article analyzes the performance of two parallel implementations (parRapply and boot) which significantly reduce the execution time of simulations of type I error probability for a goodness-of-fit test for the bivariate Poisson distribution. The results obtained demonstrate how the parallelization strategies accelerate the simulations, reducing the time by 50% to 90% when using 2 to 12 processors running in parallel. This reduction is graphically evidenced as the execution time of the analyzed parallel versions fits almost perfectly ($R^2\approx 0.999$) to the power model $y=a{p}^b$, where p is the number of processors used, and $a>0$ and $b<0$ are the constants of the model. Furthermore, it is shown that the parallelization strategies used scale with an increasing number of processors. All algorithms were implemented in the R programming language, and their code is included at the end of this article.

1. Introduction

In today’s world, it is essential to anticipate events that may occur, and to do so, having models that predict such events with the highest possible accuracy is vital. Statistics provides tools for making predictions or decisions based on the information contained in the available data sample. Typically, these tools consist of models based on probability distributions. However, before using such models, it is crucial to determine whether the sample data behave according to the probability distribution on which the probabilistic model is based. This is achieved by applying a goodness-of-fit test.

Regarding the latter, it is necessary to specify that an essential part in the development of a goodness-of-fit test is the result of simulation under the null hypothesis, which confirms if the test reaches the established nominal level. In [1], two goodness-of-fit tests were generated, and the parametric bootstrap method was used to simulate the type I error probability. However, the excessive execution time of each simulation influenced the authors to make a few variations in both the parameter vector $\theta$ and the sample sizes n considered.

In this regard, to explore more and new possible scenarios, it is fundamental and necessary to reduce the computational execution time of the involved simulation processes. Many areas pursue this goal and employ parallel programming for it, for instance in [2,3,4,5,6,7].

Therefore, the interest of this research is to use parallelization strategies to accelerate the simulation processes employed in the first of the statistical tests developed in [1]. This will allow examining new parameters $\theta$ and testing with larger sample sizes n. It will contribute to reducing computation time and confirming that as sample sizes grow, parameter estimates converge to their nominal value.

This article is organized as follows: Section 2 presents the theoretical aspects and details the first test proposed in [1]. Section 3 shows the fundamental aspects to take into account to implement a parallel algorithm and are discussed the facilities offered by the R language to make implementations of this type. Section 4 presents the experimental results. In Section 5, the obtained results are discussed, and Section 6 exhibits the conclusions.

2. Background

From here on, the following notation is considered:

• ${\mathbb{N}}_0=\{0,1,2,3,\dots \}$.
• $\mathsf{\Theta }=\{\theta =({\theta }_1,{\theta }_2,{\theta }_3)\in {\mathbb{R}}^3:{\theta }_1>{\theta }_3,{\theta }_2>{\theta }_3,{\theta }_3>0\}$.

2.1. Bivariate Poisson Distribution

There have been several definitions for the bivariate Poisson distribution. This article utilizes the data provided in [8], which is the one that has received the most attention in the statistical literature.

Definition 1.

  Let

$$X_1=Y_1+Y_{3}and{X}_2=Y_2+Y_3,$$

where $Y_1,Y_2$ and $Y_3$ are mutually independent Poisson random variables with means given by ${\theta }_1^{\prime }={\theta }_1-{\theta }_3>0$, ${\theta }_2^{\prime }={\theta }_2-{\theta }_3>0$ and ${\theta }_3\ge 0$, respectively.

The joint distribution of the vector $(X_1,X_2)$ is called bivariate Poisson distribution with parameter $\theta =({\theta }_1,{\theta }_2,{\theta }_3)$, which will be denoted by $(X_1,X_2)\sim BP\left(\theta \right)$.

The joint probability mass function of $X_1$ and $X_2$ is given by

$$P_{\theta }(X_1=x_1,X_2=x_2)=exp({\theta }_3-{\theta }_1-{\theta }_2)\sum _{i=0}^{min\{x_1,x_2\}}\frac{{({\theta }_1-{\theta }_3)}^{x_1-i}{({\theta }_2-{\theta }_3)}^{x_2-i}{\theta }_3^i}{(x_1-i)!(x_2-i)!i!},$$

where $x_1,x_2\in {\mathbb{N}}_0$.

Also, the joint probability generating function (pgf) of $X_1$ and $X_2$ is

$$g(u;\theta )=exp\{{\theta }_1(u_1-1)+{\theta }_2(u_2-1)+{\theta }_3(u_1-1)(u_2-1)\},$$

$\forall u=(u_1,u_2)\in {\mathbb{R}}^2$, $\forall \theta \in \mathsf{\Theta }$.

2.2. Goodness-of-Fit Test of Novoa-Muñoz and Jiménez-Gamero (2014)

This subsection shows, in summary, the first test proposed by these authors, hereinafter, test Q.

To do this, let ${\mathit{X}}_1=(X_{11},X_{21}),\dots ,{\mathit{X}}_n=(X_{1n},X_{2n})$ be random vectors independent and identically distributed of $\mathit{X}=(X_1,X_2)\in {\mathbb{N}}_0^2$.

The hypotheses are contrasted

$$\begin{array}{cc}\hfill H_0& :(X_1,X_2)\sim BP({\theta }_1,{\theta }_2,{\theta }_3),\mathrm{for}\mathrm{some}({\theta }_1,{\theta }_2,{\theta }_3)\in \mathsf{\Theta },\hfill \\ \hfill H_1& :(X_1,X_2)\nsim BP({\theta }_1,{\theta }_2,{\theta }_3),\forall ({\theta }_1,{\theta }_2,{\theta }_3)\in \mathsf{\Theta }.\hfill \end{array}$$

According to the definition established in [9], if $\theta \in \mathsf{\Theta }$ but the hypothesis test incorrectly decides to reject $H_0$, then the test has made a type I error.

The test statistic is of the Cramer Von Misses type and is given by

$$Q_{n,w}\left({\widehat{\theta }}_n\right)=n{\int }_0^1{\int }_0^1{\left\{g_n\left(u\right)-g(u;{\widehat{\theta }}_n)\right\}}^{2}w\left(u\right)du,$$

where a consistent estimator ${\widehat{\theta }}_n={\widehat{\theta }}_n({\mathit{X}}_1,{\mathit{X}}_2,\dots ,{\mathit{X}}_n)$ is a consistent estimator of $\theta$, $g(u;{\widehat{\theta }}_n)$ is the pgf and $g_n\left(u\right)$ its empirical version, $w\left(u\right)=u_1^{a_1}{u}_2^{a_2}$ is the weight function, in which $u\in {[0,1]}^2$ and $(a_1,a_2)$, called the weight vector, is such that $a_1>-1,a_2>-1$.

The null hypothesis will be rejected for “large” values of $Q_{n,w}\left({\widehat{\theta }}_n\right)$.

Since the null distribution of the statistic turned out to be unknown, the authors estimated it using the asymptotic null distribution, but since it depends on the true value of the $\theta$ parameter, which is unknown, it did not provide them will a useful solution and decided to estimate the null distribution of the statistic using the parametric bootstrap method, which is analyzed below.

2.3. Parametric Bootstrap

In the case under study, the distribution of the statistic depends on the parameter; then, such a model is called parametric, and therefore, the statistical methods based in this model are parametric methods [10].

As in the real world, a sample from a given population is obtained, and with this sample, the population parameter is estimated, which is expected to be an interior point of the parameter space. The parametric bootstrap method is a resampling technique that emulates the situation described above; that is, with the parameter estimator, a Bivariate Poisson is generated from which a sample is drawn, called the bootstrap sample, and with it the parameter is estimated, which is named the bootstrap parameter. With the sample and the bootstrap parameter, the statistic $Q^{*}$ is computed, which is the bootstrap version of the statistic Q.

Although the bootstrap method has been highly useful in statistical inference, [11] shows that there are situations in which it is not consistent. In the said article, examples are shown where parametric bootstrap does not yield good results, and particularly, it is demonstrated that the method is inconsistent when the true parameter value is on the boundary of the parameter space or very close to it.

3. A Parallel Implementation

There are two parameters to analyze the efficiency of an algorithm which solves a given problem: the memory space required to store the input data and results and the time of execution used in performing the task. Several technological advances allow the use of many computers that collaborate each other either to increase the size of problems to be solved through data distribution or task distribution [12]. In this latter case, parallel programs are being used. In the case of bootstrap resampling, it involves task distribution, as the iterations are independent of each other.

3.1. Parallelization in R Language

The R programming language offers the possibility of developing tasks in parallel, when they allow it, through a series of packages that provide commands for this type of processing. Among the packages that help accomplish this task, Snow, Multicore, and Parallel can be mentioned.

Currently, one of the packages that has gained great prominence in the programming in R language is Parallel [13], which takes the best of two previous versions Multicore and Snow, which is complemented by the random numbers generation [14,15].

The parApply package allows all the functions contained in Apply, such as the direct data manipulation from each vector data, matrix, list or data frames avoiding the use of cycles. It also allows finding out how many processors are available.

Thus, the parallel package allows distributing the data in the different cores available and there, simultaneously, manipulating them according to the indicated function, and then gathering all the information and returning it as a list [16].

In general terms, two ways of implementing parallelism using this package can be distinguished: through forking or sockets.

The forking method is based on the full duplication of a master process with a shared environment toward each of the parallel environments, including objects or variables defined before the start of parallel threads. This method has the advantage of being very fast, but it cannot run under the Windows operating system.

On the other hand, the sockets method is based where each thread runs separately without sharing objects or variables, which can only be passed from the master process explicitly. As a result, it runs slower due to communication overhead, but it has the advantage of being able to be implemented in any operating system [16].

Just as the Parallel package offers tools that allow parallelization in R, there is a Boot package [13] that incorporates functions that facilitate the application of the bootstrap method. For this package, we consider the possibility of performing the operations in parallel using forking or a socket.

When programming in parallel in R language [13], with the aim of reducing computation times, it is important to take into account some experts’ recommendations. In many cases, simply “quite fast” is pretty good. The additional speed that could be achieved by transitioning from R to C/C++ does not justify the significant amount of time required to write, debug, and maintain code at that level [17].

3.2. Performance Evaluation of a Parallel Program

According to [18], one way to measure the performance of a parallel program is to calculate its Speedup (S), which is the ratio between the execution time of the program in the sequential version ($T_s$) and the execution time of the parallel version ($T_p$) with p processors, which is given by $S=\frac{T_s}{T_p}$.

Furthermore, as the processors used increase, there is more additional work (and time) than expected; that is, the ratio between S and p decreases in value. Thus, the Efficiency of a parallel program is defined as $E=\frac{S}{p}$.

3.3. Methodology

To showcase the results of this work, three aspects are considered:

(a)

Preparation of the programs.

Once it was verified that the process was parallelizable, the parts of the program that could be modified to reduce computation times were identified; then, the instructions of the R language that allow for parallelizing the task were analyzed, finding parRapply from the package Parallel, and boot from the Boot package. Then, the new codes that allow for parallelizing the simulation process were developed.

(b)

Simulation with the same components used in [1]. The simulations made by these authors were replicated considering the execution sequential and parallel code. For the parallel program, the number of cores p, using $p=2,3,\dots ,12$.

(c)

Simulations with new components.

-

New weight vectors $(a_1,a_2)\in \{(1,0),(1,1)\}$, with $p=1,\dots ,12$.

-

Sample sizes $n=100\left(50\right)300,500$ with the same original population parameters and $p=10$.

-

Population parameters $\theta$, with ${\theta }_1=1,1.5$, ${\theta }_2=1$ and ${\theta }_3$ chosen so that the correlation coefficient $\rho =\frac{{\theta }_3}{\sqrt{{\theta }_1{\theta }_2}}$ is less than 0.25 or greater than 0.75, $n=30\left(20\right)70,100$$\left(50\right)300,500$ and $p=10$.

The simulations described in items (b) and (c) were carried out in a cluster of computers with 12 processors with a processing speed of 2.00 GHz, 32 GB of RAM and operating system Centos 7 (Linux).

4. Experimental Results

This section presents the results of the simulations of type I error considering the three parts detailed in  Section 3.3.

4.1. Simulations of Probability of a type I error with the Components Used in the First Test of Novoa-Muñoz and Jiménez-Gamero (2014)

Firstly, it was verified that when using parallel programming, there was no significant difference between the simulated type I error probability and its respective nominal value. Due to space, the results are not presented in this paper and can be obtained from the author upon request.

The next step was to analyze the execution times and the efficiency of the two parallel versions (parRapply and boot commands). For this, it is essential to know the sequential execution time, which is presented in 

Table 1. Average execution time (in seconds) using Sequential Version, weight vectors $(a_1,a_2)$, sample sizes n and parameters $\theta$.

	n = 30				n = 50				n = 70
$\mathsf{\theta }$	$(\mathbf{0},\mathbf{0})$	$(\mathbf{0},\mathbf{1})$	$(\mathbf{1},\mathbf{0})$	$(\mathbf{1},\mathbf{1})$	$(\mathbf{0},\mathbf{0})$	$(\mathbf{0},\mathbf{1})$	$(\mathbf{1},\mathbf{0})$	$(\mathbf{1},\mathbf{1})$	$(\mathbf{0},\mathbf{0})$	$(\mathbf{0},\mathbf{1})$	$(\mathbf{1},\mathbf{0})$	$(\mathbf{1},\mathbf{1})$
(1.0, 1, 0.75)	6851	6881	6892	6912	7558	7432	7657	7461	8108	8054	7952	8019
(1.0, 1, 0.50)	6923	7114	7006	6985	7554	7556	7513	7511	8008	8086	8055	8109
(1.0, 1, 0.25)	7451	7257	7247	7317	7856	7795	7854	7778	8264	8359	8270	8220
(1.5, 1, 0.62)	7937	7895	7969	7878	8630	8624	8495	8584	9174	9097	9382	9210
(1.5, 1, 0.92)	7995	7976	8051	8004	8804	8886	8819	8713	9398	9287	9270	9446
(1.5, 1, 0.31)	8209	8307	8175	8146	8907	8925	8913	8914	9427	9451	9398	9400

and

Table 2. Average execution time (in seconds) using boot Sequential Version, weight vectors $(a_1,a_2)$, sample sizes n and parameters $\theta$.

	n = 30				n = 50				n = 70
$\mathsf{\theta }$	$(\mathbf{0},\mathbf{0})$	$(\mathbf{0},\mathbf{1})$	$(\mathbf{1},\mathbf{0})$	$(\mathbf{1},\mathbf{1})$	$(\mathbf{0},\mathbf{0})$	$(\mathbf{0},\mathbf{1})$	$(\mathbf{1},\mathbf{0})$	$(\mathbf{1},\mathbf{1})$	$(\mathbf{0},\mathbf{0})$	$(\mathbf{0},\mathbf{1})$	$(\mathbf{1},\mathbf{0})$	$(\mathbf{1},\mathbf{1})$
(1.0, 1, 0.75)	6885	6868	6907	6962	7720	7688	7658	7691	8207	8164	8231	8243
(1.0, 1, 0.50)	7073	7042	7070	7059	7752	7687	7678	7690	8256	8239	8267	8237
(1.0, 1, 0.25)	7526	7536	7472	7485	7905	7960	8013	7993	8468	8380	8429	8385
(1.5, 1, 0.62)	7977	7926	7933	7952	8823	8789	8768	8782	9415	9403	9500	9391
(1.5, 1, 0.92)	7910	7865	7885	7889	8765	8812	8776	8817	9441	9362	9393	9492
(1.5, 1, 0.31)	8313	8335	8326	8343	9028	9138	8991	9029	9653	9553	9573	9587

. Table 1 presents the results using the algorithm implemented in this research (described in Appendix A), and Table 2 records the results of running the R package 4.0.0 boot command. These times serve as a basis for calculating the performance of parallel programming.

The execution times and efficiency of the parallelized programs using the parRapply and boot commands, which were the two parallel versions analyzed in this study, are shown graphically below.

Figure 1, Figure 2, Figure 3, Figure 4, Figure 5 and Figure 6 present the average execution times versus the number of processors used. To obtain these results and for comparison, the same parameters $\theta$ used in [1] were used.

Table A1. Average execution time (in seconds) using parRapply, $\theta =$ (1.5, 1, 0.31), weight vectors $(a_1,a_2)$, sample size n, for different numbers of processors p.

		n = 30				n = 50				n = 70
	$({\mathit{a}}_{\mathbf{1}},{\mathit{a}}_{\mathbf{2}})$	$(\mathbf{0},\mathbf{0})$	$(\mathbf{0},\mathbf{1})$	$(\mathbf{1},\mathbf{0})$	$(\mathbf{1},\mathbf{1})$	$(\mathbf{0},\mathbf{0})$	$(\mathbf{0},\mathbf{1})$	$(\mathbf{1},\mathbf{0})$	$(\mathbf{1},\mathbf{1})$	$(\mathbf{0},\mathbf{0})$	$(\mathbf{0},\mathbf{1})$	$(\mathbf{1},\mathbf{0})$	$(\mathbf{1},\mathbf{1})$
p
2		4049	4058	4063	4088	4402	4428	4413	4440	4657	4662	4663	4,674
3		2764	2762	2801	2793	2959	2979	2970	3006	3164	3142	3164	3162
4		2118	2100	2092	2112	2261	2259	2255	2248	2406	2399	2389	2404
5		1726	1761	1713	1724	1868	1888	1874	1870	2004	1970	1988	1,969
6		1445	1457	1444	1461	1575	1551	1601	1567	1679	1658	1640	1655
7		1242	1246	1257	1263	1365	1350	1361	1356	1432	1457	1420	1447
8		1102	1125	1123	1090	1203	1193	1191	1196	1288	1261	1274	1264
9		1007	1007	1012	1012	1105	1099	1112	1097	1158	1162	1160	1171
10		919	916	914	918	988	981	976	986	1044	1039	1055	1037
11		841	840	835	845	905	931	913	933	963	975	960	981
12		764	765	764	776	823	820	820	823	869	875	882	877

–

Table A12. Average execution time (in seconds) using boot, $\theta =$ (1, 1, 0.75), weight vectors $(a_1,a_2)$, sample size n, for different numbers of processors p.

		n = 30				n = 50				n = 70
	$({\mathit{a}}_{\mathbf{1}},{\mathit{a}}_{\mathbf{2}})$	$(\mathbf{0},\mathbf{0})$	$(\mathbf{0},\mathbf{1})$	$(\mathbf{1},\mathbf{0})$	$(\mathbf{1},\mathbf{1})$	$(\mathbf{0},\mathbf{0})$	$(\mathbf{0},\mathbf{1})$	$(\mathbf{1},\mathbf{0})$	$(\mathbf{1},\mathbf{1})$	$(\mathbf{0},\mathbf{0})$	$(\mathbf{0},\mathbf{1})$	$(\mathbf{1},\mathbf{0})$	$(\mathbf{1},\mathbf{1})$
p
2		3582	3601	3601	3596	3957	3978	3983	3966	4226	4631	4244	4254
3		2533	2524	2539	2532	2789	2790	2795	2783	2963	2957	2966	2972
4		1904	1900	1905	1900	2105	2099	2095	2088	2228	2335	2229	2241
5		1625	1623	1621	1622	1791	1791	1789	1785	1901	1904	1898	1897
6		1358	1356	1353	1351	1486	1492	1491	1484	1588	1589	1585	1585
7		1220	1219	1218	1218	1346	1347	1345	1347	1428	1432	1427	1427
8		1053	1050	1049	1052	1169	1166	1166	1161	1245	1249	1249	1247
9		976	978	978	979	1086	1087	1087	1084	1169	1170	1170	1169
10		895	895	894	896	983	984	985	982	1048	1050	1049	1051
11		843	844	843	846	927	925	925	924	984	986	986	985
12		782	777	775	784	856	849	860	857	903	911	914	906

, displayed in Appendix B, contain the data with which these graphs were constructed.

Figure 1, Figure 2 and Figure 3 show the execution times when $E\left(X_1\right)=E\left(X_2\right)$ and Figure 4, Figure 5 and Figure 6 show the times when $E\left(X_1\right)\ne E\left(X_2\right)$.

Since in this research two parallel implementations are analyzed, the interest lies in knowing which of them is more efficient; therefore, the efficiencies of both parallel implementations were calculated. Figure 7, Figure 8, Figure 9, Figure 10, Figure 11 and Figure 12 display the efficiencies, E, versus the number of processors used. Each figure shows graphically the efficiency of both commands (parRapply and boot) for the same parameter $\theta$. These parameters are the same ones used in [1].

Tables with these data are not shown, as they can be obtained directly using Table 1 and Table 2 (sequential time) together with Table A1–Table A12 in Appendix B (parallel time).

4.2. Simulations of Type I Error Probability with Sample Sizes Larger than Those Used in the First Test of Novoa-Muñoz and Jiménez-Gamero (2014)

For simulations with sample sizes greater than $n=$ 70, only the command parRapply was used because, as seen in the previous subsection, it is more efficient than the command boot. Furthermore, only ten processors were used.

The estimations of the type I error probabilities for the nominal values $\alpha$ = 0.01, 0.05, and 0.10 are presented in 

Table 3. Results of simulating probability of type I error. Instance $\theta =(1,1,0.75),\rho =0.75$.

	${\mathit{a}}_1={\mathit{a}}_2=0$			${\mathit{a}}_1=0$, ${\mathit{a}}_2=1$			${\mathit{a}}_1=1$, ${\mathit{a}}_2=0$			${\mathit{a}}_1={\mathit{a}}_2=1$
$\mathit{n}$	1%	5%	10%	1%	5%	10%	1%	5%	10%	1%	5%	10%
100	0.009	0.039	0.086	0.013	0.042	0.088	0.007	0.044	0.102	0.010	0.040	0.093
150	0.013	0.054	0.111	0.009	0.048	0.107	0.010	0.054	0.109	0.010	0.052	0.114
200	0.013	0.053	0.112	0.014	0.058	0.108	0.014	0.056	0.115	0.015	0.060	0.113
250	0.016	0.060	0.109	0.014	0.065	0.112	0.014	0.057	0.115	0.014	0.060	0.108
300	0.013	0.051	0.101	0.013	0.050	0.089	0.014	0.052	0.100	0.011	0.048	0.101
500	0.010	0.048	0.099	0.009	0.050	0.098	0.011	0.047	0.099	0.010	0.049	0.099

,

Table 4. Results of simulating probability of type I error. Instance $\theta =(1,1,0.5),\rho =0.5$.

	${\mathit{a}}_1={\mathit{a}}_2=0$			${\mathit{a}}_1=0$, ${\mathit{a}}_2=1$			${\mathit{a}}_1=1$, ${\mathit{a}}_2=0$			${\mathit{a}}_1={\mathit{a}}_2=1$
$\mathit{n}$	1%	5%	10%	1%	5%	10%	1%	5%	10%	1%	5%	10%
100	0.009	0.054	0.092	0.009	0.053	0.099	0.015	0.048	0.087	0.009	0.052	0.085
150	0.017	0.051	0.109	0.009	0.055	0.107	0.017	0.061	0.108	0.012	0.054	0.096
200	0.013	0.068	0.116	0.011	0.055	0.115	0.018	0.070	0.114	0.013	0.064	0.114
250	0.009	0.047	0.095	0.009	0.050	0.093	0.014	0.038	0.096	0.010	0.050	0.100
300	0.018	0.048	0.098	0.012	0.048	0.099	0.022	0.049	0.095	0.015	0.050	0.092
500	0.011	0.051	0.102	0.010	0.052	0.100	0.011	0.049	0.099	0.011	0.056	0.100

,

Table 5. Results of simulating probability of type I Error. Instance $\theta =(1,1,0.25),\rho =0.25$.

	${\mathit{a}}_1={\mathit{a}}_2=0$			${\mathit{a}}_1=0$, ${\mathit{a}}_2=1$			${\mathit{a}}_1=1$, ${\mathit{a}}_2=0$			${\mathit{a}}_1={\mathit{a}}_2=1$
$\mathit{n}$	1%	5%	10%	1%	5%	10%	1%	5%	10%	1%	5%	10%
100	0.006	0.052	0.100	0.008	0.051	0.109	0.011	0.042	0.105	0.009	0.046	0.096
150	0.011	0.047	0.097	0.012	0.052	0.101	0.007	0.047	0.099	0.008	0.043	0.093
200	0.017	0.057	0.090	0.011	0.044	0.091	0.018	0.065	0.110	0.018	0.053	0.102
250	0.013	0.056	0.108	0.009	0.051	0.097	0.013	0.068	0.124	0.012	0.056	0.115
300	0.010	0.057	0.113	0.013	0.056	0.108	0.011	0.049	0.106	0.011	0.057	0.111
500	0.009	0.049	0.100	0.011	0.051	0.101	0.009	0.049	0.099	0.010	0.048	0.100

,

Table 6. Results of simulating probability of type I error. Instance $\theta =(1.5,1,0.62),\rho \approx 0.5$.

	${\mathit{a}}_1={\mathit{a}}_2=0$			${\mathit{a}}_1=0$, ${\mathit{a}}_2=1$			${\mathit{a}}_1=1$, ${\mathit{a}}_2=0$			${\mathit{a}}_1={\mathit{a}}_2=1$
$\mathit{n}$	1%	5%	10%	1%	5%	10%	1%	5%	10%	1%	5%	10%
100	0.010	0.060	0.116	0.012	0.054	0.115	0.010	0.061	0.108	0.009	0.054	0.111
150	0.012	0.063	0.129	0.015	0.068	0.130	0.014	0.067	0.122	0.011	0.067	0.129
200	0.007	0.038	0.080	0.009	0.039	0.093	0.004	0.037	0.094	0.006	0.040	0.085
250	0.008	0.046	0.094	0.009	0.049	0.103	0.007	0.042	0.089	0.009	0.041	0.094
300	0.016	0.058	0.103	0.014	0.059	0.108	0.013	0.055	0.100	0.010	0.052	0.101
500	0.009	0.048	0.099	0.011	0.048	0.098	0.009	0.049	0.097	0.010	0.049	0.099

,

Table 7. Results of simulating probability of type I error. Instance $\theta =(1.5,1,0.92),\rho \approx 0.75$.

	${\mathit{a}}_1={\mathit{a}}_2=0$			${\mathit{a}}_1=0$, ${\mathit{a}}_2=1$			${\mathit{a}}_1=1$, ${\mathit{a}}_2=0$			${\mathit{a}}_1={\mathit{a}}_2=1$
$\mathit{n}$	1%	5%	10%	1%	5%	10%	1%	5%	10%	1%	5%	10%
100	0.007	0.039	0.088	0.008	0.037	0.090	0.009	0.043	0.094	0.006	0.035	0.091
150	0.011	0.066	0.121	0.012	0.054	0.131	0.011	0.067	0.123	0.011	0.056	0.118
200	0.017	0.060	0.110	0.021	0.052	0.101	0.016	0.057	0.110	0.018	0.057	0.106
250	0.019	0.053	0.111	0.021	0.059	0.111	0.011	0.054	0.098	0.022	0.055	0.098
300	0.009	0.054	0.111	0.013	0.053	0.110	0.011	0.060	0.103	0.012	0.052	0.101
500	0.011	0.051	0.099	0.011	0.051	0.101	0.011	0.051	0.101	0.011	0.051	0.101

and

Table 8. Results of simulating probability of type I error. Instance $\theta =(1.5,1,0.31),\rho \approx 0.25$.

	${\mathit{a}}_1={\mathit{a}}_2=0$			${\mathit{a}}_1=0$, ${\mathit{a}}_2=1$			${\mathit{a}}_1=1$, ${\mathit{a}}_2=0$			${\mathit{a}}_1={\mathit{a}}_2=1$
$\mathit{n}$	1%	5%	10%	1%	5%	10%	1%	5%	10%	1%	5%	10%
100	0.010	0.044	0.099	0.014	0.053	0.104	0.008	0.049	0.089	0.009	0.048	0.104
150	0.010	0.050	0.098	0.006	0.053	0.097	0.013	0.057	0.100	0.004	0.048	0.098
200	0.009	0.036	0.094	0.008	0.043	0.090	0.007	0.051	0.099	0.008	0.043	0.090
250	0.017	0.066	0.112	0.019	0.064	0.121	0.015	0.056	0.105	0.018	0.062	0.114
300	0.014	0.055	0.091	0.014	0.046	0.088	0.014	0.060	0.112	0.015	0.048	0.094
500	0.011	0.051	0.101	0.011	0.051	0.102	0.011	0.049	0.101	0.011	0.051	0.101

with 1%, 5% and 10%, respectively.

Figure 13 and Figure 14 present the average execution times versus different sample sizes (n) when the simulations displayed in Table 3, Table 4, Table 5, Table 6, Table 7 and Table 8 are executed.

4.3. Simulations of the Probability of a Type I Error with Parameter Vectors Different from Those Used in the First Test of Novoa-Muñoz and Jiménez-Gamero (2014)

The simulation of type I errors implemented in parallel allows to evaluate the performance of the Q test when working with vectors that have parameters different to those already studied. In particular, it allows simulating the type I error probability for the cases in which the correlation coefficient $\rho$ takes very small (≤ 0.1) or very large (≥ 0.8) values. The first of these cases takes on special relevance, since it involves parameters that are very close to the boundary of the parametric space, which according to [11] could lead to inconsistencies while increasing computation times.

For the same reasons given in Section 4.2, only the command parRapply and ten processors were used.

Table 9. Results of simulating probability of type I error. Instance $\theta =(1,1,0.1),\rho =0.1$.

	${\mathit{a}}_1={\mathit{a}}_2=0$			${\mathit{a}}_1=0$, ${\mathit{a}}_2=1$			${\mathit{a}}_1=1$, ${\mathit{a}}_2=0$			${\mathit{a}}_1={\mathit{a}}_2=1$
$\mathit{n}$	1%	5%	10%	1%	5%	10%	1%	5%	10%	1%	5%	10%
30	0.052	0.141	0.210	0.038	0.117	0.179	0.032	0.115	0.198	0.037	0.121	0.201
50	0.079	0.180	0.263	0.053	0.143	0.222	0.057	0.158	0.225	0.060	0.160	0.243
70	0.117	0.235	0.305	0.082	0.192	0.269	0.083	0.181	0.270	0.087	0.199	0.276
100	0.175	0.281	0.334	0.128	0.243	0.306	0.127	0.249	0.305	0.141	0.259	0.309
150	0.217	0.287	0.317	0.166	0.261	0.306	0.168	0.270	0.305	0.188	0.272	0.309
200	0.258	0.290	0.318	0.214	0.276	0.309	0.219	0.277	0.305	0.231	0.277	0.310

,

Table 10. Results of simulating probability of type I error. Instance $\theta =(1,1,0.2),\rho =0.2$.

	${\mathit{a}}_1={\mathit{a}}_2=0$			${\mathit{a}}_1=0$, ${\mathit{a}}_2=1$			${\mathit{a}}_1=1$, ${\mathit{a}}_2=0$			${\mathit{a}}_1={\mathit{a}}_2=1$
$\mathit{n}$	1%	5%	10%	1%	5%	10%	1%	5%	10%	1%	5%	10%
30	0.014	0.045	0.081	0.009	0.044	0.068	0.013	0.044	0.081	0.011	0.039	0.076
50	0.009	0.041	0.086	0.010	0.039	0.074	0.013	0.045	0.077	0.009	0.039	0.074
70	0.029	0.063	0.101	0.017	0.061	0.107	0.016	0.057	0.102	0.016	0.062	0.100
100	0.018	0.054	0.083	0.015	0.042	0.085	0.014	0.049	0.087	0.016	0.044	0.082
150	0.031	0.058	0.085	0.021	0.056	0.095	0.020	0.052	0.086	0.021	0.054	0.091
200	0.018	0.049	0.077	0.020	0.046	0.085	0.016	0.043	0.075	0.017	0.046	0.075

,

Table 11. Results of simulating probability of type I error. Instance $\theta =(1,1,0.8),\rho =0.8$.

	${\mathit{a}}_1={\mathit{a}}_2=0$			${\mathit{a}}_1=0$, ${\mathit{a}}_2=1$			${\mathit{a}}_1=1$, ${\mathit{a}}_2=0$			${\mathit{a}}_1={\mathit{a}}_2=1$
$\mathit{n}$	1%	5%	10%	1%	5%	10%	1%	5%	10%	1%	5%	10%
30	0.011	0.052	0.114	0.009	0.052	0.100	0.008	0.056	0.112	0.009	0.048	0.113
50	0.012	0.058	0.109	0.012	0.061	0.114	0.014	0.059	0.105	0.008	0.061	0.109
70	0.010	0.055	0.108	0.013	0.550	0.114	0.008	0.050	0.110	0.008	0.053	0.109
100	0.050	0.051	0.122	0.012	0.058	0.118	0.018	0.052	0.112	0.014	0.051	0.119
150	0.006	0.050	0.108	0.008	0.051	0.108	0.008	0.056	0.105	0.008	0.056	0.104
200	0.014	0.041	0.098	0.013	0.043	0.099	0.014	0.045	0.096	0.014	0.045	0.097

,

Table 12. Results of simulating probability of type I error. Instance $\theta =(1,1,0.9),\rho =0.9$.

	${\mathit{a}}_1={\mathit{a}}_2=0$			${\mathit{a}}_1=0$, ${\mathit{a}}_2=1$			${\mathit{a}}_1=1$, ${\mathit{a}}_2=0$			${\mathit{a}}_1={\mathit{a}}_2=1$
$\mathit{n}$	1%	5%	10%	1%	5%	10%	1%	5%	10%	1%	5%	10%
30	0.018	0.063	0.114	0.020	0.068	0.121	0.021	0.065	0.120	0.020	0.067	0.122
50	0.009	0.054	0.098	0.011	0.056	0.096	0.011	0.056	0.109	0.008	0.056	0.103
70	0.013	0.059	0.114	0.017	0.062	0.107	0.014	0.055	0.112	0.016	0.059	0.117
100	0.014	0.049	0.107	0.015	0.048	0.108	0.013	0.054	0.109	0.012	0.055	0.109
150	0.014	0.058	0.112	0.016	0.059	0.109	0.012	0.056	0.107	0.013	0.059	0.104
200	0.009	0.042	0.089	0.009	0.039	0.089	0.012	0.045	0.091	0.009	0.041	0.091

,

Table 13. Results of simulating probability of type I error. Instance $\theta =(1.5,1,0.12),\rho \approx 0.098$.

	${\mathit{a}}_1={\mathit{a}}_2=0$			${\mathit{a}}_1=0$, ${\mathit{a}}_2=1$			${\mathit{a}}_1=1$, ${\mathit{a}}_2=0$			${\mathit{a}}_1={\mathit{a}}_2=1$
$\mathit{n}$	1%	5%	10%	1%	5%	10%	1%	5%	10%	1%	5%	10%
30	0.015	0.091	0.161	0.010	0.072	0.141	0.020	0.080	0.146	0.011	0.086	0.145
50	0.051	0.142	0.219	0.028	0.109	0.175	0.042	0.139	0.219	0.043	0.140	0.211
70	0.069	0.172	0.240	0.035	0.118	0.200	0.052	0.161	0.227	0.059	0.157	0.237
100	0.095	0.183	0.255	0.051	0.135	0.213	0.088	0.184	0.251	0.081	0.177	0.242
150	0.134	0.233	0.279	0.072	0.179	0.260	0.123	0.228	0.281	0.116	0.221	0.278
200	0.192	0.259	0.299	0.129	0.220	0.273	0.169	0.255	0.295	0.168	0.250	0.290

,

Table 14. Results of simulating probability of type I error. Instance $\theta =(1.5,1,0.25),\rho \approx 0.2$.

	${\mathit{a}}_1={\mathit{a}}_2=0$			${\mathit{a}}_1=0$, ${\mathit{a}}_2=1$			${\mathit{a}}_1=1$, ${\mathit{a}}_2=0$			${\mathit{a}}_1={\mathit{a}}_2=1$
$\mathit{n}$	1%	5%	10%	1%	5%	10%	1%	5%	10%	1%	5%	10%
30	0.020	0.062	0.125	0.015	0.067	0.129	0.015	0.065	0.121	0.013	0.059	0.115
50	0.011	0.054	0.099	0.007	0.051	0.097	0.010	0.043	0.091	0.008	0.048	0.103
70	0.020	0.066	0.113	0.013	0.058	0.116	0.017	0.060	0.097	0.016	0.060	0.114
100	0.024	0.059	0.101	0.015	0.054	0.099	0.029	0.064	0.097	0.020	0.056	0.105
150	0.022	0.062	0.098	0.019	0.052	0.081	0.020	0.062	0.105	0.018	0.056	0.093
200	0.013	0.048	0.087	0.012	0.050	0.090	0.013	0.039	0.089	0.013	0.041	0.092

and

Table 15. Results of simulating probability of type I error. Instance $\theta =(1.5,1,0.98),\rho \approx 0.8$.

	${\mathit{a}}_1={\mathit{a}}_2=0$			${\mathit{a}}_1=0$, ${\mathit{a}}_2=1$			${\mathit{a}}_1=1$, ${\mathit{a}}_2=0$			${\mathit{a}}_1={\mathit{a}}_2=1$
$\mathit{n}$	1%	5%	10%	1%	5%	10%	1%	5%	10%	1%	5%	10%
30	0.017	0.070	0.138	0.016	0.073	0.130	0.021	0.068	0.129	0.018	0.072	0.127
50	0.015	0.062	0.119	0.016	0.063	0.119	0.014	0.060	0.108	0.013	0.060	0.115
70	0.012	0.043	0.091	0.013	0.043	0.090	0.010	0.049	0.092	0.010	0.049	0.097
100	0.005	0.050	0.102	0.008	0.056	0.107	0.006	0.041	0.102	0.006	0.046	0.098
150	0.009	0.049	0.106	0.011	0.052	0.098	0.012	0.050	0.096	0.009	0.047	0.102
200	0.010	0.047	0.094	0.007	0.041	0.101	0.009	0.048	0.096	0.008	0.043	0.093

present the results of the type I error simulation for the parameter vectors that meet the characteristics already mentioned, considering the cases $E\left(X_1\right)=E\left(X_2\right)$ and $E\left(X_1\right)\ne E\left(X_2\right)$ with the following sample sizes $n=30,50,70,100,150$, and 200.

Figure 15 and Figure 16 present the average execution times versus different sample sizes (n) when the simulations displayed in Table 9, Table 10, Table 11, Table 12, Table 13, Table 14 and Table 15 are executed.

5. Discussion

From Table 1 and Table 2, it is evident that the sequential version implemented in this research is faster (between 1% and 2%) than the boot version of the R package in almost all parameters $\theta$, except for $\theta =$ (1.5, 1, 0.92). This could be attributed to the sensitivity of the Maximum Likelihood Method, as ${\theta }_2=1$ is very close to ${\theta }_3=0.92$.

On the other hand, regardless of the parallel version (parRapply or boot) used, Figure 1, Figure 2, Figure 3, Figure 4, Figure 5 and Figure 6 highlight the rapid decrease in program execution time as the number of processors increases. Additionally, it is observed that the parRapply command delivers results faster than the boot command. Furthermore, when $E\left(X_1\right)=E\left(X_2\right)$, the execution time is lower than in cases where $E\left(X_1\right)\ne E\left(X_2\right)$. It is observed that the execution time increases as sample sizes grow.

It is also important to emphasize that the power model (faster than the exponential model) fits almost perfectly to the execution time versus number of processors, as approximately 99.9% ($R^2\approx 0.999$) of the variation in execution time is explained by the number of processors used.

Adding to the previous point, Figure 7, Figure 8, Figure 9, Figure 10, Figure 11 and Figure 12 show that the two parallel implementations addressed in this research exhibit linear efficiency. In all analyzed cases, the parRapply command is more efficient than the boot command and with less dispersion among the sample sizes used. Furthermore, it is observed that when two processors are employed, the efficiency of the parRapply command is superior to 1.

In this line of discussion and thanks to this research (using parallel programming), it was possible to obtain results for sample sizes greater than 70, which are displayed in Table 3, Table 4, Table 5, Table 6, Table 7 and Table 8. From these tables, it can be observed that the estimates are close to the nominal values. For example, the estimates at 10% are very close to 0.1. This becomes clearer as the sample size increases, implying that the estimates are converging to the nominal values.

In line with the previous paragraph, Figure 13 and Figure 14 graphically illustrate the positive correlation between the execution time of the programs and the sample size. Clearly, the execution time of the programs is linearly adjusted with the sample size, and this adjustment is almost perfect ($R^2\approx 0.99$). It is also evident that the execution time of the programs is lower when $E\left(X_1\right)=E\left(X_2\right)$.

Finally, from Table 9, Table 10, Table 11, Table 12, Table 13, Table 14 and Table 15, it is possible to appreciate that the estimations of the type I error probability are very close to the nominal values in all cases where the correlation coefficient $\rho >0.1$. However, as could be expected, according to what [11] suggested, in the cases in which ${\theta }_3=0.1(\rho =0.1)$ and ${\theta }_3=0.12$ ($\rho \approx 0.098$), the simulated probabilities are very far from the nominal values and, strangely, the situation worsens as the sample size grows.

In relation to the last paragraph, from Figure 15 and Figure 16, it is appreciated that the execution time of the programs is linearly adjusted with the sample size ($R^2>0.94$) with a positive slope. It is also observed that these times decrease as the correlation coefficient $\rho$ increases, and along this path, the dependence becomes more linear ($R^2$ increases). Additionally, from these graphs, the high dispersion between the lines can be observed, which is due to the large difference between the correlation coefficients $\rho$.

6. Conclusions

In this research, two parallel algorithmic implementations in the R language have been analyzed, which allow simulating the probability of type I error. The results obtained show that the R language is a good alternative to reduce the simulation time of the studied test. On the other hand, the estimates of the type I error probability of the studied test have a value close to the nominal value for both small sample sizes ($n=30,50,70$) and large sample sizes ($n=100\left(50\right)300,500$) as long as the involved correlation coefficient $\rho$ is greater than 0.1

Moreover, the execution time when simulating the type I error probability of the studied test is significantly higher in cases where the correlation coefficient $\rho$ is low ($\rho \approx 0.1$ and $\rho \approx 0.2$) compared to those where this coefficient is high ($\rho \approx 0.8$ and $\rho \approx 0.9$).

The major mathematical finding obtained in this research is that the execution time of the analyzed parallel versions (parRapply and boot) fits almost perfectly ($R^2\approx 0.999$) to the power-law model of the form $y=a{p}^b$, where p is the number of processors used, $a>0$, and $b<0$ are the constants of the model.

The other mathematical result found is that the execution time of the analyzed parallel versions (parRapply and boot) fits the linear model $y=an+b$, where n is the sample size, and $a>0$ is the slope of the line.

As future work, there are plans to deepen the research on the efficiency of the studied test in cases where the correlation coefficient is less than 0.2. Additionally, implementing the algorithm developed in this research in the C programming language is planned. Furthermore, developing a package in the R language that incorporates the goodness-of-fit test studied and others proposed in [1,19,20,21] is also planned.