A Comparison of Limited Information Estimation Methods for the Two-Parameter Normal-Ogive Model with Locally Dependent Items

Abstract

The two-parameter normal-ogive (2PNO) model is one of the most popular item response theory (IRT) models for analyzing dichotomous items. Consistent parameter estimation of the 2PNO model using marginal maximum likelihood estimation relies on the local independence assumption. However, the assumption of local independence might be violated in practice. Likelihood-based estimation of the local dependence structure is often computationally demanding. Moreover, many IRT models that model local dependence do not have a marginal interpretation of item parameters. In this article, limited information estimation methods are reviewed that allow the convenient and straightforward handling of local dependence in estimating the 2PNO model. In detail, pairwise likelihood, weighted least squares, and normal-ogive harmonic analysis robust method (NOHARM) estimation are compared with marginal maximum likelihood estimation that ignores local dependence. A simulation study revealed that item parameters can be consistently estimated with limited information methods. At the same time, marginal maximum likelihood estimation resulted in biased item parameter estimates in the presence of local dependence. From a practical perspective, there were only minor differences regarding the statistical quality of item parameter estimates of the different estimation methods. Differences between the estimation methods are also compared for two empirical datasets.

1. Introduction

Item response theory (IRT) models [1,2,3] are statistical models for analyzing a vector of binary (i.e., dichotomous) random variables. IRT models represent a high-dimensional contingency table by a low-dimensional latent factor variable (also referred to as a trait or ability variable).

Let $\mathit{X}=(X_1,\dots ,X_I)$ be the vector of I dichotomous items $X_i\in \{0,1\}$. A unidimensional IRT model [3,4] is a statistical model for the probability distribution $\mathsf{P}(\mathit{X}=\mathit{x})$ for the vector of item responses $\mathit{x}\in {\{0,1\}}^I$, where

$$\mathsf{P}(\mathit{X}=\mathit{x};\mathit{\gamma })={\int }_{-\infty }^{\infty }\prod _{i=1}^I{P}_i(x_i,\theta ;{\mathit{\gamma }}_i)f\left(\theta \right)\mathrm{d}\theta$$

(1)

and ${\mathit{\gamma }}_i$ are item parameters of item $i=1,\dots ,I$. The latent variable $\theta$ with density f in (1) summarizes the vector $\mathit{X}$ of item responses. Frequently, the density f is fixed and does not depend on the model parameters to be estimated. For convenience, f is often chosen as the standard normal density $\varphi$. The IRT model is most frequently estimated with marginal maximum likelihood (MML) estimation, in which the latent variable $\theta$ is integrated out [4,5].

The item response functions (IRFs)

$$P_i(x,\theta ;{\mathit{\gamma }}_i)=\mathsf{P}(X_i=x|\theta )(\mathrm{for}x=0,1)$$

(2)

in (1) describe the relationship of the dichotomous item $X_i$ with $\theta$. Let $\mathit{\gamma }=({\mathit{\gamma }}_1,\dots ,{\mathit{\gamma }}_I)$ be the model parameter of interest. The two-parameter logistic (2PL) model [6] (also referred to as the two-parameter logit model) uses the IRF

$$P_i(1,\theta ;a_i,b_i)=\Psi (a_i\theta -b_i)=\frac{exp(a_i\theta -b_i)}{1+exp(a_i\theta -b_i)},$$

(3)

where $\Psi$ denotes the logistic distribution function and ${\mathit{\gamma }}_i=(a_i,b_i)$. The item parameter $a_i$ is often called item discrimination, while $b_i$ is denoted as the item intercept. As an alternative IRT model, the IRF of the two-parameter normal-ogive (2PNO) model [7] (also referred to as the two-parameter probit model) is defined as

$$P_i(1,\theta ;a_i,b_i)=\Phi (a_i\theta -b_i)={\int }_{-\infty }^{a_i\theta -b_i}\varphi \left(u\right)\mathrm{d}u,$$

(4)

where $\Phi$ denotes the standard normal distribution function and ${\mathit{\gamma }}_i=(a_i,b_i)$. There is a close correspondence between the 2PL and the 2PNO model (see [8,9,10,11]) because it holds that

$$|\Phi (x)-\Phi (Dx\left)\right|<0.01\mathrm{with}D=1.701.$$

(5)

Hence, item parameters from the 2PL model can be converted to item parameters of the 2PNO model and the other way around if one defines

$$\Phi (a_i\theta -b_i)=\Psi (D{a}_i\theta -D{b}_i)=\Psi ({\tilde{a}}_i\theta -{\tilde{b}}_i)\mathrm{with}{\tilde{a}}_i=D{a}_i\mathrm{and}{\tilde{b}}_i=D{b}_i.$$

(6)

The dimensional reduction of $\mathit{X}$ to $\theta$ in the IRT model (1) is accomplished by the property of local independence [3], which means that item responses $X_i$ are independent conditionally on the latent variable $\theta$. In the IRT model, it is assumed that

$$\mathsf{P}\left(\mathit{X}\right|\theta )=\prod _{i=1}^I\mathsf{P}\left(X_i\right|\theta ).$$

(7)

In empirical datasets containing item responses, the local independence assumption could be violated. In this case, there could exist items $X_i$ and $X_j$ such that

$$\mathsf{P}(X_i,X_j|\theta )\ne \mathsf{P}\left(X_i\right|\theta )\mathsf{P}\left(X_j\right|\theta ).$$

(8)

For example, in a reading comprehension test, students are asked to respond to items that refer to several reading texts. However, items that refer to the same reading text are likely more dependent on each other than items from different reading texts. Hence, the local dependence (i.e., deviation from local independence) of item responses of items within the same reading text can be expected. The item groups that are prone to local dependence are also referred to as testlets [12]. However, local independence holds for item responses to different testlets (i.e., reading texts).

It has been shown that item discriminations are distorted and positively biased in the case of local dependence [13]. Moreover, the neglected local dependence entails inflated reliability estimates [14,15]. Hence, parameter estimation methods must be adapted to allow consistent parameter estimates for ${\mathit{\gamma }}_i$ of the IRFs.

The literature mainly treats local dependence by means of three alternative strategies. First, in testlet IRT models, additional latent variables are included to model the testlet structure [12,16]. Second, the dependency structure is covered by additional parameters within a testlet [17,18,19,20,21]. In this approach, a polytomous superitem is defined as one that decodes the combination of values of items within a testlet in their categories [22]. For example, if a testlet consists of three items, the superitem possesses $2^3=8$ categories. The first and the second approaches have the disadvantage that item parameters do not have a marginal interpretation [14,23]. The third approach employs a marginal unidimensional IRT model and models the local dependence structure employing copula models [14,24,25,26,27]. This approach allows a marginal interpretation of item parameters. However, copula models require the specification of the type of copula distribution of the testlets and estimate testlet residual dependence parameters $\mathit{\delta }$ along with item parameters $\mathit{\gamma }$.

In this article, we are interested in estimating item parameters $a_i$ and $b_i$ of the 2PNO model that possess a marginal interpretation but avoid simultaneously estimating local dependence parameters $\mathit{\delta }$. The 2PNO model can be equivalently formulated using underlying continuous latent variables $X_i^{\ast }$ for dichotomous items $X_i$. Let the dichotomous variable $X_i$ be defined due to the dichotomization of $X_i^{\ast }$

$$X_i=\mathbf{1}(X_i^{\ast }>0),$$

(9)

where $\mathbf{1}$ denotes the indicator function. The Gaussian copula 2PNO model with local dependence [14,25,28] relies on a vector of correlated item residuals $\mathit{\epsilon }=({\epsilon }_1,\dots ,{\epsilon }_I)$. Item residuals $\epsilon$ and the factor variable $\theta$ are uncorrelated. Moreover, each of the components ${\epsilon }_i$ is standard normally distributed. The correlation matrix ${\Sigma }^{\ast }=\mathrm{Var}\left(\epsilon \right)$ models deviations from local independence. Define

$$X_i^{\ast }=a_i\theta -b_i-{\epsilon }_i.$$

(10)

Due to the definition (9), we obtain

$$\mathsf{P}(X_i=1|\theta )=\mathsf{P}(X_i^{\ast }>0|\theta )=\mathsf{P}({\epsilon }_i<a_i\theta -b_i|\theta )=\Phi (a_i\theta -b_i),$$

(11)

which is the IRF of the 2PNO model. Hence, local dependence can be induced by assuming a non-diagonal correlation matrix ${\Sigma }^{\ast }$. Nevertheless, item parameters $a_i$ and $b_i$ retain their marginal interpretation irrespective of an assumed local dependence structure.

Purpose

This article compares different limited information estimation methods of the 2PNO model with correlated item residuals (i.e., the 2PNO normal copula model). The limited information estimation methods can handle local dependence but do not require the specification and estimation of parameters that refer to deviations from local independence. Previous research considered item response datasets under local independence. It indicated that the different limited information methods provided similar estimates and resulted in only minor efficiency loss compared to full information methods. This research investigates whether the findings also translate to item response data with local dependence. To our knowledge, no simulation studies have previously addressed this topic.

The rest of the article is structured as follows. Section 2 discusses the application of limited information methods to handle local dependence. Section 3 presents findings from a simulation study that compares different limited information methods with MML estimation that ignores local dependence. Differences in estimation methods are illustrated through two empirical datasets in Section 4. Section 5 discusses the main findings of our study. Finally, the paper closes with conclusions in Section 6.

2. Limited Information Methods for Local Dependence

In this section, we describe three limited information methods capable of handling local dependence in item responses.

2.1. Pairwise Maximum Likelihood Estimation (PML)

Pairwise (maximum) likelihood (PML; [29,30,31,32,33]) estimation models first-order or second-order probabilities of items. Hence, in contrast to MML, the dependence among items is only modeled up to probabilities of item responses if they are item pairs. The marginal univariate probabilities are given by

$$L_{1,i}(x;a_i,b_i)=\mathsf{P}(X_i=x)=\int P_i(x,\theta ;a_i,b_i)\varphi \left(\theta \right)\mathrm{d}\theta ,$$

(12)

where $\varphi$ denotes the density of the standard normal distribution. PML estimation also relies on the evaluation of bivariate probabilities for an item pair $(X_i,X_j)$

$$L_{2,ij}(x,y;a_i,b_i,a_j,b_j)=\mathsf{P}(X_i=x,X_j=y)=\int P_i(x,\theta ;a_i,b_i)P_j(y,\theta ;a_j,b_j)\varphi \left(\theta \right)\mathrm{d}\theta .$$

(13)

In PML estimation, item parameters $\mathit{\gamma }=(a_1,b_1,\dots ,a_I,b_I)$ are computed by maximizing a weighted sum of likelihood contributions of univariate and bivariate probabilities. In more detail, the optimization function utilized in PML is given by

$$l\left(\mathit{\gamma }\right)=\sum _{i=1}^I{w}_{1,i}\left(\sum _{x=0}^1{n}_{i,x}log{L}_{1,i}(x;a_i,b_i)\right)+\sum _{i<j}{w}_{2,ij}\left(\sum _{x=0}^1\sum _{y=0}^1{n}_{ij,xy}log{L}_{2,ij}(x,y;a_i,b_i,a_j,b_j)\right),$$

(14)

where $n_{i,x}$ denotes the univariate frequency that item $X_i$ takes the value $x\in \{0,1\}$. Furthermore, $n_{ij,xy}$ denotes the bivariate frequency in the sample that item $X_i$ has the value $x\in \{0,1\}$ and item $X_j$ has the value $y\in \{0,1\}$. The weights $w_{1,i}$ and $w_{2,ij}$ can be specifically chosen [34]. For example, the choices $w_{1,i}=1/I$ and $w_{2,ij}=2I^{-1}{(I-1)}^{-1}$ ensure that the univariate and bivariate frequencies equally contribute to the optimization function l defined in (14).

PML is particularly useful in avoiding computationally demanding multidimensional integration in multidimensional IRT models or models involving longitudinal data [35,36,37,38]. In these situations, high-dimensional integrals of latent variables in MML are reduced to lower-dimensional integrals in PML estimation.

PML can also be used to estimate IRT models with locally dependent items. In this case, item pairs $(X_i,X_j)$ that are locally dependent have an inflated associated and will bias estimated item discriminations $a_i$ and item intercepts $b_i$ in the 2PNO model. However, these item pairs can be essentially removed from the PML estimation by setting the corresponding weights $w_{2,ij}$ to zero. This technique has proven successful in estimating the 2PL model [39].

2.2. Weighted Least Squares Estimation (DWLS and ULS)

As an alternative to MML and PML estimation, weighted least squares (WLS) estimation for factor analysis of dichotomous and polytomous item responses have been proposed [40,41,42,43,44]. WLS is a two-step (or three-step) procedure that estimates thresholds and tetrachoric correlations in the first step and computes item parameters in the second step.

Assume that dichotomous items $X_i$ have an underlying normally distributed latent variable $X_i^{\ast }$ defined in the 2PNO model by

$$X_i^{\ast }=a_i\theta -b_i+{\epsilon }_i,\mathrm{and}{X}_i^{\ast }\sim \mathrm{N}(-b_i,a_i^2+1),$$

(15)

where ${\epsilon }_i$ is the item residual, and $\mathrm{N}(\mu ,{\sigma }^2)$ denotes the normal distribution with mean $\mu$ and variance ${\sigma }^2$. The proportion correct of item $X_i$ can be computed as

$$p_i=\mathsf{P}(X_i=1)=\mathsf{P}(X_i^{\ast }>0)=\Phi \left(-\frac{b_i}{\sqrt{a_i^2+1}}\right)=\Phi \left({\tau }_i\right),$$

(16)

where ${\tau }_i$ is the item threshold. The threshold can be estimated by ${\widehat{\tau }}_i={\Phi }^{-1}\left({\widehat{p}}_i\right)$, where ${\widehat{p}}_i$ is the proportion correct of item $X_i$ in the sample.

The correlation between the underlying latent variables $X_i^{\ast }$ and $X_j^{\ast }$ (i.e., the tetrachoric correlation) ${\rho }_{ij}$ can be determined from the 2PNO model as

$${\rho }_{ij}=\mathrm{Cor}(X_i^{\ast },X_j^{\ast })=\frac{\mathrm{Cov}(X_i^{\ast },X_j^{\ast })}{\sqrt{\mathrm{Var}\left(X_i^{\ast }\right)\mathrm{Var}\left(X_j^{\ast }\right)}}=\frac{a_i{a}_j}{\sqrt{a_i^2+1}\sqrt{a_j^2+1}}.$$

(17)

The tetrachoric correlation ${\rho }_{ij}$ can be estimated with maximum likelihood [43] with fixed item thresholds ${\widehat{\tau }}_i$ and ${\widehat{\tau }}_j$. The involved probabilities of item pair $(X_i,X_j)$ in the estimation are given by

$$\mathsf{P}(X_i=1,X_j=1)=\mathsf{P}(X_i^{\ast \ast }>{\tau }_i,X_j^{\ast \ast }>{\tau }_j)=\mathsf{P}(-X_i^{\ast \ast }<-{\tau }_i,-X_j^{\ast \ast }<-{\tau }_j)={\Phi }_2(-{\tau }_i,-{\tau }_j,{\rho }_{ij}),$$

(18)

where ${\Phi }_2$ denotes the distribution function of the bivariate standard normal distribution with correlation ${\rho }_{ij}$. Note that $X_i^{\ast \ast }$ and $X_j^{\ast \ast }$ are underlying normally distributed standardized latent variables. More generally, it holds that

$$\mathsf{P}(X_i=x,X_j=y)={\Phi }_2\left(-(2x-1){\tau }_i,-(2y-1){\tau }_j,{(-1)}^{x+y}{\rho }_{ij}\right)\mathrm{for}x,y\in \{0,1\}.$$

(19)

The optimization function G in diagonally weighted least squares (DWLS; [30]) relies on estimated tetrachoric correlations ${\widehat{\rho }}_{ij}$ and is defined as

$$G\left(\mathit{a}\right)=\sum _{i<j}{w}_{ij}{\left({\widehat{\rho }}_{ij}-\frac{a_i{a}_j}{\sqrt{(1+a_i^2)(1+a_j^2)}}\right)}^2.$$

(20)

The weights $w_{ij}$ are chosen as the inverse of the variances of estimated tetrachoric correlations ${\widehat{\rho }}_{ij}$ and must be estimated in the first step (see [41]). If the test contains locally dependent items, the corresponding item pair $(X_i,X_j)$ receives a weight of zero (i.e., $w_{ij}=0$) in the DWLS estimation.

Unweighted least squares (ULS; [45]) estimation is obtained by choosing all weights equal to one. Again, weights $w_{ij}$ of item pairs $(X_i,X_j)$ that are locally dependent can be set to zero.

The minimizer of G defined in (20) provides an estimate $\widehat{\mathit{a}}=({\widehat{a}}_1,\dots ,{\widehat{a}}_I)$ of item discriminations. Item intercepts $b_i$ are subsequently estimated by

$${\widehat{b}}_i=-\sqrt{1+{\widehat{a}}_i^2}·{\widehat{\tau }}_i.$$

(21)

The IRT approach and the underlying latent variable approach to dichotomous items have been shown to be (essentially) equivalent [46,47,48]. DWLS and ULS estimation showed only a minor efficiency loss compared to MML estimation for dichotomous and polytomous item responses [49,50]. The advantage of WLS estimation is that multidimensionality in latent variables does not pose an issue in the estimation because no multidimensional integrals appear in the estimation function. However, WLS estimation requires the estimation of tetrachoric correlations, which can also be computationally demanding for a large number of items.

2.3. NOHARM Estimation

As an alternative to MML, PML, and WLS estimation, McDonald [51,52,53] proposed an alternative estimation method that relies on approximating the normal-ogive IRF in the 2PNO model by a series of orthogonal Hermite polynomials of order three. This estimation method is referred to as the normal-ogive harmonic analysis robust method (NOHARM) method (see [54,55,56] for overviews). As for WLS estimation, the estimation of item intercepts $b_i$ is separated from the estimation of item discriminations $a_i$.

Let the item threshold ${\tau }_i$ be defined as

$$p_i=\mathsf{P}(X_i=1)=\Phi \left({\tau }_i\right).$$

(22)

Like in WLS estimation, the threshold ${\tau }_i$ can be estimated on the sample proportion correct:

$${\widehat{\tau }}_i={\Phi }^{-1}\left({\widehat{p}}_i\right).$$

(23)

The NOHARM estimation method starts with determining the coefficients of the Hermite series for item $X_i$:

$$c_{i0}=\Phi \left({\widehat{\tau }}_i\right),$$

(24)

$$c_{i1}=-\varphi \left({\widehat{\tau }}_i\right),$$

(25)

$$c_{i2}=\frac{1}{\sqrt{2}}{\widehat{\tau }}_i\varphi \left({\widehat{\tau }}_i\right),\mathrm{and}$$

(26)

$$c_{i3}=-\frac{1}{\sqrt{6}}({\widehat{\tau }}_i^2-1)\varphi \left({\widehat{\tau }}_i\right).$$

(27)

Due to the orthogonality of the Hermite polynomials, the pairwise probabilities passing items $X_i$ and $X_j$ can be approximated by

$$\mathsf{P}(X_i=1,X_j=1)=p_{ij}(a_i,a_j)=\sum _{k=0}^3{c}_{ik}{c}_{jk}{\gamma }_{ij}^k\mathrm{with}{\gamma }_{ij}=\frac{a_i{a}_j}{\sqrt{(1+a_i^2)(1+a_j^2)}}.$$

(28)

Using (28), estimates of the item discriminations can be obtained by minimizing

$$G\left(\mathit{a}\right)=\sum _{i<j}{w}_{ij}{\left({\widehat{p}}_{ij}-p_{ij}(a_i,a_j)\right)}^2.$$

(29)

The item intercepts $b_i$ can be obtained in the same way as in (21). The originally proposed NOHARM procedure fixes all weights $w_{ij}$ in (28) equally to one. In the case of the local dependence of item pairs $(X_i,X_j)$, the corresponding weights $w_{ij}$ can be set to zero. In this article, we also investigate a weighted NOHARM (WNOHARM) estimation method in which we define the weights as the variance of the estimated proportions ${\widehat{p}}_{ij}$ (i.e., $w_{ij}=N{\widehat{p}}_{ij}^{-1}{(1-{\widehat{p}}_{ij})}^{-1}$). To our knowledge, the WNOHARM method has not been investigated previously in the literature. In the literature, it has been demonstrated that NOHARM produces similar results to MML, DWLS, and ULS estimation [57,58,59,60,61].

3. Simulation Study

3.1. Methodology

In this simulation study, item responses under local dependence are simulated according to the 2PNO model with a normal copula model for item residuals [25]. The factor variable $\theta$ is assumed to be standard normally distributed. Two test lengths for the 2PNO model with local dependence are simulated: $I=10$ and $I=20$ items. The item discriminations $a_i$ in the case of $I=10$ items are 1.0, 0.9, 0.8, 0.6, 0.6, 1.1, 0.9, 1.0, 0.8, and 0.7. The item intercepts $b_i$ are chosen as −0.9, −0.7, 1.3, −1.9, −1.0, −1.3, −0.2, −1.3, −0.1, and 0.2. The 10 items are arranged into three testlets containing three items each and a single item. The item residuals ${\epsilon }_i$ are standard normally distributed with a correlation matrix

$${\Sigma }^{\ast }=\left(\begin{array}{cccccccccc}1& & & & & & & & & \\ \delta & 1& & & & & & & & \\ \delta & \delta & 1& & & & & & & \\ 0& 0& 0& 1& & & & & & \\ 0& 0& 0& \delta & 1& & & & & \\ 0& 0& 0& \delta & \delta & 1& & & \\ 0& 0& 0& 0& 0& 0& 1& & & \\ 0& 0& 0& 0& 0& 0& \delta & 1& & \\ 0& 0& 0& 0& 0& 0& \delta & \delta & 1& \\ 0& 0& 0& 0& 0& 0& 0& 0& 0& 1\end{array}\right)$$

(30)

The local dependence parameter $\delta$ is chosen as 0, 0.4, and 0.8, indicating local independence, moderate local dependence, and strong local dependence, respectively. In the case of $I=20$ items, the item parameters from the case of 10 items are duplicated.

The sample sizes $N=500$, 1000, and 2000 are employed in this simulation. In smaller sample sizes, estimation issues of the 2PNO model can be expected.

Each simulated dataset is analyzed with the 2PNO model assuming a standard normal distribution for the factor variable $\theta$. Six different estimation methods are investigated. First, MML estimation ignores the local dependence. Next to MML, five different limited information estimation methods (PML, DWLS, ULS, WNOHARM, NOHARM) are specified that take the local dependence structure into account. In all the approaches, the weights $w_{ij}$ referring to item pairs $(X_i,X_j)$ are set to zero if there is local dependence between $X_i$ and $X_j$.

In total, 1500 replications in each of the 2 (number of items I) × 3 (extent of local dependence $\delta$) × 3 (sample size N) = 18 cells of the simulation study are conducted. We compute the empirical bias and the empirical root mean square error (RMSE) for all estimated item discriminations $a_i$ and item intercepts $b_i$. For each item parameter, the RMSE of an estimation method is divided by the RMSE of the PML estimation method and multiplied by 100. By doing so, the estimation methods are compared relative to the performance of PML estimation. To summarize the simulation results, we compute the average absolute bias and the average relative RMSE across all item discriminations and item intercepts, respectively.

All estimation methods and the analysis of the findings of this simulation study have been carried out using the statistical software R [62]. MML and PML estimation is specified using the R function sirt::xxirt() in the R package sirt [63]. DWLS and ULS estimation is conducted with the probit_wls() function that can be found at https://osf.io/wxbfn accessed on 27 April 2024. The required tetrachoric correlations and their variances are estimated with the R package lavaan [64]. The WNOHARM and NOHARM estimation methods are specified in the R function sirt::noharm.sirt() in the R package sirt [63]. The functions sirt::xxirt(), probit_wls(), and sirt::noharm.sirt() utilize the R-internal optimizers stats::optim() or stats::nlminb(). Replication material for the simulation study can be found at https://osf.io/wxbfn accessed on 27 April 2024.

3.2. Results

Table 1. Simulation Study: Average absolute bias and average relative root mean square error (RMSE) for estimated item discriminations $a_i$ and item intercepts $b_i$ in the two-parameter normal-ogive (2PNO) model as a function of the number of items I, the extent of local dependence $\delta$, and sample size N.

			Average Absolute Bias						Average Relative RMSE
$\mathit{I}$	$\mathit{\delta }$	$\mathit{N}$	MML	PML	DWLS	ULS	WNH	NH	MML	PML	DWLS	ULS	WNH	NH
			Item discriminations $a_i$
10	0	500	0.014	0.017	0.022	0.018	0.021	0.021	91.8	100 ‡	100.5	102.7	104.6	104.3
		1000	0.007	0.008	0.011	0.010	0.011	0.011	92.7	100 ‡	100.7	104.4	104.1	104.0
		2000	0.003	0.003	0.005	0.004	0.007	0.007	92.6	100 ‡	100.4	104.9	103.6	103.6
	0.4	500	0.118	0.020	0.023	0.020	0.022	0.022	124.0	100 ‡	100.4	104.2	103.7	103.2
		1000	0.113	0.010	0.012	0.011	0.013	0.013	149.3	100 ‡	100.5	104.7	103.7	103.7
		2000	0.107	0.006	0.006	0.006	0.009	0.009	183.5	100 ‡	100.5	105.5	103.4	103.6
	0.8	500	0.334	0.025	0.025	0.024	0.026	0.026	274.7	100 ‡	99.7	103.6	102.1	102.4
		1000	0.320	0.012	0.012	0.012	0.013	0.014	350.0	100 ‡	100.2	105.4	102.1	102.6
		2000	0.316	0.008	0.007	0.007	0.009	0.009	464.0	100 ‡	100.2	105.0	102.5	102.9
20	0	500	0.007	0.008	0.015	0.008	0.012	0.012	95.7	100 ‡	101.2	101.2	103.4	103.2
		1000	0.011	0.005	0.009	0.006	0.009	0.009	97.3	100 ‡	100.6	101.9	103.1	102.9
		2000	0.001	0.003	0.004	0.003	0.006	0.006	95.7	100 ‡	100.5	102.1	103.5	103.4
	0.4	500	0.053	0.013	0.019	0.013	0.016	0.016	105.7	100 ‡	101.4	101.5	103.3	103.1
		1000	0.057	0.005	0.009	0.006	0.009	0.009	121.6	100 ‡	100.5	101.9	102.7	102.5
		2000	0.046	0.004	0.005	0.003	0.007	0.007	128.4	100 ‡	100.3	102.2	103.3	103.3
	0.8	500	0.111	0.014	0.018	0.012	0.016	0.016	139.9	100 ‡	101.1	101.5	102.8	102.9
		1000	0.119	0.007	0.010	0.008	0.011	0.011	178.5	100	100.4	102.2	102.5	102.7
		2000	0.104	0.004	0.004	0.003	0.007	0.007	207.7	100 ‡	100.2	102.4	102.8	103.1
			Item intercepts $b_i$
10	0	500	0.013	0.014	0.016	0.015	0.017	0.016	96.4	100 ‡	100.2	100.6	102.3	102.0
		1000	0.006	0.007	0.008	0.008	0.009	0.008	97.0	100 ‡	100.6	102.1	102.2	102.1
		2000	0.003	0.003	0.004	0.004	0.005	0.005	97.0	100 ‡	100.3	102.4	101.9	101.9
	0.4	500	0.056	0.017	0.017	0.017	0.018	0.017	110.4	100 ‡	100.1	101.5	102.2	101.9
		1000	0.046	0.008	0.009	0.010	0.010	0.010	116.3	100 ‡	100.6	102.5	102.2	102.2
		2000	0.044	0.005	0.005	0.005	0.006	0.006	129.2	100 ‡	100.5	102.8	102.1	102.1
	0.8	500	0.114	0.022	0.020	0.020	0.021	0.021	157.4	100 ‡	99.2	100.5	101.1	101.2
		1000	0.090	0.010	0.010	0.011	0.011	0.011	165.7	100 ‡	100.3	102.7	101.6	101.8
		2000	0.101	0.006	0.005	0.005	0.006	0.006	211.6	100 ‡	99.8	102.1	101.3	101.4
20	0	500	0.015	0.010	0.013	0.010	0.012	0.012	100.7	100 ‡	100.7	99.7	101.4	101.2
		1000	0.013	0.005	0.007	0.006	0.007	0.007	100.2	100 ‡	100.6	100.8	101.6	101.4
		2000	0.005	0.003	0.003	0.003	0.004	0.004	101.9	100 ‡	100.1	100.7	101.3	101.2
	0.4	500	0.045	0.013	0.015	0.012	0.014	0.014	109.6	100 ‡	100.5	99.6	101.4	101.2
		1000	0.020	0.006	0.007	0.006	0.007	0.007	103.8	100 ‡	100.6	100.8	101.6	101.4
		2000	0.030	0.004	0.004	0.003	0.005	0.005	118.2	100 ‡	99.9	100.6	101.3	101.1
	0.8	500	0.078	0.013	0.014	0.011	0.013	0.013	127.6	100 ‡	100.4	99.7	101.1	101.0
		1000	0.034	0.007	0.008	0.007	0.008	0.008	112.2	100 ‡	100.5	101.0	101.5	101.4
		2000	0.059	0.004	0.004	0.003	0.005	0.005	156.5	100 ‡	99.5	100.4	100.7	100.7

Note. MML = marginal maximum likelihood estimation ignoring local independence; PML = pairwise likelihood estimation; DWLS = diagonally least squares estimation; ULS = unweighted least squares estimation; WNH = weighted normal ogive harmonic analysis robust method (WNOHARM) estimation; NH = normal ogive harmonic analysis robust method (NOHARM) estimation; ^‡ = PML was defined as the reference method for computing the average RMSE. Average absolute bias values larger than 0.015 and average RMSE values larger than 105.0 are printed in bold font.

reports the average absolute bias and the average relative RMSE for estimated item discriminations and item intercepts in the 2PNO model as a function of the number of items I, the extent of local dependence $\delta$, and the sample size N.

All six estimation methods are unbiased in large samples if local independence holds (i.e., $\delta =0$). Notably, small biases occur for the sample size $N=500$. The average relative RMSE indicates an efficiency loss when using PML estimation and omitting some item pairs from estimation compared to MML under local independence. However, the efficiency loss is decreased with a larger number of items.

In the case of local dependence (i.e., $\delta =0.4$ and $\delta =0.8$), MML produces strongly biased results. The biases for MML are larger for item discriminations than for item intercepts. Note that the bias in MML estimation is reduced with an increasing number of items. Across all conditions, PML and DWLS perform similarly regarding the mean (denoted by M) RMSE and slightly outperform ULS, WNOHARM, and NOHARM estimation for item discriminations $a_i$ (PML: M = 100, DWLS: M = 100.5, ULS: M = 103.2, WNOHARM: M = 103.2, NOHARM: M = 103.2). Overall, WNOHARM resulted in similar estimates to NOHARM. However, there are slight advantages of WNOHARM compared to NOHARM only for strong local dependence in the case of $\delta =0.8$. For item intercepts $b_i$, PML and DWLS have smaller advantages over ULS, WNOHARM, and NOHARM (PML: M = 100, DWLS: M = 100.3, ULS: M = 101.1, WNOHARM: M = 101.6, NOHARM: M = 101.5).

To sum up, all limited information methods could handle locally dependent item responses and result in approximately unbiased item parameter estimates. In applied research, the differences between the estimation methods can be considered negligible.

4. Empirical Examples

In this section, two datasets with item responses with a testlet structure from the R package sirt [63] are analyzed with the 2PNO model. The same estimation methods (i.e., MML, PML, DWLS, ULS, WNOHARM, and NOHARM) as in the simulation study (see Section 4) are utilized. Item pairs that refer to the same testlet receive a zero weight in the limited information estimation methods. We compute the average absolute deviation of estimated item parameters between the estimation methods as a measure of the discrepancy between the methods.

4.1. Dataset data.read

The first dataset data.read contains item responses of $N=328$ subjects on $I=12$ items from a reading comprehension test administered to Austrian students. The twelve items are arranged into three testlets, each testlet containing four items.

Table 2. Dataset data.read: Estimated item discriminations $a_i$ and item intercepts $b_i$ in the two-parameter normal-ogive (2PNO) model for different estimation methods.

		${\mathit{a}}_{\mathit{i}}$						${\mathit{b}}_{\mathit{i}}$
Item	Testlet	MML	PML	DWLS	ULS	WNH	NH	MML	PML	DWLS	ULS	WNH	NH
A1	A	$\phantom{-}$0.59	$\phantom{-}$0.45	$\phantom{-}$0.49	$\phantom{-}$0.36	$\phantom{-}$0.54	$\phantom{-}$0.58	−1.20	−1.14	−1.16	−1.10	−1.18	−1.20
A2	A	$\phantom{-}$0.85	$\phantom{-}$0.84	$\phantom{-}$0.89	$\phantom{-}$0.88	$\phantom{-}$0.91	$\phantom{-}$0.91	−0.84	−0.83	−0.85	−0.85	−0.86	−0.86
A3	A	$\phantom{-}$0.68	$\phantom{-}$0.63	$\phantom{-}$0.64	$\phantom{-}$0.60	$\phantom{-}$0.69	$\phantom{-}$0.70	−0.21	−0.20	−0.20	−0.20	−0.21	−0.21
A4	A	$\phantom{-}$0.53	$\phantom{-}$0.52	$\phantom{-}$0.53	$\phantom{-}$0.47	$\phantom{-}$0.58	$\phantom{-}$0.59	$\phantom{-}$0.11	$\phantom{-}$0.11	$\phantom{-}$0.11	$\phantom{-}$0.11	$\phantom{-}$0.12	$\phantom{-}$0.12
B1	B	$\phantom{-}$0.36	$\phantom{-}$0.39	$\phantom{-}$0.40	$\phantom{-}$0.39	$\phantom{-}$0.40	$\phantom{-}$0.40	−0.60	−0.60	−0.61	−0.60	−0.61	−0.61
B2	B	$\phantom{-}$0.40	$\phantom{-}$0.46	$\phantom{-}$0.49	$\phantom{-}$0.52	$\phantom{-}$0.47	$\phantom{-}$0.46	−0.01	−0.02	−0.02	−0.02	−0.02	−0.02
B3	B	$\phantom{-}$0.62	$\phantom{-}$0.62	$\phantom{-}$0.65	$\phantom{-}$0.69	$\phantom{-}$0.59	$\phantom{-}$0.55	−1.56	−1.57	−1.59	−1.62	−1.55	−1.52
B4	B	$\phantom{-}$0.65	$\phantom{-}$0.86	$\phantom{-}$0.90	$\phantom{-}$0.90	$\phantom{-}$0.84	$\phantom{-}$0.84	−0.57	−0.63	−0.64	−0.64	−0.62	−0.62
C1	C	$\phantom{-}$1.50	$\phantom{-}$0.47	$\phantom{-}$0.47	$\phantom{-}$0.48	$\phantom{-}$0.44	$\phantom{-}$0.43	−2.66	−1.66	−1.65	−1.66	−1.64	−1.63
C2	C	$\phantom{-}$0.86	$\phantom{-}$0.64	$\phantom{-}$0.62	$\phantom{-}$0.64	$\phantom{-}$0.61	$\phantom{-}$0.61	−0.74	−0.67	−0.66	−0.67	−0.66	−0.66
C3	C	$\phantom{-}$0.98	$\phantom{-}$0.40	$\phantom{-}$0.40	$\phantom{-}$0.42	$\phantom{-}$0.37	$\phantom{-}$0.35	−1.59	−1.22	−1.22	−1.23	−1.21	−1.20
C4	C	$\phantom{-}$0.64	$\phantom{-}$0.54	$\phantom{-}$0.38	$\phantom{-}$0.38	$\phantom{-}$0.38	$\phantom{-}$0.38	−0.75	−0.75	−0.67	−0.67	−0.67	−0.67

Note. MML = marginal maximum likelihood estimation ignoring local independence; PML  = pairwise likelihood estimation; DWLS = diagonally least squares estimation; ULS = unweighted least squares estimation; WNH = weighted normal-ogive harmonic analysis robust method (WNOHARM) estimation; NH = normal-ogive harmonic analysis robust method (NOHARM) estimation.

displays estimated item discriminations $a_i$ and item intercepts $b_i$ in the 2PNO model.

MML substantially differs in the item discriminations $a_i$ from all limited information methods regarding the average absolute deviations (PML: 0.20, DWLS: 0.23, ULS: 0.25, WNOHARM: 0.22, NOHARM: 0.23). The average absolute deviations between the limited information methods are much smaller and range between 0.01 (WNOHARM and NOHARM) and 0.07 (ULS and NOHARM).

We also observe non-negligible differences for item intercepts $b_i$ of MML with the five specified limited information methods (PML: 0.13, DWLS: 0.14, ULS: 0.15, WNOHARM: 0.14, NOHARM: 0.14). The differences in item intercepts between the limited information methods are small, with the largest average absolute deviation of 0.03 between ULS and NOHARM estimation.

Figure 1 presents pairwise scatter plots and correlations of estimated item discriminations $a_i$ for the dataset data.read. The correlation of the MML estimates with the other estimation methods are almost zero. Obviously, one outlying observation (i.e., Item C1 with $a_i=1.50$) impacts the correlations of the MML estimates with other methods. Moreover, WNOHARM and NOHARM are correlated at 1.00, while the correlation of ULS and NOHARM is relatively small at 0.86, which indicates non-negligible differences in the estimates.

Figure 2 shows pairwise scatter plots and correlations of estimated item intercepts $b_i$. The correlations of the MML estimates for $b_i$ with the other estimation methods are far from perfect at 0.95. However, the estimates of the alternative estimation methods correlate at 1.00, indicating almost identical estimates.

4.2. Dataset data.pisaMath

The second dataset data.pisaMath contains item responses of $N=565$ Austrian students on $I=11$ mathematics items as a subset of students and items from a PISA study [65,66]. The eleven items are arranged into four testlets with two items each. Moreover, the dataset has three single items without a testlet structure.

Table 3. Dataset data.pisaMath: Estimated item discriminations $a_i$ and item intercepts $b_i$ in the two-parameter normal-ogive (2PNO) model for different estimation methods.

		${\mathit{a}}_{\mathit{i}}$						${\mathit{b}}_{\mathit{i}}$
Item	Testlet	MML	PML	DWLS	ULS	WNH	NH	MML	PML	DWLS	ULS	WNH	NH
M192Q01	—	$\phantom{-}$0.75	$\phantom{-}$0.79	$\phantom{-}$0.81	$\phantom{-}$0.80	$\phantom{-}$0.78	$\phantom{-}$0.78	$\phantom{-}$0.14	$\phantom{-}$0.14	$\phantom{-}$0.14	$\phantom{-}$0.14	$\phantom{-}$0.14	$\phantom{-}$0.14
M406Q01	M406	$\phantom{-}$1.03	$\phantom{-}$0.86	$\phantom{-}$0.87	$\phantom{-}$0.88	$\phantom{-}$0.87	$\phantom{-}$0.87	$\phantom{-}$0.21	$\phantom{-}$0.20	$\phantom{-}$0.20	$\phantom{-}$0.20	$\phantom{-}$0.20	$\phantom{-}$0.20
M406Q02	M406	$\phantom{-}$1.28	$\phantom{-}$1.04	$\phantom{-}$1.04	$\phantom{-}$1.04	$\phantom{-}$1.07	$\phantom{-}$1.06	$\phantom{-}$0.94	$\phantom{-}$0.84	$\phantom{-}$0.83	$\phantom{-}$0.83	$\phantom{-}$0.85	$\phantom{-}$0.84
M423Q01	—	$\phantom{-}$0.31	$\phantom{-}$0.31	$\phantom{-}$0.31	$\phantom{-}$0.31	$\phantom{-}$0.31	$\phantom{-}$0.31	−0.67	−0.67	−0.67	−0.67	−0.67	−0.67
M496Q01	M496	$\phantom{-}$0.82	$\phantom{-}$0.74	$\phantom{-}$0.74	$\phantom{-}$0.75	$\phantom{-}$0.74	$\phantom{-}$0.74	−0.17	−0.17	−0.17	−0.17	−0.17	−0.17
M496Q02	M496	$\phantom{-}$0.65	$\phantom{-}$0.55	$\phantom{-}$0.55	$\phantom{-}$0.55	$\phantom{-}$0.55	$\phantom{-}$0.55	−0.67	−0.65	−0.65	−0.65	−0.65	−0.65
M564Q01	M564	$\phantom{-}$0.51	$\phantom{-}$0.52	$\phantom{-}$0.52	$\phantom{-}$0.52	$\phantom{-}$0.52	$\phantom{-}$0.52	−0.04	−0.04	−0.04	−0.04	−0.04	−0.04
M564Q02	M564	$\phantom{-}$0.49	$\phantom{-}$0.50	$\phantom{-}$0.50	$\phantom{-}$0.50	$\phantom{-}$0.50	$\phantom{-}$0.50	−0.07	−0.07	−0.07	−0.07	−0.07	−0.07
M571Q01	—	$\phantom{-}$0.85	$\phantom{-}$0.91	$\phantom{-}$0.93	$\phantom{-}$0.89	$\phantom{-}$0.92	$\phantom{-}$0.93	−0.15	−0.16	−0.16	−0.16	−0.16	−0.16
M603Q01	M603	$\phantom{-}$0.70	$\phantom{-}$0.74	$\phantom{-}$0.75	$\phantom{-}$0.73	$\phantom{-}$0.73	$\phantom{-}$0.74	−0.17	−0.17	−0.17	−0.17	−0.17	−0.17
M603Q02	M603	$\phantom{-}$0.85	$\phantom{-}$0.91	$\phantom{-}$0.93	$\phantom{-}$0.89	$\phantom{-}$0.89	$\phantom{-}$0.91	$\phantom{-}$0.09	$\phantom{-}$0.09	$\phantom{-}$0.09	$\phantom{-}$0.09	$\phantom{-}$0.09	$\phantom{-}$0.09

Note. MML = marginal maximum likelihood estimation ignoring local independence; PML  = pairwise likelihood estimation; DWLS = diagonally least squares estimation; ULS = unweighted least squares estimation; WNH = weighted normal-ogive harmonic analysis robust method (WNOHARM) estimation; NH = normal-ogive harmonic analysis robust method (NOHARM) estimation.

displays estimated item discriminations and item intercepts of the 2PNO model for this dataset. Substantial deviations for item parameters between MML estimation and the limited information estimation methods are observed for items in the testlets M406 and M496. The average absolute deviation of estimated item discriminations $a_i$ of MML estimation with the five limited information methods is always 0.07. However, there are no substantial differences in item discriminations between the limited information methods, and they differ, on average, by no more than 0.01. For item intercepts $b_i$, the average absolute differences between MML and all limited information methods are smaller and at 0.02. Furthermore, there are no notable differences among the limited information estimation methods, with a maximum absolute difference of 0.002.

5. Discussion

5.1. Merits

In this article, we compared MML estimation with limited information estimation methods in the 2PNO model with locally dependent items. It turned out that limited information methods PML, DWLS, ULS, WNOHARM, and NOHARM were effective in handling local dependence and produced approximately unbiased item parameters in the 2PNO model. The employed limited information methods operate on statistical information (i.e., bivariate frequencies) of item pairs. This property allows the exclusion of item pairs prone to local dependence from estimation. By doing so, item parameters from the 2PNO model can be marginally interpreted. Overall, there were only minor differences in the quality of parameter estimates between the different limited information methods. From the perspective of an applied researcher, the different estimation methods can be considered exchangeable, and the availability of software can determine the choice of a particular estimation method. Notably, MML estimation that ignores local dependence will typically provide biased item parameter estimates.

5.2. Limitations

As with any simulation study, our study had several limitations. First, we did not simulate item responses with larger item discriminations. It could be that the performance of the limited information methods more substantially differs with more extreme item parameters. Second, we did not investigate smaller sample sizes such as $N=250$. In this case, it is more crucial how to handle estimation issues. Third, we did not treat polytomous data. However, the NOHARM procedure must be adapted from dichotomous to polytomous item responses [56]. Fourth, our analyses were restricted to unidimensional IRT models with local dependence. In a similar vein, multidimensional IRT models with locally dependent items can also be handled and investigated in future research. Fifth, the item discriminations used in the simulation study had moderate deviations. We carried out preliminary additional simulation studies in which there was more variation in item discriminations. It turned out that NOHARM (and WNOHARM) estimates performed worse than PML or DWLS (and ULS) in these situations. Sixth, we only considered a normally distributed $\theta$ variable and the probit link function in the 2PNO model. PML estimation has the advantage that it can be generalized to other link functions (such as in the 2PL model) or non-normal $\theta$ distributions [39]. However, DWLS/ULS and NOHARM rely on the probit link function and the normal distribution and cannot be generalized straightforwardly.

5.3. Meaning of Local Dependence

We would like to emphasize that any empirical dataset will necessarily have some deviations from local independence [67]. If a unidimensional IRT model is fitted to a dataset of item responses, positive and negative local dependencies will average out in some way. It is up to the researcher to define whether positive local dependence should only exist. Such a decision would imply that positive local dependence is treated as a nuisance that should be removed from model estimation. However, there might be situations in which local dependence is an in-built feature of a test, and it is not advised to control for such factors in statistical analysis [68].

6. Conclusions

Our study demonstrated that ignoring local dependence in the estimation of the 2PNO model results in biased item parameters, particularly in biases for item discriminations. We have shown in a simulation study and empirical examples that limited information methods can easily be adapted to item responses containing local dependence by only including item pairs in the estimation that are not prone to local dependence. Utilizing this modification in the estimation methods results in unbiased item parameter estimation under local dependence. Importantly, there is no need to model the local dependence structure in the modified limited information estimation methods. In this respect, limited information methods are more robust than the full information maximum likelihood estimation method, which requires a correctly specified distribution to model local dependence.