A Comparison of Mixed and Partial Membership Diagnostic Classification Models with Multidimensional Item Response Models

Abstract

Diagnostic classification models (DCM) are latent structure models with discrete multivariate latent variables. Recently, extensions of DCMs to mixed membership have been proposed. In this article, ordinary DCMs, mixed and partial membership models, and multidimensional item response theory (IRT) models are compared through analytical derivations, three example datasets, and a simulation study. It is concluded that partial membership DCMs are similar, if not structurally equivalent, to sufficiently complex multidimensional IRT models.

1. Introduction

In the social or life sciences, humans (i.e., subjects or persons) respond to multiple tasks (i.e., items) in a test. For example, students may be asked to solve items in a mathematics test, or patients may report whether particular symptoms present. These tests result in multivariate datasets with dichotomously scored item responses taking values of zero or one.

Let $\mathit{X}=(X_1,\dots ,X_I)\in {\{0,1\}}^I$ be a random vector containing I items $X_i$ ($i=1,\dots ,I$). In multivariate analysis, the dependency structure of the high-dimensional contingency table $\mathrm{P}(\mathit{X}=\mathit{x})$ (for $\mathit{x}\in {\{0,1\}}^I$) is represented by sufficiently parsimonious models. A general model is defined in latent structure analysis [1,2,3,4,5]:

$$\mathrm{P}(\mathit{X}=\mathit{x};\mathsf{\gamma })=\int \prod _{i=1}^I\mathrm{P}(X_i=x_i|\mathsf{\xi };{\mathsf{\gamma }}_i)\mathrm{d}{F}_{\mathsf{\beta }}\left(\mathsf{\xi }\right),$$

(1)

where $\mathsf{\gamma }=({\mathsf{\gamma }}_1,\dots ,{\mathsf{\gamma }}_I,\mathsf{\beta })$ is a model parameter. Note that the integration with respect to $F_{\mathsf{\beta }}$ is interpreted as any measure that can be either continuous, discrete, or a mixture of both. It should be emphasized that the dependency structure in $\mathit{X}$ is represented by a (finite-dimensional) latent variable $\mathsf{\xi }$. The dimension of $\mathsf{\xi }$ is typically lower than I to achieve a convenient model interpretation of the dependency in $\mathit{X}$. The model (1) imposes a local independence assumption, meaning that the items $X_i$ are conditionally independent of $\mathsf{\xi }$. The functions $\mathsf{\xi }\mapsto \mathrm{P}(X_i=x_i|\mathsf{\xi };{\mathsf{\gamma }}_i)$ are also referred to as item response functions (IRF).

In item response theory (IRT; Refs. [6,7,8]), the latent variable $\mathsf{\xi }$ is a unidimensional or multidimensional latent variable $\mathsf{\theta }$ that can take values between minus and plus infinity. Diagnostic classification models (DCM; Refs. [9,10,11]) are particular latent class models in which the variable $\mathsf{\xi }$ is denoted as $\mathsf{\alpha }$ and is a multidimensional latent variable. It is evident that IRT and DCM offer quite distinct interpretations of the underlying latent variable to applied researchers. Recently, the exact allocation (i.e., crisp membership) of subjects to latent classes in DCMs has been weakened, allowing for gradual membership to latent classes [12,13]. Comparisons between unidimensional IRT models and DCMs have been carried out in  [14,15,16,17,18,19,20,21]. However, comparisons of DCMs with multidimensional DCMs are scarce in the literature, and to our knowledge, there is no systematic comparison of multidimensional IRT models and mixed membership DCMs.

Purpose

In this article, DCMs, mixed membership DCMs, and multidimensional IRT models are compared using three publicly available datasets and by conducting a simulation study focusing on model selection.

The remainder of the article is structured as follows. Section 2 reviews compensatory and noncompensatory multidimensional IRT models. DCMs are reviewed in Section 3. The extension of ordinary DCMs to DCMs with mixed or partial membership is outlined in Section 4. Section 5 provides a heuristic comparison of multidimensional IRT models, DCMs, and DCM partial membership extensions. In Section 6, three empirical datasets are analyzed using different specifications of the three modeling approaches. Moreover, Section 7 presents findings from a simulation study devoted to model selection. Finally, the article closes with a discussion Section 8.

2. Multidimensional Item Response Model

Multidimensional IRT (MIRT) models [22,23] assume a vector $\mathsf{\theta }=({\mathsf{\theta }}_1,\dots ,{\mathsf{\theta }}_D)$ of real-valued ability variables. That is, abilities ${\mathsf{\theta }}_d$ can range from minus to plus infinity. The MIRT model can be written as:

$$\mathrm{P}(\mathit{X}=\mathit{x};\mathsf{\gamma })=\int \prod _{i=1}^I\mathrm{P}(X_i=x_i|\mathsf{\theta };{\mathsf{\gamma }}_i)f_{\mathsf{\beta }}\left(\mathsf{\theta }\right)\mathrm{d}\mathsf{\theta },$$

(2)

where $f_{\mathsf{\beta }}$ denotes the multivariate density function of $\mathsf{\theta }$ that depends on the unknown parameter $\mathsf{\beta }$. Most frequently, a multivariate normal distribution for $\mathsf{\theta }$ is assumed, and its density is given as:

$$f_{\mathsf{\theta }}(\mathit{x};\mathsf{\mu },\mathsf{\Sigma })={\left(2\pi \right)}^{-D/2}{\left|\mathsf{\Sigma }\right|}^{-1/2}exp\left(-\frac{1}{2}{(\mathit{x}-\mathsf{\mu })}^{\top }{\mathsf{\Sigma }}^{-1}(\mathit{x}-\mathsf{\mu })\right)\mathrm{for}x\in {\mathbb{R}}^D,$$

(3)

where $\mathsf{\mu }$ and $\mathsf{\Sigma }$ are the mean vector and the covariance matrix of $\mathsf{\theta }$, respectively. Moreover, $\left|\mathsf{\Sigma }\right|$ denotes the determinant of $\mathsf{\Sigma }$. In MIRT models that estimate item intercepts and item slopes for all items separately, all means are usually fixed to 0, and all standard deviations are fixed to 1 for identification reasons.

In this article, we confine ourselves to confirmatory MIRT models. Items are allocated to dimensions of $\mathsf{\theta }$ in the specification of the model. The allocation can be encoded in a Q-matrix [24,25] that defines which item loads on which dimension. Two types of loading structures can be distinguished: between-item dimensionality and within-item dimensionality [26,27].

2.1. Between-Item Dimensionality

In between-item dimensionality [26,28], all items load on one and only dimension. That is, item $X_i$ loads only on one dimension $d_1\left[i\right]$. In this case, the multidimensional IRF is reduced to an unidimensional IRF:

$$\mathrm{P}(X_i=1|\mathsf{\theta };{\mathsf{\gamma }}_i)=\mathrm{P}(X_i=1|{\mathsf{\theta }}_{d_1\left[i\right]};{\mathsf{\gamma }}_i).$$

(4)

2.2. Within-Item Dimensionality

In within-item dimensionality, an item is allowed to load on more than a single dimension. Let us assume that item $X_i$ loads on the two dimensions $d_1\left[i\right]$ and $d_2\left[i\right]$. Then, the IRF can be written as:

$$\mathrm{P}(X_i=1|\mathsf{\theta };{\mathsf{\gamma }}_i)=\mathrm{P}(X_i=1|{\mathsf{\theta }}_{d_1\left[i\right]},{\mathsf{\theta }}_{d_2\left[i\right]};{\mathsf{\gamma }}_i).$$

(5)

In the rest of the paper, models for within-item dimensionality are only discussed in the case that items load on at most two dimensions. The case in which items load on more than two dimensions is not conceptually different but introduces much more notation. Therefore, we restrict ourselves to only two-dimensional IRT models to simplify the technical aspects of our arguments.

The most frequent choice of the IRF in the case of between-item dimensionality is the multivariate variant of the 2PL model [29,30] and is given as:

$$\mathrm{P}(X_i=1|{\mathsf{\theta }}_{d_1\left[i\right]};{\mathsf{\gamma }}_i)=\mathsf{\Psi }\left({\mathsf{\lambda }}_{i0}+{\mathsf{\lambda }}_{i1,d_1\left[i\right]}{\mathsf{\theta }}_{d_1\left[i\right]}\right),$$

(6)

where ${\mathsf{\lambda }}_{i0}$ denotes the item intercept and ${\mathsf{\lambda }}_{i1,d_1\left[i\right]}$ the item slope, respectively, and $\mathsf{\Psi }$ denotes the logistic distribution function.

There is more flexibility in defining IRFs in the case of within-item dimensionality than for between-item dimensionality. Different alternatives are discussed in the following subsections.

2.2.1. Compensatory Multidimensional IRT Model

In the case of within-item dimensionality, the IRF of item $X_i$ depends on the $\mathsf{\theta }$ values of dimensions $d_1\left[i\right]$ and $d_2\left[i\right]$. The compensatory MIRT model [23,30] is defined as:

$$\mathrm{P}(X_i=1|{\mathsf{\theta }}_{d_1\left[i\right]},{\mathsf{\theta }}_{d_2\left[i\right]};{\mathsf{\gamma }}_i)=\mathsf{\Psi }\left({\mathsf{\lambda }}_{i0}+{\mathsf{\lambda }}_{i1,d_1\left[i\right]}{\mathsf{\theta }}_{d_1\left[i\right]}+{\mathsf{\lambda }}_{i1,d_2\left[i\right]}{\mathsf{\theta }}_{d_2\left[i\right]}\right),$$

(7)

where ${\mathsf{\lambda }}_{i1,d_1\left[i\right]}$ and ${\mathsf{\lambda }}_{i1,d_2\left[i\right]}$ are dimension-specific item slopes. It can be seen in (7) that the terms ${\mathsf{\lambda }}_{i1,d_1\left[i\right]}{\mathsf{\theta }}_{d_1\left[i\right]}$ and ${\mathsf{\lambda }}_{i1,d_2\left[i\right]}{\mathsf{\theta }}_{d_2\left[i\right]}$ can compensate for each other. Hence, for example, low values in ${\mathsf{\theta }}_{d_1\left[i\right]}$ can be compensated by large values in ${\mathsf{\theta }}_{d_2\left[i\right]}$. Nevertheless, the IRF in (7) allows a flexible relationship of the required abilities for item i. By allowing for compensation in the abilities, the abilities ${\mathsf{\theta }}_{d_1\left[i\right]}$ and ${\mathsf{\theta }}_{d_2\left[i\right]}$ can operate somehow independently, which is not necessarily the case for noncompensatory MIRT models discussed in the next subsection.

2.2.2. Noncompensatory Multidimensional IRT Model

In some applications, it can be questioned whether the compensatory MIRT model makes assumptions that are likely to be met. In contrast, it could be argued that low abilities in one dimension cannot be compensated by high abilities in another dimension. For this reason, noncompensatory MIRT models have been proposed. A noncompensatory functioning of abilities ${\mathsf{\theta }}_{d_1\left[i\right]}$ and ${\mathsf{\theta }}_{d_1\left[i\right]}$ can be defined by defining the multidimensional IRF as a product of dimension-wise IRFs [31,32,33,34]:

$$\mathrm{P}(X_i=1|{\mathsf{\theta }}_{d_1\left[i\right]},{\mathsf{\theta }}_{d_2\left[i\right]};{\mathsf{\gamma }}_i)=\mathsf{\Psi }\left({\mathsf{\lambda }}_{i0,d_1\left[i\right]}+{\mathsf{\lambda }}_{i1,d_1\left[i\right]}{\mathsf{\theta }}_{d_1\left[i\right]}\right)\mathsf{\Psi }\left({\mathsf{\lambda }}_{i0,d_2\left[i\right]}+{\mathsf{\lambda }}_{i1,d_2\left[i\right]}{\mathsf{\theta }}_{d_2\left[i\right]}\right).$$

(8)

Note that item intercepts are separately defined for each dimension. Hence, the noncompensatory MIRT model (8) has one additional item parameter compared to the compensatory MIRT model (7). An alternative approach to (8) was proposed in [35], in which the minimum instead of the product of the two probabilities was chosen.

2.2.3. Partially Compensatory Multidimensional IRT Model

It might be too restrictive to decide between the extreme cases of the compensatory MIRT model (7) and the noncompensatory MIRT model (8). The partially compensatory MIRT model has been proposed that contains the former two MIRT models as particular cases [36]. By defining the linear terms:

$$T_{i,d_1\left[i\right]}={\mathsf{\lambda }}_{i0,d_1\left[i\right]}+{\mathsf{\lambda }}_{i1,d_1\left[i\right]}{\mathsf{\theta }}_{d_1\left[i\right]}\mathrm{and}{T}_{i,d_2\left[i\right]}={\mathsf{\lambda }}_{i0,d_2\left[i\right]}+{\mathsf{\lambda }}_{i1,d_2\left[i\right]}{\mathsf{\theta }}_{d_2\left[i\right]},$$

(9)

The IRF of the partially compensatory MIRT model is given as

$$\mathrm{P}(X_i=1|{\mathsf{\theta }}_{d_1\left[i\right]},{\mathsf{\theta }}_{d_2\left[i\right]};{\mathsf{\gamma }}_i)=\frac{exp\left(T_{i,d_1\left[i\right]}+T_{i,d_2\left[i\right]}\right)}{1+exp\left(T_{i,d_1\left[i\right]}+T_{i,d_2\left[i\right]}\right)+{\mathsf{\lambda }}_{i12}\left[exp\left(T_{i,d_1\left[i\right]}\right)+exp\left(T_{i,d_2\left[i\right]}\right)\right]},$$

(10)

where the interaction parameter ${\mathsf{\lambda }}_{i12}$ ranges between 0 and 1. A value of 0 for ${\mathsf{\lambda }}_{i12}$ corresponds to the compensatory MIRT model (7), while a value of 1 corresponds to the noncompensatory MIRT model (8).

An alternative partially compensatory MIRT model has been proposed whose IRFs also involve the product of abilities ${\mathsf{\theta }}_{d_1\left[i\right]}{\mathsf{\theta }}_{d_2\left[i\right]}$ as an additional term [37]. The multidimensional IRF of this model is given as:

$$\mathrm{P}(X_i=1|{\mathsf{\theta }}_{d_1\left[i\right]},{\mathsf{\theta }}_{d_2\left[i\right]};{\mathsf{\gamma }}_i)=\mathsf{\Psi }\left({\mathsf{\lambda }}_{i0}+{\mathsf{\lambda }}_{i1,d_1\left[i\right]}{\mathsf{\theta }}_{d_1\left[i\right]}+{\mathsf{\lambda }}_{i1,d_2\left[i\right]}{\mathsf{\theta }}_{d_2\left[i\right]}+{\mathsf{\lambda }}_{i2,d_1\left[i\right]d_2\left[i\right]}{\mathsf{\theta }}_{d_1\left[i\right]}{\mathsf{\theta }}_{d_2\left[i\right]}\right).$$

(11)

The disadvantage of (11) is that the product ${\mathsf{\theta }}_{d_1\left[i\right]}{\mathsf{\theta }}_{d_2\left[i\right]}$ can positively contribute to the item response probabilities if both abilities are positive or both are negative. This property can be regarded as unreasonable.

3. Diagnostic Classification Model

In DCMs [9,38] (also referred to as cognitive diagnostic models [39,40]), the latent variables in the latent structure model (1) are discrete. In this article, we consider the case of binary latent variables, which are also referred to as skills in the literature [41]. Let $\mathsf{\alpha }=({\mathsf{\alpha }}_1,\dots ,{\mathsf{\alpha }}_D)\in {\{0,1\}}^D$ be a D-dimensional latent variable that contains binary components ${\mathsf{\alpha }}_d\in \{0,1\}$ ($d=1,\dots ,D$). A value of 1 of variable ${\mathsf{\alpha }}_d$ indicates mastery of dimension d, whereas a value of 0 indicates nonmastery. The DCM is defined as:

$$\mathrm{P}(\mathit{X}=\mathit{x};\mathsf{\gamma })=\sum _{\mathsf{\alpha }\in {\{0,1\}}^D}{\pi }_{\mathsf{\alpha }}\prod _{i=1}^I\mathrm{P}(X_i=x_i|\mathsf{\alpha };{\mathsf{\gamma }}_i),$$

(12)

where the IRFs $\mathsf{\alpha }\mapsto \mathrm{P}(X_i=x_i|\mathsf{\alpha };{\mathsf{\gamma }}_i)$ are now functions of $\mathsf{\alpha }$ and ${\pi }_{\mathsf{\alpha }}$ are probabilities. In a saturated probability model for $\mathsf{\alpha }$, there are $2^D-1$ probabilities ${\pi }_{\mathsf{\alpha }}$ that can be estimated. Obviously, the probabilities must sum to 1:

$$\sum _{\mathsf{\alpha }\in {\{0,1\}}^D}{\pi }_{\mathsf{\alpha }}=1.$$

(13)

In the case of two skills (i.e., $D=2$), the vector $\mathsf{\alpha }$ can take the $2^2=4$ values $(0,0)$, $(1,0)$, $(0,1)$, and $(1,1)$. For high-dimensional $\mathsf{\alpha }$ vectors, positing or estimating a hierarchy among skills [42,43,44] might be convenient, which results in less than $2^D$ possible combination of different $\mathsf{\alpha }$ vectors in the DCM (12).

3.1. Between-Item Dimensionality

In the case of between-item dimensionality, all items load on one and only skill $d_1\left[i\right]$. The IRF is given as:

$$\mathrm{P}(X_i=1|{\mathsf{\alpha }}_{d_1\left[i\right]};{\mathsf{\gamma }}_i)=\mathsf{\Psi }\left({\mathsf{\lambda }}_{i0}+{\mathsf{\lambda }}_{i1,d_1\left[i\right]}{\mathsf{\alpha }}_{d_1\left[i\right]}\right),$$

(14)

where the item parameters ${\mathsf{\lambda }}_{i0}$ and ${\mathsf{\lambda }}_{i1,d_1\left[i\right]}$ again denote the item intercept and the item slope, respectively. Note that (14) reparametrizes two probabilities $\mathrm{P}(X_i=1|{\mathsf{\alpha }}_{d_1\left[i\right]}=a;{\mathsf{\gamma }}_i)$ for $a=0,1$ such that:

$$p_{i1}=\mathrm{P}(X_i=1|{\mathsf{\alpha }}_{d_1\left[i\right]}=1;{\mathsf{\gamma }}_i)=\mathsf{\Psi }\left({\mathsf{\lambda }}_{i0}+{\mathsf{\lambda }}_{i1,d_1\left[i\right]}\right)\mathrm{and}$$

(15)

$$p_{i0}=\mathrm{P}(X_i=1|{\mathsf{\alpha }}_{d_1\left[i\right]}=0;{\mathsf{\gamma }}_i)=\mathsf{\Psi }\left({\mathsf{\lambda }}_{i0}\right).$$

(16)

The probability $1-p_{i1}$ is also referred to as the slipping probability, whereas $p_{i0}$ is denoted as the guessing probability [45]. The DCM with only one skill (i.e., $D=1$) appeared as the mastery model in the literature [46,47].

3.2. Within-Item Dimensionality

In the following subsections, different DCMs for items with within-item dimensionality are discussed. In this case, an item $X_i$ is assumed to load on only two dimensions $d_1\left[i\right]$ and $d_2\left[i\right]$. As argued in Section 2, the situation in which an item loads on more than two dimensions (i.e., two skills) is conceptually similar to the case of two dimensions but just introduces more notation, and it does not add more insights to this article. In analogy to MIRT models, compensatory, noncompensatory, and partially compensatory DCMs can be distinguished.

3.2.1. Compensatory DCM:ADCM

The additive diagnostic classification model (ADCM; Refs. [48,49]) allows that the two skills ${\mathsf{\alpha }}_{d_1\left[i\right]}$ and ${\mathsf{\alpha }}_{d_2\left[i\right]}$ can compensate. The IRF for the ADCM is defined as:

$$\mathrm{P}(X_i=1|{\mathsf{\alpha }}_{d_1\left[i\right]},{\mathsf{\alpha }}_{d_2\left[i\right]};{\mathsf{\gamma }}_i)=\mathsf{\Psi }\left({\mathsf{\lambda }}_{i0}+{\mathsf{\lambda }}_{i1,d_1\left[i\right]}{\mathsf{\alpha }}_{d_1\left[i\right]}+{\mathsf{\lambda }}_{i1,d_2\left[i\right]}{\mathsf{\alpha }}_{d_2\left[i\right]}\right).$$

(17)

Note that $({\mathsf{\alpha }}_{d_1\left[i\right]},{\mathsf{\alpha }}_{d_2\left[i\right]})\mapsto \mathrm{P}(X_i=1|{\mathsf{\alpha }}_{d_1\left[i\right]},{\mathsf{\alpha }}_{d_2\left[i\right]};{\mathsf{\gamma }}_i)$ imposes a constraint on the IRF because the four probabilities $\mathrm{P}(X_i=1|{\mathsf{\alpha }}_{d_1\left[i\right]}=a_1,{\mathsf{\alpha }}_{d_2\left[i\right]}=a_2;{\mathsf{\gamma }}_i)$ for $(a_1,a_2)\in {\{0,1\}}^2$ are represented by the three parameters ${\mathsf{\lambda }}_{i0}$, ${\mathsf{\lambda }}_{i1,d_1\left[i\right]}$ and ${\mathsf{\lambda }}_{i1,d_2\left[i\right]}$.

3.2.2. Noncompensatory DCM:DINA Model

A noncompensatory DCM appeared as the deterministic inputs, noisy “and” gate (DINA; Refs. [45,50]) model in the literature. The IRF of the DINA model is given by:

$$\mathrm{P}(X_i=1|{\mathsf{\alpha }}_{d_1\left[i\right]},{\mathsf{\alpha }}_{d_2\left[i\right]};{\mathsf{\gamma }}_i)=\mathsf{\Psi }\left({\mathsf{\lambda }}_{i0}+{\mathsf{\lambda }}_{i2,d_1\left[i\right]d_2\left[i\right]}{\mathsf{\alpha }}_{d_1\left[i\right]}{\mathsf{\alpha }}_{d_2\left[i\right]}\right).$$

(18)

A high probability of getting item $X_i$ correct can only be achieved if both skills ${\mathsf{\alpha }}_{d_1\left[i\right]}$ and ${\mathsf{\alpha }}_{d_2\left[i\right]}$ are mastered (i.e., they have values of 1).

3.2.3. Partially Compensatory DCM:GDINA Model

The generalized deterministic inputs, noisy “and” gate (GDINA; Refs. [50,51,52]) model is a partially compensatory DCM that has the ADCM (17) and the DINA model (18) as two particular cases. It contains the two main effects of the ADCM as well as the interaction effect of the DINA model. The IRF of the GDINA model is given as:

$$\mathrm{P}(X_i=1|{\mathsf{\alpha }}_{d_1\left[i\right]},{\mathsf{\alpha }}_{d_2\left[i\right]};{\mathsf{\gamma }}_i)=\mathsf{\Psi }\left({\mathsf{\lambda }}_{i0}+{\mathsf{\lambda }}_{i1,d_1\left[i\right]}{\mathsf{\alpha }}_{d_1\left[i\right]}+{\mathsf{\lambda }}_{i1,d_2\left[i\right]}{\mathsf{\alpha }}_{d_2\left[i\right]}+{\mathsf{\lambda }}_{i2,d_1\left[i\right]d_2\left[i\right]}{\mathsf{\alpha }}_{d_1\left[i\right]}{\mathsf{\alpha }}_{d_2\left[i\right]}\right).$$

(19)

The four item response probabilities $\mathrm{P}(X_i=1|{\mathsf{\alpha }}_{d_1\left[i\right]}=a_1,{\mathsf{\alpha }}_{d_2\left[i\right]}=a_2;{\mathsf{\gamma }}_i)$ for $(a_1,a_2)\in {\{0,1\}}^2$ are reparametrized in (19). Hence, in contrast to the ADCM and the DINA model, no constraints are imposed in the GDINA model in its most flexible form.

4. Mixed and Partial Membership Diagnostic Classification Model

Researchers have occasionally questioned that the dichotomous classification into masters and nonmasters of the skills in DCMs is empirically not always tenable [53,54,55]. DCMs assume a crisp membership; that is, students can only belong to the class $\mathsf{\alpha }=0$ or to the class $\mathsf{\alpha }=1$. Mixed membership models (also referred to as grade of membership models) or partial membership models weaken this assumption [56,57,58,59,60,61,62]. In these models, students are allowed to switch classes (i.e., the mastery and the nonmastery states) across items [63,64,65].

In mixed membership models, the vector $\mathsf{\alpha }=({\mathsf{\alpha }}_1,\dots ,{\mathsf{\alpha }}_D)\in {\{0,1\}}^D$ of binary skills is replaced by a vector of continuous bounded variables ${\mathsf{\alpha }}^{\ast }=({\mathsf{\alpha }}_1^{\ast },\dots ,{\mathsf{\alpha }}_D^{\ast })\in {[0,1]}^D$. The value ${\mathsf{\alpha }}_d^{\ast }$ can be interpreted as the degree to which a student belongs to the mastery class ${\mathsf{\alpha }}_d=1$, while $1-{\mathsf{\alpha }}_d^{\ast }$ characterizes the degree to belong to class ${\mathsf{\alpha }}_d=0$. It should be emphasized that the bounded latent membership vector variable ${\mathsf{\alpha }}^{\ast }$ can be equivalently represented by an unbounded latent vector variable $\mathsf{\theta }$ such that:

$${\mathsf{\alpha }}_d=\mathcal{L}\left({\mathsf{\theta }}_d\right)\mathrm{for}d=1,\dots ,D,$$

(20)

where $\mathcal{L}:\mathbb{R}\to [0,1]$ is a monotonically increasing and injective link function. The probit link function ${\mathsf{\alpha }}_d=\mathsf{\Phi }\left({\mathsf{\theta }}_d\right)$ has been used in [12]. In this article, the logistic link function $\mathsf{\Psi }$ is utilized [20,66]. In this case, an injective transformation of the latent membership variable ${\mathsf{\alpha }}^{\ast }$ follows a multivariate normal distribution [66]. More formally, assume that $\mathsf{\theta }=({\mathsf{\theta }}_1,\dots ,{\mathsf{\theta }}_D)$ follows a multivariate normal distribution with mean vector $\mathsf{\mu }$ and covariance matrix $\mathsf{\Sigma }$. The variable ${\mathsf{\alpha }}^{\ast }$ is obtained by applying the logistic transformation coordinate-wise:

$${\mathsf{\alpha }}_d^{\ast }=\mathsf{\Psi }\left({\mathsf{\theta }}_d\right),\mathrm{which} \mathrm{implies}{\mathsf{\theta }}_d={\mathsf{\Psi }}^{-1}\left({\mathsf{\alpha }}_d^{\ast }\right).$$

(21)

We write $\mathit{x}={\mathsf{\Psi }}^{-1}\left(\mathit{y}\right)$ as an abbreviation for the coordinate-wise application of the inverse logistic function for $\mathit{y}=(y_1,\dots ,y_D)$. By using the density transformation theorem [67], we obtain the following:

$$f_{{\mathsf{\alpha }}^{\ast }}(\mathit{y};\mathsf{\mu },\mathsf{\Sigma })=\left[\prod _{d=1}^D\frac{1}{y_d(1-y_d)}\right]f_{\mathsf{\theta }}\left({\mathsf{\Psi }}^{-1}\left(\mathit{y}\right);\mathsf{\mu },\mathsf{\Sigma }\right)\mathrm{for}\mathit{y}\in {[0,1]}^D.$$

(22)

Note that choosing a very large standard deviation for ${\mathsf{\theta }}_d^{\ast }$ (e.g., a standard deviation of 1000) corresponds to a membership variable ${\mathsf{\alpha }}^{\ast }$ whose values are concentrated nearly 0 or 1. That is, the crisp membership employed in DCMs can be obtained as a special case of mixed membership models.

As an alternative, a discrete grid of ${\mathsf{\alpha }}^{\ast }$ values can be defined, and the mixed membership model is defined based on this grid. This approach is also referred as a nonparametric assumption of the mixed membership distribution. For example, one could assume ${\mathsf{\alpha }}^{\ast }\in {\{0,0.5,1\}}^D$. In this specification, the value ${\mathsf{\alpha }}_d^{\ast }=0.5$ can be interpreted as a partial mastery of the dth skill.

In the following two sections, we compare mixed membership and partial membership DCMs. These two modeling approaches differ in how their IRFs are defined.

4.1. Mixed Membership DCM

Mixed membership DCMs have been proposed in [12]. In general, IRFs in mixed membership models are obtained by defining a weighted sum of item response probabilities from crisp membership values associated with a corresponding latent class [12,68].

4.1.1. Between-Item Dimensionality

We first discuss the case of between-item dimensionality. The IRF for the crisp membership case (i.e., using the binary latent vector variable in the DCM) is denoted as:

$$P_{i,{\mathsf{\alpha }}_{d_1\left[i\right]}}=\mathrm{P}(X_i=1|{\mathsf{\alpha }}_{d_1\left[i\right]};{\mathsf{\gamma }}_i).$$

(23)

The IRF for the mixed membership DCM is then defined as

$$\mathrm{P}(X_i=1|{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast };{\mathsf{\gamma }}_i)={\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }\mathrm{P}(X_i=1|{\mathsf{\alpha }}_{d_1\left[i\right]}=1;{\mathsf{\gamma }}_i)+(1-{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast })\mathrm{P}(X_i=1|{\mathsf{\alpha }}_{d_1\left[i\right]}=0;{\mathsf{\gamma }}_i).$$

(24)

Note that (24) can be more compactly written as:

$$\mathrm{P}(X_i=1|{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast };{\mathsf{\gamma }}_i)={\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }{P}_{i,1}+(1-{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast })P_{i,0}.$$

(25)

Using the IRF definition of the between-item dimensionality DCM in (14), we arrive at:

$$\mathrm{P}(X_i=1|{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast };{\mathsf{\gamma }}_i)=\mathsf{\Psi }\left({\mathsf{\lambda }}_{i,0}\right)+{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }\left(\mathsf{\Psi }({\mathsf{\lambda }}_{i,0}+{\mathsf{\lambda }}_{i1,d_1\left[i\right]})-\mathsf{\Psi }\left({\mathsf{\lambda }}_{i,0}\right)\right).$$

(26)

4.1.2. Within-Item Dimensionality

Now, we discuss mixed membership DCMs in the case of within-item dimensionality. In the DCM with a crisp membership, we denote the IRF as:

$$P_{i,{\mathsf{\alpha }}_{d_1\left[i\right]}{\mathsf{\alpha }}_{d_2\left[i\right]}}=\mathrm{P}(X_i=1|{\mathsf{\alpha }}_{d_1\left[i\right]},{\mathsf{\alpha }}_{d_2\left[i\right]};{\mathsf{\gamma }}_i).$$

(27)

The IRF can follow the ADCM, the DINA, or the GDINA model. The IRF for the mixed membership variable ${\mathsf{\alpha }}^{\ast }$ is defined as (see [12]):

$$\begin{array}{cc}& \mathrm{P}(X_i=1|{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast },{\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast };{\mathsf{\gamma }}_i)\hfill \\ =& {\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }{\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast }{P}_{i,11}+{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }(1-{\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast })P_{i,10}+(1-{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }){\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast }{P}_{i,01}+(1-{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast })(1-{\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast })P_{i,00}.\hfill \end{array}$$

(28)

The IRF in (28) generally holds with a specified DCM for crisp membership. That is, by using the ADCM, DINA, or the GDINA model, different specifications of the mixed membership DCM are obtained.

Note that (28) can be rewritten as:

$$\begin{array}{cc}& \mathrm{P}(X_i=1|{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast },{\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast };{\mathsf{\gamma }}_i)\hfill \\ =& P_{i,00}+{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }\left[P_{i,10}-P_{i,00}\right]+{\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast }\left[P_{i,01}-P_{i,00}\right]+{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }{\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast }\left[P_{i,11}+P_{i,00}-P_{i,10}-P_{i,01}\right].\hfill \end{array}$$

(29)

4.2. Partial Membership DCM

Like mixed membership models, partial membership models [69] also weaken the assumption of a crisp membership but define their IRFs differently. While mixed membership models define the IRF as the weighted sum of IRFs from the crisp membership case, partial membership models define the IRF as the weighted harmonic mean of the IRFs from the crisp membership case [68,70]. We now separately describe the cases of between-item and within-item dimensionality in the next two subsections.

4.2.1. Between-Item Dimensionality

The IRFs in the between-item dimensionality in the crisp membership case (i.e., the DCMs) are denoted as:

$$P_{i,{\mathsf{\alpha }}_{d_1\left[i\right]}}=\mathrm{P}(X_i=1|{\mathsf{\alpha }}_{d_1\left[i\right]};{\mathsf{\gamma }}_i)\mathrm{and}{Q}_{i,{\mathsf{\alpha }}_{d_1\left[i\right]}}=1-\mathrm{P}(X_i=1|{\mathsf{\alpha }}_{d_1\left[i\right]};{\mathsf{\gamma }}_i).$$

(30)

If the logistic link function $\mathsf{\Psi }$ is utilized for defining the IRFs in the DCM, they can be written as:

$$P_{i,{\mathsf{\alpha }}_{d_1\left[i\right]}}=\mathsf{\Psi }\left({\mathsf{\eta }}_{i,{\mathsf{\alpha }}_{d_1\left[i\right]}}\right)=\frac{exp\left({\mathsf{\eta }}_{i,{\mathsf{\alpha }}_{d_1\left[i\right]}}\right)}{1+exp\left({\mathsf{\eta }}_{i,{\mathsf{\alpha }}_{d_1\left[i\right]}}\right)}\mathrm{and}{Q}_{i,{\mathsf{\alpha }}_{d_1\left[i\right]}}=\mathsf{\Psi }\left(-{\mathsf{\eta }}_{i,{\mathsf{\alpha }}_{d_1\left[i\right]}}\right)=\frac{1}{1+exp\left({\mathsf{\eta }}_{i,{\mathsf{\alpha }}_{d_1\left[i\right]}}\right)}$$

(31)

with a linear predictor ${\mathsf{\eta }}_{i,{\mathsf{\alpha }}_{d_1\left[i\right]}}={\mathsf{\lambda }}_{i0}+{\mathsf{\lambda }}_{i1,d_1\left[i\right].}$

The IRF for the partial membership DCM is defined as:

$$\mathrm{P}(X_i=1|{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast };{\mathsf{\gamma }}_i)=\frac{P_{i,1}^{{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }}{P}_{i,0}^{1-{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }}}{C_i\left({\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }\right)}\mathrm{and}\mathrm{P}(X_i=0|{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast };{\mathsf{\gamma }}_i)=\frac{Q_{i,1}^{{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }}{Q}_{i,0}^{1-{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }}}{C_i\left({\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }\right)},$$

(32)

where $C_i\left({\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }\right)=P_{i,1}^{{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }}{P}_{i,0}^{1-{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }}+Q_{i,1}^{{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }}{Q}_{i,0}^{1-{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }}$.

In Appendix A, the IRF is derived as:

$$\mathrm{P}(X_i=1|{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast };{\mathsf{\gamma }}_i)=\mathsf{\Psi }\left({\mathsf{\lambda }}_{i0}+{\mathsf{\lambda }}_{i1,d_1\left[i\right]}{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }\right).$$

(33)

This IRF looks identical to the IRF in the between-item dimensionality MIRT model in which ${\mathsf{\theta }}_{d_1\left[i\right]}$ is replaced with ${\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }$. We elaborate on the connection of the two approaches in more detail in Section 5.

4.2.2. Within-Item Dimensionality

We now discuss the definition of the IRF for a partial membership DCM in the case of within-item dimensionality. Let again the IRF for the crisp membership DCMs be denoted by:

$$P_{i,{\mathsf{\alpha }}_{d_1\left[i\right]}{\mathsf{\alpha }}_{d_2\left[i\right]}}=\mathrm{P}(X_i=1|{\mathsf{\alpha }}_{d_1\left[i\right]},{\mathsf{\alpha }}_{d_2\left[i\right]};{\mathsf{\gamma }}_i)\mathrm{and}{Q}_{i,{\mathsf{\alpha }}_{d_1\left[i\right]}{\mathsf{\alpha }}_{d_2\left[i\right]}}=1-P_{i,{\mathsf{\alpha }}_{d_1\left[i\right]}{\mathsf{\alpha }}_{d_2\left[i\right]}}.$$

(34)

The IRFs of the DCMs discussed in this paper utilize the logistic link function. The notation in (34) includes the IRFs of the ADCM, DINA, and the GDINA model.

We can rewrite (34) as:

$$P_{i,{\mathsf{\alpha }}_{d_1\left[i\right]}{\mathsf{\alpha }}_{d_2\left[i\right]}}=\mathsf{\Psi }({\mathsf{\eta }}_{i,{\mathsf{\alpha }}_{d_1\left[i\right]}{\mathsf{\alpha }}_{d_2\left[i\right]}})=\frac{exp({\mathsf{\eta }}_{i,{\mathsf{\alpha }}_{d_1\left[i\right]}{\mathsf{\alpha }}_{d_2\left[i\right]}})}{1+exp({\mathsf{\eta }}_{i,{\mathsf{\alpha }}_{d_1\left[i\right]}{\mathsf{\alpha }}_{d_2\left[i\right]}})}.$$

(35)

The IRF for the partial membership DCM can generally be defined as:

$$\mathrm{P}(X_i=1|{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast },{\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast };{\mathsf{\gamma }}_i)=\frac{P_{i,11}^{{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }{\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast }}{P}_{i,10}^{{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }(1-{\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast })}{P}_{i,01}^{(1-{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }){\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast }}{P}_{i,11}^{{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }{\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast }}}{C_i({\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast },{\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast })}\mathrm{and}$$

(36)

$$\mathrm{P}(X_i=0|{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast },{\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast };{\mathsf{\gamma }}_i)=\frac{Q_{i,11}^{{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }{\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast }}{Q}_{i,10}^{{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }(1-{\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast })}{Q}_{i,01}^{(1-{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }){\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast }}{Q}_{i,11}^{{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }{\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast }}}{C_i({\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast },{\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast })},$$

(37)

where the constant $C_i({\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast },{\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast })$ is chosen such that the probabilities in (36) and (37) add to 1.

In Appendix B, the IRF is simplified to:

$$\mathrm{P}(X_i=1|{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast },{\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast };{\mathsf{\gamma }}_i)=\mathsf{\Psi }\left({\mathsf{\lambda }}_{i0}+{\mathsf{\lambda }}_{i1,d_1\left[i\right]}{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }+{\mathsf{\lambda }}_{i1,d_2\left[i\right]}{\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast }+{\mathsf{\lambda }}_{i2,d_1\left[i\right]d_2\left[i\right]}{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }{\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast }\right).$$

(38)

Interestingly, the IRF in (38) is the same as in the DCM in which the binary skills ${\mathsf{\alpha }}_d$ are substituted with continuous skills ${\mathsf{\alpha }}_d^{\ast }$. These kinds of models are also referred to as probabilistic (membership) DCMs [13,71,72,73,74,75]. Hence, this section demonstrated the equivalence of partial membership and probabilistic membership DCMs if the IRFs are based on the logistic link function (see also [20]).

5. Heuristic Comparison of the Different Modeling Approaches

In this section, MIRT models, (crisp membership) DCMs, and partial membership CDMs are heuristically compared (see also [70,76]). Note thats the multidimensional real-value variable $\mathsf{\theta }\in {\mathbb{R}}^D$ from a MIRT model can be obtained from the bounded variable ${\mathsf{\alpha }}^{\ast }\in {[0,1]}^D$ by applying the inverse logistic transformation (see also [77]):

$${\mathsf{\theta }}_d={\mathsf{\Psi }}^{-1}\left({\mathsf{\alpha }}_d^{\ast }\right)$$

(39)

for all dimensions $d=1,\dots ,D$. Hence, the IRFs based on $\mathsf{\theta }$ from the MIRT model can be alternatively formulated as IRFs based on the transformed latent variable ${\mathsf{\alpha }}^{\ast }$.

For two skills, an equivalent, or for at least three skills, a more parsimonious representation of $\mathsf{\alpha }\in {\{0,1\}}^D$ can be obtained by defining ${\mathsf{\alpha }}_d$ as a discretization of ${\mathsf{\alpha }}_d^{\ast }$; that is:

$${\mathsf{\alpha }}_d=\mathbf{1}({\mathsf{\alpha }}_d^{\ast }>0.5)=\mathcal{C}\left({\mathsf{\alpha }}_d^{\ast }\right),$$

(40)

where $\mathbf{1}$ denotes the indicator function. This underlying multivariate normal distribution approach to DCMs has been discussed in [78]. Hence, (crisp membership) DCMs can be interpreted as partial membership models in which the discretization operator $\mathcal{C}$ is applied in the IRFs.

We now discuss the different implications of the approaches. The GDINA model for the partial membership DCM has the IRF

$$\mathrm{P}(X_i=1|{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast },{\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast };{\mathsf{\gamma }}_i)=\mathsf{\Psi }\left({\mathsf{\lambda }}_{i0}+{\mathsf{\lambda }}_{i1,d_1\left[i\right]}{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }+{\mathsf{\lambda }}_{i1,d_2\left[i\right]}{\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast }+{\mathsf{\lambda }}_{i2,d_1\left[i\right]d_2\left[i\right]}{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }{\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast }\right).$$

(41)

The DCM GDINA specification can be formulated as:

$$\mathrm{P}(X_i=1|{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast },{\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast };{\mathsf{\gamma }}_i)=\mathsf{\Psi }\left({\mathsf{\lambda }}_{i0}+{\mathsf{\lambda }}_{i1,d_1\left[i\right]}\mathcal{C}({\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast })+{\mathsf{\lambda }}_{i1,d_2\left[i\right]}\mathcal{C}({\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast })+{\mathsf{\lambda }}_{i2,d_1\left[i\right]d_2\left[i\right]}\mathcal{C}({\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast })\mathcal{C}({\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast })\right),$$

(42)

which follows the particular step function $\mathcal{C}$, and the cut point of 0.5 is fixed across all items. While (38) assumes linear effects of ${\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }$, ${\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast }$, and ${\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }{\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast }$, the DCM specification follows a particular nonlinear step function. To some extent, the linear continuous effects offer more flexibility in modeling the IRFs.

The multidimensional compensatory IRT model can be written as:

$$\mathrm{P}(X_i=1|{\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast },{\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast };{\mathsf{\gamma }}_i)=\mathsf{\Psi }\left({\mathsf{\lambda }}_{i0}+{\mathsf{\lambda }}_{i1,d_1\left[i\right]}{\mathsf{\Psi }}^{-1}\left({\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }\right)+{\mathsf{\lambda }}_{i1,d_2\left[i\right]}{\mathsf{\Psi }}^{-1}\left({\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast }\right)\right)$$

(43)

with ${\mathsf{\theta }}_d={\mathsf{\Psi }}^{-1}\left({\mathsf{\alpha }}_d^{\ast }\right)$. The IRF in (43) looks quite similar to (38) when excluding the interaction effect for ${\mathsf{\alpha }}_{d_1\left[i\right]}^{\ast }{\mathsf{\alpha }}_{d_2\left[i\right]}^{\ast }$. However, a different transformation of ${\mathsf{\alpha }}^{\ast }$ is utilized when defining the IRF.

Alternatively, one could also start with (38) and formulate a partially compensatory IRT model as:

$$\mathrm{P}(X_i=1|{\mathsf{\theta }}_{d_1\left[i\right]},{\mathsf{\theta }}_{d_2\left[i\right]};{\mathsf{\gamma }}_i)=\mathsf{\Psi }\left({\mathsf{\lambda }}_{i0}+{\mathsf{\lambda }}_{i1,d_1\left[i\right]}\mathsf{\Psi }\left({\mathsf{\theta }}_{d_1\left[i\right]}\right)+{\mathsf{\lambda }}_{i1,d_2\left[i\right]}\mathsf{\Psi }\left({\mathsf{\theta }}_{d_2\left[i\right]}\right)+{\mathsf{\lambda }}_{i2,d_1\left[i\right]d_2\left[i\right]}\mathsf{\Psi }\left({\mathsf{\theta }}_{d_1\left[i\right]}\right)\mathsf{\Psi }\left({\mathsf{\theta }}_{d_2\left[i\right]}\right)\right).$$

(44)

Instead of (11), the logistically transformed variables $\mathsf{\Psi }\left({\mathsf{\theta }}_d\right)$ instead of the original variables ${\mathsf{\theta }}_d$ are used.

Because the latent variable vector ${\mathsf{\alpha }}^{\ast }$ in (38) is bounded, the IRF allows for partially capturing guessing and slipping effects. Hence, the partial membership DCM can be interpreted as more flexible than a MIRT model. The partial membership DCM can likely accommodate the shape of the IRF used in the four-parameter logistic (4PL; Refs. [79,80,81,82]) model and can attain lower and upper asymptotes different from 0 and 1, respectively. Note that it holds that:

$$\underset{({\mathsf{\theta }}_{d_1\left[i\right]},{\mathsf{\theta }}_{d_2\left[i\right]})\to (-\infty ,-\infty )}{lim}\mathrm{P}(X_i=1|{\mathsf{\theta }}_{d_1\left[i\right]},{\mathsf{\theta }}_{d_2\left[i\right]};{\mathsf{\gamma }}_i)=\mathsf{\Psi }\left({\mathsf{\lambda }}_{i0}\right)\mathrm{and}$$

(45)

$$\underset{({\mathsf{\theta }}_{d_1\left[i\right]},{\mathsf{\theta }}_{d_2\left[i\right]})\to (\infty ,\infty )}{lim}\mathrm{P}(X_i=1|{\mathsf{\theta }}_{d_1\left[i\right]},{\mathsf{\theta }}_{d_2\left[i\right]};{\mathsf{\gamma }}_i)=\mathsf{\Psi }\left({\mathsf{\lambda }}_{i0}+{\mathsf{\lambda }}_{i1,d_1\left[i\right]}+{\mathsf{\lambda }}_{i1,d_2\left[i\right]}+{\mathsf{\lambda }}_{i2,d_1\left[i\right]d_2\left[i\right]}\right).$$

(46)

Hence, the probabilistic membership DCMs can be interpreted as approximations of a multidimensional 4PL model [83,84,85,86,87]. Note that the probabilistic membership DCMs also estimate the mean vector $\mathsf{\mu }$ and the covariance matrix $\mathsf{\Sigma }$ in the logit-normal distribution, while standardized variables are used in MIRT models for identification reasons when using a multivariate normally distributed $\mathsf{\theta }$ variable.

Previous literature emphasized that there is likely skill continuity in DCMs. That is, the true data-generating latent variable in DCMs is ${\mathsf{\alpha }}^{\ast }$, but the discrete variable $\mathsf{\alpha }$ is applied [88,89,90] (see also [91]). The strong resemblance between MIRT models and DCMs has also been studied by researcher Matthias von Davier [92,93,94,95,96].

To sum up, the similarity of the approaches can be investigated by comparing distributional assumptions of ${\mathsf{\alpha }}^{\ast }\in {[0,1]}^D$ and IRFs by rephrasing the compensatory MIRT model and DCMs reparametrized based on the latent variable ${\mathsf{\alpha }}^{\ast }$. In this sense, the compensatory MIRT model and DCMs can be viewed as particular cases of a partial membership DCM.

6. Empirical Examples

In this section, we compare different MIRT models, (crisp membership) DCMs, and mixed and partial membership DCMs through analyzing three publicly available datasets. The first dataset has between-item dimensionality, while the second and third dataset have within-item dimensionality. We selected subdatasets that contained only two dimensions (i.e., $D=2$) to reduce model complexity and computation time. It can be expected that the main findings would change if more than two dimensions were used.

The three datasets were frequently applied in DCM applications. Therefore, it is interesting to compare DCMs with alternative model specifications.

6.1. Method

We now describe the three datasets that are used in the following empirical analyses.

The first dataset, acl, contained in the R package mokken [97,98], had 433 subjects on items for the Dutch adjective checklist. For this analysis, we chose 20 items on the scales achievement (original Items 11 to 20) and dominance (original Items 21 to 30). This subdataset had between-item dimensionality. Each of the 20 items either measured the first or the second dimension. The dataset has been dichotomized, and values of at least two were set to one, while values smaller than two were set to zero in this analysis.

The second dataset, data.ecpe, contains item responses of 2922 subjects on the grammar section of the examination for the certificate of proficiency in English (ECPE) test [99,100,101,102]. The data.ecpe dataset can be accessed from the R package CDM [103]. For this analysis, we selected items that measured the first dimension (Skill 1: morphosyntactic rules) or the third skill (Skill 3: lexical rules). In total, 22 items that had within-item dimensionality were selected. Five and ten items uniquely measured the first or the second dimension, respectively. Seven items had within-item dimensionality.

The third dataset, mcmi, contains item responses of 1208 persons on a Dutch version of the Millon clinical multiaxial inventory (MCMI), designed as a questionnaire to diagnose mental disorders. The mcmi dataset is contained in the R package mokken [97,98]. Different DCMs for the whole dataset were studied in [104,105]. In the analysis, we used items from mcmi that measured the dimensions “H” (somatoform) or “CC” (major depression). The resulting 22 items had within-item dimensionality. Eight items measure both dimensions. Four and ten items solely load on the first and the second dimension, respectively.

In the following analyses, six model specifications were applied. In the DCMs, the skill space $\mathsf{\alpha }$ is chosen as ${\{0,1\}}^2$. The mixed membership (MM) and partial membership (PM) specifications defined a distribution on ${[0,1]}^2$ for ${\mathsf{\alpha }}^{\ast }$. In the first approach, the logit-normal distribution was specified (MM-NO and PM-NO) in which the mean vector $\mathsf{\mu }$ and the covariance matrix $\mathsf{\Sigma }$ were estimated. In the second approach, a discrete nonparametric distribution on $\{0,0.5,1\}$ is used for MM and PM (denoted as MM-NP and PM-NP). This approach can be interpreted as a crude approximation of a complex continuous distribution on ${[0,1]}^2$. In the case of between-item dimensionality, there is only a single option for specifying the IRFs. In the case of within-item dimensionality, the GDINA, ADCM, and the DINA model were combined with the DCM, MM-NO, MM-NP, PM-NO, and PM-NP specifications. Finally, MIRT models for $\mathsf{\theta }$ were specified by assuming a bivariate normal distribution on ${\mathbb{R}}^2$. The mean vector $\mathsf{\mu }$ was fixed to $\mathbf{0}$, and only correlations were estimated in the covariance matrix $\mathsf{\Sigma }$, while the standard deviations of the components of $\mathsf{\theta }$ were fixed to 1. In the case of between-item dimensionality, there was only a single option for specifying the multidimensional 2PL model. In the case of within-item dimensionality, the partially compensatory (PC), the compensatory (CO), and the noncompensatory (NC) MIRT model were specified.

The different models are compared through information criteria. The information criteria rely on the deviance $-2l\left(\widehat{\mathsf{\gamma }}\right)$ of a fitted model. The most well-known information criteria are the Akaike information criterion (AIC; ref. [106]) and Bayesian information criterion (BIC; ref. [107]) and are defined as [108]:

$$\mathrm{AIC}=-2l\left(\widehat{\mathsf{\gamma }}\right)+2p\mathrm{and}$$

(47)

$$\mathrm{BIC}=-2l\left(\widehat{\mathsf{\gamma }}\right)+log\left(N\right)p,$$

(48)

where p and N denote the number of estimated model parameters and sample size, respectively. A smaller information criterion indicates a better-fitting model. Note that the information criteria balance model fit and model parsimony.

Notably, AIC and BIC depend on sample size. To obtain a standardized measure of model fit, the Gilula–Haberman penalty (GHP; refs. [109,110,111,112]) is defined as:

$$\mathrm{GHP}=\frac{\mathrm{AIC}}{2NI}=-\frac{l\left(\widehat{\mathsf{\gamma }}\right)}{NI}+\frac{p}{NI}.$$

(49)

The GHP can be interpreted as a standardized measure of a bias-corrected likelihood per case and item. In this article, we define the GHP that is normed for a hypothetical sample size of $N={10}^4=10,000$ and is defined as:

$$\mathrm{GHP}4={10}^4·\mathrm{GHP}.$$

(50)

where, again, smaller values of GHP or GHP4 indicate a better-fitting model.

For the evaluation of alternative models, model differences in statistics AIC, BIC, and $\mathrm{GHP}4$ can be computed, which are denoted as $\mathsf{\Delta }\mathrm{AIC}$, $\mathsf{\Delta }\mathrm{BIC}$, and $\mathsf{\Delta }\mathrm{GHP}4$. For $\mathsf{\Delta }\mathrm{AIC}$, model differences regarding the AIC larger than 10 (see [108]) can be interpreted as substantial. In line with previous research, $\mathrm{GHP}4$ differences larger than 10 might be considered as a moderate deviation, while $\mathsf{\Delta }\mathrm{GHP}4$ differences between 1 and 10 as a small deviation [112,113].

All models were estimated using the sirt::xxirt() function from the R (Version 4.3.1; [114]) package sirt [115]. The models were estimated using marginal maximum likelihood estimation [116]. In the first 100 iterations, the expectation-maximization algorithm [117] was utilized and the algorithm switched afterward to the Newton-Raphson approach.

The different model specifications were compared regarding model fit using the AIC and the $\mathsf{\Delta }\mathrm{GHP}4$ statistics. We did not perform model selection based on BIC because the simulation study presented in Section 7 showed inferior performance of this criterion compared to model selection based on AIC.

6.2. Results

6.2.1. Dataset acl

Table 1. Dataset acl: Model comparisons based on $\mathsf{\Delta }\mathrm{AIC}$ and $\mathsf{\Delta }\mathrm{GHP}4$.

Model	$\mathsf{\Delta }\mathbf{AIC}$	$\mathsf{\Delta }\mathbf{GHP}4$
DCM	167	97
MM-NO	79	46
MM-NP	0 ‡	0 ‡
PM-NO	5	3
PM-NP	11	6
MIRT	5	3

Note: DCM = diagnostic classification model; MM-NO = mixed membership DCM with a logistic normal distribution assumption for ${\mathsf{\alpha }}^{\ast }$; MM-NP = mixed membership DCM with the discrete distribution ${\{0,0.5,1\}}^2$ for ${\mathsf{\alpha }}^{\ast }$; PM-NO = partial membership DCM with a logistic normal distribution assumption for ${\mathsf{\alpha }}^{\ast }$; PM-NP = partial membership DCM with the discrete distribution ${\{0,0.5,1\}}^2$ for ${\mathsf{\alpha }}^{\ast }$; MIRT  = multidimensional item response model; ^‡ = reference model used for computing $\mathsf{\Delta }\mathrm{AIC}$ and $\mathsf{\Delta }\mathrm{GHP}4$; Entries with $\mathsf{\Delta }\mathrm{AIC}$ values smaller than 10 and $\mathsf{\Delta }\mathrm{GHP}4$ values smaller than 10 are printed in bold font.

presents $\mathsf{\Delta }\mathrm{AIC}$ and $\mathsf{\Delta }\mathrm{GHP}4$ for the dataset acl. It turned out that the discrete mixed membership specification (i.e., MM-NP) fitted best, followed by PM-NO and the MIRT model. Notably, the DCM resulted in a worse model fit. However, there were only small model differences between MM-NP, PM-NO, PM-NP, and the MIRT model in terms of the $\mathsf{\Delta }\mathrm{GHP}4$ statistic.

Table 2. Dataset acl: Estimated item parameters for selected models.

	DCM			MM-NP			PM-NO			MIRT
Item	${\mathsf{\lambda }}_{\mathit{i}\mathbf{0}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{1},\mathbf{1}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{1},\mathbf{2}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{0}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{1},\mathbf{1}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{1},\mathbf{2}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{0}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{1},\mathbf{1}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{1},\mathbf{2}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{0}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{1},\mathbf{1}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{1},\mathbf{2}}$
A1	−1.27	2.67		−3.21	6.86		−4.11	7.55		−0.29	2.23
A2	−0.84	1.67		−2.41	5.13		−2.68	4.88		−0.17	1.13
A3	−0.81	1.84		−2.31	5.66		−2.88	5.41		−0.06	1.11
A4	−1.22	1.70		−3.07	4.50		−2.79	4.52		−0.48	1.01
A5	−1.45	2.95		−3.47	7.22		−4.51	8.22		−0.37	2.53
A6	−0.26	1.87		−0.98	5.22		−2.28	5.16		$\phantom{-}$0.52	1.10
A7	−0.10	1.54		−0.67	4.24		−1.73	4.22		$\phantom{-}$0.59	0.95
A8	−0.88	2.27		−2.18	6.23		−3.29	6.34		$\phantom{-}$0.00	1.42
A9	−1.75	1.86		−3.55	4.11		−3.47	5.09		−0.93	1.04
A10	−1.59	2.53		−4.00	6.58		−4.53	7.84		−0.61	1.61
D1	$\phantom{-}$0.87		2.96	$\phantom{-}$0.85		2.16	−0.54		5.11	$\phantom{-}$1.80		0.84
D2	−0.66		4.16	−1.20		5.41	−3.66		7.27	$\phantom{-}$0.85		1.70
D3	−1.89		1.42	−3.81		4.07	−3.81		3.09	−1.12		0.97
D4	−1.67		3.07	−2.69		2.56	−2.79		1.41	−1.15		0.61
D5	−1.40		2.77	−2.74		6.32	−4.45		7.02	$\phantom{-}$0.14		1.80
D6	−1.45		2.48	−3.13		5.82	−4.15		6.09	−0.10		1.53
D7	−0.95		3.45	−3.19		4.62	−2.77		3.93	−0.10		1.01
D8	−1.09		1.29	−2.98		4.05	−2.63		3.16	−0.31		0.88
D9	−1.52		2.85	−2.82		6.24	−4.42		6.83	$\phantom{-}$0.07		1.73
D10	−1.98		2.88	−3.93		5.79	−4.76		6.45	−0.36		1.69

Note: DCM = diagnostic classification model; MM-NP = mixed membership DCM with the discrete distribution ${\{0,0.5,1\}}^2$ for ${\mathsf{\alpha }}^{\ast }$; PM-NO = partial membership DCM with a logistic normal distribution assumption for ${\mathsf{\alpha }}^{\ast }$; MIRT = multidimensional item response model.

presents the estimated item parameters for four selected model specifications for the acl dataset. Overall, the absolute values of MM and PM item parameters were larger compared to the DCM. The correlations of the ${\mathsf{\lambda }}_{i0}$ item parameters were moderate to large but far from perfect (DCM and MM: 0.94, DCM and PM: 0.83, MM and PM: 0.78). While the correlations for the ${\mathsf{\lambda }}_{i1,1}$ parameter were quite large (DCM and MM: 0.92, DCM and PM: 0.98, MM and PM: 0.94), they turned out to be small to moderately valued for ${\mathsf{\lambda }}_{i1,2}$ (DCM and MM: 0.14, DCM and PM: 0.42, MM and PM: 0.75).

The correlation between the two dimensions were 0.67 in the DCM (computed as a tetrachoric correlation), 0.86 in the MM-NO model, 0.98 in the PM-NO model, and 0.54 in the MIRT model. Note that the correlations in MIRT and PM were quite different, resulting in different conclusions, although the model fit was very similar.

In the DCM, the skill class proportions were 0.28 for $(0,0)$, 0.09 for $(1,0)$, 0.25 for $(0,1)$, and 0.38 for $(1,1)$, resulting in marginal skill class probabilities of 0.47 for the first skill and 0.63 for the second skill. In the MM-NP specification, the means of ${\mathsf{\alpha }}_d^{\ast }$ ($d=1,2$) were 0.48 and 0.57, and for PM-NP, they were 0.51 and 0.52, respectively. In addition,

Table 3. Dataset acl: Estimated class probabilities $\mathrm{P}({\mathsf{\alpha }}_1^{\ast },{\mathsf{\alpha }}_2^{\ast })$.

${\mathsf{\alpha }}_1^{\ast }$	${\mathsf{\alpha }}_2^{\ast }$	MM-NP	PM-NP
0	0	0.14	0.09
0.5	0	0.06	0.13
1	0	0.01	0.00
0	0.5	0.11	0.13
0.5	0.5	0.25	0.27
1	0.5	0.08	0.13
0	1	0.03	0.00
0.5	1	0.18	0.14
1	1	0.14	0.11

Note: MM-NP = mixed membership DCM with the discrete distribution ${\{0,0.5,1\}}^2$ for ${\mathsf{\alpha }}^{\ast }$; PM-NP = partial membership DCM with the discrete distribution ${\{0,0.5,1\}}^2$ for ${\mathsf{\alpha }}^{\ast }$.

displays the probabilities of ${\mathsf{\alpha }}^{\ast }$ in the MM-NP and PM-NP specification. Although the general probability patterns were similar, there were nonnegligible differences between the probabilities, indicating that the different specifications resulted in slightly different quantitative interpretations regarding the two skills.

6.2.2. Dataset data.ecpe

Table 4. Dataset data.ecpe: Model comparisons based on $\mathsf{\Delta }\mathrm{AIC}$ and $\mathsf{\Delta }\mathrm{GHP}4$.

	$\mathsf{\Delta }\mathbf{AIC}$			$\mathsf{\Delta }\mathbf{GHP}4$
Model	GDINA	ADCM	DINA	GDINA	ADCM	DINA
DCM	456	454	599	35	35	47
MM-NO	169	179	295	13	14	23
MM-NP	184	206	274	14	16	21
PM-NO	10	0 ‡	29	1	0 ‡	2
PM-NP	88	80	145	7	6	11
	PC	CO	NC	PC	CO	NC
MIRT	104	75	102	8	6	8

Note: DCM = diagnostic classification model; MM-NO = mixed membership DCM with a logistic normal distribution assumption for ${\mathsf{\alpha }}^{\ast }$; MM-NP = mixed membership DCM with the discrete distribution ${\{0,0.5,1\}}^2$ for ${\mathsf{\alpha }}^{\ast }$; PM-NO = partial membership DCM with a logistic normal distribution assumption for ${\mathsf{\alpha }}^{\ast }$; PM-NP = partial membership DCM with the discrete distribution ${\{0,0.5,1\}}^2$ for ${\mathsf{\alpha }}^{\ast }$; MIRT  = multidimensional item response model; GDINA = generalized deterministic inputs, noisy “and” gate; ADCM = additive diagnostic classification model; DINA = deterministic inputs, noisy “and” gate; PC = partially compensatory MIRT model; CO = compensatory MIRT model; NC = noncompensatory MIRT model; ^‡ = reference model used for computing $\mathsf{\Delta }\mathrm{AIC}$ and $\mathsf{\Delta }\mathrm{GHP}4$; Entries with $\mathsf{\Delta }\mathrm{AIC}$ values smaller than 10 and $\mathsf{\Delta }\mathrm{GHP}4$ values smaller than 10 are printed in bold font.

presents model comparisons of the specified DCMs, mixed membership DCMs and MIRT models for the dataset data.ecpe. It turned out that the three PM-NO specifications resulted in the best model fit in terms of AIC. The ADCM for PM-NO was the best-fitting model, slightly better than the GDINA model for PM-NO. In terms of $\mathsf{\Delta }\mathrm{GHP}4$, the model differences between all PM and MIRT model specifications turned out to be small.

Table 5. Dataset data.ecpe: Estimated item parameters for selected models.

	DCM:GDINA				MM-NO:GDINA				PM-NO:GDINA				MIRT:PC
Item	${\mathsf{\lambda }}_{\mathit{i}\mathbf{0}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{1},\mathbf{1}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{1},\mathbf{2}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{2},\mathbf{12}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{0}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{1},\mathbf{1}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{1},\mathbf{2}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{2},\mathbf{12}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{0}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{1},\mathbf{1}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{1},\mathbf{2}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{2},\mathbf{12}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{0},\mathbf{1}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{1},\mathbf{1}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{0},\mathbf{2}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{1},\mathbf{2}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{2},\mathbf{12}}$
E3	−0.31	$\phantom{-}$0.98	$\phantom{-}$0.54	$\phantom{-}$0.31	−1.40	$\phantom{-}$2.07	$\phantom{-}$0.28	$\phantom{-}$1.90	−0.56	$\phantom{-}$1.81	$\phantom{-}$1.66	$\phantom{-}$1.33	$\phantom{-}$0.49	$\phantom{-}$0.82	$\phantom{-}$1.17	$\phantom{-}$0.25	$\phantom{-}$0.51
E4	−0.12		$\phantom{-}$1.69		−1.42		$\phantom{-}$4.08		−1.11		$\phantom{-}$3.81				$\phantom{-}$1.03	$\phantom{-}$0.95
E5	$\phantom{-}$0.98		$\phantom{-}$2.31		$\phantom{-}$0.63		$\phantom{-}$3.93		−0.11		$\phantom{-}$3.87				$\phantom{-}$2.31	$\phantom{-}$1.00
E6	$\phantom{-}$0.84		$\phantom{-}$1.77		$\phantom{-}$0.27		$\phantom{-}$3.83		−0.19		$\phantom{-}$3.63				$\phantom{-}$1.94	$\phantom{-}$0.87
E7	−0.14	$\phantom{-}$1.20	$\phantom{-}$1.13	$\phantom{-}$0.70	−1.51	$\phantom{-}$3.08	$\phantom{-}$1.82	$\phantom{-}$2.57	−0.78	$\phantom{-}$2.33	$\phantom{-}$2.72	$\phantom{-}$2.01	$\phantom{-}$0.91	$\phantom{-}$1.00	$\phantom{-}$1.47	$\phantom{-}$0.72	$\phantom{-}$0.25
E9	$\phantom{-}$0.10		$\phantom{-}$1.26		−0.92		$\phantom{-}$3.32		−0.81		$\phantom{-}$3.19				$\phantom{-}$0.95	$\phantom{-}$0.74
E10	$\phantom{-}$0.10	$\phantom{-}$2.01			−0.84	$\phantom{-}$4.24			$\phantom{-}$0.39	$\phantom{-}$4.16			$\phantom{-}$0.77	$\phantom{-}$0.92
E11	−0.08	$\phantom{-}$1.00	$\phantom{-}$1.11	$\phantom{-}$0.59	−1.34	$\phantom{-}$2.77	$\phantom{-}$1.70	$\phantom{-}$2.46	−0.65	$\phantom{-}$2.02	$\phantom{-}$2.54	$\phantom{-}$1.77	$\phantom{-}$0.84	$\phantom{-}$0.84	$\phantom{-}$1.34	$\phantom{-}$0.63	$\phantom{-}$0.23
E12	−1.72	$\phantom{-}$0.69	$\phantom{-}$1.31	$\phantom{-}$1.06	−3.11	$\phantom{-}$2.06	$\phantom{-}$0.09	$\phantom{-}$3.79	−2.18	$\phantom{-}$2.15	$\phantom{-}$2.94	$\phantom{-}$2.52	$\phantom{-}$0.20	$\phantom{-}$1.27	$\phantom{-}$0.70	$\phantom{-}$1.11	$\phantom{-}$0.66
E13	$\phantom{-}$0.68	$\phantom{-}$1.59			−0.08	$\phantom{-}$3.70			$\phantom{-}$1.09	$\phantom{-}$3.75			$\phantom{-}$1.25	$\phantom{-}$0.80
E14	$\phantom{-}$0.23	$\phantom{-}$1.30			−0.77	$\phantom{-}$3.20			$\phantom{-}$0.39	$\phantom{-}$3.30			$\phantom{-}$0.68	$\phantom{-}$0.66
E15	$\phantom{-}$0.93		$\phantom{-}$2.30		$\phantom{-}$0.55		$\phantom{-}$3.98		−0.24		$\phantom{-}$3.99				$\phantom{-}$2.29	$\phantom{-}$1.05
E16	−0.13	$\phantom{-}$1.07	$\phantom{-}$1.02	$\phantom{-}$0.44	−1.37	$\phantom{-}$2.83	$\phantom{-}$1.50	$\phantom{-}$2.41	−0.68	$\phantom{-}$2.07	$\phantom{-}$2.40	$\phantom{-}$1.70	$\phantom{-}$0.87	$\phantom{-}$0.86	$\phantom{-}$1.37	$\phantom{-}$0.63	$\phantom{-}$0.28
E18	$\phantom{-}$0.86		$\phantom{-}$1.52		$\phantom{-}$0.24		$\phantom{-}$3.52		−0.08		$\phantom{-}$3.38				$\phantom{-}$1.83	$\phantom{-}$0.76
E19	−0.23		$\phantom{-}$1.96		−1.52		$\phantom{-}$4.40		−1.25		$\phantom{-}$4.12				$\phantom{-}$1.09	$\phantom{-}$1.06
E20	−1.46	$\phantom{-}$0.89	$\phantom{-}$1.02	$\phantom{-}$0.86	−2.82	$\phantom{-}$2.34	$\phantom{-}$0.08	$\phantom{-}$3.52	−1.85	$\phantom{-}$2.35	$\phantom{-}$2.68	$\phantom{-}$2.41	$\phantom{-}$0.21	$\phantom{-}$1.28	$\phantom{-}$0.93	$\phantom{-}$0.91	$\phantom{-}$0.65
E21	$\phantom{-}$0.14	$\phantom{-}$0.82	$\phantom{-}$1.24	$\phantom{-}$0.31	−1.16	$\phantom{-}$2.52	$\phantom{-}$2.16	$\phantom{-}$2.21	−0.41	$\phantom{-}$1.81	$\phantom{-}$2.60	$\phantom{-}$1.52	$\phantom{-}$1.02	$\phantom{-}$0.63	$\phantom{-}$1.34	$\phantom{-}$0.79	$\phantom{-}$0.21
E22	−0.90		$\phantom{-}$2.33		−2.58		$\phantom{-}$4.67		−1.89		$\phantom{-}$4.67				$\phantom{-}$0.72	$\phantom{-}$1.35
E25	$\phantom{-}$0.12	$\phantom{-}$1.14			−0.86	$\phantom{-}$2.77			$\phantom{-}$0.23	$\phantom{-}$3.02			$\phantom{-}$0.52	$\phantom{-}$0.57
E26	$\phantom{-}$0.15		$\phantom{-}$1.16		−0.57		$\phantom{-}$2.76		−0.59		$\phantom{-}$2.77				$\phantom{-}$0.92	$\phantom{-}$0.59
E27	−0.87	$\phantom{-}$1.76			−2.58	$\phantom{-}$3.74			−0.66	$\phantom{-}$4.01			−0.26	$\phantom{-}$0.97
E28	$\phantom{-}$0.51		$\phantom{-}$1.90		−0.16		$\phantom{-}$4.07		−0.51		$\phantom{-}$3.95				$\phantom{-}$1.74	$\phantom{-}$0.99

Note: DCM = diagnostic classification model; GDINA = generalized deterministic inputs, noisy “and” gate; MM-NO = mixed membership DCM with a logistic normal distribution assumption for ${\mathsf{\alpha }}^{\ast }$; PM-NO = partial membership DCM with a logistic normal distribution assumption for ${\mathsf{\alpha }}^{\ast }$; MIRT: PC  =  partially compensatory multidimensional item response model.

reports estimated item parameters for the partially compensatory models (i.e., GDINA and PC). We reported the NO specifications for MM and PM because they resulted in a better model fit compared to the nonparametric distribution (i.e., MM-NP and PM-NP).

There were moderate to strong correlations in the item intercept parameter ${\mathsf{\lambda }}_{i0}$ (DCM and MM: 0.98, DCM and PM: 0.78, MM and PM: 0.75). The correlations between different model specifications for the item slope parameters ${\mathsf{\lambda }}_{i1,1}$ and ${\mathsf{\lambda }}_{i1,2}$ were slightly larger (for ${\mathsf{\lambda }}_{i1,1}$: DCM and MM: 0.96, DCM and PM: 0.92, MM and PM: 0.89; for ${\mathsf{\lambda }}_{i1,2}$: DCM and MM: 0.83, DCM and PM: 0.95, MM and PM: 0.86). The correlations of the item parameter ${\mathsf{\lambda }}_{i2,12}$ were large (DCM and MM: 0.95, DCM and PM: 0.98, MM and PM: 0.98). Similar to the acl dataset, the absolute values of estimated item parameters were larger for the MM and PM DCM compared to the crisp membership DCM. The ${\mathsf{\lambda }}_{i2,12}$ parameter in the PC MIRT model were estimated substantially different from 0 and 1, indicating that the items neither function fully compensatory nor noncompensatory.

The estimated correlations for the GDINA specifications between the two dimensions were 0.90 in the DCM (tetrachoric correlation), 0.96 in MM-NO, and 0.99 in the PM-NO model. Interestingly, the correlation of the ADCM specification for PM-NO was slightly smaller at 0.96. The correlations for the different MIRT models were very similar (MIRT-PC: 0.82, MIRT-CO: 0.82, MIRT-NC: 0.81).

6.2.3. Dataset mcmi

Table 6. Dataset mcmi: Model comparisons based on $\mathsf{\Delta }\mathrm{AIC}$ and $\mathsf{\Delta }\mathrm{GHP}4$.

	$\mathsf{\Delta }\mathbf{AIC}$			$\mathsf{\Delta }\mathbf{GHP}4$
Model	GDINA	ADCM	DINA	GDINA	ADCM	DINA
DCM	979	977	1225	184	184	230
MM-NO	1125	1157	1373	212	218	258
MM-NP	281	353	466	53	66	88
PM-NO	0 ‡	16	84	0 ‡	3	16
PM-NP	250	256	483	47	48	91
	PC	CO	NC	PC	CO	NC
MIRT	31	31	65	6	6	12

Note: DCM = diagnostic classification model; MM-NO = mixed membership DCM with a logistic normal distribution assumption for ${\mathsf{\alpha }}^{\ast }$; MM-NP = mixed membership DCM with the discrete distribution ${\{0,0.5,1\}}^2$ for ${\mathsf{\alpha }}^{\ast }$; PM-NO = partial membership DCM with a logistic normal distribution assumption for ${\mathsf{\alpha }}^{\ast }$; PM-NP = partial membership DCM with the discrete distribution ${\{0,0.5,1\}}^2$ for ${\mathsf{\alpha }}^{\ast }$; MIRT  = multidimensional item response model; GDINA = generalized deterministic inputs, noisy “and” gate; ADCM = additive diagnostic classification model; DINA = deterministic inputs, noisy “and” gate; PC = partially compensatory MIRT model; CO = compensatory MIRT model; NC = noncompensatory MIRT model; ^‡ = reference model used for computing $\mathsf{\Delta }\mathrm{AIC}$ and $\mathsf{\Delta }\mathrm{GHP}4$; Entries with $\mathsf{\Delta }\mathrm{AIC}$ values smaller than 10 and $\mathsf{\Delta }\mathrm{GHP}4$ values smaller than 10 are printed in bold font.

reports model comparisons for the mcmi dataset. The GDINA specification of PM-NO fitted best, followed by ADCM of PM-NO. In terms of $\mathsf{\Delta }\mathrm{GHP}4$, the partially compensatory and compensatory MIRT models had only small differences from the PM-NO approaches. The MM-NO resulted in a worse fit, so we instead reported item parameters of the better-fitting MM-NP model. Across all models, the GDINA specifications fitted the data slightly better than the ADCM specifications.

Table 7. Dataset mcmi: Estimated item parameters for selected models.

	DCM:GDINA				MM-NP:GDINA				PM-NO:GDINA				MIRT:PC
Item	${\mathsf{\lambda }}_{\mathit{i}\mathbf{0}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{1},\mathbf{1}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{1},\mathbf{2}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{2},\mathbf{12}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{0}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{1},\mathbf{1}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{1},\mathbf{2}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{2},\mathbf{12}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{0}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{1},\mathbf{1}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{1},\mathbf{2}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{2},\mathbf{12}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{0},\mathbf{1}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{1},\mathbf{1}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{0},\mathbf{2}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{1},\mathbf{2}}$	${\mathsf{\lambda }}_{\mathit{i}\mathbf{2},\mathbf{12}}$
1	−2.51	$\phantom{-}$2.22	$\phantom{-}$2.27	$\phantom{-}$0.28	−3.33	$\phantom{-}$4.05	$\phantom{-}$3.45	$\phantom{-}$2.46	−5.40	$\phantom{-}$4.45	$\phantom{-}$2.79	$\phantom{-}$5.47	−0.21	$\phantom{-}$2.60	$\phantom{-}$0.27	$\phantom{-}$1.80	$\phantom{-}$0.22
2	−2.31	$\phantom{-}$2.08	$\phantom{-}$1.98	$\phantom{-}$0.26	−3.17	$\phantom{-}$3.85	$\phantom{-}$2.66	$\phantom{-}$2.39	−5.05	$\phantom{-}$4.32	$\phantom{-}$2.42	$\phantom{-}$5.11	−0.41	$\phantom{-}$2.14	$\phantom{-}$0.46	$\phantom{-}$1.68	$\phantom{-}$0.24
3	−3.02	$\phantom{-}$1.77			−3.75	$\phantom{-}$2.19			−3.69	$\phantom{-}$3.52			−2.39	$\phantom{-}$1.03
5	−1.09		$\phantom{-}$2.61		−1.50		$\phantom{-}$3.89		−3.20		$\phantom{-}$6.16				$\phantom{-}$0.08	$\phantom{-}$1.94
6	−2.85	$\phantom{-}$2.04			−3.24	$\phantom{-}$2.63			−3.99	$\phantom{-}$4.05			−2.07	$\phantom{-}$1.21
8	−1.54		$\phantom{-}$3.61		−2.05		$\phantom{-}$5.68		−4.54		$\phantom{-}$7.97				$\phantom{-}$0.04	$\phantom{-}$2.90
9	−2.35	$\phantom{-}$1.92	$\phantom{-}$1.79	$\phantom{-}$0.30	−3.09	$\phantom{-}$3.31	$\phantom{-}$2.23	$\phantom{-}$1.85	−4.60	$\phantom{-}$3.92	$\phantom{-}$2.18	$\phantom{-}$4.55	−0.47	$\phantom{-}$2.01	$\phantom{-}$0.41	$\phantom{-}$1.41	$\phantom{-}$0.41
15	−2.33	$\phantom{-}$1.87	$\phantom{-}$1.35	−0.01	−2.98	$\phantom{-}$3.33	$\phantom{-}$1.47	$\phantom{-}$1.19	−4.20	$\phantom{-}$3.63	$\phantom{-}$1.69	$\phantom{-}$3.69	−0.74	$\phantom{-}$2.11	$\phantom{-}$0.73	$\phantom{-}$0.89	$\phantom{-}$0.54
16	−1.73	$\phantom{-}$2.64			−2.28	$\phantom{-}$3.60			−3.73	$\phantom{-}$6.06			−0.67	$\phantom{-}$1.79
21	−2.33		$\phantom{-}$2.24		−3.06		$\phantom{-}$3.30		−3.93		$\phantom{-}$5.14				−1.42	$\phantom{-}$1.57
22	−2.82	$\phantom{-}$1.03	$\phantom{-}$1.40	$\phantom{-}$0.25	−3.39	$\phantom{-}$1.99	$\phantom{-}$1.19	$\phantom{-}$1.55	−4.35	$\phantom{-}$2.45	$\phantom{-}$1.45	$\phantom{-}$3.19	−1.05	$\phantom{-}$1.51	−0.03	$\phantom{-}$0.85	$\phantom{-}$0.76
25	−2.15	$\phantom{-}$1.03	$\phantom{-}$1.52	$\phantom{-}$0.15	−2.67	$\phantom{-}$1.50	$\phantom{-}$2.58	$\phantom{-}$1.68	−3.89	$\phantom{-}$2.41	$\phantom{-}$2.46	$\phantom{-}$3.42	−0.13	$\phantom{-}$0.83	−0.12	$\phantom{-}$1.52	$\phantom{-}$0.48
28	−3.17		$\phantom{-}$3.02		−3.98		$\phantom{-}$4.09		−4.41		$\phantom{-}$5.59				−1.79	$\phantom{-}$1.79
29	−2.78	$\phantom{-}$1.85	$\phantom{-}$2.13	$\phantom{-}$0.35	−3.80	$\phantom{-}$2.99	$\phantom{-}$2.91	$\phantom{-}$2.94	−5.19	$\phantom{-}$3.83	$\phantom{-}$2.61	$\phantom{-}$5.38	−0.44	$\phantom{-}$1.94	−0.31	$\phantom{-}$1.92	$\phantom{-}$0.31
32	−2.38		$\phantom{-}$3.58		−3.27		$\phantom{-}$5.35		−4.77		$\phantom{-}$7.85				−0.87	$\phantom{-}$2.92
33	−1.00	$\phantom{-}$2.54			−1.37	$\phantom{-}$4.07			−3.25	$\phantom{-}$6.32			$\phantom{-}$0.22	$\phantom{-}$1.89
35	−1.90	$\phantom{-}$1.13	$\phantom{-}$1.98	$\phantom{-}$0.14	−2.32	$\phantom{-}$1.50	$\phantom{-}$2.95	$\phantom{-}$2.69	−3.90	$\phantom{-}$2.53	$\phantom{-}$3.00	$\phantom{-}$4.31	−0.34	$\phantom{-}$0.15	−0.48	$\phantom{-}$1.87	$\phantom{-}$0.00
36	−2.22		$\phantom{-}$2.15		−2.83		$\phantom{-}$3.01		−3.74		$\phantom{-}$4.90				−1.32	$\phantom{-}$1.47
37	−3.60		$\phantom{-}$3.72		−4.42		$\phantom{-}$4.99		−5.41		$\phantom{-}$7.43				−2.19	$\phantom{-}$2.96
38	−2.43		$\phantom{-}$2.52		−2.93		$\phantom{-}$3.40		−4.08		$\phantom{-}$5.55				−1.34	$\phantom{-}$1.67
39	−1.26		$\phantom{-}$1.27		−1.63		$\phantom{-}$1.65		−2.21		$\phantom{-}$3.15				−0.65	$\phantom{-}$0.81
44	−2.63		$\phantom{-}$2.86		−3.21		$\phantom{-}$3.93		−4.47		$\phantom{-}$6.30				−1.41	$\phantom{-}$1.96

Note: DCM = diagnostic classification model; GDINA = generalized deterministic inputs, noisy “and” gate; MM-NP = mixed membership DCM with the discrete distribution ${\{0,0.5,1\}}^2$ for ${\mathsf{\alpha }}^{\ast }$; PM-NO = partial membership DCM with a logistic normal distribution assumption for ${\mathsf{\alpha }}^{\ast }$; MIRT: PC = partially compensatory multidimensional item response model.

displays the estimated item parameters for four selected specifications for the mcmi dataset. Overall, the item parameters correlated moderately to strongly across models.

The correlations for the GDINA specification between the two dimensions were large (DCM: 0.93, MM-NO: 0.99, PM-NO: 1.00, MIRT-PC: 0.95, MIRT-CO: 0.96, MIRT-NC: 0.93).

7. Simulation Study

In this simulation study, we evaluate the accuracy of model selection based on AIC, BIC, and GHP. For simplicity, we only considered the case of between-item dimensionality.

7.1. Method

We used estimated model parameters from the analysis of the acl dataset presented in Section 6.2.1. This dataset had between-item dimensionality and consisted of 20 items. We varied the sample size N as 500, 1000, and 2000. Six data-generating models (DGM) were simulated: the DCM, MM-NO, MM-NP, PM-NO, PM-NP, and the MIRT model. The same models were specified as analysis models for all DGMs.

Model estimation was carried out in the same way as described in Section 6 (see p. 10). We assessed the percentage rate that a particular analysis was selected based on the smallest AIC and smallest BIC. Moreover, we computed the average $\mathsf{\Delta }\mathrm{GHP}4$ statistic for $\mathrm{GHP}4$ difference between the analysis model and the DGM (see Section 6.1).

In total, 2500 replications were carried out in each of the 3 (sample size) × 6 (DGMs) = 18 cells of the simulation study.

7.2. Results

Table 8. Simulation Study: Model selection rates based on AIC and BIC and average $\mathsf{\Delta }\mathrm{GHP}4$ statistic for five different data-generating models (displayed in columns).

		DCM			MM-NO			MM-NP			PM-NO			PM-NP			MIRT
		$\mathit{N}$			$\mathit{N}$			$\mathit{N}$			$\mathit{N}$			$\mathit{N}$			$\mathit{N}$
Crit	Model	500	1000	2000	500	1000	2000	500	1000	2000	500	1000	2000	500	1000	2000	500	1000	2000
AIC	DCM	97	98	98	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
	MM-NO	0	0	0	9	39	76	0	0	0	0	0	0	0	0	0	0	0	0
	MM-NP	2	1	1	26	23	13	97	100	100	1	0	0	1	0	0	0	0	0
	PM-NO	0	0	0	5	5	4	0	0	0	31	57	83	0	0	0	3	3	2
	PM-NP	1	1	1	6	4	1	2	0	0	2	1	0	98	100	100	2	1	0
	MIRT	0	0	0	53	27	6	0	0	0	66	42	17	1	0	0	95	97	98
BIC	DCM	100	100	100	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
	MM-NO	0	0	0	0	1	15	0	0	0	0	0	0	0	0	0	0	0	0
	MM-NP	0	0	0	0	0	0	67	99	100	0	0	0	1	0	0	0	0	0
	PM-NO	0	0	0	0	0	0	0	0	0	0	1	6	1	0	0	0	0	0
	PM-NP	0	0	0	0	0	0	0	0	0	0	0	0	42	96	100	0	0	0
	MIRT	0	0	0	100	99	85	32	1	0	100	99	94	57	4	0	100	100	100
$\mathsf{\Delta }\mathrm{GHP}4$	DCM	0 ‡	0 ‡	0 ‡	16	18	19	100	103	104	35	36	37	105	109	110	64	65	64
	MM-NO	59	56	54	0 ‡	0 ‡	0 ‡	52	48	46	15	12	11	55	52	50	49	44	42
	MM-NP	4	2	1	−1	0	1	0 ‡	0 ‡	0 ‡	7	6	6	13	13	13	15	14	13
	PM-NO	30	29	28	0	1	1	20	20	20	0 ‡	0 ‡	0 ‡	19	20	20	4	2	1
	PM-NP	4	2	1	1	2	2	9	9	9	5	5	4	0 ‡	0 ‡	0 ‡	8	7	7
	MIRT	48	50	50	−2	1	2	18	20	20	−1	0	1	14	16	17	0 ‡	0 ‡	0 ‡

Note: Crit = criterion; DCM = diagnostic classification model; MM-NO = mixed membership DCM with a logistic normal distribution assumption for ${\mathsf{\alpha }}^{\ast }$; MM-NP = mixed membership DCM with the discrete distribution ${\{0,0.5,1\}}^2$ for ${\mathsf{\alpha }}^{\ast }$; PM-NO = partial membership DCM with a logistic normal distribution assumption for ${\mathsf{\alpha }}^{\ast }$; PM-NP = partial membership DCM with the discrete distribution ${\{0,0.5,1\}}^2$ for ${\mathsf{\alpha }}^{\ast }$; MIRT = multidimensional item response model; ^‡ = reference model used for computing $\mathsf{\Delta }\mathrm{GHP}4$; Entries with model selection rates larger than 50% and entries with $\mathsf{\Delta }\mathrm{GHP}4$ values smaller than 10 are printed in bold font. Cells in which the data-generating model coincided with the analysis model are highlighted with a yellow background.

reports the percentage model selection rates and the average $\mathsf{\Delta }\mathrm{GHP}4$ statistic as a function of sample size for the five DGMs.

If the DCM was the DGM, model selection based on AIC and BIC was accurate. However, there were only small average model differences of the DCM to the MM-NP and PM-NP specifications in terms of $\mathsf{\Delta }\mathrm{GHP}4$.

If the MM-NO was the DGM, model selection based on the BIC strongly favored the MIRT model. However, there were very small model differences between MM-NO and the other models, making the different analysis models difficult to distinguish. The true DGM MM-NO can only be correctly detected for a large sample size $N=2000$ and when using the AIC.

If the data were generated by MM-NP, model selection based on AIC worked well. However, BIC also chose the MIRT model at a nonnegligible rate of 32% for the sample size $N=500$. Note that the PM-NP analysis model was relatively close to the MM-NP model in terms of $\mathsf{\Delta }\mathrm{GHP}4$.

If the DGM was the PM-NO model, the MIRT model was generally favored when the BIC was used as the model selection criterion. The AIC only showed acceptable model selection rates for $N=2000$.

If the PM-NP was the DGM, AIC performed well for all sample sizes but failed for the BIC for a sample size of $N=500$.

Finally, if the DGM was the MIRT model, model selection based on AIC and BIC performed well. However, there were only small model differences between MIRT and the partial membership models PM-NO and PM-NP, but larger differences to the mixed membership specifications MM-NO and MM-NP.

8. Discussion

In this article, we compared the extension of DCMs to mixed and partial memberships with crisp membership DCMs and MIRT models through three empirical datasets and a simulation study. We clarified the different nature of the two types of memberships (i.e., mixed and partial membership). In particular, we compared the partial membership DCM to the MIRT model. Essentially, these two specifications can be interpreted as structurally very similar. Hence, it is up to the researcher to interpret dimensions as partial memberships of classes or as measures of a quantitative continuous latent variable.

In the empirical examples, the DCM extensions fitted the data substantially better than the crisp membership DCMs. Moreover, partial membership DCMs outperformed mixed membership DCMs because they offer more flexibility in modeling IRFs. An anonymous reviewer wondered why an empirical study was needed to demonstrate this finding. However, the finding has previously been pointed out in the mixed membership literature of exploratory mixture and latent class models [68]. Nevertheless, recent generalizations of DCMs only considered mixed and not partial membership [12,13]. Therefore, our simulation study and empirical analysis could give rise to more DCM research devoted to partial memberships. Furthermore, MIRT models frequently showed a similar fit to the partial membership DCMs.

The simulation study showed that model selection based on AIC turned out to be more reliable than that of BIC. This finding holds for small (i.e., $N=500$) and large (i.e., $N=2000$) sample sizes. For a fixed number of items, model differences in terms of the GHP penalty are almost independent of the sample size. However, a dependence of the GHP penalty on the number of items cannot be expected.

In this article, we confined ourselves to low-dimensional models (i.e., models with only two dimensions). In models with more than two dimensions, marginal maximum likelihood might be computationally demanding, and pairwise likelihood [118,119,120], variational approximation [121,122,123], or Markov chain Monte Carlo estimation [124,125,126] might be considered alternatively.

Given the main findings of our paper, it might be questioned whether DCMs with binary skills should be used at all compared to partial membership DCMs or MIRT models. The model-based classification of persons into masters and nonmasters might be attractive to practitioners in terms of model interpretation and using individual classifications as the outcome of the DCM. Interpreted in this way, DCMs with crisp membership are model-based confirmatory cluster analyses that automatically provide classifications into masters and nonmasters of skills. However, the classification could alternatively be determined by content experts, such as in a standard setting [127,128,129] procedure. The classifications obtained by DCMs will likely be strongly dependent on the chosen persons and items [130], particularly if DCMs with a crisp membership are not the data-generating model (i.e., in the presence of skill continuity). Importantly, the property of the absolute invariance of item parameters in DCMs only holds for correctly specified models [53,90,131,132]. In MIRT models, there is always scale indeterminacy because the mean and the standard deviations of the dimensions of $\mathsf{\theta }$ cannot be disentangled from item intercepts and item discriminations if they were freely estimated. That there is no such an indeterminacy in DCMs comes at a price that the DCM exactly holds. Hence, model-based comparisons from DCMs of groups in cross-sectional settings and change in longitudinal designs should only be carried out with caution. If researchers still want to apply DCMs for such comparisons, we think that it is advisable to impose invariant item parameters across groups or time points to enable well-defined and interpretable comparisons, even if measurement invariance is certainly violated.